A Response to "The Next Era of Assessment"

I read The Next Era of Assessment: A Global Review of AI in Assessment Design, published by the Digital Education Council in partnership with Pearson.¹ The report draws on 101 case studies and organizes AI-integrated assessment into 14 methodologies. It offers a substantial collection of activities instructors might use to teach with and about AI.

I can see the value of that collection. I also agree with its starting point: assessment must provide evidence of the abilities we intend to assess, and instructions about permitted AI use are not enough to establish who did the work.

But I did not find that the report consistently applies this principle to its own proposals. It calls for structural changes that do not depend on student compliance, then repeatedly describes activities in which students are instructed to work independently, evaluate AI output themselves, or document their thinking. It then treats the resulting submissions as though they establish that those things happened.

The report recognizes that we cannot simply instruct students to do the intellectual work themselves. But many of its methods depend on exactly that instruction.

Not every example works this way. Some include live discussion, instructor interaction, or supervised performance. Those can provide useful evidence, and we should keep them separate from submitted accounts of a process. My criticism is about what the designs establish as this report describes them. It is not a claim that every instructor using these activities lacks other ways to assess their students.

What the Report Gets Right

The report distinguishes AI-Free, AI-Assisted, and AI-Integrated assessment. It also identifies two educational priorities: developing and assuring human competence, and developing the ability to work with AI. These are useful distinctions, provided we specify the abilities involved and how they will be demonstrated.

The course-level example on page 13 includes an initial in-class quiz, opportunities to work with AI, and final supervised or oral examinations. That arrangement can make sense. Students do activities that help them learn, then demonstrate particular abilities under conditions that make their performance observable.

The report is also right that we need not put every learning activity under examination conditions. A formative exercise can be valuable without serving as reliable evidence for a high-stakes grade. An opportunity to learn is not the same as evidence of achievement, and the recommendations need to keep that distinction clear throughout.

I am not convinced when an activity's intended benefits are used to support claims about what its deliverables establish.

What Makes an Assessment AI-Resilient?

On page 11, the report defines AI-resilience in terms of designing conditions that make it difficult to outsource the core learning tasks. It explicitly says this should not depend on students complying with instructions. Table 4 then offers four strategies.

Keep core human tasks in class. This addresses the problem directly when students perform the relevant work under supervised conditions. But we still need to identify the ability being observed. In the report's example, students brainstorm and draft with AI at home, then present their arguments and answer questions live in class. Answering questions about an argument is evidence of the understanding shown in those answers. It does not establish that the student developed the argument independently beforehand.

Shift from output to process. The example asks students to log and reflect on their AI interactions or justify their solutions. These are additional outputs. A submitted explanation of a decision does not establish who made the decision. A reflection can support learning without reliably documenting how the learning occurred.

Embed checkpoints and traceable development. The report lists intermediate submissions, live discussions, feedback loops, and planning artifacts. Its example is an in-class peer discussion and feedback session, after which students revise their work. Live discussion can reveal something about students' understanding. Intermediate submissions alone do not do the same job. AI can produce a plan, a draft, a revision, and an explanation connecting them. The value of a checkpoint depends on what the student actually demonstrates at it, not simply on its position in a sequence.

Validate at the unit level rather than the task level. It is reasonable to collect evidence across several assessments. But the example says validity comes from the coherence and progression of work across four lessons. Why does coherence establish student competence? A coherent sequence of artifacts can still be produced with extensive AI assistance. If the final interactive presentation provides evidence, identify what it demonstrates. The earlier artifacts do not become evidence of independent thinking merely because the presentation is consistent with them.

Combining evidence can strengthen a conclusion when the components provide relevant information. Adding more submissions does not fix a limitation that all of them share.

AI-Free and AI-Assisted Assessment

The report's section titled "Practical Guide to Assessment Design in the Age of AI" begins with a defensible position: to assess unaided performance, remove access to AI rather than merely prohibit it. But on page 15 it places supervised assessment alongside two alternative approaches: contextualized tasks and process documentation.

These do not establish the same conditions.

A task about a recent class discussion may be unfamiliar to a model, but the student may be able to supply the relevant context. An assignment involving local circumstances is not automatically protected from delegation. If the design depends on information remaining unavailable to AI, that dependency needs to be established rather than assumed.

Process documentation has the same problem we saw above. Annotated drafts, planning notes, and voice memos are accounts of work. Their existence does not establish that the student performed the underlying reasoning independently.

The report acknowledges that these alternatives exploit current AI limitations. That makes them conditional and potentially temporary. It does not make them equivalent to observing a student work without AI.

The AI-Assisted section offers a useful map of where AI can enter an assignment: planning, research, creation, editing, and reflection. But the map also illustrates the problem. AI assistance is available at every stage, including the stages meant to reveal the student's contribution. Deciding which uses are permitted does not establish that the boundaries were followed.

The report says restricted components should be redesigned to be AI-resilient. That brings us back to the question above: which change makes the relevant student performance observable?

The Eight Methods for Enhancing Traditional Assessment

1. AI-Guided Self-Assessment and Reflection

The method: Students engage in a structured dialogue with AI, save the conversation, and reflect on their understanding and the feedback they received. (Page 20.)

This can be a useful study activity. Explaining a concept and responding to questions may help a student identify gaps.

But a submitted dialogue is not necessarily a record of unaided student answers. A student could use another AI interaction to prepare responses or generate an account of the exchange. The reflection can also be delegated.

The Feynman Technique example has AI question the student and summarize strengths and gaps. That introduces a second issue: what justifies treating the AI's appraisal as an accurate judgment of understanding? Even if the conversation is authentic, we still need an appropriate basis for interpreting the answers.

If an instructor observes the student responding, there is a direct performance to evaluate. If the instructor receives only a transcript and reflection, the evidential claim is narrower. The assignment should not treat those two situations as the same.

2. AI First, Human Revision

The method: Students obtain an AI-generated draft or solution, then critique and improve it, submitting both versions and a reflection. (Page 21.)

The key step is called human revision. What establishes that a human did it?

AI can generate the initial draft, identify its defects, revise it, and explain the changes. Tracked changes show differences between versions. They do not establish who developed those changes.

The R-code example contains a useful instructional sequence: students struggle to produce a chart with AI, receive an instructor explanation, and then rework the code manually. If that manual work is observed under appropriate conditions, it can provide evidence of the skills exercised. But the model's initial failure is not what makes the later assessment valid, and that failure cannot be assumed to persist.

Revising supplied code is also different from developing a solution independently. Either can be worth teaching. The assignment needs to say which accomplishment it measures.

3. Human First, AI Review

The method: Students complete a task without AI, obtain AI feedback, revise their work, and submit the original, feedback, revision, and reflection. (Page 22.)

This is the clearest example of the report's central inconsistency. The first instruction tells students to work independently and then says this ensures foundational understanding has been demonstrated.

No. The instruction specifies a condition. It does not establish that the condition was met.

If the original work is completed under supervision, that work provides relevant evidence. If it is completed outside a controlled setting, submitting it alongside a later version does not establish independence. Both versions, the feedback, and the reflection could be generated with AI.

The essay example includes consultation with the instructor. That interaction may reveal understanding. The report should say what evidence that interaction provides, rather than credit the evidence to the fact that an original draft exists.

4. AI-Generated Materials for Analysis

The method: Students or the instructor use AI to generate passages, solutions, or scenarios, and students apply disciplinary analysis to them. (Page 23.)

If analysis is the learning objective, giving students material to analyze makes sense. But the assessment problem is the analysis itself. AI can produce the analysis too. Asking students to analyze the material does not establish that they did.

In the literary example, students examine an AI imitation of an author's style and write an analysis. AI can also produce an analysis of stylistic features. Requiring references to course concepts does not establish that the student applied those concepts.

It is also not clear what AI adds. Instructors asked students to analyze passages, solutions, and scenarios long before AI. The material can come from the instructor, a textbook, or a published source. In the literary example, the AI origin does matter, because students judge how well the AI imitated the author. But that is a reason specific to that task. As a general method, this is the traditional assessment with a different source for the material. It is not clear how that is an enhancement, and it keeps the same weakness: the analysis can be delegated.

5. AI as a Simulated Collaborator or Role-Player

The method: Students interact with an AI playing a patient, expert, or other stakeholder, then submit a transcript and reflection. (Page 24.)

This has a clear potential use as practice. A simulation can give students opportunities to rehearse questions, decisions, and responses.

For assessment, the distinction is between observing a student participate and receiving a record of an interaction. A transcript does not by itself establish who formulated the student's responses. A reflection does not establish that the student could perform the interaction again.

The interview example also includes a subsequent interview with a real person. That is a separate activity, and it may provide its own evidence. It should be evaluated on that basis.

We also need to justify what performance in the simulation tells us about performance with real people. A convincing role-play is not automatically an adequate test of the full ability we intend to assess.

6. AI for Immersive Learning

The method: Students generate narratives, images, or simulations to explore disciplinary concepts, then explain their choices and learning. (Page 25.)

An immersive experience may be engaging. But engagement, understanding, and evidence of understanding are different things.

Generating a vivid historical narrative does not establish that the student understands the historical conditions it depicts. Producing an image of an abstract concept does not establish that the student understands that concept. AI can also generate the interpretation and the account of how revising the image improved understanding.

The report asks students to explain how AI deepened their understanding. That assumes the effect was beneficial, and the assessment should be ready to question that assumption. Students should be able to demonstrate misunderstanding, identify a misleading representation, or conclude that the exercise did not help. A persuasive account of learning is not the same as demonstrated learning.

7. Human versus AI Work Comparison

The method: Students compare human and AI responses to the same task, analyzing their reasoning, quality, and limitations. (Page 26.)

Comparison can be a useful way to teach judgment. But two issues remain.

If students supply the human response, we need to establish that it was produced under the intended conditions. If they supply the comparison, we need evidence that they performed the analysis. Labeling one response human does not establish its origin, and asking for comparison does not prevent that comparison from being delegated.

The critical-reading example explicitly places the reading in class. That offers a useful basis for an assessment if the analysis is also performed under appropriate conditions. Again, the conditions of performance provide the evidence. The human-versus-AI framing supplies the subject matter.

8. AI as an Assistant

The method: Students use AI across a task, document its contribution, and explain their own role in the final result. (Page 27.)

This describes a plausible way of working. It does not yet explain how the instructor establishes competence in that way of working.

The persuasive-task example requires at least five logged AI interactions. Why should five interactions establish meaningful engagement? The number of exchanges is not a measure of the quality of the student's decisions. A student could make one important intervention or record several superficial ones.

The interview example has students conduct interviews, then use AI to organize the material and help write an essay. Conducting interviews, interpreting evidence, and constructing an argument are different abilities. Evidence of the first does not automatically establish the others.

If the intended outcome is effective AI-assisted work, assess the student's decisions under known tool conditions. That means observing students under controlled conditions, with an appropriate common tool environment, as they make decisions and respond to unfamiliar requirements. A retrospective statement about what the student contributed is not a substitute for evidence of that contribution.

The Six Methods with AI as the Object of Study

Teaching about AI is a legitimate curricular choice. But making AI the topic of an assignment does not, by itself, change the assessment problem.

9. AI Output Critique and Evaluation

The method: Students evaluate AI outputs for accuracy, bias, relevance, and quality, sometimes comparing sources or working in groups. (Page 29.)

These are worthwhile abilities. They are also tasks students can ask AI to perform.

The small-group example could allow an instructor to hear students developing criteria and defending judgments. That would provide evidence of the contributions actually observed. The final report or reflection does not independently establish that every student exercised those abilities.

The report also says the focus of this method is "understanding why AI produces certain output." That is a different task from noticing a problem in an output. Explaining why a model produced something requires evidence about the model, not only about the output. An account that sounds plausible should not receive credit merely for supplying a confident explanation of model behavior.

10. Prompt Engineering and Process Analysis

The method: Students iteratively design prompts, examine changes in output, and develop hypotheses about the tool's behavior. (Page 30.)

This can be an exercise in experimentation. But AI can help write prompts, propose revisions, analyze results, and draft the explanation. One of the report's examples asks students to log 3 to 5 iterations per experimental step. A log of iterations does not isolate the student's contribution.

The quality of the inquiry also matters. A different output after a changed prompt is an observation, not by itself an explanation of the model's underlying logic or training data. The report appropriately calls the proposed explanations hypotheses. Assessment should preserve that distinction and ask what evidence would support or challenge them.

If we want to assess experimental reasoning, we need to see students select comparisons, interpret evidence, and acknowledge uncertainty. Counting iterations or collecting polished explanations does not establish that reasoning.

11. AI Ethics, Policy, and Societal Impact

The method: Students investigate AI-related issues, develop positions, and communicate them through discussion, essays, or policy proposals. (Page 31.)

These are recognizable disciplinary activities with AI as their subject. Their importance does not make their deliverables reliable evidence of independent judgment.

An AI-generated policy brief can contain an apparently nuanced argument. The student may or may not understand or endorse its reasoning. The topic being responsible AI use does not prevent delegation of the analysis.

The classroom-policy example includes in-class discussion and group presentations. Those can provide observable contributions to assess. But a group policy produced afterward, with AI permitted, cannot automatically establish each participant's ethical reasoning. The assessment needs to identify which contribution supports which claim.

12. Constructive Misuse

The method: Students investigate AI vulnerabilities in a controlled, ethical exercise, then analyze the results and their implications. (Page 32.)

The educational purpose is understandable: examining a failure can make a risk concrete. But successfully eliciting a problematic output is different from understanding the failure, evaluating its consequences, or proposing an effective response.

The result also depends on the particular tool and its restrictions. In the report's example, students ask a model to write a disinformation article, then note whether the tool hesitated and how they got around it. Whether a tool refuses depends on its safeguards, not on the student. A tool refusing a request does not establish poor student understanding, and a tool complying does not establish sophisticated understanding.

The analysis and reflection remain susceptible to delegation. If the goal is to assess judgment about AI risks, that judgment needs to be demonstrated. The generated artifact alone is not enough.

13. AI as a Contextual Case Study

The method: Students use AI tools, interfaces, or policies as cases through which to apply disciplinary concepts. (Page 33.)

That is a reasonable choice of subject matter. An interface can be studied through design principles, and a policy can be examined through an ethical framework.

But these remain analysis tasks. AI can assist with the comparison, annotation, interpretation, and written conclusions. Naming AI as the case study does not establish that the student applied the disciplinary framework.

The relevant question is the same as for any case-based assessment: what will the student demonstrate, under what conditions, that warrants a conclusion about their ability to use the framework?

14. AI as an Artifact

The method: Students design an AI-related tool or activity and explain its purpose, design decisions, limitations, and value. (Page 34.)

A working artifact may show that someone produced an artifact that works. A report may give a sound design rationale. Neither one automatically establishes who developed the design or who understands how it operates.

This example matters here because the students are educators designing AI-enhanced educational activities, which is the same kind of work this report does. Their report includes objectives, expected outcomes, and claims about AI's added value. Implementation and lessons learned are included only if they have had a chance to try the activity.

That can assess the quality of a design proposal if the relevant reasoning is established. It cannot establish that the proposed activity improves learning. Expected outcomes and observed outcomes are different kinds of claims. The distinction matters just as much for this report's own recommendations.

Does Blending the Methods Resolve the Problem?

The final example combines image critique with prompt experimentation. Students analyze existing AI images, recreate them through several iterations, and submit their analysis, images, prompts, and reflection. (Page 35.)

There may be a good instructional reason to combine those activities. But combining two methods does not automatically overcome a limitation they share. AI can contribute to both the production and the explanation of the production.

More stages create more opportunities to learn. They can also create more work for instructors to review. Neither establishes that the final collection gives reliable evidence of each student's understanding.

If a live discussion or supervised task supplies that evidence, identify it. Do not attribute the assurance to the number or variety of submitted artifacts.

What Do the 101 Case Studies Establish?

The collection shows that educators are experimenting with many ways of using AI. That is useful information. It can give us ideas and show which approaches come up repeatedly.

But the report does not provide a study-selection and analysis method that would let us judge how comprehensive its review is. Its activity descriptions generally do not report cohort sizes, learning-outcome measurements, or tests of whether the proposed evidence distinguishes student competence from delegated performance. It also labels methods as best suited to particular outcomes without presenting comparative evidence for those judgments.

This is a limit of the evidence presented, not proof that the activities fail. The report notes that additional material is available in a member briefing; my response concerns the report itself.

For a collection of teaching ideas, descriptions of practice can be enough to make the collection useful. For claims that those designs protect assessment validity, more is required. Having many cases is not the same as having many independently validated assessment methods.

What Each Method Should State

Throughout the report, one distinction keeps getting lost: an activity may invite students to exercise an ability without its deliverables establishing that they exercised it.

That does not make the activity worthless. It changes what we can claim for it.

I would want each method to distinguish:

1. The learning opportunity. What might students learn by participating as intended?
2. The ability being assessed. Are we assessing independent production, understanding, evaluation, communication, or performance with AI?
3. The conditions of performance. What assistance is available, and how do we establish the conditions rather than merely announce them?
4. The evidence and its limits. What does the student actually demonstrate, and what remains unestablished?

For independent performance, controlled assessment offers a direct way to collect relevant evidence. For performance with AI, the same reasoning applies, as I argued under Method 8: we can observe students under controlled conditions with a common tool environment. The intended ability determines which conditions and tasks make sense.

The report already includes elements of this approach. Its supervised assessments and live interactions deserve attention because they let us observe student performance. But those elements do not automatically validate the drafts, logs, reflections, and reports collected around them.

The report gave me a useful catalog of possible learning activities. But I do not think the catalog establishes how to assess individual achievement reliably when the work can be delegated. The report asks for designs that do not rely on student compliance. Its recommendations need to meet that same standard.