A Response to "AI Can Do Your Homework. Now What?"

I read AI Can Do Your Homework. Now What?¹ looking for ways to assess student learning when AI can produce the work we traditionally assign. I appreciated the report's candor about the difficulties instructors are encountering. It records failures, disagreements, and unresolved questions, and it explicitly presents itself as an account of a workshop rather than a research study.

The report often recognizes the limits of its own strategies. Participants acknowledge that AI can generate reflections, critique its own output, produce intermediate deliverables, and complete tasks that recently seemed beyond its capabilities. My objection is that acknowledging these limitations does not resolve them. A proposal does not become a solution simply because we have accurately described why it might fail.

I kept coming back to three questions, and I would like the discussion to distinguish them more consistently:

1. What is worth learning? Independent problem-solving, understanding existing solutions, evaluating outputs, and working with AI are possible objectives. They are not interchangeable.
2. What activities help students learn it? Open-ended projects, AI-assisted experiments, worked examples, reflection, and critique may all have a role.
3. What evidence establishes that a student learned it? The value of an activity does not automatically make its deliverables valid evidence of individual achievement.

Much of my frustration comes from proposals that answer the second question and are then discussed as though they answer the third.

The first question matters as well. Solving a problem, understanding a supplied solution, explaining that solution, and directing AI to produce a solution are different accomplishments. All may be worth teaching. But if we switch from one to another, we have to say that we have changed what we are assessing.

With those questions in mind, I did not find a solution to the central assessment problem in the strategies presented. I found proposals to change what we assess, useful ideas for learning activities and grading, and several places where the report falls back on controlled assessment. Those are worth discussing. But they do not establish that redesigned take-home work provides reliable evidence of what a student knows or can do. I have grouped the report's seven strategies along those lines instead of following the report's order.

Changing What We Assess

The first three strategies change the task students are given. Each may be a reasonable answer to the first question, about what is worth learning. But a new task does not, by itself, tell us whether the student or the AI did the work.

Open-Ended & Authentic Task Design

The proposal: Give students a goal to demonstrate rather than a tightly specified task to complete. Have them decide what to build, formulate requirements, and make design choices.

There are good educational reasons to do this. Deciding what problem to solve can itself be important work.

But what makes that work the student's? If students can ask AI to implement a specification, they can also ask it to propose the specification, select a project, identify requirements, compare alternatives, and explain the resulting choices. Moving the work upstream does not establish that the student performed it.

There seem to be two possible claims here, and they need to be separated.

• If the claim is that an open-ended task is too difficult for AI to complete, that depends on current model capabilities. The report itself recognizes how quickly AI gets past those limitations. In Part III, a cybersecurity instructor describes problems that took real problem-solving with AI assistance seven or eight months earlier, and that AI now simply solves.
• If the claim is that deciding what to build is worth learning even when AI can do it, I agree. But that gives us a learning objective. It does not give us a way to assess whether the student achieved it.

"The student is responsible for the specification" is an assignment instruction. It is not evidence that the student developed the specification.

Ambitious, AI-Leveraged Projects

The proposal: Assign work beyond what students could complete unaided, treating AI collaboration as a premise of the assignment.

This can make sense as a learning activity. An advanced course may require unfamiliar languages, infrastructure, or tooling that would otherwise consume most of the semester. AI could help students implement an idea from a research paper, explore an existing codebase, or run experiments they could not reasonably set up alone.

But making those activities possible does not answer the assessment question. What are we grading?

If we grade the resulting system, we are grading an output produced with AI. That output may reflect the student's understanding, the capability of the tool, or some combination we have not established. If we grade a report explaining the system, AI can produce that too.

If we bring the student in to answer questions under controlled conditions, we have a separate source of evidence. Good. But then we need to say that the project provides an opportunity to learn, while the controlled assessment establishes particular aspects of what the student learned.

As the report describes it, this strategy makes assignments larger than either the student or the model could manage alone. I do not think there is such a thing as an assignment too large for AI to complete on its own. Even if there is one today, it will not stay that way for long. One facilitator in the report estimated that anything built to resist AI has a shelf life of roughly one semester, and that is a generous estimate. Making something larger also does not guarantee meaningful human participation.

There is another legitimate objective here: teaching students to work effectively with AI. Articulating goals, delegating work, evaluating results, and managing an agent's progress are skills worth examining. But assigning an AI-assisted project does not establish that students exercised those skills. We still need evidence of their contribution, and an instruction to collaborate meaningfully is not that evidence.

Evaluating & Fixing AI-Produced Work

The proposal: Have students read, test, critique, debug, and repair AI-generated work rather than produce everything from scratch.

Fine. If evaluating and fixing code is what we want to teach, give students code to evaluate and fix.

But what makes this an answer to the assessment problem? If they do it at home, AI can perform the evaluation and fixing too. The report acknowledges this and returns to in-class and proctored work, debates, oral presentations, and whiteboard exams.

So we have changed the task from writing code to reviewing code, but we have not solved the problem of knowing whether the student did either.

There is also a change in the learning objective that needs to be defended on its own terms. Understanding a solution is different from developing one. I can understand Dijkstra's shortest path algorithm and explain it clearly without having invented it. That does not make understanding unimportant. It means that being able to explain an idea is not the same as being able to come up with it.

The same distinction applies at the scale of a homework problem. A student may understand a solution they could not have produced independently. Whether that is sufficient depends on what the course intends to teach.

Some approaches in the report have students prompt AI for the code themselves before critiquing it. If understanding supplied solutions is the objective, why require students to obtain those solutions through AI in the first place? We can give them worked solutions, ask them to study and critique them, allow AI as a study aid, and then assess their understanding. Besides, an AI solution may use strategies we have not taught, or strategies we do not want students to use. If we provide the solution, we show students exactly how we want the problem solved. The AI-generation step needs an educational purpose of its own.

Changing What Students Submit and How It Is Graded

The next two strategies keep the take-home work and change what is collected and what receives credit. These are mostly answers to the second question, about activities that help students learn. The report discusses them as though they also answer the third, about ensuring the integrity of the assessment.

Process Evidence & Effort Signals

The proposal: Collect interaction logs, drafts, commit histories, reflections, and intermediate deliverables to assess how students worked rather than only what they produced.

What process are we assessing? The process the student actually followed, or the account of a process they submitted?

These are not the same thing. A reflection is another artifact. One participant proposed submitting the reflection as video to make it harder to generate, but a video is also another artifact. The facilitator who reported that proposal had tried video deliverables and found students using AI to script them. A sequence of intermediate submissions can also be produced with AI. If the original problem is that an artifact no longer establishes what the student did, asking for more artifacts does not by itself repair that problem.

Breaking a project into weekly deliverables may help students manage their time. Requiring reflection may help them learn. Asking for interaction logs may give an instructor useful information. These are reasons to consider those practices. They are not sufficient reasons to treat the resulting submissions as reliable evidence for a grade.

The extra work matters too. One deliverable becomes several, each requiring attention, feedback, or grading. What additional claim about student ability does that work allow us to make, and how confidently?

The report is particularly candid here: participants value process evidence while expressing substantial doubt about its trustworthiness. That is an important finding. But it does not solve the assessment problem.

Rubric & Grading Redesign

The proposal: Change what receives credit, shift weight from product to process, and use approaches such as mastery-based or specifications-based grading.

These approaches organize grading. They do not establish who performed the work.

Giving students several opportunities to demonstrate understanding is useful only if the demonstrations establish understanding. Otherwise, we have given them several opportunities to submit something AI produced.

Likewise, in specifications-based grading, a detailed specification can make expectations clearer, but it can also make the task easier to delegate. The report notes this risk itself: students may submit the rubric directly to an AI system. A process-focused rubric has the same problem if AI can generate a convincing account of the process.

Before discussing how many attempts students get or how much weight each component carries, answer a more basic question: what makes this evidence of the student's ability?

There is a legitimate curricular decision here. If AI literacy, evaluation, communication, or managing AI-assisted work are learning objectives, name them and assess them. But that is a change in what the grade represents. It should not be presented as though we have simply found a new way to measure the same abilities as before.

The rubric ideas in the report also add written and spoken work: reflections, documented design decisions, and oral explanations of code. These add demands on writing and communication. Those may belong in the course. If they do, assess them deliberately. If they do not, we need to explain how their influence on the grade is justified.

Changing the Conditions of Assessment

The last two strategies address the third question directly. They change the conditions under which the student performs, so the instructor can observe the performance instead of receiving an account of it.

Controlled & AI-Free Assessment

The proposal: Assess students under conditions that exclude AI, using supervised work, examinations, or other controlled settings.

This is the clearest response in the report to the question of what a student can do independently. If independent performance is the objective, observing independent performance provides relevant evidence.

It still requires a well-designed assessment. An exam measures what students can do on the tasks provided, within the time and conditions provided. It does not automatically establish every ability we care about, especially extended design, collaboration, or sustained project work.

But this strategy at least addresses the attribution problem directly: knowing whether the student or the AI did the work. When asked how the instructor knows the student did the work, many of the other strategies eventually fall back on controlled assessment.

That should be stated plainly. If take-home assignments provide practice and controlled assessments carry the evidence for individual achievement, that is a coherent arrangement. We should evaluate it on those terms.

We should also separate controlled assessment from supposedly AI-resistant tasks. The report discusses both in the same section: alongside supervised work, participants describe larger projects and higher-order thinking questions that a model would still fail. A task performed without access to AI is different from a task we hope AI cannot solve. The first controls a condition of performance. The second depends on a claim about what AI cannot do, and that claim may stop being true.

Oral & Interactive Assessment

The proposal: Ask students to explain their work, justify decisions, answer questions, or modify their solutions in real time.

This can establish something useful. But be precise about what.

A student who can explain a solution has demonstrated that they can explain that solution under those conditions. That does not show that they could independently produce it, or that they followed the required process when preparing the submission.

If we ask them to solve a new problem, extend the code, or respond to an unfamiliar requirement, we are assessing additional abilities. Good. Name those abilities and design the assessment around them.

The oral assessment does not retroactively make the homework evidence of independent problem-solving. It provides its own evidence, with its own scope and limitations.

This is still substantially better than accepting a take-home artifact and hoping for the best. Requiring students to understand and adapt what they submit creates a meaningful expectation. My objection is to claiming that the assessment shows more than it does.

We also need to justify the format. If the objective can be assessed through a supervised written or practical exam, why require individual oral examinations? There may be good reasons: follow-up questions, individualized probing, or communication as an explicit learning objective. But those benefits need to justify the time, staffing, and accessibility demands.

If a recommendation requires more time than instructors have, it is not workable as it stands.

Conclusion

Whether the objective is independent performance or performance with AI, the evidence has to come from a controlled setting. For independent performance, that means a supervised assessment without AI. For performance with AI, it still means bringing students in and proctoring them, but this time with access to AI in a common tool environment. That could reduce differences in access and make the student's interaction with AI observable. It would still require us to define what competent performance means. Deliberately limiting the model's capabilities might help expose particular skills, but we would need to justify those limits and what performance under them tells us.

I do not expect a workshop report to settle all of this. Its record of uncertainty is useful. But I came away unconvinced that the proposed assignment redesigns solve the assessment problem. Before asking instructors to collect more artifacts, write more rubrics, or conduct more interviews, we need a clear account of what each addition allows us to conclude about the student, and why that conclusion is warranted.