Fraser Page/GRACE lets students make submissions to update their own data. Student submissions get stored as diffs on their own model, and merge to their target once an adviser approves the request.
It's a simple framework with a fair bit of complexity in its implementation: open and rejected requests need distinct form states, requests to create records appear to students as actual records but with a pending-approval label, different endpoints for different data types, and so on.
Our initial AI tool design explained the submission workflow and handed over all of the student's data — existing records, submissions to change those records, the status of each submission, and a separate list of open requests to create new records. The thesis was: we're using a smart model, give it all the info and let it run.
Actual use showed consistent confusion on the model's part. It struggled to make sense of everything we had dumped on it, and to explain any of it succinctly to students. It would mix up the status of a submission with the status of a college application or say “some updates are pending” when the details of those updates were what the student needed to know. It mix up the id of a submission with the id of a record and create a new submission when the user intent was to edit an existing one. It would totally ignore create requests.
Our UI shows only what a student needs to see — if they've requested a change, we don't show them the old value; requests to add new records look as much like real records as possible. We were handing the model complexity that our UI abstracted away. The mistake was thinking of the model context as analogous to our back end. The realization I came to is that model context is its UI.
I rebuilt the agent surface to mirror the UI: don't explain inner workings that aren't surfaced to the student, handle the complexity in code in the name of a simple surface, and let the agent spend its attention on its prime directive rather than on decoding our data model.
The new agent was much more effective. Confusion went away. It more accurately and consistently summarized the student's data in ways that made sense. And our simplifications reduced the payload size and cost per call.