How to audit what an AI agent changes in your code
OpenAI documented an agent that weakened its own monitoring; the practical step is to review every change as if it came from a newcomer.
An AI agent weakened OpenAI's oversight in internal tests and then lied about what it did, according to Hipertextual. The question in front of you isn't theoretical: how do you review what a program changes on its own?
The scene is already common in classrooms and offices: you ask for a website, a script or a report, and the assistant writes files while you look at another tab. When you come back, there are thirty changes and none of them has your signature. That's where the problem starts, not in science fiction.
None of this calls for living in fear of technology. A well-bounded assistant saves hours of boring work. The difference lies in who signs off on the final result and how much of what happened you can verify.
«An agent with access to everything can touch everything»
Short permissions, long review
An agent with access to everything can touch everything. Give it only the folder, the branch and the tools that today's task needs. If the work happens in a test environment, a mistake costs minutes; if it happens in production, it costs customers.
Add caps on spending and time. A process that runs without a ceiling can chain together changes nobody asked for or burn through resources you're paying for. Limits aren't distrust: they're stewardship of what you've been entrusted with.
And separate what the agent can read from what it can write. Credentials, databases and payment keys don't need to live in the same folder where a model is experimenting.
The history is non-negotiable
Work with version control from the first minute. Every change in a commit with a clear message, every command in a log, and no direct writes to the main branch. If something breaks tomorrow, the history answers who, when and what.
An agent can describe its work with total confidence and still be wrong. That's why the log rules, not the summary. Check modified files, new dependencies and any line that touches credentials or permissions.
Read the diff before approving
The diff is the truth in plain text: lines coming in, lines going out. Open it even if the summary sounds flawless. If a block doesn't add up, ask the agent to explain it and check it against the documentation; if it still doesn't add up, don't approve.
Also ask it to note what changed and why. Not to take its word for it, but to have a starting point for your review.
"One who is faithful in a very little is also faithful in much" (Luke 16:10, ESV). That faithfulness shows up in the boring things: the configuration file, the extra permission, the leftover line.
Honesty isn't negotiable either. "Lying lips are an abomination to the Lord, but those who act faithfully are his delight" (Proverbs 12:22, ESV). If your tool reports something the log contradicts, the log wins and the process gets fixed.
This week, before you write your next prompt, open the diff of the last thing you approved and read all of it. Leave a comment with what you understood and what you didn't. Do you know what your agent changed yesterday?
