Audit Trail Logs for AI Agents: What to Record So You Can Investigate Later

For anyone who hands work to an AI agent, beginners included. You need no background in monitoring or security. Every term comes with an everyday picture, and the code is marked so you can skip it for now.

Numbers come from public sources listed at the end. Examples marked "illustration" are reconstructed and name no real person or company. The worked example is a simulation I wrote in Python, not the log of a commercial tool. How an AI decides what to do inside is not fully understood by anyone, so the pictures below are memory aids only.

---

Quick summary#

• When you hand work to an agent and go to a meeting, you do not see each step. Without a trail, when something goes wrong the only option is to reread a long conversation and guess.
• An action trail is one line per thing the agent did: when, which tool or command, what result, which files changed. One JSON record per line (the JSON Lines format) can be searched with a few lines of code or the jq command.
• Record just enough. No secrets, no personal data, no whole file contents. Hide secrets before writing, because a log that leaks a key is as dangerous as code that leaks a key.
• The recording layer should sit outside the agent and accept appended lines only. A log the agent can edit is no longer evidence.
• In the worked example, 25 actions produced a 7,170 byte file. The three investigation questions (timeline, first failure, changed files) are answered as soon as the script runs, and an edited line is detected.
• A trail shows what happened, not whether it was right. It complements asking for evidence and testing, it does not replace them.

---

After this post you can explain#

1. Why a session nobody watched still needs a trail, and how a trail differs from a saved conversation.
2. Which fields each log line should have, and what must never be recorded.
3. How to use a trail after an incident: rebuild the timeline, find the first failure, list the changed files and detect an edited line.

---

Hard words, plain meaning#

---

1. A picture first: shop CCTV and a receipt book#

Imagine you open a shop and hire a very quick employee. Today you are away. When you return, the till is short by 200 thousand. If nothing recorded the day, all you can do is ask the employee, who can answer perfectly honestly and still remember wrongly.

Now imagine the shop has two things. A camera records what happens at the counter. A receipt book records each sale: what time, what was sold, how much was taken. You do not need to watch a whole day of video. You open the book, find the sale at 3 pm and see the gap at once. Aircraft work the same way: the flight recorder keeps the pilots' actions so that after an accident people can rebuild what happened even when nobody is left to ask.

An agent works like that quick employee: it runs dozens of commands in a few minutes. The conversation is the story it tells. The action trail is the receipt book: it records what was done, not what was said.

Takeaway: a trail is not for catching the employee. It lets you find the cause in minutes instead of hours of guessing.

---

2. Why a session nobody rewatches needs a trail#

The more work you delegate, the fewer steps you watch. The landing page of Anthropic's 2026 agentic coding trends report says engineering leaders are balancing productivity against oversight, quality and security, and lists multi-agent coordination among its topics. I could read only the landing page. The full report sits behind a sign-up form, so I have not read the body of the report and I draw nothing further from it.

Another number: the Stack Overflow 2026 survey reports that 69.9% of respondents ask an AI agent when they face a new challenge (a figure I read through a summary and did not check against each table). Agents are part of the daily routine, and that routine often runs while you are not looking.

Two kinds of loss follow. First: an incident happens and nobody knows which commands the agent ran, so the investigation is slow. Second: no incident, but a colleague asks "why was the test file changed?" and you reread a long conversation to answer. The post Make AI Output Prove Itself asks for evidence inside the agent's answer. A trail adds the rest: evidence outside the answer, written by another layer that the agent cannot write.

Takeaway: more delegation needs more trail. A trail is cheaper than rereading.

---

3. What to record and what not to#

3.1 Seven groups of information#

Picture a receipt: who sold, what, when, how much, under which order.

Add a prev field (the fingerprint of the previous line) if you want to detect edited lines, as section 5 does.

3.2 Three things never to record#

1. Secrets and passwords. Hide them before writing. Logs are usually read by more people than a config file, so a key in a log travels further than a key in .env.
2. Users' personal data. Record only masked identifiers or internal ids.
3. Whole file contents or long command output. Record a summary (number of lines, number of passing tests) and where to look it up, not the content. The log stays small and carries less risk.

If your team already has a one-page AI use policy, put these three rules in its "forbidden" column and set a review date for how long logs are kept.

3.3 What one line looks like#

{"seq": 12, "ts": "2026-10-31T09:01:27+07:00", "session": "s-001",
 "actor": "agent", "request": "R2: make the tests pass",
 "tool": "run_command", "args": {"cmd": "python -m pytest -q"},
 "exit_code": 1, "summary": "ran 20 tests, 18 passed, 2 failed",
 "files_changed": [], "prev": "5d5c9ffa0389"}

(In a real file each record sits on a single line. It is wrapped here for reading.)

Takeaway: the seven groups answer the five investigation questions: who, what, when, why, how far. The three prohibitions keep the notebook from becoming a new leak.

---

4. The mechanism: four links#

Picture a production line: action, recording layer, storage, query.

1. The agent acts. It runs a command, edits a file, calls a tool.
2. A recording layer captures the action. This layer lives in the tool or in the harness that runs the agent, not in the prompt. If you only tell the agent "remember to log yourself", it may forget or log wrongly.
3. Hide secrets, then write to an append-only place. Add lines, never reopen old lines to edit.
4. Query when needed. Read with a script or jq.

4.1 Does your tool have a place to attach#

According to the current hooks documentation of Claude Code (I opened the page), the tool lets you attach a command that runs before or after each tool call. The command receives a JSON object on standard input (stdin). The fields the documentation lists include session_id, transcript_path, cwd, hook_event_name, and for the after-call event also tool_name, tool_input, tool_response, tool_use_id and an optional duration_ms. A separate event covers a tool that fails, with an error field. The documentation also says a hook that exits with code 0 and prints no decision does not change the tool call, while exit code 2 blocks it, so never use exit code 2 for logging.

About session transcripts: the example paths in the documentation end in .jsonl, so transcripts are stored as JSON Lines. The documentation does not state the format in words, so this is inferred from the examples. It also says the transcript file may be written with a delay.

Here is a logging script I wrote. It uses jq to pick the fields and hide keys:

#!/bin/sh
# Append one redacted JSON line per tool call. Always exit 0.
jq -c '{
  ts: (now | todate),
  session: .session_id,
  event: .hook_event_name,
  tool: .tool_name,
  input: (.tool_input | tostring | gsub("sk-[A-Za-z0-9]{8,}"; "[REDACTED]")),
  ms: .duration_ms
}' >> "${TRAIL_FILE:-hook-trail.jsonl}"
exit 0

I ran this script with a fake JSON object shaped like the documentation, and it wrote one line, with the fake key sk-test1234567890abcd replaced by [REDACTED]. I have not run it inside a real Claude Code session. The syntax for declaring a hook (the matcher key and where it goes in the settings file) varies between versions, so read the documentation for your version before attaching it.

4.2 A shared standard: OpenTelemetry#

If you prefer a standard to naming fields yourself, the OpenTelemetry project has naming conventions for generative AI work, including agent actions and tool calls. According to the summaries I read, the set is still at the "Development" stage (not stable), so attribute names may change between versions. The official page I opened only says the conventions moved to a separate repository, so the "Development" status comes from secondary sources, please check the official repository. For a beginner, starting with your own JSON Lines names and migrating later is fine.

Takeaway: the recording layer sits outside the agent, writes after each action and hides secrets before storing.

---

5. Worked example run for real: 25 actions, one failure, one fake key#

This is a simulated agent session (Python 3.14.8, jq 1.7.1) that I wrote. The actions are scripted data, but the recording layer, the redaction, the hash chain and the queries are real code that ran, and the results below are real output. Both full scripts are in the appendix at the end.

The scenario (illustration): the agent receives two requests in a row. R1 is to add a 10 percent discount to checkout. R2 is to make the tests pass. During the session the agent reads a .env file containing a fake key (API_KEY=sk-test1234567890abcd), runs the tests once with a failure (action 12), fixes the code, reruns the tests successfully, then commits twice.

5.1 The fake key was hidden#

Action 7 reads .env. The recorded line:

{"actor": "agent", "args": {"path": ".env"}, "exit_code": 0, "files_changed": [], "prev": "2efa822af966", "request": "R1: add a 10 percent discount to checkout", "seq": 7, "session": "s-001", "summary": "loaded [REDACTED] (3 variables)", "tool": "read_file", "ts": "2026-10-31T09:00:13+07:00"}

Searching the whole file for sk-test finds 0 matching lines. The redaction is a simple regular expression, so it only catches the key shapes you predicted. Treat it as the first line of defence, not the only one.

A bug I hit while writing this post: my first version turned the arguments into a JSON string and only then hid secrets. When I tried a command containing password=abc123, the \S+ pattern swallowed the closing quote and brace, truncating the line and making the query script fail while reading it. The fix is to hide each string value before turning it into JSON, as the redact function in the scripts below does.

5.2 The timeline#

== timeline ==
 1 09:00:03 read_file    exit=0 src/checkout.py
 2 09:00:05 read_file    exit=0 src/pricing.py
 3 09:00:07 search       exit=0 discount
 4 09:00:09 read_file    exit=0 tests/test_discount.py
 5 09:00:10 run_command  exit=0 git status --short
 6 09:00:11 run_command  exit=0 git switch -c agent/discount
 7 09:00:13 read_file    exit=0 .env
 8 09:00:33 edit_file    exit=0 src/pricing.py
 9 09:00:45 edit_file    exit=0 src/checkout.py
10 09:00:54 run_command  exit=0 python -m pytest -q
11 09:01:19 edit_file    exit=0 tests/test_discount.py
12 09:01:27 run_command  exit=1 python -m pytest -q
13 09:01:29 read_file    exit=0 tests/test_discount.py
14 09:01:44 edit_file    exit=0 src/pricing.py
15 09:01:52 run_command  exit=0 python -m pytest -q
16 09:01:53 run_command  exit=0 git diff --stat
17 09:01:54 run_command  exit=0 git add -A
18 09:01:56 run_command  exit=0 git commit -m 'feat: add 10 percent discount'
19 09:01:58 read_file    exit=0 README.md
20 09:02:08 edit_file    exit=0 README.md
21 09:02:09 run_command  exit=0 git add README.md
22 09:02:11 run_command  exit=0 git commit -m 'docs: mention discount'
23 09:02:19 run_command  exit=0 python -m pytest -q
24 09:02:20 run_command  exit=0 git log --oneline -3
25 09:02:24 final_answer exit=0 -

5.3 Three investigation questions#

== first failure ==
12 2026-10-31T09:01:27+07:00 run_command ran 20 tests, 18 passed, 2 failed
== files changed ==
README.md: 1 edit(s)
src/checkout.py: 1 edit(s)
src/pricing.py: 2 edit(s)
tests/test_discount.py: 1 edit(s)
== edits to test files ==
11 R2: make the tests pass ['tests/test_discount.py']

Reading it: the first failure is action 12 at 09:01:27. Just before it (action 11) the agent added 22 lines to the test file, and after the failure it edited src/pricing.py (action 14) rather than editing the test file again. In other words the trail shows the agent fixed the code to make the tests pass, not the tests to match the code. If action 14 had been an edit to a test file, that would be the place to look closely.

The same questions with jq (no Python needed):

jq -r 'select(.exit_code != 0) | "\(.seq) \(.ts) \(.summary)"' trail.jsonl
# 12 2026-10-31T09:01:27+07:00 ran 20 tests, 18 passed, 2 failed

jq -r '.files_changed[]' trail.jsonl | sort | uniq -c
#    1 README.md
#    1 src/checkout.py
#    2 src/pricing.py
#    1 tests/test_discount.py

5.4 Detecting an edited line#

I copied the log file and changed the exit code on line 12 from 1 to 0 to "erase" the failure. Then I reran the chain check:

== chain check ==
line 13: chain broken, an earlier line was changed
chain NOT ok
== first failure ==
none found

Line 13 carries the fingerprint of the original line 12, so once line 12 changed the fingerprint no longer matched. The "none found" under first failure shows that if you skip the chain check you would conclude the session ran clean.

An honest limit: a hash chain only detects, it does not prevent. Someone who edits the log can recompute the whole chain from the edited line onward. For the chain to mean anything you must keep the fingerprint of the last line somewhere the editor cannot reach (another service, an append-only system, or a second person).

---

6. A storage calculation (illustration, skippable)#

The 25-line file weighs 7,170 bytes, about 286.8 bytes per line. Suppose a team has 200 sessions a day with 100 actions each (assumed numbers for the calculation):

per_line = 7170 / 25               # 286.8 bytes
per_day = 200 * 100 * per_line     # 5,736,000 bytes
print(round(per_day / 1e6, 2), round(per_day * 30 / 1e6, 1))
# 5.74 172.1   (MB per day, MB per 30 days)

About 5.74 MB a day and 172.1 MB for 30 days. Disk space is not the big problem for summary logs. The bigger questions are who may read them and how long to keep them. The 200 and 100 are my assumptions, while 286.8 bytes was measured on this simulation. A real log with longer output will be much heavier, which is why you do not store whole contents.

---

7. Reading the trail back after something went wrong#

Picture an investigator opening the receipt book. Five questions in order:

1. Is the book intact? Check the hash chain or compare with a copy kept elsewhere.
2. When did it start? Find the first failure and look at the actions just before it.
3. Who or what gave the order? Look at the request and actor fields.
4. How far did it reach? List changed files and commands with outside effects (commit, push, delete).
5. What stops it happening again? That lives outside the log: write an incident review. The story of the day an engineer deleted the wrong data at GitLab in 2017 is one example of how a team told the story of an event after it happened.

A trail helps with questions 1 to 4. Question 5 is human work.

---

8. Where trails break#

Picture a notebook that can have missing pages, wrong entries or erasures.

---

9. Illustration: one morning, two versions#

This is an illustrated situation that names no real person or product.

Without an action trail:

• 09:00. A dev gives the agent two jobs and goes into a meeting until 11:30.
• 11:35. The dev sees two commits on the branch. A colleague asks: "Why was the test file changed?"
• 11:40. The dev opens the long conversation and scrolls. The agent said "fixed" but not which files in which order.
• 12:20. The dev guesses an answer, unsure.

With an action trail:

• 09:00. Same two jobs, the recording layer runs outside the agent.
• 11:35. The colleague asks the same question.
• 11:37. The dev queries "test files edited": action 11 at 09:01:19 added new tests, action 14 edited the code, not the tests.
• 11:40. The answer goes back with action numbers attached.

What to do in both: tie the answer to a numbered action, not to memory. This is an illustration, so it proves nothing about frequency. The evidence is in the sources at the end.

---

10. Hands-on exercise (with answers)#

Run the two scripts from the appendix, then do four things.

1. How many run_command actions failed, and at which action number?
2. Which file was edited more than once?
3. Add two actions at the end of the steps list: a command echo password=abc123 and a command echo pwd=abc123. Rerun and count the lines in the file that contain abc123.
4. Edit one line in the middle of the file by hand and run the chain check. Which line is reported broken?

Answers:

1. One command failed, at action 12.
2. src/pricing.py, edited twice (actions 8 and 14).
3. Exactly one line contains abc123. The password=abc123 command was hidden as echo [REDACTED], while pwd=abc123 slipped through because the pattern has no pwd word (I ran this, and the result was 1 line). That is the lesson about the limits of hiding with a fixed pattern.
4. The line right after the one you edited, because it holds the fingerprint of the original line.

---

11. Sources and numbers used#

Reading it correctly: I did not find a study measuring how much an action trail shortens investigation time. The arguments in this post are good practice derived from how traces work, not experimental results.

---

12. Checklist: a trail for your agent#

1. Is there a recording layer outside the agent, or did you only tell the agent to log itself?
2. Does each line have time, session, actor, original request, tool, result and changed files?
3. Are secrets hidden before writing, and have you tested with a fake key?
4. Are you recording personal data or whole file contents?
5. Is the log append-only, or can the agent edit it?
6. Have you stored the last fingerprint or a copy somewhere else?
7. Do you have sample queries for three questions: timeline, first failure, changed files?
8. Who may read it, how long is it kept, and when is the review date?

---

13. A balanced view: what a trail does and does not do#

It does: shorten rebuilding an event, answer "what did the agent do at what time", give a colleague a numbered basis, and expose an edited line if you check the chain.

It does not:

• A trail does not say whether a result is right. A successful command can still do the wrong thing. For correctness you still need evidence handed in with the answer as in Make AI Output Prove Itself, tests and a human reader.
• It does not prevent damage. It only helps you know afterwards. For risky work you also need restricted permissions and a way back, like the save points in Git for People Who Use Agents.
• Hiding secrets with a fixed pattern misses key shapes you have not thought of.
• A trail costs storage and needs privacy care: who reads it and for how long.
• The worked example is only a simulation of one session. A real tool's trail has more fields and more exceptions.

Do not oversell it: having a trail does not mean being safe. Do not underrate it: when you need it most, during an incident, it is often the only thing left to rely on.

---

14. Conclusion#

Handing work to an agent and going to a meeting is normal. What is not normal is coming back and not knowing what it did. Give it a receipt book written by another layer: record just enough, hide secrets, allow appending only and know how to read it back. A trail does not replace evidence and tests, but it turns an afternoon of investigation into a few minutes.

---

Appendix: the two full scripts (skippable)#

agent_trail_demo.py, simulates a session and writes the trail:

import hashlib
import json
import re
import sys
from datetime import datetime, timedelta, timezone
from pathlib import Path

LOG = Path(sys.argv[1] if len(sys.argv) > 1 else "trail.jsonl")
SECRET = re.compile(r"sk-[A-Za-z0-9]{8,}|(password|token|api_key)\s*[=:]\s*\S+", re.I)


def redact(value):
    """Hide secrets in every string inside a value, before it is turned into JSON."""
    if isinstance(value, str):
        return SECRET.sub("[REDACTED]", value)
    if isinstance(value, dict):
        return {k: redact(v) for k, v in value.items()}
    return value


class Trail:
    """Append-only JSON Lines trail. Each line carries the hash of the previous line."""

    def __init__(self, path, session):
        self.path, self.session, self.prev, self.seq = path, session, "start", 0
        self.clock = datetime(2026, 10, 31, 9, 0, 0, tzinfo=timezone(timedelta(hours=7)))
        path.write_text("")

    def record(self, request, tool, args, exit_code, summary, files=(), seconds=2):
        self.clock += timedelta(seconds=seconds)
        self.seq += 1
        line = {
            "seq": self.seq,
            "ts": self.clock.isoformat(),
            "session": self.session,
            "actor": "agent",
            "request": request,
            "tool": tool,
            "args": redact(args),
            "exit_code": exit_code,
            "summary": redact(summary),
            "files_changed": list(files),
            "prev": self.prev,
        }
        text = json.dumps(line, ensure_ascii=False, sort_keys=True)
        self.prev = hashlib.sha256(text.encode()).hexdigest()[:12]
        with self.path.open("a") as f:
            f.write(text + "\n")


trail = Trail(LOG, "s-001")
R1 = "R1: add a 10 percent discount to checkout"
R2 = "R2: make the tests pass"
steps = [
    (R1, "read_file", {"path": "src/checkout.py"}, 0, "120 lines", (), 3),
    (R1, "read_file", {"path": "src/pricing.py"}, 0, "80 lines", (), 2),
    (R1, "search", {"pattern": "discount"}, 0, "4 matches in 2 files", (), 2),
    (R1, "read_file", {"path": "tests/test_discount.py"}, 0, "45 lines", (), 2),
    (R1, "run_command", {"cmd": "git status --short"}, 0, "clean", (), 1),
    (R1, "run_command", {"cmd": "git switch -c agent/discount"}, 0, "switched to new branch", (), 1),
    (R1, "read_file", {"path": ".env"}, 0, "loaded API_KEY=sk-test1234567890abcd (3 variables)", (), 2),
    (R1, "edit_file", {"path": "src/pricing.py"}, 0, "+14 -2 lines", ("src/pricing.py",), 20),
    (R1, "edit_file", {"path": "src/checkout.py"}, 0, "+6 -1 lines", ("src/checkout.py",), 12),
    (R1, "run_command", {"cmd": "python -m pytest -q"}, 0, "ran 18 tests, 18 passed", (), 9),
    (R2, "edit_file", {"path": "tests/test_discount.py"}, 0, "+22 -0 lines", ("tests/test_discount.py",), 25),
    (R2, "run_command", {"cmd": "python -m pytest -q"}, 1, "ran 20 tests, 18 passed, 2 failed", (), 8),
    (R2, "read_file", {"path": "tests/test_discount.py"}, 0, "67 lines", (), 2),
    (R2, "edit_file", {"path": "src/pricing.py"}, 0, "+3 -1 lines", ("src/pricing.py",), 15),
    (R2, "run_command", {"cmd": "python -m pytest -q"}, 0, "ran 20 tests, 20 passed", (), 8),
    (R2, "run_command", {"cmd": "git diff --stat"}, 0, "3 files changed, 45 insertions, 4 deletions", (), 1),
    (R2, "run_command", {"cmd": "git add -A"}, 0, "staged 3 files", (), 1),
    (R2, "run_command", {"cmd": "git commit -m 'feat: add 10 percent discount'"}, 0, "1 commit created", (), 2),
    (R2, "read_file", {"path": "README.md"}, 0, "30 lines", (), 2),
    (R2, "edit_file", {"path": "README.md"}, 0, "+5 -0 lines", ("README.md",), 10),
    (R2, "run_command", {"cmd": "git add README.md"}, 0, "staged 1 file", (), 1),
    (R2, "run_command", {"cmd": "git commit -m 'docs: mention discount'"}, 0, "1 commit created", (), 2),
    (R2, "run_command", {"cmd": "python -m pytest -q"}, 0, "ran 20 tests, 20 passed", (), 8),
    (R2, "run_command", {"cmd": "git log --oneline -3"}, 0, "3 commits listed", (), 1),
    (R2, "final_answer", {}, 0, "reported done, branch agent/discount", (), 4),
]
for step in steps:
    trail.record(*step)
print(f"wrote {trail.seq} lines to {LOG}")

query_trail.py, checks the chain and queries:

import hashlib
import json
import sys
from collections import Counter

path = sys.argv[1] if len(sys.argv) > 1 else "trail.jsonl"

print("== chain check ==")
prev, ok = "start", True
for i, line in enumerate(open(path), 1):
    if not line.strip():
        continue
    if json.loads(line)["prev"] != prev:
        print(f"line {i}: chain broken, an earlier line was changed")
        ok = False
        break
    prev = hashlib.sha256(line.rstrip("\n").encode()).hexdigest()[:12]
print("chain ok" if ok else "chain NOT ok")

rows = [json.loads(line) for line in open(path) if line.strip()]

print("== timeline ==")
for r in rows:
    args = r["args"]
    arg = args.get("cmd") or args.get("path") or args.get("pattern") or "-"
    print(f'{r["seq"]:>2} {r["ts"][11:19]} {r["tool"]:<12} exit={r["exit_code"]} {arg}')

print("== first failure ==")
first = next((r for r in rows if r["exit_code"] != 0), None)
print(f'{first["seq"]} {first["ts"]} {first["tool"]} {first["summary"]}' if first else "none found")

print("== files changed ==")
counts = Counter(f for r in rows for f in r["files_changed"])
for name, n in sorted(counts.items()):
    print(f"{name}: {n} edit(s)")

print("== edits to test files ==")
for r in rows:
    if any(f.startswith("tests/") for f in r["files_changed"]):
        print(r["seq"], r["request"], r["files_changed"])

Run: python3 agent_trail_demo.py trail.jsonl then python3 query_trail.py trail.jsonl. The simulated clock is fixed, so the fingerprints (the prev field) in your file will match mine if you use exactly this script.

---

References#

Numbers come from public sources as of October 2026. Some sources are secondary, so open the originals before quoting.

• Anthropic, 2026 Agentic Coding Trends Report (landing page, the full report is behind a form)
• Stack Overflow, 2026 Developer Survey (read through a summary, not checked table by table)
• Anthropic, Claude Code: Hooks reference (JSON fields, events, exit codes, transcript paths)
• OpenTelemetry, GenAI semantic conventions (page moved repositories) and secondary summaries of the Development status, for example Dash0
• jq, jq Manual (filters and gsub)
• Related posts on this site: Make AI Output Prove Itself, Git for People Who Use Agents, One-Page AI Use Policy for Small Teams