---
name: Agent Session Format Check
slug: agent-session-format-check
category: Quality
description: Agent Session Format Check verifies whether current Agent Sessions code still matches the latest agent session, storage, and usage formats. Use it when agent CLIs change, flags drift, or you need to confirm discovery paths, schemas, and usage probes.
github: "https://github.com/jazzyalex/agent-sessions/tree/main/skills/agent-session-format-check"
language: Swift
stars: 806
forks: 52
install: "npx degit https://github.com/jazzyalex/agent-sessions/tree/main/skills/agent-session-format-check ~/.claude/skills/agent-session-format-check"
installs_to: ~/.claude/skills/agent-session-format-check
source_path: skills/agent-session-format-check/SKILL.md
collection_size: 8
category_size: 1354
collection_url: "https://dirskills.com/collections/jazzyalex/agent-sessions"
added: 2026-08-22T05:22:13.346Z
last_synced: 2026-08-22T05:22:13.346Z
canonical_url: "https://dirskills.com/skills/agent-session-format-check"
---

# Agent Session Format Check

Agent Session Format Check verifies whether current Agent Sessions code still matches the latest agent session, storage, and usage formats. Use it when agent CLIs change, flags drift, or you need to confirm discovery paths, schemas, and usage probes.

**Install:**

```bash
npx degit https://github.com/jazzyalex/agent-sessions/tree/main/skills/agent-session-format-check ~/.claude/skills/agent-session-format-check
```

## README

# Agent Session Format Check

Answer one precise question for every supported agent:

> Can current Agent Sessions code support the latest available session/storage/usage
> format from the latest available agent build?

The answer must be layered. Do not collapse version checks, stale samples, schema
fingerprints, discovery contracts, and usage probes into a vague severity label.

**Evidence-first:**
- Gather a report + sample paths first.
- Do not change parsers/fixtures/docs without explicit user approval.

**Related skill:** `agent-support-matrix` — maintains the matrix YAML, ledger, and
update-checklist workflow. This skill focuses on *detection and evidence collection*;
`agent-support-matrix` focuses on *recording and gating version bumps*.

**Process doc:** `docs/agent-support/monitoring.md` — defines the compatibility verdicts,
legacy severity model, cadence, and escalation workflow that feed into this skill.

---

## 1  Quick Start (all agents)

1. Run weekly monitoring:
   ```
   ./scripts/agent_watch.py --mode weekly
   ```
   Report path prints to stdout and is written under
   `scripts/probe_scan_output/agent_watch/*/report.json`.

2. In `report.json`, check each agent under `results.<agent>`:
   - `compatibility.verdict`, `compatibility.scope`, `compatibility.blockers`,
     and `compatibility.next_action`
   - `verified_version`, `installed.parsed_version`, `upstream.parsed_version`
   - `compatibility.latest_status` to distinguish `current_fetch_known`,
     `cached_latest`, and unknown latest-source states
   - `weekly.local_schema` (newest local session used for fingerprinting)
   - `weekly.schema_diff` and `evidence.schema_matches_baseline`
   - `evidence.sample_freshness` and `evidence.fresh_evidence_source`
   - `compatibility.latest_real_session_failure` when a prebump attempt failed
   - `severity` and `recommendation` only as legacy escalation fields

3. **Usage / limits reading (Codex + Claude) — always verify every weekly run.**
   These drift independently of session schema (see §2), so a clean schema does
   **not** imply healthy usage reading. Each agent's
   `results.<agent>.weekly.probes` is a **list**; for every relevant entry confirm
   `ok == true` and `exit_code == 0`:
   - Codex — `label == "codex_status_probe"` (parse `codex_status_json`): the
     active CLI status channel (`five_hour`, `weekly` percent-left). The passive
     channel is the session JSONL `token_count` / `rate_limits` events, covered by
     the schema fingerprint above.
   - Claude — `label == "claude_usage_probe"` (parse `claude_usage_json`): the
     **authenticated** `/usage` reading (`session_5h`, `week_all_models`,
     `week_opus`). Also `label == "claude_status"` (parse `claude_status_json`):
     status.claude.com indicator/incidents.
   A failed or unparsed usage probe is a usage-format or auth regression even when
   versions match and the session schema is clean — never skip it, and report each
   probe's `ok` explicitly rather than collapsing it into the compatibility verdict.

4. **Classify every new field for value, not just for safety (§1e).** A field that parses
   cleanly is *safe*, which is not the same as *handled*. Do not close a drift finding
   until each new key or type has been called noise, watch, or feature-candidate — and
   candidates filed in `docs/backlog.md`, not left as a remark in the ledger.

Interpretation:
- `supports_latest`: latest known build is covered by
  `evidence.fresh_evidence_source == "latest_prebump_report"` and
  `compatibility.latest_real_session_evidence == true` with
  `compatibility.latest_status == "current_fetch_known"`.
- `supports_installed_only`: installed build is covered by non-stale real local
  evidence, but latest is newer, cached from a prior report, unknown, or lacks
  fresh real-session proof.
- `latest_unknown`: no configured/reachable latest source or no real-session
  driver exists; do not claim latest support.
- `blocked_stale_sample`: evidence predates the installed CLI; run prebump before claiming support.
- `blocked_no_fresh_evidence`: a version changed but no fresh matching sample proves support.
- `format_drift_detected`: unknown schema/storage/usage fields appeared; update fixtures/parsers.
- `blocked_thin_sample`: the sample was both narrow and tiny, so it evidenced nothing either
  way (§5a). Generate a session that actually uses tools — not a one-line prompt.
- `monitoring_broken`: latest source, usage probe, or discovery contract failed.
- `real_session_auth_failed` in blockers: the real-session driver ran but the
  sandboxed agent was not authenticated; re-auth or provide the configured env
  token, then rerun prebump.

---

## 1a  Real-Session Prebump Validation (required before latest claims)

Weekly scanning samples the newest on-disk session, which can predate a CLI
upgrade and give a false "safe to bump" call (the codex 0.120.0 trap and the
copilot `session.shutdown` trap). When weekly reports
`recommendation == run_prebump_validator` — or before you stage any
`max_verified_version` bump or latest-support claim — run the prebump path for
every active agent being claimed. The driver exercises the currently installed
CLI once inside a sandbox and diffs its output against the fixture baseline:

```
./scripts/agent_watch.py --mode prebump --agent codex --agent claude
```

Exit-code contract:
- `0` — every requested agent produced a fresh session and the schema
  matches baseline. Safe to bump.
- `2` — at least one fresh session's schema does **not** match baseline.
  Do **not** bump; investigate the schema diff in
  `scripts/probe_scan_output/agent_watch/<slug>-prebump/report.json`.
- `3` — at least one driver failed (timeout, auth, CLI not found, or
  discovery contract violation — wrong session root, wrong glob, or
  missing required event types).
- `4` — config/invariant error: unknown `--agent` (or one with no
  prebump block), missing/invalid `discover_session` contract,
  credential hygiene failure (oversize / mode), or sandbox breach
  (copilot hermeticity gate). Re-run with
  `--allow-real-home` only if you understand your real config dir will
  be mutated for that one invocation.

Flags:
- `--agent <name>` (repeatable) — restrict to specific agents. An
  unknown agent or one without a `prebump` config block exits 4.
- `--keep-sandbox` — preserve the temp `$HOME` for debugging.
- `--timeout-seconds N` — per-driver timeout. CLI flag overrides
  per-agent config; falls back to config, then global default (120s).
- `--force-fresh` — suppress staleness evaluation for this run only (records
  `stale_reason=forced_fresh` in the report).
- `--allow-real-home` — copilot/real-HOME opt-in after a sandbox-breach
  diagnostic; never persistent.

**A thin prebump can no longer downgrade a rich weekly union.** A passing prebump
*replaces* the weekly `schema_diff` with its own one-prompt session, and the thin-sample
gate used to score only that: on 2026-08-13 codex reported `blocked_thin_sample` off a
20-event prebump while its weekly union carried 2808 clean events — adding evidence made
the verdict worse. `_sample_is_thin()` is now applied to *every* available sample and only
blocks when all of them are thin (pinned by `test_thin_prebump_does_not_override_rich_weekly_union`).

**Give `real_home_session` agents a tool-using prompt.** Their prebump session lands in the
real store and enters the newest-`_LOCAL_SCHEMA_SAMPLE_COUNT` window, so a "Say hello"
prompt actively degrades the next weekly sample — two such runs pushed antigravity's union
down to 24 events and a genuine `blocked_thin_sample`. Claude's `"Say hi, then use the Bash
tool to run pwd."` is the pattern; verify any prompt change with a real run, since a
tool-using prompt can hang or return nothing on CLIs whose one-shot mode does not complete
a tool turn.

**A passing prebump is a floor, not a ceiling.** Drivers use one-line prompts, so a fresh
session may contain only the four most basic event types and still report
`fresh_matches_baseline=True` — it proves the CLI still writes parseable output, not that
rich event families are unchanged. For `real_home_session: true` agents that session also
lands in the real store and becomes the newest sample; §5a's multi-session union is what
stops it from masking drift. Check the fresh session's type count before treating a pass as
broad evidence.

Configured real-session drivers today are `codex`, `claude`, `antigravity`,
`copilot`, `opencode`, `hermes`, `openclaw`, `cursor`, `pi`, `kimi`, and `grok`.
Droid is legacy-only and excluded from active checks. Qwen has no driver — it
reports `no_real_session_driver_configured` and can never claim `supports_latest`;
judge it on the weekly schema diff instead. That is not a gap to fill with code:
Qwen's OAuth free tier was discontinued 2026-04-15, so no session can be generated
at all until a paid plan or alternate provider is configured (see §1c).

**Staleness short-circuits the schema verdict.** `blocked_stale_sample` is reported
*instead of* drift, so a stale agent can be hiding real drift behind it. Kimi sat at
`blocked_stale_sample` while every one of its sessions carried two unmodelled event
types; building its driver surfaced them immediately. Treat a stale verdict as
"unknown", never as "clean".

Prebump uses the hybrid env-var-first auth policy: if the relevant API-key
env var (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`,
`FACTORY_API_KEY`, `GITHUB_TOKEN`) is set it is forwarded into the sandbox
and real HOME is never read. Otherwise the driver copies the declared
credential file from real HOME into the sandbox after running three hygiene
gates (64 KiB max, mode `0600`, ≤90-day mtime warning). v1 drivers:
`codex_exec`, `claude_print`, `antigravity_print`, `copilot_prompt`,
`opencode_run`, `hermes_oneshot`, `openclaw_local_agent`,
`cursor_agent_print`, `pi_prompt`, and `kimi_prompt`. Some OAuth/keychain-backed CLIs use
`real_home_session: true`; run them with `--allow-real-home` so the session
lands in the real agent store instead of copying single-use auth state into a
sandbox.

---

## 1b  Claiming a Newer Upstream Build (install → fresh session → verify → bump)

**A version bump is never just a YAML edit.** `max_verified_version` claims that the
app parses sessions *written by that build*, and the only thing that can prove it is a
session that build actually wrote. So when `upstream.parsed_version` is newer than
`installed.parsed_version`, the sequence is fixed:

```
1. Install the newer CLI (its own updater, brew, or npm — whatever owns it).
2. ./scripts/agent_watch.py --mode prebump --agent <name>     # writes a FRESH session
3. Confirm exit 0 and fresh_matches_baseline == true.
4. Only now bump max_verified_version + ledger + tracking log (§6).
```

**Step 2 is not optional, and skipping it makes the report actively worse.** Installing
an update rewrites nothing on disk: the newest session is still the one the *old* build
wrote, so it is now older than the CLI binary and the agent flips from
`supports_installed_only` (a real claim, backed by real evidence) to
`blocked_stale_sample` (which per §1a means *unknown*, never *clean*). That is why the
2026-08-13 pass deliberately did **not** install Hermes 0.19.0: without a working
one-shot run to generate a fresh session, updating would have destroyed a verdict it
could not replace. Install only if you can complete step 2.

**An agent with no prebump driver cannot do this loop at all.** It is permanently capped
at `supports_installed_only`, however clean its weekly looks, because nothing can produce
a session from a build on demand. Build the driver first — `grok_single` (added
2026-08-17) is the worked example, and see §1d for what building one involves.

**Re-read the installed version after the run.** Several CLIs self-update *because* the
prebump invoked them — copilot, cursor and antigravity all did on 2026-08-13 — so the
build you validated may not be the build you started with. Bump to what the post-run
report says is installed, not to what you intended to install.

---

## 1c  When an Agent Cannot Produce a Session At All

Some agents are blocked for reasons no driver can fix, and the distinction matters
because it decides whether there is engineering work to do:

- **Fixable, ours** — a broken install. OpenCode sat at `installed=unknown` with an
  `Exec format error` that read like a driver bug; `/opt/homebrew/bin/opencode` was
  actually the npm package's *shell stub*, because its postinstall never ran. Run
  `file $(which <agent>)` before suspecting anything schema-shaped.
- **Fixable, theirs** — a broken agent config. OpenClaw and Hermes both fail on borrowed
  backends that are misconfigured in the user's own environment (§ledger 2026-08-13).
- **Not fixable by us** — the account or plan is gone. Qwen's OAuth free tier was
  discontinued **2026-04-15**; `qwen -p` returns "Run /auth to switch to Coding Plan,
  OpenRouter, Fireworks AI, or another provider," and `qwen auth` is itself "(removed)"
  in 0.21.x. No driver, re-login, or fixture can produce a session. Record the verdict
  honestly as `blocked_stale_sample`, say why in the matrix, and stop — it is a billing
  decision for the maintainer, not a task.

Do not let category three masquerade as category one. A missing driver looks like
engineering work right up until you try to authenticate.

---

## 1d  Building a Prebump Driver

Drivers live in `scripts/agent_watch_prebump_drivers.py` and register into `DRIVERS`;
their config block is `agents.<name>.prebump` in `agent-watch-config.json`. Copy the
nearest existing driver rather than starting from the Protocol.

Non-obvious requirements, each learned from a real failure:

- **Declare `discover_session` with the modern `roots`/`globs`/`required_types` keys.**
  The config gate rejects anything else with exit 4, and the runtime validator then
  proves the driver returned the artifact it claimed rather than some other file.
- **Check the agent is in `MATRIX_KEY_FOR_AGENT`** (`scripts/agent_watch.py`). A missing
  entry does not error: `baseline_paths` comes back empty, the diff takes its
  "no baseline → nothing diffs" branch, and the prebump reports
  `fresh_matches_baseline=true` **having compared nothing**. `grok` was missing from the
  prebump copy of that map on 2026-08-17 while present in the weekly copy, so its first
  driver would have passed vacuously. The map is now single-sourced and pinned by
  `test_every_monitored_agent_is_registered_in_the_rebuild_tool`.
- **Prefer an API-key env var; fall back to a credential file.** `prepare_auth` is the
  only auth path — never build env from `os.environ` inside a driver. Some agents have no
  key env var at all (grok's only documented one is `GROK_SANDBOX`), so the credential
  copy is their sole route and the 0600 hygiene gate always applies.
- **Give it a tool-using prompt**, and for `real_home_session: true` agents especially —
  their session lands in the real store and enters the newest-5 weekly window, so a
  "Say hello" prompt degrades next week's sample (§1a).
- **If a session is a directory, verify the sidecars before returning ok.** Grok's
  `summary.json` is a discovery precondition; returning a transcript whose sidecar never
  landed hands the fingerprinter half a session and reports the gap as schema drift.

Then verify the driver the way you would verify a finding: run it, confirm
`baseline_type_count` in the report is non-zero (proving a baseline was consulted), and
confirm the fresh session actually contains a tool call rather than four trivial events.

---

## 1e  The Value Pass (what new fields are *for*, not just whether they break us)

This check has a structural blind spot, and it is worth stating plainly: it asks only
*"does it still parse?"* Because the parsers read JSON as dictionaries, the answer is
almost always yes — an unknown key is never read, an unknown type falls to a `default:`
branch and becomes a `.meta` event. So every finding exits the funnel as "no parser
change needed," which is true and also the end of the thought. **Nothing ever asks
whether upstream just started telling us something a user would want to see.**

The cost is not hypothetical. Kimi's `turn.ended.durationMs` was noted in the
2026-08-13 ledger as "the natural source of a per-turn duration UI" and then sat there,
because a ledger note is a remark, not work. And Qwen shipped with
"usage/rate-limit tracking" listed as an *unsupported surface* while every one of its
transcripts carried full per-call token accounting — 53 records in one ordinary session.
The data was never missing. Nobody opened it.

**So classify every new key or type into exactly one of three buckets, and record which:**

| Bucket | Meaning | Action |
|---|---|---|
| **noise** | internal plumbing, ids, or telemetry nobody would look at | fixture only; say so once so it is not re-litigated |
| **watch** | meaningful but not actionable yet — a field that will matter if it starts appearing widely, or that only one source emits | fixture + a line in the ledger note |
| **candidate** | carries information a user would want on screen | fixture **+ an entry in `docs/backlog.md`** |

Cheap heuristics for spotting a candidate: it is a **number a user would ask about**
(tokens, cost, duration, context size), it **names something currently anonymous** (which
connector ran this tool, which model, which provenance), or it **records a state
transition the UI hides** (mode changes, rewinds, compaction).

Two rules that keep this honest:

- **Check the value before believing the matrix.** `unsupported_surfaces` describes what
  the *app* does, never what the *agent emits*. Qwen's entry made a present surface look
  absent for a full release. When a value pass contradicts a matrix line, the matrix line
  is the thing that is wrong.
- **Measure before promoting.** Claude's `budget_usd` carries real dollars and looked like
  a free replacement for the estimated-cost path — until counting showed it in 2 of 410
  recent sessions, because it only appears when a budget is set. Confirm how often a field
  actually occurs, and on which record types, before filing it as a candidate.

Candidates go in `docs/backlog.md` under the matching area section, using that file's own
entry format, with **verified** stamped to the date of the sweep that found them. A format
check should end with two outputs: a clean bill of health, and a short list of things
upstream started telling us that we are not yet using.

---

## 1f  Steward Check (what a community steward runs)

Each agent has a steward: a contributor who uses that agent daily and re-verifies
its format a few times a year, or after a big vendor release. A steward is not
expected to know any of the above. They run one command:

```
./scripts/steward_check.py <agent>       # ./scripts/steward_check.py --list-agents
```

It runs the ordinary weekly scan (§1) restricted to that one agent, against the
steward's own local sessions, and answers in plain sentences with one of three
exits:

- **0 — all good.** `all good: <agent> format matches the baseline (N sessions
  sampled)`, plus the matrix's verified version. If their CLI is newer than the
  verified version and the schema still matches, it says the matrix entry can be
  bumped — that is the §1b evidence a maintainer needs.
- **1 — drift.** The schema diff in plain words, a **redacted** sample written to
  `scripts/probe_scan_output/steward_check/<agent>/redacted-sample/`, and a
  ready-to-paste GitHub issue body (also saved as `issue.md`).
- **2 — cannot check.** The CLI is not installed, there are no sessions on disk,
  or the repository has no baseline fixtures for that agent yet. Says which.

What it deliberately does **not** do: write or rebuild any baseline fixture.
Deciding that drift is real and rebuilding a baseline stays a maintainer job
(`scripts/rebuild_stage0_baseline.py --agent <agent> --emit`, §5a/§7).

The sample reuses `