---
name: Caveman Experiment Manager
slug: caveman-experiment-manager
category: AI Engineering
description: "Caveman Experiment Manager inspects Caveman Cloud's eval-gated experiment lifecycle, reads current state and results, and reports one supported recommendation or blocks unsafe actions. Use it when asked to start, approve, cancel, promote, or roll back a Caveman experiment."
github: "https://github.com/JuliusBrussee/caveman/tree/main/skills/caveman-manage"
language: Go
stars: 97841
forks: 5640
install: "npx degit https://github.com/JuliusBrussee/caveman/tree/main/skills/caveman-manage ~/.claude/skills/caveman-manage"
installs_to: ~/.claude/skills/caveman-manage
source_path: skills/caveman-manage/SKILL.md
collection_size: 20
category_size: 2451
collection_url: "https://dirskills.com/collections/JuliusBrussee/caveman"
added: 2026-08-13T07:35:48.426Z
last_synced: 2026-08-13T07:35:48.426Z
canonical_url: "https://dirskills.com/skills/caveman-experiment-manager"
---

# Caveman Experiment Manager

Caveman Experiment Manager inspects Caveman Cloud's eval-gated experiment lifecycle, reads current state and results, and reports one supported recommendation or blocks unsafe actions. Use it when asked to start, approve, cancel, promote, or roll back a Caveman experiment.

**Install:**

```bash
npx degit https://github.com/JuliusBrussee/caveman/tree/main/skills/caveman-manage ~/.claude/skills/caveman-manage
```

## README

# Manage eval-gated experiments

Treat every lifecycle change as a production control action. Read current state
and results, then report one supported recommendation or block.
Current agent MCP is intentionally read-only: control-api does not yet enforce a
complete lifecycle transition table and evidence gate atomically.

## Non-negotiable gates

1. A request to review, inspect, explain, or recommend authorizes reads only.
2. Never approve an experiment whose results are pending, whose required
   guardrails are absent, or whose evidence reports a breach.
3. Never convert experiment lift into `verified_savings`. Only active real
   traffic plus provider-causal, provider-complete ledger evidence can do that.
4. Never supply an organization id. Project and tenant scope come from the
   logged-in Caveman identity and server RBAC.
5. Never execute a lifecycle mutation, even after user approval. Exact
   `<action>:<experiment_id>` strings are agent-generatable and are not proof of
   human intent.
6. Unknown states and server errors fail closed. Report exact
   `cave_snake_code`.

## Step 1 — Load project and experiment

Prefer MCP:

```text
caveman_context {}
caveman_experiment_get {"action":"get","experiment_id":"<id>"}
caveman_experiment_get {"action":"results","experiment_id":"<id>"}
```

Use `{"action":"list"}` when the user has not named an id.

CLI fallback:

```bash
caveman cloud experiments list
caveman cloud experiments show <id>
caveman cloud experiments results <id>
```

Stop if login, project, experiment, or results are unavailable.

## Step 2 — Evaluate evidence

Report:

- current lifecycle state and safety class;
- control and candidate sample sizes;
- quality or eval result;
- latency, error, cost, retry, drop, and escalation guardrails when present;
- evidence cost;
- rollback or hold reason;
- whether result is pending, failed, promotable, or active.

Absence is not a pass. If a required field is absent, state
`evidence incomplete` and do not propose approval.

## Step 3 — Propose one action

Allowed actions:

- `start` — only from a startable draft or queued state with configured graders;
- `approve` — only with complete passing evidence and a safety class the
  current role may approve;
- `cancel` — stop a non-active experiment the user no longer wants;
- `rollback` — revert an active or harmful change through the server's linked
  policy path. Current deployments may reject this honestly with
  `cave_not_implemented`; never describe that response as a rollback.

Show recommendation and id:

```text
Proposed action: approve experiment 7f...
Reason: candidate passed quality and every configured guardrail.
Execution: blocked until server-authoritative lifecycle and evidence gates ship.
```

Do not treat earlier generic statements such as "manage it" or "do what is best"
as mutation approval.

## Step 4 — Block unsafe execution

Do not emit or run an executable lifecycle command. Explain that current server
does not yet enforce every evidence/state transition atomically. CLI and MCP
agent surfaces therefore expose experiment reads only.

## Step 5 — Re-read after external operator action

If operator says they executed command, read detail and results again. Report
server-observed post-state, audit or result response, and any policy-delivery
status returned. Never infer success from operator intent alone.

Use this close:

```text
Action: <action> <experiment-id>
Before: <state>
Server response: <status and cave_snake_code if any>
After: <re-read state>
Basis: experiment evidence only. Verified savings unchanged unless the signed
ledger independently records active, provider-causal real-traffic savings.
```