---
name: Run Evals
slug: run-evals
category: Quality
description: Run Evals launches a real iPolloWork app on Daytona or local Electron and runs coded eval flows via CDP, producing machine-checkable assertions, screenshots, and reports used as PR evidence.
github: "https://github.com/Devin-AXIS/iPolloWork/tree/main/.opencode/skills/run-evals"
language: TypeScript
stars: 4292
forks: 825
install: "npx degit https://github.com/Devin-AXIS/iPolloWork/tree/main/.opencode/skills/run-evals ~/.claude/skills/run-evals"
installs_to: ~/.claude/skills/run-evals
source_path: .opencode/skills/run-evals/SKILL.md
collection_size: 25
category_size: 1354
collection_url: "https://dirskills.com/collections/Devin-AXIS/iPolloWork"
added: 2026-08-20T07:54:34.981Z
last_synced: 2026-08-20T07:54:34.981Z
canonical_url: "https://dirskills.com/skills/run-evals"
---

# Run Evals

Run Evals launches a real iPolloWork app on Daytona or local Electron and runs coded eval flows via CDP, producing machine-checkable assertions, screenshots, and reports used as PR evidence.

**Install:**

```bash
npx degit https://github.com/Devin-AXIS/iPolloWork/tree/main/.opencode/skills/run-evals ~/.claude/skills/run-evals
```

## README

# Skill: Run Evals

Launch a real iPolloWork app and run coded eval flows against it. This skill owns
**launch + run**; the prove/repair/verdict loop and evidence standard live in
the **`fraimz` skill** — load that too for anything that ends in a verdict.

## Prerequisites

- `daytona` CLI installed and logged in (`daytona login`), right org selected
  (`daytona organization use "<org-name>"`)
- `.devcontainer/` files present in the repo
- Optional provider coverage: reusable secrets volume populated once with
  `bash .devcontainer/setup-daytona-secrets-volume.sh .newtoken` (never print
  keys; sandboxes source every `/daytona-secrets/*.env` before Electron starts)

## Preferred path: Daytona sandbox

```bash
daytona organization use "<org-name>"
bash .devcontainer/test-on-daytona.sh <branch-or-commit> --artifacts-volume
```

The helper creates a fresh VNC-capable sandbox from the `ipollowork-eval-vnc`
snapshot, mounts the secrets + pnpm-store volumes, starts XFCE/noVNC, Vite, and
Electron with Daytona-safe flags, waits for CDP, then prints the CDP and noVNC
URLs. `--artifacts-volume` mounts `/daytona-artifacts` served on port 8090 for
published frame proof. Refresh the snapshot when dependencies change:
`bash .devcontainer/create-daytona-ipollowork-snapshot.sh`.

Verify the endpoint before running flows:

```
browser_list({ browser_url: "<CDP_URL>" })   # must show an "iPolloWork" target
```

If it fails, inspect `/tmp/electron.log` — the real success marker is
Chromium's `DevTools listening on ws://127.0.0.1:9825/...`.

If the app shows the Welcome page, create a workspace first (see
`evals/daytona-flows.md` Flow 1: create `/workspace/hello` on the sandbox,
"Get started" → "Local workspace" → inject the path → "Create Workspace").

## Run the flows

```bash
pnpm evals --list
pnpm evals --flow <flow-id> --cdp-url <printed-electron-cdp-url>
pnpm evals --all --stack den     # brings up MySQL + den-api + seed for cloud flows
```

The runner produces machine-checkable assertions, validated screenshots, and
writes `fraimz.html` + `report.md` / `report.json` under
`evals/results/<run-id>/`. If no coded flow exists for the behavior, add one in
`evals/flows/<id>.flow.mjs` (see the `fraimz` skill and `evals/README.md` for
the `ctx.*` API); use manual browser tools only to debug or prototype — a coded
flow is the PR evidence.

## Recording (motion only)

Frame proof is the default deliverable; record video only when motion matters
(streaming, animations). Start with
`bash .devcontainer/test-on-daytona.sh <branch> --record-video --recording-name <name>`,
stop with `daytona exec "$SANDBOX" -- 'bash .devcontainer/stop-daytona-recording.sh'`,
download via the port-8090 artifacts URL. Details: `daytona-recording-artifacts`.

## Local fallback

When Daytona is down or quota-limited:

```bash
pnpm install
pnpm --filter @ipollowork/app typecheck
IPOLLOWORK_ELECTRON_REMOTE_DEBUG_PORT=9826 pnpm dev   # then:
pnpm evals --flow <flow-id> --cdp-url http://127.0.0.1:9826
```

Report clearly whether the result came from Daytona or the local fallback — a
local run is not a Daytona validation.

## Teardown

```bash
daytona delete "$SANDBOX"
```