---
name: Human Adjudication Sheets
slug: human-adjudication-sheets
category: Quality
description: Prepares human-readable adjudication sheets from LLM pipeline outputs, recovering full evidence and handling Excel cell limits, multi-line text, and truncation issues to ensure valid human rulings.
github: "https://github.com/kennethkhoocy/applied-micro-skills/tree/main/plugins/applied-micro/skills/adjudication-sheets"
language: Python
stars: 48
forks: 0
install: "npx degit https://github.com/kennethkhoocy/applied-micro-skills/tree/main/plugins/applied-micro/skills/adjudication-sheets ~/.claude/skills/adjudication-sheets"
installs_to: ~/.claude/skills/adjudication-sheets
source_path: plugins/applied-micro/skills/adjudication-sheets/SKILL.md
collection_size: 25
category_size: 1354
collection_url: "https://dirskills.com/collections/kennethkhoocy/applied-micro-skills"
added: 2026-08-11T07:21:12.636Z
last_synced: 2026-08-11T07:21:12.636Z
canonical_url: "https://dirskills.com/skills/human-adjudication-sheets"
---

# Human Adjudication Sheets

Prepares human-readable adjudication sheets from LLM pipeline outputs, recovering full evidence and handling Excel cell limits, multi-line text, and truncation issues to ensure valid human rulings.

**Install:**

```bash
npx degit https://github.com/kennethkhoocy/applied-micro-skills/tree/main/plugins/applied-micro/skills/adjudication-sheets ~/.claude/skills/adjudication-sheets
```

## README

# Human Adjudication Sheets from LLM-Pipeline Data

## Problem
Adjudication sheets built from pipeline intermediates tend to carry truncated
"excerpt" columns (previews made for machine diffing, not human judging). A human
asked to rule "does this text show X?" on a fragment produces invalid rulings:
the decisive sentence may sit past the cutoff. Separately, multi-line text in CSV
cells renders as broken rows/empty cells in Excel, so the labeler reports "there
is no data here" even when the column is populated.

## Context / Trigger Conditions
- A labeler says the sheet has "no information" — check for embedded newlines in CSV fields first.
- Excerpt lengths cluster at an exact value (all 1,204 / 1,500 / 800 chars) or end with "[…]" — that is a hard cap, not natural length.
- The ruling question is "does the record/text show X?" — the ruling basis must be byte-identical to what the classifier/raters saw, or the ruling grades a different object.

## Solution
1. **Trace the true rating basis from the scoring code, not from intermediates.** Find the line where text is passed to the model (e.g. `prompt = ... + text`) and reproduce that construction exactly. Verify: stored excerpt == full_text[:cap] (0 mismatches), or cache-key match against the LLM request cache.
2. **Deliver .xlsx, not .csv**, for any sheet with multi-line text: wrap_text + frozen header + generous width on evidence columns; data-validation dropdown on the `ruling` column with the exact allowed codes (e.g. E_pos/E_neg/unclear, or 0/1/2/unclear).
3. **Handle the Excel cell cap (32,767 chars) explicitly:** if full text exceeds it, inline the first ~30k ending with a loud marker ("[CONTINUES — see companion file]") AND write the complete text to a per-case `.txt` (header block with case metadata + full text); add a `text_file` path column. Write companion files for ALL cases regardless — they are the comfortable reading surface.
4. Keep the original CSV untouched as the machine-readable artifact; the xlsx is the human ruling surface; harvest rulings from the xlsx.
   **Distribution:** `text_file` paths are project-relative and break the moment the sheet is emailed or copied out of the tree (the labeler asks "where are the attendant/companion files?"). If the labeler is not working inside the project folder, ship ONE zip containing the xlsx and the companion-texts folder side by side, with the instruction: extract together; the file for row X is `<texts_folder>\<case_id>.txt` next to the spreadsheet. Require the FILLED xlsx back and harvest from the returned copy, not the original.
5. Tell the labeler the epistemics: rule from the provided record only; silence = negative; outside research (Google) goes in `ruling_notes`, never the ruling — otherwise the labels leak information the classifier could never see.

## Verification
- No excerpt column has >30% of rows at one exact length; no "[…]" markers remain.
- `pd.read_excel` round-trip shows full lengths (compare min/median/max vs the old excerpts).
- Dropdown rejects free-text entries; a saved test ruling survives reopen.

## Example
Specialist Directors US, 2026-07-08: director sheet excerpts capped at ~1,200 chars (full dossiers up to 4,090); filing sheet capped at 1,500 chars while raters had scored the whole ~80k-char Item 1A — the PI caught both mid-sitting. Fix: `src/director_v1/make_sitting_xlsx.py`, `tools/make_adjudication_v2_xlsx.py` (full-text xlsx + 76 companion txts + dropdowns).

## Notes
- Uniform-length clustering is the fastest tell; check it BEFORE handing any sheet to a human.
- Harvest gotcha: when comparing harvested rulings to prior labels, coerce BOTH sides to numeric first — Excel/pandas round-trips floats as "1.0" vs the dropdown's "1", so a string compare falsely flags every row as a disagreement.
- If the pipeline caches LLM requests content-addressed, verify the rebuilt basis against the cache rather than trusting a deterministic builder to have been stable.