---
name: Agent Survey Corpus
slug: agent-survey-corpus
category: Writing
description: Agent Survey Corpus downloads open-access arXiv survey PDFs about agentic systems and extracts text for local reference. Use it to study how real surveys organize sections, subsections, and evidence-backed comparisons.
github: "https://github.com/WILLOSCAR/research-units-pipeline-skills/tree/main/.codex/skills/agent-survey-corpus"
language: Python
stars: 499
forks: 39
install: "npx degit https://github.com/WILLOSCAR/research-units-pipeline-skills/tree/main/.codex/skills/agent-survey-corpus ~/.claude/skills/agent-survey-corpus"
installs_to: ~/.claude/skills/agent-survey-corpus
source_path: .codex/skills/agent-survey-corpus/SKILL.md
collection_size: 25
category_size: 1012
collection_url: "https://dirskills.com/collections/WILLOSCAR/research-units-pipeline-skills"
added: 2026-08-26T05:12:50.281Z
last_synced: 2026-08-26T05:12:50.281Z
canonical_url: "https://dirskills.com/skills/agent-survey-corpus"
---

# Agent Survey Corpus

Agent Survey Corpus downloads open-access arXiv survey PDFs about agentic systems and extracts text for local reference. Use it to study how real surveys organize sections, subsections, and evidence-backed comparisons.

**Install:**

```bash
npx degit https://github.com/WILLOSCAR/research-units-pipeline-skills/tree/main/.codex/skills/agent-survey-corpus ~/.claude/skills/agent-survey-corpus
```

## README

# Agent Survey Corpus (arXiv PDFs → text extracts)

Goal: create a small, local reference library so you can **learn from real agent surveys** when refining:
- C2 outline structure (paper-like sectioning)
- C4 tables/claims organization
- C5 writing style and density

This is intentionally *not* part of the pipeline; it is an optional, repo-level toolkit.

## Inputs

- `ref/agent-surveys/arxiv_ids.txt`

## Outputs

- `ref/agent-surveys/pdfs/`
- `ref/agent-surveys/text/`
- `ref/agent-surveys/STYLE_REPORT.md` (tracked; auto-generated summary)

## Workflow

1) Edit `ref/agent-surveys/arxiv_ids.txt` (one arXiv id per line).
2) Run the downloader to fetch PDFs and extract the first N pages to text.
3) Skim the extracted text under `ref/agent-surveys/text/`:
   - look at section counts (H2), subsection granularity (H3), and how they transition between chapters.
   - identify repeated rhetorical patterns you want the pipeline writer to imitate.

## Script

### Quick Start

- `uv run python .codex/skills/agent-survey-corpus/scripts/run.py --help`
- `uv run python .codex/skills/agent-survey-corpus/scripts/run.py --workspace . --max-pages 20`

### All Options

- `--workspace <dir>` (use `.` to write into repo root)
- `--inputs <semicolon-separated>` (default: `ref/agent-surveys/arxiv_ids.txt`)
- `--max-pages <N>` (default: 20)
- `--sleep <seconds>` (default: 1.0)
- `--overwrite` (re-download + re-extract)

### Examples

- Download/extract into repo root `ref/`:
  - `uv run python .codex/skills/agent-survey-corpus/scripts/run.py --workspace . --max-pages 20`
- Download/extract into a specific folder (treated as workspace root):
  - `uv run python .codex/skills/agent-survey-corpus/scripts/run.py --workspace /tmp/surveys --max-pages 30`

## Troubleshooting

- **Download fails / timeout**: rerun with a larger `--sleep`, or try fewer ids.
- **Text extract is empty**: the PDF may be scanned; try another survey or increase `--max-pages`.
- **Files showing up in git status**: PDFs/text are ignored via `.gitignore` (`ref/**/pdfs/`, `ref/**/text/`).