Lit-Dedup

by kennethkhoocy

Lit-Dedup is an AI Engineering skill for Claude Code, published by kennethkhoocy in applied-micro-skills.

48 stars0 forkson kennethkhoocy/applied-micro-skillsAdded 2026/08/11Repository updated 2026/07/22

View source on GitHub Download repo ZIP

applied-microeconomicsclaude-codeclaude-skillscodexcodex-skillsempirical-research

Install in seconds

Install Lit-Dedup

Copy Lit-Dedup into your Claude Code skills folder. Run the command in your terminal, or review the source on GitHub before installing.

terminal

npx degit https://github.com/kennethkhoocy/applied-micro-skills/tree/main/plugins/applied-micro/skills/lit-review-orchestrator/lit-dedup ~/.claude/skills/lit-dedup

Requires Node.js. Downloads this skill only — not the rest of the repository — into your Claude Code skills folder.

Without Node.js

git clone https://github.com/kennethkhoocy/applied-micro-skills.git

Clones the whole repository, then copy the skill’s own directory into your skills folder yourself.

In this catalog

Source file: plugins/applied-micro/skills/lit-review-orchestrator/lit-dedup/SKILL.md in kennethkhoocy/applied-micro-skills
Installs to: ~/.claude/skills/lit-dedup
Collection: One of 25 skills cataloged from this repository
Category: AI Engineering — 2451 skills

What Lit-Dedup does

Merge and deduplicate papers from literature review pipeline stages using exact DOI matching and LLM fuzzy matching. Produces a master list with provenance tracking for research workflows.

Lit-Dedup is cataloged under AI Engineering on DirSkills. Lit-Dedup comes from a repository tagged applied-microeconomics, claude-code, claude-skills, codex and codex-skills.

Documentation

README

Excerpt

Lit-Dedup (Stage 5 — Merge & Deduplicate)

Merge multiple pipeline stage outputs into one deduplicated master list with provenance tracking.

Input: one or more JSON files from Stages 1–4 Output: merged_results.json + merged_results.ris + dedup_log.json

Quick Start

# Merge all JSON files in a directory
python scripts/lit_dedup.py --input-dir ./results/ -o merged_results.json

# Merge specific files
python scripts/lit_dedup.py --inputs stage1.json stage2.json stage4.json -o merged.json

# DOI-only dedup (skip LLM pass)
python scripts/lit_dedup.py --inputs *.json --no-llm -o merged.json

# Non-interactive (skip confirmation)
python scripts/lit_dedup.py --inputs *.json -o merged.json --yes

This is the opening of the README. Read the full README on GitHub.

Frequently asked about Lit-Dedup

What else does kennethkhoocy publish alongside Lit-Dedup?
Lit-Dedup is one of 25 skills that DirSkills catalogs from kennethkhoocy/applied-micro-skills, the repository it ships in. Its siblings there include Adversarial Empirical Review, Annotator Input Parity Check and AsyncOpenAI Concurrency Fix. Each one is a separate skill with its own page in this directory, installs the same way Lit-Dedup does, and is maintained by kennethkhoocy in that same repository. The rest of the collection is listed on the kennethkhoocy/applied-micro-skills page.
How does Lit-Dedup compare to other AI Engineering skills?
Lit-Dedup ranks #2280 by stars among the 2451 AI Engineering skills in this catalog. The most-starred ones next to it are Architecture Decision Records, AI-First Engineering and Agentic OS. DirSkills ranks by the star count of the repository each skill ships in, so that order reflects how popular those repositories are rather than any review of Lit-Dedup against them. Open each page to compare what they document and how they install.