PyTorch FSDP2

by Orchestra-Research

PyTorch FSDP2 is an AI Engineering skill for Claude Code, published by Orchestra-Research in AI-Research-SKILLs.

11.6K stars844 forkson Orchestra-Research/AI-Research-SKILLsAdded 2026/08/12+1% in starsRepository updated 2026/06/16

View source on GitHub Download repo ZIP

aiai-researchclaudeclaude-codeclaude-skillscodexgeminigpt-5grpohuggingfacemachine-leanringmegatronskillsvllm

Install in seconds

Install PyTorch FSDP2

Copy PyTorch FSDP2 into your Claude Code skills folder. Run the command in your terminal, or review the source on GitHub before installing.

terminal

npx degit https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/08-distributed-training/pytorch-fsdp2 ~/.claude/skills/pytorch-fsdp2

Requires Node.js. Downloads this skill only — not the rest of the repository — into your Claude Code skills folder.

Without Node.js

git clone https://github.com/Orchestra-Research/AI-Research-SKILLs.git

Clones the whole repository, then copy the skill’s own directory into your skills folder yourself.

In this catalog

Source file: 08-distributed-training/pytorch-fsdp2/SKILL.md in Orchestra-Research/AI-Research-SKILLs
Installs to: ~/.claude/skills/pytorch-fsdp2
Collection: One of 50 skills cataloged from this repository
Category: AI Engineering — 2451 skills

What PyTorch FSDP2 does

PyTorch FSDP2 adds fully_shard-based distributed sharding to training scripts with the right init, mixed precision, offload, optimizer, and checkpointing flow. Use it when a model does not fit on one GPU or you need DTensor-based sharding with DeviceMesh.

PyTorch FSDP2 is cataloged under AI Engineering on DirSkills. PyTorch FSDP2 comes from a repository tagged ai, ai-research, claude, claude-code and claude-skills.

Documentation

README

Excerpt

Skill: Use PyTorch FSDP2 (`fully_shard`) correctly in a training script

This skill teaches a coding agent how to add PyTorch FSDP2 to a training loop with correct initialization, sharding, mixed precision/offload configuration, and checkpointing.

FSDP2 in PyTorch is exposed primarily via torch.distributed.fsdp.fully_shard and the FSDPModule methods it adds in-place to modules. See: references/pytorch_fully_shard_api.md, references/pytorch_fsdp2_tutorial.md.

When to use this skill

This is the opening of the README. Read the full README on GitHub.

Frequently asked about PyTorch FSDP2

What else does Orchestra-Research publish alongside PyTorch FSDP2?
PyTorch FSDP2 is one of 50 skills that DirSkills catalogs from Orchestra-Research/AI-Research-SKILLs, the repository it ships in. Its siblings there include AWQ Quantization, Autoresearch and Axolotl. Each one is a separate skill with its own page in this directory, installs the same way PyTorch FSDP2 does, and is maintained by Orchestra-Research in that same repository. The rest of the collection is listed on the Orchestra-Research/AI-Research-SKILLs page.
How does PyTorch FSDP2 compare to other AI Engineering skills?
PyTorch FSDP2 ranks #356 by stars among the 2451 AI Engineering skills in this catalog. The most-starred ones next to it are Architecture Decision Records, AI-First Engineering and Agentic OS. DirSkills ranks by the star count of the repository each skill ships in, so that order reflects how popular those repositories are rather than any review of PyTorch FSDP2 against them. Open each page to compare what they document and how they install.