Megatron-Core Training

Megatron-Core Training is an AI Engineering skill for Claude Code, published by Orchestra-Research in AI-Research-SKILLs.

11.6K stars844 forkson Orchestra-Research/AI-Research-SKILLsAdded 2026/08/12+1% in starsRepository updated 2026/06/16

View source on GitHub Download repo ZIP

aiai-researchclaudeclaude-codeclaude-skillscodexgeminigpt-5grpohuggingfacemachine-leanringmegatronskillsvllm

Install in seconds

Install Megatron-Core Training

Copy Megatron-Core Training into your Claude Code skills folder. Run the command in your terminal, or review the source on GitHub before installing.

terminal

npx degit https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/08-distributed-training/megatron-core ~/.claude/skills/megatron-core

Requires Node.js. Downloads this skill only — not the rest of the repository — into your Claude Code skills folder.

Without Node.js

git clone https://github.com/Orchestra-Research/AI-Research-SKILLs.git

Clones the whole repository, then copy the skill’s own directory into your skills folder yourself.

In this catalog

Source file: 08-distributed-training/megatron-core/SKILL.md in Orchestra-Research/AI-Research-SKILLs
Installs to: ~/.claude/skills/megatron-core
Collection: One of 50 skills cataloged from this repository
Category: AI Engineering — 2451 skills

What Megatron-Core Training does

Megatron-Core Training trains large language models with NVIDIA Megatron-Core using tensor, pipeline, context, and expert parallelism. Use it for distributed pretraining of models above 1B parameters on NVIDIA GPUs.

Megatron-Core Training is cataloged under AI Engineering on DirSkills. Megatron-Core Training comes from a repository tagged ai, ai-research, claude, claude-code and claude-skills.

Documentation

README

Excerpt

Megatron-Core - Large-Scale LLM Training

Quick start

Megatron-Core trains LLMs from 2B to 462B parameters with up to 47% Model FLOP Utilization on H100 GPUs through advanced parallelism strategies.

Installation:

# Docker (recommended)
docker run --gpus all -it --rm nvcr.io/nvidia/pytorch:25.04-py3

# Or pip
pip install megatron-core

Simple distributed training:

# Train with 2 GPUs using data parallelism
torchrun --nproc_per_node=2 examples/run_simple_mcore_train_loop.py

# Or LLaMA-3 8B training
./examples/llama/train_llama3_8b_fp8.sh

Common workflows

Workflow 1: Train LLaMA-style model with 3D parallelism

Copy this checklist:

This is the opening of the README. Read the full README on GitHub.

Frequently asked about Megatron-Core Training

What else does Orchestra-Research publish alongside Megatron-Core Training?
Megatron-Core Training is one of 50 skills that DirSkills catalogs from Orchestra-Research/AI-Research-SKILLs, the repository it ships in. Its siblings there include AWQ Quantization, Autoresearch and Axolotl. Each one is a separate skill with its own page in this directory, installs the same way Megatron-Core Training does, and is maintained by Orchestra-Research in that same repository. The rest of the collection is listed on the Orchestra-Research/AI-Research-SKILLs page.
How does Megatron-Core Training compare to other AI Engineering skills?
Megatron-Core Training ranks #345 by stars among the 2451 AI Engineering skills in this catalog. The most-starred ones next to it are Architecture Decision Records, AI-First Engineering and Agentic OS. DirSkills ranks by the star count of the repository each skill ships in, so that order reflects how popular those repositories are rather than any review of Megatron-Core Training against them. Open each page to compare what they document and how they install.