Orchestra-Research/AI-Research-SKILLs

AWQ Quantization compresses large language models to 4-bit with activation-aware weight selection. Use it to deploy 7B-70B models on limited GPU memory while keeping accuracy loss low.

Orchestrates end-to-end autonomous AI research projects using a two-loop architecture for rapid experiment iteration and synthesis. Routes to domain-specific skills, supports continuous operation, and produces research presentations and papers.

Comprehensive guidance for fine-tuning LLMs using Axolotl, including YAML configuration, 100+ model support, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, and multimodal training.

BigCode Evaluation Harness evaluates code generation models on HumanEval, MBPP, MultiPL-E, and other benchmarks with pass@k metrics. Use it to benchmark coding ability, compare models, and measure multi-language code generation quality.

Bitsandbytes Model Quantization loads LLMs in 8-bit or 4-bit to cut GPU memory use and fit larger models. Use it for Hugging Face Transformers inference, QLoRA fine-tuning, or 8-bit optimizers when VRAM is limited.

Constitutional AI trains models with self-critique, revision, and AI feedback to reduce harmful outputs without human labels. Use it when you need safety alignment or a clear set of principles for model behavior.

DeepSpeed provides guidance for distributed training with ZeRO optimization, pipeline parallelism, mixed precision, 1-bit Adam, and sparse attention. Use it when implementing, tuning, or debugging DeepSpeed-based training workflows.

Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.

Fine-tune LLMs using reinforcement learning with TRL: SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training. Use when you need RLHF, align model with preferences, or train from human feedback.

Flash Attention optimizes transformer attention with Flash Attention for faster inference and lower memory use. Use it for long sequences, GPU memory pressure, or when enabling PyTorch SDPA or flash-attn features.

GGUF Quantization covers GGUF format and llama.cpp quantization for efficient CPU and GPU inference. Use it when converting models for consumer hardware, Apple Silicon, or flexible 2-8 bit local deployment.

GPTQ compresses large language models to 4-bit with minimal accuracy loss. Use it to fit 70B+ models on consumer GPUs or to speed up inference, and with Transformers plus PEFT for QLoRA fine-tuning.

Implements Group Relative Policy Optimization (GRPO) using TRL for fine-tuning language models with custom reward functions. Use for enforcing output formats, teaching verifiable tasks, improving reasoning, or aligning models without labeled preference data.

HQQ Quantization applies calibration-free half-quadratic weight quantization to LLMs at 8/4/3/2/1-bit precision. Use it when quantizing models quickly, deploying with vLLM or HuggingFace Transformers, or fine-tuning quantized models with LoRA.

HuggingFace Accelerate simplifies distributed PyTorch training with a small API and a single launch command. Use it to add multi-GPU, mixed precision, DeepSpeed, or FSDP support to one training script.

High-performance tokenization for NLP with Rust speed. Supports BPE, WordPiece, Unigram. Train custom tokenizers, handle batch encoding, padding, truncation, and alignment tracking. Integrates with Hugging Face Transformers. Use for fast tokenization or custom vocab training.

Expert guidance for fine-tuning LLMs using LLaMA-Factory, offering a no-code WebUI, support for over 100 models, and quantization with 2-8 bit QLoRA for multimodal applications.

Lambda Labs GPU Cloud reserves on-demand GPU instances for ML training and inference. Use it when you need dedicated GPUs, SSH access, persistent filesystems, or multi-node clusters.

Implements and trains LLMs with LitGPT’s clean single-file model code. Use it for understanding architectures, fine-tuning with LoRA/QLoRA, or pretraining and deploying models.

Llama Guard filters LLM inputs and outputs for unsafe content across violence, sexual content, weapons, substances, self-harm, and criminal planning. Use it to block or review prompts and responses in production chat apps.

Llama.cpp runs LLM inference locally on CPU, Apple Silicon, and non-NVIDIA GPUs with GGUF quantization. Use it for edge deployment, low-memory setups, or when CUDA is unavailable.

Llm Evaluation Harness evaluates LLMs across academic benchmarks like MMLU, GSM8K, HumanEval, and TruthfulQA. Use it to benchmark models, compare results, and track training progress.

ML Training Recipes gives PyTorch patterns for choosing architectures, setting optimizers and learning rates, and debugging training issues. Use it when training or fine-tuning neural networks, tuning GPU throughput, or diagnosing loss spikes and OOMs.

Guide for using Mamba state-space models, offering O(n) complexity and 5× faster inference for long sequences. Ideal when you need efficient processing of millions of tokens without a KV cache.

Megatron-Core Training trains large language models with NVIDIA Megatron-Core using tensor, pipeline, context, and expert parallelism. Use it for distributed pretraining of models above 1B parameters on NVIDIA GPUs.

Provides guidance for enterprise-grade RL training using miles, a production-ready fork of slime. Use when training large MoE models with FP8/INT4, needing train-inference alignment, or requiring speculative RL for maximum throughput.

Modal Serverless GPU runs GPU-backed ML workloads without managing servers. Use it to deploy models as APIs, run batch jobs, or scale inference and training on demand.

Provides guidance for interpreting and manipulating neural network internals using nnsight. Ideal for running interpretability experiments on models too large for local GPUs via remote NDIF execution, or for working with any PyTorch architecture.

GPU-accelerated toolkit for curating training data for LLMs, supporting text, image, video, and audio. Use for fuzzy deduplication, quality filtering, semantic deduplication, and PII redaction at scale.

NeMo Evaluator SDK evaluates LLMs across 100+ benchmarks and 18+ harnesses with containerized, reproducible runs. Use it to benchmark models on local Docker, Slurm HPC, or cloud backends.

NeMo Guardrails adds programmable runtime safety checks to LLM apps, including jailbreak detection, input/output validation, fact-checking, and PII filtering. Use it when you need configurable guardrails around model responses.

Train large language models (7B-70B+) using RLHF techniques like PPO, GRPO, RLOO, and DPO with distributed Ray and vLLM acceleration, achieving 2× faster training than DeepSpeedChat.

Fine-tune large language models (7B-70B) using parameter-efficient methods like LoRA and QLoRA. Ideal when memory is limited, training only a fraction of parameters with minimal quality loss.

Prompt Guard detects prompt injection and jailbreak attempts in LLM apps. It can also screen third-party text such as API responses or RAG documents before they reach a model.

PyTorch FSDP2 adds fully_shard-based distributed sharding to training scripts with the right init, mixed precision, offload, optimizer, and checkpointing flow. Use it when a model does not fit on one GPU or you need DTensor-based sharding with DeviceMesh.

PyTorch Lightning organizes PyTorch training into LightningModule and Trainer abstractions with built-in logging, checkpoints, and distributed training. Use it when you want cleaner training loops and the same code to run from a laptop to a multi-GPU system.

Provides guidance for performing causal interventions on PyTorch models using pyvene. Use for causal tracing, activation patching, interchange intervention training, or testing causal hypotheses about model behavior.

Implement and deploy the RWKV architecture, a hybrid RNN-Transformer with linear-time inference, infinite context, and no KV cache. Ideal for long-context processing, streaming generation, and memory-constrained deployment.

Ray Data is a distributed data processing library for ML workloads, supporting streaming execution on CPUs and GPUs. It handles Parquet, CSV, JSON, and images, and integrates with PyTorch, TensorFlow, and Ray Train for preprocessing, batch inference, and ETL.

Ray Train orchestrates distributed machine learning training across single-GPU and multi-node clusters. Use it to scale PyTorch, TensorFlow, or HuggingFace training and run hyperparameter sweeps with fault tolerance.

SentencePiece is a language-independent tokenizer for multilingual models and CJK languages. It supports BPE and Unigram algorithms, with fast processing (50k sentences/sec) and deterministic vocabulary.

Simple Preference Optimization for LLM alignment. Reference-free alternative to DPO with better performance (+6.4 points on AlpacaEval 2.0). No reference model needed, more efficient than DPO. Use for preference alignment when want simpler, faster training than DPO/PPO.

SkyPilot Multi-Cloud Orchestration runs ML training, batch jobs, and serving across multiple clouds with automatic cost-based placement. Use it for spot instances, auto-recovery, distributed training, and avoiding vendor lock-in.

Provides guidance for LLM post-training with RL using slime, a Megatron+SGLang framework. Use when training GLM models, implementing custom data generation workflows, or needing tight Megatron-LM integration for RL scaling.

Provides guidance for training and analyzing Sparse Autoencoders (SAEs) with SAELens to decompose neural network activations into interpretable features. Use for discovering features, analyzing superposition, or studying monosemantic representations.

Provides guidance for implementing agentic reinforcement learning (RL) using Meta's torchforge library. Use when you need clean RL abstractions, easy algorithm experimentation, or scalable training with Monarch and TorchTitan.

Provides guidance for mechanistic interpretability research using TransformerLens to inspect and manipulate transformer internals via HookPoints and activation caching. Use when reverse-engineering model algorithms, studying attention patterns, or performing activation patching experiments.

Provides expert guidance for fine-tuning large language models with Unsloth, achieving 2-5x faster training and 50-80% less memory using LoRA/QLoRA optimization. Use when implementing, debugging, or learning best practices for Unsloth.

Verl RL Training provides guidance for training LLMs with reinforcement learning using verl. Use it when implementing RLHF, GRPO, PPO, or other post-training algorithms at scale with flexible backends.

A minimal, hackable GPT implementation in ~300 lines of PyTorch for learning transformer architectures from scratch. Ideal for education, prototyping, and training small language models on limited hardware.