Browse Skills

A high-performance RLHF framework for training large language models (7B-70B+) using PPO, GRPO, RLOO, or DPO. Built on Ray and vLLM with distributed architecture and GPU resource sharing.

Provides guidance for enterprise-grade RL training using miles, a production-ready fork of slime. Use when training large MoE models with FP8 or INT4 quantization, needing train-inference alignment, or requiring speculative RL for maximum throughput.

Implement GRPO fine-tuning using the TRL library to train language models for structured outputs, verifiable tasks, and improved reasoning through reinforcement learning with custom reward functions.

Scalable data processing for ML workloads. Streaming execution across CPU/GPU, supports Parquet/CSV/JSON/images. Integrates with Ray Train, PyTorch, TensorFlow. Scales from single machine to 100s of nodes. Use for batch inference, data preprocessing, multi-modal data loading, or distributed ETL pipelines.

A GPU-accelerated toolkit for curating high-quality LLM training data. Supports fuzzy/exact/semantic deduplication, quality filtering, PII redaction, and multi-modal curation across text, image, video, and audio.

Provides guidance for mechanistic interpretability research using TransformerLens to inspect and manipulate transformer internals via HookPoints and activation caching. Use when reverse-engineering model algorithms, studying attention patterns, or performing activation patching experiments.

Provides guidance for training and analyzing Sparse Autoencoders (SAEs) using SAELens to decompose neural network activations into interpretable features. Use when discovering interpretable features, analyzing superposition, or studying monosemantic representations in language models.

Performs causal interventions on PyTorch models using pyvene's declarative framework, enabling causal tracing, activation patching, and interchange intervention training to test causal hypotheses about model behavior.

Guidance for interpreting and manipulating neural network internals using nnsight, with optional remote execution via NDIF. Use for interpretability experiments on massive models without local GPUs, or with any PyTorch architecture.

Expert guidance for fast fine-tuning with Unsloth, achieving 2-5x faster training and 50-80% less memory using LoRA/QLoRA optimization.

Fine-tune large language models (7B–70B) with limited GPU memory using LoRA and QLoRA. Train less than 1% of parameters with minimal accuracy loss, enabling efficient multi-adapter serving.

Expert guidance for fine-tuning LLMs with LLaMA-Factory: a no-code WebUI, support for 100+ models, QLoRA in 2-8 bits, and multimodal capabilities.