AI Engineering

Performs causal interventions on PyTorch models using pyvene's declarative framework, enabling causal tracing, activation patching, and interchange intervention training to test causal hypotheses about model behavior.

Provides guidance for training and analyzing Sparse Autoencoders (SAEs) using SAELens to decompose neural network activations into interpretable features. Use when discovering interpretable features, analyzing superposition, or studying monosemantic representations in language models.

Provides guidance for mechanistic interpretability research using TransformerLens to inspect and manipulate transformer internals via HookPoints and activation caching. Use when reverse-engineering model algorithms, studying attention patterns, or performing activation patching experiments.

Implement GRPO fine-tuning using the TRL library to train language models for structured outputs, verifiable tasks, and improved reasoning through reinforcement learning with custom reward functions.

Provides guidance for enterprise-grade RL training using miles, a production-ready fork of slime. Use when training large MoE models with FP8 or INT4 quantization, needing train-inference alignment, or requiring speculative RL for maximum throughput.

A high-performance RLHF framework for training large language models (7B-70B+) using PPO, GRPO, RLOO, or DPO. Built on Ray and vLLM with distributed architecture and GPU resource sharing.

SimPO is a reference-free preference optimization method for LLM alignment, outperforming DPO without a reference model. Use for simpler, faster training on preference data compared to DPO or PPO.

Guidance for LLM post-training with RL using Slime, a framework combining Megatron-LM for training and SGLang for rollout. Useful for training GLM models, custom data generation, and tight Megatron-LM integration for RL scaling.

Provides guidance for PyTorch-native agentic RL using torchforge, Meta's library separating infra from algorithms. Use when you want clean RL abstractions, easy algorithm experimentation, or scalable training with Monarch and TorchTitan.

Fine-tune language models using reinforcement learning with the TRL library. Supports SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training.

Orchestrates end-to-end autonomous AI research using a two-loop architecture for rapid experimentation and synthesis. Use for starting research projects, running autonomous experiments, or managing multi-hypothesis efforts.

Quantize large language models to 4/3/2-bit precision without calibration data, enabling faster quantization and deployment with vLLM or HuggingFace Transformers.