Browse Skills

Fine-tune LLMs using reinforcement learning with TRL: SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training. Use when you need RLHF, align model with preferences, or train from human feedback.

Flash Attention optimizes transformer attention with Flash Attention for faster inference and lower memory use. Use it for long sequences, GPU memory pressure, or when enabling PyTorch SDPA or flash-attn features.

GGUF Quantization covers GGUF format and llama.cpp quantization for efficient CPU and GPU inference. Use it when converting models for consumer hardware, Apple Silicon, or flexible 2-8 bit local deployment.

GPTQ compresses large language models to 4-bit with minimal accuracy loss. Use it to fit 70B+ models on consumer GPUs or to speed up inference, and with Transformers plus PEFT for QLoRA fine-tuning.

Implements Group Relative Policy Optimization (GRPO) using TRL for fine-tuning language models with custom reward functions. Use for enforcing output formats, teaching verifiable tasks, improving reasoning, or aligning models without labeled preference data.

HQQ Quantization applies calibration-free half-quadratic weight quantization to LLMs at 8/4/3/2/1-bit precision. Use it when quantizing models quickly, deploying with vLLM or HuggingFace Transformers, or fine-tuning quantized models with LoRA.

HuggingFace Accelerate simplifies distributed PyTorch training with a small API and a single launch command. Use it to add multi-GPU, mixed precision, DeepSpeed, or FSDP support to one training script.

High-performance tokenization for NLP with Rust speed. Supports BPE, WordPiece, Unigram. Train custom tokenizers, handle batch encoding, padding, truncation, and alignment tracking. Integrates with Hugging Face Transformers. Use for fast tokenization or custom vocab training.

Implements and trains LLMs with LitGPT’s clean single-file model code. Use it for understanding architectures, fine-tuning with LoRA/QLoRA, or pretraining and deploying models.

Lambda Labs GPU Cloud reserves on-demand GPU instances for ML training and inference. Use it when you need dedicated GPUs, SSH access, persistent filesystems, or multi-node clusters.

Llama.cpp runs LLM inference locally on CPU, Apple Silicon, and non-NVIDIA GPUs with GGUF quantization. Use it for edge deployment, low-memory setups, or when CUDA is unavailable.

Expert guidance for fine-tuning LLMs using LLaMA-Factory, offering a no-code WebUI, support for over 100 models, and quantization with 2-8 bit QLoRA for multimodal applications.