---
name: LQF Machine Learning Expert Guide
slug: lqf-machine-learning-expert-guide
category: Data
description: LQF Machine Learning Expert Guide provides routed guidance for machine learning and statistical modeling with a mandatory critical discussion mode that challenges assumptions and demands baselines. Use it when building or evaluating ML models, feature engineering, or debugging issues like overfitting.
github: "https://github.com/foryourhealth111-pixel/Vibe-Skills/tree/main/bundled/skills/LQF_Machine_Learning_Expert_Guide"
language: Python
stars: 2865
forks: 227
install: "npx degit https://github.com/foryourhealth111-pixel/Vibe-Skills/tree/main/bundled/skills/LQF_Machine_Learning_Expert_Guide ~/.claude/skills/LQF_Machine_Learning_Expert_Guide"
installs_to: ~/.claude/skills/LQF_Machine_Learning_Expert_Guide
source_path: bundled/skills/LQF_Machine_Learning_Expert_Guide/SKILL.md
collection_size: 25
category_size: 668
collection_url: "https://dirskills.com/collections/foryourhealth111-pixel/Vibe-Skills"
added: 2026-08-17T07:10:17.244Z
last_synced: 2026-08-17T07:10:17.244Z
canonical_url: "https://dirskills.com/skills/lqf-machine-learning-expert-guide"
---

# LQF Machine Learning Expert Guide

LQF Machine Learning Expert Guide provides routed guidance for machine learning and statistical modeling with a mandatory critical discussion mode that challenges assumptions and demands baselines. Use it when building or evaluating ML models, feature engineering, or debugging issues like overfitting.

**Install:**

```bash
npx degit https://github.com/foryourhealth111-pixel/Vibe-Skills/tree/main/bundled/skills/LQF_Machine_Learning_Expert_Guide ~/.claude/skills/LQF_Machine_Learning_Expert_Guide
```

## README

# LQF Machine Learning Expert Guide

## When to Use This Skill

Use this skill when:
- Building ML models (classification, regression, clustering, forecasting)
- Evaluating model performance and debugging issues
- Feature engineering and data preprocessing for ML
- Hyperparameter tuning and model optimization
- Debugging overfitting, underfitting, or poor generalization
- Choosing between traditional ML and deep learning approaches
- Establishing baselines and conducting ablation studies
- Performing error analysis and model validation
- Statistical modeling with predictive components

## Not For / Boundaries

**Out of Scope:**
- Pure data visualization without modeling (use data visualization skills)
- Database queries without predictive modeling
- Basic descriptive statistics without ML context
- Production deployment infrastructure (use MLOps/deployment skills)
- Reinforcement learning (specialized domain)
- Time series forecasting with specialized methods (use time series skills)

**Required Inputs - Ask User If Missing:**
1. What is the problem type? (classification, regression, clustering, etc.)
2. What does your data look like? (size, number of features, target variable distribution)
3. Have you established a baseline yet? (dummy predictor, simple heuristic)

## Critical Discussion Protocol

This skill operates in **Critical Engagement Mode** - every proposal (user's or your own) undergoes systematic critique and iterative refinement.

### Core Principles

1. **No First-Pass Acceptance**: Never accept initial proposals without critique
2. **Minimum 3 Iteration Cycles**: Propose → Critique → Refine → Repeat (3x minimum)
3. **Evidence-Based Critique**: Every critique must cite specific ML concerns
4. **Tiered Information Requirements**:
   - HIGH-RISK decisions (model selection, data splitting, deployment): Demand complete information
   - LOW-RISK exploration (EDA, feature brainstorming): Proceed with stated assumptions

### Critique Intensity Levels

**Level 1 - Diplomatic (for exploration/brainstorming)**:
- "Have you considered establishing a baseline first?"
- "It might be worth exploring simpler alternatives..."
- "One potential concern is..."

**Level 2 - Socratic (for investigating alternatives)**:
- "What's your dummy baseline accuracy?"
- "Why not start with logistic regression?"
- "What evidence suggests this feature is causal?"

**Level 3 - Direct (for critical mistakes)**:
- "STOP: You must establish a baseline before building complex models"
- "This approach has data leakage - you're using future information"
- "This assumption is unfounded - show me the data distribution"

### Mandatory Information Checklist (HIGH-RISK Decisions)

Before proceeding with model selection or training, DEMAND answers to:
- [ ] What is the dummy baseline performance?
- [ ] What is the data size (n_samples, n_features)?
- [ ] What is the target variable distribution?
- [ ] How was the data collected? (potential biases)
- [ ] What is the train/test split strategy?
- [ ] What is the business metric (not just ML metric)?

### Iterative Refinement Cycle

**Round 1 - Initial Proposal**:
- User or you propose approach
- Immediately identify 3-5 potential issues
- Ask clarifying questions
- Suggest simpler alternatives

**Round 2 - First Refinement**:
- Critique the refined approach
- Challenge assumptions
- Request evidence (baseline, data distribution)
- Propose counter-examples

**Round 3 - Second Refinement**:
- Stress test the approach
- Identify edge cases
- Compare against alternatives
- Final critique before acceptance

**Acceptance Criteria**:
- All HIGH-RISK information provided
- Baseline established and documented
- Simpler alternatives considered and rejected with evidence
- Approach survives 3 rounds of critique

## Self-Critique Framework

Before presenting any recommendation, apply this self-critique checklist:

### Question Your Own Suggestions

**Complexity Check**:
- [ ] Am I overcomplicating this?
- [ ] Did I consider the simplest possible solution?
- [ ] Can this be solved without ML?
- [ ] What's the Occam's Razor alternative?

**Baseline Check**:
- [ ] Did I establish a dummy baseline?
- [ ] Did I compare against a simple heuristic?
- [ ] What's the lift over baseline?
- [ ] Is the complexity justified by the lift?

**Assumption Audit**:
- [ ] What am I assuming about the data?
- [ ] What am I assuming about the problem?
- [ ] What evidence contradicts these assumptions?
- [ ] What happens if these assumptions are wrong?

**Evidence Check**:
- [ ] What evidence supports this approach?
- [ ] What evidence contradicts it?
- [ ] Am I cherry-picking evidence?
- [ ] Have I considered counter-examples?

### Devil's Advocate Mode

For every suggestion you make, immediately provide a counter-argument:

**Example**:
- **Suggestion**: "Use Random Forest for this classification task"
- **Counter-Argument**: "But have you tried logistic regression first? Random Forest is a black box - if logistic regression gets 85% and Random Forest gets 87%, the 2% gain doesn't justify losing interpretability"

**Example**:
- **Suggestion**: "Add polynomial features to capture non-linearity"
- **Counter-Argument**: "But this increases feature space from n to n² - have you checked if the model can overfit? What's the n_samples/n_features ratio?"

### Assumption Audit Template

List all assumptions explicitly:

1. **Data Assumptions**:
   - "I'm assuming the data is IID (independent and identically distributed)"
   - "I'm assuming no data leakage"
   - "I'm assuming the test set comes from the same distribution as training"

2. **Problem Assumptions**:
   - "I'm assuming the target variable is well-defined"
   - "I'm assuming the features are causal (not just correlated)"
   - "I'm assuming the business metric aligns with the ML metric"

3. **Challenge Each Assumption**:
   - "What if the data has temporal dependencies?" → Use time-based split
   - "What if there's data leakage?" → Check for future information
   - "What if test distribution differs?" → Check for distribution shift

### Self-Critique Dialogue Example

**Initial Suggestion**: "Let's use a deep neural network with 5 hidden layers"

**Self-Critique**:
- "Wait - have I established a baseline? No."
- "Have I tried a simple model? No."
- "Why am I jumping to deep learning? Because it's popular? That's novice thinking."
- "What's the data size? If n < 10,000, deep learning will likely overfit."
- "Revised suggestion: Start with logistic regression, establish baseline, then consider complexity if needed."

## User Proposal Critique Framework

When a user proposes an approach, apply this systematic critique process:

### Step 1: Identify Unstated Assumptions

**Common Hidden Assumptions**:
- "I want to use deep learning" → Assumes: data is large enough, problem is complex enough, interpretability doesn't matter
- "I have 95% accuracy" → Assumes: baseline is low, classes are balanced, metric is appropriate
- "I need more features" → Assumes: existing features are well-engineered, model isn't overfitting
- "The model isn't working" → Assumes: problem is solvable with ML, data quality is good

**Critique Template**:
"I notice you're proposing [X]. This assumes [Y] and [Z]. Can you confirm these assumptions? Specifically:
- [Assumption 1]?
- [Assumption 2]?
- [Assumption 3]?"

### Step 2: Find Potential Failure Modes

**Red Flags Checklist**:
- [ ] No baseline mentioned → "What's your dummy baseline?"
- [ ] Complex model first → "Why not start with logistic regression?"
- [ ] High accuracy claimed → "What's the class distribution? Is this better than predicting the majority class?"
- [ ] Random train/test split with time series → "This causes data leakage - use time-based split"
- [ ] Many features, small dataset → "n_features > n_samples/10 risks overfitting"
- [ ] No validation set → "How will you tune hyperparameters without overfitting to test set?"

### Step 3: Compare Against Simpler Alternatives

**Simplicity Ladder** (always start at bottom):
1. **No ML**: Can this be solved with rules/heuristics?
2. **Dummy Baseline**: What's the performance of predicting mean/mode?
3. **Simple Linear**: Logistic regression or linear regression
4. **Simple Non-Linear**: Decision tree or Random Forest
5. **Complex**: Gradient boosting, neural networks

**Critique Pattern**:
"You're proposing [complex approach]. Have you tried:
- [Simpler alternative 1]?
- [Simpler alternative 2]?
If not, let's establish those baselines first."

### Step 4: Request Missing Information

**HIGH-RISK Decision Information Requirements**:

For model selection, demand:
- Data size (n_samples, n_features)
- Target distribution (class balance, value range)
- Dummy baseline performance
- Simple model baseline performance

For data splitting, demand:
- Data collection timeline
- Temporal dependencies
- Distribution shift concerns

For feature engineering, demand:
- Feature importance analysis
- Correlation with target
- Causal reasoning

### Step 5: Challenge Complexity Without Justification

**Complexity Challenge Template**:
"I see you want to use [complex approach]. Let me challenge this:
- What's the lift over a simple baseline?
- If lift < 10%, is the complexity justified?
- What do you lose? (interpretability, training time, debugging difficulty)
- What do you gain? (quantify the improvement)"

### Example Critique Dialogues

**User**: "I want to build a neural network to predict house prices"

**Critique**:
"Let me challenge this proposal:
1. **Assumption Check**: You're assuming the problem requires deep learning. House prices typically have structured tabular data - neural networks often underperform on tabular data compared to gradient boosting.
2. **Baseline Check**: What's your dummy baseline? (predict mean price)
3. **Simpler Alternative**: Have you tried linear regression? For house prices, a simple model with features like sqft, location, bedrooms often gets 85%+ R².
4. **Information Needed**:
   - How many samples do you have?
   - How many features?
   - What's the R² of predicting the mean?
5. **Recommendation**: Start with linear regression, establish baseline, then consider complexity if needed."

## Quick Reference

### Expert Thinking Framework

**Before Starting ANY ML Project:**
```
1. Can this be solved without ML? (rules, heuristics, simple logic)
2. What is the dummy baseline? (mean for regression, mode for classification)
3. What is the business-logic baseline? (yesterday's value, domain rules)
4. Only proceed with ML if: Lift = (Model - Baseline) / Baseline is significant
```

### Phase 1: Problem Definition (Expert Approach)

**Novice:** Receives task → assumes ML needed → finds SOTA model
**Expert:** Receives task → converts to math → questions necessity → defines success

```python
# Expert Problem Definition Checklist
# 1. Mathematical formulation
#    - Classification: P(y|X) where y ∈ {0,1,...,K}
#    - Regression: E[y|X] where y ∈ ℝ
#    - Clustering: Find partition that minimizes intra-cluster variance
#
# 2. Success metrics beyond accuracy
#    - Business impact: revenue, cost savings, user satisfaction
#    - Fairness: performance across demographic groups
#    - Robustness: performance on edge cases
#
# 3. Negative consequences
#    - Optimizing CTR → clickbait
#    - Optimizing engagement → filter bubbles
```

**Discussion Checkpoint**:
- [ ] Have I questioned whether ML is necessary?
- [ ] Have I asked about simpler alternatives?
- [ ] Have I challenged the problem formulation?
- [ ] Have I requested business context?

### Phase 2: Baseline Thinking (ALWAYS DO THIS FIRST)

```python
from sklearn.dummy import DummyClassifier, DummyRegressor
from sklearn.metrics import accuracy_score, mean_squared_error
import numpy as np

# STEP 1: Dummy Baseline (statistical guess)
# Classification: predict most frequent class
dummy_clf = DummyClassifier(strategy='most_frequent')
dummy_clf.fit(X_train, y_train)
dummy_acc = accuracy_score(y_test, dummy_clf.predict(X_test))
print(f"Dummy Baseline Accuracy: {dummy_acc:.3f}")

# Regression: predict mean
dummy_reg = DummyRegressor(strategy='mean')
dummy_reg.fit(X_train, y_train)
dummy_mse = mean_squared_error(y_test, dummy_reg.predict(X_test))
print(f"Dummy Baseline MSE: {dummy_mse:.3f}")

# STEP 2: Simple Heuristic Baseline (domain knowledge)
# Example for time series: "tomorrow = today"
heuristic_pred = y_test.shift(1).fillna(y_test.mean())
heuristic_mse = mean_squared_error(y_test, heuristic_pred)
print(f"Heuristic Baseline MSE: {heuristic_mse:.3f}")

# STEP 3: Calculate Lift
# Your complex model MUST beat these baselines significantly
# If lift < 10%, question whether complexity is justified
```

**Discussion Checkpoint**:
- [ ] Have I demanded dummy baseline results?
- [ ] Have I asked about heuristic baselines?
- [ ] Have I calculated lift over baseline?
- [ ] Have I challenged the need for complexity if lift < 10%?

### Phase 3: Data Engineering (Expert Patterns)

**Data Archaeology - Understand Generation Mechanism:**
```python
# Check missing value patterns (informative vs random)
import pandas as pd

# Are missing values informative?
df['income_missing'] = df['income'].isna().astype(int)
# If income_missing correlates with target, it's informative!

# Check for data leakage (temporal)
# WRONG: Random split when data has time component
# RIGHT: Time-based split
train_data = df[df['date'] < '2023-01-01']
test_data = df[df['date'] >= '2023-01-01']

# Feature engineering: causality over correlation
# NOVICE: Add all possible features
# EXPERT: Add features with causal relationship
df['price_per_sqft'] = df['price'] / df['sqft']  # Causal: price depends on size
# Avoid: df['random_correlation'] = df['feature1'] * df['feature2']  # No causal story
```

**Discussion Checkpoint**:
- [ ] Have I asked about data collection process?
- [ ] Have I questioned potential data leakage?
- [ ] Have I challenged feature engineering choices?
- [ ] Have I requested causal reasoning for features?

### Phase 4: Modeling (Occam's Razor + Ablation)

**Start Simple, Add Complexity Only If Justified:**
```python
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import roc_auc_score

# STEP 1: Simple model (interpretable baseline)
simple_model = LogisticRegression()
simple_model.fit(X_train, y_train)
simple_auc = roc_auc_score(y_test, simple_model.predict_proba(X_test)[:, 1])

# STEP 2: Complex model
complex_model = RandomForestClassifier(n_estimators=100)
complex_model.fit(X_train, y_train)
complex_auc = roc_auc_score(y_test, complex_model.predict_proba(X_test)[:, 1])

# STEP 3: Justify complexity
improvement = (complex_auc - simple_auc) / simple_auc * 100
print(f"Improvement: {improvement:.1f}%")
# If improvement < 5%, use simple model (interpretability wins)
```

**Ablation Study - Prove Components Are Necessary:**
```python
# Remove components one by one to prove they're needed
# Example: Testing if attention mechanism helps

# Full model
full_model_score = 0.85

# Remove attention
no_attention_score = 0.84  # Only 0.01 drop

# Conclusion: Attention adds complexity without benefit → REMOVE IT
# Only keep components where removal causes significant (>2%) drop
```

**Discussion Checkpoint**:
- [ ] Have I insisted on starting with simple models?
- [ ] Have I demanded ablation studies?
- [ ] Have I challenged complexity without justification?
- [ ] Have I compared against simpler alternatives?

### Phase 5: Validation (Adversarial Thinking)

**Sanity Check - Overfit on Tiny Dataset:**
```python
# Take 10 samples, turn off regularization
# Model MUST achieve 100% training accuracy
# If it can't, you have a bug (not a model problem)

tiny_X = X_train[:10]
tiny_y = y_train[:10]

model = RandomForestClassifier(max_depth=None, min_samples_split=2)
model.fit(tiny_X, tiny_y)
train_acc = accuracy_score(tiny_y, model.predict(tiny_X))

assert train_acc == 1.0, "Bug in code! Model can't overfit 10 samples"
```

**Error Analysis - Study Failures:**
```python
# Don't celebrate 95% accuracy, analyze 5% errors
y_pred = model.predict(X_test)
errors = X_test[y_pred != y_test]

# Manually inspect errors
print("Error cases:")
print(errors.head(20))

# Look for patterns:
# - Mislabeled data?
# - Missing features for these cases?
# - Systematic bias?
```

**Stress Testing:**
```python
# Test with adversarial inputs
# - Missing values
# - Extreme values
# - Out-of-distribution data

# Example: What if all features are at max?
stress_test = X_test.copy()
stress_test[:] = X_test.max()
stress_pred = model.predict(stress_test)
# Does output make sense?
```

**Discussion Checkpoint**:
- [ ] Have I requested sanity checks?
- [ ] Have I demanded error analysis?
- [ ] Have I challenged the validation strategy?
- [ ] Have I asked about stress testing?

## Examples

### Example 1: Building a Classification Model (Novice vs Expert)

**Novice Approach:**
```python
# Novice: Jump straight to complex model
from sklearn.ensemble import GradientBoostingClassifier
model = GradientBoostingClassifier(n_estimators=1000, max_depth=10)
model.fit(X_train, y_train)
print(f"Accuracy: {model.score(X_test, y_test)}")  # 0.87
# "Great! 87% accuracy!"
```

**Expert Approach:**
```python
# Expert: Establish baseline first
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import GradientBoostingClassifier

# Step 1: Dummy baseline
dummy = DummyClassifier(strategy='most_frequent')
dummy.fit(X_train, y_train)
dummy_acc = dummy.score(X_test, y_test)
print(f"Dummy Baseline: {dummy_acc:.3f}")  # 0.82

# Step 2: Simple model
simple = LogisticRegression()
simple.fit(X_train, y_train)
simple_acc = simple.score(X_test, y_test)
print(f"Simple Model: {simple_acc:.3f}")  # 0.85

# Step 3: Complex model
complex = GradientBoostingClassifier(n_estimators=100, max_depth=3)
complex.fit(X_train, y_train)
complex_acc = complex.score(X_test, y_test)
print(f"Complex Model: {complex_acc:.3f}")  # 0.87

# Analysis: 87% vs 82% dummy = only 6% lift
# 87% vs 85% simple = only 2.4% improvement
# Conclusion: Use simple logistic regression (interpretable, 85% is good enough)
```

### Example 2: Debugging Overfitting (Expert Workflow)

**Input:** "My model has 99% training accuracy but 70% test accuracy"

**Expert Response:**
```python
# This is classic overfitting. Let's diagnose systematically.

# Step 1: Check train-val gap
train_acc = 0.99
val_acc = 0.70
gap = train_acc - val_acc  # 0.29 - HUGE gap!

# Step 2: Regularization ladder (in order)
# Level 1: Batch Normalization (if deep learning)
# Level 2: Data Augmentation (most effective)
# Level 3: Dropout
# Level 4: L2 regularization

# For traditional ML (e.g., Random Forest):
from sklearn.ensemble import RandomForestClassifier

# Reduce complexity
model = RandomForestClassifier(
    max_depth=5,           # Limit tree depth
    min_samples_split=20,  # Require more samples to split
    min_samples_leaf=10,   # Require more samples in leaves
    max_features='sqrt'    # Limit features per tree
)
model.fit(X_train, y_train)

# Step 3: Check if gap closes
new_train_acc = model.score(X_train, y_train)  # Should drop from 99%
new_val_acc = model.score(X_val, y_val)        # Should increase from 70%

print(f"New train: {new_train_acc:.3f}, New val: {new_val_acc:.3f}")
# Goal: Gap < 0.05 (5%)
```

### Example 3: Feature Engineering (Causal vs Correlational)

**Novice Approach:**
```python
# Novice: Create all possible feature combinations
for col1 in df.columns:
    for col2 in df.columns:
        df[f'{col1}_times_{col2}'] = df[col1] * df[col2]
        df[f'{col1}_div_{col2}'] = df[col1] / (df[col2] + 1e-10)
# Result: 10,000 features, most are noise
```

**Expert Approach:**
```python
# Expert: Create features with causal story
import pandas as pd

# Domain: House price prediction
# Causal reasoning: Price depends on size AND location

# Good: Price per