---
name: Building CI Pipelines
slug: building-ci-pipelines
category: DevOps
description: Building CI Pipelines constructs secure, efficient CI/CD workflows for testing, building, and deployment. Use it to choose and optimize pipelines in GitHub Actions, GitLab CI, Argo Workflows, or Jenkins, including SLSA, caching, and monorepo patterns.
github: "https://github.com/ancoleman/ai-design-components/tree/main/skills/building-ci-pipelines"
language: Python
stars: 516
forks: 73
install: "npx degit https://github.com/ancoleman/ai-design-components/tree/main/skills/building-ci-pipelines ~/.claude/skills/building-ci-pipelines"
installs_to: ~/.claude/skills/building-ci-pipelines
source_path: skills/building-ci-pipelines/SKILL.md
collection_size: 25
category_size: 798
collection_url: "https://dirskills.com/collections/ancoleman/ai-design-components"
added: 2026-08-26T05:12:23.302Z
last_synced: 2026-08-26T05:12:23.302Z
canonical_url: "https://dirskills.com/skills/building-ci-pipelines"
---

# Building CI Pipelines

Building CI Pipelines constructs secure, efficient CI/CD workflows for testing, building, and deployment. Use it to choose and optimize pipelines in GitHub Actions, GitLab CI, Argo Workflows, or Jenkins, including SLSA, caching, and monorepo patterns.

**Install:**

```bash
npx degit https://github.com/ancoleman/ai-design-components/tree/main/skills/building-ci-pipelines ~/.claude/skills/building-ci-pipelines
```

## README

# Building CI Pipelines

## Purpose

CI/CD pipelines automate testing, building, and deploying software. This skill provides patterns for constructing robust, secure, and efficient pipelines across GitHub Actions, GitLab CI, Argo Workflows, and Jenkins. Focus areas: supply chain security (SLSA), monorepo optimization, caching, and parallelization.

## When to Use This Skill

Invoke when:
- Setting up continuous integration for new projects
- Implementing automated testing workflows
- Building container images with security provenance
- Optimizing slow CI pipelines (especially monorepos)
- Implementing SLSA supply chain security
- Configuring multi-platform builds
- Setting up GitOps automation
- Migrating from legacy CI systems

## Platform Selection

**GitHub-hosted** → GitHub Actions (SLSA native, 10K+ actions, OIDC)
**GitLab-hosted** → GitLab CI (parent-child pipelines, built-in security)
**Kubernetes** → Argo Workflows (DAG-based, event-driven)
**Legacy** → Jenkins (migrate when possible)

### Platform Comparison

| Feature | GitHub Actions | GitLab CI | Argo | Jenkins |
|---------|---------------|-----------|------|---------|
| Ease of Use | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐ |
| SLSA | Native | Manual | Good | Manual |
| Monorepo | Good | Excellent | Manual | Plugins |

## Quick Start Patterns

### Pattern 1: Basic CI (Lint → Test → Build)

```yaml
# GitHub Actions
name: CI
on: [push, pull_request]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm run lint

  test:
    needs: lint
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm test

  build:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm run build
```

### Pattern 2: Matrix Strategy (Multi-Platform)

```yaml
test:
  runs-on: ${{ matrix.os }}
  strategy:
    matrix:
      os: [ubuntu-latest, windows-latest, macos-latest]
      node-version: [18, 20, 22]
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with:
        node-version: ${{ matrix.node-version }}
    - run: npm test
```

9 jobs (3 OS × 3 versions) in parallel: 5 min vs 45 min sequential.

### Pattern 3: Monorepo Affected (Turborepo)

```yaml
build:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
      with:
        fetch-depth: 0  # Required for affected detection

    - uses: actions/setup-node@v4
      with:
        node-version: 20

    - name: Build affected
      run: npx turbo run build --filter='...[origin/main]'
      env:
        TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
        TURBO_TEAM: ${{ vars.TURBO_TEAM }}
```

60-80% CI time reduction for monorepos.

### Pattern 4: SLSA Level 3 Provenance

```yaml
name: SLSA Build
on:
  push:
    tags: ['v*']

permissions:
  id-token: write
  contents: read
  packages: write

jobs:
  build:
    runs-on: ubuntu-latest
    outputs:
      digest: ${{ steps.build.outputs.digest }}
    steps:
      - uses: actions/checkout@v4
      - name: Build container
        id: build
        uses: docker/build-push-action@v5
        with:
          push: true
          tags: ghcr.io/${{ github.repository }}:${{ github.sha }}

  provenance:
    needs: build
    permissions:
      id-token: write
      actions: read
      packages: write
    uses: slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@v1.10.0
    with:
      image: ghcr.io/${{ github.repository }}
      digest: ${{ needs.build.outputs.digest }}
      registry-username: ${{ github.actor }}
    secrets:
      registry-password: ${{ secrets.GITHUB_TOKEN }}
```

Verification:
```bash
cosign verify-attestation --type slsaprovenance \
  --certificate-identity-regexp "^https://github.com/slsa-framework" \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  ghcr.io/myorg/myapp@sha256:abcd...
```

### Pattern 5: OIDC Federation (No Credentials)

```yaml
deploy:
  runs-on: ubuntu-latest
  permissions:
    id-token: write
    contents: read
  steps:
    - uses: actions/checkout@v4

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v4
      with:
        role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsRole
        aws-region: us-east-1

    - name: Deploy
      run: aws s3 sync ./dist s3://my-bucket
```

Benefits: No stored credentials, 1-hour lifetime, full audit trail.

### Pattern 6: Security Scanning

```yaml
security:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
      with:
        fetch-depth: 0

    - name: Gitleaks (secret detection)
      uses: gitleaks/gitleaks-action@v2

    - name: Snyk (vulnerability scan)
      uses: snyk/actions/node@master
      env:
        SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}

    - name: SBOM generation
      uses: anchore/sbom-action@v0
      with:
        format: spdx-json
        output-file: sbom.spdx.json
```

## Caching

### Automatic Dependency Caching

```yaml
- uses: actions/setup-node@v4
  with:
    node-version: 20
    cache: 'npm'  # Auto-caches ~/.npm
- run: npm ci
```

Supported: npm, yarn, pnpm, pip, poetry, cargo, go

### Manual Cache Control

```yaml
- uses: actions/cache@v4
  with:
    path: |
      ~/.cargo/bin
      ~/.cargo/registry
      target/
    key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
    restore-keys: |
      ${{ runner.os }}-cargo-
```

### Multi-Layer Caching (Nx)

```yaml
- name: Nx Cloud (build outputs)
  run: npx nx affected -t build
  env:
    NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}

- name: Vite Cache
  uses: actions/cache@v4
  with:
    path: '**/node_modules/.vite'
    key: vite-${{ hashFiles('package-lock.json') }}

- name: TypeScript Cache
  uses: actions/cache@v4
  with:
    path: '**/tsconfig.tsbuildinfo'
    key: tsc-${{ hashFiles('tsconfig.json') }}
```

Result: 70-90% build time reduction.

## Parallelization

### Job-Level Parallelization

```yaml
jobs:
  unit-tests:
    steps:
      - run: npm run test:unit

  integration-tests:
    steps:
      - run: npm run test:integration

  e2e-tests:
    steps:
      - run: npm run test:e2e
```

All three run simultaneously.

### Test Sharding

```yaml
test:
  strategy:
    matrix:
      shard: [1, 2, 3, 4]
  steps:
    - run: npm test -- --shard=${{ matrix.shard }}/4
```

20min test suite → 5min (4x speedup).

## Language Examples

### Python

```yaml
test:
  strategy:
    matrix:
      python-version: ['3.10', '3.11', '3.12']
  steps:
    - uses: actions/setup-python@v5
      with:
        python-version: ${{ matrix.python-version }}
    - run: pipx install poetry
    - run: poetry install
    - run: poetry run ruff check .
    - run: poetry run mypy .
    - run: poetry run pytest --cov
```

### Rust

```yaml
test:
  strategy:
    matrix:
      os: [ubuntu-latest, windows-latest, macos-latest]
      rust: [stable, nightly]
  steps:
    - uses: dtolnay/rust-toolchain@master
      with:
        toolchain: ${{ matrix.rust }}
        components: rustfmt, clippy
    - uses: Swatinem/rust-cache@v2
    - run: cargo fmt -- --check
    - run: cargo clippy -- -D warnings
    - run: cargo test
```

### Go

```yaml
test:
  steps:
    - uses: actions/setup-go@v5
      with:
        go-version: '1.23'
        cache: true
    - run: go mod verify
    - uses: golangci/golangci-lint-action@v4
    - run: go test -v -race -coverprofile=coverage.txt ./...
```

### TypeScript

```yaml
test:
  strategy:
    matrix:
      node-version: [18, 20, 22]
  steps:
    - uses: pnpm/action-setup@v3
      with:
        version: 8
    - uses: actions/setup-node@v4
      with:
        node-version: ${{ matrix.node-version }}
        cache: 'pnpm'
    - run: pnpm install --frozen-lockfile
    - run: pnpm run lint
    - run: pnpm run type-check
    - run: pnpm test
```

## Best Practices

### Security

**DO:**
- Use OIDC instead of long-lived credentials
- Pin actions to commit SHA: `actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11`
- Restrict permissions: `permissions: { contents: read }`
- Scan secrets (Gitleaks) on every commit
- Generate SLSA provenance for releases

**DON'T:**
- Expose secrets in logs
- Use `pull_request_target` without validation
- Trust unverified third-party actions

### Performance

**DO:**
- Use affected detection for monorepos
- Cache dependencies and build outputs
- Parallelize independent jobs
- Fail fast: `strategy.fail-fast: true`
- Use remote caching (Turborepo/Nx Cloud)

**DON'T:**
- Rebuild everything on every commit
- Run long tests in PR checks
- Use generic cache keys

### Debugging

```yaml
# Enable debug logging
env:
  ACTIONS_STEP_DEBUG: true
  ACTIONS_RUNNER_DEBUG: true

# SSH into runner
- uses: mxschmitt/action-tmate@v3
```

## Advanced Patterns

For detailed guides, see references:

- **github-actions-patterns.md** - Reusable workflows, composite actions, matrix strategies, OIDC setup
- **gitlab-ci-patterns.md** - Parent-child pipelines, dynamic generation, runner configuration
- **argo-workflows-guide.md** - DAG templates, artifact passing, event-driven triggers
- **slsa-security-framework.md** - SLSA Levels 1-4, provenance generation, cosign verification
- **monorepo-ci-strategies.md** - Turborepo/Nx/Bazel affected detection algorithms
- **caching-strategies.md** - Multi-layer caching, Docker optimization, cache invalidation
- **parallelization-patterns.md** - Test sharding, job dependencies, DAG design
- **secrets-management.md** - OIDC for AWS/GCP/Azure, Vault integration, rotation

## Examples

Complete runnable workflows:

- **examples/github-actions-basic/** - Starter template (lint/test/build)
- **examples/github-actions-monorepo/** - Turborepo with remote caching
- **examples/github-actions-slsa/** - SLSA Level 3 provenance
- **examples/gitlab-ci-monorepo/** - Parent-child dynamic pipeline
- **examples/argo-workflows-dag/** - Diamond DAG parallelization
- **examples/multi-language-matrix/** - Cross-platform testing

## Utility Scripts

Token-free execution:

- **scripts/validate_workflow.py** - Validate YAML syntax and best practices
- **scripts/generate_github_workflow.py** - Generate workflow from template
- **scripts/analyze_ci_performance.py** - CI metrics analysis
- **scripts/setup_oidc_aws.py** - Automate AWS OIDC setup

## Related Skills

**testing-strategies** - Test execution strategies (unit, integration, E2E)
**deploying-applications** - Deployment automation and GitOps
**auth-security** - Secrets management and authentication
**observability** - Pipeline monitoring and alerting