🧩
DataJavaScript

Engineering Data Pipelines

by telagod

Engineering Data Pipelines is a Data skill for Claude Code, published by telagod in code-abyss.

239 stars32 forkson telagod/code-abyssAdded 2026/09/03Repository updated 2026/07/19
agent-skillsai-agentai-assistantai-personalityblue-teamcharacter-cardclaude-codeclicodexcodex-cliconfigurationdeveloper-toolsdevopsgemini-clipersonaprompt-engineeringred-teamsecurityskills
Install in seconds
Install Engineering Data Pipelines
Copy Engineering Data Pipelines into your Claude Code skills folder. Run the command in your terminal, or review the source on GitHub before installing.
terminal
npx degit https://github.com/telagod/code-abyss/tree/main/skills/engineering-data-pipelines ~/.claude/skills/engineering-data-pipelines

Requires Node.js. Downloads this skill only — not the rest of the repository — into your Claude Code skills folder.

Without Node.js

git clone https://github.com/telagod/code-abyss.git

Clones the whole repository, then copy the skill’s own directory into your skills folder yourself.

In this catalog

Source file
skills/engineering-data-pipelines/SKILL.md in telagod/code-abyss
Installs to
~/.claude/skills/engineering-data-pipelines
Collection
One of 25 skills cataloged from this repository
Category
Data710 skills

What Engineering Data Pipelines does

Engineering Data Pipelines covers Airflow, Dagster, Kafka Streams, Flink, dbt, and data quality checks for building ETL workflows and stream processing systems. Use it when designing orchestration, incremental loads, or validation checks.

Engineering Data Pipelines is cataloged under Data on DirSkills. Engineering Data Pipelines comes from a repository tagged agent-skills, ai-agent, ai-assistant, ai-personality and blue-team.

Documentation

README

数据工程域 · Data Engineering

判断先于执行:决定「是否做 / 选什么 / 如何取舍」(栈、方案、架构、权衡)前,先读领域判断内核 skills/_kernel/backend/SKILL.md——它管 judgment,本秘典管 execution;冲突时以内核判断为准。

编排:Airflow(调度) | Dagster(资产) | Prefect(现代流)
流处理:Kafka Streams(嵌入式) | Flink(集群) | Spark Streaming
质量:Great Expectations | dbt tests | Soda Core

编排检查项

幂等(UPSERT/分区覆盖) | 增量(WHERE updated_at > last_run) | 事件驱动触发 | 跨 DAG 依赖 | 数据血缘(ref()/Asset deps)

流处理检查项

时间语义选择 | Watermark 乱序容忍 | 状态 TTL 防膨胀 | Checkpoint 间隔 | 端到端 Exactly-Once | 背压监控

质量检查项

分层验证(源→转换→目标) | 完整性+准确性+一致性 | 及时性阈值 | 加权评分 | 告警(Slack/PagerDuty)

工具对比、API 用法、质量维度详见 references/details.md

Frequently asked about Engineering Data Pipelines

  • What else does telagod publish alongside Engineering Data Pipelines?

    Engineering Data Pipelines is one of 25 skills that DirSkills catalogs from telagod/code-abyss, the repository it ships in. Its siblings there include Adversarial Review Orchestration, Analyzing Changes and Analyzing Spreadsheets. Each one is a separate skill with its own page in this directory, installs the same way Engineering Data Pipelines does, and is maintained by telagod in that same repository. The rest of the collection is listed on the telagod/code-abyss page.

  • How does Engineering Data Pipelines compare to other Data skills?

    Engineering Data Pipelines ranks #634 by stars among the 710 Data skills in this catalog. The most-starred ones next to it are Benchmark Methodology, Jupyter Notebook and Solana. DirSkills ranks by the star count of the repository each skill ships in, so that order reflects how popular those repositories are rather than any review of Engineering Data Pipelines against them. Open each page to compare what they document and how they install.

More from telagod/code-abyss

Engineering Data Pipelines is one of 25 skills cataloged on DirSkills from telagod/code-abyss.

See all 25 skills
🛡️
7h ago

Adversarial Review Orchestration

Adversarial Review Orchestration fans out findings, then verifies each one through multiple skeptical perspectives before accepting conclusions. Use it for security audits, code reviews, research synthesis, or migrations when one agent’s self-report cannot be trusted.
AI Engineering
23932
🔎
7h ago

Analyzing Changes

Analyzing Changes analyzes code diffs to estimate impact scope, flag documentation drift, and check whether tests or design docs need updates. Use it before commits, after refactors, or when reviewing cross-module changes.
Quality
23932
📊
7h ago

Analyzing Spreadsheets

Analyzing Spreadsheets processes Excel and CSV files to build workbooks, add formulas, preserve formatting, and analyze tabular data. Use it when working with spreadsheets or validating financial models with recalculation and zero formula errors.
Data
23932
🛡️
7h ago

Application Security Defense

Application Security Defense covers web, API, GraphQL, auth, and LLM application hardening. Use it when reviewing code, fixing CVE-class bugs, designing safer auth flows, or threat-modeling app-layer defenses.
AI Engineering
23932
🎨
7h ago

Applying UI Design System

Applying UI Design System helps choose and implement a UI aesthetic such as Glassmorphism, Liquid Glass, Neubrutalism, or Claymorphism. Use it when building components, setting design tokens, or checking contrast, focus, motion, and dark mode behavior.
Frontend
23932
🏗️
7h ago

Architecture Design

Architecture Design covers API design, security architecture, cloud-native patterns, caching, message queues, and data security. Use it when choosing system structure, integration style, or infrastructure patterns.
DevOps
23932