Review Data Pipeline

English | 中文

Review data pipeline changes with evidence-first, risk-prioritized findings. Focus on defects that can produce wrong, missing, duplicated, late, or unrecoverable data before discussing maintainability or style.

Required Inputs

Before reviewing, establish a fixed review point and obtain the exact diff:

• a base and head commit, tag, or branch;
• a merge base and head commit;
• a specific commit; or
• a complete patch supplied by the user.

If the review point is ambiguous, ask the user to choose one. Do not review a moving working tree without recording the current commit and diff. Inspect the changed files plus the minimum surrounding code, schema, tests, and configuration needed to understand their behavior.

If no diff or fixed review point is available, stop and request it. Do not infer changes from filenames, a task description, or an issue alone.

Review Priority

Always review and report in this order:

1. Data correctness and incident risk.
2. Grain, keys, join cardinality, and duplicates.
3. Time, time zones, and partitions.
4. Idempotency and reruns.
5. Performance and resource risk.
6. Observability and the minimum validation plan.

Use references/review-checklist.md to guide the review. Apply only checks relevant to the diff; the checklist is not evidence that a defect exists.

Evidence Rules

• Every finding must be supported by the diff and relevant repository context.
• Cite the narrowest useful file path and line range, function, query block, or configuration key.
• Explain the concrete execution path from changed code to impact.
• Distinguish observed behavior from assumptions. Resolve assumptions by inspecting code or asking a focused question.
• Do not invent runtime statistics, table properties, key uniqueness, scheduler behavior, or business semantics.
• If evidence is insufficient, record an open question or validation gap instead of a finding.
• Do not report pre-existing defects unless the change newly exposes or worsens them; label such context explicitly.

Severity

• Critical: The change can cause materially wrong data, destructive or unrecoverable writes, severe data loss, or a production failure with broad impact. It must be fixed before merge.
• Major: The change contains an important correctness, reliability, scalability, or operability defect that should be fixed before merge.
• Minor: The change has a bounded improvement opportunity that does not currently threaten correctness or safe operation.

Do not inflate severity. Style preferences are not findings unless they create a demonstrated maintenance or operational risk.

Output Format

Start with ## Findings. Sort findings by Critical, then Major, then Minor, while preserving the review priority within each severity.

Use this exact structure for every finding:

### [Critical|Major|Minor] <concise title>
- Location: `<path>:<line-range>` — `<symbol or query block>`
- Impact: <specific user, data, operational, or resource consequence>
- Evidence: <what the diff and surrounding code demonstrate>
- Recommendation: <smallest practical correction>
- Confidence: High | Medium | Low

Do not omit any field. A low-confidence concern without enough evidence should normally be an open question, not a finding.

If there are no findings, write No findings. Do not claim the change is risk-free.

After findings, always include:

1. ## Open Questions — unresolved facts that could change the review, or None.
2. ## Patch Summary — three to five concise bullets describing the reviewed change.
3. ## Validation Plan — the smallest executable checks needed before merge.
4. ## Residual Risks — relevant risks not eliminated by the available evidence.
5. ## Recommendation — Merge, Merge after fixes, or Do not merge.

See references/examples.md for complete examples.

Related Skills

• Live row-count, duplicate, null-key, or join-amplification incidents: investigate-data-quality.
• YARN application failed, killed, or stalled at runtime: debug-spark-yarn.
• Historical partition repair or recomputation planning: plan-safe-backfill.
• Production change request covering schema, rollout, and rollback: prepare-production-data-change.

Safety

Treat review as read-only unless the user separately asks for changes. Never run a data write, deployment, backfill, destructive command, or expensive unbounded query as part of review. Prefer static inspection and existing test results. Before proposing a validation query, bound it by partition or a small fixture and identify any possible cost or data exposure. Redact credentials, connection strings, personal data, and proprietary values from quoted evidence.