PUBLIC REPRODUCTION · 2026-08-27

Ten duplicate rows caught before publication

A generated open-source parts database provided a real test of the free Action: inspect the output, report exact duplicates, and block publication without guessing which records to rewrite.

1.1 MBpublic CSV snapshot
1,583data rows checked
10exact duplicates reported
0issues after exact-row dedupe

What was checked

The public CDFER/jlcpcb-parts-database project generates a CSV and publishes it through GitHub Pages. Running the released CSV Preflight implementation against the live snapshot found 10 data rows that were byte-for-byte equivalent after CSV parsing. The report identified their row numbers; it did not delete or merge them.

The narrow proposed fix

Public pull request #11 adds a full-row drop_duplicates() call after the generator merges its data, then runs the Action before the Pages artifact is uploaded. On the reproduced snapshot, this reduced 1,583 rows to 1,573 and the validator reported no remaining structural issue.

Status boundary: the pull request is an open contribution, not a customer result or an accepted upstream change. The repository's full multi-gigabyte generation process was not run locally, and the PR says so.

Why exact duplicates are the safe first boundary

Rows that merely share a part number or description may still represent distinct records. Automatically merging near-duplicates would introduce a business rule that a structural validator cannot justify. This check reports exact repeated rows; the owning project decides whether and where deduplication belongs.

Reusable workflow pattern

  1. Generate or update the CSV.
  2. Run the Action using an immutable commit SHA.
  3. Keep the issue report as an artifact even when validation fails.
  4. Upload or publish data only after the check passes.

The complete copy-ready workflow is in the GitHub Actions guide. It validates one generic UTF-8 CSV up to 10 MiB on the runner, requires no API key, and does not upload the input. Passing checks CSV structure only; it does not guarantee an importer accepts the file.

FIXED SCOPE · USD 99

Want this wired into one repository?

One public repository or sanitized minimal reproduction, one CSV path, one workflow, diagnostic artifacts, a usage note, and one revision. Scope is confirmed in writing before payment.

Check setup fit ↗