Score match quality and load results into Postgres
Use eval/score.py to compute precision, recall, and F1 for a reconciliation run against the ground_truth.json answer key, then optionally load the same run into a Postgres target schema.
HLAB field notes
Data Engineering
Stories and analysis filed under Data Engineering.
05 storiesReconcile ERP and CRM records with the pipeline
Run the reconcile CLI to ingest, clean, validate, and resolve ERP and CRM exports into a single crosswalk, quarantine log, and Markdown reconciliation report.
Customize messiness and build a reproducible configuration file
Tune the missing-value, bad-date, encoding, duplicate, orphan, and amount-drift ratios that control how messy a generated dataset is, and save your choices to a config file datagen can replay exactly.
Compare ERP and CRM field mappings and the ground truth answer key
Read the same customer, invoice, and payment across a flat ERP CSV export and a nested CRM JSON export, and learn what the ground_truth.json answer key does, and doesn't, record.
Generate a synthetic ERP and CRM dataset
Use the datagen CLI to produce a reproducible, deliberately messy ERP and CRM export pair, complete with a ground-truth answer key, the foundation every reconciliation lesson in this series runs against.