Score match quality and load results into Postgres
Use eval/score.py to compute precision, recall, and F1 for a reconciliation run against the ground_truth.json answer key, then optionally load the same run into a Postgres target schema.
HLAB field notes
Python
Stories and analysis filed under Python.
26 storiesReconcile ERP and CRM records with the pipeline
Run the reconcile CLI to ingest, clean, validate, and resolve ERP and CRM exports into a single crosswalk, quarantine log, and Markdown reconciliation report.
Customize messiness and build a reproducible configuration file
Tune the missing-value, bad-date, encoding, duplicate, orphan, and amount-drift ratios that control how messy a generated dataset is, and save your choices to a config file datagen can replay exactly.
Compare ERP and CRM field mappings and the ground truth answer key
Read the same customer, invoice, and payment across a flat ERP CSV export and a nested CRM JSON export, and learn what the ground_truth.json answer key does, and doesn't, record.
Generate a synthetic ERP and CRM dataset
Use the datagen CLI to produce a reproducible, deliberately messy ERP and CRM export pair, complete with a ground-truth answer key, the foundation every reconciliation lesson in this series runs against.
Test model output with deterministic rules
Add deterministic rule checks to a prompt-evaluation workflow, then measure exactly where simple substring rules produce false positives and false negatives.
Benchmark retrieval quality
Turn six labeled retrieval questions into a repeatable BM25 quality report using Hit@k, Recall@k, and mean reciprocal rank, then use the report to find what lexical retrieval misses.
Merge rankings with Reciprocal Rank Fusion
Combine BM25 and vector rankings with Reciprocal Rank Fusion, promoting chunks found by multiple retrievers without ever comparing their incompatible raw scores.
Build and search a vector index
Build a small exact vector index that stores L2-normalized vectors, ranks results by cosine similarity, and rejects vectors from an incompatible model or dimension.
Implement Okapi BM25 from scratch
Build the keyword-ranking function behind a local retrieval pipeline: preserve identifiers, calculate Okapi BM25 scores, and return deterministic ranked results without a search library.
Chunk Python and Markdown for retrieval
Build the ingestion pipeline that splits Python and Markdown files into stable, citable chunks, the foundation every retrieval guide in this series searches.
Lesson 1: A Step by Step Guide in Passing Data to Excel and Generating a Pie Chart using Python 3.7, Openpyxl and Pandas.
Write data from Python to an Excel workbook with pandas and openpyxl, then generate a pie chart from it. Covers Anaconda setup and Python 3.7.
Lesson 3: A Step by Step Tutorial on How to connect our Github Repository with Travis CI in Test-Driven Development Using Django 2.2 and Python 3.7.
Connect a Django 2.2 GitHub repository to Travis CI so your test suite runs on every push, and catch failing tests before they reach production.
Lesson 2: A Step by Step Tutorial on How to Create a Blog Application in Test-Driven Development Using Django 2.2 and Python 3.7.
Gather requirements as user stories, learn how functional tests differ from unit tests, and start the Django 2.2 blog that you build with TDD.
Lesson 5: A Step by Step Tutorial on How to Register Confirmation and Validation URL’s to M-Pesa C2B Integration on Daraja Using Django 2.2 and Python 3.7
Register M-Pesa C2B confirmation and validation URLs on the Safaricom Daraja API from Django 2.2, and build the views that receive payment callbacks.
Lesson 4: A Step by Step Tutorial on How to develop C2B Integration To M-Pesa On Daraja Using Django 2.2 and Python 3.7
Create the Django 2.2 models and migrations that store M-Pesa payments, and start building a customer-to-business (C2B) integration on Daraja.
Lesson 2: A Step by Step Tutorial on How to Generate Mpesa Access Token Using Django 2.2 and Python 3.7.
Generate an OAuth access token for the Safaricom M-Pesa Daraja API from a Django 2.2 application, the first call every M-Pesa integration makes.
Lesson 1: A Step by Step Tutorial on How to Integrate to Mpesa API Using Django 2.2 and Python 3.7
Start integrating the Safaricom M-Pesa Daraja API with Django 2.2: learn how M-Pesa works, create a developer account, and add a sandbox app.
Lesson 4: A Step by Step guide on Showing Registered Users, activating & deactivating users in Django 2.2 and Python 3.7
List registered users on a custom Django 2.2 dashboard, and let administrators activate or deactivate each account with a single click.
Lesson 3. A Step by Step guide on Creating Custom Dashboard and Login User in Django 2.2 and Python 3.7.
Build a custom dashboard in Django 2.2, redirect users there after they register, and add a sign-in view with the built-in authentication system.
Lesson 2: A Step by Step Guide on How to Use Bootstrap and Register User in Django 2.2 Authentication System and Python 3.7
Style a Django 2.2 project with the Bootstrap front-end framework, and build a registration form that creates users with the built-in auth system.
Lesson 1: A Step by Step Guide on How To Use Django 2.2 Authentication System and Python 3.7
Set up a Django 2.2 project and learn how the built-in authentication system handles user accounts, permissions, sessions, and passwords.
Lesson 1: A Step by Step Guide to Learning Data Science Using Python
Learn what data science is, see how hotels and insurers use it to make decisions, and get started with the Python tools you need for data analysis.
Learn How to Do Web Scraping Using Python 3.7 and Beautiful Soup 4
Scrape blog posts from a website with Python 3.7, Requests, and Beautiful Soup 4, then write the titles and links you extract to a CSV file.
Part II: How to Sign up User and Send confirmation Email in Django 2.1 and Python 3.6
Build a Django 2.1 sign-up form that creates inactive accounts and emails each user a token link to confirm their address and activate the account.
Part I: Email verification upon sign up in Django 2.1 and Python 3.6
Set up a Django 2.1 project for email verification on sign-up, and learn why confirming that users own their email address protects your application.