Week 6: Python Packages, FINM, and the ChartBook Catalog

Week 6: Python Packages, FINM, and the ChartBook Catalog#

Learning Outcomes#

  • See the next two final-project proposals and give substantive peer feedback.

  • Understand the ChartBook catalog: how it pins together data dependencies between repositories, and how to set one up on your own machine (The ChartBook Catalog) using cloned pipelines from the FTSFR organization.

  • Understand what changed in this week’s ChartBook 0.1.x release (manifest format v2, catalog auto-discovery, single install) and how a release like this is actually cut: changelog, semantic versioning, hatch build, publish to PyPI.

  • Understand the anatomy of a published Python package (pyproject.toml, a build backend, versioning, CLI entry points), and know the packages we’ll keep working with: chartbook, finm, and the TestPyPI practice package stockbeta.

  • Know the plan for your Midterm Report figure: build it on data produced by an FTSFR pipeline, consumed through your catalog with data.load(...).

Announcements#

  • HW 3 is due this Friday, July 24.

  • Upgrade ChartBook: a new major release shipped this week. Run pip install --upgrade chartbook (you want 0.1.1 or later). It contains breaking changes we’ll discuss tonight; see The ChartBook Catalog.

  • Midterm report: claim your topic now if you haven’t. Every candidate topic is an open issue in finm-32900/finm_midterm_report; assign one to yourself to claim it. Your merged contribution is due Monday, August 10 at 11:59 pm CT. Don’t wait to open your pull request: an early draft PR gets you review comments while there is still time to act on them.

  • Peer-feedback survey for tonight’s two groups: same as last week. Submit during the quiet time after each presentation; the form stays open until Friday at 11:59 PM and doubles as the attendance record.

  • If you present July 30, your instructor consultation must happen this week; book it now if you haven’t.

Agenda Item 1: Proposal Presentations (Two Groups)#

Two more groups present tonight, in the order on the schedule: 10 minutes to present, 5 minutes of Q&A, then ~5 minutes of quiet time to submit the peer-feedback survey. Same reminder as last week: the presentation is an advertisement for the product you’ll build (rubric).

Agenda Item 2: The Midterm Report and the ChartBook Catalog#

First, a working check-in on the Midterm Report: confirm every topic issue has an owner, then review one or two open pull requests live, applying the PR workflow from last week.

Then the architecture question the report forces on us: what should your contribution do when it needs data that another repository already produces? Copying pull code duplicates work and drifts out of date. The answer is the ChartBook catalog, covered in tonight’s new chapter, The ChartBook Catalog: Data Dependencies Across Repositories:

  • Pipelines vs. catalogs: a catalog is a chartbook.toml with a [pipelines] registry, and the FTSFR organization (~20 pipeline repos plus one catalog repo) is the production-scale example.

  • Hands-on: clone FTSFR pipeline repos, register them in your global catalog (chartbook catalog add, or members auto-discovery), build and browse the combined site, and load another repo’s dataframe by name with data.load(pipeline=..., dataframe=...).

  • For your midterm figure: build it on data an FTSFR pipeline already produces. The chapter lists exactly which repos run on free public data or WRDS alone (eleven of them); the Bloomberg-dependent ones are off the menu.

  • Live demo: wire one midterm-report contribution to consume an FTSFR dataframe this way, so the report becomes a downstream consumer of maintained pipelines rather than a pile of one-off pulls.

Agenda Item 3: Shipping a Release: ChartBook 0.1.0 as a Case Study#

You have been pip install-ing chartbook all quarter, and this week it shipped a breaking release (0.1.0 on July 22, 0.1.1 on July 23). That makes it a live case study in how real packages evolve and how a release actually happens:

  • What changed and why: read the changelog together. Manifest format v2 (one [project] table, every field optional, de-prefixed entity names), scoped pipeline IDs like ftsfr/crsp_treasury, catalog auto-discovery, and the retirement of the [data]/[plotting]/[all] extras in favor of a single install.

  • Versioning as a contract: semantic versioning, why a breaking change in a 0.x package bumps the minor version (0.0.21 → 0.1.0), and how a migration script eases users across a breaking change.

  • How the release was cut: the walk from a green test suite to a published version — changelog entry, version bump, hatch build producing the wheel and sdist, publish to PyPI. I’ll show the actual process I used for this release.

  • Anatomy of the package along the way, from the chartbook repo: pyproject.toml as the single source of truth, the Hatchling build backend, dynamic versioning, and the one-line [project.scripts] entry (chartbook = "chartbook.cli:main") that creates the CLI command.

Agenda Item 4: finm, stockbeta, and Writing Your Own Package#

With a real release fresh in mind, work through Writing and Publishing Your Own Python Packages and the packages you can practice on:

  • finm (docs) is the course’s own published package: finm.fixedincome (the yield-curve functions you imported in HW 3), finm.data (Fama-French, Federal Reserve, He-Kelly-Manela, and other loaders), and finm.analytics (factor analysis). It’s the low-stakes place to practice the contribution loop on a package you don’t own: fork, branch, PR, review, merge — the same mechanics as the midterm report, now crossing a repository boundary you don’t control.

  • stockbeta is a previous year’s practice package, published to TestPyPI but not the real PyPI. TestPyPI is the rehearsal space: the full publish workflow, with none of the consequences of claiming a real package name.

  • Where this is heading: next week’s HW 4 (a minimal options case study) is built from the same ChartBook template as your other repos, and these packaging workflows are the muscles it exercises.

Contents