[PR #209] [MERGED] test: check test data against the API schema instead of golden files #207

New issue

Closed

opened 2026-10-13 19:48:52 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-13 19:48:52 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Glitchy-Sheep/boosty-downloader/pull/209
Author: @Glitchy-Sheep
Created: 9/23/2026
Status: ✅ Merged
Merged: 9/23/2026
Merged by: @Glitchy-Sheep

Base: main ← Head: test/schema-instead-of-goldens

📝 Commits (4)

dc2647d test: build the test post in code instead of a captured fixture
c784b02 test: check the synthetic post against the observed API schema
6281862 test: check the page and console views by their lines, drop the golden files
bc4b594 docs: describe test data without golden files

📊 Changes

29 files changed (+946 additions, -974 deletions)

View changed files

📝 docs/development/02-testing.md (+7 -7)
📝 pyproject.toml (+3 -1)
📝 test/ABOUT_TESTING.md (+1 -1)
📝 test/e2e/download_flow_test.py (+17 -12)
➖ test/fixtures/rendered_post.html (+0 -355)
➖ test/fixtures/single_post.json (+0 -275)
➖ test/fixtures/single_post_domain.json (+0 -96)
➖ test/fixtures/views/blog_overview_empty.txt (+0 -3)
➖ test/fixtures/views/blog_overview_locked.txt (+0 -18)
➖ test/fixtures/views/blog_overview_own.txt (+0 -15)
➖ test/fixtures/views/blog_overview_posts.txt (+0 -29)
➖ test/fixtures/views/download_plan_full.txt (+0 -17)
➖ test/fixtures/views/download_plan_nothing.txt (+0 -10)
➖ test/fixtures/views/run_statistics_full.txt (+0 -13)
➖ test/fixtures/views/run_statistics_nothing.txt (+0 -4)
➕ test/support/__init__.py (+0 -0)
➕ test/support/synthetic_post.py (+196 -0)
📝 test/unit/boosty_api/post_url_test.py (+1 -1)
📝 test/unit/cli/views/blog_overview_test.py (+57 -9)
📝 test/unit/cli/views/conftest.py (+19 -16)

...and 9 more files

📄 Description

📝 Summary

The test tree no longer stores data captured from the live API or golden files. Test data is built in code and checked against the observed API schema from #203.

🔧 What changes

test/support/synthetic_post.py builds a post with every chunk kind in code. It replaces the captured single_post.json; the e2e run and the mapper test use it.
The mapper test checks the domain result directly: chunk order, signed links, the video quality, the unknown chunk. It replaces the single_post_domain.json snapshot.
test/unit/synthetic_data/synthetic_post_schema_test.py validates the synthetic post against docs/api/boosty-api.yaml: every key must exist in the schema with the observed type and values. Keys the schema calls required may be missing, the post carries only what the client reads. The unknown chunk must stay unknown to the schema.
A guard test fails on real post ids and real Boosty links anywhere in the tests.
The page test checks every block of the showcase page, its order and escaping, instead of comparing rendered_post.html byte by byte.
Console views are checked by their phrases in order. Spacing and line wrap are ignored, wording and numbers are not. The 8 files in test/fixtures/views/ are gone.
UPDATE_GOLDEN is gone. pytest gets test/ on its path, so support imports work when one folder runs alone.
New dev dependency: jsonschema.

💡 Why

A captured post can leak real data and freezes whatever Boosty sent on one day. Golden files break on every harmless layout change and get regenerated without a real review. The schema gives one source of truth for what Boosty sends: when it changes, the schema is regenerated and the synthetic post must follow.

🧪 Tests

353 tests pass, task check is clean: ruff, pyright strict, import-linter.
The schema test catches a made-up key, a wrong type, an unknown video quality and an extra key in a chunk.

⏭️ Next

The canary also checks live answers against the schema.

✅ Checklist

Tested live on a real blog (feature or fix)
Updated CLI --help and README if needed
Updated docs/ if the download flow or dev process changed
Added a line to ## Unreleased in CHANGELOG.md (or the ci/skip-changelog label if not user-facing)

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Glitchy-Sheep/boosty-downloader/pull/209 **Author:** [@Glitchy-Sheep](https://github.com/Glitchy-Sheep) **Created:** 9/23/2026 **Status:** ✅ Merged **Merged:** 9/23/2026 **Merged by:** [@Glitchy-Sheep](https://github.com/Glitchy-Sheep) **Base:** `main` ← **Head:** `test/schema-instead-of-goldens` --- ### 📝 Commits (4) - [`dc2647d`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/dc2647d1e15b3164f3e54b94b10759ecf4e938b4) test: build the test post in code instead of a captured fixture - [`c784b02`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/c784b02a0caa5b6d137d8bf306b42154bd415fde) test: check the synthetic post against the observed API schema - [`6281862`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/6281862ca8395b9361485ad68fc0cef022df0e75) test: check the page and console views by their lines, drop the golden files - [`bc4b594`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/bc4b5943f6ed1048105b35e7b90ceae0d6446d77) docs: describe test data without golden files ### 📊 Changes **29 files changed** (+946 additions, -974 deletions) <details> <summary>View changed files</summary> 📝 `docs/development/02-testing.md` (+7 -7) 📝 `pyproject.toml` (+3 -1) 📝 `test/ABOUT_TESTING.md` (+1 -1) 📝 `test/e2e/download_flow_test.py` (+17 -12) ➖ `test/fixtures/rendered_post.html` (+0 -355) ➖ `test/fixtures/single_post.json` (+0 -275) ➖ `test/fixtures/single_post_domain.json` (+0 -96) ➖ `test/fixtures/views/blog_overview_empty.txt` (+0 -3) ➖ `test/fixtures/views/blog_overview_locked.txt` (+0 -18) ➖ `test/fixtures/views/blog_overview_own.txt` (+0 -15) ➖ `test/fixtures/views/blog_overview_posts.txt` (+0 -29) ➖ `test/fixtures/views/download_plan_full.txt` (+0 -17) ➖ `test/fixtures/views/download_plan_nothing.txt` (+0 -10) ➖ `test/fixtures/views/run_statistics_full.txt` (+0 -13) ➖ `test/fixtures/views/run_statistics_nothing.txt` (+0 -4) ➕ `test/support/__init__.py` (+0 -0) ➕ `test/support/synthetic_post.py` (+196 -0) 📝 `test/unit/boosty_api/post_url_test.py` (+1 -1) 📝 `test/unit/cli/views/blog_overview_test.py` (+57 -9) 📝 `test/unit/cli/views/conftest.py` (+19 -16) _...and 9 more files_ </details> ### 📄 Description ## 📝 Summary The test tree no longer stores data captured from the live API or golden files. Test data is built in code and checked against the observed API schema from #203. ## 🔧 What changes - `test/support/synthetic_post.py` builds a post with every chunk kind in code. It replaces the captured `single_post.json`; the e2e run and the mapper test use it. - The mapper test checks the domain result directly: chunk order, signed links, the video quality, the unknown chunk. It replaces the `single_post_domain.json` snapshot. - `test/unit/synthetic_data/synthetic_post_schema_test.py` validates the synthetic post against `docs/api/boosty-api.yaml`: every key must exist in the schema with the observed type and values. Keys the schema calls required may be missing, the post carries only what the client reads. The unknown chunk must stay unknown to the schema. - A guard test fails on real post ids and real Boosty links anywhere in the tests. - The page test checks every block of the showcase page, its order and escaping, instead of comparing `rendered_post.html` byte by byte. - Console views are checked by their phrases in order. Spacing and line wrap are ignored, wording and numbers are not. The 8 files in `test/fixtures/views/` are gone. - `UPDATE_GOLDEN` is gone. pytest gets `test/` on its path, so `support` imports work when one folder runs alone. - New dev dependency: `jsonschema`. ## 💡 Why A captured post can leak real data and freezes whatever Boosty sent on one day. Golden files break on every harmless layout change and get regenerated without a real review. The schema gives one source of truth for what Boosty sends: when it changes, the schema is regenerated and the synthetic post must follow. ## 🧪 Tests - 353 tests pass, `task check` is clean: ruff, pyright strict, import-linter. - The schema test catches a made-up key, a wrong type, an unknown video quality and an extra key in a chunk. ## ⏭️ Next - The canary also checks live answers against the schema. ## ✅ Checklist  - [x] Tested live on a real blog (feature or fix) - [x] Updated CLI `--help` and README if needed - [x] Updated `docs/` if the download flow or dev process changed - [x] Added a line to `## Unreleased` in CHANGELOG.md (or the `ci/skip-changelog` label if not user-facing) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-13 19:48:52 +00:00