[PR #227] [MERGED] feat: persist post records as versioned JSON #225

New issue

Closed

opened 2026-10-13 19:49:04 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-13 19:49:04 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Glitchy-Sheep/boosty-downloader/pull/227
Author: @Glitchy-Sheep
Created: 10/11/2026
Status: ✅ Merged
Merged: 10/11/2026
Merged by: @Glitchy-Sheep

Base: main ← Head: feat/store-post-records-as-json

📝 Commits (4)

59d80de feat: define the JSON format for post content
5762763 feat: encode and validate versioned post records
6b023b5 feat: save post records atomically
af5721e test: intercept atomic replacement on Python 3.10

📊 Changes

8 files changed (+1942 additions, -0 deletions)

View changed files

➕ boosty_downloader/infrastructure/library/__init__.py (+1 -0)
➕ boosty_downloader/infrastructure/library/_post_json.py (+296 -0)
➕ boosty_downloader/infrastructure/library/_post_json_blocks.py (+215 -0)
➕ boosty_downloader/infrastructure/library/errors.py (+9 -0)
➕ boosty_downloader/infrastructure/library/json_store.py (+62 -0)
➕ test/unit/library/json_store_test.py (+420 -0)
➕ test/unit/library/post_json_blocks_test.py (+285 -0)
➕ test/unit/library/post_json_test.py (+654 -0)

📄 Description

📝 Summary

Add versioned post.json persistence for a complete saved post in a known folder. This lets future download runs retain the full body and media history across restarts, including when the first run downloaded only text. The current CLI download flow is unchanged.

🔧 What changes

Preserve metadata, sync state, ordered body blocks, nested text styles and all media states in readable UTF-8 JSON.
Validate schema version, identities, dates, relative paths and media references before returning a record or saving it.
Read independent snapshots without rewriting files; report invalid records and unsupported versions with file context.
Write a unique sibling temporary file, flush and sync it, close it, then atomically replace post.json. Failed or interrupted writes before replacement preserve the previous record.
Ignore abandoned temporary files and preserve media, pages and viewing marks.

🧪 Tests

task ci: 1037 tests passed; the full suite also passed on Python 3.10; Ruff, formatting, Pyright, import boundaries and wheel/sdist builds passed.
Golden JSON fixtures and round trips cover every body variant, metadata field and media state, plus records produced by the reconciliation pipeline.
Real filesystem tests cover write/flush/fsync/replace failures, overlapping writers and a child process stopped with os._exit before replacement.
Regression tests check that invalid content tags do not expose saved values in error messages or tracebacks.
Fresh independent review completed with no remaining material findings. All seven GitHub CI checks passed, including Ubuntu Python 3.10/3.12/3.14 and Windows Python 3.12.

⏭️ Next

Add folder naming and lookup by full post ID, then author records and viewing marks. Connect the completed store to downloads and dry-run planning in the download-flow integration step.

✅ Checklist

Tested live on a real blog (not applicable: storage functions are not wired into the CLI; real filesystem and process tests cover this change)
Updated CLI --help and README if needed (no CLI changes)
Updated docs/ if the download flow or dev process changed (neither changes)
Added a line to ## Unreleased in CHANGELOG.md (not user-facing; uses ci/skip-changelog)

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Glitchy-Sheep/boosty-downloader/pull/227 **Author:** [@Glitchy-Sheep](https://github.com/Glitchy-Sheep) **Created:** 10/11/2026 **Status:** ✅ Merged **Merged:** 10/11/2026 **Merged by:** [@Glitchy-Sheep](https://github.com/Glitchy-Sheep) **Base:** `main` ← **Head:** `feat/store-post-records-as-json` --- ### 📝 Commits (4) - [`59d80de`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/59d80de109607fd9dda4c7a40470ed416ac8c331) feat: define the JSON format for post content - [`5762763`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/57627639029b4a2e5bc95c0490d10b0380dca41f) feat: encode and validate versioned post records - [`6b023b5`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/6b023b550b6c7f61fb9f3a5c5ecc5a6c3e5482e0) feat: save post records atomically - [`af5721e`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/af5721efe2e34878b93a2177b7f1280116986445) test: intercept atomic replacement on Python 3.10 ### 📊 Changes **8 files changed** (+1942 additions, -0 deletions) <details> <summary>View changed files</summary> ➕ `boosty_downloader/infrastructure/library/__init__.py` (+1 -0) ➕ `boosty_downloader/infrastructure/library/_post_json.py` (+296 -0) ➕ `boosty_downloader/infrastructure/library/_post_json_blocks.py` (+215 -0) ➕ `boosty_downloader/infrastructure/library/errors.py` (+9 -0) ➕ `boosty_downloader/infrastructure/library/json_store.py` (+62 -0) ➕ `test/unit/library/json_store_test.py` (+420 -0) ➕ `test/unit/library/post_json_blocks_test.py` (+285 -0) ➕ `test/unit/library/post_json_test.py` (+654 -0) </details> ### 📄 Description ## 📝 Summary Add versioned `post.json` persistence for a complete saved post in a known folder. This lets future download runs retain the full body and media history across restarts, including when the first run downloaded only text. The current CLI download flow is unchanged. ## 🔧 What changes - Preserve metadata, sync state, ordered body blocks, nested text styles and all media states in readable UTF-8 JSON. - Validate schema version, identities, dates, relative paths and media references before returning a record or saving it. - Read independent snapshots without rewriting files; report invalid records and unsupported versions with file context. - Write a unique sibling temporary file, flush and sync it, close it, then atomically replace `post.json`. Failed or interrupted writes before replacement preserve the previous record. - Ignore abandoned temporary files and preserve media, pages and viewing marks. ## 🧪 Tests - `task ci`: 1037 tests passed; the full suite also passed on Python 3.10; Ruff, formatting, Pyright, import boundaries and wheel/sdist builds passed. - Golden JSON fixtures and round trips cover every body variant, metadata field and media state, plus records produced by the reconciliation pipeline. - Real filesystem tests cover write/flush/fsync/replace failures, overlapping writers and a child process stopped with `os._exit` before replacement. - Regression tests check that invalid content tags do not expose saved values in error messages or tracebacks. - Fresh independent review completed with no remaining material findings. [All seven GitHub CI checks passed](https://github.com/Glitchy-Sheep/boosty-downloader/actions/runs/38158635679), including Ubuntu Python 3.10/3.12/3.14 and Windows Python 3.12. ## ⏭️ Next Add folder naming and lookup by full post ID, then author records and viewing marks. Connect the completed store to downloads and dry-run planning in the download-flow integration step. ## ✅ Checklist - [x] Tested live on a real blog (not applicable: storage functions are not wired into the CLI; real filesystem and process tests cover this change) - [x] Updated CLI `--help` and README if needed (no CLI changes) - [x] Updated `docs/` if the download flow or dev process changed (neither changes) - [x] Added a line to `## Unreleased` in CHANGELOG.md (not user-facing; uses `ci/skip-changelog`) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-13 19:49:04 +00:00