[PR #196] [MERGED] [downloader] Write post.html even when a piece is missing #195

New issue

Closed

opened 2026-10-13 19:48:40 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-13 19:48:40 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Glitchy-Sheep/boosty-downloader/pull/196
Author: @Glitchy-Sheep
Created: 9/20/2026
Status: ✅ Merged
Merged: 9/20/2026
Merged by: @Glitchy-Sheep

Base: main ← Head: fix/page-with-placeholders

📝 Commits (3)

5e7e33c [html] Add a placeholder for a media piece that did not download
5405689 [downloader] Write post.html even when a piece is missing
f82eaea [docs] Describe placeholders for missing media

📊 Changes

14 files changed (+374 additions, -27 deletions)

View changed files

📝 CHANGELOG.md (+5 -0)
📝 README.md (+1 -0)
📝 boosty_downloader/application/post_retry.py (+1 -0)
📝 boosty_downloader/application/use_cases/download_single_post.py (+69 -12)
📝 boosty_downloader/infrastructure/html_generator/__init__.py (+4 -0)
📝 boosty_downloader/infrastructure/html_generator/models.py (+27 -1)
📝 boosty_downloader/infrastructure/html_generator/renderer.py (+12 -1)
📝 boosty_downloader/infrastructure/html_generator/templates/base.html (+36 -0)
➕ boosty_downloader/infrastructure/html_generator/templates/unavailable.html (+10 -0)
📝 docs/concepts/boosty/download-flow.md (+3 -1)
📝 test/e2e/download_flow_test.py (+48 -3)
📝 test/fixtures/rendered_post.html (+51 -0)
📝 test/unit/html_generator/html_templates_test.py (+39 -0)
📝 test/unit/use_cases/parallel_media_test.py (+68 -9)

📄 Description

📝 Summary

Groundwork for #76. A post with one piece that will not download stays readable.

🔧 What changes

post.html is always written from what landed.
A missing picture, video or audio track leaves a dashed card at its place: what it is, its name, the reason from the download error. An external video also links to the original.
A failed attachment stays off the page. Only downloaded files get a card.
A page with a placeholder is never complete: post_content stays out of the cache while the hole is there. The retry and the next run process the post again and rewrite the page. The cache rule of #188 is unchanged.

💡 Why

Before, one failed image, video or audio meant no post.html at all. The text of the post was saved nowhere. A permanent failure left the post unreadable forever.

Bug fixed on the way

A page rewritten after a retry lost the videos, audio and files of the first attempt: their content types were already cached, the retry skipped them, the rewrite had nothing to show. The use case now keeps the page elements of every attempt and rewrites the page from all of them. A post rebuilt with fresh signed links takes the page over. The new e2e test caught it.

🚫 Not changed on purpose

Retries and cache: 5 attempts, per-type caching, the streak breaker.
Across runs the page is still a snapshot: a piece cached by an earlier run is not on a page rewritten today. Rebuilding pages from disk is the manifest stage.

📚 Docs

docs/concepts/boosty/download-flow.md: when the page is written, what a placeholder means, why such a page is never cached, the in-run memory and the cross-run limit.
README: a missing picture or video leaves a note instead of hiding the post.

🧪 Tests

Renderer: the card names the piece, escapes the name, shows the reason, links to the original; an image has no name and no link. The showcase golden pins a missing external video and a missing image.
Use case: a failed image becomes a placeholder and keeps post_content uncached; a successful retry rewrites the page without it; a retry keeps the elements of the earlier attempt, also after a rebuild for fresh links.
e2e: an image that answers 404 on every attempt leaves a failed post with a readable page: the placeholder with the reason, the video and the file. The next run asks for the image alone, 5 times. The retried-file scenario now expects the file card after the retry.
task ci green locally: ruff, pyright strict, import-linter, 293 tests + e2e.
Live on a small blog without failures: no placeholder rendered, page bodies byte-identical to the previous run.

✅ Checklist

Tested live on a real blog (feature or fix)
Updated CLI --help and README if needed
Updated docs/ if the download flow or dev process changed
Added a line to ## Unreleased in CHANGELOG.md (or the ci/skip-changelog label if not user-facing)

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Glitchy-Sheep/boosty-downloader/pull/196 **Author:** [@Glitchy-Sheep](https://github.com/Glitchy-Sheep) **Created:** 9/20/2026 **Status:** ✅ Merged **Merged:** 9/20/2026 **Merged by:** [@Glitchy-Sheep](https://github.com/Glitchy-Sheep) **Base:** `main` ← **Head:** `fix/page-with-placeholders` --- ### 📝 Commits (3) - [`5e7e33c`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/5e7e33c4283b60b729864718c6700eb63c622da3) [html] Add a placeholder for a media piece that did not download - [`5405689`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/5405689d738e36c6d3650537fe4683e95f342edb) [downloader] Write post.html even when a piece is missing - [`f82eaea`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/f82eaeaefd0835293a56fe1837c1b8f35a3ae9f1) [docs] Describe placeholders for missing media ### 📊 Changes **14 files changed** (+374 additions, -27 deletions) <details> <summary>View changed files</summary> 📝 `CHANGELOG.md` (+5 -0) 📝 `README.md` (+1 -0) 📝 `boosty_downloader/application/post_retry.py` (+1 -0) 📝 `boosty_downloader/application/use_cases/download_single_post.py` (+69 -12) 📝 `boosty_downloader/infrastructure/html_generator/__init__.py` (+4 -0) 📝 `boosty_downloader/infrastructure/html_generator/models.py` (+27 -1) 📝 `boosty_downloader/infrastructure/html_generator/renderer.py` (+12 -1) 📝 `boosty_downloader/infrastructure/html_generator/templates/base.html` (+36 -0) ➕ `boosty_downloader/infrastructure/html_generator/templates/unavailable.html` (+10 -0) 📝 `docs/concepts/boosty/download-flow.md` (+3 -1) 📝 `test/e2e/download_flow_test.py` (+48 -3) 📝 `test/fixtures/rendered_post.html` (+51 -0) 📝 `test/unit/html_generator/html_templates_test.py` (+39 -0) 📝 `test/unit/use_cases/parallel_media_test.py` (+68 -9) </details> ### 📄 Description ## 📝 Summary Groundwork for #76. A post with one piece that will not download stays readable. ## 🔧 What changes - `post.html` is always written from what landed. - A missing picture, video or audio track leaves a dashed card at its place: what it is, its name, the reason from the download error. An external video also links to the original. - A failed attachment stays off the page. Only downloaded files get a card. - A page with a placeholder is never complete: `post_content` stays out of the cache while the hole is there. The retry and the next run process the post again and rewrite the page. The cache rule of #188 is unchanged. ## 💡 Why Before, one failed image, video or audio meant no `post.html` at all. The text of the post was saved nowhere. A permanent failure left the post unreadable forever. ## Bug fixed on the way A page rewritten after a retry lost the videos, audio and files of the first attempt: their content types were already cached, the retry skipped them, the rewrite had nothing to show. The use case now keeps the page elements of every attempt and rewrites the page from all of them. A post rebuilt with fresh signed links takes the page over. The new e2e test caught it. ## 🚫 Not changed on purpose - Retries and cache: 5 attempts, per-type caching, the streak breaker. - Across runs the page is still a snapshot: a piece cached by an earlier run is not on a page rewritten today. Rebuilding pages from disk is the manifest stage. ## 📚 Docs - `docs/concepts/boosty/download-flow.md`: when the page is written, what a placeholder means, why such a page is never cached, the in-run memory and the cross-run limit. - README: a missing picture or video leaves a note instead of hiding the post. ## 🧪 Tests - Renderer: the card names the piece, escapes the name, shows the reason, links to the original; an image has no name and no link. The showcase golden pins a missing external video and a missing image. - Use case: a failed image becomes a placeholder and keeps `post_content` uncached; a successful retry rewrites the page without it; a retry keeps the elements of the earlier attempt, also after a rebuild for fresh links. - e2e: an image that answers 404 on every attempt leaves a failed post with a readable page: the placeholder with the reason, the video and the file. The next run asks for the image alone, 5 times. The retried-file scenario now expects the file card after the retry. - `task ci` green locally: ruff, pyright strict, import-linter, 293 tests + e2e. - Live on a small blog without failures: no placeholder rendered, page bodies byte-identical to the previous run. ## ✅ Checklist - [x] Tested live on a real blog (feature or fix) - [x] Updated CLI `--help` and README if needed - [x] Updated `docs/` if the download flow or dev process changed - [x] Added a line to `## Unreleased` in CHANGELOG.md (or the `ci/skip-changelog` label if not user-facing) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-13 19:48:40 +00:00