[PR #128] [MERGED] [downloader] Survive broken posts and stop early on failure streaks #132

New issue

Closed

opened 2026-10-13 19:47:46 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-13 19:47:46 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Glitchy-Sheep/boosty-downloader/pull/128
Author: @Glitchy-Sheep
Created: 8/9/2026
Status: ✅ Merged
Merged: 8/9/2026
Merged by: @Glitchy-Sheep

Base: main ← Head: fix/per-post-containment

📝 Commits (4)

f0a675c fix: 🐛 One broken post no longer kills or hollows out the download run
0dabff7 feat: 📢 Run summary lists posts that failed to download
b49c692 feat: ✨ --skip-all-failures flag disables the failure-streak stop
58b321e test: ✅ Containment, failure streak and Ctrl+C are guarded by tests

📊 Changes

10 files changed (+497 additions, -26 deletions)

View changed files

📝 CHANGELOG.md (+8 -0)
📝 README.md (+13 -0)
📝 boosty_downloader/main.py (+6 -0)
📝 boosty_downloader/src/application/exceptions/application_errors.py (+13 -0)
➕ boosty_downloader/src/application/failure_streak.py (+32 -0)
📝 boosty_downloader/src/application/use_cases/download_all_posts.py (+147 -23)
📝 boosty_downloader/src/cli/cli_options.py (+11 -0)
📝 boosty_downloader/src/cli/commands/download.py (+5 -0)
📝 boosty_downloader/src/infrastructure/boosty_api/utils/validation_errors.py (+14 -3)
➕ test/unit/use_cases/download_all_posts_test.py (+248 -0)

📄 Description

Summary

One post with an unexpected error used to kill the whole download run (#88). Now the run skips it, remembers it, and only stops early when failures come in a row - a sign the problem is the environment, not the posts.

Fixes #88

Problem

The per-post loop caught only two known error types; anything else flew up and killed the run
Real case from #88: a very long post title -> OSError: File name too long on folder creation -> the run is dead, everything after that post is not downloaded
The user got a raw traceback, no summary, no idea what happened

Fix

An unexpected per-post error skips the post: it lands in failed_downloads.log and in the run summary, the run continues
FailureStreakBreaker: 5 failed posts in a row stop the run - scattered failures are post problems, a streak means disk, permissions or network
The stop message shows how many posts are done and that a re-run resumes from cache in seconds
--skip-all-failures turns the streak stop off for a full pass no matter what
Cancellation re-raises before the containment branch, so Ctrl+C keeps working

Before / After

Before: one broken post killed the run with a raw traceback; a dead disk meant hours of retries over every remaining post.
After: broken posts are skipped and listed in the summary; a systemic cause stops the run within 5 posts with a diagnosis and a resume hint.

Tests

test_unexpected_error_skips_the_post_and_the_run_continues - the #88 regression: the next post still downloads
test_cancellation_reraises_and_stops_immediately - Ctrl+C cannot be swallowed by the containment branch
test_failure_streak_stops_the_run_with_a_systemic_diagnosis - stops exactly at the 5th failure, names the cause and the resume path
test_success_resets_the_streak - scattered failures never stop the run
test_skip_all_failures_never_stops - the escape hatch attempts every post
Live: full task ci green, including 6 integration tests against the real Boosty API

Notes

The long-title post itself still fails and lands in failed_downloads.log: title length trimming is a separate fix on the roadmap
README got a "When downloads fail" section, so users learn this behavior before they hit it

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Glitchy-Sheep/boosty-downloader/pull/128 **Author:** [@Glitchy-Sheep](https://github.com/Glitchy-Sheep) **Created:** 8/9/2026 **Status:** ✅ Merged **Merged:** 8/9/2026 **Merged by:** [@Glitchy-Sheep](https://github.com/Glitchy-Sheep) **Base:** `main` ← **Head:** `fix/per-post-containment` --- ### 📝 Commits (4) - [`f0a675c`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/f0a675c598e9fe08375938c1d065d2f00b745100) fix: :bug: One broken post no longer kills or hollows out the download run - [`0dabff7`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/0dabff71b466ea52660958c1512eb0bbcc2d25fd) feat: :loudspeaker: Run summary lists posts that failed to download - [`b49c692`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/b49c692fc99f58115ffa1c6e61f2f68d8e04b692) feat: :sparkles: --skip-all-failures flag disables the failure-streak stop - [`58b321e`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/58b321ef45ec0f4b81c994df18a2d51b6b330d63) test: :white_check_mark: Containment, failure streak and Ctrl+C are guarded by tests ### 📊 Changes **10 files changed** (+497 additions, -26 deletions) <details> <summary>View changed files</summary> 📝 `CHANGELOG.md` (+8 -0) 📝 `README.md` (+13 -0) 📝 `boosty_downloader/main.py` (+6 -0) 📝 `boosty_downloader/src/application/exceptions/application_errors.py` (+13 -0) ➕ `boosty_downloader/src/application/failure_streak.py` (+32 -0) 📝 `boosty_downloader/src/application/use_cases/download_all_posts.py` (+147 -23) 📝 `boosty_downloader/src/cli/cli_options.py` (+11 -0) 📝 `boosty_downloader/src/cli/commands/download.py` (+5 -0) 📝 `boosty_downloader/src/infrastructure/boosty_api/utils/validation_errors.py` (+14 -3) ➕ `test/unit/use_cases/download_all_posts_test.py` (+248 -0) </details> ### 📄 Description ## Summary One post with an unexpected error used to kill the whole download run (#88). Now the run skips it, remembers it, and only stops early when failures come in a row - a sign the problem is the environment, not the posts. Fixes #88 ## Problem - The per-post loop caught only two known error types; anything else flew up and killed the run - Real case from #88: a very long post title -> `OSError: File name too long` on folder creation -> the run is dead, everything after that post is not downloaded - The user got a raw traceback, no summary, no idea what happened ## Fix - An unexpected per-post error skips the post: it lands in `failed_downloads.log` and in the run summary, the run continues - `FailureStreakBreaker`: 5 failed posts in a row stop the run - scattered failures are post problems, a streak means disk, permissions or network - The stop message shows how many posts are done and that a re-run resumes from cache in seconds - `--skip-all-failures` turns the streak stop off for a full pass no matter what - Cancellation re-raises before the containment branch, so Ctrl+C keeps working ## Before / After **Before:** one broken post killed the run with a raw traceback; a dead disk meant hours of retries over every remaining post. **After:** broken posts are skipped and listed in the summary; a systemic cause stops the run within 5 posts with a diagnosis and a resume hint. ## Tests - `test_unexpected_error_skips_the_post_and_the_run_continues` - the #88 regression: the next post still downloads - `test_cancellation_reraises_and_stops_immediately` - Ctrl+C cannot be swallowed by the containment branch - `test_failure_streak_stops_the_run_with_a_systemic_diagnosis` - stops exactly at the 5th failure, names the cause and the resume path - `test_success_resets_the_streak` - scattered failures never stop the run - `test_skip_all_failures_never_stops` - the escape hatch attempts every post - Live: full `task ci` green, including 6 integration tests against the real Boosty API ## Notes - The long-title post itself still fails and lands in `failed_downloads.log`: title length trimming is a separate fix on the roadmap - README got a "When downloads fail" section, so users learn this behavior before they hit it --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-13 19:47:46 +00:00