[PR #187] [MERGED] [downloader] Move media downloads into PostMediaDownloader #185

New issue

Closed

opened 2026-10-13 19:48:31 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-13 19:48:31 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Glitchy-Sheep/boosty-downloader/pull/187
Author: @Glitchy-Sheep
Created: 9/13/2026
Status: ✅ Merged
Merged: 9/13/2026
Merged by: @Glitchy-Sheep

Base: main ← Head: refactor/post-media-downloader

📝 Commits (2)

4111774 [downloader] Add PostMediaDownloader with one progress and error contract
9d2d9b4 [downloader] Drive media downloads through PostMediaDownloader

📊 Changes

10 files changed (+904 additions, -593 deletions)

View changed files

📝 boosty_downloader/application/use_cases/download_single_post.py (+128 -272)
➕ boosty_downloader/infrastructure/post_media_downloader.py (+258 -0)
📝 test/unit/file_downloader/extension_guessing_test.py (+5 -76)
📝 test/unit/file_downloader/filename_truncation_test.py (+4 -4)
➕ test/unit/post_media_downloader/__init__.py (+0 -0)
➕ test/unit/post_media_downloader/post_media_downloader_test.py (+331 -0)
➕ test/unit/use_cases/chunk_error_translation_test.py (+172 -0)
➖ test/unit/use_cases/ext_video_errors_test.py (+0 -97)
➖ test/unit/use_cases/ext_video_threading_test.py (+0 -138)
📝 test/unit/use_cases/video_filename_test.py (+6 -6)

📄 Description

📝 Summary

Audit stage 4, part 1 of 3: the I/O half of DownloadSinglePostUseCase moves to infrastructure. Pure refactoring: the on-disk tree, post.html and the cache rows do not change.

🔧 What changes

infrastructure/post_media_downloader.py saves the media of one post: images, files, audio, Boosty videos and external videos (yt-dlp off the event loop). It owns the subfolder layout and the filename policy per kind, and speaks one language outward: progress as MediaProgress (same shape for aiohttp and yt-dlp), a failure as MediaDownloadError with the transport error as its cause, cancellation as CancelledError with the partial file removed.
DownloadSinglePostUseCase shrinks from 568 to 424 lines and keeps orchestration only: map, cache lookup, parallel chunks, post.html, cache write, statistics. Chunk dispatch checks CHUNK_TO_FILTER once and matches on the chunk kind; the noqa: C901, PLR0911 markers are gone.
Error translation is two branches instead of seven: cancellation stays cancellation, MediaDownloadError becomes a logged ApplicationFailedDownloadError. The cause chain is kept, so the expired-link refresh and the path-too-long hint still see the transport error underneath.
Small asymmetries resolved on the way: the external-video folder is created off the loop like the others, and every saved path is relative to the same base.

Wording

Failure messages are unified: Couldn't download resource: <reason> with the file name or the URL as the resource. Reasons are the same texts as before.

🧪 Tests

New test/unit/post_media_downloader: layout and relative paths, extension policy (#75), progress shape, transport failure and cancellation contracts, yt-dlp off the loop and cancellable, every ExtVideoError mapped.
New chunk_error_translation_test: the use case's two branches, including the 400 visible through the cause chain.
ext_video_threading_test and ext_video_errors_test retired: the media downloader tests pin the same behaviour.
task ci green locally: ruff, pyright strict, import-linter, 255 tests + e2e (the e2e tree assertions are the oracle for this refactoring).
Live run on a small blog into a temporary directory: --dry-run -f post_content -f audio promised 11 images and 2 audio files, the real run saved exactly that, a rerun served everything from the cache.

⏭️ Next

PR 2: when one chunk fails, finish the others and cache the content types that completed (audit 19.1). PR 3: --post-url through the post retrier, URL parsing up front, 429 retries.

✅ Checklist

Tested live on a real blog (feature or fix)
Updated CLI --help and README if needed
Updated docs/ if the download flow or dev process changed
Added a line to ## Unreleased in CHANGELOG.md (or the ci/skip-changelog label if not user-facing)

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Glitchy-Sheep/boosty-downloader/pull/187 **Author:** [@Glitchy-Sheep](https://github.com/Glitchy-Sheep) **Created:** 9/13/2026 **Status:** ✅ Merged **Merged:** 9/13/2026 **Merged by:** [@Glitchy-Sheep](https://github.com/Glitchy-Sheep) **Base:** `main` ← **Head:** `refactor/post-media-downloader` --- ### 📝 Commits (2) - [`4111774`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/41117744434e5305392c83b2728c7cff17c52678) [downloader] Add PostMediaDownloader with one progress and error contract - [`9d2d9b4`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/9d2d9b41505f727ea4129c8fb85ca702c505c716) [downloader] Drive media downloads through PostMediaDownloader ### 📊 Changes **10 files changed** (+904 additions, -593 deletions) <details> <summary>View changed files</summary> 📝 `boosty_downloader/application/use_cases/download_single_post.py` (+128 -272) ➕ `boosty_downloader/infrastructure/post_media_downloader.py` (+258 -0) 📝 `test/unit/file_downloader/extension_guessing_test.py` (+5 -76) 📝 `test/unit/file_downloader/filename_truncation_test.py` (+4 -4) ➕ `test/unit/post_media_downloader/__init__.py` (+0 -0) ➕ `test/unit/post_media_downloader/post_media_downloader_test.py` (+331 -0) ➕ `test/unit/use_cases/chunk_error_translation_test.py` (+172 -0) ➖ `test/unit/use_cases/ext_video_errors_test.py` (+0 -97) ➖ `test/unit/use_cases/ext_video_threading_test.py` (+0 -138) 📝 `test/unit/use_cases/video_filename_test.py` (+6 -6) </details> ### 📄 Description ## 📝 Summary Audit stage 4, part 1 of 3: the I/O half of `DownloadSinglePostUseCase` moves to infrastructure. Pure refactoring: the on-disk tree, `post.html` and the cache rows do not change. ## 🔧 What changes - `infrastructure/post_media_downloader.py` saves the media of one post: images, files, audio, Boosty videos and external videos (yt-dlp off the event loop). It owns the subfolder layout and the filename policy per kind, and speaks one language outward: progress as `MediaProgress` (same shape for aiohttp and yt-dlp), a failure as `MediaDownloadError` with the transport error as its cause, cancellation as `CancelledError` with the partial file removed. - `DownloadSinglePostUseCase` shrinks from 568 to 424 lines and keeps orchestration only: map, cache lookup, parallel chunks, `post.html`, cache write, statistics. Chunk dispatch checks `CHUNK_TO_FILTER` once and matches on the chunk kind; the `noqa: C901, PLR0911` markers are gone. - Error translation is two branches instead of seven: cancellation stays cancellation, `MediaDownloadError` becomes a logged `ApplicationFailedDownloadError`. The cause chain is kept, so the expired-link refresh and the path-too-long hint still see the transport error underneath. - Small asymmetries resolved on the way: the external-video folder is created off the loop like the others, and every saved path is relative to the same base. ## Wording Failure messages are unified: `Couldn't download resource: <reason>` with the file name or the URL as the resource. Reasons are the same texts as before. ## 🧪 Tests - New `test/unit/post_media_downloader`: layout and relative paths, extension policy (#75), progress shape, transport failure and cancellation contracts, yt-dlp off the loop and cancellable, every `ExtVideoError` mapped. - New `chunk_error_translation_test`: the use case's two branches, including the 400 visible through the cause chain. - `ext_video_threading_test` and `ext_video_errors_test` retired: the media downloader tests pin the same behaviour. - `task ci` green locally: ruff, pyright strict, import-linter, 255 tests + e2e (the e2e tree assertions are the oracle for this refactoring). - Live run on a small blog into a temporary directory: `--dry-run -f post_content -f audio` promised 11 images and 2 audio files, the real run saved exactly that, a rerun served everything from the cache. ## ⏭️ Next PR 2: when one chunk fails, finish the others and cache the content types that completed (audit 19.1). PR 3: `--post-url` through the post retrier, URL parsing up front, 429 retries. ## ✅ Checklist - [x] Tested live on a real blog (feature or fix) - [x] Updated CLI `--help` and README if needed - [x] Updated `docs/` if the download flow or dev process changed - [x] Added a line to `## Unreleased` in CHANGELOG.md (or the `ci/skip-changelog` label if not user-facing) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-13 19:48:31 +00:00