Glitchy-Sheep/boosty-downloader

Fork 0

mirror of https://github.com/Glitchy-Sheep/boosty-downloader.git synced 2026-10-14 21:33:23 +00:00

[PR #33] [CLOSED] Major improvements: OAuth auth, cache refactoring, enhanced HTML reporter, and comprehensive testing (v1.0.1 to v1.5.0) #64

New issue

Closed

opened 2026-10-13 19:46:46 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-13 19:46:46 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Glitchy-Sheep/boosty-downloader/pull/33
Author: @petrouv
Created: 7/24/2025
Status: ❌ Closed

Base: main ← Head: feature/fix-tests-and-setup

📝 Commits (10+)

1998657 fix: Fix tests and improve project setup
1bd1718 chore: Update .gitignore to include macOS system files
82e255d v1.0.1: Fix tests and improve project setup
d29216d v1.0.2: Improve cache architecture
1aff7ce feat: major cache system refactoring v1.1.0
4b9233e feat: Add OAuth authentication with automatic token refresh
78b3cfd 🐛 Fix target_directory config parameter being ignored
a6ea0bf fix: OAuth token parsing issues from browser console copy-paste
52aa63c feat: Enhanced HTML Reporter with modern semantic structure and content parsing
4742fc8 feat: comprehensive code quality improvements (v1.3.1)

📊 Changes

35 files changed (+4405 additions, -230 deletions)

View changed files

📝 .gitignore (+11 -0)
📝 CHANGELOG.md (+87 -0)
📝 README.md (+26 -3)
➕ boosty_downloader/__main__.py (+6 -0)
📝 boosty_downloader/main.py (+42 -6)
➕ boosty_downloader/oauth_setup.py (+403 -0)
➕ boosty_downloader/src/boosty_api/core/auth_validator.py (+153 -0)
📝 boosty_downloader/src/boosty_api/core/client.py (+110 -13)
➕ boosty_downloader/src/boosty_api/core/oauth_client.py (+133 -0)
📝 boosty_downloader/src/boosty_api/models/__init__.py (+1 -0)
➕ boosty_downloader/src/boosty_api/models/oauth.py (+33 -0)
📝 boosty_downloader/src/boosty_api/models/post/base_post_data.py (+7 -9)
📝 boosty_downloader/src/boosty_api/models/post/posts_request.py (+3 -0)
📝 boosty_downloader/src/boosty_api/utils/auth_parsers.py (+16 -1)
➕ boosty_downloader/src/boosty_api/utils/oauth_manager.py (+159 -0)
📝 boosty_downloader/src/caching/post_cache.py (+100 -26)
📝 boosty_downloader/src/download_manager/download_manager.py (+681 -69)
📝 boosty_downloader/src/download_manager/download_manager_config.py (+3 -0)
📝 boosty_downloader/src/download_manager/utils/base_file_downloader.py (+2 -1)
📝 boosty_downloader/src/download_manager/utils/human_readable_size.py (+1 -1)

...and 15 more files

📄 Description

Summary

This PR represents a comprehensive evolution of the boosty-downloader project from v1.0.1 to v1.5.0, introducing major improvements across authentication, caching, reporting, testing, and code quality.

Version Evolution

v1.5.0 - Raw API Data Export

Raw API Data Export: Added option to save complete Boosty API responses
Text Summary Export: Added option for minimalistic text representations
Enhanced Post Information: Text summaries include post type, teaser, content, and metadata
Improved Code Quality: Fixed all type annotation issues and translated Russian comments to English

v1.4.0 - OAuth Retry Mechanism

OAuth Retry Mechanism: Added retry mechanism for handling inaccessible posts
Enhanced Reliability: Improved error handling and recovery for OAuth authentication

v1.3.1 - Code Quality Improvements

Comprehensive Refactoring: Reduced ruff errors by 89% (from 224 to 24)
UTC-Aware DateTime: Fixed all datetime operations to use timezone-aware UTC timestamps
Magic Numbers Elimination: Replaced all magic numbers with named constants
Modern Test Style: Migrated from unittest-style to pytest assertions
Code Formatting: Applied comprehensive formatting fixes and import optimization

v1.3.0 - Enhanced HTML Reporter

Modern Semantic HTML5: Complete rewrite with proper semantic structure
Post Metadata Support: Comprehensive post metadata in HTML headers
Content Type Attributes: Added data-content-type attributes for improved parsing
Files and Videos in HTML: Display with local links when available
Clickable Images: Images can be opened in new tabs
Structured Data: Added JSON-LD structured data for better machine parsing
Modern CSS Design: Responsive design with dark/light theme support
Enhanced Accessibility: Proper alt text, loading attributes, and semantic markup

v1.2.2 - OAuth Token Parsing Fix

Critical Bug Fix: Fixed OAuth token parsing from browser console
New Token Format Support: Added pipe-separated string format support
Enhanced Error Handling: Improved error messages and debug logging
Token Validation: Comprehensive validation for both legacy and new formats

v1.2.1 - Target Directory Fix

Critical Bug Fix: Fixed configuration parameter being ignored
Configuration Support: Proper support for both relative and absolute paths
Directory Creation: Automatic target directory creation

v1.2.0 - OAuth Authentication

OAuth Authentication: Added OAuth token-based authentication with automatic refresh
Enhanced Security: Automatic token refresh when expired
Browser Token Extraction: JavaScript utility for extracting OAuth tokens
Dual Authentication: Support for both OAuth and legacy cookie/header auth
OAuth Management: New commands for OAuth setup and token status checking

v1.1.0 - Cache System Refactoring

Major Cache Refactoring: Complete overhaul of post caching architecture
Fixed Cache Validation: Proper use of unique post IDs as primary keys
Better Title Handling: Automatic folder renaming when post titles change
Enhanced Error Handling: Comprehensive error handling for download failures
Separated Responsibilities: Split cache checking and folder renaming into distinct methods

v1.0.2 - Cache Architecture Improvement

BREAKING CHANGE: Improved cache architecture to separate clean post titles from folder names
Data Deduplication: Cache now stores clean post titles without dates

v1.0.1 - Project Setup and Testing

Import Path Fixes: Fixed import paths and code style issues
CLI Enhancement: Added for running package via
Code Style: Fixed ruff issues and import organization
macOS Support: Updated .gitignore for macOS system files

Key Technical Improvements

Authentication System

Complete OAuth implementation with automatic token refresh
Dual authentication support (OAuth + legacy cookie/header)
Robust token parsing and validation
Retry mechanisms for failed authentication

Caching Architecture

Major refactoring for better reliability and performance
Proper cache validation using unique post IDs
Automatic folder renaming for title changes
Comprehensive error handling

HTML Reporting

Modern semantic HTML5 structure
Responsive design with theme support
Enhanced accessibility features
Structured data for better machine parsing

Testing Infrastructure

15+ new test files covering all major functionality
Modern pytest-based testing approach
Comprehensive coverage of edge cases
All tests passing consistently

Code Quality

89% reduction in code quality issues
Modern Python practices and type hints
Comprehensive documentation
Consistent code formatting

Files Changed

35+ files modified/added including core API, models, utilities, and tests
15 new test files for comprehensive coverage
New OAuth authentication system with multiple components
Enhanced caching mechanism with improved architecture
Modern HTML reporting with semantic structure
Complete CLI enhancement with better user experience

Breaking Changes

v1.0.2: Cache architecture change (separated clean titles from folder names)
All other changes are backward compatible improvements

Testing

All changes include comprehensive test coverage with 26+ tests passing consistently.

Dependencies

Updated dependencies to latest stable versions with proper version constraints.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Glitchy-Sheep/boosty-downloader/pull/33 **Author:** [@petrouv](https://github.com/petrouv) **Created:** 7/24/2025 **Status:** ❌ Closed **Base:** `main` ← **Head:** `feature/fix-tests-and-setup` --- ### 📝 Commits (10+) - [`1998657`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/19986576124aba922afcdca6f194f87888d2ec5b) fix: Fix tests and improve project setup - [`1bd1718`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/1bd17182c69a9141eb4ff8a825058b9a375c94df) chore: Update .gitignore to include macOS system files - [`82e255d`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/82e255d2dfd88267b2fb8d3d7e2bbb0c65bb924f) v1.0.1: Fix tests and improve project setup - [`d29216d`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/d29216d2e72010fa06feae2c986979ebd209bc8d) v1.0.2: Improve cache architecture - [`1aff7ce`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/1aff7ce4553935a2ad4e88f993b3474e488550d1) feat: major cache system refactoring v1.1.0 - [`4b9233e`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/4b9233e3be46a62f0bb4c72853a5582b14c155c0) feat: Add OAuth authentication with automatic token refresh - [`78b3cfd`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/78b3cfd05ab3306ee8de5bf242a0936b2f22c5b7) 🐛 Fix target_directory config parameter being ignored - [`a6ea0bf`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/a6ea0bfea8db8db9f33d4ad788f4bc99b42e6ca1) fix: OAuth token parsing issues from browser console copy-paste - [`52aa63c`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/52aa63c29f2da24c762f77f8f7278e2e85ed6d02) feat: Enhanced HTML Reporter with modern semantic structure and content parsing - [`4742fc8`](https://github.com/Glitchy-Sheep/boosty-downloader/commit/4742fc87b823e0836ccbb09937c492e0bc5b882a) feat: comprehensive code quality improvements (v1.3.1) ### 📊 Changes **35 files changed** (+4405 additions, -230 deletions) <details> <summary>View changed files</summary> 📝 `.gitignore` (+11 -0) 📝 `CHANGELOG.md` (+87 -0) 📝 `README.md` (+26 -3) ➕ `boosty_downloader/__main__.py` (+6 -0) 📝 `boosty_downloader/main.py` (+42 -6) ➕ `boosty_downloader/oauth_setup.py` (+403 -0) ➕ `boosty_downloader/src/boosty_api/core/auth_validator.py` (+153 -0) 📝 `boosty_downloader/src/boosty_api/core/client.py` (+110 -13) ➕ `boosty_downloader/src/boosty_api/core/oauth_client.py` (+133 -0) 📝 `boosty_downloader/src/boosty_api/models/__init__.py` (+1 -0) ➕ `boosty_downloader/src/boosty_api/models/oauth.py` (+33 -0) 📝 `boosty_downloader/src/boosty_api/models/post/base_post_data.py` (+7 -9) 📝 `boosty_downloader/src/boosty_api/models/post/posts_request.py` (+3 -0) 📝 `boosty_downloader/src/boosty_api/utils/auth_parsers.py` (+16 -1) ➕ `boosty_downloader/src/boosty_api/utils/oauth_manager.py` (+159 -0) 📝 `boosty_downloader/src/caching/post_cache.py` (+100 -26) 📝 `boosty_downloader/src/download_manager/download_manager.py` (+681 -69) 📝 `boosty_downloader/src/download_manager/download_manager_config.py` (+3 -0) 📝 `boosty_downloader/src/download_manager/utils/base_file_downloader.py` (+2 -1) 📝 `boosty_downloader/src/download_manager/utils/human_readable_size.py` (+1 -1) _...and 15 more files_ </details> ### 📄 Description ## Summary This PR represents a comprehensive evolution of the boosty-downloader project from v1.0.1 to v1.5.0, introducing major improvements across authentication, caching, reporting, testing, and code quality. ## Version Evolution ### v1.5.0 - Raw API Data Export - **Raw API Data Export**: Added option to save complete Boosty API responses - **Text Summary Export**: Added option for minimalistic text representations - **Enhanced Post Information**: Text summaries include post type, teaser, content, and metadata - **Improved Code Quality**: Fixed all type annotation issues and translated Russian comments to English ### v1.4.0 - OAuth Retry Mechanism - **OAuth Retry Mechanism**: Added retry mechanism for handling inaccessible posts - **Enhanced Reliability**: Improved error handling and recovery for OAuth authentication ### v1.3.1 - Code Quality Improvements - **Comprehensive Refactoring**: Reduced ruff errors by 89% (from 224 to 24) - **UTC-Aware DateTime**: Fixed all datetime operations to use timezone-aware UTC timestamps - **Magic Numbers Elimination**: Replaced all magic numbers with named constants - **Modern Test Style**: Migrated from unittest-style to pytest assertions - **Code Formatting**: Applied comprehensive formatting fixes and import optimization ### v1.3.0 - Enhanced HTML Reporter - **Modern Semantic HTML5**: Complete rewrite with proper semantic structure - **Post Metadata Support**: Comprehensive post metadata in HTML headers - **Content Type Attributes**: Added data-content-type attributes for improved parsing - **Files and Videos in HTML**: Display with local links when available - **Clickable Images**: Images can be opened in new tabs - **Structured Data**: Added JSON-LD structured data for better machine parsing - **Modern CSS Design**: Responsive design with dark/light theme support - **Enhanced Accessibility**: Proper alt text, loading attributes, and semantic markup ### v1.2.2 - OAuth Token Parsing Fix - **Critical Bug Fix**: Fixed OAuth token parsing from browser console - **New Token Format Support**: Added pipe-separated string format support - **Enhanced Error Handling**: Improved error messages and debug logging - **Token Validation**: Comprehensive validation for both legacy and new formats ### v1.2.1 - Target Directory Fix - **Critical Bug Fix**: Fixed configuration parameter being ignored - **Configuration Support**: Proper support for both relative and absolute paths - **Directory Creation**: Automatic target directory creation ### v1.2.0 - OAuth Authentication - **OAuth Authentication**: Added OAuth token-based authentication with automatic refresh - **Enhanced Security**: Automatic token refresh when expired - **Browser Token Extraction**: JavaScript utility for extracting OAuth tokens - **Dual Authentication**: Support for both OAuth and legacy cookie/header auth - **OAuth Management**: New commands for OAuth setup and token status checking ### v1.1.0 - Cache System Refactoring - **Major Cache Refactoring**: Complete overhaul of post caching architecture - **Fixed Cache Validation**: Proper use of unique post IDs as primary keys - **Better Title Handling**: Automatic folder renaming when post titles change - **Enhanced Error Handling**: Comprehensive error handling for download failures - **Separated Responsibilities**: Split cache checking and folder renaming into distinct methods ### v1.0.2 - Cache Architecture Improvement - **BREAKING CHANGE**: Improved cache architecture to separate clean post titles from folder names - **Data Deduplication**: Cache now stores clean post titles without dates ### v1.0.1 - Project Setup and Testing - **Import Path Fixes**: Fixed import paths and code style issues - **CLI Enhancement**: Added for running package via - **Code Style**: Fixed ruff issues and import organization - **macOS Support**: Updated .gitignore for macOS system files ## Key Technical Improvements ### Authentication System - Complete OAuth implementation with automatic token refresh - Dual authentication support (OAuth + legacy cookie/header) - Robust token parsing and validation - Retry mechanisms for failed authentication ### Caching Architecture - Major refactoring for better reliability and performance - Proper cache validation using unique post IDs - Automatic folder renaming for title changes - Comprehensive error handling ### HTML Reporting - Modern semantic HTML5 structure - Responsive design with theme support - Enhanced accessibility features - Structured data for better machine parsing ### Testing Infrastructure - 15+ new test files covering all major functionality - Modern pytest-based testing approach - Comprehensive coverage of edge cases - All tests passing consistently ### Code Quality - 89% reduction in code quality issues - Modern Python practices and type hints - Comprehensive documentation - Consistent code formatting ## Files Changed - **35+ files modified/added** including core API, models, utilities, and tests - **15 new test files** for comprehensive coverage - **New OAuth authentication system** with multiple components - **Enhanced caching mechanism** with improved architecture - **Modern HTML reporting** with semantic structure - **Complete CLI enhancement** with better user experience ## Breaking Changes - v1.0.2: Cache architecture change (separated clean titles from folder names) - All other changes are backward compatible improvements ## Testing All changes include comprehensive test coverage with 26+ tests passing consistently. ## Dependencies Updated dependencies to latest stable versions with proper version constraints. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-13 19:46:46 +00:00