Commit Graph

  • cdf2ead7ed security: patch proxy SSRF in Docker server (0.8.9) main v0.8.9 docker-rebuild-v0.8.9 release/v0.8.9 develop unclecode 2026-06-04 06:17:41 +00:00
  • 930dc0a61f security(credits): credit Geo (geo-chen) for 0.8.8 base_url exfil; update secsys_codex handle (FORIMOC) v0.8.8 docker-rebuild-v0.8.8 release/v0.8.8 unclecode 2026-06-04 01:00:06 +00:00
  • 5d3d0fe2d7 docs: 0.8.8 release notes, changelog, README; pre-announce next secure-by-default release unclecode 2026-06-04 00:59:17 +00:00
  • aa81e8fe7d security: non-breaking hardening patch (0.8.8) unclecode 2026-06-02 12:39:04 +00:00
  • 72fd78e59e chore: gitignore out/ local pipeline output unclecode 2026-06-02 05:14:11 +00:00
  • 7259d734a1 security(credits): credit IcySun & Yashon (co-reporter) per their request unclecode 2026-06-01 14:44:43 +00:00
  • 7b280d36b1 chore: gitignore .security/ - advisory payloads stay private, published via GHSA only v0.8.7 docker-rebuild-v0.8.7 release/v0.8.7 unclecode 2026-06-01 14:09:23 +00:00
  • 4bf6071226 chore: bump version to 0.8.7 unclecode 2026-06-01 14:04:51 +00:00
  • 30187e6dc7 docs: 0.8.7 release notes, changelog, README highlights; finalize security credits (q1uf3ng confirmed, Velayutham Selvaraj) unclecode 2026-06-01 14:04:11 +00:00
  • d705d7c4f1 security(credits): acknowledge independent reporters Velayutham S and IcySun unclecode 2026-06-01 13:25:47 +00:00
  • 71e1667bd1 security(advisory): fold direct /crawl,/md,/llm SSRF + IPv6-mapped bypass into advisory-2; credit secsys_codex unclecode 2026-06-01 13:13:57 +00:00
  • bd20700626 merge: domain mapper + 4 core fixes cherry-picked from cloud (OSS-safe, audited) for 0.8.7 unclecode 2026-06-01 13:05:41 +00:00
  • c06a3dcf56 fix(logger): default Console width=200 for non-TTY contexts unclecode 2026-04-17 04:17:06 +00:00
  • 62c619d454 fix(prefetch): honor <head><base href> in quick_extract_links (#752) unclecode 2026-06-01 06:15:45 +00:00
  • 7059170614 fix(async_webcrawler): skip block validator when downloaded_files is set unclecode 2026-05-06 02:05:34 +00:00
  • a615d697af fix(async_webcrawler): success=True for binary downloads unclecode 2026-05-06 02:14:48 +00:00
  • 858c827145 feat: add per-source timeout to DomainMapper unclecode 2026-03-23 04:56:31 +00:00
  • ed60c6628a feat: add include_subdomains flag to DomainMapperConfig unclecode 2026-03-21 13:31:26 +00:00
  • 9d5bcf78e2 feat: Add DomainMapper for comprehensive domain URL discovery unclecode 2026-03-21 07:24:08 +00:00
  • fcaf08b3b3 merge: slot April 2026 security batch (Docker API vulns, SSRF, JWT, file-write, XSS, execute_js) into develop for 0.8.7 unclecode 2026-06-01 12:40:37 +00:00
  • 820cbb59f9 Merge pull request #1960 from NaabZer/bugfix/stealth-import-mismatch Nasrin 2026-05-25 12:50:56 +02:00
  • 1ef552f351 Merge pull request #1967 from unclecode/fix/mcp-ensure-ascii-cjk-encoding Nasrin 2026-05-25 12:26:37 +02:00
  • be71585239 Merge pull request #1969 from cgseyhan/fix/async-logger-stderr-mcp-1968 Nasrin 2026-05-25 12:20:01 +02:00
  • 532b105fc7 Merge pull request #1975 from nightcityblade/fix/issue-1973 Nasrin 2026-05-25 11:59:25 +02:00
  • 158231be60 feat: add follow_frames support for legacy HTML4 frameset redirects feat/follow-frameset Soham Kukreti 2026-05-22 17:10:42 +05:30
  • 791aae3a21 fix: allow assistant toolbar to scroll nightcityblade 2026-05-20 23:07:47 +08:00
  • 944eb1e456 fix(logger): route AsyncLogger output to stderr by default cemgo 2026-05-14 14:13:30 +03:00
  • 76f56af2dd fix: use ensure_ascii=False in MCP bridge json.dumps to preserve CJK characters fix/mcp-ensure-ascii-cjk-encoding Soham Kukreti 2026-05-13 14:10:19 +05:30
  • d709d670f9 fix: expose crawler_config on all MCP scrape tools (#1963) fix/mcp-crawler-config-passthrough Soham Kukreti 2026-05-12 21:13:17 +05:30
  • 5568a9ad38 Change browser_adapter to use Stealth import instead of stealth_* NaabZer 2026-05-07 18:10:40 +09:00
  • dfb525edec Merge pull request #1951 from unclecode/fix/bedrock-provider-prefix Nasrin 2026-05-06 10:37:05 +02:00
  • 37f6c4aace fix: arun() returns CrawlResult directly instead of CrawlResultContainer (#1898) fix/arun-return-type-1898 ntohidi 2026-05-05 08:41:38 +02:00
  • 47a4c256c9 Merge pull request #1952 from hafezparast/fix/maysam-silent-scrape-failure-1949 Nasrin 2026-05-05 07:52:44 +02:00
  • a45c678ee4 Merge pull request #1939 from unclecode/fix/preserve-tail-text-1938 Nasrin 2026-05-05 07:46:47 +02:00
  • 5e5519b1c6 fix: log failure reason before COMPLETE and fix misleading SCRAPE ✓ (#1949) hafezparast 2026-05-01 16:18:03 +08:00
  • 660f49c879 fix: add bedrock to PROVIDER_MODELS_PREFIXES so AWS credential auth works fix/bedrock-provider-prefix Soham Kukreti 2026-04-30 22:13:54 +05:30
  • 388ce3f033 Merge pull request #1940 from hafezparast/fix/mermaid-sequence-fence-1043 Nasrin 2026-04-30 13:34:00 +02:00
  • dba38c7886 fix: broaden mermaid SVG text extraction and prevent nested fences (#1043) hafezparast 2026-04-25 01:36:52 +08:00
  • 4d139247b9 Merge pull request #1934 from hafezparast/fix/maysam-dispatcher-semaphore-count-1927 Nasrin 2026-04-24 18:53:56 +02:00
  • e8f1af7c16 fix: preserve .tail text when removing empty elements (#1938) fix/preserve-tail-text-1938 ntohidi 2026-04-24 18:51:24 +02:00
  • 04985ea15e docs: update arun() docstring to match CrawlResultContainer return type ntohidi 2026-04-24 18:35:57 +02:00
  • 35ee366e28 Merge pull request #1901 from hafezparast/fix/maysam-arun-type-hint-1898 Nasrin 2026-04-24 18:33:41 +02:00
  • 244fbf7b58 Merge pull request #1929 from atomic-carpenter/listen-on-all-addressess Nasrin 2026-04-24 17:59:48 +02:00
  • 4e72f31011 fix: use semaphore_count default of 10 to match CrawlerRunConfig default hafezparast 2026-04-24 23:57:17 +08:00
  • d595679d25 Merge pull request #1925 from sevenmoonlightsteps/fix/docker-llm-table-extraction-allowlist Nasrin 2026-04-24 17:11:56 +02:00
  • 936e4470eb Merge pull request #1845 from hafezparast/fix/maysam-mermaid-svg-text-1043 Nasrin 2026-04-24 16:49:59 +02:00
  • 5e56e34840 Merge pull request #1922 from unclecode/fix/deep-crawl-streaming-contextvar-1917 Nasrin 2026-04-24 16:36:08 +02:00
  • d3c92ee3df fix: wire semaphore_count into auto-created MemoryAdaptiveDispatcher (#1927) hafezparast 2026-04-24 10:59:52 +08:00
  • e06b19ca09 docker: listen on all addresses Atomic Carpenter 2026-04-22 00:47:08 +02:00
  • 1e25edcb5c fix(security): block IPv6-mapped IPv4 SSRF bypass unclecode 2026-04-20 10:10:59 +00:00
  • f77c0a856f fix(security): SSRF protection on all crawl/md/llm URL entry points unclecode 2026-04-20 09:42:43 +00:00
  • 0e92b5e239 docs: add Privacy Policy, Terms of Service, and Support pages unclecode 2026-04-20 02:24:21 +00:00
  • c8c2dc319f fix: add LLMTableExtraction to Docker API deserialization allowlist Gab 2026-04-17 15:43:56 -04:00
  • 4e86399bfa Merge pull request #1913 from unclecode/fix/nlp-sentence-chunking-1909 Nasrin 2026-04-16 10:24:59 +02:00
  • 3d4bda122a fix(deep-crawl): use set(False) instead of reset(token) for ContextVar (#1917) fix/deep-crawl-streaming-contextvar-1917 ntohidi 2026-04-16 13:49:32 +08:00
  • 7bfc547bce fix: preserve rowspan/colspan in cleaned_html (#1920) ntohidi 2026-04-16 12:42:36 +08:00
  • c9914691db chore: add GitHub Security Advisory payloads for release day unclecode 2026-04-15 06:08:42 +00:00
  • c45ccf20f6 fix: credit wulonchia by requested handle unclecode 2026-04-15 05:45:15 +00:00
  • 0f20f8bb83 fix(security): batch 2 - JWT secret, eval removal, execute_js, hook sandbox unclecode 2026-04-15 05:42:14 +00:00
  • 7976b45817 fix(security): patch 4 vulns - file write, SSRF, monitor auth, XSS unclecode 2026-04-13 11:29:54 +00:00
  • c837c0d9cb fix(chunking): preserve sentence order in NlpSentenceChunking (#1909) fix/nlp-sentence-chunking-1909 ntohidi 2026-04-11 17:27:18 +08:00
  • c5612f7551 fix: correct arun() return type from RunManyReturn to CrawlResultContainer (#1898) hafezparast 2026-04-06 21:35:17 +08:00
  • 3d02d75edb Merge pull request #1852 from hafezparast/feat/maysam-arun-many-config-list-1837 Nasrin 2026-04-06 10:26:44 +02:00
  • ec560f13d2 fix: default LLMExtractionStrategy extraction_type to schema unclecode 2026-04-04 09:26:35 +00:00
  • e326da9166 fix(security): complete AST sandbox escape remediation (CVSS 9.8) unclecode 2026-03-31 13:01:57 +00:00
  • 2fc39cbe89 fix(security): remove eval() from computed fields, harden config deserializer unclecode 2026-03-31 12:02:28 +00:00
  • cf74312719 test: add 146 tests for _safe_eval_expression sandbox hardening fix/sandbox-escape-allowlist-attrs ntohidi 2026-03-30 15:30:35 +08:00
  • 05cc47c2d4 fix(security): harden _safe_eval_expression sandbox against frame-walking RCE ntohidi 2026-03-30 15:22:33 +08:00
  • 1debe5f5fc Merge pull request #1885 from unclecode/develop UncleCode 2026-03-30 09:59:58 +07:00
  • bcbccbea2f docs: update version references to 0.8.6 in README and Docker docs ntohidi 2026-03-30 10:57:13 +08:00
  • 7e7533ec7c Merge pull request #1882 from hafezparast/fix/crawler-config-dict-validation-1880 Nasrin 2026-03-30 04:50:32 +02:00
  • e9f832274e fix: validate markdown_generator type in CrawlerRunConfig to catch bad JSON format (#1880) hafezparast 2026-03-29 07:39:28 +08:00
  • af648e104f fix: bump Dockerfile version to 0.8.6 docker-rebuild-v0.8.6 unclecode 2026-03-24 15:19:18 +00:00
  • 4e4a996878 fix: replace litellm with unclecode-litellm due to PyPI supply chain compromise v0.8.6 unclecode 2026-03-24 14:49:36 +00:00
  • f4bda05178 release: bump version to 0.8.6 unclecode 2026-03-24 14:13:41 +00:00
  • 01c685cd3a fix: pin litellm to safe fork (v1.81.13) due to PyPI supply chain compromise unclecode 2026-03-24 14:03:26 +00:00
  • 1a40ccf093 Merge pull request #1844 from hafezparast/fix/maysam-browser-none-guard-1842 Nasrin 2026-03-24 11:37:46 +01:00
  • 6eb2530bd9 Merge pull request #1849 from hafezparast/fix/maysam-serialize-skip-non-config-1848 Nasrin 2026-03-24 11:36:03 +01:00
  • fb24ee592e Merge pull request #1851 from hafezparast/fix/maysam-mcp-sse-asgi-1850 Nasrin 2026-03-24 11:17:35 +01:00
  • 3846b738cf Merge branch 'develop' of https://github.com/unclecode/crawl4ai into main ntohidi 2026-03-24 18:10:40 +08:00
  • 1a597cb97f Merge pull request #1836 from unclecode/release/v0.8.5 UncleCode 2026-03-24 11:06:58 +01:00
  • 8995c1bbd6 feat: expose arun_many config-list support in Docker API (#1837) hafezparast 2026-03-23 09:56:53 +08:00
  • 219416e49d fix: MCP SSE endpoint crash on Starlette >=0.50 (#1850) hafezparast 2026-03-23 08:55:41 +08:00
  • e603e4a722 fix: skip non-allowlisted types in serialization/deserialization (#1848) hafezparast 2026-03-23 08:17:02 +08:00
  • 2fd0f4c6a7 fix: preserve mermaid diagram text from SVGs during scraping (#1043) hafezparast 2026-03-22 11:43:54 +08:00
  • 310b52b663 fix: improve browser None guard in create_browser_context (#1842) hafezparast 2026-03-22 10:45:38 +08:00
  • 37da8b8f97 fix: pin redis-tools version to match redis-server in Dockerfile docker-rebuild-v0.8.5 release/v0.8.5 ntohidi 2026-03-21 14:26:23 +08:00
  • 29d27ed1ae fix: install curl and gnupg in Dockerfile to support Redis repository addition ntohidi 2026-03-21 14:17:27 +08:00
  • c4389adddf fix: Prevent scan_full_page from hanging on dynamic/infinite-scroll pages unclecode 2026-03-18 15:36:12 +00:00
  • 3ecd852011 fix: Re-check is_blocked() when fallback fetch fails unclecode 2026-03-18 14:36:57 +00:00
  • 4bf17796d4 feat: add version 0.8.5 release highlights including anti-bot detection, shadow DOM support, and critical security fixes to README v0.8.5 ntohidi 2026-03-18 11:23:20 +08:00
  • 9b571bb947 feat: HTTP strategy detects and saves file downloads (CSV, PDF, etc.) unclecode 2026-03-16 14:03:43 +00:00
  • bb6406a2d0 release: Crawl4AI v0.8.5 ntohidi 2026-03-16 18:46:05 +08:00
  • dd1d4ddbe2 fix: RateLimiter burst race, Retry-After headers, deep crawl dispatcher (#1095) fix/rate-limiter-burst-and-headers-1095 ntohidi 2026-03-13 15:56:40 +08:00
  • f6ab207e25 fix: remove shared LOCK contention in monitor to prevent pod deadlock (#1754) fix/timeline-deadlock-shared-lock-1754 ntohidi 2026-03-13 12:17:52 +08:00
  • 648f36b622 Merge pull request #1827 from hafezparast/fix/maysam-llm-provider-redis-config-1611-1817 Nasrin 2026-03-13 03:59:28 +01:00
  • 6e4299577f Merge pull request #1833 from hafezparast/fix/maysam-css-selector-raw-1484 Nasrin 2026-03-13 03:38:15 +01:00
  • 8de83a3590 fix: css_selector ignored in LXML scraping for raw:// URLs (#1484) hafezparast 2026-03-12 20:00:33 +08:00
  • bf1158a61b fix: upgrade Redis to 7.2.7 for CVE-2025-49844 (CVSS 10.0) (#1671) unclecode 2026-03-12 11:24:42 +00:00
  • a73bc1c076 fix: MCP SSE endpoint crash — mount via raw ASGI Route (#1594) unclecode 2026-03-12 11:22:48 +00:00