mirror of
https://github.com/unclecode/crawl4ai.git
synced 2026-06-10 15:58:15 +00:00
Commit Graph
Select branches
Hide Pull Requests
0.3.5
0.3.6
0.3.7
0.3.72
0.3.73
0.3.74
0.3.742
0.3.743
0.3.744
0.3.745
0.3.75
0.4.0
0.4.1
0.4.2
2025-JUN-1
add-claude-github-actions-1759553116682
bug/proxy_config
bugfix/arun-many-cdp-managed-browser
claude/fix-update-pyopenssl-security-011CUPexU25DkNvoxfu5ZrnB
claude/implement-webhook-crawl-feature-011CULZY1Jy8N5MUkZqXkRVp
coderabbitai/docstrings/14vTVzYa3bH06l5wYNY9jTghrrj9FxxWL
codex/add-httpx-and-https-http2]-packages
codex/add-memory_wait_timeout-parameter-to-memoryadaptivedispatche
codex/add-use_stemming-parameter-to-bm25contentfiler
codex/add-vnc-streaming-endpoint-to-docker-server
codex/find-and-fix-a-bug
codex/fix-indexerror-in-browser-manager-py-with-use-managed-browse
copilot/modify-page-creation-and-logging
deploy
develop
devin/1748137705-fix-bm25contentfilter-docs
docker-test
docker/add_features
docker/base_config_overrides
docker/fix_sig
docs
docs-llm-strategies-update
docs-proxy-security
extract-media
feat/ahmed_dev
feat/follow-frameset
feat/undetected-browser
feature/agent-oai
feature/async-llm-extaction
feature/c4a-script
feature/configHealthMonitor
feature/content-filter
feature/content-filter-nasrin-1
feature/docker-cluster
feature/docker-hooks
feature/docker-llm-parameters
feature/marketplace-sponsor-logo
feature/nasrin-cli-deep-crawl
feature/scraper
feature/scraping-strategy
feature/telemetry
fix-async-url-seeder-redirect-verification
fix-cors-disable-web-security
fix/adaptive-crawler-llm-config
fix/arun-return-type-1898
fix/async-llm-extraction-arunMany
fix/batch-easy-issues-10
fix/bedrock-provider-prefix
fix/case_senstive_params
fix/cdp
fix/configurable-backoff
fix/deep-crawl-scoring
fix/deep-crawl-scoring-priority
fix/deep-crawl-stream-docker
fix/deep-crawl-streaming-contextvar-1917
fix/deprecated_pydantic
fix/deserialize-schema-type-false-positive
fix/dfs_deep_crawling
fix/docker
fix/docker-filter
fix/docker-jwt
fix/docker-llmEnvFile
fix/exit_with_q
fix/https-reditrect
fix/issue-1748-screenshot-scroll-delay
fix/issue-1776-adaptive-external-filter
fix/json-infinity-serialization
fix/linkPreviewScoring
fix/marketplace
fix/mcp-crawler-config-passthrough
fix/mcp-ensure-ascii-cjk-encoding
fix/n-playwright-stealth
fix/nlp-sentence-chunking-1909
fix/playwright-stealth
fix/preserve-tail-text-1938
fix/proxy_deprecation
fix/rate-limiter-burst-and-headers-1095
fix/relative_url
fix/release-notes-demo-code
fix/request-crawl-stream
fix/sandbox-escape-allowlist-attrs
fix/serialize-proxy-config
fix/sitemap_seeder
fix/timeline-deadlock-shared-lock-1754
fix/viewport_in_managed_browser
format-inline-tags
hooks
image-description
image-filterizer
implement-webhook-crawl-feature-011CULZY1Jy8N5MUkZqXkRVp
integrate-verified-prs
main
main-0.3.7
main-1
main-75
main-img-captionify
main-v0.2.72
merge-pr971
new-release-0.0.2
new-release-0.0.2-no-spacy
next
next-2-batch-crawl
next-JUN
next-MAY
next-alpine-docker
next-browser-farm
patch/generate_schema
pdf_processing
proxy-support
pull-84
release/v0.7.0
release/v0.7.1
release/v0.7.2
release/v0.7.3
release/v0.7.4
release/v0.7.5
release/v0.7.6
release/v0.7.7
release/v0.7.8
release/v0.8.0
release/v0.8.5
release/v0.8.7
release/v0.8.8
release/v0.8.9
run-many-deep-crawling
scraper-uc
scrapper
sponsors/thor_data
ssh-server
staging
unclecode-patch-1
unclecode-patch-2
unclecode-patch-3
unclecode-patch-4
unclecode-patch-5
unclecode-patch-6
unclecode-patch-7
unclecode-patch-8
unclecode/issue157
unclecode/issue167
v0.2.74
v0.2.76
v0.4.24
v0.4.241
v0.4.242
v0.4.243
v0.5.5
vr0.4.244
vr0.4.245
vr0.4.246
vr0.4.267
vr0.4.3b1
vr0.4.3b2
vr0.4.3b3
vr0.5.0.post1
vr0.5.0.post5
#1004
#1030
#1054
#1058
#1059
#1060
#1062
#1065
#1068
#1073
#1074
#1077
#1078
#108
#1081
#1083
#1085
#1085
#109
#1090
#1093
#1094
#1098
#1100
#1102
#1104
#1106
#1107
#1108
#1110
#1113
#1122
#1123
#1124
#1124
#1133
#1137
#1140
#1145
#1152
#1155
#1155
#1156
#1157
#1159
#1161
#1170
#1175
#1179
#1180
#1184
#1186
#119
#1192
#1193
#1195
#1200
#1207
#1208
#1209
#1210
#1211
#1212
#1214
#1220
#1223
#1225
#1232
#1234
#1238
#1239
#1245
#1249
#125
#1255
#1263
#1265
#1266
#1267
#1272
#1274
#128
#1281
#1282
#1285
#1289
#1289
#129
#1290
#1296
#13
#1303
#1304
#1305
#1307
#1308
#1313
#1319
#1334
#1334
#1336
#1337
#1339
#134
#135
#1351
#1356
#1358
#1361
#1364
#1366
#1368
#1369
#1371
#1372
#1373
#1376
#1378
#1381
#1383
#1384
#1386
#1387
#1388
#1389
#139
#1390
#1393
#1395
#1398
#1399
#14
#1402
#1408
#1413
#1416
#1417
#1420
#1422
#1425
#1426
#1432
#1433
#1435
#1436
#1440
#1441
#1444
#1447
#1448
#1450
#1451
#1454
#1463
#1464
#1465
#1467
#1469
#1470
#1471
#1478
#1482
#1483
#1486
#1488
#149
#1494
#1495
#1496
#1497
#1501
#1508
#1513
#1514
#1518
#1519
#1525
#1527
#1528
#1529
#1530
#1531
#1532
#1533
#1533
#1535
#1536
#1537
#1539
#1546
#1547
#1548
#1550
#1554
#1555
#1556
#1557
#1558
#1560
#1565
#1568
#1569
#1570
#1572
#1576
#158
#1580
#1588
#1589
#1590
#1592
#1595
#1596
#1597
#1598
#1599
#1600
#1605
#1607
#1609
#1612
#1613
#1617
#1617
#1619
#1620
#1622
#1623
#1624
#1628
#1630
#1633
#1637
#1640
#1641
#1643
#1645
#1648
#1650
#1653
#1655
#1661
#1662
#1667
#1668
#1674
#1676
#1677
#1681
#1683
#1685
#1689
#169
#1694
#1696
#1697
#1698
#1700
#1702
#1703
#1706
#1707
#1710
#1712
#1713
#1714
#1715
#1716
#1717
#1718
#1719
#172
#1720
#1721
#1722
#1723
#1724
#1729
#1730
#1733
#1734
#1744
#1746
#1752
#1755
#1756
#1756
#1759
#176
#1760
#1761
#1763
#1764
#1765
#1766
#1768
#1770
#1771
#1772
#1773
#1774
#1775
#1777
#1778
#1782
#1783
#1784
#1785
#1786
#1787
#1788
#1789
#1790
#1791
#1792
#1793
#1794
#1795
#1796
#1798
#1803
#1804
#1805
#1806
#1807
#1807
#1808
#1808
#1809
#1809
#1810
#1810
#1811
#1811
#1812
#1812
#1813
#1814
#1814
#1816
#1816
#1822
#1822
#1823
#1824
#1826
#1827
#1828
#1829
#1830
#1831
#1832
#1833
#1834
#1835
#1835
#1836
#1838
#1838
#1840
#1840
#1844
#1845
#1846
#1847
#1847
#1849
#1851
#1852
#1853
#1853
#1854
#1854
#1855
#1856
#1856
#1857
#1857
#1858
#1858
#1859
#1859
#1860
#1860
#1861
#1861
#1862
#1862
#1866
#1866
#1868
#1868
#1869
#1869
#1870
#1870
#1871
#1871
#1873
#1873
#1874
#1874
#1875
#1875
#1876
#1876
#1877
#1879
#1881
#1881
#1882
#1884
#1884
#1885
#1886
#1887
#1887
#1891
#1891
#1892
#1892
#1893
#1893
#1895
#1895
#1896
#1896
#1897
#1899
#1899
#1901
#1902
#1902
#1904
#1904
#1906
#1906
#1907
#1908
#1908
#1910
#1911
#1913
#1914
#1915
#1915
#1922
#1923
#1923
#1925
#1929
#1931
#1932
#1932
#1933
#1934
#1935
#1935
#1936
#1937
#1939
#194
#1940
#1941
#1941
#1943
#1944
#1944
#1946
#1946
#1947
#1951
#1952
#1953
#1955
#1955
#1957
#1957
#1960
#1965
#1965
#1967
#1969
#1970
#1970
#1971
#1975
#1976
#1977
#1977
#1978
#1979
#1981
#1983
#1983
#1984
#1984
#1985
#1985
#1986
#1986
#1987
#1987
#1988
#1988
#1989
#1990
#1991
#1991
#1993
#1993
#1994
#1994
#1995
#1995
#1997
#1997
#200
#2001
#2001
#2003
#2003
#2004
#2004
#2005
#2005
#2008
#2008
#2009
#2009
#215
#218
#229
#232
#234
#24
#249
#255
#269
#271
#279
#286
#288
#293
#294
#298
#299
#3
#300
#304
#312
#313
#314
#324
#33
#332
#335
#337
#34
#357
#358
#369
#37
#379
#387
#389
#390
#394
#403
#410
#411
#416
#419
#419
#427
#440
#444
#445
#458
#462
#465
#472
#475
#496
#510
#562
#581
#60
#605
#606
#609
#612
#617
#618
#622
#64
#640
#65
#657
#658
#66
#662
#671
#679
#680
#681
#685
#687
#706
#708
#723
#724
#729
#734
#741
#749
#75
#752
#754
#775
#776
#777
#788
#792
#799
#80
#800
#806
#808
#821
#84
#84
#846
#85
#864
#865
#868
#891
#899
#901
#903
#914
#915
#916
#918
#929
#93
#931
#945
#948
#95
#961
#967
#969
#970
#971
#973
#977
#983
#988
#988
#990
#994
#999
0.3.4
checkpoint-pre-antibot-fallback
docker-rebuild-v0.7.5
docker-rebuild-v0.7.6
docker-rebuild-v0.7.7
docker-rebuild-v0.7.8
docker-rebuild-v0.8.0
docker-rebuild-v0.8.5
docker-rebuild-v0.8.6
docker-rebuild-v0.8.7
docker-rebuild-v0.8.8
docker-rebuild-v0.8.9
v.3.72
v0.0.75
v0.1.0
v0.2.0
v0.2.1
v0.2.2
v0.2.4
v0.2.6
v0.2.7
v0.2.71
v0.2.72
v0.2.73
v0.2.74
v0.2.77
v0.3.0
v0.3.3
v0.3.6
v0.3.745
v0.3.746
v0.4.24
v0.4.243
v0.5.0.post1
v0.6.3
v0.7.0
v0.7.1
v0.7.2
v0.7.3
v0.7.4
v0.7.5
v0.7.6
v0.7.7
v0.7.8
v0.8.0
v0.8.5
v0.8.6
v0.8.7
v0.8.8
v0.8.9
vr0.6.0
vr0.6.0rc1
vr0.6.3
Select branches
Hide Pull Requests
0.3.5
0.3.6
0.3.7
0.3.72
0.3.73
0.3.74
0.3.742
0.3.743
0.3.744
0.3.745
0.3.75
0.4.0
0.4.1
0.4.2
2025-JUN-1
add-claude-github-actions-1759553116682
bug/proxy_config
bugfix/arun-many-cdp-managed-browser
claude/fix-update-pyopenssl-security-011CUPexU25DkNvoxfu5ZrnB
claude/implement-webhook-crawl-feature-011CULZY1Jy8N5MUkZqXkRVp
coderabbitai/docstrings/14vTVzYa3bH06l5wYNY9jTghrrj9FxxWL
codex/add-httpx-and-https-http2]-packages
codex/add-memory_wait_timeout-parameter-to-memoryadaptivedispatche
codex/add-use_stemming-parameter-to-bm25contentfiler
codex/add-vnc-streaming-endpoint-to-docker-server
codex/find-and-fix-a-bug
codex/fix-indexerror-in-browser-manager-py-with-use-managed-browse
copilot/modify-page-creation-and-logging
deploy
develop
devin/1748137705-fix-bm25contentfilter-docs
docker-test
docker/add_features
docker/base_config_overrides
docker/fix_sig
docs
docs-llm-strategies-update
docs-proxy-security
extract-media
feat/ahmed_dev
feat/follow-frameset
feat/undetected-browser
feature/agent-oai
feature/async-llm-extaction
feature/c4a-script
feature/configHealthMonitor
feature/content-filter
feature/content-filter-nasrin-1
feature/docker-cluster
feature/docker-hooks
feature/docker-llm-parameters
feature/marketplace-sponsor-logo
feature/nasrin-cli-deep-crawl
feature/scraper
feature/scraping-strategy
feature/telemetry
fix-async-url-seeder-redirect-verification
fix-cors-disable-web-security
fix/adaptive-crawler-llm-config
fix/arun-return-type-1898
fix/async-llm-extraction-arunMany
fix/batch-easy-issues-10
fix/bedrock-provider-prefix
fix/case_senstive_params
fix/cdp
fix/configurable-backoff
fix/deep-crawl-scoring
fix/deep-crawl-scoring-priority
fix/deep-crawl-stream-docker
fix/deep-crawl-streaming-contextvar-1917
fix/deprecated_pydantic
fix/deserialize-schema-type-false-positive
fix/dfs_deep_crawling
fix/docker
fix/docker-filter
fix/docker-jwt
fix/docker-llmEnvFile
fix/exit_with_q
fix/https-reditrect
fix/issue-1748-screenshot-scroll-delay
fix/issue-1776-adaptive-external-filter
fix/json-infinity-serialization
fix/linkPreviewScoring
fix/marketplace
fix/mcp-crawler-config-passthrough
fix/mcp-ensure-ascii-cjk-encoding
fix/n-playwright-stealth
fix/nlp-sentence-chunking-1909
fix/playwright-stealth
fix/preserve-tail-text-1938
fix/proxy_deprecation
fix/rate-limiter-burst-and-headers-1095
fix/relative_url
fix/release-notes-demo-code
fix/request-crawl-stream
fix/sandbox-escape-allowlist-attrs
fix/serialize-proxy-config
fix/sitemap_seeder
fix/timeline-deadlock-shared-lock-1754
fix/viewport_in_managed_browser
format-inline-tags
hooks
image-description
image-filterizer
implement-webhook-crawl-feature-011CULZY1Jy8N5MUkZqXkRVp
integrate-verified-prs
main
main-0.3.7
main-1
main-75
main-img-captionify
main-v0.2.72
merge-pr971
new-release-0.0.2
new-release-0.0.2-no-spacy
next
next-2-batch-crawl
next-JUN
next-MAY
next-alpine-docker
next-browser-farm
patch/generate_schema
pdf_processing
proxy-support
pull-84
release/v0.7.0
release/v0.7.1
release/v0.7.2
release/v0.7.3
release/v0.7.4
release/v0.7.5
release/v0.7.6
release/v0.7.7
release/v0.7.8
release/v0.8.0
release/v0.8.5
release/v0.8.7
release/v0.8.8
release/v0.8.9
run-many-deep-crawling
scraper-uc
scrapper
sponsors/thor_data
ssh-server
staging
unclecode-patch-1
unclecode-patch-2
unclecode-patch-3
unclecode-patch-4
unclecode-patch-5
unclecode-patch-6
unclecode-patch-7
unclecode-patch-8
unclecode/issue157
unclecode/issue167
v0.2.74
v0.2.76
v0.4.24
v0.4.241
v0.4.242
v0.4.243
v0.5.5
vr0.4.244
vr0.4.245
vr0.4.246
vr0.4.267
vr0.4.3b1
vr0.4.3b2
vr0.4.3b3
vr0.5.0.post1
vr0.5.0.post5
#1004
#1030
#1054
#1058
#1059
#1060
#1062
#1065
#1068
#1073
#1074
#1077
#1078
#108
#1081
#1083
#1085
#1085
#109
#1090
#1093
#1094
#1098
#1100
#1102
#1104
#1106
#1107
#1108
#1110
#1113
#1122
#1123
#1124
#1124
#1133
#1137
#1140
#1145
#1152
#1155
#1155
#1156
#1157
#1159
#1161
#1170
#1175
#1179
#1180
#1184
#1186
#119
#1192
#1193
#1195
#1200
#1207
#1208
#1209
#1210
#1211
#1212
#1214
#1220
#1223
#1225
#1232
#1234
#1238
#1239
#1245
#1249
#125
#1255
#1263
#1265
#1266
#1267
#1272
#1274
#128
#1281
#1282
#1285
#1289
#1289
#129
#1290
#1296
#13
#1303
#1304
#1305
#1307
#1308
#1313
#1319
#1334
#1334
#1336
#1337
#1339
#134
#135
#1351
#1356
#1358
#1361
#1364
#1366
#1368
#1369
#1371
#1372
#1373
#1376
#1378
#1381
#1383
#1384
#1386
#1387
#1388
#1389
#139
#1390
#1393
#1395
#1398
#1399
#14
#1402
#1408
#1413
#1416
#1417
#1420
#1422
#1425
#1426
#1432
#1433
#1435
#1436
#1440
#1441
#1444
#1447
#1448
#1450
#1451
#1454
#1463
#1464
#1465
#1467
#1469
#1470
#1471
#1478
#1482
#1483
#1486
#1488
#149
#1494
#1495
#1496
#1497
#1501
#1508
#1513
#1514
#1518
#1519
#1525
#1527
#1528
#1529
#1530
#1531
#1532
#1533
#1533
#1535
#1536
#1537
#1539
#1546
#1547
#1548
#1550
#1554
#1555
#1556
#1557
#1558
#1560
#1565
#1568
#1569
#1570
#1572
#1576
#158
#1580
#1588
#1589
#1590
#1592
#1595
#1596
#1597
#1598
#1599
#1600
#1605
#1607
#1609
#1612
#1613
#1617
#1617
#1619
#1620
#1622
#1623
#1624
#1628
#1630
#1633
#1637
#1640
#1641
#1643
#1645
#1648
#1650
#1653
#1655
#1661
#1662
#1667
#1668
#1674
#1676
#1677
#1681
#1683
#1685
#1689
#169
#1694
#1696
#1697
#1698
#1700
#1702
#1703
#1706
#1707
#1710
#1712
#1713
#1714
#1715
#1716
#1717
#1718
#1719
#172
#1720
#1721
#1722
#1723
#1724
#1729
#1730
#1733
#1734
#1744
#1746
#1752
#1755
#1756
#1756
#1759
#176
#1760
#1761
#1763
#1764
#1765
#1766
#1768
#1770
#1771
#1772
#1773
#1774
#1775
#1777
#1778
#1782
#1783
#1784
#1785
#1786
#1787
#1788
#1789
#1790
#1791
#1792
#1793
#1794
#1795
#1796
#1798
#1803
#1804
#1805
#1806
#1807
#1807
#1808
#1808
#1809
#1809
#1810
#1810
#1811
#1811
#1812
#1812
#1813
#1814
#1814
#1816
#1816
#1822
#1822
#1823
#1824
#1826
#1827
#1828
#1829
#1830
#1831
#1832
#1833
#1834
#1835
#1835
#1836
#1838
#1838
#1840
#1840
#1844
#1845
#1846
#1847
#1847
#1849
#1851
#1852
#1853
#1853
#1854
#1854
#1855
#1856
#1856
#1857
#1857
#1858
#1858
#1859
#1859
#1860
#1860
#1861
#1861
#1862
#1862
#1866
#1866
#1868
#1868
#1869
#1869
#1870
#1870
#1871
#1871
#1873
#1873
#1874
#1874
#1875
#1875
#1876
#1876
#1877
#1879
#1881
#1881
#1882
#1884
#1884
#1885
#1886
#1887
#1887
#1891
#1891
#1892
#1892
#1893
#1893
#1895
#1895
#1896
#1896
#1897
#1899
#1899
#1901
#1902
#1902
#1904
#1904
#1906
#1906
#1907
#1908
#1908
#1910
#1911
#1913
#1914
#1915
#1915
#1922
#1923
#1923
#1925
#1929
#1931
#1932
#1932
#1933
#1934
#1935
#1935
#1936
#1937
#1939
#194
#1940
#1941
#1941
#1943
#1944
#1944
#1946
#1946
#1947
#1951
#1952
#1953
#1955
#1955
#1957
#1957
#1960
#1965
#1965
#1967
#1969
#1970
#1970
#1971
#1975
#1976
#1977
#1977
#1978
#1979
#1981
#1983
#1983
#1984
#1984
#1985
#1985
#1986
#1986
#1987
#1987
#1988
#1988
#1989
#1990
#1991
#1991
#1993
#1993
#1994
#1994
#1995
#1995
#1997
#1997
#200
#2001
#2001
#2003
#2003
#2004
#2004
#2005
#2005
#2008
#2008
#2009
#2009
#215
#218
#229
#232
#234
#24
#249
#255
#269
#271
#279
#286
#288
#293
#294
#298
#299
#3
#300
#304
#312
#313
#314
#324
#33
#332
#335
#337
#34
#357
#358
#369
#37
#379
#387
#389
#390
#394
#403
#410
#411
#416
#419
#419
#427
#440
#444
#445
#458
#462
#465
#472
#475
#496
#510
#562
#581
#60
#605
#606
#609
#612
#617
#618
#622
#64
#640
#65
#657
#658
#66
#662
#671
#679
#680
#681
#685
#687
#706
#708
#723
#724
#729
#734
#741
#749
#75
#752
#754
#775
#776
#777
#788
#792
#799
#80
#800
#806
#808
#821
#84
#84
#846
#85
#864
#865
#868
#891
#899
#901
#903
#914
#915
#916
#918
#929
#93
#931
#945
#948
#95
#961
#967
#969
#970
#971
#973
#977
#983
#988
#988
#990
#994
#999
0.3.4
checkpoint-pre-antibot-fallback
docker-rebuild-v0.7.5
docker-rebuild-v0.7.6
docker-rebuild-v0.7.7
docker-rebuild-v0.7.8
docker-rebuild-v0.8.0
docker-rebuild-v0.8.5
docker-rebuild-v0.8.6
docker-rebuild-v0.8.7
docker-rebuild-v0.8.8
docker-rebuild-v0.8.9
v.3.72
v0.0.75
v0.1.0
v0.2.0
v0.2.1
v0.2.2
v0.2.4
v0.2.6
v0.2.7
v0.2.71
v0.2.72
v0.2.73
v0.2.74
v0.2.77
v0.3.0
v0.3.3
v0.3.6
v0.3.745
v0.3.746
v0.4.24
v0.4.243
v0.5.0.post1
v0.6.3
v0.7.0
v0.7.1
v0.7.2
v0.7.3
v0.7.4
v0.7.5
v0.7.6
v0.7.7
v0.7.8
v0.8.0
v0.8.5
v0.8.6
v0.8.7
v0.8.8
v0.8.9
vr0.6.0
vr0.6.0rc1
vr0.6.3
-
7155778eac
chore: move from faust-cchardet to chardet
Aravind Karnam
2025-04-03 17:42:51 +05:30 -
4133e5460d
typo-fix: https://github.com/unclecode/crawl4ai/pull/918
Aravind Karnam
2025-04-03 17:42:24 +05:30 -
73fda8a6ec
fix: address the PR review: https://github.com/unclecode/crawl4ai/pull/899#discussion_r2024639193
Aravind Karnam
2025-04-03 13:47:13 +05:30 -
86df20234b
fix(crawler): handle exceptions in get_page call to ensure page retrieval
UncleCode
2025-04-02 21:25:24 +08:00 -
179921a131
fix(crawler): update get_page call to include additional return value
UncleCode
2025-04-02 19:01:30 +08:00 -
9e16a4bb26
Merge next and resolve conflicts
Aravind Karnam
2025-04-02 12:18:23 +05:30 -
c5cac2b459
feat(browser): add BrowserHub for centralized browser management and resource sharing
UncleCode
2025-04-01 20:35:02 +08:00 -
555455d710
feat(browser): implement browser pooling and page pre-warming
UncleCode
2025-03-31 21:55:07 +08:00 -
765f856ed4
Merge pull request #808 from dvschuyl/bug/parse-srcset-fix-float-width
Aravind
2025-03-31 18:21:09 +05:30 -
757e3177ed
fix: https://github.com/unclecode/crawl4ai/issues/839
Aravind Karnam
2025-03-31 17:10:04 +05:30 -
d8357e80d2
Merge pull request #915 from maggie-edkey/css-selector
Aravind
2025-03-31 13:03:35 +05:30 -
ef1f0c4102
fix:https://github.com/unclecode/crawl4ai/issues/701
Aravind Karnam
2025-03-31 12:43:32 +05:30 -
1119f2f5b5
fix: https://github.com/unclecode/crawl4ai/issues/911
maggie.wang
2025-03-31 14:05:54 +08:00 -
bb02398086
refactor(browser): improve browser strategy architecture and lifecycle management
UncleCode
2025-03-30 20:58:39 +08:00 -
3ff7eec8f3
refactor(browser): consolidate browser strategy implementations
UncleCode
2025-03-28 22:47:28 +08:00 -
d8cbeff386
fix: https://github.com/unclecode/crawl4ai/issues/842
Aravind Karnam
2025-03-28 19:31:05 +05:30 -
64f20ab44a
refactor(docker): update Dockerfile and browser strategy to use Chromium
UncleCode
2025-03-28 15:59:02 +08:00 -
57e0423b3a
fix:target_element should not affect link extraction. -> https://github.com/unclecode/crawl4ai/issues/902
Aravind Karnam
2025-03-28 12:56:37 +05:30 -
c635f6b9a2
refactor(browser): reorganize browser strategies and improve Docker implementation
UncleCode
2025-03-27 21:35:13 +08:00 -
7be5427283
Merge branch 'next' into 2025-MAR-ALPHA-1
Aravind Karnam
2025-03-27 12:29:32 +05:30 -
7f93e88379
refactor(tests): remove unused imports in test_docker_browser.py
UncleCode
2025-03-26 15:19:29 +08:00 -
40d4dd36c9
chore(version): bump version to 0.5.0.post8 and update post-installation setup
UncleCode
2025-03-25 21:56:49 +08:00 -
d8f38f2298
chore(version): bump version to 0.5.0.post7
UncleCode
2025-03-25 21:47:19 +08:00 -
5c88d1310d
feat(cli): add output file option and integrate LXML web scraping strategy
UncleCode
2025-03-25 21:38:24 +08:00 -
4a20d7f7c2
feat(cli): add quick JSON extraction and global config management
UncleCode
2025-03-25 20:30:25 +08:00 -
585e5e5973
fix: https://github.com/unclecode/crawl4ai/issues/733
Aravind Karnam
2025-03-25 15:17:59 +05:30 -
e3111d0a32
fix: prevent session closing after each request to maintain connection pool. Fixes: https://github.com/unclecode/crawl4ai/issues/867
Aravind Karnam
2025-03-25 13:46:55 +05:30 -
2f0e217751
Chore: Add brotli as dependancy to fix: https://github.com/unclecode/crawl4ai/issues/867
Aravind Karnam
2025-03-25 13:44:41 +05:30 -
6405cf0a6f
Merge branch 'vr0.5.0.post5' into next
UncleCode
2025-03-25 14:51:29 +08:00 -
6eed4adc65
Merge branch 'vr0.5.0.post5'
UncleCode
2025-03-25 12:24:07 +08:00 -
bdd9db579a
chore(version): bump version to 0.5.0.post6
vr0.5.0.post5
UncleCode
2025-03-25 12:01:36 +08:00 -
1107fa1d62
feat(cli): enhance markdown generation with default content filters
UncleCode
2025-03-25 11:56:00 +08:00 -
efa73257c5
Merge branch 'next' into 2025-MAR-ALPHA-1
Aravind Karnam
2025-03-24 21:57:29 +05:30 -
4dfd270161
fix: #855
run-many-deep-crawling
UncleCode
2025-03-24 22:54:53 +08:00 -
8c08521301
feat(browser): add Docker-based browser automation strategy
UncleCode
2025-03-24 21:36:58 +08:00 -
462d5765e2
fix(browser): improve storage state persistence in CDP strategy
UncleCode
2025-03-23 21:06:41 +08:00 -
6eeb2e4076
feat(browser): enhance browser context creation with user data directory support and improved storage state handling
UncleCode
2025-03-23 19:07:13 +08:00 -
0094cac675
refactor(browser): improve parallel crawling and browser management
UncleCode
2025-03-23 18:53:24 +08:00 -
4ab0893ffb
feat(browser): implement modular browser management system
UncleCode
2025-03-21 22:50:00 +08:00 -
e01d1e73e1
fix: link normalisation in BestFirstStrategy
Aravind Karnam
2025-03-21 17:34:13 +05:30 -
471d110c5e
fix: url normalisation ref: https://github.com/unclecode/crawl4ai/issues/841
Aravind Karnam
2025-03-21 16:48:07 +05:30 -
f89113377a
fix: Move adding of visited urls to the 'visited' set, when queueing the URLs instead of after dequeuing, this is to prevent duplicate crawls. https://github.com/unclecode/crawl4ai/issues/843
Aravind Karnam
2025-03-21 13:44:57 +05:30 -
6740e87b4d
fix: remove trailing slash when the path is empty. This is causing dupicate crawls
Aravind Karnam
2025-03-21 13:41:31 +05:30 -
8b761f232b
fix: improve logged url readability by decoding encoded urls
Aravind Karnam
2025-03-21 13:40:23 +05:30 -
e0c2a7c284
chore: remove mistakenly commited deps.txt file
Aravind Karnam
2025-03-21 11:06:46 +05:30 -
ac2f9ae533
fix: streamline url status logging via single entrypoint i.e. logger.url_status
Aravind Karnam
2025-03-20 18:59:15 +05:30 -
eedda1ae5c
fix: Truncate long urls in middle than end since users are confused that same url is being scraped several times. Also remove labels on status and timer to be replaced with symbols to save space and display more URL
Aravind Karnam
2025-03-20 18:56:19 +05:30 -
8cecbec7a7
Merge branch 'next' into 2025-MAR-ALPHA-1
Aravind Karnam
2025-03-20 17:07:53 +05:30 -
6432ff1257
feat(browser): add builtin browser management system
UncleCode
2025-03-20 12:13:59 +08:00 -
4359b12003
docs + fix: Update example for full page screenshot & PDF export. Fix the bug Error: crawl4ai.async_webcrawler.AsyncWebCrawler.aprocess_html() got multiple values for keyword argument - for screenshot param. https://github.com/unclecode/crawl4ai/issues/822#issuecomment-2732602118
Aravind Karnam
2025-03-18 17:20:24 +05:30 -
5358ac0fc2
refactor: clean up imports and improve JSON schema generation instructions
UncleCode
2025-03-18 18:53:34 +08:00 -
529a79725e
docs: remove hallucinations from docs for CrawlerRunConfig + Add chunking strategy docs in the table
Aravind Karnam
2025-03-18 16:14:00 +05:30 -
9109ecd8fc
chore: Raise an exception with clear messaging when body tag is missing in the fetched html. The message should warn users to add appropriate wait_for condition to wait until body tag is loaded into DOM. fixes: https://github.com/unclecode/crawl4ai/issues/804
Aravind Karnam
2025-03-18 15:26:20 +05:30 -
84883be513
Merge branch 'next' into 2025-MAR-ALPHA-1
Aravind Karnam
2025-03-18 15:12:21 +05:30 -
79328e4292
Create main.yml (#846)
Aravind
2025-03-17 18:17:57 +05:30 -
a24799918c
feat(llm): add additional LLM configuration parameters
UncleCode
2025-03-14 21:36:23 +08:00 -
a31d7b86be
feat(changelog): update CHANGELOG for version 0.5.0.post5 with new features, changes, fixes, and breaking changes
UncleCode
2025-03-14 15:26:37 +08:00 -
7884a98be7
feat(crawler): add experimental parameters support and optimize browser handling
UncleCode
2025-03-14 14:39:24 +08:00 -
c190ba816d
refactor: Instead of custom validation of question, rely on the built in FastAPI validator, so generated API docs also reflects this expectation correctly
Aravind Karnam
2025-03-14 09:40:50 +05:30 -
a3954dd4c6
refactor: Move the checking of protocol and prepending protocol inside api handlers
Aravind Karnam
2025-03-14 09:39:10 +05:30 -
6e3c048328
feat(api): refactor crawl request handling to streamline single and multiple URL processing
UncleCode
2025-03-13 22:30:38 +08:00 -
b750542e6d
feat(crawler): optimize single URL handling and add performance comparison
UncleCode
2025-03-13 22:15:15 +08:00 -
cbb8755972
Merge branch 'next' into 2025-MAR-ALPHA-1
Aravind Karnam
2025-03-13 10:42:22 +05:30 -
dc36997a08
feat(schema): improve HTML preprocessing for schema generation
UncleCode
2025-03-12 22:40:46 +08:00 -
1630fbdafe
feat(monitor): add real-time crawler monitoring system with memory management
UncleCode
2025-03-12 19:05:24 +08:00 -
341b7a5f2a
🐛 Truncate width to integer string in parse_srcset
dvschuyl
2025-03-11 11:05:14 +01:00 -
3ea3c0520d
Add all 5 deployments solution for testing
deploy
UncleCode
2025-03-10 18:57:14 +08:00 -
9547bada3a
feat(content): add target_elements parameter for selective content extraction
UncleCode
2025-03-10 18:54:51 +08:00 -
9d69fce834
feat(scraping): add smart table extraction and analysis capabilities
UncleCode
2025-03-09 21:31:33 +08:00 -
c6a605ccce
feat(filters): add reverse option to URLPatternFilter
UncleCode
2025-03-08 18:54:41 +08:00 -
4aeb7ef9ad
refactor(proxy): consolidate proxy configuration handling
UncleCode
2025-03-07 23:14:11 +08:00 -
a68cbb232b
feat(browser): add standalone CDP browser launch and lxml extraction strategy
UncleCode
2025-03-07 20:55:56 +08:00 -
e1b3bfe6fb
Merge branch 'vr0.5.0.post4'
UncleCode
2025-03-06 22:46:44 +08:00 -
f78c46446b
feat(deep-crawling): improve URL normalization and domain filtering
UncleCode
2025-03-06 22:45:57 +08:00 -
1b72880007
chore(version): bump version to 0.5.0.post3
UncleCode
2025-03-06 20:32:32 +08:00 -
29f7915b79
fix(models): support float timestamps in CrawlStats
UncleCode
2025-03-06 20:30:57 +08:00 -
2327db6fdc
refactor(crawler): introduce CrawlResultContainer and simplify interfaces
UncleCode
2025-03-05 22:23:08 +08:00 -
fd02dc782d
Merge branch 'main' of https://github.com/unclecode/crawl4ai
UncleCode
2025-03-05 17:15:48 +08:00 -
3a234ec950
fix(auth): make JWT authentication optional with fallback
UncleCode
2025-03-05 17:14:42 +08:00 -
9e89d27fcd
chore(version): bump version to 0.5.0.post2
UncleCode
2025-03-05 14:18:29 +08:00 -
b3ec7ce960
Merge branch 'vr0.5.0.post1' into next
UncleCode
2025-03-05 14:17:19 +08:00 -
baee4949d3
refactor(llm): rename LlmConfig to LLMConfig for consistency
UncleCode
2025-03-05 14:17:04 +08:00 -
14fe5ef873
Update config.yml
UncleCode
2025-03-05 14:16:24 +08:00 -
e12d2e29e5
Update config.yml
unclecode-patch-8
UncleCode
2025-03-05 14:15:57 +08:00 -
fc425023f5
Update config.yml
UncleCode
2025-03-05 12:51:07 +08:00 -
9c58e4ce2e
fix(docs): correct section numbering in deepcrawl_example.py tutorial
v0.5.0.post1
UncleCode
2025-03-04 20:57:33 +08:00 -
df6a6d5f4f
refactor(docs): reorganize tutorial sections and update wrap-up example
UncleCode
2025-03-04 20:55:09 +08:00 -
e896c08f9c
chore(version): bump version to 0.5.0.post1
vr0.5.0.post1
UncleCode
2025-03-04 20:29:27 +08:00 -
56bc3c6e45
refactor(cli): improve CLI default command handling
UncleCode
2025-03-04 20:28:16 +08:00 -
cbef406f9b
docs: update README for version 0.5.0 release with new features and CLI commands
UncleCode
2025-03-04 19:24:46 +08:00 -
8a76563018
chore(docs): update site version to v0.5.x in mkdocs configuration
UncleCode
2025-03-04 18:30:03 +08:00 -
415c1c5bee
refactor(core): replace float('inf') with math.inf
UncleCode
2025-03-04 18:23:55 +08:00 -
f334daa979
feat(deep-crawling): add max_pages and score_threshold parameters for improved crawling control
UncleCode
2025-03-03 21:54:58 +08:00 -
504207faa6
docs: update text in llm-strategies.md to reflect new changes in LlmConfig
Aravind Karnam
2025-03-03 19:24:44 +05:30 -
d024749633
refactor(deep-crawl): add max_pages limit and improve crawl control
UncleCode
2025-03-03 21:51:11 +08:00 -
f14e4a4b67
Merge pull request #776 from jawshoeadan/patch-1
Aravind
2025-03-03 19:01:30 +05:30 -
1e819cdb26
fixes: https://github.com/unclecode/crawl4ai/issues/774
Aravind Karnam
2025-03-03 11:53:15 +05:30 -
5edfea279d
Fix LiteLLM branding and link
jawshoeadan
2025-03-02 16:58:00 +01:00 -
c612f9a852
feat(profiles): add CLI command for crawling with browser profiles
UncleCode
2025-03-02 21:33:33 +08:00 -
95175cb394
feat(cli): add browser profile management functionality
UncleCode
2025-03-02 20:54:45 +08:00