mirror of
https://github.com/unclecode/crawl4ai.git
synced 2026-06-10 15:58:15 +00:00
Commit Graph
Select branches
Hide Pull Requests
0.3.5
0.3.6
0.3.7
0.3.72
0.3.73
0.3.74
0.3.742
0.3.743
0.3.744
0.3.745
0.3.75
0.4.0
0.4.1
0.4.2
2025-JUN-1
add-claude-github-actions-1759553116682
bug/proxy_config
bugfix/arun-many-cdp-managed-browser
claude/fix-update-pyopenssl-security-011CUPexU25DkNvoxfu5ZrnB
claude/implement-webhook-crawl-feature-011CULZY1Jy8N5MUkZqXkRVp
coderabbitai/docstrings/14vTVzYa3bH06l5wYNY9jTghrrj9FxxWL
codex/add-httpx-and-https-http2]-packages
codex/add-memory_wait_timeout-parameter-to-memoryadaptivedispatche
codex/add-use_stemming-parameter-to-bm25contentfiler
codex/add-vnc-streaming-endpoint-to-docker-server
codex/find-and-fix-a-bug
codex/fix-indexerror-in-browser-manager-py-with-use-managed-browse
copilot/modify-page-creation-and-logging
deploy
develop
devin/1748137705-fix-bm25contentfilter-docs
docker-test
docker/add_features
docker/base_config_overrides
docker/fix_sig
docs
docs-llm-strategies-update
docs-proxy-security
extract-media
feat/ahmed_dev
feat/follow-frameset
feat/undetected-browser
feature/agent-oai
feature/async-llm-extaction
feature/c4a-script
feature/configHealthMonitor
feature/content-filter
feature/content-filter-nasrin-1
feature/docker-cluster
feature/docker-hooks
feature/docker-llm-parameters
feature/marketplace-sponsor-logo
feature/nasrin-cli-deep-crawl
feature/scraper
feature/scraping-strategy
feature/telemetry
fix-async-url-seeder-redirect-verification
fix-cors-disable-web-security
fix/adaptive-crawler-llm-config
fix/arun-return-type-1898
fix/async-llm-extraction-arunMany
fix/batch-easy-issues-10
fix/bedrock-provider-prefix
fix/case_senstive_params
fix/cdp
fix/configurable-backoff
fix/deep-crawl-scoring
fix/deep-crawl-scoring-priority
fix/deep-crawl-stream-docker
fix/deep-crawl-streaming-contextvar-1917
fix/deprecated_pydantic
fix/deserialize-schema-type-false-positive
fix/dfs_deep_crawling
fix/docker
fix/docker-filter
fix/docker-jwt
fix/docker-llmEnvFile
fix/exit_with_q
fix/https-reditrect
fix/issue-1748-screenshot-scroll-delay
fix/issue-1776-adaptive-external-filter
fix/json-infinity-serialization
fix/linkPreviewScoring
fix/marketplace
fix/mcp-crawler-config-passthrough
fix/mcp-ensure-ascii-cjk-encoding
fix/n-playwright-stealth
fix/nlp-sentence-chunking-1909
fix/playwright-stealth
fix/preserve-tail-text-1938
fix/proxy_deprecation
fix/rate-limiter-burst-and-headers-1095
fix/relative_url
fix/release-notes-demo-code
fix/request-crawl-stream
fix/sandbox-escape-allowlist-attrs
fix/serialize-proxy-config
fix/sitemap_seeder
fix/timeline-deadlock-shared-lock-1754
fix/viewport_in_managed_browser
format-inline-tags
hooks
image-description
image-filterizer
implement-webhook-crawl-feature-011CULZY1Jy8N5MUkZqXkRVp
integrate-verified-prs
main
main-0.3.7
main-1
main-75
main-img-captionify
main-v0.2.72
merge-pr971
new-release-0.0.2
new-release-0.0.2-no-spacy
next
next-2-batch-crawl
next-JUN
next-MAY
next-alpine-docker
next-browser-farm
patch/generate_schema
pdf_processing
proxy-support
pull-84
release/v0.7.0
release/v0.7.1
release/v0.7.2
release/v0.7.3
release/v0.7.4
release/v0.7.5
release/v0.7.6
release/v0.7.7
release/v0.7.8
release/v0.8.0
release/v0.8.5
release/v0.8.7
release/v0.8.8
release/v0.8.9
run-many-deep-crawling
scraper-uc
scrapper
sponsors/thor_data
ssh-server
staging
unclecode-patch-1
unclecode-patch-2
unclecode-patch-3
unclecode-patch-4
unclecode-patch-5
unclecode-patch-6
unclecode-patch-7
unclecode-patch-8
unclecode/issue157
unclecode/issue167
v0.2.74
v0.2.76
v0.4.24
v0.4.241
v0.4.242
v0.4.243
v0.5.5
vr0.4.244
vr0.4.245
vr0.4.246
vr0.4.267
vr0.4.3b1
vr0.4.3b2
vr0.4.3b3
vr0.5.0.post1
vr0.5.0.post5
#1004
#1030
#1054
#1058
#1059
#1060
#1062
#1065
#1068
#1073
#1074
#1077
#1078
#108
#1081
#1083
#1085
#1085
#109
#1090
#1093
#1094
#1098
#1100
#1102
#1104
#1106
#1107
#1108
#1110
#1113
#1122
#1123
#1124
#1124
#1133
#1137
#1140
#1145
#1152
#1155
#1155
#1156
#1157
#1159
#1161
#1170
#1175
#1179
#1180
#1184
#1186
#119
#1192
#1193
#1195
#1200
#1207
#1208
#1209
#1210
#1211
#1212
#1214
#1220
#1223
#1225
#1232
#1234
#1238
#1239
#1245
#1249
#125
#1255
#1263
#1265
#1266
#1267
#1272
#1274
#128
#1281
#1282
#1285
#1289
#1289
#129
#1290
#1296
#13
#1303
#1304
#1305
#1307
#1308
#1313
#1319
#1334
#1334
#1336
#1337
#1339
#134
#135
#1351
#1356
#1358
#1361
#1364
#1366
#1368
#1369
#1371
#1372
#1373
#1376
#1378
#1381
#1383
#1384
#1386
#1387
#1388
#1389
#139
#1390
#1393
#1395
#1398
#1399
#14
#1402
#1408
#1413
#1416
#1417
#1420
#1422
#1425
#1426
#1432
#1433
#1435
#1436
#1440
#1441
#1444
#1447
#1448
#1450
#1451
#1454
#1463
#1464
#1465
#1467
#1469
#1470
#1471
#1478
#1482
#1483
#1486
#1488
#149
#1494
#1495
#1496
#1497
#1501
#1508
#1513
#1514
#1518
#1519
#1525
#1527
#1528
#1529
#1530
#1531
#1532
#1533
#1533
#1535
#1536
#1537
#1539
#1546
#1547
#1548
#1550
#1554
#1555
#1556
#1557
#1558
#1560
#1565
#1568
#1569
#1570
#1572
#1576
#158
#1580
#1588
#1589
#1590
#1592
#1595
#1596
#1597
#1598
#1599
#1600
#1605
#1607
#1609
#1612
#1613
#1617
#1617
#1619
#1620
#1622
#1623
#1624
#1628
#1630
#1633
#1637
#1640
#1641
#1643
#1645
#1648
#1650
#1653
#1655
#1661
#1662
#1667
#1668
#1674
#1676
#1677
#1681
#1683
#1685
#1689
#169
#1694
#1696
#1697
#1698
#1700
#1702
#1703
#1706
#1707
#1710
#1712
#1713
#1714
#1715
#1716
#1717
#1718
#1719
#172
#1720
#1721
#1722
#1723
#1724
#1729
#1730
#1733
#1734
#1744
#1746
#1752
#1755
#1756
#1756
#1759
#176
#1760
#1761
#1763
#1764
#1765
#1766
#1768
#1770
#1771
#1772
#1773
#1774
#1775
#1777
#1778
#1782
#1783
#1784
#1785
#1786
#1787
#1788
#1789
#1790
#1791
#1792
#1793
#1794
#1795
#1796
#1798
#1803
#1804
#1805
#1806
#1807
#1807
#1808
#1808
#1809
#1809
#1810
#1810
#1811
#1811
#1812
#1812
#1813
#1814
#1814
#1816
#1816
#1822
#1822
#1823
#1824
#1826
#1827
#1828
#1829
#1830
#1831
#1832
#1833
#1834
#1835
#1835
#1836
#1838
#1838
#1840
#1840
#1844
#1845
#1846
#1847
#1847
#1849
#1851
#1852
#1853
#1853
#1854
#1854
#1855
#1856
#1856
#1857
#1857
#1858
#1858
#1859
#1859
#1860
#1860
#1861
#1861
#1862
#1862
#1866
#1866
#1868
#1868
#1869
#1869
#1870
#1870
#1871
#1871
#1873
#1873
#1874
#1874
#1875
#1875
#1876
#1876
#1877
#1879
#1881
#1881
#1882
#1884
#1884
#1885
#1886
#1887
#1887
#1891
#1891
#1892
#1892
#1893
#1893
#1895
#1895
#1896
#1896
#1897
#1899
#1899
#1901
#1902
#1902
#1904
#1904
#1906
#1906
#1907
#1908
#1908
#1910
#1911
#1913
#1914
#1915
#1915
#1922
#1923
#1923
#1925
#1929
#1931
#1932
#1932
#1933
#1934
#1935
#1935
#1936
#1937
#1939
#194
#1940
#1941
#1941
#1943
#1944
#1944
#1946
#1946
#1947
#1951
#1952
#1953
#1955
#1955
#1957
#1957
#1960
#1965
#1965
#1967
#1969
#1970
#1970
#1971
#1975
#1976
#1977
#1977
#1978
#1979
#1981
#1983
#1983
#1984
#1984
#1985
#1985
#1986
#1986
#1987
#1987
#1988
#1988
#1989
#1990
#1991
#1991
#1993
#1993
#1994
#1994
#1995
#1995
#1997
#1997
#200
#2001
#2001
#2003
#2003
#2004
#2004
#2005
#2005
#2008
#2008
#2009
#2009
#215
#218
#229
#232
#234
#24
#249
#255
#269
#271
#279
#286
#288
#293
#294
#298
#299
#3
#300
#304
#312
#313
#314
#324
#33
#332
#335
#337
#34
#357
#358
#369
#37
#379
#387
#389
#390
#394
#403
#410
#411
#416
#419
#419
#427
#440
#444
#445
#458
#462
#465
#472
#475
#496
#510
#562
#581
#60
#605
#606
#609
#612
#617
#618
#622
#64
#640
#65
#657
#658
#66
#662
#671
#679
#680
#681
#685
#687
#706
#708
#723
#724
#729
#734
#741
#749
#75
#752
#754
#775
#776
#777
#788
#792
#799
#80
#800
#806
#808
#821
#84
#84
#846
#85
#864
#865
#868
#891
#899
#901
#903
#914
#915
#916
#918
#929
#93
#931
#945
#948
#95
#961
#967
#969
#970
#971
#973
#977
#983
#988
#988
#990
#994
#999
0.3.4
checkpoint-pre-antibot-fallback
docker-rebuild-v0.7.5
docker-rebuild-v0.7.6
docker-rebuild-v0.7.7
docker-rebuild-v0.7.8
docker-rebuild-v0.8.0
docker-rebuild-v0.8.5
docker-rebuild-v0.8.6
docker-rebuild-v0.8.7
docker-rebuild-v0.8.8
docker-rebuild-v0.8.9
v.3.72
v0.0.75
v0.1.0
v0.2.0
v0.2.1
v0.2.2
v0.2.4
v0.2.6
v0.2.7
v0.2.71
v0.2.72
v0.2.73
v0.2.74
v0.2.77
v0.3.0
v0.3.3
v0.3.6
v0.3.745
v0.3.746
v0.4.24
v0.4.243
v0.5.0.post1
v0.6.3
v0.7.0
v0.7.1
v0.7.2
v0.7.3
v0.7.4
v0.7.5
v0.7.6
v0.7.7
v0.7.8
v0.8.0
v0.8.5
v0.8.6
v0.8.7
v0.8.8
v0.8.9
vr0.6.0
vr0.6.0rc1
vr0.6.3
Select branches
Hide Pull Requests
0.3.5
0.3.6
0.3.7
0.3.72
0.3.73
0.3.74
0.3.742
0.3.743
0.3.744
0.3.745
0.3.75
0.4.0
0.4.1
0.4.2
2025-JUN-1
add-claude-github-actions-1759553116682
bug/proxy_config
bugfix/arun-many-cdp-managed-browser
claude/fix-update-pyopenssl-security-011CUPexU25DkNvoxfu5ZrnB
claude/implement-webhook-crawl-feature-011CULZY1Jy8N5MUkZqXkRVp
coderabbitai/docstrings/14vTVzYa3bH06l5wYNY9jTghrrj9FxxWL
codex/add-httpx-and-https-http2]-packages
codex/add-memory_wait_timeout-parameter-to-memoryadaptivedispatche
codex/add-use_stemming-parameter-to-bm25contentfiler
codex/add-vnc-streaming-endpoint-to-docker-server
codex/find-and-fix-a-bug
codex/fix-indexerror-in-browser-manager-py-with-use-managed-browse
copilot/modify-page-creation-and-logging
deploy
develop
devin/1748137705-fix-bm25contentfilter-docs
docker-test
docker/add_features
docker/base_config_overrides
docker/fix_sig
docs
docs-llm-strategies-update
docs-proxy-security
extract-media
feat/ahmed_dev
feat/follow-frameset
feat/undetected-browser
feature/agent-oai
feature/async-llm-extaction
feature/c4a-script
feature/configHealthMonitor
feature/content-filter
feature/content-filter-nasrin-1
feature/docker-cluster
feature/docker-hooks
feature/docker-llm-parameters
feature/marketplace-sponsor-logo
feature/nasrin-cli-deep-crawl
feature/scraper
feature/scraping-strategy
feature/telemetry
fix-async-url-seeder-redirect-verification
fix-cors-disable-web-security
fix/adaptive-crawler-llm-config
fix/arun-return-type-1898
fix/async-llm-extraction-arunMany
fix/batch-easy-issues-10
fix/bedrock-provider-prefix
fix/case_senstive_params
fix/cdp
fix/configurable-backoff
fix/deep-crawl-scoring
fix/deep-crawl-scoring-priority
fix/deep-crawl-stream-docker
fix/deep-crawl-streaming-contextvar-1917
fix/deprecated_pydantic
fix/deserialize-schema-type-false-positive
fix/dfs_deep_crawling
fix/docker
fix/docker-filter
fix/docker-jwt
fix/docker-llmEnvFile
fix/exit_with_q
fix/https-reditrect
fix/issue-1748-screenshot-scroll-delay
fix/issue-1776-adaptive-external-filter
fix/json-infinity-serialization
fix/linkPreviewScoring
fix/marketplace
fix/mcp-crawler-config-passthrough
fix/mcp-ensure-ascii-cjk-encoding
fix/n-playwright-stealth
fix/nlp-sentence-chunking-1909
fix/playwright-stealth
fix/preserve-tail-text-1938
fix/proxy_deprecation
fix/rate-limiter-burst-and-headers-1095
fix/relative_url
fix/release-notes-demo-code
fix/request-crawl-stream
fix/sandbox-escape-allowlist-attrs
fix/serialize-proxy-config
fix/sitemap_seeder
fix/timeline-deadlock-shared-lock-1754
fix/viewport_in_managed_browser
format-inline-tags
hooks
image-description
image-filterizer
implement-webhook-crawl-feature-011CULZY1Jy8N5MUkZqXkRVp
integrate-verified-prs
main
main-0.3.7
main-1
main-75
main-img-captionify
main-v0.2.72
merge-pr971
new-release-0.0.2
new-release-0.0.2-no-spacy
next
next-2-batch-crawl
next-JUN
next-MAY
next-alpine-docker
next-browser-farm
patch/generate_schema
pdf_processing
proxy-support
pull-84
release/v0.7.0
release/v0.7.1
release/v0.7.2
release/v0.7.3
release/v0.7.4
release/v0.7.5
release/v0.7.6
release/v0.7.7
release/v0.7.8
release/v0.8.0
release/v0.8.5
release/v0.8.7
release/v0.8.8
release/v0.8.9
run-many-deep-crawling
scraper-uc
scrapper
sponsors/thor_data
ssh-server
staging
unclecode-patch-1
unclecode-patch-2
unclecode-patch-3
unclecode-patch-4
unclecode-patch-5
unclecode-patch-6
unclecode-patch-7
unclecode-patch-8
unclecode/issue157
unclecode/issue167
v0.2.74
v0.2.76
v0.4.24
v0.4.241
v0.4.242
v0.4.243
v0.5.5
vr0.4.244
vr0.4.245
vr0.4.246
vr0.4.267
vr0.4.3b1
vr0.4.3b2
vr0.4.3b3
vr0.5.0.post1
vr0.5.0.post5
#1004
#1030
#1054
#1058
#1059
#1060
#1062
#1065
#1068
#1073
#1074
#1077
#1078
#108
#1081
#1083
#1085
#1085
#109
#1090
#1093
#1094
#1098
#1100
#1102
#1104
#1106
#1107
#1108
#1110
#1113
#1122
#1123
#1124
#1124
#1133
#1137
#1140
#1145
#1152
#1155
#1155
#1156
#1157
#1159
#1161
#1170
#1175
#1179
#1180
#1184
#1186
#119
#1192
#1193
#1195
#1200
#1207
#1208
#1209
#1210
#1211
#1212
#1214
#1220
#1223
#1225
#1232
#1234
#1238
#1239
#1245
#1249
#125
#1255
#1263
#1265
#1266
#1267
#1272
#1274
#128
#1281
#1282
#1285
#1289
#1289
#129
#1290
#1296
#13
#1303
#1304
#1305
#1307
#1308
#1313
#1319
#1334
#1334
#1336
#1337
#1339
#134
#135
#1351
#1356
#1358
#1361
#1364
#1366
#1368
#1369
#1371
#1372
#1373
#1376
#1378
#1381
#1383
#1384
#1386
#1387
#1388
#1389
#139
#1390
#1393
#1395
#1398
#1399
#14
#1402
#1408
#1413
#1416
#1417
#1420
#1422
#1425
#1426
#1432
#1433
#1435
#1436
#1440
#1441
#1444
#1447
#1448
#1450
#1451
#1454
#1463
#1464
#1465
#1467
#1469
#1470
#1471
#1478
#1482
#1483
#1486
#1488
#149
#1494
#1495
#1496
#1497
#1501
#1508
#1513
#1514
#1518
#1519
#1525
#1527
#1528
#1529
#1530
#1531
#1532
#1533
#1533
#1535
#1536
#1537
#1539
#1546
#1547
#1548
#1550
#1554
#1555
#1556
#1557
#1558
#1560
#1565
#1568
#1569
#1570
#1572
#1576
#158
#1580
#1588
#1589
#1590
#1592
#1595
#1596
#1597
#1598
#1599
#1600
#1605
#1607
#1609
#1612
#1613
#1617
#1617
#1619
#1620
#1622
#1623
#1624
#1628
#1630
#1633
#1637
#1640
#1641
#1643
#1645
#1648
#1650
#1653
#1655
#1661
#1662
#1667
#1668
#1674
#1676
#1677
#1681
#1683
#1685
#1689
#169
#1694
#1696
#1697
#1698
#1700
#1702
#1703
#1706
#1707
#1710
#1712
#1713
#1714
#1715
#1716
#1717
#1718
#1719
#172
#1720
#1721
#1722
#1723
#1724
#1729
#1730
#1733
#1734
#1744
#1746
#1752
#1755
#1756
#1756
#1759
#176
#1760
#1761
#1763
#1764
#1765
#1766
#1768
#1770
#1771
#1772
#1773
#1774
#1775
#1777
#1778
#1782
#1783
#1784
#1785
#1786
#1787
#1788
#1789
#1790
#1791
#1792
#1793
#1794
#1795
#1796
#1798
#1803
#1804
#1805
#1806
#1807
#1807
#1808
#1808
#1809
#1809
#1810
#1810
#1811
#1811
#1812
#1812
#1813
#1814
#1814
#1816
#1816
#1822
#1822
#1823
#1824
#1826
#1827
#1828
#1829
#1830
#1831
#1832
#1833
#1834
#1835
#1835
#1836
#1838
#1838
#1840
#1840
#1844
#1845
#1846
#1847
#1847
#1849
#1851
#1852
#1853
#1853
#1854
#1854
#1855
#1856
#1856
#1857
#1857
#1858
#1858
#1859
#1859
#1860
#1860
#1861
#1861
#1862
#1862
#1866
#1866
#1868
#1868
#1869
#1869
#1870
#1870
#1871
#1871
#1873
#1873
#1874
#1874
#1875
#1875
#1876
#1876
#1877
#1879
#1881
#1881
#1882
#1884
#1884
#1885
#1886
#1887
#1887
#1891
#1891
#1892
#1892
#1893
#1893
#1895
#1895
#1896
#1896
#1897
#1899
#1899
#1901
#1902
#1902
#1904
#1904
#1906
#1906
#1907
#1908
#1908
#1910
#1911
#1913
#1914
#1915
#1915
#1922
#1923
#1923
#1925
#1929
#1931
#1932
#1932
#1933
#1934
#1935
#1935
#1936
#1937
#1939
#194
#1940
#1941
#1941
#1943
#1944
#1944
#1946
#1946
#1947
#1951
#1952
#1953
#1955
#1955
#1957
#1957
#1960
#1965
#1965
#1967
#1969
#1970
#1970
#1971
#1975
#1976
#1977
#1977
#1978
#1979
#1981
#1983
#1983
#1984
#1984
#1985
#1985
#1986
#1986
#1987
#1987
#1988
#1988
#1989
#1990
#1991
#1991
#1993
#1993
#1994
#1994
#1995
#1995
#1997
#1997
#200
#2001
#2001
#2003
#2003
#2004
#2004
#2005
#2005
#2008
#2008
#2009
#2009
#215
#218
#229
#232
#234
#24
#249
#255
#269
#271
#279
#286
#288
#293
#294
#298
#299
#3
#300
#304
#312
#313
#314
#324
#33
#332
#335
#337
#34
#357
#358
#369
#37
#379
#387
#389
#390
#394
#403
#410
#411
#416
#419
#419
#427
#440
#444
#445
#458
#462
#465
#472
#475
#496
#510
#562
#581
#60
#605
#606
#609
#612
#617
#618
#622
#64
#640
#65
#657
#658
#66
#662
#671
#679
#680
#681
#685
#687
#706
#708
#723
#724
#729
#734
#741
#749
#75
#752
#754
#775
#776
#777
#788
#792
#799
#80
#800
#806
#808
#821
#84
#84
#846
#85
#864
#865
#868
#891
#899
#901
#903
#914
#915
#916
#918
#929
#93
#931
#945
#948
#95
#961
#967
#969
#970
#971
#973
#977
#983
#988
#988
#990
#994
#999
0.3.4
checkpoint-pre-antibot-fallback
docker-rebuild-v0.7.5
docker-rebuild-v0.7.6
docker-rebuild-v0.7.7
docker-rebuild-v0.7.8
docker-rebuild-v0.8.0
docker-rebuild-v0.8.5
docker-rebuild-v0.8.6
docker-rebuild-v0.8.7
docker-rebuild-v0.8.8
docker-rebuild-v0.8.9
v.3.72
v0.0.75
v0.1.0
v0.2.0
v0.2.1
v0.2.2
v0.2.4
v0.2.6
v0.2.7
v0.2.71
v0.2.72
v0.2.73
v0.2.74
v0.2.77
v0.3.0
v0.3.3
v0.3.6
v0.3.745
v0.3.746
v0.4.24
v0.4.243
v0.5.0.post1
v0.6.3
v0.7.0
v0.7.1
v0.7.2
v0.7.3
v0.7.4
v0.7.5
v0.7.6
v0.7.7
v0.7.8
v0.8.0
v0.8.5
v0.8.6
v0.8.7
v0.8.8
v0.8.9
vr0.6.0
vr0.6.0rc1
vr0.6.3
-
cba4a466e5
feat(browser): add BrowserProfiler class for identity-based browsing
UncleCode
2025-03-02 20:32:29 +08:00 -
7c1705712d
fix: https://github.com/unclecode/crawl4ai/issues/756
Aravind Karnam
2025-03-01 18:17:11 +05:30 -
a9e24307cc
Release prep (#749)
Aravind
2025-02-28 17:23:35 +05:30 -
3a87b4e43b
fix(dependencies): update cchardet to faust-cchardet for compatibility
UncleCode
2025-02-26 18:25:58 +08:00 -
4bcd4cbda1
refactor(pdf): improve PDF processor dependency handling
UncleCode
2025-02-25 22:27:55 +08:00 -
71ce01c9e1
feat(browser): add cdp_url parameter to BrowserManager initialization
UncleCode
2025-02-24 14:48:02 +08:00 -
c6d48080a4
feat(logger): add abstract logger base class and file logger implementation
UncleCode
2025-02-23 21:23:41 +08:00 -
46d2f12851
chore: remove old Dockerfile and server script
UncleCode
2025-02-22 13:45:04 +08:00 -
367cd71db9
feat(core): release version 0.5.0 with deep crawling and CLI
UncleCode
2025-02-21 19:55:02 +08:00 -
2af958e12c
Feat/llm config (#724)
Aravind
2025-02-21 13:11:37 +05:30 -
3cb28875c3
refactor(config): enhance serialization and config handling
UncleCode
2025-02-19 17:23:25 +08:00 -
dad592c801
2025 feb alpha 1 (#685)
Aravind
2025-02-19 11:43:17 +05:30 -
c171891999
Merge branch 'main' into next
UncleCode
2025-02-19 13:26:42 +08:00 -
3b1025abbb
Merge branch 'main' of https://github.com/unclecode/crawl4ai
UncleCode
2025-02-19 13:24:18 +08:00 -
f00dcc276f
Update README.md (#562)
UncleCode
2025-01-26 04:00:28 +01:00 -
392c923980
feat(docker): add JWT authentication and improve server architecture
UncleCode
2025-02-18 22:07:13 +08:00 -
2864015469
feat(docker): implement supervisor and secure API endpoints
UncleCode
2025-02-17 20:31:20 +08:00 -
27af4cc27b
Fix "raw://" URL parsing logic
João Martins
2025-02-15 15:34:59 +00:00 -
8bb799068e
feat(crawler): add HTTP crawler strategy for lightweight web scraping
UncleCode
2025-02-15 19:26:30 +08:00 -
063df572b0
docs(examples): add SERP API project example
UncleCode
2025-02-14 23:06:16 +08:00 -
966fb47e64
feat(config): enhance serialization and add deep crawling exports
UncleCode
2025-02-13 21:45:19 +08:00 -
43e09da694
refactor(crawler): remove content filter functionality
UncleCode
2025-02-12 21:59:19 +08:00 -
69705df0b3
fix(install): ensure proper exit after running doctor command
UncleCode
2025-02-11 19:48:23 +08:00 -
91a5fea11f
feat(cli): add command line interface with comprehensive features
UncleCode
2025-02-10 16:58:52 +08:00 -
467be9ac76
feat(deep-crawling): add DFS strategy and update exports; refactor CLI entry point
UncleCode
2025-02-09 20:23:40 +08:00 -
19df96ed56
feat(proxy): add proxy rotation strategy
UncleCode
2025-02-09 18:49:10 +08:00 -
b957ff2ecd
refactor(crawler): improve HTML handling and cleanup codebase
UncleCode
2025-02-07 21:56:27 +08:00 -
91073c1244
refactor(crawling): improve type hints and code cleanup
UncleCode
2025-02-07 19:01:59 +08:00 -
926beee832
base-config structure is changed (#618)
Sezer Bozkır
2025-02-07 12:11:51 +03:00 -
a9415aaaf6
refactor(deep-crawling): reorganize deep crawling strategies and add new implementations
UncleCode
2025-02-05 22:50:39 +08:00 -
c308a794e8
refactor(deep-crawl): reorganize deep crawling functionality into dedicated module
UncleCode
2025-02-04 23:28:17 +08:00 -
bc7559586f
feat(crawler): add deep crawling capabilities with BFS strategy
UncleCode
2025-02-04 01:24:49 +08:00 -
04bc643cec
feat(api): improve cache handling and add API tests
UncleCode
2025-02-02 20:53:31 +08:00 -
33a21d6a7a
refactor(docker): improve server architecture and configuration
UncleCode
2025-02-02 20:19:51 +08:00 -
7b1ef07c41
refactor(docker): remove unused models and utilities for cleaner codebase
UncleCode
2025-02-01 20:10:13 +08:00 -
2f15976b34
feat(docker): enhance Docker deployment setup and configuration
UncleCode
2025-02-01 19:33:27 +08:00 -
20920fa17b
refactor(docker): clean up import statements in server.py
UncleCode
2025-02-01 14:28:28 +08:00 -
53ac3ec0b4
feat(docker): add Docker service integration and config serialization
UncleCode
2025-01-31 18:00:16 +08:00 -
ce4f04dad2
feat(docker): add Docker deployment configuration and API server
UncleCode
2025-01-31 15:22:21 +08:00 -
f7ce2d42c9
feat: Add deep crawl capabilities to arun_many function
feature/scraper
Aravind Karnam
2025-01-30 17:49:58 +05:30 -
f81712eb91
refactor(core): reorganize project structure and remove legacy code
UncleCode
2025-01-30 19:35:06 +08:00 -
f6edb8342e
Refactor: remove the old deep_crawl method
Aravind Karnam
2025-01-30 16:22:41 +05:30 -
ca3f0126d3
Refactor:Moved deep_crawl_strategy, inside crawler run config
Aravind Karnam
2025-01-30 16:18:15 +05:30 -
31938fb922
feat(crawler): enhance JavaScript execution and PDF processing
UncleCode
2025-01-29 21:03:39 +08:00 -
858c18df39
fix: removed child_urls from CrawlResult
Aravind Karnam
2025-01-29 18:08:34 +05:30 -
2c8f2ec5a6
Refactor: Renamed scrape to traverse and deep_crawl in a few sections where it applies
Aravind Karnam
2025-01-29 16:24:11 +05:30 -
9ef43bc5f0
Refactor: Move adeep_crawl as method of crawler itself. Create attributes in CrawlResult to reconstruct the tree once deep crawling is completed
Aravind Karnam
2025-01-29 15:58:21 +05:30 -
84ffdaab9a
Refactor: Move adeep_crawl as method of crawler itself. Create attributes in CrawlResult to reconstruct the tree once deep crawling is completed
Aravind Karnam
2025-01-29 13:06:09 +05:30 -
78223bc847
feat: create ScraperPageResult model to attach score and depth attributes to yielded/returned crawl results
Aravind Karnam
2025-01-28 16:47:30 +05:30 -
60ce8bbf55
Merge: with v-0.4.3b
Aravind Karnam
2025-01-28 12:59:53 +05:30 -
85847ff13f
feat:
Aravind Karnam
2025-01-28 12:39:45 +05:30 -
f34b4878cf
fix: code formatting
Aravind Karnam
2025-01-28 10:00:01 +05:30 -
f8fd9d9eff
feat(pdf): add PDF processing capabilities
UncleCode
2025-01-27 21:24:15 +08:00 -
d9324e3454
fix: Move the creation of crawler outside the main loop
Aravind Karnam
2025-01-27 18:31:13 +05:30 -
0ff95c83bc
feat: change input params to scraper, Add asynchronous context manager to AsyncWebScraper, Optimise filter application
Aravind Karnam
2025-01-27 18:13:33 +05:30 -
bb6450f458
Remove robots.txt compliance from scraper
Aravind Karnam
2025-01-27 11:58:54 +05:30 -
513d008de5
feat: Merge reviews from unclecode for scorers and filters & Remove the robots.txt compliance from scraper since that will be now handled by crawler
Aravind Karnam
2025-01-27 11:54:10 +05:30 -
0f00821df5
Fix version
vr0.4.3b3
UncleCode
2025-01-26 18:08:24 +08:00 -
dde14eba7d
Update README.md (#562)
UncleCode
2025-01-26 04:00:28 +01:00 -
149b69c832
Update README.md
unclecode-patch-7
UncleCode
2025-01-26 10:59:48 +08:00 -
54c84079c4
docs(api): improve formatting and readability of API documentation
UncleCode
2025-01-25 22:06:11 +08:00 -
d0586f09a9
Merge branch 'vr0.4.3b3'
UncleCode
2025-01-25 21:57:29 +08:00 -
09ac7ed008
feat(demo): uncomment feature demos and add fake-useragent dependency
UncleCode
2025-01-25 21:56:08 +08:00 -
97796f39d2
docs(examples): update proxy rotation demo and disable other demos
UncleCode
2025-01-25 21:52:35 +08:00 -
4d7f91b378
refactor(user-agent): improve user agent generation system
UncleCode
2025-01-25 21:16:39 +08:00 -
69a77222ef
feat(browser): add CDP URL configuration support
UncleCode
2025-01-24 15:53:47 +08:00 -
0afc3e9e5e
refactor(examples): update API usage in features demo
UncleCode
2025-01-23 22:37:29 +08:00 -
65d33bcc0f
style(docs): improve code formatting in features demo
UncleCode
2025-01-23 22:36:58 +08:00 -
6a01008a2b
docs(multi-url): improve documentation clarity and update examples
UncleCode
2025-01-23 22:33:36 +08:00 -
cf3e1e748d
feat(scraper): add optimized URL scoring system
UncleCode
2025-01-23 20:46:33 +08:00 -
6dc01eae3a
refactor(core): improve type hints and remove unused file
UncleCode
2025-01-23 18:53:22 +08:00 -
7b7fe84e0d
docs(readme): resolve merge conflict and update version info
UncleCode
2025-01-22 20:52:42 +08:00 -
5c36f4308f
Merge branch 'main' of https://github.com/unclecode/crawl4ai
UncleCode
2025-01-22 20:51:52 +08:00 -
45809d1c91
Merge branch 'vr0.4.3b2'
UncleCode
2025-01-22 20:51:46 +08:00 -
357414c345
docs(readme): update version references and fix links
UncleCode
2025-01-22 20:46:39 +08:00 -
260b9120c3
docs(examples): update v0.4.3 features demo to v0.4.3b2
vr0.4.3b2
UncleCode
2025-01-22 20:41:43 +08:00 -
976ea52167
docs(examples): update demo scripts and fix output formats
UncleCode
2025-01-22 20:40:03 +08:00 -
e6ef8d91ba
refactor(scraper): optimize URL validation and filter performance
UncleCode
2025-01-22 19:45:56 +08:00 -
d21ffad3a2
chore(git): update gitignore patterns
scrapper
UncleCode
2025-01-22 17:22:26 +08:00 -
2d69bf2366
refactor(models): rename final_url to redirected_url for consistency
UncleCode
2025-01-22 17:14:24 +08:00 -
dee5fe9851
feat(proxy): add proxy rotation support and documentation
UncleCode
2025-01-22 16:11:01 +08:00 -
88697c4630
docs(readme): update version and feature announcements for v0.4.3b1
vr0.4.3b1
UncleCode
2025-01-21 21:20:04 +08:00 -
6e78c56dda
Refactor: Removed all scheduling logic from scraper. From now scraper expects arun_many to handle all scheduling. Scraper will only do traversal, validations, compliance checks, URL filtering and scoring etc. Reformatted some of the scraper files with Black code formatter
Aravind Karnam
2025-01-21 18:44:43 +05:30 -
16b8d4945b
feat(release): prepare v0.4.3 beta release
UncleCode
2025-01-21 21:03:11 +08:00 -
67fa06c09b
Refactor: Removed all scheduling logic from scraper. From now scraper expects arun_many to handle all scheduling. Scraper will only do traversal, validations, compliance checks, URL filtering and scoring etc. Reformatted some of the scraper files with Black code formatter
Aravind Karnam
2025-01-21 17:49:51 +05:30 -
d09c611d15
feat(robots): add robots.txt compliance support
UncleCode
2025-01-21 17:54:13 +08:00 -
26d78d8512
Merge branch 'next' into feature/scraper
Aravind Karnam
2025-01-21 12:35:45 +05:30 -
1079965453
refactor: Remove the URL processing logic out of scraper
Aravind Karnam
2025-01-21 12:16:59 +05:30 -
9247877037
feat(proxy): add proxy configuration support to CrawlerRunConfig
UncleCode
2025-01-20 22:14:05 +08:00 -
a677c2b61d
Merge pull request #496 from aravindkarnam/scraper-uc
Aravind
2025-01-20 16:55:41 +05:30 -
2cec527a22
feat(extraction): add LLM-powered schema generation utility
UncleCode
2025-01-20 17:28:00 +08:00 -
4b1309cbf2
feat(crawler): add URL redirection tracking
UncleCode
2025-01-19 19:53:38 +08:00 -
8b6fe6a98f
docs(api): add streaming mode documentation and examples
UncleCode
2025-01-19 18:21:34 +08:00 -
91463e34f1
feat(config): add streaming support and config cloning
UncleCode
2025-01-19 17:51:47 +08:00 -
1221be30a3
feat(browser): improve browser context management and add shared data support
UncleCode
2025-01-19 17:12:03 +08:00 -
6dfa9cb703
Streamline Feature requests, bug reports and Forums with Forms & Templates (#465)
Aravind
2025-01-19 14:23:03 +05:30 -
e363234172
feat(dispatcher): add streaming support for URL processing
UncleCode
2025-01-19 14:03:34 +08:00 -
3d09b6a221
feat(content-filter): add LLMContentFilter for intelligent markdown generation
UncleCode
2025-01-18 19:31:07 +08:00 -
2d6b19e1a2
refactor(browser): improve browser path management
UncleCode
2025-01-17 22:14:37 +08:00 -
ece9202b61
fix(dispatcher): adjust memory threshold and fix dispatcher initialization
UncleCode
2025-01-16 21:58:52 +08:00