## Summary
Once
[comfy-router#22](https://github.com/Comfy-Org/comfy-router/pull/22)
ships, `comfy.org/sitemap-index.xml` will return a unified index
aggregating both the website (38 URLs) and workflow-templates sitemaps.
This PR:
1. Reverts `Sitemap:` back to `/sitemap-index.xml` (was `/sitemap-0.xml`
in #11802 as a workaround for the 404).
2. Adds explicit allow records for 21 search and AI/LLM crawlers
(GPTBot, ChatGPT-User, OAI-SearchBot, Google-Extended, ClaudeBot,
Claude-Web, anthropic-ai, PerplexityBot, Perplexity-User,
Applebot-Extended, Bytespider, Amazonbot, CCBot, Meta-ExternalAgent,
Meta-ExternalFetcher, Diffbot, etc.).
3. Adds `Disallow:` for `/_astro/`, `/_website/`, `/_vercel/` — Vercel
build artifacts that aren't useful to crawl.
## Why granular UAs
Stacked `User-agent:` records (per [RFC 9309
§2.2](https://datatracker.ietf.org/doc/html/rfc9309#section-2.2)) share
one rule block. Listing each bot explicitly:
- Signals intent to AI bots that look for their UA in robots.txt before
crawling more aggressively.
- Surfaces our crawl policy clearly to anyone inspecting the file.
- Lets us add per-bot Disallows in future without restructuring.
## Merge order
⚠️ **Do NOT merge until comfy-router#22 is deployed to production.**
Until then, `/sitemap-index.xml` returns 404 and this PR would re-break
the issue PR #11802 patched. Verification:
```bash
curl -sI https://comfy.org/sitemap-index.xml
# expect: HTTP/2 200, x-served-by: worker-sitemap-index
```
Once that returns 200, this is safe to merge.
## Verification (after merge + deploy)
```bash
# robots.txt is served and points at the unified index
curl -s https://comfy.org/robots.txt | grep '^Sitemap:'
# → Sitemap: https://comfy.org/sitemap-index.xml
# Each AI crawler can fetch it
for ua in 'GPTBot/1.0' 'ClaudeBot/1.0' 'PerplexityBot/1.0' 'Google-Extended' 'Applebot-Extended'; do
curl -s -o /dev/null -w "$ua → %{http_code}\n" -A "$ua" https://comfy.org/robots.txt
done
# Sitemap is reachable from robots.txt
SITEMAP=$(curl -s https://comfy.org/robots.txt | awk -F': ' '/^Sitemap:/ {print $2}')
curl -s "$SITEMAP" | xmllint --noout - && echo "valid XML"
```
## Linear / closes
- Closes FE-437 (AI crawler rules)
- Updates FE-432 — the robots.txt change in #11802 was a workaround
that's no longer needed once #22 ships
┆Issue is synchronized with this [Notion
page](https://app.notion.com/p/PR-11823-feat-website-point-robots-txt-at-sitemap-index-xml-AI-crawler-rules-3546d73d3650811dbceedd06c00db444)
by [Unito](https://www.unito.io)
## Summary
<!-- One sentence describing what changed and why. -->
robots.txt at [comfy.org](https://comfy.org/) references
/sitemap-index.xml which returns 404. The actual working sitemap is at
/sitemap-0.xml (200, 38 URLs). This blocks search engines from
discovering the sitemap.
## Changes
- **What**: <!-- Core functionality added/modified -->
- Update robots.txt to point to /sitemap-0.xml, OR
## Review Focus
<!-- Critical design decisions or edge cases that need attention -->
<!-- If this PR fixes an issue, uncomment and update the line below -->
<!-- Fixes #ISSUE_NUMBER -->
## Screenshots (if applicable)
<!-- Add screenshots or video recording to help explain your changes -->
┆Issue is synchronized with this [Notion
page](https://app.notion.com/p/PR-11802-update-robots-txt-to-point-to-sitemap-0-xml-3536d73d365081bb9545eb96dd1e8025)
by [Unito](https://www.unito.io)
## Summary
Update website copy: fix branding ("Comfy" → "ComfyUI"), correct pricing
runtime, remove "coming soon" seat features, and shorten use-case label.
## Changes
- **What**: Copy corrections in `translations.ts` (branding, runtime "60
min" → "30 min" for Standard plan, remove placeholder seat features for
Creator/Pro plans); trim feature arrays in `PriceSection.vue`
## Review Focus
Verify zh-CN translations still make sense after the English copy
changes (runtime string not updated in zh-CN).
┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-11567-content-website-copy-and-pricing-updates-34b6d73d365081c29af8ee1469b08358)
by [Unito](https://www.unito.io)
---------
Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: github-actions <github-actions@github.com>
## Summary
<!-- One sentence describing what changed and why. -->
Polish and fix UI for new website
## Changes
- **What**: <!-- Core functionality added/modified -->
- [x] update about video
- [x] update Moment factory story content
- [x] update homepage visual
- [x] update customer story visual
- [x] put images and videos to bucket
## Review Focus
<!-- Critical design decisions or edge cases that need attention -->
<!-- If this PR fixes an issue, uncomment and update the line below -->
<!-- Fixes #ISSUE_NUMBER -->
## Screenshots (if applicable)
<!-- Add screenshots or video recording to help explain your changes -->
┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-11363-feat-website-Polish-and-fix-UI-3466d73d365081f895aff84b594450c9)
by [Unito](https://www.unito.io)
---------
Co-authored-by: DrJKL <DrJKL0424@gmail.com>
Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
Co-authored-by: github-actions <github-actions@github.com>
## Summary
<!-- One sentence describing what changed and why. -->
## Changes
- **What**: <!-- Core functionality added/modified -->
- **Breaking**: <!-- Any breaking changes (if none, remove this line)
-->
- **Dependencies**: <!-- New dependencies (if none, remove this line)
-->
## Review Focus
<!-- Critical design decisions or edge cases that need attention -->
<!-- If this PR fixes an issue, uncomment and update the line below -->
<!-- Fixes #ISSUE_NUMBER -->
## Screenshots (if applicable)
<!-- Add screenshots or video recording to help explain your changes -->
┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10156-feat-website-add-SEO-sitemap-redirects-CI-workflow-and-Vercel-config-3266d73d3650816ab9eaebd11072d481)
by [Unito](https://www.unito.io)
---------
Co-authored-by: GitHub Action <action@github.com>
## Summary
Adds the layout shell for the marketing site: SEO head, analytics, nav,
and footer.
## Changes (incremental from #10140)
- BaseLayout.astro: SEO meta (OG/Twitter), GTM (GTM-NP9JM6K7), Vercel
Analytics, ClientRouter, i18n
- SiteNav.vue: Fixed nav with logo, Enterprise/Gallery/About/Careers
links, COMFY CLOUD + COMFY HUB CTAs, mobile hamburger with ARIA
- SiteFooter.vue: Product/Resources/Company/Legal columns, social icons
## Stack (via Graphite)
- #10140 [1/3] Scaffold ← merge first
- **[2/3] Layout Shell** ← this PR
- #10142 [3/3] Homepage Sections
┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10141-feat-add-layout-shell-BaseLayout-SiteNav-SiteFooter-2-3-3266d73d365081aeb2d7e598943a8e17)
by [Unito](https://www.unito.io)
## Summary
Scaffolds the new apps/website/ Astro 5 + Vue 3 marketing site inside
the monorepo.
## Changes
- apps/website/ with package.json, astro.config.mjs, tsconfig, Nx
targets
- @comfyorg/design-system/css/base.css — brand tokens + fonts (no
PrimeVue)
- pnpm-workspace.yaml catalog entries for Astro deps
- .gitignore and env.d.ts for Astro
## Stack (via Graphite)
- **[1/3] Scaffold** ← this PR
- #10141 [2/3] Layout Shell
- #10142 [3/3] Homepage Sections
Part of the comfy.org website refresh (replacing Framer).
┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10140-feat-scaffold-Astro-5-website-app-design-system-base-css-1-3-3266d73d365081688dcee0220a03eca4)
by [Unito](https://www.unito.io)
