[feat] Add CI checks for OSS license compliance and telemetry tree-shaking

Implements automated verification to ensure the OSS distribution:
1. Only includes open-source licensed dependencies
2. Properly tree-shakes proprietary fonts (ABCROM)
3. Removes telemetry code (Mixpanel) from OSS builds

New scripts:
- scripts/verify-licenses.js - Validates production dependency licenses
- scripts/verify-oss-build.js - Checks dist/ for violations

New CI workflow:
- .github/workflows/ci-oss-compliance.yaml - Runs compliance checks

New npm scripts:
- pnpm verify:licenses - Check dependency licenses
- pnpm verify:oss - Verify OSS build compliance
- pnpm verify:compliance - Run all checks
- pnpm build:oss - Build OSS distribution

Documentation:
- docs/OSS_COMPLIANCE.md - Complete guide for compliance checks

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
snomiao
2025-11-20 06:12:36 +00:00
parent 87d6d18c57
commit 8dcdcfce5b
5 changed files with 750 additions and 0 deletions

View File

@@ -0,0 +1,77 @@
name: "CI: OSS Compliance"
description: "Verify OSS build compliance (license and telemetry checks)"
on:
push:
branches: [main, master, dev*, core/*, desktop/*]
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
license-check:
name: License Compliance
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- name: Install pnpm
uses: pnpm/action-setup@v4
with:
version: 10
- name: Use Node.js
uses: actions/setup-node@v4
with:
node-version: "lts/*"
cache: "pnpm"
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Verify production dependency licenses
run: node scripts/verify-licenses.js
oss-build-check:
name: OSS Build Verification
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- name: Install pnpm
uses: pnpm/action-setup@v4
with:
version: 10
- name: Use Node.js
uses: actions/setup-node@v4
with:
node-version: "lts/*"
cache: "pnpm"
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Build OSS distribution
run: DISTRIBUTION=localhost pnpm build
env:
# Ensure we're building the OSS version
DISTRIBUTION: localhost
# Disable source maps for faster build
GENERATE_SOURCEMAP: false
- name: Verify OSS build compliance
run: node scripts/verify-oss-build.js
- name: Upload build artifacts for inspection (on failure)
if: failure()
uses: actions/upload-artifact@v4
with:
name: oss-build-artifacts
path: dist/
retention-days: 7

194
docs/OSS_COMPLIANCE.md Normal file
View File

@@ -0,0 +1,194 @@
# OSS Compliance Verification
This document describes the automated compliance checks that ensure the OSS (Open Source Software) distribution of ComfyUI Frontend meets licensing and privacy requirements.
## Overview
The OSS build verification system consists of two main components:
1. **License Compliance Check** - Ensures all production dependencies use approved open-source licenses
2. **OSS Build Verification** - Ensures the OSS distribution doesn't contain proprietary code or telemetry
## Quick Start
### Run All Compliance Checks
```bash
pnpm verify:compliance
```
This command will:
1. Check all production dependency licenses
2. Build the OSS distribution
3. Verify the build output doesn't contain violations
### Individual Checks
```bash
# Check licenses only
pnpm verify:licenses
# Build OSS distribution
pnpm build:oss
# Verify OSS build (requires build first)
pnpm verify:oss
```
## License Compliance
### Purpose
Verifies that all production dependencies use licenses compatible with ComfyUI's GPL-3.0-only license.
### Script Location
`scripts/verify-licenses.js`
### Approved Licenses
The following licenses are approved for use:
- **Permissive**: MIT, Apache-2.0, BSD-2-Clause, BSD-3-Clause, ISC
- **Copyleft**: GPL-2.0, GPL-3.0, LGPL-2.1, LGPL-3.0, MPL-2.0
- **Public Domain**: CC0-1.0, Unlicense, WTFPL
- And other OSI-approved licenses
### How It Works
1. Runs `pnpm licenses list --json --prod` to get all production dependencies
2. Checks each license against the approved list
3. Flags any non-compliant or unknown licenses
4. Exits with error code 1 if violations are found
### Adding New Approved Licenses
If a legitimate open-source license is being flagged, edit `scripts/verify-licenses.js` and add it to the `APPROVED_LICENSES` set.
## OSS Build Verification
### Purpose
Ensures the OSS distribution (DISTRIBUTION=localhost) doesn't contain:
1. **Proprietary licensed assets** (e.g., ABCROM font files)
2. **Telemetry code** (e.g., Mixpanel tracking)
### Script Location
`scripts/verify-oss-build.js`
### What Gets Checked
#### Proprietary Font Files
- Searches for `.woff`, `.woff2`, `.ttf`, `.otf` files containing "ABCROM"
- These fonts are proprietary and licensed only for cloud distribution
#### Telemetry Code
Searches JavaScript files for:
- `mixpanel` references
- `MixpanelTelemetryProvider` class
- Tracking method calls (`trackWorkflow`, `trackEvent`)
- Mixpanel API endpoints (`mp.comfy.org`)
### How It Works
1. Recursively scans the `dist/` directory
2. Checks font files by filename
3. Checks JavaScript files for telemetry code patterns
4. Reports all violations with file locations and matches
5. Exits with error code 1 if violations are found
### Tree-Shaking Mechanism
The codebase uses compile-time constants for tree-shaking:
```typescript
// src/platform/distribution/types.ts
const DISTRIBUTION: Distribution = __DISTRIBUTION__
export const isCloud = DISTRIBUTION === 'cloud'
// src/platform/telemetry/index.ts
if (isCloud) {
_telemetryProvider = new MixpanelTelemetryProvider()
}
```
When building with `DISTRIBUTION=localhost`:
- `isCloud` evaluates to `false`
- Dead code elimination removes all cloud-specific code
- Mixpanel library is never imported or bundled
## CI Integration
### GitHub Actions Workflow
`.github/workflows/ci-oss-compliance.yaml`
The workflow runs on all pushes to main/dev branches and pull requests:
1. **license-check** job
- Installs dependencies
- Runs license verification
2. **oss-build-check** job
- Installs dependencies
- Builds OSS distribution
- Runs build verification
- Uploads artifacts on failure for debugging
### When Checks Run
- On push to: `main`, `master`, `dev*`, `core/*`, `desktop/*`
- On pull requests (except `wip/*`, `draft/*`, `temp/*`)
## Troubleshooting
### License Check Fails
1. Review the flagged packages
2. Check if the license is genuinely non-compliant
3. If it's a false positive, add the license to `APPROVED_LICENSES`
4. If it's truly non-compliant, find an alternative package
### OSS Build Check Fails
1. Review the violations in the output
2. Check if cloud-specific code is being included
3. Verify tree-shaking is working:
- Check `vite.config.mts` for `define` configuration
- Ensure `DISTRIBUTION` is set correctly
- Check that cloud imports are conditionally loaded
### Build Artifacts
If the OSS build check fails in CI, artifacts are uploaded for 7 days:
1. Go to the failed workflow run
2. Download "oss-build-artifacts"
3. Inspect the files to identify violations
## Adding New Cloud-Specific Code
When adding code that should only be in cloud builds:
1. **Place it in `src/platform/cloud/`** - Recommended approach
2. **Use conditional imports**:
```typescript
if (isCloud) {
const { CloudFeature } = await import('./cloud/CloudFeature')
// Use CloudFeature
}
```
3. **Test locally**:
```bash
pnpm build:oss
pnpm verify:oss
```
## References
- [Vite Tree-Shaking](https://vitejs.dev/guide/features.html#build-optimizations)
- [GPL-3.0 License](https://www.gnu.org/licenses/gpl-3.0.en.html)
- [OSI Approved Licenses](https://opensource.org/licenses/)

View File

@@ -12,7 +12,11 @@
"build-storybook": "storybook build",
"build:types": "nx build --config vite.types.config.mts && node scripts/prepare-types.js",
"build:analyze": "cross-env ANALYZE_BUNDLE=true pnpm build",
"build:oss": "cross-env DISTRIBUTION=localhost GENERATE_SOURCEMAP=false pnpm build",
"build": "cross-env NODE_OPTIONS='--max-old-space-size=8192' pnpm typecheck && nx build",
"verify:licenses": "node scripts/verify-licenses.js",
"verify:oss": "node scripts/verify-oss-build.js",
"verify:compliance": "pnpm verify:licenses && pnpm build:oss && pnpm verify:oss",
"size:collect": "node scripts/size-collect.js",
"size:report": "node scripts/size-report.js",
"collect-i18n": "pnpm exec playwright test --config=playwright.i18n.config.ts",

225
scripts/verify-licenses.js Normal file
View File

@@ -0,0 +1,225 @@
/**
* CI Script: Verify License Compliance
*
* This script verifies that all production dependencies use open-source compatible licenses.
* It checks against a list of approved licenses and flags any non-compliant dependencies.
*
* Usage: node scripts/verify-licenses.js
*
* Exit codes:
* - 0: All licenses are compliant
* - 1: Non-compliant licenses found
*/
import { execSync } from 'child_process'
const COLORS = {
red: '\x1b[31m',
green: '\x1b[32m',
yellow: '\x1b[33m',
blue: '\x1b[34m',
reset: '\x1b[0m'
}
// Approved open-source licenses
// Based on OSI-approved licenses and common permissive licenses
const APPROVED_LICENSES = new Set([
'MIT',
'Apache-2.0',
'BSD-2-Clause',
'BSD-3-Clause',
'ISC',
'CC0-1.0',
'CC-BY-3.0',
'CC-BY-4.0',
'Unlicense',
'WTFPL',
'0BSD',
'BlueOak-1.0.0',
'Python-2.0',
'Zlib',
// GPL is acceptable for libraries as long as we're GPL-3.0-only
'GPL-2.0',
'GPL-3.0',
'GPL-3.0-only',
'LGPL-2.1',
'LGPL-3.0',
'MPL-2.0',
// Public domain
'Public Domain',
'Unlicensed'
])
// Known problematic licenses
const PROBLEMATIC_LICENSES = new Set([
'UNLICENSED',
'CUSTOM',
'SEE LICENSE IN LICENSE',
'PROPRIETARY'
])
/**
* Parse pnpm licenses output
*/
function getLicenses() {
console.log(
`${COLORS.blue}Fetching production dependency licenses...${COLORS.reset}`
)
try {
const output = execSync('pnpm licenses list --json --prod', {
encoding: 'utf-8',
maxBuffer: 10 * 1024 * 1024 // 10MB buffer
})
const licenses = JSON.parse(output)
return licenses
} catch (err) {
console.error(
`${COLORS.red}Error fetching licenses: ${err.message}${COLORS.reset}`
)
process.exit(1)
}
}
/**
* Normalize license names for comparison
*/
function normalizeLicense(license) {
if (!license) return 'UNKNOWN'
// Handle common variations
const normalized = license.trim().replace(/\s+/g, '-').toUpperCase()
// Handle "OR" clauses - take the first license
if (normalized.includes(' OR ')) {
return normalized.split(' OR ')[0].trim()
}
// Handle "AND" clauses - if any license is approved, consider it approved
if (normalized.includes(' AND ')) {
const licenses = normalized.split(' AND ')
for (const lic of licenses) {
const trimmed = lic.trim()
if (APPROVED_LICENSES.has(trimmed)) {
return trimmed
}
}
}
return normalized
}
/**
* Check if a license is approved
*/
function isLicenseApproved(license) {
const normalized = normalizeLicense(license)
// Check exact match
if (APPROVED_LICENSES.has(normalized)) {
return true
}
// Check if any approved license is a substring (handles variations)
for (const approved of APPROVED_LICENSES) {
if (normalized.includes(approved.toUpperCase())) {
return true
}
}
return false
}
/**
* Main verification function
*/
function main() {
console.log(
`${COLORS.blue}========================================${COLORS.reset}`
)
console.log(`${COLORS.blue}License Compliance Verification${COLORS.reset}`)
console.log(
`${COLORS.blue}========================================${COLORS.reset}\n`
)
const licenses = getLicenses()
const violations = []
const warnings = []
let totalPackages = 0
// Check each license group
for (const [license, packages] of Object.entries(licenses)) {
for (const pkg of packages) {
totalPackages++
const isApproved = isLicenseApproved(license)
const isProblematic = PROBLEMATIC_LICENSES.has(normalizeLicense(license))
if (isProblematic || !isApproved) {
violations.push({
package: pkg.name,
version: pkg.versions[0],
license: license,
isProblematic
})
} else if (license === 'UNKNOWN' || !license) {
warnings.push({
package: pkg.name,
version: pkg.versions[0],
license: 'UNKNOWN'
})
}
}
}
// Report warnings
if (warnings.length > 0) {
console.log(
`${COLORS.yellow}⚠ Packages with unknown licenses (${warnings.length}):${COLORS.reset}`
)
warnings.forEach(({ package: name, version }) => {
console.log(` ${COLORS.yellow}- ${name}@${version}${COLORS.reset}`)
})
console.log()
}
// Report violations
if (violations.length > 0) {
console.log(
`${COLORS.red}✗ Found ${violations.length} package(s) with non-compliant licenses:${COLORS.reset}\n`
)
violations.forEach(({ package: name, version, license, isProblematic }) => {
console.log(` ${COLORS.red}Package: ${name}@${version}${COLORS.reset}`)
console.log(` ${COLORS.red}License: ${license}${COLORS.reset}`)
if (isProblematic) {
console.log(
` ${COLORS.red}⚠ This license is known to be problematic${COLORS.reset}`
)
}
console.log()
})
console.log(
`${COLORS.blue}========================================${COLORS.reset}`
)
console.log(`${COLORS.red}✗ License verification failed!${COLORS.reset}`)
console.log(
`${COLORS.red}Please review and update dependencies with non-compliant licenses.${COLORS.reset}\n`
)
process.exit(1)
}
// Success
console.log(
`${COLORS.blue}========================================${COLORS.reset}`
)
console.log(
`${COLORS.green}✓ All ${totalPackages} production dependencies use approved licenses!${COLORS.reset}\n`
)
process.exit(0)
}
main()

250
scripts/verify-oss-build.js Normal file
View File

@@ -0,0 +1,250 @@
/**
* CI Script: Verify OSS Build Compliance
*
* This script verifies that the OSS build (DISTRIBUTION=localhost) does not contain:
* 1. Proprietary licensed files (e.g., ABCROM font)
* 2. Telemetry code (e.g., mixpanel library references)
*
* Usage: node scripts/verify-oss-build.js
*
* Exit codes:
* - 0: All checks passed
* - 1: Violations found
*/
import { readFileSync, readdirSync, statSync } from 'fs'
import { join, extname, dirname } from 'path'
import { fileURLToPath } from 'url'
const __filename = fileURLToPath(import.meta.url)
const __dirname = dirname(__filename)
const DIST_DIR = join(__dirname, '..', 'dist')
const COLORS = {
red: '\x1b[31m',
green: '\x1b[32m',
yellow: '\x1b[33m',
blue: '\x1b[34m',
reset: '\x1b[0m'
}
// Patterns to check for violations
const VIOLATION_PATTERNS = {
// Proprietary font checks
font: {
patterns: [/ABCROM/gi, /ABCROMExtended/gi, /ABC\s*ROM/gi],
description: 'ABCROM proprietary font references'
},
// Telemetry checks
telemetry: {
patterns: [
/mixpanel/gi,
/MixpanelTelemetryProvider/gi,
/mp\.comfy\.org/gi,
/mixpanel-browser/gi,
/trackWorkflow/g,
/trackEvent/g,
/\.track\s*\(/g
],
description: 'Mixpanel telemetry code'
}
}
// File extensions to check
const JS_EXTENSIONS = ['.js', '.mjs', '.cjs']
const FONT_EXTENSIONS = ['.woff', '.woff2', '.ttf', '.otf']
/**
* Recursively get all files in a directory
*/
function getAllFiles(dir, extensions = null) {
const files = []
try {
const items = readdirSync(dir)
for (const item of items) {
const fullPath = join(dir, item)
const stat = statSync(fullPath)
if (stat.isDirectory()) {
files.push(...getAllFiles(fullPath, extensions))
} else if (stat.isFile()) {
if (!extensions || extensions.includes(extname(fullPath))) {
files.push(fullPath)
}
}
}
} catch (err) {
console.error(
`${COLORS.red}Error reading directory ${dir}: ${err.message}${COLORS.reset}`
)
}
return files
}
/**
* Check if file content contains violation patterns
*/
function checkFileForViolations(filePath, violationConfig) {
try {
const content = readFileSync(filePath, 'utf-8')
const violations = []
for (const pattern of violationConfig.patterns) {
const matches = content.match(pattern)
if (matches && matches.length > 0) {
violations.push({
pattern: pattern.toString(),
matches: matches.length,
sample: matches[0]
})
}
}
return violations
} catch (err) {
// Binary files or read errors - skip
return []
}
}
/**
* Check for proprietary font files
*/
function checkForFontFiles() {
console.log(
`\n${COLORS.blue}Checking for proprietary font files...${COLORS.reset}`
)
const fontFiles = getAllFiles(DIST_DIR, FONT_EXTENSIONS)
const violations = []
for (const fontFile of fontFiles) {
const fileName = fontFile.toLowerCase()
if (fileName.includes('abcrom')) {
violations.push(fontFile)
}
}
if (violations.length > 0) {
console.log(
`${COLORS.red}✗ Found ${violations.length} proprietary font file(s):${COLORS.reset}`
)
violations.forEach((file) => {
console.log(` ${COLORS.red}- ${file}${COLORS.reset}`)
})
return false
} else {
console.log(
`${COLORS.green}✓ No proprietary font files found${COLORS.reset}`
)
return true
}
}
/**
* Check JavaScript files for code violations
*/
function checkJavaScriptFiles() {
console.log(
`\n${COLORS.blue}Checking JavaScript files for code violations...${COLORS.reset}`
)
const jsFiles = getAllFiles(DIST_DIR, JS_EXTENSIONS)
const allViolations = {}
for (const [violationType, config] of Object.entries(VIOLATION_PATTERNS)) {
allViolations[violationType] = []
for (const jsFile of jsFiles) {
const violations = checkFileForViolations(jsFile, config)
if (violations.length > 0) {
allViolations[violationType].push({
file: jsFile,
violations
})
}
}
}
let hasViolations = false
for (const [violationType, config] of Object.entries(VIOLATION_PATTERNS)) {
const violations = allViolations[violationType]
if (violations.length > 0) {
hasViolations = true
console.log(
`\n${COLORS.red}✗ Found ${config.description} in ${violations.length} file(s):${COLORS.reset}`
)
violations.forEach(({ file, violations: fileViolations }) => {
console.log(`\n ${COLORS.yellow}${file}${COLORS.reset}`)
fileViolations.forEach(({ pattern, matches, sample }) => {
console.log(` ${COLORS.red}Pattern: ${pattern}${COLORS.reset}`)
console.log(` ${COLORS.red}Matches: ${matches}${COLORS.reset}`)
console.log(` ${COLORS.red}Sample: "${sample}"${COLORS.reset}`)
})
})
} else {
console.log(
`${COLORS.green}✓ No ${config.description} found${COLORS.reset}`
)
}
}
return !hasViolations
}
/**
* Main verification function
*/
function main() {
console.log(
`${COLORS.blue}========================================${COLORS.reset}`
)
console.log(`${COLORS.blue}OSS Build Verification${COLORS.reset}`)
console.log(
`${COLORS.blue}========================================${COLORS.reset}`
)
console.log(`${COLORS.blue}Checking: ${DIST_DIR}${COLORS.reset}`)
// Check if dist directory exists
try {
statSync(DIST_DIR)
} catch (err) {
console.error(
`\n${COLORS.red}Error: dist/ directory not found. Please run 'pnpm build' first.${COLORS.reset}`
)
process.exit(1)
}
// Run checks
const fontCheckPassed = checkForFontFiles()
const codeCheckPassed = checkJavaScriptFiles()
// Summary
console.log(
`\n${COLORS.blue}========================================${COLORS.reset}`
)
console.log(`${COLORS.blue}Verification Summary${COLORS.reset}`)
console.log(
`${COLORS.blue}========================================${COLORS.reset}`
)
if (fontCheckPassed && codeCheckPassed) {
console.log(
`${COLORS.green}✓ All checks passed! OSS build is compliant.${COLORS.reset}\n`
)
process.exit(0)
} else {
console.log(
`${COLORS.red}✗ Verification failed! Please fix the violations above.${COLORS.reset}\n`
)
process.exit(1)
}
}
main()