Compare commits
11
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c57a90ccab | ||
|
|
03fe11c63d | ||
|
|
02336e6a0a | ||
|
|
d4d259cd23 | ||
|
|
175ed5c4e2 | ||
|
|
dc83f97be4 | ||
|
|
7f7d7588f3 | ||
|
|
b9c6d193a4 | ||
|
|
062adaf9d3 | ||
|
|
1cf4074eb3 | ||
|
|
bc6f087efb |
No files matched your search
@@ -0,0 +1,223 @@
|
||||
---
|
||||
name: garbage-collector
|
||||
description: Daily maintenance agent that reads pre-scanned gc-findings.json, then opens Tier A draft PRs (mechanical fixes like removing stale TODOs and console.logs) and Tier B GitHub Issues (architectural drift, oversized files, outdated deps). Runs headless via the garbage-collector.yml cron. Also invokable interactively as @garbage-collector for manual sweeps.
|
||||
tools: Read, Grep, Glob, Bash, Edit, Write
|
||||
model: haiku
|
||||
color: orange
|
||||
---
|
||||
|
||||
You are the WorldWideView Garbage Collector — a low-cost maintenance agent responsible for keeping the codebase clean between human-led development cycles.
|
||||
|
||||
## Step 1 — Read your charter
|
||||
|
||||
```bash
|
||||
cat .agents/rules/garbage-collection.md
|
||||
```
|
||||
|
||||
Internalize all guard rails, caps, and the never-touch list before taking any action.
|
||||
|
||||
## Step 2 — Read the findings
|
||||
|
||||
```bash
|
||||
cat gc-findings.json
|
||||
```
|
||||
|
||||
This was produced by `scripts/gc-scan.mjs` before you were invoked. You do **not** re-scan the codebase — you triage this list only.
|
||||
|
||||
If `gc-findings.json` does not exist or is empty, output "No findings to process." and stop.
|
||||
|
||||
## Step 3 — Check dry-run mode
|
||||
|
||||
```bash
|
||||
echo "${DRY_RUN:-false}"
|
||||
```
|
||||
|
||||
If `true`, jump to **Step 7 (Dry-run summary)** and skip all PR/Issue creation.
|
||||
|
||||
## Step 4 — Check for existing GC work (idempotency)
|
||||
|
||||
```bash
|
||||
# Open gc-bot PRs
|
||||
gh pr list --label gc-bot --state open --json number,title,headRefName,body
|
||||
|
||||
# Existing gc/* remote branches
|
||||
git branch -r --list 'origin/gc/*'
|
||||
|
||||
# Open gc-bot Issues
|
||||
gh issue list --label gc-bot --state open --json number,title
|
||||
```
|
||||
|
||||
Build a mental map of already-handled concerns. For each finding, skip it if an open PR or Issue already covers the same file + pattern.
|
||||
|
||||
## Step 5 — Tier A: Create draft PRs (max 3 per run)
|
||||
|
||||
Group related Tier A findings by `patternId` or `type`. Process the highest-value group first (most findings → most cleanup per PR).
|
||||
|
||||
For each group (up to 3 total):
|
||||
|
||||
### 5a — Create a branch
|
||||
|
||||
```bash
|
||||
SHA=$(git rev-parse --short HEAD)
|
||||
BRANCH="gc/<patternId>-${SHA}"
|
||||
git checkout -b "${BRANCH}"
|
||||
```
|
||||
|
||||
### 5b — Apply the mechanical fix
|
||||
|
||||
Follow these per-type rules:
|
||||
|
||||
**stale-todo (issueClosed: true)**
|
||||
Delete the entire comment line. Do not change surrounding code.
|
||||
|
||||
**stale-todo (issueClosed: false, age > 90d)**
|
||||
Add `# STALE(gc-bot):` prefix to the comment so the human sees it needs resolution. Do not delete — the human decides. This makes it Tier A-lite (no code risk, just marks it).
|
||||
|
||||
**anti-pattern: mdc-ref**
|
||||
Replace `.mdc` with `.md` in the file content. Check that the referenced file actually exists with `.md` extension before renaming.
|
||||
|
||||
**anti-pattern: console-log**
|
||||
Delete the `console.log(...)` line entirely. If it's the only statement in a block, delete the block too (only if the block has no other statements).
|
||||
|
||||
**anti-pattern: ts-ignore**
|
||||
Delete the `// @ts-ignore` line. If the next line now has a type error that is trivially fixable (e.g., adding a missing cast), fix it. If the type error is non-trivial, revert to Tier B (create an Issue instead).
|
||||
|
||||
**anti-pattern: ts-nocheck**
|
||||
Delete the `// @ts-nocheck` line. Check if the file still compiles cleanly with `npx tsc --noEmit --skipLibCheck` — if not, revert the deletion and file a Tier B Issue instead.
|
||||
|
||||
### 5c — Verify the change is safe
|
||||
|
||||
```bash
|
||||
# Count changed lines — must be ≤150
|
||||
git diff --stat
|
||||
|
||||
# Quick type check on changed files only
|
||||
npx tsc --noEmit --skipLibCheck 2>&1 | head -20
|
||||
```
|
||||
|
||||
If changed lines exceed 150 or type errors are introduced, `git checkout .` and file a Tier B Issue for this group instead.
|
||||
|
||||
### 5d — Commit and push
|
||||
|
||||
```bash
|
||||
git add -p # stage only the GC changes
|
||||
git commit -m "chore: [gc] remove <N> stale <type> findings"
|
||||
git push origin "${BRANCH}"
|
||||
```
|
||||
|
||||
### 5e — Open a draft PR
|
||||
|
||||
```bash
|
||||
gh pr create --draft --label gc-bot \
|
||||
--title "chore: [gc] <short description (≤60 chars)>" \
|
||||
--body "$(cat <<'EOF'
|
||||
## What changed
|
||||
<bullet list of files and what was removed/changed>
|
||||
|
||||
## Why
|
||||
Detector: `<patternId>` in `scripts/gc-scan.mjs`
|
||||
Rule violated: `<rule from garbage-collection.md>`
|
||||
Age / evidence: <age in days, or closed issue number>
|
||||
|
||||
## How to verify
|
||||
1. <step>
|
||||
2. Run `pnpm test` — all tests should still pass
|
||||
3. Run `pnpm build` — build must be clean
|
||||
|
||||
> This PR was opened automatically by the GC bot. Review before merging.
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
Return to main: `git checkout main`
|
||||
|
||||
## Step 6 — Tier B: Create GitHub Issues (max 5 per run)
|
||||
|
||||
Group related Tier B findings into one Issue per `type` (e.g., all oversized files in one Issue).
|
||||
|
||||
```bash
|
||||
gh issue create --label gc-bot \
|
||||
--title "[gc] <type>: <short description>" \
|
||||
--body "$(cat <<'EOF'
|
||||
## Summary
|
||||
<what was detected and why it matters>
|
||||
|
||||
## Files / packages affected
|
||||
<file list with line counts or versions>
|
||||
|
||||
## Suggested action
|
||||
<concrete next step — e.g., "split this file", "upgrade this dep", "fix this rule reference">
|
||||
|
||||
## How to close this
|
||||
Resolve the issue and close this ticket manually.
|
||||
|
||||
> Filed automatically by the GC bot on <date>.
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
## Step 7 — Dry-run summary
|
||||
|
||||
Find or create the tracking Issue:
|
||||
|
||||
```bash
|
||||
gh issue list --label gc-bot --state open --search "[gc-bot] Daily Scan Reports" --json number,title
|
||||
```
|
||||
|
||||
If it does not exist:
|
||||
```bash
|
||||
gh issue create --label gc-bot \
|
||||
--title "[gc-bot] Daily Scan Reports" \
|
||||
--body "This issue collects daily dry-run scan summaries. Enable normal mode by setting DRY_RUN=false in the workflow."
|
||||
```
|
||||
|
||||
Then post a comment:
|
||||
```bash
|
||||
gh issue comment <number> --body "$(cat <<'EOF'
|
||||
## GC Dry-run Report — <date>
|
||||
|
||||
| Tier | Count | Types |
|
||||
|---|---|---|
|
||||
| A (would fix) | <n> | <comma-separated types> |
|
||||
| B (would file) | <n> | <comma-separated types> |
|
||||
|
||||
### Tier A findings
|
||||
<table: file | line | type | description>
|
||||
|
||||
### Tier B findings
|
||||
<table: file/package | type | description>
|
||||
|
||||
No PRs or Issues were created (dry-run mode).
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
## Step 8 — Output summary
|
||||
|
||||
Print a clean markdown summary:
|
||||
|
||||
```
|
||||
## GC Run Complete
|
||||
|
||||
### PRs opened (<n>/3)
|
||||
- #<number>: <title> (<branch>)
|
||||
|
||||
### Issues opened (<n>/5)
|
||||
- #<number>: <title>
|
||||
|
||||
### Skipped (already handled)
|
||||
- <n> findings skipped — covered by existing open work
|
||||
|
||||
### Not actioned
|
||||
- <n> Tier B findings below cap — will appear in next run
|
||||
```
|
||||
|
||||
If nothing was created: "No new findings requiring action."
|
||||
|
||||
## Important invariants
|
||||
|
||||
- Never touch the never-touch list in the charter.
|
||||
- Never create a PR with more than 150 changed lines.
|
||||
- Never mark a PR as ready-for-review.
|
||||
- Never push directly to `main`.
|
||||
- If a Tier A fix risks introducing a regression and you cannot verify it is safe, file a Tier B Issue instead — safety always beats throughput.
|
||||
@@ -8,6 +8,9 @@ paths:
|
||||
- "src/core/tenant.ts"
|
||||
---
|
||||
|
||||
> **SUPERSEDED by ADR-003** (`docs/architecture/decisions/adr-0003-shared-identity-and-ecosystem-auth-host.md`).
|
||||
> The auth host is `worldwideview.dev` (worldwideview-web repo), not `app.worldwideview.dev`. Read ADR-003 first.
|
||||
|
||||
# WorldWideView — Cloud & Auth Architecture
|
||||
|
||||
|
||||
@@ -19,6 +22,16 @@ paths:
|
||||
|
||||
## Auth Strategy
|
||||
|
||||
> [!WARNING]
|
||||
> **This section is superseded by [ADR-003](../../docs/architecture/decisions/adr-0003-shared-identity-and-ecosystem-auth-host.md) (2026-05-22).**
|
||||
> The decisions below describing `app.worldwideview.dev` as the auth owner and NextAuth as the ecosystem IdP are no longer current. The new architecture is:
|
||||
> - **`worldwideview.dev`** (the `worldwideview-web` repo) owns all login, signup, and account management UI
|
||||
> - **Supabase Auth** (single shared project) is the identity provider for all products
|
||||
> - Session cookies are scoped to `.worldwideview.dev` and shared across all subdomains
|
||||
> - The Marketplace never owns auth UI and never redirects to `app.worldwideview.dev`
|
||||
>
|
||||
> The multi-tenant RLS, tier matrix, license key, and CI/CD sections below remain valid.
|
||||
|
||||
**Auth.js (NextAuth)** is the single auth API across all editions. `app.worldwideview.dev` owns **all auth UI** — login, registration, password reset. No other subdomain ever shows a login or registration form.
|
||||
|
||||
```typescript
|
||||
|
||||
@@ -13,10 +13,22 @@ When making database schema changes, you must ensure that generated migrations a
|
||||
- **Add "IF EXISTS" clauses**: Prisma can sometimes produce brittle drop statements (`DROP INDEX "my_index"`). To prevent migrations from crashing if a state desynchronization occurs, manually edit the migration file to use `DROP INDEX IF EXISTS "my_index";` or `DROP TABLE IF EXISTS "table_name";`.
|
||||
- **Data Preservation**: When dropping columns or renaming, carefully review the migration to ensure data is correctly mapped. In PostgreSQL, ensure that cascaded drops don't accidentally remove required data.
|
||||
|
||||
## 2. Managing Failed Migrations
|
||||
## 2. P3005 Auto-Baseline (Handled in `docker-entrypoint.sh`)
|
||||
|
||||
`docker-entrypoint.sh` automatically recovers from Prisma **P3005** ("schema not empty, no migration history"). This happens when `pnpm dev` (which runs `prisma db push`) populates the database before the Docker stack is started.
|
||||
|
||||
The entrypoint only auto-baselines when **both** conditions hold:
|
||||
1. The error is exactly P3005.
|
||||
2. `prisma migrate diff` reports zero schema drift between the live DB and `prisma/schema.prisma`.
|
||||
|
||||
If drift is detected the container exits with a clear error message directing you to `pnpm db:reset`.
|
||||
|
||||
> **Never add manual `migrate resolve --applied` calls** unless you are outside the container and have independently verified the above two conditions. The entrypoint already handles the common case.
|
||||
|
||||
## 3. Managing Failed Migrations
|
||||
If a migration fails in production:
|
||||
- **Never blindly mark as resolved**: Do not run `prisma migrate resolve --applied <migration_name>` unless you have explicitly verified that the database schema precisely matches the expected state.
|
||||
- If you use `resolve --applied`, Prisma will permanently skip that migration in the future, leaving your database structurally out of sync with your Prisma Client.
|
||||
|
||||
## 3. Backups Before DDL
|
||||
## 4. Backups Before DDL
|
||||
Never perform direct DDL operations (ALTER, DROP, CREATE) on a production database without first confirming an automated backup exists, or manually taking a `pg_dump` backup of the database.
|
||||
@@ -0,0 +1,74 @@
|
||||
# Garbage Collection Policy
|
||||
|
||||
## Purpose
|
||||
|
||||
The Garbage Collector (GC) is a daily Haiku-model agent that keeps the codebase clean. It detects stale TODOs, deprecated patterns, and architectural drift, then either applies a small mechanical fix (Tier A → draft PR) or files a report for human decision (Tier B → GitHub Issue). The GC **never makes architectural decisions** and **never auto-merges** anything.
|
||||
|
||||
## Two-tier output model
|
||||
|
||||
| Tier | Criteria | Output |
|
||||
|---|---|---|
|
||||
| **A — Mechanical** | Safe, codemod-like, no architectural judgement required | Draft PR with code change |
|
||||
| **B — Judgement** | Requires human architectural decision or context | GitHub Issue, no code |
|
||||
|
||||
When unsure of the tier, escalate to Tier B. Never attempt a refactor that requires understanding business intent.
|
||||
|
||||
## How the GC works
|
||||
|
||||
1. `scripts/gc-scan.mjs` runs first — deterministic shell-based detectors, zero cost, emits `gc-findings.json`.
|
||||
2. The Haiku agent reads `gc-findings.json` and the policy below. It **does not free-scan the codebase** — it only triages the pre-computed findings list.
|
||||
3. The agent creates draft PRs and Issues per the caps and guard rails below.
|
||||
|
||||
## Approved finding types (from gc-scan.mjs)
|
||||
|
||||
| Type | Tier | Description |
|
||||
|---|---|---|
|
||||
| `stale-todo` | A | TODO/FIXME/HACK/XXX comment older than 90 days, or referencing a closed issue |
|
||||
| `anti-pattern` (mdc-ref) | A | `.mdc` file reference — must be `.md` |
|
||||
| `anti-pattern` (console-log) | A | Stray `console.log(` in `.ts`/`.tsx` files |
|
||||
| `anti-pattern` (ts-ignore) | A | `@ts-ignore` suppressing a real type error |
|
||||
| `anti-pattern` (ts-nocheck) | A | `@ts-nocheck` disabling all type checking |
|
||||
| `anti-pattern` (hardcoded-url) | B | Hardcoded `localhost:5001`/`5000` engine URL |
|
||||
| `anti-pattern` (deprecated) | B | `@deprecated` symbol still active in production code |
|
||||
| `oversized-file` | B | Source file exceeding ~350 lines |
|
||||
| `orphaned-rule-ref` | B | Rule file references a path that no longer exists |
|
||||
| `outdated-dep` | B | Dependency behind latest version |
|
||||
|
||||
## PR guard rails (mandatory — no exceptions)
|
||||
|
||||
- All PRs are **DRAFT**. Never mark ready-for-review. Never auto-merge.
|
||||
- Label every PR and Issue `gc-bot`.
|
||||
- **Max 3 PRs per run. Max 5 Issues per run.**
|
||||
- **Max ~150 changed lines per PR.** One concern per PR.
|
||||
- Branch naming: `gc/<patternId-or-type>-<git-short-sha>` (e.g. `gc/console-log-a3f1b2c`)
|
||||
- Before creating anything: check open `gc-bot` PRs and Issues for the same concern — skip if already covered.
|
||||
- Commit prefix: `chore:` for removal/cleanup, `refactor:` for structural fixes. Both are patch-level per the project `/commit` rule.
|
||||
- Every PR description must include: what changed · which detector flagged it · which rule it violates · how to verify the fix is correct.
|
||||
|
||||
## Never-touch list (absolute — the agent must refuse)
|
||||
|
||||
- `prisma/migrations/**` — migrations are irreversible
|
||||
- `pnpm-lock.yaml` — lockfile integrity
|
||||
- `.env`, `.env.local`, `.env.*`, any file containing secrets or tokens
|
||||
- Generated assets: `public/cesium/**`, `.next/**`, `node_modules/**`
|
||||
- Large seed files: `packages/*/data/**`, `local-seeders/**`
|
||||
- Binary assets: images, videos, fonts, `.db` files
|
||||
- `local-scripts/` — scratch / one-off scripts, not subject to conventions
|
||||
|
||||
## Dry-run mode
|
||||
|
||||
When `DRY_RUN=true` (set in the workflow dispatch input):
|
||||
|
||||
- Post one summary comment on the tracking Issue (`[gc-bot] Daily Scan Reports`). Create that Issue if it does not exist (label: `gc-bot`).
|
||||
- List every finding with tier, file, line, description.
|
||||
- **Do not create any PRs or Issues.**
|
||||
- Use dry-run for the first 3–5 days after deployment to validate finding quality before enabling normal mode.
|
||||
|
||||
## Idempotency rules
|
||||
|
||||
A finding is already handled if **either** of these is true:
|
||||
|
||||
1. A remote branch matching `gc/<same-patternId-or-type>-*` exists (`git branch -r --list 'origin/gc/<type>-*'`).
|
||||
2. An open PR or Issue labelled `gc-bot` already covers the same file + concern.
|
||||
|
||||
Skip handled findings without error or noise.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Garbage Collector — Daily Sweep Prompt
|
||||
|
||||
You are the WorldWideView Garbage Collector agent running headless in GitHub Actions.
|
||||
|
||||
Run the garbage-collector agent sweep:
|
||||
|
||||
1. Read `.agents/rules/garbage-collection.md` for your charter and all guard rails.
|
||||
2. Read `gc-findings.json` for the pre-scanned findings produced by `scripts/gc-scan.mjs`.
|
||||
3. Check the `DRY_RUN` environment variable — if `true`, post a dry-run summary only (no PRs or Issues).
|
||||
4. Execute the full sweep procedure described in `.agents/agents/garbage-collector.md`.
|
||||
|
||||
Do not scan the codebase yourself. Work only from `gc-findings.json`.
|
||||
@@ -1,10 +1,12 @@
|
||||
name: Garbage Collector
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 6 * * *' # 06:00 UTC daily
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
dry_run:
|
||||
description: 'Dry run — post report only, open no PRs/Issues'
|
||||
description: 'Dry run — report findings only, open no PRs or Issues (recommended for first 3–5 days)'
|
||||
required: false
|
||||
default: 'true'
|
||||
type: choice
|
||||
@@ -18,97 +20,97 @@ on:
|
||||
required: false
|
||||
default: '180'
|
||||
|
||||
schedule:
|
||||
- cron: '0 3 * * 1'
|
||||
concurrency:
|
||||
group: garbage-collector
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
contents: write
|
||||
pull-requests: write
|
||||
issues: write
|
||||
id-token: write
|
||||
|
||||
jobs:
|
||||
gc-scan:
|
||||
name: Scan for stale artefacts
|
||||
collect:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
|
||||
env:
|
||||
DRY_RUN: ${{ github.event.inputs.dry_run || 'true' }}
|
||||
GH_TOKEN: ${{ secrets.GC_BOT_TOKEN }}
|
||||
DRY_RUN: ${{ github.event.inputs.dry_run || 'false' }}
|
||||
DAYS_BRANCH: ${{ github.event.inputs.days_branch || '90' }}
|
||||
DAYS_COMMENT: ${{ github.event.inputs.days_comment || '180' }}
|
||||
GH_TOKEN: ${{ secrets.GC_BOT_TOKEN }}
|
||||
|
||||
steps:
|
||||
- name: Checkout (full history for git blame)
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
token: ${{ secrets.GC_BOT_TOKEN }}
|
||||
fetch-depth: 0 # full history required for git blame and branch age
|
||||
token: ${{ secrets.GC_BOT_TOKEN }} # PAT so pushed branches trigger CI
|
||||
|
||||
- name: Set up Node
|
||||
- name: Configure git user
|
||||
run: |
|
||||
git config user.name 'WorldWideView GC Bot'
|
||||
git config user.email '[email protected]'
|
||||
|
||||
- name: Setup pnpm
|
||||
uses: pnpm/action-setup@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
node-version: '22'
|
||||
cache: 'pnpm'
|
||||
|
||||
- name: Run GC scan
|
||||
id: scan
|
||||
- name: Install dependencies
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- name: Run deterministic scan
|
||||
run: |
|
||||
node scripts/gc-scan.mjs \
|
||||
--out gc-findings.json \
|
||||
--days-branch=${{ env.DAYS_BRANCH }} \
|
||||
--days-comment=${{ env.DAYS_COMMENT }} \
|
||||
> gc-report.json
|
||||
FINDINGS_COUNT=$(node -e "const r=require('./gc-report.json');console.log(r.findings.length)")
|
||||
echo "findings_count=$FINDINGS_COUNT" >> "$GITHUB_OUTPUT"
|
||||
echo "GC scan complete — $FINDINGS_COUNT finding(s)"
|
||||
--days-comment=${{ env.DAYS_COMMENT }}
|
||||
|
||||
- name: Upload raw report
|
||||
- name: Upload findings artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: gc-report
|
||||
path: gc-report.json
|
||||
name: gc-findings-${{ github.run_id }}
|
||||
path: gc-findings.json
|
||||
retention-days: 30
|
||||
|
||||
- name: Find or create tracking issue
|
||||
id: tracking
|
||||
run: |
|
||||
# gh issue list output: number<TAB>title<TAB>...
|
||||
ISSUE_NUMBER=$(gh issue list \
|
||||
--label gc-bot \
|
||||
--state open \
|
||||
--limit 1 | awk '{print $1}')
|
||||
- name: Run Garbage Collector agent
|
||||
uses: anthropics/claude-code-action@v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
show_full_output: true
|
||||
prompt: |
|
||||
You are the WorldWideView Garbage Collector agent running headless in GitHub Actions.
|
||||
|
||||
if [ -z "$ISSUE_NUMBER" ]; then
|
||||
ISSUE_NUMBER=$(gh issue create \
|
||||
--title "🗑️ Garbage Collector — tracking issue" \
|
||||
--body "This issue is updated automatically each time the GC scanner runs." \
|
||||
--label gc-bot | grep -oE '[0-9]+$')
|
||||
echo "Created tracking issue #$ISSUE_NUMBER"
|
||||
else
|
||||
echo "Found existing tracking issue #$ISSUE_NUMBER"
|
||||
fi
|
||||
echo "issue_number=$ISSUE_NUMBER" >> "$GITHUB_OUTPUT"
|
||||
Read `.agents/rules/garbage-collection.md` for your charter and all guard rails.
|
||||
Read `gc-findings.json` for the pre-scanned findings produced by `scripts/gc-scan.mjs`.
|
||||
|
||||
- name: Build report comment
|
||||
id: report
|
||||
run: |
|
||||
node scripts/gc-format-report.mjs gc-report.json "$DRY_RUN" > report-comment.md
|
||||
echo "Built report comment ($(wc -c < report-comment.md) bytes)"
|
||||
The DRY_RUN environment variable is: ${{ env.DRY_RUN }}
|
||||
|
||||
- name: Post scan report as comment
|
||||
run: |
|
||||
gh issue comment "${{ steps.tracking.outputs.issue_number }}" --body-file report-comment.md
|
||||
## If DRY_RUN is "true":
|
||||
Do NOT create any PRs or Issues.
|
||||
Instead, find or create the tracking Issue titled "[gc-bot] Daily Scan Reports" (label: gc-bot) in
|
||||
the silvertakana/worldwideview repository using the gh CLI:
|
||||
|
||||
- name: Open stale-branch PRs (live mode only)
|
||||
if: env.DRY_RUN == 'false'
|
||||
run: |
|
||||
node -e "
|
||||
const { execSync } = require('child_process');
|
||||
const r = require('./gc-report.json');
|
||||
const merged = r.findings.filter(f => f.type === 'stale-branch' && f.isMerged);
|
||||
for (const b of merged.slice(0, 5)) {
|
||||
try {
|
||||
const body = 'Branch merged ' + b.ageDays + ' days ago — safe to delete.\\nLast commit: ' + b.lastSubject + '\\nAuthor: ' + b.author;
|
||||
execSync('gh issue create --title "chore(gc): delete merged stale branch ' + b.branch + '" --body "' + body + '" --label gc-bot', { stdio: 'inherit' });
|
||||
} catch(e) { console.error(e.message); }
|
||||
}
|
||||
"
|
||||
# Find existing tracking issue
|
||||
gh issue list --repo silvertakana/worldwideview --label gc-bot --search "[gc-bot] Daily Scan Reports" --json number,title
|
||||
|
||||
# If it exists, add a comment:
|
||||
gh issue comment <number> --repo silvertakana/worldwideview --body "..."
|
||||
|
||||
# If it doesn't exist, create it:
|
||||
gh issue create --repo silvertakana/worldwideview --title "[gc-bot] Daily Scan Reports" --label gc-bot --body "..."
|
||||
|
||||
The comment body must list every finding with: tier, file, line, description.
|
||||
Include the run ID (${{ github.run_id }}) and timestamp at the top.
|
||||
|
||||
## If DRY_RUN is "false":
|
||||
Execute the full sweep procedure described in `.claude/agents/garbage-collector.md`.
|
||||
Create Tier A draft PRs and Tier B Issues per the guard rails in `.agents/rules/garbage-collection.md`.
|
||||
|
||||
Do not scan the codebase yourself. Work only from gc-findings.json.
|
||||
claude_args: '--model claude-haiku-4-5-20251001 --allowedTools Bash,Read,Edit,Write,Grep,Glob'
|
||||
@@ -21,6 +21,7 @@ node_modules/
|
||||
/.agents/plans
|
||||
/.agents/research/
|
||||
/.agents/context/
|
||||
/.agents/settings.local.json
|
||||
|
||||
# performance
|
||||
/performance/
|
||||
@@ -28,6 +29,9 @@ node_modules/
|
||||
# artifacts
|
||||
/artifacts/
|
||||
|
||||
# gc bot — generated scan output, never commit
|
||||
gc-findings.json
|
||||
|
||||
# scripts (tracked, but temporary scratchpads should go in /local-scripts/)
|
||||
|
||||
|
||||
|
||||
+2
-1
@@ -21,6 +21,7 @@ services:
|
||||
required: false
|
||||
environment:
|
||||
- "AUTH_SECRET=${AUTH_SECRET:-default_secret_for_local_dev}"
|
||||
- "ENCRYPTION_MASTER_KEY=${ENCRYPTION_MASTER_KEY:-local_dev_encryption_key_change_in_prod}"
|
||||
- "WWV_DEMO_ADMIN_SECRET=${WWV_DEMO_ADMIN_SECRET:-}"
|
||||
- "WWV_BRIDGE_TOKEN=${WWV_BRIDGE_TOKEN:-}"
|
||||
- "DATABASE_URL=postgresql://postgres:postgres@db:5432/worldwideview?schema=public"
|
||||
@@ -58,7 +59,7 @@ services:
|
||||
command: >
|
||||
sh -c "
|
||||
echo 'Syncing workspace dependencies...' &&
|
||||
cd $$SEEDERS_DIR/community && CI=true pnpm install --ignore-scripts && pnpm -r rebuild better-sqlite3 esbuild &&
|
||||
if [ -d \"$$SEEDERS_DIR/community\" ]; then cd $$SEEDERS_DIR/community && CI=true pnpm install --ignore-scripts && pnpm -r rebuild better-sqlite3 esbuild; fi &&
|
||||
if [ -d \"$$SEEDERS_DIR/private\" ]; then cd $$SEEDERS_DIR/private && CI=true pnpm install --ignore-scripts && pnpm -r rebuild better-sqlite3 esbuild; fi &&
|
||||
cd /app && node dist/server.js"
|
||||
ports:
|
||||
|
||||
+54
-2
@@ -29,8 +29,60 @@ if [ -z "$DATABASE_URL" ]; then
|
||||
fi
|
||||
|
||||
echo "[entrypoint] Running database migrations..."
|
||||
prisma migrate deploy
|
||||
echo "[entrypoint] Migrations complete."
|
||||
set +e
|
||||
MIGRATE_OUT=$(prisma migrate deploy 2>&1)
|
||||
MIGRATE_CODE=$?
|
||||
set -e
|
||||
|
||||
if [ $MIGRATE_CODE -eq 0 ]; then
|
||||
echo "[entrypoint] Migrations complete."
|
||||
elif echo "$MIGRATE_OUT" | grep -q "P3005"; then
|
||||
# P3005: schema exists but _prisma_migrations has no history.
|
||||
# Happens when 'prisma db push' (pnpm dev) ran before 'prisma migrate deploy' (docker).
|
||||
# Safe to auto-baseline only if there is zero schema drift.
|
||||
echo "[entrypoint] P3005: schema exists without migration history."
|
||||
echo "[entrypoint] Verifying schema matches migrations before auto-baselining..."
|
||||
set +e
|
||||
DIFF_OUT=$(prisma migrate diff \
|
||||
--from-url "$DATABASE_URL" \
|
||||
--to-schema-datamodel prisma/schema.prisma \
|
||||
--script 2>&1)
|
||||
DIFF_CODE=$?
|
||||
set -e
|
||||
if [ $DIFF_CODE -ne 0 ]; then
|
||||
echo "[entrypoint] Could not verify schema state — aborting." >&2
|
||||
echo "$DIFF_OUT" >&2
|
||||
exit 1
|
||||
fi
|
||||
# Positive-match for DDL keywords: any real drift emits at least one of these.
|
||||
# Checking for presence is safer than checking for absence of content.
|
||||
if echo "$DIFF_OUT" | grep -qiE "^(CREATE|ALTER|DROP|INSERT|UPDATE|DELETE|TRUNCATE)"; then
|
||||
echo "[entrypoint] Schema drift detected — cannot auto-baseline." >&2
|
||||
echo "[entrypoint] Run 'pnpm db:reset' to wipe and re-migrate, or resolve drift manually." >&2
|
||||
echo "$MIGRATE_OUT" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "[entrypoint] No drift detected. Baselining existing schema..."
|
||||
for migration_dir in prisma/migrations/*/; do
|
||||
if [ -f "${migration_dir}migration.sql" ]; then
|
||||
migration_name=$(basename "$migration_dir")
|
||||
echo "[entrypoint] Marking as applied: $migration_name"
|
||||
set +e
|
||||
prisma migrate resolve --applied "$migration_name"
|
||||
RESOLVE_CODE=$?
|
||||
set -e
|
||||
if [ $RESOLVE_CODE -ne 0 ]; then
|
||||
echo "[entrypoint] Failed to baseline $migration_name — aborting." >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
done
|
||||
prisma migrate deploy
|
||||
echo "[entrypoint] Migrations complete."
|
||||
else
|
||||
echo "$MIGRATE_OUT" >&2
|
||||
exit $MIGRATE_CODE
|
||||
fi
|
||||
|
||||
# Generate self-signed SSL certificates for local HTTPS bridging if they don't exist
|
||||
if [ ! -f "./data/localhost.crt" ] || [ ! -f "./data/localhost.key" ]; then
|
||||
|
||||
+6
-4
@@ -75,9 +75,11 @@ For legacy plugins (e.g., Webcams) that *cannot* use WebSockets, the local proxy
|
||||
## Operational Implementation Details
|
||||
|
||||
### 1. Library Selection
|
||||
* **`jose`:** The industry-standard library for modern JS. Used by the Marketplace to generate Ed25519 keys, sign JWTs, and expose the JWKS endpoint.
|
||||
* **`jose`:** The industry-standard library for modern JS. Used by the Marketplace to generate Ed25519 keys, sign JWTs, and expose the JWKS endpoint. **Also used by Data Engines** to verify tickets — `createRemoteJWKSet` handles remote fetching, caching, and `kid` selection, and `jwtVerify` validates the signature plus `iss`/`aud`/`exp` in one call, with native OKP/Ed25519 support.
|
||||
* **`openid-client`:** The RFC-compliant library used by the Local App to execute the PKCE Authorization Code flow securely.
|
||||
* **`@fastify/jwt` + `get-jwks`:** Used by Data Engines. `@fastify/jwt` provides the middleware, while `get-jwks` (built by the Fastify core team) handles remote fetching, local LRU caching, and OKP-to-PEM conversion for Ed25519 keys.
|
||||
|
||||
> [!NOTE]
|
||||
> **Amendment (2026-05-22):** The original draft specified `@fastify/jwt` + `get-jwks` for Data Engines. This was found to be incorrect during local integration testing: `get-jwks.getPublicKey()` converts JWKs via `jwk-to-pem`, which **does not support OKP/Ed25519 keys** (throws `Unsupported key type "OKP"`). `@fastify/jwt`'s standalone `verify()` is also incompatible with a remote-JWKS async key resolver. Data Engines now use `jose` — the same library already mandated for the Marketplace — which supports Ed25519 natively.
|
||||
|
||||
### 2. Hardened First-Message Auth
|
||||
Data Engines MUST enforce strict WebSocket handshake rules:
|
||||
@@ -145,8 +147,8 @@ sequenceDiagram
|
||||
M->>L: 5-min JWT (aud: Engine A)
|
||||
L->>U: JWT
|
||||
U->>E: wss://engine (Origin validation)
|
||||
U->>E: { type: "auth", jwt: "..." }
|
||||
E->>E: Verify with get-jwks cache
|
||||
U->>E: { type: "auth", v: 1, token: "..." }
|
||||
E->>E: Verify JWT via jose (cached JWKS)
|
||||
E->>U: Stream
|
||||
end
|
||||
```
|
||||
@@ -0,0 +1,175 @@
|
||||
# ADR-0003: Shared Identity & Ecosystem Authentication Host
|
||||
|
||||
## Status
|
||||
Proposed *(Pending Review)*
|
||||
|
||||
## Date
|
||||
2026-05-22
|
||||
|
||||
## Related
|
||||
- **Builds on:** ADR-001 (Decentralized Plugin Auth & SSRF Mitigation) — specifically the PKCE flow that lets the Local App acquire tokens from the Marketplace
|
||||
- **Supersedes:** `.agents/rules/cloud-auth-architecture.md` (which incorrectly states that `app.worldwideview.dev` owns all login UI — this ADR reverses that)
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
WorldWideView is growing into a multi-product ecosystem:
|
||||
|
||||
| Product | URL | Purpose |
|
||||
|---|---|---|
|
||||
| Landing / Auth | `worldwideview.dev` | Marketing, login, signup, cloud plan registration |
|
||||
| Marketplace | `marketplace.worldwideview.dev` | Browse, install, and manage plugins |
|
||||
| Cloud App | `app.worldwideview.dev` (or `[tenant].app.worldwideview.dev`) | Hosted geospatial intelligence instance |
|
||||
| Local App | Self-hosted | User's own instance, authenticated locally |
|
||||
|
||||
Four interrelated problems required a unified decision:
|
||||
|
||||
1. **No Marketplace identity system exists.** Users have nowhere to create an account, log in, or manage subscriptions.
|
||||
2. **Fragmented account experience.** Without a plan, a user would need separate accounts for the Marketplace and the Cloud App — unacceptable for a single-vendor ecosystem.
|
||||
3. **Cookie domain fragmentation.** If each product hosts its own login page, sessions cannot be shared across subdomains. A user logged into the Marketplace would need to log in again at the Cloud App.
|
||||
4. **The `worldwideview-web` repo is a static export** (`output: "export"`). It has a `/login` page backed by client-side Supabase but cannot support server-side sessions, API routes, or cross-subdomain cookie management.
|
||||
|
||||
---
|
||||
|
||||
## Decisions
|
||||
|
||||
### ADR-003A: Federated Identity — One Account Across All Products
|
||||
|
||||
All WorldWideView products share a **single identity**. A user who creates an account anywhere in the ecosystem (Marketplace, Cloud App, or the landing site) has that same identity available everywhere else.
|
||||
|
||||
This is the standard industry pattern for multi-product SaaS — used by GitHub (one account → GitHub, Marketplace, Actions, Packages), Atlassian (one account → Jira, Confluence, Marketplace), and Stripe (one account → Dashboard, Atlas, Stripe Apps).
|
||||
|
||||
The shared identity store is a **single Supabase project**. All products point to the same `NEXT_PUBLIC_SUPABASE_URL` and read from the same `auth.users` table. No synchronisation jobs, no duplicate user records.
|
||||
|
||||
### ADR-003B: Supabase Auth as the Identity Provider
|
||||
|
||||
**Supabase Auth** is the identity layer for the entire ecosystem. It was chosen over:
|
||||
|
||||
- **Better Auth** — excellent self-hosted option, but requires a Postgres database to be provisioned separately. Since the Marketplace and `worldwideview-web` have no existing database, Supabase provides auth + Postgres in a single managed service with zero extra infrastructure.
|
||||
- **NextAuth v5** — already present in the Local App for local access control, but it is not a full identity platform. Missing 2FA, RBAC, and a proper OAuth server model. Not suitable as an ecosystem-wide IdP.
|
||||
- **Clerk** — fastest setup and best Stripe integration, but ~$25–50/month at low user counts, per-MAU pricing at scale, and user data is stored on Clerk's servers. Vendor lock-in is unacceptable for a product that holds plugin and subscription data.
|
||||
|
||||
Supabase Auth provides:
|
||||
- Email + password, Google OAuth, GitHub OAuth, and magic link — all configured in the Supabase dashboard with no provider code
|
||||
- `@supabase/ssr` package with first-class Next.js App Router support (server components, middleware, server actions)
|
||||
- Cookie-based sessions that can be scoped to the parent domain
|
||||
- Free tier is sufficient for early-stage Marketplace; Pro plan ($25/month) unlocks unlimited MAU
|
||||
|
||||
### ADR-003C: `worldwideview.dev` (worldwideview-web repo) is the Ecosystem Auth Host
|
||||
|
||||
The login and account registration UI lives at **`worldwideview.dev`** — the apex domain, served by the `worldwideview-web` repository. Three alternatives were considered:
|
||||
|
||||
**Option A — Each app has its own login page**
|
||||
Rejected. Each product renders its own login form against the shared Supabase project. Browser `localStorage` (used by client-side Supabase) is domain-scoped and cannot be shared across subdomains. This means a user logged in at `marketplace.worldwideview.dev` would not be recognised at `app.worldwideview.dev`. Solving this in each app separately creates duplicated middleware and fragile cookie coordination.
|
||||
|
||||
**Option B — Marketplace owns the login page**
|
||||
Rejected. The Cloud App would be permanently subordinate to the Marketplace for authentication. If the Marketplace is down, Cloud App users cannot log in. Also creates a confusing UX: users navigate to `app.worldwideview.dev` and are immediately bounced to `marketplace.worldwideview.dev` just to authenticate.
|
||||
|
||||
**Option C — Dedicated `accounts.worldwideview.dev` subdomain**
|
||||
Seriously considered. A neutral auth subdomain is the cleanest long-term architecture. Rejected for now because it requires a third Next.js deployment and a third Coolify application. The same result is achieved by hosting auth on the apex domain itself (`worldwideview.dev`), which is already managed by `worldwideview-web` and is the identity host by virtue of owning the root domain.
|
||||
|
||||
**Chosen: Apex domain as auth host.** The `worldwideview-web` repo becomes the auth surface for the whole ecosystem:
|
||||
|
||||
| Path | Type | Purpose |
|
||||
|---|---|---|
|
||||
| `/` | Static | Landing page (marketing) |
|
||||
| `/about`, `/docs`, `/plugins` | Static | Marketing content |
|
||||
| `/login` | Dynamic (SSR) | Shared login for all products |
|
||||
| `/signup` | Dynamic (SSR) | New account creation |
|
||||
| `/cloud` | Dynamic (SSR) | Cloud plan registration and subscription |
|
||||
| `/accounts/*` | Dynamic (SSR) | Account management, billing, API keys |
|
||||
| `/api/auth/callback` | Dynamic (SSR) | OAuth redirect handler (Google, GitHub) |
|
||||
|
||||
**Why the apex domain is correct:** A session cookie set on `worldwideview.dev` is automatically inherited by all subdomains (`marketplace.worldwideview.dev`, `app.worldwideview.dev`, and any future product). This is the same mechanism used by Stripe — `stripe.com/login` sets the session that `dashboard.stripe.com` reads.
|
||||
|
||||
### ADR-003D: Cross-Subdomain Session Sharing via Parent Domain Cookie
|
||||
|
||||
Supabase Auth session cookies are explicitly scoped to the parent domain so that all products share the same session without additional coordination:
|
||||
|
||||
```typescript
|
||||
// Applied in worldwideview-web middleware and all Supabase server clients
|
||||
cookieOptions: {
|
||||
domain: '.worldwideview.dev', // leading dot = all subdomains inherit
|
||||
path: '/',
|
||||
sameSite: 'lax',
|
||||
secure: true, // HTTPS required — see local dev below
|
||||
}
|
||||
```
|
||||
|
||||
Each product (Marketplace, Cloud App) reads this cookie to get the authenticated user, then calls `supabase.auth.getUser()` server-side to validate the session. No cross-app API calls for authentication.
|
||||
|
||||
**Logout** must clear the cookie on the parent domain, not the subdomain, or the session will persist across products.
|
||||
|
||||
### ADR-003E: `worldwideview-web` Converts from Static Export to Mixed Static/SSR
|
||||
|
||||
The current `worldwideview-web` uses `output: "export"` in `next.config.ts`, forcing every page to be a static HTML file served by Nginx. This is removed.
|
||||
|
||||
Without `output: "export"`, Next.js App Router applies per-route rendering:
|
||||
- Pages that do not call `cookies()`, `headers()`, or other dynamic functions are **statically generated at build time** — marketing pages remain fast and CDN-cached with no change required.
|
||||
- Pages that call `cookies()` (all auth pages using `@supabase/ssr`) are **server-rendered on request**.
|
||||
|
||||
The Docker deployment changes from an Nginx static image to a Node.js runtime image. The Coolify application is updated in-place — no new Coolify application is needed.
|
||||
|
||||
### ADR-003F: `wwv.local` Local Development Environment
|
||||
|
||||
Because the parent-domain cookie requires `secure: true` (HTTPS only), local development using `localhost` will not replicate production session-sharing behaviour. A local HTTPS domain is required.
|
||||
|
||||
The project adopts `wwv.local` as the local development TLD with a self-signed certificate:
|
||||
|
||||
| Local URL | Maps to | Purpose |
|
||||
|---|---|---|
|
||||
| `wwv.local` | worldwideview-web on port 3001 | Landing + auth |
|
||||
| `marketplace.wwv.local` | worldwideview-marketplace on port 3002 | Marketplace |
|
||||
| `app.wwv.local` | worldwideview main app on port 3000 | Local App / Cloud App |
|
||||
|
||||
This setup is a deliberate investment: it ensures that cross-subdomain auth, redirect flows, and cookie behaviour can be tested locally before production deployment. The setup cost is a one-time `mkcert` install and hosts file edit per developer machine.
|
||||
|
||||
Setup (documented in each repo's README):
|
||||
```bash
|
||||
# Install mkcert (once per machine)
|
||||
# macOS: brew install mkcert && mkcert -install
|
||||
# Windows: choco install mkcert && mkcert -install
|
||||
|
||||
mkcert "*.wwv.local" wwv.local
|
||||
# Add to hosts: 127.0.0.1 wwv.local marketplace.wwv.local app.wwv.local
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security Constraints (from Pre-Mortem Review)
|
||||
|
||||
The following risks were identified and mitigated before implementation:
|
||||
|
||||
| Risk | Mitigation |
|
||||
|---|---|
|
||||
| Supabase anon key can read all rows if RLS is disabled | Row Level Security (RLS) MUST be enabled on all app-specific tables (`MarketplaceUser`, `InstalledPlugin`). Policy: `using (id = auth.uid())`. Server routes use `SUPABASE_SERVICE_ROLE_KEY` which bypasses RLS by design. |
|
||||
| PKCE authorization code replay attack | The authorization code is deleted from the database transactionally at the moment of redemption. Any subsequent attempt receives `400 invalid_grant`. |
|
||||
| Supabase free tier limits magic link email to 2/hour per address | A custom SMTP provider (Resend) is configured in Supabase project settings before magic link is enabled in production. |
|
||||
| `MarketplaceUser` becomes orphaned when `auth.users` row is deleted | A Postgres trigger on `auth.users` DELETE cascades to `MarketplaceUser` and all related rows. |
|
||||
| Ed25519 signing key rotation causes token rejection at Data Engines | Signing keys are stored in a `SigningKey` DB table, not env vars. The JWKS endpoint serves all non-revoked keys simultaneously. During rotation, the old key is marked `retiring` and kept in JWKS for 10 minutes before being marked `revoked`. |
|
||||
|
||||
---
|
||||
|
||||
## Marketplace PKCE OAuth Server (Relation to ADR-001)
|
||||
|
||||
The Marketplace acts as the **OAuth 2.0 Authorization Server** in the ADR-001 PKCE flow. Supabase Auth handles user identity; a thin custom layer handles the code exchange:
|
||||
|
||||
1. Local App redirects user to `marketplace.worldwideview.dev/api/oauth/authorize`
|
||||
2. Marketplace validates the Supabase session (or redirects to `worldwideview.dev/login`)
|
||||
3. User sees consent screen and approves
|
||||
4. Marketplace issues a short-lived authorization code (60s TTL), redirects to Local App
|
||||
5. Local App exchanges code + `code_verifier` for a long-lived API Key
|
||||
6. Subsequent plugin connections use the API Key to request 5-minute EdDSA JWTs (ADR-001B)
|
||||
|
||||
This is not a new Supabase feature — it is custom Next.js API routes sitting on top of the existing Supabase Auth session.
|
||||
|
||||
---
|
||||
|
||||
## Consequences
|
||||
|
||||
- **One Supabase project** is a shared dependency across `worldwideview-web`, `worldwideview-marketplace`, and `worldwideview` (Cloud App). A Supabase outage affects all products simultaneously. This is an accepted tradeoff at current scale; the mitigation is Supabase's SLA and free-tier generous limits.
|
||||
- **`worldwideview-web` must run as a Node.js server**, not a static Nginx container. This increases Coolify resource usage marginally but enables all auth functionality.
|
||||
- **Local development requires `mkcert` setup** (one-time, ~5 minutes). This is a deliberate decision to prevent a class of production bugs that cannot be caught with `localhost`.
|
||||
- **The old `cloud-auth-architecture.md` rule is invalidated.** That rule stated `app.worldwideview.dev` owns all login UI and the Marketplace redirects there. The reverse is now true. That rules file must be updated to reference this ADR.
|
||||
- **Upgrade path is clear:** If the Marketplace matures into a platform that third-party developers build against (issuing OAuth tokens to external apps), the existing PKCE layer is extended without changing the user table or Supabase project. The identity layer is already decoupled from the token exchange layer.
|
||||
@@ -0,0 +1,258 @@
|
||||
# SECURITY.md -- Phase 3: Local App PKCE + SSRF Proxy Hardening
|
||||
|
||||
**Audit date:** 2026-05-24
|
||||
**Phase:** 3 -- Local App PKCE + SSRF Proxy Hardening
|
||||
**ASVS Level:** 2
|
||||
**Audit mode:** Retroactive-STRIDE (no formal threat model at plan time; register constructed from implementation)
|
||||
**block_on:** open
|
||||
|
||||
---
|
||||
|
||||
## Threat Register
|
||||
|
||||
| ID | Category | Description | Disposition | Status |
|
||||
|----|----------|-------------|-------------|--------|
|
||||
| T01 | SSRF | Proxy endpoints use raw `fetch` instead of `safeFetch` | mitigate | CLOSED |
|
||||
| T02 | SSRF | `PROXY_HOST_ALLOWLIST` not enforced | mitigate | CLOSED |
|
||||
| T03 | SSRF | Private IP ranges not blocked | mitigate | CLOSED |
|
||||
| T04 | SSRF/DNS-rebinding | DNS resolved multiple times; IP not pinned between lookup and connect | mitigate | CLOSED |
|
||||
| T05 | SSRF/Open-redirect | Proxy follows 3xx redirects, enabling open-redirect chains | mitigate | CLOSED |
|
||||
| T06 | SSRF | iframe proxy previously used raw `fetch` with no size cap | mitigate | CLOSED |
|
||||
| T07 | Spoofing | PKCE state not validated; forged state accepted | mitigate | CLOSED |
|
||||
| T08 | Spoofing | PKCE state cookie not `__Host-` prefixed on HTTPS | mitigate | CLOSED |
|
||||
| T09 | Spoofing | PKCE state/verifier cookies not `httpOnly`, missing `SameSite` | mitigate | CLOSED |
|
||||
| T10 | Spoofing | PKCE state/verifier cookies too long-lived (>10 min) | mitigate | CLOSED |
|
||||
| T11 | Elevation | `code_verifier` cookie reusable after exchange (replay) | mitigate | CLOSED |
|
||||
| T12 | Tampering | `ENCRYPTION_MASTER_KEY` unset falls back to zero-key silently | mitigate | CLOSED |
|
||||
| T13 | Elevation | WsClient sends `subscribe` before auth (`onopen` path) | mitigate | CLOSED |
|
||||
| T14 | Elevation | WsClient sends `subscribe` before auth (`subscribe()` method -- race, already-open connection) | mitigate | OPEN (BLOCKER) |
|
||||
|
||||
---
|
||||
|
||||
## Verification Evidence
|
||||
|
||||
### T01 -- Proxy endpoints use `safeFetch`
|
||||
|
||||
All three user-facing proxy routes import and call `safeFetch`, not raw `fetch`:
|
||||
|
||||
- `src/app/api/camera/proxy/route.ts:6,27` -- `import { safeFetch }` + `await safeFetch(targetUrl, ...)`
|
||||
- `src/app/api/camera/proxy/iframe/route.ts:6,32` -- `import { safeFetch }` + `await safeFetch(targetUrl, ...)`
|
||||
- `src/app/api/camera/proxy/stream/route.ts:6,31` -- `import { safeFetch }` + `await safeFetch(targetUrl, ...)`
|
||||
|
||||
**Status: CLOSED**
|
||||
|
||||
---
|
||||
|
||||
### T02 -- `PROXY_HOST_ALLOWLIST` enforced
|
||||
|
||||
`src/lib/security/ssrf.ts:25-42` -- `checkHostAllowlist()` is called at the top of `safeFetch` before any network I/O. Behavior:
|
||||
- Unset env var: throws `SSRF Error: PROXY_HOST_ALLOWLIST is not configured`
|
||||
- Empty string: throws same error
|
||||
- `"*"`: warns and permits (permissive mode, documented)
|
||||
- Specific list: rejects unlisted hostnames
|
||||
|
||||
Covered by unit tests in `src/lib/security/ssrf.spec.ts:63-82`.
|
||||
|
||||
**Status: CLOSED**
|
||||
|
||||
---
|
||||
|
||||
### T03 -- Private IP ranges blocked
|
||||
|
||||
`src/lib/security/ssrf.ts:4-9` -- `isPrivateIP()` covers:
|
||||
- `127.*`, `10.*`, `192.168.*`, `169.254.*` (AWS/GCP metadata), `0.*`
|
||||
- `172.16-31.*`
|
||||
- IPv6: `::1`, `fc*`, `fd*`, `fe80*`
|
||||
|
||||
Called twice: once against the literal URL hostname (line 56) and once against the resolved IP (line 66).
|
||||
|
||||
**Status: CLOSED**
|
||||
|
||||
---
|
||||
|
||||
### T04 -- DNS pinning via undici Agent
|
||||
|
||||
`src/lib/security/ssrf.ts:60-80` -- Single `dns.lookup()` call stores `resolvedIp` and `resolvedFamily`. A custom undici `Agent` is constructed with a `connect.lookup` override that always returns the pre-resolved address, preventing a second DNS resolution during TCP connect.
|
||||
|
||||
**Status: CLOSED**
|
||||
|
||||
---
|
||||
|
||||
### T05 -- Redirect following disabled
|
||||
|
||||
`src/lib/security/ssrf.ts:92` -- `redirect: "manual" as const` is passed to undici `fetch`. Verified by test at `ssrf.spec.ts:117-135` which asserts `callOpts.redirect === "manual"` and that only one fetch call is made when a 301 is returned.
|
||||
|
||||
**Status: CLOSED**
|
||||
|
||||
---
|
||||
|
||||
### T06 -- iframe proxy uses `safeFetch` with size cap
|
||||
|
||||
`src/app/api/camera/proxy/iframe/route.ts:6,32,39` -- Uses `safeFetch` with `maxSize: 5 * 1024 * 1024` and `timeout: MAX_IFRAME_DURATION_MS` (10 s). The previous raw-fetch vulnerability is no longer present.
|
||||
|
||||
**Status: CLOSED**
|
||||
|
||||
---
|
||||
|
||||
### T07 -- PKCE state validated in callback
|
||||
|
||||
`src/app/api/marketplace/callback/route.ts:10-15`:
|
||||
|
||||
```
|
||||
const stateCookie = req.cookies.get(`${cookiePrefix}pkce_state`)?.value;
|
||||
const urlState = req.nextUrl.searchParams.get("state");
|
||||
if (!stateCookie || urlState !== stateCookie) {
|
||||
return NextResponse.json({ error: "State mismatch" }, { status: 400 });
|
||||
}
|
||||
```
|
||||
|
||||
Both missing cookie and value mismatch are rejected.
|
||||
|
||||
**Status: CLOSED**
|
||||
|
||||
---
|
||||
|
||||
### T08 -- `__Host-` cookie prefix on HTTPS
|
||||
|
||||
`src/app/api/marketplace/connect/route.ts:22-23`:
|
||||
```
|
||||
const isHttps = req.nextUrl.protocol === "https:";
|
||||
const cookiePrefix = isHttps ? "__Host-" : "";
|
||||
```
|
||||
Same logic mirrored in `src/app/api/marketplace/callback/route.ts:7-8`. On HTTPS the cookies are named `__Host-pkce_state` and `__Host-pkce_verifier`, which browsers enforce must be `Secure`, `Path=/`, and no `Domain` attribute.
|
||||
|
||||
Note: the cookies use `path: "/api/marketplace/callback"`, not `path: "/"`. The `__Host-` prefix specification (RFC 6265bis) requires `Path=/`. This is a partial inconsistency -- the `__Host-` prefix technically requires `Path=/` to be valid per browsers that enforce the prefix spec strictly. In practice Chromium enforces `Path=/` for `__Host-` cookies and will silently reject the Set-Cookie header. This is flagged as an unregistered finding below but does not affect current HTTP (dev) deployments. **Treat as unregistered flag UF-01.**
|
||||
|
||||
**Status: CLOSED** (state validation logic present; see UF-01 for prefix/path mismatch)
|
||||
|
||||
---
|
||||
|
||||
### T09 -- `httpOnly` + `SameSite` on PKCE cookies
|
||||
|
||||
`src/app/api/marketplace/connect/route.ts:26-38`:
|
||||
- `httpOnly: true` -- set on both cookies
|
||||
- `sameSite: "lax"` -- set on both cookies
|
||||
- `secure: isHttps` -- conditional on protocol
|
||||
|
||||
**Status: CLOSED**
|
||||
|
||||
---
|
||||
|
||||
### T10 -- PKCE cookies short-lived (10 min)
|
||||
|
||||
`src/app/api/marketplace/connect/route.ts:30,38`: `maxAge: 60 * 10` (600 seconds = 10 minutes) on both state and verifier cookies.
|
||||
|
||||
**Status: CLOSED**
|
||||
|
||||
---
|
||||
|
||||
### T11 -- `code_verifier` cookie deleted post-exchange (single-use)
|
||||
|
||||
`src/app/api/marketplace/callback/route.ts:66-82` -- After successful token exchange, both `pkce_state` and `pkce_verifier` cookies are explicitly cleared by setting `maxAge: 0` with empty string values. This invalidates the cookies immediately on the response, preventing replay.
|
||||
|
||||
**Status: CLOSED**
|
||||
|
||||
---
|
||||
|
||||
### T12 -- `ENCRYPTION_MASTER_KEY` fails hard when unset
|
||||
|
||||
`src/lib/auth/encryption.ts:8-11`:
|
||||
```
|
||||
function getMasterKey(): string {
|
||||
const key = process.env.ENCRYPTION_MASTER_KEY;
|
||||
if (!key) throw new Error('[encryption] ENCRYPTION_MASTER_KEY env var is required');
|
||||
return key;
|
||||
}
|
||||
```
|
||||
No fallback. No zero-key. Both `encryptCredential` and `decryptCredential` call `getMasterKey()`.
|
||||
|
||||
**Status: CLOSED**
|
||||
|
||||
---
|
||||
|
||||
### T13 -- WsClient `onopen` sends auth before subscribe (ticket-auth path)
|
||||
|
||||
`src/core/data/WsClient.ts:84-107`:
|
||||
- When any subscription requires ticket auth, `engine.awaitingWelcome = true` is set and `subscribe` messages are NOT sent on `onopen`
|
||||
- Subscribes are deferred until `data.type === "welcome"` is received (lines 118-124)
|
||||
- A 3-second auth timeout closes the connection if welcome never arrives (lines 91-96)
|
||||
|
||||
**Status: CLOSED** (for the `onopen` path)
|
||||
|
||||
---
|
||||
|
||||
### T14 -- WsClient `subscribe()` method sends `subscribe` on already-open connection while `awaitingWelcome = true` (OPEN BLOCKER)
|
||||
|
||||
`src/core/data/WsClient.ts:198-211`:
|
||||
|
||||
```typescript
|
||||
public subscribe(pluginId: string, engineUrl: string) {
|
||||
engine.subscriptions.add(pluginId);
|
||||
this.connectEngine(engineUrl);
|
||||
this.send(engine, { action: "subscribe", pluginId }); // line 210
|
||||
}
|
||||
```
|
||||
|
||||
`this.send()` at line 210 checks only `ws.readyState === WebSocket.OPEN` before sending. It does **not** check `engine.awaitingWelcome`. If:
|
||||
1. Plugin A connects and triggers ticket auth (`awaitingWelcome = true`)
|
||||
2. Before the `welcome` arrives, Plugin B calls `subscribe()` on the same engine
|
||||
3. The WS is already OPEN, so `send()` fires -- Plugin B's `subscribe` message is sent to the server before auth is complete
|
||||
|
||||
This allows an unauthenticated subscribe frame to reach the data engine server. Whether the server accepts it depends on server-side enforcement, but the client does not enforce the auth-first guarantee for late joiners.
|
||||
|
||||
The `onopen` path (T13) is correctly gated. The `subscribe()` entry point is not.
|
||||
|
||||
**Impact:** A second plugin subscribing to an engine in the window between connection open and welcome receipt bypasses the client-side auth gate. The data engine must reject unauthenticated subscriptions server-side for defense in depth, but the ADR-001 intent was client-enforced first-message auth.
|
||||
|
||||
**Required fix:** In `subscribe()`, check `engine.awaitingWelcome` before calling `this.send()`. If `awaitingWelcome` is true, skip the early send -- the subscription is already in `engine.subscriptions` and will be sent when `welcome` is received in `onmessage`.
|
||||
|
||||
**Status: OPEN (BLOCKER)**
|
||||
|
||||
---
|
||||
|
||||
## Unregistered Flags
|
||||
|
||||
### UF-01 -- `__Host-` prefix with `path: "/api/marketplace/callback"` violates prefix spec
|
||||
|
||||
**File:** `src/app/api/marketplace/connect/route.ts:29,37`
|
||||
|
||||
The `__Host-` cookie prefix per RFC 6265bis requires `Path=/`. The implementation sets `path: "/api/marketplace/callback"`. Chromium-based browsers enforce this and will silently reject the `Set-Cookie` header when `__Host-` is combined with a non-root path. In practice this means on HTTPS deployments the PKCE cookies may not be set at all, breaking the OAuth flow entirely -- or the browser falls back to setting them without the prefix, losing the security guarantee.
|
||||
|
||||
This is a functional correctness issue as well as a security regression. Recommend changing `path` to `"/"` or switching the cookie name to `pkce_state` / `pkce_verifier` without the `__Host-` prefix (accepting the loss of the prefix guarantee).
|
||||
|
||||
**Not a declared threat -- flagged as unregistered surface. Does not count as BLOCKER per config, but should be fixed before HTTPS deployment.**
|
||||
|
||||
---
|
||||
|
||||
### UF-02 -- `src/app/api/camera/extract/route.ts` uses raw `fetch` against user-supplied URL
|
||||
|
||||
**File:** `src/app/api/camera/extract/route.ts:17,29`
|
||||
|
||||
This route accepts a user-supplied `url` query parameter and passes it directly to `fetch()` without going through `safeFetch`. Current code only executes the fetch if `targetUrl.includes("balticlivecam.com")`, which provides weak host filtering via substring match rather than proper hostname validation. This is outside the declared Phase 3 scope but represents new SSRF attack surface that was not mitigated. A crafted URL like `https://evil.com?x=balticlivecam.com` would pass the check.
|
||||
|
||||
**Not a declared threat -- flagged as unregistered surface.**
|
||||
|
||||
---
|
||||
|
||||
### UF-03 -- `src/app/api/camera/test/route.ts` uses raw `fetch` against user-supplied URL
|
||||
|
||||
**File:** `src/app/api/camera/test/route.ts:45,56,68` (confirmed by grep)
|
||||
|
||||
The test route uses raw `fetch` with a user-supplied URL. Its auth posture was not reviewed in this phase. Flagged for completeness.
|
||||
|
||||
**Not a declared threat -- flagged as unregistered surface.**
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Metric | Count |
|
||||
|--------|-------|
|
||||
| Total threats | 14 |
|
||||
| CLOSED | 13 |
|
||||
| OPEN (BLOCKER) | 1 |
|
||||
| Unregistered flags | 3 |
|
||||
|
||||
**Phase 3 MUST NOT SHIP until T14 is resolved.**
|
||||
|
||||
The fix is mechanical: add an `awaitingWelcome` guard in the `public subscribe()` method before the `this.send()` call at `WsClient.ts:210`.
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "worldwideview",
|
||||
"version": "2.17.4",
|
||||
"version": "2.19.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"packageManager": "[email protected]",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@worldwideview/wwv-lib-incidents",
|
||||
"version": "1.0.2",
|
||||
"version": "1.0.4",
|
||||
"description": "Shared abstract base classes for WorldWideView incident plugins (earthquakes, wildfires, conflicts)",
|
||||
"main": "src/index.ts",
|
||||
"types": "src/index.ts",
|
||||
|
||||
@@ -0,0 +1,283 @@
|
||||
/**
|
||||
* Guard tests for the BaseIncidentPlugin WebSocket payload contract.
|
||||
*
|
||||
* CONTEXT — The bug this suite guards against (now fixed):
|
||||
* WsClient.handleDataMessage (src/core/data/WsClient.ts, lines 166-190) has three branches:
|
||||
* 1. Plugin has `mapWebsocketPayload` → called with raw payload → must return GeoEntity[]
|
||||
* 2. No handler AND payload IS a flat array → used directly (timestamps normalised)
|
||||
* 3. No handler AND payload is ANY OTHER object → silently dropped (console.warn + return)
|
||||
*
|
||||
* BaseIncidentPlugin now provides a default `mapWebsocketPayload` that unwraps the three
|
||||
* common payload shapes (scheduler envelope, GeoJSON FeatureCollection, flat array).
|
||||
* Subclasses may override for domain-specific shapes.
|
||||
*/
|
||||
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { BaseIncidentPlugin } from "./index";
|
||||
import type {
|
||||
GeoEntity,
|
||||
TimeRange,
|
||||
ServerPluginConfig,
|
||||
FilterDefinition,
|
||||
PluginCategory,
|
||||
CesiumEntityOptions,
|
||||
} from "@worldwideview/wwv-plugin-sdk";
|
||||
|
||||
// ─── Minimal GeoEntity factory ────────────────────────────────────────────────
|
||||
|
||||
function makeEntity(id: string, overrides: Partial<GeoEntity> = {}): GeoEntity {
|
||||
return {
|
||||
id,
|
||||
pluginId: "test-incident-plugin",
|
||||
latitude: 35.6762,
|
||||
longitude: 139.6503,
|
||||
timestamp: new Date("2025-01-01T00:00:00Z"),
|
||||
properties: {},
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
// ─── Minimal concrete subclass ────────────────────────────────────────────────
|
||||
// We keep it as bare-bones as possible — only the abstract members required to
|
||||
// instantiate the class. We do NOT add mapWebsocketPayload here, so we're testing
|
||||
// the BaseIncidentPlugin base behaviour exactly.
|
||||
|
||||
class MinimalIncidentPlugin extends BaseIncidentPlugin {
|
||||
id = "test-incident-plugin";
|
||||
name = "Test Incident Plugin";
|
||||
description = "Test";
|
||||
icon = null;
|
||||
category: PluginCategory = "natural-disaster";
|
||||
version = "0.0.1";
|
||||
|
||||
async fetch(_timeRange: TimeRange): Promise<GeoEntity[]> {
|
||||
return [];
|
||||
}
|
||||
|
||||
getServerConfig(): ServerPluginConfig {
|
||||
return {
|
||||
apiBasePath: "/api/test-incident-plugin",
|
||||
streamUrl: "ws://localhost:5001/stream",
|
||||
pollingIntervalMs: 0,
|
||||
};
|
||||
}
|
||||
|
||||
getFilterDefinitions(): FilterDefinition[] {
|
||||
return [];
|
||||
}
|
||||
|
||||
getLegend(): { label: string; color: string; filterId?: string; filterValue?: string }[] {
|
||||
return [];
|
||||
}
|
||||
|
||||
protected getSeverityValue(_entity: GeoEntity): number {
|
||||
return 0;
|
||||
}
|
||||
|
||||
protected getSeverityColor(_value: number): string {
|
||||
return "#ef4444";
|
||||
}
|
||||
|
||||
protected getSeveritySize(_value: number): number {
|
||||
return 10;
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Subclass that DOES override mapWebsocketPayload ─────────────────────────
|
||||
|
||||
const FIXED_ENTITIES = [makeEntity("override-1"), makeEntity("override-2")];
|
||||
|
||||
class OverridingIncidentPlugin extends MinimalIncidentPlugin {
|
||||
id = "overriding-incident-plugin";
|
||||
|
||||
mapWebsocketPayload(_payload: unknown, _existingEntities?: GeoEntity[]): GeoEntity[] {
|
||||
return FIXED_ENTITIES;
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Helpers that mimic WsClient.handleDataMessage branching logic ────────────
|
||||
// We test the plugin method in isolation (unit test), not the full WsClient,
|
||||
// because WsClient has DOM/Zustand/DataBus dependencies. This mirrors the exact
|
||||
// logic from WsClient lines 172-182.
|
||||
|
||||
function simulateWsClientDispatch(
|
||||
plugin: BaseIncidentPlugin,
|
||||
rawPayload: unknown
|
||||
): GeoEntity[] | null {
|
||||
if (typeof (plugin as any).mapWebsocketPayload === "function") {
|
||||
return (plugin as any).mapWebsocketPayload(rawPayload, []);
|
||||
}
|
||||
if (!Array.isArray(rawPayload)) {
|
||||
// This is the silent-drop branch — return null to signal "dropped"
|
||||
return null;
|
||||
}
|
||||
return (rawPayload as GeoEntity[]).map((e) => ({
|
||||
...e,
|
||||
timestamp: new Date(e.timestamp || Date.now()),
|
||||
}));
|
||||
}
|
||||
|
||||
// ─── Test suite ───────────────────────────────────────────────────────────────
|
||||
|
||||
describe("BaseIncidentPlugin — WebSocket payload contract", () => {
|
||||
|
||||
describe("scheduler-wrapped payload { source, fetchedAt, items, totalCount }", () => {
|
||||
it(
|
||||
"unwraps { source, fetchedAt, items, totalCount } and returns items as GeoEntity[]",
|
||||
() => {
|
||||
const plugin = new MinimalIncidentPlugin();
|
||||
const entity1 = makeEntity("eq-1");
|
||||
const entity2 = makeEntity("eq-2");
|
||||
|
||||
const wrappedPayload = {
|
||||
source: "usgs-earthquake-feed",
|
||||
fetchedAt: "2025-01-01T00:00:00Z",
|
||||
items: [entity1, entity2],
|
||||
totalCount: 2,
|
||||
};
|
||||
|
||||
// After the fix: mapWebsocketPayload should exist and return both entities
|
||||
const result = simulateWsClientDispatch(plugin, wrappedPayload);
|
||||
|
||||
expect(result).not.toBeNull();
|
||||
expect(Array.isArray(result)).toBe(true);
|
||||
expect(result).toHaveLength(2);
|
||||
expect(result![0].id).toBe("eq-1");
|
||||
expect(result![1].id).toBe("eq-2");
|
||||
}
|
||||
);
|
||||
|
||||
});
|
||||
|
||||
describe("flat GeoEntity[] payload", () => {
|
||||
/**
|
||||
* Flat arrays bypass mapWebsocketPayload entirely in WsClient — they work today.
|
||||
* This test MUST stay green before and after the fix.
|
||||
*/
|
||||
it("passes flat GeoEntity[] through unchanged (current working behaviour)", () => {
|
||||
const plugin = new MinimalIncidentPlugin();
|
||||
const entities = [makeEntity("eq-flat-1"), makeEntity("eq-flat-2")];
|
||||
|
||||
const result = simulateWsClientDispatch(plugin, entities);
|
||||
|
||||
expect(result).not.toBeNull();
|
||||
expect(result).toHaveLength(2);
|
||||
expect(result![0].id).toBe("eq-flat-1");
|
||||
expect(result![1].id).toBe("eq-flat-2");
|
||||
});
|
||||
|
||||
it("normalises timestamp strings to Date objects for flat array payloads", () => {
|
||||
const plugin = new MinimalIncidentPlugin();
|
||||
const raw = [{
|
||||
...makeEntity("eq-ts-1"),
|
||||
// Simulate a timestamp that arrives over the wire as a string
|
||||
timestamp: "2025-06-01T12:00:00Z" as unknown as Date,
|
||||
}];
|
||||
|
||||
const result = simulateWsClientDispatch(plugin, raw);
|
||||
|
||||
expect(result).not.toBeNull();
|
||||
expect(result![0].timestamp).toBeInstanceOf(Date);
|
||||
});
|
||||
});
|
||||
|
||||
describe("GeoJSON { features: [...] } payload", () => {
|
||||
it(
|
||||
"normalises a GeoJSON FeatureCollection into GeoEntity[]",
|
||||
() => {
|
||||
const plugin = new MinimalIncidentPlugin();
|
||||
|
||||
const geojsonPayload = {
|
||||
type: "FeatureCollection",
|
||||
features: [
|
||||
{
|
||||
type: "Feature",
|
||||
geometry: { type: "Point", coordinates: [139.65, 35.68] },
|
||||
properties: { id: "gj-1", magnitude: 4.5 },
|
||||
},
|
||||
{
|
||||
type: "Feature",
|
||||
geometry: { type: "Point", coordinates: [135.50, 34.69] },
|
||||
properties: { id: "gj-2", magnitude: 3.1 },
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
const result = simulateWsClientDispatch(plugin, geojsonPayload);
|
||||
|
||||
expect(result).not.toBeNull();
|
||||
expect(Array.isArray(result)).toBe(true);
|
||||
expect(result).toHaveLength(2);
|
||||
// Each feature should have been converted to a GeoEntity with lat/lon
|
||||
expect(result![0].latitude).toBeCloseTo(35.68, 2);
|
||||
expect(result![0].longitude).toBeCloseTo(139.65, 2);
|
||||
}
|
||||
);
|
||||
|
||||
});
|
||||
|
||||
describe("subclass override of mapWebsocketPayload", () => {
|
||||
/**
|
||||
* A subclass that implements mapWebsocketPayload is handled correctly TODAY.
|
||||
* This covers the "conflict-events" / "iranwarlive" pattern and must stay green.
|
||||
*/
|
||||
it("calls the subclass override and returns its result regardless of payload shape", () => {
|
||||
const plugin = new OverridingIncidentPlugin();
|
||||
|
||||
// Even if the payload is an object (which would normally be dropped), the
|
||||
// subclass override is respected.
|
||||
const wrappedPayload = {
|
||||
source: "conflict-api",
|
||||
fetchedAt: "2025-01-01T00:00:00Z",
|
||||
items: [makeEntity("conflict-1")],
|
||||
totalCount: 1,
|
||||
};
|
||||
|
||||
const result = simulateWsClientDispatch(plugin, wrappedPayload);
|
||||
|
||||
// Should return the overriding subclass's fixed result, not the raw items
|
||||
expect(result).toStrictEqual(FIXED_ENTITIES);
|
||||
});
|
||||
|
||||
it("override is also called for flat array payloads, taking precedence over default handling", () => {
|
||||
const plugin = new OverridingIncidentPlugin();
|
||||
const flatPayload = [makeEntity("flat-1"), makeEntity("flat-2"), makeEntity("flat-3")];
|
||||
|
||||
const result = simulateWsClientDispatch(plugin, flatPayload);
|
||||
|
||||
// The override always wins — result is FIXED_ENTITIES, not the 3-element flat array
|
||||
expect(result).toStrictEqual(FIXED_ENTITIES);
|
||||
expect(result).toHaveLength(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe("BaseIncidentPlugin structural contract", () => {
|
||||
it("has mapWebsocketPayload defined", () => {
|
||||
const plugin = new MinimalIncidentPlugin();
|
||||
expect(typeof (plugin as any).mapWebsocketPayload).toBe("function");
|
||||
});
|
||||
|
||||
it("implements the required WorldPlugin lifecycle methods", () => {
|
||||
const plugin = new MinimalIncidentPlugin();
|
||||
expect(typeof plugin.initialize).toBe("function");
|
||||
expect(typeof plugin.destroy).toBe("function");
|
||||
expect(typeof plugin.fetch).toBe("function");
|
||||
expect(typeof plugin.getPollingInterval).toBe("function");
|
||||
expect(typeof plugin.getLayerConfig).toBe("function");
|
||||
expect(typeof plugin.renderEntity).toBe("function");
|
||||
});
|
||||
|
||||
it("returns polling interval of 0 (WebSocket-only mode)", () => {
|
||||
const plugin = new MinimalIncidentPlugin();
|
||||
expect(plugin.getPollingInterval()).toBe(0);
|
||||
});
|
||||
|
||||
it("returns default layer config with clustering enabled", () => {
|
||||
const plugin = new MinimalIncidentPlugin();
|
||||
const config = plugin.getLayerConfig();
|
||||
expect(config.clusterEnabled).toBe(true);
|
||||
expect(config.color).toBe("#ef4444");
|
||||
expect(typeof config.clusterDistance).toBe("number");
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -80,4 +80,69 @@ export abstract class BaseIncidentPlugin implements WorldPlugin {
|
||||
labelFont: "11px JetBrains Mono, monospace" // Adds optional label standard
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Default WebSocket payload normaliser. WsClient calls this when a data message
|
||||
* arrives; if absent, WsClient silently drops anything that isn't a flat array.
|
||||
*
|
||||
* Handles three shapes:
|
||||
* - Scheduler envelope: { source, fetchedAt, items: GeoEntity[], totalCount } → items
|
||||
* - GeoJSON FeatureCollection: { features: [{geometry, properties}, ...] } → normalised
|
||||
* - Flat array: passes through (with timestamp normalisation)
|
||||
*
|
||||
* Subclasses may override for domain-specific shapes; this default covers
|
||||
* the common scheduler-wrapped and GeoJSON cases so the bug doesn't recur.
|
||||
*
|
||||
* @param payload - Raw payload received from the WebSocket message
|
||||
* @param _existingEntities - Current entities in the layer (unused by default; available for subclass merging logic)
|
||||
* @returns Normalised GeoEntity[] ready for the layer renderer
|
||||
*/
|
||||
mapWebsocketPayload(payload: any, _existingEntities?: GeoEntity[]): GeoEntity[] {
|
||||
const items = this.extractIncidentItems(payload);
|
||||
return items.map((e) => ({
|
||||
...e,
|
||||
timestamp: e.timestamp instanceof Date ? e.timestamp : new Date(e.timestamp ?? Date.now()),
|
||||
}));
|
||||
}
|
||||
|
||||
/**
|
||||
* Extracts a raw GeoEntity array from the three common payload shapes:
|
||||
* scheduler envelope, GeoJSON FeatureCollection, or flat array.
|
||||
*
|
||||
* @param payload - The raw WebSocket payload
|
||||
* @returns Unnormalised array of GeoEntity-like objects (timestamps not yet coerced)
|
||||
*/
|
||||
protected extractIncidentItems(payload: any): GeoEntity[] {
|
||||
if (Array.isArray(payload)) {
|
||||
return payload as GeoEntity[];
|
||||
}
|
||||
if (payload && Array.isArray(payload.items)) {
|
||||
return payload.items as GeoEntity[];
|
||||
}
|
||||
if (payload && Array.isArray(payload.features)) {
|
||||
return payload.features.map((f: any, i: number) => this.geoJsonFeatureToEntity(f, i));
|
||||
}
|
||||
return [];
|
||||
}
|
||||
|
||||
/**
|
||||
* Converts a GeoJSON Feature object into a GeoEntity.
|
||||
*
|
||||
* @param feature - GeoJSON Feature with geometry.coordinates and optional properties
|
||||
* @param index - Position in the parent features array, used to generate a fallback id
|
||||
* @returns A GeoEntity with lat/lon extracted from the feature geometry
|
||||
*/
|
||||
protected geoJsonFeatureToEntity(feature: any, index: number): GeoEntity {
|
||||
const coords = feature?.geometry?.coordinates ?? [0, 0];
|
||||
const props = feature?.properties ?? {};
|
||||
return {
|
||||
id: props.id ?? `${this.id}-${index}`,
|
||||
pluginId: this.id,
|
||||
latitude: coords[1],
|
||||
longitude: coords[0],
|
||||
timestamp: props.timestamp ? new Date(props.timestamp) : new Date(),
|
||||
properties: props,
|
||||
label: props.label,
|
||||
};
|
||||
}
|
||||
}
|
||||
+26
-8
@@ -1,4 +1,9 @@
|
||||
import { defineConfig, devices } from '@playwright/test';
|
||||
import path from 'path';
|
||||
import { fileURLToPath } from 'url';
|
||||
|
||||
const __dir = path.dirname(fileURLToPath(import.meta.url));
|
||||
const MARKETPLACE_DIR = path.resolve(__dir, '../worldwideview-marketplace');
|
||||
|
||||
/**
|
||||
* See https://playwright.dev/docs/test-configuration.
|
||||
@@ -61,12 +66,25 @@ export default defineConfig({
|
||||
},
|
||||
],
|
||||
|
||||
/* Run your local dev server before starting the tests */
|
||||
webServer: {
|
||||
command: 'pnpm dev',
|
||||
env: { PORT: '3001' },
|
||||
url: 'http://localhost:3001',
|
||||
reuseExistingServer: !process.env.CI,
|
||||
timeout: 120 * 1000,
|
||||
},
|
||||
/* Run dev servers before starting the tests */
|
||||
webServer: [
|
||||
{
|
||||
command: 'pnpm dev',
|
||||
env: {
|
||||
PORT: '3001',
|
||||
NEXT_PUBLIC_MARKETPLACE_URL: 'http://localhost:3002',
|
||||
},
|
||||
url: 'http://localhost:3001',
|
||||
reuseExistingServer: !process.env.CI,
|
||||
timeout: 120 * 1000,
|
||||
},
|
||||
{
|
||||
command: 'pnpm dev',
|
||||
cwd: MARKETPLACE_DIR,
|
||||
env: { PORT: '3002' },
|
||||
url: 'http://localhost:3002',
|
||||
reuseExistingServer: !process.env.CI,
|
||||
timeout: 120 * 1000,
|
||||
},
|
||||
],
|
||||
});
|
||||
@@ -0,0 +1,46 @@
|
||||
/**
|
||||
* Marketplace-only E2E config.
|
||||
* Runs against the marketplace dev server on port 3002 only.
|
||||
* No worldwideview auth or Docker/Postgres required.
|
||||
*
|
||||
* Usage:
|
||||
* pnpm exec playwright test --config=playwright.marketplace.config.ts
|
||||
*/
|
||||
import { defineConfig, devices } from '@playwright/test';
|
||||
import path from 'path';
|
||||
import { fileURLToPath } from 'url';
|
||||
|
||||
const __dir = path.dirname(fileURLToPath(import.meta.url));
|
||||
const MARKETPLACE_DIR = path.resolve(__dir, '../worldwideview-marketplace');
|
||||
|
||||
export default defineConfig({
|
||||
timeout: 30000,
|
||||
expect: { timeout: 10000 },
|
||||
testDir: './tests',
|
||||
testMatch: '**/marketplace-from-instance.spec.ts',
|
||||
fullyParallel: false,
|
||||
retries: 0,
|
||||
workers: 1,
|
||||
reporter: 'list',
|
||||
|
||||
use: {
|
||||
baseURL: 'http://localhost:3002',
|
||||
trace: 'on-first-retry',
|
||||
},
|
||||
|
||||
projects: [
|
||||
{
|
||||
name: 'chromium',
|
||||
use: { ...devices['Desktop Chrome'] },
|
||||
},
|
||||
],
|
||||
|
||||
webServer: {
|
||||
command: 'pnpm dev',
|
||||
cwd: MARKETPLACE_DIR,
|
||||
env: { PORT: '3002' },
|
||||
url: 'http://localhost:3002',
|
||||
reuseExistingServer: true,
|
||||
timeout: 90 * 1000,
|
||||
},
|
||||
});
|
||||
+301
-75
@@ -1,36 +1,29 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* gc-scan.mjs — Garbage Collector scanner for worldwideview
|
||||
*
|
||||
* Scans the repository for:
|
||||
* 1. Stale branches (merged or idle > STALE_BRANCH_DAYS)
|
||||
* 2. TODO/FIXME/HACK comments older than STALE_COMMENT_DAYS (via git blame)
|
||||
* 3. Orphaned workspace packages (zero reverse-deps inside the monorepo)
|
||||
* 4. Very old Prisma migrations (> 1 year untouched)
|
||||
*
|
||||
* Outputs a JSON report to stdout.
|
||||
* The workflow decides whether to open Issues/PRs based on dry_run flag.
|
||||
*
|
||||
* Usage:
|
||||
* node scripts/gc-scan.mjs [--days-branch=90] [--days-comment=180]
|
||||
*/
|
||||
// gc-scan.mjs — deterministic garbage detection for the GC bot
|
||||
// Zero npm dependencies. Emits gc-findings.json consumed by the garbage-collector agent.
|
||||
//
|
||||
// Usage: node scripts/gc-scan.mjs [--out <path>] [--days-branch=90] [--days-comment=180]
|
||||
|
||||
import { execSync } from 'node:child_process';
|
||||
import { existsSync } from 'node:fs';
|
||||
import { resolve, relative } from 'node:path';
|
||||
import { argv, cwd } from 'node:process';
|
||||
import { writeFileSync, readFileSync, existsSync } from 'node:fs';
|
||||
import { join, resolve, relative } from 'node:path';
|
||||
|
||||
// ─── Tuneable thresholds ──────────────────────────────────────────────────────
|
||||
const ROOT = process.cwd();
|
||||
|
||||
// Parse CLI args: --out <path>, --days-branch=N, --days-comment=N
|
||||
const ARGS = Object.fromEntries(
|
||||
argv.slice(2).flatMap(a => {
|
||||
process.argv.slice(2).flatMap(a => {
|
||||
const m = a.match(/^--([\w-]+)=(.+)$/);
|
||||
return m ? [[m[1], m[2]]] : [];
|
||||
})
|
||||
);
|
||||
|
||||
const outIdx = process.argv.indexOf('--out');
|
||||
const OUT = outIdx !== -1 ? process.argv[outIdx + 1] : (ARGS['out'] ?? 'gc-findings.json');
|
||||
const STALE_BRANCH_DAYS = Number(ARGS['days-branch'] ?? 90);
|
||||
const STALE_COMMENT_DAYS = Number(ARGS['days-comment'] ?? 180);
|
||||
const ROOT = cwd();
|
||||
|
||||
/** @type {Array<Record<string, unknown>>} */
|
||||
const findings = [];
|
||||
|
||||
function run(cmd) {
|
||||
try {
|
||||
@@ -40,14 +33,17 @@ function run(cmd) {
|
||||
}
|
||||
}
|
||||
|
||||
function add(finding) {
|
||||
findings.push(finding);
|
||||
}
|
||||
|
||||
function daysAgo(isoDate) {
|
||||
if (!isoDate) return Infinity;
|
||||
return (Date.now() - new Date(isoDate).getTime()) / 86_400_000;
|
||||
}
|
||||
|
||||
// ─── 1. Stale branches ────────────────────────────────────────────────────────
|
||||
function scanStaleBranches() {
|
||||
const findings = [];
|
||||
// ─── Detector 1: Stale branches ──────────────────────────────────────────────
|
||||
function detectStaleBranches() {
|
||||
const out = run(
|
||||
'git for-each-ref --sort=committerdate refs/remotes/origin ' +
|
||||
'--format=%(refname:short)|%(committerdate:iso8601)|%(authorname)|%(subject)'
|
||||
@@ -60,54 +56,204 @@ function scanStaleBranches() {
|
||||
if (age < STALE_BRANCH_DAYS) continue;
|
||||
const mergeBase = run(`git merge-base origin/main "${ref}" 2>/dev/null`);
|
||||
const branchHead = run(`git rev-parse "${ref}"`);
|
||||
findings.push({
|
||||
const isMerged = mergeBase === branchHead;
|
||||
add({
|
||||
type: 'stale-branch',
|
||||
tier: 'B',
|
||||
branch,
|
||||
author,
|
||||
lastCommitDate: date?.trim(),
|
||||
ageDays: Math.floor(age),
|
||||
isMerged: mergeBase === branchHead,
|
||||
isMerged,
|
||||
lastSubject: subject,
|
||||
description: `Branch '${branch}' is ${Math.floor(age)} days old${isMerged ? ' and fully merged into main — safe to delete' : ' with no recent activity'}`,
|
||||
});
|
||||
}
|
||||
return findings;
|
||||
}
|
||||
|
||||
// ─── 2. Stale TODO/FIXME/HACK comments ───────────────────────────────────────
|
||||
function scanStaleComments() {
|
||||
const findings = [];
|
||||
const grepResult = run(
|
||||
'git grep -rn --include=*.ts --include=*.tsx --include=*.mjs --include=*.js ' +
|
||||
'-E "TODO|FIXME|HACK|XXX" -- . :(exclude)node_modules :(exclude).next :(exclude)dist :(exclude)prisma/migrations'
|
||||
// ─── Detector 2: Stale TODO/FIXME/HACK comments ──────────────────────────────
|
||||
function detectStaleTodos() {
|
||||
const raw = run(
|
||||
'git grep -rn --fixed-strings -e "TODO" -e "FIXME" -e "HACK" -e "XXX" ' +
|
||||
'-- "*.ts" "*.tsx" "*.mjs" "*.js"'
|
||||
);
|
||||
for (const line of grepResult.split('\n').filter(Boolean).slice(0, 200)) {
|
||||
const m = line.match(/^([^:]+):(\d+):(.+)$/);
|
||||
if (!m) continue;
|
||||
const [, file, lineNum, text] = m;
|
||||
const blameOut = run(`git blame -L ${lineNum},${lineNum} --porcelain "${file}" 2>/dev/null`);
|
||||
const authorTimeMatch = blameOut.match(/author-time (\d+)/);
|
||||
if (!authorTimeMatch) continue;
|
||||
const age = (Date.now() - Number(authorTimeMatch[1]) * 1000) / 86_400_000;
|
||||
if (age < STALE_COMMENT_DAYS) continue;
|
||||
const authorMatch = blameOut.match(/^author (.+)$/m);
|
||||
findings.push({
|
||||
type: 'stale-comment',
|
||||
file: relative(ROOT, resolve(ROOT, file)),
|
||||
line: Number(lineNum),
|
||||
text: text.trim().slice(0, 120),
|
||||
author: authorMatch?.[1] ?? 'unknown',
|
||||
ageDays: Math.floor(age),
|
||||
if (!raw) return;
|
||||
|
||||
// Cap at 40 matches to keep git blame calls fast on large repos
|
||||
const lines = raw.split('\n').filter(Boolean).slice(0, 40);
|
||||
|
||||
for (const line of lines) {
|
||||
const firstColon = line.indexOf(':');
|
||||
if (firstColon === -1) continue;
|
||||
const afterFile = line.slice(firstColon + 1);
|
||||
const secondColon = afterFile.indexOf(':');
|
||||
if (secondColon === -1) continue;
|
||||
|
||||
const file = line.slice(0, firstColon);
|
||||
const lineNum = afterFile.slice(0, secondColon);
|
||||
const content = afterFile.slice(secondColon + 1).trim();
|
||||
|
||||
if (file.startsWith('node_modules/') || file.includes('public/cesium')) continue;
|
||||
|
||||
const blameOut = run(`git blame -L ${lineNum},${lineNum} --porcelain "${file}"`);
|
||||
const timeMatch = blameOut.match(/^author-time (\d+)/m);
|
||||
if (!timeMatch) continue;
|
||||
|
||||
const ageInDays = (Date.now() - parseInt(timeMatch[1], 10) * 1000) / 86_400_000;
|
||||
if (ageInDays < STALE_COMMENT_DAYS) continue;
|
||||
|
||||
const issueMatch = content.match(/#(\d+)/);
|
||||
let issueClosed = false;
|
||||
if (issueMatch) {
|
||||
const state = run(`gh issue view ${issueMatch[1]} --json state -q .state 2>/dev/null`);
|
||||
issueClosed = state.toUpperCase() === 'CLOSED';
|
||||
}
|
||||
|
||||
add({
|
||||
type: 'stale-todo',
|
||||
tier: 'A',
|
||||
file,
|
||||
line: parseInt(lineNum, 10),
|
||||
content: content.slice(0, 200),
|
||||
ageInDays: Math.round(ageInDays),
|
||||
issueClosed,
|
||||
linkedIssue: issueMatch ? issueMatch[1] : null,
|
||||
description: issueClosed
|
||||
? `TODO references closed issue #${issueMatch[1]} (${Math.round(ageInDays)}d old) — safe to delete`
|
||||
: `TODO/comment is ${Math.round(ageInDays)} days old — review and resolve or delete`,
|
||||
});
|
||||
}
|
||||
return findings;
|
||||
}
|
||||
|
||||
// ─── 3. Orphaned workspace packages ──────────────────────────────────────────
|
||||
function scanOrphanedPackages() {
|
||||
const findings = [];
|
||||
// ─── Detector 3: Known anti-patterns ─────────────────────────────────────────
|
||||
const ANTI_PATTERNS = [
|
||||
{
|
||||
id: 'mdc-ref',
|
||||
tier: 'A',
|
||||
globs: ['*.ts', '*.tsx', '*.md', '*.mjs'],
|
||||
pattern: '\\.mdc',
|
||||
desc: '.mdc file reference — must be .md (project invariant: never .mdc files)',
|
||||
},
|
||||
{
|
||||
id: 'console-log',
|
||||
tier: 'A',
|
||||
globs: ['*.ts', '*.tsx'],
|
||||
pattern: 'console\\.log(',
|
||||
excludePrefix: 'scripts/',
|
||||
desc: 'stray console.log — remove or replace with structured logging',
|
||||
},
|
||||
{
|
||||
id: 'ts-ignore',
|
||||
tier: 'A',
|
||||
globs: ['*.ts', '*.tsx'],
|
||||
pattern: '@ts-ignore',
|
||||
desc: '@ts-ignore suppresses real type errors — remove and fix the underlying type',
|
||||
},
|
||||
{
|
||||
id: 'ts-nocheck',
|
||||
tier: 'A',
|
||||
globs: ['*.ts', '*.tsx'],
|
||||
pattern: '@ts-nocheck',
|
||||
desc: '@ts-nocheck disables all type checking for the file — remove and fix types',
|
||||
},
|
||||
{
|
||||
id: 'hardcoded-url',
|
||||
tier: 'B',
|
||||
globs: ['*.ts', '*.tsx'],
|
||||
pattern: 'localhost:500',
|
||||
excludePrefix: 'scripts/',
|
||||
desc: 'hardcoded engine URL (localhost:5001/5000) — use env var or plugin streamUrl instead',
|
||||
},
|
||||
{
|
||||
id: 'deprecated',
|
||||
tier: 'B',
|
||||
globs: ['*.ts', '*.tsx'],
|
||||
pattern: '@deprecated',
|
||||
desc: '@deprecated JSDoc on a symbol — audit whether all callers have migrated',
|
||||
},
|
||||
];
|
||||
|
||||
function detectAntiPatterns() {
|
||||
for (const { id, tier, globs, pattern, desc, excludePrefix } of ANTI_PATTERNS) {
|
||||
const globArgs = globs.map(g => `"${g}"`).join(' ');
|
||||
const raw = run(`git grep -En "${pattern}" -- ${globArgs}`);
|
||||
if (!raw) continue;
|
||||
|
||||
const hits = raw
|
||||
.split('\n')
|
||||
.filter(l => l && !(excludePrefix && l.startsWith(excludePrefix)))
|
||||
.slice(0, 15);
|
||||
|
||||
for (const line of hits) {
|
||||
const m = line.match(/^([^:]+):(\d+):(.*)/);
|
||||
if (!m) continue;
|
||||
if (m[1].startsWith('node_modules/') || m[1].includes('public/cesium')) continue;
|
||||
add({
|
||||
type: 'anti-pattern',
|
||||
patternId: id,
|
||||
tier,
|
||||
file: m[1],
|
||||
line: parseInt(m[2], 10),
|
||||
content: m[3].trim().slice(0, 200),
|
||||
description: desc,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Detector 4: Oversized source files ──────────────────────────────────────
|
||||
function detectOversizedFiles() {
|
||||
const fileList = run('git ls-files "*.ts" "*.tsx"').split('\n').filter(Boolean);
|
||||
for (const file of fileList) {
|
||||
if (file.startsWith('node_modules/') || file.includes('public/cesium') || file.includes('.next/')) continue;
|
||||
const fullPath = join(ROOT, file);
|
||||
if (!existsSync(fullPath)) continue;
|
||||
const lineCount = readFileSync(fullPath, 'utf8').split('\n').length;
|
||||
if (lineCount > 350) {
|
||||
add({
|
||||
type: 'oversized-file',
|
||||
tier: 'B',
|
||||
file,
|
||||
lines: lineCount,
|
||||
description: `${lineCount} lines exceeds ~300-line convention — extract helpers, split component, or use hooks`,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Detector 5: Orphaned rule file references ───────────────────────────────
|
||||
function detectOrphanedRuleRefs() {
|
||||
const ruleFiles = run('git ls-files ".agents/rules/"').split('\n').filter(f => f.endsWith('.md'));
|
||||
for (const ruleFile of ruleFiles) {
|
||||
const fullPath = join(ROOT, ruleFile);
|
||||
if (!existsSync(fullPath)) continue;
|
||||
const content = readFileSync(fullPath, 'utf8');
|
||||
const seen = new Set();
|
||||
for (const [, ref] of content.matchAll(
|
||||
/`((?:src|packages|local-plugins|local-seeders|\.agents)\/[^`\s]+)`/g
|
||||
)) {
|
||||
const base = ref.replace(/\/\*\*.*/, '').replace(/\/\*.*/, '').replace(/\{[^}]+\}.*/, '');
|
||||
if (!base || seen.has(base)) continue;
|
||||
seen.add(base);
|
||||
const tracked = run(`git ls-files "${base}" "${base}/" 2>/dev/null`);
|
||||
if (!tracked && !existsSync(join(ROOT, base))) {
|
||||
add({
|
||||
type: 'orphaned-rule-ref',
|
||||
tier: 'B',
|
||||
file: ruleFile,
|
||||
ref: base,
|
||||
description: `Rule file references '${base}' which no longer exists — update or remove`,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Detector 6: Orphaned workspace packages ─────────────────────────────────
|
||||
function detectOrphanedPackages() {
|
||||
try {
|
||||
const wsInfo = run('pnpm list --recursive --depth=0 --json 2>/dev/null');
|
||||
if (!wsInfo) return findings;
|
||||
if (!wsInfo) return;
|
||||
const pkgs = JSON.parse(wsInfo);
|
||||
const allNames = new Set(pkgs.map(p => p.name).filter(Boolean));
|
||||
const reverseDepCount = Object.fromEntries([...allNames].map(n => [n, 0]));
|
||||
@@ -120,18 +266,22 @@ function scanOrphanedPackages() {
|
||||
const pkg = pkgs.find(p => p.name === name);
|
||||
if (!pkg?.path?.includes('/packages/')) continue;
|
||||
if (count === 0) {
|
||||
findings.push({ type: 'orphaned-package', name, path: relative(ROOT, pkg.path) });
|
||||
add({
|
||||
type: 'orphaned-package',
|
||||
tier: 'B',
|
||||
name,
|
||||
path: relative(ROOT, pkg.path),
|
||||
description: `Workspace package '${name}' has no reverse dependencies inside the monorepo — consider removing`,
|
||||
});
|
||||
}
|
||||
}
|
||||
} catch { /* skip */ }
|
||||
return findings;
|
||||
} catch { /* skip if pnpm unavailable */ }
|
||||
}
|
||||
|
||||
// ─── 4. Old Prisma migrations ─────────────────────────────────────────────────
|
||||
function scanOldMigrations() {
|
||||
const findings = [];
|
||||
// ─── Detector 7: Old Prisma migrations ───────────────────────────────────────
|
||||
function detectOldMigrations() {
|
||||
const migrationsDir = resolve(ROOT, 'prisma/migrations');
|
||||
if (!existsSync(migrationsDir)) return findings;
|
||||
if (!existsSync(migrationsDir)) return;
|
||||
const dirs = run(`ls -1 "${migrationsDir}"`).split('\n').filter(Boolean);
|
||||
for (const dir of dirs) {
|
||||
const sqlFile = `prisma/migrations/${dir}/migration.sql`;
|
||||
@@ -139,26 +289,102 @@ function scanOldMigrations() {
|
||||
if (!lastTouch) continue;
|
||||
const age = daysAgo(lastTouch);
|
||||
if (age < 365) continue;
|
||||
findings.push({
|
||||
add({
|
||||
type: 'old-migration',
|
||||
tier: 'B',
|
||||
migration: dir,
|
||||
ageDays: Math.floor(age),
|
||||
lastTouched: lastTouch.trim(),
|
||||
description: `Migration '${dir}' is ${Math.floor(age)} days old — verify it is still actively needed`,
|
||||
});
|
||||
}
|
||||
return findings;
|
||||
}
|
||||
|
||||
// ─── Main ─────────────────────────────────────────────────────────────────────
|
||||
const report = {
|
||||
generatedAt: new Date().toISOString(),
|
||||
// ─── Detector 8: Outdated dependencies ───────────────────────────────────────
|
||||
function detectOutdatedDeps() {
|
||||
if (!existsSync(join(ROOT, 'package.json'))) return;
|
||||
const raw = run('pnpm outdated --no-color 2>/dev/null || true');
|
||||
if (!raw) return;
|
||||
const dataLines = raw
|
||||
.split('\n')
|
||||
.filter(l => l.trim() && !l.startsWith('Package') && !l.startsWith('Legend') && !l.startsWith(' '));
|
||||
for (const line of dataLines.slice(0, 12)) {
|
||||
const cols = line.trim().split(/\s+/);
|
||||
if (cols.length < 3) continue;
|
||||
const [pkg, current, latest] = cols;
|
||||
if (!pkg || !current || !latest || current === latest) continue;
|
||||
const isMajor = current.replace(/^[^0-9]*/, '').split('.')[0] !== latest.replace(/^[^0-9]*/, '').split('.')[0];
|
||||
add({
|
||||
type: 'outdated-dep',
|
||||
tier: 'B',
|
||||
package: pkg,
|
||||
current,
|
||||
latest,
|
||||
isMajor,
|
||||
description: `${pkg}: ${current} → ${latest}${isMajor ? ' ⚠ MAJOR bump — review breaking changes before upgrading' : ''}`,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Run all detectors ────────────────────────────────────────────────────────
|
||||
console.log('WorldWideView GC Scan');
|
||||
console.log(`Repo: ${run('git remote get-url origin') || ROOT}`);
|
||||
console.log(`Commit: ${run('git rev-parse --short HEAD') || 'unknown'}`);
|
||||
console.log(`Thresholds: branch=${STALE_BRANCH_DAYS}d comment=${STALE_COMMENT_DAYS}d`);
|
||||
console.log('');
|
||||
|
||||
detectStaleBranches();
|
||||
console.log(` stale-branch ${findings.filter(f => f.type === 'stale-branch').length}`);
|
||||
|
||||
detectStaleTodos();
|
||||
console.log(` stale-todo ${findings.filter(f => f.type === 'stale-todo').length}`);
|
||||
|
||||
detectAntiPatterns();
|
||||
console.log(` anti-pattern ${findings.filter(f => f.type === 'anti-pattern').length}`);
|
||||
|
||||
detectOversizedFiles();
|
||||
console.log(` oversized-file ${findings.filter(f => f.type === 'oversized-file').length}`);
|
||||
|
||||
detectOrphanedRuleRefs();
|
||||
console.log(` orphaned-rule-ref ${findings.filter(f => f.type === 'orphaned-rule-ref').length}`);
|
||||
|
||||
detectOrphanedPackages();
|
||||
console.log(` orphaned-package ${findings.filter(f => f.type === 'orphaned-package').length}`);
|
||||
|
||||
detectOldMigrations();
|
||||
console.log(` old-migration ${findings.filter(f => f.type === 'old-migration').length}`);
|
||||
|
||||
detectOutdatedDeps();
|
||||
console.log(` outdated-dep ${findings.filter(f => f.type === 'outdated-dep').length}`);
|
||||
|
||||
// Deduplicate
|
||||
const seen = new Set();
|
||||
const deduped = findings.filter(f => {
|
||||
const key = [
|
||||
f.type,
|
||||
f.file ?? f.package ?? f.name ?? f.migration ?? f.branch ?? '',
|
||||
f.line ?? f.ref ?? '',
|
||||
f.patternId ?? '',
|
||||
].join(':');
|
||||
if (seen.has(key)) return false;
|
||||
seen.add(key);
|
||||
return true;
|
||||
});
|
||||
|
||||
const output = {
|
||||
scanDate: new Date().toISOString(),
|
||||
thresholds: { STALE_BRANCH_DAYS, STALE_COMMENT_DAYS },
|
||||
findings: [
|
||||
...scanStaleBranches(),
|
||||
...scanStaleComments(),
|
||||
...scanOrphanedPackages(),
|
||||
...scanOldMigrations(),
|
||||
],
|
||||
repo: run('git remote get-url origin') || ROOT,
|
||||
commit: run('git rev-parse --short HEAD') || 'unknown',
|
||||
summary: {
|
||||
total: deduped.length,
|
||||
tierA: deduped.filter(f => f.tier === 'A').length,
|
||||
tierB: deduped.filter(f => f.tier === 'B').length,
|
||||
},
|
||||
findings: deduped,
|
||||
};
|
||||
|
||||
console.log(JSON.stringify(report, null, 2));
|
||||
writeFileSync(OUT, JSON.stringify(output, null, 2));
|
||||
console.log('');
|
||||
console.log(`Total: ${deduped.length} (Tier A: ${output.summary.tierA}, Tier B: ${output.summary.tierB})`);
|
||||
console.log(`Output: ${OUT}`);
|
||||
@@ -67,7 +67,7 @@ export async function GET(req: NextRequest) {
|
||||
httpOnly: true,
|
||||
secure: isHttps,
|
||||
sameSite: "lax",
|
||||
path: "/api/marketplace/callback",
|
||||
path: "/", // must match the path used at Set-Cookie time
|
||||
maxAge: 0
|
||||
});
|
||||
|
||||
@@ -75,7 +75,7 @@ export async function GET(req: NextRequest) {
|
||||
httpOnly: true,
|
||||
secure: isHttps,
|
||||
sameSite: "lax",
|
||||
path: "/api/marketplace/callback",
|
||||
path: "/", // must match the path used at Set-Cookie time
|
||||
maxAge: 0
|
||||
});
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ export async function GET(req: NextRequest) {
|
||||
httpOnly: true,
|
||||
secure: isHttps,
|
||||
sameSite: "lax",
|
||||
path: "/api/marketplace/callback",
|
||||
path: "/", // __Host- prefix requires path="/" per RFC 6265bis
|
||||
maxAge: 60 * 10 // 10 minutes
|
||||
});
|
||||
|
||||
@@ -34,7 +34,7 @@ export async function GET(req: NextRequest) {
|
||||
httpOnly: true,
|
||||
secure: isHttps,
|
||||
sameSite: "lax",
|
||||
path: "/api/marketplace/callback",
|
||||
path: "/", // __Host- prefix requires path="/" per RFC 6265bis
|
||||
maxAge: 60 * 10 // 10 minutes
|
||||
});
|
||||
|
||||
|
||||
@@ -56,7 +56,7 @@ describe("PKCE Flow", () => {
|
||||
expect(cookies).toContain("HttpOnly");
|
||||
expect(cookies).toContain("Secure");
|
||||
expect(cookies).toContain("SameSite=lax");
|
||||
expect(cookies).toContain("Path=/api/marketplace/callback");
|
||||
expect(cookies).toContain("Path=/"); // __Host- prefix requires Path=/ per RFC 6265bis
|
||||
// Check __Host- prefix if in production (assuming https testing context)
|
||||
expect(cookies).toMatch(/__Host-pkce_verifier/);
|
||||
});
|
||||
|
||||
@@ -61,17 +61,26 @@ function TrustBadge({ trust }: { trust: string }) {
|
||||
|
||||
// ─── Browse Link ────────────────────────────────────────────
|
||||
|
||||
const MARKETPLACE_BASE = process.env.NEXT_PUBLIC_MARKETPLACE_URL ?? "https://marketplace.worldwideview.dev";
|
||||
|
||||
function BrowseLink() {
|
||||
function handleClick(e: React.MouseEvent<HTMLAnchorElement>) {
|
||||
e.preventDefault();
|
||||
trackEvent("marketplace-browse-click");
|
||||
const url = `${MARKETPLACE_BASE}?from_instance=${encodeURIComponent(window.location.origin)}`;
|
||||
window.open(url, "_blank", "noopener,noreferrer");
|
||||
}
|
||||
return (
|
||||
<a
|
||||
href="https://marketplace.worldwideview.dev"
|
||||
href={MARKETPLACE_BASE}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
className="plugins-tab__browse"
|
||||
onClick={() => trackEvent("marketplace-browse-click")}
|
||||
data-testid="browse-plugins-btn"
|
||||
onClick={handleClick}
|
||||
>
|
||||
<ExternalLink size={14} />
|
||||
Marketplace
|
||||
Browse Plugins
|
||||
</a>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -207,7 +207,11 @@ class WebSocketClient {
|
||||
|
||||
engine.subscriptions.add(pluginId);
|
||||
this.connectEngine(engineUrl);
|
||||
this.send(engine, { action: "subscribe", pluginId });
|
||||
// Only send immediately if auth is not in-flight; the welcome handler will
|
||||
// replay all pending subscriptions once auth succeeds (see onmessage:121-124).
|
||||
if (!engine.awaitingWelcome) {
|
||||
this.send(engine, { action: "subscribe", pluginId });
|
||||
}
|
||||
}
|
||||
|
||||
public unsubscribe(pluginId: string, engineUrl: string) {
|
||||
|
||||
@@ -1,4 +1,19 @@
|
||||
import { describe, it, expect, afterEach } from "vitest";
|
||||
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
|
||||
|
||||
const { mockUndiciF, mockDnsLookup } = vi.hoisted(() => ({
|
||||
mockUndiciF: vi.fn(),
|
||||
mockDnsLookup: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock("dns/promises", () => ({
|
||||
default: { lookup: mockDnsLookup },
|
||||
}));
|
||||
|
||||
vi.mock("undici", async (importOriginal) => {
|
||||
const original = await importOriginal<typeof import("undici")>();
|
||||
return { ...original, fetch: mockUndiciF };
|
||||
});
|
||||
|
||||
import { safeFetch, validateOrigin, isPrivateIP } from "./ssrf";
|
||||
|
||||
describe("SSRF Protection Utility", () => {
|
||||
@@ -68,23 +83,55 @@ describe("SSRF Protection Utility", () => {
|
||||
});
|
||||
|
||||
describe("safeFetch", () => {
|
||||
const PUBLIC_IP = "93.184.216.34";
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
process.env.PROXY_HOST_ALLOWLIST = "*";
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
delete process.env.PROXY_HOST_ALLOWLIST;
|
||||
});
|
||||
|
||||
it("should reject private IPs immediately", async () => {
|
||||
process.env.PROXY_HOST_ALLOWLIST = "*"; // bypass allowlist for these tests
|
||||
await expect(safeFetch("https://127.0.0.1/data")).rejects.toThrow(/SSRF/);
|
||||
await expect(safeFetch("https://169.254.169.254/latest/meta-data")).rejects.toThrow(/SSRF/);
|
||||
});
|
||||
|
||||
it("should enforce size limits", async () => {
|
||||
// Because safeFetch will attempt DNS lookup, we'd need to mock it if we wanted to test this in isolation.
|
||||
// But since this is a unit test, we can trust the return is a standard fetch wrapped with size enforcement.
|
||||
// The size limits are enforced on the stream pull.
|
||||
const result = safeFetch;
|
||||
expect(typeof result).toBe("function");
|
||||
it("enforces maxSize — responses exceeding the limit error on body read", async () => {
|
||||
mockDnsLookup.mockResolvedValue({ address: PUBLIC_IP, family: 4 });
|
||||
const oversized = new Uint8Array(11); // 11 bytes > 10-byte limit
|
||||
const stream = new ReadableStream<Uint8Array>({
|
||||
start(controller) {
|
||||
controller.enqueue(oversized);
|
||||
controller.close();
|
||||
},
|
||||
});
|
||||
mockUndiciF.mockResolvedValueOnce(new Response(stream, { status: 200 }));
|
||||
|
||||
const response = await safeFetch("https://example.com/feed", { maxSize: 10 });
|
||||
await expect(response.text()).rejects.toThrow(/size exceeded/);
|
||||
});
|
||||
|
||||
it("should use undici dispatcher to pin the DNS resolution", async () => {
|
||||
// Again, a full mock is complex, but we know it now uses undici.
|
||||
expect(true).toBe(true);
|
||||
it("uses undici dispatcher pinned to the resolved IP and does not follow redirects", async () => {
|
||||
mockDnsLookup.mockResolvedValue({ address: PUBLIC_IP, family: 4 });
|
||||
// safeFetch uses redirect:"manual" — this 301 must not be followed
|
||||
mockUndiciF.mockResolvedValueOnce(
|
||||
new Response(null, { status: 301, headers: { Location: "https://evil-rebind.example.com" } })
|
||||
);
|
||||
|
||||
const response = await safeFetch("https://example.com/feed");
|
||||
|
||||
// DNS resolved exactly once; redirect was NOT followed (fetch still once)
|
||||
expect(mockDnsLookup).toHaveBeenCalledOnce();
|
||||
expect(mockUndiciF).toHaveBeenCalledOnce();
|
||||
expect(response.status).toBe(301);
|
||||
|
||||
// fetch was called with a dispatcher (undici Agent) and redirect:"manual"
|
||||
const callOpts = mockUndiciF.mock.calls[0][1] as Record<string, unknown>;
|
||||
expect(callOpts.dispatcher).toBeDefined();
|
||||
expect(callOpts.redirect).toBe("manual");
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,137 @@
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
/**
|
||||
* Marketplace instance auto-discovery E2E tests.
|
||||
*
|
||||
* Two test groups:
|
||||
*
|
||||
* 1. "capture side" — navigates directly to the marketplace with
|
||||
* ?from_instance=... and verifies InstanceCapture stores it.
|
||||
* Runs with playwright.marketplace.config.ts (no worldwideview / Docker needed).
|
||||
*
|
||||
* 2. "full flow" — opens worldwideview, clicks Browse Plugins, and verifies
|
||||
* the marketplace receives the param. Needs the main playwright.config.ts
|
||||
* (Docker + Postgres required for auth).
|
||||
*/
|
||||
|
||||
const MARKETPLACE_URL = 'http://localhost:3002';
|
||||
const WWV_ORIGIN = 'http://localhost:3001';
|
||||
const FROM_INSTANCE_PARAM = `from_instance=${encodeURIComponent(WWV_ORIGIN)}`;
|
||||
|
||||
// ─── Marketplace capture side (no worldwideview auth needed) ────────────────
|
||||
|
||||
test.describe('InstanceCapture — marketplace receives from_instance', () => {
|
||||
test('captures valid origin, writes localStorage, cleans URL', async ({ page }) => {
|
||||
// Navigate as the marketplace would be opened by worldwideview.
|
||||
// InstanceCapture runs during React hydration (useEffect), so by the time
|
||||
// page.goto() resolves the URL may already be clean — that is correct behavior.
|
||||
await page.goto(`${MARKETPLACE_URL}/?${FROM_INSTANCE_PARAM}`);
|
||||
|
||||
// Wait for InstanceCapture to write to localStorage (may already be set)
|
||||
await page.waitForFunction(
|
||||
() => localStorage.getItem('wwv_instance_url') !== null,
|
||||
{ timeout: 15000 }
|
||||
);
|
||||
|
||||
// localStorage has the correct origin-only value
|
||||
const stored = await page.evaluate(() => localStorage.getItem('wwv_instance_url'));
|
||||
expect(stored).toBe(WWV_ORIGIN);
|
||||
|
||||
// from_instance param was stripped by history.replaceState
|
||||
expect(page.url()).not.toContain('from_instance');
|
||||
});
|
||||
|
||||
test('rejects javascript: protocol and leaves localStorage empty', async ({ page }) => {
|
||||
// Clear any prior localStorage from the same origin
|
||||
await page.goto(MARKETPLACE_URL);
|
||||
await page.evaluate(() => localStorage.removeItem('wwv_instance_url'));
|
||||
|
||||
await page.goto(`${MARKETPLACE_URL}/?from_instance=${encodeURIComponent('javascript:alert(1)')}`);
|
||||
|
||||
// Give InstanceCapture time to run
|
||||
await page.waitForTimeout(1500);
|
||||
|
||||
const stored = await page.evaluate(() => localStorage.getItem('wwv_instance_url'));
|
||||
expect(stored).toBeNull();
|
||||
});
|
||||
|
||||
test('rejects self-referential origin and leaves localStorage empty', async ({ page }) => {
|
||||
await page.goto(MARKETPLACE_URL);
|
||||
await page.evaluate(() => localStorage.removeItem('wwv_instance_url'));
|
||||
|
||||
// from_instance points at the marketplace itself — should be rejected
|
||||
await page.goto(`${MARKETPLACE_URL}/?from_instance=${encodeURIComponent(MARKETPLACE_URL)}`);
|
||||
await page.waitForTimeout(1500);
|
||||
|
||||
const stored = await page.evaluate(() => localStorage.getItem('wwv_instance_url'));
|
||||
expect(stored).toBeNull();
|
||||
});
|
||||
|
||||
test('direct navigation without param leaves localStorage unchanged', async ({ page }) => {
|
||||
await page.goto(MARKETPLACE_URL);
|
||||
await page.evaluate(() => localStorage.removeItem('wwv_instance_url'));
|
||||
|
||||
await page.reload();
|
||||
await page.waitForLoadState('load');
|
||||
await page.waitForTimeout(1000);
|
||||
|
||||
const stored = await page.evaluate(() => localStorage.getItem('wwv_instance_url'));
|
||||
expect(stored).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Full flow (worldwideview → marketplace, requires Docker + auth) ─────────
|
||||
|
||||
test.describe('Full flow — Browse Plugins in worldwideview opens marketplace', () => {
|
||||
test.beforeEach(async ({ page, baseURL }) => {
|
||||
// Skip when running with the marketplace-only config (baseURL = 3002)
|
||||
test.skip(baseURL !== WWV_ORIGIN, 'Requires worldwideview running (main config + Docker)');
|
||||
|
||||
await page.goto('/');
|
||||
await page.waitForSelector('[data-testid="app-ready"]', { state: 'attached', timeout: 45000 });
|
||||
|
||||
// Dismiss unverified plugin dialog if it appears
|
||||
try {
|
||||
const installBtn = page.getByRole('button', { name: /Install Selected/ });
|
||||
await installBtn.waitFor({ state: 'visible', timeout: 2000 });
|
||||
await installBtn.click();
|
||||
} catch { /* not present */ }
|
||||
|
||||
// Ensure left panel is open
|
||||
const leftToggle = page.locator('[data-testid="panel-toggle-left"]');
|
||||
const isOpen = await leftToggle.evaluate((el) => el.classList.contains('panel-toggle-btn--open'));
|
||||
if (!isOpen) await leftToggle.click();
|
||||
});
|
||||
|
||||
test('Browse Plugins appends from_instance and marketplace captures it', async ({ page }) => {
|
||||
// Navigate to Plugins tab
|
||||
const pluginsTab = page.locator('button.panel-tab[title="Plugins"]');
|
||||
await expect(pluginsTab).toBeVisible({ timeout: 10000 });
|
||||
await pluginsTab.click();
|
||||
|
||||
const browseBtn = page.locator('[data-testid="browse-plugins-btn"]');
|
||||
await expect(browseBtn).toBeVisible({ timeout: 5000 });
|
||||
|
||||
// Click and capture the popup
|
||||
const popupPromise = page.waitForEvent('popup');
|
||||
await browseBtn.click();
|
||||
const popup = await popupPromise;
|
||||
|
||||
// Initial URL has from_instance
|
||||
expect(popup.url()).toContain(FROM_INSTANCE_PARAM);
|
||||
|
||||
// InstanceCapture writes to localStorage
|
||||
await popup.waitForFunction(
|
||||
() => localStorage.getItem('wwv_instance_url') !== null,
|
||||
{ timeout: 15000 }
|
||||
);
|
||||
|
||||
const stored = await popup.evaluate(() => localStorage.getItem('wwv_instance_url'));
|
||||
expect(stored).toBe(WWV_ORIGIN);
|
||||
|
||||
// URL cleaned up
|
||||
expect(popup.url()).not.toContain('from_instance');
|
||||
|
||||
await popup.close();
|
||||
});
|
||||
});
|
||||
Reference in new issue
Block a user