Compare commits

...
Author SHA1 Message Date
Silvertakana 9b3d796299 Merge branch 'main' into fix/ticket-spec-null-typecast 2026-05-23 16:11:30 +12:00
Silvertakana 7f7d7588f3 fix(docker): auto-baseline Prisma P3005 on container startup (#181)
* fix: add id-token write permission and fix invalid inputs in GC workflow

The claude-code-action@v1 uses OIDC for OAuth token auth, which requires
id-token: write in workflow permissions. Without it ACTIONS_ID_TOKEN_REQUEST_URL
is not set and the action fails after 3 retry attempts.

Also moves model and allowed_tools from unsupported top-level inputs into
claude_args where they are valid CLI flags for claude-code-action@v1.

* fix(gc): show agent output and explicit dry-run issue posting

- Add show_full_output: true so Claude responses appear in CI logs
- Replace vague dry-run instruction with explicit gh issue create/comment
  commands so output has a concrete destination (tracking issue)
- Bake DRY_RUN value into prompt at workflow-expansion time to eliminate
  ambiguity about env var reading

* docs: mark cloud-auth-architecture.md as superseded by ADR-003

* fix(docker): guard community seeder cd and add ENCRYPTION_MASTER_KEY default

- Wrap `cd $SEEDERS_DIR/community` in an existence check so the
  data-engine container no longer crashes when local-seeders/community/
  is an empty or uncloned nested git repo. The container was entering
  Docker's exponential-backoff restart loop (2s→60s) on every boot.
- Add ENCRYPTION_MASTER_KEY to the wwv environment block with a
  local-dev fallback, matching the AUTH_SECRET pattern. Without it the
  Next.js instrumentation hook throws at startup and the app never
  serves a request.

* fix(docker): auto-baseline P3005 when schema exists without migration history

Wraps prisma migrate deploy in the container entrypoint to detect P3005
(schema populated by pnpm dev / prisma db push before the Docker stack was
started). When detected, runs prisma migrate diff to confirm zero schema
drift, then marks each migration as applied via migrate resolve --applied
and retries deploy. Exits with a clear error if drift is found.

Also documents the auto-baseline behavior in database-migrations.md and
fixes duplicate section numbering in that file.
2026-05-23 16:11:13 +12:00
Antigravity AI f77f0f41a8 fix: remove stray 'r' prefix on import in places/details route
rimport typo on line 1 of src/app/api/places/details/route.ts caused
TS1435 (unknown keyword), ESLint parse error, and a webpack build failure
in Docker CI. Corrected to a valid import statement.
2026-05-23 16:04:30 +12:00
Antigravity AI 55c19756db chore: bump version for ADR-001 Workstream C 2026-05-23 16:00:06 +12:00
Antigravity AI 149468e6c1 docs: add ADR-003 document and update cloud-auth-architecture superseded notice 2026-05-23 15:58:58 +12:00
Silvertakana b9c6d193a4 fix(docker): guard community seeder cd and add ENCRYPTION_MASTER_KEY default (#179)
* fix: add id-token write permission and fix invalid inputs in GC workflow

The claude-code-action@v1 uses OIDC for OAuth token auth, which requires
id-token: write in workflow permissions. Without it ACTIONS_ID_TOKEN_REQUEST_URL
is not set and the action fails after 3 retry attempts.

Also moves model and allowed_tools from unsupported top-level inputs into
claude_args where they are valid CLI flags for claude-code-action@v1.

* fix(gc): show agent output and explicit dry-run issue posting

- Add show_full_output: true so Claude responses appear in CI logs
- Replace vague dry-run instruction with explicit gh issue create/comment
  commands so output has a concrete destination (tracking issue)
- Bake DRY_RUN value into prompt at workflow-expansion time to eliminate
  ambiguity about env var reading

* docs: mark cloud-auth-architecture.md as superseded by ADR-003

* fix(docker): guard community seeder cd and add ENCRYPTION_MASTER_KEY default

- Wrap `cd $SEEDERS_DIR/community` in an existence check so the
  data-engine container no longer crashes when local-seeders/community/
  is an empty or uncloned nested git repo. The container was entering
  Docker's exponential-backoff restart loop (2s→60s) on every boot.
- Add ENCRYPTION_MASTER_KEY to the wwv environment block with a
  local-dev fallback, matching the AUTH_SECRET pattern. Without it the
  Next.js instrumentation hook throws at startup and the app never
  serves a request.
2026-05-23 15:17:31 +12:00
Silvertakana 062adaf9d3 fix: add id-token write permission and fix invalid inputs in GC workflow (#177)
The claude-code-action@v1 uses OIDC for OAuth token auth, which requires
id-token: write in workflow permissions. Without it ACTIONS_ID_TOKEN_REQUEST_URL
is not set and the action fails after 3 retry attempts.

Also moves model and allowed_tools from unsupported top-level inputs into
claude_args where they are valid CLI flags for claude-code-action@v1.
2026-05-23 01:28:38 +12:00
Silvertakana 1cf4074eb3 fix: remove hardcoded pnpm version from GC workflow (#176) 2026-05-23 01:23:41 +12:00
Silvertakana bc6f087efb feat: add Garbage Collector agent — daily Haiku-powered codebase hygiene (#175)
Adds a daily GitHub Actions cron that runs a zero-cost deterministic scan
(gc-scan.mjs) then hands findings to a Haiku subagent for triage:
- Tier A findings (stale TODOs, console.log, @ts-ignore, .mdc refs) → draft PRs
- Tier B findings (oversized files, orphaned rule refs, outdated deps) → Issues
- Max 3 PRs + 5 Issues per run, 150 LOC cap per PR, full idempotency via gc-bot label
- Defaults to dry-run mode for safe initial validation
2026-05-23 00:23:55 +12:00
Silvertakana 8cb1756d6c fix(gc): fix gh CLI compatibility in workflow
Refactor garbage collector workflow for improved clarity and efficiency. Update issue tracking and report generation processes.
2026-05-23 00:03:17 +12:00
Silvertakana 851811eb35 feat(gc): add Garbage Collector workflow 2026-05-23 00:00:08 +12:00
Silvertakana f5802cf34a feat(gc): add Garbage Collector scan script
This script scans the repository for stale branches, old comments, orphaned packages, and outdated Prisma migrations, generating a JSON report.
2026-05-22 23:58:33 +12:00
SilvertakanaandClaude Sonnet 4.6 c9bb6efa43 fix(test): cast null mock to any in ticket route spec (#173)
auth is typed as NextMiddleware; mockResolvedValue(null) fails tsc.
Consistent with all other vi.mocked(auth) calls in the file.

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-05-21 20:39:28 +12:00
SilvertakanaandClaude Sonnet 4.6 67fcd7923e test(auth): unit tests for ADR-001 ticket route, boot gate, and edition flag (#172)
* feat(auth): implement ADR-001 Workstream C — ticket auth, SSRF, and pre-flight audit

C1: Boot-time ENCRYPTION_MASTER_KEY / AUTH_SECRET validation (instrumentation.ts);
    remove silent fallback in encryption.ts; guard marketplace token from data-plane misuse.
C2: SSRF host-allowlist (PROXY_HOST_ALLOWLIST) in ssrf.ts; route iframe proxy through safeFetch.
C3: ticketClient.ts — exchanges long-lived API key for 5-min EdDSA ticket from Marketplace;
    audience fixed to ENGINE_AUDIENCE='wwv-data-engine'; cache keyed by engine not plugin.
C4: WsClient.ts first-message auth — sends {type:'auth',v:1,token} on open; queues subscribes
    until {type:'welcome'}; gated by ticketAuthEnabledForPlugin.
C5: edition.ts — ticketAuthEnabledForPlugin reads NEXT_PUBLIC_WWV_TICKET_AUTH_PLUGINS flag.
Step 0: docs/deploys/2026-05-21-adr-001-preflight.md + docs/feature-flags.md audit artifacts.
Tests: ticketClient.spec, WsClient.spec, ssrf.spec updated (342 passing).

Co-Authored-By: Claude Opus 4.7 <[email protected]>

* fix(ci): resolve three PR #171 type-check and startup failures

- auth-contracts.spec.ts: assert WebSocketAuthMessage.token as PluginTicket
  (not SensitiveString — distinct branded types fail toEqualTypeOf equality)
- pkce.spec.ts: replace function(){return this} mock with arrow fn to fix
  TS2683 implicit-this error in strict mode
- test-local-dev.yml: add ENCRYPTION_MASTER_KEY so instrumentation.ts boot
  validation doesn't crash the startup test in CI

Co-Authored-By: Claude Opus 4.7 <[email protected]>

* fix(ci): fix pkce mock constructor and env injection for startup test

- pkce.spec.ts: use function(this: object) instead of arrow fn so the
  mock can be called with `new Configuration(...)` (arrow functions are
  not constructors)
- test-local-dev.yml: sed-replace ENCRYPTION_MASTER_KEY in .env after
  cp .env.example so Next.js instrumentation boot check passes in CI

Co-Authored-By: Claude Opus 4.7 <[email protected]>

* fix(ci): also sed-replace AUTH_SECRET in .env for startup test

.env.example has AUTH_SECRET= (empty), which Next.js loads and overrides
the system env var, causing the same boot crash as ENCRYPTION_MASTER_KEY.

Co-Authored-By: Claude Opus 4.7 <[email protected]>

* fix(setup): auto-generate ENCRYPTION_MASTER_KEY in setup.mjs

The Playwright CI workflow uses `pnpm run setup` which previously only
generated AUTH_SECRET. ENCRYPTION_MASTER_KEY remained empty from
.env.example, causing instrumentation.ts boot validation to crash.

This fix also improves the new-developer onboarding experience — both
required secrets are now generated in a single setup step.

Co-Authored-By: Claude Opus 4.7 <[email protected]>

* test(auth): add unit tests for ticket route, instrumentation boot gate, and edition flag

Covers GET /api/auth/ticket (4 cases), register() startup validation (4 cases),
and ticketAuthEnabledForPlugin() dormant-safety gate (4 cases). All 350 tests pass.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 <[email protected]>
2026-05-21 20:22:21 +12:00
SilvertakanaandClaude Opus 4.7 26568429bc feat(auth): ADR-001 Workstream C — ticket auth, SSRF protection, and pre-flight audit (#171)
* feat(auth): implement ADR-001 Workstream C — ticket auth, SSRF, and pre-flight audit

C1: Boot-time ENCRYPTION_MASTER_KEY / AUTH_SECRET validation (instrumentation.ts);
    remove silent fallback in encryption.ts; guard marketplace token from data-plane misuse.
C2: SSRF host-allowlist (PROXY_HOST_ALLOWLIST) in ssrf.ts; route iframe proxy through safeFetch.
C3: ticketClient.ts — exchanges long-lived API key for 5-min EdDSA ticket from Marketplace;
    audience fixed to ENGINE_AUDIENCE='wwv-data-engine'; cache keyed by engine not plugin.
C4: WsClient.ts first-message auth — sends {type:'auth',v:1,token} on open; queues subscribes
    until {type:'welcome'}; gated by ticketAuthEnabledForPlugin.
C5: edition.ts — ticketAuthEnabledForPlugin reads NEXT_PUBLIC_WWV_TICKET_AUTH_PLUGINS flag.
Step 0: docs/deploys/2026-05-21-adr-001-preflight.md + docs/feature-flags.md audit artifacts.
Tests: ticketClient.spec, WsClient.spec, ssrf.spec updated (342 passing).

Co-Authored-By: Claude Opus 4.7 <[email protected]>

* fix(ci): resolve three PR #171 type-check and startup failures

- auth-contracts.spec.ts: assert WebSocketAuthMessage.token as PluginTicket
  (not SensitiveString — distinct branded types fail toEqualTypeOf equality)
- pkce.spec.ts: replace function(){return this} mock with arrow fn to fix
  TS2683 implicit-this error in strict mode
- test-local-dev.yml: add ENCRYPTION_MASTER_KEY so instrumentation.ts boot
  validation doesn't crash the startup test in CI

Co-Authored-By: Claude Opus 4.7 <[email protected]>

* fix(ci): fix pkce mock constructor and env injection for startup test

- pkce.spec.ts: use function(this: object) instead of arrow fn so the
  mock can be called with `new Configuration(...)` (arrow functions are
  not constructors)
- test-local-dev.yml: sed-replace ENCRYPTION_MASTER_KEY in .env after
  cp .env.example so Next.js instrumentation boot check passes in CI

Co-Authored-By: Claude Opus 4.7 <[email protected]>

* fix(ci): also sed-replace AUTH_SECRET in .env for startup test

.env.example has AUTH_SECRET= (empty), which Next.js loads and overrides
the system env var, causing the same boot crash as ENCRYPTION_MASTER_KEY.

Co-Authored-By: Claude Opus 4.7 <[email protected]>

* fix(setup): auto-generate ENCRYPTION_MASTER_KEY in setup.mjs

The Playwright CI workflow uses `pnpm run setup` which previously only
generated AUTH_SECRET. ENCRYPTION_MASTER_KEY remained empty from
.env.example, causing instrumentation.ts boot validation to crash.

This fix also improves the new-developer onboarding experience — both
required secrets are now generated in a single setup step.

Co-Authored-By: Claude Opus 4.7 <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 <[email protected]>
2026-05-21 17:00:24 +12:00
Antigravity AIandClaude Opus 4.7 f3720ff2c1 chore(docs): untrack .agents/context/ from git and fix ecosystem repo docs
- Remove .agents/context/ from git index (already in .gitignore, files kept locally)
- Correct local-plugins/ from "sandbox" to canonical community plugin repo clone (silvertakana/wwv-plugins)
- Correct local-seeders/community/ and local-seeders/private/ as independent git clones with own remotes
- Remove deprecated worldwideview-plugins repo references; rename wwv-seeders-community → wwv-seeders throughout
- Add nested-git-clone invariant to AGENTS.md, ecosystem-repositories.md, directory-structure.md, monorepo-workflow.md, plugin-architecture.md, data-engine-architecture.md, migrate-legacy-plugin skill

Co-Authored-By: Claude Opus 4.7 <[email protected]>
2026-05-21 12:08:33 +12:00
Antigravity AI 68716b9e56 chore: update .gitignore to include .agents/context and add .claudeignore for AI agent filtering 2026-05-21 11:24:40 +12:00
SilvertakanaandClaude Sonnet 4.6 c2d850a0d0 chore: reduce agent context window bloat (165KB → 4KB baseline) (#170)
- Slim AGENTS.md from 505 to 163 lines; extract §17 Coding Principles
  and §16 Debugging Tips into .agents/context/ reference files
- Add native paths: frontmatter to all 10 .agents/rules/ domain files
  so they only auto-load when editing matching file paths
- Move 10 on-demand reference files from .agents/rules/ to .agents/context/
  (no longer eagerly scanned on every session start)
- Create .agents/context/INDEX.md navigation table for agents
- Create .agents/context/coding-principles.md (extracted from AGENTS.md)
- Remove /.agents/context from .gitignore (now holds versioned reference docs)

Baseline session context: ~165KB → ~4KB
Worst-case (plugin work): ~165KB → ~44KB

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-05-21 02:15:25 +12:00
SilvertakanaandClaude Sonnet 4.6 6d62ca5677 fix(ws): diagnose & fix plugin 404s, WS auth failure, and reconnect storms (#169)
* fix(ws): exponential backoff + jitter on WsClient reconnect

Replaces the fixed 5-second reconnect delay with an exponential backoff
(5s→10s→20s→40s→60s cap, +4s random jitter) so simultaneous browser
sessions don't thunder-herd the engine on restart.

- Backoff resets only after a stable 5-second connection, not on every
  successful open, so rapid reconnect cycles don't mask the retry.
- Stale-timer bookkeeping added to the cleanup/unsubscribe path.

Also bundles two companion fixes discovered during the same session:

fix(scripts): BOM-safe JSON parsing in sync/watch-local-plugins (Fix)
  PowerShell 5.1 emits a UTF-8 BOM that Node's JSON.parse rejects;
  readJsonFile() strips U+FEFF before parsing, matching pnpm behavior.

docs(agents): add UTF-8 BOM troubleshooting entry (Fix)
  Documents diagnosis, fix, and prevention for the BOM crash.

Co-Authored-By: Claude Opus 4.7 <[email protected]>

* test(ws): pin Math.random to 0 so reconnect backoff test is deterministic

With jitter (Math.random()*4000), advancing timers by exactly 5000ms no
longer guaranteed to trigger the reconnect. Mocking Math.random to 0 in
beforeEach makes the delay deterministic (base only, no jitter) across
the full test suite.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 <[email protected]>
2026-05-21 01:07:16 +12:00
SilvertakanaandClaude Sonnet 4.6 58aac77a0b fix(engine): resolve plugin 404s and local engine routing (#166)
* refactor(marketplace): drive auto-seeding from signed verified registry

Retires the hard-coded DEFAULT_PLUGIN_IDS list and makes the signed
verified registry the single source of truth for what lands in a fresh
install. Publishing a plugin to the verified registry now auto-seeds
it; no code change required to add new defaults.

Also fixes the diagnosed plugins-404: fortiguard and nz-traffic-cameras
were in the static list but not published to npm, causing every fresh
boot to hit manifest 404s. Those entries are gone with the list.

- Delete src/lib/marketplace/defaultPlugins.ts (legacy static list).
- Seeder iterates getVerifiedPluginIds() from registryClient.
- Skip markSeeded when the verified set is empty so a registry outage
  doesn't lock fresh installs into zero plugins; the next request
  retries (5-min cache in registryClient bounds the request rate).
- Stamp every seeded plugin as trust=verified by definition.
- Tests rewritten against a fake verified set; new test asserts
  empty-registry deferral.
- Sync marketplace-architecture rule + migrate-legacy-plugin skill
  with the new model.
- Patch bump: 2.15.15 -> 2.15.16.

* fix(engine): resolve plugin 404s and local engine routing (#fix/diagnose-plugins-404)

- fix: engineManifest + resolveEngineUrl: port 5001→5000 to match docker-compose
  binding; removed NEXT_PUBLIC_WWV_PLUGIN_DATA_ENGINE_URL from local engine
  detection — it was poisoning local routing with the production engine URL (Patch)
- fix: WsClient: added normalizePluginId() converting underscore IDs to kebab-case
  so engine WebSocket messages route to the correct frontend plugin (Patch)
- fix: docker-compose: added ./local-seeders:/app/seeders volume mount to
  wwv-data-engine so locally-developed seeders are visible to the engine (Patch)
- feat: sync-local-plugins: added wwvHostRedirectPlugin — Vite plugin that
  intercepts @/ host-app imports in local plugin builds and redirects them to
  runtime host globals, preventing full app tree from being bundled into plugins (Minor)
- docs: plugin-architecture.md: added Plugin Data Delivery Types section (REST-only,
  WS-only, hybrid), peerDependencies rule, .tsx extension rule, mapWebsocketPayload
  signature contract (Patch)
- docs: data-engine-architecture.md: added §7 WebSocket Payload Format Contract
  documenting WsClient's three payload handling cases and the silent-drop footgun
  for plugins that send object payloads without mapWebsocketPayload (Patch)
- docs: worldwideview-plugin-creation SKILL.md: fixed ISS example which was missing
  mapWebsocketPayload and would have silently dropped all WS data; clarified
  mapWebsocketPayload requirement for non-array payloads; fixed architecture
  decision tree (Patch)
- docs: cesium-rendering.md: added Sizing Conventions table with defaults/ranges
  for point/billboard/model, cross-plugin parity rules, and silent-ignore footgun
  warning for size on billboard entities (Patch)
- docs: added ADR-0002 documenting seeder exported name as canonical plugin ID (Patch)

Version bumps: fix × 3 (Patch), feat × 1 (Minor) → 2.15.16 → 2.16.0

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* test(engine): update resolveEngineUrl test assertion from port 5001 to 5000

Follows the port correction in engineManifest.ts and resolveEngineUrl.ts.
The test was asserting the old default (5001) instead of the corrected
local engine port (5000), causing the Unit Tests CI check to fail.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-05-20 22:44:47 +12:00
dependabot[bot] 0462ab1f88 build(deps-dev): bump @vitejs/plugin-react from 5.2.0 to 6.0.2
Bumps [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/tree/HEAD/packages/plugin-react) from 5.2.0 to 6.0.2.
- [Release notes](https://github.com/vitejs/vite-plugin-react/releases)
- [Changelog](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite-plugin-react/commits/[email protected]/packages/plugin-react)

---
updated-dependencies:
- dependency-name: "@vitejs/plugin-react"
  dependency-version: 6.0.2
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-20 21:18:07 +12:00
dependabot[bot] 60fca4d540 build(deps-dev): bump @types/node from 20.19.41 to 25.9.1
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 20.19.41 to 25.9.1.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-version: 25.9.0
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-20 21:17:37 +12:00
dependabot[bot] dba5d49448 build(deps): bump commander from 12.1.0 to 14.0.3
Bumps [commander](https://github.com/tj/commander.js) from 12.1.0 to 14.0.3.
- [Release notes](https://github.com/tj/commander.js/releases)
- [Changelog](https://github.com/tj/commander.js/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tj/commander.js/compare/v12.1.0...v14.0.3)

---
updated-dependencies:
- dependency-name: commander
  dependency-version: 14.0.3
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-20 21:13:39 +12:00
dependabot[bot] 1633c7bb7b build(deps): bump the dependencies group with 11 updates (#161)
Bumps the dependencies group with 11 updates:

| Package | From | To |
| --- | --- | --- |
| [@sentry/nextjs](https://github.com/getsentry/sentry-javascript) | `10.52.0` | `10.53.1` |
| [@supabase/supabase-js](https://github.com/supabase/supabase-js/tree/HEAD/packages/core/supabase-js) | `2.105.4` | `2.106.0` |
| [lucide-react](https://github.com/lucide-icons/lucide/tree/HEAD/packages/lucide-react) | `1.14.0` | `1.16.0` |
| [pg](https://github.com/brianc/node-postgres/tree/HEAD/packages/pg) | `8.20.0` | `8.21.0` |
| [resium](https://github.com/reearth/resium) | `1.21.0` | `1.21.1` |
| [satellite.js](https://github.com/shashwatak/satellite-js) | `7.0.0` | `7.0.1` |
| [undici](https://github.com/nodejs/undici) | `8.2.0` | `8.3.0` |
| [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) | `8.59.3` | `8.59.4` |
| [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) | `8.59.3` | `8.59.4` |
| [tsx](https://github.com/privatenumber/tsx) | `4.21.0` | `4.22.2` |
| [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) | `8.0.12` | `8.0.13` |


Updates `@sentry/nextjs` from 10.52.0 to 10.53.1
- [Release notes](https://github.com/getsentry/sentry-javascript/releases)
- [Changelog](https://github.com/getsentry/sentry-javascript/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-javascript/compare/10.52.0...10.53.1)

Updates `@supabase/supabase-js` from 2.105.4 to 2.106.0
- [Release notes](https://github.com/supabase/supabase-js/releases)
- [Changelog](https://github.com/supabase/supabase-js/blob/master/packages/core/supabase-js/CHANGELOG.md)
- [Commits](https://github.com/supabase/supabase-js/commits/v2.106.0/packages/core/supabase-js)

Updates `lucide-react` from 1.14.0 to 1.16.0
- [Release notes](https://github.com/lucide-icons/lucide/releases)
- [Commits](https://github.com/lucide-icons/lucide/commits/1.16.0/packages/lucide-react)

Updates `pg` from 8.20.0 to 8.21.0
- [Changelog](https://github.com/brianc/node-postgres/blob/master/CHANGELOG.md)
- [Commits](https://github.com/brianc/node-postgres/commits/[email protected]/packages/pg)

Updates `resium` from 1.21.0 to 1.21.1
- [Release notes](https://github.com/reearth/resium/releases)
- [Changelog](https://github.com/reearth/resium/blob/main/CHANGELOG.md)
- [Commits](https://github.com/reearth/resium/compare/v1.21.0...v1.21.1)

Updates `satellite.js` from 7.0.0 to 7.0.1
- [Release notes](https://github.com/shashwatak/satellite-js/releases)
- [Changelog](https://github.com/shashwatak/satellite-js/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/shashwatak/satellite-js/compare/7.0.0...7.0.1)

Updates `undici` from 8.2.0 to 8.3.0
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v8.2.0...v8.3.0)

Updates `@typescript-eslint/eslint-plugin` from 8.59.3 to 8.59.4
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.59.4/packages/eslint-plugin)

Updates `@typescript-eslint/parser` from 8.59.3 to 8.59.4
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.59.4/packages/parser)

Updates `tsx` from 4.21.0 to 4.22.2
- [Release notes](https://github.com/privatenumber/tsx/releases)
- [Changelog](https://github.com/privatenumber/tsx/blob/master/release.config.cjs)
- [Commits](https://github.com/privatenumber/tsx/compare/v4.21.0...v4.22.2)

Updates `vite` from 8.0.12 to 8.0.13
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.0.13/packages/vite)

---
updated-dependencies:
- dependency-name: "@sentry/nextjs"
  dependency-version: 10.53.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: "@supabase/supabase-js"
  dependency-version: 2.106.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: lucide-react
  dependency-version: 1.16.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: pg
  dependency-version: 8.21.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: resium
  dependency-version: 1.21.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: satellite.js
  dependency-version: 7.0.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: undici
  dependency-version: 8.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: "@typescript-eslint/eslint-plugin"
  dependency-version: 8.59.4
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: "@typescript-eslint/parser"
  dependency-version: 8.59.4
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: tsx
  dependency-version: 4.22.2
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: vite
  dependency-version: 8.0.13
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-20 21:04:04 +12:00
SilvertakanaandClaude Sonnet 4.6 7eda6c50a4 fix: disable Firefox MSAA/AT-SPI to fix CI hangs (#165)
* fix: disable Firefox MSAA/AT-SPI accessibility on CI to prevent hangs

Adds `accessibility.force_disabled: 1` to Firefox's user prefs in the
Playwright config. On Linux CI (GitHub Actions / ubuntu-latest) Firefox
attempts to connect to the AT-SPI accessibility bus, which causes the
browser to hang for ~20 minutes before timing out. Disabling the
accessibility layer at the browser level eliminates the hang without
affecting test coverage.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: set NO_AT_BRIDGE=1 for firefox in CI to prevent AT-SPI hang

The firefoxUserPrefs approach alone is insufficient — Firefox on Linux
loads the AT-SPI accessibility bridge at the GTK level before user prefs
apply. Setting NO_AT_BRIDGE=1 cuts the connection at the OS/GTK layer,
which is what actually prevents the ~20 min hang on ubuntu-latest runners.
Scoped to the firefox matrix leg only; chromium and webkit are unaffected.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: add 20s safety boot fallback for environments where globe never initialises

On headless Linux CI (ubuntu-latest), Firefox uses Mesa software rendering
which can prevent CesiumJS from mounting its WebGL context. When that
happens, the `globeReady` DataBus event never fires, `startBoot()` is
never called, and `data-testid="app-ready"` never appears — causing every
Playwright test to time out at 45 s.

The fix guards `boot.startBoot()` behind a `bootStarted` flag so only the
first caller wins, then adds a 20 s `setTimeout` safety fallback in
AppShell. Normal flow (globe loads) fires the boot immediately via
`globeReady`; the safety timer is cleared. Degraded flow (WebGL absent)
still reaches the `app-ready` state after ~23.5 s (20 s + 3.5 s
animation), well within the 45 s test timeout.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-05-20 21:02:16 +12:00
Antigravity AI f2ae2a7de7 chore: update .gitignore to remove IDE config and refine env file rules 2026-05-20 13:45:23 +12:00
Antigravity AI 3210b3bd15 chore: convert .claude from text file to git symlink pointing to .agents 2026-05-20 10:52:00 +12:00
Antigravity AI 9fad051f75 Merge branch 'main' of https://github.com/silvertakana/worldwideview 2026-05-19 23:25:23 +12:00
Antigravity AI a818822023 chore: add CLAUDE.md symlink pointing to AGENTS.md for Claude Code 2026-05-19 23:24:59 +12:00
Silvertakana 012184af30 Fix: Plugin fetch error (#159)
* fix(plugin-manager): downgrade non-fatal 'Failed to fetch' errors to warn, remove polling re-throw

* fix(codeql): resolve format string vulnerability in console logging

* fix(ci): restore playwright dependencies and configure tsconfig properly

* fix(plugins): ensure plugin fetch uses HTTP scheme instead of WSS
2026-05-18 11:10:04 +12:00
SilvertakanaandCopilot Autofix powered by AI c1f7ffb5b4 Refactor: High quality standards (#156)
* chore(merge): resolve conflicts, fix mockWs error, and apply refactor standards

* Potential fix for pull request finding 'CodeQL / Incomplete URL substring sanitization'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for pull request finding 'CodeQL / Incomplete URL substring sanitization'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for pull request finding 'CodeQL / Incomplete URL substring sanitization'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix(ci): Stabilize github actions and resolve test failures

- fix(tests): update WsClient mock to use standard function for Vitest constructor compatibility

- fix(db): correct PrismaClient import path to use generated output directory

- fix(e2e): add predev database boot step to playwright CI workflow

- chore: sync pnpm-lock.yaml after merge conflict resolution

* refactor(plugins): decouple GeoJSON plugin and implement data bus lifecycle

- feat(sdk): [Minor] Added dynamicPluginCreate, dynamicPluginRemove, and pluginUnregistered to DataBusEvents.

- feat(core): [Minor] Implemented unregisterPlugin in PluginManager.ts for automated IndexedDB caching cleanup and polling teardown.

- feat(core): [Minor] Added removeLayer to layersSlice.ts to support safe removal of plugin states.

- refactor(plugins): [Patch] Refactored useGeoJsonImport.ts and ImportedLayerList.tsx to decouple from direct manager imports, utilizing DataBus for lifecycle events.

- refactor(core): [Patch] Added pluginUnregistered listener to DataBusSubscriber.tsx to handle state and cache cleanup synchronously via event.

* fix(types): resolve PluginManager type errors

* refactor(core): decouple PluginManager and fix rendering invariants

- refactor(core): Replace Zustand mutations in PluginManager with DataBus events (Patch)
- refactor(sdk): Add layerLoadingChanged, pluginError, pluginUnregistered to DataBusEvents (Patch)
- refactor(ui): Subscribe to new DataBus events in DataBusSubscriber (Patch)
- fix(plugins): Remove size from BaseAviationPlugin to fix model rendering clipping (Patch)
- fix(plugins): Migrate BaseIncidentPlugin from size to iconScale to fix billboard clipping (Patch)

* chore: resolve lockfile conflict

* fix: restore BottomPanelManager in AppShell and fix vitest config

* fix(layout): restore bottom panel positioning and transition styling

This commit corrects the missing bottom panel issue by resolving CSS aggregation errors and fixing transition logic. The version is bumped to 2.15.5.

* refactor(core): harden typescript and fix linting violations

This commit addresses the remaining any types and linting errors across the core engine.

* fix(eslint): resolve react-hook immutability and cascading render violations

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-05-18 11:08:00 +12:00
Silvertakanaandsnyk-bot 6816081644 Fix: Snyk vulnerability fix (#158)
* fix: package.json to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JS-WS-16722635

* fix: update pnpm-lock.yaml for ws dependency bump and bump version to 2.15.11

---------

Co-authored-by: snyk-bot <[email protected]>
2026-05-18 11:06:39 +12:00
Antigravity AI 433004598d Merge branch 'fix/lint-cleanups'
# Conflicts:
#	eslint.config.mjs
#	package.json
2026-05-17 22:08:13 +12:00
Antigravity AI 02d8355a83 feat: add inverted app icon and update lockfile dependencies 2026-05-17 22:05:48 +12:00
Silvertakana 89010b2425 feat(ui): Bottom Panel System and E2E Testing (#147)
* test(bottom-panel): add e2e resize testing and data-testid

* test(e2e): fix playwright drag events skipping React state
2026-05-17 20:14:18 +12:00
Antigravity AI 0ff1d23e4a chore(lint): clean up legacy linting errors and hallucinated rules 2026-05-17 20:04:13 +12:00
Silvertakanaandsnyk-bot fd29865e7a [Snyk] Security upgrade ws from 8.20.0 to 8.20.1 (#151)
* fix: package.json to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JS-WS-16722635

* fix: update pnpm-lock.yaml for ws dependency bump and bump version to 2.15.11

---------

Co-authored-by: snyk-bot <[email protected]>
2026-05-17 19:54:03 +12:00
Silvertakana 42564b537f ci(playwright): Add multi-browser test matrix (#150)
* ci(playwright): Add multi-browser test matrix (Patch)

* fix(cesium): Disable MSAA in Firefox to prevent readPixels crash (Patch)
2026-05-17 19:14:54 +12:00
Antigravity AI 598e00171c fix(cesium): Disable MSAA in Firefox to prevent readPixels crash (Patch) 2026-05-17 18:35:09 +12:00
devangpratap b849bb2b4d feat(api): add ISS Tracker proxy routes (#143)
* feat(api): add ISS position proxy route

Server-side proxy to the Where the ISS at? API (NORAD 25544).
Returns current latitude, longitude, altitude, velocity, visibility,
and footprint with 10-second revalidation and input validation.

Closes #8

* test(iss): add unit tests for ISS position route

Covers: valid response, field stripping, upstream failure,
invalid coordinates, and network error scenarios.

* feat(api): add ISS TLE orbit data route

Exposes the latest Two-Line Element set for NORAD 25544 via
/api/iss/tle. TLE data enables client-side orbit prediction
with satellite.js. Cached for 1 hour since TLEs update infrequently.

* test(iss): add unit tests for ISS TLE route

Covers: valid TLE response, field normalization (snake_case to
camelCase), upstream failure, empty TLE lines, and network errors.

* feat(api): add ISS historical positions route with tests

Batch endpoint at /api/iss/positions?timestamps=t1,t2,... returns
multiple past positions for orbit trail rendering. Validates input
(max 10 timestamps, numeric only), filters invalid upstream data,
and strips extraneous fields. Includes 7 test cases covering
happy path, validation, overflow, and upstream failure.

* fix(iss): harden validation and add missing edge case tests

- Remove unused TypeScript interfaces from route validators,
  align with earthquake route's bare validation style
- Add velocity validation to positions route (was missing,
  inconsistent with main route)
- Add null body tests for position and TLE routes
- Add empty string timestamps test for positions route
- Add NaN velocity rejection test for positions route
- Add network error test for positions route (was missing)
2026-05-17 17:36:51 +12:00
SilvertakanaandCopilot Autofix powered by AI fc7dd02ee2 refactor(core): harden typescript and fix linting violations (#146)
* chore(merge): resolve conflicts, fix mockWs error, and apply refactor standards

* Potential fix for pull request finding 'CodeQL / Incomplete URL substring sanitization'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for pull request finding 'CodeQL / Incomplete URL substring sanitization'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for pull request finding 'CodeQL / Incomplete URL substring sanitization'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix(ci): Stabilize github actions and resolve test failures

- fix(tests): update WsClient mock to use standard function for Vitest constructor compatibility

- fix(db): correct PrismaClient import path to use generated output directory

- fix(e2e): add predev database boot step to playwright CI workflow

- chore: sync pnpm-lock.yaml after merge conflict resolution

* refactor(plugins): decouple GeoJSON plugin and implement data bus lifecycle

- feat(sdk): [Minor] Added dynamicPluginCreate, dynamicPluginRemove, and pluginUnregistered to DataBusEvents.

- feat(core): [Minor] Implemented unregisterPlugin in PluginManager.ts for automated IndexedDB caching cleanup and polling teardown.

- feat(core): [Minor] Added removeLayer to layersSlice.ts to support safe removal of plugin states.

- refactor(plugins): [Patch] Refactored useGeoJsonImport.ts and ImportedLayerList.tsx to decouple from direct manager imports, utilizing DataBus for lifecycle events.

- refactor(core): [Patch] Added pluginUnregistered listener to DataBusSubscriber.tsx to handle state and cache cleanup synchronously via event.

* fix(types): resolve PluginManager type errors

* refactor(core): decouple PluginManager and fix rendering invariants

- refactor(core): Replace Zustand mutations in PluginManager with DataBus events (Patch)
- refactor(sdk): Add layerLoadingChanged, pluginError, pluginUnregistered to DataBusEvents (Patch)
- refactor(ui): Subscribe to new DataBus events in DataBusSubscriber (Patch)
- fix(plugins): Remove size from BaseAviationPlugin to fix model rendering clipping (Patch)
- fix(plugins): Migrate BaseIncidentPlugin from size to iconScale to fix billboard clipping (Patch)

* chore: resolve lockfile conflict

* fix: restore BottomPanelManager in AppShell and fix vitest config

* fix(layout): restore bottom panel positioning and transition styling

This commit corrects the missing bottom panel issue by resolving CSS aggregation errors and fixing transition logic. The version is bumped to 2.15.5.

* refactor(core): harden typescript and fix linting violations

This commit addresses the remaining any types and linting errors across the core engine.

* fix(eslint): resolve react-hook immutability and cascading render violations

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-05-17 17:36:33 +12:00
Antigravity AI caf1638fdc ci(playwright): Add multi-browser test matrix (Patch) 2026-05-17 15:51:48 +12:00
SilvertakanaandCopilot Autofix powered by AI ad3c7845a5 fix: restore bottom panel system and resolve E2E test failure (#145)
* chore(merge): resolve conflicts, fix mockWs error, and apply refactor standards

* Potential fix for pull request finding 'CodeQL / Incomplete URL substring sanitization'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for pull request finding 'CodeQL / Incomplete URL substring sanitization'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for pull request finding 'CodeQL / Incomplete URL substring sanitization'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix(ci): Stabilize github actions and resolve test failures

- fix(tests): update WsClient mock to use standard function for Vitest constructor compatibility

- fix(db): correct PrismaClient import path to use generated output directory

- fix(e2e): add predev database boot step to playwright CI workflow

- chore: sync pnpm-lock.yaml after merge conflict resolution

* refactor(plugins): decouple GeoJSON plugin and implement data bus lifecycle

- feat(sdk): [Minor] Added dynamicPluginCreate, dynamicPluginRemove, and pluginUnregistered to DataBusEvents.

- feat(core): [Minor] Implemented unregisterPlugin in PluginManager.ts for automated IndexedDB caching cleanup and polling teardown.

- feat(core): [Minor] Added removeLayer to layersSlice.ts to support safe removal of plugin states.

- refactor(plugins): [Patch] Refactored useGeoJsonImport.ts and ImportedLayerList.tsx to decouple from direct manager imports, utilizing DataBus for lifecycle events.

- refactor(core): [Patch] Added pluginUnregistered listener to DataBusSubscriber.tsx to handle state and cache cleanup synchronously via event.

* fix(types): resolve PluginManager type errors

* refactor(core): decouple PluginManager and fix rendering invariants

- refactor(core): Replace Zustand mutations in PluginManager with DataBus events (Patch)
- refactor(sdk): Add layerLoadingChanged, pluginError, pluginUnregistered to DataBusEvents (Patch)
- refactor(ui): Subscribe to new DataBus events in DataBusSubscriber (Patch)
- fix(plugins): Remove size from BaseAviationPlugin to fix model rendering clipping (Patch)
- fix(plugins): Migrate BaseIncidentPlugin from size to iconScale to fix billboard clipping (Patch)

* chore: resolve lockfile conflict

* fix: restore BottomPanelManager in AppShell and fix vitest config

* fix(layout): restore bottom panel positioning and transition styling

This commit corrects the missing bottom panel issue by resolving CSS aggregation errors and fixing transition logic. The version is bumped to 2.15.5.

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-05-16 15:33:43 +12:00
Silvertakana 339bf6c0f3 fix(plugins): ensure plugin fetch uses HTTP scheme (#140)
* fix(plugin-manager): downgrade non-fatal 'Failed to fetch' errors to warn, remove polling re-throw

* fix(codeql): resolve format string vulnerability in console logging

* fix(ci): restore playwright dependencies and configure tsconfig properly

* fix(plugins): ensure plugin fetch uses HTTP scheme instead of WSS
2026-05-16 13:03:50 +12:00
SilvertakanaandCopilot Autofix powered by AI 16c82c9184 refactor(lint): address React 19 hook purity and immutability violations (#136)
* chore(merge): resolve conflicts, fix mockWs error, and apply refactor standards

* Potential fix for pull request finding 'CodeQL / Incomplete URL substring sanitization'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for pull request finding 'CodeQL / Incomplete URL substring sanitization'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for pull request finding 'CodeQL / Incomplete URL substring sanitization'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix(ci): Stabilize github actions and resolve test failures

- fix(tests): update WsClient mock to use standard function for Vitest constructor compatibility

- fix(db): correct PrismaClient import path to use generated output directory

- fix(e2e): add predev database boot step to playwright CI workflow

- chore: sync pnpm-lock.yaml after merge conflict resolution

* refactor(plugins): decouple GeoJSON plugin and implement data bus lifecycle

- feat(sdk): [Minor] Added dynamicPluginCreate, dynamicPluginRemove, and pluginUnregistered to DataBusEvents.

- feat(core): [Minor] Implemented unregisterPlugin in PluginManager.ts for automated IndexedDB caching cleanup and polling teardown.

- feat(core): [Minor] Added removeLayer to layersSlice.ts to support safe removal of plugin states.

- refactor(plugins): [Patch] Refactored useGeoJsonImport.ts and ImportedLayerList.tsx to decouple from direct manager imports, utilizing DataBus for lifecycle events.

- refactor(core): [Patch] Added pluginUnregistered listener to DataBusSubscriber.tsx to handle state and cache cleanup synchronously via event.

* fix(types): resolve PluginManager type errors

* refactor(core): decouple PluginManager and fix rendering invariants

- refactor(core): Replace Zustand mutations in PluginManager with DataBus events (Patch)
- refactor(sdk): Add layerLoadingChanged, pluginError, pluginUnregistered to DataBusEvents (Patch)
- refactor(ui): Subscribe to new DataBus events in DataBusSubscriber (Patch)
- fix(plugins): Remove size from BaseAviationPlugin to fix model rendering clipping (Patch)
- fix(plugins): Migrate BaseIncidentPlugin from size to iconScale to fix billboard clipping (Patch)

* chore: resolve lockfile conflict

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-05-16 13:03:05 +12:00
Silvertakana ad324a6592 feat(ui): bottom panel plugin architecture testing (#141)
* test(ui): implement playwright end-to-end testing for bottom panel architecture

* docs(e2e): improve playwright testing rules and create mock plugin guide
2026-05-16 12:29:09 +12:00
devangpratap 0a51e198d7 feat(globe): add OSM 3D Buildings overlay for non-Google imagery layers (#135)
Loads Cesium OSM Buildings (Ion asset 96188) as a 3DTileset when the
active base layer is not Google Photorealistic 3D Tiles. Buildings
only render in 3D scene mode and are automatically removed when
switching to Google 3D, 2D, or Columbus View.

Adds a "3D Buildings" toggle in Graphics Settings, persisted via
the existing wwv_graphics cookie.
2026-05-16 12:06:30 +12:00
Silvertakana 1d6381182e fix(plugin-manager): downgrade non-fatal 'Failed to fetch' errors to warn (#139)
* fix(plugin-manager): downgrade non-fatal 'Failed to fetch' errors to warn, remove polling re-throw

* fix(codeql): resolve format string vulnerability in console logging

* fix(ci): restore playwright dependencies and configure tsconfig properly
2026-05-16 11:39:41 +12:00
Antigravity AI beb295445f fix(data-engine): properly respect NEXT_PUBLIC_WWV_PLUGIN_DATA_ENGINE_URL in getLocalWsUrl and align port with engineManifest 2026-05-15 22:59:47 +12:00
Silvertakana dd62116351 feat(ui): bottom panel system refinements and visual polish (#137)
- Major: Add DESIGN.md source-of-truth document

- Minor: Update Bottom Panel plugin SDK to include 'enabled' prop

- Fix: Resolve React 19 hydration error in root layout script tag

- Fix: Refine bottom panel transparency, contrast, and resize clamps

- Fix: Transition pill styling for empty panel states
2026-05-15 21:31:39 +12:00
GodisDeadxandAntigravity AI d675d91441 Fix(Plugins): Fix sync-local-plugins looking in wrong location (#102)
Co-authored-by: Antigravity AI <[email protected]>
2026-05-15 15:15:38 +12:00
SilvertakanaandCopilot Autofix powered by AI fb07195ec2 feat(testing): stabilize Playwright E2E Tests infrastructure (#134)
* feat(testing): add Playwright E2E infrastructure and stabilize test suite

- Add Playwright E2E configuration and ui-navigation / plugin-system tests
- Add GitHub Actions workflow for Playwright CI tests
- Add test helpers and fixtures (e2e-mock-plugin)
- fix(testing): resolve UnverifiedPluginBatchDialog UI blocking issue in E2E tests
- fix(testing): limit Playwright to 2 workers locally to avoid Next.js/Prisma exhaustion
- fix(api): replace deprecated processAuthCodeResponse with authorizationCodeGrant
- test(ui): add data-testid to PanelToggleArrows and AppShell

* Potential fix for pull request finding 'CodeQL / Workflow does not contain permissions'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix(ci): restrict GITHUB_TOKEN permissions to read contents

* docs(e2e): document Playwright infrastructure and mock plugin lifecycle

* fix(e2e): explicitly load .env in teardown and fix dev command pathing

* fix(ci): resolve playwright environment setup and database initialization

Changes:
- Patch: Added pnpm run setup step to .github/workflows/playwright.yml to generate .env in CI
- Patch: Added explicit loadEnv() helper to 	ests/global.setup.ts and 	ests/global.teardown.ts to ensure database credentials are fresh
- Patch: Hardened scripts/safe-db-push.mjs to detect missing DATABASE_URL and explicitly prompt for pnpm run setup rather than assuming production
- Patch: Updated scripts/dev.mjs to execute pnpm exec next dev instead of raw
ext dev to resolve pathing issues
- Patch: Bumped version in package.json to 2.14.4

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-05-15 14:57:37 +12:00
devangpratap 1838f1a5cd fix(setup): remove broken legacy migration banner from setup page (#130)
The setup page unconditionally displays a banner telling users to run
scripts/migrate-legacy.mjs, but that script does not exist in the repo
or the Docker image. Every user hitting /setup sees a broken command,
including fresh installs with no legacy data.

The docker-entrypoint.sh already detects prisma/dev.db and logs a
migration warning for actual upgraders.

Closes #129
2026-05-15 13:20:27 +12:00
Silvertakana 28e4e67043 feat: Multi-Worktree Database Lifecycle Automation and Deterministic Ports (#133)
* feat: implement Extensible Bottom Panels and Worktree DB automation

- feat: Implemented BottomPanelManager and circular navigation dock for plugins
- feat: Added dynamic DATABASE_URL injection with deterministic port assignments
- fix: Added 3-retry mechanism to handle file lock (EBUSY / os error 32) issues
- feat: Added telemetry for boot sequence visibility

* fix(ui): resolve TS2339 by accessing plugin.name directly instead of plugin.metadata
2026-05-15 11:54:55 +12:00
Silvertakana 594f85afb2 feat(dx): implement worktree DB orchestration and graceful dev teardown (#132)
* feat(dx): implement worktree DB orchestration and graceful dev teardown

- feat(dx): [Minor] Implement deterministic port binding per git worktree
  to mitigate Docker port conflicts during parallel development.
- feat(dx): [Minor] Implement graceful development environment teardown
  using a custom node orchestrator (scripts/dev.mjs) replacing CLI concurrently,
  enabling cross-platform tree-kill and optional DB stopping on exit.
- build(deps): Add tree-kill for process termination.
- fix(dx): Prevent rm -rf from orphaning Docker volumes by adding
  a db:prune script and tying volume cleanup to git-wt remove.

* fix: update openid-client api usage to v6 and resolve db import

- Migrated processAuthCodeResponse to authorizationCodeGrant
- Updated db import to export prisma client correctly
- Updated unit tests with v6 openid-client mocks
- Bumped version to 2.13.1

* fix(ci): setup .env from example in local dev CI test
2026-05-15 11:20:29 +12:00
Antigravity AI b29133fca0 feat: add watcher script for automatic local plugin synchronization 2026-05-14 22:23:52 +12:00
Antigravity AI ac0092c8a8 ci: add workflow to test local development environment startup and predev tasks 2026-05-14 22:19:44 +12:00
Antigravity AI f0daefd53d chore: map postgres port to host for external access in docker-compose 2026-05-14 22:09:35 +12:00
Antigravity AI 94f8a47e0d feat(auth): implement PKCE token exchange and AES credential encryption (Phase 3)
- Minor: Implement /api/marketplace/callback route with openid-client
- Minor: Add invoke-code-review.ps1 script to automate code review subagent dispatch
- Fix: Remove duplicate global skills from .agents/skills/
- Docs: Update ADR-0001 references in rule files and AGENTS.md
2026-05-14 21:03:23 +12:00
Antigravity AI 161401a0af docs(architecture): update marketplace architecture rule with PKCE token exchange flow 2026-05-14 20:51:12 +12:00
Antigravity AI 399b811c7e fix: resolve typescript typecheck errors in ssrf utility 2026-05-14 20:42:54 +12:00
Antigravity AI 786e60865a fix: resolve SSRF and encryption bugs from premortem 2026-05-14 19:23:56 +12:00
Antigravity AI ded2d587e4 feat(auth): implement decentralized plugin auth and SSRF hardening
- Add PBKDF2 + AES-256-GCM encryption for credentials
- Update Prisma schema to include MarketplaceCredential
- Implement comprehensive SSRF utility with IP pinning
- Harden /api/camera/proxy and stream proxy with safeFetch
- Fix openid-client PKCE generator usage
- Update workspace configurations and ADRs
- Bump version to 2.10.0 (Minor)
2026-05-14 19:17:09 +12:00
SilvertakanaandCopilot Autofix powered by AI b177c6987d feat(wwv-plugin-sdk): add shared auth contracts for decentralized plugin auth (#127)
* refactor(docker): separate self-host Docker compose structure from development

test(ci): add github action to verify self-host scripts

* feat(wwv-plugin-sdk): add shared auth contracts for decentralized plugin auth

* Potential fix for pull request finding 'CodeQL / Workflow does not contain permissions'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix(wwv-plugin-sdk): wrap auth contract type tests in vitest suite

* fix(wwv-plugin-sdk): implement PR review feedback for auth contracts

* fix(wwv-plugin-sdk): address PR comments on auth contracts

* docs(rules): codify decentralized plugin auth contracts and security rules

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-05-14 19:01:39 +12:00
SilvertakanaandCopilot Autofix powered by AI b11125fd5f refactor(docker): separate self-host Docker compose structure from development (#125)
* refactor(docker): separate self-host Docker compose structure from development

test(ci): add github action to verify self-host scripts

* Potential fix for pull request finding 'CodeQL / Workflow does not contain permissions'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-05-14 14:22:40 +12:00
380 changed files with 13837 additions and 9236 deletions

No files matched your search

+223
View File
@@ -0,0 +1,223 @@
---
name: garbage-collector
description: Daily maintenance agent that reads pre-scanned gc-findings.json, then opens Tier A draft PRs (mechanical fixes like removing stale TODOs and console.logs) and Tier B GitHub Issues (architectural drift, oversized files, outdated deps). Runs headless via the garbage-collector.yml cron. Also invokable interactively as @garbage-collector for manual sweeps.
tools: Read, Grep, Glob, Bash, Edit, Write
model: haiku
color: orange
---
You are the WorldWideView Garbage Collector — a low-cost maintenance agent responsible for keeping the codebase clean between human-led development cycles.
## Step 1 — Read your charter
```bash
cat .agents/rules/garbage-collection.md
```
Internalize all guard rails, caps, and the never-touch list before taking any action.
## Step 2 — Read the findings
```bash
cat gc-findings.json
```
This was produced by `scripts/gc-scan.mjs` before you were invoked. You do **not** re-scan the codebase — you triage this list only.
If `gc-findings.json` does not exist or is empty, output "No findings to process." and stop.
## Step 3 — Check dry-run mode
```bash
echo "${DRY_RUN:-false}"
```
If `true`, jump to **Step 7 (Dry-run summary)** and skip all PR/Issue creation.
## Step 4 — Check for existing GC work (idempotency)
```bash
# Open gc-bot PRs
gh pr list --label gc-bot --state open --json number,title,headRefName,body
# Existing gc/* remote branches
git branch -r --list 'origin/gc/*'
# Open gc-bot Issues
gh issue list --label gc-bot --state open --json number,title
```
Build a mental map of already-handled concerns. For each finding, skip it if an open PR or Issue already covers the same file + pattern.
## Step 5 — Tier A: Create draft PRs (max 3 per run)
Group related Tier A findings by `patternId` or `type`. Process the highest-value group first (most findings → most cleanup per PR).
For each group (up to 3 total):
### 5a — Create a branch
```bash
SHA=$(git rev-parse --short HEAD)
BRANCH="gc/<patternId>-${SHA}"
git checkout -b "${BRANCH}"
```
### 5b — Apply the mechanical fix
Follow these per-type rules:
**stale-todo (issueClosed: true)**
Delete the entire comment line. Do not change surrounding code.
**stale-todo (issueClosed: false, age > 90d)**
Add `# STALE(gc-bot):` prefix to the comment so the human sees it needs resolution. Do not delete — the human decides. This makes it Tier A-lite (no code risk, just marks it).
**anti-pattern: mdc-ref**
Replace `.mdc` with `.md` in the file content. Check that the referenced file actually exists with `.md` extension before renaming.
**anti-pattern: console-log**
Delete the `console.log(...)` line entirely. If it's the only statement in a block, delete the block too (only if the block has no other statements).
**anti-pattern: ts-ignore**
Delete the `// @ts-ignore` line. If the next line now has a type error that is trivially fixable (e.g., adding a missing cast), fix it. If the type error is non-trivial, revert to Tier B (create an Issue instead).
**anti-pattern: ts-nocheck**
Delete the `// @ts-nocheck` line. Check if the file still compiles cleanly with `npx tsc --noEmit --skipLibCheck` — if not, revert the deletion and file a Tier B Issue instead.
### 5c — Verify the change is safe
```bash
# Count changed lines — must be ≤150
git diff --stat
# Quick type check on changed files only
npx tsc --noEmit --skipLibCheck 2>&1 | head -20
```
If changed lines exceed 150 or type errors are introduced, `git checkout .` and file a Tier B Issue for this group instead.
### 5d — Commit and push
```bash
git add -p # stage only the GC changes
git commit -m "chore: [gc] remove <N> stale <type> findings"
git push origin "${BRANCH}"
```
### 5e — Open a draft PR
```bash
gh pr create --draft --label gc-bot \
--title "chore: [gc] <short description (≤60 chars)>" \
--body "$(cat <<'EOF'
## What changed
<bullet list of files and what was removed/changed>
## Why
Detector: `<patternId>` in `scripts/gc-scan.mjs`
Rule violated: `<rule from garbage-collection.md>`
Age / evidence: <age in days, or closed issue number>
## How to verify
1. <step>
2. Run `pnpm test` — all tests should still pass
3. Run `pnpm build` — build must be clean
> This PR was opened automatically by the GC bot. Review before merging.
EOF
)"
```
Return to main: `git checkout main`
## Step 6 — Tier B: Create GitHub Issues (max 5 per run)
Group related Tier B findings into one Issue per `type` (e.g., all oversized files in one Issue).
```bash
gh issue create --label gc-bot \
--title "[gc] <type>: <short description>" \
--body "$(cat <<'EOF'
## Summary
<what was detected and why it matters>
## Files / packages affected
<file list with line counts or versions>
## Suggested action
<concrete next step — e.g., "split this file", "upgrade this dep", "fix this rule reference">
## How to close this
Resolve the issue and close this ticket manually.
> Filed automatically by the GC bot on <date>.
EOF
)"
```
## Step 7 — Dry-run summary
Find or create the tracking Issue:
```bash
gh issue list --label gc-bot --state open --search "[gc-bot] Daily Scan Reports" --json number,title
```
If it does not exist:
```bash
gh issue create --label gc-bot \
--title "[gc-bot] Daily Scan Reports" \
--body "This issue collects daily dry-run scan summaries. Enable normal mode by setting DRY_RUN=false in the workflow."
```
Then post a comment:
```bash
gh issue comment <number> --body "$(cat <<'EOF'
## GC Dry-run Report — <date>
| Tier | Count | Types |
|---|---|---|
| A (would fix) | <n> | <comma-separated types> |
| B (would file) | <n> | <comma-separated types> |
### Tier A findings
<table: file | line | type | description>
### Tier B findings
<table: file/package | type | description>
No PRs or Issues were created (dry-run mode).
EOF
)"
```
## Step 8 — Output summary
Print a clean markdown summary:
```
## GC Run Complete
### PRs opened (<n>/3)
- #<number>: <title> (<branch>)
### Issues opened (<n>/5)
- #<number>: <title>
### Skipped (already handled)
- <n> findings skipped — covered by existing open work
### Not actioned
- <n> Tier B findings below cap — will appear in next run
```
If nothing was created: "No new findings requiring action."
## Important invariants
- Never touch the never-touch list in the charter.
- Never create a PR with more than 150 changed lines.
- Never mark a PR as ready-for-review.
- Never push directly to `main`.
- If a Tier A fix risks introducing a regression and you cannot verify it is safe, file a Tier B Issue instead — safety always beats throughput.
-169
View File
@@ -1,169 +0,0 @@
---
trigger: model_decision
description: Architecture rules and conventions for the WorldwideView Next.js 16 frontend application, state management, and CesiumJS integration.
---
# WorldwideView Application Architecture
## 1. Core Framework
- Built on **Next.js 16 (App Router)** with `output: "standalone"`.
- Uses **React 19** and **TypeScript 5** (strict mode).
- **Styling**: Strictly Vanilla CSS. No Tailwind or CSS-in-JS.
- **Component File Size**: Max 150 lines per file. Split components, extract helpers, and use hooks for anything larger.
## 2. State & Event Bus
- **Zustand**: Slices handle global state (`globe`, `layers`, `timeline`, `ui`, `filter`, `data`, `config`, `favorites`, `geojson`).
- **DataBus**: Custom typed pub/sub singleton (`DataBus.getInstance().emit()`) decoupling React components from background WebSocket streams.
## 3. Edition System
Features are controlled by `NEXT_PUBLIC_WWV_EDITION`:
- `local`: Self-hosted, full features, auth.
- `cloud`: Managed instance.
- `demo`: Public demo, no auth.
## Source Layout
```
worldwideview/
├── .agents/ # AI agent workflows, rules, memories, skills
├── docs/ # Public-facing documentation
├── artifacts/
│ └── context/ # ← THIS FOLDER — private source of truth (not committed)
├── packages/ # Plugin packages (workspaces)
├── public/ # Static assets (Cesium assets, GeoJSON data files)
├── scripts/ # Build/utility scripts (copy-cesium.mjs, scaffold-osm-plugin.mjs, etc.)
└── src/
├── app/ # Next.js App Router (layouts, pages, API routes)
├── components/ # React UI components (HUD, LayersPanel, Panels, Sidebar)
├── core/
│ ├── data/ # DataBus, PollingManager, CacheLayer
│ ├── globe/ # Cesium hooks, viewer lifecycle, primitive collections
│ └── plugins/ # Plugin registry, lifecycle, loaders
├── lib/ # Shared utilities, server-side logic
│ └── marketplace/ # Marketplace bridge (auth, token, CORS, repository)
└── plugins/ # Built-in feature plugins (aviation, maritime, borders, etc.)
```
---
## Module Breakdown
### `src/core/data/` — The Heartbeat
- **DataBus**: Decentralized event pipeline for all system actions. Plugins emit, components subscribe.
- **PollingManager**: Intelligent scheduler for external API calls with exponential backoff.
- **CacheLayer**: 2-stage persistent caching — In-Memory + IndexedDB (`worldwideview-cache`).
### `src/core/globe/` — Rendering Engine
- **Cesium Integration**: Low-level control over the Cesium Viewer and Primitive Collections.
- **EntityRenderer**: High-performance Primitive-based renderer. See `Primitives vs. Entities` below.
- **Spiderifier & Clustering Engine** (`StackManager.ts` & `stackAnimation.ts`):
- Automatically identifies overlapping entities and groups them into geometrically balanced clusters (concentric rings for <18 density, Fermat's golden spiral for massive stacks).
- **Mobile Layout Constraints**: Integrates responsive coordinate geometry. Modifies logical icon dimensions via `iconUpscaler.ts` (32px mobile / 48px desktop) and scales the radial expansion offsets proportionally by 70% on mobile displays to ensure tight, aesthetically pleasing clusters without massive gaps.
- **Z-Fighting Mitigation**: Clustering involves highly frequent stack rebuild cycles on any zoom/pan delta. To prevent chaotic render flickering, the global `AnimationLoop` explicitly yields visibility management to `stackAnimation.ts` via `isEntityClustered()`. Items transitioning into the `collapsed` state intentionally bypass animation lerping (to prevent 220ms ghost visibility bleed) and mathematically cloak themselves instantly.
- Globe is initialized via `GlobeView.tsx`. Viewer instance passed via `CesiumContext`.
### `src/core/plugins/` — Plugin System
- **PluginManager**: Registers, boots, and destroys plugins. Enforces capability checks.
- **Loaders**: Plugins are dynamically imported at runtime as ES module bundles (`loadPluginFromManifest`). The legacy `StaticDataPlugin` and `DeclarativePlugin` runtimes are fully deprecated.
- **Manifest Requirement**: All plugins MUST include a `"worldwideview"` manifest block in their `package.json` (containing `id`, `icon`, `category`, and `format`). This is strictly required by the Marketplace in order to parse display attributes correctly.
---
## Performance: Primitives vs. Entities
**Core design decision:** Use Cesium `Primitive` API instead of the high-level `Entity` API for high-count datasets.
| Feature | Entity API | Primitive API (WWV) |
|---|---|---|
| **Abstraction** | High (easy to use) | Low (direct GPU access) |
| **JS Overhead** | Significant per-entity objects | Minimal (batched draw calls) |
| **Max Capacity** | ~1,000 entities | **100,000+ points/billboards** |
| **WWV Use Case** | Info window / detail panel content | Live data point visualization |
Specifically: `PointPrimitiveCollection` and `BillboardCollection` ensure 60FPS even with dense global data. The full migration from Entity-based rendering to Primitives was completed to eliminate severe FPS drops.
---
## Globe Occlusion & Depth Testing
- **Correct approach:** `depthTestAgainstTerrain = false` — uses GPU to depth-test against the smooth WGS84 ellipsoid. Icons visible on the near hemisphere, correctly occluded behind Earth.
- **Avoid:** `eyeOffset`, `disableDepthTestDistance` hacks that push entities in front of everything. These cause icons to bleed through the globe.
- The Plugin SDK's `CesiumEntityOptions` exposes this via `disableDepthTestDistance` when absolutely needed for specific usecases.
---
## Data Pipeline (Example: Aviation)
```
Remote API (e.g. OpenSky)
↓ [raw JSON, every ~30s via PollingManager]
Plugin (e.g. AviationPlugin)
↓ [parse, filter, sanitize → GeoEntity[]]
DataBus.emit("dataUpdated", { entities })
↓
EntityRenderer [batched Primitive update]
↓
GPU (CesiumJS) → 60FPS visualization
```
---
## Database (Unified PostgreSQL)
**Location:** We have migrated from a fragmented SQLite + PostgreSQL setup to a unified **PostgreSQL** architecture using Prisma 7 with the `@prisma/adapter-pg` driver. This enables true multi-tenant cloud capability, real-time sync, and scalable connection pooling via `DATABASE_URL`.
**Next.js Core Tables:**
| Model | Table | Purpose |
|---|---|---|
| `User` | `users` | Auth accounts (email, bcrypt password, role) |
| `InstalledPlugin` | `installed_plugins` | Marketplace plugin install records |
| `Setting` | `settings` | Key-value app config (Cesium token, map style, etc.) |
| `Tenant` | `tenants` | Multi-tenant organization boundaries |
**Data Engine Tables:**
Managed explicitly via optimized queries in `packages/wwv-data-engine/src/db.ts` targeting PostgreSQL to maximize ingest performance.
Includes tables like `iranwar_events`, `earthquakes`, `wildfires`, and `maritime_history`.
> [!WARNING]
> **Catastrophic Time-Series Polling (The Lookback Trap):**
> High-frequency polling (e.g. 6,000 aircraft every 10s) accumulates millions of rows rapidly.
> - **DO NOT** query unindexed columns (like `ts`) for lookback/freshness without a dedicated index, otherwise you will trigger full table scans and 500 Route timeout errors.
> - **REQUIRED:** Implement aggressive data retention / hourly pruning via `scheduler.ts` to prevent infinite table growth.
> - **PREFERRED:** Keep real-time / spatial lookbacks in Redis (like `getLiveSnapshot()`), and leave PostgreSQL strictly for historical chronological replays that hit standard indices.
**Hot-Caching (Redis) & Connection Rules:**
The `wwv-data-engine` polls external APIs, dumps raw payloads to PostgreSQL for historical persistence, and broadcasts active snapshots via WebSockets while persisting to Upstash Redis. The Next.js frontend fetches initial state from Redis to ensure horizontal scalability.
*TLS Connection Requirement:* The `REDIS_URL` mapped in Coolify and `.env.local` must use the `rediss://` protocol for TLS encryption. Attempting to use plain `redis://` with Upstash will cause the connection to drop (`ECONNRESET`) and throw a `MaxRetriesPerRequestError (which is 3)` inside the data engine.
*Upstash Request Limit Throttling:* Due to Upstash Serverless billing (capped at 10k requests/day or 500k/month on free tier), the data engine decouples WebSocket live broadcasting from Redis persistence. High-frequency scrapers (like Aviation) run every 15 seconds to push WebSockets, but the backend forces a **5-minute throttle** before saving the next snapshot to Upstash using `zlib` compression. This executes just 2 commands (`SET` and `EXPIRE`) per 5 minutes per scraper, completely eliminating `max requests limit exceeded` errors while retaining real-time frontend updates.
**Resilience (Data Self-Healing):**
Even with persistent PostgreSQL, critical history (like `iranwarlive` events) should be bundled as `seed.json` or `history.csv` artifacts. If the Data Engine detects an empty table, it MUST parse its built-in fallback seed to hydrate the database and Redis, ensuring 100% data recovery without manual intervention.
**Dynamic Engine URL Resolution:**
The decoupled Data Engine lives at `http://localhost:5001` during local development, and deploys to `https://dataengine.worldwideview.dev` in production. The Next.js frontend injects this context directly into the plugin via the `ctx.env` object (specifically reading from `NEXT_PUBLIC_WWV_PLUGIN_DATA_ENGINE_URL` environment variables). Plugins resolve their required backend natively by parsing `this.context.env.DATA_ENGINE_URL`. The legacy `globalThis.__WWV_ENGINE_URL__` proxy approach has been fully deprecated.
---
## Settings Architecture
User-configurable values live in the `settings` table (not `.env`). They can be updated without rebuilds.
| Setting Type | Where | Example |
|---|---|---|
| **Infrastructure** | `.env` | `DATABASE_URL`, `NEXT_PUBLIC_WWV_EDITION` |
| **User settings** | `settings` table (DB) | Cesium token, map style, API keys |
**Setup Wizard:** If `cesium_token` is missing from the DB on first load, the app shows a setup wizard. Plugin-level wizards work the same way — plugin declares required settings in its manifest; if missing, it renders a setup form instead of data.
---
## Design Principles
- **Single Responsibility:** Plugins only handle data mapping — they don't know about UI or cache.
- **Dependency Inversion:** `PluginManager` communicates via the `WorldPlugin` interface — new plugins don't require core changes.
- **Event-Driven:** Components subscribe to what they need via DataBus — no prop-drilling.
- **Defensive Programming:** All external calls wrapped in error boundaries + `PollingManager` backoff.
+21 -2
View File
@@ -1,7 +1,9 @@
---
trigger: glob
description: Strict guidelines for rendering geospatial entities, handling 3D terrain clipping, and applying dynamic animation properties against Cesium's runtime engine.
globs: "src/core/globe/**/*"
paths:
- "src/core/globe/**/*"
- "src/plugins/**/*"
- "packages/wwv-plugin-*/src/**/*"
---
# Cesium Rendering Rules
@@ -23,6 +25,23 @@ When creating primitive objects:
- **Points** (`PointPrimitive`): Can use `size`, `outlineWidth`, and `outlineColor`.
- **Billboards** (`BillboardGraphics` for SVG/Icons): Must **NOT** use `size`, `outlineWidth`, or `outlineColor`. You must use `iconScale` to resize them. Mixing point properties on billboards causes severe shader clipping.
## Sizing Conventions
| Entity type | Knob | Default | Typical band | Source |
|-------------|--------------|------------------------------------|------------------|--------|
| `point` | `size` | `8` desktop / `12` mobile | 5–12 | `src/core/globe/primitiveOps.ts` (`defaultPointSize`) |
| `billboard` | `iconScale` | `0.7` (`DEFAULT_BILLBOARD_SCALE`) | 0.6–0.9 | `src/core/globe/primitiveOps.ts` (`DEFAULT_BILLBOARD_SCALE`) |
| `model` | `modelScale` | `1.0` + `modelMinPixelSize` | varies per asset | `src/core/globe/ModelManager.ts`, `src/core/globe/hooks/useModelRendering.ts` |
> [!IMPORTANT]
> **Cross-plugin parity rule.** If your plugin renders alongside maritime / aviation / military-aviation on the globe, **leave `iconScale` unset** (or pick a value inside 0.6–0.9). A value below 0.5 will look visibly broken next to peer plugins. A value above 1.0 will dominate the globe.
> [!CAUTION]
> **Silent-ignore footgun.** `size` is a `PointPrimitive` property. If you set `size` on a `billboard` entity it is **silently dropped** — Cesium does not error and the billboard simply renders at `DEFAULT_BILLBOARD_SCALE`. When converting a point plugin to a billboard plugin, replace `size: N` with `iconScale: ~N/16` (the canvas base is `getBaseSize() = 48px` desktop) — do not delete it and do not translate it literally as `iconScale: N/10`. This footgun has shipped at least once (see commit history on `local-plugins/wwv-plugin-satellite`).
> [!NOTE]
> **Backdrop stack.** `createSvgIconUrl` (in the SDK) already wraps the icon in a 44×44 SVG with a dark backdrop circle. `src/core/globe/iconUpscaler.ts` then redraws onto a 48px canvas with **another** backdrop circle (`DEFAULT_BG = "rgba(15, 23, 42, 0.55)"`). Pass `background: false` to `createSvgIconUrl` when you want only the host's circle. Don't add a third backdrop in your own SVG.
## Performance Culling
> [!TIP]
+28 -1
View File
@@ -1,8 +1,16 @@
---
trigger: model_decision
description: Cloud edition implementation details, PostgreSQL Row-Level Security (RLS) isolation, cross-subdomain auth, and cryptographic license keys.
paths:
- "src/lib/auth*"
- "src/app/api/auth/**/*"
- "src/core/auth.ts"
- "src/core/storage.ts"
- "src/core/tenant.ts"
---
> **SUPERSEDED by ADR-003** (`docs/architecture/decisions/adr-0003-shared-identity-and-ecosystem-auth-host.md`).
> The auth host is `worldwideview.dev` (worldwideview-web repo), not `app.worldwideview.dev`. Read ADR-003 first.
# WorldWideView — Cloud & Auth Architecture
@@ -14,6 +22,16 @@ description: Cloud edition implementation details, PostgreSQL Row-Level Security
## Auth Strategy
> [!WARNING]
> **This section is superseded by [ADR-003](../../docs/architecture/decisions/adr-0003-shared-identity-and-ecosystem-auth-host.md) (2026-05-22).**
> The decisions below describing `app.worldwideview.dev` as the auth owner and NextAuth as the ecosystem IdP are no longer current. The new architecture is:
> - **`worldwideview.dev`** (the `worldwideview-web` repo) owns all login, signup, and account management UI
> - **Supabase Auth** (single shared project) is the identity provider for all products
> - Session cookies are scoped to `.worldwideview.dev` and shared across all subdomains
> - The Marketplace never owns auth UI and never redirects to `app.worldwideview.dev`
>
> The multi-tenant RLS, tier matrix, license key, and CI/CD sections below remain valid.
**Auth.js (NextAuth)** is the single auth API across all editions. `app.worldwideview.dev` owns **all auth UI** — login, registration, password reset. No other subdomain ever shows a login or registration form.
```typescript
@@ -36,6 +54,15 @@ Application code is **identical** across editions — always calls `auth()`, `si
---
## Decentralized Plugin Authentication (Token Exchange)
For cross-domain plugin communication (Local App connecting to Marketplace or Data Engines), WWV uses an **Asymmetric JWT Token Exchange (PKCE)** model.
- **PKCE Flow**: The Local App uses standard OAuth 2.0 PKCE to exchange a `code` for a long-lived API key from the Marketplace.
- **Token Exchange**: The Local App exchanges its long-lived API key for a short-lived, audience-bound JWT before connecting to a decentralized Data Engine.
- **WebSocket First-Message Auth**: Data Engines enforce a "First-Message Auth" pattern over WebSockets. The client MUST immediately send a `{ type: "auth", v: 1, token: "..." }` message upon opening the connection to prevent SSRF vulnerabilities and credential leakage.
---
## Registration & Auto-Provisioning
Registration: `app.worldwideview.dev/register`
-21
View File
@@ -1,21 +0,0 @@
---
trigger: always_on
---
# Context and Memory Synchronization
## 1. Purpose
This specification ensures that any agent maintains persistent, accurate project context by pointing them directly to the Single Source of Truth for this repository.
## 2. Synchronization Target
The agent MUST orient itself by immediately reading:
1. `c:\dev\worldwideview\AGENTS.md` (The root index of all technical stack details)
2. `c:\dev\worldwideview\.agents\rules\platform-architecture.md` (The high-level platform goals)
If deep technical rules are required for a specific domain (e.g., UI state, Map Rendering, Monorepo management), the agent MUST consult the relevant file in `c:\dev\worldwideview\.agents\rules\`.
## 3. Reflecting Changes
If the architecture fundamentally changes (e.g., migrating off Zustand, or removing Cesium), the agent MUST proactively update the `AGENTS.md` and relevant files in `.agents/rules/` to reflect this new reality.
## 4. Goal
A new chat instance MUST be able to reconstruct an accurate, comprehensive understanding of the project solely from reading the root `AGENTS.md` file and branching out to the specific rules as necessary. This prevents context bloat and ensures accuracy.
-28
View File
@@ -1,28 +0,0 @@
---
trigger: always_on
description: Guidelines for ongoing project maintenance, cleanliness, and self-correction during everyday development tasks.
---
# Continuous Improvement & Maintenance
As an AI agent working within the WorldWideView project, you are expected to practice the **Boy Scout Rule**: always leave the codebase (and its context) cleaner than you found it.
While executing your primary programming tasks, you MUST actively abide by these maintenance protocols:
## 1. Documentation & Rule Parity
- **Self-Correction**: If you notice that an architectural choice you are implementing contradicts an existing rule in `.agents/rules/`, you must halt and update the rule file to reflect the new paradigm.
- **Dead Context**: If you encounter context, plans, or architectural files referencing deprecated concepts (e.g., legacy microservices, `StaticDataPlugin`), proactively prune or update them.
## 2. Codebase Cleanliness
- **Dead Code**: If you refactor a file and leave behind unused functions, imports, or CSS classes, clean them up immediately. Do not leave "commented out" code unless explicitly requested for future use.
- **Console Logs**: Remove debugging `console.log` statements before finalizing a feature, except for structured error logging or critical startup info.
- **Sandbox Hygiene**: Any throwaway scripts or debugging tools placed in `/local-scripts/` should be deleted once the underlying issue is resolved, unless they provide permanent utility.
## 3. Dependency & Typings Checks
- **Type Safety**: Never use `any` or `@ts-ignore` as a shortcut when refactoring. Always build the proper TypeScript interfaces.
- **Manifest Validation**: Whenever you touch a plugin, ensure its `package.json` `"worldwideview"` manifest block is completely accurate and synced with its capabilities.
## 4. The "See Something, Say Something" Rule
If you spot an unrelated but obvious bug, security vulnerability, or architectural anti-pattern in the file you are editing:
1. Fix it if it's a minor, isolated issue (and document the fix in your summary).
2. For larger issues, explicitly call it out to the user so it can be added to the technical debt backlog.
+63 -3
View File
@@ -1,6 +1,9 @@
---
trigger: model_decision
description: Architecture rules and dependency management guidelines for wwv-data-engine seeders.
description: Architecture rules and dependency management guidelines for wwv-data-engine seeders, pnpm workspace dependency loading, and WebSocket payload contracts.
paths:
- "packages/**/*"
- "local-seeders/**/*"
- "docker-compose.yml"
---
# Data Engine V2 Architecture & Dependency Loading
@@ -28,4 +31,61 @@ In production, the `wwv-data-engine-v2` does NOT mount the local file system. In
4. It then runs a single **workspace-aware `pnpm install --prod`**. This ensures all custom unbundled dependencies declared by the extracted plugins are downloaded directly into the container and properly linked together.
## 5. Namespace Separation Rule
To prevent collisions and `404` or `ERR_MODULE_NOT_FOUND` errors, seeders **MUST NOT** exist simultaneously in both the community (`wwv-seeders-community`) and private (`wwv-seeders-private`) repositories. Namespace overlaps will cause module resolution failures when the V2 engine attempts to load them. Private seeders have priority.
To prevent collisions and `404` or `ERR_MODULE_NOT_FOUND` errors, seeders **MUST NOT** exist simultaneously in both the community (`wwv-seeders`, cloned to `local-seeders/community/`) and private (`wwv-seeders-private`, cloned to `local-seeders/private/`) repositories. Namespace overlaps will cause module resolution failures when the V2 engine attempts to load them. Private seeders have priority.
## 6. Plugin ID Contract — Single Source of Truth
> **See ADR-0002** (`docs/architecture/decisions/adr-0002-seeder-exported-name-as-canonical-plugin-id.md`) for full rationale and alternatives considered.
**Rule:** Every seeder MUST export a `name` field that exactly matches the corresponding frontend plugin's `id` field (kebab-case).
```typescript
// ✅ Correct — seeder src/index.ts
export default {
name: "gps-jamming", // MUST equal the frontend plugin id
cron: "0 0 * * *",
fn: seedGpsJam,
};
// ❌ Wrong — do not use folder names, camelCase, or abbreviations
export default {
name: "gpsjam", // mismatch — frontend plugin is "gps-jamming"
...
};
```
**How the engine uses it:** `seeder-loader.ts` reads each seeder's exported `name` as the canonical plugin ID. This ID is used for:
- The `/manifest` endpoint's `plugins` array (what the frontend checks for local routing)
- WebSocket data messages (`pluginId` field)
- Redis keys (`data:<name>:live`)
- Scheduler logs
**Folder names are organizational only.** The engine warns at startup if `toKebabCase(folderName) !== seeder.name`, but this is cosmetic — the seeder's exported `name` always wins. Folder renames are a future cleanup, not a functional requirement.
**Frontend side:** `resolveEngineUrl(pluginId)` checks `localEngineHasPlugin(pluginId)` against the manifest. If the names match, the plugin routes to `ws://localhost:5000/stream`. No alias maps or translation layers are needed.
## 7. WebSocket Payload Format Contract
When a seeder calls `setLiveSnapshot(pluginId, payload, ttl)`, the engine broadcasts:
```json
{ "type": "data", "pluginId": "my-plugin", "payload": <whatever you passed> }
```
**`WsClient` payload handling rules (in order):**
1. Plugin has `mapWebsocketPayload(payload, existingEntities)` → called with raw payload → must return `GeoEntity[]`
2. No handler + payload **is** a flat `GeoEntity[]` array → used directly (timestamp normalized)
3. No handler + payload is **any other object** → **silently dropped** with a console warning
This means:
| Seeder sends | Frontend needs | Notes |
|---|---|---|
| `[{id, latitude, longitude, ...}]` | Nothing (WsClient handles) | ✅ Simplest for basic point data |
| `{ items: [{...}] }` | `mapWebsocketPayload` | Standard engine snapshot format |
| `{ satellites: [{...}] }` | `mapWebsocketPayload` | Named-collection format |
**Recommendation:** If the seeder and frontend plugin are developed together, use a **flat `GeoEntity[]` array** — no `mapWebsocketPayload` needed. Use named objects (e.g. `{ satellites: [...] }`) only when the seeder data has a domain-specific shape worth preserving; in that case you **must** implement `mapWebsocketPayload` on the frontend plugin.
> [!CAUTION]
> The most common silent bug: seeder sends `{ items: [...] }` (an object), frontend plugin omits `mapWebsocketPayload`. Data arrives but WsClient drops it with a console warning. The globe stays empty. Always implement `mapWebsocketPayload` for any plugin whose seeder sends an object payload.
+20 -2
View File
@@ -1,3 +1,9 @@
---
description: Rules for safe Prisma schema changes, defensive migrations, and PostgreSQL DDL operations.
paths:
- "prisma/**/*"
---
# Database Migrations & Schema Modification Rules
## 1. Defensive Migrations
@@ -7,10 +13,22 @@ When making database schema changes, you must ensure that generated migrations a
- **Add "IF EXISTS" clauses**: Prisma can sometimes produce brittle drop statements (`DROP INDEX "my_index"`). To prevent migrations from crashing if a state desynchronization occurs, manually edit the migration file to use `DROP INDEX IF EXISTS "my_index";` or `DROP TABLE IF EXISTS "table_name";`.
- **Data Preservation**: When dropping columns or renaming, carefully review the migration to ensure data is correctly mapped. In PostgreSQL, ensure that cascaded drops don't accidentally remove required data.
## 2. Managing Failed Migrations
## 2. P3005 Auto-Baseline (Handled in `docker-entrypoint.sh`)
`docker-entrypoint.sh` automatically recovers from Prisma **P3005** ("schema not empty, no migration history"). This happens when `pnpm dev` (which runs `prisma db push`) populates the database before the Docker stack is started.
The entrypoint only auto-baselines when **both** conditions hold:
1. The error is exactly P3005.
2. `prisma migrate diff` reports zero schema drift between the live DB and `prisma/schema.prisma`.
If drift is detected the container exits with a clear error message directing you to `pnpm db:reset`.
> **Never add manual `migrate resolve --applied` calls** unless you are outside the container and have independently verified the above two conditions. The entrypoint already handles the common case.
## 3. Managing Failed Migrations
If a migration fails in production:
- **Never blindly mark as resolved**: Do not run `prisma migrate resolve --applied <migration_name>` unless you have explicitly verified that the database schema precisely matches the expected state.
- If you use `resolve --applied`, Prisma will permanently skip that migration in the future, leaving your database structurally out of sync with your Prisma Client.
## 3. Backups Before DDL
## 4. Backups Before DDL
Never perform direct DDL operations (ALTER, DROP, CREATE) on a production database without first confirming an automated backup exists, or manually taking a `pg_dump` backup of the database.
+24
View File
@@ -1,3 +1,12 @@
---
description: Docker build patterns, pnpm dev commands, Vitest strategy, Coolify deployment rules, and security headers.
paths:
- "Dockerfile"
- "docker-compose.yml"
- ".github/**/*"
- "next.config.ts"
---
# Deployment, Testing, and Operations
## Development Commands
@@ -36,8 +45,20 @@ Frontend runs at `http://localhost:3000`.
- **Framework**: Vitest with jsdom environment
- **Coverage**: `src/lib/**`, `src/core/**`, `src/plugins/**`
- **Run**: `pnpm test` (or `vitest run`)
- **Type-Level Contract Testing**: When defining shared TypeScript interfaces/contracts (e.g., in `wwv-plugin-sdk`), you MUST use `expectTypeOf` from `vitest` to assert the actual contract shapes at compile-time instead of writing literal-value identity runtime assertions. You must also enforce `@ts-expect-error` negative type tests to ensure missing or invalid fields trigger compilation errors.
- **Key test files**: `rateLimit.test.ts`, `edition.test.ts`, `demoAdmin.test.ts`, `DeclarativePlugin.test.ts`, `cors.test.ts`, `repository.test.ts`, `marketplaceToken.test.ts`
### E2E Testing (Playwright)
WorldWideView uses Playwright for End-to-End browser testing to verify the complete UI integration including NextAuth.
- **Run**: `pnpm run test:e2e` (Ensure your database is running before executing)
- **Auth Bypass**: Playwright uses a `global.setup.ts` to directly insert a test user into the database and simulate a UI login to capture a `storageState.json` cookie file. This allows all tests to run pre-authenticated without bypassing production security logic.
- **Database Access in Setup**: Because Playwright setup scripts run in a vanilla Node context, importing Next.js backend utilities like `src/lib/db.ts` will crash with `next/headers` resolution errors. You MUST instantiate `PrismaClient` directly using `@prisma/adapter-pg` in any Playwright `setup` or `teardown` scripts.
- **Component Synchronization**: Do not use arbitrary timeouts. Tests MUST wait for the application boot sequence to finish by waiting for the `[data-testid="app-ready"]` marker on the `AppShell` to be attached.
- **Stable Selectors**: Always use `data-testid` properties (e.g. `data-testid="panel-toggle-left"`) for interacting with UI components in E2E tests rather than fragile CSS classes or dynamic text.
- **Worktree Port Conflicts**: If you are testing in a git worktree, configure `playwright.config.ts` to spawn the `webServer` on an alternative port (e.g. `3001`) to avoid colliding with other running worktrees.
## Security Headers
Configured in `next.config.ts` `headers()`:
@@ -247,6 +268,9 @@ The production Coolify instance runs on a local server at `192.168.68.64`. Direc
1. Push to `silvertakana/worldwideview`
2. Coolify webhook triggers → automated BuildKit deployment on the host server.
**GitHub Actions Workflows:**
- **Minimal Permissions:** CodeQL and GitHub security best practices dictate that `GITHUB_TOKEN` permissions should always be explicitly constrained. When creating or modifying `.github/workflows/*.yml` files, you MUST always include an explicit `permissions` block (e.g., `permissions: { contents: read }` or similar required scopes) at the top level or job level to adhere to the principle of least privilege.
**Coolify Deployment Rules & Gotchas (The "Redis 500k" Rule):**
- **Coolify Pulls from GitHub:** Coolify does *not* deploy your local workspace files. It strictly clones the repository from GitHub.
- **Push Before You Deploy:** If you make a critical infrastructure or optimization fix locally (e.g., swapping a massive `HSET` pipeline for a compressed 5-min throttled `setLiveSnapshot`), you **must** `git commit` and `git push` those changes to the `main` branch before triggering a Coolify deployment.
-69
View File
@@ -1,69 +0,0 @@
# Directory Structure & Repositories
## Directory Structure
```
worldwideview/
├── src/
│ ├── app/ # Next.js App Router (pages, API routes, layouts)
│ │ ├── api/ # Server-side API routes (auth, aviation, camera, etc.)
│ │ ├── login/ # Login page
│ │ ├── setup/ # First-time setup page
│ │ └── globals.css # Root stylesheet
│ ├── components/
│ │ ├── common/ # Shared UI: BootOverlay, FloatingWindow, PluginIcon
│ │ ├── layout/ # AppShell, Header, SearchBar, DataBusSubscriber
│ │ ├── panels/ # LayerPanel, EntityInfoCard, FilterPanel, GraphicsSettings
│ │ ├── timeline/ # Timeline component
│ │ ├── marketplace/ # Plugin install/unverified dialogs
│ │ ├── ui/ # Tooltip, ReloadToast
│ │ └── video/ # Floating video manager
│ ├── core/
│ │ ├── data/ # DataBus, PollingManager, CacheLayer, SmartFetcher
│ │ ├── filters/ # filterEngine (applies plugin filters to entities)
│ │ ├── globe/ # GlobeView, EntityRenderer, AnimationLoop, StackManager,
│ │ │ │ CameraController, InteractionHandler, SelectionHandler,
│ │ │ │ ModelManager, ImageryProviderFactory
│ │ │ └── hooks/ # useCameraActions, useEntityRendering, useModelRendering, etc.
│ │ ├── hooks/ # useBootSequence, useIsMobile, useMarketplaceSync
│ │ ├── plugins/ # PluginManager, PluginRegistry, PluginManifest,
│ │ │ │ loadPluginFromManifest, validateManifest, InstalledPluginsLoader
│ │ │ └── loaders/ # mapJsonToEntities (Legacy loaders like DeclarativePlugin are deprecated)
│ │ └── state/ # Zustand store + slices (config, data, globe, layers, timeline, ui, etc.)
│ ├── lib/ # auth, db, rateLimit, analytics, AIS stream, marketplace APIs
│ ├── plugins/ # GeoJSON plugin registrations
│ ├── styles/ # HUD animations CSS
│ ├── types/ # GeoJSON types, Umami types
│ └── generated/ # Prisma generated client (gitignored)
├── local-plugins/ # Local sandbox for developing plugins (gitignored)
├── packages/ # pnpm monorepo workspace packages
│ ├── wwv-plugin-sdk/ # Plugin SDK: type definitions, manifest schema
│ ├── wwv-plugin-aviation/
│ ├── wwv-plugin-maritime/
│ ├── wwv-plugin-wildfire/
│ ├── wwv-plugin-borders/
│ ├── wwv-plugin-camera/
│ ├── wwv-plugin-military-aviation/
│ ├── wwv-plugin-satellite/
│ ├── wwv-plugin-iranwarlive/ # Standalone plugin with custom endpoints
│ └── wwv-plugin-{airports,embassies,lighthouses,nuclear,seaports,spaceports,volcanoes}/
├── prisma/ # schema.prisma, migrations/
├── public/ # Static assets, Cesium workers, plugin GeoJSON data
├── scripts/ # Build scripts (copy-cesium, scaffold-osm-plugin, setup)
├── data/ # PostgreSQL data volume (gitignored)
├── Dockerfile # Multi-stage production build
├── docker-compose.yml # Main app + data engine runner + local seeders
└── .agents/ # Agent documentation, rules, skills, workflows
```
## Related Repositories
| Repo | Purpose |
|---|---|
| `worldwideview` | Main application (this repo) |
| `wwv-data-engine` | Generic data engine runner (PUBLIC, runs via Docker) |
| `wwv-seeders-community` | Open-source community seeders (PUBLIC) |
| `wwv-seeders-private` | Proprietary seeder scripts (PRIVATE, downloaded in prod) |
| `worldwideview-marketplace` | Plugin marketplace web app |
| `worldwideview-plugins` | Published npm plugin packages (frontend source) |
| `worldwideview-web` | Marketing / landing page |
+45
View File
@@ -0,0 +1,45 @@
---
description: Guidelines and architecture for the Playwright End-to-End (E2E) testing infrastructure.
paths:
- "tests/**"
- "public/e2e-fixtures/**"
- "playwright.config.ts"
---
# E2E Testing Infrastructure
The WorldWideView project uses **Playwright** for end-to-end (E2E) testing. This document outlines the boundaries, architectural decisions, and conventions required when authoring or modifying E2E tests.
## 1. Resource Constraints & Parallelism
- **Worker Limits:** The Playwright configuration (`playwright.config.ts`) is strictly limited to a maximum of **2 workers**.
- **Reasoning:** Running Next.js compilation alongside a heavy Prisma connection pool inside multiple parallel test workers will lead to database connection exhaustion and resource contention on CI and local environments. Do not increase the worker count beyond 2 without careful consideration of the database pool limits.
## 2. Test User Lifecycle
E2E tests require an authenticated user to access core application features. We manage this test user securely to ensure the environment stays pristine:
- **Setup (`tests/global.setup.ts`):** Before tests run, the global setup script defensively deletes any existing test user (`[email protected]`) and creates a fresh instance with the `ADMIN` role. It then performs a UI login to capture the authentication state, which is shared across all test files.
- **Teardown (`tests/global.teardown.ts`):** After the entire test suite completes (regardless of success or failure), the global teardown script runs and completely purges the test user from the database.
- **Safety:** Do not use `[email protected]` for manual testing. It is volatile and intended exclusively for the CI test runner.
## 3. UI Interaction Conventions
- **Strict Selector Usage**: ALWAYS use `data-testid` selectors (e.g., `getByTestId('panel-toggle-left')`) for DOM interactions. **NEVER rely on CSS classes** (like `.layer-item__toggle` or `.dock-btn`) or deep DOM structures, as these are brittle and break during design refactors. If a `data-testid` is missing from an element you need to test, you MUST add it to the React component source code first.
- **Dialog Interception:** WorldWideView relies on heavy frontend plugin discovery and dynamic rendering. For example, the `UnverifiedPluginBatchDialog` may unexpectedly appear and block UI interactions in a fresh browser context.
- *Mitigation:* Ensure your test suites either explicitly handle these dialogs (e.g., in a `beforeEach` block) or inject mock plugins that bypass the verification flow entirely.
## 4. Mock Plugins
- **Data Interface Strictness**: WorldWideView's `PluginManager` is extremely strict. **ALL mock plugins MUST implement `getPollingInterval`, `fetch`, and `renderEntity`**, even if they are purely UI-focused (like a bottom-panel plugin). Failure to implement these will crash the renderer.
- **Capabilities**: If your mock UI plugin needs to be manually activated by the user, ensure its manifest includes the `"ui:sidebar"` capability so it appears in the layer list.
- **Injection:** We inject safe, deterministic mock plugins into the database during the `global.setup.ts` phase to test the core rendering pipeline without relying on external APIs. Ensure your mock plugin is registered in the setup and teardown cleanup arrays.
- **Location:** Mock plugin configurations and manifests are stored in `public/e2e-fixtures/`. When modifying the rendering engine, ensure you also update the mock plugin (`public/e2e-fixtures/mock-plugin.js`) to reflect any changes in the Plugin SDK capabilities.
## 5. Security Permissions
- When integrating Playwright tests into GitHub Actions (e.g., `.github/workflows/playwright.yml`), you MUST include an explicit permissions block (e.g., `permissions: { contents: read }`) to adhere to CodeQL and least-privilege scoping rules.
> [!WARNING]
> If you encounter flaky tests or `ECONNREFUSED` errors during Playwright runs, check if the local Next.js dev server or the Docker engine stack is competing for ports (e.g., port 5432 or 3000). Ensure `pnpm run dev:backends` is running and stable before launching the test suite.
## 6. Execution Environment & Tooling Limitations
The development environment operates on **Windows PowerShell**. Agents MUST adhere to the following strict execution rules when running commands or interacting with the testing infrastructure:
- **No Double Ampersands (`&&`)**: Standard Windows PowerShell (prior to v7) does not support the `&&` operator. Use the semicolon (`;`) to chain commands, or rely on `pnpm` package.json scripts which manage cross-platform execution properly.
- **Avoid Non-existent Database CLI Commands**: Do not attempt to use raw `pg` or `psql` commands unless explicitly verified to exist in the path. Instead, use Prisma (`npx prisma studio`, `npx prisma migrate`) or Docker executions (`docker exec -it wwv-db psql ...`) for database interactions.
- **Creating Test Plugins**: You will repeatedly need to create mock test plugins for verifying different platform features (e.g. bottom-panel resizing, new UI widgets). When doing so, you MUST review this `e2e-testing.md` file to ensure compliance with the `PluginManager` strict interface rules (implementing `getPollingInterval`, `fetch`, and `renderEntity`).
-34
View File
@@ -1,34 +0,0 @@
---
trigger: model_decision
description: A comprehensive guide to the WorldWideView ecosystem repositories in the local C:\dev environment. Use this to determine which repository to edit based on the feature being developed.
---
# Ecosystem Repositories & Development Structure
The WorldWideView platform is not a single monolith; it is distributed across several repositories. In the local development environment, these repositories are organized as sibling directories under `C:\dev\`.
If you are asked to implement a feature or fix a bug, you **MUST** determine the correct repository to edit before writing code.
## Ecosystem Map (C:\dev\)
| Repository Folder | Purpose & When to Edit |
|---|---|
| **`worldwideview`** | The main application, Next.js frontend, Cesium globe, UI components, and the core plugin SDK. **Edit here** for anything related to the UI, state, map rendering, or built-in plugin frontend configurations. |
| **`wwv-data-engine`** | The V2 Data Engine (Host Environment) that runs the seeders. **Edit here** if you need to modify how the engine parses WebSocket streams, implements REST APIs, downloads seeder bundles, or handles Redis caching. |
| **`wwv-seeders`** | The community (public) data seeders. **Edit here** to create or modify background polling scripts (e.g. `aviation.ts`, `wildfire.ts`) that fetch data from public APIs and normalize it into `GeoEntity` arrays. |
| **`wwv-seeders-private`** | The proprietary (private) data seeders. **Edit here** for closed-source, premium, or high-security data ingestion scripts. |
| **`worldwideview-marketplace`** | The Plugin Marketplace web application. **Edit here** if you are modifying the platform where users browse, purchase, or manage plugin installations. |
| **`worldwideview-plugins`** | Independent repository for published npm plugin packages (typically frontend source bundles). |
| **`worldwideview-web`** | The marketing site, landing pages, and documentation. |
| **`wwv-test`** | A testing ground repository (e.g., for docker-compose setups, scratch testing). |
## Cross-Repository Workflows
1. **Adding a New Data Source:**
- Write the backend fetcher inside `C:\dev\wwv-seeders` (or `private`).
- Write the frontend plugin interface inside `C:\dev\worldwideview` (or dynamically loaded via `worldwideview-plugins`).
2. **Fixing Real-Time Streaming Issues:**
- Investigate `C:\dev\worldwideview` (for frontend `WsClient` or `DataBus`).
- Investigate `C:\dev\wwv-data-engine` (for backend Fastify websocket broadcasting).
3. **Pnpm Workspaces:**
- The Data Engine V2 automatically combines `community` and `private` seeders at runtime using pnpm workspaces. When testing seeders locally, refer to `C:\dev\worldwideview\local-seeders`.
-17
View File
@@ -1,17 +0,0 @@
# Environment Variables
| Variable | Required | Description |
|---|---|---|
| `DATABASE_URL` | Yes | PostgreSQL connection string |
| `AUTH_SECRET` | Yes | JWT signing secret (generate with `openssl rand -hex 32`) |
| `NEXT_PUBLIC_CESIUM_ION_TOKEN` | No | Cesium Ion access token |
| `NEXT_PUBLIC_BING_MAPS_KEY` | No | Bing Maps imagery |
| `NEXT_PUBLIC_GOOGLE_MAPS_API_KEY` | No | Google 3D Tiles |
| `NEXT_PUBLIC_WWV_EDITION` | No | `local` / `cloud` / `demo` (default: `local`) |
| `NEXT_PUBLIC_WWV_PLUGIN_DATA_ENGINE_URL` | No | Override engine WebSocket URL (default: cloud) |
| `OPENSKY_CREDENTIALS` | No | Comma-separated `id:secret` pairs for credential rotation |
| `WWV_BRIDGE_TOKEN` | No | Shared secret for marketplace → WWV install bridge |
| `WWV_DEMO_ADMIN_SECRET` | No | Demo edition admin password |
| `IRANWARLIVE_BACKEND_URL` | No | Override for IranWarLive custom backend URL |
Secrets go in `.env.local` (gitignored). Non-secrets go in `.env` (committed).
+74
View File
@@ -0,0 +1,74 @@
# Garbage Collection Policy
## Purpose
The Garbage Collector (GC) is a daily Haiku-model agent that keeps the codebase clean. It detects stale TODOs, deprecated patterns, and architectural drift, then either applies a small mechanical fix (Tier A → draft PR) or files a report for human decision (Tier B → GitHub Issue). The GC **never makes architectural decisions** and **never auto-merges** anything.
## Two-tier output model
| Tier | Criteria | Output |
|---|---|---|
| **A — Mechanical** | Safe, codemod-like, no architectural judgement required | Draft PR with code change |
| **B — Judgement** | Requires human architectural decision or context | GitHub Issue, no code |
When unsure of the tier, escalate to Tier B. Never attempt a refactor that requires understanding business intent.
## How the GC works
1. `scripts/gc-scan.mjs` runs first — deterministic shell-based detectors, zero cost, emits `gc-findings.json`.
2. The Haiku agent reads `gc-findings.json` and the policy below. It **does not free-scan the codebase** — it only triages the pre-computed findings list.
3. The agent creates draft PRs and Issues per the caps and guard rails below.
## Approved finding types (from gc-scan.mjs)
| Type | Tier | Description |
|---|---|---|
| `stale-todo` | A | TODO/FIXME/HACK/XXX comment older than 90 days, or referencing a closed issue |
| `anti-pattern` (mdc-ref) | A | `.mdc` file reference — must be `.md` |
| `anti-pattern` (console-log) | A | Stray `console.log(` in `.ts`/`.tsx` files |
| `anti-pattern` (ts-ignore) | A | `@ts-ignore` suppressing a real type error |
| `anti-pattern` (ts-nocheck) | A | `@ts-nocheck` disabling all type checking |
| `anti-pattern` (hardcoded-url) | B | Hardcoded `localhost:5001`/`5000` engine URL |
| `anti-pattern` (deprecated) | B | `@deprecated` symbol still active in production code |
| `oversized-file` | B | Source file exceeding ~350 lines |
| `orphaned-rule-ref` | B | Rule file references a path that no longer exists |
| `outdated-dep` | B | Dependency behind latest version |
## PR guard rails (mandatory — no exceptions)
- All PRs are **DRAFT**. Never mark ready-for-review. Never auto-merge.
- Label every PR and Issue `gc-bot`.
- **Max 3 PRs per run. Max 5 Issues per run.**
- **Max ~150 changed lines per PR.** One concern per PR.
- Branch naming: `gc/<patternId-or-type>-<git-short-sha>` (e.g. `gc/console-log-a3f1b2c`)
- Before creating anything: check open `gc-bot` PRs and Issues for the same concern — skip if already covered.
- Commit prefix: `chore:` for removal/cleanup, `refactor:` for structural fixes. Both are patch-level per the project `/commit` rule.
- Every PR description must include: what changed · which detector flagged it · which rule it violates · how to verify the fix is correct.
## Never-touch list (absolute — the agent must refuse)
- `prisma/migrations/**` — migrations are irreversible
- `pnpm-lock.yaml` — lockfile integrity
- `.env`, `.env.local`, `.env.*`, any file containing secrets or tokens
- Generated assets: `public/cesium/**`, `.next/**`, `node_modules/**`
- Large seed files: `packages/*/data/**`, `local-seeders/**`
- Binary assets: images, videos, fonts, `.db` files
- `local-scripts/` — scratch / one-off scripts, not subject to conventions
## Dry-run mode
When `DRY_RUN=true` (set in the workflow dispatch input):
- Post one summary comment on the tracking Issue (`[gc-bot] Daily Scan Reports`). Create that Issue if it does not exist (label: `gc-bot`).
- List every finding with tier, file, line, description.
- **Do not create any PRs or Issues.**
- Use dry-run for the first 3–5 days after deployment to validate finding quality before enabling normal mode.
## Idempotency rules
A finding is already handled if **either** of these is true:
1. A remote branch matching `gc/<same-patternId-or-type>-*` exists (`git branch -r --list 'origin/gc/<type>-*'`).
2. An open PR or Issue labelled `gc-bot` already covers the same file + concern.
Skip handled findings without error or noise.
+37 -22
View File
@@ -1,6 +1,8 @@
---
trigger: model_decision
description: Guidelines for the WorldwideView plugin marketplace, database interactions, and dynamic plugin installation.
description: Guidelines for the WorldWideView plugin marketplace, database interactions, PKCE auth flow, and dynamic plugin installation.
paths:
- "src/lib/marketplace/**/*"
- "src/app/api/marketplace/**/*"
---
# Marketplace Architecture
@@ -16,6 +18,18 @@ Plugins from the marketplace use the `bundle` format and are imported at runtime
- `pluginManager.loadFromManifest()` validates and registers these plugins.
- Ensure any schema changes to the marketplace tables follow the `database-migrations.md` rule.
## 4. Fresh-Install Seeding
Fresh installs (no rows in `InstalledPlugin`) auto-seed a starter set so the globe isn't empty on day one. The source of truth is the **signed verified registry** (`marketplace.worldwideview.dev/api/registry`, loaded by `registryClient.getVerifiedPluginIds()`).
- **No hard-coded plugin lists.** Publishing a plugin to the verified registry is what makes it part of the fresh-install starter set.
- **Idempotent**: a `defaults_seeded` row in the `Setting` table prevents re-runs. If the registry is unreachable on first attempt, the guard is NOT set — the next request retries.
- **Existing instances** with `defaults_seeded=true` are untouched. To pull in newly-verified plugins, users install via the marketplace UI.
- **Demo edition** uses an independent path (`NEXT_PUBLIC_DEMO_DEFAULT_PLUGINS` env var, handled in `AppShell.tsx` and `useMarketplaceSync.ts`) — not this seeder.
> [!CAUTION]
> Do not reintroduce a static `DEFAULT_PLUGIN_IDS` constant or any other hard-coded plugin list. That drifted from npm reality, caused 404s for unpublished IDs, and conflated "verified" with "auto-installed."
## Current Implementation State
@@ -39,40 +53,41 @@ Plugins from the marketplace use the `bundle` format and are imported at runtime
---
## One-Click Install Flow (Redirect-Based)
## Token Exchange Authentication (PKCE)
**No static API keys.** Auth is entirely session-based. The marketplace redirects the browser to WWV, which checks the user's existing login session.
> [!IMPORTANT]
> **See ADR-0001 (Decentralized Plugin Authentication & SSRF Mitigation)**
> The PKCE flow and API key proxying MUST adhere to ADR-0001. This mandates AES-256-GCM encryption with 100,000 PBKDF2 iterations for all credentials stored in the `MarketplaceCredential` model, and DNS IP Pinning / stream chunking on all proxy requests to mitigate SSRF.
Auth is governed by an **Asymmetric JWT Token Exchange (PKCE)** model to eliminate static, long-lived, and un-rotatable tokens. The Marketplace acts as the OAuth Authorization Server.
```
User clicks "Install" on marketplace
Local App initiates connection
↓
Marketplace: check localStorage for instance URL
↓ (if not configured)
Show "Connect Your Instance" modal → user enters http://localhost:3000
Local App generates PKCE `code_verifier` and `code_challenge` (S256)
↓
MP → GET /api/marketplace/grant-token?redirectTo=...
Local App redirects to Marketplace: /oauth/authorize?response_type=code&...
↓
WWV: check Auth.js session → issue 30-day JWT → redirect ?token=<jwt>
User authenticates on Marketplace and grants access
↓
MP stores JWT in localStorage
Marketplace redirects back to Local App: /api/marketplace/callback?code=...
↓
MP: Navigate browser to /api/marketplace/install-redirect
?pluginId=borders&manifest=<b64>&redirectTo=...
Local App exchanges `code` + `code_verifier` for a long-lived API Key from Marketplace
↓
WWV: check session → upsertPlugin() → issueMarketplaceToken()
↓
Redirect back ?installed=borders&token=<jwt>
↓
MP: Show ✓ Installed — store JWT for future Manage-page calls
Local App securely stores API Key (encrypted via AES-256-GCM)
```
### Token Flow
### Data Engine Connections
After successful install, WWV issues a **30-day HS256 JWT** (`scope: "marketplace"`, signed with `AUTH_SECRET`). The marketplace stores it in `localStorage` and sends it as `Authorization: Bearer <token>` on all subsequent API calls.
Once connected, the Local App uses the long-lived API Key to fetch short-lived entitlements for specific data engines:
1. **Request JWT**: Local App requests a short-lived JWT for a specific Data Engine audience.
2. **Issue JWT**: Marketplace signs an EdDSA JWT containing the user's entitlements and the specific audience.
3. **Connect**: Local App connects to the decentralized Data Engine via WebSocket and performs "First-Message Auth" by sending the JWT.
4. **Verify**: Data Engine verifies the EdDSA signature using the Marketplace's public JWKS. No central DB lookup is required.
`validateMarketplaceAuth` accepts auth in priority order:
1. Active Auth.js session (same-origin browser)
2. Marketplace JWT Bearer token (cross-origin Manage page)
2. EdDSA JWT Bearer token (cross-origin, audience-bound)
3. Legacy static `WWV_BRIDGE_TOKEN` (backward compat)
---
@@ -181,7 +196,7 @@ tenants ─── installed_plugins
| Manifest tampering | Base64 only — no code execution. Validated via `validateManifest()` |
| Malicious plugins | Sandboxed in Web Workers (unverified). CSP headers. Code review (verified) |
| Supply chain | Plugin bundles checksummed (SHA-256), integrity verified |
| Plugin API keys | Stored encrypted in `settings` table, never exposed to browser |
| Plugin API keys | Stored encrypted (AES-256-GCM) in `MarketplaceCredential` model, never exposed to browser |
| Trust forgery | Registry signed with Ed25519 — `verified` cannot be claimed for unlisted plugins |
| Trust stamping | WWV always overwrites `manifest.trust` server-side — client/marketplace input ignored |
| Unverified consent | Warning dialog before any unverified plugin loads; approval stored per-plugin in `localStorage` |
+5 -3
View File
@@ -1,7 +1,9 @@
---
trigger: model_decision
description: Strict rules for operating within the pnpm monorepo workspace environment structure across frontend and backend services.
globs: ""
paths:
- "packages/**/*"
- "pnpm-workspace.yaml"
- "local-plugins/**/*"
---
# Monorepo Workflow Guidelines
@@ -13,7 +15,7 @@ Strict rules for operating within the `pnpm` monorepo workspace environment stru
- The Next.js frontend is located at root `.`.
- Individual standalone plugins are located at `packages/wwv-plugin-[name]`.
- For heavy plugin processing, data engine seeders are located at `local-seeders/community/[name]` or `local-seeders/private/[name]`.
- For heavy plugin processing, data engine seeders are located at `local-seeders/community/[name]` or `local-seeders/private/[name]`. **These are separate git clones** (`github.com/silvertakana/wwv-seeders` and `wwv-seeders-private` respectively) — `pnpm install` from the main repo does NOT pull seeder code. Run `git pull` inside each clone to get upstream changes.
## Critical Workspace Rule
-120
View File
@@ -1,120 +0,0 @@
---
trigger: model_decision
description: High-level platform goals, product vision, business model, multi-repository layout, and the Edition System.
---
# WorldWideView — Platform Architecture
> **This folder is the canonical source of truth for the project. It is NOT committed to git.**
> It contains live architectural decisions, future plans, and operational knowledge.
---
> [!TIP]
> **For the fundamental system architecture, component layout, and data pipelines, please read [`docs/ARCHITECTURE.md`](../../../docs/ARCHITECTURE.md) first.**
> This file focuses strictly on the advanced business vision, multi-repository layout, and the Edition System.
## Product Vision & Business Model
WorldWideView operates on a **VS Code distribution model**. It is a visualization platform, not a compute provider. Plugin authors bring their own backends to display real-time geospatial intelligence.
---
## Repository Structure
Five repositories, modelled after the n8n pattern:
| Repo | License | Purpose |
|---|---|---|
| `silvertakana/worldwideview` | Elastic License 2.0 | Core 3D globe app (Next.js + CesiumJS + Prisma) |
| `silvertakana/wwv-data-engine` | Open Source | Generic runner (Fastify + Redis + WS) |
| `silvertakana/wwv-seeders-community` | Open Source | Open-source community seeders |
| `silvertakana/wwv-seeders-private` | **Private** | Proprietary seeder scripts (aviation, maritime, etc.) |
| `silvertakana/worldwideview-marketplace` | Open Source | Plugin catalog, signed registry, admin UI |
| `silvertakana/worldwideview-plugins` | Open Source | Published plugin frontend source |
| `silvertakana/worldwideview-web` | Open Source | Landing page, docs, static content |
| `silvertakana/worldwideview-cloud` | **Private** | Cloud deployment config — Docker Compose, nginx, provisioning scripts |
| `silvertakana/plugin-sdk` | MIT | `@worldwideview/plugin-sdk` — shared types for repos + 3rd-party authors |
**n8n Parallel:**
| n8n | WorldWideView |
|---|---|
| `n8n-io/n8n` | `silvertakana/worldwideview` |
| `n8n-io/n8n-cloud` (private) | `silvertakana/worldwideview-cloud` (private) |
| `n8n.io` landing | `worldwideview.dev` (static on Vercel) |
| `app.n8n.cloud` dashboard | `app.worldwideview.dev` |
| `[user].app.n8n.cloud` | `[user].app.worldwideview.dev` |
| `@n8n/workflow` npm | `@worldwideview/plugin-sdk` npm |
---
## Domain Architecture
| URL | What | Stack |
|---|---|---|
| `worldwideview.dev` | Static marketing/landing | Next.js (static export) on Vercel |
| `app.worldwideview.dev` | Auth, accounts, dashboard | Next.js + Supabase Auth |
| `[user].app.worldwideview.dev` | User's cloud WWV instance | Next.js (multi-tenant, RLS) |
| `marketplace.worldwideview.dev` | Plugin catalog, publishing, install | Next.js (SSO via WorldWideView auth) |
| `demo.worldwideview.dev` | Public demo — pre-configured, read-only | Next.js (WWV app) |
| `docs.worldwideview.dev` | Documentation | TBD (Mintlify/Nextra) |
| `status.worldwideview.dev` | Uptime / incident | Uptime Kuma |
| `api.worldwideview.dev` | Built-in data services (aviation, maritime) | Next.js API routes or standalone |
| `cdn.worldwideview.dev` | Plugin bundle storage | Cloudflare R2 |
**Reserved subdomains:** `www`, `app`, `api`, `demo`, `marketplace`, `docs`, `status`, `admin`, `cdn`, `blog`, `changelog`
---
## Infrastructure
All on **Coolify** running on a repurposed **ThinkPad T480s** (i5-8350U, 16GB RAM, 256GB NVMe, Ubuntu). Zero monthly hosting cost.
```
ThinkPad T480s (home server, $0/mo) Supabase (hosted)
├── Coolify ──→ ├── PostgreSQL (cloud DB)
│ ├── worldwideview.dev (landing, Vercel) ├── Auth (GoTrue)
│ ├── app.worldwideview.dev (auth/dashboard) └── Storage (file uploads)
│ ├── WWV Cloud App (Docker)
│ ├── Marketplace App (Docker) Cloudflare (free)
│ ├── Demo Instance (Docker) ├── DNS + wildcard routing
│ ├── api.worldwideview.dev ├── CDN caching
│ ├── Uptime Kuma ├── DDoS protection
│ └── Plausible Analytics └── R2 (plugin bundle CDN)
└── Ubuntu Server
```
**CI/CD:** Push to `silvertakana/worldwideview` → GitHub Actions builds Docker image → pushes to `ghcr.io/silvertakana/worldwideview:latest` → Coolify webhook triggers rolling restart.
**Monitoring:** Uptime Kuma (all subdomains), Plausible/Umami analytics.
---
## Business Model
**Distribution platform, not compute provider.** Plugin authors handle their own backends.
| Revenue Stream | How |
|---|---|
| **Cloud hosting** | `[user].app.worldwideview.dev` — free/pro/enterprise tiers |
| **Marketplace cut** | % of paid plugin subscriptions via Stripe Connect |
| **Verified tier** | Plugin authors pay for review + verified badge |
| **Pro/Enterprise license** | More quota, faster cache TTL, advanced features |
| **Extension packs** | Curated bundles of related plugins |
Plugin authors can monetize independently (VS Code + GitHub Copilot model). Selling through the marketplace is **incentivized** (managed billing, featured placement, analytics) but not mandatory.
---
## Editions
Three editions; same codebase, gated by `NEXT_PUBLIC_WWV_EDITION`:
| Edition | Value | Database | Auth | Plugin Install |
|---|---|---|---|---|
| **Local** | `local` | PostgreSQL | Auth.js Credentials | Yes |
| **Cloud** | `cloud` | PostgreSQL (Supabase) | Auth.js + Supabase | Yes |
| **Demo** | `demo` | PostgreSQL (read-only) | Disabled | No |
Three thin adapter files handle edition differences: `src/core/auth.ts`, `src/core/storage.ts`, `src/core/tenant.ts`. Everything else is edition-unaware.
+101 -6
View File
@@ -1,6 +1,10 @@
---
trigger: model_decision
description: The standard operating procedure for instantiating, modifying, and registering a new data source plugin within the engine.
paths:
- "src/core/plugins/**/*"
- "packages/wwv-plugin-*/src/**/*"
- "local-plugins/**/*"
- "packages/wwv-plugin-sdk/src/**/*"
---
# Plugin Architecture & Data Flow
@@ -17,6 +21,59 @@ A valid plugin class MUST provide:
3. `renderEntity(GeoEntity)` to determine visual style.
4. `getPollingInterval()` (defaults fallback securely to standard store config).
## Plugin Data Delivery Types
Plugins receive entity data in one of three modes. Choose the right mode before writing code — mixing them causes silent data loss.
| Mode | `getPollingInterval()` | `fetch()` | `mapWebsocketPayload()` | When to use |
|---|---|---|---|---|
| **REST-only** | `> 0` (ms) | Returns `GeoEntity[]` | Not needed | Static or slow-changing data (< every 30s). No seeder required. |
| **WebSocket-only** | `0` | Return `[]` immediately | **Required** if seeder sends object payload | Live-streaming data from a seeder (satellites, AIS, flights) |
| **Hybrid** | `> 0` | Returns initial snapshot | **Required** if seeder sends object payload | Needs both immediate data on load AND live updates |
### WebSocket-Only Pattern (most live-data plugins)
```typescript
getPollingInterval(): number {
return 0; // WS-only — do NOT call REST on enable
}
async fetch(_timeRange: TimeRange): Promise<GeoEntity[]> {
return []; // Initial data arrives via mapWebsocketPayload within seconds
}
mapWebsocketPayload(payload: any, _existingEntities: GeoEntity[]): GeoEntity[] {
// Transform raw engine payload → GeoEntity[]
// payload shape depends on what the seeder calls setLiveSnapshot() with
const items = payload?.items ?? (Array.isArray(payload) ? payload : []);
return items.map((item: any): GeoEntity => ({
id: `my-plugin-${item.id}`,
pluginId: "my-plugin",
latitude: item.lat,
longitude: item.lon,
altitude: item.alt ?? 0,
timestamp: new Date(),
label: item.name,
properties: item,
}));
}
```
> [!WARNING]
> If `getPollingInterval()` returns `0` but the seeder sends an object payload, you **MUST** implement `mapWebsocketPayload`. Without it, `WsClient` drops the data silently. See `data-engine-architecture.md §7` for the exact rule.
### `mapWebsocketPayload` Signature
```typescript
mapWebsocketPayload(payload: any, existingEntities: GeoEntity[]): GeoEntity[]
```
- `payload` — raw value from `setLiveSnapshot()` in the seeder (whatever shape the seeder stores)
- `existingEntities` — current entities in the store for this plugin (use for incremental updates)
- Must return a **complete replacement** `GeoEntity[]` (not a diff)
---
## Plugin Architectures (Manifest Formats)
Plugins are defined via `PluginManifest` and operate on an **All-Bundle** architecture.
@@ -52,6 +109,33 @@ All plugins MUST define their identity and compatibility via a `"worldwideview"`
}
```
### `peerDependencies` Rule for npm-Published Plugins
Plugins published to npm **MUST** declare the following as `peerDependencies`, not `dependencies`:
```json
{
"peerDependencies": {
"@worldwideview/wwv-plugin-sdk": "*",
"react": ">=19",
"lucide-react": "*"
}
}
```
Also peer (never bundle): `react-dom`, `cesium`, `resium`, `zustand`.
Bundling these causes multiple React instances in the host app, breaking hooks (`Invalid hook call` at runtime).
### File Extension Rule
- Files containing **JSX** (any `<Component />` syntax, `return (<div>...)`) → must use `.tsx`
- Files with no JSX (pure TypeScript, even importing React types) → use `.ts`
This is enforced by the TypeScript compiler. A `.ts` file with JSX will fail to build.
---
## The Registration Pipeline
> [!NOTE]
@@ -78,9 +162,9 @@ DataBus.getInstance().emit('dataUpdated', {
});
```
## Local Plugin Development (Sandbox)
## Local Plugin Development
To rapidly iterate on plugins without polluting the core monorepo `packages/` directory or git history, use the local devkit:
`local-plugins/` is a **git clone of the canonical community plugin repo** (`github.com/silvertakana/wwv-plugins`), gitignored from the main `worldwideview` repo. It has its own remote — commits and pushes go upstream to that public repo, not to `worldwideview`. Run `cd local-plugins && git pull` before working to get the latest community plugins.
1. **Create**: Run `node packages/wwv-cli/dist/index.js create <name> --local` to scaffold a full plugin structure inside the `local-plugins/` directory.
2. **Develop**: Running `pnpm dev` or `pnpm dev:all` automatically spawns the `dev:plugins` watcher. Any changes made to plugins in `local-plugins/` will be instantly rebuilt using Vite (externalizing large globals) and synced to `public/plugins-local/` for hot-reloading.
@@ -191,6 +275,16 @@ Plugins declare auth type in the manifest:
**Recommended for scale:** `token-exchange` — your server is never in the hot path for data requests.
### Shared Auth Contracts & Security Branding
> [!IMPORTANT]
> **See ADR-0001 (Decentralized Plugin Authentication & SSRF Mitigation)**
> All proxy and token-exchange implementations must strictly adhere to ADR-0001. This includes enforcing DNS IP Pinning and stream size limits via `undici.Agent` on all proxy routes to prevent SSRF, and encrypting all long-lived credentials (API keys) using AES-256-GCM with 100k PBKDF2 iterations before storing them in the `MarketplaceCredential` model.
When implementing the `token-exchange` flow or communicating across domains, all plugins and endpoints MUST use the strictly defined shared auth contracts from `@worldwideview/wwv-plugin-sdk/src/auth-contracts.ts` (e.g., `TokenExchangeRequest`, `WebSocketAuthMessage`).
**Sensitive Credentials**: All long-lived secrets (like API keys and tokens) MUST use the `SensitiveString` branded type and be instantiated via the `sensitive(str)` factory function. This prevents accidental logging/serialization and creates a single grep target for security audits.
---
## Plugin Development Workflows
@@ -198,7 +292,7 @@ Plugins declare auth type in the manifest:
Starting in 2026, WorldWideView adopted the **Architect and Toolbox** paradigm using NPM global tools, ensuring a fully decoupled developer experience. Plugin repositories contain *pure code* with absolutely no platform hosting boilerplate (e.g. no embedded docker-compose files).
### The Developer Experience
1. **Scaffold**: `node packages/wwv-cli/dist/index.js create <name> --local` (Creates sandbox inside `local-plugins/`)
1. **Scaffold**: `node packages/wwv-cli/dist/index.js create <name> --local` (Creates plugin inside `local-plugins/` — the community plugin repo clone)
2. **Install**: `pnpm install` (Links dependencies via workspace)
3. **Develop**: `pnpm dev:all` (Starts frontend, data engine, and plugin watcher dynamically)
@@ -381,10 +475,11 @@ While avoiding monolithic coupling, WWV plans to implement the following high-UX
## Local Plugins & Seeders
### 1. Local Sandbox Workflow
### 1. Local Plugin Workflow (`local-plugins/` — community plugin repo clone)
`local-plugins/` is a git clone of `github.com/silvertakana/wwv-plugins` with its own remote. Pull latest before starting: `cd local-plugins && git pull`.
- **Scaffold**: `node packages/wwv-cli/dist/index.js create` (creates inside `local-plugins/wwv-plugin-<name>` by default, or `packages/` if `--core` is used).
- **Develop**: `pnpm dev` triggers hot-reloading via `dev:plugins` watch script.
- **Publish**: `node packages/wwv-cli/dist/index.js publish <name> [--org <your-org>]` publishes the plugin from the project root. (No linking is needed; the sandbox natively functions in the workspace).
- **Publish**: `node packages/wwv-cli/dist/index.js publish <name> [--org <your-org>]` publishes the plugin from the project root. Commit and push from inside `local-plugins/` to contribute back to the community repo.
### 2. Data Engine Seeders
Seeders in `wwv-data-engine/src/seeders/` provide the backend data.
-25
View File
@@ -1,25 +0,0 @@
---
trigger: model_decision
description: Server development and debugging rules for the production/homelab server using SSH and Coolify MCP.
---
# Server Management & Debugging
## 1. Coolify MCP (High-Level)
Use the `coolify` MCP server for managed actions:
- **Discovery**: `mcp_coolify_projects`, `mcp_coolify_environments`
- **Inspection**: `mcp_coolify_application_logs`, `mcp_coolify_get_application`
- **Actions**: `mcp_coolify_control` (restart, stop)
## 2. Direct SSH (Low-Level)
For filesystem or network issues, use direct SSH to the production server:
`ssh [email protected] "<command>"`
**Common Commands**:
- `docker ps`
- `docker logs --tail 100 <container_id>`
- `df -h`
- `top -b -n 1` (Never run interactive `top` or `nano` via agent commands).
## 3. Safety Precautions
Always use non-interactive flags for SSH. Communicate clearly before executing high-risk actions like volume deletion or critical infrastructure restarts.
@@ -1,378 +0,0 @@
---
trigger: model_decision
description: Human-centered design principles, stakeholder map, and decision matrix.
---
# Stakeholders & Human-Centered Design
> [!IMPORTANT]
> **This document is mandatory reading before any feature, UI, API, or architectural decision.**
> Every line of code in this project ultimately serves a human being. The AI agent MUST identify which stakeholder(s) a change affects and design the solution around their needs, abilities, and constraints — not the other way around.
---
## Why This Document Exists
Software that doesn't consider its humans fails. Not technically — socially. A plugin API that's elegant internally but hostile to external developers will die with zero third-party plugins. A monetization tier that ignores what users actually value will generate zero revenue. A contribution process that assumes PhD-level Cesium knowledge will attract zero contributors.
**The directive is simple:** Before you write code, ask *"Who is this for, what do they need, and what will they actually experience?"*
---
## The Stakeholder Universe
WorldWideView is not a single-user tool. It is a **platform ecosystem** with at least nine distinct human roles, each with different goals, different technical abilities, different emotional states when they interact with the product, and different definitions of "success."
```
┌──────────────────┐
│ Owner / Founder │
│ (silvertakana) │
└────────┬─────────┘
│ vision, architecture, deployment
┌───────────────────┼───────────────────┐
│ │ │
┌──────┴──────┐ ┌───────┴───────┐ ┌───────┴───────┐
│ Core Devs │ │ Contributors │ │ Plugin Devs │
│ (team) │ │ (community) │ │ (3rd party) │
└──────┬──────┘ └───────┬───────┘ └───────┬───────┘
│ │ │
└───────────┬───────┘ │
│ build the product │ extend the product
┌───────────┴────────────────────────────┘
│
┌──────┴──────────────────────────────────┐
│ The Application │
│ ┌────────┐ ┌────────┐ ┌───────────┐ │
│ │ Demo │ │ Local │ │ Cloud │ │
│ │Instance│ │Instance│ │ Instance │ │
│ └───┬────┘ └───┬────┘ └─────┬─────┘ │
└──────┼───────────┼─────────────┼────────┘
│ │ │
┌──────┴──┐ ┌─────┴────┐ ┌────┴─────┐
│Casual │ │Power User│ │Paying │
│Visitor │ │(self-host)│ │Customer │
└─────────┘ └──────────┘ └────┬─────┘
│
┌────────┴────────┐
│ Advertisers │
│ (demo edition) │
└─────────────────┘
```
---
## Stakeholder Profiles
### 1. Owner / Founder (silvertakana)
| Attribute | Description |
|---|---|
| **Role** | Solo founder, architect, deployer, and operator of the entire ecosystem |
| **Goal** | Build a sustainable, revenue-generating geospatial intelligence platform that rivals WorldMonitor |
| **Motivations** | Technical excellence, creative ownership, financial independence, community building |
| **Technical Ability** | Expert — TypeScript, Next.js, CesiumJS, Docker, infrastructure, Prisma, AI-assisted development |
| **Emotional State** | Ambitious but resource-constrained; every wasted hour counts double |
| **Definition of Success** | Growing user base, marketplace with active plugin developers, recurring revenue, industry recognition |
| **Pain Points** | Single point of failure for ops, deployment, and architecture decisions; time pressure; hosting cost sensitivity ($0/mo target) |
| **Interaction Surface** | `.agents/` rules, ROADMAP.md, Coolify dashboard, code reviews, AI pair-programming sessions |
**Design implications:**
- Automation is critical — CI/CD, self-healing databases, zero-config setup
- AI agent rules must preserve the owner's vision even when the owner isn't in the conversation
- Cost-efficient architecture (PostgreSQL, ThinkPad server, free-tier services) is a feature, not a compromise
- Every architectural decision must consider the owner's ability to maintain it solo
---
### 2. Core Development Team (WWV Devs)
| Attribute | Description |
|---|---|
| **Role** | Trusted developers with direct commit access to the monorepo |
| **Goal** | Ship features quickly without breaking existing functionality |
| **Motivations** | Clean code, clear architecture, fast iteration, not getting paged at 2am |
| **Technical Ability** | High — comfortable with TypeScript, Next.js App Router, Zustand, pnpm workspaces |
| **Emotional State** | Productive when patterns are clear; frustrated by undocumented gotchas |
| **Definition of Success** | PRs merged cleanly, features work on first deploy, zero regressions |
| **Pain Points** | File bloat (>150 lines), implicit state dependencies, CesiumJS rendering quirks, Docker build complexity |
| **Interaction Surface** | `src/core/`, `src/components/`, `packages/`, PR reviews, `pnpm dev` |
**Design implications:**
- 150-line file limit is sacred — enforce it in every code change
- Clear slice boundaries in Zustand store prevent cross-team conflicts
- `AGENTS.md` and `.agents/` context must always be current so that AI-augmented devs have accurate context
- Rendering rules (billboard vs. point, never mix properties) must be enforced systematically
- Docker gotchas documented in `06-dev-workflows.md` prevent time-sink debugging
---
### 3. Open Source Contributors
| Attribute | Description |
|---|---|
| **Role** | External developers who fork, fix bugs, add features, or create plugins via PRs |
| **Goal** | Contribute meaningfully without needing to understand the entire 50-package monorepo |
| **Motivations** | Portfolio building, learning CesiumJS/geospatial tech, improving a tool they use, open source goodwill |
| **Technical Ability** | **Variable** — ranges from junior devs submitting typo fixes to senior engineers contributing optimizations |
| **Emotional State** | Enthusiastic but easily discouraged by onboarding friction; will abandon contribution if setup takes >15 minutes |
| **Definition of Success** | PR accepted and merged; their name in the contributors list; feature actually ships |
| **Pain Points** | Complex monorepo, CesiumJS learning curve, Elastic License 2.0 implications, unclear "where to start" |
| **Interaction Surface** | `CONTRIBUTING.md`, `docs/SETUP.md`, `docs/PLUGIN_GUIDE.md`, GitHub Issues, PR templates |
**Design implications:**
- **One-command setup** (`pnpm run setup && pnpm dev:all`) is non-negotiable — if it breaks, contributors leave
- `CONTRIBUTING.md` must use `pnpm` (currently shows `npm` in places — this is a bug)
- "Good first issue" labels on GitHub attract newcomers
- Plugin creation is the #1 entry point for contributors — the scaffold script and Plugin Guide must be flawless
- The Elastic License 2.0 must be clearly communicated (it's NOT fully open source — contributors need to understand what they're signing up for)
---
### 4. Third-Party Plugin Developers
| Attribute | Description |
|---|---|
| **Role** | External developers who build and publish plugins to the WorldWideView Marketplace |
| **Goal** | Create a plugin that integrates their data source, potentially monetize it |
| **Motivations** | Revenue (marketplace cut), reach (installed on many WWV instances), reputation, data evangelism |
| **Technical Ability** | **Mid-to-high** — comfortable with TypeScript and npm, but NOT Cesium experts. They should never need to learn Cesium internals. |
| **Emotional State** | Evaluating whether the platform is worth investing development time in. Extremely sensitive to DX friction. |
| **Definition of Success** | Plugin published, visible in marketplace, installed by users, data renders correctly, optional revenue stream |
| **Pain Points** | (1) Backend hosting burden — they need a VPS for active plugins (the "VPS Problem"), (2) No distributable mini-engine yet, (3) SDK docs may not cover edge cases, (4) Rendering bugs with billboard/point property mixing |
| **Interaction Surface** | `@worldwideview/wwv-plugin-sdk`, `docs/PLUGIN_GUIDE.md`, `docs/creating-a-plugin.md`, marketplace submission UI, npm publishing |
**Design implications:**
- The Plugin SDK is the **#1 most important public API surface** — it must be impeccably documented, versioned, and stable
- `WorldPlugin` interface changes are breaking changes that affect real people with real published packages
- Three plugin formats (Declarative, Static, Code Bundle) exist specifically to lower the barrier:
- **Declarative** = zero code (JSON only) — anyone with a data URL can create a plugin
- **Static** = GeoJSON file + manifest — data analysts who aren't developers
- **Code Bundle** = full TypeScript — professional developers
- The future **Mini-Engine Docker image** (§3 of `03-plugin-system.md`) is critical for reducing the VPS burden
- Rendering must be abstracted — plugin devs define `renderEntity()`, and the engine handles GPU batching
- Error messages from the plugin lifecycle must be clear, not cryptic Cesium stack traces
- Marketplace analytics (install counts, usage) are what keep plugin developers motivated
---
### 5. Casual Visitors (Demo Users)
| Attribute | Description |
|---|---|
| **Role** | Anonymous visitors who land on `demo.worldwideview.dev` — no account, no setup |
| **Goal** | Explore the globe, see cool real-time data, understand what WorldWideView is |
| **Motivations** | Curiosity, news events (war, natural disasters, flight tracking), linked from social media or Reddit |
| **Technical Ability** | **Low to none** — they don't know what ADS-B or GeoJSON means. They just want to see planes on a globe. |
| **Emotional State** | Impatient. They will leave within 10 seconds if the globe doesn't load or the UI is confusing. |
| **Definition of Success** | "Whoa, this is cool" → bookmark/share → return later → potentially convert to self-host or cloud user |
| **Pain Points** | Slow boot time (Cesium asset loading), confusing UX, data not loading (API rate limits), ads disrupting experience |
| **Interaction Surface** | Boot overlay, 3D globe, layer toggle panel, entity hover/click, info cards, timeline, ad strip |
**Design implications:**
- **First 5 seconds are everything** — the boot overlay must feel cinematic, not broken
- Pre-activate compelling default layers (aviation is the universal crowd-pleaser)
- Globe interaction must work without instructions — left-click drag, scroll zoom, click entity
- The "Why ads?" banner (`DemoAdStrip`) is a trust-building honesty moment — keep it human and transparent
- Mobile layout must be flawless — many demo visitors come from social media on phones
- No login walls, no setup wizards, no configuration prompts — demo is zero-friction
- Performance on commodity hardware (integrated GPUs, 2017 laptops) must be acceptable
- Error states must be invisible — if OpenSky is down, the layer just shows fewer planes, not an error dialog
---
### 6. Self-Hosted Power Users (Local Edition)
| Attribute | Description |
|---|---|
| **Role** | Technical users who run WorldWideView on their own hardware (Raspberry Pi, home server, VPS) |
| **Goal** | Full control over their own geospatial intelligence dashboard, no cloud dependency |
| **Motivations** | Privacy, customization, data sovereignty, OSINT hobbyism, professional intelligence work |
| **Technical Ability** | **High** — comfortable with Docker, environment variables, CLI tools, possibly contributing plugins |
| **Emotional State** | Empowered but demanding. They chose self-hosting for a reason — they expect it to work without phoning home. |
| **Definition of Success** | Instance running 24/7, plugins installed and polling reliably, data persisted locally, zero external dependencies |
| **Pain Points** | Docker build complexity, PostgreSQL connection pooling issues, Cesium token setup, Prisma migration gotchas, plugin data not loading |
| **Interaction Surface** | `docker-compose.yml`, `.env.local`, Setup Wizard, Layers Panel, Config Panel, Marketplace Bridge, `pnpm run dev:all` |
**Design implications:**
- **Zero-config first run** — `pnpm run setup && pnpm dev` or `docker-compose up` must "just work"
- The Setup Wizard (Cesium token prompt) is the ONLY mandatory configuration step — minimize it
- PostgreSQL must handle concurrent reads from the app + data engine efficiently
- Self-hosted users are the most likely to become contributors — treat their issues as high-priority
- Data engine self-healing (fallback seeds) ensures restarts don't lose data
- Offline capability is a differentiator — cached entities in IndexedDB survive network outages
- `NEXT_PUBLIC_WWV_EDITION=local` must be the default, most-tested path
---
### 7. Paying Customers (Cloud Pro/Enterprise)
| Attribute | Description |
|---|---|
| **Role** | Users or organizations paying for `[user].app.worldwideview.dev` — the revenue engine |
| **Goal** | Reliable, always-on geospatial intelligence with history, higher rate limits, and team access |
| **Motivations** | Professional OSINT, military/defense analysis, logistics monitoring, journalism, corporate security |
| **Technical Ability** | **Variable** — from journalists who just need a globe to security analysts who want API access |
| **Emotional State** | **Paying customers have the lowest tolerance for bugs.** They expect enterprise-grade uptime and support. |
| **Definition of Success** | Data is live, history works, team members can access, exports work, it doesn't go down |
| **Pain Points** | Downtime, stale data, missing features compared to competitors, unclear what "Pro" actually gives them |
| **Interaction Surface** | Cloud dashboard, license key input, plugin marketplace, support channels, billing UI |
**Design implications:**
- **Uptime is king** — Uptime Kuma monitoring, error alerting, and rollback procedures exist for this reason
- Tier differentiation must be clear and valuable:
- Free: 3 users, 500MB, 24h history
- Pro: 20 users, 5GB, full history, snapshot capture
- Enterprise: unlimited, 50GB+, custom domains
- License key verification (RSA-signed JWT) prevents forgery but must not be onerous UX
- RLS tenant isolation is non-negotiable — data leaks between tenants are existential
- Response caching at the API proxy layer makes the $0/mo server viable for multiple tenants
- The cloud edition must feel **identical** to local — same UI, same plugins, same experience
---
### 8. Advertisers (Demo Edition Revenue)
| Attribute | Description |
|---|---|
| **Role** | Google AdSense (programmatic) — automated ad placement on the demo instance |
| **Goal** | Display relevant ads to engaged visitors, generate clicks/impressions |
| **Motivations** | ROI on ad spend; relevant audience targeting |
| **Technical Ability** | N/A — they interact through Google's ad platform, not with WWV directly |
| **Emotional State** | N/A — algorithmic. But their platform (AdSense) has strict technical requirements. |
| **Definition of Success** | Ads render correctly, non-intrusively, on viewable screen real estate |
| **Pain Points** | AdSense script crashes on resize, ads failing to fill on localhost/tunnels, ads overlapping interactive content |
| **Interaction Surface** | `DemoAdStrip.tsx`, `AdUnit.tsx`, Google AdSense publisher dashboard |
**Design implications:**
- Ad strip is a **structurally separate sibling container** — never overlaps the globe viewport
- Fixed-position elements (header, timeline, sidebars) use `--ad-strip-inset` CSS variable to avoid overlap
- AdSense `push()` errors on resize are handled by fully unmounting ad units below 850px height
- Ads are **demo edition only** — they never appear in local or cloud editions
- The "Why ads?" banner is a conscious UX choice that builds trust with users and is dismissible
- Ad revenue is a temporary bridge to sustainability, not the long-term business model
---
### 9. Data Source Providers (Upstream APIs)
| Attribute | Description |
|---|---|
| **Role** | External services that provide the raw data: OpenSky, AIS providers, NASA FIRMS, ACLED, CelesTrak, GPSJam, etc. |
| **Goal** | Serve data within their rate limits and terms of service |
| **Motivations** | Open data mission (NASA, CelesTrak), freemium conversion (OpenSky), research community |
| **Technical Ability** | N/A — they expose APIs; WWV consumes them |
| **Emotional State** | Hostile if rate-limited. Neutral if respected. |
| **Definition of Success** | API consumers stay within rate limits, attribute correctly, and don't DDOS their endpoints |
| **Pain Points** | Credential rotation failures (OpenSky), rate limit exhaustion, unattributed scraping |
| **Interaction Surface** | Data Engine seeders, API route proxies, `OPENSKY_CREDENTIALS` rotation, `pollingIntervalMs` settings |
**Design implications:**
- **Respect upstream rate limits** — this is an ethical and practical obligation
- Credential rotation (`OPENSKY_CREDENTIALS` comma-separated pairs) prevents per-key throttling
- The Data Engine's 5-minute Redis throttle (`setLiveSnapshot`) is specifically designed to stay within Upstash free-tier limits
- API proxy caching (1000 users → 1 upstream request) is architecturally essential
- Attribution (e.g., "Data: OpenSky Network") should be visible in entity detail panels
- Graceful degradation when APIs go down — show cached data, not error modals
---
## The Stakeholder Decision Matrix
When making any decision, use this matrix to identify impact:
| Decision Area | Owner | Core Devs | Contributors | Plugin Devs | Casual Visitors | Power Users | Paying Customers | Advertisers | Data Providers |
|---|---|---|---|---|---|---|---|---|---|
| **Plugin SDK change** | ⚠️ | ⚠️ | ⬜ | 🔴 | ⬜ | ⬜ | ⬜ | ⬜ | ⬜ |
| **UI/UX redesign** | ⚠️ | ⬜ | ⬜ | ⬜ | 🔴 | ⚠️ | 🔴 | ⚠️ | ⬜ |
| **Rendering pipeline change** | ⚠️ | 🔴 | ⬜ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ⬜ | ⬜ |
| **Database schema migration** | ⚠️ | ⚠️ | ⬜ | ⬜ | ⬜ | 🔴 | 🔴 | ⬜ | ⬜ |
| **New data source plugin** | ✅ | ⬜ | ✅ | ⬜ | ✅ | ✅ | ✅ | ⬜ | ⚠️ |
| **Pricing/tier change** | 🔴 | ⬜ | ⬜ | ⚠️ | ⬜ | ⚠️ | 🔴 | ⬜ | ⬜ |
| **Ad placement change** | ⚠️ | ⬜ | ⬜ | ⬜ | ⚠️ | ⬜ | ⬜ | 🔴 | ⬜ |
| **API rate limit adjustment** | ⬜ | ⬜ | ⬜ | ⬜ | ⬜ | ⬜ | ⚠️ | ⬜ | 🔴 |
| **Marketplace policy** | 🔴 | ⬜ | ⬜ | 🔴 | ⬜ | ⬜ | ⚠️ | ⬜ | ⬜ |
| **Docker/deployment change** | 🔴 | ⚠️ | ⬜ | ⬜ | ⬜ | 🔴 | ⚠️ | ⬜ | ⬜ |
| **License change** | 🔴 | ⚠️ | 🔴 | 🔴 | ⬜ | ⚠️ | ⬜ | ⬜ | ⬜ |
**Legend:** 🔴 = Primary impact (design for this person) | ⚠️ = Secondary impact (consider carefully) | ✅ = Positive impact | ⬜ = Minimal/no impact
---
## The Golden Rules
### 1. Every Feature Has a Human
Before writing a single line of code, the AI agent MUST answer:
- **WHO** is this for? (Name the stakeholder)
- **WHY** do they need it? (What problem does it solve for them?)
- **HOW** will they discover/use it? (What's their journey?)
- **WHAT** happens when it breaks? (Graceful degradation plan)
### 2. Reduce Friction, Not Features
The most dangerous failure mode is building powerful systems with hostile interfaces. Apply the **Principle of Least Effort**:
- Contributors: one-command setup, not a wiki page of prerequisites
- Plugin devs: `renderEntity()` returns an object, not a Cesium Primitive
- Casual visitors: zero onboarding, zero configuration
- Power users: sensible defaults, optional configuration
- Paying customers: it just works, always
### 3. Design for the Least Technical User in Each Cohort
| Cohort | Least Technical Member | Design Target |
|---|---|---|
| Plugin Developers | Data analyst who can write JSON | Declarative plugin format |
| Contributors | Junior dev on their first open source PR | Scaffold scripts + clear CONTRIBUTING.md |
| Casual Visitors | Non-technical person who followed a Twitter link | Auto-loading layers, intuitive globe controls |
| Power Users | Sysadmin who can `docker-compose up` but doesn't know Next.js | Pre-built Docker images, `.env` only config |
| Paying Customers | Journalist tracking a conflict | Clear UI, no jargon, exportable data |
### 4. Protect the Ecosystem Trust
The trust hierarchy is cryptographically enforced (Ed25519 signed registry), but trust is also social:
- **Built-in plugins** must always work — they're the first impression
- **Verified plugins** carry WorldWideView's reputation — review them seriously
- **Unverified plugins** get a warning dialog — users make informed decisions
- **Plugin developers** who see installs and analytics stay motivated
- **Data providers** who aren't rate-limit abused will keep serving data
### 5. Revenue Serves Sustainability, Not Extraction
The business model exists to keep the project alive:
- Ads on demo = temporary hosting cost recovery (transparent "Why ads?" banner)
- Cloud Pro = value exchange (history, higher limits, team access)
- Marketplace cut = sustainable ecosystem (Stripe Connect revenue split)
- Plugin developer monetization = VS Code model (developers set their own prices)
Never design a feature that extracts value from users without returning it. The demo must always be useful, not crippled.
---
## Applying This Document
### For the AI Agent
When the agent is asked to implement a feature, modify architecture, or solve a bug, it MUST:
1. **Identify affected stakeholders** from the list above
2. **Consider the least-technical user** in each affected cohort
3. **Check the decision matrix** for impact severity
4. **Apply the golden rules** to the implementation approach
5. **Document stakeholder considerations** in PR descriptions and implementation plans
### For Human Developers
When reviewing PRs or proposing features, ask:
- "Did we consider how this affects plugin developers?"
- "Will a first-time contributor understand this?"
- "Does this work on the demo without configuration?"
- "Will this survive a Docker restart on the ThinkPad?"
---
## Version History
| Date | Change |
|---|---|
| 2026-04-05 | Initial stakeholder map created from comprehensive project research |
+3 -2
View File
@@ -1,7 +1,8 @@
---
trigger: glob
description: Guidelines on managing the single source of truth across WorldWideView using Zustand, avoiding prop-drilling, and isolating UI rerenders.
globs: "src/core/state/**/*, src/components/**/*"
paths:
- "src/core/state/**/*"
- "src/components/**/*"
---
# Zustand State Management
@@ -1,22 +0,0 @@
---
trigger: model_decision
description: Troubleshooting and debugging guidelines for WorldWideView, covering latency, logging, and namespace collisions.
---
# Troubleshooting and Debugging
## 1. Plugin Data Latency
If there is a noticeable delay (e.g., ~1 second) between enabling a plugin and the initial data arriving on the globe, **do not assume the UI is blocking**.
- **WebSocket Handshake**: Initial latency is often caused by the time it takes to establish a new WebSocket connection to the engine's `/stream`.
- **Upstream Cooldowns**: Plugins querying external providers (like OpenSky for `aviation`) have strict rate limits. If a snapshot returns a `404`, it might mean the seeder is in a cooldown period or hasn't successfully fetched data yet. This is expected behavior.
- **Diagnosis**: Instrument the `DataBusSubscriber` and `WsClient` with timing logs (`performance.now()`) to track exactly where the bottleneck occurs (manifest discovery vs connection establishment vs first payload).
## 2. Log Verbosity and Structured Logging
We employ a **structured logging strategy** in the frontend to reduce noise and improve readability.
- **Rule**: Do not add excessive `console.log` statements that dump raw JSON payloads or stream contents directly to the console in core files like `WsClient.ts` or `PluginManager.ts`.
- **Implementation**: If detailed logs are needed, use conditional debug levels or summarize the output (e.g., logging payload sizes or entity counts instead of the full object array). This ensures critical diagnostic information remains visible without overflowing the terminal.
## 3. Namespace Collisions
If a plugin fails to load in the Data Engine with a `404` or `Error [ERR_MODULE_NOT_FOUND]` error during startup:
- **Root Cause**: The seeder may exist simultaneously in both the `wwv-seeders-community` and `wwv-seeders-private` repositories.
- **Resolution**: Seeders must have a unique namespace. Ensure that private plugins (e.g., `aviation`, `maritime`, `military-aviation`) are explicitly removed from the community repository to prevent workspace-level module resolution failures when the V2 engine attempts to load them. Private seeders take priority, but overlapping names will cause dependency conflicts.
+125
View File
@@ -0,0 +1,125 @@
param (
[Parameter(Mandatory=$true, HelpMessage="Base Git SHA")]
[string]$BaseSha,
[Parameter(Mandatory=$true, HelpMessage="Head Git SHA")]
[string]$HeadSha,
[Parameter(Mandatory=$false, HelpMessage="Path to ADR or Requirements File")]
[string]$RequirementsFile = "",
[Parameter(Mandatory=$false, HelpMessage="Brief description of the implemented changes")]
[string]$Description = "Implemented recent changes as per requirements.",
[Parameter(Mandatory=$false, HelpMessage="Model to use")]
[string]$Model = "gemini-2.5-flash"
)
# Set strict error handling
$ErrorActionPreference = "Stop"
Write-Host "Gathering Code Review Context..." -ForegroundColor Cyan
# Ensure local-scripts directory exists
$ScriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
$WorkspaceRoot = Split-Path -Parent $ScriptDir
$TempPromptFile = Join-Path $ScriptDir "pr-review-prompt.txt"
# 1. Fetch Requirements text if provided
$RequirementsText = "No specific requirements provided."
if ($RequirementsFile -ne "") {
if (Test-Path $RequirementsFile) {
$RequirementsText = Get-Content $RequirementsFile -Raw
} else {
Write-Warning "Requirements file not found at $RequirementsFile"
}
}
# 2. Extract git diff
Write-Host "Extracting diff between $BaseSha and $HeadSha..." -ForegroundColor DarkGray
try {
$DiffOutput = git diff $BaseSha..$HeadSha
if ([string]::IsNullOrWhiteSpace($DiffOutput)) {
Write-Host "No changes found between $BaseSha and $HeadSha." -ForegroundColor Yellow
exit 0
}
} catch {
Write-Error "Failed to execute git diff. Ensure you are in a git repository and the SHAs are valid."
exit 1
}
# 3. Construct the Reviewer Prompt
$ReviewerPrompt = @"
You are a Senior Code Reviewer with expertise in software architecture, design patterns, and best practices. Your job is to review completed work against its plan or requirements and identify issues before they cascade. You are a sub-agent for Antigravity. Output ONLY the review format below. DO NOT use tools.
## What Was Implemented
$Description
## Requirements / Plan
$RequirementsText
## Git Range to Review
**Base:** $BaseSha
**Head:** $HeadSha
```diff
$DiffOutput
```
## What to Check
- **Plan alignment:** Does the implementation match the requirements? Are deviations justified?
- **Code quality:** Clean separation of concerns? Proper error handling?
- **Architecture:** Sound design decisions? Security concerns?
- **Testing:** Edge cases covered? Integration tests?
- **Production readiness:** Backward compatibility? Obvious bugs?
## Output Format
### Strengths
[What's well done? Be specific.]
### Issues
#### Critical (Must Fix)
[Bugs, security issues, data loss risks, broken functionality]
#### Important (Should Fix)
[Architecture problems, missing features, poor error handling, test gaps]
#### Minor (Nice to Have)
[Code style, optimization opportunities, documentation polish]
For each issue:
- File:line reference
- What's wrong
- Why it matters
- How to fix (if not obvious)
### Recommendations
[Improvements for code quality, architecture, or process]
### Assessment
**Ready to merge?** [Yes | No | With fixes]
**Reasoning:** [1-2 sentence technical assessment]
"@
# 4. Save to temporary file
$ReviewerPrompt | Set-Content $TempPromptFile -Encoding UTF8
# 5. Dispatch to Gemini CLI
Write-Host "Dispatching to Gemini CLI ($Model)..." -ForegroundColor Cyan
Write-Host "Depending on the diff size, this may take 30-60 seconds." -ForegroundColor DarkGray
$GeminiCmd = "$env:APPDATA\npm\gemini.cmd"
if (-Not (Test-Path $GeminiCmd)) {
Write-Error "Gemini CLI not found at $GeminiCmd. Ensure it is installed globally via npm."
exit 1
}
try {
Get-Content $TempPromptFile -Raw | & $GeminiCmd -m $Model
Write-Host "`nCode Review Complete." -ForegroundColor Green
} catch {
Write-Error "Failed to execute Gemini CLI. If you received a 429 RESOURCE_EXHAUSTED error, try changing the model (e.g. -Model gemini-1.5-pro)."
} finally {
# Clean up temp file
if (Test-Path $TempPromptFile) {
Remove-Item $TempPromptFile -Force
}
}
@@ -11,7 +11,8 @@ Trigger this skill whenever you are about to introduce a new architectural patte
## Guidelines
1. **Never make a major architectural change without an ADR**.
2. Store ADRs in `docs/architecture/decisions/` (create this directory if it doesn't exist).
3. An ADR must follow this exact structure:
3. Name ADR files using the pattern `adr-00##-name.md` (e.g., `adr-0001-use-postgresql.md`).
4. An ADR must follow this exact structure:
- **Title**: Short, descriptive noun phrase.
- **Context**: What is the problem we are solving? What forces are at play? (Be objective).
- **Decision**: What is the specific change or pattern we are adopting?
@@ -1,13 +1,13 @@
---
name: migrate-legacy-plugin
description: Use when moving a WorldWideView plugin from packages/ (or ../worldwideview-plugins/packages/) to local-plugins/, converting legacy plugins to the new Engine & Payload architecture, or when a plugin hardcodes engine URLs, bundles Node.js built-ins, or uses import.meta.url for file paths inside Docker containers
description: Use when moving a WorldWideView plugin from packages/ to local-plugins/ (the wwv-plugins community repo clone), converting legacy plugins to the new Engine & Payload architecture, or when a plugin hardcodes engine URLs, bundles Node.js built-ins, or uses import.meta.url for file paths inside Docker containers
---
# Migrate Legacy Plugin to Modern Architecture
## Overview
Moves a WorldWideView plugin from the old `packages/` monorepo location (or the external `worldwideview-plugins` repository) into the `local-plugins/` directory (which is itself a publishable Git repository). Simultaneously fixes two **independent** concerns that legacy plugins always have:
Moves a WorldWideView plugin from the old `packages/` monorepo location into the `local-plugins/` directory (which is a git clone of `github.com/silvertakana/wwv-plugins` — the canonical community plugin repo). Simultaneously fixes two **independent** concerns that legacy plugins always have:
1. **Frontend routing** — hardcoded engine URLs must be replaced with `resolveEngineUrl()`
2. **Backend seeder build** — ESM bundling and container-aware file paths
@@ -16,7 +16,7 @@ Moves a WorldWideView plugin from the old `packages/` monorepo location (or the
## When to Use
- Moving a plugin from `packages/wwv-plugin-<name>/` (or `../worldwideview-plugins/packages/wwv-plugin-<name>/`) to `local-plugins/wwv-plugin-<name>/`
- Moving a plugin from `packages/wwv-plugin-<name>/` to `local-plugins/wwv-plugin-<name>/`
- Plugin's `fetch()` hardcodes a URL like `https://dataengine.worldwideview.dev`
- Plugin uses `this.context?.apiBaseUrl` instead of `resolveEngineUrl()`
- Backend seeder crashes with `Dynamic require` or `ERR_INVALID_FILE_URL_PATH` in Docker
@@ -38,7 +38,7 @@ Moves a WorldWideView plugin from the old `packages/` monorepo location (or the
> ```
> Example: `node scripts/migrate-plugin.mjs gps-jamming`
>
> The script will automatically copy the frontend plugin, inject peerDependencies, refactor `index.tsx` routing, fix the backend seeder's build settings and `SEEDERS_DIR` paths, and register the plugin in `defaultPlugins.ts`.
> The script will automatically copy the frontend plugin, inject peerDependencies, refactor `index.tsx` routing, and fix the backend seeder's build settings and `SEEDERS_DIR` paths. (Fresh-install seeding no longer involves a hard-coded list — once published to npm with a valid `worldwideview` block, the plugin is registered via the signed verified registry.)
> **You still need to run `pnpm install` and verify the builds manually as described below.**
> [!IMPORTANT]
@@ -59,14 +59,13 @@ Copy the plugin from `packages/` (or the external plugins repo) to `local-plugin
# From project root (if in local packages/)
cp -r packages/wwv-plugin-<name> local-plugins/wwv-plugin-<name>
# OR from external plugins repository:
cp -r ../worldwideview-plugins/packages/wwv-plugin-<name> local-plugins/wwv-plugin-<name>
```
The `local-plugins/` directory is:
- Part of the pnpm workspace (`pnpm-workspace.yaml` includes `local-plugins/*`)
- Gitignored from the main worldwideview repo (it IS its own Git repo — `worldwideview-plugins`)
- Gitignored from the main worldwideview repo — it IS its own git repo (`github.com/silvertakana/wwv-plugins`, the canonical community plugin repo)
- Auto-discovered by the `dev:plugins` watcher during `pnpm dev`
- Run `cd local-plugins && git pull` before working to fetch the latest community plugins
## Step 2: Fix Frontend Routing (getEngineUrl)
@@ -220,7 +219,7 @@ local-seeders/community/packages/gps-jamming/
After moving to `local-plugins/`, remove the old references:
- [ ] Delete `packages/wwv-plugin-<name>/` directory (or the source directory in `../worldwideview-plugins/packages/`)
- [ ] Delete `packages/wwv-plugin-<name>/` directory
- [ ] Remove from `transpilePackages` in `next.config.ts` (if present)
- [ ] Remove path alias from `tsconfig.json` `paths` (if present)
- [ ] Remove any registration in `AppShell.tsx` (now loaded dynamically)
@@ -0,0 +1,72 @@
---
name: playwright-testing
description: Use when creating or modifying Playwright end-to-end tests, especially when injecting mock plugins to test platform features.
---
# Playwright E2E Testing & Mock Plugin Guide
When you are tasked with testing a new feature or plugin in WorldWideView using Playwright, you must use this skill to ensure you follow the strict architecture rules of the platform.
## 1. Mock Plugin Requirements
When creating a mock plugin (e.g., `public/e2e-fixtures/my-mock-plugin.js`) to test a specific UI feature, **you MUST implement the full Data Plugin interface**.
WorldWideView's `PluginManager` throws fatal runtime errors if a plugin is missing core data fetching methods, even if your plugin only tests a UI component like a bottom panel.
**Required Methods for EVERY Mock Plugin:**
```javascript
export default {
id: "my-mock-plugin",
name: "My Mock Plugin",
// ...
// MUST BE INCLUDED:
fetch: async (timeRange) => { return []; },
getPollingInterval: () => { return 60000; },
getLayerConfig: () => { return { color: "#FFF", clusterEnabled: false, clusterDistance: 50 }; },
renderEntity: (entity) => { return { type: "point", color: "#FFF", size: 5 }; },
// Your UI testing component (e.g., Sidebar, Bottom Panel)
getBottomPanelComponent: () => { ... }
}
```
## 2. Mock Plugin Manifest Capabilities
If your mock plugin needs to be toggled by the user in the Layers list, you MUST include the `"ui:sidebar"` capability in its `manifest.json`.
```json
{
"capabilities": ["ui:bottom-panel", "ui:sidebar"],
"extends": ["sidebar"]
}
```
## 3. Database Injection
The frontend reads plugins from the database. You MUST inject your mock plugin during `tests/global.setup.ts` using Prisma:
```typescript
const manifestPath = path.join(process.cwd(), 'public', 'e2e-fixtures', 'my-mock-plugin-manifest.json');
const manifestStr = fs.readFileSync(manifestPath, 'utf-8');
await prisma.installedPlugin.create({
data: {
pluginId: 'my-mock-plugin',
version: '1.0.0',
config: manifestStr,
enabled: true
}
});
```
Make sure to add your new mock plugin ID to the cleanup logic in `global.setup.ts` as well:
```typescript
await prisma.installedPlugin.deleteMany({
where: { pluginId: { in: ['e2e-mock-plugin', 'my-mock-plugin'] } }
});
```
## 4. UI Selectors
**DO NOT use CSS classes for your locators.** CSS classes change frequently (e.g., `.dock-btn`, `.layer-item__toggle`).
You MUST either:
1. Add explicit `data-testid` attributes to the React components you are testing.
2. Use stable accessibility locators (e.g., `getByRole('button', { name: /Text/ })`).
## 5. Execution Environment
- Ensure `pnpm run dev:backends` is running before you execute tests locally.
- Run tests using `pnpm run test:e2e` or `npx playwright test tests/your-test.spec.ts`.
- **PowerShell Warning**: Remember you are running in Windows PowerShell. Do NOT chain commands with `&&`. Use `;` instead.
@@ -1,64 +0,0 @@
---
name: supabase-postgres-best-practices
description: Postgres performance optimization and best practices from Supabase. Use this skill when writing, reviewing, or optimizing Postgres queries, schema designs, or database configurations.
license: MIT
metadata:
author: supabase
version: "1.1.1"
organization: Supabase
date: January 2026
abstract: Comprehensive Postgres performance optimization guide for developers using Supabase and Postgres. Contains performance rules across 8 categories, prioritized by impact from critical (query performance, connection management) to incremental (advanced features). Each rule includes detailed explanations, incorrect vs. correct SQL examples, query plan analysis, and specific performance metrics to guide automated optimization and code generation.
---
# Supabase Postgres Best Practices
Comprehensive performance optimization guide for Postgres, maintained by Supabase. Contains rules across 8 categories, prioritized by impact to guide automated query optimization and schema design.
## When to Apply
Reference these guidelines when:
- Writing SQL queries or designing schemas
- Implementing indexes or query optimization
- Reviewing database performance issues
- Configuring connection pooling or scaling
- Optimizing for Postgres-specific features
- Working with Row-Level Security (RLS)
## Rule Categories by Priority
| Priority | Category | Impact | Prefix |
|----------|----------|--------|--------|
| 1 | Query Performance | CRITICAL | `query-` |
| 2 | Connection Management | CRITICAL | `conn-` |
| 3 | Security & RLS | CRITICAL | `security-` |
| 4 | Schema Design | HIGH | `schema-` |
| 5 | Concurrency & Locking | MEDIUM-HIGH | `lock-` |
| 6 | Data Access Patterns | MEDIUM | `data-` |
| 7 | Monitoring & Diagnostics | LOW-MEDIUM | `monitor-` |
| 8 | Advanced Features | LOW | `advanced-` |
## How to Use
Read individual rule files for detailed explanations and SQL examples:
```
references/query-missing-indexes.md
references/query-partial-indexes.md
references/_sections.md
```
Each rule file contains:
- Brief explanation of why it matters
- Incorrect SQL example with explanation
- Correct SQL example with explanation
- Optional EXPLAIN output or metrics
- Additional context and references
- Supabase-specific notes (when applicable)
## References
- https://www.postgresql.org/docs/current/
- https://supabase.com/docs
- https://wiki.postgresql.org/wiki/Performance_Optimization
- https://supabase.com/docs/guides/database/overview
- https://supabase.com/docs/guides/auth/row-level-security
@@ -1,170 +0,0 @@
# Writing Guidelines for Postgres References
This document provides guidelines for creating effective Postgres best
practice references that work well with AI agents and LLMs.
## Key Principles
### 1. Concrete Transformation Patterns
Show exact SQL rewrites. Avoid philosophical advice.
**Good:** "Use `WHERE id = ANY(ARRAY[...])` instead of
`WHERE id IN (SELECT ...)`" **Bad:** "Design good schemas"
### 2. Error-First Structure
Always show the problematic pattern first, then the solution. This trains agents
to recognize anti-patterns.
```markdown
**Incorrect (sequential queries):** [bad example]
**Correct (batched query):** [good example]
```
### 3. Quantified Impact
Include specific metrics. Helps agents prioritize fixes.
**Good:** "10x faster queries", "50% smaller index", "Eliminates N+1"
**Bad:** "Faster", "Better", "More efficient"
### 4. Self-Contained Examples
Examples should be complete and runnable (or close to it). Include `CREATE TABLE`
if context is needed.
```sql
-- Include table definition when needed for clarity
CREATE TABLE users (
id bigint PRIMARY KEY,
email text NOT NULL,
deleted_at timestamptz
);
-- Now show the index
CREATE INDEX users_active_email_idx ON users(email) WHERE deleted_at IS NULL;
```
### 5. Semantic Naming
Use meaningful table/column names. Names carry intent for LLMs.
**Good:** `users`, `email`, `created_at`, `is_active`
**Bad:** `table1`, `col1`, `field`, `flag`
---
## Code Example Standards
### SQL Formatting
```sql
-- Use lowercase keywords, clear formatting
CREATE INDEX CONCURRENTLY users_email_idx
ON users(email)
WHERE deleted_at IS NULL;
-- Not cramped or ALL CAPS
CREATE INDEX CONCURRENTLY USERS_EMAIL_IDX ON USERS(EMAIL) WHERE DELETED_AT IS NULL;
```
### Comments
- Explain _why_, not _what_
- Highlight performance implications
- Point out common pitfalls
### Language Tags
- `sql` - Standard SQL queries
- `plpgsql` - Stored procedures/functions
- `typescript` - Application code (when needed)
- `python` - Application code (when needed)
---
## When to Include Application Code
**Default: SQL Only**
Most references should focus on pure SQL patterns. This keeps examples portable.
**Include Application Code When:**
- Connection pooling configuration
- Transaction management in application context
- ORM anti-patterns (N+1 in Prisma/TypeORM)
- Prepared statement usage
**Format for Mixed Examples:**
````markdown
**Incorrect (N+1 in application):**
```typescript
for (const user of users) {
const posts = await db.query("SELECT * FROM posts WHERE user_id = $1", [
user.id,
]);
}
```
````
**Correct (batch query):**
```typescript
const posts = await db.query("SELECT * FROM posts WHERE user_id = ANY($1)", [
userIds,
]);
```
---
## Impact Level Guidelines
| Level | Improvement | Use When |
|-------|-------------|----------|
| **CRITICAL** | 10-100x | Missing indexes, connection exhaustion, sequential scans on large tables |
| **HIGH** | 5-20x | Wrong index types, poor partitioning, missing covering indexes |
| **MEDIUM-HIGH** | 2-5x | N+1 queries, inefficient pagination, RLS optimization |
| **MEDIUM** | 1.5-3x | Redundant indexes, query plan instability |
| **LOW-MEDIUM** | 1.2-2x | VACUUM tuning, configuration tweaks |
| **LOW** | Incremental | Advanced patterns, edge cases |
---
## Reference Standards
**Primary Sources:**
- Official Postgres documentation
- Supabase documentation
- Postgres wiki
- Established blogs (2ndQuadrant, Crunchy Data)
**Format:**
```markdown
Reference:
[Postgres Indexes](https://www.postgresql.org/docs/current/indexes.html)
```
---
## Review Checklist
Before submitting a reference:
- [ ] Title is clear and action-oriented
- [ ] Impact level matches the performance gain
- [ ] impactDescription includes quantification
- [ ] Explanation is concise (1-2 sentences)
- [ ] Has at least 1 **Incorrect** SQL example
- [ ] Has at least 1 **Correct** SQL example
- [ ] SQL uses semantic naming
- [ ] Comments explain _why_, not _what_
- [ ] Trade-offs mentioned if applicable
- [ ] Reference links included
- [ ] `pnpm test` passes
@@ -1,39 +0,0 @@
# Section Definitions
This file defines the rule categories for Postgres best practices. Rules are automatically assigned to sections based on their filename prefix.
Take the examples below as pure demonstrative. Replace each section with the actual rule categories for Postgres best practices.
---
## 1. Query Performance (query)
**Impact:** CRITICAL
**Description:** Slow queries, missing indexes, inefficient query plans. The most common source of Postgres performance issues.
## 2. Connection Management (conn)
**Impact:** CRITICAL
**Description:** Connection pooling, limits, and serverless strategies. Critical for applications with high concurrency or serverless deployments.
## 3. Security & RLS (security)
**Impact:** CRITICAL
**Description:** Row-Level Security policies, privilege management, and authentication patterns.
## 4. Schema Design (schema)
**Impact:** HIGH
**Description:** Table design, index strategies, partitioning, and data type selection. Foundation for long-term performance.
## 5. Concurrency & Locking (lock)
**Impact:** MEDIUM-HIGH
**Description:** Transaction management, isolation levels, deadlock prevention, and lock contention patterns.
## 6. Data Access Patterns (data)
**Impact:** MEDIUM
**Description:** N+1 query elimination, batch operations, cursor-based pagination, and efficient data fetching.
## 7. Monitoring & Diagnostics (monitor)
**Impact:** LOW-MEDIUM
**Description:** Using pg_stat_statements, EXPLAIN ANALYZE, metrics collection, and performance diagnostics.
## 8. Advanced Features (advanced)
**Impact:** LOW
**Description:** Full-text search, JSONB optimization, PostGIS, extensions, and advanced Postgres features.
@@ -1,34 +0,0 @@
---
title: Clear, Action-Oriented Title (e.g., "Use Partial Indexes for Filtered Queries")
impact: MEDIUM
impactDescription: 5-20x query speedup for filtered queries
tags: indexes, query-optimization, performance
---
## [Rule Title]
[1-2 sentence explanation of the problem and why it matters. Focus on performance impact.]
**Incorrect (describe the problem):**
```sql
-- Comment explaining what makes this slow/problematic
CREATE INDEX users_email_idx ON users(email);
SELECT * FROM users WHERE email = '[email protected]' AND deleted_at IS NULL;
-- This scans deleted records unnecessarily
```
**Correct (describe the solution):**
```sql
-- Comment explaining why this is better
CREATE INDEX users_active_email_idx ON users(email) WHERE deleted_at IS NULL;
SELECT * FROM users WHERE email = '[email protected]' AND deleted_at IS NULL;
-- Only indexes active users, 10x smaller index, faster queries
```
[Optional: Additional context, edge cases, or trade-offs]
Reference: [Postgres Docs](https://www.postgresql.org/docs/current/)
@@ -1,55 +0,0 @@
---
title: Use tsvector for Full-Text Search
impact: MEDIUM
impactDescription: 100x faster than LIKE, with ranking support
tags: full-text-search, tsvector, gin, search
---
## Use tsvector for Full-Text Search
LIKE with wildcards can't use indexes. Full-text search with tsvector is orders of magnitude faster.
**Incorrect (LIKE pattern matching):**
```sql
-- Cannot use index, scans all rows
select * from articles where content like '%postgresql%';
-- Case-insensitive makes it worse
select * from articles where lower(content) like '%postgresql%';
```
**Correct (full-text search with tsvector):**
```sql
-- Add tsvector column and index
alter table articles add column search_vector tsvector
generated always as (to_tsvector('english', coalesce(title,'') || ' ' || coalesce(content,''))) stored;
create index articles_search_idx on articles using gin (search_vector);
-- Fast full-text search
select * from articles
where search_vector @@ to_tsquery('english', 'postgresql & performance');
-- With ranking
select *, ts_rank(search_vector, query) as rank
from articles, to_tsquery('english', 'postgresql') query
where search_vector @@ query
order by rank desc;
```
Search multiple terms:
```sql
-- AND: both terms required
to_tsquery('postgresql & performance')
-- OR: either term
to_tsquery('postgresql | mysql')
-- Prefix matching
to_tsquery('post:*')
```
Reference: [Full Text Search](https://supabase.com/docs/guides/database/full-text-search)
@@ -1,49 +0,0 @@
---
title: Index JSONB Columns for Efficient Querying
impact: MEDIUM
impactDescription: 10-100x faster JSONB queries with proper indexing
tags: jsonb, gin, indexes, json
---
## Index JSONB Columns for Efficient Querying
JSONB queries without indexes scan the entire table. Use GIN indexes for containment queries.
**Incorrect (no index on JSONB):**
```sql
create table products (
id bigint primary key,
attributes jsonb
);
-- Full table scan for every query
select * from products where attributes @> '{"color": "red"}';
select * from products where attributes->>'brand' = 'Nike';
```
**Correct (GIN index for JSONB):**
```sql
-- GIN index for containment operators (@>, ?, ?&, ?|)
create index products_attrs_gin on products using gin (attributes);
-- Now containment queries use the index
select * from products where attributes @> '{"color": "red"}';
-- For specific key lookups, use expression index
create index products_brand_idx on products ((attributes->>'brand'));
select * from products where attributes->>'brand' = 'Nike';
```
Choose the right operator class:
```sql
-- jsonb_ops (default): supports all operators, larger index
create index idx1 on products using gin (attributes);
-- jsonb_path_ops: only @> operator, but 2-3x smaller index
create index idx2 on products using gin (attributes jsonb_path_ops);
```
Reference: [JSONB Indexes](https://www.postgresql.org/docs/current/datatype-json.html#JSON-INDEXING)
@@ -1,46 +0,0 @@
---
title: Configure Idle Connection Timeouts
impact: HIGH
impactDescription: Reclaim 30-50% of connection slots from idle clients
tags: connections, timeout, idle, resource-management
---
## Configure Idle Connection Timeouts
Idle connections waste resources. Configure timeouts to automatically reclaim them.
**Incorrect (connections held indefinitely):**
```sql
-- No timeout configured
show idle_in_transaction_session_timeout; -- 0 (disabled)
-- Connections stay open forever, even when idle
select pid, state, state_change, query
from pg_stat_activity
where state = 'idle in transaction';
-- Shows transactions idle for hours, holding locks
```
**Correct (automatic cleanup of idle connections):**
```sql
-- Terminate connections idle in transaction after 30 seconds
alter system set idle_in_transaction_session_timeout = '30s';
-- Terminate completely idle connections after 10 minutes
alter system set idle_session_timeout = '10min';
-- Reload configuration
select pg_reload_conf();
```
For pooled connections, configure at the pooler level:
```ini
# pgbouncer.ini
server_idle_timeout = 60
client_idle_timeout = 300
```
Reference: [Connection Timeouts](https://www.postgresql.org/docs/current/runtime-config-client.html#GUC-IDLE-IN-TRANSACTION-SESSION-TIMEOUT)
@@ -1,44 +0,0 @@
---
title: Set Appropriate Connection Limits
impact: CRITICAL
impactDescription: Prevent database crashes and memory exhaustion
tags: connections, max-connections, limits, stability
---
## Set Appropriate Connection Limits
Too many connections exhaust memory and degrade performance. Set limits based on available resources.
**Incorrect (unlimited or excessive connections):**
```sql
-- Default max_connections = 100, but often increased blindly
show max_connections; -- 500 (way too high for 4GB RAM)
-- Each connection uses 1-3MB RAM
-- 500 connections * 2MB = 1GB just for connections!
-- Out of memory errors under load
```
**Correct (calculate based on resources):**
```sql
-- Formula: max_connections = (RAM in MB / 5MB per connection) - reserved
-- For 4GB RAM: (4096 / 5) - 10 = ~800 theoretical max
-- But practically, 100-200 is better for query performance
-- Recommended settings for 4GB RAM
alter system set max_connections = 100;
-- Also set work_mem appropriately
-- work_mem * max_connections should not exceed 25% of RAM
alter system set work_mem = '8MB'; -- 8MB * 100 = 800MB max
```
Monitor connection usage:
```sql
select count(*), state from pg_stat_activity group by state;
```
Reference: [Database Connections](https://supabase.com/docs/guides/platform/performance#connection-management)
@@ -1,41 +0,0 @@
---
title: Use Connection Pooling for All Applications
impact: CRITICAL
impactDescription: Handle 10-100x more concurrent users
tags: connection-pooling, pgbouncer, performance, scalability
---
## Use Connection Pooling for All Applications
Postgres connections are expensive (1-3MB RAM each). Without pooling, applications exhaust connections under load.
**Incorrect (new connection per request):**
```sql
-- Each request creates a new connection
-- Application code: db.connect() per request
-- Result: 500 concurrent users = 500 connections = crashed database
-- Check current connections
select count(*) from pg_stat_activity; -- 487 connections!
```
**Correct (connection pooling):**
```sql
-- Use a pooler like PgBouncer between app and database
-- Application connects to pooler, pooler reuses a small pool to Postgres
-- Configure pool_size based on: (CPU cores * 2) + spindle_count
-- Example for 4 cores: pool_size = 10
-- Result: 500 concurrent users share 10 actual connections
select count(*) from pg_stat_activity; -- 10 connections
```
Pool modes:
- **Transaction mode**: connection returned after each transaction (best for most apps)
- **Session mode**: connection held for entire session (needed for prepared statements, temp tables)
Reference: [Connection Pooling](https://supabase.com/docs/guides/database/connecting-to-postgres#connection-pooler)
@@ -1,46 +0,0 @@
---
title: Use Prepared Statements Correctly with Pooling
impact: HIGH
impactDescription: Avoid prepared statement conflicts in pooled environments
tags: prepared-statements, connection-pooling, transaction-mode
---
## Use Prepared Statements Correctly with Pooling
Prepared statements are tied to individual database connections. In transaction-mode pooling, connections are shared, causing conflicts.
**Incorrect (named prepared statements with transaction pooling):**
```sql
-- Named prepared statement
prepare get_user as select * from users where id = $1;
-- In transaction mode pooling, next request may get different connection
execute get_user(123);
-- ERROR: prepared statement "get_user" does not exist
```
**Correct (use unnamed statements or session mode):**
```sql
-- Option 1: Use unnamed prepared statements (most ORMs do this automatically)
-- The query is prepared and executed in a single protocol message
-- Option 2: Deallocate after use in transaction mode
prepare get_user as select * from users where id = $1;
execute get_user(123);
deallocate get_user;
-- Option 3: Use session mode pooling (port 5432 vs 6543)
-- Connection is held for entire session, prepared statements persist
```
Check your driver settings:
```sql
-- Many drivers use prepared statements by default
-- Node.js pg: { prepare: false } to disable
-- JDBC: prepareThreshold=0 to disable
```
Reference: [Prepared Statements with Pooling](https://supabase.com/docs/guides/database/connecting-to-postgres#connection-pool-modes)
@@ -1,54 +0,0 @@
---
title: Batch INSERT Statements for Bulk Data
impact: MEDIUM
impactDescription: 10-50x faster bulk inserts
tags: batch, insert, bulk, performance, copy
---
## Batch INSERT Statements for Bulk Data
Individual INSERT statements have high overhead. Batch multiple rows in single statements or use COPY.
**Incorrect (individual inserts):**
```sql
-- Each insert is a separate transaction and round trip
insert into events (user_id, action) values (1, 'click');
insert into events (user_id, action) values (1, 'view');
insert into events (user_id, action) values (2, 'click');
-- ... 1000 more individual inserts
-- 1000 inserts = 1000 round trips = slow
```
**Correct (batch insert):**
```sql
-- Multiple rows in single statement
insert into events (user_id, action) values
(1, 'click'),
(1, 'view'),
(2, 'click'),
-- ... up to ~1000 rows per batch
(999, 'view');
-- One round trip for 1000 rows
```
For large imports, use COPY:
```sql
-- COPY is fastest for bulk loading
copy events (user_id, action, created_at)
from '/path/to/data.csv'
with (format csv, header true);
-- Or from stdin in application
copy events (user_id, action) from stdin with (format csv);
1,click
1,view
2,click
\.
```
Reference: [COPY](https://www.postgresql.org/docs/current/sql-copy.html)
@@ -1,53 +0,0 @@
---
title: Eliminate N+1 Queries with Batch Loading
impact: MEDIUM-HIGH
impactDescription: 10-100x fewer database round trips
tags: n-plus-one, batch, performance, queries
---
## Eliminate N+1 Queries with Batch Loading
N+1 queries execute one query per item in a loop. Batch them into a single query using arrays or JOINs.
**Incorrect (N+1 queries):**
```sql
-- First query: get all users
select id from users where active = true; -- Returns 100 IDs
-- Then N queries, one per user
select * from orders where user_id = 1;
select * from orders where user_id = 2;
select * from orders where user_id = 3;
-- ... 97 more queries!
-- Total: 101 round trips to database
```
**Correct (single batch query):**
```sql
-- Collect IDs and query once with ANY
select * from orders where user_id = any(array[1, 2, 3, ...]);
-- Or use JOIN instead of loop
select u.id, u.name, o.*
from users u
left join orders o on o.user_id = u.id
where u.active = true;
-- Total: 1 round trip
```
Application pattern:
```sql
-- Instead of looping in application code:
-- for user in users: db.query("SELECT * FROM orders WHERE user_id = $1", user.id)
-- Pass array parameter:
select * from orders where user_id = any($1::bigint[]);
-- Application passes: [1, 2, 3, 4, 5, ...]
```
Reference: [N+1 Query Problem](https://supabase.com/docs/guides/database/query-optimization)
@@ -1,50 +0,0 @@
---
title: Use Cursor-Based Pagination Instead of OFFSET
impact: MEDIUM-HIGH
impactDescription: Consistent O(1) performance regardless of page depth
tags: pagination, cursor, keyset, offset, performance
---
## Use Cursor-Based Pagination Instead of OFFSET
OFFSET-based pagination scans all skipped rows, getting slower on deeper pages. Cursor pagination is O(1).
**Incorrect (OFFSET pagination):**
```sql
-- Page 1: scans 20 rows
select * from products order by id limit 20 offset 0;
-- Page 100: scans 2000 rows to skip 1980
select * from products order by id limit 20 offset 1980;
-- Page 10000: scans 200,000 rows!
select * from products order by id limit 20 offset 199980;
```
**Correct (cursor/keyset pagination):**
```sql
-- Page 1: get first 20
select * from products order by id limit 20;
-- Application stores last_id = 20
-- Page 2: start after last ID
select * from products where id > 20 order by id limit 20;
-- Uses index, always fast regardless of page depth
-- Page 10000: same speed as page 1
select * from products where id > 199980 order by id limit 20;
```
For multi-column sorting:
```sql
-- Cursor must include all sort columns
select * from products
where (created_at, id) > ('2024-01-15 10:00:00', 12345)
order by created_at, id
limit 20;
```
Reference: [Pagination](https://supabase.com/docs/guides/database/pagination)
@@ -1,50 +0,0 @@
---
title: Use UPSERT for Insert-or-Update Operations
impact: MEDIUM
impactDescription: Atomic operation, eliminates race conditions
tags: upsert, on-conflict, insert, update
---
## Use UPSERT for Insert-or-Update Operations
Using separate SELECT-then-INSERT/UPDATE creates race conditions. Use INSERT ... ON CONFLICT for atomic upserts.
**Incorrect (check-then-insert race condition):**
```sql
-- Race condition: two requests check simultaneously
select * from settings where user_id = 123 and key = 'theme';
-- Both find nothing
-- Both try to insert
insert into settings (user_id, key, value) values (123, 'theme', 'dark');
-- One succeeds, one fails with duplicate key error!
```
**Correct (atomic UPSERT):**
```sql
-- Single atomic operation
insert into settings (user_id, key, value)
values (123, 'theme', 'dark')
on conflict (user_id, key)
do update set value = excluded.value, updated_at = now();
-- Returns the inserted/updated row
insert into settings (user_id, key, value)
values (123, 'theme', 'dark')
on conflict (user_id, key)
do update set value = excluded.value
returning *;
```
Insert-or-ignore pattern:
```sql
-- Insert only if not exists (no update)
insert into page_views (page_id, user_id)
values (1, 123)
on conflict (page_id, user_id) do nothing;
```
Reference: [INSERT ON CONFLICT](https://www.postgresql.org/docs/current/sql-insert.html#SQL-ON-CONFLICT)
@@ -1,56 +0,0 @@
---
title: Use Advisory Locks for Application-Level Locking
impact: MEDIUM
impactDescription: Efficient coordination without row-level lock overhead
tags: advisory-locks, coordination, application-locks
---
## Use Advisory Locks for Application-Level Locking
Advisory locks provide application-level coordination without requiring database rows to lock.
**Incorrect (creating rows just for locking):**
```sql
-- Creating dummy rows to lock on
create table resource_locks (
resource_name text primary key
);
insert into resource_locks values ('report_generator');
-- Lock by selecting the row
select * from resource_locks where resource_name = 'report_generator' for update;
```
**Correct (advisory locks):**
```sql
-- Session-level advisory lock (released on disconnect or unlock)
select pg_advisory_lock(hashtext('report_generator'));
-- ... do exclusive work ...
select pg_advisory_unlock(hashtext('report_generator'));
-- Transaction-level lock (released on commit/rollback)
begin;
select pg_advisory_xact_lock(hashtext('daily_report'));
-- ... do work ...
commit; -- Lock automatically released
```
Try-lock for non-blocking operations:
```sql
-- Returns immediately with true/false instead of waiting
select pg_try_advisory_lock(hashtext('resource_name'));
-- Use in application
if (acquired) {
-- Do work
select pg_advisory_unlock(hashtext('resource_name'));
} else {
-- Skip or retry later
}
```
Reference: [Advisory Locks](https://www.postgresql.org/docs/current/explicit-locking.html#ADVISORY-LOCKS)
@@ -1,68 +0,0 @@
---
title: Prevent Deadlocks with Consistent Lock Ordering
impact: MEDIUM-HIGH
impactDescription: Eliminate deadlock errors, improve reliability
tags: deadlocks, locking, transactions, ordering
---
## Prevent Deadlocks with Consistent Lock Ordering
Deadlocks occur when transactions lock resources in different orders. Always
acquire locks in a consistent order.
**Incorrect (inconsistent lock ordering):**
```sql
-- Transaction A -- Transaction B
begin; begin;
update accounts update accounts
set balance = balance - 100 set balance = balance - 50
where id = 1; where id = 2; -- B locks row 2
update accounts update accounts
set balance = balance + 100 set balance = balance + 50
where id = 2; -- A waits for B where id = 1; -- B waits for A
-- DEADLOCK! Both waiting for each other
```
**Correct (lock rows in consistent order first):**
```sql
-- Explicitly acquire locks in ID order before updating
begin;
select * from accounts where id in (1, 2) order by id for update;
-- Now perform updates in any order - locks already held
update accounts set balance = balance - 100 where id = 1;
update accounts set balance = balance + 100 where id = 2;
commit;
```
Alternative: use a single statement to update atomically:
```sql
-- Single statement acquires all locks atomically
begin;
update accounts
set balance = balance + case id
when 1 then -100
when 2 then 100
end
where id in (1, 2);
commit;
```
Detect deadlocks in logs:
```sql
-- Check for recent deadlocks
select * from pg_stat_database where deadlocks > 0;
-- Enable deadlock logging
set log_lock_waits = on;
set deadlock_timeout = '1s';
```
Reference:
[Deadlocks](https://www.postgresql.org/docs/current/explicit-locking.html#LOCKING-DEADLOCKS)
@@ -1,50 +0,0 @@
---
title: Keep Transactions Short to Reduce Lock Contention
impact: MEDIUM-HIGH
impactDescription: 3-5x throughput improvement, fewer deadlocks
tags: transactions, locking, contention, performance
---
## Keep Transactions Short to Reduce Lock Contention
Long-running transactions hold locks that block other queries. Keep transactions as short as possible.
**Incorrect (long transaction with external calls):**
```sql
begin;
select * from orders where id = 1 for update; -- Lock acquired
-- Application makes HTTP call to payment API (2-5 seconds)
-- Other queries on this row are blocked!
update orders set status = 'paid' where id = 1;
commit; -- Lock held for entire duration
```
**Correct (minimal transaction scope):**
```sql
-- Validate data and call APIs outside transaction
-- Application: response = await paymentAPI.charge(...)
-- Only hold lock for the actual update
begin;
update orders
set status = 'paid', payment_id = $1
where id = $2 and status = 'pending'
returning *;
commit; -- Lock held for milliseconds
```
Use `statement_timeout` to prevent runaway transactions:
```sql
-- Abort queries running longer than 30 seconds
set statement_timeout = '30s';
-- Or per-session
set local statement_timeout = '5s';
```
Reference: [Transaction Management](https://www.postgresql.org/docs/current/tutorial-transactions.html)
@@ -1,54 +0,0 @@
---
title: Use SKIP LOCKED for Non-Blocking Queue Processing
impact: MEDIUM-HIGH
impactDescription: 10x throughput for worker queues
tags: skip-locked, queue, workers, concurrency
---
## Use SKIP LOCKED for Non-Blocking Queue Processing
When multiple workers process a queue, SKIP LOCKED allows workers to process different rows without waiting.
**Incorrect (workers block each other):**
```sql
-- Worker 1 and Worker 2 both try to get next job
begin;
select * from jobs where status = 'pending' order by created_at limit 1 for update;
-- Worker 2 waits for Worker 1's lock to release!
```
**Correct (SKIP LOCKED for parallel processing):**
```sql
-- Each worker skips locked rows and gets the next available
begin;
select * from jobs
where status = 'pending'
order by created_at
limit 1
for update skip locked;
-- Worker 1 gets job 1, Worker 2 gets job 2 (no waiting)
update jobs set status = 'processing' where id = $1;
commit;
```
Complete queue pattern:
```sql
-- Atomic claim-and-update in one statement
update jobs
set status = 'processing', worker_id = $1, started_at = now()
where id = (
select id from jobs
where status = 'pending'
order by created_at
limit 1
for update skip locked
)
returning *;
```
Reference: [SELECT FOR UPDATE SKIP LOCKED](https://www.postgresql.org/docs/current/sql-select.html#SQL-FOR-UPDATE-SHARE)
@@ -1,45 +0,0 @@
---
title: Use EXPLAIN ANALYZE to Diagnose Slow Queries
impact: LOW-MEDIUM
impactDescription: Identify exact bottlenecks in query execution
tags: explain, analyze, diagnostics, query-plan
---
## Use EXPLAIN ANALYZE to Diagnose Slow Queries
EXPLAIN ANALYZE executes the query and shows actual timings, revealing the true performance bottlenecks.
**Incorrect (guessing at performance issues):**
```sql
-- Query is slow, but why?
select * from orders where customer_id = 123 and status = 'pending';
-- "It must be missing an index" - but which one?
```
**Correct (use EXPLAIN ANALYZE):**
```sql
explain (analyze, buffers, format text)
select * from orders where customer_id = 123 and status = 'pending';
-- Output reveals the issue:
-- Seq Scan on orders (cost=0.00..25000.00 rows=50 width=100) (actual time=0.015..450.123 rows=50 loops=1)
-- Filter: ((customer_id = 123) AND (status = 'pending'::text))
-- Rows Removed by Filter: 999950
-- Buffers: shared hit=5000 read=15000
-- Planning Time: 0.150 ms
-- Execution Time: 450.500 ms
```
Key things to look for:
```sql
-- Seq Scan on large tables = missing index
-- Rows Removed by Filter = poor selectivity or missing index
-- Buffers: read >> hit = data not cached, needs more memory
-- Nested Loop with high loops = consider different join strategy
-- Sort Method: external merge = work_mem too low
```
Reference: [EXPLAIN](https://supabase.com/docs/guides/database/inspect)
@@ -1,55 +0,0 @@
---
title: Enable pg_stat_statements for Query Analysis
impact: LOW-MEDIUM
impactDescription: Identify top resource-consuming queries
tags: pg-stat-statements, monitoring, statistics, performance
---
## Enable pg_stat_statements for Query Analysis
pg_stat_statements tracks execution statistics for all queries, helping identify slow and frequent queries.
**Incorrect (no visibility into query patterns):**
```sql
-- Database is slow, but which queries are the problem?
-- No way to know without pg_stat_statements
```
**Correct (enable and query pg_stat_statements):**
```sql
-- Enable the extension
create extension if not exists pg_stat_statements;
-- Find slowest queries by total time
select
calls,
round(total_exec_time::numeric, 2) as total_time_ms,
round(mean_exec_time::numeric, 2) as mean_time_ms,
query
from pg_stat_statements
order by total_exec_time desc
limit 10;
-- Find most frequent queries
select calls, query
from pg_stat_statements
order by calls desc
limit 10;
-- Reset statistics after optimization
select pg_stat_statements_reset();
```
Key metrics to monitor:
```sql
-- Queries with high mean time (candidates for optimization)
select query, mean_exec_time, calls
from pg_stat_statements
where mean_exec_time > 100 -- > 100ms average
order by mean_exec_time desc;
```
Reference: [pg_stat_statements](https://supabase.com/docs/guides/database/extensions/pg_stat_statements)
@@ -1,55 +0,0 @@
---
title: Maintain Table Statistics with VACUUM and ANALYZE
impact: MEDIUM
impactDescription: 2-10x better query plans with accurate statistics
tags: vacuum, analyze, statistics, maintenance, autovacuum
---
## Maintain Table Statistics with VACUUM and ANALYZE
Outdated statistics cause the query planner to make poor decisions. VACUUM reclaims space, ANALYZE updates statistics.
**Incorrect (stale statistics):**
```sql
-- Table has 1M rows but stats say 1000
-- Query planner chooses wrong strategy
explain select * from orders where status = 'pending';
-- Shows: Seq Scan (because stats show small table)
-- Actually: Index Scan would be much faster
```
**Correct (maintain fresh statistics):**
```sql
-- Manually analyze after large data changes
analyze orders;
-- Analyze specific columns used in WHERE clauses
analyze orders (status, created_at);
-- Check when tables were last analyzed
select
relname,
last_vacuum,
last_autovacuum,
last_analyze,
last_autoanalyze
from pg_stat_user_tables
order by last_analyze nulls first;
```
Autovacuum tuning for busy tables:
```sql
-- Increase frequency for high-churn tables
alter table orders set (
autovacuum_vacuum_scale_factor = 0.05, -- Vacuum at 5% dead tuples (default 20%)
autovacuum_analyze_scale_factor = 0.02 -- Analyze at 2% changes (default 10%)
);
-- Check autovacuum status
select * from pg_stat_progress_vacuum;
```
Reference: [VACUUM](https://supabase.com/docs/guides/database/database-size#vacuum-operations)
@@ -1,44 +0,0 @@
---
title: Create Composite Indexes for Multi-Column Queries
impact: HIGH
impactDescription: 5-10x faster multi-column queries
tags: indexes, composite-index, multi-column, query-optimization
---
## Create Composite Indexes for Multi-Column Queries
When queries filter on multiple columns, a composite index is more efficient than separate single-column indexes.
**Incorrect (separate indexes require bitmap scan):**
```sql
-- Two separate indexes
create index orders_status_idx on orders (status);
create index orders_created_idx on orders (created_at);
-- Query must combine both indexes (slower)
select * from orders where status = 'pending' and created_at > '2024-01-01';
```
**Correct (composite index):**
```sql
-- Single composite index (leftmost column first for equality checks)
create index orders_status_created_idx on orders (status, created_at);
-- Query uses one efficient index scan
select * from orders where status = 'pending' and created_at > '2024-01-01';
```
**Column order matters** - place equality columns first, range columns last:
```sql
-- Good: status (=) before created_at (>)
create index idx on orders (status, created_at);
-- Works for: WHERE status = 'pending'
-- Works for: WHERE status = 'pending' AND created_at > '2024-01-01'
-- Does NOT work for: WHERE created_at > '2024-01-01' (leftmost prefix rule)
```
Reference: [Multicolumn Indexes](https://www.postgresql.org/docs/current/indexes-multicolumn.html)
@@ -1,40 +0,0 @@
---
title: Use Covering Indexes to Avoid Table Lookups
impact: MEDIUM-HIGH
impactDescription: 2-5x faster queries by eliminating heap fetches
tags: indexes, covering-index, include, index-only-scan
---
## Use Covering Indexes to Avoid Table Lookups
Covering indexes include all columns needed by a query, enabling index-only scans that skip the table entirely.
**Incorrect (index scan + heap fetch):**
```sql
create index users_email_idx on users (email);
-- Must fetch name and created_at from table heap
select email, name, created_at from users where email = '[email protected]';
```
**Correct (index-only scan with INCLUDE):**
```sql
-- Include non-searchable columns in the index
create index users_email_idx on users (email) include (name, created_at);
-- All columns served from index, no table access needed
select email, name, created_at from users where email = '[email protected]';
```
Use INCLUDE for columns you SELECT but don't filter on:
```sql
-- Searching by status, but also need customer_id and total
create index orders_status_idx on orders (status) include (customer_id, total);
select status, customer_id, total from orders where status = 'shipped';
```
Reference: [Index-Only Scans](https://www.postgresql.org/docs/current/indexes-index-only-scans.html)
@@ -1,48 +0,0 @@
---
title: Choose the Right Index Type for Your Data
impact: HIGH
impactDescription: 10-100x improvement with correct index type
tags: indexes, btree, gin, gist, brin, hash, index-types
---
## Choose the Right Index Type for Your Data
Different index types excel at different query patterns. The default B-tree isn't always optimal.
**Incorrect (B-tree for JSONB containment):**
```sql
-- B-tree cannot optimize containment operators
create index products_attrs_idx on products (attributes);
select * from products where attributes @> '{"color": "red"}';
-- Full table scan - B-tree doesn't support @> operator
```
**Correct (GIN for JSONB):**
```sql
-- GIN supports @>, ?, ?&, ?| operators
create index products_attrs_idx on products using gin (attributes);
select * from products where attributes @> '{"color": "red"}';
```
Index type guide:
```sql
-- B-tree (default): =, <, >, BETWEEN, IN, IS NULL
create index users_created_idx on users (created_at);
-- GIN: arrays, JSONB, full-text search
create index posts_tags_idx on posts using gin (tags);
-- GiST: geometric data, range types, nearest-neighbor (KNN) queries
create index locations_idx on places using gist (location);
-- BRIN: large time-series tables (10-100x smaller)
create index events_time_idx on events using brin (created_at);
-- Hash: equality-only (slightly faster than B-tree for =)
create index sessions_token_idx on sessions using hash (token);
```
Reference: [Index Types](https://www.postgresql.org/docs/current/indexes-types.html)
@@ -1,43 +0,0 @@
---
title: Add Indexes on WHERE and JOIN Columns
impact: CRITICAL
impactDescription: 100-1000x faster queries on large tables
tags: indexes, performance, sequential-scan, query-optimization
---
## Add Indexes on WHERE and JOIN Columns
Queries filtering or joining on unindexed columns cause full table scans, which become exponentially slower as tables grow.
**Incorrect (sequential scan on large table):**
```sql
-- No index on customer_id causes full table scan
select * from orders where customer_id = 123;
-- EXPLAIN shows: Seq Scan on orders (cost=0.00..25000.00 rows=100 width=85)
```
**Correct (index scan):**
```sql
-- Create index on frequently filtered column
create index orders_customer_id_idx on orders (customer_id);
select * from orders where customer_id = 123;
-- EXPLAIN shows: Index Scan using orders_customer_id_idx (cost=0.42..8.44 rows=100 width=85)
```
For JOIN columns, always index the foreign key side:
```sql
-- Index the referencing column
create index orders_customer_id_idx on orders (customer_id);
select c.name, o.total
from customers c
join orders o on o.customer_id = c.id;
```
Reference: [Query Optimization](https://supabase.com/docs/guides/database/query-optimization)
@@ -1,45 +0,0 @@
---
title: Use Partial Indexes for Filtered Queries
impact: HIGH
impactDescription: 5-20x smaller indexes, faster writes and queries
tags: indexes, partial-index, query-optimization, storage
---
## Use Partial Indexes for Filtered Queries
Partial indexes only include rows matching a WHERE condition, making them smaller and faster when queries consistently filter on the same condition.
**Incorrect (full index includes irrelevant rows):**
```sql
-- Index includes all rows, even soft-deleted ones
create index users_email_idx on users (email);
-- Query always filters active users
select * from users where email = '[email protected]' and deleted_at is null;
```
**Correct (partial index matches query filter):**
```sql
-- Index only includes active users
create index users_active_email_idx on users (email)
where deleted_at is null;
-- Query uses the smaller, faster index
select * from users where email = '[email protected]' and deleted_at is null;
```
Common use cases for partial indexes:
```sql
-- Only pending orders (status rarely changes once completed)
create index orders_pending_idx on orders (created_at)
where status = 'pending';
-- Only non-null values
create index products_sku_idx on products (sku)
where sku is not null;
```
Reference: [Partial Indexes](https://www.postgresql.org/docs/current/indexes-partial.html)
@@ -1,80 +0,0 @@
---
title: Add Constraints Safely in Migrations
impact: HIGH
impactDescription: Prevents migration failures and enables idempotent schema changes
tags: constraints, migrations, schema, alter-table
---
## Add Constraints Safely in Migrations
PostgreSQL does not support `ADD CONSTRAINT IF NOT EXISTS`. Migrations using this syntax will fail.
**Incorrect (causes syntax error):**
```sql
-- ERROR: syntax error at or near "not" (SQLSTATE 42601)
alter table public.profiles
add constraint if not exists profiles_birthchart_id_unique unique (birthchart_id);
```
**Correct (idempotent constraint creation):**
```sql
-- Use DO block to check before adding
do $$
begin
if not exists (
select 1 from pg_constraint
where conname = 'profiles_birthchart_id_unique'
and conrelid = 'public.profiles'::regclass
) then
alter table public.profiles
add constraint profiles_birthchart_id_unique unique (birthchart_id);
end if;
end $$;
```
For all constraint types:
```sql
-- Check constraints
do $$
begin
if not exists (
select 1 from pg_constraint
where conname = 'check_age_positive'
) then
alter table users add constraint check_age_positive check (age > 0);
end if;
end $$;
-- Foreign keys
do $$
begin
if not exists (
select 1 from pg_constraint
where conname = 'profiles_birthchart_id_fkey'
) then
alter table profiles
add constraint profiles_birthchart_id_fkey
foreign key (birthchart_id) references birthcharts(id);
end if;
end $$;
```
Check if constraint exists:
```sql
-- Query to check constraint existence
select conname, contype, pg_get_constraintdef(oid)
from pg_constraint
where conrelid = 'public.profiles'::regclass;
-- contype values:
-- 'p' = PRIMARY KEY
-- 'f' = FOREIGN KEY
-- 'u' = UNIQUE
-- 'c' = CHECK
```
Reference: [Constraints](https://www.postgresql.org/docs/current/ddl-constraints.html)
@@ -1,46 +0,0 @@
---
title: Choose Appropriate Data Types
impact: HIGH
impactDescription: 50% storage reduction, faster comparisons
tags: data-types, schema, storage, performance
---
## Choose Appropriate Data Types
Using the right data types reduces storage, improves query performance, and prevents bugs.
**Incorrect (wrong data types):**
```sql
create table users (
id int, -- Will overflow at 2.1 billion
email varchar(255), -- Unnecessary length limit
created_at timestamp, -- Missing timezone info
is_active varchar(5), -- String for boolean
price varchar(20) -- String for numeric
);
```
**Correct (appropriate data types):**
```sql
create table users (
id bigint generated always as identity primary key, -- 9 quintillion max
email text, -- No artificial limit, same performance as varchar
created_at timestamptz, -- Always store timezone-aware timestamps
is_active boolean default true, -- 1 byte vs variable string length
price numeric(10,2) -- Exact decimal arithmetic
);
```
Key guidelines:
```sql
-- IDs: use bigint, not int (future-proofing)
-- Strings: use text, not varchar(n) unless constraint needed
-- Time: use timestamptz, not timestamp
-- Money: use numeric, not float (precision matters)
-- Enums: use text with check constraint or create enum type
```
Reference: [Data Types](https://www.postgresql.org/docs/current/datatype.html)
@@ -1,59 +0,0 @@
---
title: Index Foreign Key Columns
impact: HIGH
impactDescription: 10-100x faster JOINs and CASCADE operations
tags: foreign-key, indexes, joins, schema
---
## Index Foreign Key Columns
Postgres does not automatically index foreign key columns. Missing indexes cause slow JOINs and CASCADE operations.
**Incorrect (unindexed foreign key):**
```sql
create table orders (
id bigint generated always as identity primary key,
customer_id bigint references customers(id) on delete cascade,
total numeric(10,2)
);
-- No index on customer_id!
-- JOINs and ON DELETE CASCADE both require full table scan
select * from orders where customer_id = 123; -- Seq Scan
delete from customers where id = 123; -- Locks table, scans all orders
```
**Correct (indexed foreign key):**
```sql
create table orders (
id bigint generated always as identity primary key,
customer_id bigint references customers(id) on delete cascade,
total numeric(10,2)
);
-- Always index the FK column
create index orders_customer_id_idx on orders (customer_id);
-- Now JOINs and cascades are fast
select * from orders where customer_id = 123; -- Index Scan
delete from customers where id = 123; -- Uses index, fast cascade
```
Find missing FK indexes:
```sql
select
conrelid::regclass as table_name,
a.attname as fk_column
from pg_constraint c
join pg_attribute a on a.attrelid = c.conrelid and a.attnum = any(c.conkey)
where c.contype = 'f'
and not exists (
select 1 from pg_index i
where i.indrelid = c.conrelid and a.attnum = any(i.indkey)
);
```
Reference: [Foreign Keys](https://www.postgresql.org/docs/current/ddl-constraints.html#DDL-CONSTRAINTS-FK)
@@ -1,55 +0,0 @@
---
title: Use Lowercase Identifiers for Compatibility
impact: MEDIUM
impactDescription: Avoid case-sensitivity bugs with tools, ORMs, and AI assistants
tags: naming, identifiers, case-sensitivity, schema, conventions
---
## Use Lowercase Identifiers for Compatibility
PostgreSQL folds unquoted identifiers to lowercase. Quoted mixed-case identifiers require quotes forever and cause issues with tools, ORMs, and AI assistants that may not recognize them.
**Incorrect (mixed-case identifiers):**
```sql
-- Quoted identifiers preserve case but require quotes everywhere
CREATE TABLE "Users" (
"userId" bigint PRIMARY KEY,
"firstName" text,
"lastName" text
);
-- Must always quote or queries fail
SELECT "firstName" FROM "Users" WHERE "userId" = 1;
-- This fails - Users becomes users without quotes
SELECT firstName FROM Users;
-- ERROR: relation "users" does not exist
```
**Correct (lowercase snake_case):**
```sql
-- Unquoted lowercase identifiers are portable and tool-friendly
CREATE TABLE users (
user_id bigint PRIMARY KEY,
first_name text,
last_name text
);
-- Works without quotes, recognized by all tools
SELECT first_name FROM users WHERE user_id = 1;
```
Common sources of mixed-case identifiers:
```sql
-- ORMs often generate quoted camelCase - configure them to use snake_case
-- Migrations from other databases may preserve original casing
-- Some GUI tools quote identifiers by default - disable this
-- If stuck with mixed-case, create views as a compatibility layer
CREATE VIEW users AS SELECT "userId" AS user_id, "firstName" AS first_name FROM "Users";
```
Reference: [Identifiers and Key Words](https://www.postgresql.org/docs/current/sql-syntax-lexical.html#SQL-SYNTAX-IDENTIFIERS)
@@ -1,55 +0,0 @@
---
title: Partition Large Tables for Better Performance
impact: MEDIUM-HIGH
impactDescription: 5-20x faster queries and maintenance on large tables
tags: partitioning, large-tables, time-series, performance
---
## Partition Large Tables for Better Performance
Partitioning splits a large table into smaller pieces, improving query performance and maintenance operations.
**Incorrect (single large table):**
```sql
create table events (
id bigint generated always as identity,
created_at timestamptz,
data jsonb
);
-- 500M rows, queries scan everything
select * from events where created_at > '2024-01-01'; -- Slow
vacuum events; -- Takes hours, locks table
```
**Correct (partitioned by time range):**
```sql
create table events (
id bigint generated always as identity,
created_at timestamptz not null,
data jsonb
) partition by range (created_at);
-- Create partitions for each month
create table events_2024_01 partition of events
for values from ('2024-01-01') to ('2024-02-01');
create table events_2024_02 partition of events
for values from ('2024-02-01') to ('2024-03-01');
-- Queries only scan relevant partitions
select * from events where created_at > '2024-01-15'; -- Only scans events_2024_01+
-- Drop old data instantly
drop table events_2023_01; -- Instant vs DELETE taking hours
```
When to partition:
- Tables > 100M rows
- Time-series data with date-based queries
- Need to efficiently drop old data
Reference: [Table Partitioning](https://www.postgresql.org/docs/current/ddl-partitioning.html)
@@ -1,61 +0,0 @@
---
title: Select Optimal Primary Key Strategy
impact: HIGH
impactDescription: Better index locality, reduced fragmentation
tags: primary-key, identity, uuid, serial, schema
---
## Select Optimal Primary Key Strategy
Primary key choice affects insert performance, index size, and replication
efficiency.
**Incorrect (problematic PK choices):**
```sql
-- identity is the SQL-standard approach
create table users (
id serial primary key -- Works, but IDENTITY is recommended
);
-- Random UUIDs (v4) cause index fragmentation
create table orders (
id uuid default gen_random_uuid() primary key -- UUIDv4 = random = scattered inserts
);
```
**Correct (optimal PK strategies):**
```sql
-- Use IDENTITY for sequential IDs (SQL-standard, best for most cases)
create table users (
id bigint generated always as identity primary key
);
-- For distributed systems needing UUIDs, use UUIDv7 (time-ordered)
-- Requires pg_uuidv7 extension: create extension pg_uuidv7;
create table orders (
id uuid default uuid_generate_v7() primary key -- Time-ordered, no fragmentation
);
-- Alternative: time-prefixed IDs for sortable, distributed IDs (no extension needed)
create table events (
id text default concat(
to_char(now() at time zone 'utc', 'YYYYMMDDHH24MISSMS'),
gen_random_uuid()::text
) primary key
);
```
Guidelines:
- Single database: `bigint identity` (sequential, 8 bytes, SQL-standard)
- Distributed/exposed IDs: UUIDv7 (requires pg_uuidv7) or ULID (time-ordered, no
fragmentation)
- `serial` works but `identity` is SQL-standard and preferred for new
applications
- Avoid random UUIDs (v4) as primary keys on large tables (causes index
fragmentation)
Reference:
[Identity Columns](https://www.postgresql.org/docs/current/sql-createtable.html#SQL-CREATETABLE-PARMS-GENERATED-IDENTITY)
@@ -1,54 +0,0 @@
---
title: Apply Principle of Least Privilege
impact: MEDIUM
impactDescription: Reduced attack surface, better audit trail
tags: privileges, security, roles, permissions
---
## Apply Principle of Least Privilege
Grant only the minimum permissions required. Never use superuser for application queries.
**Incorrect (overly broad permissions):**
```sql
-- Application uses superuser connection
-- Or grants ALL to application role
grant all privileges on all tables in schema public to app_user;
grant all privileges on all sequences in schema public to app_user;
-- Any SQL injection becomes catastrophic
-- drop table users; cascades to everything
```
**Correct (minimal, specific grants):**
```sql
-- Create role with no default privileges
create role app_readonly nologin;
-- Grant only SELECT on specific tables
grant usage on schema public to app_readonly;
grant select on public.products, public.categories to app_readonly;
-- Create role for writes with limited scope
create role app_writer nologin;
grant usage on schema public to app_writer;
grant select, insert, update on public.orders to app_writer;
grant usage on sequence orders_id_seq to app_writer;
-- No DELETE permission
-- Login role inherits from these
create role app_user login password 'xxx';
grant app_writer to app_user;
```
Revoke public defaults:
```sql
-- Revoke default public access
revoke all on schema public from public;
revoke all on all tables in schema public from public;
```
Reference: [Roles and Privileges](https://supabase.com/blog/postgres-roles-and-privileges)
@@ -1,50 +0,0 @@
---
title: Enable Row Level Security for Multi-Tenant Data
impact: CRITICAL
impactDescription: Database-enforced tenant isolation, prevent data leaks
tags: rls, row-level-security, multi-tenant, security
---
## Enable Row Level Security for Multi-Tenant Data
Row Level Security (RLS) enforces data access at the database level, ensuring users only see their own data.
**Incorrect (application-level filtering only):**
```sql
-- Relying only on application to filter
select * from orders where user_id = $current_user_id;
-- Bug or bypass means all data is exposed!
select * from orders; -- Returns ALL orders
```
**Correct (database-enforced RLS):**
```sql
-- Enable RLS on the table
alter table orders enable row level security;
-- Create policy for users to see only their orders
create policy orders_user_policy on orders
for all
using (user_id = current_setting('app.current_user_id')::bigint);
-- Force RLS even for table owners
alter table orders force row level security;
-- Set user context and query
set app.current_user_id = '123';
select * from orders; -- Only returns orders for user 123
```
Policy for authenticated role:
```sql
create policy orders_user_policy on orders
for all
to authenticated
using (user_id = auth.uid());
```
Reference: [Row Level Security](https://supabase.com/docs/guides/database/postgres/row-level-security)
@@ -1,57 +0,0 @@
---
title: Optimize RLS Policies for Performance
impact: HIGH
impactDescription: 5-10x faster RLS queries with proper patterns
tags: rls, performance, security, optimization
---
## Optimize RLS Policies for Performance
Poorly written RLS policies can cause severe performance issues. Use subqueries and indexes strategically.
**Incorrect (function called for every row):**
```sql
create policy orders_policy on orders
using (auth.uid() = user_id); -- auth.uid() called per row!
-- With 1M rows, auth.uid() is called 1M times
```
**Correct (wrap functions in SELECT):**
```sql
create policy orders_policy on orders
using ((select auth.uid()) = user_id); -- Called once, cached
-- 100x+ faster on large tables
```
Use security definer functions for complex checks:
```sql
-- Create helper function (runs as definer, bypasses RLS)
create or replace function is_team_member(team_id bigint)
returns boolean
language sql
security definer
set search_path = ''
as $$
select exists (
select 1 from public.team_members
where team_id = $1 and user_id = (select auth.uid())
);
$$;
-- Use in policy (indexed lookup, not per-row check)
create policy team_orders_policy on orders
using ((select is_team_member(team_id)));
```
Always add indexes on columns used in RLS policies:
```sql
create index orders_user_id_idx on orders (user_id);
```
Reference: [RLS Performance](https://supabase.com/docs/guides/database/postgres/row-level-security#rls-performance-recommendations)
-112
View File
@@ -1,112 +0,0 @@
---
name: supabase
description: "Use when doing ANY task involving Supabase. Triggers: Supabase products (Database, Auth, Edge Functions, Realtime, Storage, Vectors, Cron, Queues); client libraries and SSR integrations (supabase-js, @supabase/ssr) in Next.js, React, SvelteKit, Astro, Remix; auth issues (login, logout, sessions, JWT, cookies, getSession, getUser, getClaims, RLS); Supabase CLI or MCP server; schema changes, migrations, security audits, Postgres extensions (pg_graphql, pg_cron, pg_vector)."
metadata:
author: supabase
version: "0.1.2"
---
# Supabase
## Core Principles
**1. Supabase changes frequently — verify against changelog and current docs before implementing.**
Do not rely on training data for Supabase features. Function signatures, config.toml settings, and API conventions change between versions.
First, fetch `https://supabase.com/changelog.md` (a lightweight summary index — not a heavy pull), scan for `breaking-change` tags relevant to your task, and follow the linked page for any that apply. Then look up the relevant topic using the documentation access methods below.
**2. Verify your work.**
After implementing any fix, run a test query to confirm the change works. A fix without verification is incomplete.
**3. Recover from errors, don't loop.**
If an approach fails after 2-3 attempts, stop and reconsider. Try a different method, check documentation, inspect the error more carefully, and review relevant logs when available. Supabase issues are not always solved by retrying the same command, and the answer is not always in the logs, but logs are often worth checking before proceeding.
**4. Exposing tables to the Data API:** Depending on the user's [Data API settings](https://supabase.com/dashboard/project/<ref>/integrations/data_api/settings), newly created tables may not be automatically exposed via the Data (REST) API. If this is the case, `anon` and `authenticated` roles will need to be explicitly granted access.
> Note that this is separate from RLS, which controls which _rows_ are visible once a table is accessible, not whether the table is accessible at all.
When a user reports a SQL-created table is unexpectedly inaccessible, check their Data API settings and whether the roles have been granted access via explicit `GRANT` SQL. When granting public (`anon`/`authenticated`) access, always enable RLS too. See [Exposing a Table to the Data API](https://supabase.com/docs/guides/api/securing-your-api.md) for the full setup workflow.
**5. RLS in exposed schemas.**
Enable RLS on every table in any exposed schema, which includes `public` by default. This is critical in Supabase because tables in exposed schemas can be reachable through the Data API when the `anon`/`authenticated` roles have access (see [Exposing a Table to the Data API](https://supabase.com/docs/guides/api/securing-your-api.md)). For private schemas, prefer RLS as defense in depth. After enabling RLS, create policies that match the actual access model rather than defaulting every table to the same `auth.uid()` pattern.
**6. Security checklist.**
When working on any Supabase task that touches auth, RLS, views, storage, or user data, run through this checklist. These are Supabase-specific security traps that silently create vulnerabilities:
- **Auth and session security**
- **Never use `user_metadata` claims in JWT-based authorization decisions.** In Supabase, `raw_user_meta_data` is user-editable and can appear in `auth.jwt()`, so it is unsafe for RLS policies or any other authorization logic. Store authorization data in `raw_app_meta_data` / `app_metadata` instead.
- **Deleting a user does not invalidate existing access tokens.** Sign out or revoke sessions first, keep JWT expiry short for sensitive apps, and for strict guarantees validate `session_id` against `auth.sessions` on sensitive operations.
- **If you use `app_metadata` or `auth.jwt()` for authorization, remember JWT claims are not always fresh until the user's token is refreshed.**
- **API key and client exposure**
- **Never expose the `service_role` or secret key in public clients.** Prefer publishable keys for frontend code. Legacy `anon` keys are only for compatibility. In Next.js, any `NEXT_PUBLIC_` env var is sent to the browser.
- **RLS, views, and privileged database code**
- **Views bypass RLS by default.** In Postgres 15 and above, use `CREATE VIEW ... WITH (security_invoker = true)`. In older versions of Postgres, protect your views by revoking access from the `anon` and `authenticated` roles, or by putting them in an unexposed schema.
- **UPDATE requires a SELECT policy.** In Postgres RLS, an UPDATE needs to first SELECT the row. Without a SELECT policy, updates silently return 0 rows — no error, just no change.
- **Do not put `security definer` functions in an exposed schema.** Keep them in a private or otherwise unexposed schema.
- **Storage access control**
- **Storage upsert requires INSERT + SELECT + UPDATE.** Granting only INSERT allows new uploads but file replacement (upsert) silently fails. You need all three.
For any security concern not covered above, fetch the Supabase product security index: `https://supabase.com/docs/guides/security/product-security.md`
## Supabase CLI
Always discover commands via `--help` — never guess. The CLI structure changes between versions.
```bash
supabase --help # All top-level commands
supabase <group> --help # Subcommands (e.g., supabase db --help)
supabase <group> <command> --help # Flags for a specific command
```
**Supabase CLI Known gotchas:**
- `supabase db query` requires **CLI v2.79.0+** → use MCP `execute_sql` or `psql` as fallback
- `supabase db advisors` requires **CLI v2.81.3+** → use MCP `get_advisors` as fallback
- When you need a new migration SQL file, **always** create it with `supabase migration new <name>` first. Never invent a migration filename or rely on memory for the expected format.
**Version check and upgrade:** Run `supabase --version` to check. For CLI changelogs and version-specific features, consult the [CLI documentation](https://supabase.com/docs/reference/cli/introduction) or [GitHub releases](https://github.com/supabase/cli/releases).
## Supabase MCP Server
For setup instructions, server URL, and configuration, see the [MCP setup guide](https://supabase.com/docs/guides/getting-started/mcp).
**Troubleshooting connection issues** — follow these steps in order:
1. **Check if the server is reachable:**
`curl -so /dev/null -w "%{http_code}" https://mcp.supabase.com/mcp`
A `401` is expected (no token) and means the server is up. Timeout or "connection refused" means it may be down.
2. **Check `.mcp.json` configuration:**
Verify the project root has a valid `.mcp.json` with the correct server URL. If missing, create one pointing to `https://mcp.supabase.com/mcp`.
3. **Authenticate the MCP server:**
If the server is reachable and `.mcp.json` is correct but tools aren't visible, the user needs to authenticate. The Supabase MCP server uses OAuth 2.1 — tell the user to trigger the auth flow in their agent, complete it in the browser, and reload the session.
## Supabase Documentation
Before implementing any Supabase feature, find the relevant documentation. Use these methods in priority order:
1. **MCP `search_docs` tool** (preferred — returns relevant snippets directly)
2. **Fetch docs pages as markdown** — any docs page can be fetched by appending `.md` to the URL path.
3. **Web search** for Supabase-specific topics when you don't know which page to look at.
## Making and Committing Schema Changes
**To make schema changes, use `execute_sql` (MCP) or `supabase db query` (CLI).** These run SQL directly on the database without creating migration history entries, so you can iterate freely and generate a clean migration when ready.
Do NOT use `apply_migration` to change a local database schema — it writes a migration history entry on every call, which means you can't iterate, and `supabase db diff` / `supabase db pull` will produce empty or conflicting diffs. If you use it, you'll be stuck with whatever SQL you passed on the first try.
**When ready to commit** your changes to a migration file:
1. **Run advisors** → `supabase db advisors` (CLI v2.81.3+) or MCP `get_advisors`. Fix any issues.
2. **Review the Security Checklist above** if your changes involve views, functions, triggers, or storage.
3. **Generate the migration** → `supabase db pull <descriptive-name> --local --yes`
4. **Verify** → `supabase migration list --local`
## Reference Guides
- **Skill Feedback** → [references/skill-feedback.md](references/skill-feedback.md)
**MUST read when** the user reports that this skill gave incorrect guidance or is missing information.
@@ -1,17 +0,0 @@
## What happened
**Task:** <!-- e.g., "Set up MFA on patient records" -->
**Skill said:** <!-- e.g., "Use auth.jwt()->'app_metadata' in the RLS policy" -->
**Expected:** <!-- e.g., "The function also needs SECURITY DEFINER + grant to supabase_auth_admin" -->
## Source
**File:** <!-- e.g., references/security-model.md -->
**Section:** <!-- e.g., "Trust Boundaries > user_metadata vs app_metadata" -->
## Fix suggestion
<!-- Leave blank if unsure -->
@@ -1,17 +0,0 @@
# Skill Feedback
Use this when the user reports that the skill gave incorrect guidance, is missing information, or could be improved. This is about the skill (agent instructions), not about Supabase the product.
## Steps
1. **Ask permission** — Ask the user if they'd like to submit feedback to the skill maintainers. If they decline, move on.
2. **Draft the issue** — Use the template at [assets/feedback-issue-template.md](../assets/feedback-issue-template.md) to structure the feedback. Fill in the fields based on the conversation. Always identify which specific reference file and section caused the problem.
3. **Submit** — Create a GitHub Issue on the `supabase/agent-skills` repository using the draft as the issue body. The title must follow this format: `user-feedback: <summary of the problem>`.
4. **Share the result** — Share the issue URL with the user after submission. If submission fails, give the user this link to create the issue manually:
```
https://github.com/supabase/agent-skills/issues/new
```
-32
View File
@@ -1,32 +0,0 @@
---
name: typescript-docs
description: Use when writing new TypeScript code or refactoring existing code to ensure strict TSDoc/JSDoc hygiene is applied to all methods and interfaces.
---
# Strict TypeScript Documentation
## When to use this skill
Trigger this skill automatically whenever you are creating a new function, class, or interface, OR when you are modifying an existing undocumented function. This is the "Nutrition Label" rule.
## Guidelines
1. **JSDoc/TSDoc format is mandatory**. Always use `/** ... */` above the definition.
2. The block MUST include a short, plain-English description of *what* the function does and *why* it exists.
3. MUST explicitly list all parameters using `@param name - Description`.
4. MUST explicitly define the return value using `@returns Description`.
5. If the function intentionally throws an error, use `@throws {ErrorType} Description`.
## Example
```ts
/**
* Resolves the appropriate WebSocket engine URL for a given plugin.
* Prioritizes the local engine during development before falling back to cloud.
*
* @param pluginId - The unique string identifier of the plugin.
* @returns The fully qualified WebSocket URL for the stream.
* @throws {Error} If the plugin ID is undefined or missing.
*/
export function resolveEngineUrl(pluginId: string): string { ... }
```
## Actionable Steps
Never consider a PR or code edit "done" unless the new or modified functions have these strict labels attached. If you spot an undocumented function nearby while working, clean it up!
@@ -0,0 +1,65 @@
---
name: using-git-worktrees
description: Use when tasked with creating a new git worktree, building features in isolation, setting up parallel branches, or provisioning a safe sandbox environment to test architectural changes.
---
# Using Git Worktrees (Worktrunk)
## Overview
Git worktrees allow you to check out multiple branches of a repository simultaneously without cloning the entire repository from scratch. However, creating worktrees in nested subdirectories breaks relative path resolutions to peer monorepo packages. We use **Worktrunk** (`git-wt`) to automatically provision sibling worktrees and handle environment setup.
## When to Use
- When asked to create an isolated environment for a feature or branch.
- When you need a safe sandbox to test destructive or complex architectural changes.
- When working on multiple GitHub issues in parallel.
## Core Directives
### 1. NEVER Nest Worktrees
If you create a worktree inside a subdirectory of the current repo (e.g., `.agents/worktrees/feature`), all relative paths going UP and OUT of the repository (like `../../wwv-data-engine`) will immediately break due to the altered directory depth.
**Worktrees MUST be siblings of the main repository root.**
### 2. ALWAYS Use Worktrunk
Do not use raw `git worktree add`. Always use the `git-wt` alias provided by Worktrunk. It is pre-configured via `.config/wt.toml` to:
- Automatically place the worktree in the correct sibling path.
- Automatically run `pnpm install`.
- Automatically copy `.env` files so the environment works immediately.
> [!IMPORTANT]
> **For AI Agents:** You MUST append the `--yes` flag when creating worktrees to bypass the interactive approval prompt for these hooks, otherwise the command will fail in your non-interactive shell.
### 3. Database Lifecycle & Port Assignment
Each worktree gets a deterministic PostgreSQL port assigned automatically based on its directory hash (to avoid collisions between multiple running worktrees).
> [!CAUTION]
> **Orphaned Volumes:** You MUST use `git-wt remove` to delete worktrees. This triggers a `pre-remove` hook that cleanly tears down the Docker database. If you manually delete the folder (`rm -rf`), the Docker volume is orphaned. If this happens, run `pnpm run db:prune` in the main repository to garbage collect orphaned database volumes.
## Quick Reference
| Action | Command | Details |
| :--- | :--- | :--- |
| **Create Worktree** | `git-wt switch --create <branch> --yes` | Provisions the sibling directory, branch, env files, and dependencies. |
| **Delete Worktree** | `git-wt remove` | Deletes the worktree and the associated branch cleanly. Run from within the worktree. |
## Implementation Steps
1. Run the create command from the main repository (always include `--yes`):
```bash
git-wt switch --create feature/my-new-idea --yes
```
2. **CRITICAL:** Change your working directory to the newly created worktree before editing files. Worktrunk creates it as a sibling.
```bash
# Example if the main repo is c:\dev\worldwideview
cd ../worldwideview.feature-my-new-idea
```
3. Do your work, run tests, and commit.
4. When completely finished and merged, clean up by running `git-wt remove` from inside the worktree directory.
## Common Mistakes & Red Flags
| Excuse / Rationalization | Reality & Fix |
| :--- | :--- |
| "I'll just put the worktree in `.agents/worktrees` to keep the project folder clean." | **Red Flag:** This breaks relative paths to other monorepo packages. Delete the worktree and use `git-wt` to create a sibling worktree instead. |
| "I'll use `git worktree add` instead of `git-wt` because it's standard." | **Red Flag:** Raw git won't copy `.env` files or run the `pre-start` hooks for `pnpm install`, leaving you with a broken environment. Delete and use `git-wt`. |
| "I ran the command, but my file edits are applying to the main branch." | You forgot to `cd` into the newly created sibling directory. |
@@ -29,20 +29,15 @@ A WorldWideView plugin is a self-contained data source that renders geospatial e
## Architecture Decision
```dot
digraph plugin_type {
rankdir=TB;
node [shape=diamond];
"Does it need live\nexternal API data?" -> "Is polling\nfrequency > 30s?" [label="yes"];
"Does it need live\nexternal API data?" -> "Static GeoJSON\n(wwvStaticCompiler)" [label="no"];
"Is polling\nfrequency > 30s?" -> "Cron Seeder\n(engine)" [label="yes"];
"Is polling\nfrequency > 30s?" -> "Init Seeder\n(engine)" [label="no"];
"Static GeoJSON\n(wwvStaticCompiler)" [shape=box];
"Cron Seeder\n(engine)" [shape=box];
"Init Seeder\n(engine)" [shape=box];
}
```
Does it need live external API data?
├── NO → Static GeoJSON plugin (wwvStaticCompiler in vite.config.ts)
│ No seeder needed. Place data in data/data.json.
└── YES → Does it update more often than every 30s?
├── NO → Cron Seeder (engine, registerSeeder with cron string)
│ getPollingInterval() = 0, mapWebsocketPayload() required
└── YES → Init Seeder (engine, registerSeeder with init function)
getPollingInterval() = 0, mapWebsocketPayload() required
```
---
@@ -146,7 +141,7 @@ export default class WildfiresPlugin implements WorldPlugin {
| `getLayerConfig()` | Layer appearance (color, clustering, max entities) |
| `renderEntity(entity)` | Per-entity Cesium rendering options |
**Optional:**
**Optional (but `mapWebsocketPayload` is REQUIRED for WS-only plugins with object payloads):**
| Method | Purpose |
|---|---|
@@ -158,7 +153,7 @@ export default class WildfiresPlugin implements WorldPlugin {
| `getDetailComponent()` | Custom detail panel for selected entity |
| `getSettingsComponent()` | Plugin settings UI |
| `getGlobeComponent()` | React component injected into globe view |
| `mapWebsocketPayload(payload, existing)` | Transform raw WS data into `GeoEntity[]` |
| `mapWebsocketPayload(payload, existing)` | **Required if seeder payload is an object** (not a flat array). Transforms raw engine payload → `GeoEntity[]`. Without it, `WsClient` drops object payloads silently. |
| `requiresConfiguration(settings)` | Return `true` if plugin needs setup first |
### 1.3 Package Metadata
@@ -518,14 +513,30 @@ export default class IssPlugin implements WorldPlugin {
destroy(): void {}
async fetch(timeRange: TimeRange): Promise<GeoEntity[]> {
return []; // WS-only plugin
async fetch(_timeRange: TimeRange): Promise<GeoEntity[]> {
return []; // WS-only — data arrives via mapWebsocketPayload
}
getPollingInterval(): number {
return 0; // WS-only
}
// REQUIRED: seeder sends { source, fetchedAt, items: [...], totalCount } (an object).
// Without this, WsClient drops the payload silently — the globe stays empty.
mapWebsocketPayload(payload: any, _existingEntities: GeoEntity[]): GeoEntity[] {
const items: any[] = payload?.items ?? (Array.isArray(payload) ? payload : []);
return items.map((item: any): GeoEntity => ({
id: `iss-${item.id}`,
pluginId: "iss",
latitude: item.lat,
longitude: item.lon,
altitude: item.alt ?? 0,
timestamp: new Date(),
label: "International Space Station",
properties: { speed: item.speed },
}));
}
getLayerConfig(): LayerConfig {
return {
color: "#ffffff",
@@ -644,7 +655,9 @@ Categories are **lowercase**: `"aviation"` not `"Aviation"`, `"natural-disaster"
- [ ] **Engine:** Seeder file in `wwv-seeders/src/seeders/<name>.ts` (or private repo)
- [ ] **Engine:** `registerSeeder({ name })` matches frontend plugin `id`
- [ ] **Engine:** Calls `setLiveSnapshot()` with correct plugin ID key
- [ ] **Engine:** Data shape is an object with `items` array (or plugin implements `mapWebsocketPayload`)
- [ ] **Engine:** Seeder payload is either a flat `GeoEntity[]` array OR plugin implements `mapWebsocketPayload` (any other object is silently dropped by WsClient)
- [ ] **Frontend (WS-only):** `getPollingInterval()` returns `0` and `fetch()` returns `[]`
- [ ] **Frontend (WS-only):** `mapWebsocketPayload(payload, existingEntities)` implemented if seeder sends an object
- [ ] **Engine:** SQLite table created in `db.ts` if historical data needed
- [ ] **Verification:** `GET localhost:5000/manifest` includes the plugin ID
- [ ] **Verification:** `GET localhost:5000/health` shows seeder last-run timestamp
-1
View File
@@ -1 +0,0 @@
.agents
Symlink
+1
View File
@@ -0,0 +1 @@
.agents
+50
View File
@@ -0,0 +1,50 @@
node_modules
**/node_modules
**/dist
**/.next
.git
# Database files at ANY depth
/data
packages/*/data
**/*.db
**/*.sqlite
**/*.sqlite3
**/*-journal
**/*.db-wal
**/*.db-shm
# ============================================================
# UNIGNORE FOR AI AGENT (FILTERED PATHS ONLY)
# ============================================================
# Code Frameworks, Configurations & Typings
!/out/
!*.tsbuildinfo
!next-env.d.ts
!/build
!*.code-workspace
# Generated Codebase Assets & Database Schemas (e.g., Prisma Client)
!/src/generated/
# Permanent Local Automation and Scripts Shared in Project
!/local-scripts/
!/public/plugins-local/
# Local Structural Repositories & Workspaces
!local-plugins/*
!/worldmonitor/
!local-seeders/*
# Complete Ecosystem Repositories (Allows Agent to Cross-Reference Layouts)
!/packages/wwv-data-engine/
!/worldwideview-marketplace/
!/worldwideview-plugins/
!/worldwideview-web/
!/wwv-data-engine/
!/wwv-data-engine-public/
!/wwv-seeders/
!/wwv-seeders-private/
!/wwv-test/
!worldwideview.code-workspace
+14
View File
@@ -0,0 +1,14 @@
[[pre-start]]
install_deps = "pnpm install"
[[pre-start]]
copy_env = """powershell -Command "if (Test-Path '../worldwideview/.env') { Copy-Item '../worldwideview/.env' '.env' -Force }""""
[[pre-start]]
copy_local_plugins = """powershell -Command "if (Test-Path '../worldwideview/local-plugins') { robocopy '../worldwideview/local-plugins' 'local-plugins' /E /XD node_modules dist .next coverage /XF '*.log' /NFL /NDL /NJH /NJS | Out-Null }""""
[[pre-start]]
copy_local_seeders = """powershell -Command "if (Test-Path '../worldwideview/local-seeders') { robocopy '../worldwideview/local-seeders' 'local-seeders' /E /XD .pnpm-store node_modules /NFL /NDL /NJH /NJS | Out-Null }""""
[[pre-remove]]
docker_clean = "docker compose down -v"
+30
View File
@@ -19,6 +19,12 @@ DATABASE_URL=postgresql://postgres:postgres@localhost:5432/worldwideview
# at the project root so docker-compose can inject it.
AUTH_SECRET=
# Encryption Master Key (REQUIRED — generate with: openssl rand -hex 16)
# Must be exactly 32 characters. Used to derive the AES-256-GCM key for credentials at rest.
# PRE-DEPLOY GATE: verify this is set in Coolify for every environment before merging C1 changes.
# If unset the server will refuse to start (enforced in src/instrumentation.ts).
ENCRYPTION_MASTER_KEY=
# Cesium Ion Access Token (optional — get one free at https://ion.cesium.com/)
NEXT_PUBLIC_CESIUM_ION_TOKEN=
@@ -48,6 +54,25 @@ ACLED_PASSWORD=
# Generate any random string, then set the same value in the marketplace instance config.
WWV_BRIDGE_TOKEN=
# Marketplace URL — used by ticketClient.ts for token exchange (server-side only)
# Falls back to NEXT_PUBLIC_WWV_MARKETPLACE_URL then https://app.worldwideview.dev
MARKETPLACE_URL=
# Marketplace URL — used for PKCE connect/callback and client-side links
NEXT_PUBLIC_WWV_MARKETPLACE_URL=
# Plugin Ticket Auth — comma-separated plugin IDs that require WS first-message auth (ADR-001).
# Empty (default) = ticket auth dormant for all plugins. Set AFTER Marketplace + Data Engine deploy.
# Example: NEXT_PUBLIC_WWV_TICKET_AUTH_PLUGINS=aviation,maritime
NEXT_PUBLIC_WWV_TICKET_AUTH_PLUGINS=
# SSRF Proxy Host Allowlist (REQUIRED in production)
# "*" = permissive (logs WARN per request — use to observe traffic for one week, then tighten).
# "" = deny all (safe default when not set).
# Comma-separated list = only those hostnames allowed (e.g. "192.168.1.100,camera.example.com").
# Ship with "*" in Coolify; replace with concrete list after one week of log observation.
PROXY_HOST_ALLOWLIST=*
# Default plugins enabled on boot for demo edition
# Comma-separated list of plugin IDs
NEXT_PUBLIC_DEMO_DEFAULT_PLUGINS=
@@ -55,6 +80,11 @@ NEXT_PUBLIC_DEMO_DEFAULT_PLUGINS=
# Edition: "local" | "cloud" | "demo"
NEXT_PUBLIC_WWV_EDITION=local
# Optional Database Teardown
# If set to true, terminating 'pnpm run dev' (Ctrl+C) will automatically
# stop the local database container (freeing up port 5432).
WWV_TEARDOWN_DB_ON_EXIT=false
# Demo Admin Secret (demo edition only)
# Allows operator plugin management on the demo instance via x-wwv-admin-secret header.
# Generate any random string. Only set this on the demo deployment.
+12
View File
@@ -0,0 +1,12 @@
# Garbage Collector — Daily Sweep Prompt
You are the WorldWideView Garbage Collector agent running headless in GitHub Actions.
Run the garbage-collector agent sweep:
1. Read `.agents/rules/garbage-collection.md` for your charter and all guard rails.
2. Read `gc-findings.json` for the pre-scanned findings produced by `scripts/gc-scan.mjs`.
3. Check the `DRY_RUN` environment variable — if `true`, post a dry-run summary only (no PRs or Issues).
4. Execute the full sweep procedure described in `.agents/agents/garbage-collector.md`.
Do not scan the codebase yourself. Work only from `gc-findings.json`.
+116
View File
@@ -0,0 +1,116 @@
name: Garbage Collector
on:
schedule:
- cron: '0 6 * * *' # 06:00 UTC daily
workflow_dispatch:
inputs:
dry_run:
description: 'Dry run — report findings only, open no PRs or Issues (recommended for first 3–5 days)'
required: false
default: 'true'
type: choice
options: ['true', 'false']
days_branch:
description: 'Days before a branch is considered stale'
required: false
default: '90'
days_comment:
description: 'Days before a TODO/FIXME comment is flagged'
required: false
default: '180'
concurrency:
group: garbage-collector
cancel-in-progress: true
permissions:
contents: write
pull-requests: write
issues: write
id-token: write
jobs:
collect:
runs-on: ubuntu-latest
timeout-minutes: 15
env:
GH_TOKEN: ${{ secrets.GC_BOT_TOKEN }}
DRY_RUN: ${{ github.event.inputs.dry_run || 'false' }}
DAYS_BRANCH: ${{ github.event.inputs.days_branch || '90' }}
DAYS_COMMENT: ${{ github.event.inputs.days_comment || '180' }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0 # full history required for git blame and branch age
token: ${{ secrets.GC_BOT_TOKEN }} # PAT so pushed branches trigger CI
- name: Configure git user
run: |
git config user.name 'WorldWideView GC Bot'
git config user.email '[email protected]'
- name: Setup pnpm
uses: pnpm/action-setup@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '22'
cache: 'pnpm'
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Run deterministic scan
run: |
node scripts/gc-scan.mjs \
--out gc-findings.json \
--days-branch=${{ env.DAYS_BRANCH }} \
--days-comment=${{ env.DAYS_COMMENT }}
- name: Upload findings artifact
uses: actions/upload-artifact@v4
with:
name: gc-findings-${{ github.run_id }}
path: gc-findings.json
retention-days: 30
- name: Run Garbage Collector agent
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
show_full_output: true
prompt: |
You are the WorldWideView Garbage Collector agent running headless in GitHub Actions.
Read `.agents/rules/garbage-collection.md` for your charter and all guard rails.
Read `gc-findings.json` for the pre-scanned findings produced by `scripts/gc-scan.mjs`.
The DRY_RUN environment variable is: ${{ env.DRY_RUN }}
## If DRY_RUN is "true":
Do NOT create any PRs or Issues.
Instead, find or create the tracking Issue titled "[gc-bot] Daily Scan Reports" (label: gc-bot) in
the silvertakana/worldwideview repository using the gh CLI:
# Find existing tracking issue
gh issue list --repo silvertakana/worldwideview --label gc-bot --search "[gc-bot] Daily Scan Reports" --json number,title
# If it exists, add a comment:
gh issue comment <number> --repo silvertakana/worldwideview --body "..."
# If it doesn't exist, create it:
gh issue create --repo silvertakana/worldwideview --title "[gc-bot] Daily Scan Reports" --label gc-bot --body "..."
The comment body must list every finding with: tier, file, line, description.
Include the run ID (${{ github.run_id }}) and timestamp at the top.
## If DRY_RUN is "false":
Execute the full sweep procedure described in `.claude/agents/garbage-collector.md`.
Create Tier A draft PRs and Tier B Issues per the guard rails in `.agents/rules/garbage-collection.md`.
Do not scan the codebase yourself. Work only from gc-findings.json.
claude_args: '--model claude-haiku-4-5-20251001 --allowedTools Bash,Read,Edit,Write,Grep,Glob'
+56
View File
@@ -0,0 +1,56 @@
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
permissions:
contents: read
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
browser: [chromium, firefox, webkit]
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v3
with:
version: 9.15.4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'pnpm'
- name: Install dependencies
run: pnpm install
- name: Install Playwright Browsers
run: pnpm exec playwright install --with-deps
- name: Run Project Setup
run: pnpm run setup
- name: Boot Database
run: |
docker network create coolify || true
pnpm run predev
- name: Run Playwright tests
env:
# Prevent Firefox from loading the AT-SPI/MSAA accessibility bridge on
# Linux CI — without this, Firefox hangs for ~20 min waiting for a bus
# that doesn't exist on ubuntu-latest runners.
NO_AT_BRIDGE: ${{ matrix.browser == 'firefox' && '1' || '0' }}
run: pnpm test:e2e --project=${{ matrix.browser }}
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: playwright-report-${{ matrix.browser }}
path: playwright-report/
retention-days: 30
+64
View File
@@ -0,0 +1,64 @@
name: Test Local Dev Environment
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
test-dev:
name: Test Local Dev Startup
runs-on: ubuntu-latest
env:
NEXT_PUBLIC_WWV_EDITION: local
AUTH_SECRET: ci-placeholder-secret-not-used-in-tests
ENCRYPTION_MASTER_KEY: ci-placeholder-key-not-used-in-tests
steps:
- uses: actions/checkout@v6
- uses: pnpm/action-setup@v6
- uses: actions/setup-node@v6
with:
node-version: 22
cache: pnpm
- name: Create coolify docker network
# Needed because docker-compose.yml references external coolify network
run: docker network create coolify || true
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Setup environment
run: |
cp .env.example .env
sed -i 's/^ENCRYPTION_MASTER_KEY=.*/ENCRYPTION_MASTER_KEY=ci-placeholder-key-not-used-in-tests/' .env
sed -i 's/^AUTH_SECRET=.*/AUTH_SECRET=ci-placeholder-secret-not-used-in-tests/' .env
- name: Test predev setup
# Tests that Docker DB boots, port mappings work, Prisma pushes schema, and plugins sync
run: pnpm run predev
- name: Test dev server startup
run: |
# Start the dev server in the background
pnpm run dev &
DEV_PID=$!
echo "Waiting for Next.js to respond on port 3000..."
# Wait up to 120 seconds for the dev server to return a response
timeout 120 bash -c 'until curl -s http://localhost:3000 > /dev/null; do sleep 5; done'
# Check if the process is still running
if ! kill -0 $DEV_PID 2>/dev/null; then
echo "Dev server crashed or failed to start within timeout!"
exit 1
fi
echo "Dev server started successfully!"
# Graceful shutdown
kill $DEV_PID
+121
View File
@@ -0,0 +1,121 @@
name: Test Setup Scripts
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
permissions:
contents: read
jobs:
test-bash:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.x'
- name: Start local HTTP server
run: |
python3 -m http.server 8080 &
sleep 2 # Wait for server to start
- name: Patch setup.sh to use local server
run: |
sed -i 's|https://raw.githubusercontent.com/silvertakana/worldwideview/main|http://localhost:8080|g' self-host/setup.sh
- name: Run setup.sh
run: |
cd self-host
bash setup.sh
- name: Verify .env generation
run: |
cd self-host
if [ ! -f .env ]; then
echo ".env file was not created"
exit 1
fi
if ! grep -q "^AUTH_SECRET=[a-f0-9]\{64\}$" .env; then
echo "AUTH_SECRET was not generated correctly or is missing"
exit 1
fi
- name: Verify docker-compose.yml download
run: |
cd self-host
if [ ! -f docker-compose.yml ]; then
echo "docker-compose.yml was not downloaded"
exit 1
fi
- name: Verify Docker containers are running
run: |
cd self-host
docker compose ps
if ! docker compose ps | grep -q "Up"; then
echo "Docker containers failed to start"
exit 1
fi
test-powershell:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.x'
- name: Start local HTTP server
run: |
python3 -m http.server 8080 &
sleep 2 # Wait for server to start
- name: Patch setup.ps1 to use local server
run: |
sed -i 's|https://raw.githubusercontent.com/silvertakana/worldwideview/main|http://localhost:8080|g' self-host/setup.ps1
- name: Run setup.ps1
run: |
cd self-host
pwsh ./setup.ps1
- name: Verify .env generation
run: |
cd self-host
if [ ! -f .env ]; then
echo ".env file was not created"
exit 1
fi
if ! grep -q "^AUTH_SECRET=[a-f0-9]\{64\}$" .env; then
echo "AUTH_SECRET was not generated correctly or is missing"
exit 1
fi
- name: Verify docker-compose.yml download
run: |
cd self-host
if [ ! -f docker-compose.yml ]; then
echo "docker-compose.yml was not downloaded"
exit 1
fi
- name: Verify Docker containers are running
run: |
cd self-host
docker compose ps
if ! docker compose ps | grep -q "Up"; then
echo "Docker containers failed to start"
exit 1
fi
+14 -7
View File
@@ -19,12 +19,9 @@ node_modules/
# agents
/.agents/plans
/.agents/context
/.agents/research/
# claude code IDE config
/.claude/
/.cline/
/.agents/context/
/.agents/settings.local.json
# performance
/performance/
@@ -32,6 +29,9 @@ node_modules/
# artifacts
/artifacts/
# gc bot — generated scan output, never commit
gc-findings.json
# scripts (tracked, but temporary scratchpads should go in /local-scripts/)
@@ -64,7 +64,8 @@ yarn-error.log*
.pnpm-debug.log*
# env files (can opt-in for committing if needed)
.env*
.env
.env.local
# vercel
.vercel
@@ -96,6 +97,7 @@ packages/wwv-data-engine/
# IDE workspace files
*.code-workspace
*.vscode
# Build outputs and generated assets
/public/cesium/
@@ -127,4 +129,9 @@ skills-lock.json
/wwv-seeders/
/wwv-seeders-private/
/wwv-test/
worldwideview.code-workspace
worldwideview.code-workspace
# Playwright
/playwright-report/
/test-results/
/playwright/.auth/
+82 -418
View File
@@ -2,11 +2,8 @@
## 1. Project Identity
WorldWideView is a **real-time geospatial intelligence engine** that visualizes live global data on an interactive 3D globe. Built with **Next.js 16**, **CesiumJS**, **React 19**, and **Zustand**, it renders everything from live aircraft and maritime vessels to conflict events, satellites, and environmental data — all through a modular plugin architecture.
WorldWideView is a **real-time geospatial intelligence engine** visualizing live global data on an interactive 3D globe. Built with **Next.js 16**, **CesiumJS**, **React 19**, and **Zustand**. Design/feature target: `www.worldmonitor.app` ([reference repo](https://github.com/koala73/worldmonitor)).
### Target Inspiration
Our primary design, feature-set, and operational layout goal is to mimic the structure and capabilities of `www.worldmonitor.app`.
- **Reference Codebase**: [GitHub - koala73/worldmonitor](https://github.com/koala73/worldmonitor)
---
## 2. Tech Stack
@@ -16,485 +13,152 @@ Our primary design, feature-set, and operational layout goal is to mimic the str
| Framework | Next.js 16 (App Router, `output: "standalone"`) |
| Language | TypeScript 5, strict mode |
| 3D Engine | CesiumJS + Resium (Google Photorealistic 3D Tiles) |
| State | Zustand (slice-based: globe, layers, timeline, UI, filters, data, config, favorites, geojson) |
| State | Zustand (9 slices: globe, layers, timeline, ui, filter, data, config, favorites, geojson) |
| Event Bus | Custom typed `DataBus` (pub/sub singleton) |
| Styling | Vanilla CSS — **no Tailwind** |
| Database | PostgreSQL via Prisma 7 (local and cloud) |
| Database | PostgreSQL via Prisma 7 |
| Auth | NextAuth v5 beta (Credentials provider, JWT sessions) |
| Package Manager | pnpm (monorepo with `pnpm-workspace.yaml`) |
| Testing | Vitest + jsdom + React Testing Library |
| Deployment | Docker multi-stage build → Coolify |
| Analytics | Vercel Analytics / custom `trackEvent` |
---
## 3. Directory Structure
## 3. Critical Invariants
> [!NOTE]
> See `.agents/rules/directory-structure.md` for the full project directory map and related repositories.
---
## 4. Architecture Patterns
> [!NOTE]
> See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for the human-facing architecture deep-dive — plugin system, state management, data pipeline, data engine & seeders, rendering pipeline, and edition system.
Key invariants agents MUST respect:
Agents MUST respect these at all times:
- **Plugin source of truth**: `@worldwideview/wwv-plugin-sdk` — never define plugin types locally.
- **All-Bundle Model**: every plugin (seeder, CDN-loaded, static-compiled, active-proxied) is dynamically imported via `loadPluginFromManifest` using `import(/* webpackIgnore: true */ entry)`. Legacy `StaticDataPlugin` / `DeclarativePlugin` runtimes are deprecated.
- **Agnostic frontend**: the renderer has no concept of a "unified" data engine. Each plugin **MUST declare its own `streamUrl`** in its manifest or config; do not assume one shared pipe.
- **Nine Zustand slices** under `src/core/state/`: `globe`, `layers`, `timeline`, `ui`, `filter`, `data`, `config`, `favorites`, `geojson`. Access via `useStore` in React, `useStore.getState()` elsewhere.
- **Primitive-based rendering** (not Cesium Entity API). Point/Billboard/Label/Polyline collections only. Never mix `size`/`outlineWidth`/`outlineColor` onto billboard entities — GPU silently clips.
- **Three editions** controlled by `NEXT_PUBLIC_WWV_EDITION` (`local` / `cloud` / `demo`); feature flags derived in `src/core/edition.ts`.
- **All-Bundle Model**: every plugin is dynamically imported via `loadPluginFromManifest` using `import(/* webpackIgnore: true */ entry)`. Legacy `StaticDataPlugin` / `DeclarativePlugin` runtimes are deprecated.
- **Agnostic frontend**: each plugin **MUST declare its own `streamUrl`**; do not assume one shared pipe.
- **Nine Zustand slices** under `src/core/state/`: access via `useStore` in React, `useStore.getState()` elsewhere.
- **Primitive-based rendering**: Point/Billboard/Label/Polyline collections only. Never mix `size`/`outlineWidth`/`outlineColor` onto billboard entities — GPU silently clips.
- **Three editions** via `NEXT_PUBLIC_WWV_EDITION` (`local` / `cloud` / `demo`); feature flags in `src/core/edition.ts`.
- **Nested git clones**: `local-plugins/` (community plugins) and `local-seeders/community/` + `local-seeders/private/` (seeders) are **independent git repos cloned inside this repo, gitignored from it**. Each has its own remote. Run `git pull` inside each before editing; commits/pushes there go to their own upstream — not to `worldwideview`. See `.agents/context/ecosystem-repositories.md`.
---
## 5. Critical Conventions
## 4. Conventions
### 5.1 File Size
- **File size**: ~300 lines max. Extract helpers, split components, use hooks.
- **Import aliases**: `@/*` → `./src/*`; `@worldwideview/wwv-plugin-sdk` → `./packages/wwv-plugin-sdk/src`
- **CSS**: Vanilla CSS only. Global: `src/app/globals.css`. Scoped: CSS Modules. HUD: `src/styles/hud-animations.css`.
- **Rendering entities**: Points use `type: "point"` + `size`/`outlineColor`/`outlineWidth`. Billboards use `type: "billboard"` + `iconUrl`/`iconScale`. NEVER mix.
- **Plugin registration**: Built-ins via `AppShell.tsx` → `PluginRegistry` → `PluginManager`. Marketplace plugins via `InstalledPluginsLoader`.
- **Workspace**: Use `"workspace:*"` (not `"*"`) for internal deps. New `packages/` plugins need `transpilePackages` in `next.config.ts`.
- **Temp files**: Save debugging scripts/outputs exclusively in `/local-scripts/` — never in root.
- **Cleanliness**: Remove dead code, unused imports, debug `console.log` before finalising. Never use `any` or `@ts-ignore`. Never create `.mdc` files.
**Guideline: ~300 lines per file.** If a file grows excessively large, consider modularizing it. Extract helpers, split components, or use hooks.
---
### 5.2 Import Aliases
- `@/*` → `./src/*`
- `@worldwideview/wwv-plugin-sdk` → `./packages/wwv-plugin-sdk/src`
- Plugin modules are dynamically imported and do not require individual `tsconfig.json` aliases
### 5.3 CSS Rules
- **Vanilla CSS only** — no Tailwind, no CSS-in-JS
- Global styles in `src/app/globals.css`
- Component-scoped styles use CSS Modules (`.module.css`) or co-located `.css` files
- HUD animations in `src/styles/hud-animations.css`
### 5.4 Rendering Entity Rules
When returning `CesiumEntityOptions` from `renderEntity()`:
- **Points**: Use `type: "point"` with `color`, `size`, `outlineColor`, `outlineWidth`
- **Billboards**: Use `type: "billboard"` with `iconUrl`, `color`, `iconScale`
- **NEVER mix**: Do not use `size`/`outlineWidth`/`outlineColor` on billboard entities — causes GPU clipping
### 5.5 Plugin Registration
Built-in plugins are instantiated in `AppShell.tsx` and registered via `PluginRegistry` → `PluginManager`. Marketplace-installed plugins are loaded from the database via `InstalledPluginsLoader`.
### 5.6 Workspace Rules
- Always run `pnpm install` from project root after creating new packages or linking local plugins
- Official plugin packages go in `packages/wwv-plugin-<name>/`
- Experimental/local plugin sandboxes go in `local-plugins/wwv-plugin-<name>/`
- Globs for `packages/*`, `packages/*/backend`, and `local-plugins/*` are mapped in `pnpm-workspace.yaml` and `tsconfig.json` paths
- Add new `packages/` plugins to `transpilePackages` in `next.config.ts` if required
### 5.7 AI Meta-Directives: Antigravity Standard
> [!NOTE]
> This repository is orchestrated via the **Antigravity open standard**. The entry point for the agent framework is `CLAUDE.md` / `AGENTS.md` at the project root.
## 5. AI Meta-Directives
> [!WARNING]
> - **Always** use standard `.md` file extensions for rules, skills, and workflows.
> - **Never** use proprietary `.mdc` extensions.
> - **Never** reference Cursor IDE rules; we use the open `.agents/` standard.
> - **MUST**: You MUST update Semantic Versioning numbering inside the relevant `package.json` file prior to executing any code commits, adhering strictly to the `[/commit]` workflow rules (`feat:` -> Minor, `fix/refactor/perf:` -> Patch).
> - **MUST Detail Commit Levels & Bumps**: On description changes or release notes, you must detail the level of commit (Major/Minor/Fix) for *each* individual change. If there are multiple accumulated changes, you MUST EITHER commit them individually and bump the version each time, OR commit them all at once and bump the version multiple times.
> - **MUST Explain Complex Concepts Simply**: Whenever providing a complicated technical explanation to the user, you MUST include a simple explanation below it. Use an analogy with reference to the correct terminology, comparing the concept to something from everyday life to ensure the user easily understands it.
> - **MUST Be Transparent & Narrate Actions (Gemini 3.1 Agents)**: If you are a Gemini 3.1 agent, you MUST always be fully transparent. **Whenever you do anything, you must explicitly say what you are going to do in a visible chat message to the user, and ONLY THEN do it.** This ensures the user can actually see and verify what you are doing in real time. Do not jump to destructive actions without stating your intent first. Keep this narration conversational and natural. Avoid stiff, robotic templates. Just explicitly drop a brief, casual note about what you are checking, fixing, or deleting *before* you run the tool.
> - **MUST Require Explicit Authorization**: Do NOT execute any state-changing tools (e.g., modifying files, writing code, running scripts, executing git commands) without first obtaining clear, explicit permission from the user to proceed with that specific action, **UNLESS** the action is a simple, straightforward, and safe operation (e.g., no deletion or dangerous operations). For safe operations, you may proceed without explicit permission. Otherwise, inform the user of your proposed plan and WAIT for their approval before acting. **Crucially, if you realize you have violated this rule by taking an unauthorized action, DO NOT automatically revert it. Reverting is itself an action that requires authorization. Just answer the question.**
> - **MUST Ask Clarifying Questions**: Never assume anything. If requirements are unclear, if you encounter an unexpected roadblock, or if the user's intent could be interpreted in multiple ways, you MUST pause and ask clarifying questions. Do NOT proceed until the developer has explicitly answered your question or requested you to proceed anyway.
### 5.8 Workspace Hygiene
Whenever agents generate temporary debugging scripts, test REST endpoints via `.mjs`, or dump traces/JSON outputs, they **MUST** save these exclusively inside `/local-scripts/`. The root directory is strictly for production configuration files.
### 5.9 Rule Adherence & Proactive Updates
> [!CAUTION]
> **Agents will trust the `.agents/rules/` files unconditionally.**
> Whenever you execute a major architectural shift or make a change that invalidates an existing rule, you **MUST** immediately update the corresponding rule file(s) and/or `AGENTS.md` to reflect the new reality. Do not leave rules untouched after a change; operating on outdated information will cause future agents to break the codebase.
> - Always use `.md` extensions. Never `.mdc`. Never reference Cursor IDE rules.
> - **MUST** bump semver in `package.json` before every commit via `/commit` workflow (`feat:` → Minor, `fix/refactor/perf:` → Patch).
> - **MUST** explain complex concepts simply — include an everyday-life analogy.
> - **MUST** require explicit user authorization before any state-changing action that isn't simple/safe.
> - **MUST** ask clarifying questions rather than assume when requirements are unclear.
> - **MUST** update `.agents/rules/` files immediately whenever an architectural shift invalidates them.
---
## 6. Environment & Configuration
> [!NOTE]
> See `.agents/rules/environment-config.md` for required environment variables and secrets.
See `.agents/context/environment-config.md` for required environment variables and secrets.
---
## 7. Development, Deployment & Testing
## 7. Development & Deployment
> [!NOTE]
> See `.agents/rules/deployment-and-testing.md` for development commands, Docker architecture, Vitest strategy, and CSP security headers.
```bash
pnpm dev # Frontend only (auto-runs prisma db push + copy-cesium)
pnpm dev:all # Frontend + data engine via Docker Compose
pnpm build # Production build
pnpm test # Vitest
pnpm db:reset # Wipe + re-migrate DB (destructive)
```
See `.agents/context/` → [deployment and testing details in `.agents/rules/deployment-and-testing.md`] for Docker architecture, Coolify rules, and CSP headers.
---
## 12. On-Demand Rules
## 8. On-Demand Rules (path-scoped, auto-load on file access)
Read the relevant rule file when working in that domain:
These load automatically when you read/edit files matching their paths:
| Rule | When to use | Path |
|---|---|---|
| `platform-architecture` | High-level platform goals, product vision, business model, and Edition System | `.agents/rules/platform-architecture.md` |
| `application-architecture` | Next.js frontend, Zustand state management, and CesiumJS integration | `.agents/rules/application-architecture.md` |
| `plugin-architecture` | Creating/modifying plugins, lifecycle, capability declarations, and seeders | `.agents/rules/plugin-architecture.md` |
| `marketplace-architecture` | Dynamic plugin installation, DB sync, and CDN loading | `.agents/rules/marketplace-architecture.md` |
| `cloud-auth-architecture` | Cloud edition, PostgreSQL RLS, multi-tenancy, and licensing | `.agents/rules/cloud-auth-architecture.md` |
| `server-management` | Server development and debugging using SSH and Coolify MCP | `.agents/rules/server-management.md` |
| `stakeholders-and-human-centered-design` | Human-centered design principles and stakeholder map | `.agents/rules/stakeholders-and-human-centered-design.md` |
| `directory-structure` | Project structure and related repositories | `.agents/rules/directory-structure.md` |
| `ecosystem-repositories` | WorldWideView ecosystem repositories and cross-repository workflows | `.agents/rules/ecosystem-repositories.md` |
| `deployment-and-testing` | Docker build patterns, testing strategies, security headers | `.agents/rules/deployment-and-testing.md` |
| `environment-config` | Secrets and environment variables | `.agents/rules/environment-config.md` |
| `monorepo-workflow` | pnpm commands, adding packages, workspace config | `.agents/rules/monorepo-workflow.md` |
| `data-engine-architecture` | Data Engine backend seeder loading, pnpm workspace dependencies | `.agents/rules/data-engine-architecture.md` |
| `cesium-rendering` | Globe rendering, entity types, primitives, LOD, culling | `.agents/rules/cesium-rendering.md` |
| `state-management` | Zustand slices, store access, plugin settings | `.agents/rules/state-management.md` |
| `database-migrations` | Prisma schema changes, migrations, PostgreSQL | `.agents/rules/database-migrations.md` |
| `continuous-improvement` | When to create/update rules, skills, or workflows | `.agents/rules/continuous-improvement.md` |
| `context-and-memory` | How to orient and maintain project context between sessions | `.agents/rules/context-and-memory.md` |
| `troubleshooting-and-debugging` | Resolving plugin latency, namespace collisions, or deployment issues | `.agents/rules/troubleshooting-and-debugging.md` |
| Rule | Triggers on |
|---|---|
| `.agents/rules/cesium-rendering.md` | `src/core/globe/**`, `src/plugins/**`, `packages/wwv-plugin-*/src/**` |
| `.agents/rules/state-management.md` | `src/core/state/**`, `src/components/**` |
| `.agents/rules/plugin-architecture.md` | `src/core/plugins/**`, `packages/wwv-plugin-*/src/**`, `local-plugins/**` |
| `.agents/rules/marketplace-architecture.md` | `src/lib/marketplace/**`, `src/app/api/marketplace/**` |
| `.agents/rules/cloud-auth-architecture.md` | `src/lib/auth*`, `src/app/api/auth/**`, `src/core/auth.ts` |
| `.agents/rules/database-migrations.md` | `prisma/**` |
| `.agents/rules/monorepo-workflow.md` | `packages/**`, `pnpm-workspace.yaml`, `local-plugins/**` |
| `.agents/rules/data-engine-architecture.md` | `packages/**`, `local-seeders/**`, `docker-compose.yml` |
| `.agents/rules/deployment-and-testing.md` | `Dockerfile`, `docker-compose.yml`, `.github/**`, `next.config.ts` |
| `.agents/rules/e2e-testing.md` | `tests/**`, `public/e2e-fixtures/**`, `playwright.config.ts` |
---
## 13. Slash Commands / Workflows
## 9. On-Demand Context (read explicitly when needed)
Invoke by name. Read the skill/workflow file and follow its steps.
See `.agents/context/INDEX.md` for the full navigation table. Key files:
| When to read | File |
|---|---|
| Product vision, business model, Edition system | `.agents/context/platform-architecture.md` |
| Finding files, repo layout | `.agents/context/directory-structure.md` |
| Routing fix to correct repository | `.agents/context/ecosystem-repositories.md` |
| Next.js, data pipeline, Redis, DB schema | `.agents/context/application-architecture.md` |
| Debugging plugin data, namespace collisions | `.agents/context/troubleshooting-and-debugging.md` |
| SSH, Coolify MCP | `.agents/context/server-management.md` |
| Coding principles, Definition of Done | `.agents/context/coding-principles.md` |
| `.env` variables and secrets | `.agents/context/environment-config.md` |
---
## 10. Slash Commands
| Command | Description | File |
|---|---|---|
| `/commit` | **Required before every commit** — bump semver + conventional commit | `.agents/skills/commit/SKILL.md` |
| `/remember` | Save a lesson, constraint, or fact into `.agents/` permanent memory | `.agents/skills/remember/SKILL.md` |
| `/remember` | Save a lesson or fact into permanent memory | `.agents/skills/remember/SKILL.md` |
| `/pr-review` | 6-role comprehensive pull request review | `.agents/skills/pr-review/SKILL.md` |
| `/update-context` | Sync `.agents/context/` with current project state | Global skill |
| `/local-dev` | Check, start, and troubleshoot local dev environment | `.agents/workflows/local-dev.md` |
| `/local-dev` | Check, start, troubleshoot local dev environment | `.agents/workflows/local-dev.md` |
| `/data-engine-cli` | Use the wwv-data-engine CLI wrapper | `.agents/workflows/data-engine-cli.md` |
| `/debugging-coolify` | Troubleshoot deployed apps on Coolify via MCP/SSH | `.agents/workflows/debugging-coolify.md` |
| `/five` | Five Whys root cause analysis | `.agents/workflows/five.md` |
| `/stitch-to-nextjs` | Generate UI with Stitch MCP, port into Next.js | `.agents/workflows/stitch-to-nextjs.md` |
| `/bing-news-hydration` | Hydrate event attributes with Bing RSS news | `.agents/workflows/bing-news-hydration.md` |
| `/generate-user-roadmap` | Generate updated user-facing roadmap | `.agents/workflows/generate-user-roadmap.md` |
---
## 14. Agent Skills Reference
Refer to these skill documents for specialized tasks:
### Project Skills (`.agents/skills/`)
## 11. Agent Skills Reference
| Skill | When to Use |
|---|---|
| `worldwideview-plugin-creation` | **Use when creating any plugin** — strict architectural checklist |
| `plugin-creation-master-guide.md` | Decision matrix for choosing plugin architecture |
| `osm-static-plugin-creation.md` | Creating static GeoJSON plugins from OpenStreetMap |
| `microservice-plugin-creation.md` | Legacy guide for standalone Fastify microservices |
| `database-operations.md` | Prisma schema changes, migrations, database queries |
| `database-incident-recovery-procedures.md` | Authoritative protocol for safely restoring a broken production database |
| `index-documentation.md` | Maintaining project documentation index |
| `context7` | Fetch up-to-date library docs via Context7 API |
| `cesium-context7` | CesiumJS-specific documentation lookup |
| `database-incident-recovery-procedures.md` | Safely restoring a broken production database |
### Global Skills
52 skills are available across all projects. See `.agents/global-skills-index.md` for the full list and invocation paths.
52 global skills available. See `.agents/global-skills-index.md`.
---
## 15. Pull Request & Commit Guidelines
## 12. Pull Request & Commit Guidelines
- **Commit Format**: We strictly enforce Conventional Commits (`feat:`, `fix:`, `refactor:`, `perf:`).
- **Workflow**: You **MUST** use the `/commit` workflow before every git commit to ensure proper semantic versioning bumps.
- **Required Checks**: Ensure `pnpm test` and `pnpm build` complete successfully before proposing a merge.
- **Review Process**: Use `/pr-review` to conduct a comprehensive multi-role review on any pull request.
- **Commit format**: Conventional Commits (`feat:`, `fix:`, `refactor:`, `perf:`). Use `/commit` workflow.
- **Required checks**: `pnpm test` and `pnpm build` must pass before merge.
- **Review**: Use `/pr-review` for comprehensive multi-role review.
- **Worktrees**: Use `git-wt switch --create <branch>` and `git-wt remove` (never `rm -rf` a worktree — orphans the PostgreSQL Docker volume).
---
## 16. Debugging and Troubleshooting
- **Prisma & PostgreSQL Sync Issues**: If the local database state falls out of sync with Prisma schema, do not manually drop tables. Instead, run `pnpm db:reset` to cleanly wipe and re-apply all migrations.
- **Cesium Entity Clipping**: If `billboard` entities are clipping or failing to render correctly, verify that you are not mixing `point` primitive properties (like `size` or `outlineWidth`) into the `billboard` options.
- **Build Exhaustion (Docker)**: Multi-stage pnpm builds generate massive cache layers. If a Coolify deployment fails silently or PostgreSQL crashes abruptly, check host disk space and run `docker builder prune -a -f`.
- **Next.js Typechecking Failures**: If backend-only `scripts/` fail during Next.js build, verify that the scripts directory is properly listed in the `exclude` array of `tsconfig.json`.
## 17. Coding Principles
You are about to write or modify code. **Read this file first.** It exists to prevent the failure mode where every fix makes the next harder — the kind of accumulated coupling and band-aid-on-band-aid that turns a clean codebase into a fragile one over months. This file is the constitution that keeps that from happening.
### The hard rule
## Hard Rule
> **Always interface-based, extensible, composable, modular. Never band-aids on band-aids.**
If you find yourself adding a flag, an `if`-check, or a special case to make a problem go away, you are probably band-aiding. Stop and find the seam where the abstraction broke. Fix it there.
### Core principles
#### 1. Interfaces before implementations
Define the contract first. The contract is what other components depend on; the implementation is replaceable. Public functions accept the interface, not the concrete type.
```ts
// Yes
function attachSource(source: DataSource) { ... }
// No
function attachPostgresSource(pg: PostgresInstance) { ... }
```
When a pattern repeats across two modules, refactor it into an interface immediately. Don't wait for the third copy to "earn" the abstraction — by then the costs have compounded.
#### 2. Loose coupling through explicit contracts
Components communicate through well-defined contracts: typed function signatures, events on a bus, message queues, or capability registries. Pick the form that fits, but pick *one* and make it the only path.
Components do **not** communicate through:
- Direct imports of each other's internal modules
- Shared mutable state outside the contract
- Synchronous request/response when async fits the actual data flow
- "Just this once" backdoors that bypass the contract
If you're tempted to write `import { internalThing } from '../other-module/internal'`, you are doing it wrong. If the symbol you need isn't part of the other module's public contract, either it should be — propose adding it — or you're solving the problem in the wrong place.
#### 3. Domain boundaries are sacred
Each domain (module, service, bounded context, plugin) owns its own state and its own surface. Cross-domain access is **explicit, gated, and audited** — never a shortcut.
Forbidden:
- Reading another domain's storage directly
- Calling another domain's private functions
- Hardcoded `if (tenant === 'X')` or `if (mode === 'special')` branches in shared code
Allowed:
- A documented capability or API on the other domain that you call
- A configuration field on the relevant record that drives the strategy
- Cross-domain access gated by explicit policy (default **deny**, opt-in per case), with the access itself logged
The rule of thumb: if removing a domain would require code archaeology in three other domains to figure out what depends on its internals, the boundary has already been violated. Treat that as a bug.
#### 4. Don't bend your architecture to fit a dependency
When you bring in a library or framework, use it where it fits cleanly; build natively where it doesn't.
Let the dependency handle (don't reinvent):
- The thing it's specifically good at, in the shape it expects
Your code owns (the dependency can't represent):
- Anything that's specifically your domain
- Anything that requires identity, lifecycle, or invariants the dependency doesn't model
- The integration seams between the dependency and your domain
If you find yourself bending your design to fit a library convenience, stop. Either the library has a hook for what you need, or you build it natively. Don't compromise architecture for the convenience of pulling in someone else's defaults.
#### 5. Data lifetimes are deliberate, not incidental
Different data has different lifetimes. Get this wrong and you'll either retain garbage forever or delete things users care about. Common categories:
- **Operational telemetry** (logs, metrics, internal events) — short rolling window. Useful for crash recovery and immediate investigation; not useful enough to keep forever.
- **User content** (messages, documents, records the user created or that represent their lived experience with the system) — **permanent unless the user explicitly deletes it.** Never pruned by a sweep, never deleted by a retention policy authored for telemetry.
- **Resumable session state** — bounded window appropriate to the use case; archived (state flag flipped) rather than deleted.
Name each category in your codebase. Write the retention policy for each one separately. Never propose a sweep that crosses categories — "delete events older than 30 days" is fine if events are telemetry; the same policy applied to user content is data loss masquerading as housekeeping. If storage pressure ever becomes real for content, introduce cold-tiered storage, not deletion.
#### 6. Optional features must be truly removable
Every optional feature (plugin, extension, integration, side module) declares its surface explicitly and communicates through the same contracts as everything else. No optional feature reaches into another's internals.
Test for whether something is core vs. optional: **if removing this feature should break the system, it's core. Otherwise it's optional, and it should be removable without leaving residue.**
If you can't delete an "optional" module's directory and have the system keep running, it isn't optional. Either fix the coupling or move it into core honestly.
#### 7. Put code where it belongs
When a feature spans layers (a trigger lives here, the action lives there, the data model lives somewhere else), resist the urge to fold it all into one place "for convenience." The feature lives in three places because each piece belongs to a different domain. Routing across them is the job — not collapsing them.
The general rule: if a feature needs to read/write durable state, it's a tool/extension on top of the layer that owns that state; if it's "the system should know about this concept," it's a configuration or contract change. Often it's both, with a thin handler that wires them together. That thin handler is fine — it's load-bearing — but it doesn't justify smashing the layers together.
### Antipatterns to refuse
These are common failure modes that, repeated over time, calcify a codebase. Refuse them when you catch yourself reaching for them.
| Antipattern | Better answer |
|---|---|
| Adding a new flag to an existing function instead of extracting a strategy | Extract the strategy interface; existing call sites pass the default strategy |
| Patching around a bug at the call site | Find the seam where the abstraction broke; fix it there |
| Direct database access from a feature module | Go through the layer that owns the data; if no surface exists, propose one |
| Hardcoded paths, model names, IDs, environment specifics in business logic | Configuration; or look up via registry |
| `if (tenant === 'X') { ... }` in shared code | Strategy on the record, or a per-tenant config field |
| Inheriting one layer's concerns into another (or vice versa) | Cross the boundary through the documented contract |
| "It works for now" — inline coupling between layers | If the layers are real, draw the contract; if they're not, merge them honestly |
| Adding a defensive `try/catch` to swallow an error | Find why the error happens; gate cleanly at the right boundary |
| Writing a comment to explain what code does | Rename and refactor so the code reads itself |
| Copying a snippet from one module into another "to avoid the dependency" | Either it belongs in a shared module, or the duplicate will drift — pick |
### Debugging discipline
When something breaks:
1. **Reproduce it.** Don't trust intuition about what's failing.
2. **Find the seam.** Where did the abstraction break? Which contract is being violated, and by whom?
3. **Fix at the seam.** Not at the call site. Not at the consumer. At the boundary that's wrong.
4. **If you can't find the seam, surface it.** Tell the user explicitly: "this is a design gap; the right fix needs an architectural decision, not a patch." Do not paper over.
If a fix doesn't work, **don't add another guard on top.** Find the real seam and gate cleanly. Stacked defensive layers are how clean codebases become unmaintainable.
### Lifecycle invariants and state-changing operations
Most recurring bugs cluster around the same shape: an operation crosses lifecycle state without checking, and the result is silent corruption, lost data, or orphan rows. The recipe that prevents recurrences:
1. **Name the invariant.** "Closed sessions don't accept work." "Scoped grants need a scope id." "Subscriptions past retention can't deliver swept events." Write it down before reaching for code.
2. **Throw a typed error at the seam.** The single function that crosses the invariant refuses the bad case with a typed, named error — not a boolean return, not a silent skip, not a generic `Error`.
3. **Pre-check at every call site that mutates *before* reaching the seam.** If the seam check fires after `store.append()`, you've left an orphan row. The pre-check is hygiene; the seam check is correctness — both are required. When the same pre-check appears at a second call site, extract a helper.
4. **Validate, then mutate — never the reverse.** Within a single function, resolve all inputs that could fail (key derivation, lookups, permission checks) before touching durable state. A validation throw should leave nothing half-applied. If you can't separate validation from mutation, the function is doing two jobs.
5. **A documented unsafe API is still unsafe.** A docstring that says "don't use this when X" is a foot-gun, not a warranty disclaimer. Either make the API safe in case X (drain, throw, refuse) or remove it.
If you're tempted to skip step 3 because "the seam check covers it," check whether the call site writes anything before reaching the seam. If yes, you need the pre-check.
### Definition of done — checklist for adding or updating a function
Every code change must satisfy the relevant checklist before merge. Refusal triggers below mean **stop and surface to the user** — do not ship a half-met checklist.
#### Adding a new function
All must be true:
- [ ] Belongs in this module (not somewhere else)
- [ ] Fulfills an existing interface, OR a new interface is added with it (and reasons documented in a spec doc / ADR)
- [ ] Has a doc-comment header (purpose + params + returns + thrown errors if relevant)
- [ ] Has at least one test that targets the contract (not implementation details)
- [ ] Test passes locally and in CI
- [ ] Doesn't introduce direct cross-module imports of internals — communication through public contracts only
- [ ] Doesn't touch storage owned by another component — route through that component's surface
- [ ] Matches the spec doc's contract for this component (if no spec covers it, update the spec first)
- [ ] Name is clear enough that the header comment is a summary, not an explanation
#### Updating an existing function
All must be true:
- [ ] Reason for change is clear (bug fix / new requirement / refactor / spec change)
- [ ] Doc-comment still accurate after change (update if not)
- [ ] Existing tests cover the new behavior (add new tests if not)
- [ ] Build is green (all tests passing)
- [ ] If this changes the contract: spec doc updated AND an ADR added if the change is load-bearing
- [ ] Fix is at the seam, not at the symptom (no band-aid patches)
- [ ] No new flags or special cases added in lieu of strategy extraction
#### Refusal triggers (any one means stop)
- Adding `if (tenant === 'X')` to shared code → use per-tenant config instead
- Adding a flag to make a problem go away → extract the abstraction the flag implies
- Direct DB or filesystem access from a feature module → route through the owning component
- `try`/`catch` swallowing an error → find why the error happens, gate cleanly
- Test omitted because "it's trivial" → no
- Function-header skipped because "it's obvious" → no
- "Just add another guard here" when a guard didn't work → find the real seam
- Patch shipped to "unblock the build" without root-cause fix → no, surface the design gap
If any criterion isn't met, stop. Surface to the user as a design question. Don't paper over.
### Code-style minimums
#### Comments
**Every function gets a doc-comment header** in your language's idiomatic format (TSDoc, JSDoc, Python docstrings, Rust `///`, Javadoc, etc.). Not optional. Long-running codebases get re-read after gaps; function-header docs are how re-onboarding stays cheap.
Required content:
- One-line summary at a contract level (what the function guarantees, not what the code does line-by-line)
- Each parameter and what it represents
- Return value
- Thrown errors / failure conditions if relevant
- Example usage when the call shape is non-obvious
Apply this to public, internal, exported, and private functions. Apply to methods on classes too.
**Inline comments inside function bodies** still follow the standard rule: only for non-obvious WHY (hidden constraints, workarounds, surprising behavior, references to bug reports). Don't comment WHAT the code does — well-named identifiers and the function-header doc cover that.
#### Tests
**Every function ships with at least one test. Tests passing is a build requirement.**
- Tests target the contract (function signature + documented behavior + invariants), not implementation details. Refactoring should not break tests; behavior changes should.
- A new implementation of an interface must pass the same contract test suite (write tests against the interface, not against a specific implementation).
- Test names describe the case being verified, not the function: `it('refuses resume after archival window expires')` not `it('resumeSession')`.
- For storage code: tests use a real (temporary) instance of the storage layer, created and torn down per test or per suite. Don't mock the storage engine — you're testing the wrong thing if you do.
- For integration code that depends on an external library's hooks: contract tests against that library's surface so a dependency bump catches behavior changes.
- Build pipeline (CI or local) runs all tests; non-zero exit = build fails. There is no "build succeeded but tests failed" state.
#### Logging
**Every new feature with a non-happy-path ships with a log line at each rare branch.** Use a structured logger with module-scoped child loggers. The log file is where "something's not working" turns into "what actually happened."
Level guidance:
- **info** — expected-but-noteworthy edges: a UI race that hits a closed resource, a permission bypass for a privileged principal. Correct behavior an operator might want to confirm fired.
- **warn** — should be rare and worth investigating: race past a pre-check, lost events past retention, programmer/operator errors (calling a method after stop, a key that should have resolved didn't).
- **error** — actual failures the operator must respond to: handler crashes, dispatch loop crashes, integration failures.
- **debug** — high-volume internal state changes; always pair with a `module` field so it can be filtered.
Pass structured fields, not interpolated strings — fields are what makes a log queryable:
```ts
// Yes
logger.warn({ session_id, state }, "rejected append against non-live session");
// No
logger.warn(`session ${session_id} (${state}) rejected append`);
```
If you add a new typed error or a new event topic, also log it. The typed error tells the immediate caller; the event tells subscribers; the log tells the operator. The three are independent surfaces for the same fact, and all three are needed — events don't show up in the log, logs don't show up on the bus, errors don't survive the catch.
#### Structure
- Keep modules small. If a file is over ~500 lines, ask whether it has more than one responsibility.
- Prefer explicit composition (caller wires things together) over implicit dependency-injection magic.
- One public concept per module. If a file exports both `Foo` and `unrelatedHelperUsedNowhereElse`, the helper has the wrong home.
#### Dependencies
**Always check the registry for the current version before adding or updating a dependency.** Don't pick versions from memory.
The simple test: did you actually run `<package-manager> view <pkg> version` (or its equivalent) before pinning? If not, you're guessing — and the guess is almost always behind, sometimes by a major version with active deprecation.
Why this matters:
- **Long-running codebases.** Stale dependency anchors compound. A v3 you picked because that's what was on your mind will still be v3 a year later, and the v4 migration gets harder every month it waits.
- **Active deprecations are real.** Common libraries are mid-deprecation cycles for older majors. Pinning to a soon-deprecated version on day one means the eventual upgrade is forced rather than chosen.
- **"Latest" doesn't mean "use without thinking."** Read the changelog for breaking changes; decide whether to take them now or pin behind explicitly. But the *default* posture should be: pick latest, and justify in writing if pinning behind.
If you must pin behind latest, leave a comment in the manifest (or a CHANGELOG note) explaining why — a known incompatibility with another dep, an in-progress migration, a tested-and-rejected major bump. *"I haven't checked"* is not an acceptable reason.
Apply to: adding a new dep, updating an existing one, scaffolding a new package. *Not* required for routine no-op installs that don't touch versions.
#### Doc-vs-spec discipline
Specification documents (the deeper ones that describe how a component must behave) are **functional specifications**, not narrative essays. The discipline:
- **Concrete data structures** — type definitions, schemas, DDL, JSON examples
- **Concrete function signatures** — input/output types, error types
- **Concrete event/message payloads** — name + schema + example
- **Concrete API signatures** — input/output schemas, side effects, permission requirements
- **Flow pseudocode** — when orchestration is non-obvious
- **Invariants** — explicitly listed: "after X happens, Y is always true"
- **Failure modes** — enumerated: "if X fails, the system does Y"
- **Acceptance tests** — statements another engineer can implement: "given input X, output Y"
Avoid in spec docs:
- Long prose explaining motivation (that goes in ADRs / decision records)
- Discussion of alternatives considered (also ADRs)
- Background context (README handles this)
- Marketing/aspirational tone
Spec docs are for implementation. Read one and you should know what to build.
### When in doubt
Ask the human user. Specifically:
- "I don't see a clean seam for this; the options I see are A or B. Which fits the architecture you have in mind?"
- "This requires a new contract / capability / interface. Should I draft it before implementing?"
- "I'm about to add a special case here; that smells like a missing abstraction. Do you want me to extract the abstraction or note the smell and defer?"
Don't ship band-aids and ask forgiveness later. The user has explicitly said: prevent that pattern.
Read `.agents/context/coding-principles.md` before any non-trivial code change.
Symlink
+1
View File
@@ -0,0 +1 @@
AGENTS.md
+51
View File
@@ -0,0 +1,51 @@
# Design System: WorldWideView
**Project ID:** local-worldwideview-app
## 1. Visual Theme & Atmosphere
The WorldWideView platform employs an **"Airy, High-Tech, and Geospatial-Focused"** aesthetic. The design philosophy centers around "Glassmorphism Pro Max"—keeping the user's primary focus on the rich, underlying 3D globe visualization while providing highly legible, structured, and floating Heads-Up Display (HUD) elements.
The mood is utilitarian yet premium, heavily relying on dense background blurring, delicate inner borders, and vibrant glowing accent states to establish depth, interactivity, and a sense of "live" intelligence.
## 2. Color Palette & Roles
WorldWideView operates using a multi-theme architecture (Default Dark, Pure Black, Legacy OLED, and Light). The core semantic roles for the **Default Dark Theme** are outlined below:
* **Obsidian Canvas** (`#171717`) - `--bg-primary`: The foundational layer behind the application shell and globe.
* **Translucent Slate Glass** (`rgba(38, 38, 38, 0.85)`) - `--bg-glass`: The primary background for floating HUD panels, sidebars, and dialogs. Provides a frosted glass effect that allows the map to bleed through.
* **Frost White** (`#ffffff`) - `--text-primary`: Primary typography for headers, active states, and high-emphasis data.
* **Muted Ash** (`#a3a3a3`) - `--text-muted`: Secondary typography used for labels, subtitles, and inactive states to establish visual hierarchy.
* **Gossamer Edge** (`rgba(255, 255, 255, 0.15)`) - `--border-subtle`: Defines the subtle, crisp edges of glass panels, dividers, and default button states.
* **Starlight Cyan** (`#ffffff` / *Legacy:* `#22d3ee`) - `--accent-cyan`: Used for primary actions, active tabs, timeline scrubbers, and play buttons. Often paired with a diffuse glow (`0 0 24px rgba(255, 255, 255, 0.15)`) for emphasis and technical flair.
* **Telemetry Green** (`#22c55e`) - `--accent-green`: Used for live status indicators, pulsing connectivity badges, and positive data deltas.
* **Warning Amber** (`#f59e0b`) - `--accent-amber`: Used for warnings, "Alpha" badges, or unavailable data states.
## 3. Typography Rules
The system uses a dual-font strategy to balance clean user interfaces with technical precision.
* **UI Font (Inter):** Used for all general interface elements, headers, and buttons.
* Headers and brand elements use a bold `600/700` weight with tight letter-spacing (`-0.02em`) for a sleek, modern feel.
* Section titles, keys, and subtitles use a very small size (`11px`), a medium weight, uppercase transformation, and wide letter-spacing (`0.05em` to `0.1em`) to mimic aerospace or military HUD interfaces.
* **Monospace Font (JetBrains Mono):** Used strictly for data values, coordinates, telemetry, and exact timestamps (`11px` to `12px`). Ensures tabular alignment, clear distinction between similar characters (e.g., 0 and O), and technical precision.
## 4. Component Stylings
* **Glass Panels (Cards/Containers):**
* **Shape:** Generously rounded corners (`16px` / `var(--radius-lg)`).
* **Surface:** Dense frosted background blur (`24px`).
* **Depth:** Elevated significantly above the map using a dual-layered drop shadow (`0 8px 32px rgba(0, 0, 0, 0.4)` combined with `0 2px 8px rgba(0, 0, 0, 0.2)`). Features a crisp inner border (`inset 0 1px 1px rgba(255, 255, 255, 0.1)`) to simulate physical glass thickness.
* **Buttons:**
* **Shape:** Subtly rounded corners (`12px` / `var(--radius-md)`) with a standard height of `36px`.
* **Surface:** Translucent glass background with a subtle border.
* **Behavior:** Smooth, spring-based transitions. On hover, the background brightens, the element scales up/translates slightly (`translateY(-1px)`), and primary buttons emit a vivid neon glow. On click (active), buttons physically depress (`scale(0.97)`).
* **Tabs & Selectors:**
* **Shape:** Softly rounded (`8px` / `var(--radius-sm)`).
* **Surface:** Clean, flat, and transparent by default.
* **Behavior:** Transitions to a soft glass hover or active state. Active tabs use a highlighted background (`rgba(var(--accent-rgb), 0.1)`) and bright cyan text with increased font weight.
* **Timeline Scrubber:**
* **Track:** Slender `4px` tall track with softly rounded edges.
* **Handle:** A circular `14px` handle that emits a cyan glow, expanding dynamically on hover (`scale(1.2)`).
## 5. Layout Principles
* **Absolute Overlays:** The UI does not push the map; instead, it floats over a full-screen, `z-index: 0` globe canvas. Elements are anchored to the edges (Header to top, Sidebar to sides, Timeline to bottom).
* **Strict 4/8dp Rhythm:** All spacing adheres to an 8-point grid base (`4px`, `8px`, `12px`, `16px`, `24px`, `32px`, `48px`).
* **Dynamic Containment:** Sidebars and bottom panels push each other dynamically to prevent overlap and optimize map visibility (e.g., when the bottom panel opens, sidebars slide upward).
* **Whitespace & Density:** Panels use a dense `16px` padding globally to maximize the amount of intelligence data visible while maintaining clear breathing room between individual data rows. Scrollable areas hide scrollbars until hovered to maintain the clean glass aesthetic.
+2 -2
View File
@@ -87,13 +87,13 @@ WorldWideView uses a multi-stage Dockerfile designed for standalone output. To d
**Mac/Linux:**
```bash
mkdir worldwideview && cd worldwideview
curl -fsSL https://raw.githubusercontent.com/silvertakana/worldwideview/main/setup.sh | bash
curl -fsSL https://raw.githubusercontent.com/silvertakana/worldwideview/main/self-host/setup.sh | bash
```
**Windows (PowerShell):**
```powershell
mkdir worldwideview; cd worldwideview
Invoke-WebRequest -Uri https://raw.githubusercontent.com/silvertakana/worldwideview/main/setup.ps1 -UseBasicParsing | Invoke-Expression
Invoke-WebRequest -Uri https://raw.githubusercontent.com/silvertakana/worldwideview/main/self-host/setup.ps1 -UseBasicParsing | Invoke-Expression
```
> [!NOTE]
-36
View File
@@ -1,36 +0,0 @@
services:
wwv:
build:
context: .
args:
- NEXT_PUBLIC_WWV_EDITION=${NEXT_PUBLIC_WWV_EDITION:-local}
- NEXT_PUBLIC_WWV_PLUGIN_DATA_ENGINE_URL=${NEXT_PUBLIC_WWV_PLUGIN_DATA_ENGINE_URL:-}
- NEXT_PUBLIC_SUPABASE_URL=${NEXT_PUBLIC_SUPABASE_URL:-}
- NEXT_PUBLIC_SUPABASE_ANON_KEY=${NEXT_PUBLIC_SUPABASE_ANON_KEY:-}
- NEXT_PUBLIC_GOOGLE_MAPS_API_KEY=${NEXT_PUBLIC_GOOGLE_MAPS_API_KEY:-}
- NEXT_PUBLIC_BING_MAPS_KEY=${NEXT_PUBLIC_BING_MAPS_KEY:-}
- NEXT_PUBLIC_CESIUM_ION_TOKEN=${NEXT_PUBLIC_CESIUM_ION_TOKEN:-}
- NEXT_PUBLIC_ADSENSE_CLIENT_ID=${NEXT_PUBLIC_ADSENSE_CLIENT_ID:-}
pull_policy: build
ports:
- "3000:3000"
db:
profiles:
- "${WWV_SKIP_LOCAL_DB:-}"
ports:
- "5432:5432"
wwv-data-engine:
command: >
sh -c "
echo 'Syncing workspace dependencies...' &&
cd $$SEEDERS_DIR/community && CI=true pnpm install --ignore-scripts && pnpm -r rebuild better-sqlite3 esbuild &&
if [ -d \"$$SEEDERS_DIR/private\" ]; then cd $$SEEDERS_DIR/private && CI=true pnpm install --ignore-scripts && pnpm -r rebuild better-sqlite3 esbuild; fi &&
cd /app && node dist/server.js"
ports:
- "5000:5000"
networks:
coolify:
external: true
+17 -8
View File
@@ -1,6 +1,5 @@
services:
wwv:
image: ghcr.io/silvertakana/worldwideview:latest
build:
context: .
args:
@@ -16,11 +15,13 @@ services:
- "3000:3000"
volumes:
- wwv-data:/app/data
- ./public/plugins-local:/app/public/plugins-local:ro
env_file:
- path: .env
required: false
environment:
- "AUTH_SECRET=${AUTH_SECRET:-default_secret_for_local_dev}"
- "ENCRYPTION_MASTER_KEY=${ENCRYPTION_MASTER_KEY:-local_dev_encryption_key_change_in_prod}"
- "WWV_DEMO_ADMIN_SECRET=${WWV_DEMO_ADMIN_SECRET:-}"
- "WWV_BRIDGE_TOKEN=${WWV_BRIDGE_TOKEN:-}"
- "DATABASE_URL=postgresql://postgres:postgres@db:5432/worldwideview?schema=public"
@@ -42,8 +43,8 @@ services:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: worldwideview
expose:
- "5432"
ports:
- "${WWV_DB_PORT:-5432}:5432"
volumes:
- postgres-data:/var/lib/postgresql/data
healthcheck:
@@ -55,20 +56,24 @@ services:
wwv-data-engine:
image: ghcr.io/silvertakana/wwv-data-engine:latest
command: >
sh -c "
echo 'Syncing workspace dependencies...' &&
if [ -d \"$$SEEDERS_DIR/community\" ]; then cd $$SEEDERS_DIR/community && CI=true pnpm install --ignore-scripts && pnpm -r rebuild better-sqlite3 esbuild; fi &&
if [ -d \"$$SEEDERS_DIR/private\" ]; then cd $$SEEDERS_DIR/private && CI=true pnpm install --ignore-scripts && pnpm -r rebuild better-sqlite3 esbuild; fi &&
cd /app && node dist/server.js"
ports:
- "5000:5000"
volumes:
- ./local-seeders:/app/seeders
environment:
- PORT=5000
- REDIS_URL=redis://wwv-redis:6379
- SEEDERS_DIR=/mnt/host/c/dev/worldwideview/local-seeders
- SKIP_SEEDER_DOWNLOAD=true
- SEEDERS_DIR=/app/seeders
- DB_PATH=/app/data/engine.db
depends_on:
wwv-redis:
condition: service_healthy
volumes:
- ../wwv-seeders:/mnt/host/c/dev/worldwideview/local-seeders/community
- ../wwv-seeders-private:/mnt/host/c/dev/worldwideview/local-seeders/private
restart: unless-stopped
profiles:
- engine
@@ -83,6 +88,10 @@ services:
profiles:
- engine
networks:
coolify:
external: true
volumes:
wwv-data:
postgres-data:
+54 -2
View File
@@ -29,8 +29,60 @@ if [ -z "$DATABASE_URL" ]; then
fi
echo "[entrypoint] Running database migrations..."
prisma migrate deploy
echo "[entrypoint] Migrations complete."
set +e
MIGRATE_OUT=$(prisma migrate deploy 2>&1)
MIGRATE_CODE=$?
set -e
if [ $MIGRATE_CODE -eq 0 ]; then
echo "[entrypoint] Migrations complete."
elif echo "$MIGRATE_OUT" | grep -q "P3005"; then
# P3005: schema exists but _prisma_migrations has no history.
# Happens when 'prisma db push' (pnpm dev) ran before 'prisma migrate deploy' (docker).
# Safe to auto-baseline only if there is zero schema drift.
echo "[entrypoint] P3005: schema exists without migration history."
echo "[entrypoint] Verifying schema matches migrations before auto-baselining..."
set +e
DIFF_OUT=$(prisma migrate diff \
--from-url "$DATABASE_URL" \
--to-schema-datamodel prisma/schema.prisma \
--script 2>&1)
DIFF_CODE=$?
set -e
if [ $DIFF_CODE -ne 0 ]; then
echo "[entrypoint] Could not verify schema state — aborting." >&2
echo "$DIFF_OUT" >&2
exit 1
fi
# Positive-match for DDL keywords: any real drift emits at least one of these.
# Checking for presence is safer than checking for absence of content.
if echo "$DIFF_OUT" | grep -qiE "^(CREATE|ALTER|DROP|INSERT|UPDATE|DELETE|TRUNCATE)"; then
echo "[entrypoint] Schema drift detected — cannot auto-baseline." >&2
echo "[entrypoint] Run 'pnpm db:reset' to wipe and re-migrate, or resolve drift manually." >&2
echo "$MIGRATE_OUT" >&2
exit 1
fi
echo "[entrypoint] No drift detected. Baselining existing schema..."
for migration_dir in prisma/migrations/*/; do
if [ -f "${migration_dir}migration.sql" ]; then
migration_name=$(basename "$migration_dir")
echo "[entrypoint] Marking as applied: $migration_name"
set +e
prisma migrate resolve --applied "$migration_name"
RESOLVE_CODE=$?
set -e
if [ $RESOLVE_CODE -ne 0 ]; then
echo "[entrypoint] Failed to baseline $migration_name — aborting." >&2
exit 1
fi
fi
done
prisma migrate deploy
echo "[entrypoint] Migrations complete."
else
echo "$MIGRATE_OUT" >&2
exit $MIGRATE_CODE
fi
# Generate self-signed SSL certificates for local HTTPS bridging if they don't exist
if [ ! -f "./data/localhost.crt" ] || [ ! -f "./data/localhost.key" ]; then
@@ -0,0 +1,154 @@
# ADR-001: Decentralized Plugin Authentication & SSRF Mitigation
## Status
Proposed *(Pending Review)*
## Date
2026-05-14
## Related
- **Dependencies:** `docs/ARCHITECTURE.md` (Agnostic Client principle), `prisma/schema.prisma` (existing Local App Postgres integration).
---
## Context
The WorldWideView ecosystem faces four interlocking security and scalability problems:
1. **The SSRF Liability:** The Local App uses generic proxy routes (`/api/camera/proxy`) to bypass browser CORS restrictions. These lack robust SSRF hardening (no DNS rebinding defense, no private IP blocking), exposing the host LAN.
2. **The CORS Barrier:** Browser-enforced Cross-Origin Resource Sharing (CORS) prevents the agnostic frontend from connecting directly to independent Data Engines via standard HTTP.
3. **The Centralization Bottleneck:** If every Data Engine queries the Marketplace database to verify a user's subscription on every frame, the Marketplace becomes a catastrophic bottleneck.
4. **The Credential Interception Threat:** Passing credentials via simple URL redirects during setup exposes them to browser history theft and MITM attacks.
To resolve these, we are adopting an umbrella "Token Exchange" architecture, broken down into four specific decisions.
---
## Decisions
### ADR-001A: Direct WebSocket Data Plane
We mandate that heavy data streams utilize WebSockets (`wss://`). Browsers do not apply CORS preflight to WebSockets, allowing direct connections.
* **Security Constraint:** Because CORS is bypassed, Data Engines MUST explicitly validate the `Origin`, `Host`, TLS state, and expected tenant/plugin metadata upon connection.
### ADR-001B: Marketplace-Issued JWT Tickets
To enforce monetization without centralized database queries, we implement a stateless Token Exchange via Asymmetric Cryptography (Ed25519).
* **The JWT Contract:** The token MUST contain the following claims:
* `iss`: `https://marketplace.worldwideview.dev`
* `sub`: The user's unique ID.
* `aud`: **Crucial.** The specific Data Engine or plugin ID. Tokens MUST be audience-bound to prevent a valid token for Engine A being replayed against Engine B.
* `exp`: Expiration (exactly 5 minutes from issuance).
* `iat`: Issued at.
* `jti`: Unique token identifier.
* `tier`: Subscription level (e.g., "pro").
* `scope`: Specific entitlements.
### ADR-001C: PKCE Bootstrap (The API Key)
The Local App acts as the credential holder. It uses the OAuth 2.0 Authorization Code flow with PKCE to securely acquire a **long-lived refresh credential** (The API Key).
* **API Key Lifetime:** The API Key does not expire automatically but is bound to the specific device/installation. It rotates manually via the Marketplace UI or automatically upon suspicion of compromise.
### ADR-001D: SSRF Proxy Hardening
For legacy plugins (e.g., Webcams) that *cannot* use WebSockets, the local proxy must be hardened.
* **Constraints:** We will remove generic proxying where possible. Where proxying remains, we enforce:
1. Exact HTTPS origin allowlists.
2. Blocking of private / link-local / loopback IPs *after* DNS resolution.
3. Re-resolving or pinning DNS to defend against DNS rebinding.
4. Rejecting redirects to non-allowlisted hosts.
5. Strict payload size and time limits.
---
## Threat Model & Non-Goals
**In Scope:**
* Defending against external attackers on the user's LAN (via SSRF hardening).
* Preventing interception of the long-lived API key during setup (via PKCE).
* Preventing URL/Header credential leakage (via First-Message Auth).
**Non-Goals (Accepted Residual Risks):**
* Defending against compromised Local App hosts (root OS access).
* Defending against XSS in the Local App frontend (token theft via JS).
* Defending against malicious browser extensions (full session access).
* Real-time subscription revocation for *already issued* 5-minute tickets.
---
## Operational Implementation Details
### 1. Library Selection
* **`jose`:** The industry-standard library for modern JS. Used by the Marketplace to generate Ed25519 keys, sign JWTs, and expose the JWKS endpoint. **Also used by Data Engines** to verify tickets — `createRemoteJWKSet` handles remote fetching, caching, and `kid` selection, and `jwtVerify` validates the signature plus `iss`/`aud`/`exp` in one call, with native OKP/Ed25519 support.
* **`openid-client`:** The RFC-compliant library used by the Local App to execute the PKCE Authorization Code flow securely.
> [!NOTE]
> **Amendment (2026-05-22):** The original draft specified `@fastify/jwt` + `get-jwks` for Data Engines. This was found to be incorrect during local integration testing: `get-jwks.getPublicKey()` converts JWKs via `jwk-to-pem`, which **does not support OKP/Ed25519 keys** (throws `Unsupported key type "OKP"`). `@fastify/jwt`'s standalone `verify()` is also incompatible with a remote-JWKS async key resolver. Data Engines now use `jose` — the same library already mandated for the Marketplace — which supports Ed25519 natively.
### 2. Hardened First-Message Auth
Data Engines MUST enforce strict WebSocket handshake rules:
1. **Silence until Auth:** The server MUST accept no subscription or data messages before auth succeeds.
2. **Timeout:** Auth MUST complete within a short timeout (e.g., 3 seconds of connection).
3. **Drop on Failure:** A failed auth immediately closes the socket with a `4003` code.
4. **No Re-Auth:** Re-authentication on the same socket is forbidden.
5. **Log Redaction:** All server logs MUST redact the JWT payload from capture.
### 3. API Key Encryption at Rest (Local App)
The Local App stores the API Key in PostgreSQL. It MUST NOT be stored in plaintext.
* **Algorithm:** AES-256-GCM.
* **Key Derivation (KDF):** The encryption key is derived from `AUTH_SECRET` using PBKDF2 (100,000+ iterations) and a unique salt per installation.
* **Nonce/Salt Storage:** The 96-bit nonce and the KDF salt are stored alongside the encrypted payload in the DB.
* **Versioning:** The DB column must support key versioning (e.g., `v1:base64_nonce:base64_ciphertext`) to allow future `AUTH_SECRET` rotation without permanently bricking the DB.
---
## Failure-Mode Behavior
| Scenario | System Behavior |
| :--- | :--- |
| **Marketplace is down** | Local App cannot get new JWTs. Existing streams continue until the 5-min JWT expires. |
| **Data Engine Cold Start (Marketplace down)** | Engine cannot fetch `jwks.json`. Engine **fails closed** and rejects all connections. |
| **JWKS Cache expires (Marketplace down)** | Data Engine retains stale cache for a grace period, but eventually **fails closed**. |
| **Redis is down (Marketplace)** | Ticket issuance falls back to PostgreSQL (slower, but functional). |
| **Unknown `kid` arrives** | Data Engine immediately triggers a background re-fetch of `jwks.json`. If still unknown, rejects token. |
| **Clocks drift beyond leeway** | Token is rejected (Next.js and Fastify default to 60s leeway). NTP sync is a hard operational requirement. |
---
## Revocation Latency
| Event | Time to take effect |
| :--- | :--- |
| **User cancels subscription** | Up to 5 minutes (bounded strictly by active JWT lifetime). |
| **Admin rotates Ed25519 key** | Zero downtime (handled via key overlap in JWKS array). |
| **User revokes API Key** | Immediate for *new* issuances, up to 5 minutes for active streams. |
---
## The End-to-End Auth Flow
```mermaid
sequenceDiagram
participant U as User Browser
participant L as Local App
participant M as Marketplace
participant E as Data Engine
Note over U,M: ADR-001C: PKCE bootstrap (once)
U->>L: Click "Connect to Marketplace"
L->>L: Generate code_verifier + challenge
L->>U: Redirect to Marketplace with challenge
U->>M: Authorize
M->>U: Redirect with temp code
U->>L: Forward temp code
L->>M: POST temp code + code_verifier
M->>L: API Key (Long-lived)
L->>L: PBKDF2 + AES-GCM Encrypt & Store
Note over L,E: ADR-001B & A: Ongoing Data Plane
loop every 4.5 minutes
L->>M: POST API Key (Request audience: Engine A)
M->>L: 5-min JWT (aud: Engine A)
L->>U: JWT
U->>E: wss://engine (Origin validation)
U->>E: { type: "auth", v: 1, token: "..." }
E->>E: Verify JWT via jose (cached JWKS)
E->>U: Stream
end
```
@@ -0,0 +1,158 @@
# ADR-0002: Source-of-truth for plugin identity and registration
## Status
Accepted
## Date
2026-05-20
## Context
Two related problems in the plugin system surfaced in the same period, and both have the same root cause: canonical plugin information was being derived in middleware translation layers (folder names, hard-coded lists) rather than at its proper source. Each problem is fixed by pushing the canonical truth back to its rightful owner and removing the static translation in the middle. Documenting them together makes the shared principle explicit and discourages future regressions of the same shape.
### Problem 1 — Engine seeder ID derived from folder name
The WorldWideView local data engine discovers seeders by scanning the filesystem under `/app/seeders`. Historically, the engine assigned each seeder an ID equal to its **folder name** (e.g., the folder `gpsjam/` became plugin ID `"gpsjam"`).
The frontend, however, identifies plugins by a separate **`id` field** declared in each plugin class (e.g., `"gps-jamming"`). The engine's `/manifest` endpoint listed available plugins using folder-derived IDs, and the frontend's `localEngineHasPlugin(pluginId)` performed an exact string match against that list.
This created a silent failure mode: six seeders had folder names that did not match their corresponding frontend plugin IDs:
| Folder name | Frontend plugin `id` |
|---|---|
| `gpsjam/` | `gps-jamming` |
| `sanctions/` | `international-sanctions` |
| `wildfires/` | `wildfire` |
| `civilUnrest/` | `civil-unrest` |
| `conflictEvents/` | `conflict-events` |
| `cyberAttacks/` | `cyber-attacks` |
When `localEngineHasPlugin('gps-jamming')` returned `false` (because the manifest listed `"gpsjam"`), the frontend silently fell through to the production cloud engine, bypassing the local engine entirely. The bug was invisible — no error, just incorrect routing.
A short-term patch introduced `SEEDER_ID_ALIASES` in `src/core/data/engineManifest.ts` — a frontend translation map from folder names to plugin IDs. This was explicitly a band-aid: it duplicated knowledge in a consumer rather than fixing the contract at the source, and would go stale as new seeders were added.
The engine's TypeScript source (`seeder-loader.ts`) also had a `toKebabCase()` helper intended to normalize camelCase folder names (e.g., `civilUnrest` → `civil-unrest`), but its compiled `dist/` was stale (the production image had not been rebuilt to include it), and `toKebabCase` alone cannot resolve abbreviation mismatches like `gpsjam` → `gps-jamming`.
### Problem 2 — Fresh-install plugin set hard-coded
The marketplace auto-seeder ([src/lib/marketplace/seedDefaultPlugins.ts](../../../src/lib/marketplace/seedDefaultPlugins.ts)) writes a starter set of plugins to the database on first boot so the globe isn't empty on day one. The list of plugins to seed lived in a hard-coded constant `DEFAULT_PLUGIN_IDS` ([src/lib/marketplace/defaultPlugins.ts](../../../src/lib/marketplace/defaultPlugins.ts)).
Two failure modes followed from the static list:
1. **Drift**: ten plugins were published to npm under `@worldwideview/*` and verified — surveillance-satellites, military-bases, embassies, seaports, lighthouses, spaceports, iranwarlive, undersea-cables, mineral-mines, nuclear-facilities — but never added to `DEFAULT_PLUGIN_IDS`. Fresh installs missed them silently.
2. **Phantom 404s**: two entries in the list (`fortiguard`, `nz-traffic-cameras`) were dev-only sandboxes in `local-plugins/`, never published to npm. Every fresh boot hit `https://marketplace.worldwideview.dev/api/plugins/fortiguard` → 404, logged a warning, and continued. The branch `fix/diagnose-plugins-404` was opened to diagnose those.
A separate, signed verified-plugins registry already existed at `marketplace.worldwideview.dev/api/registry`, loaded by [registryClient.ts](../../../src/lib/marketplace/registryClient.ts) with Ed25519 verification. It was used for trust-stamping installed plugins but not for selecting which plugins to seed. That registry is the authoritative list of plugins that have been reviewed and verified — exactly the list a starter set should come from.
### Shared shape
Both problems wrap the same anti-pattern: a consumer (frontend, seeder) maintains a static translation/list derived from the producer's actual identity (seeder export, npm publication). The translation drifts. The fix in both cases is the same: stop translating in the consumer; let the producer declare the truth and consume it directly.
## Decisions
### Decision 1 — The seeder's exported `name` field is the canonical plugin ID
The engine reads each seeder's `dist/index.mjs` default export's `name` property and uses it as the plugin ID everywhere:
- `/manifest` endpoint `plugins` array
- WebSocket welcome message plugin list
- WebSocket data message `pluginId` field
- Redis key (`data:<name>:live`)
- Scheduler logs
The folder name is used only as a filesystem fallback if the seeder exports no `name`. The engine logs a warning at startup whenever the kebab-cased folder name differs from the seeder's exported `name`, making drift visible without blocking startup.
**Contract:** Every seeder MUST export a `name` field that exactly equals the corresponding frontend plugin's `id` field (kebab-case). This is a two-place contract: declare the same ID string in the seeder export and in the frontend plugin class.
Implementation: `seeder-loader.ts` was updated to set `id = seederConfig.name || toKebabCase(folderName)` instead of `id = toKebabCase(folderName)`. The frontend `SEEDER_ID_ALIASES` map was removed entirely. No translation layer exists between the engine's manifest and the frontend's plugin lookup.
### Decision 2 — The signed verified registry is the source-of-truth for fresh-install auto-seeding
The marketplace auto-seeder iterates the set returned by `getVerifiedPluginIds()` (the signed registry) instead of a hard-coded `DEFAULT_PLUGIN_IDS` constant. Publishing a plugin to the verified registry automatically makes it part of the fresh-install starter set on subsequent boots, with no code change.
The static `DEFAULT_PLUGIN_IDS` list is deleted. There is no fallback list in code — if the registry is unreachable on first attempt, the seeder defers without marking-seeded so the next request retries (the 5-min in-memory cache in `registryClient.ts` bounds the request rate during outages).
Trust stamping is collapsed: every plugin landed by the seeder is `verified` by definition, so the per-plugin `verified.has(pluginId)` check inside the seeder is dropped — `manifest.trust = "verified"` is set directly.
**Contract:** A plugin is part of the fresh-install starter set iff it appears in the signed verified-plugins registry. There is no other list to maintain.
## Shared principle
> **Canonical plugin truth lives at the source. Reject static translation/lookup layers in the middle.**
This is the architectural rule both decisions encode. Concrete corollaries for future contributions:
- Don't introduce a static map in a consumer to compensate for a producer that mis-labels itself. Fix the producer.
- Don't introduce a static list in code to enumerate what a registry or filesystem already enumerates. Read from the registry/filesystem.
- A translation layer that "just needs an entry added each time we publish a plugin" is a foot-gun — it will silently drift the moment someone forgets.
## Alternatives Considered
### For Decision 1
#### Folder renames (rejected as primary fix)
Rename the six mismatched folders to match their frontend plugin IDs (e.g., `gpsjam/` → `gps-jamming/`). This would make the folder name the canonical ID.
- **Rejected as primary fix** because Windows Docker Desktop holds NTFS volume mount locks on running containers, making renames impossible without a full Docker Desktop shutdown. More importantly, it makes the canonical ID fragile — any accidental folder rename breaks routing silently.
- Folder renames remain a cosmetic cleanup option, now that the seeder export is the actual source of truth.
#### Frontend alias map (`SEEDER_ID_ALIASES`)
Maintain a static mapping in `engineManifest.ts` that translates engine-reported IDs to frontend plugin IDs.
- **Rejected** because it puts translation responsibility on the consumer rather than the producer. Every new seeder with a naming mismatch requires a manual alias entry. The map goes stale silently — no warning if an alias is wrong or missing.
#### `toKebabCase()` normalization only
Apply `toKebabCase()` to folder names in the engine and rely on the convention that folder names encode the plugin ID in camelCase.
- **Rejected** because `toKebabCase` cannot resolve abbreviation mismatches (`gpsjam` → still `gpsjam`, not `gps-jamming`) or singular/plural differences (`wildfires` → still `wildfires`, not `wildfire`). It only handles camelCase-to-kebab conversion.
#### Strict validation with engine startup failure
Refuse to load any seeder whose folder name doesn't match its exported `name`, blocking startup until inconsistencies are resolved.
- **Rejected** as too aggressive for a local dev environment. A warning is sufficient — startup continues, mismatches are visible, and the system keeps working.
### For Decision 2
#### Add the missing 10 IDs to `DEFAULT_PLUGIN_IDS` (rejected)
Extend the static list to include the newly-published plugins.
- **Rejected** because it ships a fix that resolves the immediate symptom while preserving the underlying drift mechanism. The list would go stale again the next time a plugin is published.
#### Hybrid: registry as default + static fallback list
Read from the registry; fall back to a hard-coded list if the registry is empty/unreachable.
- **Rejected** because it reintroduces the static list as a permanent shadow source-of-truth. Fallback lists go stale silently. The chosen "defer seeding on empty registry" behavior is safer — fresh installs retry on the next request rather than locking in a stale set.
#### Delete auto-seeding entirely (rejected)
Drop the seeder and let users install everything from the marketplace UI starting from an empty globe.
- **Rejected** because fresh installs should have something to look at on day one. The user explicitly stated this requirement: "we want the user to have something to get started with."
## Consequences
### Easier
**Decision 1 — engine seeder identity**
- Self-documenting identity. Reading any seeder's `src/index.ts` immediately tells you its plugin ID. No external registry to consult.
- Filesystem layout is irrelevant. Folders can be named for readability or organized in subdirectories without affecting routing.
- No translation layers. The frontend receives the correct ID directly from the manifest; `localEngineHasPlugin()` works with a direct string comparison.
- Drift is visible. The engine warns on startup when folder names diverge from exported names, catching mistakes during development rather than silently at runtime.
- Adding a new seeder requires only: (1) export `name: '<plugin-id>'` matching the frontend plugin's `id`, (2) no registration step, no alias map entry.
**Decision 2 — marketplace seeding**
- Publishing a plugin to the verified registry is sufficient to add it to the starter set. No code change, no PR to add an ID to a constant.
- Phantom 404s are gone. The seeder only queries IDs that exist in the verified registry, which by construction reflects what's actually published.
- Trust semantics align. "Verified" and "auto-seeded on fresh install" are now the same condition, not two near-overlapping lists that can drift.
- Registry outages no longer brick fresh installs. The deferral semantics ensure the seeder retries instead of marking-seeded with zero rows.
### Harder
**Decision 1 — engine seeder identity**
- Seeder author bears responsibility. If a seeder exports the wrong `name`, routing fails silently (same failure mode as before, but now localized to the seeder rather than the folder). The startup warning mitigates this.
- Contract validation is advisory, not enforced. There is no build-time check that the seeder's `name` matches an actual frontend plugin `id`. A validation script (`scripts/validate-seeder-ids.mjs`) is the recommended mitigation.
- Engine image must be kept up to date. The fix lives in `dist/seeder-loader.js` inside the Docker image. If the published `ghcr.io/silvertakana/wwv-data-engine:latest` image is behind the local source, the behavior reverts to folder-name-based IDs. Local changes require `docker cp` injection or an image rebuild until the fix is published.
**Decision 2 — marketplace seeding**
- The verified registry must be kept accurate. A newly-published plugin doesn't auto-seed until it's added to the registry and the registry is re-signed. Operational responsibility shifts from "remember to add the ID to a code constant" to "remember to add the ID to the registry" — a similar discipline, but now at the right layer (the marketplace, not the consumer).
- Existing instances are not back-filled. Users whose `Setting` table already has `defaults_seeded=true` will not automatically receive newly-verified plugins; they must install via the marketplace UI. This is by design — re-seeding could reinstall plugins a user explicitly uninstalled.
- Dev-only sandboxes (`local-plugins/<name>/`) no longer "tag along" via the default list. `fortiguard` and `nz-traffic-cameras` only reach users once published to npm and added to the verified registry. The local-plugins folder discovery in `/api/marketplace/load` still surfaces them in dev (`NODE_ENV=development` or `WWV_PLUGIN_DEV=true`).
@@ -0,0 +1,175 @@
# ADR-0003: Shared Identity & Ecosystem Authentication Host
## Status
Proposed *(Pending Review)*
## Date
2026-05-22
## Related
- **Builds on:** ADR-001 (Decentralized Plugin Auth & SSRF Mitigation) — specifically the PKCE flow that lets the Local App acquire tokens from the Marketplace
- **Supersedes:** `.agents/rules/cloud-auth-architecture.md` (which incorrectly states that `app.worldwideview.dev` owns all login UI — this ADR reverses that)
---
## Context
WorldWideView is growing into a multi-product ecosystem:
| Product | URL | Purpose |
|---|---|---|
| Landing / Auth | `worldwideview.dev` | Marketing, login, signup, cloud plan registration |
| Marketplace | `marketplace.worldwideview.dev` | Browse, install, and manage plugins |
| Cloud App | `app.worldwideview.dev` (or `[tenant].app.worldwideview.dev`) | Hosted geospatial intelligence instance |
| Local App | Self-hosted | User's own instance, authenticated locally |
Four interrelated problems required a unified decision:
1. **No Marketplace identity system exists.** Users have nowhere to create an account, log in, or manage subscriptions.
2. **Fragmented account experience.** Without a plan, a user would need separate accounts for the Marketplace and the Cloud App — unacceptable for a single-vendor ecosystem.
3. **Cookie domain fragmentation.** If each product hosts its own login page, sessions cannot be shared across subdomains. A user logged into the Marketplace would need to log in again at the Cloud App.
4. **The `worldwideview-web` repo is a static export** (`output: "export"`). It has a `/login` page backed by client-side Supabase but cannot support server-side sessions, API routes, or cross-subdomain cookie management.
---
## Decisions
### ADR-003A: Federated Identity — One Account Across All Products
All WorldWideView products share a **single identity**. A user who creates an account anywhere in the ecosystem (Marketplace, Cloud App, or the landing site) has that same identity available everywhere else.
This is the standard industry pattern for multi-product SaaS — used by GitHub (one account → GitHub, Marketplace, Actions, Packages), Atlassian (one account → Jira, Confluence, Marketplace), and Stripe (one account → Dashboard, Atlas, Stripe Apps).
The shared identity store is a **single Supabase project**. All products point to the same `NEXT_PUBLIC_SUPABASE_URL` and read from the same `auth.users` table. No synchronisation jobs, no duplicate user records.
### ADR-003B: Supabase Auth as the Identity Provider
**Supabase Auth** is the identity layer for the entire ecosystem. It was chosen over:
- **Better Auth** — excellent self-hosted option, but requires a Postgres database to be provisioned separately. Since the Marketplace and `worldwideview-web` have no existing database, Supabase provides auth + Postgres in a single managed service with zero extra infrastructure.
- **NextAuth v5** — already present in the Local App for local access control, but it is not a full identity platform. Missing 2FA, RBAC, and a proper OAuth server model. Not suitable as an ecosystem-wide IdP.
- **Clerk** — fastest setup and best Stripe integration, but ~$25–50/month at low user counts, per-MAU pricing at scale, and user data is stored on Clerk's servers. Vendor lock-in is unacceptable for a product that holds plugin and subscription data.
Supabase Auth provides:
- Email + password, Google OAuth, GitHub OAuth, and magic link — all configured in the Supabase dashboard with no provider code
- `@supabase/ssr` package with first-class Next.js App Router support (server components, middleware, server actions)
- Cookie-based sessions that can be scoped to the parent domain
- Free tier is sufficient for early-stage Marketplace; Pro plan ($25/month) unlocks unlimited MAU
### ADR-003C: `worldwideview.dev` (worldwideview-web repo) is the Ecosystem Auth Host
The login and account registration UI lives at **`worldwideview.dev`** — the apex domain, served by the `worldwideview-web` repository. Three alternatives were considered:
**Option A — Each app has its own login page**
Rejected. Each product renders its own login form against the shared Supabase project. Browser `localStorage` (used by client-side Supabase) is domain-scoped and cannot be shared across subdomains. This means a user logged in at `marketplace.worldwideview.dev` would not be recognised at `app.worldwideview.dev`. Solving this in each app separately creates duplicated middleware and fragile cookie coordination.
**Option B — Marketplace owns the login page**
Rejected. The Cloud App would be permanently subordinate to the Marketplace for authentication. If the Marketplace is down, Cloud App users cannot log in. Also creates a confusing UX: users navigate to `app.worldwideview.dev` and are immediately bounced to `marketplace.worldwideview.dev` just to authenticate.
**Option C — Dedicated `accounts.worldwideview.dev` subdomain**
Seriously considered. A neutral auth subdomain is the cleanest long-term architecture. Rejected for now because it requires a third Next.js deployment and a third Coolify application. The same result is achieved by hosting auth on the apex domain itself (`worldwideview.dev`), which is already managed by `worldwideview-web` and is the identity host by virtue of owning the root domain.
**Chosen: Apex domain as auth host.** The `worldwideview-web` repo becomes the auth surface for the whole ecosystem:
| Path | Type | Purpose |
|---|---|---|
| `/` | Static | Landing page (marketing) |
| `/about`, `/docs`, `/plugins` | Static | Marketing content |
| `/login` | Dynamic (SSR) | Shared login for all products |
| `/signup` | Dynamic (SSR) | New account creation |
| `/cloud` | Dynamic (SSR) | Cloud plan registration and subscription |
| `/accounts/*` | Dynamic (SSR) | Account management, billing, API keys |
| `/api/auth/callback` | Dynamic (SSR) | OAuth redirect handler (Google, GitHub) |
**Why the apex domain is correct:** A session cookie set on `worldwideview.dev` is automatically inherited by all subdomains (`marketplace.worldwideview.dev`, `app.worldwideview.dev`, and any future product). This is the same mechanism used by Stripe — `stripe.com/login` sets the session that `dashboard.stripe.com` reads.
### ADR-003D: Cross-Subdomain Session Sharing via Parent Domain Cookie
Supabase Auth session cookies are explicitly scoped to the parent domain so that all products share the same session without additional coordination:
```typescript
// Applied in worldwideview-web middleware and all Supabase server clients
cookieOptions: {
domain: '.worldwideview.dev', // leading dot = all subdomains inherit
path: '/',
sameSite: 'lax',
secure: true, // HTTPS required — see local dev below
}
```
Each product (Marketplace, Cloud App) reads this cookie to get the authenticated user, then calls `supabase.auth.getUser()` server-side to validate the session. No cross-app API calls for authentication.
**Logout** must clear the cookie on the parent domain, not the subdomain, or the session will persist across products.
### ADR-003E: `worldwideview-web` Converts from Static Export to Mixed Static/SSR
The current `worldwideview-web` uses `output: "export"` in `next.config.ts`, forcing every page to be a static HTML file served by Nginx. This is removed.
Without `output: "export"`, Next.js App Router applies per-route rendering:
- Pages that do not call `cookies()`, `headers()`, or other dynamic functions are **statically generated at build time** — marketing pages remain fast and CDN-cached with no change required.
- Pages that call `cookies()` (all auth pages using `@supabase/ssr`) are **server-rendered on request**.
The Docker deployment changes from an Nginx static image to a Node.js runtime image. The Coolify application is updated in-place — no new Coolify application is needed.
### ADR-003F: `wwv.local` Local Development Environment
Because the parent-domain cookie requires `secure: true` (HTTPS only), local development using `localhost` will not replicate production session-sharing behaviour. A local HTTPS domain is required.
The project adopts `wwv.local` as the local development TLD with a self-signed certificate:
| Local URL | Maps to | Purpose |
|---|---|---|
| `wwv.local` | worldwideview-web on port 3001 | Landing + auth |
| `marketplace.wwv.local` | worldwideview-marketplace on port 3002 | Marketplace |
| `app.wwv.local` | worldwideview main app on port 3000 | Local App / Cloud App |
This setup is a deliberate investment: it ensures that cross-subdomain auth, redirect flows, and cookie behaviour can be tested locally before production deployment. The setup cost is a one-time `mkcert` install and hosts file edit per developer machine.
Setup (documented in each repo's README):
```bash
# Install mkcert (once per machine)
# macOS: brew install mkcert && mkcert -install
# Windows: choco install mkcert && mkcert -install
mkcert "*.wwv.local" wwv.local
# Add to hosts: 127.0.0.1 wwv.local marketplace.wwv.local app.wwv.local
```
---
## Security Constraints (from Pre-Mortem Review)
The following risks were identified and mitigated before implementation:
| Risk | Mitigation |
|---|---|
| Supabase anon key can read all rows if RLS is disabled | Row Level Security (RLS) MUST be enabled on all app-specific tables (`MarketplaceUser`, `InstalledPlugin`). Policy: `using (id = auth.uid())`. Server routes use `SUPABASE_SERVICE_ROLE_KEY` which bypasses RLS by design. |
| PKCE authorization code replay attack | The authorization code is deleted from the database transactionally at the moment of redemption. Any subsequent attempt receives `400 invalid_grant`. |
| Supabase free tier limits magic link email to 2/hour per address | A custom SMTP provider (Resend) is configured in Supabase project settings before magic link is enabled in production. |
| `MarketplaceUser` becomes orphaned when `auth.users` row is deleted | A Postgres trigger on `auth.users` DELETE cascades to `MarketplaceUser` and all related rows. |
| Ed25519 signing key rotation causes token rejection at Data Engines | Signing keys are stored in a `SigningKey` DB table, not env vars. The JWKS endpoint serves all non-revoked keys simultaneously. During rotation, the old key is marked `retiring` and kept in JWKS for 10 minutes before being marked `revoked`. |
---
## Marketplace PKCE OAuth Server (Relation to ADR-001)
The Marketplace acts as the **OAuth 2.0 Authorization Server** in the ADR-001 PKCE flow. Supabase Auth handles user identity; a thin custom layer handles the code exchange:
1. Local App redirects user to `marketplace.worldwideview.dev/api/oauth/authorize`
2. Marketplace validates the Supabase session (or redirects to `worldwideview.dev/login`)
3. User sees consent screen and approves
4. Marketplace issues a short-lived authorization code (60s TTL), redirects to Local App
5. Local App exchanges code + `code_verifier` for a long-lived API Key
6. Subsequent plugin connections use the API Key to request 5-minute EdDSA JWTs (ADR-001B)
This is not a new Supabase feature — it is custom Next.js API routes sitting on top of the existing Supabase Auth session.
---
## Consequences
- **One Supabase project** is a shared dependency across `worldwideview-web`, `worldwideview-marketplace`, and `worldwideview` (Cloud App). A Supabase outage affects all products simultaneously. This is an accepted tradeoff at current scale; the mitigation is Supabase's SLA and free-tier generous limits.
- **`worldwideview-web` must run as a Node.js server**, not a static Nginx container. This increases Coolify resource usage marginally but enables all auth functionality.
- **Local development requires `mkcert` setup** (one-time, ~5 minutes). This is a deliberate decision to prevent a class of production bugs that cannot be caught with `localhost`.
- **The old `cloud-auth-architecture.md` rule is invalidated.** That rule stated `app.worldwideview.dev` owns all login UI and the Marketplace redirects there. The reverse is now true. That rules file must be updated to reference this ADR.
- **Upgrade path is clear:** If the Marketplace matures into a platform that third-party developers build against (issuing OAuth tokens to external apps), the existing PKCE layer is extended without changing the user table or Supabase project. The identity layer is already decoupled from the token exchange layer.
@@ -0,0 +1,167 @@
# ADR-001 Pre-flight Audit — 2026-05-21
**Auditor:** Claude (ADR-001 Local App workstream) &nbsp; **Branch:** `feat/adr-001-local-app`
**Status:** Complete — see gate summary in §6
This document records the six step-0 checklist items from the master deploy plan
(`i-want-you-to-delightful-candle.md`, §"Deploy Order") before any ADR-001 code
lands on production.
---
## 1. Coolify Env-Var Audit
Swept every `JWT_*`, `TICKET_*`, `JWKS_*`, and `AUTH_*` variable across the three
live services. Values are masked; only presence is recorded.
### Debris check
| Service | Coolify resource | `JWT_*` | `TICKET_*` | `JWKS_*` | `AUTH_SECRET` | Decision |
|---|---|---|---|---|---|---|
| Local App | `worldwideview-demo` | — | — | — | ✓ set | nothing to remove |
| Data Engine | `wwv-engine-stack` | — | — | — | — | nothing to remove |
| Marketplace | `worldwideview-marketplace` | — | — | — | — | nothing to remove |
**Result: No debris from the previous failed deploy.** All three services are clean.
No removals required.
### ADR-001 vars — gaps to fill in deploy steps 1–2
| Service | Missing var | Required by | When to set | Notes |
|---|---|---|---|---|
| `worldwideview-demo` | `ENCRYPTION_MASTER_KEY` | **C1 hard gate** | **Step 1 — first action** | Without it the app boots but step-1 code will refuse to start |
| `worldwideview-demo` | `MARKETPLACE_URL` | C3 ticketClient | Step 1 | Server-side token exchange endpoint |
| `worldwideview-demo` | `NEXT_PUBLIC_WWV_MARKETPLACE_URL` | C3/C5 client | Step 1 | Client-side Marketplace link |
| `worldwideview-demo` | `PROXY_HOST_ALLOWLIST` | C2 SSRF | Step 1 | Set to `"*"` (observation mode); tighten in step 7c |
| `worldwideview-demo` | `NEXT_PUBLIC_WWV_TICKET_AUTH_PLUGINS` | C5 flag | Step 6 | Leave empty (dormant) until step 6 cutover |
| `worldwideview-marketplace` | `DATABASE_URL`, `AUTH_SECRET` | A basics | Step 2 | Marketplace cannot function without these |
| `worldwideview-marketplace` | EdDSA signing key + JWKS endpoint | A1–A2 | Step 2 | Required for `/api/auth/exchange` to sign tickets |
| `wwv-engine-stack` | JWKS / JWT public-key config | B JWKS wiring | Step 4 | See §5 — not blocking until cutover |
---
## 2. Data Engine `main` Branch State
**Repo:** `silvertakana/wwv-data-engine`
**Branch audited:** `main` (HEAD matches `feat/adr-001-data-engine`)
The `main` branch is **not** in a rolled-back clean state. The decentralised auth
work from the prior attempt was **retained and fully merged** (PR #5
`feature/decentralised-plugin-auth-data-engine`, PR #6 `chore/standardise-ci`), with
four additional fix commits on top.
**Auth code present in `main`:**
- Dependencies: `@fastify/jwt ^10`, `get-jwks ^11`, `jose ^6`
- `websocket.ts` verifies EdDSA JWTs:
```
allowedIss: 'https://marketplace.worldwideview.dev'
algorithms: ['EdDSA']
clockTolerance: 60
expectedAud: process.env.ENGINE_ID || 'wwv-data-engine'
```
**Env-gated bypass (commit `e3e2940`):**
`WWV_SKIP_WS_AUTH=true` is set on `wwv-engine-stack`. When true, every WS connection
is pre-authenticated with no JWT required. **This bypass is currently load-bearing:**
the JWKS endpoint is not yet wired (`websocket.ts:23` comment confirms), so the
engine cannot verify real tokens. `WWV_SKIP_WS_AUTH=true` must stay until Workstream
B wires JWKS (deploy step 4).
**Uncommitted working-tree WIP (not blocking steps 1–3):**
`src/server.ts`, `src/websocket.ts`, `src/websocket.spec.ts` modified; untracked
`HANDOFF.md`, `src/startup-checks.ts`, `vitest.config.ts`. The Data Engine
workstream has unfinished work not yet committed. Must be resolved before step 4.
---
## 3. Dormant Routes / Flags Already in Production
| Flag / Route | Service | Current value | Introduced | Notes |
|---|---|---|---|---|
| `WWV_SKIP_WS_AUTH` | `wwv-engine-stack` | `true` (load-bearing) | 2026-05-20 | Engine bypass — must stay `true` until JWKS wired (step 4) |
No other ADR-001 feature flags are active in production. The old no-auth data path
is still the only path through the Data Engine.
---
## 4. `ENCRYPTION_MASTER_KEY` Gate (C1)
**Status: FAIL — not set on `worldwideview-demo`**
The `ENCRYPTION_MASTER_KEY` env var is absent from the live Local App. Once
`src/instrumentation.ts` boot validation ships (deploy step 1), the app will
**refuse to start** if this var is missing.
**Action required before deploy step 1:**
Generate a 32-character key (`openssl rand -hex 16`) and set `ENCRYPTION_MASTER_KEY`
on `worldwideview-demo` in Coolify. This is the **first action of deploy step 1** and
blocks all subsequent steps.
---
## 5. SDK Delivery Decision (Gate for Workstream B)
**Status: OPEN — operator to confirm**
No evidence of a published `@worldwideview/wwv-plugin-sdk` package in the deployed
services. The `auth-contracts.ts` types (`TokenExchangeRequest`,
`TokenExchangeResponse`, `PluginTicket`, `MarketplaceSessionToken`) currently live in
the Local App's bundled SDK package.
Workstream B JWKS wiring requires a decision on how the engine receives the
Marketplace public key:
- **Option A:** Ed25519 public key as a Coolify env var (simplest)
- **Option B:** Remote JWKS URL env var — engine fetches key from
`https://marketplace.worldwideview.dev/.well-known/jwks.json` at startup
**Cross-repo issuer alignment note:** The engine hard-codes
`allowedIss: 'https://marketplace.worldwideview.dev'`. The Local App's `MARKETPLACE_URL`
config defaults to `https://app.worldwideview.dev`. Confirm that the live Marketplace
issues JWTs with `iss: https://marketplace.worldwideview.dev` before step 5 smoke test.
This decision must be made and documented before deploy step 4.
---
## 6. Gate Summary
| Gate | Status | Blocks deploy step | Action |
|---|---|---|---|
| No legacy JWT/TICKET debris | ✅ PASS | — | None |
| `ENCRYPTION_MASTER_KEY` set on Local App | ❌ FAIL | **Step 1** | Set as first action of step 1 |
| Marketplace configured (DB, auth, signing key) | ❌ FAIL | **Step 2** | Configure `worldwideview-marketplace` in Coolify |
| SDK delivery decision confirmed | ⚠️ OPEN | Step 4 | Operator decision needed |
| Data Engine JWKS wired | ❌ FAIL | Step 4 / cutover | `WWV_SKIP_WS_AUTH` bypass active until then |
| Data Engine WIP committed | ⚠️ OPEN | Step 4 | Not blocking steps 1–3 |
| Issuer alignment confirmed | ⚠️ OPEN | Step 5 smoke test | Verify `iss` claim matches engine `allowedIss` |
**Top pre-cutover risks:**
1. **JWKS not wired on the engine** — `WWV_SKIP_WS_AUTH=true` is the only thing
keeping streams alive. If this flag is ever accidentally unset, all WebSocket
streams break immediately.
2. **Marketplace under-configured** — token exchange will error until step 2 env vars
are set. `worldwideview-marketplace` currently has only infra vars (NIXPACKS,
REDIS, REGISTRY_PRIVATE_KEY).
3. **`ENCRYPTION_MASTER_KEY` missing** — deploy step 1 will trigger a boot refusal
if not set first. This is the riskiest item in the pre-cutover phase per the
master plan.
---
## 7. Cleanup Log
**Removed this step (2026-05-21):**
| Resource | Type | UUID | Reason |
|---|---|---|---|
| `wwv-app-stack` | Coolify service | `h3gtnyotl6b7007hudlfqyf9` | Operator-confirmed deprecated |
**Remaining cleanup decisions (operator to confirm before step 4):**
| Resource | Type | UUID | Notes |
|---|---|---|---|
| `worldwideview web` | Application | `cisr3bruyyvvoeym41dexxa3` | Repo `silvertakana/worldwideview-web`; only REDIS+AISSTREAM vars — appears stale. Verify before removing. |
| `wwv-data-engine-legacy` | Application | `sj8519kr561va8ktcdq166hn` | Repo `wwv-data-engine-internal`, branch `master`; appears superseded by `wwv-engine-stack`. Verify before removing. |
+79
View File
@@ -0,0 +1,79 @@
# ADR-001 Feature Flags
Single source of truth for all feature flags introduced by
[ADR-0001](architecture/decisions/adr-0001-decentralized-plugin-auth-and-ssrf-mitigation.md)
(Decentralized Plugin Auth + SSRF Mitigation).
**Keep the per-environment table current** whenever a flag is set, changed, or
removed in Coolify. Drift between staging and production is the primary paper-cut
this doc exists to prevent.
---
## Flag Inventory
### 5 Master-Plan Flags
| Flag | Repo | Default | Meaning |
|---|---|---|---|
| `NEXT_PUBLIC_WWV_TICKET_AUTH_PLUGINS` | **Local App** | `""` (empty = dormant) | Comma-separated plugin IDs that require WS first-message auth (e.g. `aviation,maritime`). Empty or unset → ticket auth disabled for all plugins. Set *after* Marketplace + Data Engine are deployed (step 6). |
| `REQUIRE_TICKET_AUTH` | **Data Engine** | `false` | When `true`, the engine rejects any WS connection that does not present a valid ticket. Global — applies to all plugins. Replaces `WWV_SKIP_WS_AUTH=false` once JWKS is wired (step 7a). |
| `REQUIRE_TICKET_AUTH_PLUGINS` | **Data Engine** | `""` | Comma-separated plugin IDs for per-plugin enforcement before enabling the global flag. Not yet implemented (see Divergence Note below). |
| `ENFORCE_ORIGIN_ALLOWLIST` | **Data Engine** | `false` | When `true`, the engine rejects connections from Origins not in its configured allowlist. Enable after populating the allowlist from observed traffic (step 7b). |
| `PROXY_HOST_ALLOWLIST` | **Local App** | `""` (deny all) | SSRF allowlist for the camera / iframe proxy route. `""` or unset = deny all (safe default). `"*"` = permissive + logs WARN per host (observation mode). Comma-separated list = explicit allowlist. Ship `"*"` initially; tighten to an explicit list after one week of log observation (step 7c). |
### Deployed Reality: `WWV_SKIP_WS_AUTH`
> **Divergence note:** The Data Engine workstream implemented a single global bypass
> flag (`WWV_SKIP_WS_AUTH`) instead of the three granular master-plan flags
> (`REQUIRE_TICKET_AUTH`, `REQUIRE_TICKET_AUTH_PLUGINS`, `ENFORCE_ORIGIN_ALLOWLIST`).
> The master-plan flags remain the target architecture and must be implemented before
> step 7. `WWV_SKIP_WS_AUTH` is the current interim control.
| Flag | Repo | Default | Meaning |
|---|---|---|---|
| `WWV_SKIP_WS_AUTH` | **Data Engine** | `false` | When `true`, every WS connection is pre-authenticated — no JWT is checked. **Currently `true` in production and load-bearing** (JWKS not yet wired; the engine cannot verify real tokens). Must stay `true` until deploy step 4 wires JWKS. Flip to `false` as part of step 5 smoke test, replacing it with `REQUIRE_TICKET_AUTH=true`. |
---
## Per-Environment Table
Update this table whenever a flag changes in Coolify.
| Flag | Local dev | Staging | Production | Last updated |
|---|---|---|---|---|
| `NEXT_PUBLIC_WWV_TICKET_AUTH_PLUGINS` | `""` (.env.example) | `""` (not set) | `""` (not set) | 2026-05-21 |
| `REQUIRE_TICKET_AUTH` | N/A | N/A | N/A (not yet implemented) | — |
| `REQUIRE_TICKET_AUTH_PLUGINS` | N/A | N/A | N/A (not yet implemented) | — |
| `ENFORCE_ORIGIN_ALLOWLIST` | N/A | N/A | N/A (not yet implemented) | — |
| `PROXY_HOST_ALLOWLIST` | `"*"` (.env.example) | TBD — set before step 5 | not set (= deny all) | 2026-05-21 |
| `WWV_SKIP_WS_AUTH` | `false` | TBD — set before step 4 | `true` (**load-bearing**) | 2026-05-20 |
---
## Rollout Sequence
The sequence in which flags are set during the ADR-001 deploy order:
| Deploy step | Service | Flag change | Notes |
|---|---|---|---|
| Step 1 | Local App | Set `PROXY_HOST_ALLOWLIST="*"` | Observation mode — watch WARN logs for one week |
| Step 3 | Local App | `NEXT_PUBLIC_WWV_TICKET_AUTH_PLUGINS=""` confirmed empty | Dormant — no change needed, just confirm |
| Step 4 | Data Engine | JWKS wired; `WWV_SKIP_WS_AUTH` remains `true` | Both paths exist, neither enforced |
| Step 5 (staging) | Local App | `NEXT_PUBLIC_WWV_TICKET_AUTH_PLUGINS=test-synthetic` | Smoke test only — revert after |
| Step 5 (staging) | Data Engine | `WWV_SKIP_WS_AUTH=false` | Prove real JWT verification end-to-end |
| Step 6 | Local App | `NEXT_PUBLIC_WWV_TICKET_AUTH_PLUGINS=<first-real-plugin>` | One plugin at a time, 24 h soak |
| Step 6 | Data Engine | `REQUIRE_TICKET_AUTH_PLUGINS=<same-plugin>` | Mirror Local App change |
| Step 7a | Data Engine | `REQUIRE_TICKET_AUTH=true` | Global default-deny; 24 h soak |
| Step 7b | Data Engine | `ENFORCE_ORIGIN_ALLOWLIST=true` | After populating allowlist from observed connections |
| Step 7c | Local App | `PROXY_HOST_ALLOWLIST=<observed-list>` | Replace `"*"` with explicit host list |
---
## Rollback
Every step is independently revertable by flipping its single flag back. The old
no-auth data path remains wired throughout steps 1–6 and is only superseded at
step 7a. If anything breaks, removing the plugin from
`NEXT_PUBLIC_WWV_TICKET_AUTH_PLUGINS` immediately restores the old path for that
plugin.
+7 -9
View File
@@ -1,7 +1,7 @@
import nextCoreWebVitals from "eslint-config-next/core-web-vitals";
import nextTypeScript from "eslint-config-next/typescript";
export default [
const config = [
{
ignores: [
"node_modules/**",
@@ -16,6 +16,8 @@ export default [
"local-seeders/**",
".agents/**",
".claude/**",
".gemini/**",
"*.cjs",
],
},
...nextCoreWebVitals,
@@ -33,20 +35,16 @@ export default [
// Pre-existing — flagged in the original code review (#96):
"@typescript-eslint/no-explicit-any": "warn",
"no-console": "warn",
"no-console": "off",
// React 19 hook-purity rules from eslint-plugin-react-hooks 7+:
"react-hooks/purity": "warn",
"react-hooks/refs": "warn",
"react-hooks/set-state-in-effect": "warn",
"react-hooks/immutability": "warn",
"react-hooks/static-components": "warn",
// (These were previously hallucinated or require extra plugins)
// Misc one-offs across the codebase:
"react/no-unescaped-entities": "warn",
"@typescript-eslint/ban-ts-comment": "warn",
"@next/next/no-assign-module-variable": "warn",
"prefer-const": "warn",
},
},
];
export default config;
-2
View File
@@ -93,6 +93,4 @@ const nextConfig: NextConfig = {
},
};
import { withSentryConfig } from "@sentry/nextjs";
export default nextConfig;
+29 -12
View File
@@ -1,6 +1,6 @@
{
"name": "worldwideview",
"version": "2.9.2",
"version": "2.18.1",
"private": true,
"type": "module",
"packageManager": "[email protected]",
@@ -8,7 +8,7 @@
"setup": "node scripts/setup.mjs",
"test-worktree": "node scripts/test-worktree.mjs",
"clean": "node -e \"const fs=require('fs');fs.rmSync('.next',{recursive:true,force:true})\"",
"dev": "concurrently \"pnpm run dev:plugins\" \"next dev\"",
"dev": "node scripts/dev.mjs",
"dev:all": "concurrently -n \"dev,backends\" -c \"magenta,blue\" \"pnpm dev\" \"pnpm dev:backends\"",
"dev:backends": "docker compose --profile engine up wwv-data-engine wwv-redis",
"dev:plugins": "node scripts/watch-local-plugins.mjs",
@@ -18,9 +18,11 @@
"build": "next build --webpack",
"start": "next start",
"test": "vitest run",
"test:e2e": "playwright test",
"test:mutate": "stryker run",
"lint": "eslint .",
"db:reset": "npx prisma migrate reset --force",
"db:prune": "node scripts/db-prune.mjs",
"generate": "prisma generate"
},
"dependencies": {
@@ -29,8 +31,8 @@
"@prisma/adapter-pg": "^7.8.0",
"@prisma/client": "^7.8.0",
"@prisma/client-runtime-utils": "7.8.0",
"@sentry/nextjs": "^10.52.0",
"@supabase/supabase-js": "^2.105.4",
"@sentry/nextjs": "^10.53.1",
"@supabase/supabase-js": "^2.106.0",
"@types/pg": "^8.20.0",
"@vercel/analytics": "^1.6.1",
"@worldwideview/wwv-plugin-sdk": "workspace:*",
@@ -40,22 +42,28 @@
"hls.js": "^1.6.16",
"insecam-api": "^0.0.3",
"jose": "^6.2.3",
"lucide-react": "^1.14.0",
"lucide-react": "^1.16.0",
"next": "16.2.6",
"next-auth": "^5.0.0-beta.30",
"pg": "^8.20.0",
"openid-client": "^6.8.4",
"pg": "^8.21.0",
"prisma": "^7.8.0",
"react": "19.2.6",
"react-dom": "19.2.6",
"react-player": "^3.4.0",
"resium": "1.21.0",
"satellite.js": "^7.0.0",
"resium": "1.21.1",
"satellite.js": "^7.0.1",
"stripe": "^22.1.1",
"tslib": "^2.8.1",
"ws": "^8.14.2",
"undici": "^8.3.0",
"ws": "^8.20.1",
"zod": "^4.4.3",
"zustand": "^5.0.13"
},
"devDependencies": {
"@eslint/compat": "^2.1.0",
"@eslint/eslintrc": "^3.3.5",
"@playwright/test": "^1.60.0",
"@stryker-mutator/core": "^9.6.1",
"@stryker-mutator/vitest-runner": "^9.6.1",
"@testing-library/dom": "^10.4.1",
@@ -66,18 +74,27 @@
"@types/react": "^19",
"@types/react-dom": "^19",
"@types/ws": "^8.18.1",
"@vitejs/plugin-react": "^5.1.4",
"@typescript-eslint/eslint-plugin": "^8.59.4",
"@typescript-eslint/parser": "^8.59.4",
"@vitejs/plugin-react": "^6.0.2",
"@vitest/coverage-v8": "4.1.6",
"concurrently": "^9.2.1",
"copy-webpack-plugin": "^14.0.0",
"eslint": "^9",
"eslint-config-airbnb": "^19.0.4",
"eslint-config-airbnb-typescript": "^18.0.0",
"eslint-config-next": "^16.2.6",
"eslint-plugin-import": "^2.32.0",
"eslint-plugin-jsx-a11y": "^6.10.2",
"eslint-plugin-react": "^7.37.5",
"eslint-plugin-react-hooks": "^7.1.1",
"fast-check": "^4.8.0",
"jsdom": "^28.1.0",
"rollup-plugin-external-globals": "^0.13.0",
"tsx": "^4.21.0",
"tree-kill": "^1.2.2",
"tsx": "^4.22.2",
"typescript": "^5",
"vite": "^8.0.12",
"vite": "^8.0.13",
"vitest": "^4.1.6"
},
"pnpm": {
+2 -2
View File
@@ -13,11 +13,11 @@
"clean": "rm -rf dist"
},
"dependencies": {
"commander": "^12.1.0",
"commander": "^14.0.3",
"prompts": "^2.4.2"
},
"devDependencies": {
"@types/node": "^20.0.0",
"@types/node": "^25.9.1",
"@types/prompts": "^2.4.9",
"typescript": "^5.0.0"
}
+1
View File
@@ -1,3 +1,4 @@
/* eslint-disable no-console */
import { Command } from 'commander';
import { saveConfig } from '../utils/config.js';
+1
View File
@@ -1,3 +1,4 @@
/* eslint-disable no-console */
import { Command } from 'commander';
import fs from 'fs';
import path from 'path';
+3 -2
View File
@@ -1,3 +1,4 @@
/* eslint-disable no-console */
import { Command } from 'commander';
import { execSync } from 'child_process';
import path from 'path';
@@ -50,7 +51,7 @@ export const publishCommand = new Command('publish')
process.exit(1);
}
let originalName = pkgContent.name;
const originalName = pkgContent.name;
let publishedName = originalName;
// Determine organization
@@ -66,7 +67,7 @@ export const publishCommand = new Command('publish')
try {
const whoami = execSync('npm whoami', { stdio: 'pipe' }).toString().trim();
targetOrg = whoami;
} catch (e) {
} catch (_e) {
// Not logged in or errored, ignore
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@worldwideview/wwv-lib-aviation",
"version": "1.0.1",
"version": "1.0.2",
"description": "Shared abstract base classes for WorldWideView aviation plugins (commercial, military, specialized)",
"main": "src/index.ts",
"types": "src/index.ts",
-1
View File
@@ -69,7 +69,6 @@ export abstract class BaseAviationPlugin implements WorldPlugin {
return {
type: "model",
iconUrl: this.iconUrl,
size: isAirborne ? 8 : 5,
modelUrl: this.modelUrl,
modelScale: 2.56,
modelMinPixelSize: 16,
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@worldwideview/wwv-lib-incidents",
"version": "1.0.1",
"version": "1.0.2",
"description": "Shared abstract base classes for WorldWideView incident plugins (earthquakes, wildfires, conflicts)",
"main": "src/index.ts",
"types": "src/index.ts",
Loaded 100 of 380 files, more files were not shown because too many files have changed in this diff. Show more