Skip to content

feat(cli): add 'appkit add' and 'appkit registry' commands - #462

Open
MarioCadenas wants to merge 31 commits into
mainfrom
feat/registry-cli
Open

feat(cli): add 'appkit add' and 'appkit registry' commands#462
MarioCadenas wants to merge 31 commits into
mainfrom
feat/registry-cli

Conversation

@MarioCadenas

@MarioCadenas MarioCadenas commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Adds a component/plugin registry workflow to the AppKit CLI. Items are fetched directly from the databricks/appkit-registry repo and written into the consumer's app — no shadcn and no components.json required. Registry commands are wired up but hidden from --help while the feature is still in development.

New CLI surface

  • appkit add <item...> (top-level, the primary entry point) — adds UI components and/or server plugins from the registry. Auto-detects each item's type and routes it to the right place:

    • UI components → <frontend>/src/components/appkit/
    • Server plugins → <server>/plugins/<name>/, then runs appkit plugin sync --write and registers the plugin in the server's createApp call (--no-register to skip the edit).

    Frontend/server roots are detected from common monorepo layouts, so it can be run from the repo root. Transitive registryDependencies are resolved breadth-first (deduped, fetched concurrently per level), and npm dependencies are installed with the detected package manager (pnpm/yarn/bun/npm).

  • appkit registry list — enumerates items from the registry index, showing each item's kind (component/plugin/hook/…) and a marker for verified items (colorized output).

  • appkit registry search <terms...> — filters items by name / title / description / type / category; all terms must match.

  • appkit registry info <item> — shows an item's resource requirements plus its npm and registry dependencies (--json for machine output).

Resource-aware install

Server plugins declare the Databricks resources they need. On add, the CLI:

  • reconciles the resource env vars into .env (and their names into .env.example);
  • patches the deploy config — app.yaml and databricks.yml resource bindings — to match, never clobbering existing entries;
  • prompts interactively by default, with a workspace picker to resolve resource IDs; --yes runs non-interactively for agents/CI, --env KEY=VALUE (repeatable) pre-supplies values, and --profile runs bundle validate after writing;
  • warns when a resource type needs a user_api_scope before deploy.

Security hardening

Registry items are untrusted remote data, so add gates and sanitizes them:

  • Verified gate — refuses items the index doesn't mark verified, evaluated over the entire resolved graph so an unverified transitive dep can't ride in on a verified item. Fails closed if the index can't be read; --allow-unverified overrides.
  • Identity pinning — an item is pinned to the fetch key, not the body's self-reported name, so it can't claim a verified name to slip past the gate.
  • Name validation — item refs must be plain slugs before use as a fetch path or on-disk dir (blocks /, .., control chars).
  • Path containment — file targets are resolved within the destination dir; absolute or escaping targets are rejected (blocks arbitrary write → RCE).
  • Dependency specs — only [@scope/]name[@range] specs are installed (blocks tarball/git-URL install-script RCE and flag/argument injection; -- guard as defense in depth).
  • Code interpolation — a plugin's export name is only spliced into the user's server source when it's a valid JS identifier.

Tests & supporting changes

  • Unit tests + fixtures (analytics, lakebase) covering add, client, config-plan, config-writer, env-reconcile, env-writer, requirements, server-register, and workspace-picker.
  • schemas/manifest.ts: single source for field-origin derivation (used by requirements).
  • apps/scratch/: gitignored dir for local scratch apps; pnpm workspace/lock updates.

Paired change

The registry itself is renamed to @databricks-appkit in databricks/appkit-registry#1; this PR's REGISTRY_NAMESPACE matches it.

@github-actions

Copy link
Copy Markdown
Contributor

🔬  Run evals on this PR  ·  Go to Evals Monitor →

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

📦 Bundle size report

Compared against bundle-size-baseline.json (main).

@databricks/appkit

npm tarball (packed): 840 KB (+165 B) — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist raw gzip
JS (runtime) 869 KB (-553 B) 303 KB (-89 B)
Type declarations 315 KB (+379 B) 109 KB (+212 B)
Source maps 1.7 MB (+326 B) 566 KB (+137 B)
Other 11 KB 3.7 KB
Total 2.9 MB (+152 B) 982 KB (+260 B)
Per-entry composition (own code — deps external (as shipped))
Entry Initial (gz) Lazy (gz) Total (gz) node_modules (min) Own code (min)
. 88 KB (+18 B) 2.5 KB 91 KB (+18 B) external 288 KB (+48 B)
./beta 49 KB (+7 B) 456 B (-2 B) 49 KB (+5 B) external 143 KB (+48 B)
./type-generator 21 KB (+41 B) 0 B 21 KB (+41 B) external 61 KB (+48 B)

Chunks:

Entry Chunk Load Size (gz)
. index.js initial 84 KB
. utils.js initial 4.0 KB
. remote-tunnel-manager.js lazy 2.5 KB
./beta beta.js initial 33 KB
./beta stream-manager.js initial 5.8 KB
./beta wide-event-emitter.js initial 3.2 KB
./beta databricks.js initial 3.0 KB
./beta configuration.js initial 2.1 KB
./beta service-context.js initial 1.3 KB
./beta client.js initial 434 B
./beta client-options.js initial 219 B
./beta supervisor-api.js lazy 192 B
./beta databricks.js lazy 141 B
./beta index.js lazy 123 B
./type-generator index.js initial 21 KB

@databricks/appkit-ui

npm tarball (packed): 342 KB (+351 B) — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist raw gzip
JS (runtime) 390 KB 130 KB
Type declarations 228 KB (+412 B) 83 KB (+360 B)
Source maps 753 KB 248 KB
CSS 16 KB 3.3 KB
Total 1.4 MB (+412 B) 465 KB (+360 B)
Per-entry composition (consumer bundle — deps bundled, peerDeps external)
Entry Initial (gz) Lazy (gz) Total (gz) node_modules (min) Own code (min)
./js 5.3 KB 49 KB 55 KB 208 KB 14 KB
./js/beta 20 B 0 B 20 B 0 B 0 B
./react 432 KB 49 KB 480 KB 1.3 MB 175 KB
./react/beta 1.0 KB 0 B 1.0 KB 0 B 1.9 KB

Chunks:

Entry Chunk Load Size (gz)
./js index.js initial 5.2 KB
./js chunk initial 120 B
./js apache-arrow lazy 49 KB
./js/beta beta.js initial 20 B
./react index.js initial 430 KB
./react tslib initial 2.1 KB
./react apache-arrow lazy 49 KB
./react/beta beta.js initial 1.0 KB

@github-actions

Copy link
Copy Markdown
Contributor

This pull request has had no activity for 23 days and has been marked as stale. It will be closed in 7 days if there is no further activity. Add a comment, push a commit, or apply the no-stale label to keep it open.

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

🤖 AppKit PR bot

🔬 Run evals

Start an eval for this PR from the evals-monitor app: Go to Evals Monitor →

📦 Try this PR's app template

Scaffolds a new app from this PR's SDK build. Run it in any folder (requires the GitHub CLI — gh auth login — and the Databricks CLI):

gh run download 31817420861 -R databricks/appkit -n appkit-template-0.60.0-pr.f677545-feat-registry-cli-462 -D appkit-pr-462 \
  && unzip -o "appkit-pr-462/appkit-template-0.60.0-pr.f677545-feat-registry-cli-462.zip" -d "appkit-pr-462" \
  && databricks apps init --template "appkit-pr-462"

The template pins @databricks/appkit and @databricks/appkit-ui to tarballs built from this branch, so the scaffolded app runs against this PR's code.

@github-actions github-actions Bot removed the stale label Aug 4, 2026
@MarioCadenas
MarioCadenas force-pushed the feat/registry-cli branch 2 times, most recently from b2816be to 15bb1be Compare August 13, 2026 10:30
@MarioCadenas
MarioCadenas marked this pull request as ready for review August 13, 2026 13:59
@MarioCadenas
MarioCadenas requested a review from a team as a code owner August 13, 2026 13:59
@MarioCadenas
MarioCadenas requested a review from pkosiec August 13, 2026 13:59
Add a shadcn-namespaced component registry workflow to the AppKit CLI:

- 'appkit add <component...>' ensures the @AppKit registry namespace in the
  consumer's components.json, then delegates to 'shadcn add @appkit/<name>'.
- 'appkit registry list' enumerates components from the registry index.

Registry components import primitives from @databricks/appkit-ui (npm peer),
so they stay in sync with the installed AppKit version and design tokens.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Serve the registry directly from the public GitHub repo over
raw.githubusercontent.com instead of a placeholder host — no separate
hosting infra needed.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Fetch registry items ourselves (token-aware) and hand a local file to
'shadcn add', rather than relying on a shadcn namespace — so we control
the auth headers and the internal/private repo works today.

- Token resolved from gh auth token, then APPKIT_REGISTRY_TOKEN /
  GITHUB_TOKEN / GH_TOKEN.
- Private fetch uses the GitHub Contents API (Accept: raw); falls back to
  raw.githubusercontent.com when no token / once the repo is public.
- 'appkit registry list' is token-aware too.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
AppKit registry items are self-contained (import from @databricks/appkit-ui,
no shadcn @/ aliases or registryDependencies), so shadcn's alias resolution
isn't needed. Write item files to their target path directly and install npm
deps with the detected package manager — no components.json required.

- Targets resolved under src/ when present.
- --force to overwrite existing files (refuses by default).
- Warns on any registryDependency rather than silently dropping it.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
'appkit add' is typically run from the repo root, where the frontend lives
in a client/ subdir. Detect the frontend root (dir with components.json or
src/, including common client/frontend/web/app subdirs) and write components
under <frontend>/src/<target>. Install deps into the nearest package.json
(single root package.json in the AppKit app layout). Add --cwd to override.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Server plugins can now be distributed through the same registry as UI
components. 'appkit plugin add <name>' fetches a plugin item, writes it to
plugins/<name>/ (verbatim targets, not under client/src), installs npm deps,
runs 'plugin sync' to register it, and prints the createApp snippet plus any
required env vars.

- Extract shared fetch into registry/client.ts (reused by add + plugin add).
- Detect plugin items by the presence of manifest.json.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
…server/plugins

Remove the separate 'appkit plugin add' command. 'appkit add' now detects the
item kind (plugin = has manifest.json) and routes:
  • UI components → <frontend>/src/components/appkit/
  • server plugins → <server>/plugins/<name>/ (server/ subdir detected)
then installs deps, runs plugin sync for plugins, and prints next steps.
A single call can mix components and plugins.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
After placing a plugin and running sync, 'appkit add' edits the server entry
(server/index.ts etc.) to add the import and insert the plugin into the
createApp({ plugins: [...] }) array, reusing the ast-grep machinery from
'plugin sync'. Best-effort and safe:
  • idempotent — skips if the plugin is already registered
  • only edits the standard plugins:[...] array literal; otherwise falls back
    to printing the manual snippet
  • --no-register opts out of the server edit

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
toPlugin exports are factories, so register as `hello()` (matching server(),
analytics()), not a bare identifier. Insert before the first array element with
its indentation so multi-line plugins arrays keep their formatting instead of
jamming onto the opening-bracket line.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
- 'appkit registry list' shows a VERIFIED ✓ column (from item meta.verified)
  with a --verified filter; JSON output gains a top-level verified field.
- Colorize list + add output with picocolors (green created/registered/✓,
  yellow updated/warnings, red errors, dim hints). Padding is applied before
  coloring so columns stay aligned; picocolors no-ops on non-TTY/NO_COLOR.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Derive a friendly kind per item: a manifest.json marks a plugin; otherwise map
the shadcn registry:* type (component/hook/lib/theme/...). Adds a colorized TYPE
column to the table and a 'kind' field to --json output.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Adds a search subcommand so agents (and people) can match intent to items:
matches all query terms against name, title, description, derived kind, and
item categories. Shares the token-aware index fetch + table/JSON rendering
with 'registry list'. --verified and --json supported.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Match the registry rename (registry.json name: appkit -> databricks-appkit)
so `appkit add` writes the @databricks-appkit namespace into components.json
and stripNamespace() strips the same prefix.

Co-authored-by: Isaac
…l resource env vars

- Plugins declaring registryDependencies now pull their full dependency
  graph on 'appkit add' (previously ignored on the plugin branch and never
  resolved transitively).
- declaredEnvVars walks optional resources too, not just required.
- Add regression tests for the registry add resolver (dir had none).

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
The registry-distributed 'add' and 'registry' commands are executable but
hidden from top-level --help while the feature is still in development.
Registered with { hidden: true } — no behavior change, only help visibility.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
…, workspace picker

Folds the full registry resource workflow into the base:

- registry info + requirements table on add
- .env / .env.example reconciliation (origin-classified from the authored
  contract: platform fields skipped, static defaults pre-filled, already-set
  values preserved and fed to deploy config)
- app.yaml + databricks.yml resource-binding generation (verified against
  golden fixtures; writes target-variable values even when binding/var pre-exist)
- workspace picker: SDK-backed listing (auto-paginating) with a type-to-filter
  autocomplete over the full list; flat types via the SDK, parent-context types
  (volume/uc_function/secret/vector_search_index) via CLI drill-down; free-text
  fallback when the workspace can't be reached

Relocates the Databricks SDK facade from packages/appkit/src/workspace-client to
packages/shared/src/workspace-client so both appkit and the CLI (in shared) reach
the SDK through one sanctioned import site. appkit's workspace-client becomes a
thin re-export (its 47 importers are unchanged); the noRestrictedImports allow-list
points at the new location. Adds a `profile` option to WorkspaceClientOptions.

Adds @databricks/sdk-experimental + yaml deps to shared.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Scratch/test apps placed under apps/scratch/ resolve their @databricks/*
dependencies against the monorepo (workspace-linked via the new apps/scratch/*
entry in pnpm-workspace.yaml) and are skipped by knip (apps/** is already in
ignoreWorkspaces). Contents are gitignored; .gitkeep keeps the dir tracked so
the location exists for everyone without committing any scratch app.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Registry items are untrusted remote data. Close the review findings on
the resource-aware `appkit add` path:

- Path traversal: writeItemFile routes through resolveWithinBase, which
  rejects absolute/escaping file targets (arbitrary write -> RCE).
- Export-name / import-path injection: validate against a JS-identifier /
  clean-module-path allowlist before editing the user's server source.
- Dependency injection: partitionDeps allowlists name@version specs,
  rejects flag-like and URL/git specs; install uses a `--` separator.
- .env newline injection: isSafeEnvValue rejects CR/LF values that would
  inject an extra .env line (e.g. host override -> exfil).
- CLI arg injection: drill-down refuses `-`-prefixed parent picks so a
  crafted resource name can't become a databricks CLI flag.
- Integrity gate: add refuses items the registry index doesn't mark
  verified unless --allow-unverified is passed.

Correctness/perf alongside:
- Postgres binding fields with no env (project/branch/database) are now
  collected so databricks.yml target variables are assigned.
- Secret drill-down composes scope/key instead of dropping the scope.
- Flat picker caps pagination at MAX_PICKER_RESULTS; Genie listSpaces
  follows next_page_token.
- resolveItems fetches each dependency level concurrently.
- add lazy-loads server-register (@ast-grep) and env-writer (SDK) so
  unrelated CLI commands don't pay their load cost.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Follow-up to the registry-add hardening, from a second review pass:

- Verified gate now covers the full resolved set (requested items plus
  their transitive registryDependencies), not just the top-level names.
  A verified item could otherwise declare an unverified registryDependency
  whose code gets written/installed/wired without passing the gate. The
  gate moved after resolveItems (fetching item JSON is read-only; nothing
  is written until after the check) and still fails closed on an
  unreadable index.
- Env-var NAMES from a manifest are now validated with isValidEnvName
  (^[A-Za-z_][A-Za-z0-9_]*$) before reaching .env or app.yaml. The prior
  fix guarded the value but not the key, so a field named
  "PORT=x\nDATABRICKS_HOST=..." could still inject a second .env line.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
The verified gate checks items.map(i => i.name), but item.name came
straight from the fetched JSON body — untrusted remote data never checked
against the key the item was fetched under. A verified:false item
published at key "evil" could self-report "name": "analytics" (a verified
name) and slip past the gate, then have its own files written and wired
into the server. Same spoof could point item.name at another item's
plugin dir.

Pin item.name to the fetch key inside resolveItems, so the trustworthy
identity (what the user requested / a parent listed / the index keys
verified on) is what every downstream consumer sees — the gate, the
plugins/<name> write path, and dedup.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Address the remaining review findings on appkit add:

- Item names (user refs and untrusted registryDependencies) are now
  slug-validated via isValidItemName (^[A-Za-z0-9._-]+$, rejecting `.`
  and `..`) before use. A name is both the fetch path
  (public/r/<name>.json) and the on-disk plugins/<name> dir, so a crafted
  ref like "../../attacker/repo/payload" could otherwise redirect the
  fetch (SSRF) or escape the destination dir. Rejecting at the source
  also keeps control chars (ANSI escapes) out of any printed name.
- Fetch the verified index and resolve the item graph concurrently — the
  two are independent round-trips, previously serialized (~1 RTT on every
  add). The gate still evaluates verification before any write.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Final review cleanups:

- registry info now validates its ref with isValidItemName before
  fetching, matching the add guard, so `/` or `..` can't redirect the
  request to another path in the repo.
- Reword code/test comments: the fetch templates interpolate the name
  into the URL path only (same host), so a bad name is a repo-path
  redirect, not cross-host SSRF. Corrected to avoid overstating it.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Cleanup pass over the registry CLI (no behavior change):

- Reuse: parseEnv now delegates to dotenv.parse (the parser the app loads
  .env with at runtime); extract registryAuthHeaders as the single source
  for the GitHub auth/Accept headers (was duplicated 3x); share the
  JS_IDENTIFIER regex from constants instead of two copies.
- Simplify: fold VARIABLE_FIELDS into BINDING_SPECS.variableFields so each
  resource type is declared in one place (drops the parallel-map sync
  hazard and dead ?? fallbacks); drop a redundant register re-check and
  two identity .map() copies; flatten a pointless intersection type.
- Trim AI-verbose comments to their load-bearing line, drop transitional
  "Fix #N"/"Bugs #" test-comment prefixes, and de-duplicate rationale that
  was stated in multiple places.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
fieldOrigin (registry CLI) re-implemented the localOnly>value>resolve>user
cascade that computeOriginFromField (schemas/manifest) already owns — two
copies of one contract that had already drifted (truthy `resolve` vs
`resolve !== undefined`). Export computeOriginFromField and have fieldOrigin
delegate to it, keeping only its wrapper role (trust a synced manifest's
stamped origin, else derive). One rule, one place.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
add wrote plugin files under findServerRoot (server/api/backend) but called
registerPluginInServer(cwd, ...), which re-searched for the entry from cwd
with candidates covering only server/, src/, and the repo root. For an app
under api/ or backend/, wiring found no entry and silently fell back to
printed instructions whose import path (relative to the server root) didn't
match — so auto-wire failed for two of its own supported layouts.

Pass the resolved serverRoot into registerPluginInServer and look up entry
candidates within it; the printed path is rebased to cwd for display. Adds a
server-register test covering the api/ subdir, src/, idempotency, and the
skip fallbacks.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
The rebase's conflict resolution kept the pre-rebase lockfile, which
predates main's new `size-sensor` pnpm override — so `pnpm install
--frozen-lockfile` failed with ERR_PNPM_LOCKFILE_CONFIG_MISMATCH, breaking
every install-dependent CI job. Regenerate the lockfile against the merged
package.json (scratch workspace excluded so it stays out of the committed
lockfile). Verified with a frozen install.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
- Replace the internal staging workspace host in the databricks.yml golden
  fixtures with a neutral example.cloud.databricks.com.
- Replace the internal environment codename used as a test profile name in
  workspace-picker.test.ts with a generic "my-profile".

Fixtures/tests otherwise use placeholders only; no emails, tokens, or real
resource IDs.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
… add and doctor

Extract the deploy-config binding shape into a shared module
(packages/shared/src/cli/deploy-config.ts): the databricks.yml/app.yaml
file-name constants, the AppYamlEnvEntry/ResourceBinding shapes, and the
inverse bindingToNode/bindingTypeOf pair.

'appkit add' (registry/config-writer) generates this deploy config and
'appkit doctor' (doctor/bundle) reads and validates it. Sharing the shape
plus the encode/decode pair keeps the writer from drifting off the reader.
No behavior change.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
The import de-dup keyed on the module path, so an existing import from the
same path under a different binding (e.g. a type or renamed import) suppressed
the plugin import while the array still gained `exportName()` — emitting a
server that references an unimported symbol yet reporting status: wired.
De-dup on the local binding (default/namespace/named specifiers, aliases
resolved) instead. Adds regression tests for the different-binding and
already-bound cases.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
…guard

The already-bound case seeded the pre-existing import from the same path being
registered, so it passed under both the old path-based and new binding-based
de-dup. Seed it from a different path so the old logic would add a conflicting
second import and fail — the test now actually guards the fix.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
…p reasons

The resource picker fell through to free-text on any workspace-listing
failure: listWorkspaceResources swallowed the error, and its SDK client only
used a profile when --profile was passed — so it couldn't reach a workspace
configured only via the app's .env. Now:
- resolve the profile from the app's .env DATABRICKS_CONFIG_PROFILE (via the
  shared dotenv parseEnv) when --profile is absent, and use it for the picker,
  binding-value collection, and bundle validate;
- surface the listing-failure reason and point at --profile/DATABRICKS_CONFIG_PROFILE
  instead of reporting the workspace as empty;
- print why server auto-registration was skipped when falling back to the
  manual snippet.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant