feat(cli): add 'appkit add' and 'appkit registry' commands - #462
feat(cli): add 'appkit add' and 'appkit registry' commands#462MarioCadenas wants to merge 31 commits into
Conversation
|
📦 Bundle size reportCompared against
|
| dist | raw | gzip |
|---|---|---|
| JS (runtime) | 869 KB (-553 B) | 303 KB (-89 B) |
| Type declarations | 315 KB (+379 B) | 109 KB (+212 B) |
| Source maps | 1.7 MB (+326 B) | 566 KB (+137 B) |
| Other | 11 KB | 3.7 KB |
| Total | 2.9 MB (+152 B) | 982 KB (+260 B) |
Per-entry composition (own code — deps external (as shipped))
| Entry | Initial (gz) | Lazy (gz) | Total (gz) | node_modules (min) | Own code (min) |
|---|---|---|---|---|---|
. |
88 KB (+18 B) | 2.5 KB | 91 KB (+18 B) | external | 288 KB (+48 B) |
./beta |
49 KB (+7 B) | 456 B (-2 B) | 49 KB (+5 B) | external | 143 KB (+48 B) |
./type-generator |
21 KB (+41 B) | 0 B | 21 KB (+41 B) | external | 61 KB (+48 B) |
Chunks:
| Entry | Chunk | Load | Size (gz) |
|---|---|---|---|
. |
index.js |
initial | 84 KB |
. |
utils.js |
initial | 4.0 KB |
. |
remote-tunnel-manager.js |
lazy | 2.5 KB |
./beta |
beta.js |
initial | 33 KB |
./beta |
stream-manager.js |
initial | 5.8 KB |
./beta |
wide-event-emitter.js |
initial | 3.2 KB |
./beta |
databricks.js |
initial | 3.0 KB |
./beta |
configuration.js |
initial | 2.1 KB |
./beta |
service-context.js |
initial | 1.3 KB |
./beta |
client.js |
initial | 434 B |
./beta |
client-options.js |
initial | 219 B |
./beta |
supervisor-api.js |
lazy | 192 B |
./beta |
databricks.js |
lazy | 141 B |
./beta |
index.js |
lazy | 123 B |
./type-generator |
index.js |
initial | 21 KB |
@databricks/appkit-ui
npm tarball (packed): 342 KB (+351 B) — gzipped download (dist + bin; excludes release-only docs/NOTICE).
| dist | raw | gzip |
|---|---|---|
| JS (runtime) | 390 KB | 130 KB |
| Type declarations | 228 KB (+412 B) | 83 KB (+360 B) |
| Source maps | 753 KB | 248 KB |
| CSS | 16 KB | 3.3 KB |
| Total | 1.4 MB (+412 B) | 465 KB (+360 B) |
Per-entry composition (consumer bundle — deps bundled, peerDeps external)
| Entry | Initial (gz) | Lazy (gz) | Total (gz) | node_modules (min) | Own code (min) |
|---|---|---|---|---|---|
./js |
5.3 KB | 49 KB | 55 KB | 208 KB | 14 KB |
./js/beta |
20 B | 0 B | 20 B | 0 B | 0 B |
./react |
432 KB | 49 KB | 480 KB | 1.3 MB | 175 KB |
./react/beta |
1.0 KB | 0 B | 1.0 KB | 0 B | 1.9 KB |
Chunks:
| Entry | Chunk | Load | Size (gz) |
|---|---|---|---|
./js |
index.js |
initial | 5.2 KB |
./js |
chunk |
initial | 120 B |
./js |
apache-arrow |
lazy | 49 KB |
./js/beta |
beta.js |
initial | 20 B |
./react |
index.js |
initial | 430 KB |
./react |
tslib |
initial | 2.1 KB |
./react |
apache-arrow |
lazy | 49 KB |
./react/beta |
beta.js |
initial | 1.0 KB |
|
This pull request has had no activity for 23 days and has been marked as stale. It will be closed in 7 days if there is no further activity. Add a comment, push a commit, or apply the |
1bf0f75 to
bf0bf52
Compare
🤖 AppKit PR bot🔬 Run evalsStart an eval for this PR from the evals-monitor app: Go to Evals Monitor → 📦 Try this PR's app templateScaffolds a new app from this PR's SDK build. Run it in any folder (requires the GitHub CLI — gh run download 31817420861 -R databricks/appkit -n appkit-template-0.60.0-pr.f677545-feat-registry-cli-462 -D appkit-pr-462 \
&& unzip -o "appkit-pr-462/appkit-template-0.60.0-pr.f677545-feat-registry-cli-462.zip" -d "appkit-pr-462" \
&& databricks apps init --template "appkit-pr-462"The template pins |
b2816be to
15bb1be
Compare
Add a shadcn-namespaced component registry workflow to the AppKit CLI: - 'appkit add <component...>' ensures the @AppKit registry namespace in the consumer's components.json, then delegates to 'shadcn add @appkit/<name>'. - 'appkit registry list' enumerates components from the registry index. Registry components import primitives from @databricks/appkit-ui (npm peer), so they stay in sync with the installed AppKit version and design tokens. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Serve the registry directly from the public GitHub repo over raw.githubusercontent.com instead of a placeholder host — no separate hosting infra needed. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Fetch registry items ourselves (token-aware) and hand a local file to 'shadcn add', rather than relying on a shadcn namespace — so we control the auth headers and the internal/private repo works today. - Token resolved from gh auth token, then APPKIT_REGISTRY_TOKEN / GITHUB_TOKEN / GH_TOKEN. - Private fetch uses the GitHub Contents API (Accept: raw); falls back to raw.githubusercontent.com when no token / once the repo is public. - 'appkit registry list' is token-aware too. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
AppKit registry items are self-contained (import from @databricks/appkit-ui, no shadcn @/ aliases or registryDependencies), so shadcn's alias resolution isn't needed. Write item files to their target path directly and install npm deps with the detected package manager — no components.json required. - Targets resolved under src/ when present. - --force to overwrite existing files (refuses by default). - Warns on any registryDependency rather than silently dropping it. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
'appkit add' is typically run from the repo root, where the frontend lives in a client/ subdir. Detect the frontend root (dir with components.json or src/, including common client/frontend/web/app subdirs) and write components under <frontend>/src/<target>. Install deps into the nearest package.json (single root package.json in the AppKit app layout). Add --cwd to override. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Server plugins can now be distributed through the same registry as UI components. 'appkit plugin add <name>' fetches a plugin item, writes it to plugins/<name>/ (verbatim targets, not under client/src), installs npm deps, runs 'plugin sync' to register it, and prints the createApp snippet plus any required env vars. - Extract shared fetch into registry/client.ts (reused by add + plugin add). - Detect plugin items by the presence of manifest.json. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
…server/plugins Remove the separate 'appkit plugin add' command. 'appkit add' now detects the item kind (plugin = has manifest.json) and routes: • UI components → <frontend>/src/components/appkit/ • server plugins → <server>/plugins/<name>/ (server/ subdir detected) then installs deps, runs plugin sync for plugins, and prints next steps. A single call can mix components and plugins. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
After placing a plugin and running sync, 'appkit add' edits the server entry
(server/index.ts etc.) to add the import and insert the plugin into the
createApp({ plugins: [...] }) array, reusing the ast-grep machinery from
'plugin sync'. Best-effort and safe:
• idempotent — skips if the plugin is already registered
• only edits the standard plugins:[...] array literal; otherwise falls back
to printing the manual snippet
• --no-register opts out of the server edit
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
toPlugin exports are factories, so register as `hello()` (matching server(), analytics()), not a bare identifier. Insert before the first array element with its indentation so multi-line plugins arrays keep their formatting instead of jamming onto the opening-bracket line. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
- 'appkit registry list' shows a VERIFIED ✓ column (from item meta.verified) with a --verified filter; JSON output gains a top-level verified field. - Colorize list + add output with picocolors (green created/registered/✓, yellow updated/warnings, red errors, dim hints). Padding is applied before coloring so columns stay aligned; picocolors no-ops on non-TTY/NO_COLOR. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Derive a friendly kind per item: a manifest.json marks a plugin; otherwise map the shadcn registry:* type (component/hook/lib/theme/...). Adds a colorized TYPE column to the table and a 'kind' field to --json output. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Adds a search subcommand so agents (and people) can match intent to items: matches all query terms against name, title, description, derived kind, and item categories. Shares the token-aware index fetch + table/JSON rendering with 'registry list'. --verified and --json supported. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Match the registry rename (registry.json name: appkit -> databricks-appkit) so `appkit add` writes the @databricks-appkit namespace into components.json and stripNamespace() strips the same prefix. Co-authored-by: Isaac
…l resource env vars - Plugins declaring registryDependencies now pull their full dependency graph on 'appkit add' (previously ignored on the plugin branch and never resolved transitively). - declaredEnvVars walks optional resources too, not just required. - Add regression tests for the registry add resolver (dir had none). Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
The registry-distributed 'add' and 'registry' commands are executable but
hidden from top-level --help while the feature is still in development.
Registered with { hidden: true } — no behavior change, only help visibility.
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
…, workspace picker Folds the full registry resource workflow into the base: - registry info + requirements table on add - .env / .env.example reconciliation (origin-classified from the authored contract: platform fields skipped, static defaults pre-filled, already-set values preserved and fed to deploy config) - app.yaml + databricks.yml resource-binding generation (verified against golden fixtures; writes target-variable values even when binding/var pre-exist) - workspace picker: SDK-backed listing (auto-paginating) with a type-to-filter autocomplete over the full list; flat types via the SDK, parent-context types (volume/uc_function/secret/vector_search_index) via CLI drill-down; free-text fallback when the workspace can't be reached Relocates the Databricks SDK facade from packages/appkit/src/workspace-client to packages/shared/src/workspace-client so both appkit and the CLI (in shared) reach the SDK through one sanctioned import site. appkit's workspace-client becomes a thin re-export (its 47 importers are unchanged); the noRestrictedImports allow-list points at the new location. Adds a `profile` option to WorkspaceClientOptions. Adds @databricks/sdk-experimental + yaml deps to shared. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Scratch/test apps placed under apps/scratch/ resolve their @databricks/* dependencies against the monorepo (workspace-linked via the new apps/scratch/* entry in pnpm-workspace.yaml) and are skipped by knip (apps/** is already in ignoreWorkspaces). Contents are gitignored; .gitkeep keeps the dir tracked so the location exists for everyone without committing any scratch app. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Registry items are untrusted remote data. Close the review findings on the resource-aware `appkit add` path: - Path traversal: writeItemFile routes through resolveWithinBase, which rejects absolute/escaping file targets (arbitrary write -> RCE). - Export-name / import-path injection: validate against a JS-identifier / clean-module-path allowlist before editing the user's server source. - Dependency injection: partitionDeps allowlists name@version specs, rejects flag-like and URL/git specs; install uses a `--` separator. - .env newline injection: isSafeEnvValue rejects CR/LF values that would inject an extra .env line (e.g. host override -> exfil). - CLI arg injection: drill-down refuses `-`-prefixed parent picks so a crafted resource name can't become a databricks CLI flag. - Integrity gate: add refuses items the registry index doesn't mark verified unless --allow-unverified is passed. Correctness/perf alongside: - Postgres binding fields with no env (project/branch/database) are now collected so databricks.yml target variables are assigned. - Secret drill-down composes scope/key instead of dropping the scope. - Flat picker caps pagination at MAX_PICKER_RESULTS; Genie listSpaces follows next_page_token. - resolveItems fetches each dependency level concurrently. - add lazy-loads server-register (@ast-grep) and env-writer (SDK) so unrelated CLI commands don't pay their load cost. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Follow-up to the registry-add hardening, from a second review pass: - Verified gate now covers the full resolved set (requested items plus their transitive registryDependencies), not just the top-level names. A verified item could otherwise declare an unverified registryDependency whose code gets written/installed/wired without passing the gate. The gate moved after resolveItems (fetching item JSON is read-only; nothing is written until after the check) and still fails closed on an unreadable index. - Env-var NAMES from a manifest are now validated with isValidEnvName (^[A-Za-z_][A-Za-z0-9_]*$) before reaching .env or app.yaml. The prior fix guarded the value but not the key, so a field named "PORT=x\nDATABRICKS_HOST=..." could still inject a second .env line. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
The verified gate checks items.map(i => i.name), but item.name came straight from the fetched JSON body — untrusted remote data never checked against the key the item was fetched under. A verified:false item published at key "evil" could self-report "name": "analytics" (a verified name) and slip past the gate, then have its own files written and wired into the server. Same spoof could point item.name at another item's plugin dir. Pin item.name to the fetch key inside resolveItems, so the trustworthy identity (what the user requested / a parent listed / the index keys verified on) is what every downstream consumer sees — the gate, the plugins/<name> write path, and dedup. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Address the remaining review findings on appkit add: - Item names (user refs and untrusted registryDependencies) are now slug-validated via isValidItemName (^[A-Za-z0-9._-]+$, rejecting `.` and `..`) before use. A name is both the fetch path (public/r/<name>.json) and the on-disk plugins/<name> dir, so a crafted ref like "../../attacker/repo/payload" could otherwise redirect the fetch (SSRF) or escape the destination dir. Rejecting at the source also keeps control chars (ANSI escapes) out of any printed name. - Fetch the verified index and resolve the item graph concurrently — the two are independent round-trips, previously serialized (~1 RTT on every add). The gate still evaluates verification before any write. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Final review cleanups: - registry info now validates its ref with isValidItemName before fetching, matching the add guard, so `/` or `..` can't redirect the request to another path in the repo. - Reword code/test comments: the fetch templates interpolate the name into the URL path only (same host), so a bad name is a repo-path redirect, not cross-host SSRF. Corrected to avoid overstating it. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Cleanup pass over the registry CLI (no behavior change): - Reuse: parseEnv now delegates to dotenv.parse (the parser the app loads .env with at runtime); extract registryAuthHeaders as the single source for the GitHub auth/Accept headers (was duplicated 3x); share the JS_IDENTIFIER regex from constants instead of two copies. - Simplify: fold VARIABLE_FIELDS into BINDING_SPECS.variableFields so each resource type is declared in one place (drops the parallel-map sync hazard and dead ?? fallbacks); drop a redundant register re-check and two identity .map() copies; flatten a pointless intersection type. - Trim AI-verbose comments to their load-bearing line, drop transitional "Fix #N"/"Bugs #" test-comment prefixes, and de-duplicate rationale that was stated in multiple places. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
fieldOrigin (registry CLI) re-implemented the localOnly>value>resolve>user cascade that computeOriginFromField (schemas/manifest) already owns — two copies of one contract that had already drifted (truthy `resolve` vs `resolve !== undefined`). Export computeOriginFromField and have fieldOrigin delegate to it, keeping only its wrapper role (trust a synced manifest's stamped origin, else derive). One rule, one place. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
add wrote plugin files under findServerRoot (server/api/backend) but called registerPluginInServer(cwd, ...), which re-searched for the entry from cwd with candidates covering only server/, src/, and the repo root. For an app under api/ or backend/, wiring found no entry and silently fell back to printed instructions whose import path (relative to the server root) didn't match — so auto-wire failed for two of its own supported layouts. Pass the resolved serverRoot into registerPluginInServer and look up entry candidates within it; the printed path is rebased to cwd for display. Adds a server-register test covering the api/ subdir, src/, idempotency, and the skip fallbacks. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
The rebase's conflict resolution kept the pre-rebase lockfile, which predates main's new `size-sensor` pnpm override — so `pnpm install --frozen-lockfile` failed with ERR_PNPM_LOCKFILE_CONFIG_MISMATCH, breaking every install-dependent CI job. Regenerate the lockfile against the merged package.json (scratch workspace excluded so it stays out of the committed lockfile). Verified with a frozen install. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
- Replace the internal staging workspace host in the databricks.yml golden fixtures with a neutral example.cloud.databricks.com. - Replace the internal environment codename used as a test profile name in workspace-picker.test.ts with a generic "my-profile". Fixtures/tests otherwise use placeholders only; no emails, tokens, or real resource IDs. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
5df17ae to
aab8a35
Compare
… add and doctor Extract the deploy-config binding shape into a shared module (packages/shared/src/cli/deploy-config.ts): the databricks.yml/app.yaml file-name constants, the AppYamlEnvEntry/ResourceBinding shapes, and the inverse bindingToNode/bindingTypeOf pair. 'appkit add' (registry/config-writer) generates this deploy config and 'appkit doctor' (doctor/bundle) reads and validates it. Sharing the shape plus the encode/decode pair keeps the writer from drifting off the reader. No behavior change. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
The import de-dup keyed on the module path, so an existing import from the same path under a different binding (e.g. a type or renamed import) suppressed the plugin import while the array still gained `exportName()` — emitting a server that references an unimported symbol yet reporting status: wired. De-dup on the local binding (default/namespace/named specifiers, aliases resolved) instead. Adds regression tests for the different-binding and already-bound cases. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
…guard The already-bound case seeded the pre-existing import from the same path being registered, so it passed under both the old path-based and new binding-based de-dup. Seed it from a different path so the old logic would add a conflicting second import and fail — the test now actually guards the fix. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
…p reasons The resource picker fell through to free-text on any workspace-listing failure: listWorkspaceResources swallowed the error, and its SDK client only used a profile when --profile was passed — so it couldn't reach a workspace configured only via the app's .env. Now: - resolve the profile from the app's .env DATABRICKS_CONFIG_PROFILE (via the shared dotenv parseEnv) when --profile is absent, and use it for the picker, binding-value collection, and bundle validate; - surface the listing-failure reason and point at --profile/DATABRICKS_CONFIG_PROFILE instead of reporting the workspace as empty; - print why server auto-registration was skipped when falling back to the manual snippet. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Adds a component/plugin registry workflow to the AppKit CLI. Items are fetched directly from the
databricks/appkit-registryrepo and written into the consumer's app — noshadcnand nocomponents.jsonrequired. Registry commands are wired up but hidden from--helpwhile the feature is still in development.New CLI surface
appkit add <item...>(top-level, the primary entry point) — adds UI components and/or server plugins from the registry. Auto-detects each item's type and routes it to the right place:<frontend>/src/components/appkit/<server>/plugins/<name>/, then runsappkit plugin sync --writeand registers the plugin in the server'screateAppcall (--no-registerto skip the edit).Frontend/server roots are detected from common monorepo layouts, so it can be run from the repo root. Transitive
registryDependenciesare resolved breadth-first (deduped, fetched concurrently per level), and npmdependenciesare installed with the detected package manager (pnpm/yarn/bun/npm).appkit registry list— enumerates items from the registry index, showing each item's kind (component/plugin/hook/…) and a marker for verified items (colorized output).appkit registry search <terms...>— filters items by name / title / description / type / category; all terms must match.appkit registry info <item>— shows an item's resource requirements plus its npm and registry dependencies (--jsonfor machine output).Resource-aware install
Server plugins declare the Databricks resources they need. On
add, the CLI:.env(and their names into.env.example);app.yamlanddatabricks.ymlresource bindings — to match, never clobbering existing entries;--yesruns non-interactively for agents/CI,--env KEY=VALUE(repeatable) pre-supplies values, and--profilerunsbundle validateafter writing;user_api_scopebefore deploy.Security hardening
Registry items are untrusted remote data, so
addgates and sanitizes them:--allow-unverifiedoverrides.name, so it can't claim a verified name to slip past the gate./,.., control chars).[@scope/]name[@range]specs are installed (blocks tarball/git-URL install-script RCE and flag/argument injection;--guard as defense in depth).Tests & supporting changes
analytics,lakebase) coveringadd,client,config-plan,config-writer,env-reconcile,env-writer,requirements,server-register, andworkspace-picker.schemas/manifest.ts: single source for field-origin derivation (used byrequirements).apps/scratch/: gitignored dir for local scratch apps; pnpm workspace/lock updates.Paired change
The registry itself is renamed to
@databricks-appkitin databricks/appkit-registry#1; this PR'sREGISTRY_NAMESPACEmatches it.