* initial proof-of-concept for data apps
POC 2
POC 3
POC 4
Revert "POC 4"
This reverts commit 1f62e574dc.
POC 5
POC 6
* Expose all SDK components to child realm
* Setup app publishing and serving
* Refactor distortion helpers
* Adjust skill
* Add useQuestionQuery SDK hook (#74900)
Co-authored-by: Timofei <timofei@metabase.com>
* Mock METABASE_EMBEDDING_SDK_BUNDLE for data app sdk hooks/components
* Adjust skill for TS project
* Update skill to clearly mention TS
* Add blocked APIs list to skill
* Prototype schema exports and typed hooks for custom data apps (#74978)
* add prototype for schema generation
* add formatting for generated schemas
* add skill file
* useQuestionQuery should take only ID based on @sanex3339 feedback
* add segments, measures, tables and useMetabaseQuery hook
* Add useMetabaseQuery to endowments
* Fix queryDataset
---------
Co-authored-by: Timofei <timofei@metabase.com>
* Add ability to update app bundle
* Simplify hook
* Add spinner instruction
* Mode Data Apps to OSS
* Instruct to not not use blocked event handlers
* Improve API, add styles isolation, add router components (#75418)
* Prepare skills for data app template + adjust router components (#75490)
* Fixes after rebase
* Move migration to v63
* Move migration to v63 #2
* Fix config.edn
* Fix BE tests
* Fix team name
* Fix duplicated config
* Adjust version text in skill
* Fix data app bundle uploading after legacy client removal
* Remove `actions` from config edn
* Fix failing test
* Fix failing test #2
* Add custom actions for data apps (#75627)
* Fix imports
* Fix type annotations
* Adjust migration
* Adjust permissions
* Fixed iframe security
* Data Apps publishing via remote repository POC (#75702)
* Update gitignore with data-app.html
* [PoC] Type safety and modularity improvements for custom data apps (#75381)
* allow separate libraries in schemas
* add metric type checking
* add collections parameter
* check that segments and measures belongs to table in useMetabaseQuery
* require less type information for measures
* require less type information for segments
* use metric dataset endpoint when querying measures in metrics
* add validation for breakout buckets
* fix filter operation type scoping
* expose filter type helper for nicer autocompletions
* update skill file
* add breakout schema helper for better type errors
* update breakout schema helper to accept binning and bucket
* move fields that are not used in compile and runtime to comments to shorten type errors
* enforce data truthfulness in skill
* enforce semantic layer richness in skill
* handle dimension key collisions mapped from many tables
* update skill description
* prevent hallucinated measures and breakouts
* fix field serialization
* add guidance on using filter matcher
* add skill guidance on long results and units
* accept query object in StaticQuestion and InteractiveQuestion
* extract use-metabase-query to multiple files
* update skill file to use sdk question by default
* update skill file to avoid hallucinated derived calculations
* add new functions into sandbox
* kondo and cljfmt lint fixes
* update CSP rules to allow style tags for data apps
* address review feedback on scoping and metric filters
* address code review feedback
* Add aggregations support for breakouts in query hooks (#75723)
* add aggregations support
* add additional aggregation helpers
* Add metrics support in useMetabaseQueryObject hook (#75728)
* add metric queries
* indicate metric support in skill file for useMetabaseQueryObject
* add new version for release
* fix query dataset tests
* update release workflow temporarily
* update skill description
* update skill to use more generic examples
* hotfix: allow esm usage in vite dev mode
* skill file update for table caveat
* update sandbox
* fix type errors in useMetabaseQuery
* add v0.63.2 to release workflow
* update skill file to use the 63-data-apps version
* Move data apps to the EE directory and under `data-apps` token feature (#75871)
* remove skill file alias
* Add error handling for data apps
* Add spinner while iframe is loading
* Add OSS/no token upsell banner placeholder for data apps (#75893)
* ensure a fixed height is set if container has height
* Allow including everything in data and metric library in schema endpoint (#75944)
* add filter for including library and data collections
* address review on skill file
* remove id field from rendered comment
* remove useQuestionQuery and useMetricQuery hook
* Improve env files lookup in skills (#75955)
* Allow `unsafe-eval` for data app iframe (#76004)
* Fix lint
* Fixes after rebase
* Fix duplicated test
* Improve Providers and typesafety (#75934)
* chore: reformat single expression functions to arrow
* Use entity id in questionCollections when generating typed schemas (#76230)
* allow using entity ids for question collection filter
* guide the agent towards using entity ids
* add failing test for invalid entity id
* Make custom viz work in data apps (#76237)
* Allow blob: images in data-app CSP (EMB-1923)
SDK custom visualizations load their icon assets via blob: URLs (to avoid
the cross-origin auth issue), but the /embed/data-app/ route allowed
'unsafe-eval' without the blob: scheme in img-src, so icons were blocked
under a production CSP. Add blob: to img-src scoped to the data-app iframe
route only; the main app img-src stays strict.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* EMB-1880: support for `providerProps` in data apps
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Reduce schema bloat in dimensions of generated metrics when many tables are involved (#76003)
* Add useQuestionQuery SDK hook (#74900)
Co-authored-by: Timofei <timofei@metabase.com>
* Prototype schema exports and typed hooks for custom data apps (#74978)
* add prototype for schema generation
* add formatting for generated schemas
* add skill file
* useQuestionQuery should take only ID based on @sanex3339 feedback
* add segments, measures, tables and useMetabaseQuery hook
* Add useMetabaseQuery to endowments
* Fix queryDataset
---------
Co-authored-by: Timofei <timofei@metabase.com>
* [PoC] Type safety and modularity improvements for custom data apps (#75381)
* allow separate libraries in schemas
* add metric type checking
* add collections parameter
* check that segments and measures belongs to table in useMetabaseQuery
* require less type information for measures
* require less type information for segments
* use metric dataset endpoint when querying measures in metrics
* add validation for breakout buckets
* fix filter operation type scoping
* expose filter type helper for nicer autocompletions
* update skill file
* add breakout schema helper for better type errors
* update breakout schema helper to accept binning and bucket
* move fields that are not used in compile and runtime to comments to shorten type errors
* enforce data truthfulness in skill
* enforce semantic layer richness in skill
* handle dimension key collisions mapped from many tables
* update skill description
* prevent hallucinated measures and breakouts
* fix field serialization
* add guidance on using filter matcher
* add skill guidance on long results and units
* accept query object in StaticQuestion and InteractiveQuestion
* extract use-metabase-query to multiple files
* update skill file to use sdk question by default
* update skill file to avoid hallucinated derived calculations
* add new functions into sandbox
* kondo and cljfmt lint fixes
* update CSP rules to allow style tags for data apps
* address review feedback on scoping and metric filters
* address code review feedback
* Add aggregations support for breakouts in query hooks (#75723)
* add aggregations support
* add additional aggregation helpers
* Add metrics support in useMetabaseQueryObject hook (#75728)
* add metric queries
* indicate metric support in skill file for useMetabaseQueryObject
* add new version for release
* fix query dataset tests
* remove useQuestionQuery and useMetricQuery hook
* Data Apps publishing via remote repository POC (#75702)
* avoid duplicating dimension fields
namespace dimensions by table
add dimension id to mapping
pick from table fields for only allowed dimensions
fix incorrect type guards
allow querying metric using question as source
remove createMetabaseQuery from public interface
add sourceFieldId to metric
* doc generation type fixes
---------
Co-authored-by: Timofei <timofei@metabase.com>
* Add allowed hosts handling for data apps (#76094)
* Update version
* Properly include data app changes in `pull changes` modal (#76294)
* Move app template into main repo, handle skill versioning flow (#76288)
* feat(data-apps): allow passing down a card with viz settings (#75993)
support card prop in sdk question component
* Rename data app env variables and read them only in dev mode (#76604)
* Add custom data apps layout with sliding toolbar (#76600)
* Remove unused helper
* Dev env Sandboxing, Human Readable Errors and Copy Helper (#76597)
* Revert wrong rebase changes
* Improve creating filtering interfaces in custom data apps (#76401)
* update skill files to guide filter usages
* update skill file to prevent filtering mistakes
* move filtering ui patterns into a separate file
* do not treat mb_replace_me as valid creds
* use combobox for filters
* pass title false to sdk question
* combobox trigger before typing
* Add missing exported types
* Apply visualizations and visualization settings from SDK package (#76659)
* ask the agent to pull possible visualizations
* ask the agent to use card prop
* teach the agent when to use viz settings
* Support query-only SDK cards
* Remove cross references between data apps skills (#76661)
Remove data app skill cross references
* Move Data App Dev Vite App setup into SDK package (#76695)
* Don't navigate on ad-hoc question chart clicks (#76751)
* Fix sandboxing crashes and update skill file mistakes (#76858)
Fix data app sandboxing and skill guidance
* Block form submissions in data apps via CSP (#76882)
* Fix SDK theming hover text colors in generated data apps (#77083)
update guidance on colors
* Remove SDK question query prop for data apps use cases (#77080)
* Remove SDK question query prop
* update tests to use card prop instead
* Adopt Lib.createTestQuery for table sources when querying in data apps (#75957)
* Add base metabase-lib query implementation
* switch to Lib.createTestQuery DSL
* Load query metadata at runtime for metabase-lib instead of hard-coding (#76874)
* load query metadata dynamically
* accept id-only source references
* Wait for SDK login before resolving query metadata
* refactor: use useAsyncFn to handle states
* pass source-name to createTestQuery
* Add metric aggregations for useMetabaseQuery via Lib.createTestQuery (#76947)
* add metric sources for useMetabaseQuery via Lib.createTestQuery
* use MetabaseQueryObject instead of opaque type marker
* Fix metric schema and join source validation
* add metric library filtering to skill files
* make metric an aggregation expression instead of join sources
* mark tableId as never in joined fields
* export public MetabaseQueryObject type
* use camelCase for source name and field id
* Add initial data app e2e setup (#77101)
* Add sorting and fields to useMetabaseQuery via Lib.createTestQuery (#76966)
* add sorting, limiting and fields to useMetabaseQuery
* refactor: cleanup sorting and aggregations
* Fix metric order by validation
* Support saved questions for useMetabaseQuery via Lib.createTestQuery (#77073)
* add saved questions as useMetabaseQuery input
* remove extra operators for saved cards
* Add unit fe/be tests for data apps (#76979)
* Fix lint
* Use modern api in query-dataset.ts
* Fix BG color
* Fix execute-action tests
* Update SDK-data-apps version
* Move migrations to v64 folder
* Adjust router components to use metabase/router
* Allow opening a data app for any signed user
* Rename data app skills (#77266)
* Update DataAppLayout.tsx `go back` button logic
* Harden data apps skills and action validation errors (#77261)
* harden data apps skills and actions error validation
* do not display models as questions in data apps
* clarify action result types
* Agents should report errors when models, actions or questions cannot be schema generated (#77260)
* return an error if models or saved questions are invalid
* return an error if actions are invalid
* update skill file guidance on model generation
* fix model boundary
* Add data apps admin UI + routing/sandboxing/isolation e2e (#77271)
* Rename data-apps paths to /apps (#77298)
* Include all action-included models in the typed schema when a mutation is specified (#77291)
* add includeModels=true flag to include all models
* omit model columns and models without actions
* remove extra metadata
* Adjust admin UI for Data Apps (#77420)
* Add more e2e/unit tests for Data Apps (#77524)
* Update bun.lock
* Fix errors after rebase
* Parallelize dts rollup (#77663)
* revert EMB-1840 measures support in metric dataset endpoint
* chore: add data app module (#77705)
* test ci
* apply code review suggestion
* Setup separate entries for client and node dev data app entries (#77687)
* Fix type casts lint errors (#77696)
* Adjust data apps setup + add proper analytics setup (#77604)
* Extract common and scope helpers + tests for typed schema module (#77403)
* refactor: extract common and query param helpers
* refactor: use kebab-case for query params
* refactor: cleanup schema mapping
ci: fix module config
* refactor: rename schema scope helpers to scope.clj
* refactor: extract scope_test.clj
* Extract typed schema rendering helpers to render.clj (#77703)
* extract schema render function
* move runtime compact test out
* remove javascript and json endpoints from schema generator
* refactor: cleanup javascript.clj and render.clj
* Extract typed schema generators for tables and questions (#77729)
* refactor: extract schema generator modules
* remove question-details from model-schemas
* fix: batch queries in table and question schema generator
* remove N+1 in typed schema field table lookup
* Extract typed schema generators for models and actions (#77739)
* extract model schema generator
* batch model action querying to avoid n+1 queries
* Extract typed schema generator for metrics (#77806)
* Extract typed schema generators for models and actions (#77739)
* extract model schema generator
* batch model action querying to avoid n+1 queries
* extract metric schema generator
* test: simplify fields in metric_test fixtures
* refactor: clean up metric schema generation
* remove n+1 from metric field dimension read
* disable default temporal dimension breakout
* surface errors if metric schema cannot be built
* revert metrics/api.clj to master to remove outdated comments
* Fix routes
* Take a data app's slug from its directory name (#77688)
* Merge data-apps specific migrations into one
* ci: fix failing backend test and cljfmt lint
* Add a beta badge in data apps admin page heading (#77877)
marks the data app as beta
* Revert "Merge data-apps specific migrations into one"
This reverts commit 383a5fa28d.
* Revert "Revert "Merge data-apps specific migrations into one""
This reverts commit 948d904b62.
* ci: actionlint fix
* Fix data apps route
* Use getSubpathSafeUrl in attachIframeUrlMirror
* Fix comment
* Do not send apps with sync error for token check
* Add data app listing sidebar section for non-admins (#77977)
* show data apps in sidebar for non-admins
* add app icon to data apps
* use available=true to filter no sync error and enabled apps
* fix navbar keys in data apps
* test: use non-admin user to assert non-admin response
* fix toucan query in data apps listing endpoint
* re-export entity detail helpers and result column to metabot core
* open data apps in new tab from the sidebar
* Prevent race conditions in responses of useMetabaseQuery hook (#78023)
prevent stale responses in useMetabaseQuery
* ci: remove tools.util from module declaration
* Merge migrations into a single one
* reset release-embedding-sdk.yml to master
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Update to typescript 7.0 (beta)
* Update moduleResolution to bundler
* Switch to tsgo in scripts
* Remove unneeded ts-expect-error
* Add aliases for sdk framework imports
* Add aliases for node_modules types
* Resolve csstype to 3.2.3
* Resolve codemirror types since the package does not export .d.ts files
* Be more precise in SegmentedControl type
* Fix types in CssVarsDeclarationPlugin
* Use correct type for keys in WeakMap
* Use declare context instead of context! to narrow context type
* Import from top-level mantine package
* Remove unused card prop
* Add missing props
* Fix import
* Be more precise in SegmentedControl type
* Convert CardRenderer to TypeScript
* Remove unused @ts-expect-error
* Remove infinite type instantiations
* Do not use declare in type
* Set module to preserve for bundlers
* Add types for tinykeys
* Switch to TypeScript 6.0.3
* Switch back to tsc
* Resolve ts config path directly
* Update @microsoft/api-extractor
* Fix Modal types
* Remove redundant ext-apps ambient module shim
Under the old moduleResolution 'node', the package exports field was
ignored, so @modelcontextprotocol/ext-apps/react could not be resolved
to its real types and this ambient shim re-exported them from the deep
dist path.
With moduleResolution 'bundler', the subpath now resolves natively via
the exports map, while that same exports map blocks the deep dist import
the shim relied on. The shim therefore resolved to nothing and shadowed
the real module, breaking fe-type-check. Deleting it lets the native
types take over.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Remove deprecated baseUrl from next-15 host app tsconfigs
next build runs a full type-check using the repo's TypeScript (now 6.0),
which hard-errors (TS5101) on the deprecated baseUrl option. Both
next-15 production-bundle e2e jobs failed on it.
baseUrl is unnecessary here: the apps only use @/* path mappings and
real package imports, and under moduleResolution 'bundler' paths
resolve relative to the tsconfig location without baseUrl. This also
matches the modern create-next-app scaffold, which no longer emits it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Remove ts 7 from this branch
* Upgrade typescript-eslint to a version compatible with TS6
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Convert `app-embed` to TypeScript
Part of the Accelerate TypeScript Migration project.
Closes DEV-1516
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Use bun in package.json
* Update workflows
* Update more workflows
* Kick ci
* Fix yaml
* Fix type errors
* Compat with snyk
* Fix jest resolutions
* Add another esm package
* Fix aliases and next js missing deps
* Update next js embed tests
* Try fix embed test config
* Fix
* Fix requires in e2e support
* Use bun not yarn in embed bin
* Revert config changes
* Use bun in more scripts and use prepare-release action
* Replace more yarn usages
* Remove overrides from bunfig move to package json
* Revert e2e support changes
* Add back comment
* [ci skip] comment typo
* Update bun lock
* Update to reference bun
* Update to reference bun
* Update license disclaimer
* Tidy tests
* Update to use bun in workflows
* Change aliases
* Update codemirrort selector
* Tidy license generation
* Remove .only
* Fix yarn references
* Checkout .github for release scripts
* Update lockfile
* Re-add package
* update release bunlock
* fix checkout check
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Convert plugin system to deferred initialization with undo capability
* fixed wrong rebase
* better initialization
* .
* fixed unit tests
* .
* duh
* added back whitelabel overrides for SDK
* fixup
* fixed unit test
* cleanup
* [Admin settings] Show help documentation to connect DB in a side panel
* Add unit tests
* Add more tests / tweaks
* Add e2e tests
* Make sure help content is not rendered if not available
* Self-review tweaks
* Use useDocsUrl in DatabaseHelpSidePanel.tsx
* Use aside element for the side panel
* Use await findBy when possible
* Use JSDoc / Remove unnecessary comment
* Add some style to blockquote
* Add test file for useHelpLink
* Fix failing test
* Add analytics event
This also moves the CLJS output out of `frontend/src/cljs{,_release}` to
`target/cljs_{dev,release}`. If that causes problems for FE tooling I
don't know about, it could be moved back.
Hot reloading:
CLJS changes are hot-reloaded into the running app by Shadow CLJS - no
action is needed. Note that the UI will not refresh; you usually have to
click around so the updated MLv2 code gets executed.
Usage of the CLJS REPL:
- Open the browser (you should see `shadow-cljs: #N ready!` in console)
- Connect to nrepl on 50655
- "Select" the `:app` build;
- https://shadow-cljs.github.io/docs/UsersGuide.html#_editor_integration
- Conjure: `:ConjureShadowSelect app`
- Plain REPL: Eval in CLJ mode `(shadow/repl :app)`
- Consider running `yarn build-hot:cljs` and `yarn build-hot:js` in
separate terminals; then when `yarn build-hot:js` crashes you don't
have to reconnect your REPL.
- The REPL works beautifully, even loading files outside the :entries
of the shadow-cljs build like tests!
"Stale" error in browser:
- This can happen if the webpack bundle is created before the CLJS is
done building.
- If you ran `yarn build-hot:cljs` and `:js` separately, let `:cljs`
finish compiling first.
- If you see this otherwise: (a) contact Braden so it can be fixed; and
(b) either restart `build-hot:js` or force webpack to rebuild by
hitting http://localhost:8080/webpack-dev-server/invalidate
Yarn commands:
`dev`, `dev-ee`, `build-hot`, and `build-static-viz:watch` all now wait
for the CLJS build to be complete by watching for files to be generated.
They should work the same from a user's POV.
(Since the webpack build no longer watches the CLJS output,
it won't notice the changes and immediately rebuild after the first
build. This avoids errors on the clean build, or "stale build" messages
in the app from Shadow CLJS.)
Fixes#32674.
Previously, JS files isn't picked up by VS Code, so they'll have limited
intellisence. Also, Jest and Cypress has conflicting names e.g.
`describe` or `expect`. So, this commit attempt to resolve that as well.
* add basic typescript configuration
* convert a tiny hook use-debounced-value to typescript
* allow any for now?
* update terser and static viz configs
* adjust config
* change extension of QuestionActivityTimeline from jsx to js to match jest config
* adjust config
* skip failing tests
* Fix missing question method in unit tests (#18314)
* review fix
* fix
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
