`entity_id` columns on Databases, Tables and Fields are new in #53076.
This PR uses `entity_id` on Fields as their `:ident` in the metadata,
rather than the concatenated names previously used.
The names were a bit too fragile, since occasionally people rename
Fields or Tables in the appdb to retain the identity of something that
got renamed in place in their data warehouse. (That isn't exactly
supported, but it's also not blocked.)
With random (or hashed and saved) `entity_id`s for Fields used as their
`:ident`s instead, references to fields will not need to be updated if
fields or tables are renamed but their `entity_id`s preserved.
**On memoization**
Since the `entity_id` is `NULL` for existing fields and they're only
being populated in the background, we need to populate them on demand
when reading metadata. Since fields depend on their tables and tables on
their databases, this meant a lot of extra appdb reads at first.
This change includes memoization added to `serdes/hydrated-hash`, such
that any table or database only gets read once, and memoized forever.
There's a big comment about the memory impact of this; in summary it
should be safe since this is on demand, and holds only DBs/tables
and not fields.
A later PR will add a background job as mentioned, that will populate
all the `entity_id` columns with hashed values. After that, the cache
will always be empty.
Note that this memoization is not going to work if a table or field is
renamed before the background job catches up! I consider this an
acceptable loss - renaming fields is an admin action that requires
editing the appdb. The guide for doing that can include making sure all
`entity_id` fields are populated first.
* fix: use standard user order sort for group list
* fixup! remove alba from user fixtures
* fixup! better user model creation
* fixup! remove test-user? filter
* fixup! remove added test and use exisiting sort test
* fix: allow password reset if google sso disabled
* fixup! correct arg type to keyword in message test
* fixup! remove empty line
* fixup! add public docstring
* fixup! add temp google client id
* Use NativeEditor.type in test
* Use NativeEditor.type in all tests that aren't
* Stop using .as(editor) for native editor
* Do not return anything from startNewNativeQuestion to avoid confusion
* Fix more tests
* Fix movetostart/movetoend
- Ctrl + A is "select all" on non-apple machines, but on apple machines it's same as "home"
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Error explicitly when sandboxing/impersonation configs conflict
* Fix tests
* Fix kondo warning, use consistent ns aliases
* Fix another test
* Cleanup, consistent aliases
* Add an extra check of *current-user-id*
* Wrap east/west filterBounds to [-180,180] in LeafletMap.jsx
* Split filters that cross the antimerdian into two parts in update-lat-lon-filter
If the bounds cross the 180th meridian, split the filter into two parts ORed togehter. One part covering the
[west, 180] range and the other covering [-180, east].
* Add tests for bounds crossing the antimerdian
* Rename longDelta to lngDelta to conform to standard leaflet abbreviation
* Review feedback: use shorthand notation for east and west filterBounds
* Add e2e tests
* Fix existing e2e test
realMouseDown expects to have x and y coords passed in an options map.
* Use common pinMapSelectRegion test helper in new e2e tests
* wip logic to handle 'personal' for collectionId when creating dashboards
* unit test for useCreateDashboardApi supporting 'personal'
* add e2e test for createdashboardmodal
* collection browser support for 'personal'
* some actual decent logic
* change jest tests to use root collcetion
* fix type error in stories
* fix create dashboard modal support for root
* yeah i shouldn't have commited a .only
* self review, better docs + removed un-used type
* typo
* another step of self review: changed comments and removed useless
beforeEach
* updated docs
* Apply suggestions from code review
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* re-format file after suggestions were applied from gh
* refactor(e2e): use cy.wait().then instead of let =
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Enable pivot tables with only pivot columns
* Re-add one condition so that tests pass
* Tweak logic
* Fix logic again, and add cypress test
* Try again to fix conditional
* Enable results metadata during qp/process-query as it seems to also control insights which we need for slack alert trendlines
* Removing outdated alerts w/ models test
* fix: add logging during connection health checks
Trying to get ahead of #52458, add logs around startup health checks
* Add log for when health check is queued
* Use with-generator
* Replace for tu.rng
* Add ref to workflow, rename env
* Rename vars
* Remove former random ns
* Typo
* Adjust log level
* random-query-from
* Use random-query-from
* Adjust random exception handling
* Add with-gentest kondo hook
* Initial version of with-gentest
* Logging in test assertions
* Workflow, env var names, logging
* Remove redundant code
* Newline
* Seed for non exceptional queries
* Naming
* Cleanup
* Atom for seed
* Cleanup
* Whitespace
* Kondo
* Temporarily fix schema and suppress full-join, to be rebased out later
* Update .github/workflows/generative-query-test.yml
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* Update test/metabase/test/gentest.clj
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* Remove stale comment
* Use tu.rng
---------
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* Improve ID test
* Improve ID test
* Improve ID test
* Improve ID test
* Add a test
* Between test
* Between test
* Between test
* Between test
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* Make the UpsellBigCard more flexible
The product requirements for the Cloud upsell dictate that
the main CTA doesn't directly lead to Metabase website like in
all other cases. It opens a Migration modal instead.
UpsellBigCard had to be adjusted to enable that.
* Add Upsell Cloud component
* Resolve CLO-3576
* Use the cloud upsell instead of the previous CTA
* Adjust the UI
* Update stories
* Update related test
* Fix e2e test
* Add context for translators
* Adjust the vertical spacing
* Don't :return-drills-for-dimensions? for header-context-only drills
Dimensions can never have a "missing" value, i.e. (nil? value), and so none of these drills will ever apply when
called with a context built from dimension-contexts.
* Add comment with updated table for Pivot Cell clicks
#53076 added `:hook/entity-id` for databases, tables and fields, but this is incorrect behaviour.
The sync process periodically detects new tables and fields and adds them to the appdb. If multiple instances of Metabase are connected via serdes, and linked to the same data warehouses, then eg. a new table would get duplicated before the serdes process would be run.
Instead, the `entity_id`s should be based only on the hashes, the same as serdes for databases, tables and fields.
* Cast date binning back to date in postgres
* Remove unnecessary conversion to timestamp
* Fix existing test.
* Test that temporal columns don't change type when binned.
* Fix bucketing for postgres days
* Improve comment about casting interval addition
* Select single field in test for robustness
* Tag :to_timestamp to avoid other areas adding a type
* Whitespace
* Remove blank line at end of file
* Don't cast if type is nil.
* Add alias to native query in test
* Fix redshift
* Pass temporal unit when listing quick-filter options.
* Use with-temporal-bucket like a gentleman
* Use lib.temporal-bucket/temporal-bucket
* Whitespace
* check if the project is a nextjs project
* add nextjs imports and exports for nextjs projects
* add comments
* add notice on analytics.css and link to guide
* fix typo
* update snippet
* normalize path
* quote import path
* update generated directory
* use fileName for sampleComponents
* remove getImportSnippet
* use nextjs imports from the compatibility layer
* remove use client and export default from snippets
* fix checkIsInNextJsProject import
* add use client directive to pages with useReducer
* add suggestions on module resolution
* make import paths reasonable for next.js
* add example _app.tsx snippet
* trim snippets for Next.js
* show custom app snippets separately for next
* auto-generate custom app
* fix file extension
* do not show the react import step in nextjs
* add default root layout or app
* refine root layout generation logic
* add comments on root layout generation
* add next.js route to app and pages router
* add notes on API routes
* only generate AnalyticsPage when not using nextjs
* use writeFileSync
* create directory if it does not exist
* add messages on route created
* remove AppProps type from snippets
* inform users that a custom root layout has been added
* wait for root layout snippets to be ready
* use the correct layout file name in message
* cleanup snippet message
* highlight the layout file
* temporarily remove the moduleResolution warning
* cleanup messages helper
* cleanup exports
* cleanup analytics page snippet
* restore analytics.css
* add more info on the default provider
* fix whitespace in imports
* add a use-client directive to EmbeddingProvider
* highlight the analytics demo route
* add a sans-serif font by default
* do not show the Express server message if Express server is not there
* use glob to simplify path matching
* make analytics page snippet function pure
* make custom app snippet function pure
* add support for src directory
* add source prefix in router resolution
* trim layout snippet header and add eta
* strip the source prefix from import paths
* make src dir a constant
* add lightweight margin reset to sample analytics.css
* consistent source prefix
* parameterize the route name
* cleanup snippet whitespace
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* rename to getNextJsPagesWrapperOrAppWrapperSnippet
* consolidate snippets file
* inline hasNextJsCustomAppOrRootLayout
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Fix#50731
* Revert first fix as design team said there's a better way
* Remove HackyInvisibleTargetFiller
* Use max-width instead of width to prevent overflow if there's less than 300px available
* Add a repro for #50731
When wrapping literal values in :value clauses, fields with string-type ids (for example, fields after an aggregation)
were getting their type info from source-metadata. When the field in question was a postgres enum, the source
metadata was missing important data. Fetching info from both source-metadata and the type info of field-ref fixes
that.
These three entities have previously used their names as their serdes
keys. The names are still used for serdes but we want opaque, stable
`entity_id`s as well so that they can be used as `:ident`s in the field
refs overhaul. Then the name of a database, schema, table or field can
be changed without its `:ident` changing, and therefore without breaking
any questions that reference it.
* Updating the v2 serdes baseline export - it has `entity_id`s now
* Updating the exported audit database; it has entity_ids now
* Fix MySQL appdbs
Use lower case for all table/field names; hard-code the schema
"PUBLIC". (The actual schema for appdb is ignored.)
* run all smoke tests on OSS tests regularly
* skip certain smoke tests for oss v EE
* or not and
* union
* update prerelease tests
* run smoke tests on OSS except EE
* Add upsell for whitelabel/appearance
The most important change this PR introduces is a whitelabel route that's now present in the OSS and starter plans. How this works now:
- When the whitelabel token feature is not present, the route `/admin/settings/whitelabel` shows the upsell page
- When the whitelabel token feature gets activated, the route `/admin/settings/whitelabel` gets redirected to `/admin/settings/whitelabel/branding` and the upsell page is not visible anymore.
* Mantine v7-1 squash
update deps
update UI configs
easy fixes
happy little typescript
updates after rebase
test fixes
fix rebase issues
admin-2, collections, custom-columns, dashbaord and dashboard-cards suites
fix clause popovers
fixing rebase issue
more rebase sadness:
more e2e
autocomplete adjustment
fixing nested popovers
adjusting a couple more pickers
more e2e fixes
fix mantine provider, searchbox errors
unit test adjustments
theme colors, filter labels
Relative Date Picker
more unit test fixes
* e2e adjustments
* [MANTINE] ci: find css variables that are used but never defined (#52947)
* ci: find css variables that are used but never defined
* only search in frontend and enterprise/frontend
* whitelist mantine css variables
* temporarily allow some css variables
* read mantine variables directly from mantine css file
* UI Regressions
* lint and type fixes. green unit test
* Cherry pick updates to the never defined variables script (#53049)
* Update bin/find-never-defined-css-variables/find-never-defined-css-variables.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* test: checks that it doesn't confuse definitions with usages
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* more adjustments for e2e tests
* adjustments
* embedding sdk green, maybe unit tests?
* loki regressions
* test(sdk): migrate theme provider test to e2e to work with mantine v7 (#53290)
* test: migrate theme provider test to e2e to work with mantine v7
* re-put there things i removed by mistake
* preview and type fixes
* I think I'm making this better?
* fixing colors, unit tests
* scoping css vars and static classes, other things
* class prefix, scoping input css
* small adjustments
* shamelessly poach Nicolo's commit
* small fixes, should make Kamil happy :)
* finally green!
* type check
* chartsettingsinput fixes
* silly number types
* Adjustments for loki tests
* type fix
* undo Question View Download changes
* adjusting smart scalar
* updating loki reference
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* smooth out some rough edges for local e2e tests
* note for the normies
* remove premium embedding token
* add note about h2 app db
* update config file stuff
* Allow passing native extras to withDifferentDatabase
* Pass empty nativeExtras when switching databases
* Add reproduction for 53299
* Do not pass nativeExtras to withDifferentDatabase
* Do not update native-extras when chaning databases
* Remove with-extras overload on with-different-database
* Remove native-extras for with-different-database call sites
* Remove required throw from test
* Remove collection requirement from test
* convert pgarray to java array
* convert array to vector early
* add test and todo comment
* fix whitespace
* add tests for caching and emailing
* whitespace
* use feature and test other drivers
* update test names
* deduplicate native array methods
* remove vertica
* remove http from array agg test
* remove let
* fixes offset images when dnd-ing columns in dashcards
* we should be grateful for e2e tests
* -> mantine style prop
* fix this in the sortable list while I'm at it
* add clarification message to the cli
* update cli startup message
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* line-wrap the cli startup message
* use metaba.se/sdk for docs link
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Remove silent flag and re-enable multi-arch build
Check what's going on here
* test code
* fix test code
* Update Dockerfile_ubuntu
Fixes the multi-arch build
* Update Dockerfile_ubuntu
Have a feeling that the ca-certificates-java is making a mess here
* Update uberjar.yml
Add the logging to the process
* Update Dockerfile_ubuntu
* Update uberjar.yml
* Update run_metabase.sh
ensure the path lives with the user
* Update uberjar.yml
Alpine 21 is multi-arch, we should have that as well in the build script along ubuntu
* Update uberjar.yml
push explicitly all platforms
* Update uberjar.yml
Update actions to see if platform flag works
* Update uberjar.yml
This should push the multi-arch images without building again 🤞🏻
* Update uberjar.yml
return to old versions, just to keep stuff more consistent
* Update uberjar.yml
fix typo
* remove test code
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* feat: use liquibase session lock instead of transaction lock
Prevent the migration lock class of errors in non-h2 dbs by using a liquibase session lock instead of a transaction lock.
This type of lock would clear automatically when the connection closes for any reason.
Using this approach there is no need for the metabase release lock command except with h2.
The force migration command still is applicable though, as it does more things than just release the lock.
See https://github.com/blagerweij/liquibase-sessionlockFix#50212
* fix: mark migration 109 as none db
From Nathan Voxland:
From a liquibase-standpoint, deleting the changesets are just fine. As long as you don't accidentally reuse the id+author+path combo in the future. Databases that already ran it will not care that it's missing and new databases will just never see it to know. So it's more of an our-standards questions.
Alternately, you can set dbms="none" to keep it in the file but not have it actually apply anywhere. That doesn't change the checksum
* fix: error when force-releasing non-h2 locks
* test: add session lock test
* WIP - lots of streamlining
* revert dotenv addition
* ci fix
* some fixes
* make component testing work
* set some default ports
* fix component test ci
* fix all the e2e tests
* fix the other sdk tests
* update e2e docs
* fix embedding more
* fix tagged tests
* documentation
* split user and derived options
* better exit behavior
* more error handling
* revert saml change
* ❤️ typescript and booleans
* fix some stuff
* fixes
* stupid lockfile
* maybe keep the lockfile
* remove unnecessary diff
When fetching a param's possible values, we were originally looking for concrete columns and checking result metadata.
However, if the query is against a model and has an aggregation, the thing you are filtering on isn't a concrete field
and probably isn't in result metadata. Checking filterable-columns lets us find the appropriate field in that case. The practical result is that we actually find remapped values for a param when it is linked to an aggregated query against a model.
* Check cycles on PUT to /card/:id
* Remove extra protection from saving cycles.
* cljfmt
* kondo
* Better threading
* Cleaner tests
* Fix test
* Better naming in metabase.api.card/dataset-query->query
* Rename `dataset-query` arg to `dq-fragment`.
* Normalize query on backend.
* Throw exceptions with user messages.
Also rework `dataset-query->query`.
* Unshadow card-type in doseq
* Make consistent use of query/dataset-query
* Remove normalization from lib.query
* Don't shadow `query`.
* Rename `dq-fragment` to `query-fragment`
* `timelines` module
* Move some render-related timeline stuff into the timeline module
* Remove unused /api/card/:id/timelines endpoint
* PR feedback and test fixes 🔧
* Create dashboard questions for x-rays
* Don't make subcollections for autogened dashboards
Before, for an admin we'd create a structure like:
```
- Our analytics
- Automatically Generated Dashboards
- A look at Accounts
- Dashboard
- Card 1
- Card 2
- ...
```
Or, for regular users, a structure like:
```
- Your personal collection
- A look at Accounts
- Dashboard (A look at Accounts)
- Card 1
- ...
```
Now, we'll create the following structure:
```
- Our Analytics (OR Your personal collection, for non-admins)
- Automatically Generated Dashboards
- Dashboard (A look at Accounts)
```
Other autogenerated dashboards will be placed alongside.
* Fix question not found error showing after navigating from dashboards
* Add interactive dashboard tests
* Add editable dashboard tests
* Fix failed tests
* Add an assertion to ensure we display ad-hoc question
* update enable-embedding-sdk based on embedding-app-origins-sdk
|enable-embedding-sdk key exists? | then we:
|---------------------------------|---------
|false | no-op
|true | set enable-embedding-sdk to true when `embedding-app-origins-sdk` exists
* actually need to set it to false when embedding-app-origins-sdk dne
* reformat liquibase migration
* make the query run on h2, too
* per-db sql + update comment
* use preconditions to simplify update sql
* Allow private slack channels to be listed for alerts/subscriptions.
Requires the groups:read scope to see private channels
* manifest.yaml changed so new installations get it by default
* things are kept compatible if the token does not have the scope
* you can manually add the scope to the app in slack.
private channels will then up when the channels are refreshed
* Update the slack manifest.yaml inlined in the docs
* Add re-installation guidance to slack docs for `groups:read`
* Ensure slack token is set in conversations-list test
* Refactor conversations-list to take :private-channels opt
* Add refresh-channels-and-usernames! test to check groups:read behaviour
* prevent duplicated SET_EDITING_DASHBOARD dispatches from overriding edit dashboard state if the editing dashboard is the same dashboard
* small improvements
* add test coverage
* impl pr feedback
* unmark as flaky: should not reload the model for record in the same model
* unmark as flaky: scenarios > search should allow users to paginate results
* unmark as flaky: scenarios > model indexes should be able to see details of a record outside the first 2000
* unmark as flaky: should be able to search model index values and visit detail records
* Remove the custom permissions upsell component
* Implement upsell for advanced permissions
This applies to both:
- `/admin/permissions/data/group` and
- `/admin/permissions/data/database/` routes.
* Remove the old feature from the enterprise feature level permissions
* make dashboard dirty if tabs are edited and add test coverage for leaving dashboard after editing a dashboard or its tabs
* handle rename tab, add rename tab test coverage, simplify test coverage
* makes ci ts happy
* clean up test a bit
* rename helper
* Return column corresponding to aggregation value in SmartScalar getCurrentMetricData
Closes#46168
The returned clicked object had the `value` from the aggregation cell, but the `column` from the date dimension, which
breaks the quick filter drill (and probably others).
Instead, return the `column` that corresponds to the `value`.
* Add e2e test for smartscalar quick filter drill
* remove unsafe lifecycles from visualization component
* add e2e tests
* remove extra comma in test helpers
* only run tests under strict mode when a flag is passed
* extract console assertion to a test helper
* update tests for known console errors
* swap new and previous props
* combine old state for transform props to state
* combine initial state
* use getDerivedStateFromProps to fix failures
* only store a subset of lastProps to compare
* remove timeline events check from getDerivedStateFromProps
* reset state when timeline events changed
* get rid of mutation in strict mode
* remove existing state spread from state updates
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* remove deriveStateFromProps from constructor
* use sdkProviderProps to pass provider props
* replace UNSAFE_componentWillReceiveProps with getDerivedStateFromProps
* add unsafe lifecycle check for DashboardGrid
* lastProps is already part of DashboardGrid state
* optional chain state.initialCardSizes
* remove state comparison logic
* fix misleading hasVisibleDashcardsChanged condition
* fix editor issue
* remove unused dashcardData from lastProps
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Add generative test
* Use per test generator with adjustable seed
* Add workflow file
* Temporarily add trigger
* Make rand function work with default generator
* Update log4j2 test config
* Info log level
* Whitespace
* Add iteration count and env to make run on-demand
* cljfmt
* cljfmt
* remove unsafe lifecycles from visualization component
* add e2e tests
* remove extra comma in test helpers
* only run tests under strict mode when a flag is passed
* extract console assertion to a test helper
* update tests for known console errors
* swap new and previous props
* combine old state for transform props to state
* combine initial state
* use getDerivedStateFromProps to fix failures
* only store a subset of lastProps to compare
* remove timeline events check from getDerivedStateFromProps
* reset state when timeline events changed
* get rid of mutation in strict mode
* remove existing state spread from state updates
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* remove deriveStateFromProps from constructor
* use sdkProviderProps to pass provider props
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
When replacing a join, we were taking a valid join (with a correctly set up join alias), dropping the alias, creating
a new alias, and then inferring which fields belonged to the new alias. This caused issues when joining models using
a field that could plausibly belong to either model, because we inferred that the lhs join condition belonged to the
rhs model. This in turn confused the join naming code and caused us to refer to the lhs as "previous results" instead
of the correct table name.
With the new code, if we want to rename the new join, we explicitly replace the old alias with the new alias instead
of just removing the new alias and inferring what goes with what. This avoids potentially incorrect inferences and
fixes this issue (and possibly others?).
Last year we had a P1 where a high-visibility page in Brazilian Portuguese was broken. Let's add some basic e2e tests that check high-visibility admin/webapp pages (accessible in one click from the homepage) in our most popular locales.
* update ui to show trash icon, tooltip, and undo message if the dashcard being removed is the last dashcard for a dashboard question
* add e2e test, fix types
* Add pivot-drill-details
Similar to filter-drill-details and combine-columm-drill-details.
pivot-drill-details returns the info needed by FE to render the pivot drill popover UI.
* Merge pivot-types into pivot-drill-details
* Add e2e test
* Revert unintentional whitespace change
* Re-enable Breakout by e2e test
* Re-enable multi-stage pivot drill tests
* Add pivot drill test for query with explict join
* Add hostname support to BigQuery
* Fix formatting
---------
Co-authored-by: Case Nelson <snoe@snoe.ca>
Co-authored-by: Case Nelson <case@metabase.com>
Fixes#52843.
* Redshift: Adding an explicit dep on `aws-java-sdk-core`
This is listed as an optional dep, but it seems that at least one class
from it (`com.amazonaws.util.StringUtils`) is now dynamically required
by our driver - tests were throwing `NoClassDefFoundError`s without it.
Redshift tests are passing locally with this dependency added.
* fix(entity-id): show 404 for entity id that starts with digits
* simplify logic by removing support for sequential ids with slugs for the
tab
* clean up the tests and rename things
Year of era aggregations weren't working because our legacy malli schema didnt recognize it as a valid temporal unit.
Adding it to the schema makes it work.
* partition indexes and use separate queries
* test no exception thrown
* improve helper func name
* improve test
* remove test debugging
* address pr comments
This should ensure tasks will find classes in our plugin system. The
issue arose when `submit-task!` was called to sync an unhidden bigquery
table before any query was run on the main thread before hand.
This ensures that if either of these tasks runs before classloader has
been set, that their threads will inherit it.
`task.cache/pool` was found as an audit of other threads that may
exhibit this behavior.
Testing this from source is not possible right away since the
classloaders behave so differently between jars and sources.
* ci: find css variables that are used but never defined
* only search in frontend and enterprise/frontend
* Update bin/find-never-defined-css-variables/find-never-defined-css-variables.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* test: checks that it doesn't confuse definitions with usages
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
These date from the beginnings of the `metabase.lib` codebase,
and are unused outside of a couple of tests.
I deleted the tests for these functions specifically, and updated a few
other tests that used these functions to retrieve columns on a query.
Prework for #49182.
* Start refactoring Field to TS
* Revert "Start refactoring Field to TS"
This reverts commit 88af47fdac.
* Add description input
* Use translation
* Make placeholder match data model page UI
* Add repro for #37907
* Fixing xlsx pivot table exports to respect table column names
* Adding fallback to column title to use display_name if custom title is empty; fixing kondo error
* Fix cljfmt spacing
* Adding unit test for xlsx pivot exports respecting custom column name
* Improving logic to be more idiomatic
* OpenAPI dox generation part 2
* Fix docstring
* metabase.billing.billing => metabase.billing.api
* Fix dox generation code fr fr
* Revert changes to dox for now
* Remove define-routes
* Fix missing routes
* Fix even more missing routes
* Third time's a charm
* Remove extra newlines at end of file
Databricks has static test data, by creating views we run the risk of
concurrent test runs seeing the temporary views or those views not
getting cleaned up if a job gets killed.
This is part of ENG-14545
* Add upsell component for better support
* Implement upsell for better support on a troubleshooting page
* Snow better support upsell only for OSS
* Update import path
* Remove whitespace
* Account for smaller screens
* fix(entity-id): don't make call to the card endpoint if the entity id
starts with digits
* prevent `isXXXPath` to return true if the path is the one of entity ids
* Remove flaky tags in e2e/test/scenarios/native/native.cy.spec.js
* Remove flaky tags in e2e/test/scenarios/native/native_subquery.cy.spec.js
* Remove flaky tags in e2e/test/scenarios/native/native-reproductions.cy.spec.js
* Remove flaky tag from admin test that uses native editor
* Remove flaky tag from visualization test that uses native editor
* Remove flaky tag from dashboard test that uses native editor
* Add helper to clear browser cache
* Reduce flakiness in autocomplete test
* Remove flaky tag in e2e/test/scenarios/native-filters/native-filters-reproductions.cy.spec.js
* Fix SDK cross-version testing failure after test util refactor (#52900)
* Revert the code before the fix
* Revert "Revert the code before the fix"
This reverts commit be11146bf1.
* Another attempt by checking out the whole e2e dir
* Can't checkout twice it will override everything 😭
* Reset everything back to the start
* Add missing files
* Keep only the latest webpack config
* Clarify the temporary bandaid with a comment
* Ensure when we cut 54, we won't ran into SDK cross-version testing
Future work on the refs overhaul #49182 changes how queries are
represented. To enable testing that new behaviour at a per-test level,
we can't let global queries like `lib.tu/venues-query` be defined once
and reused forever: in that case, tests of the new refs would see global
queries defined under old refs, and vice versa.
This change transforms the global queries, cards and other helpers in
`lib.test-util` from static `def`s to functions, so they build the
queries etc. in the context of the current test.
(If this introduces a performance hit for the lib tests, we can update
the functions to cache the two versions up front and choose between them
at runtime. So far it does not seem to have made a significant impact.)
* Use CodeMirror 6 (#50484)
* Add @uiw/react-codemirror
* Add language plugins for codemirror
* Add @codemirror/autocomplete
* Add basic codemirror editor
* Add selection handler to CodeMirror editor
* Set height
* Add basic styling to codemirror editor
* Add custom selection background
* Add simple theme with font settings
* Move extensions to a separate file
* Implement right click on selection
* Add language support
* Add @codemirror/language
* Add @lezer/highlight
* Add custom syntax colors
* Hightlight tags
* Add highlight for matching bracket
* Add completions
* Add snippet completions
* Make the editor full height by default
* Remove duplicate dependencies
* Match autocomplete colors with ace editor
* Render completion items in subgrid
* Tweak completion margin
* Use nesting
* Add hover state on completions
* Make getTagAtCursor more sueful
* Move matchTagAtCursor to utils
* Avoid work when there's no need
* Implement onCursorMoveOnCardTag
* Move autocomplete into language helper
* Disable language completions when inside a tag
* Allow open ended tags to return without suffix
* Add testid to codemirror editor
* Update test helpers
* Fix native subquery tests
* Fix native reproductions
* Fix snippet tests
* Remove ensureAceEditorClosed helper
* Add nativeEditorType helper
* Fix action tests
* Do not use native editor helper for custom expression test
* Add more native editor helpers
* Fix model metadata tests
* Use better type helpers
* Add more TODO comments
* Fix sharing test
* Simplify e2e helper
* Use native editor type helper
* Fix clearNativeEditor helper
* Fix missing namespace
* Update Loki Snapshots
* Remove unused lang extension
* Be explicit about h2 engine
* Nest language plugin return
* Export useLazyGetCardQuery
* Add completions for referenced card columns
* Split up autocompleters
* Simplify completion info
* Use cached cards
* Move useMemoized to util
* Add TODO comment about engine type
* Fix subquery test
* Use helpers in native reproduction
* Fix e2e test meta helper
* Do not assume we've seen all suggestions when narrowing down
* Fix selectAll
* Unskip unbroken test
* Fix broken reproduction
* Increase the delay for autocompletions to kick in
* Fix nativeEditorSelectAll on non-Mac systems
* Rename completer
* Move completers to separate file and make them handle their own dependencies
* Add autocomplete endpoints to metabase/api
* Use autocompletion hooks from metabase/api
* Reuse AutocompleteMatchStyle type
* Simplify helper
* Whitespace formatting
* Test sub-limit completion
* Fix reproduction
* Add tests for matchTagAtCursor
* Remove commented out code
* Typos
* Clarify why we disable rtk-query's cache
* Simplify cx call
* Use Cypress.platform over navigator.platform
* Tweak fold caret position
* Pass font to cm-content too
* Add synteax highlighting for the name token
* Rename AutocompleteSuggestionsRequest to AutocompleteRequest
* Fix broken overflow on notebook preview
* Add more fonts to monospaceFontFamily
* Use the same line height as the ACE editor
* Autocomplete much faster
* Handle overflow in completion meta
* Use shared object
* Add padding to editor scroll pane
* Add more fonts to monospaceFontFamily
* Set line-height to normal
* Fold on parens, braces and brackets
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Namespace native editor e2e helpers (#51093)
* Add namespace for helpers
* Replace H.nativeEditor with namespaced version
* Replace H.focusNativeEditor with namespaced H.NativeEditor.focus()
* Replace H.nativeEditor completions helpers with namespaced versions
* Replace H.nativeEditorSelectAll with namespaced version
* Replace H.nativeEditorClear with namespaced version
* Replace H.nativeEditorType with namespaced version
* Use NativeEditor.type where possible
* Fix imports
* Replace H.focusNativeEditor with namespaced H.NativeEditor.focus()
* Use NativeEditor.type where possible
* Use NativeEditor.clear where possible
* Do not call NativeEditor
* Add missing backspace in helpers
* Fix case of selectall
* Remove use of focus().as(...)
* Fix NativeEditor.get call
* Fix e2e test
* Remove focus where it is wrong
* Fix text replacement test
* Add small delay
* Remove ACE version of the Native editor (#51437)
* Remove sql_behaviour
* Remove getEngineNativeAceMode
* Remove CARD_TAG_REGEX
* Do not re-export ACE_EDITOR_ID
* Remove AceEditor
* Remove ACE_EDITOR_ID from NativeEdiror mock
* Remove AceEditor reference in test
* Remove intermediate editor
* Update Loki Snapshots
* Fix formatting
* Fix broken test
* Various fixes for the native editor revamp (#52321)
* Deduplicate schema completions
* Align description to the right
* Add keyword description to comletions
* Render completion description in normal font
* Set indentation to use tabs
* Remove boost from completions
* Stop cmd+enter from entering a newline
* Use same indentation as previous line when pressing Enter
* Remove empty resize handler in CodeMirrorEditor
* Add defaults to NativeEditor.type
* Make config parameter optional
* Add custom keymaps for next and previous completions
* Add custom interpolations for previous and next completion
* Add helpers up and down arrows
* Add reproduction for #48712
* Add tests for suggestion issues
* Allow completing from inside a word
* Add unit test for useSchemaCompletions
* Add unit test for useSnippetCompletions
* Add unit test for useCardTagCompletion
* Add unit test for useReferencedCardCompletion
* Accept completion with tab
* Do not indent with tab, but just insert a tab
* Use Lib.Query in CodeMirrorEditor
* Simplif CodeMirror editor prop and ref names
* Add simple unit test for CodeMirrorEditor
* Use isNotNull helper
* Use Lib.Query in NotebookNativePreview
* Handle nil in engine
* Handle empty engine
* Use correct meta kay in test
* Reduce flakiness in dashboard test
* Reduce flakiness in native test
* Add completion for query locals
* Make database optional
* Add test for locals
* Add test for command enter
* Add tests for indenting with tab
* Handle non-darwin platform in test
* Allow typing {tab}
* Add test for indentation
* Handle quoted locals
* Add typing to source
* Improve syntax highlighting for mongo
* WIP: add custom keywords completion for mongo
* Make collection_id optional
* Remove test return
* Handle SQLDialct shape
* Handle keyword completion all in the completer
* Lift completers out of language plugin
* Move language helpers to separate file
* Simplify keywords extraction
* Add tests for keyword completion
* Add more mongo keywords
* Fix quoted local test
* Add suggestion tests for mongo
* Fix e2e test with NativeEditor
* Update Loki Snapshots
* Fix e2e test with NativeEditor
* Revert "Update Loki Snapshots"
This reverts commit 7d1ea82579.
* Revert "Revert "Update Loki Snapshots""
This reverts commit fd7b552770.
* update screenshot
* Fix autocomplete API calls in the new native query editor (#52688)
* Enable autoFocus
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Sync state in useStringFilter
* Sync state in useBooleanOperatorFilter
* Sync state in useBooleanOptionFilter
* Add FilterModalResult type
* Fix setTab type
* Make comments work as JSDocs
* Move FilterModalResult
* Add filter modal context
* Drop FilterModalHeader and use FilterSearchInput directly
* Start to use context in BooleanFilterEditor
* Start to use context in TimeFilterEditor
* Start to use context in DateFilterEditor
* Start to use context in CoordinateFilterEditor
* Start to use context in NumberFilterEditor
* Start to use context in StringFilterEditor
* Start to use context in DefaultFilterEditor
* Stop drilling isSearching
* Fix tests
* Stop drilling version prop
* Remove FilterModalBody and FilterModalFooter exports
* Make naming consistent
* Remove props drilling
* Remove props drilling
* Remove props drilling
* Inline FilterTabContent into FilterModalBody
* Revert "Inline FilterTabContent into FilterModalBody"
This reverts commit cfd5738e52.
* Remove props drilling
* Sort attributes
* Consolidate imports
* Fix a mistake
* Improve JSDoc
* Revert "Sync state in useBooleanOptionFilter"
This reverts commit 33a979dfdf.
* Update approach after merging refactoring branch
* Add a repro for string filters
* Go back and forth on search input
* Assert operator
* Move effect to concrete hook
* Add test for boolean filters
* Fix time filters
* Revert "Move effect to concrete hook"
This reverts commit 82b94f4345.
* Move effect outside use-time-filter
* Improve naming
* Remove TODO
* Extract setup48319
* Rename setup48319 to setupCustomColumns
* Rename handle* to on*
* Fix coordinate filters
* Fix default filters
* Fix number filters
* Revert 1st approach
* Use key prop
* Remove detailed tests and leave only one
* Use current value
* Add a test for "version" hack
* Use only version for the key
* Rename version to remountKey and add explanatory comments
* Rename forceDescendantsRemount to forceRemountDescendants
* Rename
* fix ultra wide tooltips (closes#52199)
* all tooltips wrap now, and better test
* sturdier test
* use setup() in test case
* removed unncessary code path
* use features instead of hard coding drivers in tests
* fix formatting
* fix details-name-is-db feature
* bad conn details helper instead of features
* address pr comments
* exclude vertica from materialized view tests
* fix whitespace
Update YAMLs for 53 release.
Move all questions into usage analytics dashboards. In some cases, questions had to be duplicated in order to be moved to dashboards.
* update copy and avoid showing two messages about moving when going between collections and dashboards in different collections
* pr feedback
* impl pr feedback
* Add dashboard empty states
* Separate empty states from the actual dashboard content
* Fix public dashboards view
* Add translation context
* Localize the `alt` tag
* Make sure the content doesn't get cut off
In sql server, language settings change what day weeks start on. The sql server driver assumes that weeks always
start on sunday (ie the US default), which means that if you change the db to french, our math is off and we start
calculating weeks and days of the week wrong. This adds back in an offset based on DATEFIRST to fix that.
* POC of entity id aware route
* wip e2e tests
* clean up logic
* changing approach as everything was broken
* revert unwanted changes
* Revert "revert unwanted changes"
This reverts commit f6ce10ac7b.
* reset routes to master
* /by-entity-id/ routes
* update tests to check for subpaths and search param preservation
* cleaning up un-needed code
* simplified even more the logic by merging the two config maps to one
array
* pass down router instead of single props
* split the tests to their files
* fix typo in test
* adds a test for a unlikely to be needed scenario
* fix import after merging master
* fix e2e copy
* /by-entity-id/ -> /entity/ as per product decision
* tag the test as flaky as it fails depending on the order of test files
being run
* Auto-rewrite defendpoint forms in dataset.clj
* Auto-rewrite defendpoint forms in dataset.clj
* Auto-rewrite defendpoint forms in dataset.clj
* Test fixes 🔧
* More test fixes 🔧
* A few more test fixes.
* Test fixes 🔧
* Test fixes 🔧
* Finally handle form-encoded bodies correctly
- Cleaned up session management
- Don't send "new device" email from embedded logins
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* Avoid cartesian products in pivot csv exports
* Get row and column sorting working properly
* Add a bunch of unit tests
* Test fixes, various tweaks to better match the FE output
* Fix another test
* address feedback
* Add FilterModalResult type
* Fix setTab type
* Make comments work as JSDocs
* Move FilterModalResult
* Add filter modal context
* Drop FilterModalHeader and use FilterSearchInput directly
* Start to use context in BooleanFilterEditor
* Start to use context in TimeFilterEditor
* Start to use context in DateFilterEditor
* Start to use context in CoordinateFilterEditor
* Start to use context in NumberFilterEditor
* Start to use context in StringFilterEditor
* Start to use context in DefaultFilterEditor
* Stop drilling isSearching
* Fix tests
* Stop drilling version prop
* Remove FilterModalBody and FilterModalFooter exports
* Make naming consistent
* Remove props drilling
* Remove props drilling
* Remove props drilling
* Inline FilterTabContent into FilterModalBody
* Revert "Inline FilterTabContent into FilterModalBody"
This reverts commit cfd5738e52.
* Remove props drilling
* Sort attributes
* Consolidate imports
* Fix a mistake
* Improve JSDoc
* Rename handle* to on*
* Auto-rewrite defendpoint forms in session.clj
* Auto-rewrite defendpoint forms in session.clj
* Remove &request anaphor and make it a normal arg
* Add test
* Defendpoint 2.1
* base-handler => handler
* Fix Kondo errors
* Make sure async exception handling is done correctly
* Remove defendpoint-async
* Rewrite all the rest of the remaining defendpoints except for those in public
* Remove card endpoints from this PR
* Update src/metabase/setup/api.clj
Co-authored-by: bryan <bryan.maass@gmail.com>
* cljfmt
* Test fixes 🔧
---------
Co-authored-by: bryan <bryan.maass@gmail.com>
* Drop Java 11/17 Support
* Update Dockerfile
Change the Java version of the builder as well
* mention that we are dropping support for earlier Java in docs
---------
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>
Co-authored-by: Alexander Solovyov <alexander@solovyov.net>
* throw down ui
* pixel freaking perfect
* wire in real endpoint for fetching dashboard question candidates
* wire in move endpoint
* loading and error handling for mutation
* persist info step acknowledgement
* optionally show item in menu, fix flickering of info modal is acked previously, add a toast on successful bulk move
* fix some of the failing existing tests
* add unit tests for collection menu
* add unit tests for main migration modal
* add e2e test
* fix failing unit test
* fix another failing unit test
* fix candidates result being stale if dashcards added/removed from dashboards creates/removes candidates
* adjust url of feature
* copy feedback
* fix type-check
* scope feature to only admins
* use pagination to fetch count but not items when detecting if to show the migration tool menu item
* test + style improvements, impl pr feedback
* fix unit tests, add unit coverage for is admin
* Add lib.native/has-template-tag-variables?
* Disable drills for native queries with template tag variables
Closes#49051
* Re-export lib.native/has-template-tag-variables? from lib.core
* Add tests for disabled drills for native queries with variables
* Review feedback
Closes: https://github.com/metabase/metabase/issues/52116
Note that of icu4j was already a dependency, via graalvm.
We have just made it explicit now, ~~and updated to the latest version.~~ ([upgrading version breaks rendering](https://github.com/metabase/metabase/actions/runs/12926546646/job/36049844305?pr=52581))
```
org.graalvm.js/js 22.3.5
. org.graalvm.regex/regex 22.3.5
. org.graalvm.truffle/truffle-api 22.3.5
. com.ibm.icu/icu4j 71.1
. org.graalvm.truffle/truffle-api 22.3.5
. org.graalvm.sdk/graal-sdk 22.3.5
. org.graalvm.sdk/graal-sdk 22.3.5
. com.ibm.icu/icu4j 71.1
```
* Auto-rewrite defendpoint forms in src/metabase/api/api_key.clj
* Auto-rewrite defendpoint forms in src/metabase/api/api_key.clj
* Fix formatting
* Auto-rewrite defendpoint forms in api_key.clj
* Auto-rewrite defendpoint forms in api_key.clj
* updated some flaky screenshots
* simplified ci job
* updated types
* registered visualizations and added a way to register multiple visualizations in storybook
* replaced @storybook/testing-library with @storybook/test
* Paginate the DQ candidates endpoints
* Paginate in Clojure
I think this should be fine - the number of cards we're working with
should be relatively small, since this is working one collection at a time.
* CR response
* Auto-rewrite defendpoint forms in database.clj
* Some cleanup
* Update src/metabase/api/database.clj
Co-authored-by: bryan <bryan.maass@gmail.com>
* cljfmt
* Auto-rewrite defendpoint forms in database.clj
* Test fix?
---------
Co-authored-by: bryan <bryan.maass@gmail.com>
* Exclude DQs from DQ candidate endpoint
... oops
I also moved the `(= (:collection_id card) (:collection_id dashboard))`
check out of `card/sole-dashboard-id`, since it's not really part of
being a sole dashboard.
* Add `dashboard_id IS NULL` to the query
... instead of filtering them out in clojure.
This was already legal-ish, as [:* 1 [:sum ...]] was a valid argument to coalesce. However, you couldn't pass in the
sum directly. Under the hood, NumericExpressionArg allowed aggregations, but ExpressionArg didn't. Presumably, the
one is supposed to be a superset of the other, and this change made that actually be true.
* filter out collections and dashboards where can_write is false from recents tab in collection picker
* move modal filter out locations you can not save to from recents tab
* add e2e test
* pr feedback - make cleaner and add comment
* Auto-rewrite defendpoint forms in src/metabase/api/automagic_dashboards.clj
* Manual cleanup
* Auto-rewrite defendpoint forms in bookmark.clj
* Manual cleanup
* Auto-rewrite defendpoint forms in automagic_dashboards.clj
* Fix endpoints with no route params but no regexes
* Auto-rewrite defendpoint forms in automagic_dashboards.clj
* Auto-rewrite defendpoint forms in automagic_dashboards.clj
* Auto-rewrite defendpoint forms in automagic_dashboards.clj
* Don't include changes to dev namespace
* Revert changes to bookmark
* Some manual cleanup
* Show should be optional
* Auto-rewrite defendpoint forms in user_key_value.clj
* Auto-rewrite defendpoint forms in user_key_value.clj
* Auto-rewrite defendpoint forms in user_key_value.clj
* Cleanup + fixes 🔧
* Auto-rewrite defendpoint forms in cache.clj
* Auto-rewrite defendpoint forms in cache.clj
* Auto-rewrite defendpoint forms in cache.clj
* Auto-rewrite defendpoint forms in cache.clj
* Fix schema for PUT /api/cache
* Fix cache schema (part 2)
* Do not render "Make public" toggle in Create Action Modal
* Disable the "make public" toggle and add a tooltip explaining why it's disabled
- Use Mantine's Switch
- Use Mantine's Tooltip
* Add a repro for #51282
* Update test name
* Fix tests
* Update unit tests
* Add unit test repro for #51282
* Remove e2e repro for #51282
* Format code
* Format code
* fix unknown attribute error
* fix another error message about unknow HTML attribute
* fix invisible ellipsis when title is empty string
* added e2e test
* Update labels
* Revert "Update labels"
This reverts commit be632aec2e.
* Simplify the fix
* Update unit tests
* Update test
* Revert "Simplify the fix"
This reverts commit a9c267c4c0.
* Update labels
* Update test
* Make it work in native editor
* Revert "Update unit tests"
This reverts commit 3614a8ff2f.
* Update test
* Update test
* Update test
* Update test
* Using 1 as max value in minibar chart rendering logic if the column is pure percent
* Adding MiniBar unit spec
* Prettier --write
* fixing eslint errors
* polish specs
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
* Defendpoint 2 misc fixes
* More fixes 🔧
* Remove some requires for faster loading; always make all query params optional
* Fix comment
* Make :boolean default false as well
* Auto-rewrite defendpoint forms in activity.clj
* Auto-rewrite defendpoint forms in activity.clj
* Formatting
* Auto-rewrite defendpoint forms in activity.clj
* Auto-rewrite defendpoint forms in activity.clj
* Auto rewrite defendpoint ones
* My bird is typing
* Make sure response format matches old defendpoint format
* Cleanup
* Clean up alert tests
* Test fixes 🔧
* One last test fix 🔧
* Make actions execute body optional
As a follow-up to #52284, this PR adds lint rules banning the pattern that caused a P1 (#48144). These rules require us to use the browser default with `String.localeCompare` and related functions in the `Intl` library. Since we cannot pass these functions our own locale codes, since some of these are in the wrong format.
This is designed to be a good-enough, "80 percent of the value with 20 percent of the effort" solution, even though better solutions could be pursued such as aligning FE and BE around an international standard (BCP 47).
- [x] Tests have been added/updated to cover changes in this PR
This is a strange one. But in practice, this was still doing a seq scan
on postgres.
```sql
clean=# explain DELETE FROM query_execution WHERE id IN (SELECT id FROM query_execution WHERE started_at <= '2025-01-02 01:10:15.651839+00' LIMIT 50000);
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------
Delete on query_execution (cost=526084.17..1066675.96 rows=0 width=0)
-> Nested Loop (cost=526084.17..1066675.96 rows=50000 width=34)
-> Unique (cost=526076.73..526326.73 rows=50000 width=32)
-> Sort (cost=526076.73..526201.73 rows=50000 width=32)
Sort Key: "ANY_subquery".id
-> Subquery Scan on "ANY_subquery" (cost=0.00..522174.32 rows=50000 width=32)
-> Limit (cost=0.00..521674.32 rows=50000 width=4)
table scan >>>>> -> Seq Scan on query_execution query_execution_1 (cost=0.00..1704153.49 rows=163335 width=4)
Filter: (started_at <= '2025-01-01 19:10:15.651839-06'::timestamp with time zone)
-> Bitmap Heap Scan on query_execution (cost=7.44..11.46 rows=1 width=10)
Recheck Cond: (id = "ANY_subquery".id)
-> Bitmap Index Scan on query_execution_pkey (cost=0.00..7.44 rows=1 width=0)
Index Cond: (id = "ANY_subquery".id)
(13 rows)
```
The odd thing is that it is sensitive to what timestamp you use. Here's
the same query but with a timestamp in 2024:
```sql
clean=# explain DELETE FROM query_execution WHERE id IN (SELECT id FROM query_execution WHERE started_at <= '2024-12-19 01:10:15.651839+00' LIMIT 50000);
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------
Delete on query_execution (cost=225736.06..641819.90 rows=0 width=0)
-> Nested Loop (cost=225736.06..641819.90 rows=28490 width=34)
-> HashAggregate (cost=225724.95..226009.85 rows=28490 width=32)
Group Key: "ANY_subquery".id
-> Subquery Scan on "ANY_subquery" (cost=122917.34..225653.72 rows=28490 width=32)
-> Limit (cost=122917.34..225368.82 rows=28490 width=4)
-> Bitmap Heap Scan on query_execution query_execution_1 (cost=122917.34..225368.82 rows=28490 width=4)
Recheck Cond: (started_at <= '2024-12-18 19:10:15.651839-06'::timestamp with time zone)
-> Bitmap Index Scan on idx_query_execution_started_at (cost=0.00..122910.22 rows=28490 width=0)
Index Cond: (started_at <= '2024-12-18 19:10:15.651839-06'::timestamp with time zone)
-> Bitmap Heap Scan on query_execution (cost=11.11..15.12 rows=1 width=10)
Recheck Cond: (id = "ANY_subquery".id)
-> Bitmap Index Scan on query_execution_pkey (cost=0.00..11.11 rows=1 width=0)
Index Cond: (id = "ANY_subquery".id)
(14 rows)
```
And this is because the db engine knows that most of my data is around
2025 and will just hit the whole table rather than hit an index and then
go return most of the table.
The logic we want here is basically, "work up the index on created_at
until we get to a notion of recent".
If we add an order by, we're close to that and things go much faster
again
```sql
clean=# explain DELETE FROM query_execution WHERE id IN (SELECT id FROM query_execution WHERE started_at <= '2025-01-02 01:10:15.651839+00' order by id asc LIMIT 50000);
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Delete on query_execution (cost=851615.84..1392207.62 rows=0 width=0)
-> Nested Loop (cost=851615.84..1392207.62 rows=50000 width=34)
-> Unique (cost=851608.39..851858.39 rows=50000 width=32)
-> Sort (cost=851608.39..851733.39 rows=50000 width=32)
Sort Key: "ANY_subquery".id
-> Subquery Scan on "ANY_subquery" (cost=0.55..847705.98 rows=50000 width=32)
-> Limit (cost=0.55..847205.98 rows=50000 width=4)
-> Index Scan using query_execution_pkey on query_execution query_execution_1 (cost=0.55..2767566.54 rows=163335 width=4)
Filter: (started_at <= '2025-01-01 19:10:15.651839-06'::timestamp with time zone)
-> Bitmap Heap Scan on query_execution (cost=7.44..11.46 rows=1 width=10)
Recheck Cond: (id = "ANY_subquery".id)
-> Bitmap Index Scan on query_execution_pkey (cost=0.00..7.44 rows=1 width=0)
Index Cond: (id = "ANY_subquery".id)
(13 rows)
```
It caused problems as lint-staged stashes files and if developer is not familiar with the mechanism, he thinks that his work is lost. To avoid confusion we remove this git hook
* skips model popup
* fix types
* port to ts
* reorganize acknowledgement code
* improve ack api, add unit test, fix missing key prop console warning for question action menu
* move schema file to correct location, improve e2e
* Make sure we can load User KV schemas when in a JAR (#52246)
* Make sure we can load user KV schemas in a JAR
* A few minor tweaks
- removed some unused code
- added some more comments to clarify when each function is used
- move `update-user-key-value-schema!` outside the `doseq`
* Fix tests
* A bit more clarity
* actually fix tests
* move prod implementation below dev implementation; couple other tweaks
* fix fn name
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* remove only usage in test
* lint and unit fixes
* fix unit test for real this time
* but actually... unit tests are fixed this time
---------
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
When we replace a breakout, we also update any related order-by. However, we weren't updating all fields. This
meant that the order-by and breakout weren't identical. This was fine-ish on postgres, because they compiled to the
same thing. However, on google bigquery, it expects that group bys and order bys will refer to the AS alias instead
of just duplicating the code for the aliased thing. The differences between order-by and breakout meant that we
didn't realize that the one was supposed to match the other, and so we duplicated the code and bigquery barfed.
Syncing more fields fixes that.
**Joined columns**
- `join__${IDENT_OF_JOIN_CLAUSE}__${IDENT_OF_COLUMN}` for explicit
joins
- `implicit_via__${IDENT_OF_FK_COLUMN}__->__${IDENT_OF_TARGET_COLUMN}`
for implicit joins
These fulfill the contract for column `:ident`s:
- Unique identity within one stage of a query
- Globally unique
- Note that the same implicitly joined column (eg.
`Products via Orders.PRODUCT_ID`) might be referenced in
many queries, but it's the **same column** - same `display_name`,
types, etc.
**New test metadata**
In order to test these properly, I needed test metadata with more
permutations of FKs.
I've added three new tables to `metabase.lib.test-metadata`, which are a
simplified version of Github issues, users and comments. The tables are
`:gh/issues`, `:gh/users` and `:gh/comments`.
Some notable things:
- `:gh/users` has a `:type/Text` PK, ie. Github usernames
- `:gh/issues` and `:gh/comments` have `:type/UUID` PKs
- `:gh/issues` has **two** FKs to `:gh/users`: `:reporter-id` and
`:assignee-id`.
- `:gh/comments` has a **self FK**: `:reply-to`
- `:gh/issues :closed-at` is a `NULL`able `:type/DateTime` column, which
we lacked previously
* phase #1: remove all uses of openNativeEditor without a parameter
* phase #2: remove all uses with parameter
* remove useless test
* fix failing test cases
* fix failing tests
* add db id and native filter interaction
* fix flaky tests
* Support multiple operators in expectOp
* Refactor
* Show errors from parser
* Revert recursive-parser changes
* Pass human-friendly operator name to expectOp
* Fix typo
* Require comma if next token is not closing parenthesis
* Add a repro for #15892
* Update existing test
This drill thru was recently enabled for multi-stage queries in #51950, but there is a bug which causes the
QueryColumnPicker to fail if a join is present. Disable the drill thru for now while we work on a fix.
Closes#52221
Related to #52236
* Upgrade postgres app db version
The more the marrier
* Update docs/installation-and-operation/configuring-application-database.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Add repro for #50925
* Add a test
* Fix the case with whitespace following an incomplete token
* Add test case with bracket identifier following an incomplete token
* Add comments
* Thanks vscode
* Fix the case with bracket identifier following an incomplete token
* Simplify unit tests
* Fix the logic
* Update comment
* Fix it
* Refactor
* Throw different error when using a known name that is not a metric
* Drop checking for segments as they cannot be used in aggregation expressions anyway
* Add tests
* Remove redundant withName: true
* Wrap with describe
* Make naming consistent
* Improve copy
If a card with a default value doesn't actually require a value, letting people clear the default value can be useful.
However, if the card does require a value, letting people clear the default just throws an error. This is a
completely useless result, so we should use the default value in that case. See [here](https://metaboat.slack.com/archives/C0645JP1W81/p1719233020129649) for more info.
* Add a warning/disclaimer for H2 uploads
* Expand the component test
* Use the built in margin bottom
* Show the additional warning on hosted instances only
* Remove `mb/once` tag since we don't use it anymore
* Add `^:mb/old-migrations-tests` tag.
* Update GH Actions config
* Cleanup
* Keep the migrations test files in order
* Quote excluded tags
* Update running.md
Some little details about running Java apps on prod
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
* Update mysql.md
Add a note about vitess and planetscale
* Update docs/databases/connections/mysql.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* [Backend modularization] Remove `metabase.models`
* I guess we don't really need an ordered map
* Remove unneeded require of models
* Small fixes 🔧
* Test fix 🔧
* Fix metabase.api.embed.common
* Fix some insane code that was using symbols for raw table names
* We need to do model resolution for table names
* Don't test EE namespaces in OSS
* Secret handling in details
Secret handling in details was very confusing. This PR consolidates most
of the logic into models.secret around toucan2 lifecycles of
model/Database.
Some of the problems fixed here:
- path and value confusion. It was possible to swap between uploaded
files and file-paths and end up with the uploaded file being treated
as a file-path and leaking to users in logs, APIs, and UIs.
- Unclear management of secret properties in details. The
responsibility for taking in secrets in details, saving to a Secret,
and then ensuring those incoming secrets do not get saved in details
was spread out all over the place. This made it possible to save a
secret-value in details.
- Unclear hydration of secret values for connection testing. How a
driver was supposed to get a secret and which properties might be in
details was unclear and inconsistent. Often mangling strings and
files returned by Secret api was necessary.
- Encoding problems. Uploaded files were being mangled by taking bytes
into a string and back to bytes to save. Ensure that the bytes given
are the bytes saved.
* Fix merge
* Fix tests
* Fix db creation
* Fix db creation in other tests
* Manually add updated_at to database
* Fix tests
* Fix tests
* Fix after-select input
* Wrap in data-uri
* Fallback for -value if base64 encoding failed
* If not data-uri, then just turn string to utf8 bytes
* Poking at oracle failures
* Oracle wrap bytes in data-uri
* Don't clean secret properties from oracle
* Fix base64 typo and add test util for data-uri wrap
* Fix snowflake test
* Another snowflake test
* Update src/metabase/models/secret.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update src/metabase/models/secret.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update src/metabase/models/secret.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update test/metabase/test/util.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update src/metabase/util/jvm.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update src/metabase/util/jvm.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
---------
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update comment in automatic-insights drill to point to new github issue
* Move helpers from lib.drill-thru.common to lib.underlying
These funcs deal with "underlying" columns, so move them into lib.underlying.
aggregation-sourced?
breakout-sourced?
strictly-underlying-aggregation?
* Consistent naming for drill-thru multi-stage query tests
* Update multi-stage query drills tests to use new test helpers
* Homogenize apis for drill thru test helpers
Rename prepend-filter-to-stage to prepend-filter-to-test-expectation-stage and make the calling interface match
that of append-filter-stage-to-test-expectation.
* cljfmt
* Revise new table permissions logic to handle sandboxes
* tweak new perm logic
* make sure cond has a default
* fix comments
* tweak defenterprise conditions
* [Backend modularization] `query-analysis` cleanup
* Update Kondo config
* Add CODEOWNERS
* metabase.task.sweep-query-analysis => metabase.query-analysis.task.sweep-query-analysis
* Move test util
* Module linter should ignore my test-setup namespace
* Make sure query analysis init is loaded in tests that need it
* Deparallelize some of the tests
This change happened in peer
https://github.com/metabase/metabase/pull/51023 and is manifesting now
it seems.
The removal of the call to `(classloader/the-classloader)` is what is
hurting us.
Only clickhouse cloud instances can be set as a suitable upload spot so
i'm doing the following changes in the local jar repro to fix it up:
```clojure
(in-ns 'metabase.driver.clickhouse)
(defmethod driver/database-supports? [:clickhouse :uploads] [_driver _feature db]
true)
(in-ns 'metabase.public-settings)
(let [db_id 2, schema_name "default", table_prefix "upload_"]
(t2/update! :model/Database db_id {:uploads_enabled true
:uploads_schema_name schema_name
:uploads_table_prefix table_prefix}))
```
* [debug] add storybook helper
* Make error on InteractiveQuestion consistent
* Make error on StaticQuestion consistent
* [debug] static question stories
* [debug] InteractiveDashboard Stories
* Make error on InteractiveDashboard consistent
* [debug] StaticDashboard stories
* Make error on StaticDashboard consistent
* [debug] EditableDashboard stories
* Make error on EditableDashboard consistent
* Fix CreateQuestion not working after fixing entity ID states
* Add tests for InteractiveQuestion
* Add StaticQuestion tests
* Add InteractiveDashboard tests
* Add StaticDashboard tests
* Add EditableDashboard tests
* Make variable names consistent
* Extract resource not found error to a component
* Fix missing return
* Make storybook usable and reduce duplications
* Don't roll my own util
* Update error colors following the new design
* Update loki snapshots
* Fix unit test expectation to hex
* Make colors easier to understand with comments
* Update SDK error style
* Update error components copy
* Use Code from metabase/ui
* Remove change accidentally commited
* Fix InteractiveQuestion renders question not found briefly
metabase#51969
* WIP First attempt at passing start of week option to static viz bundle
* Cleaning debug lines
* set moment and dayjs start of week day setting in static viz
* reverting index.ts
* Adding backend test for checking first day of week setting is respected in static viz
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
* Update mysql.md
Add an additional flag as a note, since I've hit this one today with a customer
* Update docs/databases/connections/mysql.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Update activating-the-enterprise-edition.md
another little nitpick, since people tend to get creative with these things and then we end up debugging endless hours
* edit
---------
Co-authored-by: jeff bruemmer <jeff.bruemmer@gmail.com>
* initial impl
* better error handling, smooth scroll for adding existing cards, clean up of code in general
* add e2e test
* fix type error
* pr feedback
* reworks implementation to consume value from url then clear it, follows existing code patterns better
* clean up and test improvements
* fix e2e tests
* fix type issues
* pr feedback
* mostly working solution w/ tests
* add basic type impl, change interface to return obj from hook, update tests to account for both
* add types to setup mock endpoint fns
* hacks solution together to test if everything passes CI
* real implementation, removes hacked approach
* patch types
* add test coverage for patched package
* add one more unit test
* Update error colors following the new design
* Update loki snapshots
* Fix unit test expectation to hex
* Make colors easier to understand with comments
* Fix E2E tests
* Make setupModelPersistenceEndpoints work with multiple models
* Make test wait for data
* Improve tags invalidation
* Improve types
* Remove entity.ListLoader usage in UploadSettingsForm (#51909)
* Remove entity.ListLoader usage from ModelCacheRefreshJobs (#51910)
* Remove entity.ListLoader usage from SavedEntityList (#51914)
* Remove entity.ListLoader usage from SearchApp (#51915)
* Remove entity.ListLoader usage from QuestionList (#51912)
* Remove entity.ListLoader usage from SearchResults (#51913)
* Remove `entity.ListLoader` from `CollectionContentView` (#51916)
* Remove entity.ListLoader usage from CollectionContentView
* CI Failure notice improvements (#51885)
* don't notify slack on flaky job failing
also add more info to failure notice
* this is nicer
* add entity insertion library (#47895)
* add entity insertion library
- this lets us use `add-load/from-script`
* update add load examples
* Update year in the README (#51923)
* reset cache when starting cypress locally (#51921)
* Don't build FE for driver tests that don't need it; shave 2 minutes off of driver tests (#51884)
* Don't build FE for driver tests
* Try this again
* Consolidate MySQL job configs
* Oops fix conditional flags for test-driver
* More MySQL consolidation
* Oops
* Add a new job that waits for all other jobs (so we can require one single job going forward)
* Restore deleted code
* Try adding a Driver Tests (Summary) workflow
* How about a nice job name
* Don't love YAML inline array syntax
* Try fixing the build-static-viz configuration again
* Try fixing the build-static-viz configuration again
* Try this again
* Fix MySQL SSL config
* Try fixing the drivers summary workflow
* re-remove docs update block (#51928)
* upload title (#51931)
* Fix formatting of month-of-year when there are 53 weeks in the year (#51892)
* New deactivated_at field on users (#51765)
* New deactivated_at field on users
* fix migration syntax
* Setting alignment to top for icons in subscription and alert emails (#51886)
* Correct docstring for series (#51840)
Minor nitpicks on some endpoint docstrings
* Dont escape chars of header dashcards (#51880)
* fix rerun job
* feat(sdk): Interactive Question Chart Settings Dropdown #50976
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: bryan <bryan.maass@gmail.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Remove `entity.ListLoader` from `CollectionItemsTable` - approach v3 (#51958)
* Break CollectionItemsTable down to avoid using Search.ListLoader
* ESLint
* Oops
* Fix auto-upload effect in ModelUploadModal
* Fix types
* Fix the effect running too often
* Remove redundant code
* Reduce diff
* Remove old EntityListLoader and rename new EntityListLoaderRtkQuery to EntityListLoader
* Remove PaginationState
* Remove EntityType
* Fix effect
* Fix effect
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: bryan <bryan.maass@gmail.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Make setupModelPersistenceEndpoints work with multiple models
* Make test wait for data
* Improve tags invalidation
* Improve types
* Remove entity.ListLoader usage in UploadSettingsForm (#51909)
* Remove entity.ListLoader usage from ModelCacheRefreshJobs (#51910)
* Remove entity.ListLoader usage from SavedEntityList (#51914)
* Remove entity.ListLoader usage from SearchApp (#51915)
* Remove entity.ListLoader usage from QuestionList (#51912)
* Remove entity.ListLoader usage from SearchResults (#51913)
* Remove `entity.ListLoader` from `CollectionContentView` (#51916)
* Remove entity.ListLoader usage from CollectionContentView
* CI Failure notice improvements (#51885)
* don't notify slack on flaky job failing
also add more info to failure notice
* this is nicer
* add entity insertion library (#47895)
* add entity insertion library
- this lets us use `add-load/from-script`
* update add load examples
* Update year in the README (#51923)
* reset cache when starting cypress locally (#51921)
* Don't build FE for driver tests that don't need it; shave 2 minutes off of driver tests (#51884)
* Don't build FE for driver tests
* Try this again
* Consolidate MySQL job configs
* Oops fix conditional flags for test-driver
* More MySQL consolidation
* Oops
* Add a new job that waits for all other jobs (so we can require one single job going forward)
* Restore deleted code
* Try adding a Driver Tests (Summary) workflow
* How about a nice job name
* Don't love YAML inline array syntax
* Try fixing the build-static-viz configuration again
* Try fixing the build-static-viz configuration again
* Try this again
* Fix MySQL SSL config
* Try fixing the drivers summary workflow
* re-remove docs update block (#51928)
* upload title (#51931)
* Fix formatting of month-of-year when there are 53 weeks in the year (#51892)
* New deactivated_at field on users (#51765)
* New deactivated_at field on users
* fix migration syntax
* Setting alignment to top for icons in subscription and alert emails (#51886)
* Correct docstring for series (#51840)
Minor nitpicks on some endpoint docstrings
* Dont escape chars of header dashcards (#51880)
* fix rerun job
* feat(sdk): Interactive Question Chart Settings Dropdown #50976
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: bryan <bryan.maass@gmail.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Remove `entity.ListLoader` from `CollectionItemsTable` - approach v3 (#51958)
* Break CollectionItemsTable down to avoid using Search.ListLoader
* ESLint
* Oops
* Fix auto-upload effect in ModelUploadModal
* Fix types
* Fix the effect running too often
* Remove redundant code
* Reduce diff
* Fix effect
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: bryan <bryan.maass@gmail.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Don't select entire collections for permission checks
* Pull collection perm checks out as a CTE (wip)
* Cljfmt
* Refactor, fix tests
* Move comment, formatting fixes
* do a single query to fetch both the limited results and the total row count
* select more fields from collections
* inline values in the CTE to get it to work on H2
* fix e2e tests
* fix handling of limit = 0 edge case
* inline last ID
* Split Kondo lint fixes off into #51747
* Merge changes from #51660
* Reworked API documentation generation code
* Oops Markdown not YAML
* BIG documentation generation improvements
* Don't try to style tables ourselves
* Fix lint errors
* Fix tests 🔧
* Fix Kondo warnings
* Update OpenAPI dox generator to handle Defendpoint v2
* Update OpenAPI dox generator to handle Defendpoint v2
* Test fixes 🔧
* Fix uberjar compilation
* Static OpenAPI documentation generation
* change output to docs/api.html
* Update test
* PR feedback from @escherize
* add api.html
* Deprecate the old `defendpoint`
* Nuke the old version of the API documentation code (new version will be merged in #51872)
* Remove empty file
* Disable output validation in prod
* Fix the way we define defendpoint 2 handlers
* Fix deps.edn
---------
Co-authored-by: jeff bruemmer <jeff.bruemmer@gmail.com>
* Update docstring in zoom-in.timeseries
The comment is stale. Construction of dimensions from row data is now done in available-drill-thrus.
* Fix zoom-in.geographic drill thru for mutli-stage queries
* Fix return type schema for lib.underlying/top-level-column
* Add append-filter-stage-to-test-expectation to drill-thru test-utils
* Add tests for multi-stage queries to zoom-in.geographic tests
* add New Dashboard button and rework Create a new collection button to New Collection
* fix failing e2e tests
* fix unit tests
* allows users to create a new dashboard to save their dashboard question into
* remove usage of ref
* adds e2e test coverage for creating a dashboard on the go for a new question
* add unit tests
* fix type error
* clean up
* make test less flaky
* Fix quick-filter drill thru for multi-staged queries
* Add column-by-name and append-filter-stage to drill-thru test utils
* Add quick-filter tests for multi-stage queries
* Allow drill-thru tests to pass an update fn for :custom-{query,native,row}
The update fn will get passed the default value for the test case based on query-table and query-type. This allows
callers to easily modify or extend the default, rather than having to look it up or recreate it from scratch.
;; same as the standard test query, but with an extra filter stage
:custom-query #(lib.drill-thru.tu/append-filter-stage % "count")
* Use mu/defmethod and mu/defn for column-by-name and append-filter-stage
* Add lib.drill-thru.test-util/prepend-filter-to-stage
* Add lib.drill-thru.tu/test-drill-variants-with-merged-args
A helper function to make it easy to call test-returns-drill or test-drill-application with minor variations of some
base case (for example, drill behavior should remain unchanged when called with a query with filter stage after
aggregations).
* Use test-drill-variants-with-merged-args for quick-filter multi-stage tests
* Add multi-stage quick-filter tests for a couple of unaggregated queries
* Use lib.drill-thru.tu/column-by-name in metabase.lib.drill-thru-test
* Use lib.drill-thru.tu/column-by-name in lib.drill-thru.test-util.canned
* Support skipping tests in test-drill-variants-with-merged-args
* Allow variant-case to be a fn in test-drill-variants-with-merged-args
* Clean up test formatting
* Reduce diff
* feat(QB): combine chart type selector with viz settings sidebar
* fix typecheck
* fix unit tests
* chore(e2e): create helpers for viz settings and viz type buttons
* fix some tests
* fix most of the e2e tests
* fixup
* more fixes
* another test
* add a warning
* revert cross test changes
* chore: cleanup unused code
* render component directly
* Don't build FE for driver tests
* Try this again
* Consolidate MySQL job configs
* Oops fix conditional flags for test-driver
* More MySQL consolidation
* Oops
* Add a new job that waits for all other jobs (so we can require one single job going forward)
* Restore deleted code
* Try adding a Driver Tests (Summary) workflow
* How about a nice job name
* Don't love YAML inline array syntax
* Try fixing the build-static-viz configuration again
* Try fixing the build-static-viz configuration again
* Try this again
* Fix MySQL SSL config
* Try fixing the drivers summary workflow
* feat(QB): combine chart type selector with viz settings sidebar
* fix typecheck
* fix unit tests
* chore(e2e): create helpers for viz settings and viz type buttons
* fix some tests
* fix most of the e2e tests
* fixup
* more fixes
* another test
* add a warning
* revert cross test changes
* chore(QB): show small Join and CC icons for aggregation without breakout
* fix linting
* tests
* use compact prop and add another test
* revert provider
* fix memo
* Fix typo
* Fix broken memoization
* Use undefined instead of 0
* Introduce entityOptions
* Fix memoization
* Enable EntityListLoaderRtkQuery for alerts (#51501)
* Port bookmarks to use new `EntityListLoaderRtkQuery` (#51502)
* Fix types
* Enable EntityListLoaderRtkQuery for bookmarks
* Revert "Fix types"
This reverts commit f855c3a460.
* Enable EntityListLoaderRtkQuery for dashboards (#51503)
* Enable EntityListLoaderRtkQuery for persisted models (#51505)
* Enable EntityListLoaderRtkQuery for pulses (#51507)
* Enable EntityListLoaderRtkQuery for revisions (#51509)
* Enable EntityListLoaderRtkQuery for persisted segments (#51512)
* Enable EntityListLoaderRtkQuery for snippets (#51513)
* Enable EntityListLoaderRtkQuery for persisted questions (#51508)
* Enable EntityListLoaderRtkQuery for actions (#51515)
* Port tables to use new `EntityListLoaderRtkQuery` (#51518)
* Enable EntityListLoaderRtkQuery for tables
* Fix polling
* Port timelines to use new `EntityListLoaderRtkQuery` (#51516)
* Enable EntityListLoaderRtkQuery for timelines
* Add missing options
* Fix flaky test
* Port groups to use new `EntityListLoaderRtkQuery` (#51504)
* Enable EntityListLoaderRtkQuery for groups
* Bind action creators
* Fix flaky test
* Port users to use new `EntityListLoaderRtkQuery` (#51514)
* Enable EntityListLoaderRtkQuery for users
* Add missing options
* Fix crash
* Reimplement pagination
* Enable EntityListLoaderRtkQuery for search (#51568)
* Port schemas to use new `EntityListLoaderRtkQuery` (#51511)
* Enable EntityListLoaderRtkQuery for persisted schemas
* Add missing options
* Temporary commit
- just to save work at EOY
* Fix file upload
* Fix unit tests
* Fix data being set for different entityQuery
* Port collections to use new `EntityListLoaderRtkQuery` (#51519)
* Enable EntityListLoaderRtkQuery for collections
* Add missing options
* Improve typing
* Fix data being set for different entityQuery
* Fix destructuring crash
* Fix typing
* Pass options to RTK query where missing
* Use undefined instead of 0 for reloadInterval
* Port snippet collections to use new `EntityListLoaderRtkQuery` (#51847)
* Enable EntityListLoaderRtkQuery for snippet collections
* Introduce custom useListQuery
* Reduce diff
* Reduce diff
* Split Kondo lint fixes off into #51747
* Merge changes from #51660
* Reworked API documentation generation code
* Oops Markdown not YAML
* BIG documentation generation improvements
* Don't try to style tables ourselves
* Fix lint errors
* Fix tests 🔧
* Fix Kondo warnings
* Update OpenAPI dox generator to handle Defendpoint v2
* Update OpenAPI dox generator to handle Defendpoint v2
* Test fixes 🔧
* Fix uberjar compilation
* PR feedback from @escherize
* Deprecate the old `defendpoint`
* Nuke the old version of the API documentation code (new version will be merged in #51872)
* Remove empty file
* Disable output validation in prod
* Fix the way we define defendpoint 2 handlers
* start moving code around
* WIP fixes
* Test reorg, some more fixes
* Test fixes
* cljfmt, fix a test
* more cljfmt and linting
* cleanup, formatting fixes
* remove debugging code
* format again
* add some ns docstrings
* fix: Change pool acquireRetryAttemps to 1
Fixes: #51176
* Address PR review and only test dbs with username auth
* Fix model ref
* Exclude hive-like connections
* Ensure user and password
Updates JVM, JS and unit test `MetadataProvider` implementations to include `:ident` fields for all columns.
These idents need to be globally unique and serdes portable; therefore they are not randomized.
Instead they are built from the database, schema, table and field names, eg.
```clojure
{:ident "field__Sample Database__PUBLIC__Orders__CREATED_AT"
#_...}
```
**Note:** These `:ident` fields are not included on `/query_metadata` and other such API calls (currently).
They could be, but they can just as easily be synthesized from the DB, table and field in the JS metadata
implementation.
When you order by an aggregation, mbql v2 references the aggregation by uuid. However, that makes the query hash to a different value each time, because you get a different uuid each time you hash it. Converting the query to reference by index during the hashing process fixes this.
* Ensure database is fully encrypted if key is set, fail f database is encrypted and key is not set
* Added "remove-encryption" command as a more obvious alternative to rotate-encryption-key with an empty key
* fix broken link on BE
* Frontend
* update switch styles
* Fix type errors
* Fix BE tests
* fix malli schema
* logic tweak - don't show the switch if the target model is database
* cljfmt
* Add cypress test
* localize strings
* simplify form updates
* remove unneeded description style
* remove unused enabled? prop
* revert unintended change to cypress helper
* Make renaming options optional in lib.underlying/top-level-column
* Construct dimensions from row data in available-drill-thrus
Construct dimensions from row data once in available-drill-thrus, rather than doing it in each drill that needs it
separately. The dimensions must be constructed in available-drill-thrus since some drills are expected to
return-drills-for-dimensions?, but this also has the side benefit of reducing code duplication by lifting the
rows->dimensions conversion up into a single place.
* Fix the zoom-in.binning drill thru for multi-stage queries
* Add tests for zoom-in.binning with multi-stage queries
* cljfmt
* Explicitly disable the automatic-insights drill for multi-stage queries
Previously, this drill will was implicitly disabled for multi-stage queries because the FE would not pass dimensions
for queries with underlying breakouts / aggregations. Now that dimensions are reconstructed in available-drill-thrus,
automatic-insights drill is available again, but if you try to actually run the drill you get a big scary "Something
went wrong" error, so just explicitly disable the drill until we fix it.
* Update test/metabase/lib/drill_thru/zoom_in_bins_test.cljc
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Review feedback: not-empty -> seq
* Reivew feeback: shadow rather than using a single-quote suffix
* Review feedback: underyling -> underlying
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Fix date widgets dark theme
* Update more outdated loki snapshots
* Add a story to theme date filters missing dark theme
* Theme relative current widget and co.
* Theme date exclude widget
* Theme relative date interval widget and co.
* Update more snapshots from border color change
* Fix comment on the wrong line
* Fix light theme story switching + mock relative date stories
* Review: Remove unused function arg
* Update snapshots that change once a year 🤦
That is: aggregations, breakouts, expressions and joins.
Nothing is consuming these yet, but the BE populates them when reading cards.
MBQL changes
=============
These new `:ident`s have been present for new clauses for a few weeks, but now they'll be generated for existing queries on read.
The new idents are stable, including across serialization. They aren't written back until the card is saved again.
Backfilling
===========
Here's the constraints for generating idents for pre-existing clauses:
- If the same query exists across serdes instances, produce the same ident
- Globally unique, even if:
- two cards have identical queries
- two stages of the same card have identical `:count` aggregations.
- a single stage has multiple aggregations of the same type
- a single stage has duplicate breakouts with different time granularity
- Never change, even if the query does
- mbql-lib assumes these are (in theory) randomly generated, and never examines or updates them.
- If new clauses are added, they get random idents.
- If clauses are removed, that's no problem.
- If clauses are edited (eg. changing an expression name) then the BE *would* generate a new ident!
- But when the card gets saved, the original ident with the original expression name is enshrined forever as the ident - it just needs to be unique!
The backfilled idents have this form:
```clojure
"${kind}_${entity_id of the card}@${stage-number}__${name or index}"
;; eg.
"aggregation_ThisIsAValidNanoIdxxx@0__1"
"breakout_ThisIsAValidNanoIdxxx@1__1"
"expression_ThisIsAValidNanoIdxxx@0__exprname"
"join_ThisIsAValidNanoIdxxx@1__alias"
```
* Add M2M Oauth to Databricks Driver
* Add toggle for PAT/M2M Oauth
Introduces a 'Use Personal Access Token' toggle button which shows different UI components when setting up the connection. The implementation was inspired by @snoe in #51249
* Address feedback from @ibrdnk
Addresses the feedback related to #pullrequestreview-2515818008
* Default M2M, Add conditional Merge, Reduce whitespace
* Update docs/databases/connections/databricks.md
Co-authored-by: Jeff Bruemmer <jeffbruemmer@gmail.com>
* Reset FieldValuesWidget to v50 state
* Revert some changes in SingleSelectListField to v50
* Make onSearchChange optional
* Use RowValue instead of string
* Add test-ids to FieldValuesWidget inputs
* Add testing matrix for all parameter combinations
* Add filter matrix test to github workflow
* Fix unit test
* Render custom option value when it exists
* Fix e2e test
* Fix remove value helper
* Fix e2e test
* Render custom label on TokenField
* Update Loki Snapshots
* Use css colors for TokenFieldItem
* Make TokenField dark-mode aware
* Make TokenFieldItem dark-mode aware
* Update Loki Snapshots
* Fix e2e test after merge
* Add listbox role to tokenfield values
* Fix more e2e tests
* Fix e2e test for listbox
* Remove commented out code
* Fix e2e test
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* fixes showing admin links to not admins in the command palette
* updates calculation to take into account EE advanced permissions allowing certain users permissions to change settings who are not admins
* admin routes for non-admins
* fix unit tests
* actually fix unit tests
* adding checks for other permissions
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Azure Synapse doesn't like it when you give it a uuid parameter, even though synapse does support uuids in general.
Converting those uuids to strings beforehand fixes this error, and regular SQL Server is also fine with that.
* prevent dashboard from appearing in breadcrumbs when not on a DQ page
* updating unit test with location
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* fix: add supports for xx-YY locales in embed/public-links
* Generate locales.json
* [WIP] Read locales.json
* wip FE implementation
* refactor + unit and sdk component tests
* fixes after the merge, plus tests and cleand up logic
* fix file name in string
* change FE logic to use locales from the instance settings
* revert json generation
* remove leftover comment
* add comment about the code to keep in sync
* migrate e2e tests for static/public to use #locale=de-CH so we test that
it's using the fallback logic
* fix kondo error
Co-authored-by: dpsutton <dan@dpsutton.com>
* fix unit test
* fix more jest tests
* cy.request -> cy.intercept
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* fix: add supports for xx-YY locales in embed/public-links
* Generate locales.json
* [WIP] Read locales.json
* wip FE implementation
* refactor + unit and sdk component tests
* fixes after the merge, plus tests and cleand up logic
* fix file name in string
* change FE logic to use locales from the instance settings
* revert json generation
* remove leftover comment
* add comment about the code to keep in sync
* fix kondo error
Co-authored-by: dpsutton <dan@dpsutton.com>
* fix unit test
* fix more jest tests
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* chore: rework pickers in querying folder to css modules
* more css
* more css
* more css
* fix typecheck
* fix ui
* fix ui
* drop unused code
* fix ui
* fix css
* fix unit test
* revert color changes
* fix join color
* revert previous solution and move code into question question picker
* convert QuestionPicker to typescript
* port to css module
* refine button styles
* fix some e2e
* fix last failing e2e
* fe implementation for sessions
* fix styles if only one button is showing
* user css va for border colors
* fix failing unit tests and move new unit tests into existing file i had not noticed
* fix lint
* short circuit tracking for ignored requests
- add fall-through for ignored embedding requests
* add test for ignored status codes
* remove cruft
* add logging for unknown clients or statuses
* Fix some e2e tests in custom-column and view-data (#51431)
* warn on reflection!
---------
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
Using secrets.METABASE_AUTOMATION_USER_TOKEN causes the auto-approve PR step to fail with an error like
Can not approve your own pull request
Revert to using secrets.GITHUB_TOKEN instead, which matches what the auto-backport.yml workflow does and what this
workflow did previously.
* Add EntityListLoaderRtkQuery boilerplate
* Add loaded prop
* Add metadata prop to children
* Improve typing for entity selectors
* Add missing selector typing
* Adjust list loader
* Drop dispatchApiErrorEvent, entityAlias
* Add listName prop
* Conditionally use entityListLoaderRtkQuery
* Add transformResponse
* Enable new loader for database entity
* Update typing
* Add todo
* Remove actionCreators
* Update TODO
* Use loadListLegacy
* Fix condition
* Add dynamic reload* prop to children
* Handle onLoaded prop
* Reproduce isReloading behavior with isLoading
* Add support for reloadInterval prop
* Fix list state path when query is empty
* Migrate sample database endpoint
* Update type
* Simplify transformResponse
* Use modern syntax
* Type the wrapper
* Name the wrapper components
* Improve typing
* chore: rework mix components to css modules
* more css
* more css
* more css
* more css
* rework last pieces of emotion to css modules
* remove failing test
* address review
* fix view button styling
* fix view button styling
* drop not needed change
This PR builds on #50420 by adding an e2e test that the user can press the back button to return from the notebook editor to the homepage. Previously this was impossible due to a pop-under bug (#50971).
This PR introduces a simple way of avoiding pop-under bugs:
* First, we **give all overlays the same z-index.** It's 200, above everything else. SDK users can change this value.
* Second, we **prevent eager portal rendering**. On master, overlays are appended to the DOM inside an empty div, called a "portal". This portal is appended on mount. It stays hidden until the overlay is opened. In this branch, we stop portals from rendering eagerly. Instead, portals are only appended when the overlay is opened, and removed when the overlay is closed. This way, the most recently opened overlay is always last in the DOM.
Since the most recently opened overlay is last in the DOM, and since all overlays have the same z-index, the most recently opened overlay always appears on top.
This PR:
* closes#51256
* closes#45468
* closes#49466
* closes#50943
* closes#50971
* provides a more robust solution for two issues that previously received a quick, "minimum viable fix": #45469 and #48256
* improves the health of the codebase by removing hacks
* step1: add MetabaseReduxProvider and make our hooks + connect fn use it
step1: add MetabaseReduxProvider and make our hooks + connect fn use it
* step 2: run codemod to change the imports
step 2: run codemod to change the imports
* step 3: manual fixes
* step 4: e2e test for sdk
* adjusting webhook test response ui
* missed i18n, updating test to check for test response
* show response on error, not success
* parse the request as json response
---------
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Add EntityObjectLoaderRtkQuery boilerplate
* Incorporate EntityType logic into the new component
* Improve comment
* Add childrenProps
* Move component into its own directory, extract types to a separate file
* Add entityObjectLoaderRtkQuery and temporary mappings
* Hook up the useGetQuery hook (tables only)
* Pass all the props to children
* Add Playground
* Gather more info in the Playground
* Integrate RTK with redux store
* Adjust props passed down
* Improve typing
* Make the request hook dynamic
* Add missing id in the query
* Handle fetchMetadata
* Change requestType to fetchType
* Add rtk mappings for databases in entity framework
* Use requestType to get request state path
* Enable new component everywhere
* Use legacy loader for entities without rtk mappings
* Improve comment
* Handle fetchMetadataAndForeignTables
* Remove unused function
* Handle reload and remove
* Add TODO
* Add transformResponse option and use it for fetchIdFields
* Format code
* Remove Playground
* Remove unused getUseGetListQuery
* Add rtk mapping for alerts
* Add rtk mapping for dashboards
* Add rtk mapping for groups
* Add rtk mapping for pulses
* Remove options
* Fix permissions tests
* Fix notifications tests
* Add rtk mapping for segments
* Use params instead of body
- It's leftover because this code got reverted after body -> params refactoring for GET endpoint has been merged to master
* Add rtk mapping for snippet collections
* Add TODO
* ESLint
* ESLint
* Update TODO
* Deprecate entity HOCs
* Update TODO
* Revert other entities for now
* Revert other entities for now
* Bring tables back
* Improve todos
* Add comment
* Improve comment
* Remove TODO
* Use reload param
* Remove properties prop
This was used to short-circuit and avoid triggering api calls if we already had an object with given attributes in cache.
With RTK query now managing the cache, we don't want to reproduce this behavior.
* Remove reloadInterval
This is used in EntityListLoader, not EntityObjectLoader
* Don't use setRequestPromise action
* Fix deprecations
* Format code
* Fix flake :(
* Try to fix flake
* Revert "Try to fix flake"
This reverts commit 6408a22a37.
* Try to fix flake
* Fix UseQuery type
* Implement useGetMetadataAndForeignTables
* Fix useMemo and move things down
* Fix typo
* Bring back original hack
* Move file
* Enable RTK for alerts
* Undeprecate a test
* Enable RTK for dashboards
* Port Segments to use new EntityObjectLoaderRtkQuery (#50635)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* [WIP] Port Groups to use new EntityObjectLoaderRtkQuery (#50631)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Fix failing test
* Enable RTK for dashboards
* Port Snippet collections to use new EntityObjectLoaderRtkQuery (#50636)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* Revert "Enable RTK for segments"
This reverts commit ef65fe78f3.
* Enable RTK for snippet collections
* Fix skipToken passing
* Enable RTK for dashboards
* Port Pulses to use new EntityObjectLoaderRtkQuery (#50633)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Fix hook usage
* Enable RTK for dashboards
* Port Fields to use new EntityObjectLoaderRtkQuery (#50639)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* Revert "Enable RTK for segments"
This reverts commit ef65fe78f3.
* Enable RTK for snippet collections
* Revert "Enable RTK for snippet collections"
This reverts commit e1439cb28b.
* Enable RTK for fields
* Account for skipToken
* Fix fetchFieldValues
* Optimize hooks
* Format code
* Fix loading state not kicking in during first render
* Rename
* Add comment
* Use consistent reference name for revisions
* Port Questions to use new EntityObjectLoaderRtkQuery (#50634)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Rename import
* Fix weirdness
- Apparently what makes the difference here is the lack of children render pattern ({childProps => <ComposedComponent {...childProps} />})
* Refactor
* Use isFetching instead of isLoading in EntityObjectLoaderRtkQuery, replace useEffect+useState with useMemo in SavedQuestionLoader
* Change order of HOCs to prevent loading state from redundantly kicking in
* Rename import
* [WIP] Port Persisted Models to use new EntityObjectLoaderRtkQuery (#50853)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Rename import
* Fix weirdness
- Apparently what makes the difference here is the lack of children render pattern ({childProps => <ComposedComponent {...childProps} />})
* Refactor
* Use isFetching instead of isLoading in EntityObjectLoaderRtkQuery, replace useEffect+useState with useMemo in SavedQuestionLoader
* Change order of HOCs to prevent loading state from redundantly kicking in
* Port persisted models
* Port schemas (#50854)
* Port snippets (#50855)
* Port timelines (#50858)
* Do TODOs
* Port timeline events (#50857)
* Port users (#50866)
* Port Collections to use new EntityObjectLoaderRtkQuery (#50869)
* Port collections
* Fix imports
* Drop unused params
* Add missing property
* Fix collection not getting loaded
* Replace useCollectionQuery with useGetCollectionQuery
* Remove useCollectionQuery
* Format code
* Skip request on null id
* Revert "Remove useCollectionQuery"
This reverts commit 9b1068e611.
* Revert "Replace useCollectionQuery with useGetCollectionQuery"
This reverts commit 0c5ee660a3.
* Fix test
* Implement EntityName with RTK query
* git merge --squash 50323-port-other-entities
* Use entity getName selector
* Update comment
* Remove BaseEntity
* Improve typing
* git merge --squash 50323-port-other-entities
* Enable RTK for dashboards
* Deprecate the component
* Remove entityObjectLoader usage
* Remove old EntityObjectLoader and replace it with new one
* git merge --squash 51213-entity-loader
* Fix unexpected rounding in csv & json exports when column multiplier is non integral
* Fixing table unit test
* Adding WIP test
* Adding test case for validating new scaling logic is applied appropriately in exports
* Reverting id-or-name change and adjusting new scaling unit test
* Updating formatting to fix cljfmt in CI
* removing debug def
* Preserving integral behavior if scale is nil
`query=` logic was returning false negatives when two queries were identical except for their `:ident`s.
This broke Metrics UI, for example, since it makes up a query based on the "old" and "new" metrics, and
compares those queries.
Now `:ident`s are ignored by `query=`, so two queries of the same shape are equal.
* PopoverWithRef, plus some other stuff
* removing commented code
* I've done terrible things
* cleanup
* adding comments
* lets see what this breaks
* e2e adjustmnet
* removing unused component
* Add EntityObjectLoaderRtkQuery boilerplate
* Incorporate EntityType logic into the new component
* Improve comment
* Add childrenProps
* Move component into its own directory, extract types to a separate file
* Add entityObjectLoaderRtkQuery and temporary mappings
* Hook up the useGetQuery hook (tables only)
* Pass all the props to children
* Add Playground
* Gather more info in the Playground
* Integrate RTK with redux store
* Adjust props passed down
* Improve typing
* Make the request hook dynamic
* Add missing id in the query
* Handle fetchMetadata
* Change requestType to fetchType
* Add rtk mappings for databases in entity framework
* Use requestType to get request state path
* Enable new component everywhere
* Use legacy loader for entities without rtk mappings
* Improve comment
* Handle fetchMetadataAndForeignTables
* Remove unused function
* Handle reload and remove
* Add TODO
* Add transformResponse option and use it for fetchIdFields
* Format code
* Remove Playground
* Remove unused getUseGetListQuery
* Add rtk mapping for alerts
* Add rtk mapping for dashboards
* Add rtk mapping for groups
* Add rtk mapping for pulses
* Remove options
* Fix permissions tests
* Fix notifications tests
* Add rtk mapping for segments
* Use params instead of body
- It's leftover because this code got reverted after body -> params refactoring for GET endpoint has been merged to master
* Add rtk mapping for snippet collections
* Add TODO
* ESLint
* ESLint
* Update TODO
* Deprecate entity HOCs
* Update TODO
* Revert other entities for now
* Revert other entities for now
* Bring tables back
* Improve todos
* Add comment
* Improve comment
* Remove TODO
* Use reload param
* Remove properties prop
This was used to short-circuit and avoid triggering api calls if we already had an object with given attributes in cache.
With RTK query now managing the cache, we don't want to reproduce this behavior.
* Remove reloadInterval
This is used in EntityListLoader, not EntityObjectLoader
* Don't use setRequestPromise action
* Fix deprecations
* Format code
* Fix flake :(
* Try to fix flake
* Revert "Try to fix flake"
This reverts commit 6408a22a37.
* Try to fix flake
* Fix UseQuery type
* Implement useGetMetadataAndForeignTables
* Fix useMemo and move things down
* Fix typo
* Bring back original hack
* Move file
* Enable RTK for alerts
* Undeprecate a test
* Enable RTK for dashboards
* Port Segments to use new EntityObjectLoaderRtkQuery (#50635)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* [WIP] Port Groups to use new EntityObjectLoaderRtkQuery (#50631)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Fix failing test
* Enable RTK for dashboards
* Port Snippet collections to use new EntityObjectLoaderRtkQuery (#50636)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* Revert "Enable RTK for segments"
This reverts commit ef65fe78f3.
* Enable RTK for snippet collections
* Fix skipToken passing
* Enable RTK for dashboards
* Port Pulses to use new EntityObjectLoaderRtkQuery (#50633)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Fix hook usage
* Enable RTK for dashboards
* Port Fields to use new EntityObjectLoaderRtkQuery (#50639)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* Revert "Enable RTK for segments"
This reverts commit ef65fe78f3.
* Enable RTK for snippet collections
* Revert "Enable RTK for snippet collections"
This reverts commit e1439cb28b.
* Enable RTK for fields
* Account for skipToken
* Fix fetchFieldValues
* Optimize hooks
* Format code
* Fix loading state not kicking in during first render
* Rename
* Add comment
* Use consistent reference name for revisions
* Port Questions to use new EntityObjectLoaderRtkQuery (#50634)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Rename import
* Fix weirdness
- Apparently what makes the difference here is the lack of children render pattern ({childProps => <ComposedComponent {...childProps} />})
* Refactor
* Use isFetching instead of isLoading in EntityObjectLoaderRtkQuery, replace useEffect+useState with useMemo in SavedQuestionLoader
* Change order of HOCs to prevent loading state from redundantly kicking in
* Rename import
* [WIP] Port Persisted Models to use new EntityObjectLoaderRtkQuery (#50853)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Rename import
* Fix weirdness
- Apparently what makes the difference here is the lack of children render pattern ({childProps => <ComposedComponent {...childProps} />})
* Refactor
* Use isFetching instead of isLoading in EntityObjectLoaderRtkQuery, replace useEffect+useState with useMemo in SavedQuestionLoader
* Change order of HOCs to prevent loading state from redundantly kicking in
* Port persisted models
* Port schemas (#50854)
* Port snippets (#50855)
* Port timelines (#50858)
* Do TODOs
* Port timeline events (#50857)
* Port users (#50866)
* Port Collections to use new EntityObjectLoaderRtkQuery (#50869)
* Port collections
* Fix imports
* Drop unused params
* Add missing property
* Fix collection not getting loaded
* Replace useCollectionQuery with useGetCollectionQuery
* Remove useCollectionQuery
* Format code
* Skip request on null id
* Revert "Remove useCollectionQuery"
This reverts commit 9b1068e611.
* Revert "Replace useCollectionQuery with useGetCollectionQuery"
This reverts commit 0c5ee660a3.
* Fix test
* Implement EntityName with RTK query
* git merge --squash 50323-port-other-entities
* Use entity getName selector
* Update comment
* Remove BaseEntity
* Improve typing
* git merge --squash 50323-port-other-entities
* Enable RTK for dashboards
* Deprecate the component
* Fix flaky test
* Fix flake in all tests
* Use useGetActionQuery in ActionExecuteModal
* Remove use-action-query
* Remove Actions.load usage in ActionCreator
* Remove unused initialAction prop
* Remove redundant prop
* Use entityCompatibleQuery
* Update unit tests to account for RTK Query refetching
* Fix unit tests
* Fix unit tests
* Fix unit tests
* don't check for enterprise release tag anymore
* use oss tag for release channel tagging
* add error catcher
* don't trigger release log from tag job
* wip nextjs compat docs
* explain how to do manual wrapping
* doocs
* better explanation of the index file
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Update docs/embedding/sdk/next-js.md
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Get rid of enitity.Loader in PinnedQuestionLoader
* Set errorIcon
* Get rid of enitity.Loader in TargetClickMappings
* Remove duplication
* Get rid of enitity.Loader in LinkedField
* Convert metabase-lib/v1/metadata/utils/schema.js to TS
* Extract useGetSchemas
* Revert "Extract useGetSchemas"
This reverts commit 2271bd3aa7.
* Mark optional props as optional
* Properly handle skipToken
* Remove Schema.Loader from FieldMappingSelect
- This was added to fix metabase#19451 but seems no longer needed as the repro passes without it
- See 83f4097e87
* Update unit tests to use number as DatabaseId
* Extract DataSourceCrumbs and refactor it to TS
* Extract SourceDatasetBreadcrumbs and refactor it to TS
* Extract SourceQuestionBreadcrumbs and refactor it to TS
* Remove Questions.Loader from SourceQuestionBreadcrumbs
* Drop entity.Loader
* Update unit tests
* Drop checkNotNull usage
* Add missing -test suffix to zoom-in-timeseries test name
* Add missing -test suffix to underlying-records test name
* Fix distribution drill thru for multi-stage queries
Ensure the distribution drill thru is not returned by available-drill-thrus when a user clicks on an aggregation or
breakout column, even if there are additional query stages after the last aggregation / breakout stage.
* only create a single release for each release
* remove dead file
* little bug fixes
* update tests
* add link to metabase cloud in release notes
* remove checksums from release notes
Some funcs were added to lib.drill-thru.common for the recent fix to the zoom-in.timeseries drill. Update the
underlying-records drill to use the common lib code rather than duplicating it.
`replace-clause` is for editing an existing clause, which should
*preserve* its identity. But the new clause was getting its own
random ident which didn't match the original.
I've added tests for this, and fixed the `lib.remove-replace` logic to
make them pass.
Hat tip to Mike for catching this case!
Previously, cumulative aggregators were being done in clojure after the mongo query was finished. However, this
didn't work on multi-step queries where the cumulative aggregator wasn't the last step (for example, do a cumulative
sum and then filter the result). Implementing cumulative operators directly in mongo fixes this issue.
* Fix#51020
* Add a repro for #51020
* Improve test name
* Update comment and naming
* Improve diff
* Add repro for when primary key is not called 'id'
* Update repro
* Drop prefetchesInitialValues & getValuesForUpdate
It's unclear what the real intention behind these two was, but it's clear that it made two differences:
(1) It decreased the probability of overwriting someone else's changes (if someone else saved them after we opened the form)
(2) It decreases the payload sent over the wire
(1) We can ignore this, as this problem is prevalent everywhere in our app
(2) This is negligible
Dropping these is simpler than enhancing getValuesForUpdate with extra logic for primary keys (and who knows what else)
* Break beforeEach into 2
* Remove no longer valid test
* save only grid on fixed width dashboards
* split export pdf into pages
* trim dashboard export pdf into pages
* specs
* add padding to pdf pages
* dark mode
* skip tracking non react-sdk x-metabase-clients
- Do not do promethus tracking for sdk.
- tracking ignores x-metabase-client headers when they're not == "embedding-sdk-react"
* cleanup indentation
* add prometheus tracking counters for iframe
* remedy 24h query execution inaccuracy
We were getting the `dashboard_tab_id` from the existing cards on the
first tab... but if no such card exists (the first tab is empty) then we
end up with a dashboard with tabs, but a card with
`dashboard_tab_id=null`, which doesn't show up.
This also fixes a bug in the revision history: we were publishing an
event for the dashboard being updated. The object was the updated
dashboard, but it had the old dashcards that we'd hydrated in order to
autoplace. We can dissoc the dashcards to ensure that the handler
hydrates them with the fresh, updated dashcards.
* checks if there are dashboards in the personal collection to check if we should show items below
* adding e2e test
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* add bug report to command palette and profile menu
add bug report to command palette and profile menu
* refactor and update diagnostic window modal
* add analytics to track bug reporter modal activity
* add file id to to the upload-file method
* add the ability to collect browser information for diagnostic window
* implement feedback from PR comments
* Update frontend/src/metabase/components/ErrorPages/ErrorDiagnosticModal.tsx
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* implement feedback from PR comments #2
* rename "file a bug" to "report an issue"
* resolve conflict
* fix lint issues
* fix bad merge conflict resolution
* fix failing onboarding tests
---------
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* wip: better documentation for developers that need to run the sdk
locally
* morning fixes
* remove developers-guide/sdk in favor of dev.md in the sdk folder
* add note about using the sdk with link to the docs
* fix mising word
* addressed feedback
* added notes about the envs for sdk e2e test
* Prevent newly opened items from expanding outside viewport
* Expand the unti test coverage
* Use the existing helper
* Change scroll view to center
This works better because it ensures that the top bar doesn't cover the element.
* Update frontend/src/metabase/home/components/Onboarding/Onboarding.tsx
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Extract the `stopVideo` function
* Fix the type name
* Set the top scroll margin on the `main` element
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Closes#49682. Closes#49683.
Updates `lib.convert` and legacy MBQL normalization to allow
`:ident`s, and fixes some QP middleware and utils so they can
still match up refs correctly.
No user-visible impact.
Newly created queries will have `:ident`s generated for joins,
aggregations, etc. by the previous PR. This starts saving them
into appDBs. It is still acceptable to have missing idents at
this stage, including for new queries created without the lib.
Pre-existing clauses don't have refs added yet.
* Fix loading in segments
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Our revision system has a race condition in the case of two concurrent requests to modify the same object. You can reproduce the exact issue here by opening two `psql` shells. In each one, run
```
begin;
insert into revision (model, model_id, user_id, timestamp, object, most_recent, metabase_version) values ('Dashboard', 1, 1, '2024-01-01T00:00:00Z', '{}', true, 'v1.0.0') returning id;
update revision set most_recent = false where model='Dashboard' and model_id = 10 and id != [[the id from above]];
```
after running all of the above, `commit` the transaction and note that **two** revisions are now marked as `most_recent`.
This is the way our revision system works: `t2/before-insert` sets `most_recent=true`, while `t2/after-insert` updates all the *other* revisions with `most_recent=false`.
The issue here is that when each transaction is running, it doesn't see
the uncommitted result of the other transaction (the new revision), so
the UPDATE does not set `most_recent` to `false` for either new
revision.
The solution here is to just select all existing `revision`s for this
model `FOR UPDATE`. This effectively locks the revisions for a single
entity, so we can't concurrently update revisions for the same entity.
* Add EntityObjectLoaderRtkQuery boilerplate
* Incorporate EntityType logic into the new component
* Improve comment
* Add childrenProps
* Move component into its own directory, extract types to a separate file
* Add entityObjectLoaderRtkQuery and temporary mappings
* Hook up the useGetQuery hook (tables only)
* Pass all the props to children
* Add Playground
* Gather more info in the Playground
* Integrate RTK with redux store
* Adjust props passed down
* Improve typing
* Make the request hook dynamic
* Add missing id in the query
* Handle fetchMetadata
* Change requestType to fetchType
* Add rtk mappings for databases in entity framework
* Use requestType to get request state path
* Enable new component everywhere
* Use legacy loader for entities without rtk mappings
* Improve comment
* Handle fetchMetadataAndForeignTables
* Remove unused function
* Handle reload and remove
* Add TODO
* Add transformResponse option and use it for fetchIdFields
* Format code
* Remove Playground
* Remove unused getUseGetListQuery
* Add rtk mapping for alerts
* Add rtk mapping for dashboards
* Add rtk mapping for groups
* Add rtk mapping for pulses
* Remove options
* Fix permissions tests
* Fix notifications tests
* Add rtk mapping for segments
* Use params instead of body
- It's leftover because this code got reverted after body -> params refactoring for GET endpoint has been merged to master
* Add rtk mapping for snippet collections
* Add TODO
* ESLint
* ESLint
* Update TODO
* Deprecate entity HOCs
* Update TODO
* Revert other entities for now
* Revert other entities for now
* Bring tables back
* Improve todos
* Add comment
* Improve comment
* Remove TODO
* Use reload param
* Remove properties prop
This was used to short-circuit and avoid triggering api calls if we already had an object with given attributes in cache.
With RTK query now managing the cache, we don't want to reproduce this behavior.
* Remove reloadInterval
This is used in EntityListLoader, not EntityObjectLoader
* Don't use setRequestPromise action
* Fix deprecations
* Format code
* Fix flake :(
* Try to fix flake
* Revert "Try to fix flake"
This reverts commit 6408a22a37.
* Try to fix flake
* Fix UseQuery type
* Implement useGetMetadataAndForeignTables
* Fix useMemo and move things down
* Fix typo
* Bring back original hack
* Move file
* Enable RTK for alerts
* Undeprecate a test
* Enable RTK for dashboards
* Port Segments to use new EntityObjectLoaderRtkQuery (#50635)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* [WIP] Port Groups to use new EntityObjectLoaderRtkQuery (#50631)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Fix failing test
* Enable RTK for dashboards
* Port Snippet collections to use new EntityObjectLoaderRtkQuery (#50636)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* Revert "Enable RTK for segments"
This reverts commit ef65fe78f3.
* Enable RTK for snippet collections
* Fix skipToken passing
* Enable RTK for dashboards
* Port Pulses to use new EntityObjectLoaderRtkQuery (#50633)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Fix hook usage
* Enable RTK for dashboards
* Port Fields to use new EntityObjectLoaderRtkQuery (#50639)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* Revert "Enable RTK for segments"
This reverts commit ef65fe78f3.
* Enable RTK for snippet collections
* Revert "Enable RTK for snippet collections"
This reverts commit e1439cb28b.
* Enable RTK for fields
* Account for skipToken
* Fix fetchFieldValues
* Optimize hooks
* Format code
* Fix loading state not kicking in during first render
* Rename
* Add comment
* Use consistent reference name for revisions
* Port Questions to use new EntityObjectLoaderRtkQuery (#50634)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Rename import
* Fix weirdness
- Apparently what makes the difference here is the lack of children render pattern ({childProps => <ComposedComponent {...childProps} />})
* Refactor
* Use isFetching instead of isLoading in EntityObjectLoaderRtkQuery, replace useEffect+useState with useMemo in SavedQuestionLoader
* Change order of HOCs to prevent loading state from redundantly kicking in
* Rename import
* [WIP] Port Persisted Models to use new EntityObjectLoaderRtkQuery (#50853)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Rename import
* Fix weirdness
- Apparently what makes the difference here is the lack of children render pattern ({childProps => <ComposedComponent {...childProps} />})
* Refactor
* Use isFetching instead of isLoading in EntityObjectLoaderRtkQuery, replace useEffect+useState with useMemo in SavedQuestionLoader
* Change order of HOCs to prevent loading state from redundantly kicking in
* Port persisted models
* Port schemas (#50854)
* Port snippets (#50855)
* Port timelines (#50858)
* Do TODOs
* Port timeline events (#50857)
* Port users (#50866)
* Port Collections to use new EntityObjectLoaderRtkQuery (#50869)
* Port collections
* Fix imports
* Drop unused params
* Add missing property
* Fix collection not getting loaded
* Replace useCollectionQuery with useGetCollectionQuery
* Remove useCollectionQuery
* Format code
* Skip request on null id
* Revert "Remove useCollectionQuery"
This reverts commit 9b1068e611.
* Revert "Replace useCollectionQuery with useGetCollectionQuery"
This reverts commit 0c5ee660a3.
* Fix test
* Extract dedicated providePermissionsGroupListQueryTags
* Format code
* Fix test setup
* Get rid of Database.Link usage in TableBrowser
* Fix browseDatabase typing
* Get rid of Collection.Link usage in DashboardMoveModal
* Get rid of Collection.Link usage in QuestionMoveToast
* Style links consistently
* Remove EntityLink
* Use database from the store if it's already there
* Remove filter
* Fix typo
* Use ts-pattern
* Bring spacing back
* Extract components for links
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* BE: Dashboard Questions
Setting this makes the card a "dashboard internal card" that should only
be used within a single dashboard, is moved along with the dashboard, etc.
prevent invalid inserts/updates
POST /api/card for dashboard internal cards
GET /api/collection/:id/items
Search
PUT /api/card/:id
WIP: last bits of card-in-dashboard API
Sticking point here is the copy, I think I need to do a small refactor
to make the code less hairy.
rm line
optionally show DQs for collection items
copying dashboards 🎉
wip
quick fix
Disallow moving dashboard questions & test archive
remove unused columns
fix argument order 🤦
serdes for dashboard questions
serialization baseline tests
Fix tests in `metabase.models.card`
Fix download exports test
Lint migrations file
cljfmt
cljfmt, again
the cljfmt job in CI doesn't run on this branch, it runs on the PR
branch merged with `origin/master` - which is nice, because it catches
cljfmt issues that would happen if you merged, but don't happen on your
branch, but also confusing, because it catches issues like
"metabase.api.common is never used", and then you stare at the file for
a while like "YES IT IS" before realizing what's going on.
migration to add idx for report_card.dashboard_id
don't rollback the `report_card.dashboard_id` bits
a couple missed API changes
- no DQs w/o DashboardCards in search
- no DQs in autocomplete
- fix check for "currently a dashboard question" in card validation
Fix some comments, clean up some existing code
Before, `cards-to-copy` returned just two lists of cards: those to
discard (no permissions) and those to copy. But `maybe-duplicate-cards!`
had some extra logic: if the card was a `model`, don't copy it, just
reference it.
I modified `cards-to-copy` to return three lists instead: cards to copy,
cards to discard, *and* cards to reference. This was initially just used
for making sure we copy Dashboard Questions instead of referencing them
- but I realized we could use the same for Models as well. I moved the
logic for models to `cards-to-copy` instead: if the card is a Model,
return it in the `reference` list instead of the `copy` list.
Fix commented-out test
cljfmt
Optionally include dashboard questions in search
Add a test for failing card name change
Opt. include dashboard questions for autocomplete
Update src/metabase/api/dashboard.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
PR review response
lint migrations
quick impl of dashboard items
copied from collection items. should be firmed up for real going
forward. notable divergences:
- i'm not doing any limit/offset in the query
- i'm not including limit/offset in the response
- no reuse. just grabbed it to unblock nick
```
❯ http get "localhost:3000/api/dashboard/22/items" x-api-key:$API_KEY -pb
{
"data": [
{
"archived": false,
"archived_directly": false,
"collection_id": 20,
"collection_position": null,
"collection_preview": true,
"database_id": 48,
"dataset_query": {
"database": 48,
"query": {
"source-table": 360
},
"type": "query"
},
"description": null,
"display": "table",
"entity_id": "_TmF-T7m_bBVxV_jfaNtM",
"id": 192,
"last-edit-info": {
"email": "dan@metabase.com",
"first_name": "dan",
"id": 1,
"last_name": "sutton",
"timestamp": "2024-09-13T22:13:30.220311Z"
},
"last_used_at": "2024-10-01T17:48:36.128915Z",
"model": "card",
"moderated_status": null,
"name": "Cynthia Example"
}
],
"models": [
"card"
],
"total": 1
}
```
contrast with collection items:
```
❯ http get "localhost:3000/api/collection/root/items?models=card&limit=1" x-api-key:$API_KEY -pb
{
"data": [
{
"archived": false,
"can_delete": false,
"can_restore": false,
"can_write": true,
"collection_id": null,
"collection_position": null,
"collection_preview": true,
"database_id": 4,
"description": null,
"display": "table",
"entity_id": "e_jABu05u9nT9ocgfQWfZ",
"fully_parameterized": true,
"id": 143,
"last-edit-info": {
"email": "dan@metabase.com",
"first_name": "dan",
"id": 1,
"last_name": "sutton",
"timestamp": "2024-08-27T14:11:15.214254Z"
},
"last_used_at": "2024-08-27T14:11:27.138735Z",
"location": null,
"model": "card",
"moderated_status": null,
"name": "Collection + query on sleep question"
}
],
"limit": 1,
"models": [
"card"
],
"offset": 0,
"total": 13
}
```
Clean up last-edit-info and exclude archived cards
before: over-selecting, copy-paste function to clean it up.
after: reuse (bulk-aware) query that annotates with last-edit-info, ALSO
EXCLUDE CARDS
also condensed query a bit. was each field on a separate line for no
real reason.
`location` on dashboards
dashcards are still active for archived DQs
WIP: autoplace dashcards for dashboard cards
more wip for autoplacement
WIP for moving DQs
more WIP on moving DQs/autoplacement
delete old dashcards when moving to new dashboard
populate `here` on dashboards in coll items
reject moving card if in another dashboard
fix silly typo
allow optionally deleting old dashcards on move
WIP
add dashboard details to recents
untested wip
fix auto delete of old dashcards for move
Add another test for DQs getting unarchived
Fix dashboard details on recents, with a test
Archive and unarchive DQs when used/unused
including in revisions
Handle reversions containing unowned DQs
lint
Tests and fixes for dashboard items endpoint
remove stray #p
fix a test
Add more tests for DQs/reversions
Fix/update tests for updating DQs
Don't revert to dashcards with archived cards
This is the existing behavior, we should keep it.
lint
small fixes for recents tests
add more tests
Autoplace Dashboard Questions on creation
Improve autoplace to be like frontend
I'd hacked together something - this actually makes autoplacement work
the same way on the frontend and backend.
fix a couple tests
`delete_old_dashboards` as a query param
Fix autoplace for tabbed dashboards
and add a test
do not autoplace DQs when duplicating a dashboard
This was resulting in us accidentally creating two dashcards for a
duplicated DQ.
add effective location to collections on dash
add dashboard_count to (card) collection items
autoplace on unarchive, better error messages
Exclude DQs from dashboard if archived directly
DOH
workaround a bug with "changing" a type
Fix bug causing dashboard cards to be removed
small refactor autoplacing / removing dashcards
Create Dashboard revisions on autoplace/remove
When we modify Dashboard Questions, we autoplace or autoremove relevant
cards from the Dashboards involved. For example, we remove DashCards
from the old dashboard if we're moving to a new dashboard, and we
autoplace DashCards into the new dashboard.
When we do this, publish an event that the dashboard itself has changed.
This updates the revision history for the dashboard so that you can
revert correctly.
Note that when using the revision history, DQs will automatically be
archived or unarchived as necessary. So e.g. if you manually archive a
DQ, then go to the Dashboard and revert, the DQ will be auto-unarchived.
If you go to the dashboard and remove all cards for a DQ, the DQ will be
archived - if you then manually unarchive it, it'll be autoplaced on its
dashboard again.
Fix bug on card modification
The conditions here were wrong. Say you had a card on dashboard 1 that
was archived. If you specified `{:dashboard_id 1 :archived false}`, it
would be correctly autoplaced on the dashboard. But if you just
specified `{:archived false}`, the "new dashboard ID" is nil.
The logic here is pretty painful, because to figure out the resulting
value for something like `dashboard_id` we can't just get it from the
old version or the changes. Instead, we need to:
- check to see whether it's changing, then
- if it changed, use the version from the update, otherwise
- use the old version
The same is true for `archived`.
After we figure out the old dashboard_id, new dashboard_id, and whether
the card will end up archived or not, we can actually do the checks we
need:
- if we're ending up an unarchived card on a dashboard, autoplace
- if we're moving from one dashboard to another, or moving from a
dashboard to a collection (and the user told us to), auto-remove
offset/limits for dashboard items
Reuse some code between dashboard/collection items
Fix some failing tests in api.dashboard
I was getting stack overflow errors when running these tests. I followed
along with the demonstartion
[here](https://github.com/metabase/metabase/pull/43496) to fix.
cljfmt
add additional tests
Always delete dashcards when archiving a DQ
If we're archiving a DQ, let's delete the dashcards so this appears in
the revision history.
publish revision when autoplacing
Publish dashboard events when cards are autoplaced
new endpoint to get dashboards a card is in
allow moving to a dashboard deleting old dashcards
fix the migrations 😭
fix migration IDs
* work on dedicated dashboard question page
* wip on dashboard page for handling dashboard questions
* hacking linking to dashboard for question and books marks - does not handle navigating to the correct tab yet
* make it easier to change the dashboard_id for hacking
* misc. changes
* action menu and dashbcard action updates
* enables trashing dashboard questions from the dashboard view, enables undoing of that case, updates archive state for dedicated dashboard question page
* wip - mostly done creation flow
* wip
* add question directly on dashboard flow
* significant clean up of fe code, revert some wierd differences between base branch, allows selecting dashboard when duplicating a question
* type fixes
* clean up isDashboardQuestion checks, fixes bug where save question form name and description state would get cleared after selecting a dashboard from the picker, fixed bug where editing a dashboard question and choosing save as new question would only let you save it to the dashboard the original question was on
* implement bulk of team feedback
This commit accomplishes the following things:
- Updates copy of the dashboard/collection picker
- Fixes the tooltip copy for the add question button in the edit dashboard view
- Removes the ability to trash a dashboard question + dashboard view removes rather than trashes the question
- Adds option in dashboard edit view to add a native or notebook question
* prompt to save rather than discard changes when leaving a dashboard while editing to create a new question
* include dashboard questions in search results
* fix lint & kick CI
* type fixes
* fix unit tests
* two small fixes
* wip on failing e2e tests
* bulk e2e test fix
* fixed most tests worth fixing - leaving the ones that need to be replaced
* Dashboard Question as a source
* replace dashcard flow
* starting to update collection picker to treat dashboards as folders
* fix spelling mistake
* fix bad resolution and stop double adding newly created dashboard questions to their dashboards
* allow dashboards in collection picker
* remove checkbox to only clone dashboard w/o cards, fix collection id calculation for saving a question when defaults to a dashboard
* rough pass at allowing questions to be moved to, from, and between dashboards
* building state path with initial value as a dashboard
* simplify moving code and fix encountering BE error
* add confirm modals when moving a dashboard question to a collection or another dashboard, allow users to keep or remove dashcards for a dashboard question getting moved to a collection
* improve the breadcrumbs data consistency, fix bug in moving logic and dump user in edit state of dashboard when moving from collection to dashboard
* dynamic select button text
* Search showing available models
* hopefully this resolves some type sadness
* handle inconsistent state issues between latest question and query builder internal state
* default question picker to showing dashboards as folders
* silly type things
* disallow saving metrics | models | events to dashboards, added back ability to keep or remove dashboard ca
rds when moving a DQ to a collection, fixed dedicated DQ page not appearing archived after archiving the parent dashboard
* use currentData instead of data for collection breadcrumbs
* dashboards are not folders when they are empty
* fix moving question to Our Analytics, Update sidesheet question into for DQs, always allowed to select a dashboard for moving question
* confirm moving questions into dashboards
* trying to sort out types
* types are still a mess
* only regular questions can be moved into a dashboard
* adjusting some unit tests
* add e2e tests for dashboard questions
* should have happy linter and unit tests
* fix some type errors
* loading state
* e2e test fixes
* more fixed tests
* loading state message
* filter series options, fix create new collection logic
* fix some more e2e tests
* e2e test adjustments
* even more e2e tests fixes
* fix named export
* more e2e test fixes
* fixing some e2e stuff
* hopefully final e2e adjustments
* last failing e2e (for now)
* More BE: Dashboard Questions
add dashboard question dimension to stats
Handle series correctly
and test for them
Don't restore archived-directly questions
* unit test adjustments, linter changes
* obey the linter
* unit tests for new breadcumb functionality
* allow creating a question directly to a dashboard
* More BE: Dashboard Questions
use dashboard constants for default size
add missing sql clause
add dummmy alias to subquery
* more cards in dashboards e2e tests
* rebase whoopsie
* more tests
* fix types
* More BE: Dashboard Questions
include dashboard info on coll items & cards
include dashboard info in search
also modify the moderation info a bit, now all three endpoints return
the same shape for `dashboard` (if present):
```
{:id dashboard-id
:name dashboard-name
:moderation_status moderation-status-if-present}
```
Bulk endpoints for checking dashboards and moving
For now the move endpoint just runs everything in a transaction and
spits out an error if anything failed. I'd like to take more time to
think of the exact shape of the result we want in the case of an error,
even though I think ideally we should allow partial success.
* reproduce card-pocalypse
* adding dashboard link to search results
* more trash tests
* lint fixes
* fix crash when archiving dashboard with virtual cards
* add loading state to archive modal
* obey the clojure linter
* More BE: Dashboard Questions
cljfmt and fix tests
small test fix
* more e2e tests
* embedding tests
* type fix
* More BE: Dashboard Questions
two more small test fixes
more test fixes
* add sdk tests
* another test
* add series removal e2e test
* More BE: Dashboard Questions
yet another test 😭
work around old MariaDB limitations
* permission check and confirmation modal when moving cards into dashboards
* e2e adjustments
* add some missing tests
* prettier + rebase fixes
* fix search result bug
* Entity syncing when moving a question from a collection to a dashboard
* hello rebase hades
* gree dashboard-questions suite
* fix more e2e tests
* happy little test fixes
* Sloan Feedback
* fix move modal error swallowing
* fix embedding test
* fixes breadcrumbs for embedding, fixes associated failing tests, and updates some tests to account for reduction in # of requests
* fix embedding e2e tests
* Bryan's review response
* fix usage of `offset-paging`
* fix request count errors
* fix unit test
* finish feedback from FE code review
* CR response for Noah
* bad sloan
* no need for dashboards in ad hoc
* get the dash name from the question object
* fix migrations again
* fix test + kondo
* lint fix
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Add new user-level setting
* Add selector
* Make link dismissible from the sidebar
* Transfer the onboarding link to the main "gear" menu
* Add E2E test to cover the link dismissal behavior
* Add tests for the settings menu link
* Fix tests
* Provide useful comment in the e2e test
* Provide contextual explanations to translators
* We need to look at the context field to categorize public views
* pull poll-unil into util ns
* fix comment typo
* add test
* update docstring
* remove true? call, it will return `true` on pass
* improve poll-until
* Fix the zoom-in.timeseries drill for multi-stage queries
If no dimensions are provided to the zoom-in.timeseries drill, then attempt to find the matching dimension from the FE-provided row data instead, if present.
Related to https://github.com/metabase/metabase/issues/46932
* Only return zoom-in.timeseries drills for clicks on aggregated columns
This PR removes the deprecated `BodyComponent` HOC in favor of Mantine's `Portal` component. This helps pave the way for https://github.com/metabase/metabase/pull/49442
### How to verify
1. Perform an action that triggers an undo toast (e.g., move a question into the trash)
2. Verify the undo toast appears correctly at the bottom of the screen
3. In a collection, drag and drop items to verify the drag preview renders correctly
### Testing
This change is covered by the following tests:
* `collections.cy.spec.js > "should be able to drag an item to the root collection (metabase#16498)"`
* `collection-pinned-overview.cy.spec.js:269: "should be able to pin a visualization by dragging it up"`
Note the following comment, however, which mentions the limitations of our drag-n-drop e2e tests:
(from https://github.com/metabase/metabase/blob/92d6150/e2e/test/scenarios/collections/collection-pinned-overview.cy.spec.js#L282-L289)
```
// this test can give us some degree of confidence, but its effectiveness is limited
// because we are manually firing events on the correct elements. It doesn't seem that there's
// a way to actually simulate the raw user interaction of dragging a certain distance in cypress.
// this will not guarantee that the drag and drop functionality will work in the real world, e.g
// when our various drag + drop libraries start interfering with events on one another.
// for example, this test would not have caught https://github.com/metabase/metabase/issues/30614
// even libraries like https://github.com/dmtrKovalenko/cypress-real-events rely on firing events
// on specific elements rather than truly simulating mouse movements across the screen
```
Closes#49682. Closes#49683.
Updates `lib.convert` and legacy MBQL normalization to allow
`:ident`s, and fixes some QP middleware and utils so they can
still match up refs correctly.
No user-visible impact.
Newly created queries will have `:ident`s generated for joins,
aggregations, etc. by the previous query, and this will start
saving them into appDBs.
Pre-existing clauses don't have refs added yet.
* Fix loading in segments
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
For now, we've decided not to trigger index updates when elements are deleted, due to the overhead that toucan2 would add.
We have the code to calculate what *should* be done, so we might as well keep it around in a test that documents these relationships.
It's good to be mindful of the implications, and we could even use this information to do manual calls from the relevant deletion call sites.
* Add list/create/update/delete to actions api
* Add api mappings in entity framework
* Port actions to RTK
* Remove old api mappings
* Fix a test
* Fix test
* Fix test
* Fix test
* Fix method
* Fix types
* Define request types
* Remove ActionsApi.create
* Fix deleting actions
* Fix invalidation
* Fix invalidation
* Remove redundant spreads
* Remove type attribute from UpdateActionRequest
The underlying issue is that nest-query/nest-expression included temporal units when copying over joined fields into the nested query. This caused both the nested query and the parent query to include the "extract the unit from the date" code. In cases where the extraction didn't change the type, this duplication wasn't a big deal. However, for temporal units like hour-of-day, the first extraction changed the type of the field from date to string, and then the second extraction broke. This change ensures that the inner query doesn't try to extract temporal units, which should avoid the duplicate extractions.
* wip next sdk compat
* better code generation, dts works
* Last line of file should end in a newline character
* feat(sdk): make MetabaseProvider not break ssr
* aUpdate enterprise/frontend/src/embedding-sdk/package.template.json
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* address pr comments
* refactor(sdk): use relative paths when fixing d.ts files
* make types work on moduleResolution: node
* add loading text
* fix(sdk): #50736 make sure we don't break the host app if a sdk
component is rendered outside of the provider for a render
* fix: this should make types work for real?
* self review fixes
* update test for PublicComponentWrapper
* Update enterprise/frontend/src/embedding-sdk/components/private/PublicComponentWrapper/PublicComponentWrapper.tsx
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* remove unused component
* eslint-disable-next-line for MetabaseProvider
* list tsx as dependency
* /next -> /nextjs
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* introduce public metabase question type
* revise internal question type
* update stories
* use public question type in dashboard card plugins
* add tests for transforming question properties
* add tests for question description
* re-export the MetabaseQuestion type
* fix lint imports
* change publicQuestion to sdkQuestion
* name cannot be null if the card is present
* Fix embed loader not translated with #locale
* Fix unit tests
* Add E2E tests
* Fix a case where the lcoale is failed to load
* Simplify conditions in `LocaleProvider`
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Make E2E tests faster
Co-authored-by: Nicolò Pretto <info@npretto.com>
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
* WIP
* metabase.request.cookies namespace
* Server API namespace
* `metabase.server` => `metabase.server.instance`
* Reformat code
* Revert indentation changes
* Unrevert comment
* Oops delete moved file
* Don't be lazy CAM!!!
* Add TODO
* Please cljfmt
* Bump Kondo version to fix metadata ignores in namespaces being ignored
* Update Kondo and enable a few more linters
* Update Clojure linux install script location
* Ok so apparently the old URL does work
* Make sure things live in the right places
* Why do I always have to clean up other people's messes
* Fix formatting
* People need to knocc it off with writing slow tests
* Fix another test that took 45 SECONDS
* Fix Kondo warning
* fix scatter chart tooltip shows irrelevant info
* remove outdated spec as we always show the tooltip_column setting
* fix missing tooltip on the total bar on waterfall charts
* Always show embed sharing option for admins (#50513)
* Always render the public link option
This is possible because metabase#50504
* Always render the embed option
* Fix unit tests
* Fix E2E tests
* Correct test semantic
* Hide Embed option for non-admins
* New Embed modal (#50580)
* Open "Compare options" in a new tab
* Add an external link icon to interactive embed option when it should
* Add disabled state to SDK
* Add a disable state to static and interactive embedding
* Add interactive embedding disabled state
* Implement disabled link
* Prepare folder for testing
* Fix tests and add SDK tests
* Add interactive embedding tests
* Correct test desribe block levels
* Add static embedding tests
* Add premium interactive embedding tests
* Add enterprise tests (EE without tokens)
* Fix the rest of the failed unit tests
* Update E2E tests
* Fix a case where OSS see disabled interactive embedding
This should be impossible as interactive embedding isn't available on
OSS, so we should not display the disabled state.
* Address review
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Address review
Co-Authored-by: Poom poom@poom.dev
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Update Interactive Embedding Admin Settings page (#50693)
* Add a quick start link to interactive embedding settings page
* Update UTM tags for interactive embedding card
* Don't show interactive embedding option as enabled on OSS
* Fix unit tests
* Add a new test
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* fix the script, and add the new edn
* make the migrations run in the proper order
* it works (needs cleanup)
* remove broken binning
- These bins rely on fingerprinting for the custom Age column
- The fingerprinting for that custom column are not properly exported in
serialization, hence do not get loaded into metabase, causing binning by
Age to fail.
* cleanup / linting
* cleanup
* update migration test ids to test
* realign test name to check for to see if sample content was created
* remove Product collection + move Examples into root
* update collection_ids for the content too
* remove click behavior for 'Revenue goal for this quarter'
* Apply suggestions from code review
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
* for the sample collection is_sample should be true
* Drop CreateInternalUserNoOp
* typo
* update migration comments
* 2 small fixes
* fix data perms for example coll
* add a comment so we don't have 2 blank lines in a row
---------
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
* Add delay helper
* Remove delays in storybook
* Use skippable delay in ListField and SingleSelectListField
* Use skippable delay in ExplicitSize
* Remove waitTimeContext in storybook
* Remove waitTimeContext altogether
* Add documentation to delay
* Remove all context references in ExplicitSize
* Update flaky snapshots
Fixes: #50771
The new sync process failed to qualify enums outside public and
therefore failed to properly type them in database_type. This could
either result in matching the wrong type if there was a conflict or not
being typed as an enum.
* test: see which drivers pass these tests
* Fix build
* Change test to use standard dataset, add initial implementation for bigquery
* Use fully qualified names for views
* Don't use transactions for databricks
* Make drop table only if exists, fix describe-index in mongo for views
* don't sync system.views
* Fix oracle drop-views, disable non-syncing views
* Use metabase-instance to find the table, this is important because redshift and oracle munge table name
* Fix oracle tests
* Fix oracle views
* Fix formatting
* Fix snowflake qualified components
* Change tests to opt out
* Fix sqlsserver
* Fix postgres test
* Disable h2 describe views test
* Address review
* Add trial banner
* Use the semantic "warning" color for background
* Simplify the "reappearance" logic
* Refine the banner logic
- Pass the bare minimum information (expiry timestamp) to the `TrialBanner`
- Take timezones into account
- Introduce utilities
- Add unit tests
* Update the remaining tests
* Clean up
* Unbreak my `defsetting` 🎶
* Add comments
* Defer var assignment
* Fix failing tests 🤦
* Adjust the types in order to match `master`
* Move the logic up to the parent component
* Convert the `index` component to TS
* Fix typo in the JSDoc
* Fix another typo
* Make the return statement more readable
* Remove redundant test
* Move modals out of resizable area
* Split off ace editor from NativeQueryEditor
* Query for selection target in AceEditor
* Rename editor ref
* Add custom type for selection range
* Move more items into shared types file
* Add shared interface for editors
* Add shared Editor wrapper
* Add SQL colors in core/colors.module.css
* Allow importing global css with .global.css
* Move ace styles to AceEditor.global.css
* Fix Range type
* Fix Ace editor tests
* Scope the ace editor styles
* Remove EditorRoot styled component
* Use parent wrapper instead of classname on editor
* Remove ace editor styles from global styles
* Remove custom ace styles from embedding sdk
* Move NativeQueryEditor styles to css modules
* Fix styles for notebook sql preview
* Remove unused props
* Remove unused variable warnings
* Remove width prop
* Rename openDataReferenceAtQuestion and make it optional
* Rename setNativeEditorSelectedRange and make it optional
* Move selection popover logic out of AceEditor
* Make autocompleteResultsFn optional
* Make cardAutocompleteResultsFn optional
* Remove fetchQuestion prop
* Remove nativeEditorSelectedText prop
* Make props explicit
* Make onChange optional
* Set editor height
* Use Editor in NotebookNativePreview
* Remove cardAutocompleteResultsFn and autocompleteResultsFn props from Editor
* Remove unused snippetCollections prop
* Add missing native-query-autocomplete-match-style setting
* Remove snippets from Editor props
* Resize ace editor on load
* Add custom snippets wrapper
* Flatter completer styles
* Add container class to autocomplete popover
* Fix flex size for editor
* Fix AceEditor types
* Flatten ace styles
* Avoid hard-coded color
* Set height and width on editor
* Flatten ace wrapper
* Do not call api with empty prefix
* Add helpers to target native editor
* Use nativeQueryEditor in action helper
* Use nativeQueryEditor in ad hoc helper
* Remove ace_* references from tests
* Align nativeEditor helpers
* Revert "Align nativeEditor helpers"
This reverts commit e629aecb18.
* Move focusNativeEditor to native editor helpers
* Use focusNativeEditor where applicable
* Remove ace_editor references in test
* Fix test
* Remove references to native-query-editor testid in test
* Remove duplicate test assertion
* Fix nesting in css
* Use font-family from variable
* Use default module.css with :global instead of custom .global.css file
* Move AceEditor into subdirectory
* Fix import path
* More strongly type native-query-autocomplete-match-style
* Remove duplicate default
* Rename EditorHandle to EditorRef
* Add more specificity to ace syntax overwrites
* Simplify EditorRef
* Remove HOC
* Do not perform visibility check inside nativeEditor helper
* Explicitly assert elements are visible
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix formatting issue
* Add highlight for identifier
* Improve gutter cell
* Use relative imports for helpers
* Focus editor before typing
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* rename onClose to onCancel
* rename questionHeight to drillThroughQuestionHeight
* rename showVisualizationSelector to withVisualizationSelector
* rename parameter value props
* doc updates for parameter values
* rename the chart type selector prop
* document the withChartTypeSelector prop
* do not show column selection error on empty datasets
* add more specs to chart-definition logic
* fix entry nodes tooltip and click behavior values
* fix nodes formatting
* do not show sankey data settings on dashboards
* do not initiallt open sankey data columns selects
* hide sankey tooltip on scroll
* drop incorrect types
* set sankey node limit
* prevent clicks on edges close tooltips on native sankey questions
* tweak copy
* update sankey spec according to fixes
* revert changed settings behavior
* fix unit specs
* Pivot Exports Can Handle `nil` in Breakout Col, and Multiple Cols of the Same Aggregation Type
Fixes#50551
If a pivot table has several measures configured, they might both use the same kind of
aggregation (eg. 'sum'). Previously, this would lead to columns with the same name preventing the pivot-measures from
properly making it through the export post processing. Now, the correct aliased/deduped name is used and all columns
can be properly included in the pivot export.
Also fixes the case where a breakout column (pivot-row) contains `nil` values, which caused the pivot export to
fail. Now, all nil values are grouped and handled appropriately.
* when pivot is disabled, header should still be included
* min and max don't default to 0, add a test for the aggregations
* Add EntityObjectLoaderRtkQuery boilerplate
* Incorporate EntityType logic into the new component
* Improve comment
* Add childrenProps
* Move component into its own directory, extract types to a separate file
* Add entityObjectLoaderRtkQuery and temporary mappings
* Hook up the useGetQuery hook (tables only)
* Pass all the props to children
* Add Playground
* Gather more info in the Playground
* Integrate RTK with redux store
* Adjust props passed down
* Improve typing
* Make the request hook dynamic
* Add missing id in the query
* Handle fetchMetadata
* Change requestType to fetchType
* Add rtk mappings for databases in entity framework
* Use requestType to get request state path
* Enable new component everywhere
* Use legacy loader for entities without rtk mappings
* Improve comment
* Handle fetchMetadataAndForeignTables
* Remove unused function
* Handle reload and remove
* Add TODO
* Add transformResponse option and use it for fetchIdFields
* Format code
* Remove Playground
* Remove unused getUseGetListQuery
* Add rtk mapping for alerts
* Add rtk mapping for dashboards
* Add rtk mapping for groups
* Add rtk mapping for pulses
* Remove options
* Fix permissions tests
* Fix notifications tests
* Add rtk mapping for segments
* Use params instead of body
- It's leftover because this code got reverted after body -> params refactoring for GET endpoint has been merged to master
* Add rtk mapping for snippet collections
* Add TODO
* ESLint
* ESLint
* Update TODO
* Deprecate entity HOCs
* Update TODO
* Revert other entities for now
* Revert other entities for now
* Bring tables back
* Improve todos
* Add comment
* Improve comment
* Remove TODO
* Use reload param
* Remove properties prop
This was used to short-circuit and avoid triggering api calls if we already had an object with given attributes in cache.
With RTK query now managing the cache, we don't want to reproduce this behavior.
* Remove reloadInterval
This is used in EntityListLoader, not EntityObjectLoader
* Don't use setRequestPromise action
* Fix deprecations
* Format code
* Fix flake :(
* Try to fix flake
* Revert "Try to fix flake"
This reverts commit 6408a22a37.
* Try to fix flake
* Fix UseQuery type
* Implement useGetMetadataAndForeignTables
* Fix useMemo and move things down
* Fix typo
* Bring back original hack
* Move file
* Add comment
* match against any hexadecimal bundle name
* add e2e tests skeleton
* Support display-name for ISO get-day-of-week exclusion expressions (#50606)
* Support display-name for ISO get-day-of-week exclusion expressions
Fixes#50574
* Fix light theme embeddings has different text colors (#50678)
* add cache-control tests
* support local runs
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* add withVisualizationSelector prop
* add tests
* add to docs
* use default layout in tests
* only use custom layout when needed
* make withChartTypeSelector false by default
Adds `dev.portal/diff->`, a drop-in replacement for `->` that `tap>`s
a diff at each stage. (With the metadata for Portal to default to diff viewer)
I found myself wanting this several times while debugging the refs overhaul.
Specifically, for expressions, aggregations, breakouts, and joins.
Adds MBQL lib unit tests for this behaviour.
Currently these idents are discarded during `lib.convert` and don't
survive a round trip to legacy. That will come with a later change.
This protects against us trying to read or write from an incompatible search index. In particular it prevents existing instances from being switched over to an incompatible table during an HA upgrade or downgrade.
~~It doesn't yet get rid of the needless re-initialization, but that will be a fast follow.~~ It breaks up the jobs so that we won't reindex on startup unless necessary.
See https://www.notion.so/metabase/Search-Index-Maintenance-14b69354c9018047b670cbba1c3be50d
This ensures that when one instance starts or finishes reindexing, the other instances pick up and switch to the same tables.
Note: this does not stop blowing away the currently active table on deployment yet, we need to have a notion of version for that first.
* Enable underlying-records drill in table viz for multi-stage queries
* Fix e2e test
* Add tests for underyling-records drill for multi-stage queries
* Review feedback: filterv not-empty etc
* Review feedback: add a breakout-only query to drill-thru canned-queries
* do not show column selection error on empty datasets
* persist default sankey column settings when saving
* add more specs to chart-definition logic
* more specs
* Table Column Ordering Should Not Prevent Pivot Exports
Prior, when viz-settings existed for table column ordering, or if some columns were hidden in a regular table viz, the
pivot export could fail.
Now, if a pivot table is being exported, the table column sorting is properly ignored and the pivot export should work.
* address feedback
* use css variables for skeleton background color
* apply theme to folder color
* replace theme.white with themeColor
* use name of folder chevron color
* add background-inverse color mapping
* Revert "add background-inverse color mapping"
This reverts commit d0b546c583.
* avoid overriding folder chevron color in emotion
* use Mantine style overrides for NavLink
* temporarily hiding group into other
* hide group into other
* remove assertions related to group into other
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Adding .png version of no_results.svg to fix#49012
* Adjusting text color to to match existing color scheme
* Removing old unused no results assets; adding new png version of no_results.svg at matching resolution to pulse_no_results@2x.png
* Fixing namespace import cycle
Fixes: #50422
Although we passed additional-options through subname in url parameters
to DriverManager, we were also passing `:schema nil` as connection property.
* Pivot Export Aggregations Improved and Won't Error
Fixes#50207
WIP
- Totals now have appropriate aggregation implementation
- Totals for different row and column partial paths could collide before (eg. if a pivot-col value was equal to the
- measure idx), now the paths are unique and will not collide, causing errors
* add a test
* multiple measures need to have the correct order in each row
* clean up, add some comments explaining totals paths
* fix tests
* add test to show add-rows working correctly after :rows-part :cols-part added
* remove pivot-grouping row and columns in tandem
* add pivot card scalar test for csvs and xlsx.
* improve test feedback
* add a 'todo' for pivoted json exports
* indent
* Add stage-number parameter to NativeParameterDimensionTarget
* Update unit test
* Recognize native query dimension parameters with stage-number
* Update test setup
---------
Co-authored-by: Tamás Benkő <tamas@metabase.com>
* Don't dissoc `:id` in `before-update`
There seems to be a nasty footgun in toucan2 here.
Say you're executing an `update!` command on a set of IDs, e.g.:
```
(t2/update! :model/Card :id [:in 1 2] {:view_count 1})
```
This works if:
- both cards 1 and 2 have `view_count=1` already
- both cards 1 and 2 have `view_count!=1`
However.
If one of the two cards has `view_count=1` and another has a different
view_count, then (if the `before-update` method doesn't have the primary
key attached) Toucan emits a call to update *every card in the
database*, without a where clause at all.
* Trigger CI tests
* Add dummy job generator
* [DO NOT MERGE] Remove driver test config from the list of backend changes
* Add missing runs-on
* Avoid too much nesting
* Revert changes to file paths and README
* FE - Allow to use in dashboard filters both pre- and post- last aggregation columns (#46670)
* Extract getFilterStageIndexes
* Update comment
* Include 2 last stages in filterable columns in getParameterColumns
* Move getJoinQueryHelpers to test-helpers
* Add basic query creation
* Add breakout columns
* Make tableName optional
* Implement 1-stage test, refactor assertions
* Rename
* Move filter utils up
* Use getGroupItems and appendStageIfAggregated in getFilterableColums
* Introduce Lib.filterGroups
* Revert "Introduce Lib.filterGroups"
This reverts commit 5f8b2d098e.
* Don't reuse getGroupItems or appendStageIfAggregated
* Move getFilterStageIndexes out of Lib
* Format code
* Revert types move
* Revert functions move
* Improve diff
* Group dashcard mapping options
* Adjust existing getParameterColumns usages (except unit tests)
* Remove redundant call
* Fix filtering
* Update unit tests
* Format code, remove TODO
* Add stage index assertions
* Add model test for 1-stage query
* Add partial boilerplate for 2-stage query
* Add a 2-stage case
* Add a 2-stage case for model
* Update unit test
* Fix click behavior mapping
* Remove duplicated functions
* Remove complex return type
* Move appendStageIfAggregated and getFilterStageIndexes to metabase-lib/filter.ts
* Re-group columns based on stage index instead of using ML-returned references for comparison
* Add new options to dimension types
* Use the new stage-number option
* Rename appendStageIfSummarized to ensureFilterStage and add explanatory comment
* Remove duplicated ensureFilterStage after merge
* FE - e2e - Cover dashboard filters with tests (#46958)
* Extract getFilterStageIndexes
* Update comment
* Include 2 last stages in filterable columns in getParameterColumns
* Move getJoinQueryHelpers to test-helpers
* Add basic query creation
* Add breakout columns
* Make tableName optional
* Implement 1-stage test, refactor assertions
* Rename
* Move filter utils up
* Use getGroupItems and appendStageIfAggregated in getFilterableColums
* Introduce Lib.filterGroups
* Revert "Introduce Lib.filterGroups"
This reverts commit 5f8b2d098e.
* Don't reuse getGroupItems or appendStageIfAggregated
* Move getFilterStageIndexes out of Lib
* Format code
* Revert types move
* Revert functions move
* Improve diff
* Group dashcard mapping options
* Adjust existing getParameterColumns usages (except unit tests)
* Remove redundant call
* Fix filtering
* Update unit tests
* Format code, remove TODO
* Add stage index assertions
* Add model test for 1-stage query
* Add partial boilerplate for 2-stage query
* Add a 2-stage case
* Add a 2-stage case for model
* Update unit test
* Fix click behavior mapping
* Remove duplicated functions
* Remove complex return type
* Remove unused @ts-expect-error directive
- use createQuestionAndDashboard helper instead of command
- sort imports
* Add boilerplate for query stages test suite
* Add createDashboard
* Try out the new function
* Organize code
* Add descriptions
* Move visitDashboard call out of createDashboard
* Move appendStageIfAggregated and getFilterStageIndexes to metabase-lib/filter.ts
* Re-group columns based on stage index instead of using ML-returned references for comparison
* Add boilerplate for base questions
* Add assertions for mapping options
* Simplify verifyDashcardMappingOptions interface
* Add assertions for all types of parameters
* Update test name
* Format code
* Add new options to dimension types
* Use the new stage-number option
* Add Q2 creation
* Introduce helper constants
* Rename appendStageIfSummarized to ensureFilterStage and add explanatory comment
* Add cases for models
* Extract subroutines
* Add Q3
* Extract createQ2Query and createQ3Query
* Extract createQ1uery, remove createM1
* Optimize verifyPopoverMappingOptions
* Add more assertions
* Add model assertions
* Add tooltip assertion
* Break large functions down
* Restructure tests to prevent memory-related crashes
* Refactor
* Update comment
* Update comment
* Remove createQ1
* Remove createQ2, createQ3
* Refactor
* Refactor
* Comment out failing assertions
* Add Q3 assertions
* Add Q3 assertions
* Add comment
* Introduce createAndVisitDashboardWithQueryMatrix
* Add Q4
* Fix assertions
* Add repro tags
* Add comment
* De-hardcode indexes
* Add Q5-Q8
* Rename
* Add Q5
* Add Q6
* Add Q7 & Q8
* Add Q9
* Fix structure
* Reuse Lib.ensureFilterStage
* Fix unit of time parameters
* Support explicit and implicit stage numbers for temporal-unit params
* Extend test with temporal unit parameter with stage number
* Update unit tests
* Let stage-path handle nil (default) stage-number
---------
Co-authored-by: Tamás Benkő <tamas@metabase.com>
* Allow filtering on columns from all stages in dashboards (#48254)
* Allow filtering on columns from all stages in dashboards
* Update tests
* Clean up
* Update unit tests
* Distinguish multiple "Summaries" column groups in dashboard filter mapping (#48261)
* Allow filtering on columns from all stages in dashboards
* Update tests
* Clean up
* Update unit tests
* Distinguish multiple "Summaries" column groups
* Fix imports
* Remove redundant attribute
* Update import
* Allow filter params targeting native stages (#48379)
* Move filters on nested native stages to parent stage in expand-mbql-params
Similar to the existing handling in move-join-condition-to-source-query.
This fixes filter params that directly target a native stage with a non-negative :stage-number, e.g. when applying
dashboard filters to a card where the card query is native.
Fixes#48258
* Add test for filter params explicitly targeting nested native queries
* FE - e2e - Cover dashboard drills with tests (#48289)
* Allow filtering on columns from all stages in dashboards
* Update tests
* Clean up
* Update unit tests
* Distinguish multiple "Summaries" column groups
* Fix imports
* Remove redundant attribute
* Fix type
* Add a test
* Add a test
* Reduce flakiness
* Fix race conditions with last_used_param_values
- see https://metaboat.slack.com/archives/C010L1Z4F9S/p1727947712609579
* Add a temporary workaround
* Fix name
* Fix applying parameters in Question
* Revert "Fix type"
This reverts commit 67f6778d50.
* Rename and reorder tests
* Add more tests
* Update tests to avoid hitting https://github.com/metabase/metabase/issues/46774
* Add Q8 tests and fix tests for Q9
* Update assertions
* Update assertions because of https://github.com/metabase/metabase/issues/48339
* Improve test names
* Add test
* Skip failing test
* Add more tests
* Add test
* Remove redundant code
* Remove invalid cases
* Add test
* Add tests
* Add more assertions
* Add more assertions
* Add assertions
* Add assertions
* Use helper functions
* Use helper functions
* Nest helpers consistently throughout the file
* Decrease flakiness
* Revert "Add a temporary workaround"
This reverts commit 48ed8770d6.
* Add a temporary workaround
* Add tests
* Add assertions
* Add tests
* Update unit tests
* Make temporary workaround safer
* Add TODOs to github links
* Extract a function to improve readability
* Simplify getParametersMappedToDashcard
* Fix failing tests by calling Lib.ensureFilterStage in the correct place and properly identifying the originalQuestion so that Question.prototype.isDirtyComparedTo returns expected result
* Remove redundant code
* Quick dirty fix to run in CI
* Format code
* Format code
* Update unit tests
* Fix test setup
* Revert "Remove redundant code"
This reverts commit 35b47a10f2.
* Use Lib.areLegacyQueriesEqual
* Create a new e2e test group: dashboard-filters-2 (#48610)
* Allow filtering on columns from all stages in dashboards
* Update tests
* Clean up
* Update unit tests
* Distinguish multiple "Summaries" column groups
* Fix imports
* Remove redundant attribute
* Fix type
* Add a test
* Add a test
* Reduce flakiness
* Fix race conditions with last_used_param_values
- see https://metaboat.slack.com/archives/C010L1Z4F9S/p1727947712609579
* Add a temporary workaround
* Fix name
* Fix applying parameters in Question
* Revert "Fix type"
This reverts commit 67f6778d50.
* Rename and reorder tests
* Add more tests
* Update tests to avoid hitting https://github.com/metabase/metabase/issues/46774
* Add Q8 tests and fix tests for Q9
* Update assertions
* Update assertions because of https://github.com/metabase/metabase/issues/48339
* Improve test names
* Add test
* Skip failing test
* Add more tests
* Add test
* Remove redundant code
* Remove invalid cases
* Add test
* Add tests
* Add more assertions
* Add more assertions
* Add assertions
* Add assertions
* Use helper functions
* Use helper functions
* Nest helpers consistently throughout the file
* Decrease flakiness
* Revert "Add a temporary workaround"
This reverts commit 48ed8770d6.
* Add a temporary workaround
* Add tests
* Add assertions
* Add tests
* Update unit tests
* Make temporary workaround safer
* Add TODOs to github links
* Extract a function to improve readability
* Simplify getParametersMappedToDashcard
* Fix failing tests by calling Lib.ensureFilterStage in the correct place and properly identifying the originalQuestion so that Question.prototype.isDirtyComparedTo returns expected result
* Remove redundant code
* Quick dirty fix to run in CI
* Format code
* Format code
* Update unit tests
* Fix test setup
* Create a new e2e test group because the old one was timing out in CI
* Add the group to GH action
* Revert "Remove redundant code"
This reverts commit 35b47a10f2.
* Use Lib.areLegacyQueriesEqual
* FE - e2e - Cover combination of unit of time parameter + non-last-stage filters with tests (#48726)
* Add JSDoc for entityPickerModalItem
* Add addToDashboard parameter to saveQuestion
* Add JSDoc for saveQuestion
* Fix import
* Add a test reproducing the bug
* Try to fix unit of time parameters
* Fix wrong stage-index for temporal unit parameters on composed ad hoc questions
* Fix ad-hoc question naming when there is an empty extra filter stage
* Fix unit tests
* FE - Do not add extra filtering stage for pivot tables (#48975)
* Avoid using Lib.ensureFilterStage for pivoted questions
* Fix unit test setup
* Add tests for pivot tables
* Optimize tests
* Test filter modal
* Revert redundant changes
* [WIP] Dashboard filter columns be fixes (#48906)
* Revert "Add a temporary workaround"
This reverts commit 4ea374870d.
* Fix some of the issues and unskip some of the E2E tests
* Acknowledge that metrics are transparent
* Fix breakouts by the same column with different temporal-units
* Use non-negative stageIndex for temporal unit parameters
* Add stage if there is an explicit reference to stage after the last
Instead of duplicating the ensure-filter-stage logic, just check if there is a parameter referencing the filter
stage (which is always one after the last in the query). If there is such a reference, add that stage.
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Require only breakouts when adding new stage in ensure-filter-stage (#49067)
* Only require breakouts when adding new stage in ensure-filter-stage
Previously, this function required both breakouts and aggregations.
Closes#48339
* Update e2e tests after changes to Lib.ensureFilterStage
This resolves all-but-one of the TODOs related to #49339
Related to #49022
* Implementing parameter_mappings migration (#49256)
* FE - e2e - Update tests after `Lib.ensureFilterStage` changes (#49222)
* Unskip #19744 repro
* Remove sanity checks
* Update comment
* Fix typo
* Remove invalid test
* Ignore pivoted "table" viz and and inline Question.prototype.isPivoted
* Remove test for pivoted table
* FE - e2e - Click behavior (#49239)
* Unskip #19744 repro
* Remove sanity checks
* Update comment
* Fix typo
* Remove invalid test
* Ignore pivoted "table" viz and and inline Question.prototype.isPivoted
* Remove test for pivoted table
* Add a test
* Disable columns from non-last stage in click behavior
* Finish the test
* Refactor
* FE - e2e - Cover public & embedded dashboards with tests (#49280)
* Add test for public dashboard
* Add test for embedded dashboard
* Add comment
* Add test for 2nd stage column
* Add test for 2nd stage aggregation column
* Add test for 2nd stage breakout column
* Add assertions for models
* Handle no auto-pivoting of ad-hoc questions from drill thrus
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Add missing test setup
* Address some review comments (#49429)
* FE - Show columns from all stages in click behavior (target question) (#48828)
* Add JSDoc for entityPickerModalItem
* Add addToDashboard parameter to saveQuestion
* Add JSDoc for saveQuestion
* Fix import
* Add a test reproducing the bug
* Try to fix unit of time parameters
* Fix wrong stage-index for temporal unit parameters on composed ad hoc questions
* Fix ad-hoc question naming when there is an empty extra filter stage
* Fix unit tests
* Fix non-unique keys
* Add stage-number to click behavior dimension target
* Call Lib.ensureFilteringStage in click behavior
* Use describeEE because click behavior tests use setTokenFeatures("all")
* Stop testing implementation details
* Stop testing implementation details
* Stop testing implementation details
* Modernize test and make it less flaky
* Address flake source
* Fix SAMPLE_DB_ID
* Unskip repro for #19744
* Add a test
* Update test name
* Introduce verifyNotebookQuery
* Use verifyNotebookQuery
* Use verifyNotebookQuery
* Move verifyNotebookQuery to helpers
* Assert items count in verifyNotebookQuery
* Extract helper functions
* Rename helpers
* Refactor
* Consistent comments
* Extract verifyVizTypeIsLine
* Add scrollIntoView calls for extra safety
* Add TODO
* Use verifyNotebookQuery
* Make stages optional
* Add createMultiStageQuery
* Remove redundant scrollIntoView calls
- We don't need them because the assertions are not using "be.visible"
* Improve test setup
* Add more assertions
* Add more assertions
* Add basic join support to verifyNotebookJoins
* Unnest code
* Implement join assertions
* Implement expressions and sort assertions
* Simplify code
* Add commented-out assertions
* Yes, it is correct
- it's stageIndex which sourceColumn comes from in sourceQuery
* Conditionally apply filter stage
* Allow mapping to all stages in click behavior
* Remove redundant argument
* Simplify diff
* Simplify diff
* Update test setup
* Remove TODO related to metabase#49282
- Closes#49282
* Support fetching param values on filter stage (#50120)
* Support fetching param values on filter stage
Fixes#48613
* Fix temporal-unit-parameters
* Enrich dimension targets in click_behavior with stage-numbers (#49911)
* Enrich dimension targets in click_behavior with stage-numbers
Fixes#49110
* Add missing stage-number
* Remove changeSynchronousBatchUpdateSetting references
* Update assertions to include time bucket
* Use mbql.u/legacy-last-stage-number in metabase.query-processor.card
---------
Co-authored-by: Tamás Benkő <tamas@metabase.com>
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* prevent the UI from breaking if there is an invalid enum value set by MB_LOADING_MESSAGE
* add test case
* add enum validation on backend
* oops - fix set-value-of-type! call
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* use -1 for stage index for breakout and aggregation list
* add e2e tests
* remove sum of total in e2e
* use test id matcher
* add issue to e2e name
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* move stageIndex to a separate variable
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* hide pivoted exports under env var
* only export pivoted when the env var is true
Also added tests to confirm.
* must specify :export for public settings
* spec
* change setting to :internal since it's only via env var
* use correct visibility add setting to settings.yaml for serialization
---------
Co-authored-by: Adam James <adam.vermeer2@gmail.com>
On MySQL 8.0.17, the following is prohibited:
```
WITH RECURSIVE MyRecursiveCTE (...) AS
(
SELECT ... FROM bar UNION ALL
SELECT ... FROM bar JOIN MyRecursiveCTE ON ...
)
```
Instead, you need to do:
```
WITH RECURSIVE MyRecursiveCTE (...) AS
(
SELECT ... FROM bar UNION ALL
SELECT ... FROM MyRecursiveCTE JOIN bar ON ...
)
```
Basically, just putting the CTE on the left vs. the right side of the
join (which is exactly equivalent under the hood) fixes the issue.
I can't actually find any documentation of this change, but it seems
that sometime between MySQL 8.0.17 and MySQL 8.0.33, MySQL started
allowing this, perhaps just an optimization where it now notices the
equivalence and does the right thing.
I originally tested this on what I believed was the oldest version of
MySQL we supported, but didn't read carefully enough. We recommend
8.0.33 or higher here:
https://www.metabase.com/docs/latest/databases/connections/mysql
But that's not for the AppDB - for that, we recommend 8.0.17 or higher.
* Fix serdes of missing cards in click behavior
When exporting a dashboard, we export its dashcards, including cards
referenced inside its viz_settings.
The problem is that viz_settings might have references to
now-nonexistent cards.
This is an initial stab at cleaning up the problem, specifically for
click behavior. If Dashboard D has Dashcard DC, which has a
click-behavior that links to Card C, then when we collect the
descendants of D, we'll include a reference to C - even if C has been
deleted.
So this makes two changes: first, when calculating the descendants of a
dashcard, don't include links to cards that don't exist. And, when
exporting the click behavior, remove the whole thing entirely if the
card it points to doesn't exist.
* Put `#locale` behind the feature flag
* Prepare folder structure for unit testing
* Refactor: extract a setup function as a shared util
* Add #locale tests
* Add all tests
* Prepare folder structure for unit testing
* Refactor: extract a setup function as a shared util
* Add locale tests
* Add all questions tests
* Fix failed E2E
* Tidy up questions tests
* Tidy up dashboards tests
* Fix test name typo
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Allow searching a single select list picker
* Fix unit tests
* Add repro for 49642
* Remove selection when searching
* Rename parameter to value because it does not need to be a prefix
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Simplify loading state for ListField
* Use same DEBOUNCE_FILTER_TIME in ListField
* Don't rely on waitTimeContext in test
* Fix lint
* Add initialization state for filter widget
* Update Loki Snapshots
* Update Loki Snapshots
* Allow picking other values when selecting a single value dropdown value
* Add comment about initialization
* Use setup in test
* Update Loki Snapshots
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* revamp the create question component
* docs updates
* add stories and update styles
* prevent switching to visualization when it is not ready
* ensure CreateQuestion works in flex parents
* add basic e2e tests for CreateQuestion
* ensure that switching between editor and visualization works
* use the save button for the disabled state
* update the question title when the question is saved
* implement updating questions in place
* rename replaceQuestion
* make CreateQuestion props all optional
* add background color to modal header
* hide the save button if question is dirty
* hide save button instead of disabling per design feedback
* update e2e tests
* revert e2e helpers
* add title update assertions
* whitespace changes
* Add new strings for v51.4
* Update translations
* kick ci
---------
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Don't try to fetch dbs for zero cards in api.table/batch-fetch-card-query-metadatas
Closes#48461
Ensure we don't throw an exception by attempting to fetch the Databases associated with potentially-deleted card ids in
api.table/batch-fetch-card-query-metadatas.
In particular, this ensures we don't generate an exception if the user does the following
1. Create a card "Source Card"
2. Create another card "Derived Card" with a :source-table of "Source Card"
3. Add Derived Card to a dashboard
4. Permanently delete (not just archive) the Source Card
* Add test for virtual-table query-metadata with deleted source card
* Add test for dataset query_metadata with deleted source card
* Add test for dashboard query-metadata with deleted source card
* Add test for card query-metadata with deleted source card
* Add metabase.api.test-util and use it in query-metadata-with-deleted-card tests
* Replace copy/pasted debug helpers with select-query-metadata-keys-for-debugging
* trigger save handlers on question create
* add skeleton example stories for save forms
* update storybook stories
* add tests for saving questions
* docs update
* add createCard assertions
* rollback docs
* feat(sdk): detect mismatch between sdk version and mb version
* refactor: log utils to print the big headers
* Update enterprise/frontend/src/embedding-sdk/lib/version-utils.unit.spec.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* refactor: call -> callArguments
* add link to the docs
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* remove LegendVertical logic that preserves legend width
* memoize pie legend items
* prevent pie from jumping when toggle legend
* fix responsive legend logic, cleanup
* drop hyphens from the pie legend (#29597)
* eliminate flickering on render due to not having dimensions
* shortcut
* specs
* test vertical layout has no hyphens
* setup func
* set explicit showLegend=true default
* [xlsx-export] Cache the custom datetime formatter
* [xlsx-export] Optimize functions in m.q_p.pivot.postprocess
* cljfmt
---------
Co-authored-by: Adam James <adam.vermeer2@gmail.com>
* Revert "fix(sdk): put a bandage on the flashing error on static question in strict mode (#49659)"
This reverts commit c57b3a38e0.
* refactor(sdk): use ignore flag instead of mounted ref
* Add temporal unit to display name
* Remove temporal unit adjustments from FE
* formatColumn to displayNameForColumn
* Factor out getFriendlyName
* Adjust BE tests
* Add require
* Adjust FE unit tests
* Adjust temporal-unit-parameters.cy.spec.js
* Adjust dashboard-reproductions.cy.spec
* Convert formatColumn occurrence
* Adjust legacy display name logic generation
After `getFriendlyName` was removed from FE, capitalization
of display name should happen here. Logic is similar, should
be adjusted prior merging the PR.
* Adjust embedding-linked-filters.cy.spec.js
* Adjust joins-reproductions.cy.spec.js
* Adjust metrics-editing.cy.spec.js
* Adjust timelines-question.cy.spec.js
* Adjust offset.cy.spec.ts
* TMP Avoid duplication of temporal unit in display name in MLv2 code
* Probably FE fix: add default temporal bucket only if not present
* WIP Add ends-with-temporal-unit? function
Current strategy is to add temporal units to display names if same is not already present
* Adjust display-name-method :metadata/column
Append TU to display name only if different to what's already present
* Adjust FE: show temporal unit that's present on field instead of default on hover on QueryColumnPicker element
* Return also temporal-unit from visible-columns
* Enable tempora-unit to be returned from previous-stage-or-source-visible-columns
* WIP Adjust multi breakout multi stage e2e multiple-column-breakouts.cy.spec
* Adjust multiple-column-breakouts.cy.spec.ts
* It was not a bug.
* Revert changes in _BaseBucketPickerPopover
* Restore temporal-unit for next stages removal in stage.cljc
* Add inherent-temporal-unit
* Adjust display name temporal bucket logic
* Rename to inherited-temporal-unit
* Shutdown too broad display name capitalization
* Eastwood
* Adjust pivot-rows-order-test
* Adjust pivot-non-numeric-values-in-aggregations
* Adjust inherent to inherited
* Rename to inherited
* Add inherent-temporal-unit only to returned-columns
* Update e2e/test/scenarios/dashboard-filters/temporal-unit-parameters.cy.spec.js
* Update e2e/test/scenarios/question/multiple-column-breakouts.cy.spec.ts
* Update consistent-date-formatting-test
* Update dashboard/x-rays.cy.spec.js
* Update waterfall.cy.spec.js
* Return month default for default inherited-temporal-unit
* Update reproductions-3.cy.spec.js
* Update downloads/downloads.cy.spec.js
* Update legend.cy.spec.js
* Update pie_chart.cy.spec.js
* Update click-behavior.cy.spec.js
* Update embedding-linked-filters.cy.spec.js
* Return QueryColumnPicker's condition to original state
* Remove reundant export
* Update e2e/test/scenarios/question/multiple-column-breakouts.cy.spec.ts
* Comment
* Update nested.cy.spec.js
* Update e2e/test/scenarios/sharing/sharing-reproductions.cy.spec.js
* Adjust to TS types
* Adjust for cljfmt
* Capitalize
* Add propagation tests
* Add inherited-temporal-unit propagation for expressions
* Add default inherited-temporal-unit test cases
* Add qp temporal-bucket-test ns
* Simplify default unit logic
* Comment
* Use NULL_DISPLAY_VALUE
* Comment
* Comment
* Comment
* Comment
* Comment
* Comment
* Remove redundant code (for now)
* Update comments
* Comment
* Restore way too daring simplification
* cljfmt
* Adjust capitalize
* Update comment
* Update test
* Address remarks in tests
* Update test
* Address remarks
* typo
* Use Flex instead of Group for items in button
* Allow column picker to stretch to match the height of the element
* Fix height of data picker buttons
* Make sure the whole button is clickable
* Add e2e test for metabase/metabase#50038
* Use correct id for issue in e2e test
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* attempt to fix 49537 by removing position relative from the tooltip
container
* Make echart tooltip container cover all iframe area
* Update the E2E test
* Update loki snapshots from CI
* Re-enable echarts tooltip assertion.
I couldn't find a better way to fail these assertions on the previous
version of the code, but these assertions are better than nothing.
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* XLSX Pivot Export - Initialize Pivot Table with Small Area Ref
This PR changes the native pivot export so that we first initialize the pivot table with an area ref that's only the
first 2 rows. Prior, I used `AreaReference/GetWholecolumn` which had the side effect of using lots of memory.
I think this is related to how the Pivot Table's Cache is built up as you add rows/cols to the pivot definition. So,
if you keep the area ref only as wide as the number of columns in the raw data, and just 2 rows, then the cache can
stay small.
After adding row/column data to the pivot table, you can then modify the area ref to be all rows in the relevant
columns (`A:E` for example).
This keeps the pivot-table object much smaller in size.
* need the extra xml schemas to get this to work
I'd like to try find a way around including this extra dep, it's a 13mb jar (ish), and I think that's a little bigger
than I'd like
* oops, didnt mean to have this in
* Add test for pivot table initialization being fast and lean on memory
* remove time based assertion, add comments in impl
* cljfmt
* fix test
* Pesky little whitespace
---------
Co-authored-by: Oleksandr Yakushev <alex@bytopia.org>
* XLSX Pivot Exports Now Use The Sort Settings from Viz Settings
Prior to this PR, the xlsx pivot exports worked correctly except for the fact that no sorting (ascending or
descending) was applied to the pivot rows or columns. Now, those settings are used when they exist.
* only try to set sort when a correct setting exists
* add ooxml-full dep for correct classes
* Update src/metabase/query_processor/streaming/xlsx.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
---------
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Fixes: #50072
When we `describe-fields` we are passing db details, but sometimes we
have the database name in `dbname` rather than in `db` depending on the
connection specs and possibly environment. So we check for both now.
== Goal ==
Hide attached DWH database details from anyone incl. admins:
* Do not show them in the UI
* Do not permit to change them
* Do not serialize them
- This is what we're adding for H2 snapshots (dumps) with this PR.
The aim is that customers cannot gain access to (parts of) credentials,
and they cannot break a feature they are paying for by changing
connection details.
== Implementation ==
In 592360c9fa I wrongly understood that
database details would be omitted when dumping to H2, but this is only
true for dumping H2 databases.
Fix this by omitting database details also when dumping databases with
`is_attached_dwh` set.
== How to test ==
To prepare, download a H2 JAR matching the H2 version used by Metabase:
```
wget https://repo1.maven.org/maven2/com/h2database/h2/2.1.214/h2-2.1.214.jar
```
If the H2 version does not match, you will get an error when trying to
open the H2 shell:
```
Exception in thread "main" org.h2.jdbc.JdbcSQLNonTransientConnectionException: Unsupported database file version or invalid file header in file "[REDACTED]" [90048-232]
```
=== New behaviour ===
Setting the `is_attached_dwh` field hides the database details from H2 dumps:
1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
- In addition to the fields you would normally set, also set
`is_attached_dwh: true`.
- This also works when adding this flag to a database that previously
did not have this flag set.
2. Start your Metabase instance.
3. Run `${metabase} dump-to-h2 ./dump-file-h2` to create the H2 snapshot file.
4. Run `java -cp h2-2.1.214.jar org.h2.tools.Shell -url "jdbc:h2:file:./dump-file-h2;ifexists=true"` to open a H2 shell.
5. Verify that `SELECT name,details FROM metabase_database;` shows `{}` for the database you added in step 1
=== Original behaviour ===
Behaviour without setting the `is_attached_dwh` field is unchanged:
1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
- Only set the fields you would normally set. Do not set
`is_attached_dwh` (or set it to `false`).
2. Start your Metabase instance.
3. Run `${metabase} dump-to-h2 ./dump-file-h2` to create the H2 snapshot file.
4. Run `java -cp h2-2.1.214.jar org.h2.tools.Shell -url "jdbc:h2:file:./dump-file-h2;ifexists=true"` to open a H2 shell.
5. Verify that `SELECT name,details FROM metabase_database;` shows a non-empty object (i.e. not `{}`) for the database you added in step 1
Fixes: 592360c9fa
Closes: https://github.com/metabase/harbormaster/issues/5526
* fix(sdk): remove `Cannot update a component XX while rendering a
different component` error
* test(sdk): introduce jest test to make sure api.basename is set before
loading the locale
* sync the attached datawarehouse
requires either "sync_db" true or a table name
it will error
```shell
❯ http post 'localhost:3000/api/notify/db/attached_datawarehouse' x-metabase-apikey:apikey -pb
Must provide `sync_db` or a `table_name` to sync
```
It will sync the db when sync_db is true [NOTE: this is outdated. now just omitting the table name will sync the db]
```shell
❯ echo '{"synchronous?": true, "sync_db": true}' | http post 'localhost:3000/api/notify/db/attached_datawarehouse' x-metabase-apikey:apikey -pb
{
"success": true
}
```
It will sync existing tables
```shell
❯ http post 'localhost:3000/api/notify/db/attached_datawarehouse' table_name=existing schema_name=public x-metabase-apikey:apikey -pb
{
"success": true
}
```
it will error if it cannot find a table
```shell
❯ http post 'localhost:3000/api/notify/db/attached_datawarehouse' table_name=new schema_name=public x-metabase-apikey:apikey -pb
{
"cause": "Unable to identify table 'public.new'",
"data": {
"schema_name": "public",
"status-code": 404,
"table_name": "new"
},
"message": "Unable to identify table 'public.new'",
"schema_name": "public",
"table_name": "new",
"trace": [],
"via": [
{
"data": {
"schema_name": "public",
"status-code": 404,
"table_name": "new"
},
"message": "Unable to identify table 'public.new'",
"type": "clojure.lang.ExceptionInfo"
}
]
}
```
if i create that table
```sql
attached=# create table new (id int);
CREATE TABLE
```
it will then find and sync it
```shell
❯ http post 'localhost:3000/api/notify/db/attached_datawarehouse' table_name=new schema_name=public x-metabase-apikey:apikey -pb
{
"success": true
}
```
* formatting
* Tests for attached datawarehouse
* PR nits
typo in `thorw` -> `throw`
comment explaining why `find-and-sync-new-table` is always sync
gensym the db name in test
remove a binding for a fn `sync!` that's only called once
test `{:sync_db true}` syncs the whole db
* stupid formatting 🦾
* Use absence of table_name as indicator to sync database
* Fix smartscalar texts to use user locale
* Fix Pie chart texts to use user locale
* Fix line chart tooltip texts to use user locale
* Add trend chart assertions
* Add pie chart assertions
* Add line chart other series tooltip assertions
* Update frontend/src/metabase/visualizations/echarts/pie/constants.ts
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Fix failed E2E tests
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Apply Column Sort To Pivot Sections
Fixes#49437
This PR fixes the dataset API so that column sorts (ascending/descending settings on pivot-rows and pivot-cols) are
properly parsed and passed through the QP.
After that, I've also modified the post processor to use the sort orders properly in pivoted exports.
* println debugging :)
* fixing a few tests
* fix a few more tests
* Update workflow on master
* [DEBUG] Disable steps for testing
* Add alberto as a reviewer
* [DEBUG] modify the branch name for testing
* Revert "[DEBUG] modify the branch name for testing"
This reverts commit 7d5d8bf802.
* Revert "[DEBUG] Disable steps for testing"
This reverts commit a8662c9e8b.
* Try snowflake 3.19.0
* Deafault to java 21 for drivers
* Trigger CI
* Try running things with --add-opens to see if it solves our problems
* Fix `update-view-dashboard-timestamp-test`
* Try snowflake 3.19.0
* Deafault to java 21 for drivers
* Try running things with --add-opens to see if it solves our problems
* Fix `update-view-dashboard-timestamp-test`
* Update deps.edn
* Switch to Java 21
* Docker image needs to use --add-opens option
* Add note about change to Java 21 to driver changelog
---------
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>
* add filter picker skeleton
* add filter picker skeleton
* add picker in popover use case
* reuse FilterColumnPicker
* add question filter logic
* remove fixed width from popover
* add onClose handler
* add e2e test skeleton
* contain FilterBar in Box in story
* add a test for adding filters
* update e2e tests
* add filter picker to the docs
* inline props and functions
* update the props
* Allow using the arrow buttons to change the limit value
* Add a min prop to the number input
* Use base-10
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Ensure number is integer
* Add issue name to unit test
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Use ParameterValueWidget for default value picker in native parameter settings
* Fix e2e test
* Fix native filter tests
* Fix e2e test
* Fix e2e test
* Add repro for #44665
* Assert that filter values are shown in the popover.
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
This also cleans up the testing to check everything more carefully,
and to expect the right things across multiple stages.
(This is a bit tricky since the order-bys get removed when the stage
gets aggregated, because the columns they were using are now gone.)
* Remove dead code from browse models page
* Use the grid to control the gap
* Implement "Add database" button
* Add basic unit tests
* Make sure the feature works when there are no databases
* Add analytics to the db list
* Add "Add database" button to the sidebar
* Add E2E Snowplow tests
* Add unit tests
* Deal with authentication in each beforeEeach separately
* Add Snowplow E2E test for a database card
* Slightly tweak the title line height
Search currently returns a detailed list of the scores for each item, which is pretty hefty, and the type is also coupled to the current rankers, which will change in future.
Removing from the type to start to just validate that nothing uses it.
Best to just not model this. Going to see about just removing it from the BE as well, and put in some other way to debug. Worst case an opt-in query parameter.
* Add blurb from C3P0 docs to comment for unreturnedConnectionTimeout
* Enable debugUnreturnedConnectionStackTraces in data-warehouse-connection-pool-properties
Closes#47981
* Add defsetting for jdbc-data-warehouse-debug-unreturned-connection-stack-traces
And only enable debugUnreturnedConnectionStackTraces in the C3P0 config if the setting is enabled. There is some
memory cost to enabling the setting, and since maxPoolSize is user-configurable, it seems safer to disable
debugUnreturnedConnectionStackTraces by default.
* Add metabase.logger/level-enabled?
* Log a warning if C3P0 logs are not configured correctly to log stack traces
Log a warning if the user enables the jdbc-data-warehouse-debug-unreturned-connection-stack-traces but does not raise
the log level for the com.mchange package to INFO or higher.
* Add docs for MB_JDBC_DATA_WAREHOUSE_DEBUG_UNRETURNED_CONNECTION_STACK_TRACES
* Improve markdown docs for MB_JDBC_DATA_WAREHOUSE_DEBUG_UNRETURNED_CONNECTION_STACK_TRACES
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Keep Pivoted Option works in Subscriptions
Fixes: #49525
Even though the user could toggle 'Keep data pivoted` in the Sharing Sidebar UI, the setting was not being propagated
to the backend, so attachments would not be pivoted.
* add a cypress test
* add viz settings to get the pivot working
* fix for linter
* remove eslint ignore
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
I had to add `only-safe-columns` to filter out columns with known issues
in current pMBQL. In particular, implicitly joining on a column which
appears twice (eg. because of a double- or self-join) is ambiguous with
classic `:field` refs, and will cause errors.
There's a `:dynamic` var to re-enable all the columns for tests that are
only exercising new `:column` refs.
* Revert "[streaming-response] Use HttpEndpoint.isOpen() instead of reading a byte from TCP socket (#49460)"
This reverts commit 060baf0ce4.
* Disable connection reuse for requests that serve streaming responses
This introduces a reusable `weighted-choice` function and a "weight"
system for the various steps. `:order-by` should be more rare than
filters, etc., `:limit` and `:append-stage` should be more rare still.
Adds basic expressions to the query generators.
There's a lot more to do to robustly generate realistic expressions! But
this is enough to make expressions exist on queries, which then get
consumed as inputs to filters etc.
* build on java 21
* update docker image
* update test
* more using 21
* back to 11 for build
* update docker command
* update containerization
* update docker actions
* revert dockerfile change
* update other docker action versions
* use release commit for containerization
* Remove @flaky tag
* Unskip test and remove @flaky tag
* Remove @flaky tag
* Remove @flaky tag
* Revert "Remove @flaky tag"
This reverts commit 08c57add88.
* Try to fix flake
* See if timeout is necessary
* Revert "See if timeout is necessary"
This reverts commit 181f3c5e63.
* Revert change for coordinates filter
* Remove @flaky tag
* Make workaround test pass
* Disable workaround test
* Uncomment and unquarantine the workaround test
* Disable normal test for now
* Revert "Disable normal test for now"
This reverts commit 68fa2175a7.
* Bring back quarantine
* Update comment
* Move the upload CSV button to the main nav bar
* Remove the old `SidebarOnboardingSection`
* Remove obsolete unit tests
* Rework E2E tests
* Add some spacing around the button
* Fix E2E test
* Break `MainNavbar` unit test down
* Expand unit tests
* Update E2E tests
* Fix E2E test failure
* Move JSDoc where it belongs
* Provide context for translators
* Simplify the condition
* Pivot Measures Order Used in Pivot Exports
Fixes#48442
A pivot table can have any number of measures, and the user can order these by dragging in the UI. Before this PR,
that order was ignored and measures would alway be in index order, which is confusing for any user who needs the
measures to be displayed in a particular order, especially if they've re-ordered them in the pivot viz settings UI.
A test has been added to check that measure order is used.
A few minor changes to the pivot qp and post-processor
- measure indices are looked up in the pivot qp and added in viz-settings order
- the pivot measures are only added if the qp has not already added them.
- pivot-opts Malli spec has been made in the namespace, adjusted to allow `nil` as a valid pivot-opts output, and
used in relevant functions
* address review points.
* add a rows order test
* Filter out Empty Rows from Pivot Exports
Fixes#49353
The linked issue is not actually related to pivot export size but is instead related to the 'Min of Created At: Month'
aggregation; the default aggregation function was `+`, so it broke when the date string was encountered.
That was fixed.
As I was trynig to keep the export small if possible, I noticed that in some cases empty rows are appended, so I added
the filter so that if a pivot row's values are completely empty, it doesn't add it.
Finally, this PR also adds the 'sub section totals' which I noticed were missing from the exports. This comes up when
you have 3+ pivot-rows configured, so you can see the subtotals for the first pivot row and the subtotals nested
within those sections for the second pivot row, and so on.
* add test for non-numeric values
* Make sure the new name refs match on aggregations not just breakouts
* cljfmt
* move the database prompt to the very beginning
* ability to use sample database for cli
* print username and password in console
* no tenancy isolation as a default option
* highlight sample credentials file
* update credential log messages
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* continue without tenancy isolation
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Fix :pivot-measures with column names
* Fix the test
* Update src/metabase/query_processor/pivot.clj
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Add a test
* Add a test
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Adds filters to generative testing for queries.
There's a lot of new code here, but don't panic, it's quite straightforward:
generating filter values, in the right shapes for various filter expressions.
* refactor + more specific auth errors
* fix linting issues
* make tests pass, again
* make test pass for real?
* continues the refactor and hopefully makes tests pass
* try to simplify the logic a little bit more
* also logs to the console for the edge case of invalid session
* remove setLoginStatus
* fix typo
* fix debug line that should have been commented
* remove dead code
* remove test for code that doesnt exist
* address feedback from PR
* update test copy
* Update enterprise/frontend/src/embedding-sdk/store/auth.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Update enterprise/frontend/src/embedding-sdk/store/auth.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* update copy for site settings error
* ex -> exception
* Update enterprise/frontend/src/embedding-sdk/store/auth.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Update enterprise/frontend/src/embedding-sdk/store/auth.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* update tests to use the updated copy + small typos
* refactor: use jest spy on to check for console.error
Co-authored-by: "Mahatthana (Kelvin) Nomsawadi" <me@bboykelvin.dev>
* use full error message in test
* renamed response to session
* mock implementation of console.log
* fix(49482): don't create a new store on each MetabaseProvider render (#49492)
* fix(49482): don't create a new store on each MetabaseProvider render
* remove store prop as it seems to not be used
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
The query can contain all kinds of things, in particular the whole original query in the case of pivot questions, so
we should check only the part that actually gets resolved by the middleware.
* perf: Implement faster sync methods for postgres
Fixes#48575
Pulls work from redshift into the common postgres driver.
* Fix tests and formatting
* Move nested-field-column sync to sync functions so describe-fields will also get them
* Fix test
* Fix test
* Remove fixed safety test
* Add test specific database-supports feature for pk metadata
* Fix test
* perf: faster mysql sync with describe-fields
Fixes: #49010
* Adrress PR feedback
* Fix tests
* Fix test
* Add nil table-schema
* Don't use subselect for field-comment
* Fix quoting weird identifiers
* Make format string inline
* Update src/metabase/driver/mysql.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update src/metabase/driver/mysql.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Fix tests
* Fix database-type
* Fix tests
* Fix test
* Fix tests
* Exclude mysql table_schema
* Handle tinyInt1IsBit
* Fix test
* Only get one db at a time
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
These are helpful for creating literal dates and times in a
cross-platform way. Useful for generating queries, in particular for
expression and filter values.
* add base color palette
* add some more comments
* fix formatting and color copy-pasta errors
* ignore existing bad base color usage
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
The ref (usually) contains enough information to unambiguously find the
matching implictly-joined column, even if there are several FKs that
could join the target table.
The remaining ambiguous case is if the *same* FK column exists
multiple times - current MBQL uses raw field IDs for the FK field, so it
can't distinguish between eg. an FK in the main table and the one from a
self-join.
* Update saveDashboard helper to wait for the GET /api/dashboard/:id request
- update aliases to be unique so that they don't interfere with aliases used in tests
* Remove no longer needed intercept + wait
* only tag nightly on the current version
* add release channel setting to publish job
* clean up release workflow
* remove code from something else
* reduce depth of the release process stack
* update build job too
* remove dead arg
* Pivot Measures Order Used in Pivot Exports
Fixes#48442
A pivot table can have any number of measures, and the user can order these by dragging in the UI. Before this PR,
that order was ignored and measures would alway be in index order, which is confusing for any user who needs the
measures to be displayed in a particular order, especially if they've re-ordered them in the pivot viz settings UI.
A test has been added to check that measure order is used.
A few minor changes to the pivot qp and post-processor
- measure indices are looked up in the pivot qp and added in viz-settings order
- the pivot measures are only added if the qp has not already added them.
- pivot-opts Malli spec has been made in the namespace, adjusted to allow `nil` as a valid pivot-opts output, and
used in relevant functions
* address review points.
* add a rows order test
* only tag nightly on the current version
* add release channel setting to publish job
* clean up release workflow
* remove code from something else
* fix step title
Also fix a bug in `lib.breakout/existing-breakouts` that treated
breakouts on duplicate columns from different joins as existing, even
though they're not.
* add ability to hide the collection picker
* add save options
* make sure collectionId is fresh
* update sdk question form component
* make collectionId accept null
* apply saveOptions to sdk components
* revert to use initialCollectionId as context name
* target the correct collection id
* add e2e tests
* update e2e tests
* move into within
* update test
* remove collectionId from default args
* revert e2e of hide columns
* make e2e helper more generic
* use saveToCollectionId prop to save to a specific collection
* update e2e tests
* update comment formatting
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* clean up leftover changes from an earlier version
* readme update
* indicate that saveToCollectionId is only applicable to static questions
* revert QuestionEditor.isSaveEnabled defaults
* update comment format
* remove unused initialCollectionId prop
* experiment: try to fix failing custom-columns e2e
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Adding moderated status to entity picker
* adjusting verified icon size
* size adjustments
* e2e test adjustments
* removing verified model icon
* fixing e2e test
* unit test
* unit test adjustment
* Adding moderation status to Dashboards
* react sins
* verification coll: subselect on card + dashboard
* fix the 3 failing BE tests
* add an api test for dashboard verification
* verified dashboards are never stale
* e2e tests
* code cleanup
* unit test and e2e test cleanup
* PR Feedback
* backend test fixes
* Nick shouldn't touch clojure
* type fix
* indent
* fix last 2 BE tests
---------
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* WIP: use list picker when searching a single value
* Add test case for multi vs non-multi search fields
* Update loki snapshots
* Add special case to test for multiautocomplete filter
* Use single select value in test
* Add test for using list picker
* Explicitly make sure elements are visible
* Do not use list selector when search is disabled
* Add margin to multi autocomplete dropdown
* Update loki snapshots
* Update loki snapshots
* Fix scroll for EditableDashboard in fullscreen mode
* Fix scroll for InteractiveDashboard in fullscreen mode
* Display fullscreen control to dashboards by default
* Fix issue with public shared questions empty footer
Previously these were omitted, and it caused invalid SQL to be
generated, since the ORDER BY (time unit not updated) and GROUP BY
(time unit updated) did not line up.
Fixes#49263.
Lays the foundation for a weighted probability approach to generating infinite sequences of valid queries.
Only has aggregations so far, but we can expand it to filters, breakouts, order-by, removals etc. etc. in later PRs.
* Update fullscreen dashboard filter on dark theme
* Use the latest base color name
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
* perf: Implement faster sync methods for postgres
Fixes#48575
Pulls work from redshift into the common postgres driver.
* Fix tests and formatting
* Move nested-field-column sync to sync functions so describe-fields will also get them
* Fix test
* Fix test
* Remove fixed safety test
* Add test specific database-supports feature for pk metadata
* Fix test
* Adrress PR feedback
* Fix test
* Don't use subselect for field-comment
* Fix quoting weird identifiers
* Make format string inline
* Remove unused component
* Remove extra space
* Remove cross-component dependency on the `Banner`
* Use named export
* Convert the Banner to TS
* Make `ReadOnlyBanner` use `Banner` under the hood
* Add test id to the banner
* Simplify render method and tests
* Add the missing test for ReadOnly banner
* Avoid using special ASCII characters for the common apostrophe
* Remove E2E spec
* Use `Banner` for `DatabaseInfoField`
* Use colors defined in the theme
* Remove the dependency on the `Banner` component again!
* Add a comment
* Use pattern matching
* Mongo objects should download as JSON, not EDN
Fixes#48198
Prior to this change, object columns (base or effective type of :type/Dictionary) were just formatted with `(str
value)` which results in a csv or json download containing EDN formatted objects.
This is a bug because we present object column values as json in the app, so the expected formatting of the download
should match this.
The formatter function now takes this type into account. As well, since this is a type of formatting that should be
always applied (even when format_rows is false), the function is modified to unconditionally apply the json/encode
formatting to dictionary types when encountered.
* add a test
* add proper condition to test
* card-download should be public
* uncomment json encoding formatter
* set-cell! should keep encoded json string for Objects
I think this is the correct change; I don't really understand the reason for wrapping, encoding, decoding, and then
string-ing that value. Maybe I'm missing something.
* Adjusted xlsx Object set-cell! implementation
* forgot the not... inverted
* Update cursor position when selection changes
* Add reproduction for #33640
* Revert "Add reproduction for #33640"
This reverts commit 573f6320f5.
We cannot reproduce the bug with tests, so having a test is useless.
* Add reproduction for #12586
* Do not show the dirty overlay when an error occurred
* Fix tests that rely on dirty overlay
* Add issue tag to test case
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add max-height to the click actions popover to prevent overflow and infinite scroll
* Add reproduction for #48851
* Update reproduction to speed it up
* Use existing helper for table header click
* Add max height to string filter picker
* Revert "Add max-height to the click actions popover to prevent overflow and infinite scroll"
This reverts commit 166478d892.
The audit collection is a bit weird. Users have permissions on the
collection but even superusers don't actually have write permissions on
the elements within it.
For now, I'm just excluding anything in the audit collection from the
"stale" endpoint, so we won't suggest cleaning these up, but I wonder if
maybe there's a cleaner way here. It feels weird to fake permissions
like this.
Fixes https://github.com/metabase/metabase/issues/49164
* Add new route
* Add the navigation link
* Guard the route
* Add first pass of the page structure
* Add the bottom section
* Add placeholder dummy YT videos
* Add images
* Track checklist being opened
* Track checklist item expanded
* Track checklist item CTA clicked
* PoC: Stop the video on accordion change
* Use actual onboarding videos
* Fix missing unique `key` prop
* Final content tweaks
* Override the accordion styles defined in the `theme`
* Use predefined t-shirt sizes where possible
* Update copy
* Conditionally display the example dashboard CTA
* Add very basic unit test coverage
* Show onboarding link to new instances only
* Remember the last position in the onboarding
* Fix the mock settings types
* Scroll last opened item into view
* Properly display docs links
* Another accordion style override
* Add tests for the Onboarding page contents
* Add E2E Snowplow tests
* Conditionally display xray item contents based on settings
* Split tests
* Make sure items scroll into view when expanded
* Do not show example question CTA if example dashboard doesn't exist
* Allow YT videos to play in fullscreen
* Add comments about the example question
* Update rules for when we show the onboarding nav link
* Test the conditional documentation links display
* Reduce E2E test flakiness
* document loader and error states
* document loader and error states in the docs
* revert changes to README.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* pie chart crashes when light or dark accents of instance colors overriden
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Add verifyNotebookQuery helper
* Use describeEE
* Fix DB id
* Refactor test
* Refactor tests
* Introduce customizeLinkText helper
* Use verifyNotebookQuery
* Fix test setup
* Fix static embed card download when param has accentuation
* Fix E2E util type
* Add a dashboard E2E test
* test question download with params
* Fix some names regarding tests
* Fix failed E2E tests
* Fix csp directives for embed previews
We set content security directives to allow for iframes on
dashboards. This list did not include 'self' so we can't actually host
an iframe pointing at our, well, self.
Embed previews work by just embedding an iframe with the dashboard and
this breaks if we don't allow iframes from our self.
* e2e test
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
### Description
This PR updates Macaw to the latest version, which exposes a new analyzer which handles compound queries.
It also finishes the migration to using tagged-union result maps consistently across all the interfaces.
This should also remove the atrociously verbose analysis error messages in CI.
* investigate hide columns
* add update viz settings in the sdk
* cleanup implementation
* add storybook for interactive question
* use the question callback to update viz settings
* remove onUpdateVisualizationSettings
* revert change to onUpdateVisualizationSettings
* add click action behaviour
* fix misspelling
* add onUpdateQuestion mock function
* add unit test for click action behaviour
* add e2e test for hiding question column header
This standards on a "just maps" approach to returning disabled, disallowed, or failed query analysis. This is as opposed to the hodgepodge approach of significant `nil`, untagged unions with keywords, and exceptions.
Essentially we have a classic "either" style tagged union, where `:error` is the tag.
* test(sdk): add tests for auth flow
* make the fetch token fn return something that will pass the validations
we'll add in the next project
* waitForRequest helper + add expects for card endpoints on api key test
* upgrades is-airgapped? defn to defsetting airgap-enabled
- uses the airgap-enabled settings value to remove Cloud section in
SettingsEditor
- "Metabase Enterprise Airgap" -> "Enterprise Airgap"
* hide still need help section in airgap mode
* move flag checking into selectors
* fe lint
* ts type check
* use optional chaining and nullish coalescing
* hide StillNeedHelp in ag mode
* adding airgap-enabled to createMockSettings
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* add allowed iframe host setting wip
* use allowed-iframe-hosts setting in the CSP header
* add a test for the frame-src csp directive
* Update allowed-iframe-hosts setting definition
* Add error state for forbidden iframe url domains
* Move out iframe e2e test suite
* Add e2e test
* Update error message in view mode
* Fix unit tests
* Update setting on the admin page
* Update error state
* Add links to error states
* Update docs links
* Update link anchors
* Add default allowed hosts to public setting
* Update allowed domain check logic
* Fix default value display in admin page
* Don't update setting without changes
* Update error message spacing
* correct the parsing of allowed-hosts string for CSP header entries
* fix test
* fix not handling wildcard ports
* Fix failing e2e test
* Fix subdomain test
* address review
- the parse-allowed-iframe-hosts fn is now memoized
- a * entry is handled and doesn't produce a weird *:* entry
- no more try/catch, errors in parsing will be logged but the list returns all valid entries
- when www. is encountered, an entry including www. is added
- trailing / is 'cleaned' and the entry is used as if there was no trailing /
* Fixup test for expecting a few more frame sources
* indentation fix for linter 🥲
* Fix type error
---------
Co-authored-by: Adam James <adam.vermeer2@gmail.com>
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
* ui implementation of new emb homepage, tests will come later
* change padding of the last card to be consistent with the top one
* update beta badge spacing
* update tests
* fix stories type error
* use quick-start for url
* update HomeContent tests
* update events + related e2e
* Update frontend/src/metabase/home/components/EmbedHomepage/StaticEmbedContent.tsx
* Add `graph.max_categories` viz setting
* Allow grouping series into "other" category
* Add tooltip
* Toggle "other" series visibility
* WIP: new implementation for `graph.series_order`
* Remove not used function
* Make "other" series a regular `SeriesModel`
* Fix stacked bar charts
* Remove colors from "other" tooltip
* Sort "other" tooltip values
* Update `graph.max_categories` setting
- negative number validation
- 0 value turns off grouping
* Fix Loki test
* Add basic loki test
* Add loki test for stacked chart
* Add loki test for stacked normalized chart
* slight refactor of series_order setting
* Add screenshots
* Handle different kinds of aggregations for "other"
* Handle different aggregation kinds in tooltips
* Add `graph.other_series_aggregation_fn` viz setting
* Fix type errors
* Fix unit tests
* settings ui
* remove sorting by value to match series order setting, other color
* hide grouped series controls
* Add e2e test
* Update screenshots
* Fix incorrect series grouping for stacked charts
* group series into other settings
* Update e2e test to work with new viz settings
* Disable drills for the "Other" series
* fix max_categories setting popover positioning
* fix total row has misaligned columns on other series values
* Remove redundant `click({ force: true })`
* Rename aggregation fn viz setting
* Move aggregation fn setting to the popover
* WIP
* Update popover
* Fix `it.only`
* Fix legend sync issue
* Disable "Other" category by default (temporary)
* Enable aggregation fn picker for MBQL queries (temporary)
* Move "Other" category summary to a tooltip row
* Add `graph.max_categories_enabled` to viz settings type
* Reuse already computed "Other" value in tooltips
* Add null check
* Fix series length check
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
This is to allow for the possibiliy of adding more metricsv2 usage and error metrics in the future while still
distinguishing the source of the errors.
### Description
This is a low level step towards applying table-level permissions to native queries.
It replaces the more generic and powerful `:ast-walker-1` analyzer with the more conservative `:basic-select` version we recently landed. The advantage of this version is that it protects against dynamic table references and false negatives due to masking.
This was throwing errors trying to `nippy/freeze!` an Atom.
That was because in certain circumstances a pMBQL query can end up in
`(:json_query metadata)` for a query, and those can contain an atom
due to `metabase.lib.cache` and both `CachedMetadataProvider` and
`InvocationTracker`.
I added `lib.query/serializable` as a single location to strip out
anything that isn't serializable, such as atoms. (And the entire
`:lib/metadata`, which is a wasted effort to serialize.)
* Fix reference in deprecation notice
* Add lock_filled icon
* Add view-only tag to the question header for questions with hidden source tables
* Simplify the useHasHiddenSourceTables helper
* Add typings to useHasHiddenSourceTables
* Show 'view-only' badge when the question is not editable
* Set up card endpoints
* Add test for hidden tables
* Add e2e test for View-only tag
* Add visibilityType to TableDisplayInfo
* Use unfiltered table display info to determine which tables are hidden
* Rename util to hooks
* Use rem for border-radius
* Sign in in beforeEach
* Add tests for more visibility types
* Fix unit tests
* Remove empty file
* Add test for unavailable cards
* Remove unnecessary card endpoint
* Add unit test for inaccessible card
* Move ViewOnlyTag to it's own file
* Add more unit tests for ViewOnly tag
* Use MBQL to create queryies where possible
* Increase the size of the overlay spacer
* Increase the delay of the popover opening
* Hard-code the popover delays
* Remove delay props from info components
* Fix unit test
* Shorten popover timeout to 500ms
* Make popover delay configurable again
* Increase openDelay for header info
* Fix a Couple bugs related to pivot exports
- pivot-grouping value, when not an int (in the case of Oracle dbs), would cause exports to contain zero rows. This
fixes that
- json formatted/unformatted for the dataset API endpoints (unsaved questions) was not being used, so all exports
were formatted
* json and csv use (int group) to check row inclusion
* grouping change fix tests
* json formatted/unformatted test
* xlsx formatters are correct when pivot-grouping col is removed
* cljfmt
* fix up failing tests
* fix test
* Update src/metabase/api/dataset.clj
* Reset the question's viz settings when converting to a model
* Update e2e/test/scenarios/models/reproductions.cy.spec.ts
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Test model metadata
* Test model metadata
* Test model metadata
* Test model metadata
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix#48878 by avoiding to use entity framework which can serve incomplete card objects
* Make sure canWrite is a boolean
* Remove redundant optional chaining
* Add repro for #48878
* Optimize
* Improve comment
* Fix test setup
* Automate npm version
* Make troubleshooting SDK release process easier
* Fix quote situation
* Allow deploying SDK to different branches at the same time
Because they won't be in conflict with each other
* Make checkout a bit faster
* Remove unnecessary console.log
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Tidy up `jq` command
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Tidy up multiple GitHub actions summary log
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Add helper to add utm params to url
* Add utm param to getDocsUrl
* Allow passing utm to docs url
* Add utm parameters to expression widget docs url
* Add useUrlWithUtm hook
* Make more utm props optional
* Use the utm hook and docs hook instead of adding utm tags manually to urls
* Simplify getDocsUrl
* Update unit tests for expression editor to contain utm params
* Only add parameters if they have a value
* Fix unit test
* Fix unit test
* Simplify docs url
* Use useDocsUrl on embedding page
* Use named arguments in useDocsUrl
* Modularize metabase.pulse; add metabase.pulse.core API namespace
* Rename metabase.pulse.send-test
* Split email result attachment stuff into its own namespace
* Update dashboard-subscription-test
* Allow metabase.email.result-attachment externally... for now
* More code cleanup
* Decouple models.card and email.messages
* Decouple models.collection and metabase.email.messages
* Hopefully the last fix 🔧
* Fix#48647🙁
* Fix event handler
* :render/text, not :pulse/text
* Test fixes 🔧
* Test fix 🔧
* RESPECT the modules
* Remove unused namespace
* defineEmbeddingSdkConfig to make typing easier when the config is created outside of the jsx
* also add defineEmbeddingSdkTheme, move them to index.ts as i don't have a good name for what file to put them into :shrugs:
* updated the readme to use the defineXXX functions
== Behaviour without this fix ==
See #48881
== Behaviour with this fix ==
1. Metabase starts with an empty application database, and a config file
containing configuration for a connected database
3. By the time it reaches "INFO metabase.task :: Task scheduler started",
`qrtz_triggers` contains `metabase.task.sync-and-analyze.trigger.1`
(sample DB) and `metabase.task.sync-and-analyze.trigger.2` (DB from
config file)
Closes: https://github.com/metabase/metabase/issues/48881
Co-authored-by: Noah Moss <noah@metabase.com>
* Use type date for case expression when there are Date and DateTime args
* Use logic for getting case expr type from annotate middleware
* Add e2e test
* Add bigquery test
* Update test
* cljfmt and comment
* Adjust e2e test
clojure.core/pr{:message"Use metabase.util.log instead of clojure.core/pr"}
clojure.core/tap>{:message"Don't use tap> is source code"}
clojure.core/time{:message"Don't use time in source code"}
clojure.test.check/quick-check{:message"You probably meant to use clojure.test.check.clojure-test/defspec instead -- quick check will run the tests when it is evaluated rather than when we run the test suite."}
clojure.java.jdbc/get-connection{:message"Use sql-jdbc.execute/do-with-connection-with-options (for drivers) or t2/with-connection (for the app DB) instead of jdbc/get-connection"}
clojure.java.jdbc/with-db-connection{:message"Use sql-jdbc.execute/do-with-connection-with-options (for drivers) or t2/with-connection (for the app DB) instead of jdbc/with-db-connection"}
clojure.java.jdbc/with-db-metadata{:message"Use sql-jdbc.execute/do-with-connection-with-options + .getMetaData instead of jdbc/with-db-metadata"}
clojure.pprint/pprint{:message"Use metabase.util.log instead of clojure.pprint/pprint"}
clojure.string/lower-case{:message"Use metabase.util/lower-case-en instead of clojure.string/lower-case"}
clojure.string/upper-case{:message"Use metabase.util/upper-case-en instead of clojure.string/upper-case"}
compojure.core/context{:message"Use metabase.api.util.handlers/route-map-handler instead of compojure.core/context, it is much faster and supports OpenAPI spec generation"}
compojure.core/defroutes{:message"Use metabase.api.util.handlers/defroutes instead of compojure.core/routes, it supports OpenAPI spec generation."}
compojure.core/routes{:message"Use metabase.api.util.handlers/routes instead of compojure.core/routes, it supports OpenAPI spec generation."}
fipp.edn/pprint{:message"Use metabase.util.log instead of fipp.edn/pprint"}
honeysql.core/call{:message"Use hx/call instead because it is Honey SQL 2 friendly"}
honeysql.core/raw{:message"Use hx/raw instead because it is Honey SQL 2 friendly"}
java-time/with-clock{:message"Use mt/with-clock"}
java.util.UUID/randomUUID{:message"Use clojure.core/random-uuid instead of java.util.UUID/randomUUID"}
malli.core/explainer{:message"Use metabase.util.malli.registry/explainer instead of malli.core/explainer"}
malli.core/explain{:message"Use metabase.util.malli.registry/explain instead of malli.core/explain because it has built-in caching"}
malli.core/explainer{:message"Use metabase.util.malli.registry/explainer instead of malli.core/explainer because it has built-in caching"}
malli.core/validate{:message"Use metabase.util.malli.registry/validate instead of malli.core/validate because it has built-in caching"}
malli.core/validator{:message"Use metabase.util.malli.registry/validator instead of malli.core/validator because it has built-in caching"}
me.flowthing.pp/pprint{:message"Use metabase.util.log instead of me.flowthing.pp/pprint"}
medley.core/random-uuid{:message"Use clojure.core/random-uuid instead of medley.core/random-uuid"}
metabase.driver/database-supports?{:message"Use metabase.driver.util/supports? instead of metabase.driver/database-supports?"}
metabase.lib.equality/find-column-indexes-for-refs{:message"Use lib.equality/closest-matches-in-metadata or lib.equality/find-closest-matches-for-refs instead of lib.equality/find-column-indexes-for-refs"}
metabase.test/with-temp*{:message"Use mt/with-temp instead of mt/with-temp*"}
toucan2.tools.with-temp{:message"Use mt/with-temp instead of t2.with-temp/with-temp"}}
metabase.lib.equality/find-column-indexes-for-refs{:message"Use lib.equality/closest-matches-in-metadata or lib.equality/find-closest-matches-for-refs instead of lib.equality/find-column-indexes-for-refs"}}
:discouraged-namespace
{camel-snake-kebab.core{:message"CSK is not Turkish-safe, use the versions in metabase.util instead."}
cheshire.core{:message"Use metabase.util.json instead of cheshire.core directly."}
clojure.tools.logging{:message"Use metabase.util.log instead of clojure.tools.logging directly"}
grouper.core{:message"Use metabase.util.grouper instead of grouper.core"}
honeysql.core{:message"Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}
honeysql.format{:message"Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}
honeysql.helpers{:message"Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}
honeysql.types{:message"Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}
honeysql.util{:message"Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}
metabase.util.jvm{:message"All of metabase.util.jvm is re-exported from metabase.util; prefer that"}}
honeysql.util{:message"Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}}
:unresolved-var
{:exclude
[colorize.core]}
;; A "module" is any `metabase.x` namespace in `src`.
:metabase/ns-module-checker
{:level:warning
;; Map of module name => the "core" external public-facing API namespace(s). You have three options here:
;;
;; 1. Special sentinel value `:any` means means this module does not (yet) have external public-facing API
;; namespace(s). This is mostly a temporary placeholder until we go in and create module namespaces, which means
;; you should go create one.
;;
;; 2. A set of namespace symbols. All namespaces in other modules will only be allowed to use namespaces from
;; this set. Ideally this set should only have one namespace, but restricting it to a set of several is still
;; better than `:any`.
;;
;; 3. `nil` or not listed here -- we default to assuming there is one API namespace that matches the name of the
;; module itself, e.g. the module namespace for `metabase.setup` would be `metabase.setup`.
;;
;; `nil` or an empty set Otherwise this should be a set of namespace symbols.
;;
;; PRO TIP: Check out the `dev.deps-graph` namespace for helpful tools to see where a module is used externally.
:api-namespaces
{metabase.actions#{metabase.actions.core}
metabase.analytics#{metabase.analytics.prometheus
metabase.analytics.snowplow
metabase.analytics.stats
metabase.analytics.sdk}; TODO -- consolidate these into a real API namespace.
metabase.analyze#{metabase.analyze}
metabase.api#{metabase.api.common
metabase.api.dataset
metabase.api.permission-graph
metabase.api.routes}; TODO -- consolidate these into a real API namespace. I think the `*current-user*` type stuff might need to be moved into a separate module.
metabase.async#{metabase.async.streaming-response
metabase.async.streaming-response.thread-pool
metabase.async.util}; TODO -- consolidate these into a real API namespace.
metabase.audit#{metabase.audit}
metabase.bootstrap#{metabase.bootstrap}
metabase.cmd#{}; there are no namespaces here since you shouldn't be using this in any other module.
metabase.channel#{metabase.channel.core}
metabase.config#{metabase.config}
metabase.core#{metabase.core.initialization-status}; TODO -- only namespace used outside of EE, this probably belongs in `metabase.server` anyway since that's the only place it's used.
metabase.db#{metabase.db
metabase.db.metadata-queries; TODO this should probably be separate from metabase.db
metabase.db.query; TODO this is mostly util stuff like `metabase.db.query/query` that we don't even need anymore.
metabase.db.setup}; TODO these are only calling `metabase.db.setup/setup-db!` and there's a slightly different version in `metabase.db`
metabase.driver:any; TODO -- 19 namespaces!!!! CRY
metabase.email#{metabase.email
metabase.email.messages}; TODO -- consolidate these into a real API namespace.
metabase.legacy-mbql.util}; TODO -- consolidate these into a real API namespace.
metabase.lib:any; TODO -- :cry: 34 externally referenced namespaces, but maybe half of them are schema namespaces which technically don't need to be required.
metabase.logger#{metabase.logger}
metabase.metabot#{metabase.metabot}
metabase.models:any; TODO -- scream, 62 namespaces used elsewhere, but to be fair a lot of these don't *need* to be required.
:message(format"The name of this %s should end with `!` because it contains calls to non thread safe form `%s`. [:metabase/test-helpers-use-non-thread-safe-functions]"
:message(format"The name of this %s should end with `!` because it contains calls to non thread safe form `%s`. [:metabase/test-helpers-use-non-thread-safe-functions]"
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.