* update people nav
* type fix
* use modern upsell
* fix e2e test, cleanup
* wider pages
* update tests
* update unit tests
* update titles and nav
* limit description width
* little updates
* Localize the trash name on memoized collection
* just defer translation on the `trash-collection`
When we're getting it explicitly rather than for all collections - for
the sake of speed here, since this was causing some issues with
serialization.
* parallelize
* docstring
* VIZ-162 Updates percent calculation for stacked bar charts’ tooltips for bars with a mix of positive and negative values. Positive values are a percentage of the positive total and negative values are a percentage of the negative total.
* VIZ-162 Updates e2e tests to verify new stacked bar chart behavior
shorten filename to not overflow byte limit in some filesystems
in particular ext4 limits file name by 255 *bytes*, not chars, which is doable by using 3-byte chars
fixes#59233
* set embedding-homepage visible as soon as we can
* cleanup test
* reorder because it makes more sense
* Update frontend/src/metabase/setup/components/EmbeddingSetup/steps/UserCreationStep.tsx
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Fix can't download static SQL question\
when a parameter only expects a single value
* Ignore `?locale` search param in result downloads
* Finish the comment
* Fix public action tests failing
* Address review: Call side-effect explicitly
* Preserve same behavior
* Flip boolean to make it easier to read
* Improve boolean condition
* revert the use of `isEmbeddingSdk` env until #59602 is fixed
* Add missing condition to retain the same behavior
* Fix more failing tests duh!
* Fix should sync logic for real this time.
* Update upgrading-metabase.md
Adding a note about upgrading between major versions
* Update upgrading-metabase.md
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Add embedding setup flow with multi-step wizard. Add EmbeddingSetup component with welcome, data connection, processing, and final steps. Create setupEmbeddingSettings action for configuring embedding and JWT. Add CodeSnippet component for displaying code examples. Set up proper routing for the embedding setup flow.
* Update embedding setup flow to actually create the database and handle errors. Update DataConnectionStep to display error messages.
* Fix linter errors in EmbeddingSetup component and setupEmbeddingSettings action
* Setup embedding flow: Remove SchemaSelectionStep, fix imports and types
* Fix linter errors and finalize embedding setup flow with navigation and original content restored
* Add EmbeddingSetupWrapper to provide context for embedding setup flow
* Fix table fetching in TableSelectionStep to use metadata endpoint
* Implement model and dashboard creation in ProcessingStep using /api/card endpoint
* Fix useEffect and useCallback dependencies and import order in TableSelectionStep
* working flow
* small tweaks
* sidebar with steps
* update snippet to use iframe embed
* better "Need assistance" positioning
* fix and refactor database and tables steps
* style final step
* bottom part of /final
* done step and idk i'm tired
* remove chevron and add syntax highlight as per feedback on the loom
* fix user not being loaded
* welcome step and final step
* remove dead code
* add query builder
* more stuff
* add comment about keepMounted
* set the proper settings during setup, fixes EMB-462 and EMB-461
* hide DatabaseStatus, fixes EMB-458
* "do later" -> link to /, closes EMB-456
* attempt at setting a max width, maybe closes EMB-454
* fix icons
* fix scrollbar on tall pages
* to refactor: wip link between setup and onboarding
* fix jumpy layout while moving across pages
* fix DatabaseStatus showing up after landing to the homepage
* Revert "to refactor: wip link between setup and onboarding"
This reverts commit b4c19da37d.
* wip user step
* revert changes to database status, we don't need those changes now that we're not in the routes that use the App component
* prevent going to setup/embedding if setup is already done
* table search
* restore original position of metabot to keep diff cleaner
* category + icon on search input
* restore routes structures
* add widht and height, fixes EMB-480
* remove "Need assistance" link, closes EMB-479
* change content in DoneStep, closes EMB-478
* removed /embed from the url
* empty set for search + layout adjustements, closes EMB-481
* removing things i commited by mistake
* locale selector, closes EMB-466
* cleanup unused code
* fix loading not being centered in table selection loading
* fix processingStatus i18n issue
* Remove undefined CSS custom property
* Revert "fix loading not being centered in table selection loading"
This reverts commit 8fda19338a.
* re-do the centering
* close EMB-473, refactor to be only one page
* move CodeSnippet in the file where it's used, it's not a generic
component for now
* Add a basic embedding onboarding E2E test (#59323)
* Add a basic E2E test
* Add missing import
* Finish assertions
* Wait for DB sync
* Hide DB sync status, so Cypress could click a button
* Fix test on CI
* Fix E2E after upstream change
* Some adjustment to make CI passes
* Wait for loader to disappear to avoid failing CI
* Simulate being on cloud
* Addresses review feedback
* fix sidebar steps, titles and visibility
* fix another wrong string
* Update frontend/src/metabase/setup/components/EmbeddingSetup/EmbeddingSetupSidebar.tsx
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* address pr feedbback
* fix error ui in table selection
* add useForceLocaleRefresh to table step
* update label on test too
* more label updates
* Properly refetch database API
* Add DB sync retries assertion
* fix the retry of the database sync
---------
Co-authored-by: Kyle Doherty <5248953+kdoh@users.noreply.github.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* add regression tests
* Add issue references
* make double comment test driver agnostic
* make tests more agnostic
* format
* make another test agnostic
* add prometheus metrics
* add driver query metrics test
* test against failed count
* move clickhouse native tests back, remove parallel from metrics query test
* fix kondo
* add todo attribution
* disable 0.8.4 broken tests, re-enable 0.8.6 broken tests
* Manually close popover on escape
* Add repro for metabase#55673
* Revert "Manually close popover on escape"
This reverts commit cce85f0882.
* Trap focus in click actions popover
* Remove unneeded closeOnEscape
* Add missing :ident to :model/products-and-reviews mock card
* Add tests for implicitly-joinable visible-columns
* Add more context and cross-reference the comment about implicit-self-join-test
* Put keyword first when looking up entries in lib.tu/mock-cards
* process-query supports temporal-unit template tags in native queries (#57295)
The query processor understands temporal-unit template tags and can parse and compile
`{{mb.time_grouping('param', 'table.column')}}` appropriately. This does not include lib or frontend support and this
feature is not yet usable outside of manually making a query and passing it to qp/process-query.
* Update metabase.lib.native to understand mb.time_grouping parameters (#57506)
This adds lib support for time_grouping parameters. To do that, it pulls out the parsing logic from the driver code and pulls it into lib, where both the driver and lib can use it. The frontend still doesn't support these parameters, so while you can technically type in and run a query with a time_grouping parameter, the frontend behaves oddly if you try.
* implement frontend for time grouping in native queries (#57849)
* Update metabase.lib.native to understand mb.time_grouping parameters
* move parse-param tests
* keep picky behavior of old tagfinding
* fix ns alias
* fix ns declaration for cljs
* add lib tests for finding function tags
* remove resurrected file
* fix rebased code
* fix linting
* update doc string
* fix modules
* implement frontend for time grouping in native queries
* fix modules
* fix validation bug
* update sidebar
* fix typecheck
* fix styles
* use correct icon
---------
Co-authored-by: William Zimrin <william@metabase.com>
* implement dashboard mapping to time grouping parameters (#57931)
* Update metabase.lib.native to understand mb.time_grouping parameters
* move parse-param tests
* keep picky behavior of old tagfinding
* fix ns alias
* fix ns declaration for cljs
* add lib tests for finding function tags
* remove resurrected file
* fix rebased code
* fix linting
* update doc string
* fix modules
* implement frontend for time grouping in native queries
* fix modules
* fix validation bug
* update sidebar
* fix typecheck
* fix styles
* implement dashboard mapping to time grouping parameters
* filter columns properly
* fixup
* fixup
* temporal-unit tags use variable targets
* fix backend tests
* use variable
* cleanup
* add handling for required params
* update copy and fix required toggle usage
---------
Co-authored-by: William Zimrin <william@metabase.com>
* tests for native query time grouping (#58091)
* Update metabase.lib.native to understand mb.time_grouping parameters
* move parse-param tests
* keep picky behavior of old tagfinding
* fix ns alias
* fix ns declaration for cljs
* add lib tests for finding function tags
* remove resurrected file
* fix rebased code
* fix linting
* update doc string
* fix modules
* implement frontend for time grouping in native queries
* fix modules
* fix validation bug
* update sidebar
* fix typecheck
* fix styles
* implement dashboard mapping to time grouping parameters
* filter columns properly
* fixup
* fixup
* temporal-unit tags use variable targets
* fix backend tests
* use variable
* cleanup
* add handling for required params
* update copy and fix required toggle usage
* reset default value if time grouping option is excluded from the list of available options
* add tests
* drop not planned test
---------
Co-authored-by: William Zimrin <william@metabase.com>
* add tests for mapping of native query to temporal unit parameter (#58161)
* rework tag editor sidebar and related files to typescript (#58283)
* revert default required logic (#58568)
* revert default required logic
* fix test
* add native query validation of time grouping syntax (#58426)
* add basic fe validation for native queries
* fixup
* add cljs function for validating native queries
* make validate function take a full query and work in cljs
* add backend/cljc tests for validate-native-query
* improved error messages for validate-native-query tests
* use a new dianostics method from lib
* fix clj linting
* update sidebar styling
* fix initial state
* adjust error styling
* fix tests and drop not valid anymore code
* fix linting
* favor first template tag found and update tags when type changes
* validate that temporal unit function parameters aren't empty
* add e2e tests
* convert day/week/month/quarter/year groupings to dates
* fix empty param
* only convert days to dates
* finally actually make date conversion work correctly?
* address review
* adjust tests to the latest changes
---------
Co-authored-by: William Zimrin <william@metabase.com>
* rename test
* update native query time grouping tests to be more comprehensive (#58814)
* Fix time grouping tests (#58878)
* validate time unit parameters and fix bug (#58993)
* Fix and test nil values for native time groupings (#59062)
* add e2e tests for optional usage of time grouping (#59045)
* add e2e tests for optional usage of time grouping
* cleanup
* fixup
* update test
* fix class reference
---------
Co-authored-by: William <william@metabase.com>
* fix: improve effective children performance
on the /items endpoint by reusing the permission CTE
Co-Authored-by: John Swanson <john.swanson@metabase.com>
* fix: remove n+1 hydration call for card children
* fix: remove case/whens in sort statements
* fix: scope search results for modal to collections
Co-Authored-By: Nathan Voxland <nathan@metabase.com>
* fix: function to convert to collection items
* fixup! PR suggestion
Co-Authored-By: Ryan Laurie <ryan@metabase.com>
* fixup! add row-lock fixture for mariadb flake
* add debug for failing test
* fixup! make sure api/bit->boolean is called first
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: Ryan Laurie <ryan@metabase.com>
* Support type deduction for `case` and `coalesce`
* Revert accidental changes
* Fix CLJS tests
* More misc QP/Lib cleanup
* Test fixes 🔧
* Test fixes 🔧
* Add `:type/HasDate` and `:type/HasTime` to the type hierarchy
* Additional fix 🔧
* Improved type calculation logic
* Fix QUE-1358
* Test fixes 🔧
* Revert "Address review comments from #58408 (#58785)"
This reverts commit ed6e5591e3.
* Revert "Introduce the option to add data from the sidebar via "Add data" modal (#58408)"
This reverts commit de780f53cd.
* Reflect changes from #58405
Fixes#59273.
Fixes#48306.
Swapping two clauses can change their names, eg. two sum aggregations
`sum` and `sum_2`. Since refs in later stages are based on those names,
the refs need updating to track the same column, rather than change
targets.
`lib.remove-replace` already has logic for this when replacing a clause,
it was readily reused here. I added tests for swapping aggregations and
multiple breakouts of the same column.
Note that viz settings (also based on the names) are not updated as part
of this PR. There are separate bugs open for that issue both when
removing a clause and reordering them.
* a bit lazier table->unfold-json-fields
* remove need for dynamic var
* more optimised
* don't leak json parsers
* code style
* better perf
* short circuit earlier in case of a lot of nested fields
* add to changelog
* rename
* Add icon color support to the banner componet and story
* Add token missing banner rules
* Add token missing banner rules
* Add useLicenseTokenMissingBanner tests
* Dismiss license banner in e2e tests by default
* Improve flag naming in LicenseTokenMissingBanner
* Add banner story - non-closable
* Use regular double quotes for simple string
* Remove duplicated CSS property
* Improve docstring for license token missing banner setting
* Fix wrong property name in the test
* Remove empty align property
* Use existing setupUpdateSettingEndpoint mock
* Extract constants out of the function
* Use ready mock helpers in tests
* Refactor mock setup function to conform to existing standards
* Share the dismissal date between all admins
* Use admin settings APIs to interact with dissmisal state
* Improve banner dismissal test
* Use regular settings API to access license token banner dismissal fetch
* fix: set-table/db-permissions! in single txn
Update set-table-permissions! and set-database-permissions! to build
lists of all necessary changes based on the requested permissions
updates and then execute those changes in a single transaction to avoid
possible deadlocks in the recursive calls.
* fixup! incorrect doc string
We can have tables in migrated instances that have two copies of the
audit db tables one with a 'public' schema and one with a 'PUBLIC'
schema. This handles that case.
* FE wip
* more FE progress
* last big pass at UI
* url page param sync, show model name, move new code to new file
* remove pagination in url and add link to new metabot convo
* unit testing
* finish tests
* move prompt result fetching outside of metabot agent hook and into chat sidebar
* fix constant reference
* fix
* Add metabot prompts and wire up AI service
* Return prompt suggestions for cards visible by the user
* pr feedback
* Test metabot-scope
* Add BE tests
* Fix tests and kondo issues
* Fix tests and kondo issues
* Fix tests more
* Fix serialization
* Fix serdes more
* Use MySQL compatible delete syntax
* have FE use new regenerate endpoint instead of delete
* change mocked status code
* fix flaking unit test and fix failing MetabotAdminPage spec from wrong mocked pagination value
* Add prompt regeneration endpoint
* Document prompt regeneration endpoint
* fix test flake
* doh... fix the right test this time
* Use the collection configured int the app-db in answer-sources
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
In postgres, json fields are hidden by default, and we just show their children instead. However, mongo fields and
bigquery struct fields work differently -- we default to just showing the parent, even though the ui claims that all
fields are selected. This makes mongo and bigquery work like postgres.
In addition, this fixes a presumably-longstanding bug where mongo had incorrect column names when selecting both a
parent its child. Basically, you can't ask mongo for both a parent and its child. The returned data is just json, so
asking for the parent field includes all of its children. Instead, mongo just errors when you try to make a request
like that. As a workaround, the mongo driver was just requesting and returning the child fields. This meant that it
was returning a different number of fields than expected, which confused annotate.
Now, the mongo query processor only selects for parent fields, because as mentioned, parent fields include their
children. However, it also keeps track of all the fields we asked for in :projections, so when we get data back, we
know to pull all of those fields out of the returned data.
* fix: check for row existence in parameter upsert
Do an exists check instead of relying on the result of `update!` being
zero. `update!` can return zero either when a row does not exist or when
there are no applicable changes in the request.
* fixup! use medley/mapply instead
* test logs as artifact
* better test logs
* debugging improvements
* fix `with-context` query to work with non-literals
```
(with-context {:foo "bar"}
...)
```
worked,
```
(with-context (merge {:foo "bar"} {:bin "baz"})
...)
```
did not, now it does!
* add compiled query args to context
* kondo
* compact logs
* use an `:after` method
* use defonce to avoid repeated `test-var` overwrite
This reverts commit 9dcbcba3f6.
snowflake is now fixed from being flaky and is now required again
the fix was in checking old dataset names to prune. Shared databases
like cloud databases don't get destroyed when docker terminates, so we
have to remove old databases. The way we did it in the past would error
if more than 10,000 rows were returned. The fix is the following from
https://github.com/metabase/metabase/pull/58818 :
```clojure
(defn- old-dataset-names
"Return a collection of all dataset names that are old -- prefixed with a date two days ago or older?"
[]
;; modified the query from the snowflake driver:
;; fa58e34968/src/main/java/net/snowflake/client/jdbc/SnowflakeDatabaseMetaData.java (L1644C21-L1644C90)
;; We run into issues where this returns more than 10,000 items and we get an error: "The result set size exceeded
;; the max number of rows(10000) supported for SHOW statements. Use LIMIT option to limit result set"
(let [query "show /* JDBC:DatabaseMetaData.getCatalogs() with limit */ databases in account limit 10000"]
(into []
(comp (map :name)
(filter sql.tu.unique-prefix/old-dataset-name?))
(jdbc/reducible-query (no-db-connection-spec) [query] {:raw?
true}))))
```
Some mysql -> postgresql migrated users still had adjust to source
failing because they had two tables with downcased names but one with a
"public" schema column and the other with a null schema column. This
handles that case by only selecting tables if there is not already a
table with a "public" schema column.
* VIZ-192 Updates goal tooltip to use same formatting as y-axis (but ignores compact/auto formatting to ensure the goal value is represented accurately)
* VIZ-192 Adds support for formatting percentage-based goals (e.g. bar charts with "Stack - 100%" selected)
* VIZ-192 Updates getGoalLineHoverData to accept a formatGoal param rather than a chartModel per PR feedback
* VIZ-192 Adds e2e tests for goal value formatting
* VIZ-181 Adds boundaryGap to bar chart y-axis if it has negative values to ensure bar labels don’t overlap axis labels
* VIZ-181 Minor performance consideration; `ensureRoomForLabels` only checks `dataSeriesOptions` since "bar" series won't show up in other seriesOptions
* VIZ-181 Tweaks ensureRoomForLabels calculation
* VIZ-181 Adds unit tests for ensureRoomForLabels
* Don't open the database picker when there is only one database
* Allow null for database
* Add repro for metabase#57644
* Remove duplicate it
* Use == to check for undefined
* Fix test description
* Update e2e/test/scenarios/native/native-reproductions.cy.spec.ts
Wording
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add check for correct clauses for expression mode
* Add support helper to expressions/mode
* Do not show help text for unsupported functions
* Add repro for metabase#58230
* Add e2e test for not showing helptext for unsopported functions
* Allow value operator
* Pass expressionMode where needed
* Tranlate mode name in error message
* Avoid throwing in checkExpressionModeSupportsClause
* Add repro for metabase#57388
* Fix update button helper
* Set to default when value is empty
* Update repro
* Make unit test more robust
* Do not pass default value to default value picker
* Fix unit test
* fix info formatting
* ensure exception is not swallowed
* Ensure we use a different connection per [[have-select-privilege?]] call.
This is to guard against databases (like Oracle) closing a connection when attempting
to execute a priviledged select, causing subsequent calls to fail.
* adjust usage of snowflake driver
* adjust usage of postgres driver
* update tests
* more efficient: move filtering within db-tables
* add test
* add changes to driver changelog
* route through driver rather than jdbc, so athena doesn't fail
* ensure we're testing only the drivers using default jdbc impl
* expose wrapper, rename
* PRG-53 add community translations notice
Enhance localization features by adding community translation links and descriptions across user profile, localization settings, and setup components.
* PRG-53 Refactor localization components to use CommunityLocalizationNotice
Replaced direct translation links and descriptions in UserProfileForm, LocalizationSettingsPage, and LanguageStep with the new CommunityLocalizationNotice component for improved code reuse and consistency.
Signed-off-by: Jakub Chodorowicz <jakub@chodorowicz.com>
* PRG-53 Use external link in the notice and add tests
* PRG-53 Fix e2e i18n test
* PRG-53 Prefer stack over brs, add a11y label, refactor export
* PRG-53 Improve cy selector to a semantic one
* PRG-53 Use hardcoded Metabase name in the setup flow
* PRG-53 use color name in color property
* PRG-53 use show-metabase-link setting
* PRG-53 Use Metabase original name for admin screens
* PRG-53 Add storybook stories
* PRG-53 Don't mention custom name in the translation when white labelling
* PRG-53 use static for crowdin link
* PRG-53 remove CommunityLocalizationNotice story
* PRG-53 Fix e2e i18n test
---------
Signed-off-by: Jakub Chodorowicz <jakub@chodorowicz.com>
* fix: lookup cloud migration part keys
Mapping over multipart-urls map entries wasn't ordered or deterministic, but it happened to be for less than 8 elements.
Changed to a lookup on advice from @dpsutton.
Fix https://metaboat.slack.com/archives/C010ZSXQY87/p1748611922362219
* test: update cloud migration settings test vars
* fix: add early multipart errors
* Update Static and Interactive Embedding Settings
* update sdk settings
* remove useMergeSetting
* remove more dead code
* remove more unnecessary stuff
* update unit tests
* more test updates
* update e2e tests
* Move embedding unit tests to local components
* address review comments
* Fix a flaky test for max-refingerprint-field-count
The code here goes through tables and refingerprints them (with all
their fields). After each table it checks the
`max-refingerprint-field-count` and stops if we've hit it.
The test gave some wiggle-room - we'd allow 10 extra fields. But in a
fresh run of my tests, the biggest table has... 13 fields. So if the
last table we hit before hitting the limit was *that* table, we exceed
our wiggle-room and the test fails.
Instead, the test now actually looks to see what's the biggest table we
have (in terms of number of fields), and asserts that the number of
fields refingerprinted is, at the most, the max plus that number of
fields.
* fix two more flaky tests
one is clearly a symptom of another test being super naughty but I can't
figure out which one. It's marked `parallel` and doing
```
(mt/with-temp [:model/Card {id :id} ...]
(do-something-with (t2/select-one :model/Card id)))
```
which is failing because `(t2/select-one :model/Card id)` is returning
nil. We can just pass the card that was actually returned by
`mt/with-temp` here and that should be more resilient to horrible tests
being horrible.
the other is checking that we're caching something by checking the call
count to toucan on the first and subsequent call to a function. But it
asserts that the first call count is exactly 6, which isn't relevant.
If I start a fresh REPL I get more than 6 the first time. It doesn't
matter! Just assert that we don't get MORE than the first time on the
second run!
* fix flaky tests freal
* fix: use t2/query to skip card callbacks
* fix: retry on failed pk insert
* fix: log when we have the cluster lock
* fix: try catching all sqlexceptions
* fix: add a fixture to pre-create the cluster_lock
* fixup! remove with-connection from update-used-cards
---------
Co-authored-by: Edward Paget <ed.paget@gmail.com>
* Add testing context around before-and-after gen tests
To ensure you can see the before and after queries as well as the steps that produced them when a test fails.
* Fix before-and-after :join in generative tests
Pass stage-number to check for aggregations and breakouts on the stage where the join was added rather than the final
stage.
* Add an option to force multi-staged data picker
* Move type to a proper location
* Add `question` to `entity_types`
* Add ability to select saved questions back to multi-staged data picker
* Auto select Questions or Models when there is only a single option
* Prevent accidental types
* Remove unreachable code
* Default the join step after selecting a question to the question step
* Fix type errors
* Improve typing + fix unit tests
* Remove unnecessary await
* Add unit tests
* Update the default simple data picker value
* Fix the picker name typo
* Ignore `question` in Simple data picker
* Add E2E tests
* Move a constant out of a type file
* Fix types
* Address review. Make code more sane
* Address review. Improve variable name clarity.
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* add tests for SEM-3
* let fingerprinters respect ->Number coercers
* explicitely assert for supported coercions
* add test on fingerprinted supported coercions
This was failing because we call
```
(session/create-session! :sso user (request/device-info request))
```
with the result of this function, and `session/create-session!` requires
a user with at least `:id` and `:last_login`. We didn't have a
`:last_login` in this case so `create-session!` was barfing.
we didn't have any tests that actually logged in as an existing user.
* update sync error handling to take into account `do-not-retry-exceptions?` as per its docstring
* add test
* introduce `with-returning-throwable`, use consistently
* update test
* add remember me to google sign in
* UI adjustments
* assert that remember me checkbox is rendered with google button
* backend test that cooke expiration is set
* rollback model changes in BE test
* create user model and use with-model-cleanup
---------
Co-authored-by: Edward Paget <ed.paget@gmail.com>
* Define the number of engines in a constant
* Improve callback naming
* Use t-shirt sizing
* Remove superfluous e2e test
We're already covering this scenario using Jest.
* Update prop naming
* Simplify CSS
* Change tab as well when url changes
* Do not get confused about editor height
* Add repro for metabase#55486
* Scope loading query
* Test forward button too
* Limit query
* Remove loading check altogether
* Wait for queries to complete after saving changes
* Wait for `dc.series` queries in `visitDashboard`
* Wait until visualizer loads after it's open
* Fix flaky tests
* Remove flaky tag
* Revert "Wait for `dc.series` queries in `visitDashboard`"
This reverts commit 3f6a606442.
Error we were getting:
```
1. Unhandled net.snowflake.client.jdbc.SnowflakeSQLException
The result set size exceeded the max number of rows(10000)
supported for SHOW statements. Use LIMIT option to limit result set
to a smaller number.
SnowflakeUtil.java: 170 net.snowflake.client.jdbc.SnowflakeUtil/checkErrorAndThrowExceptionSub
SnowflakeUtil.java: 102 net.snowflake.client.jdbc.SnowflakeUtil/checkErrorAndThrowException
StmtUtil.java: 488 net.snowflake.client.core.StmtUtil/pollForOutput
StmtUtil.java: 394 net.snowflake.client.core.StmtUtil/execute
SFStatement.java: 502 net.snowflake.client.core.SFStatement/executeHelper
SFStatement.java: 211 net.snowflake.client.core.SFStatement/executeQueryInternal
SFStatement.java: 145 net.snowflake.client.core.SFStatement/executeQuery
SFStatement.java: 779 net.snowflake.client.core.SFStatement/execute
SFStatement.java: 697 net.snowflake.client.core.SFStatement/execute
SnowflakeStatementV1.java: 293 net.snowflake.client.jdbc.SnowflakeStatementV1/executeQueryInternal
SnowflakeStatementV1.java: 149 net.snowflake.client.jdbc.SnowflakeStatementV1/executeQuery
SnowflakeDatabaseMetaData.java: 1648 net.snowflake.client.jdbc.SnowflakeDatabaseMetaData/getCatalogs
REPL: 107 metabase.test.data.snowflake/old-dataset-names/fn
```
We do this in `old-dataset-names` by just calling `(.getCatalogs
metadata)` which runs
fa58e34968/src/main/java/net/snowflake/client/jdbc/SnowflakeDatabaseMetaData.java (L1644C21-L1644C90)
or
```sql
String showDB = "show /* JDBC:DatabaseMetaData.getCatalogs() */ databases in account";
```
I've modified that to have a limit and then run that
`jdbc/reducible-query` to get the names
* Fix zoom drill for clicks on a joined PK column
* Add lib.drill-thru.common.primary-key? and foreign-key? helpers
* Fix zoom and quick-filter drill for clicks on a joined PK column
* Remove unused imports
* Move source fields check into lib.drill-thru.common and revert changes in lib.metadata.calculation
* Remove trailing whitespace from docstring
* Lookup primary key in visible-columns ignoring joins rather than source-table fields
* Fix docstring
* Fixing mapped user facing strings for relative date params in subscriptions and alerts
* Adding frontend fix + spec
* Changing 'is' to '->' in UI
* pr feedback
* e2e test fix
* use badges in oss sidebar; update pulse card rendering; test fixes
* test fix
* test fix
* pr feedback
Currently, if a user is set to `is_active: false`, we still allow them
to log in via JWT or SAML. When they do, we also don't update the value
of `is_active`, which could potentially lead to inaccurate user counts.
After this change, the behavior when discovering an existing user upon
login is:
- if `is_active` is `false` and JWT/SAML user provisioning is on, we'll
log the user in and set `is_active` to `true`.
- if `is_active` is `false` and JWT/SAML user provisioning is off, we'll
give the user an error and not change the existing user.
- if `is_active` is `true`, we'll log the user in like normal.
* Remove binning from pinned items
* Add reproduction for metabase#57697
* Revert "Remove binning from pinned items"
This reverts commit aa2d4abbb4472b835dee901ce7ac36c4bc43cd5e.
* Remove binning from column when calculating displayname
* Add models; liquibase migration; ...
and other model incantations
* Add fk constraint after indexes
* Update model tests for column names
* serdes fixes
* Replace remaining missing metabot_model_entity_id
* Update generated baselines
* fix serialization tests
* initial implementation
* make the api match
* wip
* align types with API
* improved routing
* empty state
* cleanup and tests
* pivot to collection picking
* no personal collection
* update tests
* remove defunct table changes
* fix conflicts
* add copy for embedded metabot
* linter appeasement program
* linter appeasement program
* linter appeasement program
* update api
* no thank you migration
* un-un-un revert some stuff
* address review comments
---------
Co-authored-by: Edward Paget <ed.paget@gmail.com>
* Add lib.util/find-clause-by-uuid
* Lookup clause by uuid to generate the :long display-name for a :metadata/column
* Fix display-name expectations in targets.unit.spec.ts
* Fix display-name expectations in click-behavior.cy.spec.js
* Fix display-name expectations in filter-sources.cy.spec.js
* Review feedback: ensure last-stage? throw when given an invalid stage-number
* Review feedback: use lib.tu/venues-query
* Review feedback: explict true? and false? checks for first-stage? and last-stage?
* WIP
* Adjust process-multiple-queries to also support single pivot queries
* cleanup
* Seemingly working backend support for only running necessary subqueries
* pivot unit test adjustments
* Test fixes
* Rerun pivot query when settings change
* wip fix for pivots as email attachments in subscriptions
* alternate single query qp support; unit test fix
* fix ts
* fix
* fix frontend spec
* pr feedback
* adding api test coverage for pivots with totals disabled
* fix cljkondo
* typo
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* keep backend pivot exports uncollapsed
* fix kondo errors
* update tests - only run collapsing tests on cljs, add a new test for clj that rows are not collapsed
* fix more kondo warnings
* cleanup
* feat: api and models for managing metabot entities
* Add stubs
* Add models; liquibase migration; ...
and other model incantations
* tests for api
* Update h2 copy
* rename model to dataset
* Add fk constraint after indexes
* Wrap tests with with premium feature
* Add seed migration; drop unused index
* Update API to wrap in items instead of data
* Update model tests for column names
* Pass remaining serdes tests
* Reorder migrations
* serdes fixes
* correct api to remove metabot_model_entity_id
Handle root collection
* fix mistake in merge
* Replace remaining missing metabot_model_entity_id
* Update generated baselines
* fix rollback sql syntax
* update api to support collections
* update serdes for collections in metabot entities
* fix precondition for loading metabots
* fix cleanup metabot table before list endpoint test
* fix whitespace
* fix serialization tests
* fixup! change migration version to 56
* fixup! remove remaining no-metabot tags
* fixup! super nit
* fixup! move api into new namespace
make /metabot API routes singular
* fixup! better combine routes
* cljs: Make currency definitions more compact
Useful for reducing CLJS bundle size.
* cljs: Use unquoted functions in metabase.lib.drill-thru
* cljs: Make var re-exporting leaner in CLJS
* prevent Metabot from making a request for suggested prompts if no user is logged in to prevent auth redirect
* this fix is better
* add code comment
* reference PR in code comment
* revert unneeded change
* Fix sharing e2e test
* Simplify `shouldSplitVisualizerSeries` check
* Handle empty/erroring datasets
* Use card names for series names
* Port e2e tests
* Don't use entity IDs
* Unskip test
* Include metrics and models in data importer search
* Return result_metadata for metrics and datasets
* Fix drillthrough test
* Update comment
* Bring back removed metric tests
* Remove reundant test
* include result_metadata in recents api request
* Fix e2e test
* Fix typo
Co-authored-by: Sébastien <sebastien@metabase.com>
* Fix e2e test
* fix be test
* Fix e2e tests
* Update src/metabase/activity_feed/models/recent_views.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
---------
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Sébastien <sebastien@metabase.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Mega App DB memory usage improvements
* Add a test
* Minor comment formatting
* Fix test name
* This test can be ^:parallel
* Actually no it can not
* Fix test name
* Don't create mega queries when deleting databases (#58491)
* Improved fix
* Don't require new migrations to have preconditions (this is bad)
* Remove test
* Revert migrations
* Manually set Field.parent_id = NULL for all Fields in a Database when deleting a Database
* Tweak query for MySQL
* Yet another fix 🔧
* `IS NULL` => `IS NOT NULL`
* Give up on trying to find a nice fix
* MySQL fix
* MySQL fix (for real)
* Fix SDK cross version tests getting version number from other prerelease labels
* Update code from #58638
* Remove unnecessary console.log
* Simplify code even moarrrrrrr
* fix: set download-results to no on table block
When a view-data permission for a table is set to blocked set the
create-queries and download-results permissions to no. Also includes a
migration script read to backport to v52 to update databases to be
consistent with this change
* fixup! remove precondition
* fixup! join in update to work on mysql
* fixup! separate migrations on mysql/postgres
* fixup! test order
* Setup sdk-based embedding iframe route (#56915)
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* restore routes.clj to fix merge conflicts
* Tests for new iframe embedding's script tag and route (#57269)
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* remove the safe-to-remove warning on rspack css ordering
* Rename embed.v1.js to embed.js for new iframe embedding (#58415)
* Revert "remove the safe-to-remove warning on rspack css ordering"
This reverts commit 4feae48604.
* Remove redundant sdk-ee-plugins import from the SDK in new iframe embedding (#58416)
* Ignore CSS import order warnings from new iframe embedding (#58417)
* Follow-up validation improvements for new iframe embedding (#58516)
* Explicitly disallow using API keys in production for new iframe embedding (#58520)
* fix wrong merge conflict - do not destructure H.cy in component tests
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Fix static question embed download when parameters have multiple values
* Simplify type
* Don't wanna deal with outdated query builder types 🤷
* Use a proper value when asserting
Before, nest-query relied on the preprocessor (specifically, the add-implicit-clauses middleware) to add all possible
fields to the newly created inner query (and then removes unnecessary fields later). However, add-implicit-fields
ignores fields with a parent id. The main user of parent-id is mongo, and mongo doesn't use nest-query, so this
usually isn't an issue. However, bigquery also uses parent-id, and bigquery is a sql driver that uses nest-query. As
a result, nest-query wasn't adding bigquery struct member fields to the inner query, which caused some queries to
break. Now, nest-query explicitly finds all non-sensitive/retired fields (including fields with parent ids) and
ensures that they are in the inner query.
* used `entityTypeFilter` for new embedding data pickers
* Rename `entityTypeFilter` to be consistent with `entity_types`
* Migrate `entity_types` to a new reducer
* Add E2E test
* remove unused prop
* Fix interactive embedding's `entity_types`
* Fix unit tests
* Avoid using quadratic search in FkTargetPicker
* Reduce further parsing and stringification
* Avoid calling JSON.parse in FkTargetPicker
* Assert fieldId
* Rename getters to be more clear
* use user locale setting if available for lang prop in <html>
* add a test
* fix my broken test
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* use mantine menu and reorganize options
* remove comments
* fix unit tests
* e2e test fixes, replace "a" with ForwardRefLink to fix sidebar not getting closed
* fix external links
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* Add prompt to the query
* Add prompt to the query
* Add prompt to the query
* Add prompt to the query
* Add prompt to the query
* Add prompt to the query
* Add prompt to the query
* Tests
* Tests
* Tests
* Tests
* Tests
* fix: audit db not working after import form h2
Removes conditions from the `adjust-audit-db-to-source!` function so we
make sure we always restoring the audit db tables to their original form
regardless of what type of app-db is used.
* fixup! test setting up db triggers
* fixup! cljfmt
run breaking changes tests on multiple runners
TO BE REVERTED BEFORE MERGING
force production fe build
build-release as setting node_env fucks up everything
exclude @OSS tests
Revert "TO BE REVERTED BEFORE MERGING"
This reverts commit c907a2f492.
* Add additional check on rendering native preview panel
* adding e2e test
* adjusting e2e test
* do not show native panel when no data is selected
* move e2e test to instance analytics suite
* simplify embedding data picker code
* Show the data picker step based on starting query source
* Set starting model collection ID when joining a model
* Improve initial collection loading UX
* Fix failed unit tests and types
* Fix failed E2E tests
* remove unused var
* Fix sometimes the starting collection is incorrect
* Add an E2E test
* Add a comment to clarify
* Clarify a new prop
* VIZ-329 Updates ChartSettingInput and ChartSettingInputNumeric to apply changes when the user pauses typing in addition to on blur
* VIZ-329 Removes now-redundant blur handler in ChartSettingInputNumeric and updates unit tests
* Revert "fix serialization for visualizer entities (#57467)"
This reverts commit d56e791077.
* drop columns from test data
* fix types, tests
* Revert "[VISUALIZER] Fixing dashboard duplication for visualizer cards (#58106)"
This reverts commit 6cf7606754.
* keep preferred true?
* serdes transformation support for visualizer settings
* Update entity ids in visualizer dashcards on DB read to be card ids
* unit test for the serdes transformations
* cljkondo
* fix on-read to only run if viz settings exists
---------
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* Empty
* Add schema for datetime()
* Add datetime() function; works on iso strings
* Implement datetime() for sql to test all drivers
* Remove Z from datetime
* Change the format of the datetime string in tests
* Be more lenient on types
* have multiple formats for the test
* Implement datetime() for mongo
* Use ISO string in test
* Datetime for string with no mode
* Add changelog entry for :expressions/datetime
* Add docs for datetime
* Add e2e tests for datetime
* Update e2e test and datetime text
* Fix :expressions/datetime feature
* Remove reference to iso
* Remove unused schema
* Fix frontend test
* Use multiple arities for datetime()
* Replace T with ' ' in databases that don't like T in datetime timestamps
* Try removing visibility requirement in test
* Fix brittle test
* Test dates with space and with T
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* fix: explicitly mark when fields are gtapped
* add debug
* fixup! tests setup to set advanced feature
* fixup! strip any user-added gtapped-tables keys
* fix a bad import
* Refactor `browse` navigation section
- Removes the dependency on the `CollapseSection` custom component
- Prepares the section for the "Add Data" project
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
* added an ... submenu to open questions
* fix
* menu is now in the title
* addressed feedback
* reverted submenu change
* added title menu items to funnel
* added e2e test
* added chevron
* unit test fix
* remove auto-close logic
* add metabot icon to searchbar, clicking icon opens metabot instead of search
* make chat ui appear in the sidebar
* add header section to the chat sidebar
* progress on conversation ui
* few tiny additions
* fetch mock suggested prompts from an endpoint, animate thinking text, fix search button touch target size, do no clear chat history when closing the chat sidebar
* messages rendered as markdown
* restyles
* more restyles
* fix input autofocus
* impl empty state, remove more textarea code
* hide copy button actions
* handle user aborted requests better
* design iteration
* latest design feedback, better auto-scrolling, better markdown rendering
* fix types
* increase unit test coverage
* impl long convo calc, add unit tests, fix focus issue if clicking the metabot option from the command palette
* move code to enterprise folder, fix some e2e, adjust header alignment, fix some types
* small clean ups, fix failing tests and erroring types
* add unit test and fix selector for getting latest agent messages by type
* review feedback, turn on an e2e test again
* try getting metabot e2e working
* fix embedding sdk
* impl pr feedback
* Make default toast color darker
* Avoid remounting the UndoList on overy toast change
* Tweak the undo animation to come in from the left
* Move more undo state into UndoListingOverlay
* Reverse toast order
* Manually manage undo target
* Remove unused props
* Store pause state in undo itself
* Use react-transition-group to render toast notifications
* Measure height so we can animate toast position
* Use css transforms only and avoid animating bottom
* Mock react-transition-group in test
* Unmount elements on exit
* Wait for animation in test
* Make toast ref optional
* Clean up transition
* Fix e2e test
* Disable transition group in cypress
* Add check_filled icon
* Default to check_filled
* Add text-secondary-inverse
* Use text-secondary-inverse color for icon on tooltip
* Remove min-width from toast
* Control the animation state of the todo's after moving the portal
* Fix icon in unit tests
* Simplify undo hooks
* Avoid overwriting state
* Add missing import
* Allow transitioning new todos even when not moving the portal
* Fix e2e test
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* fix typo
* don't error out during fields-metadata fallback if table is known to not exist
* table known to not exist impl for drivers
* add test
* add debug log
* add to changelog
* address review comments
This commit adds the conditional branding to dashboard email subscriptions and to question alerts. It does so by altering the handlebars templates we use for each, respectively. It follows the same logic as the rest of the PRs from this project. If a token has a whitelabel feature flag (Pro/Enterprise), the branding is not shown. It should be shown for OSS and for Starter plans.
* Add branding content to dashboard subscriptions
* Add branding content to alerts
* docstring typo fixes
* [SEM-8] Exclude all numeric values from Category semantic type consideration
* add unit test for SEM-8
* more typo fixes
* adjust existing tests for new sync behavior
* extend category classifier restriction to any non-text(-like)
* unit test for bolean classification
* further adjust existing tests for new sync behavior
* missed one test
* Fix e2e test by filtering
* fix e2e test by setting field semantic-type before hand
* don't classify int/bool values to category by name either
* rename tests
* extend tests
* style
* remove unused var
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Fix the sorting of custom column from a saved card in a new question
* Remove the expression_name that is not actually used.
* Use col.name instead of expression_name in sanatizeResultData
* Remove additional :expression_name references in tests
* Remove comment that was attached to now-removed :expression_name
* Remove expression_name from static-viz components
* Remove expression_name from instance_analytics export
* Add comment to now-deprecated expression_name field in DatasetColumn
* Add e2e test for sorting on nested custom column
---------
Co-authored-by: Lifeng Lv <lvlifen9@gmail.com>
* unlimit pivot exports
* fixups, fix default for collapsed rows setting
* address import issue
* simple unit test
* simplify test a bit
---------
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Running `yarn dev-ee` was broken on the latest master. (introduced in #58054)
The compiler was complaining about the `\` in the docstring because it was interpreting it as an escape character followed by a newline.
We're greatly simplifying and speeding up a Metabase setup
when the use case is "embedding". This currently applies only to
hosted instances. The FE in the main app will expect the query param
`use_case=embedding` to exist. If it does, we're showing the simplified
setup flow.
* Empty
* kick ci
* Allow temporal type in addition to string for `date()`
* Add date(datetime) test
* Use cast for date for postgres
* Extend date() on mysql
* Extend date() on bigquery
* Extend date() at snowflake
* Extend date() to datetime on clickhouse
* Generalize `date()` to :sql
* Update changelog with `date(datetime)`
* Empty
* DateTime->Date Coercion + test
* Slight cleanup
* Whitespace
* Cleaner implementation and test
* Test timestamp as well as timestamptz
* Correct implementation to get subtypes of DateTime
* Update helper text string for date
* Add e2e tests for date
* Also check to correct coercion of dates
* Add customization points to enable clickhouse implementation.
* Remove changelog-test/ignore true from ->date
* Fix defn ordering
* Fix expected type for datetime coercions
* Fix tests for sqlite
* Implement coercion for Mongo
* Implement specialization for oracle
* Add :expressions/date feature to all sql dbs
* Add :expressions/date feature to mongo
* Mongo: add text() and date()
* Better check
* Better check
* kick ci
* Add text() to all sql dbs
* Empty
* kick ci
* kick ci
* Format code
* Format code
* Add :text implementation to sql driver
* More robust tests across drivers
* Correct changelog text
* Ors on one line
* Or on same line
* kick ci
* Add `text-dbtype` multi-method
* Add athena implementation of text-dbtype
* Add databricks implementation of text-dbtype
* Remove date expression in tests that breaks some sql dbs
* Improve cast test for dates on other dbs
* Remove duplicate test
* Remove test that won't work for many dbs
* Use VARCHAR for oracle
* Use STRING for SparkSQL
* Use VARCHAR on Vertica
* Remove test that converts date to text before parsing it
* Add mongo exception
* Only run this test for SQL databases (requires native sql query)
* Add SQL Server text type
* Add Starburst text type
* Try varchar2(100)
* Simpler implementation
* Don't support date() for Oracle
* Try selecting from people to get oracle to work.
* Compile nested query for `text()` test
* Don't test empty string because Oracle treats it as NULL.
* Appease the kondo
* Implement Presto cast to text
* Implement cast-to-text in terms of cast (for presto)
* Correct test name
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
We're using the `computed.filters` value in the handlebars email template.
Previously, if `parameters` were an empty list `[]` we were returning a truthy value,
and handlebars was rendering an empty row in an email.
@ngoc figured out that we need to wrap parameters with `seq` in order to have a
a proper `nil` assigned to `:parameters`.
* Cast boolean expressions in :fields clauses to type bit on sqlserver
* Compile boolean literals :inline on sqlserver
* Add some tests for non-booleans to boolean_to_comparison_test.clj
* Add tests for boolean-expression-clause?
* Mention sql.qp/*inner-query* in boolean-expression-clause? docstring
* Split separate cases into separate tests
* Add cast in sql.qp/as if field options contain ::sql.qp/add-cast
Rather than duplicating logic for :select-top in apply-top-level-clause [:sqlserver :fields], just decorate boolean
:fields clauses with ::sql.qp/add-cast and let the parent method handle it. In this way, the :sqlserver method can
pre-process the MBQL :fields clauses rather than post-processing the raw honeysql.
* Add tests for {sum,count,distinct}-where with literal booleans
* Fix typo in literal expressions tests: filters -> filter
* Fix boolean->comparison for boolean expressions coming from nested queries
* Add sql.u/->honeysql-parent-method util
* Remove ::boolean-is-comparison abstract driver
* Lift ->honeysql call out of boolean-to-comparison
* Pass in boolean-types to boolean->comparison from driver
* Update comments and docstrings
* Require :left-join feature in nested-and-joined-literal-boolean-expressions-in-conditions-test
* Revert whitespace changes
* Implement sql.qp/apply-top-level-clause :filter in terms of parent method
* Fix typo in docstring
* Remove unused sql.helpers import
* Add comment about Oracle 23+ support to boolean_to_comparison.clj
* Rename local has-types? to has-some-types? in boolean-field-clause?
* [Backend modularization] `parameters` module (first pass)
* Make sure DB is initialized
* Oops revert the change I made to demo something during BE Guild Sync
* feat: generic data permissions visibility queries
Creates a new namespace with a generically applicable set of sql queries
for getting only models controlled by data permissions that a given user
has access to.
It also creates a new model.interface defmulti `visible-filter-clause`
that allows us to generate a filter clause for a given model to filter a
query on a supplied expression to only those models allowed for the
supplied user. There is a default implementation that is always true.
the `visible-filter-clause` method is only implemented for the Table
model so far since that is used in a search.
This also fixes a bug in the previous implementation of the sql
filtering for Table models that did not correctly handle the logic in
`data-perms/coalesce` for `:perms/view-data` where `:blocked` is ranked
ahead of `:legacy-no-self-service` for the purposes of selecting the
applicable user permission but is ranked below when testing permission levels.
Co-Authored-By: John Swanson <john.swanson@metabase.com>
* fixup! default for visible-fliter-clause to false
* fixup! add cast back for search permissions
* fixup! correct db type test
* fixup! more verbose table names in perms test
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Closes#41265.
The schema for is-empty and not-empty was updated in #56855 to allow StringExpressionArg as well as Field. This
updates desugar-is-empty-and-not-empty to also understand non-field args.
* fix: move quartz metric listeners to analytics
To make sure the task module does not need to depend on the analytics module
* fixup! remove analytics from task uses
* fixup! export listener adder methods
* fixup! missing docstring
* fixup! put the comment back in the right spot
* Update upgrading-metabase.md
Add changes that were needed for a long time :)
* suggested edits (#57913)
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Remove `expectGoodSnowplowEvents` entirely - it was an antipattern.
Replace it with a good set of snowplow helper functions that actually
accomplish what we want:
- `expectSnowplowEvent`, which matches on the *entire* event rather than
assuming the event is an unstructured event (most of ours are, but some
are not).
- `expectUnstructuredSnowplowEvent`, which behaves the same as
`expectGoodSnowplowEvent` used to, except I removed an unused behavior
where it took a function instead of a partial-match object.
- `assertNoUnstructuredSnowplowEvent`, which asserts the inverse of
`expectGoodSnowplowEvent` - there are zero matching snowplow events.
I removed `Good` from all these names as I think that's implied - if you
want to make an assertion that a bad event is being received you can
write your own helper :)
* Only send id and value when downloading csv
* Add repro for metabase#51813
* Merge in template tag parameters to parameters when running cards
* Fix erroneous test
* Do not pass all parameter data for cards
* Remove commented out code
* Only pass sparse parameters on public dashboards or cards
* Remove unused import
* Fix lint
---------
Co-authored-by: Tamás Benkő <tamas@metabase.com>
* [Backend modularization] `collections` module (DEV-444)
* Fix wrong namespace
* Fix new metabot v3 code
* Some test fixes. 🔧
* Move test for EE-specific behavior to EE
* Make `/api/collection` tests work correctly from the REPL
* Try to debug failing test
* Fix sanity checc
* Fix Kondo warnings
* Test fix 🔧
* Attempt to debug test failures
* More test failure debugging
* Cleanup
* More test failure debugging
* More test debugging
* Fix private var
* Test fix? 🔧
* adding get-visualization-columns in clojure land
* remove inline defs
* add util unit tests
* update comment
* linter fixes
* Fix render visualizer utils to work with entity ids, add test
* Fix body test
* Remove unused endpoints
* Resolve implicit FK Field -> PK Field -> [Name] Field remapping
This affects the field and dashboard APIs.
* New endpoints
* Remove unused code
* Always return value, possibly without label
* Stop requiring FK entry in FK -> PK -> Name resolution
* Relax FK requirement in remapped value resolution
* Remove remaining /field/ endpoints
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* use existing col settings helper
* forward declaration of viz-settings-for-col to prevent compilation issues
* fixing typo in parens
* fix unit tests
* Fix clj kondo warnings
* fix: wrap un-translated strings in t``
* fix more untranslated strings
* translate some aria-labels
* use interpolation instead of having to provide context
* address feedback and add context
* more t calls added
* revert changes to slider component as it's not used and may get removed
* remove file that i re-created when rebasing
* more context
* more context
* Update frontend/src/metabase/query_builder/components/MoveQuestionModal/MoveQuestionModal.tsx
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* Update frontend/src/metabase/query_builder/components/expressions/Editor/Listbox/Listbox.tsx
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* address feedback
* add context to what is SSL short for
* remove whitespace that could possibly create issues
* add missing import for c
---------
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* Empty
* kick ci
* Allow temporal type in addition to string for `date()`
* Add date(datetime) test
* Use cast for date for postgres
* Extend date() on mysql
* Extend date() on bigquery
* Extend date() at snowflake
* Extend date() to datetime on clickhouse
* Generalize `date()` to :sql
* Update changelog with `date(datetime)`
* Empty
* DateTime->Date Coercion + test
* Slight cleanup
* Whitespace
* Cleaner implementation and test
* Test timestamp as well as timestamptz
* Correct implementation to get subtypes of DateTime
* Update helper text string for date
* Add e2e tests for date
* Also check to correct coercion of dates
* Add customization points to enable clickhouse implementation.
* Remove changelog-test/ignore true from ->date
* Fix defn ordering
* Fix expected type for datetime coercions
* Fix tests for sqlite
* Implement coercion for Mongo
* Implement specialization for oracle
* Better check
* kick ci
* kick ci
* Format code
* Format code
* Correct changelog text
* Ors on one line
* Or on same line
* Remove zh from the list of supported translations (#57672)
We don't support the `zh` (Traditional Chinese) locale anymore – only `zh-CN` (Simplified Chinese), and `zh-HK` (Hong Kong), and `zh-TW` (Taiwan). Here's a [Slack discussion](https://metaboat.slack.com/archives/C6UBFU9GW/p1741983889165599).
So let's remove `zh` from the list of supported translations.
For some reason the list wasn't sorted alphabetically by language name - this PR fixes that too.
* [Backend modularization] Mega Renaming PR 1 (#57561)
* auth-provider => auth-provider.core
* enterprise/auth-provider
* `metabase.events.core`
* Move `sql-parsing-enabled` Setting into `query-analysis` (DEV-449)
* `metabase.events.core` (DEV-455)
* `metabase-enterprise.stats` => `metabase-enterprise.analytics.stats` (DEV-475)
* `enterprise/stale` cleanup (DEV-474)
* `enterprise/airgap` => `enterprise/premium-features`
* Appease Kondo
* Remove unused
* Mark setup tests flaky (#57710)
* mark setup tests flaky
* add flake
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* Fix parameter types! (#57677)
* [Notification] reduce module apis (#57678)
* fix audit log to record external emails when sending notifications (#57679)
* Remove "Model" and "Action" from the "New" button menu (#57647)
* feat: add a link to field filter docs from dashboard sidebar with native question (#57687)
* feat: add a link to field filter docs from dashboard sidebar with native question
* oops
* add missing translation
* Metabot v3 PoC (#52436)
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Mihael Konjević <konjevic@gmail.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: github-automation-metabase <166700802+github-automation-metabase@users.noreply.github.com>
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
* Patch the metabot-v3 merge (#57731)
* Remove unused props from the "New" menu
* Remove Metabot code related to the "New" menu
* Fix flaky e2e test in the onboarding `setup` spec (#57735)
* remove unused slider component (#57745)
* [Admin] Modernize components for People + Groups pages (#57365)
* update components in frontend/src/metabase/admin/people/components to use Mantine components and remove deprecated component usage
* update people and group pages/modals to use more modern components
* use relative sizes
* fix user success modal not closing on "Done" confirm button click
* remove CS from PeopleListRow
* clean up admin/people/colors.ts
* lint fix
* people list footer fix
* pr feedback
* [Admin] Convert people / group pages to RTKQuery (#57490)
* update components in frontend/src/metabase/admin/people/components to use Mantine components and remove deprecated component usage
* update people and group pages/modals to use more modern components
* use relative sizes
* fix user success modal not closing on "Done" confirm button click
* remove CS from PeopleListRow
* clean up admin/people/colors.ts
* lint fix
* people list footer fix
* pr feedback
* wip
* fix issues, people page updates, some clean up
* add unsaved files
* type fix, fetch user list from rktquery for PeopleList
* remove entity usage from unsubscribe modal, lint fix
* remove all remaining usages of user entity
* remove user entity
* test fixes, remove unused user schema
* bit of clean up
* qa pass - invalidate groups cache if user is deactivated/reactivated
* address pr feedback
* Bump org.apache.poi/poi-ooxml-full from 5.4.0 to 5.4.1 (#56357)
Inspect dependency changes here:
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
* Add analytics for new entity creation being initiated (#57673)
This commit adds analytics to the:
1. New button being clicked
- In the top app bar
- From the empty collection
2. New menu individual items being clicked:
- Dashboard
- ~Question~ (not possible at the moment)
- ~Native query~ (not possible at the moment)
3. An entity creation being initiated
- Model from the /browse/models page
- Metric from the /browse/metrics page
- Collection from the collection header
- Collection from the main nav sidebar
* Bump io.github.eerohele/pp from 9598f4c72b528480a85ea41651c45b08d1717428 to 4998a1fdd9d3713d7034d04509e9212204773bf2 (#57577)
Inspect dependency changes here: 9598f4c72b...4998a1fdd9
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
* Bump commons-io/commons-io from 2.18.0 to 2.19.0 (#56654)
Inspect dependency changes here:
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
* Shutdown index sync while its results are unused (#56885)
* Shutdown index sync while its results are unused
* Update tests
* Update mongo tests
* Bump com.google.guava/guava from 33.4.6-jre to 33.4.8-jre (#56952)
Inspect dependency changes here:
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
* Bump com.amazonaws/aws-java-sdk-core from 1.12.782 to 1.12.783 (#57586)
Inspect dependency changes here:
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
* Bump io.github.tonsky/clj-reload from 0.9.4 to 0.9.5 (#57578)
Inspect dependency changes here: https://github.com/tonsky/clj-reload/blob/0.9.5/CHANGELOG.md
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump babashka/fs from 0.5.24 to 0.5.25 (#57575)
Inspect dependency changes here: https://github.com/babashka/fs/blob/v0.5.25/CHANGELOG.md
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump org.apache.calcite.avatica/avatica from 1.25.0 to 1.26.0 (#54171)
Inspect dependency changes here: https://github.com/apache/calcite-avatica/compare/avatica-1.25.0-rc0...avatica-1.26.0-rc0
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* fix: do not render gauge if grouping col is date (#56715)
* fix: toolbar button color not changing on hover (#56694)
* Bump refactor-nrepl/refactor-nrepl from 3.10.0 to 3.11.0 (#57584)
Inspect dependency changes here: https://github.com/clojure-emacs/refactor-nrepl/blob/v3.11.0/CHANGELOG.md
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump org.jsoup/jsoup from 1.19.1 to 1.20.1 (#57583)
Inspect dependency changes here: https://github.com/jhy/jsoup/blob/jsoup-1.20.1/CHANGES.md
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* [Backend modularization] `view-log` module (DEV-427) (#57628)
* [Backend modularization] Move anonymous-stats-related stuff into `analytics` module (DEV-437) (#57631)
* Bump org.eclipse.jetty.ee9.websocket/jetty-ee9-websocket-jetty-server from 12.0.19 to 12.0.20 (#57579)
* Bump org.eclipse.jetty.ee9.websocket/jetty-ee9-websocket-jetty-server from 12.0.19 to 12.0.20
Inspect dependency changes here:
* Update deps.edn
---------
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump com.taoensso/nippy from 3.4.2 to 3.5.0 (#56953)
Inspect dependency changes here: https://github.com/taoensso/nippy/blob/v3.5.0/CHANGELOG.md
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
* Bump com.ibm.icu/icu4j from 76.1 to 77.1 (#55241)
Inspect dependency changes here:
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump thheller/shadow-cljs from 2.28.21 to 2.28.23 (#56657)
Inspect dependency changes here:
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* `date(datetime)` integration branch (#57448)
* Empty
* kick ci
* Allow temporal type in addition to string for `date()`
* Add date(datetime) test
* Use cast for date for postgres
* Extend date() on mysql
* Extend date() on bigquery
* Extend date() at snowflake
* Extend date() to datetime on clickhouse
* Generalize `date()` to :sql
* Update changelog with `date(datetime)`
* Update helper text string for date
* Add e2e tests for date
* Also check to correct coercion of dates
* Better check
* kick ci
* Format code
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Bump org.apache.poi/poi from 5.4.0 to 5.4.1 (#56355)
* Bump org.apache.poi/poi from 5.4.0 to 5.4.1
Inspect dependency changes here:
* Update deps.edn
---------
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump cider/cider-nrepl from 0.55.4 to 0.55.7 (#57576)
Inspect dependency changes here: https://github.com/clojure-emacs/cider-nrepl/blob/v0.55.7/CHANGELOG.md
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump org.postgresql/postgresql from 42.7.4 to 42.7.5 (#57314)
Inspect dependency changes here: https://github.com/pgjdbc/pgjdbc/blob/REL42.7.5/CHANGELOG.md
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump clj-kondo/clj-kondo from 2025.02.20 to 2025.04.07 (#56650)
* Bump clj-kondo/clj-kondo from 2025.02.20 to 2025.04.07
Inspect dependency changes here: https://github.com/clj-kondo/clj-kondo/blob/v2025.04.07/CHANGELOG.md
* Add kondo ignore to tests for new :missing-protocol-method linter
---------
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump thheller/shadow-cljs from 2.28.21 to 3.0.4 (#57585)
Inspect dependency changes here:
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Remove unused code in embedding data picker (#57700)
* Remove unused exported components
* Remove references to metrics
* Remove references to saved questions
* Remove irrelevant tests
* Remove unnecessary prop
* Fix more unit tests
* Remove more unnecessary code
* Fix failed unit tests
* Remove unnecessary prop
* speed up notification tests (#57763)
* Update logger presets (#57667)
* Update presets
* Update presets
* Update test
* Apply suggestions from code review
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
---------
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* better snowplow errors when expecting the wrong number of good events (#57730)
* add ?. to make it more resilient
* add more logssss
* cy.log
* some nicer error message
* fix eventType
* removes leftover lines
* typos on comments
* Update e2e/support/helpers/e2e-snowplow-helpers.js
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Mihael Konjević <konjevic@gmail.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: github-automation-metabase <166700802+github-automation-metabase@users.noreply.github.com>
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
Co-authored-by: Adithya Vardhan <imadithyavardhan@gmail.com>
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
* add ?. to make it more resilient
* add more logssss
* cy.log
* some nicer error message
* fix eventType
* removes leftover lines
* typos on comments
* Update e2e/support/helpers/e2e-snowplow-helpers.js
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Empty
* kick ci
* Allow temporal type in addition to string for `date()`
* Add date(datetime) test
* Use cast for date for postgres
* Extend date() on mysql
* Extend date() on bigquery
* Extend date() at snowflake
* Extend date() to datetime on clickhouse
* Generalize `date()` to :sql
* Update changelog with `date(datetime)`
* Update helper text string for date
* Add e2e tests for date
* Also check to correct coercion of dates
* Better check
* kick ci
* Format code
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
This commit adds analytics to the:
1. New button being clicked
- In the top app bar
- From the empty collection
2. New menu individual items being clicked:
- Dashboard
- ~Question~ (not possible at the moment)
- ~Native query~ (not possible at the moment)
3. An entity creation being initiated
- Model from the /browse/models page
- Metric from the /browse/metrics page
- Collection from the collection header
- Collection from the main nav sidebar
* update components in frontend/src/metabase/admin/people/components to use Mantine components and remove deprecated component usage
* update people and group pages/modals to use more modern components
* use relative sizes
* fix user success modal not closing on "Done" confirm button click
* remove CS from PeopleListRow
* clean up admin/people/colors.ts
* lint fix
* people list footer fix
* pr feedback
* wip
* fix issues, people page updates, some clean up
* add unsaved files
* type fix, fetch user list from rktquery for PeopleList
* remove entity usage from unsubscribe modal, lint fix
* remove all remaining usages of user entity
* remove user entity
* test fixes, remove unused user schema
* bit of clean up
* qa pass - invalidate groups cache if user is deactivated/reactivated
* address pr feedback
* update components in frontend/src/metabase/admin/people/components to use Mantine components and remove deprecated component usage
* update people and group pages/modals to use more modern components
* use relative sizes
* fix user success modal not closing on "Done" confirm button click
* remove CS from PeopleListRow
* clean up admin/people/colors.ts
* lint fix
* people list footer fix
* pr feedback
We don't support the `zh` (Traditional Chinese) locale anymore – only `zh-CN` (Simplified Chinese), and `zh-HK` (Hong Kong), and `zh-TW` (Taiwan). Here's a [Slack discussion](https://metaboat.slack.com/archives/C6UBFU9GW/p1741983889165599).
So let's remove `zh` from the list of supported translations.
For some reason the list wasn't sorted alphabetically by language name - this PR fixes that too.
* convert UnsubscribeUserModal and UnsubscribeUserForm components
* convert UserActivationModal to TS
* convert UserPasswordResetModal to TS
* convert UserSuccessModal to TS
* convert PeopleList and PeopleListingApp components to TS
* convert PeopleListRow to TS
* convert use-people-query to TS
* fix deprecated usages of google_auth and ldap_auth with sso_source field
* fix lint
* convert GroupsListing, GroupsListingApp, and AddRow components to TS
* convert lib/groups, AdminContentTable, and AddMemberRow to TS
* convert GroupDetailApp and GroupDetail to TS
* fix import
* convert GroupSummary to TS, make adjustments to other converted files to make types happy
* remove people/components/SearchInput and convert usage to mantine component to avoid TS migration
* remove styled component usage from people and groups pages and components
* adjustment to people listing app
* refactor: rename test file to tsx to have hopefully better diff in the
next commit
* fix EMB-360: Make Mantine components use the locale from LocaleProvider
* close EMB-373: make useLocale return the locale from LocaleProvider if
present
* Use Mantine dropdown for snippet menu
* Fix search icon in snippet header too
* Remove default ICON_SIZE
* Fix MIN_SNIPPETS_FOR_SEARCH
* Add createSnippet helper
* Add repro for metabase#57441
* Add test for snippet search
* Move snippet helper to api
* Use actual ref for input
* Simplify header search rendering
* Use SidebarHeader for snippet sidebar
* Use Flex for most components
* Simplify padding
* Use class component for archived snippets
* Use class for header button
* Use direct properties
* Rely less on custom css
* Fix header padding
* Fix e2e tests
* Remove unused eslint-disable
* Add type annotation
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Move SimpleDataPicker to a new directory preparing for new components
* Use the embedding data picker from a new folder
This requires turning the component into a plugin, which we'll do in the
next commit.
* Move plugin property to a more appropriate place
* Implement embedding multi-stage data picker as an EE plugin
* Add DataSourceSelectorProps
* Remove reference to query builder data picker
* Fix failing unit tests
* Move embedding data picker components to a proper location
* Update Appearance Settings
* unit test additions + updates
* update e2e tests
* test updates
* unit test fix
* fix lint errors
* obey the linter
* add test coverage
* add more tests
* support env var override ui
* remove dead code
* nicer routing
* useDisclosure
* stabilize test
* adding watermark for dynamic viz
* starting to put together e2e tests for watermarks
* adding unit test
* adding loki tests
* adding more e2e tests
* changing test id
* adding e2e tests on chart interactivity
* fixing type error
* Split up config into categories
* Break out clauses into groups
* Add duplicate name check
* Allow setting shared options
* Add category to mbql clause config
* Rename groups
* Use enum for category
* Build examples using helper functions
* Build ExpressionParts directly in helper-text
* Flatten examples
* Wrap example in function
* Enable late-binding for args
* Unwrap example
* Remove get name and get description wrappers
* Remove getHelpDocsUrl helper
* Simplify example fn
* Add missing export
* Remove getFunctionByStructure
* Simplify help text helpers
* Rename structure to displayName
* Make args required
* Manually type clause definition type
* Merge HELPER_TEXT_STRINGS into config
* Move help text helpers to separate file
* Use data from MBQL_CLAUSES instead of HELPER_TEXT_STRINGS
* Allow defining stuff in config
* Move help text into config
* Remove fallback to HELPER_TEXT_STRINGS
* Remove helper-text-strings
* Allow more stuff to be optional
* Build clause in getClauseDefinition
* Avoid self-referential types
* Use getClauseDefinition in completer
* Handle new args definition
* Handle null instead of undefined
* Enforce type on all argument definitions
* Enforce name on all argument definitions
* Get clause defns
* Fix types
* Simplify tests
* Remove unnecessary guard
* Avoid creating unnecessary functions
* Move defineClauses to separate module
* Remove any in config
* Rename helper-text-strings test to help-text
* Implement Tenant Groups
- normal users cannot be added to a tenant group
- tenant users cannot be added to a normal group
- tenant groups cannot receive write access to any collection
- tenant groups cannot receive any access to the audit collection
- tenant groups cannot receive any application permissions
* Add a setting to turn tenants off and on
When the setting is turned on:
- create the All External Users group, with
`magic_group_type=all-exsternal-users` and `is_tenant_group=true`
When the setting is turned off:
- make sure there are no remaining tenants or tenant users
- delete the All External Users group
* Remove the need for memoized `tenant?` functions
* Fix tests
* allowe using with-temp with perms group membership
* fix more tests
* remove unnecessary bindings
* Put setting behind a feature flag
* Don't read setting from env
closes adm-648
closes adm-657
closes adm-649
Resolves PRG-25
This PR adds a new button to the collection header menu. The button simply opens a new collection modal (lets you create a new collection). There are two important things to note about this change:
1. This button shows only when a user has a curate permissions for the current collection
2. This change required adding a new icon to our design system, which was approved by Maz.
Resolves PRG-20
This PR replaces the old `...` menu next to the "Collections" sidebar header. As a reminder, this menu was previously triggering a popup with two items:
1. New collection
2. Other users' personal collections (in case the user is an admin)
We're now improving the UX and replacing this menu with two distinct items:
1. The `+` icon that creates a new collection directly
2. The new individual sidebar link that leads to other users' personal collections (again, only if admin)
* update title sizes to match figma - add multi line title stories for lineheight examples
* little more line height in modal titles
* little fixes
* more little fixes
* reduce smartscalar title size
* Update Loki Snapshots
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* fix: add table-level sql based permission query
Some places in our code we may be returning fewer tables than a user has
access to because we load a limited number of models from the database
then apply permissions filters to those. This provides a subquery based
check for table permissions that we can use in other queries to do the
same kind of test for table access that we would do on the clojure-side.
Co-Authored-By: Nathan Voxland <nathan@metabase.com>
* fixup! restore check-permissions-for-model impl
* Update src/metabase/models/table.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* fixup! caluse -> clause spelling
* fixup! reformat malli annotations; add docstring
* fixup! add Literal type to honeysql helpers
* fixup! restore accidentally deleted clause
---------
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* fix: filter button labels
* fix: part of EMB-335, uiName: t => getUiName: () => t
* fix linting issue
* fixed unit test mock missing getUiName
frontend/src/metabase/visualizations/components/Visualization/Visualization.unit.spec.js
missing
* make application-permissions.cy.spec.js pass
* cursor says this could fix the issue
* harder vibe coding
* remove handling of made up by llm elements that do not exist
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
* Rename ALL_NODES to NODE_TYPE
* Remove unnecessary name override
* Remove unused EXPRESSION_OPERATOR_WITHOUT_ORDER_PRIORITY
* Use precedence from pratt compiler
* Move EDITOR_FK_SYMBOLS to identifier module
* Lift const descriptor
* Remove unused characters config
* Remove identifierAlwaysQuoted from EDITOR_QUOTES config and make the behaviour the default
* Move identifierDefaultQuote out of config and hard-code the behaviour
* Flatten default quotes config and move default into string module
* Remove wrong comment
* Use parseOperatorType to find operator type
* Remove separate operator groups
* Remove unused FUNCTIONS variable
* Remove barely used getExpressionName
* Use Boolean token to parse booleans
* Remove True and False from operators
* Rename Star and Slash to Multiply and Divide
* Remove FIELD_MARKERS
* Simplify config definition
* Inline getDescriptionForNow
* Move OPERATOR check to formatter
* Use normal quotes in config
* Split up MBQL_CLAUSES to avoid duplicating names
* Split clause helpers into separate files
* Fix sort order in completion tests
* Move operators to a separate group in config
* Use EXPRESSION_OPERATORS in formatter
* Export using star
* Rename OPERATOR to PUNCTUATOR
* Rename parseOperatorType to parsePunctuator
* Rename parseOperatorType to parsePunctuator
* Add helper for potential function prefixes
* Remove unused OPERATORS
* Remove superflouous helper
* Inline formatCallExpression
* Use Lib.ExpressionOperator where possible
* Remove impossible case for expression
* Inline isLogicOperator
* Rename variable for clarity
* Deduplicate helpers
* Move formatter helpers into utils
* Simplify helper
* Fix import order
* Remove reliance on PUNCTUATOR type
* Fix variable name
* Simplify punctuator definition
* Clean up lexifier
* Flatten lexifier results
* Remove errors for parser results
* Flatten parser result
* Simplify parser tests
* Remove cast
* Remove unused import
* Translate all error messages in formatter
* Simplify type
* Use sentence case name for type
When making sure stuff can load even with a loop, the circuit breaker was removed, leading to endless loops. That
would happen if the data causing the loop wasn't stripped properly. Right now if stripping things once did not help
and the entity is in the pipeline again, the process will break (with hopefully enough data to detect why).
This also changes the logic from exception to just recursive calls. Plus it's back at skipping all dashcards in a
Dashboard instead of just skipping visualization logic, which should make for a way faster processing (less deep for
sure). Plus it tries to improve logging a bit so it's easier to understand what's going on in complex cases.
* Remove Legacy Metrics code from the backend
* Code cleanup
* Test fix 🔧
* Test fix 🔧
* PR feedback
* Extra test
* Revert some X-Rays changes
* Remove commented-out test
* Test fixes 🔧
* feat: add prometheus metric for quartz task states
Adds a new quartz_task_state metric that will report to prometheus the
states of all quartz jobs running on a metabase instance using a guage.
Each task state is reported as a label and we also report a state for
tasks that are executing.
All hosts on a metabase cluster _should_ report the same value for these
metrics if they are scraped at the same time, and we need to consider if
the additional DB load is okay since this does query the database during
a prometheus scrape to report this metric.
* fixup! add job-name tag and histogram of job time
* try to fix cypress download format flakes
* handle unset last_table_download_format
* more test tweaks
* make tests shorter, refactor, and add wait periods
* rearrange more code
* remove the waits, quarantine tests as flaky
* move cy.intercept to right before reload
* skip tests again
* unravel .then calls
* feat: Implement multi-catalog for databricks
* Fix schema for nil
* Add tests for joins on multi-catalog
* Fix multicatalog driver name
* Remove require
* Alphabetize drivers
* Add changelog
* Fix tests
* Fix tests
* Do not sync system catalogs
* Address review comments
* Update connection properties
* Update example to make clear its a schema
* [SEM-219] Humanize error message even more
To provide the customized error message, the type check built into the defendpoint framework has to be disabled, which
results in less specific API documentation.
(Malli has an experimental feature we currently don't use for looking up humanized errors from parent schemas, that
would let us configure the error message AND keep the input type information.)
* Test comparing case vs filtered ags
* Add test and update case definition
* Add logging to check CI, mark as driver ns
* Remove metadata to see which driver tests run
* add failing op for some drivers
* Use case to filter metric ag
* Filter in case
* Existing tests update
* WIP Respect expression deduplication
* Move transformation right after metric fetch
* Update test
* Aggregation maps to be used in tests
* Simplify metric transform
* New tests
* Cleanup
* Cleanup
* Cleanup
* Fix equality
* Cleanup
* Cleanup equality
* Simplify test
* Tolerate float deviation
* Feature dependent aggregations
* Update test
* Fix value formatting
* Add var to std dev ags
* Update the filtering logic
* Speculate
* Adjust tests
* Update test
* Kondo
* Adjust test for starburst
* Remove debug def
* Apply suggestions from code review
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Adjust naming
* Comment
* fresh-uuids into definition
* Add shortcut for empty condition
* Adjust meta and case default
* Remove now redundant condition
* Docstring
* Indentation
* Remove dev leftover
* Remove dev leftover
* Update tests and distinct-where
* Nullary aggregations map
* Distinct-where feature
* Fix formatting
* Adjust e2e test
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* convert UnsubscribeUserModal and UnsubscribeUserForm components
* convert UserActivationModal to TS
* convert UserPasswordResetModal to TS
* convert UserSuccessModal to TS
* convert PeopleList and PeopleListingApp components to TS
* convert PeopleListRow to TS
* convert use-people-query to TS
* fix deprecated usages of google_auth and ldap_auth with sso_source field
* fix lint
* convert GroupsListing, GroupsListingApp, and AddRow components to TS
* convert lib/groups, AdminContentTable, and AddMemberRow to TS
* convert GroupDetailApp and GroupDetail to TS
* fix import
* convert GroupSummary to TS, make adjustments to other converted files to make types happy
* remove people/components/SearchInput and convert usage to mantine component to avoid TS migration
* Add float cases to integer cast testing
* Update types of integer mbql expression
* Better messages for tests
* Add rounding test and specialize for redshift
* Don't use extreme values for redshift
* Add e2e tests for float -> integer casts
* Update helper text for integer
* Change test message.
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* Helper function to convert to integer
* Appease the kondo
* More accurate comment
* Update arithmetic.cljc
* Refactor to use better ->integer implementation for SQL dbs
* kick ci
* Missed an argument
* Round properly for inline constants
* Add String->Integer and Float->Integer coercion strategies
* Add MongoDB string->integer and float->integer coercions
* Fix types in coercion test
* Also cast in mongodb coercion
* Use NUMBER(p) type in Oracle
* Spark SQL needs explicit round()
* SQLite needs explicit round()
* SQL Server needs explicit round()
* Clickhouse cast to float first
* SQLite needs to cast to float first so it can round
* SQL Server needs to cast to float so it can round
* Mongo needs to cast to float before rounding
* Spark SQL needs to cast to float so it can round
* Revert converting multimethod ->integer to defn
* Remove cast-integer multi-method and restore ->integer
* Update `integer-dbtype` in changelog
* Fix oracle integer type
* Fix spark sql casting
* Databricks driver needs to round
* kick ci
* Use ->integer in coercion
* Helper function for cast to integer with round
* kick ci
* Remove unnecessary ->float implementation
* Fix cljfmt precommmit hook
* Fix clickhouse returning bigdecimal
* Handle raw values in untyped-inline-value
* This came in with the merge
* Allow integer(float) and integer(string)
* Remove stray println
* Work for known types only
* Test for converting literals at compile time
* Update mysql
* Fix timezone test failing on null week-of-year
* Update fe test for float coercion
* Doh! Typo.
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* add viz setting fe
* ClJS changes
* Fix up some BE tests
* fix last test
* spec, hint
* specs
* review
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
Avoid the possiblity of deadlocks during cached query processing by no
taking the connection using with-connection and instead just selecting
the cached result using toucan.
I _think_ this connection holding was introduced under the assumption
that we could 'stream' a binary column from postgresql using the
.getBinaryStreaming jdbc method. However postgres requires binary
columns to be full read and written and so we are unable to stream
results from that column, so this operation ends up being the same as
just querying the column normally.
Because we are not streaming the result in it seems safe to just yeild
the connection back to pool so it can shared with other requests.
* initial implementation
* e2e test
* tweaks
* fix e2e test + refactor
* reduce unneeded comments
* clean ns
* fix unit tests
* fix e2e test
* disable userKV when no user is logged in
* fix up a number of unit tests
* refactor out BaseQuestionDownloadPopoverProps with two wrapper components to avoid using the user kv hook in public/embedded contexts
* fix up types
* fix up e2e test
* use ?. syntax
* hide email domains for users without settings access
* fix test
* fix up e2e test
* update BE setting visibility
* Revert "update BE setting visibility"
This reverts commit b3728ad21d.
* remove commented-out line
* fix: scheduler state management in cloud migration
When we create the h2 database to dump a metabase instance to during
cloud migration we need to stop and start the quartz scheduler instance
running on the instance during the migration.
Some liquibase migrations handling quartz job changes expect to control
the scehduler and start/stop it while those migrations run. Recent
changes to guard against this behaviour in testing case these
migrations to throw an exception when they run during the cloud
migration task.
This also fixes an older bug that the quartz scheduler would remain off
on the instance being migrated after the cloud migration task finishes.
* fixup! stop scheduler after all calls to migrate!
* fixup! formatting
* fix: wrong syntax on `if:` was making 'cross-version-for-breaking-changes'
always run
* Update .github/workflows/embedding-sdk.yml
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Rename OPERATORS in tokenizer to FIELD_PUNCTUATORS to avoid confusion with expression operators
* Rename Reference to Field in tokenizer for consistency
* Allow specifying the string delimiter to unquote
* Reduce state in tokenizer
* Allow the tokenizer to parse field tokens that are missing an opening bracket
* Add more comments and rename variable for clarity
* Implement proper escaping in field tokenizer
* Remove duplicate OPEN_BRACKET from punctuators
* Simplify lexifier now that tokenizer returns the correct token for fields
* Reorder loop to reduce duplication
* Simplify lexifier now that tokenizer returns the correct token for fields
* Add unit tests for unquoting with specific character
* Make invalid token error message more precise
* Punctuation
* Typo
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Resolves PRG-1 by introducing the branding content to the dashboard PDF exports
Resolves PRG-7 by making the branding content clickable
Additionally resolves PRG-14
* convert UnsubscribeUserModal and UnsubscribeUserForm components
* convert UserActivationModal to TS
* convert UserPasswordResetModal to TS
* convert UserSuccessModal to TS
* convert PeopleList and PeopleListingApp components to TS
* convert PeopleListRow to TS
* convert use-people-query to TS
* fix deprecated usages of google_auth and ldap_auth with sso_source field
* fix lint
* float() tests + tier 1 (postgres)
* Add more parsing test.
* Better float comparison
* Add expression/float feature
* Add float function to config
* Add float help text
* Add e2e test for float()
* Add `:expressions/float` to list of feature flags
* Only allow casting text to float (not number)
* Better test message
* Refactor for nicer code
* `float()` for snowflake (#56927)
* Implement float() for snowflake
Also add text and date feature keys because somehow they were left off.
* Don't use Long/MAX_VALUE in tests because it breaks some dbs.
* Fix copy-paste error
* kick ci
* Implement float() for mysql (#56919)
Also, add :expressions/date and :expressions/text features that were missing somehow.
* Cast and split custom expression functions for Clickhouse (#56924)
* Implement integer() for clickhouse
ClickHouse is recently tier 2!
* Implement float() for clickhouse
* Implement text() for clickhouse
* Add date() to clickhouse
* Implement splitPart() for clickhouse
* Cast to Date32 after parsing date
* Use a nullable type
* Use nullable types for casts
* Try parseDateTime instead to avoid null
* Use case sensitive function names
* Fix argument order of splitByString()
* Don't use Long/MAX_VALUE in Clickhouse
* Clickhouse needs a lot of type hints!
* Cast position to int
* Better tests for split part
* Negative indexes should return empty string
* kick ci
* Convert index to long (for ClickHouse)
* `float()` implementation for BigQuery (#56920)
* Add float() support for bigquery.
Also add features for date and text that mistakenly got left out.
* Correct argument order for parse_date
* Refactor ->float
* Remove cast-float implementations
* Reinstate Clickhouses' cast to float function.
* Remove specialized :float functions for sql drivers
Replace them with a single :sql implementation
* kick ci
* More refactoring to get rid of questionable recursion
* Fix three tests
* Add null case for literal ->float compilation
* Add type info
* Revert making typed? public (don't need it no more)
* Revert converting multimethod ->float to defn
Also, use a new function called `coerce-float` that does the work.
* Try clickhouse using cast() instead of toFloat64()
* Revert presto jdbc ->float implementation
* Use keyword type name for float-dbtype
* Revert removing ->float method from bigquery clickhouse and starburst
* Add `float-dbtype`, `cast-float`, and `:expressions/float` to changelog
* Remove cast-float multimethod and redundant ->float methods
* REmove mt/dataset test-data from split-part tests.
Is it really necessary?
* Revert qpt changes
* Properly handle NULL in clickhouse cast to float
* Fix presto and starburst
With an improved safe-denominator function
* Change to double as default sql type for casting
* Fix test for casting to double
* Use ->float instead of coerce-float for coercion
* Use DOUBLE PRECISION for Vertica driver
* Use BINARY_DOUBLE for oracle
* Use float for sqlserver
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
and use it as the source of truth for whether a group is the "All Users"
or "Administrators" group, instead of just the name.
This is a unique field, but it's nullable. H2, MariaDB, MySQL, and
Postgres all treat `null` values as unequal, so it's fine to have many
permissions groups without a `magic_group_type`, but any row *with* a
`magic_group_type` must be distinct from the others. In other words, we
can have lots of non-magic groups and exactly one magic group of each
type.
For now, we'll just have the same Administrators and All Users groups
and the names won't change. At some point, when tenants are enabled,
we'll want to rename "All Users" to "All Internal Users" and create a
separate "All External Users" group.
Fixes QUE-989, QUE-905.
Stop backfilling idents in `result_metadata` on reading cards. That's costly and recursive, and was the source
of the perf issues on stats.
We're planning instead to support optional, best-effort idents for a longer period, and let the background job do
the work.
Fixes QUE-904.
With (1) a fresh column like an aggregation, and (2) when the source was a model:
`annotate` was wrapping the column's ident as if it came from the source model, but that's incorrect.
This adjusts the order of operations so that only columns inherited from the source get their idents
wrapped with the source model.
* Updating text wrap and minibar logic to gracefully handle structure of column widths; updated fallback column width when no width is present in viz settings
* removing inline defs
* updating shortcuts, adding shortcut in profile menu, add hotkeys to close shortcut modal and sidesheets
* updating tests
* no one will know
* updating to use mantine hooks
* teach claude how to write backend code
* Add solid instructions for calling into the repl
* teach it how to behave when there's no repl
* remove mcp stuff for now
* add lint pr quick command
* fix: Add missing docstrings in mage/be_dev.clj
Added docstrings to parse-int, nrepl-port, and nrepl-eval functions to fix clj-kondo linting warnings.
* flesh out Lint PR quick command
* feat: enable equals-true Kondo linter and fix warnings
This change enables the `:equals-true` warning in the clj-kondo linter configuration,
which identifies code patterns where `(= true x)` is used instead of the more idiomatic
`(true? x)` form in Clojure.
Key changes:
- Enable `:equals-true` linter with warning level in .clj-kondo/config.edn
- Fix all instances of `(= true x)` throughout the codebase, replacing with `(true? x)`
- Add a new utility to check balanced parentheses in Clojure forms
- Add a new mage command `-balance-parens-check` to validate balanced parentheses
- Update CLAUDE.md with documentation about the balanced parentheses utility
This change improves code quality by promoting idiomatic Clojure usage and adds
a helpful tool for ensuring proper Clojure syntax when editing code.
* fix: update code to resolve equals-true Kondo linter warnings
This commit addresses the newly enabled equals-true linter warnings by:
- Adding #_{:clj-kondo/ignore [:equals-true]} to are forms that legitimately need to test both true/false values
- Replacing (= true x) with (true? x) and (= false x) with (false? x) in test assertions where appropriate
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix more, refine claude instructions
* fix more = true -> true? linter errors automatically
* fix remaining (= true x) -> (true? x) linter warnings by adding clj-kondo ignores
We need the ignores for 'are' macro clj-kondo tests where (= expected (fn x)) is the idiomatic pattern.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* move the linter to the things-we-use section
* update claude.md from github copliot's review
* balanced-parens? -> is-readable-code?
- Returns explicit errors, usually with line numbers.
* update claude.md (mostly formatting)
* Fix kondo-updated when there are no matching files
* Update CLAUDE.md to replace -balance-parens-check with -code-is-readable
* Refactor mage -eval to mage -repl with improved usability and reliability
The implementation uses eval and read-string strategically to control when and where code is evaluated:
1. The code from the command line must first be transported as a string to the running REPL server
2. read-string allows parsing the code without executing it
3. eval then executes the code in the correct namespace context
4. This creates a "REPL inside a REPL" where namespace switching and evaluation happen in a controlled environment
5. This approach prevents premature evaluation and ensures code runs in the intended scope
* -check-readable task naming
* cleanup -check-readable
* fix output + fix user.clj
* shuffle code around
* Update CLAUDE.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Use Resolver type from resolver module
* Move getKindForType to resolver
* Inline compileDimension
* Simplify resolver and make data flow requirements more obvious
* Avoid fetching unused dimensions
* Cache metadata displayInfo while resolving
* Move compiler-based resolver tests to compiler
* Rename variables to be more generic
* Refactor resolver
* Merge ExpressionType and MBQLClauseReturnType
* Add tests for resolver
* Be more precise in the resolver error messages
* Be more specific about expected field type in errors
* Add tests for unknown dimensions
* Rename any_ to any
* Add tests for mixed case matching
* Allow matching column names case-insensitively
* Replace ResolveError with CompilerError in resolver
* Use expressionMode in tests
* Fix types
* Remove cast in test
* Add more comments
* Add e2e tests for field matching
* Fix lint
* Rename Field to column in resolver error messages
* Remove TODO's and replace them with comments
* Fix error message in e2e test
Resolves PRG-2
This commit adds Metabase branding to the PNG downloads
for instances that **do not** have the whitelabel feature flag.
Even though it is possible to download the dark mode PNG,
this commit does not fully resolve PRG-6 because it doesn't
turn the text white.
* Check all mismatched sibling tokens, not just missing comma's
* Rename siblings check to checkMismatchedSiblings
* Merge imports
* Also catch call expressions
result_metadata for a query on a model returns the underlying columns, not the model's columns. This was causing
drills to break in certain circumstances, because the query processor was trying to cast a particular value twice.
Making drills default to using the column from lib/returned-columns fixes the issue.
* update copy w/ description, update db routing docs link, impl prefill for new destination dbs, fix bug where editing db details would not let you tab through the form
* fixes
* make no user attributes error message link to people page, db routing section expanded by default if feature enabled
* pull out fix for edit db form not being tab-able
* fix failing tests
* copy feedback
* wip
* More robust checking for timeout exceptions
Overview:
We enumerate items in the database and then we want to see which ones
the user can actually run queries against. To do this we run a query
like `select false from TABLE where 1 != 1 limit 0`. The hope is that
the query analyzer realizes it only needs to do permission checks and
doesn't try to satisfy these conditions (either because limit says no
rows, or 1 != 1 is easily proven to filter all rows). So we _usually_
get very quick feedback if we can select from the table or not.
Snowflake seems to not do this on views. If a view takes a long time to
query, snowflake seems to take a long time to query. The original fix
was to add a 15 second timeout to the query. The expectation here is
that Snowflake can quickly fail if there are no permissions for the
table or view, but might take a long time to run the probe query. But we
can interpret a timeout as a successful permission check.
The original way we checked for timeouts was a very naive check for
`java.sql.SQLTimeoutException`. It seems basically no one uses this when
you do:
```clojure
(with-open [stmt (...)]
(doto stmt
(.setQueryTimeout 15)
(.execute)))
```
For instance, here's some logs from Snowflake:
```
WARN net.snowflake.client.core.StmtUtil Cancelling query 0e8987ee-2d22-40f9-b30d-0129d38ce976 with reason TIMEOUT
INFO metabase.driver.sql-jdbc.sync.describe-database <schema>:<table-name>: Assuming no SELECT privileges: caught exception,net.snowflake.client.jdbc.SnowflakeSQLException: SQL execution canceled, at ...
```
Note It's a regular `net.snowflake.client.jdbc.SnowflakeSQLException`.
Here's a true permissions failure for comparison:
```
sync.describe-database <schema>:<table>: Assuming no SELECT privileges: caught exception,net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error:,Failure during expansion of view 'SOME_VIEW': SQL compilation error:,Object 'DB.SCHEMA.TABLE' does not exist or not authorized.,
```
So we use some regex checks against the messages to try to detect when
we have a permissions failure (cannot select) vs a timeout of the
probe (presumably can select).
And a reminder of the balances here: it's better to show someone a table
that they get an error if they try to query than to hide a table that
they can query. So this change should bias towards reporting can-select
in cases where it might not be clear.
* Only run tests when classes are present
if you don't have :drivers on classpath it would fail
* ensure boolean is returned
tests assert false? on a nil
* CI fails based on the indentation of comments
* empty commit for CI
* Move postgres test into postgres test namespace
* Replace ex-msg matching with sqlerrorcode/sqlstate
Use jdbc error codes and sql state matching to figure out if a
SQLException is caused by user action canceling the request (essentially
how setQueryTimeout works). This adds a multimethod dispatching off the
driver keyword to match up either the SQLState code or the SQLErrorCode
with the appropriate value for a given database driver. Some of these
were found by looking how QueryTimeoutException is implemented in
Hibernate and others were found from JDBC driver source code or documentation.
* fixup! handle multiple codes for mariadb
* fixup! add partial query timeout test
* fixup! split apart sql timeout tests
* fixup! use correct sqlstate for snowflake timeout
* fixup! remove split out tests
* fixup! remove when-class
* fixup! snowflake sql state
* fixup! move query-canceled? into driver code
Removes the jdbc-exceptions namespace and instead makes query-canceled a
driver multimethod that extracts and dispatches on the SQLException
class within the sql-jdbc driver.
Non-jdbc drivers can implement this method, but it is only used for
handling errors in the cluster lock using the appdb and for the sql-jdbc
describe-database sync task.
Co-Authored-By: Nathan Voxland <nathan@metabase.com>
* fixup! update driver changelog add default method
* fixup! merge mysql/mariadb methods
* fixup! add added metadata
* fixup! make test throw instead of is false
---------
Co-authored-by: Edward Paget <ed.paget@gmail.com>
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
Co-authored-by: Nathan Voxland <nathan@metabase.com>
* Fix deleting a DB with database routing on
* Fix database sync with a user attribute set
When an admin has the relevant user attribute set and tries to sync the
database schema, we use `(with-database-routing-off ...)` to explicitly
turn off database routing for the purposes of sync. However, there are
two problems here:
First, this is just informational for the purposes of error checking, it
didn't actually turn off database routing, it just made us throw an
exception if a driver touched a mirror database (in dev).
Second, during query processing, when we're actually doing the database
routing "swap" to replace the router with the mirror DB, we were doing
`with-database-routing-on`. This essentially means that if you do
```
(with-db-routing-off
(qp/process-query query rff))
```
you *don't* actually turn off DB routing. 🤦
Both underlying issues get fixed by the same change: if database-routing
is OFF, don't DO any database routing in the QP middleware. This seems
more reasonable to me anyway.
* CR resp
* add ttag/no-module-declaration rule + eslint ignore comments
* change the reference to github issue instead of linear
* add new ignores after rebase
* remove formatting changes from eslint files
* fix eslint rule mising from a file
* convert one field to getter as that got added after I opened the codemod
pr and it's failing the check here
* more //eslint-disable-next-line
* Fix 66k statement params in sync-table-privileges and sync-fks
* Test for privileges
* Test privileges
* Update and test set-initial-table-sync-complete-for-db!
* Move :expression-literals aggregation tests into expressions_test.clj
These tests all use :basic-aggregations not :expression-aggregations, so move them into expressions_test.clj to
consolidate the :expression-literals tests in one place.
* Add clause with top-level boolean literal in case-with-literal-expression-test
* Prototype sharing CSV pivot export code with Excel instead of Excel native pivots
* Cleanup some code, fix reflection warnings
* clean ns
* halfway to getting native excel formatting working
* native formatting working, but no auto-sizing
* fix up formatting, get auto-sizing working (+ refactor)
* Fix BE tests
* remove an old test case no longer needed
* lint
* remove final broken tests which is no longer needed
* lint, and clean up/refactor CSV code a bit
* clean up/refactor XLSX code
* remove debug symbol
* fix whitespace
* address feedback
* Add missing t to error messages
* Pull out lexification from diagnoseSyntaxChecks
* Add hooks to compiler
* Define diagnose and diagnoseAndCompile using compiler hooks
* Add maybe helper to throw the first error
* Use maybe helper to make sure parsing and lexifying throws
* Remove throwOnError argument to parser
* Add error hook
* Use error hook to make lexifier and parser throw
* Pass token and source in object
* Return position information for mismatching parenthesis
* Add length to Token
* Handle unknown string quotes
* Simplify lexifier
* Move malformed number check to diagnostics
* Move malformed string check to diagnostics
* Move malformed field name check to diagnostics
* Move bad token check to diagnostics
* Stop returning errors from lexifier
* Merge nodes when unopened bracket occurs
* Add error helper for diagnostics
* Use error helper for diagnostics
* Remove unused helpers
* Remove unused ParseError
* Remove errors fuzz test
* Add default hooks to parser
* Remove maybe helper
* Split up checks into syntax and expression folders
* Format no-argument expression clauses with empty ()
* Suggest no-arg clauses with empty ()
* Remove fallback for non-arg functions in resolver
* Fix unit tests
* Fix e2e test
* Add test for no-arg suggestion completions
* Update examples in docs
* Render parens for no-arg clauses in helptext header
* Make function browser insert parens for no-arg functions
* Do not show help text when a function call is not made
* Add repro for metabase#55300
* Fix function browser e2e test
* Remove unnecessary branch
* Remove fallback for non-arg functions in resolver
* Add () to now in one more location in docs
* Fix unit tests
Fixes#56434.
A join clause's `:fields` might contain refs to columns that no longer exist on the joined table/card.
Before this change, a malformed column was being passed to `with-join-alias`, causing it to throw.
Now the bad fields are dropped from `returned-columns`, and if they all get dropped then it is treated as `:all`.
Resolves PRG-10.
This commit keeps the DOM node that we use for downloads
deterministic, while ensuring that we always target the correct
chart in the dashboard.
The same question can be added to the dashboard multiple times
so the only unique identifier we can use is the dashcard id.
But this brings us to the weird situation that the node we're sending
to the canvas to render before downloading is different when we're
downloading the chart from a dashboard vs when we're downloading
it directly from the question page.
This pseudo DOM code shows what we did before:
Dashboard:
```
dashcard-id <--- we were grabbing this
visualization-root
card-id
```
Question:
```
div
visualization-root
card-id <-- we were grabbing this
```
Compare that to what we're doing now
Dashboard:
```
dashcard-id
visualization-root
card-id <--- we are now grabbing this*
```
* but only when it is under the specific dashcard-id
Question:
```
div
visualization-root
card-id <-- we continue grabbing this like before
```
* Fix literal-expressions-inside-nested-and-filtered-aggregations-test
* Use mt/formatted-rows in order-by-integer-literal-expression-test
* Support selecting an empty string literal in select-clause-deduplicate-aliases
For drivers like Oracle that inherit from ::sql.qp.empty-string-is-null/empty-string-is-null, if a user adds an empty
string as a custom literal expression, that literal "" value will get compiled to nil, so ensure that
select-clause-deduplicate-aliases can handle such nil values in the :fields list.
Basically, we want to ensure that a query like
{:expressions {"empty": [:value ""]},
:fields [[:expression "empty"]]
:limit 1}
gets compiled to
SELECT NULL [AS] "EMPTY" ...
* Add sql.qp.expression-literals/boolean->comparison abstract driver
This is similar in spirit to the existing sql.qp.empty-string-is-null driver and can be used for drivers like oracle
and sqlserver that require booleans to be converted to comparison expressions in logical operators like :and, :or, and
:not, and in the top-level of a :filter clause.
* Enable :expression-literals for sqlserver
* Enable :expression-literals for oracle
* Review feedback: not -> nil?
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Implement ->honeysql :sum-where and :distinct-where in terms of :case for :sql driver
Change the ->honeysql methods for :sum-where and :distinct-where to build an MBQL :case form and compile that, rather
than building a honeysql :case form directly. In this way, drivers that need special handling for :case will get
:sum-where and :distinct-where for free. Note that :count-where was already implemented this way. See, e.g. the
:metabase.driver.sql.query-processor.expression-literals/boolean->comparison abstract driver which converts booleans
in the case conditions to equivalent := expressions.
* Fix typo in test: case clause -> expression
* Rename boolean->comparison abstract driver to boolean-is-comparison
Rename
::sql.qp.expression-literals/boolean->comparison to
::sql.qp.boolean-is-comparison/boolean-is-comparison
This makes it clear that the namespace should only contain the abstract driver, and is also more consistent with the
existing driver in sql.qp.empty-string-is-null.
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
the view_count and last_view_at updates of report cards can deadlock
against each other, so introduce a shared keyword they can use to each
take a cluster lock when updating these values to prevent the deadlocks.
* Adjust missing base type logging for jdbc drivers
* Adjust base type logging for Athena
* Add schema and table to missing base type logging
* Comments
* Update test
* Drop only preceding nils
* Use Mantine instead of styled components
* Fix centering
* Add task and status params
* Add TaskStatusPicker
* Make picker clearable
* Add task picker
* Put filters and pagination together
* Translate task status
* Break down logic
* Move loading and error state
* Always render something
* Make task picker searchable
* Add empty state to the table
* Fix clear button being transparent
* Add useUrlState
* Use useUrlState in TasksApp
* Move UrlState to its own file
* Sort
* Rename file
* Rename
* Clean invalid query params on mount
* Prettier
* Use Action
* Ignore invalid pages
* Reuse variable
* Fix tests
* Add test
* Add label to clear button
* Remove redundant condition
* Add assertions
* Improve typing
* Be explicit about empty string handling
* Add sorting params to ListTasksRequest
* Add sorting params to url state
* Render icon for the first column
* Add SortableColumnHeader
* Extract TasksTable
* Move use-url-state to its own directory
* Add unit tests for useUrlState
* Add test
* Refactor
* Add more tests
* Fix missing import
* Add task endpoints mocks
* Add TasksApp tests boilerplate
* Add pagination test
* Add tests for filters
* Add more tests
* Tmp
* Make SortColumn generic
* Use SortingOptions
* Remove unused type
* Make BaseItemsTableProps non-generic
* Integrate sortingOptions
* Prevent CLS
* Prevent table content from disappearing on sort or filter
* Improve typing
* Use isLoading
* Get fresh data
* Update comment
* Reset page on sorting change
* Update tests
* Add tests for page reset
* Update tests
* Mark test as flaky
* Reuse SortingOptions
* Extract getFirstParamValue
* add tests for mb-wrapper leaks
* revert me - temporarily check diffs
* Revert "revert me - temporarily check diffs"
This reverts commit f474fe7b25.
* only apply mb-wrapper when in the sdk
* use cy.log to show e2e messages
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Remove legacy expression in the return type from compiler
* Move test cases to formatter tests
* Move op and opt helpers to test utils
* Move value helper to test utils
* Rewrite compiler tests to use expression parts
* Remove unused test helpers
* Use actual invalid metadata instead of broken references
* Remove Lib.legacyExpressionForExpressionClause and Lib.expressionClauseForLegacyExpression
* Remove unused mock in fuzz test
* Remove legacy-expression-for-expression-clause and expression-clause-for-legacy-expression helpers
* Merge __support__ and test folders
* Reuse query from shared testing data
* Remove legacy-expression-for-expression-clause and expression-clause-for-legacy-expression tests
* Fix import
* Rename util to utils
* float() tests + tier 1 (postgres)
* Add more parsing test.
* Better float comparison
* Add expression/float feature
* Add float function to config
* Add float help text
* Add e2e test for float()
* Add `:expressions/float` to list of feature flags
* Only allow casting text to float (not number)
* Better test message
* Refactor for nicer code
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
To avoid deadlocks on postgresql appdbs caused by overlapping ids in the
in clause of these updates.
I also had to change the `call-count` implementation for one of the
tests to spy on t2/query instead. The call-count pipeline middleware
will register either 2 or 4 depending on if we need to insert rows into
the lock table or not.
Fixes QUE-755
Since killing connections this way is the mechanism we use to terminate long running JDBC driver queries, this timeout
shouldn't differ from the db-query-timeout-minutes setting.
* Resolve dimensions while compiling
* Use resolver in pratt compiler instead of resolver pass
* Remove legacy resolver
* Add missing imports
* Add startRule to compiler in test
* Use helper to calculate argtype
* Provide a default implementation of argType
* Return expressionParts from compiler
* Convert resolver tests to use compiler
* Remove superflouous comment
* Convert MAP_TYPE to a function
* Rename field-resolver to resolver
* Simplify resolver check
* Move resolver-related code into resolver module
* pinned questions show moderated status, handle their own caption title
* unit tests, description
* fixing grid spacing
* e2e test adjustments
* fix for verified skeletons
* update quetion embed test, dashboard embed test
Fixes [QUE-701](https://linear.app/metabase/issue/QUE-701), [QUE-778](https://linear.app/metabase/issue/QUE-778), [QUE-777](https://linear.app/metabase/issue/QUE-777), [QUE-776](https://linear.app/metabase/issue/QUE-776), [QUE-703](https://linear.app/metabase/issue/QUE-703), [QUE-704](https://linear.app/metabase/issue/QUE-704).
### Description
Directly implements `:ident` calculation for legacy MBQL in the QP!
Also changes the spelling of compound idents to eg. `join[JoinClauseIdent]__InnerColumnIdent`, `native[CardEntityId]__COLUMN_NAME`, etc. The advantage of this is that NanoIDs cannot contain `[]`, whereas they do contain `_`s. Then we can make sure there's no missing `entity_id`s (a common issue) by looking for `native[]`. `native____` might have happened by accident is the NanoID started with two underscores.
This is an unfortunately large PR, but I don't think much can be done to help that.
The output of `annotate` becomes `Card.result_metadata`, which is a key **input** to
`annotate`. So we have to tackle legacy metadata end to end to produce a healthy PR.
(A prior attempt to use the lib and match up the results turned into a cluster of hacks,
and required half of these changes anyway, so this direct approach seems more sound.)
Most of this logic lives in the `annotate` post-processing middleware, with
some support from `add-source-metadata` pre-processing.
#### Idents
This generates correct idents for proper fields, expressions,
aggregations, breakouts, explicit and implicit joins, native queries and
models (native and MBQL).
#### Safety checks and Guarantees
Also adds and strengthens several safety checks in the library and
elsewhere. The guarantees I hope we can count on at this point are:
- Cards are saved with correct `:ident`s in their `:result_metadata`.
- Cards with old `:result_metadata` get (correct) `:ident`s added on read.
- Lib can now rely on getting correct idents for all input cards!
- Therefore the lib (BE + FE) can compute the ident for **every column**.
#### Placeholder idents
For native queries, we can only get `result_metadata` from executing them.
The usual flow is:
- Write the query
- Send it as an ad-hoc `/api/dataset` request
- Get back `result_metadata`
- Save it, passing the `result_metadata` back to the BE.
We need a card `entity_id` to create idents for a native query, but an ad-hoc
query doesn't have a card yet! The BE now generates a placeholder `entity_id`,
which gets replaced on save with the new card's actual `entity_id`. These
placeholders contain (1) illegal NanoID characters, and (2) a NanoID, which
means they cannot be mistaken for non-placeholder NanoID, and that they are
still globally unique, so there's no caching concerns.
* Move diagnostics to separate dir
* Use correct type for startRule arg
* Split up diagnostics into sepearate files per check
* Use throw for checks that don't yet
* Pull out syntax checks into separate function
* Split up lib.diagnoseExpression into helper
* Return expressionParts from compiler
* Add vist helper that walks Lib.ExpressionParts
* Convert check-known-functions to use Lib.ExpressionParts
* Convert check-supported-functions to use Lib.ExpressionParts
* Convert check-comparison-operator-args to use Lib.ExpressionParts
* Convert check-arg-count to use Lib.ExpressionParts
* Convert check-case-or-if-arg-count to use Lib.ExpressionParts
* Convert check-arg-validators to use Lib.ExpressionParts
* Remove legacy visitor
* Also pass expression clause to diagnostics
* Use pMBQL in diagnose-expression
* Stop passing expression to diagnostic checkers
* Rename visit to visitor
* replace the export button in dashboard header
* use the same download button style for both
* add styles for compact download button
* change the compact export button style
* add download button logic
* update data-testid
* update dashboard tests in public-resource-downloads
* use getDashboardCard helper instead
* move question result download to header
* remove PDF export localized test
* add PublicOrEmbeddedDashboardView loki diffs
* use text-primary instead of text-dark for download icons
* simplify downloadAndAssert embed logic
* simplify classnames merging
* revert ExportPdfMenuItem
* cleanup TitleAndDescriptionContainer
* revert LegendCaption
* show the header if headerButtons is not null
* standalone question download results only shows up on card hover
* always show header if there is a header button
* add floating headers when title is hidden
* add comment on floating header buttons
* use accessible label selectors instead of data-testid
* replace question-results-download-button testid with accessible label
* cleanup downloadAndAssert logic
* update tests
* support shared selector
* update unit test for PublicOrEmbeddedDashboardPage.enterprise
* update unit test for PublicOrEmbeddedDashboardPage.premium
* update unit test for PublicOrEmbedded
* update localization tests
* localize aria-label and re-add label tests
* make aria-label "Download as PDF" for all instances
* fix question tests on locale hash params
* fix tests on disabling downloads
* add realHover case to downloadAndAssert
* update testid in LegendDescriptionIcon
* update public-resource-downloads cypress test
* use visibility checks in tests
* restore ellipsis button size
* restore hover parent state to standalone question
* update localization tests that referenced PDF export button
* remove eslint-disable directives
* update ChartSkeleton
* update ChartSkeleton
* lift spacing to LegendCaption
* use lighter colors for dashcard downloads
* add new loki snapshots
* use the auto refresh text for locale tests as pdf export text is gone
* use the lighter color for all download icons
* add new loki snapshots
* update test assertions for locale tests
* fix(sdk): use instance locale if no locale is passed
* update docs
* we do support country codes, we have tests for it
* Update e2e/test-component/scenarios/embedding-sdk/locale-on-provider.cy.spec.tsx
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* remove manual fallback to "en", site-locale is never undefined as it has
a default
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Avoids hitting the postgresql query parameter limit when there are more
than MAX_UINT16 tables in a database, also should be more efficient to
fetch this in a single query.
* Use Mantine instead of styled components
* Fix centering
* Add task and status params
* Add TaskStatusPicker
* Make picker clearable
* Add task picker
* Put filters and pagination together
* Translate task status
* Break down logic
* Move loading and error state
* Always render something
* Make task picker searchable
* Add empty state to the table
* Fix clear button being transparent
* Add useUrlState
* Use useUrlState in TasksApp
* Move UrlState to its own file
* Sort
* Rename file
* Rename
* Clean invalid query params on mount
* Prettier
* Use Action
* Ignore invalid pages
* Reuse variable
* Fix tests
* Add test
* Add label to clear button
* Remove redundant condition
* Add assertions
* Improve typing
* Be explicit about empty string handling
* Add url state to logs
* Add download button
* Disable the button instead of hiding it
* Add query input
* Add clear button
* Improve copy
* Typo
* Fix unit tests setup
* Improve empty state message
* Support location in setup
* Add a test for empty state message when filtering
* Add a test
* Add download button assertions
* Add e2e tests
* Simplify assertion
* Remove redundant element
* Add debounce
* Fix timezone issue
* Extract TasksTable
* Move use-url-state to its own directory
* Add unit tests for useUrlState
* Add test
* Refactor
* Add more tests
* Fix missing import
* Add task endpoints mocks
* Add TasksApp tests boilerplate
* Add pagination test
* Add tests for filters
* Add more tests
* Update tests
* Update tests
* Add more assertions
* Use Mantine instead of styled components
* Fix centering
* Add task and status params
* Add TaskStatusPicker
* Make picker clearable
* Add task picker
* Put filters and pagination together
* Translate task status
* Break down logic
* Move loading and error state
* Always render something
* Make task picker searchable
* Add empty state to the table
* Fix clear button being transparent
* Add useUrlState
* Use useUrlState in TasksApp
* Move UrlState to its own file
* Sort
* Rename file
* Rename
* Clean invalid query params on mount
* Prettier
* Use Action
* Ignore invalid pages
* Reuse variable
* Fix tests
* Add test
* Add label to clear button
* Remove redundant condition
* Add assertions
* Improve typing
* Be explicit about empty string handling
* Extract TasksTable
* Move use-url-state to its own directory
* Add unit tests for useUrlState
* Add test
* Refactor
* Add more tests
* Fix missing import
* Add task endpoints mocks
* Add TasksApp tests boilerplate
* Add pagination test
* Add tests for filters
* Add more tests
* Convert pratt compiler to use ExpressionParts
* Simplify compile by using compileNode helper
* Add todo's about casts
* Simplify error creation
* Make assertions more consistent and translate error messages
* Remove any check
* Simplify compiler by moving stuff into infix/unary helpers
* Remove unused assertions
* Remove casts on constant operator names
* Add assertions for operators
* Wrap top-level literals in a value clause
* Handle bigints in pratt compiler
* Accept ExpressionParts in Lib.expressionClause
* Remove null from ExpressionArgs
* Add field and metric to expression operator
* Simplify pratt compiler types
* Add more tests for equivalent clauses
* Return metadata objects directly from field-resolver
* Add more operators to type
* Create type for field type
* Remove resolverPass
* Convert resolver to Lib.ExpressionParts
* Convert compiler to expressionParts
* Fix syntax error
* Fix types
* Remove unused passes
* Fix resolver tests
* Add types to top-level literals
* WIP: convert untagged expression-parts from #js
* Manually unwrap field, segment and metric metadata
* Unwrap options in pratt compiler
* Remove empty options
* Handle normalized expression when diagnosing
* Allow customizing field resolver
* Fix fuzz tests
* Add value to ExpressionOperator
* Remove unused TODO's
* Use constants in test so types match
* Wrap clauses in defineClause and extract list of defined clauses
* Forbid setting name on clause config
* Use defined clause name as ExpressionOperator
* Use getClauseDefinition helper to get clause definitions
* Ignore type in filters
* Move expression-parts transformation to lib.js
* Add type-of method implementations for field, metrics and segments
* Trigger change when fast setting custom expression
* Remove superflouous casts
* Simplify resolver return type
* Use reference for top-level fields
* Be more specific about types in compiler
* Convert operator to string
* Add unit test for specific expressions
* Fix literals icons
* Remove unused matchers
* Remove unused option from compiler
* Negate number literals directly in compiler
* Add expression-clause implementation that takes one argument
* Use single-arg version of Lib.expressionParts
* Remove TODO
* Use helper to compile top level literal
* Add effective-type to top-level literals
* Remove unused MBQLCLauseMap type
* Remove unused import
* Add two-argument version of expressionClause
* Wait for button to be visible
* Add operator to invalid operator assertion
* Remove incorrect operator check
* Add more specific type to clause name
* Always throw if function name is not found
* Add overloads to getDefinedClause for known function names
* Document the allowFastSet typing hack
* Lift operator check
* Convert nil to {} in body
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Make expression-parts js conversion more clear
* Pass through lib.metadata.calculation/type-of for the type-of implementations
* Always throw assertion errors
* Use helper to check query invariants
* Remove lib.schema.expression/type-of methods
* Add test for normalization of uppercase keyword arguments
* Allow some normalizers to ignore case
* Use threading for lowercase converter
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Structure test better
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Remove unused var
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Log metric data for failures in qp.middleware.metrics/adjust
When qp.middleware.metrics/adjust fails to expand a metric, log the metric's name and dataset_query, not just the
metric id.
Closes QUE-862
* Update test to assert that metric data is present in ExceptionInfo
* wip script to build the frankenjar
* build/run/test functions that seem to be working
* preparing for ci
* wip ci workflow
* need to run the job "manually" at least one to be able to dispatch it
from the gh cli
* add workflow_call
* bash instead of sh to solve issue of illegal option -o pipefail?
* use correct version of node
* prepare backend
* more envs to ci
* make jar run on the background in ci
* trying to run a list of specs in ci, tagging with tags probably won't
work as tags potentially need to be there on "older" versions of the
test
* fewer tests so it maybe doesn't timeout
* run on prs targerting the pr branch and the target base one
:wq
* clean up
* fix: without --env it wasn't really using the grepTags param:wq
* refactor: check for the actual files instead of doing ls*
* add back some more test files
* add log for ci
* remove colors as they're not shown in ci
* test: wait for data to load in a flaky test
* test: wait for data to load in a flaky test
* make sure we're building the EE jar
* skip licnese step as we don't need it
* move script out of embedding-sdk folder
* make curl -s to spam less in ci
* refactor: move jops to keep the result job last
---------
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
* Correct output for dashboard filter variables
* Whitespace
* Add node script for testing clojurescript
* Enhance format-unit and add parse-unit.
parse-unit is new.
format-unit now works with moment (cljs) or java.time (clj) values, not just strings and numbers. I also added two new
units (:day-of-week-abbrev and :month-of-year-full).
* Refactor to use time/parse-unit and time/format-unit
this happens because we're not hashing `now` in this case, but what is coming from DB; which is OffsetDateTime and can
have slightly different precision, leading to different hash.
* Replace confirm content in use-confirmation hook
* Ensure modal closes after confirmation
* Update to mantine popover
* Toggle closeOnClickOutside when modal is open/closed
Fixes#51175
BigQuery sometimes returns timestamp values as floating point values (as strings, of course). We used to pass the
parsed double value to java-time.api/instant, but that seems to have microsecond precision. In this change we use
Java's built-in Instant construction function to keep the full precision.
* Add repro for #56596
* Allow escaping backslash
* Use unquoteString everywhere to parse strings and references
* Do string quoting and unquoting more consistently
* Add repro for #56596
* Make string parsing more straightforward and consistent
* Use quoteString to find errors with quoting
* Type string helpers more strictly
* Add fuzz tests for string quoting
* Move fuzz helper to test folder
* Make string fuzzing repeatable
* Add sanity check
* Fuzz all three quoting modalities
* Add [-quote tests
* Rename quote to delimiter
* Fix path for lexifier fuzz testing
* Fix name for compiler fuzz testing
* Use constant to store delimiter pairs
* Fix option name in formatter
* Add more cases around half-quoted strings
* Rename quote to delimiter in comment
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add isExpressionParts helper to check
* Export Lib.isColumnMetadata
* Add Lib.isColumnMetadata to check
* Mark is-column-metadata as Single use instead of smelly
* Add is-metric-metadata helper
* Add is-segment-metadata helper
* Add isSegmentMetadata and isMetricMetadata to checks
* Use expressionParts and expressionClause in formatter
* Avoid stringifying the ast
* Allow passing literals to filter-clause
* Simplify test case for now
* Handle [:value x] in formatter
* Unnest expression-parts
* Add metric-clause? and segment-clause? helpers
* Handle metric and segment clauses in expression-parts
* Flatten case/if statement args in expression-parts
* Remove special case for case/if in formatter
* Support keywords in expression-parts
* Fix duplicate negation test case
* Prefer desctructuring over get
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Use set to match case and if
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Fix typo
* Use conditional threading in maybe-expand-*-expression
* Use destructuring to match case/if clauses
* Use cond-> instead of maybe-expand- helpers
* Fix literal in formatter tests
* Rename is-*-metadata to *-metadata?
* Swap test case order
* Use multi-method for expression-parts
* Inline recurse fn
* Accept ExpressionClause in the formatter
* Pass expression clause in HighlightExpression
* Build expressionParts for examples manually
* Be more strict in formatter typing
* Be more strict in formatter typing
* Fix FunctionBrowser example
* Add test case for 1 + 1
* Pass expression clause to format in Editor
* Add docstring to expression-parts-method
* Remove superflouous check
* Use threading macro in column-metadata-from-ref
* Unwrap case/if into pairs and a fallback argument
* Remove reference to unknown function
* Add stage-index-less version of expression-parts
* Add MetricMetadata and SegmentMetadata to ExpressionArg
* Fix case/if expansion not to have nil argument when fallback is nil
* Add test for case and if
* Fix case/if defop
* Add more test cases to case and if test
* Add ExpressionArg type
* Allow passing Expressionable to formatter
* Add more tests for case and if
* Move case and if conversions to fe_util
* Deduplicate case-or-if matcher
* Document case-or-if helpers more
* Be more precise in the type of ExpressionParts args
* Fix args for expression-clause in exclude-date-filter
* Remove faulty test
* Add test for nested parts
* Move fix-expression into ->op-arg
* Move ensure-uuid into fix-expression
* Wrap operator in keyword
* Correctly recurse through expression-parts
* Add test for nested case parts round-trip
* Add source-field to test data
* Comment out broken test case
* Add source-field to ReferenceOptions type
* Remove unused imports
* Allow more arguments to expression-parts
* Use expression-parts in filter test
* Handle bigint in formatter
* Add missing import
* Remove unused import
* Normalize mbql
* Allow lib.expression/expression as arg to expression-clause
* Handle expression refs
* Add tests for unknown fields, metrics and segments
* Add metrics to formatting tests
* Handle non-existent metrics and segments
* Add workaround for unknown fields
* Use cond-> and destructuring for unknown field
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Remove superflouous call to lib.field/extend-column-metadata-from-ref
* Inline segment-metadata-from-ref and metric-metadata-from-ref
* Collapse all identity expression-parts-methods in a doseq
* Call expression-parts instead of expression-parts-method
* Use multi-method to handle case and if
* Pull out temporal expression expansion into separate multimethod
* Move check closer to where it is used
* Remove constant conditional
* Use threading macro in expression-parts
* Move nested case/if test out of deftest
* Fix formatting
* Fix metadata method when metric is not found
* Add display-name to unknown field stub
* Use lib.metadata helpers directly for field, metric and segments in expression-parts
* Fix e2e test
* Handle segments in filter parts directly
* Fix e2e test
* Add TODO
* Use doseq in literals test
* Add test for bigint value clause in expression-parts
* Remove TODO
* Use id from sample database
* Use vec
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Fix types
* Use simpler assertion helper
* Move metric to aggregations tests
* Do not wrap value clauses in parentheses
* Return metric metadata from expression-parts
* Emit expression reference in the case of expressions
* Strip expression-name from columns metadata in expression-parts
* Add e2e test for nested references
* Return stub metric metadata if metric is not found
* Use expression-name from the reference
* Add tests for field reference in expression-parts
* Add tests for segment reference in expression-parts
* Add tests for metric reference in expression-parts
* Add tests for expression reference in expression-parts
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Support drivers that need special handling for literal string expressions
* Enable the :expression-literals feature for vertica
* Enable the :expression-literals feature for redshift
* Review feedback: lift comment about inlined literals from h2 to sql driver
- if you have bb on your path, then this would never be hit.
Testing by uninstalling bb will not uncover this issue if you have
repo/path/metabase/bin on your PATH!
* Rename True and False expressions in literal expression tests
Speculative fix for sqllite failing the tests.
* Update literal expression tests to work with drivers that treat empty string as null
* Add empty-string-literal-expression-test and remove empty string form standard-literal-expression-defs
* Fix logical merge conflicts in case-with-literal-expression-test
* Enable the :expression-literals feature for sqlite and starburst
* Review feedback: drop format-nil->empty-str
As pointed out by metamben in code review `(str nil)` is already the empty string. In fact, the culprit here was a
missing `format-nil-values?` arg to `format-rows-by`.
* Add route prop to disable command palette, Disable in setup routes
* increase wait timeout
* refresh user on setup
* Lets try a unit test instead
* removing refresh user from UserStep
We need to ensure that we consistently have entity-ids in metadata (and potentially elsewhere) even before we've finished backfilling them in. To do this, we check for entity-ids in the field/table/database/card after-select and run calculation/deduplication logic to get them when necessary. We then cache the results (because these calculations can be slow), and a separate job then drains the cache and writes the results back to the db.
* Migrations for database sandboxing (#53125)
Two changes here:
- add a self-referential foreign key,
`metabase_database.router_database_id`, to allow marking a mirror
database's router database. Databases with this set are mirror
databases, databases without it are primary databases.
- add a `db_sandbox` table with only three columns:
- `id`, primary ID,
- `db_id`, the `metabase_database.id` this sandbox is for, and
- `user_attribute`, the user attribute used to map users to a
particular mirror database.
* CRUD API for database routing (#54102)
* Migrations for database sandboxing
Two changes here:
- add a self-referential foreign key,
`metabase_database.primary_database_id`, to allow marking a mirror
database's primary database. Databases with this set are mirror
databases, databases without it are primary databases.
- add a `db_sandbox` table with only three columns:
- `id`, primary ID,
- `db_id`, the `metabase_database.id` this sandbox is for, and
- `user_attribute`, the user attribute used to map users to a
particular mirror database.
* Bulk endpoint for creating mirror dbs
The error case is a bit wonky right now, it just returns the errors for
the details that failed... without the details or name.
Co-authored-by: Ed Paget <ed@metabase.com>
* Add endpoint for marking a database as a RouterDB
Co-authored-by: Ed Paget <ed@metabase.com>
* Rebase fixes
* Exclude mirrordbs from regular db endpoints
* kondo fixes
* mark `DatabaseRouter` as not serialized
* serdes fix
* copy fix
* delete and update routers
Also reorganized things to be a bit less cattywompus.
Now we have:
- `POST api/ee/database-routing/mirror-database` to create a new mirror database
- `POST api/ee/database-routing/router` to create a new router using an
existing database
- `PUT api/ee/database-routing/router/:id` to update a router to use a
new user attribute
- `DELETE api/ee/database-routing/router/:id` to delete a router and all
of the associated mirror databases.
* GET endpoint for mirrors of a particular router db
* Don't test database conn for mirror dbs
---------
Co-authored-by: Ed Paget <ed@metabase.com>
* Modify the API - don't care about database routers (#55036)
* Modify the API - don't care about database routers
Add a `router_user_attribute` to the `GET /api/database/:id` API
endpoint.
Modifies the database routing API to address only by the database ID.
There's now just two endpoints:
- POST /ee/database-routing/mirror-database to create a mirror database
- PUT /ee/database-routing/router-database/:id to change the
`user_attribute` used for routing an existing database. Can be set to
`null` to turn of database routing for the database.
* fix test
* kondo
* DB Routing: Sloan's API improvements (#55438)
* router_user_attribute on `/database/`
* unique database names for mirror dbs
* Don't delete mirror databases when deleting router
* query param to check conn details
* no making a mirror a router
* Require `:manage-database` perms to get dest dbs
(rather than superuser.)
* set `visible-if`, NA on dest dbs
* Add conditions when making db router
* fix db migrations
* fix tests
* kondo
* reorder migrations for mysql/mariadb
* Implement database routing (#55053)
* Swap DB for DB routing
Caching should work, field values does not currently.
* Rename the metadata replacement dynamic var
for clarity. Since we're using it outside of tests now.
* Refactor Advanced Field Values
Previously we had 3 types of Advanced field values:
- sandbox,
- impersonation,
- and linked-filter
Each of these three had their own function, `hash-key-for-*`, and the
caller of this function was responsible for generating the correct hash
key for the field provided.
For example, `hash-key-for-advanced-field-values` took a `fv-type` and
called either `hash-key-for-linked-filters` or `hash-key-for-sandbox` or
`hash-key-for-impersonation` depending on the type.
The big problem here is that the hash keys aren't composable. You can
see this being an issue in `hash-key-for-linked-filters`, where the
enterprise version needs to also check whether the field is sandboxed,
and if so, include sandboxing data in the generated hash as well as the
linked filter data.
If we add more orthogonal things to the hash (like database routing) we
would end up with a rat's nest where each hashing function partially
implements a bunch of other hashing functions to make things work. E.g.
`hash-key-for-database-routing` might check if the field is sandboxed,
and if so include data about the sandbox, but also
`hash-key-for-linked-filters` should check if the field is
sandboxed *or* impersonated *or* subject to db-routing and include data
about those, and ...
Instead, I've replaced all these functions with `hash-input-for-*`
functions, which return self-contained maps that reflect
their *internal* knowledge of what should/should not be cached.
For example, the `hash-input-for-sandbox` function returns either
`nil` (if the field is not sandboxed for the current user) or
`{:sandbox-attributes {<the user attributes used for the sandbox>}}` (if
it is).
`hash-input-for-field-values` simply merges all of these together into
one lump.
To tell whether we're generating a `full` or `advanced` field value, we
can just check to see whether the hash input has any data other than the
field ID.
The one downside here is that we'll no longer be able to look at the DB
and distinguish between different types of FieldValues. As far as I can
tell, the code itself does not ever do this, so I think this is an
acceptable tradeoff.
* database routing for field vals
* move things around a bit
* fix tests
* Lint
* WIP: database routing
* fix test
* more tests, caching tests
* kondo
* stray #p
* Exclude mirror databases from search
* Exclude mirror databases from sync
* Do not create permissions for mirror databases
* fix more tests
* lint
* Add another e2e test for field values
* Don't grab database from query in h2 driver
* Fix my rebase error
* [DB Routing] Implement UI (#55132)
* save
* wip
* wip
* lots of progress
* iteration
* more progress, add delete mirror modal
* type clean up, bug fixes, code organization
* address lots of TODOs
* prevent configuring uploads to routed dbs, prevent db routing if db is upload destination db, replace all usages of routed database and mirror database in favor of destination database
* clean up url usage, add health check to destination database lists
* support hidden fields for database details
* fix types and TODOs
* render warning that model features can not be enabled when db routing is enabled, add unit tests for DestinationDatabasesList
* add e2e coverage for db routing CRUD
* move db routing e2e to a new file, make tests nicer, add test coverage for not showing destination databases outside of database settings
* test that db routing can not be enabled if other conflicting features are enabled
* finish e2e tests as much as possible, add feature token to session properties payload
* move more code into enterprise folder, add redirects and permissions checks for destination db routes, handle no user attributes on instance state, fix failing unit tests, make alerts tooltips when features cant be enabled based on other features being enabled
* test fixes, impl error states kyle provided
* split up tests by ee and oss
* error state updates, fix failing e2e tests
* fix failing oss e2e test, beef up db management perms test
* PR feedback
* impl more feedback
* Fix BE tests
* another be test
* pr feedback
* fix one of the failing e2e tests
* include_mirror_databases -> router_database_id
* skip test
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* fix deletion of router dbs with dest dbs (#55831)
* Don't require superuser perms to get dest db (#55754)
* [DB Routing] Database pages tech debt - RTKQuery + TS migrations (#55744)
* convert DatabaseList to typescript
* make DatabaseEditApp use rtkquery, remove old initialize approach, remove related redux code, port code to use api db rather than metabase lib Database class
* remove dismissDatabaseSyncSpinner redux code in favor of rtkquery
* fix some mistakes causes tests to fail
* port DatabaseList to typescript
* port adding sample database to rtkquery
* misc fixes
* fix failing unit test
* convert admin/databases/database.js and admin/databases/selectors.js to TS
* editParamsForUserControlledScheduling.js to editParamsForUserControlledScheduling.ts
* remove all usages of metabase lib Database class from Database pages
* fix type errors
* fix failing e2e
* prettier
* [DB Routing] Port Admin DB page components to Mantine (#55811)
* port components to mantine for admin db pages
* fix failing tests
* remove unused comment
* fix type issue
* Require opt in/out of database routing (#55841)
I was considering possible errors here and I think this will work well
as a safety harness.
In dev/testing (non-prod) environments, let's add an assertion to the
JDBC driver that if we're about to connect to a database. Either:
- we're *explicitly* trying to only connect to Router Databases. For
example, sync should *never* connect to mirror databases or go through
Database Routing. OR
- we've enabled Database Routing, and we *don't* want to access the
Router Database (unless of course DB routing has been configured to send
us there)
Another way of putting this is that there are two obvious ways for this
feature to go sideways: we accidentally sync a mirror database or do
something else that shouldn't work for mirror databases, OR we
accidentally skip the Database Routing process and connect to the router
database when we shouldn't.
Of course, we can't guarantee that all drivers will call
`(database-routing/check-allowed-access!)` before connecting! But we can
just call it from the JDBC router and that should be sufficient to catch
any issues in development in the vast vast majority of cases.
* [DB Routing] UI feedback part 1 (#55893)
* make dest db validate on creation, invalidate cache of user attributes after user is updated
* add test coverage for updates and uncomment e2e tests parts that were previously failing
* fix type issue
* Fix audit/query logging for DB routing (#56103)
[ADM-585: Handle audit logging and query logging](https://linear.app/metabase/issue/ADM-585/handle-audit-logging-and-query-logging)
* exclude destination dbs from serialization (#56076)
* Decrease the pool size for mirror DBs (#56104)
DB metadata seems like an odd thing - it's the Metabase Database
converted to a snake-hating map, so `router_database_id` becomes
`router-database-id`.
We can't write a function of `:model/Database` that decides whether
we're looking at a mirror database (because we're not looking at a
`:model/Database` here). So we're reduced to just checking the key.
Feels weird, but I think the cleanest way available here?
* Fix sandbox on a routed DB with question as filter (#56393)
This only affected turning on sandboxing, and was not an error for
sandboxing itself.
When we configure sandboxing, we want to make sure the column types for
the query we're using match the columns in the sandboxed table.
As part of this, we ran the query anonymously with
`(request/with-current-user nil ...)`. With Database Routing, however,
we added a new constraint: you can't run any queries against a database
with routing configured anonymously.
Instead just run the query as a superuser. This is effectively the same,
and it's consistent with what we do when actually *enforcing* the row
level restrictions anyway: we preprocess the source query as an admin
and then splice it into the user's query.
The one thing that gives me pause: I'm not really sure why we would have
done `(request/with-current-user nil ...)` in the first place here.
* Add database-router usage e2e tests (#56255)
* Improve error messages and UI for config (#56392)
* Improve error messages and UI for config
- improve error messages when a dest db can't be found
- improve the admin UI for configuring DB routing: https://metaboat.slack.com/archives/C07L6VCCYQL/p1744044752380479
* changes suggested by Jeff
* fix error msg tests
* disable linting for DB Routing description
* fix getting a destination db
db `can-read/write?` use router DB if necessary, since destination
databases don't actually have permissions saved.
* i18n for errors
* spelling 😁
* fix tests
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Fix options for Destination DBs (#56450)
* Fix options for Destination DBs
This may still require a bit of frontend work.
* Set destination-database=true for destination db modal
---------
Co-authored-by: Fraser McIntosh <fraser@metabase.com>
* Fix impersonation with Database Routing (#56503)
* Fix impersonation with Database Routing
Database routing was not working with impersonation, because by the time
the driver calls `set-role-if-supported!` the router database (which is
configured for impersonation) has already been swapped out for the
mirror database (which is not).
To fix this, have a new impersonation middleware set a dynamic var,
`*impersonation-role*`, which is preferred by `set-role-if-supported!`
if set. This way, in queries, `set-role-if-supported!` will use the
pre-calculated role (which seems better than computing it twice anyway).
Co-authored-by: Ed Paget <ed@metabase.com>
* Add impersonation tests
* Add sandbox question test
---------
Co-authored-by: Ed Paget <ed@metabase.com>
Co-authored-by: Fraser McIntosh <fraser@metabase.com>
* keep metadata store on metadata provider swap (#56502)
* keep metadata store on metadata provider swap
Before, we would blow away the existing QP store entirely when using
`*DANGER-allow-replacing-metadata-provider* to swap out the existing
metadata provider.
Instead we can only initialize the QP store when it was truly
uninitialized, and replace *only* the metadata provider when we're
trying to do that.
This is just in case something in QP post-processing needs data from the
original store, e.g. something that was stored with
`qp.store/store-miscellaneous-value!`.
Co-authored-by: Ed Paget <ed@metabase.com>
* only allow replacing md provider once
---------
Co-authored-by: Ed Paget <ed@metabase.com>
---------
Co-authored-by: Ed Paget <ed@metabase.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Fraser <fraser@metabase.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* feat: upgrade ring-jetty to 1.14.1
Moves us onto Jetty 12 which has some features (pthreading, QoSHanlder)
that we may want to take advantage of.
* fixup! customize jetty metrics
* fixup! license info for jetty ee9
* fixup! make a var private
* fixup! don't use deprecated method
* fixup! create custom statistics handler
Replicate the Jetty 11 statistics handler that is able to report on
asynchronously handled requests separately from synchronously handled
ones. Also ports it to directly report statistics to our prometheus
rather than needing to build a connector between the stats handler
object and our metrics reporting
Co-Authored-By: John Swanson <john.swanson@metabase.com>
* fixup! fix race condition in stats test
* fixup! missing private annotation
* fixup! use normal jetty release with logging fix
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* call e2e-component-tests-embedding-sdk-cross-version from
embedding-sdk.yml
* add e2e-component-tests-embedding-sdk-cross-version to the tests-results
so it needs to pass (or be skipped)
* Reset all errored triggers to `WAITING` on startup
Co-authored-by: Ed Paget <ed@metabase.com>
* Add explanatory comments
---------
Co-authored-by: Ed Paget <ed@metabase.com>
Metabase jars would crash when running on JVM 24 because the getReourse
method on the ClassLoader object was changed to return null if there is
no file at the provided path. This adds a ".keep-me" file to the
resources directory that Metabase can use to check if it is running in a
jar.
* fix: resolve snowflake sync cost issue
Using information_schema queries will cause snowflake to wake an
instance and thus incur costs. Using `show` queries will not do so.
In addition this adds a new multi-method for manipulating the results of
`describe-fields-sql`.
* Fix database-position and sorting
* Fix kondo
* Add order-by to mysql describe-fields
* Sort deps
* Use identity not map identity
* Fix databricks
* Address PR comments
* scaffold separate download options for dashboard static embeds
* parse the display options to hash options
* add toggles to the embed preview
* workaround for duplicate types
* cleanup the dashboard workaround
* cleanup the dashcard workaround
* downloads parameter should not show up on OSS
* replace the download button tests
* update existing tests
* fix test for Download buttons
* remove unnecessary fallback
* add a new snowplow schema
* update unit tests
* fix snowplow test assertions
* add snowplow tests
* use results instead of dashcard for consistency
* make the JSON schema backwards-compatible
* split download settings into two components
* use downloadsEnabled.results instead
* snowplow type must be results, not dashcard
* revert StaticEmbedSetupPane changes as it is unused
* create a new field for snowplow tracking
* Enable expression literals on drivers that pass tests
Closes QUE-780
* Update .devcontainer/Dockerfile for java 21 and node 22
This Dockerfile is apparently used by the "metabase assembler", which in turn is used by
.github/workflows/presto-kerberos-integration-test.yml to run the Presto Kerberos tests.
* Add case-with-literal-expression-test
* Review feedback: remove extra whitespace
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* feat: make cluster locking reusable
Builds a more generic interface for cluster locking using our existing
retry mechanism and jdbc level timeouts that will work with any database
rather than just postgresql.
* fixup! handle alt error text on mysql/mariadb
* fixup! don't lock on h2
* fixup! move malli annotations to retry
* Store all native editor selected ranges in redux
* Pass all selected ranges to redux
* Add repro for metabase#54799
* Fix unit tests
* Fix typo
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Do not intersperse space in when running selected text
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
This test was flaking if it was run close to the boundary of a UTC day
where the models could be created one day and then the clock ticked over
to a new day before the query ran.
* Don't update card metadata if it has not changed
* Store stored-metadata as a different field on info
* Fixed failing tests
* Update src/metabase/query_processor/middleware/results_metadata.clj
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
* Addressed review comments
* Update src/metabase/query_processor/middleware/results_metadata.clj
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Addressed review comments
* Update src/metabase/query_processor/middleware/results_metadata.clj
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Addressed review comments
* fixup! exclude info map from legacy normalization
* fixup! test that metadata equality skips update
* fixup! make adding the info map conditional
* fixup! use qp.store instead of adding to info map
* fixup! remove #p
* fixup! remove cyclic dep
* fixup! move load and store fns to same ns
* fixup! test name
---------
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Co-authored-by: Edward Paget <ed.paget@gmail.com>
The new `:card_schema` field on `:model/Card` was wrongly excluded from Revisions.
This PR includes them, and adds a default value for old Revisions which predate `:card_schema`.
* fix: snowflake can-connect with ssh without touching other drivers
* Adjust host-and-port for can-connect without calling incorporate-ssh-tunnel-details
We were inadvertently logging db-details in the rarely used
`maybe-test-and-migrate-details`. This changes to more of a data
diff where we can still get important information on how it changed.
* Rename checkCompiledExpression to diagnoseExpression
* Add tests for arg count validation
* Add arg validator tests
* Add test for unsupported functions to diagnostics
* Add isCallExpression matcher
* Add visit helper
* Move unsupported function check to diagnostics
* Move arg validator to diagnostics
* Move arg count validation to diagnostics
* Remove EQUIVALENT_CLAUSES since we have them in config.js
* Remove database argument where no longer needed
* Add COMPARISON_OPERATORS to config
* Add LOGICAL_OPS to config
* Add NUMBER_OPERATORS to config
* Add FIELD_MARKERS to config
* Move comparison validator to diagnostics
* Simplify resolver
* Move checkers before Lib.diagnoseExpression
* Fix diagnoseExpression return type
* Move case or if operator argument check to diagnostics
* Remove unused tokenName
* Make expression types more specific
* Convert resolver to typescript
* Add test for multi-arg infix operators
* Add checker for unknown functions
* Add more tests for coalesce typing
* Add more tests for comparison operator resolving
* Add test for number operators
* Add test for logic operators
* Remove branches that are handled in the config already
* Allow mbql clause config to define the arg types
* Allow case/if to define argType too
* Move getNode and getToken to utils
* Make resolver fn required
* Validate field names in resolver
* Move trim into getMBQLName
* Fix top-level literals
* Handle expression refs
* Allow number operators to be multiple
* Add more tests for custom arg validators
* Test operator resolvers more
* Add more tests for multi arg functions with options
* Use better formatting in test
* Be more generic in getNode and getToken
* Use correct location for errors
* Make it work
* Use Mantine
* Update tests
* Display db type instead of mbql type
* Show db type instead of semantic type
* Revert "Show db type instead of semantic type"
This reverts commit 5a51c5abca.
* Update tests
* Prevent CLS
* Revert change
* Revert "Prevent CLS"
This reverts commit f74b6e80af.
This commit introduces empty states for visualizations
in order to help less experienced users create their first chart.
These new empty states come in two flavors:
1. With a "Summarize" button CTA that will open the summarize sidebar
2. With a link to our helpful documentation (for more complex charts)
We have been previously relying on the `placeholderSeries` dummy data to display a greyed out chart preview when we cannot render a chart. That functionality is now replaced with a new property called `hasEmptyState`.
`checkRenderable()` util will try to render a chart based on the series and settings. If it is unable to do so, it will throw a `ChartSettingsError`. This, in combination with hasEmptyState and a few other properties, is a cue to render a new, empty state for that visualization.
* Initial BE changes for custom expression literals (#55206)
* Empty
* Add `value` support in the expression editor (#55203)
* Literals in filters tests (#55283)
* E2E tests for literals in custom columns (#55288)
* BE changes for :value wrapped expression literals
* Extend BE test with more literal values
* Add float value to qp test for expression literals
* Add Float and Boolean cases to literal-expression-test
* Remove ^:internal from value clause
:value clauses are now generated by the frontend when creating a top-level expression literal and preserved when
converting legacy <-> pMBQL.
* Add some lib.convert tests for literal custom expressions
* Add legacy schema tests for literal custom expressions
* Add normalize / canonicalize tests for literal custom expressions
* Remove vestigial "date" expression from fields clause in query-with-literal-custom-expression-test
* Fix literal-expression-test for cljs
CLJS parses 1.00 as 1 and hence :type/Integer not :type/Float
* Fix js_test expression-clause-<->-legacy-expression-test for wrapped literals
* Move QP tests into expressions_test.clj and add more tests
* Remove commented out :value and update comment in stage-expressions->legacy-MBQL
* Restrict QP literal expressison tests to supported drivers
* Update boolish->bool to accept BigDecimal inputs
Apparently some DBs return BigDecimal results for boolean expressions.
* Temporarily remove literal-expression-case-test
Needs more debugging
* Move aggregations tests into expression_aggregations_test.clj
* Add some basic filter tests to QP expressions_test.clj
* E2E tests for literal custom columns in other clauses and fixes (#55295)
* E2E tests for literal custom columns in other clauses and fixes (#55295)
* Move changes from the FE branch
* Move changes from the FE branch
* Move changes from the FE branch
* Fix filter schema and false = false filter (#55417)
* Fix date/string literals effective type (#55427)
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix malformed order-by clauses
* Remove tests for offset expressions as non-aggregation expressions
These tests were initially added in #42302, but we disabled offset in custom columns in #43224.
Remove these tests so we can re-enabled nest-expressions when expressions appear in an order-by clause.
https://metaboat.slack.com/archives/C0645JP1W81/p1742317450874479
* Add order-by to the list of places to check in should-nest-expressions?
This is to support order-by of custom expressions, in particular literal custom expressions.
https://metaboat.slack.com/archives/C0645JP1W81/p1742317450874479
* Rename round-trip-expression-literal-test and move it next to existing literal tests in the same file
* Uncomment filter-literal-boolean-expression-with-no-operator-test
The schema was relaxed in bd82763beb, so now the test passes.
* Uncomment filter-literal-expression-with-and-or-test
* Add round-trip-filter-expression-test
* Add more cases to legacy schema filter-test
* Remove unused import
* Review feedback: set/difference -> disj
* Add :expression-literals driver feature (#55662)
* Add the :expression-literals database feature
And enabled it for postgres, mysql, bigquery, and snowflake.
These are the tier 1 and tier 2 DBs, minus redshift which is currently failing some tests.
* Add lib.metadata/database-supports? helper
* Check if :expression-literals feature is enabled in diagnose-expression
* Use standard-literal-expression-values in order-by test expectation
* Add comment about compiler warning to boolish->bool
* Add breakout-by-literal-expression-test
* Add expression-literals-test to nested-queries-test
* Add compound expressions involving literal custom columns to e2e filters tests
* Use lib.schema.common/is-clause? to detect :value expr in diagnose-expression
* Review feedback: fix order-by-integer-literal-expression-test
* Review feedback: conj -> vector
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Add explicit CAST when compiling string literal expressions in H2 driver (#55920)
* Add explicit CAST when compiling string literal expressions in H2 driver
Closes QUE-726
* Add e2e test for nested question with literal custom columns
* Add :expression-literals feature to various mock providers used in tests
* Update diagnose-expression tests
* Ensure legacy :value options are normalized in mbql.normalize
Values coming from the FE in expression-clause-for-legacy-expression call normalize and expect to have :value options
converted from strings to keywords.
* Add base_type to value literals in adjustTopLevelLiteral
* Consistent spelling of base_type
* Use lib.metadata/database-supports? to check for database :features (#56114)
* Use lib.metadata/database-supports? to check for database :features
Closes QUE-836
* Convert a couple more tests to use meta/updated-metadata-provider
* Add e2e test to ensure that sandboxing on literal custom column fails closed (#56190)
* Add e2e test to ensure that sandboxing on literal custom column fails closed
Closes QUE-844
* Review feedback: dedup CustomColumnType definition
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* scrape releases page to find latest major version
e.g.:
```
$./bin/mage jar-download 50
Warning: JARS not found in env.
Looking for latest version of 50 on release page 1
Downloading from: https://downloads.metabase.com/enterprise/v1.50.35/metabase.jar
to: /Users/bcm/dv/mb/metabase/jars/metabase_1.50.35.jar ...
Downloaded 1.50.35.jar to /Users/bcm/dv/mb/metabase/jars
```
* better feedback
* Make segment name clickable
* Upadte e2e test
* Convert SegmentItem to TS
* Modernize SegmentItem
* Add cells
* Add TableBreadcrumbs component
* Use TableBreadcrumbs in SegmentEditor
* Use real data
* Improve data handling
* Handle ellispis
* Add tests
* Add test for no schema
* Rename
* Prettier
* Update icon color
* Fix icon shrinking
* Use TableBreadcrumbs in SegmentItem
* Fix icon shrinking
* Update max width
* Update tests
* Sort
* Remove css file
* Add TableBreadcrumbs component
* Use TableBreadcrumbs in SegmentEditor
* Use real data
* Improve data handling
* Handle ellispis
* Add tests
* Add test for no schema
* Rename
* Prettier
* Update icon color
* Fix icon shrinking
* Remove css file
* scaffold setting up individual downloads
* use downloadsEnabled as boolean for result downloads
* allow SDK to use a single withDownloads props for compatibility
* sdk static dashboard should continue to support single withDownloads prop
* update downloads prop to be string compatible
* fix the areDownloadsEnabled matcher
* refactor download plugin matcher
* do not modify EmbedDisplayParams for now
* fix misleading types
* remove ConnectedDashboard default
* undo LookAndFeelSettings
* revert embed flow and analytics changes
* add test cases for public resource downloads
* add test cases for public question download
* do not support PDF downloads on questions
* remove redundant showDashboardCardActions
* create types for download results
* rename cardResult to dashcard
* replace cardResult with dashcard
* add downloadsEnabled to storybook
* update unit test for areDownloadsEnabled
* update unit test for no-token case
* update unit test for oss case
* try to disable downloads in storybook
* update story defaults
* make downloadsEnabled a component default
* add default to downloadsEnabled in PublicOrEmbeddedDashboardView for loki
* revert changes as loki diffs seems good now
* fix loki diff issue in stories
* Empty
* Date generalized tests (#55953)
* Centralize and generalize tests for `integer()`
Also make room for other casting functions to be similarly tested
* Remove unsued imports
* Move postgres date cast test into cast_test.clj
* Rename :cast to :expressions/date
* Remove old imports
* Remove unused features
* Remove more!
* Fix test files
* Centralize and generalize tests for `integer()` (#55950)
* Centralize and generalize tests for `integer()`
Also make room for other casting functions to be similarly tested
* Remove unsued imports
* Example tests for integer()
* Remove empty line
* Use field type
* Appease the Kondo
* Check for Number instead of Integer
* Cast to big-integer before comparing (for snowflake)
* Remove split-part-test. It doesn't belong in this branch.
* Rename :cast to :expressions/integer
* Fix parens after merge
* Unified text() tests
* Appease the kondo
* Appease the whitespace demons
* Oops, wasn't filtering for drivers
* Use literal string for redshift
* Add a repro for #55951
* Fix typos
* Refactor repro
* Fix#55951
* Remove unused components
* Use 'loading'
* Format code
* Fix test setup
* Rename isLoading back to isSearchLoading since it indicates Search.loadList usage
* Use 1 /search API call instead of 2
* Get rid of rawData
* Update unit tests
* fix: deadlock in last used at
This prevents bulk updates to last used at from deadlocking by creating
a new table in metabase's database that can be used by instances to take
cluster wide locks. It adds an initial implementation of a customizeable
db-based locking mechanism.
For the last_used_at update, a metabase instance will first set a
session variable to control locking timesout then take a cluster lock
using SELECT * FROM metabase_cluster_lock WHERE lock_name = 'name' then
execute the update_at.
If another table is holding this lock metabase will wait .5s for the
lock to be released then timeout its SQL transaction, sleep the Thread
for 1s then retry taking the lock. This makes sure we're not holding DB
connections waiting for locks.
* fixup! kondo
* Update resources/migrations/001_update_migrations.yaml
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update src/metabase/query_processor/middleware/update_used_cards.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* fixup! finish method rename
---------
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Fix formatter support for when both field ids and col names are used in viz settings
* Apply fix in qp common as well, which date formatter consumes
* Addressing PR feedback and breaking up large test
* Fixing or->merge reversion; adding datetime formatter tests
* Improving viz setting merges based on types of field_ref and column setting keys
* Addressing PR feedback
* Unified tests for `splitPart()`
* Remove postgres specific split-part-test
* Remove Redshift split-part-test
* Remove snowflake split-part-test
* Remove mysql split-part-test
* Remove bigquery-specific split-part test
* Fix MySQL implementation of splitPart()
* Only allow string literals for 2nd and 3rd args of replace()
* Only allow non-empty string for delimiter of splitPart()
* Better non-blank string
* Make position in splitPart() positive literal or numeric expression
* WIP new examples tests + corner cases
* Oops, go back to non-empty string
* Improve tests
* Improve test message
* Empty string on non-positive position in splitPart() for mysql and postgre
* Fix Redshift error
* Snowflake: return empty string with non-positive position
* Also test zero position
* Correct misplaced paren
* Fix `splitPart()` for snowflake
* Nicer comment on weird mysql implementation
* Rename positive-integer schema
* Clearer parameter comments
* dont skip mage tests when BE job is skipped
* checkout metabase for mage tests
* make mage tests its own thing
* add mage test workflow
* if it aint broke dont fix it
* remove dupe mage test from backend workflow
* fix: correct setting name check for static stats
also fix namespace for the defenterprise in token check to load the
embedding stats module
* fixup! use get-raw-value
* fixup! use-get-value-of-type
Fixes [QUE-838](https://linear.app/metabase/issue/QUE-838/linked-filters-are-not-working-on-public-dashboards).
Query params were getting parsed incorrectly to a map like `{:17fdc812 ...}`
with a keyword rather than a string. These failed to match any of the
dashboard's parameters (which are keyed by strings like `"17fdc812"`)
and returned no values (but did not throw any errors in prod).
Before, when you had an outer query referring to something in a nested field from the inner query, the query processor
would assume the field is still nested and generate invalid sql. Now, it will just refer to the alias.
* Move the lexifier to it's own file
* Remove duplicate type union member
* Remove unused typeinferencer
* Add ParseError type to errors
* Lexify straight in to pratt.Token
* Remove unused TOKEN type
* Use pratt.Token type everywhere
* Fix type
* Remove unused __support__/expressions helper file
* Ignore END_OF_INPUT when matching suggestions
Fixes#55929. (Requires a new export for 54 that also gets backported.)
Adds a new setting `install-analytics-database` that skips loading the
database at all, if set to false. That allows the dev instance that
exports the analytics content to have no real audit DB, and therefore to
generate the right `entity_id` hashes for the audit export!
Existing audit DBs are fine; the checksum will see these new entity_ids
in the export and overwrite any existing content.
* Implement splitPart() for BigQuery
* Always return string instead of null
* Better test failure message
It's running fine on my machine ;)
* Use coalesce instead
* fix: Ensure that local timestamps params work
Fixes#53371
By filtering a timestamp or date column on an OffsetDateTime it results
in an index being missed since the lhs needs to be upcast by the db engine
to a `timestamp with timezone`.
This ensures we only pass OffsetDateTime to `timestamp with timezone`
columns.
Context about decision here:
https://metaboat.slack.com/archives/C0645JP1W81/p1743111964032089
* Handle sending a datetime range rather than just a date
* Feature gate to dynamic dataset loading
* Fix kondo
* Remove comment
* Rename caseInsensitive to caseSensitivity
* Use literal string for case-insensitive template
* Update argument description to be more clear
* Support inline expression highlights
* Render help text description with Markdown
* Add more markdown to help text strings
* Use markdown code blocks to mark code in help text descriptions
* Highlight argument references in help text descriptions
* Fix arg alignment
* Fix HelpText unit tests
* Fix FunctionBrowser tests
* Render FunctionBrowser descriptions using Markdown too
* Mark true
* Fix e2e tests
* Fix e2e tests
* Remove commented-out code
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Auto-categorize truncation units as datetime columns in insights code
* detect datetime units on the FE as well when rendering trend visualization
* prettier
* add unit test
* typo
* fix comment alignment
* Initial support for text wrapping in static viz
* Add text-align support
* Add row index support
* Fixing broken unit tests
* Adding new unit test coverage for Tanstack viz settings
* Fixing typo in whitespace; removing debug def
* Removing defs & outdated comment
* Addressing PR feedback
* Adding column width support to minibar columns; improving doc strings
* Adding minibar column width support; improving doc strings; moving consts to style namespace
* Fixing broken unit test after font size change
* Remove trailing whitespace
# Support Environment Variables for API Keys
### Overview
This PR adds support for using environment variables to define API keys in the configuration file, enhancing security and flexibility for deployment environments.
### Changes
- Modified API key validation to support environment variable substitution in the key field
- Added runtime validation for API keys from environment variables to ensure proper formatting
- Refactored tests to properly validate environment variable integration
- All API keys (including those from environment variables) still must follow the required format:
- Must be between 11-254 characters
- Must start with mb_
- Must match the pattern mb_[A-Za-z0-9+/=]+
### Usage
```
api-keys:
- name: "ENV API Key"
key: "{{env API_KEY_FROM_ENV}}"
creator: "admin@example.com"
group: "admin"
```
### Testing
- Creation of API keys from environment variables
- Validation of API key format from environment variables
- Error handling for improperly formatted environment variable values
# Skip keys if they exist
To guarantee that it is possible to keep the same yaml file on each instance initialization, if there is a key in the instance and the config file has a key with the same name, the creation is skipped.
* Minibar chart support in table static viz
* Fixing unit test; adding new minibar unit test
* Fixing clj kondo warnings
* Addressing PR feedback
* Removing dead minibar code
* Removing old minibar unit tests
* Empty
* Implement and test snowflake integer()
* Use card-with-source-metadata-for-query in snowflake tests
* Use lib/functions instead of lib/expression-clause in snowflake
* integer() for BigQuery
* Fix BigQuery tests for integer()
* Change to backticks
* kick ci
* Fix for BigQuery
* Capitalize name of field
* Use STRING type for casting in BigQuery
* Sometimes we get an integer back, and that's okay
* Use mt/card-with-source-metadata-for-query in bigquery tests
* use lib/functions instead of lib/expression-clause in bigquery
* feat: add remaining token check stats and settings
Adds the remaining five data points we want to report during the token
check request. Adds a new module to internal stats that does the
embedding counts and setting checks.
* fixup! log stats
* fixup! move enterprise settings into defenterprise
* fixup! remove row count
* fix: only remove vis_settings from dashboardcard
Fixes loading serialized dashboard cards where we would previously
delete and recreate them on each load.
In order to handle circular dependencies caused by links to other
dashboards in a dashboardcard's visualization_settings we were removing
the dashboardcards entirely from the dashboard model to break the cycle.
Later when we traversed back through the dependency path we would load
the original dashboard a second time with the restored dashboardcards.
However, on the load of the cycle breaking change we could entirely drop
dashcards belonging to the dashboard because the serialization code was
treating our cycle breaking changes as an edit to the dashboard to drop those
cards. This meant when we got back to the the second load of the
dashboard we would create the dashcards fresh instead of updating
existing instances.
The fix here is that we only remove the values from the dasboard card
model that can cause the cycle -- in this case just the
visualization_settings column. This seemed the most straightforward fix,
but there are other possibilities here.
One is that we could _just_ preserve the entity_id column in dashcards
which would eliminate the possibility of any other cycles here.
Another option would be to change how we handle nested models in serialization.
There is not really a foreign key dependency here between dashboard A
and dashboard B linked by the dashboard card. If we instead just created
those dashboards first _then_ went back and created their dashcards we
don't have any problems at all. We could move nested models up to the
same level of handling in serialization that we do for models, which
would let us create the dashboards first then create any dashboard cards
in a subsequent step.
* fixup! update docstring
Closes QUE-794 and QUE-705.
This will enable us to *know*, rather than *infer*, what version of
`report_card` we're looking at.
One example: in the near future, I'm going to add some values to
`result_metadata` on cards. If we later discovered a bug in that logic
and could no longer trust those fields written to AppDB in
`result_metadata`, the `card_schema` will tell us with certainty
whether the card was saved with the broken logic (`:card_schema N`)
or the new logic (`:card_schema (+ N k)`).
The upgrade steps could drop the invalid fields and, if desired, infer
the correct ones.
Note that there's nothing here about writing back! This is about
cleaning up old cards during `after-select`.
There are no upgrades yet; the current version is still the default.
```
2025-03-27 15:25:21,322 INFO sync.describe-database :: Checking "PUBLIC"."LARGE_NUMBERS"
2025-03-27 15:25:21,757 INFO sync.describe-database :: "PUBLIC"."LARGE_NUMBERS": SELECT privileges confirmed
2025-03-27 15:25:21,759 INFO sync.describe-database :: Checking "PUBLIC"."SINGLE_ROW_TABLE"
2025-03-27 15:25:21,975 INFO sync.describe-database :: "PUBLIC"."SINGLE_ROW_TABLE": SELECT privileges confirmed
2025-03-27 15:25:21,976 INFO sync.describe-database :: Checking "PUBLIC"."SLOW_QUERY_VIEW"
2025-03-27 15:25:22,219 INFO sync.describe-database :: "PUBLIC"."SLOW_QUERY_VIEW": SELECT privileges confirmed
```
This will list all tables it enumerates and then which ones it
removes. There are also two error handling states:
```
2025-03-27 15:29:25,418 INFO sync.describe-database :: "schema"."some_table": SELECT privileges confirmed
2025-03-27 15:29:25,419 INFO sync.describe-database :: "schema"."some_table": Assuming SELECT privileges: caught timeout exception
2025-03-27 15:29:25,419 INFO sync.describe-database :: "schema"."some_table": Assuming no SELECT privileges: caught exception
java.lang.Exception: Some exception
at metabase.driver.sql_jdbc.sync.describe_database$eval194718.invokeStatic(NO_SOURCE_FILE:33)
at metabase.driver.sql_jdbc.sync.describe_database$eval194718.invoke(NO_SOURCE_FILE:33)
[more stacktrace from the exception]
```
This is essentially just changing the log level. We can and perhaps
should change these back to debug or trace in the future but this is
useful for now.
* WIP Implement `integer()` for all :sql descendants
* Add test-data to tests
* Just do MySQL
* Undo change to query_processor_test
* Remove unused imports
* Remove extra blank line in query_processor
* Convert postgres tests to mysql
* Appease MySQL syntax
* Remove old tests for mysql integer()
* Remove unsupported postgres tests
* Fix kondo
* kick ci
* Fix up for MariaDB
* Fix queries in test
* Try bigger numbers for MariaDB
* Remove ability to cast integer to integer
* Check that boolean = 1 in mysql
* Use mt/card-with-source-metadata-for-query
* Use lib/functions instead of lib/expression-clause
* Use lib/functions instead of lib/expression-clause in postgres
* Use card-with-source-metadata-for-query for postgres tests
* Update some function names to match modern casing
* Fix test cases
* Update casing in docs for isEmpty, notEmpty, lTrim, rTrim, isNull, and notNull
* Fix isNull and isEmpty casing in tests
* Fix case in unit test
* Fix casing in e2e tests
* Fix couldn't find Slack channel
When the first `conversations.list` doesn't return the Slack channel we
want, we need to continue to paginate until we found it.
* [debug] for testing
* Revert "[debug] for testing"
This reverts commit 67dd2c0405.
* Simplify logic
Co-Authored-By: Timofey Kachalov <timofei@metabase.com>
---------
Co-authored-by: Timofey Kachalov <timofei@metabase.com>
* rename and reorganize
# Conflicts:
# enterprise/frontend/src/metabase-enterprise/google_drive/GdriveConnectionModal.tsx
# enterprise/frontend/src/metabase-enterprise/google_drive/GdriveSyncStatus.tsx
* resolve conflicts
* only show sync status to admin who created it
* Add connection info to disconnect modal (#55588)
* rename and reorganize
# Conflicts:
# enterprise/frontend/src/metabase-enterprise/google_drive/GdriveConnectionModal.tsx
# enterprise/frontend/src/metabase-enterprise/google_drive/GdriveSyncStatus.tsx
* resolve conflicts
* update disconnect modal
* better spacing
Improve the performance of the collection permission query by using a
semi-join instead of an inner join to the permission tables. This
ensures each collection only needs to be tested once for a route that
gives a user permission to access it rather than multiple times.
This reduces the query time for users that are members of lots of
collections since we dont need postgresql to create large join tables or
return the same collection id more than once from this query.
* test: make dashboard flaky test more robust and stable
* revert flaky
* add additional check
* fix that works locally
* revert prev version
* add missing line
* Add baseType prop to SemanticTypePicker
* Update unit tests
* Fix legacy v1/ Field['base_type'] typing, to make it same as Field['base_type'] from api/
* Use isa
* Add a hack for numerical types
* Fix typo
* Pass field
* Reuse type
* Use effective type
* First pass on the tech doc
* Handle booleans
* Use isa
* Refactor
* Update test
* Update test
* Assume no type can have 2 level-one ancestors
* Add comments
* Extract getCompatibleSemanticTypes
* Refactor isDeprecated
* Refactor
* Add fieldId param
* Add test for booleans
* Make initialValue required
* Do not assume a field cannot have multiple level-one ancestors
- these types have 2 level-one ancestors: type/JSON, type/XML, type/DruidJSON, type/OracleCLOB
* Extract FIELDS
* Add test
* Test Entity Name
* Remove '
* Add test
* Update test
* Add test for base_type fallback
* Update comment
* Refactor tests
* Add Field containing JSON tests
* Refactor
* Add a comment
* Improve naming
* Improve naming
* Use SemanticTypePicker in DatasetFieldMetadataSidebar
* Use Field from types/api
* Extract DatasetFieldMetadataSemanticTypePicker
* Fix typing
* Pass label, tabIndex, and onKeyDown
* Remove old picker
* Use SemanticTypePicker in FieldTypeDetail
* Remove unused prop
* Remove redundant margin
* Fix#55617
* Use SemanticTypePicker in Field
* Remove old SemanticTypePicker
* Reuse FkTargetPicker
* Introduce new top-level frontend directory: frontend/src/metabase/metadata
- Move semantic type picker + subcomponents in there
* Add readme
* Get rid of global CSS classes
* Remove old FKTargetPicker component
* Pass databaseId to components
* Don't check for text-like fields
* Merge branch 'SEM-97-limit-semantic-types' into SEM-176-new-semantic-type-picker-everywhere
* Update test
* Update tests
* Sort options by label
* Update test
* Fix docs links
* Change reproductions suite to TypeScript
* Repro #55618
* Repro #55617
* Make sure combobox flips if there is no space underneath
* fix: add index to field_usage query_execution_id
Supports faster deletes on the query_execution table. Without this index
deleting rows from that table leads to a seq scan on the field usage
table for each row deleted to handle the ON DELETE CASCADE trigger.
* fixup! drop empty preconditions
* Add baseType prop to SemanticTypePicker
* Update unit tests
* Fix legacy v1/ Field['base_type'] typing, to make it same as Field['base_type'] from api/
* Use isa
* Add a hack for numerical types
* Fix typo
* Pass field
* Reuse type
* Use effective type
* First pass on the tech doc
* Handle booleans
* Use isa
* Refactor
* Update test
* Update test
* Assume no type can have 2 level-one ancestors
* Add comments
* Extract getCompatibleSemanticTypes
* Refactor isDeprecated
* Refactor
* Add fieldId param
* Add test for booleans
* Make initialValue required
* Do not assume a field cannot have multiple level-one ancestors
- these types have 2 level-one ancestors: type/JSON, type/XML, type/DruidJSON, type/OracleCLOB
* Extract FIELDS
* Add test
* Test Entity Name
* Remove '
* Add test
* Update test
* Add test for base_type fallback
* Update comment
* Refactor tests
* Add Field containing JSON tests
* Refactor
* Add a comment
* Improve naming
* Improve naming
* Use Field from types/api
* Fix typing
* Don't check for text-like fields
Previously we looked at 1000 users and got the set of unique user
attribute names from them. So if 1000 users each had a `name` and user
1001 had `some-other-attribute`, we would give you back `name` as the
complete set of available attributes.
This modifies the magic number (to 5000 from 1000) and the logic. We
select the first 5000 unique *attributes*. So if 1000 users each have a
`name`, that only counts as 1, and we'll continue looking for other
unique attributes before returning to the client.
Implemented validation that ensures API keys when created with config.yml:
Start with the mb_ prefix
Have a minimum length of 12 characters and max 254
Prefix (first 7 characters) must be unique
String must be Base64
* Use field refs instead of field ids/names for tiles queries
* Fix up embedded + public endpoints
* use field_refs
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
* resetting qb state before running card on back navigation
* try something else
* adding e2e test for notebook navigation
* wait for fk call in test
* increase timeout
* e2e adjustment
* Control visible entities in multi-stage data picker (#54845)
* Add props to DataSelector, and keeps the same behaviors.
* Read entity_types from URL search params
* Fix type errors
* Fix SDK not rendering multi-stage data picker
* Add redux slice tests
* Add E2E tests
* WIP add integration tests
* Add all integration tests
* Put tests file in a proper folder
So the SDK build process doesn't check test file types
* Cover more E2E scenarios
* Cover the new code with a test
* Add more assertions to make sure more paths are tested
* Fix imports
* Make functions easier to understand
* Fix test case name typo
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Clarify embed state type
* Address review: Make code more concise
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Control visible entities in simple data picker (#55329)
* Filter simple data picker by "entity_types"
* Add a happy path for multi-stage data picker E2E test
* Fix loading a wrong entity
* Control visible entities on new item menu dropdown (#55331)
* Control visible entities on new item menu dropdown
* Hide action following product discussion
* Make a new menu item more accessible
* Add tests
* Control visible entities in Browse sidebar (#55332)
* Control visible entities in Browse sidebar
* Add relevant tests
* Fix test case typos
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Add comment to clarify magic value
Co-authored-by: Nicolò Pretto <info@npretto.com>
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Empty
* Add split-part backend (#55425)
* feat: prepare fe part of split-part (#55483)
* Add split-part backend
* feat: prepare fe part of split-part
* rename function and add test
* use postgres for testing
* add test for an error
* add db flag
* remove docs page
* update copy
---------
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
* feat: prepare fe part of url-pathname
* fix dropdown styling
* fixup example
* rename function to path and update example
* Backend for url-pathname
* Rename to path()
* Backend tests for `path()`
* Frontend and Backend for url-pathname (#55353)
* feat: prepare fe part of url-pathname
* fix dropdown styling
* fixup example
* rename function to path and update example
* Backend for url-pathname
* Rename to path()
* Backend tests for `path()`
---------
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
* Fix kondo
* Fix kondo
* feat: prepare fe part of url-pathname (#55291)
* fix description
* fix styling
* cleanup
* fix rebase
* fix rebase
* add tests
* add tests
* update test
* fix test
* address review
* prevent url translations
* make tests less flaky
* update copy
* add category
* remove docs page and add feature flag
---------
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Convert typeinferencer to typescript
* Split up diagnostics
* Split of lib check from diagnostics function
* Move getDatabase to util file
* Move isErrorWithMessage to utils
* Add getExpressionMode helper
* Simplify branches
* Move resolveMBQLField to separate module
* Simplify expressions import
* Remove unused options
* Fix type of mock env
* Move compiler to separate file
* Remove unused arguments to compiler
* Move passes to separate file
* Return expression from diagnose
* Remove unused processSource helper
* Fix types in passes
* Make resolving optional in compiler
* Add fuzz testing for compiler
* Mock legacy mbql to clause in fuzz test
* Remove need for compare
* Use default parameters
* Remove compare and compareSeed
* Remove unused recursive-parser
* Remove fuzz testing for recursive parser
* Remove unused arguments
* Colocate helper with the tests that use it
* Use compileExpression in formatter tests
* Use compileExpression in typeinferencer tests
* Order compiler passes
* Move StartRule and ClauseType to metabase-lib/v1/expressions
* Remove useShorthands pass
* Fix token escaping in pratt compiler #54751
* Remove POPULAR_ functions and aggregations
* Move compiler passes out of pratt compiler
* Make getMBQLName optional
* Add missing adjustBooleans pass
* Type startRule in compiler
* Move resolverPass to field-resolver
* Add isCompilerPass helper
* Allow use of short-circuiting for disabling passes
* Only render friendly errors
* Mark ExpressionError and ResolverError as user-friendly
* Add check for missing comma in argument list
* Simplify diagnostic checks
* Always return error and expression
* Make compileExpression and diagnoseAndCompile generic to StartRule
* Remove diagnoseAndCompileExpression helper
* Remove unused completer utils
* Move enclosingFunction to expressions module
* Fix typeinferencer tests
* Fix e2e test
* Detect cycles
* Inline passes
* Flatten compiler
* Remove prevError
* Remove default from diagnoseAndCompile type definition
* Use try/catch to simplify diagnoseAndCompile
* Wrap diagnostic errors in their own type
* Remove unnecessary wrapping of diagnostic error
* Move errors to top-level expression module
* Remove duplicate error types in expressions
* Fix error format in tests
* Add comments to passes
* Add tests for passes
* Fix lint
* Fix type imports
* Flesh out references in test data with base-type
* Use fleshed out field reference in all test cases
* Add boolean expression to test data
* Port recursive-parser tests for compiler
* Fix support for unary +
* Add cast support to test database
* Move cast calls to separate test
* Add test for escaping identifiers
* Unskip test
* Fix compiler tests to support top-level literal
* Typo
* Use normal quotes
* Fix typeinferencer for value clause
* Use all passes by default
* Fix pratt compiler tests
* Unskip garbage test
* Use dedent from ts-dedent
* Remove ClauseType helper for expression and make everything use Lib.ExpressionClause
* Coerce clause to be expression clause in filter and aggregation pickers
* Fix diagnostic test
* Wrap diagnostic error in a DiagnosticError
* Use instanceof to check type of error
* Do not assume all diagnostic errors are human-friendly
* Update translation sources 2025-03-17
* Update translations
* don't add lithuanian yet
* language is always locale
* Fix the `ttag` i18n calls for pluralized strings
Pluralized strings *without* the number interpolated into them (eg.
"Month" and "Months") were not getting found by `i18n/trun` in CLJS,
which made them fall back to English.
This fixed the `ttag/msgid` call to be correctly formed in both cases.
* update some localized strings
---------
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
Co-authored-by: Braden Shepherdson <braden@metabase.com>
We are running a probe query on snowlfake to see if the current db user
can see views
```
select true from X where 1 <> 1 limit 0
```
Something like this.
But on large views, it seems snowflake does not quickly determine this
has no rows and it can take a while.
This leads to the connection being closed. Then in the error handler we
attempt to rollback. But if the connection is closed _this_ throws and
we have now thrown from the error handler and we break sync.
This does two things
1. set a timeout on sql statements. This does 10 seconds.
2. If a timeout occurs, assume that they _can_ see the object
The idea is that timeouts might happen but it's trying to run a query
that it would quickly reject if the permissions were lacking. So
timeouts mean you can see the data, it's just taking a while
* feat: add embedding stats to token request
Adds the embedding_users, internal_users, domains, and query_execution_*
stats to the token check request in order to track embedding usage.
This creates a new internal_stats module to share code between the
anonymously reported stats the stats for the token check request to
avoid some circular depedencies. This also seperates creation of a stat
and how it is reported, so we could move over other stats calculated for
the phone home metric.
* temp! add a logging statement to figure out why CI is different
* fixup! exclude api-key email
* fixup! exclude api-key/internal type users
* fix localhost:* in embedding app origins sdk
We need to call the setting's getter directly, and not use
`setting/get-value-of-type`. The setter it adds localhost:* which is required.
* add test
* test fix
* update module config
* improve test feedback
* fix ci
* use whitelabel colors in the static viz
* Fix backend whitelabel test for app colors
* update CI
---------
Co-authored-by: Tyler Plude <tsplude@gmail.com>
* Fix cond fmt of null vs empty str in table static viz
* Adding unit test
* Add TextWrapper support in json exports
* Updating unit tests with TextWrapper
* Empty
* Add `value` support in the expression editor (#55203)
* Literals in filters tests (#55283)
* E2E tests for literals in custom columns (#55288)
* E2E tests for literal custom columns in other clauses and fixes (#55295)
* Move changes from the FE branch
* Move changes from the FE branch
* Move changes from the FE branch
* Fix filter schema and false = false filter (#55417)
* Fix date/string literals effective type (#55427)
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Forbid top-level constants until the BE is fixed (#55459)
* Use matches
* Make test flaky
---------
Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
* ci(sdk): adds embedding-sdk-tests-result
* ci: should run the sdk tests as part of the embedding-sdk workflow
* let's try without concurrency
* rename "frontend" -> "embedding SDK"
* Integration
* Support for `text` MBQL function on the FE (#55274)
* Cast text backend on postgres (#55292)
* Add `text()` custom expression that casts to text
* Add :cast to list of all possible features
* Complete testing plan
* Fix kondo warnings
* Fix test
* Empty
* Add e2e tests (#55361)
* Fix failed tests for text() function (#55374)
* Add UI for integer() function, with e2e tests (#55366)
* Set category
* Set category
* Set category
* Set category
* Fix tests
* Cast integer backend (#55381)
* integer() custom expression with simple tests
(more tests coming)
* MOAR tests
* EVEN MOAR tests
* Change type to BigInteger
---------
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
This adds coercion support for serialization and adds a very limited partial schema for a Dashboard, in hopes that
this grows to be a full schema for our data models at some point.
fixes#54941
* sync translations with crowdin
* remove poeditor integration
* fix run name
* update to new project
* sync
* path
* move to config
* not master lol
* kick ci
* update crowdin config
* remove croatian
* roflcopter
* token
* maybe fetch harder
* test code - revert this
* test
* more tokens
* just use the cli
* line breaks
* le facepalm
* no verify
* token swap
* auto approve
* revert
* remove zh e2e test
* remove test code
* don't backport translations
* don't backport translations
* Add FunctionBrowser component
* Move header into CombineColumns for consistency
* Render the function browser
* Allow wrapping shortcuts
* Render toolbar buttons on top again
* Add missing Lib import
* Fix e2e test
* Use ellipsis in placeholder
* Add test-id to function browser
* Add e2e tests for function browser
* Add category to help text configs
* Only render supported functions in function browser
* Convert to helptext in getFilteredClauses
* Group functions by category
* More e2e tests
* Add empty state to function browser search
* Add more breakpoints for expression editor layout
* Move function browser item into its own component
* Increase width of function browser
* Add case to functions
* Remove unnecessary maw prop
* Sort functions
* Add tests for FunctionBrowser
* Remove nested component definitions
* Use standard breakpoints
* Add spacing between toolbar buttons
* Fix functions test
* Add category to distinct-if
* Fix tests that were broken by sorting
* Fix unit test examples
* Add more unit tests for filtering
* Do not enter parentheses when there are no arguments to the clause
* Sort by displayName
* Sort by structure
* Add e2e about clauses with no args
* Fix sorting test
* Fix font sizes for function browser
* Fix function browser spacing
* Allow replacing selection with function browser
* Context Driven Indicator Menu
* add unit tests
* unit test adjustments
* Ho
C time
* hasSeenAll
* cleanup, passing tests
* adjusting some tests, PR feedback
* more small adjustments. Starting to add docs
* hopfully final adjustments
* adjusting storybook path
Closes QUE-764.
Adds support for returning "remapped" columns, which have not been
supported in the library previously.
- Refactor `lib.field/remapped-field` to `lib.metadata`
- Add `lib.metadata.calculation/remapped-columns`
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* fix: improve startup by not scheduling all dbs
Improve long startup times by not scheduling tasks for all dbs on
startup. Due to the lack of bulk operations in quartz if an installation
has 500 dbs we need to load and scan through all associated db scanning
triggers 500 times for all dbs which means loading about 500k records in
1000 operations.
This is generally pretty fast on a per operation basis (about .5s
in local development), but on some postgresql installations this can
blow up to a few seconds per db, which makes the whole operation take 40
minutes plus. Additionally there maybe both lock contention and resource
contention between the quartz scheduler and the main app thread that
draws out this time.
Instead we want to init the schedule and put it into standby mode -- where
we can schedule tasks but are not executing them -- before we process
the configuration file. That we can schedule all necessary tasks as part
of loading the database from the config file and not need to then
iterate all database to schedule their tasks later in the startup.
Long term, it would be good to support rescheduling tasks using a bulk
operation in quartz (there are TODOs in their code base for supporting
this) rather than needing to update each trigger individually.
* fixup! doc string for check-health! function
* fix some mage installer logic
* make `cd bin && ./mage` work
get the absolute path, + cd to mb repo root in subshell
---------
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* do not setup sandboxing if using a sample database
* do not ask for tenancy columns if using a sample database
* alter copywriting when in sample database mode
* improve copywriting for scenarios with a sample database
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* fix misleading comments
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
The backfill job originally retried up to 20 times and then gave up. However, some instances apparently have ~150
duplicates or more, so 20 retries is insufficient. In addition, 2 db calls per conflict is sort of a problem when the
conflict count is O(duplicates^2). This bumps up the retry limit and reduces the db calls to 1 per 50 conflicts.
* Disallow navigating from model/{slug}/query and model/{slug}/metadata to model/{slug} when there are unsaved changes
* Reset state
* Simplify reset
* Use appropriate redux action when programmatically invoking navigation
* Disallow navigating from metric/{slug}/query and metric/{slug}/metadata to metric/{slug} when there are unsaved changes
* Remove mentions on inexisting /metric/metadata and /metric/{slug}/metadata routes
* Rename test
* Add repro for #32037 (models)
* Improve assertions
* Fix state cleanup
* Fix flaky test
* Deal with race conditions
* Add repro for metrics
* Correctly reset ui state when cancelling editing a model/metric
* Add repro for editing model's query
* Remove log
* Add a comment
* Revert "Skip tests that deal with the click outside and escape behaviour of the custom expression editor"
This reverts commit 2c05bc08c2.
* Revert "Remove CloseModal component"
This reverts commit ad1795e05a.
* Revert "Move usePreventClosePopover into utils"
This reverts commit de31b21282.
* Add PreventPopoverExit util
* Remove use of non-existant hook
* Prevent popover from closing on outside click or escape
* Rename argument to avoid confusion with double negatives
* Prevent custom expression editor from closing in aggregation sidebar
* Allow exiting the component when navigating away from custom expression editor
* Add a comment explaining why the useClickOutside does not work
* Add comment on how CloseModal works
* Remov empty file
* Add comment on what children should be
* Allow overriding cancel button text on ConfirmModal
* Allow passing more props to ConfirmModal
* Fix primary button on ConfirmModal
* Use ConfirmModal for custom expression editor
* Fix e2e test
* Remove second message in closemodal
* Update loki
If you run an uberjar with `MB_RUN_MODE=dev`, you get an exception
because you try to run the `watch-directory!` bit (which only works for
actual files) in an uberjar environment.
Let's just print a warning but continue along.
* Rename arg to caseInsensitive
* Add matcher for internal [:dimension name]
* Add formatting support for internal dimension clause
* Use raw MBQL in examples
* Format examples at certain width
* Remove unused suggest code
* Fix unit test for help text
* Add unit test for rendering examples on multiple lines in HelpText
* Improve spacing
* Make sure that all help texts can be formatted properly
* Add custom formatExample helper to format example expressions
* Use useAsync to format example
* fixup! Add custom formatExample helper to format example expressions
* Add HighlightExpression component
* Export all modules
* Fix `row` type to be `RowValue[]`
* Use richer `clicked` object in object detail view
* Fix column settings not available in detail view
* Add e2e test
* Rename Select to LegacySelect
* Convert frontend/src/metabase/lib/core.js to TypeScript
* Format code
* Extract SemanticTypePicker
* Fix type
* Fix styling
* Fix nulls
* Add nothingFoundMessage
* Extract DefaultSelectItem
* Add CurrencyPicker
* Optimize
* Add checked icon to CurrencyPicker
* Improve overflow in SelectItem
* Add FkTargetPicker
* Style font-weights to be the same as in other inputs around
* Fix colors
* Add a comment
* Fix imports
* Style dropdowns
* Make currency input full width
* Add Combobox component
* Show currency symbol next to value
* Modernize field settings button
* Modernize FieldVisibilityPicker
* Update tests
* Update test
* Update and merge tests
* Update tests
* Remove bold, fix padding and move components to their own dirs
* Update unit tests
* Update e2e tests
* Update tests
* Reduce flakiness
* Add comment
* Repro #53595
* Replace CSS modules overrides with props
* Fix symbol styling for selected option in CurrencyPicker
* POC - moving two pivot functions into CLJC
* Debug logs in frontend pivot impl
* POC continued
* setting isCollapsed consistently
* Working replication of valuesByKey map construction
* sorting logic
* hook up new code to the FE impl
* Fixing small potential bug in valuesByKey
* Fixing bug in clojure vs javascript string key for valuesByKey
* fix valueColumns key name
* Some performance improvements + cleanup
* A couple more attempts at perf improvements
* Fix bugs
* port format-values-in-tree
* WIP add-subtotals
* @tsp - Improved build-values-by-key perf; some debug perf lines in frontend land
* @tsp - in progress conversion of add-subtotals
* @tsp - working converted Pivot.add_subtotals
* @tsp - fixing bug in add-subtotal mapcat recursion
* @tsp - add-subtotal restoring map to mapcat (oops)
* @tsp - adding working Pivot.add_value_column_nodes to replace addValueColumnNodes
* @tsp - removing debug logs & perf timing logs
* Appease whitespace linter
* More formatting code ported, cleanup
* Fix bugs, cleanup
* Fix some tests
* Start hooking up the BE, fix some bugs
* Fix lint/formatting errors
* Fix comparator + i18n
* Fix another module issue
* Test using js-specific i18n function
* Revert "Test using js-specific i18n function"
This reverts commit 9a8604b583885f1c35f48507112e3a142ca0d3a0.
* Bump up timeout on cypress assertion
* WIP - migrate row section getter to cljc
* fix tests
* Fix e2e test
* Switch exports over to fully using CLJC code paths, delete now-unused code
* Delete outdated postprocess tests
* Bug fix
* Fix build
* cleanup, consolidation
* fix function call
* more cleanup
* empty test ns
* misc cleanup/docstrings
* WIP tests
* more tests and refactor
* small fix
* fix getting column title for value columns
* Fix some tests
* Fix another test
* Revert a couple changes
* Fix some more tests
* Fix more tests, fix default values for pivot viz settings
* Ensure headers respect format-rows? settings in exports, and fix more
tests
* Fix lint issues
* fix another test
* Fix a test, a little cleanup
* Try to fix another test by setting format-rows middleware to false for subscription attachments
* Revert "Try to fix another test by setting format-rows middleware to false for subscription attachments"
This reverts commit d860522a6d17c27e67afabccf119039508c96cca.
* Update formatting logic to fix another test
* Fix conditional formatting
* Fix last BE test
* remove test not needed anymore
* Fix drill through + more cleanup
* clean ns
* remove old code from .clj-kondo/config.edn
* add a comment + tweak keywords
* perf improvements and refactoring the CSV export path
* Performance improvements
* Fix bad rebase
* more perf improvements
* fix test
* lint
* oops, fix test
* more optimizations, memoize backend formatting
* fix range-based conditional formatting
* remove unused function
* add some new backend unit tests
* fix FE type checks - caused by an experiment that I didn't revert fully
* address PR review feedback
* fix up tests
---------
Co-authored-by: Tyler Plude <tsplude@gmail.com>
We are adding entity_ids to dbs, tables, and fields, but existing rows don't have entity_ids yet, and newly created dbs/tables/fields don't have entity_ids when created. This job adds those missing ids.
Ideally, we'd copy them too, but we need to decide if we should duplicate them or not (do a deep or shallow copy) is a
too hard to make immediately, and this at least will unblock people being unable to duplicate dashboard with actions
defined.
somewhat improves on #38224
* setup discouraged-tag linting for #p
- this is fine to use in dev, but shouldn't get comitted
so this will fail kondo linting.
* sample #p commit (check linting)
* Revert "sample #p commit (check linting)"
This reverts commit 171130c668.
I added it to double check the linter will complain correctly
API Key Configuration via config.yml
Overview
This PR adds support for configuring API keys through the config.yml file in Metabase Enterprise. This enhancement allows administrators to define and manage API keys declaratively through configuration, making it easier to set up and maintain API access across different environments.
Features
Adds support for defining API keys in config.yml under the api-keys section
API keys can be configured with:
name (required)
group ("admin" or "all-users")
creator (email of the user to assign it, user must exist in the instance or be created in the config file, must be superuser)
Automatically creates API keys on Metabase startup if they don't already exist
Skips creation of API keys that already exist (based on name)
Validates configuration including:
Required fields
Valid group values
Creator permissions
Example Configuration
version: 1
config:
api-keys:
- name: "My Admin API Key"
group: admin
description: "API key for admin operations"
creator: admin@metabase.com
- name: "My All Users API Key"
group: all-users
description: "API key for general operations"
creator: admin@metabase.com
Security Considerations
API keys are only created by admin users
Keys are properly hashed before storage
Existing keys are preserved to maintain continuity
Group validation ensures keys have appropriate permission levels
Use Cases
Automated deployment and configuration
Infrastructure as code
Consistent API key setup across environments
DevOps automation
Testing
Unit tests cover key creation, validation, and error cases
Tests for configuration parsing and application
Validation of group permissions
Error handling for invalid configurations
This enhancement is part of Metabase Enterprise's advanced configuration capabilities, providing more flexibility in managing API access through configuration files.
* Use timezoned timestamp for export filenames
* clean ns
* tweak docstring
* add unit test + make namespace aliases consistent
* use new function in xlsx exports
* feat: add malli schema, test converage to ldap api
Restores the malli schema that was causing e2e tests to fail on the
first attempt and updates those e2e tests to look for the new error
message returned when the api request is rejected due to schema
validation issues.
Expand test cases around ldap connection testing errors so those
messages can be caught on future api changes.
* fixup! error capitalization
* Add reproduction for #54638
* Add reproduction for #54722
* Add reproduction for #31964
* Correct user of metabase issue in test name
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Correct use of issue name in test title
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Use addCustomColumn helper
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Use helpers to open filter and summarize popovers
* Check for textbox role instead on cm-content class
* Just use enter
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
I enforced this by checking the return values of `visible-columns` and
`returned-columns`. I want to check that in (for dev and test only) but
it fails tests currently.
The trouble is that cards don't have `:ident` in their `result_metadata`.
That's coming later, so the enforcement flag is disabled for now.
* Truncate mrkdwn links
They could exceed block limits
* Truncate header block text
* Remove incorrect (testing) block in escaping test
* Make length consts under test public
* Add link truncation test
* Add header truncation tests
* Use lezer parser for custom expressions (#53540)
* Add @lezer/generator
* Add missing value on Token
* Add script to generate lezer grammar parser
* Add @lezer/lr
* Add more tests to tokenizer
* Add lezer grammar for custom expressions
* Generate lezer parser for custom expressions
* Convert TOKEN and OPERATOR to enums
* Make Token type more specific
* Add isReference to identifier token
* Typo
* Add more tests for tokenizer
* Add tokenize alternative that uses lezer grammar
* Remove old tokenizer
* Update location of grammer in generate rule
* Add test cases for tokens that share a prefix
* Merge tokens when necessary
* Only add character error when text is 1 character long
* Reduce the need to manually merge nodes
* Stop the iteration when node is found
* Update tests to be more specific
* Simplify tokenize
* Uncommit edge cases
* Generate parser as js
* Move previous tokenizer tests into expressions folder
* Add typing to unit test
* Allow escaping [] in reference
* Add test for escaping closing bracket
* Handle invalid parses manually
* Add support for signed expressions
* Document parser more
* Add custom tokenizer for Reference
* Make tests more explicit
* Fix e2e test
* Merge old test suite into tokenizer test suite
* Move @lezer/generator to devDependencies
* Custom expression suggestions (#53679)
* Add new suggesters for custom expressions
* Add test for joined fields
* Move complete into separate folder
* Split up completions into separate files
* Split up completion tests too
* Copy more tests from legacy completer
* Fix types
* Lezer highlight for custom expressions (#53651)
* Rename generated parser to lezer
* Add lezer highlighting rules to parser
* Add HighlightExpression component
* Use HighlightExpression in expression HelpText
* Add type annotation to highlighter
* Add test for highlight
* Simplify unit test
* Move syntax highlighting to shared module
* Rename util.tsx to util.ts
* Add monospace font variable
* Use monospace font from theme for syntax block
* Use relative font size in HighlightExpression
* Use mantine var for border-radius
* Add test id to example
* Fix tests that rely on example content
* Move highlighting code to metabase/ui/syntax
* Replace ACE with codemirror in custom expression editor (#53746)
* Rename generated parser to lezer
* Add lezer highlighting rules to parser
* Add HighlightExpression component
* Use HighlightExpression in expression HelpText
* Add type annotation to highlighter
* Add test for highlight
* Simplify unit test
* Move syntax highlighting to shared module
* Rename util.tsx to util.ts
* Add monospace font variable
* Use monospace font from theme for syntax block
* Use relative font size in HighlightExpression
* Use mantine var for border-radius
* Add test id to example
* Fix tests that rely on example content
* Move highlighting code to metabase/ui/syntax
* Replace custom expression editor with CodeMirror editor
* Add test mock for CodeMirror
* Colocate custom expression tests (#53812)
* Remove unused DISPLAY_QUOTES
* Colocate custom expressions diagnostic tests
* Colocate custom expressions process tests
* Colocate custom expressions recursive-parser tests
* Colocate custom expressions typeinferencer tests
* Colocate custom expressions completer tests
* Colocate custom expressions format tests
* Colocate custom expressions resolver tests
* Colocate custom expressions index tests
* Colocate custom expressions pratt compiler tests
* Colocate fuzz tests with custom expressions code
* Update paths for fuzz tests
* Fix types in fuzz test
* Fix types in process test
* Remove unused var
* Remove unused var in generator
* Merge types for custom expressions (#53902)
* Add missing expression types to metabase-types
* Add more explicit types to matchers
* Use Expression type from metabase-types everywhere instead of custom type
* Move matchers to separate file
* Further use Expression types for matchers
* Add types for Metric and Segment matchers
* Fix types in format
* Fix missing types
* Clean up imports
* Split string helpers into separate submodule
* Split identifier helpers into separate submodule
* Avoid importing from index file in expressions module
* Render suggestions and help text in new custom tooltip (#53760)
* Add new help text component for custom expression editor
* Add new suggestions listbox for custom expression editor
* Add helper to find help text by function name
* Add enclosingFunction helper that is more forgiving to broken syntax
* Add new tooltip component for custom expression editor
* Add extension to render custom react tooltips
* Render suggestions in custom tooltip
* Export custom types
* Remove old expression editor popover components
* Add typings to process
* Add missing shortcuts
* Fix example font size
* Fix type imports
* Fix click on helptext
* Stop click propagation
* Fix process types
* Remove superflouous optional chain
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix types whith optional expressionIndex
* Use imports instead of React namespace
* Use index for key
* Rename util to utils
* Remove unused box shadow
* Remove brackets around placeholder names
* Resize tooltip to avoid it overflowing the window
* Fix line height for args
* Fix missing key icons
* Fix chevron alignment
* Fix types
* Move Listbox resizing classes to Tooltip
* Fix completion template for functions with no arguments
* Add unit tests for enclosingFunction
* Add darker background to expression example
* Remove bold from function call
* Fix imports
* Fix HelpText tests
* Add custom color for highlighted syntax argument
* Use colors from design system
* Fix blurring when switching to a different tab
* Remove trailing comma
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Split up effect
* Remove background from template arguments
* Improve tooltip position
* Fix Popover position when selection changes
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix custom expression editor e2e tests (#53761)
* Fix e2e tests for custom expressions
* Fix unit test
* Fix e2e tests
* Fix types
* Fix moveToStart and moveToEnd by using Home/End
* Add test for popover position
* Remove JSDoc in favor of inline comments
* Add examples of side effects
* Remove shouldContain helper in favour of value()
* Add comment to wait
* Use CustomExpressionEditor helper instead of @formula
* Add wait to reduce flakiness
* Add back in the useful comment about popover position
* Remove superflouous cy.viewport call
* Add helper to accept completion
* Rename helper to verifyHelptextPosition
* Convert comments to cy.logs
* Make sure we don't avoid testing the regression
* Allow customising key with which to accept completion
* Use acceptCompletion helper
* Fix tooltip position bug
* Remove unnused wait
* Add selectCompletion helper
* Fix helper using H
* Add better emulation of copy/paste in reproduction
* Ignore ts error
* Multiline formatter for the custom expression editor (#53973)
* Make type name meaningful
* Add more tests for not isnull, etc
* Remove unreachable code
* Add isEmpty matcher
* Pass query to format tests
* Add stronger types to formatStringLiteral
* Fix CallOptions import
* Implement multiline formatter as a prettier plugin
* Reimplement legacy formatter as new formatter with infinite print width
* Use the new formatter in the expression editor
* Remove legacy expression format files
* Document utils
* Simplify Expression type further
* Fully type test cases
* Remove cast in test
* Add more tests and fix formatting
* Use recurse helper to recurse into path children
* Prefix logical operators
* Improve formatting of logical connectors
* Wait for expression editor to be ready
* Avoid passing unknown html props
* Use selectCompletion helper
* Remove unused @ts-expect-error
* Align UI more to design
* Remove shortcuts in suggestions popover
* Disable popular functions in completions
* Render shortcuts as separate component
* Move NameInput to separate component
* Fix button location when no name input is present
* Remove unused ExpressionWidgetInfo component
* Avoid flicker of shortcut buttons
* Tweak name input padding
* Deduplicate styleTags in Editor
* Do not format on blur and pass through all valid changes
* Split of diagnose and compile step
* Remove unused props
* Do not trigger completion on focus
* Show update button when no name is passed
* Expose expression types
* Use diagnoseAndCompileExpression in lint extension too
* Add toolbar with format button
* Only show popover when user has interacted with editor
* Throw ResolverError when types mismatch
* Fix missing highligh for boolean
* Render errors
* Disable format button when the expression contains an error
* Remove shortcut and popular function completion
* Move Shortcut type to component that uses it
* Fix blur for CustomExpression helper
* Add format helper
* Fix e2e tests
* Fix format helper
* Fix blur helper
* Use actual ellipsis in placeholder
* Add helper for nameInput
* Remove buttons from tab sequence once hidden
* Do not autofocus shortcuts
* Hide shortcuts when error occurs
* Allow reading multiline value
* Fix e2e tests
* Fix unit tests
* Add more tests for multiline custom expression editor
* Debounce error checking
* Add max height to tooltip
* Fix formatting
* Document changing reproduction
* Test wording
* Disable format button when expression is empty
* Do not try compiling when source is empty
* Cancel changes when source changes
* Autofocus editor
* Use Tooltip to mark button
* Remove default autoFocus on button
* Simplify shortcuts show and hide style
* Move shortcuts styles to parent element
* Fix line number alignment
* Add a gap between Cancel and Done buttons
* Always close helptext when suggestions are being shown
* Add usePreventClosePopover hook
* Prevent closing popover from the custom expression editor
* Add tests for not closing on click outside and escape
* Move toolbar down
* Fix e2e tests
* Add format_code icon
* Use correct icon for formatting code
* Use new format code icon in native editor too
* Update Loki Snapshots
* Merge useEffect cases into one
* Allow clicking the shortcuts
* Debounce validation
* Fix React update loop
* Prevent flicker in the listbox completions
* Tweak the linting delay
* Hide shortcuts when there is an error
* Only prevent closing popover when editor is rendered
* Fix width and height for editor widget
* Fix format button positioning
* Fix button alignment in expression editor
* Allow blurring tooltip by clicking nothing
* Add button role to header
* Show modal when clicking outside of editor
* Allow closing editor when it's empty
* Add tests for clicking outside of expression editor
* Fix missing onClose handler
* Add bigger timeout
* Rename Format to Auto-format
* Hide format button when there is an error or editor is empty
* Fix label names in test
* Fix format button test
* Deduplicate tests for exiting the editor
* Fix test
* Fix repro
* Only prevent closing popover when the source has changed
* Fix e2e test
* Hide lint point
* Fix types
* Fix tooltip helptext layout
* Update loki reference
* Only show format button when the expression was validated
* Scale the expression editor with the screen width
* Move usePreventClosePopover into utils
* Remove CloseModal component
* Remove custom PreventCloseContext from Popover
* Skip tests that deal with the click outside and escape behaviour of the custom expression editor
* Remove canPreventCloseHandlers from ClausePopover
* Add lineWrapping to editor
* Fix loki
* Fix loki screenshots
* Fix e2e test
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Apparently, a filter like `blah in (cast([uuid] as uuid))` is fine in postgres, but sqlserver doesn't like it very
much. This gives sql server the syntax it needs.
* first crack at linear pr link bot
* try it again
* bump actions versions
* bump
* try again
Added Node.js setup step
Added dependency installation step for node-fetch
Changed the ES module import to a CommonJS require statement
Changed variable name from fetch to nodeFetch for clarity
* extract script
* use the right graphql query
* add debug
* look at the right part of the linear api response
* lets see the issue numbers
* again
* again
* again
* again
* again
* again
* again
* again
* again
* fix formatting
* remove ???'s
* improve logging
* check all magic closing words
* refine checking logic
* test
* some review responses
* again
* make github object a thing
* install octokit
* install octokit again
* install octokit again
* running
* running
* running
* running
* running
* delimit message with spaces
* delimit message with spaces
* delimit message with spaces
* delimit message with spaces
* test
* only try to close metabase/metabase issues
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* support react 19 version range in embedding sdk
* sync supported react versions
* add oxford comma
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* revert docs and cli updates as we want to do a silent release
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* add use-toast-hook
* use the new toast hook
# Conflicts:
# frontend/src/metabase/admin/performance/components/ModelPersistenceConfiguration.tsx
* bonus change slipped in
* use those callbacks
* add new database header, make layout adjustments
* start building connection section
* reorganize connection info section content, migrate danger info section
* reorg
* flow rewired together mostly, connection details form is now in a modal
* make section content components the entire section for better hide/show logic, remove old code comments, show proper engine name in edit db header, hide model actions section if no actions can be taken
* lots of polishing
* fix BE punctuation
* test e2e fixes in ci
* more test fixes
* more test fixes
* fix unit tests
* fix last failing unit test
* fix last e2e test?
* final polish pass from self review
* fix model persistence section and its related tests, reorganize the tests
* fix model persistence section and its related tests, reorganize the tests
* impl feedback from arakaki around creation flow and date format
* clean up last commit
* make modal scroll container limited to the body, remove use of force to find inputs in cypress tests
* impl PR feedback
* fix: question stats are calculated in sql query
Previously instances could OOM when calculating these question stats if
they had too many cards because we load all these cards into memory.
We were doing this full calculation because we have some data we want to
calculate metrics over stored as json in a text column. This uses native
mysql/mariadb and postgresql json fuctions to calculate those metrics
inside the database prevent us from needing to load everything. That
also means we no longer calculate these json-based metrics for metabase
instances running h2.
* fixup! use older path_exists json function
* Add a native editor placeholder text
Resolves CLO-3674
* Keep the "data reference" panel open by default
Resolves CLO-3675
[ci skip]
* Add empty state for snippet sidebar
Resolves CLO-3684
* Empty state for query results panel
Resolves CLO-3676
* More refined way to initialize data reference
* Update copy in e2e tests
* Fix more tests
* Open data reference whenever native editor is open
* Do not alter the initializeQB flow
* Close the data reference sidebar when question settings are open
* Fix a failing Mongo test
* Address review comment
* Improve _.compose type + fix SDK types
* Use a simpler type overloading
* Fix Englishz
* Fix type errors
* Add a comment to clarify extrat type overload
* Clarify the source of code
* add fall-through for ignored embedding requests
* Gsheets mb api (#51307)
* send flag to show "connect gsheets" button to FE
* clean up docstring + add etl-connections condition
* metabase api WIP
* align APIs
* include site-url in the HM temp-url api call
* cleanup
* update HM<->MB api to be working (minus temp-url)
* cleanup
* remove some notes
* indentation + linting
* add tests
* fix typo
* remove illustrative (when true ...)
* add frontend for gsheets sync
* update frontend to use changed API
* Gsheets integration: more api alignment (#51492)
* add frontend for gsheets sync
* update frontend to use changed API
* wip
* cleanup
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* fix: declare gsheet setting fn before using in getter
* fix: delay u.secret instantiation to prevent build race condition
See https://metaboat.slack.com/archives/CKZEMT1MJ/p1734605793293989
* build: push ee-extra to dev repo, not ee
Cloud wants ee-extra, not just ee
* move settings out of auth middleware ns
- update kondo modules
- add metabase.api.auth ns
* More cleaning up
- update docstring
- remove declares + use {set,get}-value-of-type{!,}
- remove nil state from gsheets setting
* rely on more hm stubs
* save folder_owner to gsheets
* gsheets and hm http client => enterprise
+ move tests too
* require the right namespace
* add migration for is_gsheet to metabase_table
* Gsheets service account be flow (#51997)
* update setup flow to use service-account
* align GET api/ee/gsheets/service-account response
* align the HM service account endpoint
* fix migration yaml
* update test for service accounts
* indentation + migration quibble
* improve error messages + kondo housekeeping
* remove unreachable if-let branch + fix serdes test
* add is_gsheet to mock and test data
* add gsheets setting to serialization test
* fix fe-type-check
* typechecker
* [kondo] defendpoint -> defendpoint2
* Gsheets table management (#52145)
* include dwh ∈ GET /api/ee/upload-management/tables
* cljfmt
* simplify preliminary gsheet definition
* archive cards is a query param
* gsheets: update polling logic + better errors (#52162)
* update polling logic + better errors
* codify protocol for polling folder done-ness
* align on FE polling logic + the real sync complete
* [WIP] testing happy path
* status should be "active"
- remove a bunch of extra debugging
* remove outdated tests + cljfmt
* check admin + use the connection's id w/ delete
- cleanup
* fix :delete /folder + schema on :get /folder
* Gsheets frontend (#51946)
* non-oauth UI
* update api and add disconnect modal
* update illustration and add error displays
* add gsheets sync status
* better copy
* remove test code'
* fix bad api error
* updates to sync status
* Update sidebar entrypoint
* gate things behind the right features
- only show gsheets button
- http requests to /api/ee/gsheets/*
* model/Table has a db_id column not database_id
* add and use simple events
* differentiate failure modes
* + tests and error status codes
* make form submit on enter
* reset gsheet status when there is no gdrive conn
* add debug info
* FIXME FOR TESTING - disable feature checks
* fail on sync error + partially allow deletes
* add database page entrypoint
* FIXME FOR TESTING - set token-status-cache-ttl to 1 second (vs 12 hrs)
* only show _active_ dwh tables
* fix config.edn (howd it break tho?)
* handle multiple gdrive connections
- on delete, delete them all
- when looking at syncing, look at the most recent one
* add auto-cruft-tables to metabase_database.setting
- Here's the contract:
There should be an array of strings located at: db > settings >
"auto-cruft-tables". Each string will get fed into re-pattern. The table
names get lowercased before getting matched on. That list of regexes is
combined with our built in list of crufty regex patterns. We use re-find
to match them against the lower-cased table names. If there's a match we
will mark the table crufty (this is called a "Hidden Table" in the UI).
This way, if someone names their sheet `I'mUnique____dlt_loads` or
whatever they can at least fish it out of the hidden section.
Note: It will get set back to being crufty (aka what the ui calls
hidden) on the next sync... but this is configurable so we can fine-tune
the regexes to match the tables we want to hide more closely.
Examples:
If you put ["a"] in the config.yml: any table with A or a in it will be
marked hidden. So be sure to use anchors for the suffix or what-not. For
our thing, I think we want something like:
* add bunch of tests for auto-cruft
* mocked api tests
* add commment + mock hm responses
* documentation
* when multiple gdrive conns, sort by status, then created-at time
* cleanup
* joda time Instant/ofEpochSecond 0 -> t/instant 0
* if no gdrive conns are found, reset state
- add gdrive sorting tests
* indentation
* fix complete sync state and add error state
* update sync unit tests
* update sync unit tests
* add frontend analytics events
* don't show reconnect copy from DB page
* Add 10 minute timeout for polling connection
- return errors as json-encoded :error in the response body
- minor cleanup how we auto cruft tables (no change to behavior)
- add script for running the flow locally: from the repl
* better FE sync state and form validation
* refresh setting on error
* remove fake error state
* Delete connections on reset
- Change order of timeout checks in :get /folder handler
- add auto-cruft-columns setting
* update gsheets routes to use the new ns-handler
* loosen up gdrive link validation
* update regex
* add docstring to dynamic timeout var
* simplify api use via gdrive conn id
- move ->config into hm.client ns
* recover from suddenly-missing gdrive conn @ id
* cleanup hm.client
- indentation
fix [[make-request]] args
* fail to load invalid + present cruft patterns
* remove some extra comments + use hm-delete-conn
* fix update-table-metadata
- (no longer update nil -> nil)
* auto cruft columns
- pull the regex comparrison code into crufty ns
- reuse it on 2
- add defsetting so having the setting on the database doesn't complain
- pass database down into field syncing + grab vis types
* indent
* fix a few tests
* fix gsheets tests
* add harbormaster client test
* kondo + cljfmt
* migrate to underscores for auto_cruft_*
- add tests for field + fix schema for table syncs
* use kebab-case for settings but accept snake_cased
* cljfmt + kebab case stragglers
* fix update-field-metadata-if-needed! signature
* fixing tests
* rename macro double -> run-twice
* remove test code
* prettier
* cruft settings are db-local
* Fix search index update job restart issue (#53229)
* more idiomatic crufty db-local settings
* add auto cruft fields test
* bump gsheets sync timeout to 10 seconds
* handle going back to loading state
* kondo + add empty cruft tables test
* add get folder timeout test, update dox,
- remove regex folder check
* add cruft columns config.yml setup tests
* remove def + actually run the test
* test code for bruyan
* reset gsheets status on every error response
* save dbId and actually show it
* bust cache on gsheets setting
* solve initial-sync-status-table-only-test failure
* Update src/metabase/sync/sync_metadata/crufty.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update enterprise/backend/src/metabase_enterprise/advanced_config/file/databases.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update enterprise/backend/src/metabase_enterprise/advanced_config/file/databases.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update enterprise/backend/src/metabase_enterprise/gsheets.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* respond to Noah's review comments
* kondo
* update modal copy
* update error copy
* status click target update
* improve crufty/name? api
* modularize gsheets
* remove temp gsheets
- token status checks happen once per 12h
- remove hm/conn and mb/sync-info from :get /folder response
* gdrive-conn is now unused
* remove comment
* fix modules
* add docstring
* add harbormaster module
* kondo
* fix the tests
* module linter
* fix typo
* cljfmt
* setup api-key setting before reading it
* sort ns
* ::snowplow/simple_event -> :snowplow/simple_event
* hard code error message
* remove test code
* update frontend tests
* update module linter config
* add simple schema snowplow event schema
* fix schema name
* update some tests
* add a test
* better docstring
* better docstrings
* fix e2e tests
* fix fe token mock
* respond to review comments
+ fix timeout 50000 -> 5000
+ make internal regex function private
* build: push ee-extra to dev repo, not ee
Cloud wants ee-extra, not just ee
* fix strings
* remove resolved comment
* adds prometheus metrics to gsheets api
- :metabase-gsheets/sync-creation-begin
- :metabase-gsheets/sync-creation-ok
- :metabase-gsheets/sync-creation-error
- failure-reason is one of:
- "timeout" - gsheets.api/sync-complete? didn't return true after 10 minutes
- "status_error" - HM said the sync connection had status == error
- :metabase-gsheets/connection-deleted
* Refines prometheus events
They are now:
- :metabase-gsheets/connection-creation-began
- :metabase-gsheets/connection-creation-ok
- :metabase-gsheets/connection-creation-error {:reason "status_error"}
- :metabase-gsheets/connection-creation-error {:reason "timeout"}
- :metabase-gsheets/connection-deleted
* comment
* allow resolution of snake_cased setting names
* review responses
* add logging if any part of the batch-delete fails
* remove unused schema
* Revert "remove unused schema"
This schema is actually used! 🎉
This reverts commit f5dbd78051.
* update loki icon reference image
* switch off flaky test setup
* mark date filter picker tests flaky (#54866)
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Filipe Silva <filipematossilva@gmail.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* WRK-114,WRK-115: Replace SchedulePicker with Schedule in Alerts.
Integrated a newer feature-rich component for Alerts scheduling.
Fixed bug related to incorrect handling of '*' values in minute and hour
alerts.
* Remove scaling transition from popInFromBottom animation.
This transition used for all modals led to a bug with sizing in some
Mantine components, when they were used inside a modal:
https://github.com/mantinedev/mantine/issues/6967#issuecomment-2417439117
* Reuse Personal Access Token on all tasks
* Fix permission when pushing to GitHub
* Use annotated tag for better information
* Remove steps for faster testing
* Revert "Remove steps for faster testing"
This reverts commit 8bd5d2e07f.
* Prefer slack_file for images, so metabase no longer requires a public files channel
This removes the requirement to have a public files channel so share
images or other files in slack bot posts. Simplifying the integration
and lessening requirements to get setup.
It enables the posting of images files directly to public or private
channels without sharing - improving privacy and security.
* Files channel deprecation note
* Drop slack-files-channel requirement from slack /settings API
* Remove remaining files-channel calls
* Remove files channel input from SlackSetupForm
* Remove slack-files-channel from SettingsManagerSettings
* Remove slack-files-channel from API docs
* Drop files channel configuration guidance
* Remove slack channel prompt/description
* Instead of polling files.read reuse the complete call
There was an accidental introduction of files.read.
The goal is to preserve the previous behaviour of reusing
completeUploadExternal.
This limits the oauth scope requirements of the bot token.
* If more than 50 slack blocks, send using multiple messages
* Escape mkdwn special-processing chars in card links
* Simplify slack attachment uploading
- Its easy to mapv
- (cond) over nested (if)s
- Fix out of date docstring
* Remove slack-files-channel from config-template.md
* Remove metabase_files check from e2e suite
* replace use of findDOMNode in core components
* do not wrap input in div
* always populate scrollElementRef
* remove scrollElement from TokenField
* do not export _TokenField
* remove scrollElementRef
* use appropriate ref container
* cleanup root container
* use the scrollingContainerRef instead
* pass the elementRef directly
* use full-width container
* remove full-width container
* use display contents to preserve layout
* attach listRootRef conditionally
* try to update Loki snapshots
* modify snapshots
* fallback to scrollingContainer instead of elementRef
* revert loki snapshot
* revert loki snapshot to master
* Do not use originalQuestion when changing the template tag's type
* Do not use originalQuestion when changing the template tag's type
* Fix test
* Fix
* Fix
* Fix
* Fix
* Fix
* build multi-arch arm image
* refactor artifact fetching and containerizing
* fixes after testing
* use old master for caution's sake
* enable multi-arch in publish workflow
* sync
Part of #54559
This adds `trace` level logging to:
- `metabase.driver.athena`
- `metabase.sync.sync-metadata.fields.sync-instances`
That should help us diagnose sync issues where describe-fields and
fingerprinting seem to be out of sync.
{clojure.core/print{:message"Use metabase.util.log instead of clojure.core/print"}
@@ -273,6 +273,7 @@
compojure.core/context{:message"Use metabase.api.util.handlers/route-map-handler instead of compojure.core/context, it is much faster and supports OpenAPI spec generation"}
compojure.core/defroutes{:message"Use metabase.api.util.handlers/defroutes instead of compojure.core/routes, it supports OpenAPI spec generation."}
compojure.core/routes{:message"Use metabase.api.util.handlers/routes instead of compojure.core/routes, it supports OpenAPI spec generation."}
clojurewerkz.quartzite.jobs/defjob{:message"Use metabase.task/defjob instead of clojurewerkz.quartzite.jobs/defjob"}
fipp.edn/pprint{:message"Use metabase.util.log instead of fipp.edn/pprint"}
honeysql.core/call{:message"Use hx/call instead because it is Honey SQL 2 friendly"}
honeysql.core/raw{:message"Use hx/raw instead because it is Honey SQL 2 friendly"}
@@ -285,7 +286,8 @@
me.flowthing.pp/pprint{:message"Use metabase.util.log instead of me.flowthing.pp/pprint"}
medley.core/random-uuid{:message"Use clojure.core/random-uuid instead of medley.core/random-uuid"}
metabase.driver/database-supports?{:message"Use metabase.driver.util/supports? instead of metabase.driver/database-supports?"}
metabase.lib.equality/find-column-indexes-for-refs{:message"Use lib.equality/closest-matches-in-metadata or lib.equality/find-closest-matches-for-refs instead of lib.equality/find-column-indexes-for-refs"}}
metabase.lib.equality/find-column-indexes-for-refs{:message"Use lib.equality/closest-matches-in-metadata or lib.equality/find-closest-matches-for-refs instead of lib.equality/find-column-indexes-for-refs"}
next.jdbc/active-tx?{:message"Use metabase.app-db.core/in-transaction? to see if we are in a transaction"}}
:discouraged-namespace
{camel-snake-kebab.core{:message"CSK is not Turkish-safe, use the versions in metabase.util instead."}
@@ -298,223 +300,98 @@
honeysql.types{:message"Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}
honeysql.util{:message"Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}}
:discouraged-tag{p{:message"You can use it, but don't commit it."}}
> If that is your first contribution to Metabase, please sign the [Contributor License Agreement](https://docs.google.com/a/metabase.com/forms/d/1oV38o7b9ONFSwuzwmERRMi9SYrhYeOrkbmNaq9pOJ_E/viewform) (unless it's a tiny documentation change). Also, if you're attempting to fix a translation issue, please submit your changes to our [POEditor project](https://poeditor.com/join/project/ynjQmwSsGh) instead of opening a PR.
> If that is your first contribution to Metabase, please sign the [Contributor License Agreement](https://docs.google.com/a/metabase.com/forms/d/1oV38o7b9ONFSwuzwmERRMi9SYrhYeOrkbmNaq9pOJ_E/viewform) (unless it's a tiny documentation change). Also, if you're attempting to fix a translation issue, please submit your changes to our [Crowdin project](https://crowdin.com/project/metabase-i18n) instead of opening a PR.
- name:Comment on PR if 'not included in the documentation' error happens
uses:./.github/actions/notify-pull-request
with:
remove-previous-comment:true
add-new-comment:${{ steps.docs-generation.outcome == 'failure' && contains(steps.docs-generation.outputs.output, 'not included in the documentation') }}
message:"📝 Some exported from `@metabase/embedding-sdk-react` types reference other types that are not exported, i.e. internal types.<br><br>The internal type reference breaks API documentation generation for the SDK and also may unexpectedly increase the output size of the type declaration file.<br><br>Please check the CI logs to find the name of such type.<br><br>- If a type should be publicly available in the SDK, add a description to it or a TypeDoc annotation and export it from SDK.<br>- If a type is internal and should _not_ be publicly exposed, refactor types to prevent referencing other types that are not exported from SDK.<br><br>You can run `yarn embedding-sdk:docs:generate` locally to verify that the issue has been resolved."
include-log:true
- name:Comment on PR if 'Failed to resolve link' error happens
message:"📝 A typedoc `{@link <TYPE_NAME>}` annotation is found that can't be resolved.<br><br>Most likely it happens when the referenced type is not exported. Please check the CI logs to find the name of such type.<br><br>- If a referenced in `{@link}` type should be publicly available in the SDK, add a description to it or a TypeDoc annotation and export it from SDK.<br>- If a type is internal and should _not_ be publicly exposed, change types to prevent unexported type reference in `{@link}` annotation.<br><br>You can run `yarn embedding-sdk:docs:generate` locally to verify that the issue has been resolved."
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.