Compare commits

..

51 Commits

Author SHA1 Message Date
Eric Normand
197b1c3016 Empty 2025-05-05 15:47:44 -05:00
Kyle Doherty
1b0ceb2c48 Update Title component sizes to match Figma (#54750)
* update title sizes to match figma - add multi line title stories for lineheight examples

* little more line height in modal titles

* little fixes

* more little fixes

* reduce smartscalar title size

* Update Loki Snapshots

---------

Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
2025-05-05 13:41:48 -06:00
Nick Fitzpatrick
c42252c802 Add watermark to static viz chart renderings (#57553)
* add watermark to static viz chart renderings

* adjusting unit test
2025-05-05 22:38:28 +05:30
Edward Paget
dd9c170d8d fix: add table-level sql based permission query (#57541)
* fix: add table-level sql based permission query

Some places in our code we may be returning fewer tables than a user has
access to because we load a limited number of models from the database
then apply permissions filters to those. This provides a subquery based
check for table permissions that we can use in other queries to do the
same kind of test for table access that we would do on the clojure-side.

Co-Authored-By: Nathan Voxland <nathan@metabase.com>

* fixup! restore check-permissions-for-model impl

* Update src/metabase/models/table.clj

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

* fixup! caluse -> clause spelling

* fixup! reformat malli annotations; add docstring

* fixup! add Literal type to honeysql helpers

* fixup! restore accidentally deleted clause

---------

Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
2025-05-05 11:48:38 -05:00
Ryan Laurie
c88a5b5bc5 remove use modal hook (#57601) 2025-05-05 10:39:07 -06:00
github-automation-metabase
aa0bd12ed3 Bump net.snowflake/snowflake-jdbc from 3.23.2 to 3.24.0 (#57587)
Inspect dependency changes here:

Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Riley Thompson <riley@metabase.com>
2025-05-05 11:20:53 -04:00
github-automation-metabase
a0bee28c26 Bump org.eclipse.jetty/jetty-server from 12.0.19 to 12.0.20 (#57581)
Inspect dependency changes here:

Co-authored-by: Metabase bot <metabase-bot@metabase.com>
2025-05-05 14:04:27 +00:00
Ngoc Khuat
ea279ca19b Email domain check when adding notification recipients (#57416) 2025-05-05 13:59:54 +00:00
github-automation-metabase
a8656deb25 Bump org.eclipse.jetty.ee9/jetty-ee9-servlet from 12.0.19 to 12.0.20 (#57580)
Inspect dependency changes here:

Co-authored-by: Metabase bot <metabase-bot@metabase.com>
2025-05-05 13:48:41 +00:00
Nick Fitzpatrick
8ebc12b933 Register Dynamic Shortcuts (#56812)
* Register Dynamic Shortcuts

* regex keys

* things more-less working

* adding admin shortcuts, patching kbar to pass keyboard event

* fixing type issues

* only emit events on keyboard shortcut executions

* remove wait
2025-05-05 17:35:30 +05:30
Nick Fitzpatrick
edb69d6dbb properly invalidate collections when x-raying a table (#57495) 2025-05-05 17:34:15 +05:30
Kamil Mielnik
8aac6aac61 Add label to CodeEditor in LogLevelsForm (#57524)
* Add label to CodeEditor in LogLevelsForm

* Remove required for consistency
2025-05-05 17:56:16 +07:00
Nemanja Glumac
691d1430f8 Add a new metric button to browse metrics page (#57496)
Resolves PRG-22
2025-05-05 12:53:00 +02:00
Nicolò Pretto
2adc027178 fix(sdk): manual fixes for module level ttag calls causing broken translations (#57391)
* fix: filter button labels

* fix: part of EMB-335, uiName: t => getUiName: () => t

* fix linting issue

* fixed unit test mock missing getUiName
frontend/src/metabase/visualizations/components/Visualization/Visualization.unit.spec.js
missing

* make application-permissions.cy.spec.js pass
2025-05-05 10:11:02 +02:00
Aleksandr Lesnenko
e3e035ac6e remove old static pie chart interface (#57545) 2025-05-03 00:56:14 +00:00
Alexander Polyankin
c7ee8b29d2 Fix isDate check in legacy (#57185) 2025-05-03 00:04:44 +00:00
Vamsi Peri
a23481188c Fix for Grey areas around the cards when exporting to PDF (#56928)
* cursor says this could fix the issue

* harder vibe coding

* remove handling of made up by llm elements that do not exist

---------

Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
2025-05-02 22:49:49 +00:00
Oleksandr Yakushev
0a4625fdbe [svg] Pool static-viz-context objects (#56648) 2025-05-02 23:34:37 +03:00
Nathan Voxland
ac81ee5c25 Fixed catching of 403 errors from harbormaster (#57504) 2025-05-02 14:04:34 -05:00
Cam Saul
99613cfe0d [Backend modularization] sample-data module (#57511)
* [Backend modularization] `sample-data` module

* Remove duplicate ns form
2025-05-02 11:51:47 -07:00
Nathan Voxland
e515a2d6d0 Further limit tsvector size since it's concatinated (#57514) 2025-05-02 13:20:26 -05:00
Riley Thompson
0c0aaf5490 druid jdbc localdate set param impl (#57507) 2025-05-02 10:56:28 -04:00
Riley Thompson
0bc9646ca0 set timezone for identity-hash-test tests (#57530) 2025-05-02 10:56:14 -04:00
Case Nelson
40507a0e21 fix: include system catalog in databricks multicatalog (#57484)
* fix: include system catalog in databricks multicatalog

* Fix formatting
2025-05-02 08:53:37 -06:00
github-automation-metabase
562617890f Bump com.facebook.presto/presto-jdbc from 0.291 to 0.292 (#56659)
Inspect dependency changes here:

Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Riley Thompson <riley@metabase.com>
2025-05-02 10:43:22 -04:00
github-automation-metabase
9e4fb1181c Bump io.trino/trino-jdbc from 431 to 475 (#57319)
Inspect dependency changes here:

Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Riley Thompson <riley@metabase.com>
2025-05-02 10:43:11 -04:00
github-automation-metabase
6422238118 Bump com.clickhouse/clickhouse-jdbc from 0.8.4 to 0.8.5 (#57318)
* Bump com.clickhouse/clickhouse-jdbc from 0.8.4 to 0.8.5

Inspect dependency changes here:

* Fix test for jdbc driver upgrade

---------

Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Riley Thompson <rileythompson99@gmail.com>
2025-05-02 10:42:04 -04:00
Romeo Van Snick
c4cb9263f6 Do not set card when using browser back button (#57389)
* Add reproduction for metabase#56775

* Pass the correct card to the location change

* Unwrap card when navigating to model or metric
2025-05-02 13:42:05 +02:00
Romeo Van Snick
003777b77d Clean up expression config (#57109)
* Rename ALL_NODES to NODE_TYPE

* Remove unnecessary name override

* Remove unused EXPRESSION_OPERATOR_WITHOUT_ORDER_PRIORITY

* Use precedence from pratt compiler

* Move EDITOR_FK_SYMBOLS to identifier module

* Lift const descriptor

* Remove unused characters config

* Remove identifierAlwaysQuoted from EDITOR_QUOTES config and make the behaviour the default

* Move identifierDefaultQuote out of config and hard-code the behaviour

* Flatten default quotes config and move default into string module

* Remove wrong comment

* Use parseOperatorType to find operator type

* Remove separate operator groups

* Remove unused FUNCTIONS variable

* Remove barely used getExpressionName

* Use Boolean token to parse booleans

* Remove True and False from operators

* Rename Star and Slash to Multiply and Divide

* Remove FIELD_MARKERS

* Simplify config definition

* Inline getDescriptionForNow

* Move OPERATOR check to formatter

* Use normal quotes in config

* Split up MBQL_CLAUSES to avoid duplicating names

* Split clause helpers into separate files

* Fix sort order in completion tests

* Move operators to a separate group in config

* Use EXPRESSION_OPERATORS in formatter

* Export using star

* Rename OPERATOR to PUNCTUATOR

* Rename parseOperatorType to parsePunctuator

* Rename parseOperatorType to parsePunctuator

* Add helper for potential function prefixes

* Remove unused OPERATORS

* Remove superflouous helper

* Inline formatCallExpression

* Use Lib.ExpressionOperator where possible

* Remove impossible case for expression

* Inline isLogicOperator

* Rename variable for clarity

* Deduplicate helpers

* Move formatter helpers into utils

* Simplify helper

* Fix import order

* Remove reliance on PUNCTUATOR type

* Fix variable name

* Simplify punctuator definition

* Clean up lexifier

* Flatten lexifier results

* Remove errors for parser results

* Flatten parser result

* Simplify parser tests

* Remove cast

* Remove unused import

* Translate all error messages in formatter

* Simplify type

* Use sentence case name for type
2025-05-02 13:19:47 +02:00
Alexander Solovyov
76133ade49 [serdes] break endless loops (#57482)
When making sure stuff can load even with a loop, the circuit breaker was removed, leading to endless loops. That
would happen if the data causing the loop wasn't stripped properly. Right now if stripping things once did not help
and the entity is in the pipeline again, the process will break (with hopefully enough data to detect why).

This also changes the logic from exception to just recursive calls. Plus it's back at skipping all dashcards in a
Dashboard instead of just skipping visualization logic, which should make for a way faster processing (less deep for
sure). Plus it tries to improve logging a bit so it's easier to understand what's going on in complex cases.
2025-05-02 10:30:38 +00:00
Nicolò Pretto
f29902368b sdk-storybook: disable reactDocGen to speed up storybook (#57440) 2025-05-02 10:47:58 +02:00
Sébastien
2c106817a4 closes viz-360 esc key should not add a recipient (#57253)
* closes viz-360 esc key should not add a recipient

* better code in e2e test to avoid eslint-disable-next-line
2025-05-02 09:47:19 +02:00
Sébastien
da5cd4e7cf Closes VIZ-349 unknown date style warnings (#57263)
* closes VIZ-349 unknown date style warnings

* safer approach
2025-05-02 07:20:03 +02:00
Alexander Polyankin
04bf793795 Drop param_values (#57508) 2025-05-02 00:39:30 +00:00
Fraser
fa9ea2a5a9 Create EmailSettingsPage (#56760)
* Create EmailSettingsPage

* Add email settings to mocks

* Create ReplyToWidget

* Add unit tests for email page

* Handle boolean conversion in AdminSettingInput
2025-05-02 11:45:51 +12:00
Cam Saul
a04e9bd9b0 [Backend modularization] testing-api module (#57503) 2025-05-01 15:53:14 -07:00
Cam Saul
85b3ab5d5f [Backend modularization] Move remaining Slack namespaces into channel module (#57502) 2025-05-01 15:09:14 -07:00
Cam Saul
41e3110f20 [Backend modularization] create new api-keys module (#57497)
* [Backend modularization] create new `api-routes` module

* Fix test 🔧

* [Backend modularization] `api-keys` module
2025-05-01 14:22:52 -07:00
Cam Saul
213535773b [Backend modularization] create new api-routes module (#57492)
* [Backend modularization] create new `api-routes` module

* Fix test 🔧
2025-05-01 20:16:03 +00:00
Jeff Bruemmer
c1e301eda6 docs - note on role setting (#57268)
* note on role setting

* format
2025-05-01 16:15:16 -04:00
Edward Paget
d82bf686b6 deps: remove simpleclient_jetty and javax.servlet (#57487) 2025-05-01 18:58:56 +00:00
Oisin Coveney
fe3792cda9 Convert AutomaticDashboardApp to a functional component (#57390) 2025-05-01 17:53:37 +00:00
lbrdnk
8278732860 Fix cumulative aggregations legacy transform (#57258)
* Fix cum-sum and cum-count legacy transform

* Address review remarks

* Adjsut test

* ag idents

* dissoc-in
2025-05-01 09:34:57 +02:00
Oisin Coveney
0fb515cbc6 TS Conversion: Automatic Dashboard App (#57075) 2025-05-01 08:09:35 +01:00
Cam Saul
2d038e518e Remove Legacy Metrics code from the backend (#57401)
* Remove Legacy Metrics code from the backend

* Code cleanup

* Test fix 🔧

* Test fix 🔧

* PR feedback

* Extra test

* Revert some X-Rays changes

* Remove commented-out test

* Test fixes 🔧
2025-04-30 19:06:26 -07:00
Cam Saul
fc56c810f9 [Backend modularization] native-query-snippets (#57460) 2025-04-30 23:18:03 +00:00
Riley Thompson
fd18b7708f Try specifying tz for identity hash test flake (#57408) 2025-04-30 22:40:06 +00:00
Cam Saul
1c113a608e [Backend modularization] upload module cleanup (#57414)
* `upload` module cleanup

* Fix MySQL

* More keyword fixes 🔧

* Keep uploads sorted
2025-04-30 15:15:58 -07:00
Noah Moss
5fabb3e1ec Fix subtotal label formatting for xlsx pivot exports (#57452)
* Fix subtotal label formatting for xlsx pivot exports

* untangle circular dep
2025-04-30 14:57:49 -07:00
Nathan Voxland
6da77fce6f Reindex immediately on start-up if existing table is old (#57159) 2025-04-30 16:39:46 -05:00
Riley Thompson
f947b70c35 Restrict Mongo sandboxing with saved questions (#56983)
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
2025-04-30 17:32:52 -04:00
456 changed files with 5446 additions and 4285 deletions

View File

@@ -585,7 +585,7 @@
metabase.test/with-temp-file clojure.core/let
metabase.test/with-user-in-groups clojure.core/let
metabase.test.gentest/defgentest clojure.test/deftest
metabase.upload-test/with-upload-table! clojure.core/let
metabase.upload.impl-test/with-upload-table! clojure.core/let
metabase.util/with-timer-ms clojure.core/fn
metabase.util.files/with-open-path-to-resource clojure.core/let
metabase.util.malli.defn/defn schema.core/defn
@@ -682,7 +682,7 @@
metabase.request.core/with-current-request hooks.common/with-used-first-arg
metabase.request.core/with-current-user hooks.common/with-used-first-arg
metabase.request.core/with-limit-and-offset hooks.common/with-used-first-two-args
metabase.sample-data-test/with-temp-sample-database-db hooks.common/with-one-binding
metabase.sample-data.impl-test/with-temp-sample-database-db hooks.common/with-one-binding
metabase.search.api-test/with-search-items-in-collection hooks.metabase.search.api-test/with-search-items-in-collection
metabase.search.api-test/with-search-items-in-root-collection hooks.common/do*
metabase.server.streaming-response-test/with-start-execution-chan hooks.common/with-one-binding

View File

@@ -46,11 +46,11 @@
metabase.analytics.prometheus
metabase.analytics.snowplow}
:uses #{api
channel
config
db
driver
eid-translation
integrations
internal-stats
models
premium-features
@@ -63,21 +63,69 @@
api
{:api #{metabase.api.auth
metabase.api.card
metabase.api.cards
metabase.api.collection
metabase.api.common
metabase.api.common.validation
metabase.api.dashboard
metabase.api.database
metabase.api.dataset
metabase.api.docs
metabase.api.field
metabase.api.geojson
metabase.api.logger
metabase.api.macros
metabase.api.open-api
metabase.api.query-metadata
metabase.api.routes
metabase.api.routes.common
metabase.api.table
metabase.api.user
metabase.api.util
metabase.api.util.handlers}
:uses #{actions
analytics
analyze
api-routes
channel
config
database-routing
db
driver
eid-translation
embedding
events
legacy-mbql
lib
logger
model-persistence
models
permissions
plugins
premium-features
pulse
query-processor
request
revisions
sample-data
search
server
session
settings
sso
sync
types
upload
util
xrays
enterprise/advanced-permissions}}
api-keys
{:api #{metabase.api-keys.api
metabase.api-keys.core}
:uses #{api db events models permissions util}}
api-routes
{:api #{metabase.api-routes.core}
:uses :any}
audit
@@ -98,6 +146,7 @@
metabase.cache.init}
:uses #{api config db events models premium-features settings util}}
;; TODO -- way too many API namespaces.
channel
{:api #{metabase.channel.api
metabase.channel.core
@@ -106,6 +155,8 @@
metabase.channel.init
metabase.channel.models.channel
metabase.channel.render.core
metabase.channel.settings
metabase.channel.slack
metabase.channel.template.core}
:uses #{analytics
api
@@ -114,7 +165,6 @@
driver
events
formatter
integrations
lib
models
notification
@@ -122,6 +172,7 @@
premium-features
settings
query-processor
task
timeline
types
util}}
@@ -136,7 +187,7 @@
{:api #{metabase.cmd
metabase.cmd.copy
metabase.cmd.dump-to-h2}
:uses #{api config db legacy-mbql models plugins query-processor search settings util}}
:uses #{api api-routes config db legacy-mbql models plugins query-processor search settings util}}
;; technically this 'uses' `enterprise/core` and `test` since it tries to load them to see if they exists so we know
;; if EE/test code is available; however we can ignore them since they're not real usages.
@@ -231,8 +282,8 @@
util}}
events
{:api #{metabase.events metabase.events.init metabase.events.notification}
:uses #{api channel driver models notification premium-features task-history util}}
{:api #{metabase.events metabase.events.init}
:uses #{api channel driver models premium-features util}}
formatter
{:api #{metabase.formatter}
@@ -242,10 +293,6 @@
{:api #{metabase.indexed-entities.api metabase.indexed-entities.init}
:uses #{analytics api driver legacy-mbql lib models query-processor search sync task util}}
integrations
{:api #{metabase.integrations.slack}
:uses #{events settings util}}
internal-stats
{:api #{metabase.internal-stats.core}
:uses #{db models}}
@@ -318,8 +365,7 @@
:uses #{api db driver events lib models premium-features settings query-processor request task task-history util}}
models
{:api #{metabase.models.api-key
metabase.models.application-permissions-revision
{:api #{metabase.models.application-permissions-revision
metabase.models.audit-log
metabase.models.card
metabase.models.card.metadata
@@ -340,8 +386,6 @@
metabase.models.init
metabase.models.interface
metabase.models.moderation-review
metabase.models.native-query-snippet
metabase.models.native-query-snippet.permissions
metabase.models.params
metabase.models.params.chain-filter
metabase.models.params.custom-values
@@ -361,6 +405,7 @@
:uses #{analytics
analyze
api
api-keys
audit
cache
channel
@@ -391,12 +436,19 @@
{:api #{metabase.moderation}
:uses #{models util}}
native-query-snippets
{:api #{metabase.native-query-snippets.api
metabase.native-query-snippets.core}
:uses #{api models permissions premium-features util}}
;; TODO -- way too many API namespaces. We need to prune these down.
notification
{:api #{metabase.notification.api
metabase.notification.api.notification
metabase.notification.core
metabase.notification.events.notification
metabase.notification.init
metabase.notification.models
metabase.notification.api.notification
metabase.notification.payload.core
metabase.notification.send}
:uses #{analytics
@@ -450,7 +502,7 @@
metabase.pulse.core
metabase.pulse.init
metabase.pulse.task.send-pulses}
:uses #{api channel config db driver events integrations models notification permissions plugins premium-features
:uses #{api channel config db driver events models notification permissions plugins premium-features
query-processor request task task-history util
enterprise/advanced-config enterprise/advanced-permissions enterprise/sandbox}}
@@ -524,8 +576,9 @@
:uses #{api config events models util}}
sample-data
{:api #{metabase.sample-data}
:uses #{plugins sync util}}
{:api #{metabase.sample-data.core
metabase.sample-data.init}
:uses #{plugins settings sync util}}
search
{:api #{metabase.search.api
@@ -536,7 +589,7 @@
metabase.search.ingestion
metabase.search.init
metabase.search.spec}
:uses #{analytics api audit config db driver models permissions premium-features settings request task util}}
:uses #{analytics api audit config db driver models permissions premium-features settings request startup task util}}
segments
{:api #{metabase.segments.api}
@@ -547,7 +600,7 @@
metabase.server.middleware.json
metabase.server.middleware.session
metabase.server.streaming-response}
:uses #{analytics api config core db driver embedding models premium-features settings session request util enterprise/sso}}
:uses #{analytics api api-keys api-routes config core db driver embedding models premium-features settings session request util enterprise/sso}}
session
{:api #{metabase.session.core
@@ -584,6 +637,10 @@
metabase.sso.init}
:uses #{api models permissions premium-features settings util}}
startup
{:api #{metabase.startup.core}
:uses #{util}}
sync
{:api #{metabase.sync.api
metabase.sync.core
@@ -602,7 +659,6 @@
config
db
driver
integrations
plugins
premium-features
settings
@@ -622,6 +678,12 @@
task
util}}
testing-api
{:api #{metabase.testing-api.api
metabase.testing-api.core
metabase.testing-api.init}
:uses #{analytics api config db premium-features search util}}
tiles
{:api #{metabase.tiles.api
metabase.tiles.init}
@@ -636,7 +698,9 @@
:uses #{util}}
upload
{:api #{metabase.upload}
{:api #{metabase.upload.api
metabase.upload.core
metabase.upload.init}
:uses #{analytics api driver events legacy-mbql lib model-persistence models permissions settings sync util}}
user-key-value
@@ -655,9 +719,10 @@
enterprise/advanced-config
{:api #{metabase-enterprise.advanced-config.api.logs
metabase-enterprise.advanced-config.models.pulse-channel
metabase-enterprise.advanced-config.models.notification
metabase-enterprise.advanced-config.init
metabase-enterprise.advanced-config.file}
:uses #{api db driver models permissions premium-features setup settings sync util}}
:uses #{api api-keys db driver models permissions premium-features setup settings sync util}}
enterprise/advanced-permissions
{:api #{metabase-enterprise.advanced-permissions.api.routes
@@ -757,6 +822,7 @@
:uses #{api
audit
db
driver
legacy-mbql
lib
models
@@ -770,7 +836,7 @@
enterprise/scim
{:api #{metabase-enterprise.scim.api metabase-enterprise.scim.routes}
:uses #{analytics api models permissions settings server util enterprise/serialization}}
:uses #{analytics api api-keys models permissions settings server util enterprise/serialization}}
enterprise/search
{:api #{}
@@ -784,13 +850,13 @@
enterprise/snippet-collections
{:api #{}
:uses #{models permissions premium-features util}}
:uses #{models native-query-snippets permissions premium-features util}}
enterprise/sso
{:api #{metabase-enterprise.sso.api.routes
metabase-enterprise.sso.integrations.sso-settings
metabase-enterprise.sso.integrations.sso-utils}
:uses #{api channel embedding events premium-features request session settings sso util enterprise/scim}}
:uses #{api channel embedding events notification premium-features request session settings sso util enterprise/scim}}
enterprise/stale
{:api #{metabase-enterprise.stale.routes}

View File

@@ -91,7 +91,7 @@
metabase.query-processor.streaming.interface/begin!
metabase.query-processor.streaming.interface/finish!
metabase.query-processor.streaming.interface/write-row!
metabase.sample-data/try-to-extract-sample-database!
metabase.sample-data.impl/try-to-extract-sample-database!
metabase.settings.models.setting-test/test-user-local-allowed-setting!
metabase.settings.models.setting-test/test-user-local-only-setting!
metabase.settings.models.setting.cache/restore-cache!
@@ -126,7 +126,7 @@
metabase.test/initialize-if-needed!
metabase.test/test-helpers-set-global-values!
metabase.test/with-temp-env-var-value!
metabase.upload-test/set-local-infile!
metabase.upload.impl-test/set-local-infile!
metabase.util.files/create-dir-if-not-exists!
metabase.util.ssh-test/start-mock-servers!
metabase.util.ssh-test/stop-mock-servers!}}}}

View File

@@ -190,7 +190,7 @@
metabase.request.core/with-limit-and-offset [[:inner 0]]
metabase.request.session/as-admin [[:block 0]]
metabase.request.session/with-current-user [[:inner 0]]
metabase.sample-data-test/with-temp-sample-database-db [[:block 1]]
metabase.sample-data.impl-test/with-temp-sample-database-db [[:block 1]]
metabase.search.api-test/do-test-users [[:block 1]]
metabase.server.middleware.session/as-admin [[:block 0]]
metabase.server.middleware.session/with-current-user [[:inner 0]]
@@ -274,7 +274,7 @@
metabase.test/with-user-in-groups [[:inner 0]]
metabase.timeseries-query-processor-test.util/test-timeseries-drivers [[:block 0]]
metabase.timeseries-query-processor-test/run-mbql-query [[:inner 0]]
metabase.upload-test/with-upload-table! [[:inner 0]]
metabase.upload.impl-test/with-upload-table! [[:inner 0]]
metabase.upload.types/with-currency [[:default]]
metabase.upload.types/with-parens [[:default]]
metabase.util.async/cancelable-thread [[:block 0]]

1
.gitignore vendored
View File

@@ -6,6 +6,7 @@
*.po~
*.sqlite
*.trace.db
*.trace.db.old
*.xcworkspacedata
.DS_Store
.\#*

Binary file not shown.

Before

Width:  |  Height:  |  Size: 70 KiB

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 69 KiB

After

Width:  |  Height:  |  Size: 69 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.7 KiB

After

Width:  |  Height:  |  Size: 2.3 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.1 KiB

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 26 KiB

After

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 34 KiB

After

Width:  |  Height:  |  Size: 37 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 43 KiB

After

Width:  |  Height:  |  Size: 47 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 37 KiB

After

Width:  |  Height:  |  Size: 43 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 38 KiB

After

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 37 KiB

After

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 37 KiB

After

Width:  |  Height:  |  Size: 40 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 38 KiB

After

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 35 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 35 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 53 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 33 KiB

View File

@@ -25,7 +25,7 @@ module.exports = {
options: {},
},
typescript: {
reactDocgen: "react-docgen-typescript",
reactDocgen: false,
},
webpackFinal: storybookConfig => ({
...storybookConfig,

View File

@@ -86,10 +86,8 @@
:git/url "https://github.com/eerohele/pp"}
io.github.metabase/macaw {:mvn/version "0.2.25"} ; Parse native SQL queries
io.github.resilience4j/resilience4j-retry ^:antq/exclude ; 2.x do not support Java 8
{:mvn/version "1.7.1"} ; Support for retrying operations
{:mvn/version "1.7.1"} ; Support for retrying operations
io.prometheus/simpleclient_hotspot {:mvn/version "0.16.0"} ; prometheus jvm collector
io.prometheus/simpleclient_jetty {:mvn/version "0.16.0"} ; prometheus jetty collector
javax.servlet/servlet-api {:mvn/version "2.5"} ; used by ring's multipart-params (file upload)
junegunn/grouper {:mvn/version "0.1.1"} ; Batch processing helper
kixi/stats {:mvn/version "0.5.7" ; Various statistic measures implemented as transducers
:exclusions [org.clojure/data.avl]}
@@ -139,6 +137,7 @@
org.bouncycastle/bcpkix-jdk18on {:mvn/version "1.80"} ; Bouncy Castle crypto library -- explicit version of BC specified to resolve illegal reflective access errors
org.bouncycastle/bcprov-jdk18on {:mvn/version "1.80"}
org.clj-commons/claypoole {:mvn/version "1.2.2"} ; Threadpool tools for Clojure
org.clj-commons/dirigiste {:mvn/version "1.0.4"} ; Instrumented object pooling
org.clj-commons/hickory {:mvn/version "0.7.7" ; Parse HTML into Clojure data structures
:exclusions [org.jsoup/jsoup]}
org.clojars.pntblnk/clj-ldap {:mvn/version "0.0.17"} ; LDAP client
@@ -161,9 +160,9 @@
org.clojure/tools.namespace {:mvn/version "1.5.0"}
org.clojure/tools.reader {:mvn/version "1.5.2"}
org.clojure/tools.trace {:mvn/version "0.8.0"} ; function tracing
org.eclipse.jetty/jetty-server {:mvn/version "12.0.19"}
org.eclipse.jetty/jetty-server {:mvn/version "12.0.20"}
org.eclipse.jetty/jetty-unixdomain-server {:mvn/version "12.0.19"}
org.eclipse.jetty.ee9/jetty-ee9-servlet {:mvn/version "12.0.19"}
org.eclipse.jetty.ee9/jetty-ee9-servlet {:mvn/version "12.0.20"}
org.eclipse.jetty.ee9.websocket/jetty-ee9-websocket-jetty-server {:mvn/version "12.0.19"}
org.flatland/ordered {:mvn/version "1.15.12"} ; ordered maps & sets
org.graalvm.polyglot/js-community {:mvn/version "24.2.0" :extension "pom"} ; JavaScript engine

View File

@@ -94,6 +94,8 @@ Note that only the `*` wildcard is supported; you can't use other special charac
## Role (optional)
> **WARNING:** Metabase gets _all the permissions_ combined from _every_ role you grant the user in Snowflake. Setting the 'Role' here just adds the role to the connection string to tell Snowflake which role to default to – it **won't restrict** access on its own. So **be careful which roles you assign to the connection user**: if down the line you grant that user broader roles in Snowflake, Metabase will automatically gets those new permissions, potentially showing data you didn't mean for it to see. If instead you want Metabase to issue `SET ROLE` commands before querying anything, check out [connection impersonation](../../permissions/impersonation.md).
Specify a role to override the database user's default role. For example, if the database user `METABASE` has the roles:
- Default role `APPLICATION`.

View File

@@ -4,6 +4,10 @@ title: API changelog
# Breaking changes to the API interface
## Metabase 0.55.0
- `POST /api/card/from-csv` has been renamed to `POST /api/upload/csv`.
## Metabase 0.54.0
- The alert system has been migrated from the legacy pulse infrastructure to the new notification system. This migration includes the following changes:

View File

@@ -20,6 +20,11 @@ title: Driver interface changelog
- Added the multi-method `integer-dbtype` that allows the driver to control which type integers are cast to.
- The `metabase.upload` namespace has been replaced with `metabase.upload.core`, but upload type keywords e.g.
`:metabase.upload/varchar-255` remain unchanged. Make sure you weren't using `::` keywords inside methods like
`metabase.driver/upload-type->database-type` or `metabase.driver/allowed-promotions` -- make sure you use
`:metabase.upload/varchar-255` rather than something like `::upload/varchar-255`.
## Metabase 0.54.0
- Added the multi-method `allowed-promotions` that allows driver control over which column type promotions are supported for uploads.

View File

@@ -3,7 +3,7 @@ export const shortcutModal = () =>
cy.findByRole("dialog", { name: "Shortcuts" });
export const openCommandPalette = () => cy.get("body").type("{ctrl+k}{cmd+k}");
export const openShortcutModal = () => cy.get("body").type("?");
export const openShortcutModal = () => cy.get("body").type("{shift+?}");
export const commandPaletteButton = () =>
cy.findByTestId("app-bar").findByRole("button", { name: /Search/ });

View File

@@ -56,7 +56,7 @@ export function uploadFile(
testFile,
uploadMode = "upload",
) {
cy.intercept("POST", "/api/card/from-csv").as("uploadCSV");
cy.intercept("POST", "/api/upload/csv").as("uploadCSV");
cy.intercept("POST", "/api/table/*/append-csv").as("appendCSV");
cy.intercept("POST", "/api/table/*/replace-csv").as("replaceCSV");
@@ -104,7 +104,7 @@ export function headlessUpload(collectionId, file) {
formData.append("collection_id", collectionId);
cy.request({
url: "/api/card/from-csv",
url: "/api/upload/csv",
method: "POST",
headers: {
"content-type": "multipart/form-data",

View File

@@ -548,9 +548,12 @@ describe("scenarios > admin > settings > email settings", () => {
Cypress.config().baseUrl + "/admin/settings/email",
);
cy.findByTestId("smtp-connection-card").should("exist");
// Non SMTP-settings should save automatically
cy.findByLabelText("From Address").type("mailer@metabase.test").blur();
cy.findByLabelText("From Address")
.clear()
.type("mailer@metabase.test")
.blur();
cy.findByLabelText("From Name").type("Sender Name").blur();
cy.findByLabelText("Reply-To Address")
.type("reply-to@metabase.test")

View File

@@ -187,6 +187,12 @@ describe("scenarios > x-rays", { tags: "@slow" }, () => {
cy.findAllByTestId("dashcard").contains("18,760");
// eslint-disable-next-line no-unscoped-text-selectors -- deprecated usage
cy.findByText("How these transactions are distributed");
H.openNavigationSidebar();
H.navigationSidebar()
.findByRole("link", { name: /Automatically generated dashboards/i })
.should("exist");
});
it("should start loading cards from top to bottom", () => {

View File

@@ -1,13 +1,15 @@
const { H } = cy;
import { USERS } from "e2e/support/cypress_data";
import { SAMPLE_DATABASE } from "e2e/support/cypress_sample_database";
import {
FIRST_COLLECTION_ID,
ORDERS_MODEL_ID,
} from "e2e/support/cypress_sample_instance_data";
import type { StructuredQuestionDetails } from "e2e/support/helpers";
const { ORDERS_ID, ORDERS, PRODUCTS_ID, PRODUCTS } = SAMPLE_DATABASE;
type StructuredQuestionDetailsWithName = H.StructuredQuestionDetails & {
type StructuredQuestionDetailsWithName = StructuredQuestionDetails & {
name: string;
};
@@ -117,6 +119,35 @@ describe("scenarios > browse > metrics", () => {
).should("be.visible");
cy.findByText("Create metric").should("not.exist");
});
cy.log("New metric header button should not show either");
cy.findByTestId("browse-metrics-header")
.findByLabelText("Create a new metric")
.should("not.exist");
});
it("user without a collection access should still be able to create and save a metric in his own personal collection", () => {
cy.intercept("POST", "/api/card").as("createMetric");
cy.signIn("nocollection");
cy.visit("/browse/metrics");
cy.findByTestId("browse-metrics-header")
.findByLabelText("Create a new metric")
.click();
cy.findByTestId("entity-picker-modal").findByText("People").click();
cy.findByTestId("edit-bar")
.should("contain", "New metric")
.button("Save")
.click();
H.modal()
.should("contain", "Save metric")
.and("contain", H.getPersonalCollectionName(USERS["nocollection"]))
.button("Save")
.click();
cy.wait("@createMetric");
cy.location("pathname").should("match", /^\/metric\/\d+-.*$/);
});
});

View File

@@ -1461,3 +1461,34 @@ describe("issue 56698", () => {
H.assertQueryBuilderRowCount(1);
});
});
describe("issue 56775", () => {
const MODEL_NAME = "Model 56775";
beforeEach(() => {
H.restore();
cy.signInAsNormalUser();
H.createQuestion(
{
type: "model",
name: MODEL_NAME,
query: {
"source-table": PRODUCTS_ID,
},
},
{ visitQuestion: true },
);
});
it("should render the correct query after using the back button in a model (metabase#56775)", () => {
H.openNotebook();
cy.button("Visualize").click();
cy.go("back");
H.openQuestionActions("Edit query definition");
cy.log("verify that the model definition is visible");
H.getNotebookStep("data").findByText(MODEL_NAME).should("not.exist");
H.getNotebookStep("data").findByText("Products").should("be.visible");
});
});

View File

@@ -5,10 +5,32 @@ import {
ORDERS_BY_YEAR_QUESTION_ID,
ORDERS_COUNT_QUESTION_ID,
ORDERS_DASHBOARD_ID,
ORDERS_QUESTION_ID,
} from "e2e/support/cypress_sample_instance_data";
import { createMockDashboardCard } from "metabase-types/api/mocks";
const { admin } = USERS;
const TAB_1 = {
id: 1,
name: "Tab 1",
};
const TAB_2 = {
id: 2,
name: "Tab 2",
};
const TAB_3 = {
id: 3,
name: "Tab 3",
};
const TAB_4 = {
id: 4,
name: "Tab 4",
};
describe("command palette", () => {
beforeEach(() => {
H.restore();
@@ -413,13 +435,31 @@ H.describeWithSnowplow("shortcuts", { tags: ["@actions"] }, () => {
event: "keyboard_shortcut_performed",
event_detail: "create-new-collection",
});
H.openCommandPalette();
H.commandPalette().findByRole("option", { name: "New dashboard" }).click();
cy.findByRole("dialog", { name: /dashboard/i }).should("exist");
cy.realPress("Escape");
// Using a command palette action registered as a shortcut should only
// emit snowplow events when using keyboard shortcuts, not command palette
H.expectGoodSnowplowEvent(
{
event: "keyboard_shortcut_performed",
event_detail: "create-new-dashboard",
},
0,
);
cy.realPress("c").realPress("d");
cy.findByRole("dialog", { name: /dashboard/i }).should("exist");
cy.realPress("Escape");
H.expectGoodSnowplowEvent({
event: "keyboard_shortcut_performed",
event_detail: "create-new-dashboard",
});
H.expectGoodSnowplowEvent(
{
event: "keyboard_shortcut_performed",
event_detail: "create-new-dashboard",
},
1,
);
cy.realPress("g").realPress("d");
cy.location("pathname").should("contain", "browse/databases");
@@ -488,18 +528,54 @@ H.describeWithSnowplow("shortcuts", { tags: ["@actions"] }, () => {
cy.realPress("Escape");
cy.realPress("[");
H.navigationSidebar().should("not.visible");
cy.findByLabelText("Settings menu").click();
H.popover().findByText("Admin settings").click();
cy.findByTestId("site-name-setting").should("exist");
cy.location("pathname").should("contain", "/admin/settings");
cy.realPress("3");
cy.location("pathname").should("contain", "/admin/datamodel");
cy.realPress("7");
cy.location("pathname").should("contain", "/admin/tools");
});
it("should support dashboard shortcuts", () => {
H.visitDashboard(ORDERS_DASHBOARD_ID);
H.createDashboardWithTabs({
tabs: [TAB_1, TAB_2, TAB_3, TAB_4],
dashcards: [
createMockDashboardCard({
id: -1,
card_id: ORDERS_QUESTION_ID,
dashboard_tab_id: TAB_1.id,
}),
createMockDashboardCard({
id: -2,
card_id: ORDERS_QUESTION_ID,
dashboard_tab_id: TAB_2.id,
}),
createMockDashboardCard({
id: -3,
card_id: ORDERS_QUESTION_ID,
dashboard_tab_id: TAB_3.id,
}),
createMockDashboardCard({
id: -4,
card_id: ORDERS_QUESTION_ID,
dashboard_tab_id: TAB_4.id,
}),
],
}).then((dashboard) => H.visitDashboard(dashboard.id));
H.openShortcutModal();
H.shortcutModal().should("exist");
cy.realPress("Escape");
cy.realPress("o");
H.openNavigationSidebar();
H.navigationSidebar()
.findByRole("tab", { name: /bookmarks/i })
.should("contain.text", "Orders in a dashboard");
.should("contain.text", "Test Dashboard");
cy.realPress("o");
H.navigationSidebar()
.findByRole("tab", { name: /bookmarks/i })
@@ -520,6 +596,31 @@ H.describeWithSnowplow("shortcuts", { tags: ["@actions"] }, () => {
cy.findByRole("dialog", { name: "Info" }).should("exist");
cy.realPress("]");
cy.findByRole("dialog", { name: "Info" }).should("not.exist");
cy.findByRole("tab", { name: "Tab 1" }).should(
"have.attr",
"aria-selected",
"true",
);
cy.realPress("3");
cy.findByRole("tab", { name: "Tab 3" }).should(
"have.attr",
"aria-selected",
"true",
);
cy.realPress("1");
cy.findByRole("tab", { name: "Tab 1" }).should(
"have.attr",
"aria-selected",
"true",
);
// Doesn't error on pressing numbers out of bounds
cy.realPress("7");
cy.findByRole("tab", { name: "Tab 1" }).should(
"have.attr",
"aria-selected",
"true",
);
});
it("should support query builder shortcuts", () => {

View File

@@ -146,10 +146,8 @@ describe("scenarios > alert", () => {
const allowedDomain = "metabase.test";
const deniedDomain = "metabase.example";
const deniedEmail = `mailer@${deniedDomain}`;
const subscriptionError =
"You're only allowed to email subscriptions to allowed domains";
const alertError =
"You're only allowed to email alerts to allowed domains";
// We're not exposing allowed domains to normal users.
const normalError = `The following email addresses are not allowed: ${deniedEmail}`;
const adminSubscriptionError = `You're only allowed to email subscriptions to addresses ending in ${allowedDomain}`;
const adminAlertError = `You're only allowed to email alerts to addresses ending in ${allowedDomain}`;
@@ -213,8 +211,11 @@ describe("scenarios > alert", () => {
addEmailRecipient(deniedEmail);
});
cy.findByText(alertError);
cy.button("Done").should("be.disabled");
cy.button("Done").click();
});
cy.findByTestId("toast-undo").within(() => {
cy.root().should("have.attr", "color", "error");
cy.findByText(normalError);
});
H.visitDashboard(ORDERS_DASHBOARD_ID);
@@ -223,9 +224,11 @@ describe("scenarios > alert", () => {
H.sidebar().within(() => {
addEmailRecipient(deniedEmail);
cy.button("Send email now").should("be.disabled");
cy.button("Done").should("be.disabled");
cy.findByText(subscriptionError);
cy.button("Done").click();
});
cy.findByTestId("toast-undo").within(() => {
cy.root().should("have.attr", "color", "error");
cy.findByText(normalError);
});
});
},

View File

@@ -107,6 +107,22 @@ describe("scenarios > dashboard > subscriptions", () => {
cy.findByText("Emailed hourly");
});
it("should not add a recipient when Escape is pressed (metabase#24629)", () => {
openDashboardSubscriptions(ORDERS_DASHBOARD_ID);
H.sidebar().findByText("Email it").click();
const input = cy.findByPlaceholderText(
"Enter user names or email addresses",
);
input.click().type(`${admin.first_name}`);
input.type("{esc}");
input.should("have.value", `${admin.first_name}`);
cy.findByTestId("token-field-popover").should("not.exist");
});
it("should not render people dropdown outside of the borders of the screen (metabase#17186)", () => {
openDashboardSubscriptions();
@@ -739,10 +755,15 @@ function assignRecipient({
} = {}) {
openDashboardSubscriptions(dashboard_id);
cy.findByText("Email it").click();
cy.findByPlaceholderText("Enter user names or email addresses")
.click()
.type(`${user.first_name} ${user.last_name}{enter}`)
.blur(); // blur is needed to close the popover
const input = cy.findByPlaceholderText("Enter user names or email addresses");
input.click().type(`${user.first_name} ${user.last_name}`);
cy.findByTestId("token-field-popover").within(() => {
cy.findByText(`${user.first_name} ${user.last_name}`).click();
});
input.blur(); // blur is needed to close the popover
}
function assignRecipients({

View File

@@ -3,7 +3,7 @@
(:require
[clojure.spec.alpha :as s]
[metabase-enterprise.advanced-config.file.interface :as advanced-config.file.i]
[metabase.models.api-key :as api-key]
[metabase.api-keys.core :as api-key]
[metabase.models.user :as user]
[metabase.permissions.core :as perms]
[metabase.util :as u]

View File

@@ -0,0 +1,3 @@
(ns metabase-enterprise.advanced-config.init
(:require
[metabase-enterprise.advanced-config.models.notification]))

View File

@@ -0,0 +1,49 @@
(ns metabase-enterprise.advanced-config.models.notification
(:require
[clojure.string :as str]
[metabase.premium-features.core :refer [defenterprise]]
[metabase.settings.core :as setting :refer [defsetting]]
[metabase.util :as u]
[metabase.util.i18n :refer [deferred-tru tru]]))
(defsetting subscription-allowed-domains
(deferred-tru
(str "Allowed email address domain(s) for new Dashboard Subscriptions and Alerts."
"To specify multiple domains, separate each domain with a comma, with no space in between."
"To allow all domains, leave the field empty. This setting doesn’t affect existing subscriptions."))
:encryption :no
:visibility :settings-manager
:export? true
:feature :email-allow-list
;; this is a comma-separated string but we're not using `:csv` because it gets serialized to an array which makes it
;; inconvenient to use on the frontend.
:type :string
:setter (fn [new-value]
(when (not-empty new-value)
(when-let [domains (str/split new-value #",")]
(assert (every? u/domain? domains) (format "Each domain must be a valid email domain. %s" domains))))
(setting/set-value-of-type! :string :subscription-allowed-domains new-value))
:audit :getter)
(defn- allowed-domains-set
"Parse [[subscription-allowed-domains]] into a set. `nil` if the Setting is not set or empty."
[]
(some-> (subscription-allowed-domains)
(str/split #",")
set
not-empty))
(defenterprise validate-email-domains!
"Check that whether `email-addresses` are allowed based on the value of the [[subscription-allowed-domains]] Setting, if set.
This function no-ops if `subscription-allowed-domains` is unset or if we do not have a premium token with the `:email-allow-list` feature."
:feature :email-allow-list
[email-addresses]
(when-let [allowed-domains (allowed-domains-set)]
(let [disallowed-emails (->> email-addresses
(remove #(->> % u/email->domain allowed-domains))
seq)]
(when (seq disallowed-emails)
(throw (ex-info
(tru "The following email addresses are not allowed: {0}"
(str/join ", " disallowed-emails))
{:status-code 403}))))))

View File

@@ -1,48 +0,0 @@
(ns metabase-enterprise.advanced-config.models.pulse-channel
(:require
[clojure.string :as str]
[metabase.premium-features.core :as premium-features]
[metabase.settings.core :refer [defsetting]]
[metabase.util :as u]
[metabase.util.i18n :refer [deferred-tru tru]]))
(defsetting subscription-allowed-domains
(deferred-tru "Allowed email address domain(s) for new Dashboard Subscriptions and Alerts. To specify multiple domains, separate each domain with a comma, with no space in between. To allow all domains, leave the field empty. This setting doesn’t affect existing subscriptions.")
:encryption :no
:visibility :public
:export? true
:feature :email-allow-list
;; this is a comma-separated string but we're not using `:csv` because it gets serialized to an array which makes it
;; inconvenient to use on the frontend.
:type :string
:audit :getter)
(defn- allowed-domains-set
"Parse [[subscription-allowed-domains]] into a set. `nil` if the Setting is not set or empty."
[]
(some-> (subscription-allowed-domains)
(str/split #",")
set
not-empty))
(defn validate-email-domains
"Check that `email-addresses` associated with a [[metabase.pulse.models.pulse-channel]] are allowed based on the value
of the [[subscription-allowed-domains]] Setting, if set. This function no-ops if `subscription-allowed-domains` is
unset or if we do not have a premium token with the `:email-allow-list` feature.
This function is called by [[metabase.pulse.models.pulse-channel/validate-email-domains]] when Pulses are created
and updated."
[email-addresses]
(when (premium-features/enable-email-allow-list?)
(when-let [allowed-domains (allowed-domains-set)]
(doseq [email email-addresses
:let [domain (u/email->domain email)]]
(assert (u/email? email)
(tru "Invalid email address: {0}" (pr-str email)))
(when-not (contains? allowed-domains domain)
(throw (ex-info (tru "You cannot create new subscriptions for the domain {0}. Allowed domains are: {1}"
(pr-str domain)
(str/join ", " allowed-domains))
{:email email
:allowed-domains allowed-domains
:status-code 403})))))))

View File

@@ -6,5 +6,6 @@
See https://metaboat.slack.com/archives/CKZEMT1MJ/p1736556522733279 for rationale behind this pattern."
(:require
[metabase-enterprise.advanced-config.init]
[metabase-enterprise.cache.init]
[metabase-enterprise.enhancements.init]))

View File

@@ -277,7 +277,7 @@
cannot-check-message
{:gdrive/conn-id conn-id
:hm/exception e})))))
[hm-status {hm-status-code :status hm-body :body}] hm-response]
[hm-status {hm-status-code :status hm-body :body hm-err-body :ex-data}] hm-response]
(if (= :ok hm-status)
(let [{:keys [status status-reason error last-sync-at last-sync-started-at]
:as _} (normalize-gdrive-conn hm-body)]
@@ -309,7 +309,7 @@
(empty? saved-setting)
{:status "not-connected"}
(= 403 hm-status-code)
(= 403 (or hm-status-code (:status hm-err-body)))
(let [[list-status list-response] (hm-get-all-connections)]
(if (and (= :ok list-status) (not (some (partial = conn-id) (set (map :id (:body list-response))))))
(u/prog1 {:status "not-connected"}

View File

@@ -18,8 +18,7 @@
:dashcard/action
:dashcard/linkcard-info]
:tabs
:param_fields
:param_values)
:param_fields)
param-id->param (zipmap
(map :id parameters)
(map (fn [param]

View File

@@ -5,6 +5,7 @@
[metabase.api.common :as api]
[metabase.api.macros :as api.macros]
[metabase.api.open-api :as open-api]
[metabase.driver.util :as driver.u]
[metabase.premium-features.core :as premium-features]
[metabase.util :as u]
[metabase.util.i18n :refer [tru]]
@@ -75,6 +76,15 @@
{:keys [table_id card_id]} :- [:map
[:table_id ms/PositiveInt]
[:card_id {:optional true} [:maybe ms/PositiveInt]]]]
(when card_id
(let [db (t2/select-one :model/Database
:id {:select [:t.db_id]
:from [[(t2/table-name :model/Table) :t]]
:where [:= :t.id table_id]})]
(when (not (driver.u/supports? (:engine db) :saved-question-sandboxing db))
(throw (ex-info (tru "Sandboxing with a saved question is not enabled for this database.")
{:status-code 400
:message (tru "Sandboxing with a saved question is not enabled for this database.")})))))
(gtap/check-columns-match-table {:table_id table_id
:card_id card_id}))

View File

@@ -2,9 +2,9 @@
"/api/ee/scim/ endpoints"
(:require
[metabase-enterprise.serialization.v2.backfill-ids :as serdes.backfill]
[metabase.api-keys.core :as api-key]
[metabase.api.common :as api]
[metabase.api.macros :as api.macros]
[metabase.models.api-key :as api-key]
[metabase.settings.core :refer [defsetting]]
[metabase.settings.deprecated-grab-bag :as public-settings]
[metabase.util.i18n :refer [deferred-tru]]

View File

@@ -13,8 +13,14 @@
(set! *warn-on-reflection* true)
(def ^:private ignored-entity-id-table-names
"Legacy (V1) Metrics are no longer supported, and all their code has been removed. However the Tables are still in the
app DB (for now)... ignore them."
#{"metric" "METRIC" "metric_important_field" "METRIC_IMPORTANT_FIELD"})
(defn- entity-id-table-names
"Return a set of lower-cased names of all application database tables that have an `entity_id` column, excluding views."
"Return a set of lower-cased names of all application database tables that have an `entity_id` column, excluding
views."
[]
(with-open [conn (.getConnection (mdb/app-db))]
(let [dbmeta (.getMetaData conn)]
@@ -24,7 +30,8 @@
:h2 "ENTITY_ID"
(:mysql :postgres) "entity_id"))]
(let [entity-id-tables (into #{} (map (comp u/lower-case-en :table_name)) (resultset-seq rset))]
(set/intersection non-view-tables entity-id-tables))))))))
(-> (set/intersection non-view-tables entity-id-tables)
(set/difference ignored-entity-id-table-names)))))))))
(defn toucan-models
"Return a list of all toucan models."

View File

@@ -2,6 +2,7 @@
"Loading is the interesting part of deserialization: integrating the maps \"ingested\" from files into the appdb.
See the detailed breakdown of the (de)serialization processes in [[metabase.models.serialization]]."
(:require
[clojure.string :as str]
[medley.core :as m]
[metabase-enterprise.serialization.v2.backfill-ids :as serdes.backfill]
[metabase-enterprise.serialization.v2.ingest :as serdes.ingest]
@@ -22,25 +23,16 @@
This map works around this: given a model (e.g. `Card`) that triggered a dependency loop, it provides a set of paths to
keys to remove from the model so that we'll be able to successfully load it. You can remove keys in vectors using :* to
indicate that all items in that vector should have a key removed."
{"Dashboard" #{[:dashcards :* :visualization_settings]}
"Card" #{[:dashboard_id]}})
{"Dashboard" #{:dashcards}
"Card" #{:dashboard_id}})
(defn- remove-by-path
[entity [first-key & rest]]
(cond
(empty? rest) (dissoc entity first-key)
(= :* first-key) (mapv #(remove-by-path % rest) entity)
:else (update entity first-key remove-by-path rest)))
(defn- without-references
"Remove references to other entities from a given one. Used to break circular dependencies when loading."
[model entity]
(let [keys-to-remove (or (model->circular-dependency-keys model)
(throw (ex-info "Don't know which keys to remove to break circular dependency!"
{:entity entity
:model model
:error ::no-known-references})))]
(reduce remove-by-path entity keys-to-remove)))
(defn- keys-to-strip [ingested]
(let [model (-> ingested :serdes/meta last :model)]
(or (model->circular-dependency-keys model)
(throw (ex-info "Don't know which keys to remove to break circular dependency!"
{:entity ingested
:model model
:error ::no-known-references})))))
(defn- load-deps!
"Given a list of `deps` (hierarchies), [[load-one]] them all.
@@ -59,13 +51,6 @@
(serdes/load-find-local dep))
ctx
;; It's a circular dep, strip off probable cause and retry. This will store an incomplete version
;; of an entity, but this is not a problem - a full version is waiting to be stored up the stack.
(= (:error (ex-data e)) ::circular)
(let [model (:model (ex-data e))]
(log/debug "Detected circular dependency" (serdes/log-path-str dep))
(load-one! (update ctx :expanding disj dep) dep (partial without-references model)))
:else
(throw e)))))]
(reduce loader ctx deps))))
@@ -73,7 +58,7 @@
(defn- path-error-data [error-type expanding path]
(let [last-model (:model (last path))]
{:path (mapv (partial into {}) path)
:deps-chain (set (map #(mapv (partial into {}) %) expanding))
:deps-chain expanding
:model last-model
:table (some->> last-model (keyword "model") t2/table-name)
:error error-type}))
@@ -102,55 +87,73 @@
This is mostly bookkeeping for the overall deserialization process - the actual load of any given entity is done by
[[metabase.models.serialization/load-one!]] and its various overridable parts, which see.
Error is thrown on a circular dependency, should be handled and retried at the caller. `modfn` is an optional
parameter to modify entity data after reading and before other processing (before loading dependencies, finding
local version, and storing in the db)."
[{:keys [expanding ingestion seen] :as ctx} path & [modfn]]
(log/infof "Loading %s" (serdes/log-path-str path))
The only tangling stuff is handling circular dependencies: parts of this is handled by the `serdes/load-one!`
function, just outright skipping processing for parts of ingested data."
[{:keys [expanding seen circular ingestion] :as ctx} path]
(log/info "Loading" (cond-> {:path (serdes/log-path-str path)}
(circular path) (assoc :stripped true)))
(cond
(expanding path) (throw (ex-info (format "Circular dependency on %s" (serdes/log-path-str path))
(path-error-data ::circular expanding path)))
(seen path) ctx ; Already been done, can skip it.
:else (let [ingested (try
(serdes.ingest/ingest-one ingestion path)
(catch Exception e
(throw (ex-info (format "Failed to read file for %s" (serdes/log-path-str path))
(path-error-data ::not-found expanding path)
e))))
;; Use the abstract path as attached by the ingestion process, not the original one we were passed.
rebuilt-path (serdes/path ingested)
;; If nil or absent :entity_id is taken as a signal to create a new entity
;; To get a nil entity_id, a user has to manually set the entity_id to null or remove it
;; in the yaml file.
;; In all other cases we should expect an :entity_id:
;; - exported entities have a :entity_id for every model that can have one
;; - backfill (pre import) guarantees all entities have ids in the appdb
expect-entity-id (some-> rebuilt-path peek :model exported-with-entity-id?)
require-new-entity (and expect-entity-id (nil? (:entity_id ingested)))
ingested (cond-> ingested
require-new-entity (assoc :entity_id (u/generate-nano-id))
modfn modfn)
deps (serdes/dependencies ingested)
_ (log/debug "Loading dependencies" deps)
ctx (-> ctx
(update :expanding conj path)
(load-deps! deps)
(update :seen conj path)
(update :expanding disj path))
local-or-nil (when-not require-new-entity (serdes/load-find-local rebuilt-path))]
(try
(serdes/load-one! ingested local-or-nil)
ctx
(catch Exception e
;; ugly mapv here to convered #ordered/map into normal map so it's readable in the logs
(throw (ex-info (format "Failed to load into database for %s" (serdes/log-path-str path))
(path-error-data ::load-failure expanding path)
e)))))))
(and (expanding path)
(circular path)) (throw (ex-info (format "Circular dependency on %s" (serdes/log-path-str path))
(path-error-data ::circular expanding path)))
(expanding path) (do
(log/debug "Detected circular dependency" (serdes/log-path-str path))
(load-one! (-> ctx
(update :expanding disj path)
(update :circular conj path))
path))
(seen path) ctx ; Already been done, can skip it.
:else
(let [ingested (try
(serdes.ingest/ingest-one ingestion path)
(catch Exception e
(throw (ex-info (format "Failed to read file for %s" (serdes/log-path-str path))
(path-error-data ::not-found expanding path)
e))))
;; Use the abstract path as attached by the ingestion process, not the original one we were passed.
rebuilt-path (serdes/path ingested)
;; If nil or absent :entity_id is taken as a signal to create a new entity
;; To get a nil entity_id, a user has to manually set the entity_id to null or remove it
;; in the yaml file.
;; In all other cases we should expect an :entity_id:
;; - exported entities have a :entity_id for every model that can have one
;; - backfill (pre import) guarantees all entities have ids in the appdb
expect-entity-id (some-> rebuilt-path peek :model exported-with-entity-id?)
require-new-entity (and expect-entity-id (nil? (:entity_id ingested)))
ingested (cond-> ingested
require-new-entity (assoc :entity_id (u/generate-nano-id))
;; `::strip` is handled in `metabase.serialization`
(circular path) (assoc ::serdes/strip (keys-to-strip ingested)))
;; we need less deps when trying to load "stripped" data
deps (->> (serdes/dependencies (apply dissoc ingested (::serdes/strip ingested)))
(remove seen))
_ (when (seq deps)
(log/debug "Loading dependencies"
{:entity_id (:entity_id ingested)
:level (count expanding)
:deps (str "[" (str/join ", " (map serdes/log-path-str deps)) "]")}))
ctx (-> ctx
(update :expanding conj path)
(load-deps! deps)
(update :seen conj path)
(update :expanding disj path))
_ (when (seq deps)
(log/debug "Ended loading dependencies" {:entity_id (:entity_id ingested)
:level (count expanding)}))
local-or-nil (when-not require-new-entity (serdes/load-find-local rebuilt-path))]
(try
(serdes/load-one! ingested local-or-nil)
ctx
(catch Exception e
;; ugly mapv here to convered #ordered/map into normal map so it's readable in the logs
(throw (ex-info (format "Failed to load into database for %s" (serdes/log-path-str path))
(path-error-data ::load-failure expanding path)
e)))))))
(defn load-metabase!
"Loads in a database export from an ingestion source, which is any Ingestable instance."
[ingestion & {:keys [backfill? continue-on-error]
:or {backfill? true
:or {backfill? true
continue-on-error false}}]
(t2/with-transaction [_tx]
;; We proceed in the arbitrary order of ingest-list, deserializing all the files. Their declared dependencies
@@ -160,6 +163,7 @@
(let [contents (serdes.ingest/ingest-list ingestion)
ctx {:expanding #{}
:seen #{}
:circular #{}
:ingestion ingestion
:from-ids (m/index-by :id contents)
:errors []}]

View File

@@ -56,8 +56,6 @@
"GroupTableAccessPolicy"
"HTTPAction"
"ImplicitAction"
"LegacyMetric"
"LegacyMetricImportantField"
"LoginHistory"
"ModelIndex"
"ModelIndexValue"

View File

@@ -26,7 +26,7 @@
(apply io/file (:root-dir ctx) (map escape-segment (concat dirnames [basename])))))
(defn- store-entity! [opts entity]
(log/infof "Storing %s" (serdes/log-path-str (:serdes/meta entity)))
(log/info "Storing" {:path (serdes/log-path-str (:serdes/meta entity))})
(spit-yaml! (file opts entity) entity)
(:serdes/meta entity))

View File

@@ -2,7 +2,7 @@
"EE implementation of NativeQuerySnippet permissions."
(:require
[metabase.models.interface :as mi]
[metabase.models.native-query-snippet.permissions :as snippet.perms]
[metabase.native-query-snippets.core :as snippets]
[metabase.permissions.models.permissions :as perms]
[metabase.permissions.util :as perms-util]
[metabase.premium-features.core :refer [defenterprise]]
@@ -21,7 +21,7 @@
([snippet]
(and
(not (perms-util/sandboxed-user?))
(snippet.perms/has-any-native-permissions?)
(snippets/has-any-native-permissions?)
(has-parent-collection-perms? snippet :read)))
([model id]
(can-read? (t2/select-one [model :collection_id] :id id))))
@@ -32,7 +32,7 @@
([snippet]
(and
(not (perms-util/sandboxed-user?))
(snippet.perms/has-any-native-permissions?)
(snippets/has-any-native-permissions?)
(has-parent-collection-perms? snippet :write)))
([model id]
(can-write? (t2/select-one [model :collection_id] :id id))))
@@ -43,7 +43,7 @@
[_model m]
(and
(not (perms-util/sandboxed-user?))
(snippet.perms/has-any-native-permissions?)
(snippets/has-any-native-permissions?)
(has-parent-collection-perms? m :write)))
(defenterprise can-update?
@@ -52,7 +52,7 @@
[snippet changes]
(and
(not (perms-util/sandboxed-user?))
(snippet.perms/has-any-native-permissions?)
(snippets/has-any-native-permissions?)
(has-parent-collection-perms? snippet :write)
(or (not (contains? changes :collection_id))
(has-parent-collection-perms? changes :write))))

View File

@@ -7,7 +7,7 @@
(comment metabase-enterprise.sso.api.saml/keep-me
metabase-enterprise.sso.api.sso/keep-me)
;; This needs to be injected into [[metabase.server.routes/routes]] -- not [[metabase.api.routes/routes]] !!!
;; This needs to be injected into [[metabase.server.routes/routes]] -- not [[metabase.api-routes.core/routes]] !!!
;;
;; TODO -- should we make a `metabase-enterprise.routes` namespace where this can live instead of injecting it
;; directly?

View File

@@ -5,7 +5,7 @@
[metabase.api.common :as api]
[metabase.channel.email.messages :as messages]
[metabase.events :as events]
[metabase.events.notification :as events.notification]
[metabase.notification.events.notification :as events.notification]
[metabase.settings.deprecated-grab-bag :as public-settings]
[metabase.sso.core :as sso]
[metabase.util :as u]

View File

@@ -1,6 +1,6 @@
(ns metabase-enterprise.stats
(:require
[metabase-enterprise.advanced-config.models.pulse-channel :as advanced-config.models.pulse-channel]
[metabase-enterprise.advanced-config.models.notification :as advanced-config.models.notification]
[metabase-enterprise.scim.api :as scim-api]
[metabase-enterprise.sso.integrations.sso-settings :as sso-settings]
[metabase.driver :as driver]
@@ -27,4 +27,4 @@
:enabled (t2/exists? :model/GroupTableAccessPolicy)}
{:name :email-allow-list
:available (premium-features/enable-email-allow-list?)
:enabled (boolean (some? (advanced-config.models.pulse-channel/subscription-allowed-domains)))}])
:enabled (boolean (some? (advanced-config.models.notification/subscription-allowed-domains)))}])

View File

@@ -5,7 +5,7 @@
[metabase.api.routes.common :refer [+auth]]
[metabase.models.interface :as mi]
[metabase.premium-features.core :as premium-features]
[metabase.upload :as upload]
[metabase.upload.core :as upload]
[metabase.util.i18n :refer [tru]]
[metabase.util.malli.schema :as ms]
[toucan2.core :as t2]))

View File

@@ -1,64 +0,0 @@
(ns metabase-enterprise.advanced-config.api.pulse-test
(:require
[clojure.test :refer :all]
[metabase.test :as mt]
[metabase.test.fixtures :as fixtures]
[metabase.util :as u]))
(use-fixtures :once (fixtures/initialize :plugins))
(deftest test-pulse-endpoint-should-respect-email-domain-allow-list-test
(testing "POST /api/pulse/test"
(mt/with-temp [:model/Card card {:name "Test card"
:dataset_query (mt/mbql-query venues)}]
;; make sure we validate raw emails whether they're part of `:details` or part of `:recipients` -- we
;; technically allow either right now
(doseq [channel [{:details {:emails ["test@metabase.com"]}}
{:recipients [{:email "test@metabase.com"}]
:details {}}]]
(testing (format "\nChannel = %s\n" (u/pprint-to-str channel))
(letfn [(send! [expected-status-code]
(mt/with-fake-inbox
{:response (mt/user-http-request
:rasta :post expected-status-code "pulse/test"
{:name (mt/random-name)
:cards [{:id (u/the-id card)
:include_csv false
:include_xls false
:dashboard_card_id nil}]
:channels [(merge {:enabled true
:channel_type "email"
:schedule_type "daily"
:schedule_hour 12
:schedule_day nil}
channel)]
:alert_condition "rows"
:skip_if_empty false})
:recipients (set (keys (mt/regex-email-bodies (re-pattern "Test card"))))}))]
(testing "allowed email -- should pass"
(mt/with-premium-features #{:email-allow-list}
(mt/with-temporary-setting-values [subscription-allowed-domains "metabase.com"]
(let [{:keys [response recipients]} (send! 200)]
(is (= {:ok true}
response))
(is (contains? recipients "test@metabase.com"))))
(testing "No :email-allow-list token"
(mt/with-premium-features #{}
(let [{:keys [response recipients]} (send! 200)]
(is (= {:ok true}
response))
(is (contains? recipients "test@metabase.com")))))))
(testing "disallowed email"
(mt/with-premium-features #{:email-allow-list}
(mt/with-temporary-setting-values [subscription-allowed-domains "example.com"]
(testing "should fail when :email-allow-list is enabled"
(let [{:keys [response recipients]} (send! 403)]
(is (= "You cannot create new subscriptions for the domain \"metabase.com\". Allowed domains are: example.com"
(:message response)))
(is (not (contains? recipients "test@metabase.com")))))
(testing "No :email-allow-list token -- should still pass"
(mt/with-premium-features #{}
(let [{:keys [response recipients]} (send! 200)]
(is (= {:ok true}
response))
(is (contains? recipients "test@metabase.com"))))))))))))))

View File

@@ -1,8 +1,8 @@
(ns metabase-enterprise.advanced-config.models.pulse-channel-test
(ns metabase-enterprise.advanced-config.models.notification-test
(:require
[clojure.string :as str]
[clojure.test :refer :all]
[metabase-enterprise.advanced-config.models.pulse-channel :as advanced-config.models.pulse-channel]
[metabase-enterprise.advanced-config.models.notification :as advanced-config.models.notification]
[metabase.test :as mt]
[metabase.util :as u]
[toucan2.core :as t2]))
@@ -40,7 +40,7 @@
(testing "should fail"
(is (thrown-with-msg?
clojure.lang.ExceptionInfo
#"You cannot create new subscriptions for the domain \"[\w@\.-]+\". Allowed domains are: .+"
#"The following email addresses are not allowed: .*"
(thunk))))
(testing "should succeed"
(is (thunk)))))))))))
@@ -49,10 +49,10 @@
(testing "Should be able to set the subscription-allowed-domains setting with the email-allow-list feature"
(mt/with-premium-features #{:email-allow-list}
(is (= "metabase.com"
(advanced-config.models.pulse-channel/subscription-allowed-domains! "metabase.com")))))
(advanced-config.models.notification/subscription-allowed-domains! "metabase.com")))))
(testing "Should be unable to set the subscription-allowed-domains setting without the email-allow-list feature"
(mt/with-premium-features #{}
(is (thrown-with-msg?
clojure.lang.ExceptionInfo
#"Setting subscription-allowed-domains is not enabled because feature :email-allow-list is not available"
(advanced-config.models.pulse-channel/subscription-allowed-domains! "metabase.com"))))))
(advanced-config.models.notification/subscription-allowed-domains! "metabase.com"))))))

View File

@@ -4,7 +4,7 @@
[clojure.test :refer :all]
[metabase.api.geojson-test :as geojson-test]
[metabase.channel.email :as email]
[metabase.integrations.slack :as slack]
[metabase.channel.slack :as slack]
[metabase.permissions.models.permissions :as perms]
[metabase.test :as mt]
[metabase.test.fixtures :as fixtures]))

View File

@@ -15,7 +15,7 @@
[metabase.test :as mt]
[metabase.test.data.sql :as sql.tx]
[metabase.test.fixtures :as fixtures]
[metabase.upload-test :as upload-test]
[metabase.upload.impl-test :as upload-test]
[metabase.util :as u]
[metabase.util.quick-task :as quick-task]
[toucan2.core :as t2]))

View File

@@ -134,10 +134,9 @@
(defmacro with-sample-db-as-dwh [& body]
"We need an attached dwh for these tests, so let's have the sample db fill in for us:"
`(try
(t2/update! :model/Database :id 1 {:is_attached_dwh true})
~@body
(finally (t2/update! :model/Database :id 1 {:is_attached_dwh false}))))
(let [db-sym (gensym "db-")]
`(mt/with-temp [:model/Database ~db-sym {:is_attached_dwh true}]
~@body)))
(deftest post-folder-test
(with-sample-db-as-dwh

View File

@@ -48,8 +48,6 @@
:model/LoginHistory
:model/FieldUsage
:model/FieldValues
:model/LegacyMetric
:model/LegacyMetricImportantField
:model/ModelIndex
:model/ModelIndexValue
:model/ModerationReview

View File

@@ -13,53 +13,6 @@
[metabase.util :as u]
[toucan2.core :as t2]))
(deftest params-values-test
(testing "Return sandboxed `param_values` for Fields to which the current User only has sandboxed access."
(met/with-gtaps! {:gtaps {:venues
{:remappings {:cat [:variable [:field-id (mt/id :venues :category_id)]]}
:query (mt.tu/restricted-column-query (mt/id))}}
:attributes {:cat 50}}
(mt/with-no-data-perms-for-all-users!
(data-perms/set-database-permission! &group (mt/id) :perms/view-data :unrestricted)
(data-perms/set-table-permission! &group (mt/id :categories) :perms/create-queries :query-builder)
(mt/with-temp [:model/Dashboard {dashboard-id :id} {:name "Test Dashboard"}
:model/Card {card-id :id} {:name "Dashboard Test Card"}
:model/DashboardCard {_ :id} {:dashboard_id dashboard-id
:card_id card-id
:parameter_mappings [{:card_id card-id
:parameter_id "foo"
:target [:dimension
[:field (mt/id :venues :name) nil]]}
;; should be returned normally since user has non-sandbox perms
{:card_id card-id
:parameter_id "bar"
:target [:dimension
[:field (mt/id :categories :name) nil]]}
;; shouldn't be returned since user has no perms
{:card_id card-id
:parameter_id "bax"
:target [:dimension
[:field (mt/id :users :name) nil]]}]}]
;; Manually activate Field values since they are not created during sync (#53387)
(field-values/get-or-create-full-field-values! (t2/select-one :model/Field :id (mt/id :venues :name)))
(field-values/get-or-create-full-field-values! (t2/select-one :model/Field :id (mt/id :categories :name)))
(field-values/get-or-create-full-field-values! (t2/select-one :model/Field :id (mt/id :users :name)))
(is (= {(mt/id :venues :name) {:values ["Garaje"
"Gordo Taqueria"
"La Tortilla"]
:human_readable_values []
:field_id (mt/id :venues :name)}
(mt/id :categories :name) {:values ["African"
"American"
"Artisan"]
:human_readable_values []
:field_id (mt/id :categories :name)}}
(let [response (:param_values (mt/user-http-request :rasta :get 200 (str "dashboard/" dashboard-id)))]
(into {} (for [[field-id m] response]
[field-id (update m :values (partial take 3))]))))))))))
(deftest chain-filter-sandboxed-field-values-test
(testing "When chain filter endpoints would normally return cached FieldValues (#13832), make sure sandboxing is respected"
(met/with-gtaps! {:gtaps {:categories {:query (mt/mbql-query categories {:filter [:< $id 3]})}}}

View File

@@ -1,6 +1,7 @@
(ns metabase-enterprise.sandbox.api.gtap-test
(:require
[clojure.test :refer :all]
[metabase.driver.util :as driver.util]
[metabase.http-client :as client]
[metabase.permissions.models.data-permissions.graph :as data-perms.graph]
[metabase.premium-features.core :as premium-features]
@@ -154,7 +155,17 @@
{:table_id table-id
:group_id group-id
:card_id card-id
:attribute_remappings {"foo" 1}})))))))))
:attribute_remappings {"foo" 1}}))))))
(testing "A database without the saved question sandboxing features returns a 400 error"
(with-redefs [driver.util/supports? (fn [_ feature _] (not= feature :saved-question-sandboxing))]
(mt/with-temp [:model/Card {card-id :id}]
(with-gtap-cleanup!
(is (=? {:message "Sandboxing with a saved question is not enabled for this database."}
(mt/user-http-request :crowberto :post 400 "mt/gtap/validate"
{:table_id table-id
:group_id group-id
:card_id card-id
:attribute_remappings {"foo" 1}}))))))))))
(deftest delete-gtap-test
(testing "DELETE /api/mt/gtap/:id"

View File

@@ -4,11 +4,8 @@
(:require
[clojure.test :refer :all]
[metabase-enterprise.test :as met]
[metabase.integrations.slack]
[metabase.test :as mt]))
(comment metabase.integrations.slack/keep-me) ; so the Setting exists
(deftest segmented-users-pulse-test
(testing "GET /api/pulse/form_input"
(testing (str "Non-segmented users are able to send pulses to any slack channel that the configured instance can "

View File

@@ -11,6 +11,8 @@
[metabase.driver.sql.query-processor :as sql.qp]
[metabase.driver.util :as driver.u]
[metabase.legacy-mbql.normalize :as mbql.normalize]
[metabase.lib.core :as lib]
[metabase.lib.metadata :as lib.metadata]
[metabase.lib.util.match :as lib.util.match]
[metabase.models.query.permissions :as query-perms]
[metabase.permissions.models.data-permissions :as data-perms]
@@ -1235,3 +1237,30 @@
(is (nil? (t2/select-one-fn :result_metadata :model/Card sandbox-card-id)))
(is (= 10 (count (mt/rows (streaming.test-util/process-query-basic-streaming :api (mt/mbql-query venues))))))
(is (not (nil? (t2/select-one-fn :result_metadata :model/Card sandbox-card-id))))))))
(deftest filter-by-column-sandboxing-test
(mt/test-drivers (mt/normal-drivers)
(testing "Sandboxing with filtering by a column works for all supported drivers"
(met/with-gtaps! {:gtaps {:venues {:remappings {:cat ["variable" [:field (mt/id :venues :category_id) nil]]}}
:checkins {:remappings {:user ["variable" [:field (mt/id :checkins :user_id) nil]]
:venue ["variable" [:field (mt/id :checkins :venue_id) nil]]}}},
:attributes {:cat 10
:user 1
:venue 47}}
(let [mp (mt/metadata-provider)
venues (lib.metadata/table mp (mt/id :venues))
venues-id (lib.metadata/field mp (mt/id :venues :id))
venues-query (-> (lib/query mp venues)
(lib/order-by venues-id :asc))
checkins (lib.metadata/table mp (mt/id :checkins))
checkins-query (-> (lib/query mp checkins)
(lib/aggregate (lib/count)))]
(is (= [[34 "Beachwood BBQ & Brewing" 10 33.7701 -118.191 2]
[99 "Golden Road Brewing" 10 34.1505 -118.274 2]]
(->> venues-query
qp/process-query
(mt/formatted-rows [int str int 4.0 3.0 int]))))
(is (= [[2]]
(->> checkins-query
qp/process-query
(mt/formatted-rows [int])))))))))

View File

@@ -5,7 +5,7 @@
[clojure.test :refer :all]
[clojure.walk :as walk]
[metabase-enterprise.serialization.api :as api.serialization]
[metabase-enterprise.serialization.v2.load :as v2.load]
[metabase-enterprise.serialization.v2.ingest :as v2.ingest]
[metabase.analytics.snowplow-test :as snowplow-test]
[metabase.models.serialization :as serdes]
[metabase.test :as mt]
@@ -48,7 +48,7 @@
"Find out entity type by log message"
[lines]
(->> lines
(keep #(second (re-find #"(?:Extracting|Loading|Storing) (\w+)" %)))
(keep #(second (re-find #"(?:Extracting|Loading|Storing) \{:path (\w+)" %)))
set))
(defn- tar-file-types [f & [raw?]]
@@ -230,14 +230,11 @@
"error_message" nil}
(-> (snowplow-test/pop-event-data-and-user-id!) last :data))))))
(mt/with-dynamic-fn-redefs [v2.load/load-one! (let [load-one! (mt/dynamic-value #'v2.load/load-one!)]
(fn [ctx path & [modfn]]
(load-one! ctx path
(or modfn
(fn [ingested]
(cond-> ingested
(= (:entity_id ingested) (:entity_id card))
(assoc :collection_id "DoesNotExist")))))))]
(mt/with-dynamic-fn-redefs [v2.ingest/ingest-file (let [ingest-file (mt/dynamic-value #'v2.ingest/ingest-file)]
(fn [^File file]
(cond-> (ingest-file file)
(str/includes? (.getName file) (:entity_id card))
(assoc :collection_id "DoesNotExist"))))]
(testing "ERROR /api/ee/serialization/import"
(let [res (binding [api.serialization/*additive-logging* false]
(mt/user-http-request :crowberto :post 500 "ee/serialization/import"

View File

@@ -1,11 +1,12 @@
(ns metabase-enterprise.serialization.cmd-test
(:require
[clojure.java.io :as io]
[clojure.string :as str]
[clojure.test :refer :all]
[metabase-enterprise.serialization.load :as load]
[metabase-enterprise.serialization.test-util :as ts]
[metabase-enterprise.serialization.v2.extract :as v2.extract]
[metabase-enterprise.serialization.v2.load :as v2.load]
[metabase-enterprise.serialization.v2.ingest :as v2.ingest]
[metabase-enterprise.serialization.v2.storage :as v2.storage]
[metabase.analytics.snowplow-test :as snowplow-test]
[metabase.cmd :as cmd]
@@ -198,8 +199,8 @@
(filter #(= "serialization" (get % "event")))
first))))
(cmd/import dump-dir)
(testing "Snowplow import event was sent"
(cmd/import dump-dir)
(is (=? {"event" "serialization"
"direction" "import"
"duration_ms" pos?
@@ -232,14 +233,13 @@
(filter #(= "serialization" (get % "event")))
first)))))
(let [load-one! @#'v2.load/load-one!]
(with-redefs [v2.load/load-one! (fn [ctx path & [modfn]]
(load-one! ctx path
(or modfn
(fn [ingested]
(cond-> ingested
(= (:entity_id ingested) (:entity_id card))
(assoc :collection_id "DoesNotExist"))))))]
(let [ingest-file @#'v2.ingest/ingest-file]
;; overriding ingest-file is weird, but ingest-one is a protocol function and with-redefs won't
;; override that reliably
(with-redefs [v2.ingest/ingest-file (fn [file]
(cond-> (ingest-file file)
(str/includes? (.getName file) (:entity_id card))
(assoc :collection_id "DoesNotExist")))]
(is (thrown? Exception
(cmd/import dump-dir)))
(testing "Snowplow import event about error was sent"

View File

@@ -1173,30 +1173,33 @@
(t2/select-one-fn :name :model/Collection :id (:id coll))))))))))
(deftest circular-links-test
(mt/with-empty-h2-app-db
(let [coll (ts/create! :model/Collection :name "coll")
card (ts/create! :model/Card :name "card" :collection_id (:id coll))
dash1 (ts/create! :model/Dashboard :name "dash1" :collection_id (:id coll))
dash2 (ts/create! :model/Dashboard :name "dash2" :collection_id (:id coll))
dash3 (ts/create! :model/Dashboard :name "dash3" :collection_id (:id coll))
dc1 (ts/create! :model/DashboardCard :dashboard_id (:id dash1) :card_id (:id card)
:visualization_settings {:click_behavior {:type "link"
:linkType "dashboard"
:targetId (:id dash2)}})
dc2 (ts/create! :model/DashboardCard :dashboard_id (:id dash2) :card_id (:id card)
:visualization_settings {:click_behavior {:type "link"
:linkType "dashboard"
:targetId (:id dash3)}})
dc3 (ts/create! :model/DashboardCard :dashboard_id (:id dash2) :card_id (:id card)
:visualization_settings {:click_behavior {:type "link"
:linkType "dashboard"
:targetId (:id dash1)}})
card-2 (ts/create! :model/Card :name "card-2" :dashboard_id (:id dash1))
ser (into [] (serdes.extract/extract {:no-settings true
:no-data-model true}))]
(testing "Circular dependencies are loaded correctly"
(is (serdes.load/load-metabase! (ingestion-in-memory ser)))
(let [select-target #(-> % :visualization_settings :click_behavior :targetId)]
(ts/with-dbs [source-db dest-db]
(ts/with-db source-db
(let [coll (ts/create! :model/Collection :name "coll")
card (ts/create! :model/Card :name "card" :collection_id (:id coll))
dash1 (ts/create! :model/Dashboard :name "dash1" :collection_id (:id coll))
dash2 (ts/create! :model/Dashboard :name "dash2" :collection_id (:id coll))
dash3 (ts/create! :model/Dashboard :name "dash3" :collection_id (:id coll))
dc1 (ts/create! :model/DashboardCard :dashboard_id (:id dash1) :card_id (:id card)
:visualization_settings {:click_behavior {:type "link"
:linkType "dashboard"
:targetId (:id dash2)}})
dc2 (ts/create! :model/DashboardCard :dashboard_id (:id dash2) :card_id (:id card)
:visualization_settings {:click_behavior {:type "link"
:linkType "dashboard"
:targetId (:id dash3)}})
dc3 (ts/create! :model/DashboardCard :dashboard_id (:id dash2) :card_id (:id card)
:visualization_settings {:click_behavior {:type "link"
:linkType "dashboard"
:targetId (:id dash1)}})
card-2 (ts/create! :model/Card :name "card-2" :dashboard_id (:id dash1))
_dc2-1 (ts/create! :model/DashboardCard :dashboard_id (:id dash1) :card_id (:id card-2))
ser (into [] (serdes.extract/extract {:no-settings true
:no-data-model false}))
select-target #(-> % :visualization_settings :click_behavior :targetId)]
;; loaded to the same db so we know ids won't be corrupted
(testing "Dashcard ids won't be changed by loading on top of them"
(is (serdes.load/load-metabase! (ingestion-in-memory ser)))
(is (= (:id dc1)
(t2/select-one-fn :id :model/DashboardCard :entity_id (:entity_id dc1))))
(is (= (:id dc2)
@@ -1209,17 +1212,21 @@
(t2/select-one-fn select-target :model/DashboardCard :entity_id (:entity_id dc2))))
(is (= (:id dash1)
(t2/select-one-fn select-target :model/DashboardCard :entity_id (:entity_id dc3)))))
(testing "Circular dependencies work for Dashboard Questions as well"
(is (= (:id dash1)
(t2/select-one-fn :dashboard_id :model/Card :entity_id (:entity_id card-2)))))))))
;; loaded to a different db so we know it works for sure instead of relying on data being present in the
;; database
(ts/with-db dest-db
(is (serdes.load/load-metabase! (ingestion-in-memory ser)))
(testing "Circular dependencies work for Dashboard Questions as well"
(is (= (t2/select-one-fn :id :model/Dashboard :entity_id (:entity_id dash1))
(t2/select-one-fn :dashboard_id :model/Card :entity_id (:entity_id card-2))))))))))
(deftest continue-on-error-test
(let [change-ser (fn [ser changes] ;; kind of like left-join, but right side is indexed
(vec (for [entity ser]
(merge entity (get changes (:entity_id entity))))))
logs-extract (fn [re logs]
(keep #(rest (re-find re %))
(map :message logs)))]
(let [change-ser (fn [ser changes] ;; kind of like left-join, but right side is indexed
(vec (for [entity ser]
(merge entity (get changes (:entity_id entity))))))
logs-extract (fn [re logs]
(keep #(rest (re-find re %))
(map :message logs)))]
(mt/with-empty-h2-app-db
(mt/with-temp [:model/Collection coll {:name "coll"}
:model/Card c1 {:name "card1" :collection_id (:id coll)}

View File

@@ -4,7 +4,7 @@
[metabase-enterprise.test :as met]
[metabase.models.collection :as collection]
[metabase.models.interface :as mi]
[metabase.models.native-query-snippet.permissions :as snippet.perms]
[metabase.native-query-snippets.models.native-query-snippet.permissions :as snippet.perms]
[metabase.permissions.models.permissions :as perms]
[metabase.permissions.models.permissions-group :as perms-group]
[metabase.test :as mt]))

View File

@@ -4,8 +4,8 @@
[java-time.api :as t]
[metabase.api.table-test :as oss-test]
[metabase.test :as mt]
[metabase.upload :as upload]
[metabase.upload-test :as upload-test]))
[metabase.upload.core :as upload]
[metabase.upload.impl-test :as upload-test]))
(def list-url "ee/upload-management/tables")

View File

@@ -5,7 +5,7 @@
[metabase.driver :as driver]
[metabase.test :as mt]
[metabase.test.fixtures :as fixtures]
[metabase.upload-test :as upload-test]))
[metabase.upload.impl-test :as upload-test]))
(set! *warn-on-reflection* true)

View File

@@ -40,7 +40,7 @@ export const ChartTypeDropdown = ({
visualizationType: CardDisplayType,
): {
value: CardDisplayType;
label: Visualization["uiName"];
label: ReturnType<Visualization["getUiName"]>;
iconName: IconName;
} | null => {
const visualization = visualizations.get(visualizationType);
@@ -50,7 +50,7 @@ export const ChartTypeDropdown = ({
return {
value: visualizationType,
label: visualization.uiName,
label: visualization.getUiName(),
iconName: visualization.iconName,
};
};

View File

@@ -55,7 +55,7 @@ export const SdkSaveQuestionForm = ({
targetCollection={id}
>
<Stack p="md">
<Title>
<Title order={2}>
<SaveQuestionTitle />
</Title>
<SaveQuestionForm onCancel={() => onCancel?.()} />

View File

@@ -47,6 +47,7 @@ export class AuditTableVisualization extends Component {
static identifier = "audit-table";
static noHeader = true;
static hidden = true;
static getUiName = () => "Audit Table";
// copy Table's settings and columnSettings
static settings = Table.settings;

View File

@@ -1,25 +0,0 @@
import { updateIn } from "icepick";
import { t } from "ttag";
import type { ADMIN_SETTINGS_SECTIONS } from "metabase/admin/settings/selectors";
import type { SettingElement } from "metabase/admin/settings/types";
import { PLUGIN_ADMIN_SETTINGS_UPDATES } from "metabase/plugins";
import { hasPremiumFeature } from "metabase-enterprise/settings";
if (hasPremiumFeature("email_allow_list")) {
PLUGIN_ADMIN_SETTINGS_UPDATES.push(
(sections: typeof ADMIN_SETTINGS_SECTIONS) =>
updateIn(
sections,
["email", "settings"],
(settings: SettingElement[]) => [
...settings,
{
key: "subscription-allowed-domains",
display_name: t`Approved domains for notifications`,
type: "string",
},
],
),
);
}

View File

@@ -1,34 +0,0 @@
import { updateIn } from "icepick";
import { t } from "ttag";
import type { ADMIN_SETTINGS_SECTIONS } from "metabase/admin/settings/selectors";
import type { SettingElement } from "metabase/admin/settings/types";
import { PLUGIN_ADMIN_SETTINGS_UPDATES } from "metabase/plugins";
import { hasPremiumFeature } from "metabase-enterprise/settings";
if (hasPremiumFeature("email_restrict_recipients")) {
PLUGIN_ADMIN_SETTINGS_UPDATES.push(
(sections: typeof ADMIN_SETTINGS_SECTIONS) =>
updateIn(
sections,
["email", "settings"],
(settings: SettingElement[]) => [
...settings,
{
key: "user-visibility",
display_name: t`Suggest recipients on dashboard subscriptions and alerts`,
type: "select",
options: [
{ value: "all", name: t`Suggest all users` },
{
value: "group",
name: t`Only suggest users in the same groups`,
},
{ value: "none", name: t`Don't show suggestions` },
],
defaultValue: "all",
},
],
),
);
}

View File

@@ -21,8 +21,6 @@ import "./embedding-sdk";
import "./snippets";
import "./sharing";
import "./moderation";
import "./email_allow_list";
import "./email_restrict_recipients";
import "./advanced_permissions";
import "./audit_app";
import "./license";

View File

@@ -18,7 +18,7 @@ import { EntityName } from "metabase/entities/containers/EntityName";
import { useToggle } from "metabase/hooks/use-toggle";
import { GTAPApi } from "metabase/services";
import type { IconName } from "metabase/ui";
import { Button, Icon } from "metabase/ui";
import { Button, Center, Icon, Loader } from "metabase/ui";
import type {
GroupTableAccessPolicyDraft,
GroupTableAccessPolicyParams,
@@ -125,40 +125,65 @@ const EditSandboxingModal = ({
(!_.isEqual(originalPolicy, normalizedPolicy) ||
normalizedPolicy.id == null);
const { data: policyCard } = useGetCardQuery(
const { data: policyCard, isFetching: loadingCard } = useGetCardQuery(
policy.card_id != null ? { id: policy.card_id } : skipToken,
);
const { data: policyTable } = useGetTableQuery(
const { data: policyTable, isFetching: loadingTabe } = useGetTableQuery(
policy.table_id != null ? { id: policy.table_id } : skipToken,
);
const hasSavedQuestionSandboxingFeature = policyTable?.db?.features?.includes(
"saved-question-sandboxing",
);
if (loadingCard || loadingTabe) {
return (
<Center p="2rem">
<Loader data-testid="loading-indicator" />
</Center>
);
}
return (
<div>
<h2 className={CS.p3}>{t`Restrict access to this table`}</h2>
<div>
<div className={cx(CS.px3, CS.pb3)}>
<div className={CS.pb2}>
{t`When the following rules are applied, this group will see a customized version of the table.`}
</div>
<div className={CS.pb4}>
{t`These rules don’t apply to native queries.`}
</div>
<h4 className={CS.pb1}>{t`How do you want to filter this table?`}</h4>
<Radio
value={!shouldUseSavedQuestion}
options={[
{ name: t`Filter by a column in the table`, value: true },
{
name: t`Use a saved question to create a custom view for this table`,
value: false,
},
]}
onChange={(shouldUseSavedQuestion) =>
setShouldUseSavedQuestion(!shouldUseSavedQuestion)
}
vertical
/>
{hasSavedQuestionSandboxingFeature ? (
<div>
<div className={CS.pb2}>
{t`When the following rules are applied, this group will see a customized version of the table.`}
</div>
<div className={CS.pb4}>
{t`These rules don’t apply to native queries.`}
</div>
<h4
className={CS.pb1}
>{t`How do you want to filter this table?`}</h4>
<Radio
value={!shouldUseSavedQuestion}
options={[
{ name: t`Filter by a column in the table`, value: true },
{
name: t`Use a saved question to create a custom view for this table`,
value: false,
},
]}
onChange={(shouldUseSavedQuestion) =>
setShouldUseSavedQuestion(!shouldUseSavedQuestion)
}
vertical
/>
</div>
) : (
<div>
<div className={CS.pb2}>
{t`Users in this group will only see rows where the selected column matches their user attribute value.`}
</div>
<div>{t`This rule doesn't apply to native queries`}</div>
</div>
)}
</div>
{shouldUseSavedQuestion && (
<div className={cx(CS.px3, CS.pb3)}>

View File

@@ -14,10 +14,15 @@ import {
renderWithProviders,
screen,
waitFor,
waitForLoaderToBeRemoved,
} from "__support__/ui";
import { ROOT_COLLECTION } from "metabase/entities/collections";
import type { GroupTableAccessPolicy } from "metabase-types/api";
import type {
DatabaseFeature,
GroupTableAccessPolicy,
} from "metabase-types/api";
import {
COMMON_DATABASE_FEATURES,
createMockCard,
createMockCardQueryMetadata,
createMockCollection,
@@ -57,15 +62,17 @@ const TEST_CARD = createMockCard({
},
});
const setup = ({
const setup = async ({
shouldMockQuestions = false,
policy = undefined,
features = COMMON_DATABASE_FEATURES,
}: {
shouldMockQuestions?: boolean;
policy?: GroupTableAccessPolicy;
features?: DatabaseFeature[];
} = {}) => {
mockGetBoundingClientRect();
const database = createSampleDatabase();
const database = createSampleDatabase({ features: features });
setupDatabasesEndpoints([database]);
setupCollectionsEndpoints({
@@ -113,6 +120,8 @@ const setup = ({
/>,
);
await waitForLoaderToBeRemoved();
return { onSave };
};
@@ -124,12 +133,65 @@ describe("EditSandboxingModal", () => {
describe("EditSandboxingModal", () => {
describe("creating new policy", () => {
it("should allow creating a new policy", async () => {
const { onSave } = setup();
const { onSave } = await setup();
expect(
screen.getByText("Restrict access to this table"),
).toBeInTheDocument();
expect(
screen.getByText("Filter by a column in the table"),
).toBeInTheDocument();
expect(
await screen.findByText(
"Use a saved question to create a custom view for this table",
),
).toBeInTheDocument();
expect(screen.getByRole("button", { name: "Save" })).toBeDisabled();
await userEvent.click(await screen.findByText("Pick a column"));
await userEvent.click(await screen.findByText("ID"));
await userEvent.click(screen.getByText("Pick a user attribute"));
await userEvent.click(await screen.findByText("foo"));
await userEvent.click(screen.getByText("Save"));
await waitFor(() =>
expect(onSave).toHaveBeenCalledWith({
attribute_remappings: {
foo: [
"dimension",
["field", PEOPLE.ID, { "base-type": "type/BigInteger" }],
{ "stage-number": 0 },
],
},
card_id: null,
group_id: 1,
table_id: PEOPLE_ID,
}),
);
});
it("should not allow sandboxing with a question if that feature is not enabled", async () => {
const { onSave } = await setup({ features: [] });
expect(
screen.getByText("Restrict access to this table"),
).toBeInTheDocument();
expect(
screen.queryByText("Filter by a column in the table"),
).not.toBeInTheDocument();
expect(
screen.queryByText(
"Use a saved question to create a custom view for this table",
),
).not.toBeInTheDocument();
expect(screen.getByRole("button", { name: "Save" })).toBeDisabled();
await userEvent.click(await screen.findByText("Pick a column"));
@@ -157,7 +219,7 @@ describe("EditSandboxingModal", () => {
});
it("should allow creating a new policy based on a card", async () => {
const { onSave } = setup({ shouldMockQuestions: true });
const { onSave } = await setup({ shouldMockQuestions: true });
expect(
screen.getByText("Restrict access to this table"),
@@ -166,7 +228,7 @@ describe("EditSandboxingModal", () => {
expect(screen.getByRole("button", { name: "Save" })).toBeDisabled();
await userEvent.click(
screen.getByText(
await screen.findByText(
"Use a saved question to create a custom view for this table",
),
);
@@ -177,7 +239,7 @@ describe("EditSandboxingModal", () => {
await screen.findByRole("link", { name: /sandbox question/i }),
);
await userEvent.click(screen.getByText("Save"));
await userEvent.click(await screen.findByText("Save"));
await waitFor(() => {
expect(screen.queryByText("Saving...")).not.toBeInTheDocument();
@@ -195,7 +257,7 @@ describe("EditSandboxingModal", () => {
describe("editing policies", () => {
it("should allow editing an existing policy", async () => {
const { onSave } = setup({
const { onSave } = await setup({
shouldMockQuestions: true,
policy: {
id: 1,
@@ -216,7 +278,7 @@ describe("EditSandboxingModal", () => {
expect(screen.getByRole("button", { name: "Save" })).toBeDisabled();
await userEvent.click(
screen.getByText(
await screen.findByText(
"Use a saved question to create a custom view for this table",
),
);
@@ -227,7 +289,7 @@ describe("EditSandboxingModal", () => {
await screen.findByRole("link", { name: /sandbox question/i }),
);
await userEvent.click(screen.getByText("Save"));
await userEvent.click(await screen.findByText("Save"));
await waitFor(() => {
expect(screen.queryByText("Saving...")).not.toBeInTheDocument();

View File

@@ -1,3 +1,4 @@
import { useDisclosure } from "@mantine/hooks";
import { useMemo } from "react";
import { P, isMatching } from "ts-pattern";
import { t } from "ttag";
@@ -8,7 +9,6 @@ import { SettingToggle } from "metabase/admin/settings/components/widgets/Settin
import type { SettingElement } from "metabase/admin/settings/types";
import { useSetting } from "metabase/common/hooks";
import { LoadingAndErrorWrapper } from "metabase/components/LoadingAndErrorWrapper";
import { useModal } from "metabase/hooks/use-modal";
import {
Box,
Button,
@@ -91,6 +91,15 @@ export const UserProvisioning = ({
const isScimEnabled = !!settingValues["scim-enabled"];
const isScimInitialized = !!maskedTokenRequest.data;
const [
showFirstEnabledModal,
{ open: openFirstEnabledModal, close: closeFirstEnabledModal },
] = useDisclosure(false);
const [
showRegenerateModal,
{ open: openRegenerateModal, close: closeRegenerateModal },
] = useDisclosure(false);
// since the request to enable scim and create the first token are seperate requests, it's possible
// that the token request failed or wasn't issued due to some interuption (network issue, instance down, etc.)
// we can detect this case by seing the scim is enabled without any token after all requests for token have settled
@@ -112,7 +121,7 @@ export const UserProvisioning = ({
onChanged: async () => {
if (enabled && !isScimInitialized) {
await regenerateToken();
firstEnabledModal.open();
openFirstEnabledModal();
}
},
});
@@ -130,9 +139,6 @@ export const UserProvisioning = ({
settingValues["saml-enabled"] ||
settingValues["jwt-enabled"]);
const firstEnabledModal = useModal(false);
const regenerateModal = useModal(false);
return (
<>
<AuthTabs activeKey="user-provisioning" />
@@ -200,7 +206,7 @@ export const UserProvisioning = ({
<Button
disabled={isLoadingToken || !isScimEnabled}
variant="filled"
onClick={regenerateModal.open}
onClick={openRegenerateModal}
>
{regenerateTokenReq.isLoading
? t`Regenerating...`
@@ -246,16 +252,16 @@ export const UserProvisioning = ({
</LoadingAndErrorWrapper>
<UserProvisioningFirstEnabledModal
opened={firstEnabledModal.opened}
onClose={firstEnabledModal.close}
opened={showFirstEnabledModal}
onClose={closeFirstEnabledModal}
unmaskedScimToken={regenerateTokenReq.data?.unmasked_key ?? ""}
scimBaseUrl={fields["scim-base-url"].value?.toString() || ""}
scimError={regenerateTokenReq.error}
/>
<UserProvisioningRegenerateTokenModal
opened={regenerateModal.opened}
onClose={regenerateModal.close}
opened={showRegenerateModal}
onClose={closeRegenerateModal}
/>
</>
);

View File

@@ -0,0 +1,38 @@
import { MBQL_CLAUSES } from "./config";
import type { MBQLClauseFunctionConfig } from "./types";
export type DefinedClauseName = keyof typeof MBQL_CLAUSES;
export function isDefinedClause(name: string): name is DefinedClauseName {
return name in MBQL_CLAUSES;
}
export function getClauseDefinition(
name: DefinedClauseName,
): MBQLClauseFunctionConfig;
export function getClauseDefinition(
name: string,
): MBQLClauseFunctionConfig | undefined;
export function getClauseDefinition(
name: string,
): MBQLClauseFunctionConfig | undefined {
if (isDefinedClause(name)) {
return MBQL_CLAUSES[name];
}
return undefined;
}
const EXPRESSION_TO_MBQL_NAME = new Map(
Object.entries(MBQL_CLAUSES).map(([mbql, { displayName }]) => [
// case-insensitive
displayName.toLowerCase(),
mbql as DefinedClauseName,
]),
);
export function getMBQLName(
expressionName: string,
): DefinedClauseName | undefined {
// case-insensitive
return EXPRESSION_TO_MBQL_NAME.get(expressionName.trim().toLowerCase());
}

View File

@@ -41,11 +41,11 @@ export function compileExpression({
hooks?: Hooks;
}): CompileResult {
try {
const { tokens } = lexify(source);
const tokens = lexify(source);
hooks.lexified?.({ tokens });
const { root } = parse(tokens, { hooks });
const root = parse(tokens, { hooks });
const expressionParts = compile(root, {
expressionMode,
resolver,

View File

@@ -1,10 +1,9 @@
import type { CompletionContext } from "@codemirror/autocomplete";
import { isNotNull } from "metabase/lib/types";
import type * as Lib from "metabase-lib";
import type Metadata from "metabase-lib/v1/metadata/Metadata";
import { AGGREGATION_FUNCTIONS, getClauseDefinition } from "../config";
import { AGGREGATION_FUNCTIONS } from "../config";
import { GROUP } from "../pratt";
import { getDatabase } from "../utils";
@@ -35,9 +34,8 @@ export function suggestAggregations({
}
const database = getDatabase(query, metadata);
const aggregations = Array.from(AGGREGATION_FUNCTIONS)
.map(getClauseDefinition)
.filter(isNotNull)
const aggregations = Object.values(AGGREGATION_FUNCTIONS)
.sort((a, b) => a.name.localeCompare(b.name))
.filter((clause) => clause && database?.hasFeature(clause.requiresFeature))
.map((agg) =>
expressionClauseCompletion(agg, {

View File

@@ -61,19 +61,19 @@ describe("suggestAggregations", () => {
options: [
{
apply: expect.any(Function),
detail: "Only counts rows where the condition is `true`.",
displayLabel: "CountIf",
detail: "Returns the count of rows in the selected data.",
displayLabel: "Count",
icon: "function",
label: "CountIf",
label: "Count",
matches: [[0, 3]],
type: "aggregation",
},
{
apply: expect.any(Function),
detail: "Returns the count of rows in the selected data.",
displayLabel: "Count",
detail: "Only counts rows where the condition is `true`.",
displayLabel: "CountIf",
icon: "function",
label: "Count",
label: "CountIf",
matches: [[0, 3]],
type: "aggregation",
},

View File

@@ -1,11 +1,16 @@
import type { CompletionContext } from "@codemirror/autocomplete";
import { isNotNull } from "metabase/lib/types";
import type * as Lib from "metabase-lib";
import type Metadata from "metabase-lib/v1/metadata/Metadata";
import { EXPRESSION_FUNCTIONS, getClauseDefinition } from "../config";
import { GROUP } from "../pratt";
import { EXPRESSION_FUNCTIONS } from "../config";
import {
GROUP,
LOGICAL_AND,
LOGICAL_NOT,
LOGICAL_OR,
type Token,
} from "../pratt";
import { getDatabase } from "../utils";
import {
@@ -14,7 +19,6 @@ import {
fuzzyMatcher,
isFieldReference,
isIdentifier,
isOperator,
tokenAtPos,
} from "./util";
@@ -36,9 +40,7 @@ export function suggestFunctions({
}
const database = getDatabase(query, metadata);
const functions = [...EXPRESSION_FUNCTIONS]
.map(getClauseDefinition)
.filter(isNotNull)
const functions = Object.values(EXPRESSION_FUNCTIONS)
.filter((clause) => clause && database?.hasFeature(clause.requiresFeature))
.filter(function disableOffsetInFilterExpressions(clause) {
const isOffset = clause.name === "offset";
@@ -46,6 +48,7 @@ export function suggestFunctions({
const isOffsetInFilterExpression = isOffset && isFilterExpression;
return !isOffsetInFilterExpression;
})
.sort((a, b) => a.name.localeCompare(b.name))
.map((func) =>
expressionClauseCompletion(func, {
type: "function",
@@ -62,7 +65,7 @@ export function suggestFunctions({
if (
!token ||
!(isIdentifier(token) || isOperator(token)) ||
!isPotentialFunctionPrefix(token) ||
isFieldReference(token)
) {
return null;
@@ -82,3 +85,9 @@ export function suggestFunctions({
};
};
}
const PREFIX_OPERATORS = new Set([LOGICAL_OR, LOGICAL_AND, LOGICAL_NOT]);
function isPotentialFunctionPrefix(token: Token) {
return isIdentifier(token) || PREFIX_OPERATORS.has(token.type);
}

View File

@@ -88,13 +88,13 @@ describe("suggestFunctions", () => {
{
apply: expect.any(Function),
detail:
'Returns the localized short name (eg. `"Apr"`) for the given month number (eg. `4`)',
displayLabel: "monthName",
"Looks at the values in each argument in order and returns the first non-null value for each row.",
displayLabel: "coalesce",
icon: "function",
label: "monthName",
label: "coalesce",
matches: [
[1, 2],
[5, 5],
[0, 1],
[6, 6],
],
type: "function",
},
@@ -111,13 +111,13 @@ describe("suggestFunctions", () => {
{
apply: expect.any(Function),
detail:
"Looks at the values in each argument in order and returns the first non-null value for each row.",
displayLabel: "coalesce",
'Returns the localized short name (eg. `"Apr"`) for the given month number (eg. `4`)',
displayLabel: "monthName",
icon: "function",
label: "coalesce",
label: "monthName",
matches: [
[0, 1],
[6, 6],
[1, 2],
[5, 5],
],
type: "function",
},
@@ -270,18 +270,18 @@ describe("suggestFunctions", () => {
const completer = setup({ expressionMode: "expression" });
const results = await completer("not|");
expect(results?.options.map((result) => result.displayLabel)).toEqual([
"notEmpty",
"notIn",
"notNull",
"notEmpty",
"doesNotContain",
"now",
"interval",
"intervalStartingFrom",
"interval",
"contains",
"minute",
"month",
"length",
"monthName",
"month",
"minute",
"contains",
]);
});
});

View File

@@ -9,7 +9,6 @@ import {
END_OF_INPUT,
FIELD,
IDENTIFIER,
OPERATORS,
STRING,
type Token,
lexify,
@@ -91,7 +90,7 @@ export function fuzzyMatcher(options: Completion[]) {
}
export function tokenAtPos(source: string, pos: number): Token | null {
const { tokens } = lexify(source);
const tokens = lexify(source);
const idx = tokens.findIndex(
(token) => token.start <= pos && token.end >= pos,
@@ -123,10 +122,6 @@ export function isIdentifier(token: Token | null) {
return token != null && (token.type === IDENTIFIER || token.type === CALL);
}
export function isOperator(token: Token | null) {
return token != null && OPERATORS.has(token.type);
}
export function isFieldReference(token: Token | null) {
return token != null && token.type === FIELD;
}

View File

@@ -1,40 +1,7 @@
import { t } from "ttag";
import { OPERATOR } from "./tokenizer";
import type { MBQLClauseFunctionConfig } from "./types";
export const EDITOR_QUOTES = {
// specifies where different quoting is used:
characters: {
"[": "identifier",
"'": "literal",
'"': "literal",
},
// specifies the default quoting style:
literalQuoteDefault: '"' as const,
identifierQuoteDefault: "[" as const,
// always quote identifiers even if they have non-word characters or conflict with reserved words
identifierAlwaysQuoted: true,
};
export const EDITOR_FK_SYMBOLS = {
// specifies which symbols can be used to delimit foreign/joined fields
symbols: [".", " → "],
// specifies the default/canonical symbol
default: " → ",
};
// copied relevant parts from https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Operator_Precedence
export const OPERATOR_PRECEDENCE: Record<string, number> = {
not: 17,
"*": 15,
"/": 15,
"+": 14,
"-": 14,
and: 6,
or: 5,
};
type Optional<T, K extends keyof T> = Pick<Partial<T>, K> & Omit<T, K>;
type Forbidden<T, K extends keyof T> = Omit<T, K> & { [P in K]?: never };
@@ -60,70 +27,84 @@ function defineClauses<const T extends Record<string, ConfigInput>>(
return result;
}
// `type` and `args` types have no effect. Type checking is done by MBQL lib.
export const MBQL_CLAUSES = defineClauses({
export const AGGREGATION_FUNCTIONS = defineClauses({
// aggregation functions
count: { displayName: `Count`, type: "aggregation", args: [] },
count: { displayName: "Count", type: "aggregation", args: [] },
"cum-count": {
displayName: `CumulativeCount`,
displayName: "CumulativeCount",
type: "aggregation",
args: [],
},
sum: { displayName: `Sum`, type: "aggregation", args: ["number"] },
sum: { displayName: "Sum", type: "aggregation", args: ["number"] },
"cum-sum": {
displayName: `CumulativeSum`,
displayName: "CumulativeSum",
type: "aggregation",
args: ["number"],
},
distinct: {
displayName: `Distinct`,
displayName: "Distinct",
type: "aggregation",
args: ["expression"],
},
stddev: {
displayName: `StandardDeviation`,
displayName: "StandardDeviation",
type: "aggregation",
args: ["number"],
requiresFeature: "standard-deviation-aggregations",
},
avg: { displayName: `Average`, type: "aggregation", args: ["number"] },
avg: { displayName: "Average", type: "aggregation", args: ["number"] },
median: {
displayName: `Median`,
displayName: "Median",
type: "aggregation",
args: ["number"],
requiresFeature: "percentile-aggregations",
},
min: { displayName: `Min`, type: "aggregation", args: ["expression"] },
max: { displayName: `Max`, type: "aggregation", args: ["expression"] },
share: { displayName: `Share`, type: "aggregation", args: ["boolean"] },
min: { displayName: "Min", type: "aggregation", args: ["expression"] },
max: { displayName: "Max", type: "aggregation", args: ["expression"] },
share: { displayName: "Share", type: "aggregation", args: ["boolean"] },
"count-where": {
displayName: `CountIf`,
displayName: "CountIf",
type: "aggregation",
args: ["boolean"],
},
"distinct-where": {
displayName: `DistinctIf`,
displayName: "DistinctIf",
type: "aggregation",
args: ["number", "boolean"],
requiresFeature: "distinct-where",
},
"sum-where": {
displayName: `SumIf`,
displayName: "SumIf",
type: "aggregation",
args: ["number", "boolean"],
},
var: {
displayName: `Variance`,
displayName: "Variance",
type: "aggregation",
args: ["number"],
requiresFeature: "standard-deviation-aggregations",
},
percentile: {
displayName: `Percentile`,
displayName: "Percentile",
type: "aggregation",
args: ["number", "number"],
requiresFeature: "percentile-aggregations",
},
offset: {
displayName: "Offset",
type: "any", // ideally we'd dynamically infer it from the first argument
args: ["any", "number"],
requiresFeature: "window-functions/offset",
validator(_expr: any, offset: number) {
if (offset === 0) {
return t`Row offset cannot be zero`;
}
},
hasOptions: true,
},
});
export const EXPRESSION_FUNCTIONS = defineClauses({
// cast functions
text: {
displayName: "text",
@@ -150,13 +131,13 @@ export const MBQL_CLAUSES = defineClauses({
requiresFeature: "expressions/float",
},
// string functions
lower: { displayName: `lower`, type: "string", args: ["string"] },
upper: { displayName: `upper`, type: "string", args: ["string"] },
lower: { displayName: "lower", type: "string", args: ["string"] },
upper: { displayName: "upper", type: "string", args: ["string"] },
substring: {
displayName: `substring`,
displayName: "substring",
type: "string",
args: ["string", "number", "number"],
validator: function (_arg: any, start: number, _length: any) {
validator(_arg: any, start: number, _length: any) {
if (start <= 0) {
return t`Expected positive integer but found ${start}`;
}
@@ -166,7 +147,7 @@ export const MBQL_CLAUSES = defineClauses({
displayName: "splitPart",
type: "string",
args: ["string", "string", "number"],
validator: function (_arg: any, _delimeter: string, position: number) {
validator(_arg: any, _delimeter: string, position: number) {
if (position < 1) {
return t`Expected positive integer but found ${position}`;
}
@@ -180,139 +161,139 @@ export const MBQL_CLAUSES = defineClauses({
requiresFeature: "regex",
},
path: {
displayName: `path`,
displayName: "path",
type: "string",
args: ["string"],
requiresFeature: "regex",
},
concat: {
displayName: `concat`,
displayName: "concat",
type: "string",
args: ["expression", "expression"],
multiple: true,
},
replace: {
displayName: `replace`,
displayName: "replace",
type: "string",
args: ["string", "string", "string"],
},
length: { displayName: `length`, type: "number", args: ["string"] },
trim: { displayName: `trim`, type: "string", args: ["string"] },
rtrim: { displayName: `rTrim`, type: "string", args: ["string"] },
ltrim: { displayName: `lTrim`, type: "string", args: ["string"] },
length: { displayName: "length", type: "number", args: ["string"] },
trim: { displayName: "trim", type: "string", args: ["string"] },
rtrim: { displayName: "rTrim", type: "string", args: ["string"] },
ltrim: { displayName: "lTrim", type: "string", args: ["string"] },
domain: {
displayName: `domain`,
displayName: "domain",
type: "string",
args: ["string"],
requiresFeature: "regex",
},
subdomain: {
displayName: `subdomain`,
displayName: "subdomain",
type: "string",
args: ["string"],
requiresFeature: "regex",
},
host: {
displayName: `host`,
displayName: "host",
type: "string",
args: ["string"],
requiresFeature: "regex",
},
"month-name": {
displayName: `monthName`,
displayName: "monthName",
type: "string",
args: ["number"],
},
"quarter-name": {
displayName: `quarterName`,
displayName: "quarterName",
type: "string",
args: ["number"],
},
"day-name": {
displayName: `dayName`,
displayName: "dayName",
type: "string",
args: ["number"],
},
// numeric functions
abs: {
displayName: `abs`,
displayName: "abs",
type: "number",
args: ["number"],
requiresFeature: "expressions",
},
floor: {
displayName: `floor`,
displayName: "floor",
type: "number",
args: ["number"],
requiresFeature: "expressions",
},
ceil: {
displayName: `ceil`,
displayName: "ceil",
type: "number",
args: ["number"],
requiresFeature: "expressions",
},
round: {
displayName: `round`,
displayName: "round",
type: "number",
args: ["number"],
requiresFeature: "expressions",
},
sqrt: {
displayName: `sqrt`,
displayName: "sqrt",
type: "number",
args: ["number"],
requiresFeature: "advanced-math-expressions",
},
power: {
displayName: `power`,
displayName: "power",
type: "number",
args: ["number", "number"],
requiresFeature: "advanced-math-expressions",
},
log: {
displayName: `log`,
displayName: "log",
type: "number",
args: ["number"],
requiresFeature: "advanced-math-expressions",
},
exp: {
displayName: `exp`,
displayName: "exp",
type: "number",
args: ["number"],
requiresFeature: "advanced-math-expressions",
},
// boolean functions
contains: {
displayName: `contains`,
displayName: "contains",
type: "boolean",
args: ["string", "string"],
multiple: true,
hasOptions: true,
},
"does-not-contain": {
displayName: `doesNotContain`,
displayName: "doesNotContain",
type: "boolean",
args: ["string", "string"],
multiple: true,
hasOptions: true,
},
"starts-with": {
displayName: `startsWith`,
displayName: "startsWith",
type: "boolean",
args: ["string", "string"],
multiple: true,
hasOptions: true,
},
"ends-with": {
displayName: `endsWith`,
displayName: "endsWith",
type: "boolean",
args: ["string", "string"],
multiple: true,
hasOptions: true,
},
between: {
displayName: `between`,
displayName: "between",
type: "boolean",
args: ["expression", "expression", "expression"],
},
@@ -322,7 +303,7 @@ export const MBQL_CLAUSES = defineClauses({
args: ["number", "string"],
},
"time-interval": {
displayName: `interval`,
displayName: "interval",
type: "boolean",
args: ["expression", "number", "string"],
hasOptions: true,
@@ -338,28 +319,28 @@ export const MBQL_CLAUSES = defineClauses({
args: ["number", "string"],
},
"is-null": {
displayName: `isNull`,
displayName: "isNull",
type: "boolean",
args: ["expression"],
},
"not-null": {
displayName: `notNull`,
displayName: "notNull",
type: "boolean",
args: ["expression"],
},
"is-empty": {
displayName: `isEmpty`,
displayName: "isEmpty",
type: "boolean",
args: ["expression"],
},
"not-empty": {
displayName: `notEmpty`,
displayName: "notEmpty",
type: "boolean",
args: ["expression"],
},
// other expression functions
coalesce: {
displayName: `coalesce`,
displayName: "coalesce",
type: "expression",
args: ["expression", "expression"],
argType(_index, _args, type) {
@@ -368,7 +349,7 @@ export const MBQL_CLAUSES = defineClauses({
multiple: true,
},
case: {
displayName: `case`,
displayName: "case",
type: "expression",
multiple: true,
args: ["expression", "expression"], // ideally we'd alternate boolean/expression
@@ -384,7 +365,7 @@ export const MBQL_CLAUSES = defineClauses({
},
},
if: {
displayName: `if`,
displayName: "if",
type: "expression",
multiple: true,
args: ["expression", "expression"],
@@ -399,21 +380,100 @@ export const MBQL_CLAUSES = defineClauses({
return "boolean";
},
},
offset: {
displayName: `Offset`,
type: "any", // ideally we'd dynamically infer it from the first argument
args: ["any", "number"],
requiresFeature: "window-functions/offset",
validator: function (_expr: any, offset: number) {
if (offset === 0) {
return t`Row offset cannot be zero`;
}
},
hasOptions: true,
//"in` and `not-in` are aliases for `=` and `!="
in: {
displayName: "in",
type: "boolean",
args: ["expression", "expression"],
multiple: true,
},
"not-in": {
displayName: "notIn",
type: "boolean",
args: ["expression", "expression"],
multiple: true,
},
"get-year": {
displayName: "year",
type: "number",
args: ["datetime"],
},
"get-quarter": {
displayName: "quarter",
type: "number",
args: ["datetime"],
},
"get-month": {
displayName: "month",
type: "number",
args: ["datetime"],
},
"get-week": {
displayName: "week",
type: "number",
args: ["datetime"],
hasOptions: true, // optional mode parameter
},
"get-day": {
displayName: "day",
type: "number",
args: ["datetime"],
},
"get-day-of-week": {
displayName: "weekday",
type: "number",
args: ["datetime"],
hasOptions: true, // optional mode parameter
},
"get-hour": {
displayName: "hour",
type: "number",
args: ["datetime"],
},
"get-minute": {
displayName: "minute",
type: "number",
args: ["datetime"],
},
"get-second": {
displayName: "second",
type: "number",
args: ["datetime"],
},
"datetime-diff": {
displayName: "datetimeDiff",
type: "number",
args: ["datetime", "datetime", "string"],
requiresFeature: "datetime-diff",
},
"datetime-add": {
displayName: "datetimeAdd",
type: "datetime",
args: ["datetime", "number", "string"],
},
"datetime-subtract": {
displayName: "datetimeSubtract",
type: "datetime",
args: ["datetime", "number", "string"],
},
now: {
displayName: "now",
type: "datetime",
args: [],
},
"convert-timezone": {
displayName: "convertTimezone",
type: "datetime",
args: ["datetime", "string"],
hasOptions: true,
requiresFeature: "convert-timezone",
},
});
export const EXPRESSION_OPERATORS = defineClauses({
// boolean operators
and: {
displayName: `AND`,
displayName: "AND",
type: "boolean",
multiple: true,
args: ["boolean", "boolean"],
@@ -422,7 +482,7 @@ export const MBQL_CLAUSES = defineClauses({
},
},
or: {
displayName: `OR`,
displayName: "OR",
type: "boolean",
multiple: true,
args: ["boolean", "boolean"],
@@ -431,7 +491,7 @@ export const MBQL_CLAUSES = defineClauses({
},
},
not: {
displayName: `NOT`,
displayName: "NOT",
type: "boolean",
args: ["boolean"],
},
@@ -484,7 +544,7 @@ export const MBQL_CLAUSES = defineClauses({
return "number";
},
},
// comparison operators
// equality operators
"=": {
displayName: "=",
type: "boolean",
@@ -495,19 +555,7 @@ export const MBQL_CLAUSES = defineClauses({
type: "boolean",
args: ["expression", "expression"],
},
// `in` and `not-in` are aliases for `=` and `!=`
in: {
displayName: "in",
type: "boolean",
args: ["expression", "expression"],
multiple: true,
},
"not-in": {
displayName: "notIn",
type: "boolean",
args: ["expression", "expression"],
multiple: true,
},
// comparison operators
"<=": {
displayName: "<=",
type: "boolean",
@@ -528,259 +576,10 @@ export const MBQL_CLAUSES = defineClauses({
type: "boolean",
args: ["expression", "expression"],
},
"get-year": {
displayName: `year`,
type: "number",
args: ["datetime"],
},
"get-quarter": {
displayName: `quarter`,
type: "number",
args: ["datetime"],
},
"get-month": {
displayName: `month`,
type: "number",
args: ["datetime"],
},
"get-week": {
displayName: `week`,
type: "number",
args: ["datetime"],
hasOptions: true, // optional mode parameter
},
"get-day": {
displayName: `day`,
type: "number",
args: ["datetime"],
},
"get-day-of-week": {
displayName: `weekday`,
type: "number",
args: ["datetime"],
hasOptions: true, // optional mode parameter
},
"get-hour": {
displayName: `hour`,
type: "number",
args: ["datetime"],
},
"get-minute": {
displayName: `minute`,
type: "number",
args: ["datetime"],
},
"get-second": {
displayName: `second`,
type: "number",
args: ["datetime"],
},
"datetime-diff": {
displayName: `datetimeDiff`,
type: "number",
args: ["datetime", "datetime", "string"],
requiresFeature: "datetime-diff",
},
"datetime-add": {
displayName: `datetimeAdd`,
type: "datetime",
args: ["datetime", "number", "string"],
},
"datetime-subtract": {
displayName: `datetimeSubtract`,
type: "datetime",
args: ["datetime", "number", "string"],
},
now: {
displayName: `now`,
type: "datetime",
args: [],
},
"convert-timezone": {
displayName: `convertTimezone`,
type: "datetime",
args: ["datetime", "string"],
hasOptions: true,
requiresFeature: "convert-timezone",
},
});
export type DefinedClauseName = keyof typeof MBQL_CLAUSES;
export function isDefinedClause(name: string): name is DefinedClauseName {
return name in MBQL_CLAUSES;
}
export function getClauseDefinition(
name: DefinedClauseName,
): MBQLClauseFunctionConfig;
export function getClauseDefinition(
name: string,
): MBQLClauseFunctionConfig | undefined;
export function getClauseDefinition(
name: string,
): MBQLClauseFunctionConfig | undefined {
if (isDefinedClause(name)) {
return MBQL_CLAUSES[name];
}
return undefined;
}
// Reserved token names
const MBQL_TO_EXPRESSION_NAME = new Map(
Object.entries(MBQL_CLAUSES).map(([mbql, { displayName }]) => [
mbql,
displayName,
]),
);
const EXPRESSION_TO_MBQL_NAME = new Map(
Object.entries(MBQL_CLAUSES).map(([mbql, { displayName }]) => [
// case-insensitive
displayName.toLowerCase(),
mbql as DefinedClauseName,
]),
);
export function getExpressionName(mbqlName: string) {
return MBQL_TO_EXPRESSION_NAME.get(mbqlName);
}
export function getMBQLName(
expressionName: string,
): DefinedClauseName | undefined {
// case-insensitive
return EXPRESSION_TO_MBQL_NAME.get(expressionName.trim().toLowerCase());
}
export const AGGREGATION_FUNCTIONS = new Set([
// count-where/sum-where must come before count/sum
"count-where",
"sum-where",
"count",
"cum-count",
"sum",
"cum-sum",
"distinct",
"distinct-where",
"stddev",
"offset",
"avg",
"median",
"min",
"max",
"share",
"var",
"percentile",
]);
export const EXPRESSION_FUNCTIONS = new Set([
// cast
"text",
"integer",
"float",
"date",
// string
"lower",
"upper",
"substring",
"split-part",
"regex-match-first",
"path",
"concat",
"replace",
"trim",
"rtrim",
"ltrim",
"length",
"domain",
"subdomain",
"host",
"month-name",
"quarter-name",
"day-name",
// number
"abs",
"floor",
"ceil",
"round",
"sqrt",
"power",
"log",
"exp",
"datetime-diff",
// date/time
"get-year",
"get-quarter",
"get-month",
"get-week",
"get-day",
"get-day-of-week",
"get-hour",
"get-minute",
"get-second",
"datetime-add",
"datetime-subtract",
"now",
"convert-timezone",
// boolean
"in",
"not-in",
"contains",
"ends-with",
"starts-with",
"between",
"time-interval",
"relative-time-interval",
"relative-datetime",
"interval",
"is-null",
"not-null",
"is-empty",
"not-empty",
"does-not-contain",
// other
"if",
"case",
"coalesce",
]);
export const NUMBER_OPERATORS = new Set([
OPERATOR.Plus,
OPERATOR.Minus,
OPERATOR.Star,
OPERATOR.Slash,
]);
// operators in which order of operands doesn't matter
export const EXPRESSION_OPERATOR_WITHOUT_ORDER_PRIORITY = new Set(["+", "*"]);
export const COMPARISON_OPERATORS = new Set([
OPERATOR.Equal,
OPERATOR.NotEqual,
OPERATOR.GreaterThan,
OPERATOR.LessThan,
OPERATOR.GreaterThanEqual,
OPERATOR.LessThanEqual,
]);
export const LOGICAL_OPERATORS = new Set([
OPERATOR.Not,
OPERATOR.And,
OPERATOR.Or,
]);
export const FUNCTIONS = new Set([
...EXPRESSION_FUNCTIONS,
export const MBQL_CLAUSES = {
...AGGREGATION_FUNCTIONS,
]);
export const OPERATORS = new Set([
...NUMBER_OPERATORS,
...COMPARISON_OPERATORS,
...LOGICAL_OPERATORS,
]);
export const FIELD_MARKERS = new Set([
"dimension",
"field",
"segment",
"metric",
]);
...EXPRESSION_FUNCTIONS,
...EXPRESSION_OPERATORS,
};

View File

@@ -2,7 +2,7 @@ import { msgid, ngettext } from "ttag";
import * as Lib from "metabase-lib";
import { getClauseDefinition } from "../../config";
import { getClauseDefinition } from "../../clause";
import { visit } from "../../visitor";
import { error } from "../utils";

View File

@@ -1,6 +1,6 @@
import * as Lib from "metabase-lib";
import { getClauseDefinition } from "../../config";
import { getClauseDefinition } from "../../clause";
import { visit } from "../../visitor";
import { error } from "../utils";

View File

@@ -2,8 +2,8 @@ import { t } from "ttag";
import * as Lib from "metabase-lib";
import { COMPARISON_OPERATORS } from "../../config";
import type { OPERATOR } from "../../tokenizer";
import { COMPARISON, EQUALITY } from "../../pratt";
import { parsePunctuator } from "../../punctuator";
import { visit } from "../../visitor";
import { error } from "../utils";
@@ -16,10 +16,13 @@ export function checkComparisonOperatorArgs({
if (!Lib.isExpressionParts(node)) {
return;
}
const { operator, args } = node;
if (!COMPARISON_OPERATORS.has(operator as OPERATOR)) {
const type = parsePunctuator(operator);
if (type !== COMPARISON && type !== EQUALITY) {
return;
}
const [firstOperand] = args;
if (typeof firstOperand === "number") {
error(node, t`Expecting field but found ${firstOperand}`);

View File

@@ -2,7 +2,7 @@ import { t } from "ttag";
import * as Lib from "metabase-lib";
import { FIELD_MARKERS, getClauseDefinition } from "../../config";
import { getClauseDefinition } from "../../clause";
import { visit } from "../../visitor";
import { error } from "../utils";
@@ -17,7 +17,7 @@ export function checkKnownFunctions({
}
const { operator } = node;
if (FIELD_MARKERS.has(operator) || operator === "value") {
if (operator === "value") {
return;
}

View File

@@ -3,7 +3,7 @@ import { t } from "ttag";
import * as Lib from "metabase-lib";
import type Metadata from "metabase-lib/v1/metadata/Metadata";
import { getClauseDefinition } from "../../config";
import { getClauseDefinition } from "../../clause";
import { getDatabase } from "../../utils";
import { visit } from "../../visitor";
import { error } from "../utils";

View File

@@ -1,6 +1,6 @@
import { t } from "ttag";
import { getClauseDefinition, getMBQLName } from "../../config";
import { getClauseDefinition, getMBQLName } from "../../clause";
import { GROUP, IDENTIFIER, type Token } from "../../pratt";
import { error } from "../utils";

View File

@@ -6,25 +6,22 @@ import { parseNumber } from "metabase/lib/number";
import * as Lib from "metabase-lib";
import { isa } from "metabase-lib/v1/types/utils/isa";
import {
type EDITOR_QUOTES,
EXPRESSION_OPERATOR_WITHOUT_ORDER_PRIORITY,
FIELD_MARKERS,
OPERATORS,
OPERATOR_PRECEDENCE,
getClauseDefinition,
getExpressionName,
} from "../config";
import { getClauseDefinition } from "../clause";
import {
formatDimensionName,
formatIdentifier,
formatMetricName,
formatSegmentName,
} from "../identifier";
import { formatStringLiteral } from "../string";
import type { OPERATOR } from "../tokenizer";
import { parsePunctuator } from "../punctuator";
import { type StartDelimiter, formatStringLiteral } from "../string";
import { pathMatchers as check } from "./utils";
import {
pathMatchers as check,
isDimensionOperator,
isExpressionOperator,
isValueOperator,
} from "./utils";
export type ExpressionNode = Lib.ExpressionParts | Lib.ExpressionArg | null;
@@ -50,7 +47,7 @@ type FormatOptions = {
stageIndex?: number;
expressionIndex?: number | undefined;
printWidth?: number;
delimiters?: typeof EDITOR_QUOTES;
stringDelimiter?: StartDelimiter;
};
export async function formatExpressionParts(
@@ -134,25 +131,21 @@ function print(
} else if (check.isBooleanLiteral(path)) {
return formatBooleanLiteral(path.node);
} else if (check.isStringLiteral(path)) {
return formatStringLiteral(path.node, options.extra);
return formatStringLiteral(path.node, options.extra.stringDelimiter);
} else if (check.isColumnMetadata(path)) {
return formatColumn(path, options.extra);
} else if (check.isMetricMetadata(path)) {
return formatMetric(path, options.extra);
} else if (check.isSegmentMetadata(path)) {
return formatSegment(path, options.extra);
} else if (check.isExpressionOperator(path)) {
return formatOperator(path, print);
} else if (check.isDimensionOperator(path)) {
return formatDimension(path);
} else if (check.isValueOperator(path)) {
return formatValueExpression(path, print);
} else if (check.isExpressionParts(path)) {
if (isOperator(path.node.operator)) {
return formatOperator(path, print);
} else if (isExpression(path.node.operator)) {
return formatExpression(path);
} else if (isDimension(path.node.operator)) {
return formatDimension(path);
} else if (isValueOperator(path.node.operator)) {
return formatValueExpression(path, print);
} else {
return formatFunctionCall(path, print);
}
return formatFunctionCall(path, print);
}
throw new Error(`Unknown MBQL clause: ${JSON.stringify(path.node)}`);
@@ -200,7 +193,7 @@ function formatColumn(
assert(Lib.isColumnMetadata(column), "Expected column");
const info = Lib.displayInfo(query, stageIndex, column);
return formatDimensionName(info.longDisplayName, options);
return formatDimensionName(info.longDisplayName);
}
function formatMetric(
@@ -215,7 +208,7 @@ function formatMetric(
assert(Lib.isMetricMetadata(metric), "Expected metric");
const displayInfo = Lib.displayInfo(query, stageIndex, metric);
return formatMetricName(displayInfo.displayName, options);
return formatMetricName(displayInfo.displayName);
}
function formatSegment(
@@ -230,31 +223,12 @@ function formatSegment(
assert(Lib.isSegmentMetadata(segment), "Expected segment");
const displayInfo = Lib.displayInfo(query, stageIndex, segment);
return formatSegmentName(displayInfo.displayName, options);
}
function isExpression(op: string): op is "expression" {
return op === "expression";
}
function formatExpression(path: AstPath<Lib.ExpressionParts>): Doc {
const { node } = path;
assert(isExpression(node.operator), "Expected expression");
const name = node.args[0];
assert(typeof name === "string", "Expected expression name to be a string");
return formatIdentifier(name);
}
function isDimension(op: string): op is "dimension" {
return op === "dimension";
return formatSegmentName(displayInfo.displayName);
}
function formatDimension(path: AstPath<Lib.ExpressionParts>): Doc {
const { node } = path;
assert(isDimension(node.operator), "Expected dimension");
assert(isDimensionOperator(node), "Expected dimension");
const name = node.args[0];
assert(typeof name === "string", "Expected expression name to be a string");
@@ -267,7 +241,6 @@ function formatFunctionCall(
print: Print,
): Doc {
const { node } = path;
const name = node.operator;
const args = node.args.map((arg: ExpressionNode) =>
recurse(path, print, arg),
@@ -278,7 +251,28 @@ function formatFunctionCall(
args.push(options);
}
return formatCallExpression(name, args);
// render a call expression as
//
// callee(arg1, arg2, ...)
//
// or, when not enough space
//
// callee(
// arg1,
// arg2,
// ...
// )
return group([
displayName(node.operator),
"(",
indent([
// indent args
softline,
join([",", line], args),
]),
softline,
")",
]);
}
function formatExpressionOptions(options: Lib.ExpressionOptions): Doc | null {
@@ -291,25 +285,22 @@ function formatExpressionOptions(options: Lib.ExpressionOptions): Doc | null {
return null;
}
function isOperator(operator: string): operator is OPERATOR {
return OPERATORS.has(operator as OPERATOR);
}
function formatOperator(path: AstPath<Lib.ExpressionParts>, print: Print): Doc {
const { node } = path;
assert(
isOperator(node.operator),
isExpressionOperator(node),
`Expected operator but got ${node.operator}`,
);
const shouldPrefixOperator = isLogicOperator(node.operator);
const operatorGoesOnNewline =
node.operator === "and" || node.operator === "or";
const args = node.args.map((arg, index) => {
const ln = index === 0 || shouldPrefixOperator ? "" : line;
const ln = index === 0 || operatorGoesOnNewline ? "" : line;
function ind(doc: Doc) {
if (index === 0 || shouldPrefixOperator) {
if (index === 0 || operatorGoesOnNewline) {
return doc;
}
return indent(doc);
@@ -317,8 +308,8 @@ function formatOperator(path: AstPath<Lib.ExpressionParts>, print: Print): Doc {
if (
!Lib.isExpressionParts(arg) ||
isValueOperator(arg.operator) ||
FIELD_MARKERS.has(arg.operator)
isValueOperator(arg) ||
isDimensionOperator(arg)
) {
// Not a call expression so not an operator
return ind([ln, recurse(path, print, path.node.args[index])]);
@@ -327,18 +318,18 @@ function formatOperator(path: AstPath<Lib.ExpressionParts>, print: Print): Doc {
const argOperator = arg.operator;
const formattedArg = recurse(path, print, path.node.args[index]);
const isLowerPrecedence =
OPERATOR_PRECEDENCE[node.operator] > OPERATOR_PRECEDENCE[argOperator];
const operatorPrecedence = precedence(node.operator);
const argPrecedence = precedence(argOperator);
const isLowerPrecedence = operatorPrecedence > argPrecedence;
const isSamePrecedence = operatorPrecedence === argPrecedence;
// "*","/" always have two arguments.
// If the second argument of "/" is an expression, we have to calculate it first.
// Hence, adding parenthesis.
// "a / b * c" vs "a / (b * c)", "a / b / c" vs "a / (b / c)"
// "a - b + c" vs "a - (b + c)", "a - b - c" vs "a - (b - c)"
const isSamePrecedenceWithExecutionPriority =
index > 0 &&
OPERATOR_PRECEDENCE[node.operator] === OPERATOR_PRECEDENCE[argOperator] &&
!EXPRESSION_OPERATOR_WITHOUT_ORDER_PRIORITY.has(node.operator);
const isSamePrecedenceWithExecutionPriority = index > 0 && isSamePrecedence;
const shouldUseParens =
isLowerPrecedence || isSamePrecedenceWithExecutionPriority;
@@ -366,24 +357,19 @@ function formatOperator(path: AstPath<Lib.ExpressionParts>, print: Print): Doc {
return [displayName(node.operator), " ", args[0]];
}
if (shouldPrefixOperator) {
if (operatorGoesOnNewline) {
return group(join([line, displayName(node.operator), " "], args));
} else {
return group(join([" ", displayName(node.operator)], args));
}
}
function isLogicOperator(op: string) {
return op === "and" || op === "or";
function precedence(op: string): number {
return parsePunctuator(op)?.precedence ?? Infinity;
}
function isUnaryOperator(op: string) {
const clause = getClauseDefinition(op);
return clause && clause?.args.length === 1;
}
function isValueOperator(op: string): op is "value" {
return op === "value";
function isUnaryOperator(op: Lib.ExpressionOperator) {
return getClauseDefinition(op)?.args.length === 1;
}
function formatValueExpression(
@@ -391,14 +377,14 @@ function formatValueExpression(
print: Print,
): Doc {
const { node } = path;
assert(isValueOperator(node), "Expected value");
const {
operator,
args: [value],
options,
} = node;
assert(isValueOperator(operator), "Expected value");
const baseType = options?.["base-type"];
if (
typeof value === "string" &&
@@ -413,34 +399,7 @@ function formatValueExpression(
return recurse(path, print, node.args[0]);
}
function formatCallExpression(callee: string, args: Doc[]): Doc {
// render a call expression as
//
// callee(arg1, arg2, ...)
//
// or, when not enough space
//
// callee(
// arg1,
// arg2,
// ...
// )
return group([
displayName(callee),
"(",
indent([
// indent args
softline,
join([",", line], args),
]),
softline,
")",
]);
}
/**
* Format the name of an expression clause.
*/
function displayName(name: string): string {
return getExpressionName(name) ?? name;
function displayName(name: Lib.ExpressionOperator): string {
assert(name !== "value", "Unexpected value clause");
return getClauseDefinition(name)?.displayName ?? name;
}

View File

@@ -100,6 +100,45 @@ describe("format", () => {
expression`
90071992547409901
`,
expression`
1 + 2 - (3 + 4)
`,
expression`
1 - 2 + (3 - 4)
`,
expression`
1 - 2 - 3 + 4
`,
expression`
1 * 2 / (3 + 4)
`,
expression`
1 / 2 * (3 / 4)
`,
expression`
1 / 2 / 3 * 4
`,
expression`
1 * (2 + 3)
`,
expression`
1 * (2 - 3)
`,
expression`
1 * (2 / 3)
`,
expression`
1 / (2 + 3)
`,
expression`
1 / (2 - 3)
`,
expression`
1 / (2 / 3)
`,
expression`
1 - (2 - 3)
`,
]);
});
@@ -205,6 +244,12 @@ describe("format", () => {
)
OR [User ID] = 1
`,
expression`
NOT (
[Product ID] > 10
AND [Tax] < 10
)
`,
]);
});
});
@@ -329,12 +374,6 @@ describe("if printWidth = Infinity, it should return the same results as the sin
});
});
it("should format expression references", async () => {
await all({
"[foo]": op("expression", "foo"),
});
});
it("should format function calls", async () => {
await all({
"now()": op("now"),

Some files were not shown because too many files have changed in this diff Show More