Compare commits

..

7 Commits

Author SHA1 Message Date
dan sutton
f8a74f8c30 new api method to unconditionally delete routing
this doesn't make a ton of sense in the current world where there's only
one way to route a db. But in the future, when you want to delete a
database, you don't care about how a db was routed; tear it all down and
delete all routers, all children, and then the database.
2025-08-14 21:55:55 -05:00
dan sutton
0491ca2fbd ensure we forward whether to check connection details 2025-08-14 21:55:06 -05:00
dan sutton
ce5e2d7a0a remove references to workspaces in api keys 2025-08-14 17:08:22 -05:00
dan sutton
942000bd8d delete workspace related files 2025-08-14 16:46:10 -05:00
dan sutton
d520489b0b workspace related stuff 2025-08-14 16:44:30 -05:00
dan sutton
e83c0944c0 more db routing stuff 2025-08-14 16:43:26 -05:00
dan sutton
85dd11f187 workspace creation
```
workspaces=> (binding [api/*current-user-id* 1]
               (database-routing/route-database 4
                                                [{:name "workspace_01" :details {:ssl false,
                                                                                 :password "secure_password",
                                                                                 :destination-database false,
                                                                                 :port nil,
                                                                                 :advanced-options false,
                                                                                 :schema-filters-type "all",
                                                                                 :use-auth-provider false,
                                                                                 :dbname "flight_data",
                                                                                 :host nil,
                                                                                 :tunnel-enabled false,
                                                                                 :user "workspace_user"}}]
                                                {:check-connection-details? true}))
[{:uploads_schema_name nil,
  :description nil,
  :features #{:describe-fks
              :database-routing
              :full-join
              :window-functions/cumulative
              :test/dynamic-dataset-loading
              :expressions/float
              :describe-fields
              :basic-aggregations
              :temporal-extract
              :identifiers-with-spaces
              :actions/custom
              :window-functions/offset
              :now
              :parameterized-sql
              :convert-timezone
              :expression-literals
              :nested-field-columns
              :expressions/today
              :standard-deviation-aggregations
              :test/jvm-timezone-setting
              :date-arithmetics
              :persist-models
              :expressions/integer
              :expressions/date
              :actions
              :expression-aggregations
              :percentile-aggregations
              :connection-impersonation
              :saved-question-sandboxing
              :distinct-where
              :table-privileges
              :right-join
              :left-join
              :expressions/datetime
              :expressions/text
              :native-parameters
              :uuid-type
              :schemas
              :nested-queries
              :test/uuids-in-create-table-statements
              :describe-indexes
              :expressions
              :uploads
              :set-timezone
              :regex
              :case-sensitivity-string-filter-options
              :binning
              :metadata/key-constraints
              :actions/data-editing
              :database-replication
              :transforms/table
              :transforms/view
              :datetime-diff
              :upload-with-auto-pk
              :inner-join
              :advanced-math-expressions
              :native-temporal-units
              :fingerprint
              :native-parameter-card-reference
              :split-part},
  :uploads_table_prefix nil,
  :cache_field_values_schedule "0 0 12 * * ? *",
  :router_database_id 4,
  :timezone nil,
  :is_attached_dwh false,
  :auto_run_queries true,
  :metadata_sync_schedule "0 43 * * * ? *",
  :name "workspace_01",
  :settings nil,
  :caveats nil,
  :creator_id 1,
  :is_full_sync false,
  :updated_at #object[java.time.OffsetDateTime
                      "0x7c64a1d"
                      "2025-08-06T23:33:53.726200Z"],
  :cache_ttl nil,
  :details {:ssl false,
            :password "secure_password",
            :destination-database false,
            :port nil,
            :advanced-options false,
            :schema-filters-type "all",
            :use-auth-provider false,
            :dbname "flight_data",
            :host nil,
            :tunnel-enabled false,
            :user "workspace_user"},
  :is_sample false,
  :id 7,
  :is_on_demand false,
  :engine :postgres,
  :initial_sync_status "incomplete",
  :is_audit false,
  :dbms_version nil,
  :uploads_enabled false,
  :refingerprint nil,
  :created_at #object[java.time.OffsetDateTime
                      "0x11e896b9"
                      "2025-08-06T23:33:53.726200Z"],
  :points_of_interest nil}]
```
2025-08-14 16:21:22 -05:00
2897 changed files with 24717 additions and 111008 deletions

View File

@@ -49,7 +49,7 @@
;; disabled linters
;;
:redundant-ignore {:level :off} ; lots of false positives, see https://github.com/clj-kondo/clj-kondo/issues/2414
:redundant-ignore {:level :off} ; lots of false positives, see https://github.com/clj-kondo/clj-kondo/issues/2414
:unexpected-recur {:level :off} ; TODO (cam): I think we just need to tell it how to handle MBQL match and we can enable this?
;;
@@ -173,6 +173,7 @@
metabase.driver.sql-jdbc.execute/execute-prepared-statement!
metabase.notification.models/create-notification!
metabase.pulse.send/send-pulse!
metabase.query-processor.store/store-database!
;; Safe in tests, where we typically use a private, temporary index per thread.
metabase.search.core/reindex!
metabase.util/run-count!
@@ -214,7 +215,7 @@
;; misc config for built-in linters
;;
:invalid-arity {:skip-args [metabase.lib.util.match/match]} ; TODO (cam): can we fix these?
:invalid-arity {:skip-args [metabase.lib.util.match/match]} ; TODO (cam): can we fix these?
:refer-all {:level :warning, :exclude [clojure.test]}
:unused-referred-var {:exclude {compojure.core [GET DELETE POST PUT]}}
@@ -249,15 +250,16 @@
(taoensso.nippy/extend-freeze)
(taoensso.nippy/extend-thaw)]}
;; TODO (Cam 8/22/25) -- what's the point of marking stuff deprecated if we don't have Kondo enforce it? We should
;; remove all of these and add individual ignores as needed.
:deprecated-var
{:exclude
{metabase.driver.sql-jdbc.execute.old-impl/set-timezone-sql {:namespaces ["metabase\\.driver.*"]}
metabase.driver.sql-jdbc.execute/set-time-zone-if-supported! {:namespaces ["metabase\\.driver.*"]}
metabase.driver/current-db-time {:namespaces ["metabase.*"]}
metabase.driver/supports? {:namespaces ["metabase.*"]}
metabase.query-processor.util/normalize-token {:namespaces ["metabase.*"]}}}
{metabase.cmd.core/dump {:namespaces ["metabase\\.cmd-test" "metabase-enterprise\\.serialization\\.cmd-test"]}
metabase.cmd.core/load {:namespaces ["metabase\\.cmd-test" "metabase-enterprise\\.serialization\\.cmd-test"]}
metabase.driver.sql-jdbc.execute.old-impl/set-timezone-sql {:namespaces ["metabase\\.driver.*"]}
metabase.driver.sql-jdbc.execute/set-time-zone-if-supported! {:namespaces ["metabase\\.driver.*"]}
metabase.driver/current-db-time {:namespaces ["metabase.*"]}
metabase.driver/supports? {:namespaces ["metabase.*"]}
metabase.query-processor.middleware.resolve-joins/maybe-resolve-source-table {:namespaces ["metabase.*"]}
metabase.query-processor.util/normalize-token {:namespaces ["metabase.*"]}}}
:discouraged-var
{clojure.core/print {:message "Use metabase.util.log instead of clojure.core/print"}
@@ -290,8 +292,6 @@
me.flowthing.pp/pprint {:message "Use metabase.util.log instead of me.flowthing.pp/pprint"}
medley.core/random-uuid {:message "Use clojure.core/random-uuid instead of medley.core/random-uuid"}
metabase.driver/database-supports? {:message "Use metabase.driver.util/supports? instead of metabase.driver/database-supports?"}
metabase.lib.convert/->legacy-MBQL {:message "Conversion to legacy MBQL is discouraged, please use Lib to manipulate MBQL queries"}
metabase.lib.core/->legacy-MBQL {:message "Conversion to legacy MBQL is discouraged, please use Lib to manipulate MBQL queries"}
metabase.lib.equality/find-column-indexes-for-refs {:message "Use lib.equality/closest-matches-in-metadata or lib.equality/find-closest-matches-for-refs instead of lib.equality/find-column-indexes-for-refs"}
next.jdbc/active-tx? {:message "Use metabase.app-db.core/in-transaction? to see if we are in a transaction"}
toucan2.core/debug {:message "Don't commit code using t2/debug"}
@@ -300,7 +300,6 @@
:discouraged-namespace
{camel-snake-kebab.core {:message "CSK is not Turkish-safe, use the versions in metabase.util instead."}
cheshire.core {:message "Use metabase.util.json instead of cheshire.core directly."}
clj-time.core {:message "Use java-time.api or metabase.util.date-2 instead of clj-time"}
clojure.tools.logging {:message "Use metabase.util.log instead of clojure.tools.logging directly"}
grouper.core {:message "Use metabase.util.grouper instead of grouper.core"}
honeysql.core {:message "Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}
@@ -409,16 +408,11 @@
clojure.test.check.clojure-test/defspec clojure.test/deftest
clojure.test.check.properties/for-all clojure.core/let
clojurewerkz.quartzite.jobs/defjob clojure.core/defn
com.climate.claypoole/with-shutdown! clojure.core/let
instaparse.core/defparser clojure.core/def
metabase-enterprise.action-v2.test-util/with-test-tables! clojure.core/let
metabase-enterprise.content-translation.routes-test/with-new-secret-key! clojure.core/fn
metabase-enterprise.serialization.test-util/with-dbs clojure.core/fn
metabase-enterprise.serialization.test-util/with-random-dump-dir clojure.core/let
metabase-enterprise.transforms.api-test/with-transform-tags clojure.core/let
metabase-enterprise.transforms.api-test/with-worker-runs clojure.core/let
metabase-enterprise.transforms.test-util/with-transform-cleanup! clojure.core/let
metabase-enterprise.transforms.execute/with-transform-lifecycle clojure.core/let
metabase-enterprise.content-translation.routes-test/with-new-secret-key! clojure.core/fn
metabase.actions.test-util/with-actions clojure.core/let
metabase.api.common/let-404 clojure.core/let
metabase.app-db.custom-migrations.util/with-temp-scheduler! clojure.core/fn
@@ -518,7 +512,6 @@
metabase-enterprise.advanced-permissions.models.permissions.application-permissions-test/with-new-group-and-current-graph hooks.common/with-two-top-level-bindings
metabase-enterprise.audit-app.pages-test/with-temp-objects hooks.common/with-one-binding
metabase-enterprise.cache.config-test/with-temp-persist-models hooks.common/with-seven-bindings
metabase-enterprise.semantic-search.db.connection/with-migrate-tx hooks.common/let-one-with-optional-value
metabase-enterprise.serialization.test-util/with-temp-dpc hooks.toucan2.tools.with-temp/with-temp
metabase.api.common.internal-test/with-jetty-server hooks.common/with-one-binding
metabase.api.macros/defendpoint hooks.metabase.api.macros/defendpoint
@@ -650,7 +643,6 @@
:macroexpand
{clojurewerkz.quartzite.jobs/build macros.quartz/build-job
clojurewerkz.quartzite.schedule.calendar-interval/build macros.quartz/build-job
clojurewerkz.quartzite.schedule.cron/schedule macros.quartz/schedule
clojurewerkz.quartzite.schedule.simple/schedule macros.quartz/simple-schedule
clojurewerkz.quartzite.triggers/build macros.quartz/build-trigger
@@ -725,19 +717,12 @@
{:pattern "metabase\\.driver.*$"
:name driver-namespaces}
;;
;; Perf-sensitive namespaces are currently the union of metabase.lib.*, driver namespaces, and QP (without tests).
;;
{:pattern "^(?!.*-test(?:\\..*)?$)(?!.*test-util)(?!^metabase\\.test.*)^metabase\\.(lib|driver|legacy-mbql|query-processor).*"
:name performance-sensitive}
;;
;; general QP tests are ones that run across multiple drivers. Anything `metabase.query-processor*-test`,
;; `metabase.driver.sql*-test` (excluding SQLServer and SQLite), or
;; `metabase-enterprise.sandbox.query-processor*-test`
;;
{:pattern "^metabase(?:((?:-enterprise\\.sandbox)?\\.query-processor)|(?:\\.driver\\.sql(?!server)(?!ite))).*-test$"
:name general-qp-and-driver-test-namespaces}
{:pattern "^metabase\\.python-runner\\..*"
:name python-runner-namespaces}]
:name general-qp-and-driver-test-namespaces}]
:config-in-ns
{test-namespaces
@@ -745,13 +730,9 @@
{:inline-def {:level :off}
:missing-docstring {:level :off}
:private-call {:level :off}
:discouraged-var {metabase.driver/database-supports? {:level :off}}
:metabase/defsetting-must-specify-export {:level :off}
:metabase/defsetting-namespace {:level :off}
:discouraged-var
{metabase.driver/database-supports? {:level :off}
metabase.lib.convert/->legacy-MBQL {:level :off}
metabase.lib.core/->legacy-MBQL {:level :off}}}
:metabase/defsetting-namespace {:level :off}}
:hooks
{:analyze-call
@@ -764,13 +745,9 @@
{:metabase/validate-escaped-single-quotes-in-i18n {:level :warning}
:metabase/validate-string-or-str-args-to-i18n {:level :warning}
:metabase/discourage-millis-duration {:level :warning}
:discouraged-var
{clojure.core/with-redefs {:message "Don't use with-redefs outside of tests"}
clojure.core/eval {:message "Don't use eval outside of tests"}}
:discouraged-namespace
{metabase.query-processor.store {:message "Please do not use the QP Store outside of legacy QP or driver code. Use MBQL 5/Lib or lib.metadata.jvm/applicaton-database-metadata-provider instead."}}}}
clojure.core/eval {:message "Don't use eval outside of tests"}}}}
printable-namespaces
{:linters
@@ -795,11 +772,9 @@
;; eventually we should do this for the whole codebase, but the args are in the opposite order so switching them
;; all at once isn't trivial, at least we can stop using it in new code.
:discouraged-var
{medley.core/map-keys {:message "Use metabase.util.performance/update-keys"}
medley.core/map-vals {:message "Use clojure.core/update-vals"}
metabase.lib.convert/->legacy-MBQL {:level :off}
metabase.lib.core/->legacy-MBQL {:level :off}
metabase.test/with-temp {:message "Don't use application database inside Lib code"}}
{medley.core/map-keys {:message "Use clojure.core/update-keys"}
medley.core/map-vals {:message "Use clojure.core/update-vals"}
metabase.test/with-temp {:message "Don't use application database inside MLv2 code"}}
:discouraged-namespace
{metabase.driver {:message "MLv2 code should be independent of driver code/methods"}
@@ -821,36 +796,26 @@
{:discouraged-var
{metabase.driver/database-supports? {:level :off}}}}
performance-sensitive
{:linters
{:discouraged-var
{clojure.core/update-keys {:message "Use metabase.util.performance/update-keys"}}}}
qp-and-driver-source-namespaces
{:linters
{:discouraged-namespace
{metabase.query-processor.store {:level :off}
metabase.util.malli.schema {:message "Prefer metabase.lib.schema or metabase.legacy-mbql.schema schemas in QP/driver code"}
toucan.db {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
toucan.models {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
toucan.util.test {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
toucan2.core {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}}
{metabase.util.malli.schema {:message "Prefer metabase.lib.schema or metabase.legacy-mbql.schema schemas in QP/driver code"}
toucan.db {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
toucan.models {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
toucan.util.test {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
toucan2.core {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}}
:discouraged-var
{medley.core/map-keys {:message "Use metabase.util.performance/update-keys"}
{medley.core/map-keys {:message "Use clojure.core/update-keys"}
medley.core/map-vals {:message "Use clojure.core/update-vals"}
metabase.app-db.core/query {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
metabase.app-db.core/reducible-query {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
metabase.lib.convert/->legacy-MBQL {:level :off}
metabase.lib.core/->legacy-MBQL {:level :off}}}}
metabase.app-db.core/reducible-query {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}}}}
general-qp-and-driver-test-namespaces
{:linters
{:discouraged-var
{metabase.test/with-column-remappings {:message "Use lib.tu/remap-metadata-provider instead of with-column-remappings in QP tests"}
metabase.test.util/with-column-remappings {:message "Use lib.tu/remap-metadata-provider instead of with-column-remappings in QP tests"}
metabase.test/with-temp {:message "Use lib.tu/mock-metadata-provider in QP tests. Exceptions: tests for code that doesn't use MPs or models not supported by MPs"}
toucan2.tools.with-temp/with-temp {:message "Use lib.tu/mock-metadata-provider in QP tests. Exceptions: tests for code that doesn't use MPs or models not supported by MPs"}}
{metabase.test/with-column-remappings {:message "Use metabase.lib.test-util/remap-metadata-provider instead of with-column-remappings in QP tests"}
metabase.test.util/with-column-remappings {:message "Use metabase.lib.test-util/remap-metadata-provider instead of with-column-remappings in QP tests"}}
:metabase/disallow-hardcoded-driver-names-in-tests
{:level :warning}}}}}

View File

@@ -68,21 +68,17 @@
query-processor
search
settings
util
warehouses}}
util}}
activity-feed
{:team "UX West"
:api #{metabase.activity-feed.api
metabase.activity-feed.core
metabase.activity-feed.init}
:api #{metabase.activity-feed.api metabase.activity-feed.init}
:uses #{api
app-db
collections
config
events
models
premium-features
util}}
analytics
@@ -127,17 +123,17 @@
;; TODO -- consolidate these into a `.core` API namespace.
api
{:team "DevEx"
:api #{metabase.api.common
metabase.api.common.internal
metabase.api.docs
metabase.api.init
metabase.api.macros
metabase.api.open-api
metabase.api.response
metabase.api.routes.common
metabase.api.settings
metabase.api.util
metabase.api.util.handlers}
:api #{metabase.api.common
metabase.api.common.internal
metabase.api.docs
metabase.api.init
metabase.api.macros
metabase.api.open-api
metabase.api.response
metabase.api.routes.common
metabase.api.settings
metabase.api.util
metabase.api.util.handlers}
:uses #{config
events
models
@@ -147,8 +143,7 @@
api-keys
{:team "UX West"
:api #{metabase.api-keys.api
metabase.api-keys.core
metabase.api-keys.schema}
metabase.api-keys.core}
:uses #{api
app-db
events
@@ -213,8 +208,6 @@
:api #{metabase.bookmarks.api}
:uses #{api
app-db
config
premium-features
queries
util}}
@@ -303,7 +296,6 @@
models
premium-features
settings
store-api
task
util}}
@@ -430,37 +422,37 @@
driver
{:team "Drivers"
:api #{metabase.driver
metabase.driver.common
metabase.driver.common.parameters
metabase.driver.common.parameters.dates
metabase.driver.common.parameters.operators
metabase.driver.common.parameters.parse
metabase.driver.common.parameters.values
metabase.driver.ddl.interface
metabase.driver.h2
metabase.driver.impl
metabase.driver.init
metabase.driver.mysql
metabase.driver.postgres
metabase.driver.settings
metabase.driver.sql
metabase.driver.sql-jdbc.connection
metabase.driver.sql-jdbc.execute
metabase.driver.sql-jdbc.execute.diagnostic
metabase.driver.sql-jdbc.sync
metabase.driver.sql.query-processor
metabase.driver.sql.query-processor.deprecated
metabase.driver.sql.util
metabase.driver.sync
metabase.driver.util}
:api #{metabase.driver
metabase.driver.common
metabase.driver.common.parameters
metabase.driver.common.parameters.dates
metabase.driver.common.parameters.operators
metabase.driver.common.parameters.parse
metabase.driver.common.parameters.values
metabase.driver.ddl.interface
metabase.driver.h2
metabase.driver.impl
metabase.driver.init
metabase.driver.mysql
metabase.driver.postgres
metabase.driver.settings
metabase.driver.sql
metabase.driver.sql-jdbc.connection
metabase.driver.sql-jdbc.execute
metabase.driver.sql-jdbc.execute.diagnostic
metabase.driver.sql-jdbc.sync
metabase.driver.sql.query-processor
metabase.driver.sql.query-processor.deprecated
metabase.driver.sql.util
metabase.driver.sync
metabase.driver.util}
:uses #{driver-api
util}}
driver-api
{:team "Drivers"
:api #{metabase.driver-api.core
metabase.driver-api.init}
:api #{metabase.driver-api.core
metabase.driver-api.init}
:uses #{actions
api
app-db
@@ -516,7 +508,6 @@
public-sharing
queries
query-processor
server
settings
tiles
util}}
@@ -565,16 +556,17 @@
internal-stats
{:team "UX West"
:api #{metabase.internal-stats.core}
:api #{metabase.internal-stats.core}
:uses #{app-db
models}}
legacy-mbql
{:team "Querying"
:api #{metabase.legacy-mbql.normalize
metabase.legacy-mbql.predicates
metabase.legacy-mbql.schema
metabase.legacy-mbql.util}
:api #{metabase.legacy-mbql.normalize
metabase.legacy-mbql.predicates
metabase.legacy-mbql.schema
metabase.legacy-mbql.schema.helpers
metabase.legacy-mbql.util}
:uses #{lib
models
types
@@ -582,54 +574,50 @@
lib
{:team "Querying"
:api #{metabase.lib.binning
metabase.lib.binning.util
metabase.lib.card
metabase.lib.core
metabase.lib.equality
metabase.lib.field
metabase.lib.field.resolution
metabase.lib.filter
metabase.lib.filter.desugar.jvm ; only so [[metabase.legacy-mbql.jvm-util]] can use defined regexes
metabase.lib.init
metabase.lib.join.util
metabase.lib.metadata
metabase.lib.metadata.cached-provider
metabase.lib.metadata.invocation-tracker
metabase.lib.metadata.protocols
metabase.lib.metadata.result-metadata
metabase.lib.normalize
metabase.lib.options
metabase.lib.parse
metabase.lib.query
metabase.lib.schema
metabase.lib.schema.actions
metabase.lib.schema.binning
metabase.lib.schema.common
metabase.lib.schema.constraints
metabase.lib.schema.expression
metabase.lib.schema.expression.temporal
metabase.lib.schema.expression.window
metabase.lib.schema.id
metabase.lib.schema.info
metabase.lib.schema.join
metabase.lib.schema.literal
metabase.lib.schema.mbql-clause
metabase.lib.schema.metadata
metabase.lib.schema.metadata.fingerprint
metabase.lib.schema.middleware-options
metabase.lib.schema.order-by
metabase.lib.schema.parameter
metabase.lib.schema.ref
metabase.lib.schema.settings
metabase.lib.schema.template-tag
metabase.lib.schema.temporal-bucketing
metabase.lib.schema.util
metabase.lib.temporal-bucket
metabase.lib.types.isa
metabase.lib.util
metabase.lib.util.match
metabase.lib.walk}
:api #{metabase.lib.binning
metabase.lib.binning.util
metabase.lib.card
metabase.lib.core
metabase.lib.equality
metabase.lib.field
metabase.lib.field.resolution
metabase.lib.filter
metabase.lib.init
metabase.lib.join.util
metabase.lib.metadata
metabase.lib.metadata.cached-provider
metabase.lib.metadata.invocation-tracker
metabase.lib.metadata.protocols
metabase.lib.metadata.result-metadata
metabase.lib.normalize
metabase.lib.options
metabase.lib.parse
metabase.lib.query
metabase.lib.schema
metabase.lib.schema.actions
metabase.lib.schema.binning
metabase.lib.schema.common
metabase.lib.schema.expression
metabase.lib.schema.expression.temporal
metabase.lib.schema.expression.window
metabase.lib.schema.id
metabase.lib.schema.info
metabase.lib.schema.join
metabase.lib.schema.literal
metabase.lib.schema.mbql-clause
metabase.lib.schema.metadata
metabase.lib.schema.metadata.fingerprint
metabase.lib.schema.order-by
metabase.lib.schema.parameter
metabase.lib.schema.ref
metabase.lib.schema.template-tag
metabase.lib.schema.temporal-bucketing
metabase.lib.schema.util
metabase.lib.temporal-bucket
metabase.lib.types.isa
metabase.lib.util
metabase.lib.util.match
metabase.lib.walk}
:uses #{config
legacy-mbql
types
@@ -637,9 +625,9 @@
lib-be
{:team "Querying"
:api #{metabase.lib-be.core
metabase.lib-be.init
metabase.lib-be.metadata.jvm}
:api #{metabase.lib-be.core
metabase.lib-be.init
metabase.lib-be.metadata.jvm}
:uses #{lib
models
settings
@@ -659,9 +647,9 @@
login-history
{:team "UX West"
:api #{metabase.login-history.api
metabase.login-history.core
metabase.login-history.init}
:api #{metabase.login-history.api
metabase.login-history.core
metabase.login-history.init}
:uses #{analytics
api
channel
@@ -671,9 +659,9 @@
model-persistence
{:team "Querying"
:api #{metabase.model-persistence.api
metabase.model-persistence.core
metabase.model-persistence.init}
:api #{metabase.model-persistence.api
metabase.model-persistence.core
metabase.model-persistence.init}
:uses #{api
app-db
driver
@@ -693,14 +681,14 @@
models
{:team "DevEx"
:api #{metabase.models.dispatch
metabase.models.humanization
metabase.models.init
metabase.models.interface
metabase.models.resolution
metabase.models.serialization
metabase.models.util.spec-update
metabase.models.visualization-settings}
:api #{metabase.models.dispatch
metabase.models.humanization
metabase.models.init
metabase.models.interface
metabase.models.resolution
metabase.models.serialization
metabase.models.util.spec-update
metabase.models.visualization-settings}
:uses #{api
app-db
classloader
@@ -716,7 +704,6 @@
metabase.native-query-snippets.core}
:uses #{api
collections
lib
models
permissions
premium-features
@@ -788,7 +775,6 @@
collections
config
events
lib
models
premium-features
request
@@ -798,7 +784,7 @@
pivot
{:team "UX West"
:api #{metabase.pivot.core}
:api #{metabase.pivot.core}
:uses #{models
util}}
@@ -885,26 +871,13 @@
enterprise/advanced-permissions
enterprise/sandbox}}
python-runner
{:team "Workflows"
:api #{metabase.python-runner.api
metabase.python-runner.core
metabase.python-runner.init}
:uses #{api
driver
models
query-processor
request
util
enterprise/transforms}}
queries
{:team "Querying"
:api #{metabase.queries.api
metabase.queries.core
metabase.queries.init
metabase.queries.models.query ; TODO FIXME
metabase.queries.schema} ; TODO FIXME
metabase.queries.schema} ; TODO FIXME
:uses #{analytics
analyze
api
@@ -951,42 +924,41 @@
query-processor
{:team "Querying"
:api #{metabase.query-processor
metabase.query-processor.api
metabase.query-processor.card
metabase.query-processor.compile
metabase.query-processor.dashboard
metabase.query-processor.debug
metabase.query-processor.error-type
metabase.query-processor.init
metabase.query-processor.interface
metabase.query-processor.metadata
metabase.query-processor.middleware.annotate
metabase.query-processor.middleware.cache-backend.db
metabase.query-processor.middleware.constraints
metabase.query-processor.middleware.fetch-source-query-legacy
metabase.query-processor.middleware.limit
metabase.query-processor.middleware.permissions
metabase.query-processor.middleware.wrap-value-literals
metabase.query-processor.parameters.operators
metabase.query-processor.pipeline
metabase.query-processor.pivot
metabase.query-processor.preprocess
metabase.query-processor.reducible
metabase.query-processor.schema
metabase.query-processor.setup
metabase.query-processor.store
metabase.query-processor.streaming
metabase.query-processor.streaming.common
metabase.query-processor.streaming.interface
metabase.query-processor.timezone
metabase.query-processor.util
metabase.query-processor.util.add-alias-info
metabase.query-processor.util.nest-query
metabase.query-processor.util.persisted-cache
metabase.query-processor.util.relative-datetime
metabase.query-processor.util.transformations.nest-breakouts
metabase.query-processor.writeback}
:api #{metabase.query-processor
metabase.query-processor.api
metabase.query-processor.card
metabase.query-processor.compile
metabase.query-processor.dashboard
metabase.query-processor.debug
metabase.query-processor.error-type
metabase.query-processor.init
metabase.query-processor.interface
metabase.query-processor.metadata
metabase.query-processor.middleware.annotate
metabase.query-processor.middleware.cache-backend.db
metabase.query-processor.middleware.constraints
metabase.query-processor.middleware.fetch-source-query-legacy
metabase.query-processor.middleware.limit
metabase.query-processor.middleware.permissions
metabase.query-processor.middleware.wrap-value-literals
metabase.query-processor.pipeline
metabase.query-processor.pivot
metabase.query-processor.preprocess
metabase.query-processor.reducible
metabase.query-processor.schema
metabase.query-processor.setup
metabase.query-processor.store
metabase.query-processor.streaming
metabase.query-processor.streaming.common
metabase.query-processor.streaming.interface
metabase.query-processor.timezone
metabase.query-processor.util
metabase.query-processor.util.add-alias-info
metabase.query-processor.util.nest-query
metabase.query-processor.util.persisted-cache
metabase.query-processor.util.relative-datetime
metabase.query-processor.util.transformations.nest-breakouts
metabase.query-processor.writeback}
:uses #{analytics
analyze
api
@@ -1021,11 +993,10 @@
request
{:team "UX West"
:api #{metabase.request.core
metabase.request.init
metabase.request.schema}
metabase.request.init}
:uses #{api
config
permissions ; FIXME
permissions ; FIXME
session
settings
users
@@ -1056,16 +1027,14 @@
search
{:team "UX West"
:api #{metabase.search.api
metabase.search.appdb.scoring
metabase.search.config
metabase.search.core
metabase.search.engine
metabase.search.in-place.scoring
metabase.search.ingestion
metabase.search.init
metabase.search.scoring
metabase.search.spec}
:api #{metabase.search.api
metabase.search.appdb.scoring
metabase.search.config
metabase.search.core
metabase.search.in-place.scoring
metabase.search.ingestion
metabase.search.init
metabase.search.spec}
:uses #{analytics
api
app-db
@@ -1107,11 +1076,10 @@
server
{:team "UX West"
:api #{metabase.server.core
metabase.server.init
metabase.server.middleware.session
metabase.server.settings
metabase.server.streaming-response} ; TODO -- too many API namespaces
:api #{metabase.server.core
metabase.server.init
metabase.server.middleware.session
metabase.server.streaming-response} ; TODO -- too many API namespaces
:uses #{analytics
api
api-keys
@@ -1136,7 +1104,7 @@
metabase.session.core
metabase.session.init
metabase.session.models.session ; TODO -- shouldn't be exposed as an API namespace
metabase.session.settings} ; TODO -- shouldn't be exposed as an API namespace
metabase.session.settings} ; TODO -- shouldn't be exposed as an API namespace
:uses #{api
app-db
channel
@@ -1205,22 +1173,14 @@
:api #{metabase.startup.core}
:uses #{util}}
store-api
{:team "Cloud"
:api #{metabase.store-api.core
metabase.store-api.init}
:uses #{config
settings
util}}
sync
{:team "Querying"
:api #{metabase.sync.api
metabase.sync.core
metabase.sync.init
metabase.sync.schedules
metabase.sync.task.sync-databases
metabase.sync.util}
:api #{metabase.sync.api
metabase.sync.core
metabase.sync.init
metabase.sync.schedules
metabase.sync.task.sync-databases
metabase.sync.util}
:uses #{analyze
api
app-db
@@ -1246,8 +1206,8 @@
task
{:team "UX West"
:api #{metabase.task.bootstrap
metabase.task.core}
:api #{metabase.task.bootstrap
metabase.task.core}
:uses #{app-db
classloader
util}}
@@ -1521,7 +1481,6 @@
metabase-enterprise.advanced-permissions.common
metabase-enterprise.advanced-permissions.models.permissions.group-manager}
:uses #{api
legacy-mbql
lib
permissions
premium-features
@@ -1548,7 +1507,7 @@
enterprise/metabot-v3}}
enterprise/ai-sql-generation
{:team "Metabot"
{:team "Metabot"
:api #{metabase-enterprise.ai-sql-generation.api}
:uses #{api
driver
@@ -1563,7 +1522,6 @@
premium-features
enterprise/advanced-config
enterprise/scim
enterprise/semantic-search
enterprise/sso}}
enterprise/api
@@ -1580,7 +1538,6 @@
app-db
audit-app
classloader
config
driver
lib
models
@@ -1616,15 +1573,6 @@
task
util}}
enterprise/cloud-add-ons
{:team "Cloud"
:api #{metabase-enterprise.cloud-add-ons.api}
:uses #{api
events
premium-features
util
enterprise/harbormaster}}
enterprise/content-translation
{:team "UX West"
:api #{metabase-enterprise.content-translation.dictionary
@@ -1668,7 +1616,6 @@
metabase-enterprise.database-replication.init}
:uses #{analytics
api
driver
premium-features
settings
util
@@ -1691,28 +1638,6 @@
warehouse-schema
warehouses}}
enterprise/documents
{:team "UX West"
:api #{metabase-enterprise.documents.api
metabase-enterprise.documents.core
metabase-enterprise.documents.init}
:uses #{activity-feed
api
app-db
batch-processing
collections
events
models
permissions
premium-features
queries
query-permissions
revisions
search
users
util
view-log}}
enterprise/email
{:team "UX West"
:api #{metabase-enterprise.email.api}
@@ -1724,7 +1649,7 @@
util}}
enterprise/gsheets
{:team "Cloud"
{:team "Cloud"
:api #{metabase-enterprise.gsheets.api
metabase-enterprise.gsheets.init}
:uses #{analytics
@@ -1738,7 +1663,7 @@
{:team "Cloud"
:api #{metabase-enterprise.harbormaster.client}
:uses #{api
store-api
cloud-migration
util}}
enterprise/impersonation
@@ -1764,8 +1689,7 @@
enterprise/llm
{:team "Metabot"
:api #{metabase-enterprise.llm.api
metabase-enterprise.llm.init
metabase-enterprise.llm.settings}
metabase-enterprise.llm.init}
:uses #{analytics
analyze
api
@@ -1776,11 +1700,10 @@
enterprise/metabot-v3
{:team "Metabot"
:api #{metabase-enterprise.metabot-v3.api
metabase-enterprise.metabot-v3.client
metabase-enterprise.metabot-v3.core
metabase-enterprise.metabot-v3.init
metabase-enterprise.metabot-v3.tools.api}
:api #{metabase-enterprise.metabot-v3.api
metabase-enterprise.metabot-v3.core
metabase-enterprise.metabot-v3.init
metabase-enterprise.metabot-v3.tools.api}
:uses #{api
app-db
collections
@@ -1801,13 +1724,11 @@
system
users
util
warehouse-schema
warehouses
enterprise/documents}}
warehouse-schema}}
enterprise/permission-debug
{:team "UX West"
:api #{metabase-enterprise.permission-debug.api}
:api #{metabase-enterprise.permission-debug.api}
:uses #{api
lib
models
@@ -1823,8 +1744,8 @@
enterprise/sandbox
{:team "UX West"
:api #{metabase-enterprise.sandbox.api.routes
metabase-enterprise.sandbox.api.util}
:api #{metabase-enterprise.sandbox.api.routes
metabase-enterprise.sandbox.api.util}
:uses #{api
app-db
audit-app
@@ -1867,32 +1788,11 @@
:uses #{premium-features
search}}
enterprise/semantic-search
{:team "Metabot"
:api #{metabase-enterprise.semantic-search.api
metabase-enterprise.semantic-search.core
metabase-enterprise.semantic-search.init}
:uses #{activity-feed
analytics
api
app-db
config
connection-pool
models
permissions
premium-features
search
settings
task
util
enterprise/llm
enterprise/metabot-v3}}
enterprise/serialization
{:team "Workflows"
:api #{metabase-enterprise.serialization.api
metabase-enterprise.serialization.cmd
metabase-enterprise.serialization.v2.backfill-ids}
:api #{metabase-enterprise.serialization.api
metabase-enterprise.serialization.cmd
metabase-enterprise.serialization.v2.backfill-ids}
:uses #{analytics
api
app-db
@@ -1922,9 +1822,9 @@
enterprise/sso
{:team "UX West"
:api #{metabase-enterprise.sso.api.routes
metabase-enterprise.sso.init
metabase-enterprise.sso.settings}
:api #{metabase-enterprise.sso.api.routes
metabase-enterprise.sso.init
metabase-enterprise.sso.settings}
:uses #{api
appearance
channel
@@ -1955,43 +1855,6 @@
settings
util}}
enterprise/transforms
{:team "Querying"
:api #{metabase-enterprise.transforms.api
metabase-enterprise.transforms.core
metabase-enterprise.transforms.init
;; TODO: Remove this from API once we remove the python-runner callback to metabase
metabase-enterprise.transforms.settings}
:uses #{analytics
api
app-db
config
driver
events
lib
lib-be
models
permissions
premium-features
query-processor
request
settings
sync
task
task-history
;; TODO: Remove this once we're using a better file format than CSV
upload
util}}
enterprise/python-transform
{:team "Transform"
:api #{metabase-enterprise.python-transform.api
metabase-enterprise.python-transform.core
metabase-enterprise.python-transform.init}
:uses #{api
models
util}}
enterprise/upload-management
{:team "UX West"
:api #{metabase-enterprise.upload-management.api}
@@ -2011,7 +1874,6 @@
{:ignored-namespace-patterns
#{"-test$" ; anything ending in `-test`
"test[-.]util" ; anything that contains `test.util` or `test-util`
"test[-.]dataset" ; anything that contains `test.dataset` or `test-dataset`
"test\\.impl" ; anything that contains `test.impl`
"test-setup" ; anything that contains `test-setup`
"^metabase(?:-enterprise)?\\.test"}}}} ; anything starting with `metabase.test` or `metabase-enterprise.test`

View File

@@ -52,13 +52,7 @@
(defn- lint-require* [node current-ns config]
(let [[_require & args] (:children node)
required-namespaces (->> (keep unwrap-require args)
;; `require` runs through `cc/load-libs` which allows
;; #{:as :reload :reload-all :require :use :verbose :refer :as-alias}
;; as args. If you use a REPL tool that applies clj-kondo linting (e.g., Clojure MCP)
;; you may wish to `:reload` or `:reload-all` a namespace. So we shouldn't assume all
;; args are keywords:
(remove keyword?))]
required-namespaces (keep unwrap-require args)]
(lint* node current-ns config required-namespaces)))
(defn- lint-requiring-resolve* [node current-ns config]

View File

@@ -14,7 +14,6 @@ if (shouldLintCssModules) {
}
module.exports = {
ignorePatterns: ["!.storybook"],
rules: {
strict: [2, "never"],
"no-undef": 2,
@@ -180,7 +179,6 @@ module.exports = {
{
files: ["*.js", "*.jsx", "*.ts", "*.tsx"],
rules: {
"jtag-missing-key": "error",
"no-unconditional-metabase-links-render": "error",
"no-color-literals": "error",
"no-literal-metabase-strings": "error",
@@ -205,7 +203,6 @@ module.exports = {
"enterprise/frontend/src/metabase-enterprise/whitelabel/**/*",
"frontend/lint/**/*",
"*.stories.*",
"**/.storybook/*",
"stories-data.*",
"e2e/**/*",
"**/tests/*",

4
.github/CODEOWNERS vendored
View File

@@ -1,9 +1,7 @@
# Before changing this file, make sure you have read how we use it:
# https://www.notion.so/metabase/Pull-Request-Code-Review-Guide-374f05df889748b69d67af753f9bc9b8?pvs=4#36550c04ceab40b5a8b8bee6264b090c
enterprise/frontend/src/embedding-sdk-package @metabase/embedding
enterprise/frontend/src/embedding-sdk-bundle @metabase/embedding
enterprise/frontend/src/embedding-sdk-shared @metabase/embedding
enterprise/frontend/src/embedding-sdk @metabase/embedding
frontend/src/metabase/embedding-sdk @metabase/embedding
frontend/src/metabase/public/containers/PublicOrEmbeddedQuestion @metabase/embedding
frontend/src/metabase/public/containers/PublicOrEmbeddedDashboard @metabase/embedding

View File

@@ -11,18 +11,6 @@ runs:
- name: Wait for port ${{ inputs.port }} to be ready
shell: bash
run: |
echo "Waiting for port ${{ inputs.port }} to be ready..."
timeout=300 # 5 minutes timeout
elapsed=0
while ! nc -z localhost ${{ inputs.port }}; do
if [ $elapsed -ge $timeout ]; then
echo "❌ Timeout waiting for port ${{ inputs.port }} after ${timeout} seconds"
exit 1
fi
sleep 1
elapsed=$((elapsed + 1))
if [ $((elapsed % 10)) -eq 0 ]; then
echo " Still waiting for port ${{ inputs.port }}... ($elapsed seconds)"
fi
sleep 0.1;
done
echo "✅ Port ${{ inputs.port }} is ready!"

View File

@@ -33,7 +33,7 @@ runs:
});
if (!artifacts.data?.artifacts?.[0]?.id) {
throw new Error(`No artifacts found for ${{ inputs.name }}`);
throw new Error(`No artifacts found for: ${{ inputs.name }}`);
}
const artifact_id = artifacts.data.artifacts[0].id;
@@ -47,51 +47,53 @@ runs:
fs.writeFileSync(`${process.env.GITHUB_WORKSPACE}/mb.zip`, Buffer.from(download.data));
- name: Extract and rename JAR
- name: unzip uberjar artifact
shell: bash
run: unzip mb.zip
- name: Process the Uberjar
shell: bash
run: | # bash
OUTPUT_NAME="${{ inputs.output-name }}"
EXTRACT_PATH="${{ inputs.extract-path }}"
# Create a temporary directory for extraction to avoid conflicts
TEMP_DIR=$(mktemp -d)
# Extract to temporary directory
unzip mb.zip -d "$TEMP_DIR"
# Create target directory
mkdir -p "$EXTRACT_PATH"
# Find the jar and move all files from its directory
if [ -e "$TEMP_DIR/metabase.jar" ]; then
# Move all files to extract path
mv "$TEMP_DIR"/* "$EXTRACT_PATH/" 2>/dev/null || true
# Rename the jar to custom name if different
if [ "$OUTPUT_NAME" != "metabase.jar" ]; then
mv "$EXTRACT_PATH/metabase.jar" "$EXTRACT_PATH/$OUTPUT_NAME"
fi
elif [ -e "$TEMP_DIR/target/uberjar/metabase.jar" ]; then
# Move all files from uberjar directory to extract path
mv "$TEMP_DIR/target/uberjar"/* "$EXTRACT_PATH/" 2>/dev/null || true
# Rename the jar to custom name if different
if [ "$OUTPUT_NAME" != "metabase.jar" ]; then
mv "$EXTRACT_PATH/metabase.jar" "$EXTRACT_PATH/$OUTPUT_NAME"
# Function to determine the final JAR path
get_jar_path() {
if [ "$EXTRACT_PATH" == "." ]; then
echo "$OUTPUT_NAME"
else
echo "$EXTRACT_PATH/$OUTPUT_NAME"
fi
}
# Create target directory if it doesn't exist
if [ "$EXTRACT_PATH" != "." ]; then
mkdir -p "$EXTRACT_PATH"
fi
# Find the source jar file
if [ -e "metabase.jar" ]; then
SOURCE_JAR="metabase.jar"
elif [ -e "target/uberjar/metabase.jar" ]; then
SOURCE_JAR="target/uberjar/metabase.jar"
else
echo "Could not find metabase.jar in extracted files"
echo "Could not find the metabase.jar file"
exit 1
fi
# Clean up temporary directory
rm -rf "$TEMP_DIR"
# Get the target path
TARGET_PATH=$(get_jar_path)
# Set the JAR_PATH environment variable
if [ "$EXTRACT_PATH" == "." ]; then
echo "JAR_PATH=$OUTPUT_NAME" >> $GITHUB_ENV
# Move the jar if source and target differ
if [ "$SOURCE_JAR" == "$TARGET_PATH" ]; then
echo "Jar is already at the correct location: $TARGET_PATH"
else
echo "JAR_PATH=$EXTRACT_PATH/$OUTPUT_NAME" >> $GITHUB_ENV
echo "Moving $SOURCE_JAR to $TARGET_PATH"
mv "$SOURCE_JAR" "$TARGET_PATH"
fi
# Export the path for subsequent steps
echo "JAR_PATH=$TARGET_PATH" >> $GITHUB_ENV
- name: Verify that this is a valid JAR file
shell: bash
run: |

View File

@@ -1,7 +1,4 @@
name: Prepare back-end environment
description: |
Prepare the backend. Installs jdk, Clojure, and fetches the cache for m2. The CI config fetchs `ci-test-config.json`
from master and lets us ignore tests at the var level if they are flaking.
inputs:
java-version:
required: true
@@ -24,14 +21,9 @@ runs:
distribution: 'temurin'
- name: Install Clojure CLI
shell: bash
continue-on-error: true
run: |
curl -O https://download.clojure.org/install/linux-install-${{ inputs.clojure-version }}.sh &&
sudo bash ./linux-install-${{ inputs.clojure-version }}.sh
- name: Fetch CI config
shell: bash
run: |
curl -o ci-test-config.json https://raw.githubusercontent.com/metabase/metabase/refs/heads/master/ci-test-config.json
- name: Check to see if dependencies should be cached
if: ${{ contains(github.event.head_commit.message, '[ci nocache]') }}
run: echo "Commit message includes [ci nocache]; dependencies will NOT be cached"

View File

@@ -40,7 +40,7 @@ runs:
list-tests: failed
fail-on-error: false
- name: Upload Logs on Failure
if: always() && !success()
if: failure()
uses: actions/upload-artifact@v4
with:
name: test-logs-${{ github.job }}

View File

@@ -41,17 +41,10 @@ frontend_sources: &frontend_sources
- "enterprise/frontend/!(test)**/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
- "frontend/!(test)**/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
frontend_embedding_sources:
- "enterprise/frontend/src/{embedding,embedding-*}/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
- "enterprise/frontend/src/metabase-enterprise/{embedding,embedding-*}**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
frontend_embedding_sdk_sources:
- *frontend_configs
- "enterprise/frontend/!(test)**/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
frontend_embedding_sdk_package_sources:
- "enterprise/frontend/src/embedding-sdk-package/**/!(*.spec).{js,jsx,ts,tsx,css}"
frontend_specs: &frontend_specs
- *shared_specs
- "frontend/test/**"
@@ -145,17 +138,16 @@ e2e_embedding_sdk_compatibility:
- "e2e/runner/embedding-sdk/**"
- "e2e/test-host-app/**/*.cy.*.{js,ts}"
snowplow: &snowplow
- ".github/workflows/snowplow.yml"
- "snowplow/**"
e2e_all:
- *default
- *e2e_ci
- *sources
- *e2e_specs
- "e2e/**"
- *snowplow
snowplow:
- ".github/workflows/snowplow.yml"
- "snowplow/**"
documentation:
- "docs/**"

View File

@@ -1,66 +1,24 @@
const TYPES = new Set([
"feat",
"fix",
"perf",
"docs",
"style",
"refactor",
"test",
"build",
"ci",
]);
/**
* @param title {string}
*/
function isConventionalTitle(title) {
const match = /^(?<type>\w+)\((?<rawScopes>[^)]+)\):/.exec(title);
if (!match) {
return false;
}
const [, type, rawScopes] = match;
const scopes = rawScopes
.split(",")
.map((scope) => scope.trim())
.filter(Boolean);
return TYPES.has(type) && scopes.length > 0;
}
exports.updateComment = updateComment;
/** @param {import('@types/github-script').AsyncFunctionArguments} AsyncFunctionArguments */
async function updateCommentForEmbedding({ github, context }) {
async function updateComment({ github, context }) {
const githubUsername = context.payload.sender.login;
const comment = `${CONVENTIONAL_COMMENT_IDENTIFIER}
@${githubUsername} You have modified embedding code. Please make sure the PR title follows [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) style.
Here are all the supported types:
const comment = `${CONVENTIONAL_COMMIT_REMINDER_COMMENT_IDENTIFIER}
@${githubUsername} You have modified embedding SDK code. Please make sure the PR title follows [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) style.
Here are all the supported types that will be shown in the changelog:
- \`feat\`
- \`fix\`
- \`perf\`
- \`docs\`
- \`style\`
- \`refactor\`
- \`test\`
- \`build\`
- \`ci\`
Please also make sure to include a scope.
These types \`docs\`, \`style\`, \`refactor\`, \`test\`, \`build\`, and \`ci\` are also encouraged for non-changelog related tasks.
Please also make sure to include \`sdk\` scope, otherwise, the changelog will not include this task.
For example, these are valid PR titles:
- \`feat(sdk-bundle): Add interactive dashboard component\`
- \`fix(sdk-package): Fixed MetabaseProviderPropsStore logic\`
- \`fix(sdk-package, embed-js): Fix API\``;
- \`feat(sdk): Add interactive dashboard component\`
- \`feat(sdk): Support theming pie chart\`
- \`fix(sdk): Fix static dashboard crash\``;
return updateComment(comment, { github, context });
}
/**
* @param comment {string}
* @param {import('@types/github-script').AsyncFunctionArguments} AsyncFunctionArguments
*/
async function updateComment(comment, { github, context }) {
const owner = context.repo.owner;
const repo = context.repo.repo;
const pullRequestNumber = context.issue.number;
@@ -91,19 +49,20 @@ async function updateComment(comment, { github, context }) {
}
}
const CONVENTIONAL_COMMENT_IDENTIFIER =
"<!---conventional comment identifier-->";
const CONVENTIONAL_COMMIT_REMINDER_COMMENT_IDENTIFIER =
"<!---conventional commit reminder comment ID-->";
/**
* @param comments {{body: string}[]}
* @typedef Comment
* @property {string} body
*
* @param {Comment[]} comments
*/
function getExistingConventionalCommitReminderComment(comments) {
for (const comment of comments) {
if (comment.body.startsWith(CONVENTIONAL_COMMENT_IDENTIFIER)) {
if (
comment.body.startsWith(CONVENTIONAL_COMMIT_REMINDER_COMMENT_IDENTIFIER)
) {
return comment;
}
}
}
exports.isConventionalTitle = isConventionalTitle;
exports.updateCommentForEmbedding = updateCommentForEmbedding;

View File

@@ -1,45 +0,0 @@
const { isConventionalTitle } = require("./utils");
describe("isConventionalTitle", () => {
it("returns true for a simple conventional commit", () => {
expect(isConventionalTitle("feat: add new feature")).toBe(false);
});
it("returns true for a commit with a scope", () => {
expect(isConventionalTitle("fix(api): correct API bug")).toBe(true);
});
it("returns true for a commit with multiple scopes, comma separated", () => {
expect(isConventionalTitle("feat(build,deps): update dependencies")).toBe(
true,
);
});
it("returns true for a commit with multiple scopes, comma separated with space", () => {
expect(isConventionalTitle("feat(build, deps): update dependencies")).toBe(
true,
);
});
it("returns false for a commit with round brackets in a message", () => {
expect(isConventionalTitle("feat: (something) update dependencies")).toBe(
false,
);
});
it("returns false for missing colon", () => {
expect(isConventionalTitle("feat add feature")).toBe(false);
});
it("returns false for invalid type", () => {
expect(isConventionalTitle("invalidType: some message")).toBe(false);
});
it("returns false for empty message", () => {
expect(isConventionalTitle("feat:")).toBe(false);
});
it("returns false for empty input", () => {
expect(isConventionalTitle("")).toBe(false);
});
});

View File

@@ -1,84 +0,0 @@
const githubSlackMap = require("../../release/github-slack-map.json");
async function getIssueData({ context, github, manualIssueNumber }) {
let issueNumber;
let issueData;
let prReviews = [];
const triggerType = context.eventName;
if (context.eventName === "issues") {
issueNumber = context.payload.issue.number;
issueData = context.payload.issue;
} else if (context.eventName === "workflow_dispatch") {
issueNumber = parseInt(manualIssueNumber);
const { data } = await github.rest.issues.get({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
});
issueData = data;
} else if (context.eventName === "pull_request_review") {
const prNumber = context.payload.pull_request.number;
const { data: prData } = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: prNumber,
});
// Extract issue number from branch name (e.g., "claude-fix/issue-12345-some-title")
const branchMatch = prData.head.ref.match(/claude-fix\/issue-(\d+)/);
if (!branchMatch) {
throw new Error(
`Could not extract issue number from branch name: ${prData.head.ref}`,
);
}
issueNumber = parseInt(branchMatch[1]);
const { data } = await github.rest.issues.get({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
});
issueData = data;
const { data: reviews } = await github.rest.pulls.listReviews({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: prNumber,
});
prReviews = reviews.filter(
(review) =>
review.state === "CHANGES_REQUESTED" &&
isMetabaseMember(review.user.login),
);
}
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
});
const employeeComments = comments.filter((comment) =>
isMetabaseMember(comment.user.login),
);
return {
issueNumber,
issueData,
employeeComments,
prReviews,
triggerType,
isIssueAuthorEmployee: isMetabaseMember(issueData.user.login),
};
}
function isMetabaseMember(username) {
return username in githubSlackMap;
}
module.exports = {
getIssueData,
};

View File

@@ -154,7 +154,7 @@ jobs:
fail-on-error: false
- name: Upload Logs on Failure
if: always() && !success()
if: failure()
uses: actions/upload-artifact@v4
with:
name: test-logs-${{ matrix.java-version }}-${{ matrix.job.edition }}-${{ github.job }}
@@ -195,7 +195,6 @@ jobs:
- name: Run cljfmt
run: ./bin/mage cljfmt-all --force-check
be-tests-result:
needs:
- be-linter-eastwood

View File

@@ -1,14 +0,0 @@
name: Block Master Branch PRs
on:
pull_request:
types: [opened, synchronize]
jobs:
check-source-branch:
runs-on: ubuntu-latest
steps:
- name: Block master branch PRs
if: github.head_ref == 'master'
run: |
echo "PRs from `master` branches are not allowed, please use a new branch name for your PR."
exit 1

View File

@@ -1,327 +0,0 @@
name: "Claude Code Bug Fix"
on:
issues:
types:
- labeled
pull_request_review:
types:
- submitted
workflow_dispatch:
inputs:
issue_number:
description: "Issue number to fix"
required: true
type: string
override_employee_check:
description: "Override employee check for issue creator"
required: false
default: false
type: boolean
jobs:
auto-fix-bug:
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: write
issues: write
pull-requests: write
id-token: write
if: |
(github.event_name == 'issues' && github.event.label.name == 'Claude Bug Fix' && github.event.issue.state == 'open') ||
(github.event_name == 'workflow_dispatch') ||
(github.event_name == 'pull_request_review' && github.event.review.state == 'changes_requested' && startsWith(github.event.pull_request.head.ref, 'claude-fix/'))
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }}
- name: Get issue data and check employee status
id: get_issue
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { getIssueData } = require('${{ github.workspace }}/.github/scripts/get-issue-data.js');
try {
const result = await getIssueData({
context,
github,
manualIssueNumber: '${{ github.event.inputs.issue_number }}'
});
const { issueNumber, issueData, employeeComments, prReviews, triggerType, isIssueAuthorEmployee } = result;
// Check employee status and fail if not authorized
if (!result.isIssueAuthorEmployee && '${{ github.event.inputs.override_employee_check }}' !== 'true') {
core.setFailed(`User ${result.issueData.user.login} is not a Metabase employee. Skipping bug fix automation.`);
}
core.setOutput('issue_number', issueNumber);
core.setOutput('issue_data', JSON.stringify(issueData));
core.setOutput('issue_comments', JSON.stringify(employeeComments));
core.setOutput('pr_reviews', JSON.stringify(prReviews));
core.setOutput('trigger_type', triggerType);
} catch (error) {
core.setFailed(error.message);
}
- name: Set up git configuration
run: |
git config --global user.name "Claude Code[bot]"
git config --global user.email "noreply@anthropic.com"
- name: Prepare issue context and bug fix prompt
id: prepare_prompt
uses: actions/github-script@v7
env:
ISSUE_DATA: ${{ steps.get_issue.outputs.issue_data }}
ISSUE_COMMENTS: ${{ steps.get_issue.outputs.issue_comments }}
PR_REVIEWS: ${{ steps.get_issue.outputs.pr_reviews }}
TRIGGER_TYPE: ${{ steps.get_issue.outputs.trigger_type }}
with:
script: |
const issueData = JSON.parse(process.env.ISSUE_DATA);
const issueComments = JSON.parse(process.env.ISSUE_COMMENTS);
const prReviews = JSON.parse(process.env.PR_REVIEWS);
const triggerType = process.env.TRIGGER_TYPE;
const issueNumber = '${{ steps.get_issue.outputs.issue_number }}';
// Convert issue title to kebab case and create branch name
const issueTitle = issueData.title || 'untitled';
const kebabTitle = issueTitle
.toLowerCase()
.replace(/[^a-z0-9]+/g, '-')
.replace(/^-+|-+$/g, '')
.replace(/-+/g, '-')
.substring(0, 50);
const branchName = `claude-fix/issue-${issueNumber}-${kebabTitle}`;
// Format data for bash environment
const formattedComments = issueComments.length > 0
? issueComments.map(comment =>
`**Comment by @${comment.user.login} on ${comment.created_at}:**\n${comment.body}`
).join('\n\n---\n\n')
: '';
const formattedReviews = prReviews.length > 0
? prReviews.map(review =>
`**Review by @${review.user.login} on ${review.submitted_at} (Changes Requested):**\n${review.body || 'No review comment provided'}`
).join('\n\n---\n\n')
: '';
core.exportVariable('BRANCH_NAME', branchName);
core.exportVariable('ISSUE_NUMBER', issueNumber);
core.exportVariable('ISSUE_TITLE', issueData.title);
core.exportVariable('ISSUE_AUTHOR', issueData.user.login);
core.exportVariable('ISSUE_CREATED', issueData.created_at);
core.exportVariable('ISSUE_BODY', issueData.body || 'No description provided');
core.exportVariable('ISSUE_COMMENTS_FORMATTED', formattedComments);
core.exportVariable('PR_REVIEWS_FORMATTED', formattedReviews);
core.exportVariable('TRIGGER_TYPE', triggerType);
- name: Generate bug fix prompt
run: |
cat << 'EOF' >> $GITHUB_ENV
BUG_FIX_PROMPT<<PROMPT_END
You are helping to fix a bug reported in GitHub issue #${{ env.ISSUE_NUMBER }}.
## Bug Report Details
**Title:** ${{ env.ISSUE_TITLE }}
**Reporter:** ${{ env.ISSUE_AUTHOR }}
**Created:** ${{ env.ISSUE_CREATED }}
**Description:**
${{ env.ISSUE_BODY }}
**Comments:**
${{ env.ISSUE_COMMENTS_FORMATTED }}
EOF
# Add review feedback section only if this is a review trigger
if [ "${{ env.TRIGGER_TYPE }}" = "pull_request_review" ]; then
cat << 'EOF' >> $GITHUB_ENV
## Previous Fix Attempt - Review Feedback (MUST ADDRESS)
Your previous automated fix attempt received the following feedback that needs to be addressed:
${{ env.PR_REVIEWS_FORMATTED }}
EOF
fi
# Add the rest of the prompt (always the same)
cat << 'EOF' >> $GITHUB_ENV
## Instructions
1. **Analyze the issue description and comments** - Understand what the issue is describing
a. Pay particular attention to reproduction steps and if there's an agreed upon fix.
2. **Search the codebase** - Use Grep and Glob tools to find relevant files
3. **Implement a fix** - Follow existing code patterns and conventions
4. **Test the fix** - Run appropriate tests using the commands in CLAUDE.md:
- JavaScript/TypeScript: `yarn test-unit path/to/file.unit.spec.js`
- Clojure: `clojure -X:dev:test :only namespace/test-name`
- Lint: `yarn lint-eslint-pure` for JS/TS, `./bin/mage kondo <file>` for Clojure
5. **Verify the fix** - Ensure tests pass and code follows conventions
## Important Guidelines
- Follow the development patterns in CLAUDE.md
- Write failing tests first, then fix them
- Work in small, testable increments
- Don't commit changes - leave that for the workflow
- IMPORTANT: Focus only on fixing the reported bug, don't make unrelated changes
## Scratchpad Directory
A `.claude-scratchpad/` directory has been created for your temporary files:
- Use this directory for ALL temporary verification files. However, prefer using production test files rather than adhoc ones.
- Only modify actual production code in the main repository directories
- The scratchpad will be cleaned up automatically - nothing from it will be committed
Start by understanding the bug and then systematically work toward a solution.
PROMPT_END
EOF
- name: Determine branch strategy
run: |
BRANCH_NAME="${{ env.BRANCH_NAME }}"
if git branch -r --list "origin/${BRANCH_NAME}" | grep -q .; then
echo "Branch already exists: $BRANCH_NAME"
echo "Will push to existing branch and skip PR creation"
echo "BRANCH_EXISTS=true" >> $GITHUB_ENV
else
echo "Creating new branch: $BRANCH_NAME"
echo "Will create new PR"
echo "BRANCH_EXISTS=false" >> $GITHUB_ENV
fi
# Export the branch name
echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV
- name: Create scratchpad directory for Claude
run: |
# Create a scratchpad directory for Claude to use for temporary files
mkdir -p .claude-scratchpad
echo "Scratchpad directory created for Claude's temporary verification files"
- name: Generate GitHub App Token for Claude
uses: actions/create-github-app-token@v1
id: claude-app-token
with:
app-id: ${{ secrets.METABASE_BOT_APP_ID }}
private-key: ${{ secrets.METABASE_BOT_APP_PRIVATE_KEY }}
- name: Generate Bug Fix with Claude Code
uses: anthropics/claude-code-action@beta
with:
mode: agent
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
github_token: ${{ steps.claude-app-token.outputs.token }}
direct_prompt: ${{ env.BUG_FIX_PROMPT }}
timeout_minutes: "25"
allowed_tools: "Bash(yarn test-unit*),Bash(yarn lint-eslint-pure),Bash(./bin/mage*),Bash(clojure*),Bash(find*),Bash(ls*),Bash(grep*),Write,Read,Glob,Grep,Edit,MultiEdit"
disallowed_tools: "Bash(git commit*),Bash(git push*),Bash(gh*)"
- name: Clean up scratchpad directory
run: |
# Remove the scratchpad directory so its contents don't get committed
rm -rf .claude-scratchpad/
# Remove output file generated by Claude Code action
rm -f output.txt
echo "Scratchpad directory and action output files cleaned up"
- name: Check for changes and analysis
id: check_changes
run: |
# Now check for actual code changes (excluding analysis file)
if [ -n "$(git status --porcelain)" ]; then
echo "has_changes=true" >> $GITHUB_OUTPUT
echo "Changes detected:"
git status --short
else
echo "has_changes=false" >> $GITHUB_OUTPUT
echo "No changes made by Claude."
fi
- name: Get issue team labels and prepare reviewers
if: steps.check_changes.outputs.has_changes == 'true'
id: prepare_reviewers
uses: actions/github-script@v7
env:
ISSUE_DATA: ${{ steps.get_issue.outputs.issue_data }}
with:
script: |
const issueData = JSON.parse(process.env.ISSUE_DATA);
const issueLabels = issueData.labels.map(label => label.name);
const teamLabels = issueLabels.filter(label => label.startsWith('.Team/'));
const issueCreator = issueData.user.login;
core.setOutput('team_labels', teamLabels.join(','));
core.setOutput('reviewers', issueCreator);
- name: Create Pull Request
if: steps.check_changes.outputs.has_changes == 'true'
uses: peter-evans/create-pull-request@v6
with:
token: ${{ secrets.GITHUB_TOKEN }}
branch: ${{ env.BRANCH_NAME }}
base: master
commit-message: |
Fix: ${{ env.ISSUE_TITLE }}
Fixes #${{ env.ISSUE_NUMBER }}
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
committer: "Claude Code[bot] <noreply@anthropic.com>"
author: "Claude Code[bot] <noreply@anthropic.com>"
title: "Fix: ${{ env.ISSUE_TITLE }}"
body: |
This PR automatically fixes the bug reported in #${{ env.ISSUE_NUMBER }}.
## Bug Report
**Original Issue:** #${{ env.ISSUE_NUMBER }}
**Reporter:** @${{ env.ISSUE_AUTHOR }}
**Issue Title:** ${{ env.ISSUE_TITLE }}
## Automated Fix
🤖 This fix was generated automatically by Claude Code based on the issue description.
**Please review carefully:**
- [ ] The fix addresses the reported issue
- [ ] Tests pass and code follows conventions
- [ ] No unrelated changes were introduced
- [ ] The solution is appropriate and maintainable
## Next Steps
- Review the changes and test thoroughly
- If the fix looks good, approve and merge
- If issues are found, either fix them or close this PR
---
Closes #${{ env.ISSUE_NUMBER }}
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
labels: |
${{ steps.prepare_reviewers.outputs.team_labels }}
automated-fix
ci skip
reviewers: ${{ steps.prepare_reviewers.outputs.reviewers }}
- name: Log existing branch update
if: steps.check_changes.outputs.has_changes == 'true' && env.BRANCH_EXISTS == 'true'
uses: actions/github-script@v7
with:
script: |
console.log('🔄 Updated existing branch: ${{ env.BRANCH_NAME }}');
console.log('The changes have been pushed to the existing PR for this issue.');
- name: Log analysis for no changes
if: steps.check_changes.outputs.has_changes == 'false'
uses: actions/github-script@v7
with:
script: |
console.log('🤖 No code changes were made by Claude.');

View File

@@ -4,53 +4,39 @@ on:
pull_request:
branches:
- master
paths:
- enterprise/frontend/src/embedding-sdk/**
jobs:
files-changed:
name: Check which files changed
runs-on: ubuntu-22.04
timeout-minutes: 3
outputs:
frontend_embedding_sources: ${{ steps.changes.outputs.frontend_embedding_sources }}
steps:
- uses: actions/checkout@v4
- name: Test which files changed
uses: dorny/paths-filter@v3.0.0
id: changes
with:
token: ${{ github.token }}
filters: .github/file-paths.yaml
pr-title-reminder:
runs-on: ubuntu-22.04
timeout-minutes: 5
permissions:
pull-requests: write
needs: [ files-changed ]
if: |
!cancelled() &&
needs.files-changed.outputs.frontend_embedding_sources == 'true'
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.head_ref }}
- name: Check if the PR title is already following Conventional Commits style
uses: actions/github-script@v7
id: is-conventional-commit
with:
result-encoding: string
script: | # js
const { isConventionalTitle } = require(".github/scripts/conventional-commit-pr-title-reminder/utils.js")
const pullRequestTitle = context.payload.pull_request.title
const conventionalCommitRegex = /^(?<type>\w+)\((?<scope>\w+)\):/
const match = pullRequestTitle.match(conventionalCommitRegex)
if (match) {
const { type, scope } = match.groups
return ['feat', 'fix', 'perf', 'docs', 'style', 'refactor', 'test', 'build', 'ci'].includes(type) && scope === 'sdk'
}
return isConventionalTitle(pullRequestTitle)
- name: Checkout repository so `gh` CLI works
uses: actions/checkout@v4
with:
ref: ${{ github.head_ref }}
- name: Remind PR authors to use conventional commit in the title
if: ${{ steps.is-conventional-commit.outputs.result != 'true' }}
uses: actions/github-script@v7
with:
script: | #js
const { updateCommentForEmbedding } = require(".github/scripts/conventional-commit-pr-title-reminder/utils.js")
await updateCommentForEmbedding({github, context})
script: |
const { updateComment } = require(".github/scripts/conventional-commit-pr-title-reminder/utils.js")
await updateComment({github, context})

View File

@@ -1,182 +0,0 @@
# .github/workflows/reusable-doc-update.yml
name: Reusable - Update Documentation
# This workflow is designed to be called by other workflows.
on:
workflow_call:
inputs:
target_branch:
description: "The branch to check out and create a PR against"
required: true
type: string
m2_cache_key_suffix:
description: "A unique suffix for the Maven cache key"
required: true
type: string
secrets:
automation_token:
description: "A token with permissions to create branches and PRs"
required: true
permissions:
contents: write
pull-requests: write
jobs:
update-docs:
name: Update documentation for ${{ inputs.target_branch }}
runs-on: ubuntu-22.04
timeout-minutes: 30
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.automation_token }}
ref: ${{ inputs.target_branch }}
- name: Prepare backend environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: "auto-update-docs-${{ inputs.m2_cache_key_suffix }}"
- name: Generate environment variables documentation
id: generate_env_docs
run: |
set -e # Exit on any error
# Ensure we're on the target branch for documentation generation
git checkout ${{ inputs.target_branch }}
echo "Generating environment variables documentation from branch: ${{ inputs.target_branch }}"
clojure -M:ee:doc environment-variables-documentation
continue-on-error: false
- name: Generate config template documentation
id: generate_config_docs
run: |
set -e # Exit on any error
echo "Generating config template documentation..."
clojure -M:ee:doc config-template
continue-on-error: false
- name: Generate API documentation
id: generate_api_docs
run: |
set -e # Exit on any error
echo "Generating API documentation..."
clojure -M:ee:doc api-documentation
continue-on-error: false
- name: Check for changes
id: check_changes
run: |
if git diff --quiet docs/; then
echo "changes=false" >> $GITHUB_OUTPUT
echo "No changes detected in docs directory"
else
echo "changes=true" >> $GITHUB_OUTPUT
echo "Changes detected in docs directory"
fi
- name: Create branch and commit changes
id: commit_and_push
if: steps.check_changes.outputs.changes == 'true'
run: |
git config user.name github-actions
git config user.email github-actions@github.com
BRANCH_NAME="docs/auto-update-${{ inputs.target_branch }}-$(date +%Y-%m-%d-%H%M%S)"
echo "Creating branch: $BRANCH_NAME"
# Create and switch to new branch from current target branch
git checkout -b "$BRANCH_NAME"
# Stage and commit changes
git add docs/
if git diff --cached --quiet; then
echo "No changes to commit"
exit 1
fi
git commit -m "Update generated docs (${{ inputs.target_branch }})" --no-verify
git push -u origin "$BRANCH_NAME"
echo "branch_name=$BRANCH_NAME" >> $GITHUB_OUTPUT
echo "Successfully created and pushed branch"
- name: Create pull request
id: create_pr
if: steps.check_changes.outputs.changes == 'true'
env:
GITHUB_TOKEN: ${{ secrets.automation_token }}
GH_REPO: ${{ github.repository }}
run: |
CHANGES_SUMMARY=$(git diff --name-only HEAD~1 docs/ 2>/dev/null | head -10 | sed 's/^/- /' || echo "- No specific files detected")
TOTAL_CHANGES=$(git diff --name-only HEAD~1 docs/ 2>/dev/null | wc -l || echo "0")
BRANCH_NAME="${{ steps.commit_and_push.outputs.branch_name }}"
echo "Creating PR with branch: $BRANCH_NAME"
echo "Changes detected: $TOTAL_CHANGES files"
for i in {1..3}; do
echo "Attempt $i of 3..."
if gh pr create \
--base "${{ inputs.target_branch }}" \
--head "$BRANCH_NAME" \
--title "Auto-update documentation for ${{ inputs.target_branch }}" \
--body "This PR contains automatically generated documentation updates.
## Changes detected
The following documentation files were updated:
$CHANGES_SUMMARY
$([ "$TOTAL_CHANGES" -gt 10 ] && echo "- ... and $((TOTAL_CHANGES - 10)) more files")
## Generation commands
Documentation was generated using:
- \`clojure -M:ee:doc environment-variables-documentation\`
- \`clojure -M:ee:doc config-template\`
- \`clojure -M:ee:doc api-documentation\`
Please review the changes and merge if they look correct." \
--label "Type:Documentation" \
--label "no-backport" \
--reviewer "metabase/tech-writers"; then
echo "PR created successfully on attempt $i"
break
else
echo "PR creation failed on attempt $i"
if [ $i -eq 3 ]; then
echo "All attempts failed. Exiting with error."
exit 1
fi
echo "Waiting 10 seconds before retry..."
sleep 10
fi
done
- name: Skip PR creation
if: steps.check_changes.outputs.changes == 'false'
run: |
echo "No documentation changes detected, skipping PR creation"
- name: Workflow summary
if: always()
run: |
if [ "${{ steps.check_changes.outputs.changes }}" == "true" ]; then
if [ "${{ steps.commit_and_push.outcome }}" == "success" ]; then
if [ "${{ steps.create_pr.outcome }}" == "success" ]; then
echo "## ✅ Documentation Updated Successfully" >> $GITHUB_STEP_SUMMARY
echo "- Branch: \`${{ steps.commit_and_push.outputs.branch_name }}\`" >> $GITHUB_STEP_SUMMARY
echo "- PR created targeting **${{ inputs.target_branch }}**" >> $GITHUB_STEP_SUMMARY
else
echo "## ⚠️ Branch Created but PR Failed" >> $GITHUB_STEP_SUMMARY
echo "- Branch: \`${{ steps.commit_and_push.outputs.branch_name }}\`" >> $GITHUB_STEP_SUMMARY
echo "- Manual PR creation required" >> $GITHUB_STEP_SUMMARY
fi
else
echo "## ❌ Failed to Create Branch" >> $GITHUB_STEP_SUMMARY
echo "- Check workflow logs for details" >> $GITHUB_STEP_SUMMARY
fi
else
echo "## ℹ️ No Changes Detected" >> $GITHUB_STEP_SUMMARY
echo "- Documentation is up to date on **${{ inputs.target_branch }}**" >> $GITHUB_STEP_SUMMARY
fi

View File

@@ -1,36 +0,0 @@
name: Check doc links
on:
pull_request:
paths:
- "docs/**"
workflow_dispatch:
jobs:
link-checker:
name: Check documentation links
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
sparse-checkout: |
docs
.lychee
- name: Cache lychee
uses: actions/cache@v3
with:
path: ~/.cache/lychee
key: lychee-${{ runner.os }}-${{ hashFiles('.lychee/config.toml') }}
- name: Run link checker
id: link-check
uses: lycheeverse/lychee-action@v2
with:
fail: true
args: docs --config ./.lychee/config.toml --offline
format: markdown

View File

@@ -1,16 +0,0 @@
name: Auto-update documentation (master)
on:
schedule:
- cron: "0 1 * * 1-5" # Every Monday through Friday at 1 AM UTC
workflow_dispatch:
jobs:
update-master-docs:
name: Call documentation update for master
uses: ./.github/workflows/docs-generate-base.yml
with:
target_branch: "master"
m2_cache_key_suffix: "master"
secrets:
automation_token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}

View File

@@ -1,16 +0,0 @@
name: Auto-update documentation (release)
on:
schedule:
- cron: "0 2 * * 1-5" # Every Monday through Friday at 2 AM UTC
workflow_dispatch:
jobs:
update-release-docs:
name: Call documentation update for release branch
uses: ./.github/workflows/docs-generate-base.yml
with:
target_branch: "release-x.56.x"
m2_cache_key_suffix: "release"
secrets:
automation_token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}

View File

@@ -689,25 +689,24 @@ jobs:
junit-name: 'be-tests-starburst-ee'
test-args: ":only ${{ inputs.test }} :times ${{ inputs.times }}"
# TODO(rileythomp, 2025-08-20): Re-enable vertica tests once the docker image is updated
# be-tests-vertica-ee:
# if: ${{ false && inputs.driver == 'Vertica' }}
# runs-on: ubuntu-22.04
# timeout-minutes: 90
# env:
# CI: 'true'
# DRIVERS: vertica
# services:
# vertica:
# image: vertica/vertica-ce:12.0.2-0
# ports:
# - "5433:5433"
# steps:
# - uses: actions/checkout@v4
# - name: Make plugins directory
# run: mkdir plugins
# - name: Test Vertica driver
# uses: ./.github/actions/test-driver
# with:
# junit-name: 'be-tests-vertica-ee'
# test-args: ":only ${{ inputs.test }} :times ${{ inputs.times }}"
be-tests-vertica-ee:
if: inputs.driver == 'Vertica'
runs-on: ubuntu-22.04
timeout-minutes: 90
env:
CI: 'true'
DRIVERS: vertica
services:
vertica:
image: vertica/vertica-ce:12.0.2-0
ports:
- "5433:5433"
steps:
- uses: actions/checkout@v4
- name: Make plugins directory
run: mkdir plugins
- name: Test Vertica driver
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-vertica-ee'
test-args: ":only ${{ inputs.test }} :times ${{ inputs.times }}"

View File

@@ -44,7 +44,7 @@ jobs:
echo "A relevant team label is present. Tier 3 driver tests will run."
else
echo "Checking for changes in relevant folders..."
CHANGED_FILES=$(git diff --name-only $(git merge-base ${{ github.event.pull_request.base.sha }} ${{ github.sha }}) ${{ github.sha }})
CHANGED_FILES=$(git diff --name-only ${{ github.event.pull_request.base.sha }} ${{ github.sha }})
# Initialize an empty array to store folders with changes
CHANGED_FOLDERS=()
@@ -338,12 +338,6 @@ jobs:
name: H2
steps:
- uses: actions/checkout@v4
with:
submodules: true
token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
- name: Start python-runner
working-directory: python-runner
run: make run
- name: Test H2 driver
uses: ./.github/actions/test-driver
with:
@@ -448,12 +442,6 @@ jobs:
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
steps:
- uses: actions/checkout@v4
with:
submodules: true
token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
- name: Start python-runner
working-directory: python-runner
run: make run
- name: Test ${{ matrix.version.name }}
uses: ./.github/actions/test-driver
with:
@@ -757,12 +745,6 @@ jobs:
POSTGRES_HOST_AUTH_METHOD: trust
steps:
- uses: actions/checkout@v4
with:
submodules: true
token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
- name: Start python-runner
working-directory: python-runner
run: make run
- name: Test ${{ matrix.version.name }} (${{ matrix.job.name }})
uses: ./.github/actions/test-driver
with:
@@ -1058,81 +1040,79 @@ jobs:
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
# TODO(dpsutton, 2025-08-20): Re-enable starburst when we figure out why it randomly dies in CI
# be-tests-starburst-ee:
# needs: determine-driver-skips
# if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
# runs-on: ubuntu-22.04
# timeout-minutes: 60
# env:
# CI: 'true'
# DRIVERS: starburst
# services:
# starburst:
# image: trinodb/trino:latest
# env:
# CATALOG_MANAGEMENT: dynamic
# ports:
# - "8080:8080"
# name: Starburst
# steps:
# - uses: actions/checkout@v4
# - name: Make plugins directory
# run: mkdir plugins
# - name: Test Starburst driver
# uses: ./.github/actions/test-driver
# with:
# junit-name: 'be-tests-starburst-ee'
# test-args: >-
# :only-tags [:mb/driver-tests]
# - name: Upload Test Results
# uses: ./.github/actions/upload-test-results
# if: always()
# with:
# input-path: ./target/junit/
# output-name: ${{ github.job }}
# bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
# aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
# aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
# aws-region: ${{ vars.AWS_REGION }}
# trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-starburst-ee:
needs: determine-driver-skips
if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
runs-on: ubuntu-22.04
timeout-minutes: 60
env:
CI: 'true'
DRIVERS: starburst
services:
starburst:
image: trinodb/trino:latest
env:
CATALOG_MANAGEMENT: dynamic
ports:
- "8080:8080"
name: Starburst
steps:
- uses: actions/checkout@v4
- name: Make plugins directory
run: mkdir plugins
- name: Test Starburst driver
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-starburst-ee'
test-args: >-
:only-tags [:mb/driver-tests]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
# TODO(rileythomp, 2025-08-20): Re-enable vertica tests once the docker image is updated
# be-tests-vertica-ee:
# needs: determine-driver-skips
# if: ${{ false && !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
# runs-on: ubuntu-22.04
# timeout-minutes: 60
# env:
# CI: 'true'
# DRIVERS: vertica
# services:
# vertica:
# image: vertica/vertica-ce:12.0.2-0
# ports:
# - "5433:5433"
# name: Vertica
# steps:
# - uses: actions/checkout@v4
# - name: Make plugins directory
# run: mkdir plugins
# - name: Test Vertica driver
# uses: ./.github/actions/test-driver
# with:
# junit-name: 'be-tests-vertica-ee'
# test-args: >-
# :only-tags [:mb/driver-tests]
# - name: Upload Test Results
# uses: ./.github/actions/upload-test-results
# if: always()
# with:
# input-path: ./target/junit/
# output-name: ${{ github.job }}
# bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
# aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
# aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
# aws-region: ${{ vars.AWS_REGION }}
# trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-vertica-ee:
needs: determine-driver-skips
if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
runs-on: ubuntu-22.04
timeout-minutes: 60
env:
CI: 'true'
DRIVERS: vertica
services:
vertica:
image: vertica/vertica-ce:12.0.2-0
ports:
- "5433:5433"
name: Vertica
steps:
- uses: actions/checkout@v4
- name: Make plugins directory
run: mkdir plugins
- name: Test Vertica driver
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-vertica-ee'
test-args: >-
:only-tags [:mb/driver-tests]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
drivers-tests-result:
needs:
@@ -1155,9 +1135,8 @@ jobs:
- be-tests-sparksql-ee
- be-tests-sqlite-ee
- be-tests-sqlserver
# - be-tests-starburst-ee
# TODO(rileythomp, 2025-08-20): Re-enable vertica tests once the docker image is updated
# - be-tests-vertica-ee
- be-tests-starburst-ee
- be-tests-vertica-ee
- be-tests-clickhouse-ee
runs-on: ubuntu-latest
timeout-minutes: 5
@@ -1179,7 +1158,7 @@ jobs:
}));
// are all jobs skipped or successful?
if (jobs.every(job => (job.result === 'skipped' || job.result === 'success' || job.name == 'be-tests-starburst-ee'))) {
if (jobs.every(job => (job.result === 'skipped' || job.result === 'success'))) {
console.log("");
console.log(" _------. ");
console.log(" / , \_ ");

View File

@@ -1,22 +1,15 @@
# Thi workflow does the backward compatibility tests
# between the Embedding SDK package of version N and Embedding SDK Bundle of version N - 1
# N - 1 is a target of a PR
name: Embedding SDK backward-compatibility E2E Component Tests
name: E2E Cross-version Component Tests for Embedding SDK
on:
workflow_call:
inputs:
cached-embedding-sdk-dist-artifact-name:
required: true
type: string
jobs:
files-changed:
name: "SDK backward-compatibility E2E > Check which files changed"
name: "SDK cross-version E2E > Check which files changed"
runs-on: ubuntu-22.04
timeout-minutes: 3
outputs:
e2e_embedding_sdk: ${{ steps.changes.outputs.e2e_embedding_sdk }}
e2e_all: ${{ steps.changes.outputs.e2e_all }}
steps:
- uses: actions/checkout@v4
- name: Test which files changed
@@ -26,31 +19,11 @@ jobs:
token: ${{ github.token }}
filters: .github/file-paths.yaml
decide-target-ref:
runs-on: ubuntu-latest
outputs:
target-ref: ${{ steps.get-target-ref.outputs.value }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 2
- name: Decide ref to checkout
id: get-target-ref
run: |
if [[ "${{ github.event_name }}" == "pull_request" ]]; then
echo "value=${{ github.base_ref }}" >> "$GITHUB_OUTPUT"
else
echo "value=$(git rev-parse HEAD^1)" >> "$GITHUB_OUTPUT"
fi
# if this is a test on a release branch, we need to check the build requirements
get-build-requirements:
name: "SDK backward-compatibility E2E > [Backport only] Get build requirements"
needs: [decide-target-ref]
name: "SDK cross-version E2E > [Backport only] Get build requirements"
if: |
!cancelled() &&
contains(github.base_ref || github.ref, 'release-x')
!cancelled()
runs-on: ubuntu-22.04
timeout-minutes: 10
outputs:
@@ -59,7 +32,6 @@ jobs:
steps:
- uses: actions/checkout@v4
with:
ref: ${{ needs.decide-target-ref.outputs.target-ref }}
sparse-checkout: release
- name: Prepare build scripts
run: cd ${{ github.workspace }}/release && yarn && yarn build
@@ -82,12 +54,76 @@ jobs:
node_version: requirements.node,
};
e2e-tests:
name: "SDK backward-compatibility E2E > Run e2e tests"
needs: [files-changed, decide-target-ref, get-build-requirements]
build:
name: "SDK cross-version E2E > Build uberjar"
needs: [files-changed, get-build-requirements]
if: |
!cancelled() &&
needs.files-changed.outputs.e2e_embedding_sdk == 'true'
!cancelled()
runs-on: ubuntu-22.04
timeout-minutes: 25
env:
MB_EDITION: ee
INTERACTIVE: false
# make sure that builds on release branches get licenses, because we use them for releases
SKIP_LICENSES: ${{ github.event_name == 'pull_request' }}
steps:
- uses: actions/checkout@v4
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
with:
node-version: "${{ needs.get-build-requirements.outputs.node_version }}"
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: uberjar
java-version: "${{ needs.get-build-requirements.outputs.java_version || 11 }}"
- name: Build uberjar with ./bin/build.sh
run: ./bin/build.sh
- name: Prepare uberjar artifact
uses: ./.github/actions/prepare-uberjar-artifact
with:
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar-cross-version
resolve-sdk-version:
name: "SDK cross-version E2E > Resolve SDK version"
if: |
!cancelled()
runs-on: ubuntu-22.04
timeout-minutes: 10
outputs:
sdk_version: ${{ fromJson(steps.dependencies.outputs.result).sdk_version }}
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: release
- name: Prepare build scripts
run: cd ${{ github.workspace }}/release && yarn && yarn build
- name: Get latest SDK release for this branch
uses: actions/github-script@v7
id: dependencies
with:
script: | # js
const { getSdkVersionFromReleaseBranchName } = require('${{ github.workspace }}/release/dist/index.cjs');
const branchName = '${{ github.base_ref || github.ref }}';
const sdk_version = await getSdkVersionFromReleaseBranchName({
github,
owner: context.repo.owner,
repo: context.repo.repo,
branchName,
});
return {
sdk_version,
};
e2e-tests:
name: "SDK cross-version E2E > Run e2e tests"
needs: [get-build-requirements, build, resolve-sdk-version]
if: |
!cancelled() && needs.build.result == 'success'
runs-on: ubuntu-22.04
timeout-minutes: 45
env:
@@ -106,33 +142,45 @@ jobs:
steps:
- uses: actions/checkout@v4
with:
ref: ${{ needs.decide-target-ref.outputs.target-ref }}
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
# Needed to omit newly added tests for functionality that is not yet released
- name: Sparse checkout tests from SDK release commit
uses: actions/checkout@v4
with:
m2-cache-key: uberjar
java-version: "${{ needs.get-build-requirements.outputs.java_version || 21 }}"
ref: embedding-sdk-${{ needs.resolve-sdk-version.outputs.sdk_version }}
path: sdk-release-e2e
sparse-checkout: e2e
- name: Move tests from the release commit into the repo
run: |
rm -rf ./e2e/support/*
mv ./sdk-release-e2e/e2e/support/* ./e2e/support
rm -rf ./e2e/test-component/*
mv ./sdk-release-e2e/e2e/test-component/* ./e2e/test-component
shell: bash
- name: Retrieve uberjar artifact for ee
uses: actions/download-artifact@v4
with:
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar-cross-version
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
with:
node-version: "${{ needs.get-build-requirements.outputs.node_version }}"
- name: Build uberjar with ./bin/build.sh
run: ./bin/build.sh
- name: Retrieve Embedding SDK dist artifact
uses: actions/download-artifact@v4
- name: Prepare JDK ${{ needs.get-build-requirements.outputs.java_version || 11 }}
uses: actions/setup-java@v4
with:
name: ${{ inputs.cached-embedding-sdk-dist-artifact-name }}
path: ${{ github.workspace }}/resources/embedding-sdk
java-version: ${{ needs.get-build-requirements.outputs.java_version || 11 }}
distribution: "temurin"
- name: Prepare Cypress environment
id: cypress-prep
uses: ./.github/actions/prepare-cypress
- name: Install Embedding SDK package v${{ needs.resolve-sdk-version.outputs.sdk_version }}
run: yarn add @metabase/embedding-sdk-react@${{ needs.resolve-sdk-version.outputs.sdk_version }}
- name: Run Metabase
run: node e2e/runner/run_cypress_ci.js start
@@ -140,10 +188,7 @@ jobs:
run: node e2e/runner/run_cypress_ci.js snapshot --browser ${{ steps.cypress-prep.outputs.chrome-path }}
- name: Run component tests for Embedding SDK
run: |
node e2e/runner/run_cypress_ci.js component \
--browser ${{ steps.cypress-prep.outputs.chrome-path }} \
--env grepTags="-@skip-backward-compatibility",grepOmitFiltered=true
run: node e2e/runner/run_cypress_ci.js component --browser ${{ steps.cypress-prep.outputs.chrome-path }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results

View File

@@ -0,0 +1,162 @@
name: E2E cross-version for breaking changes
# See `bin/backward-compatibility-test.js` for the documentation of this workflow.
on:
workflow_dispatch:
inputs:
fe-ref:
description: "Frontend git ref to use"
type: string
required: true
default: "master"
be-ref:
description: "Backend git ref to use"
type: string
required: true
default: "master"
total-shards:
description: "Total number of parallel test shards"
type: number
required: false
default: 4
workflow_call:
inputs:
fe-ref:
description: "Frontend git ref to use"
type: string
required: true
be-ref:
description: "Backend git ref to use"
type: string
required: true
total-shards:
description: "Total number of parallel test shards"
type: number
required: false
default: 4
jobs:
generate-matrix:
name: "Breaking changes E2E > Generate matrix"
runs-on: ubuntu-latest
outputs:
matrix_json: ${{ steps.set_matrix.outputs.matrix_json }}
shards_total: ${{ steps.set_matrix.outputs.shards_total }}
steps:
- name: Set up shard matrix
id: set_matrix
uses: actions/github-script@v7
with:
# generate an output like `{"matrixJson": "{\"shard\": [1, 2, 3, 4]}", "shardsTotal": 4}` to be used in the other jobs
script: |
const totalShards = parseInt(process.env.TOTAL_SHARDS_INPUT, 10);
const matrix = { shard: Array.from({ length: totalShards }, (_, i) => i + 1) };
console.log("Generated matrix JSON", matrix);
console.log(`Total shards: ${totalShards}`);
core.setOutput('matrix_json', JSON.stringify(matrix));
core.setOutput('shards_total', totalShards);
env:
TOTAL_SHARDS_INPUT: ${{ inputs.total-shards }}
build-jar:
name: "Breaking changes E2E > Build jar"
runs-on: ubuntu-22.04
timeout-minutes: 45
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Prepare Node.js
# We're not using `actions/prepare-frontend` because we don't want to install deps from the root repo
# nor we want to touch the cache as we'll be potentially be installing different versions of packages
uses: actions/setup-node@v4
with:
node-version-file: ".nvmrc"
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: cross-version-uberjar
- name: Build jar with different frontend and backend version
env:
MB_EDITION: ee
CYPRESS_CI: true
run: |
node bin/backward-compatibility-test.js \
${{ inputs.fe-ref }} ${{ inputs.be-ref }} \
build
- name: Upload Uberjar
uses: actions/upload-artifact@v4
with:
name: metabase-jar-cross-version
path: .tmp/metabase-be/target/uberjar/metabase.jar
if-no-files-found: error
run-breaking-changes-tests:
name: "Breaking changes E2E > Run breaking changes tests"
needs: [build-jar, generate-matrix]
runs-on: ubuntu-22.04
timeout-minutes: 45
env:
TOTAL_SHARDS: ${{ needs.generate-matrix.outputs.shards_total }}
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.generate-matrix.outputs.matrix_json) }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Prepare Node.js
uses: actions/setup-node@v4
with:
node-version-file: ".nvmrc"
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: cross-version-uberjar
- name: Download Uberjar
uses: actions/download-artifact@v4
with:
name: metabase-jar-cross-version
path: .tmp/metabase-be/target/uberjar/
- name: Download FE repo
run: |
git clone --depth 1 -b ${{ inputs.fe-ref }} https://github.com/metabase/metabase.git .tmp/metabase-fe
- name: Install FE dependencies
working-directory: .tmp/metabase-fe
run: |
yarn install
- name: Prepare Cypress environment
id: cypress-prep
uses: ./.github/actions/prepare-cypress
with:
working-directory: .tmp/metabase-fe
- name: Run the tests
env:
MB_EDITION: ee
CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.MB_ALL_FEATURES_TOKEN }}
CYPRESS_MB_STARTER_CLOUD_TOKEN: ${{ secrets.MB_STARTER_CLOUD_TOKEN }}
CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.MB_PRO_CLOUD_TOKEN }}
CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.MB_PRO_SELF_HOSTED_TOKEN }}
CYPRESS_CI: true
run: |
node bin/backward-compatibility-test.js \
${{ inputs.fe-ref }} ${{ inputs.be-ref }} \
start > /dev/null & \
node bin/backward-compatibility-test.js \
${{ inputs.fe-ref }} ${{ inputs.be-ref }} \
test ${{ matrix.shard}}/${{ env.TOTAL_SHARDS }}

View File

@@ -192,7 +192,7 @@ jobs:
# - starts it
# - runs its Cypress tests against the local metabase.jar and local @metabase/embedding-sdk-react package.
# The job is not required, though if it's failing better to be sure that it's not caused by a breaking change.
# For more details see `Sample Apps compatibility with Embedding SDK tests` in `enterprise/frontend/src/embedding-sdk-package/dev.md`
# For more details see `Sample Apps compatibility with Embedding SDK tests` in `enterprise/frontend/src/embedding-sdk/dev.md`
e2e-sample-apps-tests:
needs: [get-build-requirements, get-sample-apps-data, build]
strategy:
@@ -316,7 +316,7 @@ jobs:
# - starts an app
# - runs its Cypress tests against the e2e test metabase instance and local @metabase/embedding-sdk-react package.
# The job is not required, though if it's failing better to be sure that it's not caused by a breaking change.
# For more details see `Embedding SDK integration tests with Host Apps` in `enterprise/frontend/src/embedding-sdk-package/dev.md`
# For more details see `Embedding SDK integration tests with Host Apps` in `enterprise/frontend/src/embedding-sdk/dev.md`
e2e-host-apps-tests:
needs: [get-build-requirements, build]
strategy:

View File

@@ -131,7 +131,7 @@ jobs:
run: |
echo "Running tests with specs: $SPEC_PATH"
node e2e/runner/run_cypress_ci.js e2e \
--env grepTags="-@mongo+-@flaky+-@OSS+-@skip",grepOmitFiltered=true \
--env grepTags="-@mongo+-@flaky+-@OSS --@quarantine",grepOmitFiltered=true \
--spec "$SPEC_PATH" \
--browser ${{ steps.cypress-prep.outputs.chrome-path }}
@@ -143,7 +143,7 @@ jobs:
run: |
echo "Running tests with specs: $SPEC_PATH"
node e2e/runner/run_cypress_ci.js e2e \
--env grepTags="${{inputs.tags}}+-@skip",grepOmitFiltered=true \
--env grepTags="${{inputs.tags}} --@quarantine",grepOmitFiltered=true \
--spec "$SPEC_PATH" \
--browser ${{ steps.cypress-prep.outputs.chrome-path }}

View File

@@ -1,49 +0,0 @@
name: Embedding SDK Labels Reminder
on:
pull_request:
types: [auto_merge_enabled, review_requested, labeled, unlabeled, synchronize]
branches:
- "master"
merge_group:
jobs:
files-changed:
name: Check which files changed
runs-on: ubuntu-22.04
timeout-minutes: 3
outputs:
frontend_embedding_sdk_package_sources: ${{ steps.changes.outputs.frontend_embedding_sdk_package_sources }}
steps:
- uses: actions/checkout@v4
- name: Test which files changed
uses: dorny/paths-filter@v3.0.0
id: changes
with:
token: ${{ github.token }}
filters: .github/file-paths.yaml
embedding-sdk-package-no-release-notes-reminder:
needs: [ files-changed ]
if: |
!cancelled() &&
(needs.files-changed.outputs.frontend_embedding_sdk_package_sources == 'true')
name: Embedding SDK package release-notes/no-release-notes label reminder
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- uses: jesusvasquez333/verify-pr-label-action@v1.4.0
id: verify-release-notes-labels
with:
github-token: '${{ secrets.GITHUB_TOKEN }}'
valid-labels: 'release-notes, no-release-notes'
disable-reviews: true
- name: Comment on PR if label should be added
if: ${{ always() }}
uses: ./.github/actions/notify-pull-request
with:
remove-previous-comment: true
add-new-comment: ${{ steps.verify-release-notes-labels.outcome == 'failure' }}
message: "⚠️ Changes were made to the <b>Embedding SDK Package</b>. Review this PR and apply one of the following labels.<br><br>- <code>release-notes</code> — select if this PR must be included in the main Metabase changelog release notes.<br>- <code>no-release-notes</code> — select if this PR should not be included in the main Metabase changelog release notes."
include-log: true

View File

@@ -137,7 +137,7 @@ jobs:
if: ${{ steps.docs-generation.outcome == 'failure' }}
run: exit 1
# See `Component e2e tests` in `enterprise/frontend/src/embedding-sdk-package/dev.md`
# See `Component e2e tests` in `enterprise/frontend/src/embedding-sdk/dev.md`
sdk-e2e-tests:
name: "SDK e2e tests"
uses: ./.github/workflows/e2e-component-tests-embedding-sdk.yml
@@ -146,13 +146,24 @@ jobs:
with:
cached-embedding-sdk-dist-artifact-name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
e2e-component-tests-embedding-sdk-backward-compatibility:
name: "SDK e2e backward compatibility tests"
uses: ./.github/workflows/e2e-component-tests-embedding-sdk-backward-compatibility.yml
needs: [build]
e2e-component-tests-embedding-sdk-cross-version:
name: "[Backport only] SDK e2e cross version tests"
# run on: every push on release branches, and every pr that targets a release branch
if: ${{ startsWith(github.ref, 'refs/heads/release-x.') || (github.event_name == 'pull_request' && startsWith(github.base_ref, 'release-x.')) }}
uses: ./.github/workflows/e2e-component-tests-embedding-sdk-cross-version.yml
secrets: inherit
with:
cached-embedding-sdk-dist-artifact-name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
# Temporarily skipped because the cypress runner crashes
# See `bin/backward-compatibility-test.js` for the documentation of this workflow.
# cross-version-for-breaking-changes:
# name: "[NOT REQUIRED - Experimental - Backport only] detect breaking changes with cross versions tests"
# # run on: every pr that targets a release branch, it does NOT run for pr targeting master
# if: ${{ github.event_name == 'pull_request' && startsWith(github.base_ref, 'release-x.') }}
# uses: ./.github/workflows/e2e-cross-version-for-breaking-changes.yml
# secrets: inherit
# with:
# fe-ref: ${{ github.base_ref }}
# be-ref: ${{ github.head_ref }}
embedding-sdk-tests-result:
needs:
@@ -160,7 +171,7 @@ jobs:
- embedding-sdk-docs-snippets-type-check
- embedding-sdk-api-documentation-generation-validation
- sdk-e2e-tests
- e2e-component-tests-embedding-sdk-backward-compatibility
- e2e-component-tests-embedding-sdk-cross-version
if: always() && !cancelled()
runs-on: ubuntu-latest
timeout-minutes: 5

View File

@@ -52,8 +52,6 @@ jobs:
run: yarn run lint-prettier-pure
- name: Run ESLint linter
run: yarn run lint-eslint-pure
- name: Run CSS linter
run: yarn run lint-css
fe-type-check:
if: ${{ !inputs.skip }}

32
.github/workflows/links.yml vendored Normal file
View File

@@ -0,0 +1,32 @@
name: Links
on:
repository_dispatch:
workflow_dispatch:
schedule:
- cron: "0 0 * * 0"
jobs:
linkChecker:
runs-on: ubuntu-latest
permissions:
issues: write # required for peter-evans/create-issue-from-file
steps:
- uses: actions/checkout@v4
- name: Link Checker
id: lychee
uses: lycheeverse/lychee-action@v2
with:
fail: false
args: docs --include-fragments --config ./.lychee/config.toml --verbose --no-progress
format: markdown
- name: Create Issue From File
if: steps.lychee.outputs.exit_code != 0
uses: peter-evans/create-issue-from-file@v5
with:
title: Link Checker Report
content-filepath: ./.lychee/out.md
labels: "Type:Documentation"
assignees: jeff-bruemmer

View File

@@ -9,13 +9,9 @@ concurrency:
cancel-in-progress: true
jobs:
uberjar:
if: ${{ github.event.label.name == 'Run-perf' }}
uses: ./.github/workflows/uberjar.yml
secrets: inherit
run-perf-test:
needs: [uberjar]
if: ${{ github.event.label.name == 'Run-perf' }}
uses: ./.github/workflows/perf-test.yml
with:
wait_for_uberjar: true
secrets: inherit

View File

@@ -3,6 +3,11 @@ run-name: Run perf test for ${{ github.ref_name }} by @${{ github.actor }}
on:
workflow_call:
inputs:
wait_for_uberjar:
description: "Wait for uberjar build"
required: false
type: boolean
jobs:
build:
runs-on: ubuntu-latest
@@ -26,6 +31,63 @@ jobs:
uses: aws-actions/amazon-ecr-login@v2
with:
registries: "${{ secrets.PR_ENV_AWS_ACCOUNT_ID }}"
- name: Wait for uberjar
id: wait_for_uberjar
if: ${{ inputs.wait_for_uberjar == true }}
run: |
## Get workflow run id for uberjar build
curl -Ls --output e2e-tests.json \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/${{ github.repository }}/actions/workflows/run-tests.yml/runs?head_sha=${{ github.event.pull_request.head.sha || github.sha }}
ID=$(jq -r '.workflow_runs[0].id' e2e-tests.json)
echo "Run ID: ${ID}"
## Get workflow run id for uberjar build
while [ -z "$JOB_ID" ]; do
NEXT_URL="https://api.github.com/repos/${{ github.repository }}/actions/runs/${ID}/jobs?filter=latest"
# Search for build job
echo "Searching for build job..."
while [ -z "$JOB_ID" ] && [ -n "$NEXT_URL" ]; do
# Get one page of jobs
RESPONSE=$(curl -s -i \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
-H "X-GitHub-Api-Version: 2022-11-28" \
"$NEXT_URL")
# Extract headers and body
HEADERS=$(echo "$RESPONSE" | sed -n '1,/^\r$/p')
BODY=$(echo "$RESPONSE" | sed '1,/^\r$/d')
LINK_HEADER=$(echo "$HEADERS" | grep -i '^link:' | sed 's/^link: //')
# Check for next page in Link header
if echo "$LINK_HEADER" | grep -q 'rel="next"'; then
NEXT_URL=$(echo "$LINK_HEADER" | sed -E 's/.*<([^>]+)>; rel="next".*/\1/')
else
NEXT_URL=""
fi
# Check for build job
JOB_ID=$(echo "$BODY" | jq -r '.jobs[] | select(.name | contains("build (ee)")) | .id')
done
done
echo "Job ID: ${JOB_ID}"
## Wait for uberjar build to complete
while [ true ]; do
curl -Ls --output uberjar.json \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/${{ github.repository }}/actions/jobs/${JOB_ID}
jq -r '.steps[] | select(.name == "Prepare uberjar artifact") | .status' uberjar.json | grep -q "completed" && break
echo "Waiting for uberjar build..."
sleep 10
done
echo "run_id=$(jq -r '.workflow_runs[0].id' e2e-tests.json)" >> $GITHUB_OUTPUT
- name: Retrieve uberjar artifact for ee
uses: ./.github/actions/fetch-artifact
with:

View File

@@ -5,12 +5,19 @@ on:
workflow_dispatch:
inputs:
branch:
description: "Branch we want to release the SDK from (to release from other branches use the workflow from those branches)"
description: "Branch we want to release the SDK from"
type: choice
required: true
default: "master"
options:
- master
- release-x.51.x
- release-x.52.x
- release-x.53.x
- release-x.54.x
- release-x.55.x
- release-x.56.x
- esbuild-main
concurrency:
# We want to ensure only one job is running at a time per branch because
@@ -171,39 +178,41 @@ jobs:
outputs:
sdk_version: ${{ fromJson(steps.new-sdk-version.outputs.result).version }}
major_version: ${{ fromJson(steps.new-sdk-version.outputs.result).majorVersion }}
pre_release_label: ${{ fromJson(steps.new-sdk-version.outputs.result).preReleaseLabel }}
steps:
- name: Check out the code using the provided branch
uses: actions/checkout@v4
with:
ref: ${{ inputs.branch }}
- name: Prepare build scripts
run: yarn --cwd release --frozen-lockfile && yarn --cwd release build
- name: Get current SDK version
id: current-sdk-version
run: |
VERSION=$(jq -r .version ./enterprise/frontend/src/embedding-sdk-package/package.template.json)
VERSION=$(jq -r .version ./enterprise/frontend/src/embedding-sdk/package.template.json)
echo "sdk_current_version=$VERSION" >> $GITHUB_OUTPUT
- name: Get next SDK patch or pre-release version
- name: Get next SDK patch version
id: new-sdk-version
uses: actions/github-script@v7
with:
script: | #js
const { getNextSdkVersion } = require('${{ github.workspace }}/release/dist/index.cjs');
const branch = "${{ inputs.branch }}";
script: |
const currentVersion = "${{ steps.current-sdk-version.outputs.sdk_current_version }}";
const [currentVersionWithoutSuffix, suffix] = currentVersion.split("-");
const versionParts = currentVersionWithoutSuffix.split(".");
versionParts[versionParts.length - 1] = parseInt(versionParts.at(-1)) + 1;
const newVersion = versionParts.join(".");
const [_, majorVersion] = versionParts
const result = getNextSdkVersion(branch, currentVersion);
if (!suffix) {
return {
version: newVersion,
majorVersion
};
}
return {
version: result.version,
preReleaseLabel: result.preReleaseLabel,
majorVersion: result.majorVersion
};
version: `${newVersion}-${suffix}`,
majorVersion
}
- name: Append workflow summary
run: |
@@ -285,6 +294,10 @@ jobs:
run: |
./bin/embedding-sdk/release_utils.bash update_package_json_template ${{ needs.determine-sdk-version.outputs.sdk_version }}
- name: Update changelog
run: |
yarn embedding-sdk:generate-changelog -o changelog-diff
- name: Build SDK package
run: yarn run build-embedding-sdk-package
@@ -297,6 +310,12 @@ jobs:
name: metabase-sdk
path: ./resources/embedding-sdk
- name: Upload changelog diff
uses: actions/upload-artifact@v4
with:
name: sdk-changelog-diff
path: ./changelog-diff
publish-npm:
needs: [build-sdk, determine-sdk-version]
runs-on: ubuntu-22.04
@@ -317,7 +336,6 @@ jobs:
git config --global user.name "Metabase Automation"
- name: Update readme
if: inputs.branch != 'master'
run: |
bash ./bin/embedding-sdk/release_utils.bash update_readme ${{ env.sdk_version }}
@@ -325,26 +343,25 @@ jobs:
run: |
bash ./bin/embedding-sdk/release_utils.bash update_package_json_template ${{ env.sdk_version }}
- name: Create a PR updating readme + published version (using GitHub Metabase Automation account)
- name: Retrieve SDK changelog diff
uses: actions/download-artifact@v4
with:
name: sdk-changelog-diff
- name: Update changelog
run: |
cat changelog-diff enterprise/frontend/src/embedding-sdk/CHANGELOG.md > new-changelog
mv new-changelog enterprise/frontend/src/embedding-sdk/CHANGELOG.md
- name: Create a PR updating readme + published version, and changelog (using GitHub Metabase Automation account)
run: |
git checkout -b update-sdk-version-${{ env.sdk_version }}
if [ "${{ inputs.branch }}" = "master" ]; then
COMMIT_MSG="chore(sdk): Update published npm version to ${{ env.sdk_version }}"
PR_TITLE="chore(sdk): Update SDK version to ${{ env.sdk_version }}"
PR_BODY="Update published npm package version to ${{ env.sdk_version }}"
else
COMMIT_MSG="docs(sdk): Update Readme version references and published npm version to ${{ env.sdk_version }}"
PR_TITLE="docs(sdk): Update SDK version to ${{ env.sdk_version }}"
PR_BODY="Update Readme version references and published npm package version to ${{ env.sdk_version }}"
fi
git commit -a -m "$COMMIT_MSG"
git commit -a -m 'Update Readme version references and published npm version to ${{ env.sdk_version }}'
git push origin HEAD
gh pr create --base "${{ inputs.branch }}" \
--assignee "${GITHUB_ACTOR}" \
--title "$PR_TITLE" \
--body "$PR_BODY"
gh pr create --base ${{ inputs.branch }}\
--assignee "${GITHUB_ACTOR}"\
--title "docs(sdk): Update SDK version to ${{ env.sdk_version }}"\
--body "Update Readme version references and published npm package version to ${{ env.sdk_version }}"
env:
GH_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
@@ -372,36 +389,21 @@ jobs:
name: metabase-sdk
path: sdk
- name: Compute master tag
id: compute-tag
run: |
if [ -n "${{ needs.determine-sdk-version.outputs.pre_release_label }}" ]; then
echo "masterTag=${{ needs.determine-sdk-version.outputs.pre_release_label }}" >> $GITHUB_OUTPUT
else
echo "masterTag=nightly" >> $GITHUB_OUTPUT
fi
- name: Publish to NPM
working-directory: sdk
run: |
echo //registry.npmjs.org/:_authToken=${{ secrets.NPM_RELEASE_TOKEN }} > .npmrc
TAG=$(jq -r --null-input \
--arg branch "${{ inputs.branch }}" \
--arg masterTag "${{ steps.compute-tag.outputs.masterTag }}" '
{
"master": $masterTag,
"release-x.51.x": "51-stable",
"release-x.52.x": "52-stable",
"release-x.53.x": "53-stable",
"release-x.54.x": "54-stable",
"release-x.55.x": "55-stable",
"release-x.56.x": "56-stable"
}[$branch]'
)
echo "Resolved npm tag: $TAG"
npm publish --tag "$TAG"
# Please keep the value in sync with `inputs.branch`'s release branch
npm publish --tag ${{fromJson('{
"master": "nightly",
"release-x.51.x": "51-stable",
"release-x.52.x": "52-stable",
"release-x.53.x": "53-stable",
"release-x.54.x": "54-stable",
"release-x.55.x": "55-stable",
"release-x.56.x": "56-stable",
"esbuild-main": "56-esbuild"
}')[inputs.branch]}}
- name: Add `latest` tag to the latest release branch (`release-x.55.x`) deployment
if: ${{ inputs.branch == 'release-x.56.x' }}

View File

@@ -72,6 +72,8 @@ jobs:
if: ${{ !cancelled() && !(needs.files-changed.outputs.documentation == 'true' && needs.files-changed.outputs.backend_all == 'false' && needs.files-changed.outputs.frontend_all == 'false' && needs.files-changed.outputs.e2e_all == 'false') }}
uses: ./.github/workflows/uberjar.yml
secrets: inherit
with:
source_changed: ${{ needs.files-changed.outputs.backend_all == 'true' || needs.files-changed.outputs.frontend_all == 'true' || needs.static-viz-files-changed.outputs.static_viz == 'true' }}
backend-tests:
if: ${{ !cancelled() }}
@@ -81,13 +83,6 @@ jobs:
with:
skip: ${{ needs.files-changed.outputs.backend_all != 'true' && needs.static-viz-files-changed.outputs.static_viz != 'true' }}
semantic-search-tests:
needs: files-changed
uses: ./.github/workflows/semantic-search.yml
secrets: inherit
with:
skip: ${{ needs.files-changed.outputs.backend_all != 'true' }}
app-db-tests:
needs: files-changed
uses: ./.github/workflows/app-db.yml

View File

@@ -1,348 +0,0 @@
name: Semantic Search Tests
on:
workflow_call:
inputs:
skip:
type: boolean
default: false
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}-semantic-search
cancel-in-progress: true
jobs:
semantic-search-tests-h2:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
version:
- name: H2 with pgvector pg17
junit-name: semantic-search-tests-h2-pgvector-17
pgvector-image: pgvector/pgvector:pg17
job:
- name: Semantic Search Tests
build-static-viz: false
test-args: >-
:only '"enterprise/backend/test/metabase_enterprise/semantic_search"'
exclude-tag: ':mb/driver-tests'
services:
pgvector:
image: ${{ matrix.version.pgvector-image }}
ports:
- "5432:5432"
env:
POSTGRES_DB: mb_semantic_search
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
env:
MB_PGVECTOR_DB_URL: 'jdbc:postgres://localhost:5432/mb_semantic_search?user=postgres&password=postgres'
CI: 'true'
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
steps:
- uses: actions/checkout@v4
- name: Test ${{ matrix.version.name }}
uses: ./.github/actions/test-driver
with:
build-static-viz: ${{ matrix.job.build-static-viz }}
junit-name: ${{ matrix.version.junit-name }}
test-args: >-
${{ matrix.job.test-args }}
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
semantic-search-tests-mariadb:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
version:
- name: MariaDB 10.2 with pgvector pg17
junit-name: semantic-search-tests-mariadb-10-2
mariadb-image: circleci/mariadb:10.2.23
pgvector-image: pgvector/pgvector:pg17
- name: MariaDB Latest with pgvector pg17
junit-name: semantic-search-tests-mariadb-latest
mariadb-image: circleci/mariadb:latest
pgvector-image: pgvector/pgvector:pg17
job:
- name: Semantic Search Tests
build-static-viz: false
test-args: >-
:only '"enterprise/backend/test/metabase_enterprise/semantic_search"'
exclude-tag: ':mb/driver-tests'
services:
pgvector:
image: ${{ matrix.version.pgvector-image }}
ports:
- "5432:5432"
env:
POSTGRES_DB: mb_semantic_search
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
mysql:
image: ${{ matrix.version.mariadb-image }}
ports:
- "3306:3306"
env:
MYSQL_ALLOW_EMPTY_PASSWORD: true
MYSQL_DATABASE: circle_test
env:
CI: 'true'
DRIVERS: mysql
MB_DB_TYPE: mysql
MB_DB_HOST: localhost
MB_DB_PORT: 3306
MB_DB_DBNAME: circle_test
MB_DB_USER: root
MB_MYSQL_TEST_USER: root
MB_PGVECTOR_DB_URL: 'jdbc:postgres://localhost:5432/mb_semantic_search?user=postgres&password=postgres'
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
steps:
- uses: actions/checkout@v4
- name: Test ${{ matrix.version.name }}
uses: ./.github/actions/test-driver
with:
build-static-viz: ${{ matrix.job.build-static-viz }}
junit-name: ${{ matrix.version.junit-name }}
test-args: >-
${{ matrix.job.test-args }}
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
semantic-search-tests-mysql:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
version:
- name: MySQL 8.0 with pgvector pg17
junit-name: semantic-search-tests-mysql-8-0
mysql-image: cimg/mysql:8.0
pgvector-image: pgvector/pgvector:pg17
- name: MySQL Latest with pgvector pg17
junit-name: semantic-search-tests-mysql-latest
mysql-image: mysql:latest
pgvector-image: pgvector/pgvector:pg17
job:
- name: Semantic Search Tests
build-static-viz: false
test-args: >-
:only '"enterprise/backend/test/metabase_enterprise/semantic_search"'
exclude-tag: ':mb/driver-tests'
services:
pgvector:
image: ${{ matrix.version.pgvector-image }}
ports:
- "5432:5432"
env:
POSTGRES_DB: mb_semantic_search
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
mysql:
image: ${{ matrix.version.mysql-image }}
ports:
- "3306:3306"
env:
MYSQL_ALLOW_EMPTY_PASSWORD: true
MYSQL_DATABASE: circle_test
env:
CI: 'true'
DRIVERS: mysql
MB_DB_TYPE: mysql
MB_DB_HOST: localhost
MB_DB_PORT: 3306
MB_DB_DBNAME: circle_test
MB_DB_USER: root
MB_MYSQL_TEST_USER: root
MB_PGVECTOR_DB_URL: 'jdbc:postgres://localhost:5432/mb_semantic_search?user=postgres&password=postgres'
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
steps:
- uses: actions/checkout@v4
- name: Test ${{ matrix.version.name }}
uses: ./.github/actions/test-driver
with:
build-static-viz: ${{ matrix.job.build-static-viz }}
junit-name: ${{ matrix.version.junit-name }}
test-args: >-
${{ matrix.job.test-args }}
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
semantic-search-tests-postgres:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
version:
- name: Postgres 12.x with pgvector pg17
junit-name: semantic-search-tests-postgres-12
postgres-image: postgres:12-alpine
pgvector-image: pgvector/pgvector:pg17
- name: Postgres Latest with pgvector pg17
junit-name: semantic-search-tests-postgres-latest
postgres-image: postgres:latest
pgvector-image: pgvector/pgvector:pg17
job:
- name: Semantic Search Tests
build-static-viz: false
test-args: >-
:only '"enterprise/backend/test/metabase_enterprise/semantic_search"'
exclude-tag: ':mb/driver-tests'
services:
pgvector:
image: ${{ matrix.version.pgvector-image }}
ports:
- "5432:5432"
env:
POSTGRES_DB: mb_semantic_search
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
postgres:
image: ${{ matrix.version.postgres-image }}
ports:
- "5433:5432"
env:
POSTGRES_USER: mb_test
POSTGRES_DB: mb_test
POSTGRES_HOST_AUTH_METHOD: trust
env:
CI: 'true'
DRIVERS: postgres
MB_DB_TYPE: postgres
MB_DB_PORT: 5433
MB_DB_HOST: localhost
MB_DB_DBNAME: mb_test
MB_DB_USER: mb_test
MB_POSTGRESQL_TEST_USER: mb_test
MB_PGVECTOR_DB_URL: 'jdbc:postgres://localhost:5432/mb_semantic_search?user=postgres&password=postgres'
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
steps:
- uses: actions/checkout@v4
- name: Test ${{ matrix.version.name }}
uses: ./.github/actions/test-driver
with:
build-static-viz: ${{ matrix.job.build-static-viz }}
junit-name: ${{ matrix.version.junit-name }}
test-args: >-
${{ matrix.job.test-args }}
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
semantic-search-tests-result:
needs:
- semantic-search-tests-h2
- semantic-search-tests-mariadb
- semantic-search-tests-mysql
- semantic-search-tests-postgres
runs-on: ubuntu-latest
timeout-minutes: 5
name: semantic-search-tests-result
if: always() && !cancelled()
steps:
- name: Check job status
uses: actions/github-script@v7
env:
needs: ${{ toJson(needs) }}
with:
script: | # js
const needs = JSON.parse(process.env.needs);
const jobs = Object.entries(needs).map(
([jobName, jobValues]) => ({
name: jobName,
result: jobValues.result
}));
// are all jobs skipped or successful?
if (jobs.every(job => (job.result === 'skipped' || job.result === 'success'))) {
console.log("");
console.log(" _------. ");
console.log(" / , \_ ");
console.log(" / / /{}\ |o\_ ");
console.log(" / \ `--' /-' \ ");
console.log(" | \ \ | ");
console.log(" | |`-, | ");
console.log(" / /__/)/ ");
console.log(" | | ");
console.log("");
console.log("All semantic search tests have passed (or have been skipped). Cam is very proud of you.");
process.exit(0);
}
// otherwise, something failed
console.log("");
console.log(" .::::::::::. .::::::::::. ");
console.log(" .::::''''''::::. .::::''''''::::. ");
console.log(" .:::' `::::.... ....::::' `:::. ");
console.log(" .::' `:::::::| |:::::::' `::. ");
console.log(" .::| |::::::|_ ___ __|::::::| |::. ");
console.log(" `--' |::::::|_()__()_|::::::| `--' ");
console.log(" ::: |::-o::| |::o-::| ::: ");
console.log(" `::. .|::::::| |::::::|. .::' ");
console.log(" `:::. .::\-----' `-----/::. .:::' ");
console.log(" `::::......::::' `::::......::::' ");
console.log(" `::::::::::' `::::::::::' ");
console.log("");
console.log("Semantic search tests have failed. You have been sentenced to MetaJail.");
jobs.forEach((job) => {
if (job.result !== 'success') {
console.log(`${job.name} - ${job.result}`);
}
});
process.exit(1);

View File

@@ -2,7 +2,7 @@ name: Uberjar Containerize labeled
on:
pull_request:
types: [ labeled, opened, synchronize, reopened ]
types: [ labeled ]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -10,12 +10,14 @@ concurrency:
jobs:
uberjar:
if: ${{ contains(github.event.pull_request.labels.*.name, 'build-docker-uberjar') || github.event.label.name == 'build-docker-uberjar' }}
if: ${{ github.event.label.name == 'build-docker-uberjar' }}
uses: ./.github/workflows/uberjar.yml
secrets: inherit
with:
source_changed: false
containerize:
needs: [uberjar]
if: ${{ contains(github.event.pull_request.labels.*.name, 'build-docker-uberjar') || github.event.label.name == 'build-docker-uberjar' }}
if: ${{ github.event.label.name == 'build-docker-uberjar' }}
uses: ./.github/workflows/containerize-uberjar.yml
secrets: inherit

View File

@@ -4,13 +4,14 @@ name: Build Uberjar
# or automatically for the listed branches
concurrency:
group: uberjar-${{ github.event.inputs.commit || github.event.pull_request.head.sha || github.sha }}
group: uberjar-${{ github.ref }}
cancel-in-progress: false
on:
push:
branches:
- master
- metabot-v3-main
- internal-tools
paths-ignore:
# config files
@@ -27,6 +28,11 @@ on:
commit:
description: "Optional full-length commit SHA-1 hash"
workflow_call:
inputs:
source_changed:
description: "Whether source code has changed"
type: boolean
default: true
jobs:
check-existing-artifacts:
@@ -58,20 +64,21 @@ jobs:
- name: Decide if build is needed
id: decide
run: |
SOURCE_CHANGED="${{ inputs.source_changed == true }}"
EE_EXISTS="${{ steps.check-ee.outcome == 'success' }}"
OSS_EXISTS="${{ steps.check-oss.outcome == 'success' }}"
echo "Source changed: $SOURCE_CHANGED"
echo "EE exists: $EE_EXISTS"
echo "OSS exists: $OSS_EXISTS"
# Build only if artifacts don't exist for this commit hash
# Since we build by hash, if artifacts exist for this hash, we don't need to rebuild
if [[ "$EE_EXISTS" == "false" ]] || [[ "$OSS_EXISTS" == "false" ]]; then
# Build if source changed OR artifacts don't exist
if [[ "$SOURCE_CHANGED" == "true" ]] || [[ "$EE_EXISTS" == "false" ]] || [[ "$OSS_EXISTS" == "false" ]]; then
SHOULD_BUILD="true"
echo "Will build: ee_exists=$EE_EXISTS, oss_exists=$OSS_EXISTS"
echo "Will build: source_changed=$SOURCE_CHANGED, ee_exists=$EE_EXISTS, oss_exists=$OSS_EXISTS"
else
SHOULD_BUILD="false"
echo "Skipping build: artifacts exist for this commit hash"
echo "Skipping build: artifacts exist and no source changes"
fi
echo "should_build=$SHOULD_BUILD" >> $GITHUB_OUTPUT

8
.gitignore vendored
View File

@@ -98,13 +98,6 @@ xcuserdata
dev/src/dev/nocommit/
dev/serialization_deltas/
*~
# TypeScript compilation artifacts
/build/
*.tsbuildinfo
/trace-output/
/temp-types/
**/cypress_sample_database.json
**/cypress_sample_instance_data.json
/frontend/src/cljs
@@ -163,4 +156,3 @@ CLAUDE.local.md
/logs
/.clojure-mcp
__pycache__/

4
.gitmodules vendored
View File

@@ -1,4 +0,0 @@
[submodule "python-runner"]
path = python-runner
url = https://github.com/metabase/python-runner-container.git
branch = prototype

Binary file not shown.

Before

Width:  |  Height:  |  Size: 65 KiB

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 65 KiB

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 58 KiB

After

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 58 KiB

After

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 64 KiB

After

Width:  |  Height:  |  Size: 65 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 65 KiB

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 58 KiB

After

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 58 KiB

After

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 63 KiB

After

Width:  |  Height:  |  Size: 64 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 65 KiB

After

Width:  |  Height:  |  Size: 65 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 58 KiB

After

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 56 KiB

After

Width:  |  Height:  |  Size: 57 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 82 KiB

After

Width:  |  Height:  |  Size: 83 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 71 KiB

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 73 KiB

After

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 88 KiB

After

Width:  |  Height:  |  Size: 89 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 77 KiB

After

Width:  |  Height:  |  Size: 78 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 83 KiB

After

Width:  |  Height:  |  Size: 84 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 69 KiB

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 74 KiB

After

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 73 KiB

After

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 73 KiB

After

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 69 KiB

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 69 KiB

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 73 KiB

After

Width:  |  Height:  |  Size: 73 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 81 KiB

After

Width:  |  Height:  |  Size: 82 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 74 KiB

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 70 KiB

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 72 KiB

After

Width:  |  Height:  |  Size: 73 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 87 KiB

After

Width:  |  Height:  |  Size: 88 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 76 KiB

After

Width:  |  Height:  |  Size: 77 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 83 KiB

After

Width:  |  Height:  |  Size: 84 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 69 KiB

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 73 KiB

After

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 73 KiB

After

Width:  |  Height:  |  Size: 73 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 73 KiB

After

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 69 KiB

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 69 KiB

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 72 KiB

After

Width:  |  Height:  |  Size: 73 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 27 KiB

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 108 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 212 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 8.5 KiB

After

Width:  |  Height:  |  Size: 8.4 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 17 KiB

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 110 KiB

After

Width:  |  Height:  |  Size: 110 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 133 KiB

After

Width:  |  Height:  |  Size: 133 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 40 KiB

After

Width:  |  Height:  |  Size: 40 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 116 KiB

After

Width:  |  Height:  |  Size: 116 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.6 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.8 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.4 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.3 KiB

Some files were not shown because too many files have changed in this diff Show More