🤖 backported "Remove old llms.txt superseded by Jekyll plugin on docs site" (#69134)

Remove old llms.txt superseded by Jekyll plugin on docs site (#69132)

remove llms.txt and llms-embedding-full.txt superseded by Jekyll build plugin

Co-authored-by: Phoomparin Mano <poom@metabase.com>

.claude/skills/add-malli-schemas/SKILL.md

@@ -0,0 +1,420 @@
+---
+name: add-malli-schemas
+description: Efficiently add Malli schemas to API endpoints in the Metabase codebase with proper patterns, validation timing, and error handling
+---
+
+# Add Malli Schemas to API Endpoints
+
+This skill helps you efficiently and uniformly add Malli schemas to API endpoints in the Metabase codebase.
+
+## Reference Files (Best Examples)
+
+- `src/metabase/warehouses/api.clj` - Most comprehensive schemas, custom error messages
+- `src/metabase/api_keys/api.clj` - Excellent response schemas
+- `src/metabase/collections/api.clj` - Great named schema patterns
+- `src/metabase/timeline/api/timeline.clj` - Clean, simple examples
+
+## Quick Checklist
+
+When adding Malli schemas to an endpoint:
+
+- [ ] Route params have schemas
+- [ ] Query params have schemas with `:optional true` and `:default` where appropriate
+- [ ] Request body has a schema (for POST/PUT)
+- [ ] Response schema is defined (using `:-` after route string)
+- [ ] Use existing schema types from `ms` namespace when possible
+- [ ] Consider creating named schemas for reusable or complex types
+- [ ] Add contextual error messages for validation failures
+
+## Basic Structure
+
+### Complete Endpoint Example
+
+```clojure
+(mr/def ::Color [:enum "red" "blue" "green"])
+
+(mr/def ::ResponseSchema
+  [:map
+   [:id pos-int?]
+   [:name string?]
+   [:color ::Color]
+   [:created_at ms/TemporalString]])
+
+(api.macros/defendpoint :post "/:name" :- ::ResponseSchema
+  "Create a resource with a given name."
+  [;; Route Params:
+   {:keys [name]} :- [:map [:name ms/NonBlankString]]
+   ;; Query Params:
+   {:keys [include archived]} :- [:map
+                                   [:include  {:optional true} [:maybe [:= "details"]]]
+                                   [:archived {:default false} [:maybe ms/BooleanValue]]]
+   ;; Body Params:
+   {:keys [color]} :- [:map [:color ::Color]]
+   ]
+  ;; endpoint implementation, ex:
+  {:id 99
+   :name (str "mr or mrs " name)
+   :color ({"red" "blue" "blue" "green" "green" "red"} color)
+   :created_at (t/format (t/formatter "yyyy-MM-dd'T'HH:mm:ssXXX") (t/zoned-date-time))}
+  )
+```
+
+## Common Schema Patterns
+
+1. Route Params (the 5 in `api/user/id/5`)
+2. Query Params (the sort+asc pair in `api/users?sort=asc`)
+3. Body Params (the contents of a request body. Almost always decoded from json into edn)
+4. The Raw Request map
+
+Of the 4 arguments, deprioritize usage of the raw request unless necessary.
+
+### Route Params
+
+Always required, typically just a map with an ID:
+
+```clojure
+[{:keys [id]} :- [:map [:id ms/PositiveInt]]]
+```
+
+For multiple route params:
+
+```clojure
+[{:keys [id field-id]} :- [:map
+                           [:id ms/PositiveInt]
+                           [:field-id ms/PositiveInt]]]
+```
+
+### Query Params
+
+Add properties for `{:optional true ...}` and `:default` values:
+
+```clojure
+{:keys [archived include limit offset]} :- [:map
+                                            [:archived {:default false} [:maybe ms/BooleanValue]]
+                                            [:include  {:optional true}   [:maybe [:= "tables"]]]
+                                            [:limit    {:optional true} [:maybe ms/PositiveInt]]
+                                            [:offset   {:optional true} [:maybe ms/PositiveInt]]]
+```
+
+### Request Body (POST/PUT)
+
+```clojure
+{:keys [name description parent_id]} :- [:map
+                                         [:name        ms/NonBlankString]
+                                         [:description {:optional true} [:maybe ms/NonBlankString]]
+                                         [:parent_id   {:optional true} [:maybe ms/PositiveInt]]]
+```
+
+### Response Schemas
+
+#### Simple inline response:
+
+```clojure
+(api.macros/defendpoint :get "/:id" :- [:map
+                                        [:id pos-int?]
+                                        [:name string?]]
+  "Get a thing"
+  ...)
+```
+
+#### Named schema for reuse:
+
+```clojure
+(mr/def ::Thing
+  [:map
+   [:id pos-int?]
+   [:name string?]
+   [:description [:maybe string?]]])
+
+(api.macros/defendpoint :get "/:id" :- ::Thing
+  "Get a thing"
+  ...)
+
+(api.macros/defendpoint :get "/" :- [:sequential ::Thing]
+  "Get all things"
+  ...)
+```
+
+## Common Schema Types
+
+### From `metabase.util.malli.schema` (aliased as `ms`)
+
+Prefer the schemas in the ms/* namespace, since they work better with our api infrastructure. 
+
+For example use `ms/PositiveInt` instead of `pos-int?`.
+
+```clojure
+ms/PositiveInt                  ;; Positive integer
+ms/NonBlankString               ;; Non-empty string
+ms/BooleanValue                 ;; String "true"/"false" or boolean
+ms/MaybeBooleanValue            ;; BooleanValue or nil
+ms/TemporalString               ;; ISO-8601 date/time string (for REQUEST params only!)
+ms/Map                          ;; Any map
+ms/JSONString                   ;; JSON-encoded string
+ms/PositiveNum                  ;; Positive number
+ms/IntGreaterThanOrEqualToZero  ;; 0 or positive
+```
+
+**IMPORTANT:** For response schemas, use `:any` for temporal fields, not `ms/TemporalString`!
+Response schemas validate BEFORE JSON serialization, so they see Java Time objects.
+
+### Built-in Malli Types
+
+```clojure
+:string                     ;; Any string
+:boolean                    ;; true/false
+:int                        ;; Any integer
+:keyword                    ;; Clojure keyword
+pos-int?                    ;; Positive integer predicate
+[:maybe X]                  ;; X or nil
+[:enum "a" "b" "c"]         ;; One of these values
+[:or X Y]                   ;; Schema that satisfies X or Y
+[:and X Y]                  ;; Schema that satisfies X and Y
+[:sequential X]             ;; Sequential of Xs
+[:set X]                    ;; Set of Xs
+[:map-of K V]               ;; Map with keys w/ schema K and values w/ schema V
+[:tuple X Y Z]              ;; Fixed-length tuple of schemas X Y Z
+```
+
+Avoid using sequence schemas unless completely necessary.
+
+## Step-by-Step: Adding Schemas to an Endpoint
+
+### Example: Adding return schema to `GET /api/field/:id/related`
+
+**Before:**
+```clojure
+(api.macros/defendpoint :get "/:id/related"
+  "Return related entities."
+  [{:keys [id]} :- [:map [:id ms/PositiveInt]]]
+  (-> (t2/select-one :model/Field :id id) api/read-check xrays/related))
+```
+
+**Step 1:** Check what the function returns (look at `xrays/related`)
+
+**Step 2:** Define response schema based on return type:
+
+```clojure
+(mr/def ::RelatedEntity
+  [:map
+   [:tables [:sequential [:map [:id pos-int?] [:name string?]]]]
+   [:fields [:sequential [:map [:id pos-int?] [:name string?]]]]])
+```
+
+**Step 3:** Add response schema to endpoint:
+
+```clojure
+(api.macros/defendpoint :get "/:id/related" :- ::RelatedEntity
+  "Return related entities."
+  [{:keys [id]} :- [:map [:id ms/PositiveInt]]]
+  (-> (t2/select-one :model/Field :id id) api/read-check xrays/related))
+```
+
+## Advanced Patterns
+
+### Custom Error Messages
+
+```clojure
+(def DBEngineString
+  "Schema for a valid database engine name."
+  (mu/with-api-error-message
+   [:and
+    ms/NonBlankString
+    [:fn
+     {:error/message "Valid database engine"}
+     #(u/ignore-exceptions (driver/the-driver %))]]
+   (deferred-tru "value must be a valid database engine.")))
+```
+
+### Enum with Documentation
+
+```clojure
+(def PinnedState
+  (into [:enum {:error/message "pinned state must be 'all', 'is_pinned', or 'is_not_pinned'"}]
+        #{"all" "is_pinned" "is_not_pinned"}))
+```
+
+### Complex Nested Response
+
+```clojure
+(mr/def ::DashboardQuestionCandidate
+  [:map
+   [:id ms/PositiveInt]
+   [:name ms/NonBlankString]
+   [:description [:maybe string?]]
+   [:sole_dashboard_info
+    [:map
+     [:id ms/PositiveInt]
+     [:name ms/NonBlankString]
+     [:description [:maybe string?]]]]])
+
+(mr/def ::DashboardQuestionCandidatesResponse
+  [:map
+   [:data [:sequential ::DashboardQuestionCandidate]]
+   [:total ms/PositiveInt]])
+```
+
+### Paginated Response Pattern
+
+```clojure
+(mr/def ::PaginatedResponse
+  [:map
+   [:data [:sequential ::Item]]
+   [:total integer?]
+   [:limit {:optional true} [:maybe integer?]]
+   [:offset {:optional true} [:maybe integer?]]])
+```
+
+## Common Pitfalls
+
+### Don't: Forget `:maybe` for nullable fields
+
+```clojure
+[:description ms/NonBlankString]  ;; WRONG - fails if nil
+[:description [:maybe ms/NonBlankString]]  ;; RIGHT - allows nil
+```
+
+### Don't: Forget `:optional true` for optional query params
+
+```clojure
+[:limit ms/PositiveInt]  ;; WRONG - required but shouldn't be
+[:limit {:optional true} [:maybe ms/PositiveInt]]  ;; RIGHT
+```
+
+### Don't: Forget `:default` values for known params
+
+```clojure
+[:limit ms/PositiveInt]  ;; WRONG - required but shouldn't be
+[:limit {:optional true :default 0} [:maybe ms/PositiveInt]]  ;; RIGHT
+```
+
+
+### Don't: Mix up route params, query params, and body
+
+```clojure
+;; WRONG - all in one map
+[{:keys [id name archived]} :- [:map ...]]
+
+;; RIGHT - separate destructuring
+[{:keys [id]} :- [:map [:id ms/PositiveInt]]
+ {:keys [archived]} :- [:map [:archived {:default false} ms/BooleanValue]]
+ {:keys [name]} :- [:map [:name ms/NonBlankString]]]
+```
+
+### Don't: Use `ms/TemporalString` for Java Time objects in response schemas
+
+```clojure
+;; WRONG - Java Time objects aren't strings yet
+[:date_joined ms/TemporalString]
+
+;; RIGHT - schemas validate BEFORE JSON serialization
+[:date_joined :any]  ;; Java Time object, serialized to string by middleware
+[:last_login [:maybe :any]]  ;; Java Time object or nil
+```
+
+**Why:** Response schemas validate the internal Clojure data structures BEFORE they are serialized to JSON. Java Time objects like `OffsetDateTime` get converted to ISO-8601 strings by the JSON middleware, so the schema needs to accept the raw Java objects.
+
+### Don't: Use `[:sequential X]` when the data is actually a set
+
+```clojure
+;; WRONG - group_ids is actually a set
+[:group_ids {:optional true} [:sequential pos-int?]]
+
+;; RIGHT - matches the actual data structure
+[:group_ids {:optional true} [:maybe [:set pos-int?]]]
+```
+
+**Why:** Toucan hydration methods often return sets. The JSON middleware will serialize sets to arrays, but the schema validates before serialization.
+
+### Don't: Create anonymous schemas for reused structures
+
+Use `mr/def` for schemas used in multiple places:
+
+```clojure
+(mr/def ::User
+  [:map
+   [:id pos-int?]
+   [:email string?]
+   [:name string?]])
+```
+
+## Finding Return Types
+
+1. **Look at the function being called**
+
+```clojure
+(api.macros/defendpoint :get "/:id"
+  [{:keys [id]}]
+  (t2/select-one :model/Field :id id))  ;; Returns a Field instance
+```
+
+2. **Check Toucan models for structure**
+
+Look in `src/metabase/*/models/*.clj` for model definitions.
+
+3. **Use clojure-mcp or REPL to inspect**
+
+```bash
+./bin/mage -repl '(require '\''metabase.xrays.core) (doc metabase.xrays.core/related)'
+```
+
+4. **Check tests**
+
+Tests often show the expected response structure.
+
+## Understanding Schema Validation Timing
+
+**CRITICAL CONCEPT:** Schemas validate at different points in the request/response lifecycle:
+
+### Request Parameter Schemas (Query/Body/Route)
+- Validate AFTER JSON parsing
+- Data is already deserialized (strings, numbers, booleans)
+- Use `ms/TemporalString` for date/time inputs
+- Use `ms/BooleanValue` for boolean query params
+
+### Response Schemas
+- Validate BEFORE JSON serialization
+- Data is still in Clojure format (Java Time objects, sets, keywords)
+- Use `:any` for Java Time objects
+- Use `[:set X]` for sets
+- Use `[:enum :keyword]` for keyword enums
+
+### Serialization Flow
+
+```
+Request:  JSON string → Parse → Coerce → Handler
+Response: Handler → Schema Check → Encode → Serialize → JSON string
+```
+
+## Workflow Summary
+
+1. **Read the endpoint** - understand what it does
+2. **Identify params** - route, query, body
+3. **Add parameter schemas** - use existing types from `ms`
+4. **Determine return type** - check the implementation
+5. **Define response schema** - inline or named with `mr/def`
+6. **Test** - ensure the endpoint works and validates correctly
+
+## Testing Your Schemas
+
+After adding schemas, verify:
+
+1. **Valid requests work** - test with correct data
+2. **Invalid requests fail gracefully** - test with wrong types
+3. **Optional params work** - test with/without optional params
+4. **Error messages are clear** - check validation error responses
+
+## Tips
+
+- **Start simple** - begin with basic types, refine later
+- **Reuse schemas** - if you see the same structure twice, make it a named schema
+- **Be specific** - use `ms/PositiveInt` instead of `pos-int?`
+- **Document intent** - add docstrings to named schemas
+- **Follow conventions** - look at similar endpoints in the same namespace
+- **Check the actual data** - use REPL to inspect what's actually returned before serialization
+
+## Additional Resources
+
+- [Malli Documentation](https://github.com/metosin/malli)
+- Metabase Malli utilities: `src/metabase/util/malli/schema.clj`
+- Metabase schema registry: `src/metabase/util/malli/registry.clj`

.clj-kondo/config.edn

@@ -25,6 +25,7 @@
   :reduce-without-init          {:level :warning}
   :redundant-call               {:level :warning}
   :redundant-fn-wrapper         {:level :warning}
+  :redundant-str-call           {:level :warning}
   :self-requiring-namespace     {:level :warning}
   :single-key-in                {:level :warning}
   :unsorted-imports             {:level :warning}
@@ -55,20 +56,22 @@
   ;;
   ;; globally-enabled custom linters
   ;;
-  :metabase/check-defmulti-arglists                    {:level :warning}
-  :metabase/check-me-humanize                          {:level :warning}
-  :metabase/defsetting-must-specify-export             {:level :warning}
-  :metabase/defsetting-namespace                       {:level :warning}
-  :metabase/mbql-query-first-arg                       {:level :warning}
-  :metabase/missing-test-expr-requires-in-cljs         {:level :warning}
-  :metabase/modules                                    {:level :warning} ; more config in `.clj-kondo/config/modules/config.edn`
-  :metabase/namespace-name                             {:level :warning}
-  :metabase/require-shape-checker                      {:level :warning}
-  :metabase/test-helpers-use-non-thread-safe-functions {:level :warning} ; more config in `.clj-kondo/config/test-helpers/config.edn`
-  :metabase/validate-logging                           {:level :warning}
-  :metabase/validate-mb-driver-tests                   {:level :warning}
-  :metabase/validate-mb-once                           {:level :warning}
-  :metabase/discourage-millis-duration                 {:level :warning}
+  :metabase/check-defmulti-arglists                          {:level :warning}
+  :metabase/check-me-humanize                                {:level :warning}
+  :metabase/defsetting-must-specify-export                   {:level :warning}
+  :metabase/defsetting-namespace                             {:level :warning}
+  :metabase/discourage-millis-duration                       {:level :warning}
+  :metabase/mbql-query-first-arg                             {:level :warning}
+  :metabase/missing-test-expr-requires-in-cljs               {:level :warning}
+  :metabase/modules                                          {:level :warning} ; more config in `.clj-kondo/config/modules/config.edn`
+  :metabase/namespace-name                                   {:level :warning}
+  :metabase/require-shape-checker                            {:level :warning}
+  :metabase/test-helpers-use-non-thread-safe-functions       {:level :warning} ; more config in `.clj-kondo/config/test-helpers/config.edn`
+  :metabase/validate-defendpoint-route-uses-kebab-case       {:level :warning}
+  :metabase/validate-defendpoint-query-params-use-kebab-case {:level :warning}
+  :metabase/validate-logging                                 {:level :warning}
+  :metabase/validate-mb-driver-tests                         {:level :warning}
+  :metabase/validate-mb-once                                 {:level :warning}
 
   :metabase/validate-deftest
   {:level :warning
@@ -226,9 +229,6 @@
   {:exclude
    [(build.uberjar/with-duration-ms)
     (cljs.test/is [=? malli=])
-    (clojure.core.logic/fresh)
-    (clojure.core.logic/matcha)
-    (clojure.core.logic/run)
     (clojure.test/is [=? malli= qv=])
     (clojure.tools.macro/macrolet)
     instaparse.core/transform
@@ -273,6 +273,7 @@
    clojure.java.jdbc/with-db-connection               {:message "Use sql-jdbc.execute/do-with-connection-with-options (for drivers) or t2/with-connection (for the app DB) instead of jdbc/with-db-connection"}
    clojure.java.jdbc/with-db-metadata                 {:message "Use sql-jdbc.execute/do-with-connection-with-options + .getMetaData instead of jdbc/with-db-metadata"}
    clojure.pprint/pprint                              {:message "Use metabase.util.log instead of clojure.pprint/pprint"}
+   clojure.pprint/print-table                         {:message "Use metabase.util.log instead of printing directly to the console"}
    clojure.string/lower-case                          {:message "Use metabase.util/lower-case-en instead of clojure.string/lower-case"}
    clojure.string/upper-case                          {:message "Use metabase.util/upper-case-en instead of clojure.string/upper-case"}
    compojure.core/context                             {:message "Use metabase.api.util.handlers/route-map-handler instead of compojure.core/context, it is much faster and supports OpenAPI spec generation"}
@@ -406,7 +407,6 @@
  :lint-as
  {cljs.cache/defcache                                                                   clojure.core/deftype
   clojure.core.cache/defcache                                                           clojure.core/deftype
-  clojure.core.logic/defne                                                              clj-kondo.lint-as/def-catch-all
   clojure.test.check.clojure-test/defspec                                               clojure.test/deftest
   clojure.test.check.properties/for-all                                                 clojure.core/let
   clojurewerkz.quartzite.jobs/defjob                                                    clojure.core/defn
@@ -488,8 +488,9 @@
   metabase.util.malli/defn-                                                             schema.core/defn
   metabase.util.malli/fn                                                                schema.core/fn
   metabase.util.namespaces/import-fns                                                   potemkin/import-vars
+  metabase.util.performance/doseq                                                       clojure.core/doseq
+  metabase.util.performance/for                                                         clojure.core/for
   metabase.util/with-timer-ms                                                           clojure.core/fn
-  metabase.xrays.domain-entities.malli/defn                                             schema.core/defn
   metabuild-common.core/with-open-jar-file-system                                       clojure.core/let
   metabuild-common.files/with-open-jar-file-system                                      clojure.core/let
   monger.operators/defoperator                                                          clojure.core/def
@@ -519,6 +520,7 @@
    metabase-enterprise.advanced-permissions.models.permissions.application-permissions-test/with-new-group-and-current-graph hooks.common/with-two-top-level-bindings
    metabase-enterprise.audit-app.pages-test/with-temp-objects                                                                hooks.common/with-one-binding
    metabase-enterprise.cache.config-test/with-temp-persist-models                                                            hooks.common/with-seven-bindings
+   metabase-enterprise.metabot-v3.tools.dependencies-test/with-dependent-transforms!                                         hooks.common/with-two-bindings
    metabase-enterprise.semantic-search.db.connection/with-migrate-tx                                                         hooks.common/let-one-with-optional-value
    metabase-enterprise.serialization.test-util/with-temp-dpc                                                                 hooks.toucan2.tools.with-temp/with-temp
    metabase-enterprise.transforms-python.drivers-test/with-test-table                                                        hooks.common/with-two-bindings
@@ -547,6 +549,8 @@
    metabase.embedding.api.embed-test/with-temp-card                                                                          hooks.common/let-one-with-optional-value
    metabase.embedding.api.embed-test/with-temp-dashcard                                                                      hooks.common/let-one-with-optional-value
    metabase.legacy-mbql.schema.macros/defclause                                                                              hooks.metabase.legacy-mbql.schemas.macros/defclause
+   metabase.legacy-mbql.schema.macros/defclause*                                                                             hooks.metabase.legacy-mbql.schemas.macros/defclause*
+   metabase.legacy-mbql.schema.macros/one-of                                                                                 hooks.metabase.legacy-mbql.schemas.macros/one-of
    metabase.lib.schema.mbql-clause/define-catn-mbql-clause                                                                   hooks.metabase.lib.schema.mbql-clause/define-mbql-clause
    metabase.lib.schema.mbql-clause/define-mbql-clause                                                                        hooks.metabase.lib.schema.mbql-clause/define-mbql-clause
    metabase.lib.schema.mbql-clause/define-tuple-mbql-clause                                                                  hooks.metabase.lib.schema.mbql-clause/define-mbql-clause
@@ -573,8 +577,8 @@
    metabase.pulse.models.pulse-test/with-pulse-in-collection!                                                                hooks.common/with-four-bindings
    metabase.pulse.test-util/checkins-query-card                                                                              hooks.metabase.test.data/$ids
    metabase.queries.api.card-test/with-card-param-values-fixtures                                                            hooks.common/let-one-with-optional-value
-   metabase.queries.api.card-test/with-temp-native-card!                                                                     hooks.common/with-two-bindings
-   metabase.queries.api.card-test/with-temp-native-card-with-params!                                                         hooks.common/with-two-bindings
+   metabase.queries.api.card-test/with-temp-native-card                                                                      hooks.common/with-two-bindings
+   metabase.queries.api.card-test/with-temp-native-card-with-params                                                          hooks.common/with-two-bindings
    metabase.query-processor-test.expressions-test/calculate-bird-scarcity                                                    hooks.metabase.query-processor-test.expressions-test/calculate-bird-scarcity
    metabase.query-processor-test.filter-test/count-with-filter-clause                                                        hooks.metabase.test.data/$ids
    metabase.query-processor.middleware.cache-test/with-mock-cache!                                                           hooks.common/with-two-bindings
@@ -624,10 +628,11 @@
    metabase.test/with-non-admin-groups-no-root-collection-perms                                                              hooks.common/do*
    metabase.test/with-persistence-enabled!                                                                                   hooks.metabase.model-persistence.test-util/with-persistence-enabled!
    metabase.test/with-prometheus-system!                                                                                     hooks.common/with-two-bindings
+   metabase.test/with-random-premium-token!                                                                                  hooks.common/with-one-binding
    metabase.test/with-temp                                                                                                   hooks.toucan2.tools.with-temp/with-temp
    metabase.test/with-temp-file                                                                                              hooks.metabase.test.util/with-temp-file
    metabase.test/with-temporary-setting-values                                                                               hooks.metabase.test.util/with-temporary-setting-values
-   metabase.users.api-test/with-temp-user-email!                                                                              hooks.common/with-one-binding
+   metabase.users.api-test/with-temp-user-email!                                                                             hooks.common/with-one-binding
    metabase.util.i18n/deferred-trs                                                                                           hooks.metabase.util.i18n/deferred-trs
    metabase.util.i18n/deferred-tru                                                                                           hooks.metabase.util.i18n/deferred-tru
    metabase.util.i18n/trs                                                                                                    hooks.metabase.util.i18n/trs
@@ -687,7 +692,6 @@
    metabase.test/with-temporary-raw-setting-values                                         macros.metabase.test.util/with-temporary-raw-setting-values
    metabase.users.models.user-test/with-groups!                                            macros.metabase.users.models.user-test/with-groups!
    metabase.util.namespaces/import-fn                                                      macros.metabase.util.namespaces/import-fn
-   metabase.xrays.domain-entities.malli/define-getters-and-setters                         macros.metabase.xrays.domain-entities.malli/define-getters-and-setters
    metabase.xrays.related-test/with-world                                                  macros.metabase.xrays.related-test/with-world}}
 
  :config-in-comment
@@ -715,6 +719,8 @@
    :name    metabase-lib}
   {:pattern "^metabase\\.lib\\..*"
    :name    metabase-lib-excluding-legacy-mbql}
+  {:pattern "^metabase\\.legacy-mbql\\..*"
+   :name    legacy-mbql-namespaces}
   ;;
   ;; QP and driver source namespaces: `metabase.query-processor.*`, `metabase-enterprise.sandbox.query-processor.*`,
   ;; and `metabase.driver.*`, excluding `-test` namespaces.
@@ -729,7 +735,7 @@
   ;;
   ;; driver namespaces, including tests
   ;;
-  {:pattern "metabase\\.driver.*$"
+  {:pattern "metabase\\.driver(?:-api)?.*$"
    :name    driver-namespaces}
   ;;
   ;; Perf-sensitive namespaces are currently the union of metabase.lib.*, driver namespaces, and QP (without tests).
@@ -775,7 +781,14 @@
      clojure.core/eval        {:message "Don't use eval outside of tests"}}
 
     :discouraged-namespace
-    {metabase.query-processor.store {:message "Please do not use the QP Store outside of legacy QP or driver code. Use MBQL 5/Lib or lib.metadata.jvm/applicaton-database-metadata-provider instead."}}}}
+    {metabase.legacy-mbql.jvm-util       {:message "Avoid legacy MBQL utils, use Lib instead"}
+     metabase.legacy-mbql.normalize      {:message "Avoid legacy MBQL normalization, use MBQL 5 + lib/normalize instead"}
+     metabase.legacy-mbql.predicates     {:message "Avoid legacy MBQL utils, use Lib instead"}
+     metabase.legacy-mbql.schema         {:message "Avoid the legacy MBQL schema, use MBQL 5 + lib.schema instead"}
+     metabase.legacy-mbql.schema.helpers {:message "Avoid the legacy MBQL schema, use MBQL 5 + lib.schema instead"}
+     metabase.legacy-mbql.schema.macros  {:message "Avoid the legacy MBQL schema, use MBQL 5 + lib.schema instead"}
+     metabase.legacy-mbql.util           {:message "Avoid legacy MBQL utils, use Lib instead"}
+     metabase.query-processor.store      {:message "Please do not use the QP Store outside of legacy QP or driver code. Use MBQL 5/Lib or lib.metadata.jvm/applicaton-database-metadata-provider instead."}}}}
 
   printable-namespaces
   {:linters
@@ -817,34 +830,89 @@
      toucan2.core               {:message "Don't use application database inside Lib code"}}}}
 
   metabase-lib-excluding-legacy-mbql
-  {:linters
-   {:discouraged-var
-    {metabase.legacy-mbql.util/unique-name-generator {:message "Use metabase.lib.util/unique-name-generator or non-truncating-unique-name-generator"}}}
+  {}
 
-   :discouraged-namespace
-   {metabase.legacy-mbql.js         {:message "Avoid legacy MBQL utils in Lib; move things into Lib if needded."}
-    metabase.legacy-mbql.jvm-util   {:message "Avoid legacy MBQL utils in Lib; move things into Lib if needded."}
-    metabase.legacy-mbql.predicates {:message "Avoid legacy MBQL utils in Lib; move things into Lib if needded."}
-    metabase.legacy-mbql.util       {:message "Avoid legacy MBQL utils in Lib; move things into Lib if needded."}}}
+  legacy-mbql-namespaces
+  {:linters
+   {:discouraged-namespace
+    {metabase.legacy-mbql.jvm-util       {:level :off}
+     metabase.legacy-mbql.normalize      {:level :off}
+     metabase.legacy-mbql.predicates     {:level :off}
+     metabase.legacy-mbql.schema         {:level :off}
+     metabase.legacy-mbql.schema.helpers {:level :off}
+     metabase.legacy-mbql.schema.macros  {:level :off}
+     metabase.legacy-mbql.util           {:level :off}
+     metabase.query-processor.store      {:level :off}}
+
+    ;; the `metabase.legacy-mbql` stuff is all deprecated, so we don't need to warn inside the module about other stuff
+    ;; in the module being marked deprecated or discouraged.
+    :deprecated-namespace {:level :off}
+    :deprecated-var       {:level :off}}}
 
   driver-namespaces
   {:linters
    {:discouraged-var
-    {metabase.driver/database-supports? {:level :off}}}}
+    {metabase.driver/database-supports? {:level :off}}
+
+    ;; don't warn about the legacy MBQL namespaces being deprecated or discouraged in driver code now, we don't have a
+    ;; real alternative YET so it's not a helpful warning.
+    :discouraged-namespace
+    {metabase.legacy-mbql.jvm-util       {:level :off}
+     metabase.legacy-mbql.normalize      {:level :off}
+     metabase.legacy-mbql.predicates     {:level :off}
+     metabase.legacy-mbql.schema         {:level :off}
+     metabase.legacy-mbql.schema.helpers {:level :off}
+     metabase.legacy-mbql.schema.macros  {:level :off}
+     metabase.legacy-mbql.util           {:level :off}
+     metabase.query-processor.store      {:level :off}}
+
+    :deprecated-namespace
+    {:exclude
+     [metabase.driver.common.parameters
+      metabase.driver.common.parameters.dates
+      metabase.driver.common.parameters.operators
+      metabase.driver.common.parameters.parse
+      metabase.driver.common.parameters.values
+      metabase.legacy-mbql.jvm-util
+      metabase.legacy-mbql.normalize
+      metabase.legacy-mbql.predicates
+      metabase.legacy-mbql.schema
+      metabase.legacy-mbql.schema.helpers
+      metabase.legacy-mbql.schema.macros
+      metabase.legacy-mbql.util
+      metabase.query-processor.store]}
+
+    :deprecated-var
+    {:exclude
+     {metabase.legacy-mbql.schema.helpers/is-clause?   {:namespaces ["metabase\\.driver.*"]}
+      metabase.legacy-mbql.schema.helpers/mbql-clause? {:namespaces ["metabase\\.driver.*"]}
+      metabase.legacy-mbql.util/query->source-table-id {:namespaces ["metabase\\.driver.*"]}
+      metabase.legacy-mbql.util/aggregation-at-index   {:namespaces ["metabase\\.driver.*"]}
+      metabase.legacy-mbql.util/assoc-field-options    {:namespaces ["metabase\\.driver.*"]}
+      metabase.legacy-mbql.util/expression-with-name   {:namespaces ["metabase\\.driver.*"]}
+      metabase.legacy-mbql.util/field-options          {:namespaces ["metabase\\.driver.*"]}
+      metabase.legacy-mbql.util/negate-filter-clause   {:namespaces ["metabase\\.driver.*"]}
+      metabase.legacy-mbql.util/normalize-token        {:namespaces ["metabase\\.driver.*"]}
+      metabase.legacy-mbql.util/update-field-options   {:namespaces ["metabase\\.driver.*"]}}}}}
 
   performance-sensitive
   {:linters
    {:discouraged-var
-    {clojure.core/every?          {:message "Use metabase.util.performance/every?"}
+    {clojure.core/empty?          {:message "Use metabase.util.performance/empty?"}
+     clojure.core/not-empty       {:message "Use metabase.util.performance/not-empty"}
+     clojure.core/every?          {:message "Use metabase.util.performance/every?"}
      clojure.core/mapv            {:message "Use metabase.util.performance/mapv"}
      clojure.core/run!            {:message "Use metabase.util.performance/run!"}
+     clojure.core/get-in          {:message "Use metabase.util.performance/get-in"}
      clojure.core/select-keys     {:message "Use metabase.util.performance/select-keys"}
      clojure.core/some            {:message "Use metabase.util.performance/some"}
      clojure.core/update-keys     {:message "Use metabase.util.performance/update-keys"}
      clojure.walk/keywordize-keys {:message "Use metabase.util.performance/keywordize-keys"}
      clojure.walk/postwalk        {:message "Use metabase.util.performance/postwalk"}
      clojure.walk/prewalk         {:message "Use metabase.util.performance/prewalk"}
-     clojure.walk/walk            {:message "Use metabase.util.performance/walk"}}}}
+     clojure.walk/walk            {:message "Use metabase.util.performance/walk"}
+     clojure.walk/for             {:message "Use metabase.util.performance/for"}
+     clojure.walk/doseq           {:message "Use metabase.util.performance/doseq"}}}}
 
   qp-and-driver-source-namespaces
   {:linters
@@ -865,16 +933,7 @@
      metabase.lib.core/->legacy-MBQL      {:level :off}}}}
 
   qp-source-namespaces
-  {:linters
-   {:discouraged-namespace
-    {metabase.legacy-mbql.js             {:message "Avoid legacy MBQL namespaces in the QP, use Lib instead"}
-     metabase.legacy-mbql.jvm-util       {:message "Avoid legacy MBQL namespaces in the QP, use Lib instead"}
-     metabase.legacy-mbql.normalize      {:message "Avoid legacy MBQL namespaces in the QP, use Lib instead"}
-     metabase.legacy-mbql.predicates     {:message "Avoid legacy MBQL namespaces in the QP, use Lib instead"}
-     metabase.legacy-mbql.schema         {:message "Avoid legacy MBQL namespaces in the QP, use Lib instead"}
-     metabase.legacy-mbql.schema.helpers {:message "Avoid legacy MBQL namespaces in the QP, use Lib instead"}
-     metabase.legacy-mbql.schema.macros  {:message "Avoid legacy MBQL namespaces in the QP, use Lib instead"}
-     metabase.legacy-mbql.util           {:message "Avoid legacy MBQL namespaces in the QP, use Lib instead"}}}}
+  {}
 
   general-qp-and-driver-test-namespaces
   {:linters

.clj-kondo/config/modules/config.edn

@@ -46,7 +46,7 @@
 ;; you need to make
 {:metabase/modules
  {actions
-  {:team "Workflows"
+  {:team "Gadget"
    :api  #{metabase.actions.api
            metabase.actions.args
            metabase.actions.core
@@ -62,6 +62,7 @@
            lib-be
            model-persistence
            models
+           parameters
            permissions
            public-sharing
            queries
@@ -118,9 +119,9 @@
            metabase.analyze.query-results}
    :uses #{config
            driver
-           legacy-mbql
            lib
            models
+           query-processor
            sync
            util}}
 
@@ -159,7 +160,8 @@
 
   api-routes
   {:team "UX West"
-   :api  #{metabase.api-routes.core}
+   :api  #{metabase.api-routes.cmd
+           metabase.api-routes.core}
    :uses :any}
 
   app-db
@@ -248,7 +250,7 @@
 
   ;; TODO -- way too many API namespaces.
   channel
-  {:team "Workflows"
+  {:team "Gadget"
    :api  #{metabase.channel.api
            metabase.channel.core
            metabase.channel.email
@@ -317,7 +319,7 @@
            app-db
            classloader
            config
-           legacy-mbql
+
            models
            query-processor
            search
@@ -328,6 +330,8 @@
   collections
   {:team "UX West"
    :api  #{metabase.collections.api
+           metabase.collections.core
+           metabase.collections.init
            metabase.collections.models.collection
            metabase.collections.models.collection.root}
    :uses #{api
@@ -335,19 +339,19 @@
            app-db
            audit-app
            config
-           driver
            eid-translation
            events
-           legacy-mbql
            lib-be
            models
            notification
            permissions
            premium-features
            queries
+           remote-sync
            request
            revisions
            search
+           settings
            upload
            util}}
 
@@ -366,10 +370,8 @@
 
   content-translation
   {:team "UX West"
-   :api  #{metabase.content-translation.api
-           metabase.content-translation.models}
-   :uses #{api
-           premium-features
+   :api  #{metabase.content-translation.models}
+   :uses #{premium-features
            util}}
 
   content-verification
@@ -393,7 +395,8 @@
            metabase.dashboards.init
            metabase.dashboards.models.dashboard
            metabase.dashboards.models.dashboard-card
-           metabase.dashboards.models.dashboard-tab}
+           metabase.dashboards.models.dashboard-tab
+           metabase.dashboards.schema}
    :uses #{actions
            analytics
            api
@@ -433,10 +436,7 @@
    :api  #{metabase.driver
            metabase.driver.common
            metabase.driver.common.parameters
-           metabase.driver.common.parameters.dates
-           metabase.driver.common.parameters.operators
            metabase.driver.common.parameters.parse
-           metabase.driver.common.parameters.values
            metabase.driver.ddl.interface
            metabase.driver.h2
            metabase.driver.impl
@@ -450,22 +450,34 @@
            metabase.driver.sql-jdbc.execute.diagnostic
            metabase.driver.sql-jdbc.sync
            metabase.driver.sql.query-processor
-           metabase.driver.sql.query-processor.deprecated
            metabase.driver.sql.util
            metabase.driver.sync
            metabase.driver.util}
-   :uses #{driver-api
-           util}}
+   :uses #{app-db
+           appearance
+           auth-provider
+           classloader
+           config
+           driver-api
+           events
+           legacy-mbql
+           lib
+           lib-be
+           premium-features
+           query-processor
+           settings
+           system
+           util
+           warehouse-schema
+           warehouses}}
 
   driver-api
   {:team "Drivers"
-   :api  #{metabase.driver-api.core
-           metabase.driver-api.init}
+   :api  #{metabase.driver-api.core}
    :uses #{actions
            api
            app-db
            appearance
-           auth-provider
            classloader
            config
            connection-pool
@@ -484,8 +496,7 @@
            system
            upload
            util
-           warehouse-schema
-           warehouses}}
+           warehouse-schema}}
 
   eid-translation
   {:team "UX West"
@@ -507,7 +518,7 @@
    :uses #{analytics
            api
            config
-           driver
+           dashboards
            eid-translation
            events
            models
@@ -536,7 +547,6 @@
    :uses #{appearance
            models
            query-processor
-           system
            types
            util}}
 
@@ -550,12 +560,18 @@
            util}}
 
   glossary
-  {:team "Workflows"
+  {:team "Gadget"
    :api  #{metabase.glossary.api}
    :uses #{api
+           events
            models
            util}}
 
+  graph
+  {:team "Querying"
+   :api  #{metabase.graph.core}
+   :uses #{util}}
+
   indexed-entities
   {:team "UX West"
    :api  #{metabase.indexed-entities.api
@@ -581,11 +597,9 @@
   legacy-mbql
   {:team "Querying"
    :api  #{metabase.legacy-mbql.normalize
-           metabase.legacy-mbql.predicates
            metabase.legacy-mbql.schema
            metabase.legacy-mbql.util}
    :uses #{lib
-           models
            types
            util}}
 
@@ -594,12 +608,15 @@
    :api  #{metabase.lib.binning
            metabase.lib.binning.util
            metabase.lib.card
+           metabase.lib.computed ; temporarily so that `qp.compile` can use it as lib memoization evolves
+           metabase.lib.convert  ; Only so [[metabase.queries.models.card]] can set dynamic vars.
            metabase.lib.core
            metabase.lib.equality
            metabase.lib.field
            metabase.lib.field.resolution
            metabase.lib.filter
            metabase.lib.filter.desugar.jvm ; only so [[metabase.legacy-mbql.jvm-util]] can use defined regexes
+           metabase.lib.hierarchy
            metabase.lib.init
            metabase.lib.join.util
            metabase.lib.metadata
@@ -609,10 +626,10 @@
            metabase.lib.metadata.result-metadata
            metabase.lib.normalize
            metabase.lib.options
-           metabase.lib.parse
            metabase.lib.query
            metabase.lib.schema
            metabase.lib.schema.actions
+           metabase.lib.schema.aggregation
            metabase.lib.schema.binning
            metabase.lib.schema.common
            metabase.lib.schema.constraints
@@ -647,8 +664,7 @@
   lib-be
   {:team "Querying"
    :api  #{metabase.lib-be.core
-           metabase.lib-be.init
-           metabase.lib-be.metadata.jvm}
+           metabase.lib-be.init}
    :uses #{lib
            models
            settings
@@ -702,8 +718,7 @@
 
   models
   {:team "DevEx"
-   :api  #{metabase.models.dispatch
-           metabase.models.humanization
+   :api  #{metabase.models.humanization
            metabase.models.init
            metabase.models.interface
            metabase.models.resolution
@@ -715,7 +730,6 @@
            classloader
            legacy-mbql
            lib
-           lib-be
            settings
            util}}
 
@@ -725,6 +739,7 @@
            metabase.native-query-snippets.core}
    :uses #{api
            collections
+           events
            lib
            models
            permissions
@@ -732,7 +747,7 @@
            util}}
 
   notification
-  {:team "Workflows"
+  {:team "Gadget"
    :api  #{metabase.notification.api
            metabase.notification.core
            metabase.notification.init
@@ -765,6 +780,7 @@
   parameters
   {:team "Querying"
    :api  #{metabase.parameters.chain-filter
+           metabase.parameters.core
            metabase.parameters.custom-values
            metabase.parameters.dashboard
            metabase.parameters.field
@@ -775,13 +791,10 @@
    :uses #{api
            app-db
            classloader
-           driver
-           legacy-mbql
            lib
            lib-be
            models
            query-processor
-           types
            util
            warehouse-schema
            warehouses}}
@@ -800,6 +813,7 @@
            lib
            models
            premium-features
+           remote-sync
            request
            settings
            util
@@ -824,7 +838,8 @@
    :api  #{metabase.premium-features.api
            metabase.premium-features.core
            metabase.premium-features.init}
-   :uses #{api
+   :uses #{analytics
+           api
            app-db
            classloader
            config
@@ -841,7 +856,6 @@
            app-db
            channel
            config
-           driver
            premium-features
            settings
            task
@@ -859,7 +873,6 @@
            dashboards
            events
            lib
-           lib-be
            models
            parameters
            queries
@@ -870,7 +883,7 @@
            util}}
 
   pulse
-  {:team "Workflows"
+  {:team "Gadget"
    :api  #{metabase.pulse.api
            metabase.pulse.core
            metabase.pulse.init}
@@ -882,6 +895,7 @@
            config
            driver
            events
+           lib
            models
            notification
            permissions
@@ -894,19 +908,6 @@
            enterprise/advanced-permissions
            enterprise/sandbox}}
 
-  python-runner
-  {:team "Workflows"
-   :api  #{metabase.python-runner.api
-           metabase.python-runner.core
-           metabase.python-runner.init}
-   :uses #{api
-           driver
-           models
-           query-processor
-           request
-           util
-           enterprise/transforms}}
-
   queries
   {:team "Querying"
    :api  #{metabase.queries.api
@@ -928,7 +929,6 @@
            eid-translation
            embedding
            events
-           legacy-mbql
            lib
            lib-be
            models
@@ -974,10 +974,10 @@
            metabase.query-processor.middleware.annotate
            metabase.query-processor.middleware.cache-backend.db
            metabase.query-processor.middleware.constraints
-           metabase.query-processor.middleware.fetch-source-query-legacy
            metabase.query-processor.middleware.limit
            metabase.query-processor.middleware.permissions
            metabase.query-processor.middleware.wrap-value-literals
+           metabase.query-processor.parameters.dates
            metabase.query-processor.parameters.operators
            metabase.query-processor.pipeline
            metabase.query-processor.pivot
@@ -1006,6 +1006,7 @@
            batch-processing
            cache
            config
+           dashboards
            driver
            events
            formatter
@@ -1020,14 +1021,20 @@
            premium-features
            queries
            query-permissions
-           request
            server
            settings
            system
-           types
            users
            util}}
 
+  remote-sync
+  {:team "UX West"
+   :api  #{metabase.remote-sync.core
+           metabase.remote-sync.init}
+   :uses #{api
+           models
+           premium-features}}
+
   request
   {:team "UX West"
    :api  #{metabase.request.core
@@ -1084,12 +1091,14 @@
            audit-app
            collections
            config
-           driver
            events
+           lib
+           lib-be
            models
            permissions
            premium-features
            queries
+           query-processor
            request
            settings
            startup
@@ -1104,7 +1113,8 @@
 
   segments
   {:team "Querying"
-   :api  #{metabase.segments.api}
+   :api  #{metabase.segments.api
+           metabase.segments.schema}
    :uses #{api
            events
            legacy-mbql
@@ -1132,12 +1142,14 @@
            core
            driver
            embedding
+           lib-be
            premium-features
            query-processor
            request
            session
            settings
            system
+           users
            util
            enterprise/sso}}
 
@@ -1152,7 +1164,6 @@
            app-db
            channel
            config
-           driver
            events
            login-history
            request
@@ -1187,10 +1198,8 @@
            api
            app-db
            appearance
-           channel
            config
            events
-           permissions
            request
            session
            settings
@@ -1237,6 +1246,7 @@
            app-db
            audit-app
            config
+           database-routing
            driver
            events
            lib
@@ -1296,6 +1306,9 @@
            metabase.tiles.init}
    :uses #{api
            legacy-mbql
+           lib
+           lib-be
+           parameters
            query-processor
            settings
            util}}
@@ -1307,6 +1320,8 @@
    :uses #{analytics
            api
            collections
+           config
+           events
            models
            util}}
 
@@ -1324,10 +1339,8 @@
    :uses #{analytics
            api
            appearance
-           collections
            driver
            events
-           legacy-mbql
            lib
            model-persistence
            models
@@ -1350,9 +1363,12 @@
   users
   {:team "UX West"
    :api  #{metabase.users.api
+           metabase.users.core
            metabase.users.init
            metabase.users.models.user
-           metabase.users.models.user-parameter-value}
+           metabase.users.models.user-parameter-value
+           metabase.users.schema
+           metabase.users.settings}
    :uses #{analytics
            api
            appearance
@@ -1492,18 +1508,19 @@
            queries
            query-permissions
            query-processor
+           segments
            settings
            util
            warehouse-schema}}
 
   enterprise/action-v2
-  {:team "Workflows"
+  {:team "Gadget"
    :api  #{metabase-enterprise.action-v2.api
-           metabase-enterprise.action-v2.core
            metabase-enterprise.action-v2.init}
    :uses #{actions
            api
            driver
+           events
            lib
            models
            query-processor
@@ -1534,7 +1551,6 @@
            metabase-enterprise.advanced-permissions.common
            metabase-enterprise.advanced-permissions.models.permissions.group-manager}
    :uses #{api
-           legacy-mbql
            lib
            permissions
            premium-features
@@ -1556,6 +1572,8 @@
    :api  #{metabase-enterprise.ai-sql-fixer.api}
    :uses #{api
            driver
+           lib
+           lib-be
            query-processor
            util
            enterprise/metabot-v3}}
@@ -1565,7 +1583,6 @@
    :api  #{metabase-enterprise.ai-sql-generation.api}
    :uses #{api
            driver
-           models
            util
            enterprise/metabot-v3}}
 
@@ -1603,6 +1620,8 @@
            query-permissions
            query-processor
            settings
+           setup
+           startup
            sync
            users
            util
@@ -1618,7 +1637,14 @@
   {:team "UX West"
    :api  #{metabase-enterprise.billing.api.routes}
    :uses #{api
-           premium-features util}}
+           premium-features
+           store-api
+           util}}
+
+  enterprise/bookmarks
+  {:team "UX West"
+   :api  #{}
+   :uses #{}}
 
   enterprise/cache
   {:team "UX West"
@@ -1635,14 +1661,16 @@
    :uses #{api
            events
            premium-features
+           store-api
            util
            enterprise/harbormaster}}
 
   enterprise/comments
-  {:team "Workflows"
+  {:team "UX West"
    :api  #{metabase-enterprise.comments.api
            metabase-enterprise.comments.init}
-   :uses #{api
+   :uses #{analytics
+           api
            channel
            events
            models
@@ -1652,8 +1680,7 @@
 
   enterprise/content-translation
   {:team "UX West"
-   :api  #{metabase-enterprise.content-translation.dictionary
-           metabase-enterprise.content-translation.routes}
+   :api  #{metabase-enterprise.content-translation.routes}
    :uses #{api
            content-translation
            embedding
@@ -1675,24 +1702,15 @@
    :uses :any}
 
   enterprise/dashboard-subscription-filters
-  {:team "Workflows"
+  {:team "Querying"
    :api  #{}
    :uses #{premium-features}}
 
-  enterprise/data-editing
-  {:team "Workflows"
-   :api  #{metabase-enterprise.data-editing.api}
-   :uses #{actions
-           api
-           models
-           util}}
-
   enterprise/database-replication
   {:team "Cloud"
    :api  #{metabase-enterprise.database-replication.api
            metabase-enterprise.database-replication.init}
-   :uses #{analytics
-           api
+   :uses #{api
            driver
            premium-features
            settings
@@ -1716,11 +1734,37 @@
            warehouse-schema
            warehouses}}
 
+  enterprise/dependencies
+  {:team "Querying"
+   :api  #{metabase-enterprise.dependencies.api
+           metabase-enterprise.dependencies.core
+           metabase-enterprise.dependencies.init}
+   :uses #{analyze
+           api
+           collections
+           driver
+           events
+           graph
+           lib
+           lib-be
+           models
+           native-query-snippets
+           permissions
+           premium-features
+           queries
+           query-processor
+           revisions
+           settings
+           task
+           util
+           enterprise/documents}}
+
   enterprise/documents
   {:team "UX West"
    :api  #{metabase-enterprise.documents.api
            metabase-enterprise.documents.core
-           metabase-enterprise.documents.init}
+           metabase-enterprise.documents.init
+           metabase-enterprise.documents.prose-mirror}
    :uses #{activity-feed
            api
            app-db
@@ -1728,13 +1772,13 @@
            collections
            events
            models
-           permissions
            premium-features
+           public-sharing
            queries
            query-permissions
+           query-processor
            revisions
            search
-           users
            util
            view-log}}
 
@@ -1756,7 +1800,6 @@
            audit-app
            embedding
            premium-features
-           util
            enterprise/sso}}
 
   enterprise/gsheets
@@ -1819,6 +1862,7 @@
            metabase-enterprise.metabot-v3.tools.api}
    :uses #{api
            app-db
+           audit-app
            collections
            config
            driver
@@ -1826,6 +1870,7 @@
            lib
            lib-be
            models
+           native-query-snippets
            permissions
            premium-features
            pulse
@@ -1834,6 +1879,7 @@
            search
            server
            settings
+           store-api
            sync
            system
            task
@@ -1841,7 +1887,10 @@
            util
            warehouse-schema
            warehouses
-           enterprise/documents}}
+           enterprise/dependencies
+           enterprise/documents
+           enterprise/transforms
+           enterprise/transforms-python}}
 
   enterprise/permission-debug
   {:team "UX West"
@@ -1859,6 +1908,35 @@
    :uses #{premium-features
            util}}
 
+  enterprise/public-sharing
+  {:team "UX West"
+   :api  #{metabase-enterprise.public-sharing.api}
+   :uses #{api
+           events
+           premium-features
+           public-sharing
+           query-processor
+           util}}
+
+  enterprise/remote-sync
+  {:team "UX West"
+   :api  #{metabase-enterprise.remote-sync.api
+           metabase-enterprise.remote-sync.core
+           metabase-enterprise.remote-sync.init}
+   :uses #{analytics
+           api
+           app-db
+           collections
+           events
+           models
+           premium-features
+           settings
+           startup
+           task
+           task-history
+           util
+           enterprise/serialization}}
+
   enterprise/sandbox
   {:team "UX West"
    :api  #{metabase-enterprise.sandbox.api.routes
@@ -1866,14 +1944,13 @@
    :uses #{api
            app-db
            audit-app
-           classloader
            driver
+           events
            legacy-mbql
            lib
            models
            permissions
            premium-features
-           query-permissions
            query-processor
            request
            users
@@ -1927,9 +2004,10 @@
            enterprise/metabot-v3}}
 
   enterprise/serialization
-  {:team "Workflows"
+  {:team "Querying"
    :api  #{metabase-enterprise.serialization.api
            metabase-enterprise.serialization.cmd
+           metabase-enterprise.serialization.core
            metabase-enterprise.serialization.v2.backfill-ids}
    :uses #{analytics
            api
@@ -1937,17 +2015,13 @@
            appearance
            collections
            config
-           legacy-mbql
-           lib
            logger
            models
            plugins
            premium-features
            search
-           settings
            setup
-           util
-           warehouse-schema}}
+           util}}
 
   enterprise/snippet-collections
   {:team "Querying"
@@ -1975,7 +2049,6 @@
            settings
            sso
            system
-           users
            util
            enterprise/scim}}
 
@@ -1999,39 +2072,38 @@
            metabase-enterprise.transforms.core
            metabase-enterprise.transforms.init
            metabase-enterprise.transforms.instrumentation
+           metabase-enterprise.transforms.interface
            metabase-enterprise.transforms.util}
    :uses #{analytics
            api
            app-db
-           config
+           channel
            driver
            events
            lib
            lib-be
            models
-           permissions
            premium-features
+           queries
            query-processor
            request
+           revisions
+           search
            settings
            sync
            task
            task-history
-           upload
-           util
-           enterprise/transforms-python}}
+           util}}
 
   enterprise/transforms-python
   {:team "Querying"
    :api  #{metabase-enterprise.transforms-python.api
-           metabase-enterprise.transforms-python.execute
-           metabase-enterprise.transforms-python.init
-           metabase-enterprise.transforms-python.settings}
-   :uses #{api
+           metabase-enterprise.transforms-python.init}
+   :uses #{analytics
+           api
            app-db
            config
            driver
-           models
            query-processor
            settings
            util

.clj-kondo/macros/metabase/xrays/domain_entities/malli.clj

@@ -1,16 +0,0 @@
-(ns macros.metabase.xrays.domain-entities.malli)
-
-(defmacro define-getters-and-setters [schema & defs]
-  `(do
-     (comment ~schema) ;; Reference the schema
-     ~@(for [[sym _path] (partition 2 defs)]
-         `(do
-            (defn ~(vary-meta sym
-                              assoc :export true)
-              "docs" [_x#] nil)
-            (defn ~(vary-meta (symbol (str "with-" (name sym)))
-                              assoc :export true)
-              "docs" [x# _new#] x#)
-            (defn ~(vary-meta (symbol (str (name sym) "-js"))
-                              assoc :export true)
-              "docs" [x#] x#)))))

.clj-kondo/src/hooks/metabase/api/macros.clj

@@ -1,7 +1,67 @@
 (ns hooks.metabase.api.macros
   (:require
    [clj-kondo.hooks-api :as api]
-   [hooks.common]))
+   [clojure.string :as str]))
+
+(defn- validate-route!
+  "Make sure the route uses kebab-case rather than snake_case."
+  [route]
+  (letfn [(validate-route-string! [route-string]
+            (when (and (api/string-node? route-string)
+                       (str/includes? (api/sexpr route-string) "_"))
+              (api/reg-finding! (assoc (meta route-string)
+                                       :message "REST API routes and query params should use kebab-case, not snake_case"
+                                       :type    :metabase/validate-defendpoint-route-uses-kebab-case))))]
+    (cond
+      (api/string-node? route)
+      (validate-route-string! route)
+      ;; otherwise route is a vector like ["/abc/:d/" :d <regex>]
+      (api/vector-node? route)
+      (validate-route-string! (first (:children route))))))
+
+(defn- validate-query-params!
+  "Make sure query params use kebab-case rather than snake_case.
+
+  This only works on inline schemas for now. If you define the schema elsewhere then we'd have to implement this
+  validation in Clojure-land since we can't resolve the schemas in Kondo. It should however still work on complicated
+  schemas like
+
+    [:merge <map-1> <map-2>]"
+  [query-param-schema-node]
+  (letfn [(map-schema? [node]
+            (and (api/vector-node? node)
+                 (when-let [first-child (first (:children node))]
+                   (and (api/keyword-node? first-child)
+                        (= (api/sexpr first-child) :map)))))
+          (map-schema-keys [{:keys [children], :as _node}]
+            (let [child-schemas (if (api/map-node? (second children))
+                                  (drop 2 children)
+                                  (drop 1 children))]
+              (keep (fn [child-schema]
+                      (when (api/vector-node? child-schema)
+                        (when-let [k (first (:children child-schema))]
+                          (when (api/keyword-node? k)
+                            k))))
+                    child-schemas)))
+          (validate-map-schema! [form]
+            ;; only warn once per endpoint -- just once on the first bad node
+            (when-let [bad-node (some (fn [key-node]
+                                        (when (str/includes? (str (api/sexpr key-node)) "_")
+                                          key-node))
+                                      (map-schema-keys form))]
+              (api/reg-finding! (assoc (meta bad-node)
+                                       :message "REST API routes and query params should use kebab-case, not snake_case"
+                                       :type    :metabase/validate-defendpoint-query-params-use-kebab-case))))
+          (validate! [node]
+            (cond
+              (and (api/vector-node? node)
+                   (map-schema? node))
+              (validate-map-schema! node)
+
+              (api/vector-node? node)
+              (doseq [child (:children node)]
+                (validate! child))))]
+    (validate! query-param-schema-node)))
 
 (defn defendpoint
   [arg]
@@ -18,16 +78,21 @@
                                       (cons nil more))
                   [params & body] more
                   [bindings schemas] (when (api/vector-node? params)
-                                       (loop [bindings [], schemas [], [x y & more] (:children params)]
+                                       (loop [bindings [], schemas [], [x y z & more] (:children params)]
                                          (cond
                                            (not x)
                                            [bindings schemas]
 
-                                           (= (api/sexpr x) :-)
-                                           (recur bindings (conj schemas y) more)
+                                           (and (api/keyword-node? y)
+                                                (= (api/sexpr y) :-))
+                                           (recur (conj bindings x) (conj schemas z) more)
 
                                            :else
-                                           (recur (conj bindings x) schemas (cons y more)))))]
+                                           (recur (conj bindings x) (conj schemas nil) (list* y z more)))))]
+              (validate-route! route)
+              (let [[_route-param-schema query-param-schema _body-param-schema] schemas]
+                (when query-param-schema
+                  (validate-query-params! query-param-schema)))
               (-> (api/list-node
                    (list
                     (api/token-node 'do)
@@ -54,3 +119,27 @@
                       body))))
                   (with-meta (meta node)))))]
     (update arg :node update-defendpoint)))
+
+(comment
+  (defn -defendpoint [form]
+    (-> {:node (-> form pr-str api/parse-string)}
+        defendpoint
+        :node
+        api/sexpr))
+
+  (defn x []
+    (-defendpoint '(api.macros/defendpoint :get "/:id/syncable_schemas" ; snake_case route should trigger a warning
+                     "Returns a list of all syncable schemas found for the database `id`."
+                     [{:keys [id]} :- [:map
+                                       [:id ms/PositiveInt]]
+                      ;; snake_case query parameter, should trigger a warning
+                      {:keys [filter_by]} :- [:map
+                                              [:filter_by {:optional true} :string]]]
+                     (let [db (get-database id)]
+                       (api/check-403 (or (:is_attached_dwh db)
+                                          (and (mi/can-write? db)
+                                               (mi/can-read? db))))
+                       (->> db
+                            (driver/syncable-schemas (:engine db))
+                            (vec)
+                            (sort)))))))

.clj-kondo/src/hooks/metabase/legacy_mbql/schemas/macros.clj

@@ -33,18 +33,36 @@
     (defclause [ag:var var] field-or-expression FieldOrExpressionDef)
     =>
     (def ag:var [FieldOrExpressionDef])"
-  [{:keys [node]}]
-  (let [[_defclause clause-name-form & arg-specs] (:children node)]
-    {:node (-> (hooks/list-node
-                (list
-                 (hooks/token-node 'def)
-                 (unwrap-defclause-clause-name clause-name-form)
-                 (hooks/string-node "Docstring.")
-                 (collect-defclause-body-schemas arg-specs)))
-               (with-meta (meta node)))}))
+  [{:keys [node], :as x}]
+  (let [[_defclause clause-name & arg-specs] (:children node)]
+    (assoc x :node (-> (hooks/list-node
+                        (list
+                         (hooks/token-node 'do)
+                         (-> (hooks/keyword-node (keyword "metabase.legacy-mbql.schema" (name (hooks/sexpr clause-name))))
+                             (with-meta (meta clause-name))
+                             (hooks/reg-keyword! 'metabase.legacy-mbql.schemas.macros/defclause))
+                         (collect-defclause-body-schemas arg-specs)))
+                       (with-meta (meta node))))))
+
+(defn defclause*
+  "e.g.
+
+    (defclause [ag:var var] field-or-expression FieldOrExpressionDef)
+    =>
+    (def ag:var [FieldOrExpressionDef])"
+  [{:keys [node], :as x}]
+  (let [[_defclause clause-name schema] (:children node)]
+    (assoc x :node (-> (hooks/list-node
+                        (list
+                         (hooks/token-node 'do)
+                         (-> (hooks/keyword-node (keyword "metabase.legacy-mbql.schema" (name (hooks/sexpr clause-name))))
+                             (with-meta (meta clause-name))
+                             (hooks/reg-keyword! 'metabase.legacy-mbql.schemas.macros/defclause*))
+                         schema))
+                       (with-meta (meta node))))))
 
 (comment
-  (defn- defclause* [form]
+  (defn- expand-defclause [form]
     (hooks/sexpr
      (:node
       (defclause
@@ -55,15 +73,35 @@
              form)))}))))
 
   (defn- x []
-    (defclause* '(defclause [ag:var var]
-                   field-or-expression FieldOrExpressionDef)))
+    (expand-defclause '(defclause var
+                         field-or-expression FieldOrExpressionDef)))
   ;; =>
-  (def ag:var "Docstring." [FieldOrExpressionDef])
+  (do :metabase.legacy-mbql.schema/var [FieldOrExpressionDef])
 
   (defn- y []
-    (defclause* '(defclause var
-                   x (some-list-schema)
-                   y (optional FieldOrExpressionDef)
-                   z (rest (some-other-schema)))))
+    (expand-defclause '(defclause var
+                         x (some-list-schema)
+                         y (optional FieldOrExpressionDef)
+                         z (rest (some-other-schema)))))
   ;; =>
-  (def var "Docstring." [(some-list-schema) FieldOrExpressionDef (some-other-schema)]))
+  (do :metabase.legacy-mbql.schema/var [(some-list-schema) FieldOrExpressionDef (some-other-schema)]))
+
+(defn one-of [{:keys [node], :as x}]
+  (let [[_one-of & schema-nodes] (:children node)
+        node' (-> (hooks/vector-node
+                   (mapv (fn [schema-node]
+                           (if (and (hooks/token-node? schema-node)
+                                    (simple-symbol? (hooks/sexpr schema-node)))
+                             (-> (hooks/keyword-node (keyword "mteabase.legacy-mbql.schema" (name (hooks/sexpr schema-node))))
+                                 (with-meta (meta schema-node)))
+                             schema-node))
+                         schema-nodes))
+                  (with-meta (meta node)))]
+    (assoc x :node node')))
+
+(comment
+  (defn- expand-one-of [form]
+    (hooks/sexpr (:node (one-of {:node (hooks/parse-string (with-out-str (clojure.pprint/pprint form)))}))))
+
+  (defn- z []
+    (expand-one-of '(one-of absolute-datetime relative-datetime time))))

.clj-kondo/src/hooks/metabase/settings/models/setting.clj

@@ -119,10 +119,10 @@
      reset-token-ttl-hours
      retired-setting
      retry-initial-interval
-     retry-max-attempts
+     retry-max-retries
      retry-max-interval-millis
      retry-multiplier
-     retry-randomization-factor
+     retry-jitter-factor
      saml-application-name
      saml-attribute-email
      saml-attribute-firstname

.clj-kondo/src/hooks/metabase/test/util.clj

@@ -1,6 +1,6 @@
 (ns hooks.metabase.test.util
-  (:require [clj-kondo.hooks-api :as hooks]
-            [hooks.common :as common]))
+  (:require
+   [clj-kondo.hooks-api :as hooks]))
 
 (defn- namespaced-symbol-node? [node]
   (when (hooks/token-node? node)

.clj-kondo/test/hooks/metabase/api/macros_test.clj

@@ -1,6 +1,7 @@
 (ns hooks.metabase.api.macros-test
   (:require
    [clj-kondo.hooks-api :as api]
+   [clj-kondo.impl.utils]
    [clojure.test :refer :all]
    [hooks.metabase.api.macros]))
 
@@ -69,3 +70,41 @@
              (respond :x)
              (catch Throwable e
                (raise e))))))))
+
+(defn- defendpoint [form]
+  (hooks.metabase.api.macros/defendpoint
+    {:node (-> form pr-str api/parse-string)}))
+
+(defn- defendpoint-warnings
+  [form]
+  (binding [clj-kondo.impl.utils/*ctx* {:config     {:linters {:metabase/validate-defendpoint-route-uses-kebab-case       {:level :warning}
+                                                               :metabase/validate-defendpoint-query-params-use-kebab-case {:level :warning}}}
+                                        :ignores    (atom nil)
+                                        :findings   (atom [])
+                                        :namespaces (atom {})}]
+    (defendpoint form)
+    @(:findings clj-kondo.impl.utils/*ctx*)))
+
+(deftest ^:parallel validate-route-kebab-case-test
+  (is (=? [{:message "REST API routes and query params should use kebab-case, not snake_case"
+            :type    :metabase/validate-defendpoint-route-uses-kebab-case
+            :row     pos-int?
+            :col     pos-int?}]
+          (defendpoint-warnings
+            '(api.macros/defendpoint :get "/:id/syncable_schemas" ; snake_case route should trigger a warning
+               "Returns a list of all syncable schemas found for the database `id`."
+               [{:keys [id]}]
+               id)))))
+
+(deftest ^:parallel validate-query-params-kebab-case-test
+  (is (=? [{:message "REST API routes and query params should use kebab-case, not snake_case",
+            :type    :metabase/validate-defendpoint-query-params-use-kebab-case
+            :row     pos-int?
+            :col     pos-int?}]
+          (defendpoint-warnings
+            '(api.macros/defendpoint :get "/:id/syncable-schemas"
+               "Returns a list of all syncable schemas found for the database `id`."
+               [{:keys [id]}
+                {:keys [filter_by]} :- [:map
+                                        [:filter_by {:optional true} :string]]]
+               id)))))

.cljfmt.edn

@@ -43,9 +43,6 @@
   with-meta                                                                            [[:default]]
   ;; library stuff. Libraries that specify their own `:style/indent` specs are included in the section after this
   ;; since it is generated automatically.
-  clojure.core.logic/fresh                                                             [[:block 1]]
-  clojure.core.logic/matcha                                                            [[:block 1]]
-  clojure.core.logic/run                                                               [[:block 2]]
   clojure.core.match/match                                                             [[:block 1]]
   clojure.test.check.properties/for-all                                                [[:block 1]]
   clojure.test.check.properties/for-all*                                               [[:block 1]]
@@ -163,8 +160,8 @@
   metabase.models.interface/define-batched-hydration-method                              [[:inner 0]]
   metabase.models.interface/define-simple-hydration-method                               [[:inner 0]]
   metabase.parameters.chain-filter-test/with-clean-field-values-for-field                [[:block 1]]
-  metabase.parameters.chain-filter-test/with-fk-field-to-field-remapping                 [[:block 0]]
-  metabase.parameters.chain-filter-test/with-human-readable-values-remapping             [[:block 0]]
+  metabase.parameters.chain-filter-test/with-fk-field-to-field-remapping!                [[:block 0]]
+  metabase.parameters.chain-filter-test/with-human-readable-values-remapping!            [[:block 0]]
   metabase.permissions.core/allow-changing-all-users-group-members                       [[:block 0]]
   metabase.permissions.core/disable-perms-cache                                          [[:block 0]]
   metabase.public-sharing.api-test/with-sharing-enabled-and-temp-card-referencing        [[:block 3]]
@@ -179,8 +176,8 @@
   metabase.queries.api.card-test/with-cards-in-readable-collection                       [[:block 1]]
   metabase.queries.api.card-test/with-cards-in-writeable-collection                      [[:block 1]]
   metabase.queries.api.card-test/with-ordered-items                                      [[:inner 0]]
-  metabase.queries.api.card-test/with-temp-native-card!                                  [[:block 1]]
-  metabase.queries.api.card-test/with-temp-native-card-with-params!                      [[:block 1]]
+  metabase.queries.api.card-test/with-temp-native-card                                   [[:block 1]]
+  metabase.queries.api.card-test/with-temp-native-card-with-params                       [[:block 1]]
   metabase.query-processor-test.pipeline-queries-test/pmbql-query                        [[:block 1]]
   metabase.query-processor-test.pipeline-queries-test/run-pmbql-query                    [[:block 1]]
   metabase.query-processor.debug/debug>                                                  [[:block 0]]
@@ -232,7 +229,6 @@
   metabase.test.data/$ids                                                                [[:inner 0]]
   metabase.test.data/dataset                                                             [[:inner 0]]
   metabase.test.data/mbql-query                                                          [[:inner 0]]
-  metabase.test.data/native-query                                                        [[:block 0]]
   metabase.test.data/query                                                               [[:block 1]]
   metabase.test.data/run-mbql-query                                                      [[:inner 0]]
   metabase.test.data/with-empty-h2-app-db!                                               [[:block 0]]
@@ -253,7 +249,6 @@
   metabase.test/format-rows-by                                                           [[:default]]
   metabase.test/let-url                                                                  [[:block 1]]
   metabase.test/mbql-query                                                               [[:inner 0]]
-  metabase.test/native-query                                                             [[:block 0]]
   metabase.test/query                                                                    [[:block 1]]
   metabase.test/run-mbql-query                                                           [[:inner 0]]
   metabase.test/test-driver                                                              [[:inner 0]]

.codespellignore

@@ -0,0 +1,23 @@
+Brite
+afterAll
+ags
+altho
+cancelation
+cancelations
+datas
+edn
+eduction
+hasTables
+horizontalX
+horizontalY
+juxt
+methodd
+notIn
+pass-thru
+ser
+serieses
+sortings
+trs
+trsn
+tru
+trun

.codespellrc

@@ -0,0 +1,3 @@
+[codespell]
+skip = *.csv,*.edn,*.geojson,*.json,*.tsv,*.xml,docs/developers-guide/drivers/driver-tests.md,docs/developers-guide/e2e-tests.md,docs/questions/query-builder/expressions-list.md,docs/questions/visualizations/country-codes.md,e2e/test/scenarios/admin/i18n/content-translation/constants.ts,frontend/src/cljs,frontend/src/metabase/querying/expressions/tokenizer/lezer.js,frontend/src/metabase/static-viz/constants/char-sizes.ts,frontend/src/metabase/visualizations/lib/mapping_codes.ts,modules/drivers/mongo/src/metabase/driver/mongo/operators.clj
+ignore-words = .codespellignore

.devcontainer/Dockerfile

@@ -4,7 +4,6 @@ FROM mcr.microsoft.com/vscode/devcontainers/java:21
 
 RUN ( curl -fsSL https://deb.nodesource.com/setup_22.x | bash ) \
   && export DEBIAN_FRONTEND=noninteractive \
-  && apt-key adv --refresh-keys --keyserver keyserver.ubuntu.com \
   && apt-get update && apt-get -y install --no-install-recommends nodejs yarn rlwrap fontconfig
 
 # install Clojure

.dockerignore

@@ -1,5 +1,4 @@
 .babel_cache
-docs/*
 target/*
 .circleci
 .cpcache

.eslintrc.js

@@ -118,7 +118,6 @@ module.exports = {
           "And",
           "When",
           "Then",
-          "describeWithSnowplow",
         ],
       },
     ],
@@ -136,7 +135,7 @@ module.exports = {
   },
   env: {
     browser: true,
-    es6: true,
+    es2020: true,
     commonjs: true,
     jest: true,
     "jest/globals": true,
@@ -203,6 +202,7 @@ module.exports = {
         "frontend/src/metabase/admin/**/*",
         "frontend/src/metabase/setup/**/*",
         "enterprise/frontend/src/metabase-enterprise/whitelabel/**/*",
+        "enterprise/frontend/src/metabase-enterprise/embedding/**/*",
         "frontend/lint/**/*",
         "*.stories.*",
         "**/.storybook/*",
@@ -347,5 +347,11 @@ module.exports = {
         "import/no-commonjs": "off",
       },
     },
+    {
+      files: ["**/*.stories.tsx", "**/preview.tsx"],
+      rules: {
+        "import/no-default-export": "off",
+      },
+    },
   ],
 };

.github/CODEOWNERS

@@ -1,12 +1,12 @@
 # Before changing this file, make sure you have read how we use it:
 # https://www.notion.so/metabase/Pull-Request-Code-Review-Guide-374f05df889748b69d67af753f9bc9b8?pvs=4#36550c04ceab40b5a8b8bee6264b090c
 
-enterprise/frontend/src/embedding-sdk-package @metabase/embedding
-enterprise/frontend/src/embedding-sdk-bundle @metabase/embedding
-enterprise/frontend/src/embedding-sdk-shared @metabase/embedding
-frontend/src/metabase/embedding-sdk @metabase/embedding
-frontend/src/metabase/public/containers/PublicOrEmbeddedQuestion @metabase/embedding
-frontend/src/metabase/public/containers/PublicOrEmbeddedDashboard @metabase/embedding
+# enterprise/frontend/src/embedding-sdk-package @metabase/embedding
+# enterprise/frontend/src/embedding-sdk-bundle @metabase/embedding
+# enterprise/frontend/src/embedding-sdk-shared @metabase/embedding
+# frontend/src/metabase/embedding-sdk @metabase/embedding
+# frontend/src/metabase/public/containers/PublicOrEmbeddedQuestion @metabase/embedding
+# frontend/src/metabase/public/containers/PublicOrEmbeddedDashboard @metabase/embedding
 frontend/src/metabase/ui @metabase/core-frontend-ux-west
 # snowplow/* @metabase/data
 src/metabase/analytics/prometheus.clj @metabase/cloud-ops

.github/actions/cleanup-python-runner/action.yml

@@ -0,0 +1,36 @@
+name: Cleanup Python Runner
+description: Cleanup python-runner services and upload logs as artifacts
+
+inputs:
+  upload-logs:
+    description: 'Whether to upload logs (typically on failure)'
+    required: false
+    default: 'false'
+  logs-artifact-suffix:
+    description: 'The unique name of the .zip artifact containing the logs'
+    required: false
+    default: ''
+
+runs:
+  using: composite
+  steps:
+    - name: Cleanup python-runner services
+      if: always()
+      run: |
+        echo "Stopping python-runner services..."
+        if [ -d "python-runner" ]; then
+          cd python-runner
+          make stop || echo "WARNING: Failed to stop python runner"
+        fi
+      shell: bash
+
+    - name: Upload python-runner logs
+      if: ${{inputs.upload-logs == 'true' && inputs.logs-artifact-suffix != ''}}
+      uses: actions/upload-artifact@v4
+      with:
+        name: python-runner-logs-${{inputs.logs-artifact-suffix}}
+        path: |
+          python-runner.out.log
+          python-runner.err.log
+        retention-days: 1
+        if-no-files-found: warn

.github/actions/cross-branch-migration-test/action.yml

@@ -17,8 +17,6 @@ runs:
 
     - name: Prepare backend environment
       uses: ./.github/actions/prepare-backend
-      with:
-        m2-cache-key: cross-branch-migrations
 
     - name: Run backend on base branch
       run: |

.github/actions/get-cache-keys/action.yml

@@ -0,0 +1,37 @@
+name: Get cache keys
+description: |
+  Get deterministic cache keys for Maven and Node.js caches based on the centrally controlled cache prefix,
+  and the suffix based on the runner OS and the current week of the year.
+
+outputs:
+  m2-cache-key:
+    description: "The cache key for the Maven cache."
+    value: ${{ steps.keys-factory.outputs.m2-cache-key }}
+  node-cache-key:
+    description: "The cache key for the Node.js cache."
+    value: ${{ steps.keys-factory.outputs.node-cache-key }}
+  eslint-cache-key:
+    description: "The cache key for Eslint cache"
+    value: ${{ steps.keys-factory.outputs.eslint-cache-key }}
+  cypress-cache-key:
+    description: "The cache key for Cypress cache"
+    value: ${{ steps.keys-factory.outputs.cypress-cache-key }}
+
+runs:
+  using: "composite"
+  steps:
+    - name: Generate and output cache keys
+      id: keys-factory
+      run: |
+        M2_CACHE_PREFIX='M2'
+        NODE_CACHE_PREFIX='node-modules'
+        ESLINT_CACHE_PREFIX='eslint'
+        CYPRESS_CACHE_PREFIX='cypress'
+        WEEK_OF_YEAR=$(/bin/date "+%G-%V")
+        CACHE_SUFFIX="${{ runner.os }}-$WEEK_OF_YEAR"
+
+        echo "m2-cache-key=$M2_CACHE_PREFIX-$CACHE_SUFFIX" >> $GITHUB_OUTPUT
+        echo "node-cache-key=$NODE_CACHE_PREFIX-$CACHE_SUFFIX" >> $GITHUB_OUTPUT
+        echo "eslint-cache-key=$ESLINT_CACHE_PREFIX-$CACHE_SUFFIX" >> $GITHUB_OUTPUT
+        echo "cypress-cache-key=$CYPRESS_CACHE_PREFIX-$CACHE_SUFFIX" >> $GITHUB_OUTPUT
+      shell: bash

.github/actions/maximize-disk-space/action.yml

@@ -0,0 +1,19 @@
+name: Maximize disk space
+description: Removes unused stuff from GH runner
+
+runs:
+  using: "composite"
+  steps:
+    - name: Maximize build space
+      shell: bash
+      run: |
+        echo "-- Dotnet"
+        sudo rm -rf /usr/share/dotnet
+        echo "-- Android"
+        sudo rm -rf /usr/local/lib/android
+        echo "-- Haskell"
+        sudo rm -rf /opt/ghc
+        echo "-- CodeQL"
+        sudo rm -rf /opt/hostedtoolcache/CodeQL
+
+        df -h

.github/actions/prepare-backend/action.yml

@@ -1,7 +1,7 @@
 name: Prepare back-end environment
 description: |
-  Prepare the backend. Installs jdk, Clojure, and fetches the cache for m2. The CI config fetchs `ci-test-config.json`
-  from master and lets us ignore tests at the var level if they are flaking.
+  Prepare the backend. Installs jdk, Clojure, and fetches the cache for m2.
+  The CI config fetches `ci-test-config.json` from master and lets us ignore tests at the var level if they are flaking.
 inputs:
   java-version:
     required: true
@@ -9,10 +9,6 @@ inputs:
   clojure-version:
     required: true
     default: '1.12.0.1488'
-  m2-cache-key:
-    description: 'Key to cache M2 packages from Maven Central'
-    required: true
-    default: 'm2'
 
 runs:
   using: "composite"
@@ -28,21 +24,15 @@ runs:
       run: |
         curl -O https://download.clojure.org/install/linux-install-${{ inputs.clojure-version }}.sh &&
         sudo bash ./linux-install-${{ inputs.clojure-version }}.sh
-    - name: Fetch CI config
-      shell: bash
-      run: |
-        curl -o ci-test-config.json https://raw.githubusercontent.com/metabase/metabase/refs/heads/master/ci-test-config.json
-    - name: Check to see if dependencies should be cached
-      if: ${{ contains(github.event.head_commit.message, '[ci nocache]') }}
-      run: echo "Commit message includes [ci nocache]; dependencies will NOT be cached"
-      shell: bash
-    - name: Get M2 cache
-      uses: actions/cache@v4
-      if: ${{ !contains(github.event.head_commit.message, '[ci nocache]') }}
+
+    - name: Get M2 cache key
+      uses: ./.github/actions/get-cache-keys
+      id: get-cache-keys
+
+    - name: Restore M2 cache
+      uses: actions/cache/restore@v4
       with:
         path: |
           ~/.m2
           ~/.gitlibs
-        key: ${{ runner.os }}-${{ inputs.m2-cache-key }}-${{ hashFiles('**/deps.edn') }}
-        restore-keys: |
-          ${{ runner.os }}-${{ inputs.m2-cache-key }}-
+        key: ${{ steps.get-cache-keys.outputs.m2-cache-key }}

.github/actions/prepare-cypress/action.yml

@@ -22,16 +22,16 @@ runs:
         chrome-version: 1097615
       id: setup-chrome
 
-    - name: Check to see if dependencies should be cached
-      if: ${{ contains(github.event.head_commit.message, '[ci nocache]') }}
-      run: echo "Commit message includes [ci nocache]; dependencies will NOT be cached"
-      shell: bash
+    - name: Get cache keys
+      uses: ./.github/actions/get-cache-keys
+      id: get-cache-keys
+
     - name: Get Cypress cache
-      uses: actions/cache@v4
-      if: ${{ !contains(github.event.head_commit.message, '[ci nocache]') }}
+      uses: actions/cache/restore@v4
       with:
         path: ~/.cache/Cypress
-        key: ${{ runner.os }}-Cypress-${{ hashFiles('**/yarn.lock') }}
+        key: ${{ steps.get-cache-keys.outputs.cypress-cache-key }}
+
     - name: Ensure that Cypress executable is ready
       run: |
         yarn cypress install

.github/actions/prepare-frontend/action.yml

@@ -3,6 +3,10 @@ inputs:
   node-version:
     description: "The version of Node.js to use, overriding .nvmrc"
     required: false
+  apply-patches:
+    description: "Whether to patch frontend dependencies with patch-package."
+    required: true
+    default: "true"
 runs:
   using: "composite"
   steps:
@@ -12,26 +16,42 @@ runs:
         node-version: ${{ inputs.node-version }} # this overrides, if present
         node-version-file: ".nvmrc"
 
-    - name: Check to see if dependencies should be cached
-      if: ${{ contains(github.event.head_commit.message, '[ci nocache]') }}
+    - name: Get cache keys
+      uses: ./.github/actions/get-cache-keys
+      id: get-cache-keys
+
+    - name: Restore M2 cache
+      uses: actions/cache/restore@v4
+      with:
+        path: |
+          ~/.m2
+          ~/.gitlibs
+        key: ${{ steps.get-cache-keys.outputs.m2-cache-key }}
+
+    - name: Get yarn cache directory
+      id: yarn-cache-dir
       run: |
-        echo "Commit message includes [ci nocache]; dependencies will NOT be cached"
-        yarn cache clean
+        YARN_CACHE_DIR=$(yarn cache dir)
+        echo "Yarn cache directory: $YARN_CACHE_DIR"
+        echo "dir=$YARN_CACHE_DIR" >> $GITHUB_OUTPUT
       shell: bash
-    - name: Get M2 cache
-      uses: actions/cache@v4
-      if: ${{ !contains(github.event.head_commit.message, '[ci nocache]') }}
-      with:
-        path: ~/.m2
-        key: ${{ runner.os }}-cljs-${{ hashFiles('**/shadow-cljs.edn') }}
 
-    - name: Get node_modules cache
-      uses: actions/cache@v4
+    - name: Restore node_modules cache
+      uses: actions/cache/restore@v4
       id: node-modules-cache
-      if: ${{ !contains(github.event.head_commit.message, '[ci nocache]') }}
       with:
-        path: node_modules
-        key: ${{ runner.os }}-node-modules-${{ hashFiles('**/yarn.lock', '**/patches/*.patch') }}
+        path: |
+          ${{ steps.yarn-cache-dir.outputs.dir }}
+          node_modules
+        key: ${{ steps.get-cache-keys.outputs.node-cache-key }}
 
-    - run: yarn install --frozen-lockfile --prefer-offline
+    # postinstall lifecycle hook can be dangerous in CI so we only run it locally (see package.json)
+    # If a job wants to patch packages, it does so explicitly in the CI. This setting is opt-out.
+    # For example, we never want to cache node_modules with patches already applied so cache-generator.yml skips that step.
+    - name: Clean install
+      run: yarn install --frozen-lockfile --prefer-offline
+      shell: bash
+    - name: Patch frontend dependencies
+      if: inputs.apply-patches == 'true'
+      run: yarn patch-package
       shell: bash

.github/actions/run-embedding-sdk-storybook/action.yml

@@ -1,33 +0,0 @@
-name: Run Embedding SDK Storybook
-runs:
-  using: "composite"
-  steps:
-    - name: Prepare frontend environment
-      uses: ./.github/actions/prepare-frontend
-
-    - name: Prepare back-end environment
-      uses: ./.github/actions/prepare-backend
-      with:
-        m2-cache-key: "cljs"
-
-    - name: Compile CLJS
-      run: yarn build-pure:cljs
-      shell: bash
-
-    - name: Build Storybook
-      run: yarn build-storybook-embedding-sdk
-      shell: bash
-
-    - name: Serve Storybook
-      run: yarn http-server storybook-static -p 6006 &
-      shell: bash
-
-    - name: Wait for Storybook to become available
-      run: |
-        echo "Waiting for Storybook to be served..."
-        while ! curl --silent --fail http://localhost:6006; do
-          sleep 5
-          echo "Retrying..."
-        done
-        echo "Storybook is being served!"
-      shell: bash

.github/actions/setup-python-runner/action.yml

@@ -0,0 +1,14 @@
+name: Setup Python Runner
+description: Setup python-runner for tests
+inputs:
+  metabase-automation-token:
+    description: Token for accessing the python-runner repository
+    required: true
+runs:
+  using: composite
+  steps:
+    - name: Setup python-runner
+      run: ./.github/scripts/setup-python-runner.sh
+      shell: bash
+      env:
+        METABASE_AUTOMATION_USER_TOKEN: ${{ inputs.metabase-automation-token }}

.github/actions/test-driver/action.yml

@@ -9,29 +9,71 @@ inputs:
   build-static-viz:
     required: true
     default: false
+  logs-artifact-suffix:
+    required: false
+    description: Suffix to append to the test logs artifact name to avoid conflicts in matrix jobs
+
+outputs:
+  quarantine-skipped:
+    description: Whether the driver test was skipped due to quarantine
+    value: ${{ steps.check-driver.outputs.skip }}
 
 runs:
   using: "composite"
   steps:
+    - name: Check if driver is ignored
+      id: check-driver
+      shell: bash
+      run: |
+        # If DRIVERS env var is not set, proceed with tests
+        if [ -z "$DRIVERS" ]; then
+          echo "No DRIVERS env var set. Proceeding with tests."
+          echo "skip=false" >> $GITHUB_OUTPUT
+          exit 0
+        fi
+
+        # Fetch CI config and check if driver should be skipped
+        curl -sSf -o ci-test-config.json https://raw.githubusercontent.com/metabase/ci-test-config/refs/heads/master/ci-test-config.json || true
+
+        # Check if the driver is in the ignored list (gracefully handle missing/invalid config)
+        IS_IGNORED=$(jq --arg driver "$DRIVERS" '.ignored.drivers // [] | contains([$driver])' ci-test-config.json 2>/dev/null || echo "false")
+
+        if [ "$IS_IGNORED" = "true" ]; then
+          echo "⚠️  Driver '$DRIVERS' is in the ignored list. Skipping tests."
+          echo "View the ignored list here: https://github.com/metabase/ci-test-config/blob/master/ci-test-config.json"
+          echo "skip=true" >> $GITHUB_OUTPUT
+        else
+          echo "Driver '$DRIVERS' is not ignored. Proceeding with tests."
+          echo "skip=false" >> $GITHUB_OUTPUT
+        fi
+
     - name: Prepare back-end environment
+      if: steps.check-driver.outputs.skip != 'true'
       uses: ./.github/actions/prepare-backend
-      with:
-        m2-cache-key: driver
     - name: Prepare front-end environment
+      if: ${{ steps.check-driver.outputs.skip != 'true' && inputs.build-static-viz == 'true' }}
       uses: ./.github/actions/prepare-frontend
-      if: ${{ inputs.build-static-viz == 'true' }}
     - name: Build static viz frontend
+      if: ${{ steps.check-driver.outputs.skip != 'true' && inputs.build-static-viz == 'true' }}
       run: yarn build-static-viz
       shell: bash
-      if: ${{ inputs.build-static-viz == 'true' }}
       env:
         MB_EDITION: ee
     - name: Test database driver
+      if: steps.check-driver.outputs.skip != 'true'
       run: clojure -X:dev:ci:ee:ee-dev:drivers:drivers-dev:test ${{ inputs.test-args }}
       shell: bash
+      id: run-test
+      # Trunk Quarantine controls the final state of the run in the CI!
+      # Even if this step fails we let it pass, and then `trunk-io/analytics-uploader` determines
+      # whether to fail or to pass the whole job, based on the quarantine list.
+      continue-on-error: true
     - name: Publish Test Report (JUnit)
       uses: dorny/test-reporter@v1
-      if: failure()
+      # As per the documentation: https://docs.github.com/en/actions/reference/workflows-and-actions/contexts#steps-context
+      # test.outcome gives us access to the result before `continue-on-error` overrides it and sets it to 'success'.
+      # We want test reports even if the test step gets cancelled (e.g. times out), hence we match anything that is not a 'success'.
+      if: ${{ steps.run-test.outcome != 'success' && steps.check-driver.outputs.skip != 'true' }}
       with:
         path: "target/junit/**/*_test.xml"
         name: JUnit Test Report ${{ inputs.junit-name }}
@@ -40,9 +82,11 @@ runs:
         list-tests: failed
         fail-on-error: false
     - name: Upload Logs on Failure
-      if: always() && !success()
+      if: always() && steps.run-test.outcome != 'success' && steps.check-driver.outputs.skip != 'true'
       uses: actions/upload-artifact@v4
       with:
-        name: test-logs-${{ github.job }}
+        name: ${{ format('test-logs-{0}{1}', github.job, (inputs.logs-artifact-suffix == '' || inputs.logs-artifact-suffix == null) && '' || format('-{0}', inputs.logs-artifact-suffix)) }}
         path: logs/test-log.json
         retention-days: 1
+      # Failing to upload the artifact should never affect the outcome of the whole test run.
+      continue-on-error: true

.github/actions/update-cache/action.yml

@@ -0,0 +1,41 @@
+name: Update cache
+description: |
+  Update a GitHub Actions cache based on a provided path and key.
+  If a cache entry already exists for the given key, it is first removed.
+  This ensures immutability of cache entries, and prevents runtime errors.
+
+inputs:
+  path:
+    description: "The path(s) to cache, separated by newlines."
+    required: true
+  key:
+    description: "The cache key to use."
+    required: true
+
+runs:
+  using: "composite"
+  steps:
+    - name: Check if the cache entry exists
+      id: cache-lookup
+      uses: actions/cache/restore@v4
+      with:
+        path: ${{ inputs.path }}
+        key: ${{ inputs.key }}
+        lookup-only: true # Important bit: the action will not fetch anything, just check for the cache existence.
+    - name: Delete the cache entry if it exists
+      if: ${{ steps.cache-lookup.outputs.cache-hit == 'true' }}
+      run: |
+        echo "Cache entry found for key '${{ inputs.key }}'. Deleting it to ensure immutability."
+        gh api \
+          --method DELETE \
+          -H "Accept: application/vnd.github+json" \
+          -H "X-GitHub-Api-Version: 2022-11-28" \
+          /repos/${{ github.repository }}/actions/caches?key=${{ inputs.key }}
+      env:
+        GH_TOKEN: ${{ github.token }}
+      shell: bash
+    - name: Save the new cache entry
+      uses: actions/cache/save@v4
+      with:
+        path: ${{ inputs.path }}
+        key: ${{ inputs.key }}

.github/actions/upload-test-results/action.yml

@@ -20,7 +20,7 @@ inputs:
     required: true
   variant:
     required: false
-    default: ''
+    default: ""
 
 runs:
   using: "composite"
@@ -48,11 +48,12 @@ runs:
         aws s3 cp ${OUTPUT_FILE}.zip s3://$BUCKET/$DATE/$GITHUB_RUN_ID/$GITHUB_RUN_ATTEMPT/
 
     - name: Upload results to Trunk
-      if: ${{ always() && !(github.event_name == 'pull_request' && github.event.pull_request.draft == true) }}
+      if: always() # Important! This step must always run because it controls job's exit code.
       uses: trunk-io/analytics-uploader@main
       with:
         junit-paths: ${{ inputs.input-path }}
         variant: ${{ inputs.variant || inputs.output-name }}
         org-slug: metabase
+        dry-run: ${{ github.event_name == 'pull_request' && github.event.pull_request.draft == true }} # quarantine, but do not upload results
         token: ${{ inputs.trunk-api-token }}
-      continue-on-error: true
+        use-cache: true

.github/codeql/codeql-config.yml

@@ -1,3 +1,5 @@
+version: 1
+
 name: "Metabase CodeQL config"
 
 paths-ignore:

.github/file-paths.yaml

@@ -11,15 +11,19 @@ shared_specs: &shared_specs
   - "test/**/*.cljc"
   - "test/**/*.cljs"
 
-frontend_ci: &frontend_ci
+shared_ci: &shared_ci
   - ".github/actions/prepare-frontend/**"
   - ".github/actions/prepare-backend/**"
-  - ".github/workflows/frontend.yml"
+  - ".github/workflows/run-tests.yml"
 
-frontend_embedding_sdk_ci: &frontend_embedding_sdk_ci
-  - ".github/actions/prepare-frontend/**"
-  - ".github/actions/prepare-backend/**"
-  - ".github/workflows/*embedding-sdk*.yml"
+shared_e2e: &shared_e2e
+  - "e2e/runner/**"
+  - "e2e/snapshot-creators/**"
+  - "e2e/support/**"
+
+frontend_ci: &frontend_ci
+  - *shared_ci
+  - ".github/workflows/frontend.yml"
 
 frontend_configs: &frontend_configs
   - "yarn.lock"
@@ -41,14 +45,12 @@ frontend_sources: &frontend_sources
   - "enterprise/frontend/!(test)**/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
   - "frontend/!(test)**/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
 
+# Used only in .github/workflows/conventional-commit-pr-title-reminder.yml
 frontend_embedding_sources:
   - "enterprise/frontend/src/{embedding,embedding-*}/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
   - "enterprise/frontend/src/metabase-enterprise/{embedding,embedding-*}**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
 
-frontend_embedding_sdk_sources:
-  - *frontend_configs
-  - "enterprise/frontend/!(test)**/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
-
+# Used only in .github/workflows/embedding-sdk-labels-reminder.yml
 frontend_embedding_sdk_package_sources:
   - "enterprise/frontend/src/embedding-sdk-package/**/!(*.spec).{js,jsx,ts,tsx,css}"
 
@@ -76,8 +78,7 @@ frontend_loki_ci: &frontend_loki_ci
   - ".storybook/**"
 
 backend_ci: &backend_ci
-  - ".github/actions/prepare-frontend/**"
-  - ".github/actions/prepare-backend/**"
+  - *shared_ci
   - ".github/actions/test-driver/**"
   - ".github/workflows/backend.yml"
   - ".github/workflows/drivers.yml"
@@ -117,9 +118,9 @@ sources: &sources
   - *backend_sources
 
 e2e_ci: &e2e_ci
-  - ".github/actions/prepare-frontend/**"
-  - ".github/actions/prepare-backend/**"
+  - *shared_ci
   - ".github/actions/prepare-uberjar-artifact/**"
+  - ".github/actions/fetch-artifact/action.yml"
   - ".github/actions/e2e-prepare-containers/**"
   - ".github/actions/prepare-cypress/**"
   - ".github/workflows/e2e-*"
@@ -127,22 +128,27 @@ e2e_ci: &e2e_ci
 e2e_specs: &e2e_specs
   - e2e/!(test-host-app)**/*.cy.*.{js,ts}
 
-e2e_cross_version: &e2e_cross_version
-  - ".github/workflows/e2e-cross-version.yml"
-  - "e2e/test/scenarios/cross-version/**"
+embedding_sdk_ci: &embedding_sdk_ci
+  - *shared_ci
+  - ".github/actions/prepare-uberjar-artifact/**"
+  - ".github/actions/fetch-artifact/action.yml"
+  - ".github/actions/prepare-cypress/**"
+  - ".github/workflows/embedding-sdk*.yml"
 
-e2e_embedding_sdk:
-  - *frontend_embedding_sdk_ci
-  - *sources
-  - "e2e/component/**/*.cy.*.{js,ts}"
-
-e2e_embedding_sdk_compatibility:
+embedding_sdk_components:
   - *default
-  - *frontend_embedding_sdk_ci
+  - *embedding_sdk_ci
+  - *sources
+  - *shared_e2e
+  - "e2e/test-component/**/*.cy.*.{js,jsx,ts,tsx}"
+
+embedding_sdk_host_sample_apps:
+  - *default
+  - *embedding_sdk_ci
   - *frontend_sources
   - *backend_sources
+  - *shared_e2e
   - "e2e/embedding-sdk-host-apps/**"
-  - "e2e/runner/embedding-sdk/**"
   - "e2e/test-host-app/**/*.cy.*.{js,ts}"
 
 snowplow: &snowplow
@@ -170,6 +176,7 @@ yaml:
   - "**/*.yaml"
 
 codeql:
+  - ".github/workflows/codeql.yml"
   - "frontend/src/**"
   - "enterprise/frontend/src/**"
 

.github/scripts/average-e2e-timings.js

@@ -21,7 +21,6 @@ const collectAndAverageTimings = async ({
 
     const timingData = await downloadAndExtractArtifacts(
       artifacts,
-      token,
       github,
       context,
     );

.github/scripts/build-e2e-matrix.js

@@ -3,14 +3,8 @@
 // grouping some specific tests together, other tests are split into chunks
 
 const DEFAULT_SPEC_PATTERN = "./e2e/test/scenarios/**/*.cy.spec.*";
-const EMBEDDING_SDK_SPEC_PATTERN =
-  "./e2e/test/scenarios/embedding-sdk/**.cy.spec.*";
 
 const specialTestConfigs = [
-  {
-    name: "embedding-sdk",
-    specs: EMBEDDING_SDK_SPEC_PATTERN,
-  },
   {
     name: "oss-subset",
     edition: "oss",
@@ -18,6 +12,7 @@ const specialTestConfigs = [
     specs: DEFAULT_SPEC_PATTERN,
   },
   { name: "mongo", tags: "@mongo", specs: DEFAULT_SPEC_PATTERN },
+  { name: "python", tags: "@python", specs: DEFAULT_SPEC_PATTERN },
 ];
 
 /**

.github/scripts/build-e2e-matrix.unit.spec.js

@@ -21,10 +21,10 @@ describe("buildMatrix", () => {
   });
 
   it("should apply default specs path when empty entry is provided", () => {
-    const result = testBuildMatrix("", 5);
+    const result = testBuildMatrix("", 6);
 
-    expect(result.regularChunks).toBe(2);
-    expect(result.config.length).toBe(5);
+    expect(result.regularChunks).toBe(3);
+    expect(result.config.length).toBe(6);
   });
 
   it("should name regular test groups sequentially", () => {
@@ -43,9 +43,9 @@ describe("buildMatrix", () => {
 
     expect(result.regularChunks).toBe(0);
     expect(result.config.length).toBe(3);
-    expect(result.config[0].name).toBe("embedding-sdk");
-    expect(result.config[1].name).toBe("oss-subset");
-    expect(result.config[2].name).toBe("mongo");
+    expect(result.config[0].name).toBe("oss-subset");
+    expect(result.config[1].name).toBe("mongo");
+    expect(result.config[2].name).toBe("python");
   });
 
   it("should correctly calculate regularChunks for non-default spec pattern", () => {
@@ -83,9 +83,9 @@ describe("buildMatrix", () => {
   it("should return special test configs when default spec pattern is provided", () => {
     const result = testBuildMatrix("./e2e/test/scenarios/**/*.cy.spec.*", 50);
 
-    expect(result.config[47].name).toBe("embedding-sdk");
-    expect(result.config[48].name).toBe("oss-subset");
-    expect(result.config[49].name).toBe("mongo");
+    expect(result.config[47].name).toBe("oss-subset");
+    expect(result.config[48].name).toBe("mongo");
+    expect(result.config[49].name).toBe("python");
   });
 
   it("should not return special test configs when non-default spec pattern is provided", () => {

.github/scripts/setup-python-runner.sh

@@ -24,6 +24,7 @@ git clone --depth 1 https://${METABASE_AUTOMATION_USER_TOKEN}@github.com/metabas
 if [ "$SKIP_START" = false ]; then
   echo "Starting python-runner..."
   cd python-runner
-  make run-ci
+  make run-ci 1> ../python-runner.out.log 2> ../python-runner.err.log
+  make logs 1>> ../python-runner.out.log 2>> ../python-runner.err.log &
   cd ..
 fi

.github/team.json

@@ -32,20 +32,6 @@
         "lorem--ipsum"
       ]
     },
-    {
-      "name": "Workflows",
-      "label": ".Team/Workflows",
-      "members": [
-        "qnkhuat",
-        "piranha",
-        "crisptrutski",
-        "spalt08",
-        "stasgavrylov",
-        "uladzimirdev",
-        "kulyk",
-        "kamilmielnik"
-      ]
-    },
     {
       "name": "Admin Webapp",
       "label": ".Team/AdminWebapp",
@@ -58,7 +44,10 @@
       "label": ".Team/DevEx",
       "members": [
         "camsaul",
-        "escherize"
+        "dpsutton",
+        "escherize",
+        "fraserdrops",
+        "nemanjaglumac"
       ]
     },
      {
@@ -70,19 +59,10 @@
         "retro",
         "sloansparger",
         "somtom",
-        "wotbrew",
         "tsplude",
         "lbrdnk",
-        "bronsa"
-      ]
-    },
-    {
-      "name": "Product Growth",
-      "label": ".Team/ProductGrowth",
-      "members": [
-        "nemanjaglumac",
-        "chodorowicz",
-        "andsalves"
+        "piranha",
+        "crisptrutski"
       ]
     },
     {
@@ -111,7 +91,22 @@
         "edpaget",
         "deniskaber",
         "alxnddr",
-        "bpander"
+        "bpander",
+        "andsalves"
+      ]
+    },
+    {
+      "name": "Gadget",
+      "label": ".Team/Gadget",
+      "members": [
+        "nmezzopera",
+        "bronsa",
+        "wotbrew",
+        "qnkhuat",
+        "chodorowicz",
+        "kamilmielnik",
+        "stasgavrylov",
+        "uladzimirdev"
       ]
     }
   ]

.github/workflows/app-db.yml

@@ -255,12 +255,6 @@ jobs:
             env:
               enable-ssl-tests: 'true'
         job:
-          - name: Driver Tests
-            build-static-viz: false
-            test-args: >-
-              :only-tags [:mb/driver-tests]
-              :exclude-tags [:mb/transforms-python-test]
-            exclude-tag: ':mb/transforms-python-test'
           - name: Enterprise Tests
             build-static-viz: false
             test-args: >-

.github/workflows/auto-assign.yml

@@ -23,7 +23,7 @@ jobs:
         uses: actions/github-script@v7
         with:
           github-token: ${{ steps.generate-token.outputs.token }}
-          script: |
+          script: | #js
             const fs = require('fs');
             const teamConfig = JSON.parse(fs.readFileSync('.github/team.json', 'utf-8'));
             const prAuthor = context.payload.pull_request.user.login;
@@ -65,10 +65,3 @@ jobs:
               issue_number: prNumber,
               labels: [team.label]
             });
-
-            return team.projectUrl || null;
-      - uses: actions/add-to-project@v0.5.0
-        if: ${{ steps.auto-assign.outputs.result != 'null' }}
-        with:
-          project-url: ${{ steps.auto-assign.outputs.result }}
-          github-token: ${{ steps.generate-token.outputs.token }}

.github/workflows/backend-cloverage.yml

@@ -15,8 +15,6 @@ jobs:
       - uses: actions/checkout@v4
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "cloverage"
       - name: Build static viz frontend
         run: yarn build-static-viz
         env:

.github/workflows/backend.yml

@@ -21,8 +21,6 @@ jobs:
       - uses: actions/checkout@v4
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: 'kondo'
       - name: Run clj-kondo
         run: ./bin/mage kondo
 
@@ -35,8 +33,6 @@ jobs:
       - uses: actions/checkout@v4
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "eastwood"
       - run: clojure -X:dev:ee:ee-dev:drivers:drivers-dev:test:eastwood
         name: Run Eastwood linter
 
@@ -51,33 +47,33 @@ jobs:
         job:
           - name: Enterprise Tests
             edition: ee
-            build-static-viz: 'false'
+            test-group-name: ee
             test-args: >-
               :only '"enterprise/backend/test"'
           - name: EE App DB Tests (Part 1)
             edition: ee
-            build-static-viz: 'true'
+            test-group-name: ee-app-db-1
             test-args: >-
               :only '["test" ".clj-kondo/test"]'
               :partition/total 2
               :partition/index 0
           - name: EE App DB Tests (Part 2)
             edition: ee
-            build-static-viz: 'true'
+            test-group-name: ee-app-db-2
             test-args: >-
               :only '["test" ".clj-kondo/test"]'
               :partition/total 2
               :partition/index 1
           - name: OSS App DB Tests (Part 1)
             edition: oss
-            build-static-viz: 'true'
+            test-group-name: oss-app-db-1
             test-args: >-
               :only '["test" ".clj-kondo/test"]'
               :partition/total 2
               :partition/index 0
           - name: OSS App DB Tests (Part 2)
             edition: oss
-            build-static-viz: 'true'
+            test-group-name: oss-app-db-2
             test-args: >-
               :only '["test" ".clj-kondo/test"]'
               :partition/total 2
@@ -102,26 +98,15 @@ jobs:
     steps:
       - uses: actions/checkout@v4
       - name: Prepare front-end environment
-        if: ${{ matrix.job.build-static-viz == 'true' }}
         uses: ./.github/actions/prepare-frontend
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
         with:
-          m2-cache-key: be-tests
           java-version: "${{ matrix.java-version }}"
-      # Depending on which files changed, we might either have to build static-viz
-      # from scratch or download the previously built artifact
       - name: Build static-viz frontend
-        if: ${{ matrix.job.build-static-viz == 'true' && !inputs.skip }}
         run: yarn build-static-viz
         env:
           MB_EDITION: ${{ matrix.job.edition }}
-      - name: Download Static Viz Bundle Artifact
-        if: ${{ matrix.job.build-static-viz == 'true' && !inputs.skip }}
-        uses: actions/download-artifact@v4
-        with:
-          name: static-viz-bundle-${{ github.sha }}
-          path: resources/frontend_client/app/dist
 
       - name: "Test Java ${{ matrix.java-version }} ${{ matrix.job.name }}"
         id: run-java-tests
@@ -129,6 +114,10 @@ jobs:
           clojure -X:dev:ci:test:${{ matrix.job.edition }}:${{ matrix.job.edition }}-dev
           :exclude-tags '[:mb/driver-tests ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
           ${{ matrix.job.test-args }}
+        # Trunk Quarantine controls the final state of the run in the CI!
+        # Even if this step fails we let it pass, and then `trunk-io/analytics-uploader` determines
+        # whether to fail or to pass the whole job, based on the quarantine list.
+        continue-on-error: true
 
       - name: Upload Test Results
         uses: ./.github/actions/upload-test-results
@@ -157,7 +146,7 @@ jobs:
         if: always() && !success()
         uses: actions/upload-artifact@v4
         with:
-          name: test-logs-${{ matrix.java-version }}-${{ matrix.job.edition }}-${{ github.job }}
+          name: test-logs-${{ matrix.java-version }}-${{ matrix.job.edition }}-${{ github.job }}-${{ matrix.job.test-group-name }}
           path: logs/test-log.json
           retention-days: 1
 
@@ -175,8 +164,6 @@ jobs:
       - uses: actions/checkout@v4
       - name: Prepare backend
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: be-check
       # TODO -- we should probably also check WITHOUT ee -- do a check with only OSS namespaces on the classpath
       - name: Check namespaces
         run: clojure -M:ee:drivers:check
@@ -190,8 +177,6 @@ jobs:
       - uses: actions/checkout@v4
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: 'cljfmt'
       - name: Run cljfmt
         run: ./bin/mage cljfmt-all --force-check
 

.github/workflows/bugs-triage-slack-notification.yml

@@ -1,52 +0,0 @@
-name: Bugs Slack Notification
-
-on:
-  issues:
-    types:
-      - labeled
-jobs:
-  on-issue-labeled:
-    runs-on: ubuntu-22.04
-    timeout-minutes: 5
-    if: |
-      github.event.label.name == '.Needs Triage' && contains(github.event.issue.labels.*.name, 'Type:Bug')
-    steps:
-      - name: Setting title
-        uses: actions/github-script@v7
-        id: vars
-        with:
-          script: |
-            core.setOutput('issue_title', ${{ toJson(github.event.issue.title) }}.replaceAll(/"/g, '\\"'));
-      - name: Check for labels containing team name
-        id: check_labels
-        run: |
-          labels_json=$(sed 's/&/\$/g' <<< '${{ toJson(github.event.issue.labels) }}')
-          echo "has_team_label=false" >> "$GITHUB_OUTPUT"
-          for label in "$labels_json"
-          do
-            if [[ $label == *".Team"* ]]; then
-              echo "Label contains the team name."
-              echo "has_team_label=true" >> "$GITHUB_OUTPUT"
-              break
-            fi
-          done
-      - name: Send the issue ${{ github.event.issue.number }} to Slack
-        id: slack
-        if: steps.check_labels.outputs.has_team_label == 'false'
-        uses: slackapi/slack-github-action@v1.26.0
-        with:
-          payload: |
-            {
-              "blocks": [
-                {
-                  "type": "section",
-                  "text": {
-                    "type": "mrkdwn",
-                    "text": "${{ steps.vars.outputs.issue_title }}\n${{ github.event.issue.html_url }}"
-                  }
-                }
-              ]
-            }
-        env:
-          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_BUG_TRIAGE_WEBHOOK_URL }}
-          SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK

.github/workflows/build-scripts.yml

@@ -16,8 +16,6 @@ jobs:
     - name: Prepare back-end environment
       timeout-minutes: 5
       uses: ./.github/actions/prepare-backend
-      with:
-        m2-cache-key: 'build-scripts'
 
     - name: Run build and release script tests
       run: clojure -X:dev:drivers:build:build-dev:build-test:ci

.github/workflows/cache-generator.yml

@@ -0,0 +1,132 @@
+# This workflow is the single source of truth for generating and updating various caches used by our CI pipeline.
+# It sets the fresh cache each Monday but provides a manual trigger to update it on-demand.
+#
+# All other jobs only restore from this cache; *they never update it*!
+# They should always use `actions/cache/restore@v4` because `actions/cache@v4` would attempt to save the cache.
+#
+# Our agreed-upon strategy is to use a deterministic cache key based on the week of the year.
+# Jobs can utilize the prior week's cache if the current week's cache is not found.
+#
+#
+# PRINCIPLES:
+#   - The cache is never wrong!
+#   - Caches are immutable once created.
+#   - Caches are for speeding up builds, not for correctness.
+#   - It is faster to download a large cache than to re-resolve missing dependencies.
+#
+#
+# DESIGN:
+#
+# 1. Prepare
+#     - A job first prepares the local environment (e.g., installs JDK, Clojure, Node.js, etc.)
+#     - This fetches the dependencies from the cache (if it exists) and warms up the local cache for this workflow.
+# 2. Resolve
+#     - The next step pre-resolves all dependencies, ensuring that the local cache is fully populated and up-to-date.
+# 3. Update
+#     - The final step saves the local cache back to GitHub Actions cache with a deterministic key.
+#     - If a cache entry already exists for that key, it is deleted first to ensure immutability.
+#
+name: Cache Generator
+
+on:
+  schedule:
+    - cron: "0 3 * * 1" # Mondays at 03:00 UTC
+  workflow_dispatch: # manually update the cache
+
+concurrency:
+  group: cache-generator
+  cancel-in-progress: true
+
+jobs:
+  get-cache-keys:
+    runs-on: ubuntu-22.04
+    steps:
+      - uses: actions/checkout@v4
+      - name: Get cache keys
+        uses: ./.github/actions/get-cache-keys
+        id: get-cache-keys
+    outputs:
+      m2-cache-key: ${{ steps.get-cache-keys.outputs.m2-cache-key }}
+      node-cache-key: ${{ steps.get-cache-keys.outputs.node-cache-key }}
+      eslint-cache-key: ${{ steps.get-cache-keys.outputs.eslint-cache-key }}
+      cypress-cache-key: ${{ steps.get-cache-keys.outputs.cypress-cache-key }}
+
+  generate-m2-cache:
+    runs-on: ubuntu-22.04
+    timeout-minutes: 15
+    needs: get-cache-keys
+    env:
+      M2_CACHE_KEY: ${{ needs.get-cache-keys.outputs.m2-cache-key }}
+    steps:
+      - uses: actions/checkout@v4
+      - name: Prepare backend
+        uses: ./.github/actions/prepare-backend
+
+      # Add or remove aliases to affect the cache size and its contents.
+      # Adjust to taste, then redeploy this workflow.
+      - name: Pre-resolve Clojure dependencies
+        run: clojure -A:build:dev:ee:ee-dev:drivers:drivers-dev:cljs -P
+
+      - name: Update M2 cache
+        uses: ./.github/actions/update-cache
+        with:
+          path: |
+            ~/.m2
+            ~/.gitlibs
+          key: ${{ env.M2_CACHE_KEY }}
+
+  generate-frontend-caches:
+    runs-on: ubuntu-22.04
+    timeout-minutes: 15
+    needs: get-cache-keys
+    env:
+      NODE_CACHE_KEY: ${{ needs.get-cache-keys.outputs.node-cache-key }}
+      ESLINT_CACHE_KEY: ${{ needs.get-cache-keys.outputs.eslint-cache-key }}
+      CYPRESS_CACHE_KEY: ${{ needs.get-cache-keys.outputs.cypress-cache-key }}
+    steps:
+      - uses: actions/checkout@v4
+      # `prepare-frontend` action (1) prepares Node.js and (2) fetches the node_modules cache if it exists.
+      # It then (3) runs `yarn install --frozen-lockfile --prefer-offline` which ensures the local cache is fully populated.
+      # IMPORTANT: Caching node_modules after patches are already applied will break any PR that attempts to either:
+      #   - upgrade a patched library
+      #   - change a patch
+      # Hence why we explicitly exclude patches from the cache (controlled by the `apply-patches` action input).
+      - name: Install front-end dependencies
+        uses: ./.github/actions/prepare-frontend
+        with:
+          apply-patches: "false"
+
+      - name: Get yarn cache directory
+        id: yarn-cache-dir
+        run: |
+          YARN_CACHE_DIR=$(yarn cache dir)
+          echo "Yarn cache directory: $YARN_CACHE_DIR"
+          echo "dir=$YARN_CACHE_DIR" >> $GITHUB_OUTPUT
+
+      - name: Ensure that Cypress executable is ready
+        run: |
+          yarn cypress install
+          yarn cypress cache path
+          yarn cypress cache list
+          yarn cypress verify
+        shell: bash
+
+      - name: Update node_modules cache
+        uses: ./.github/actions/update-cache
+        with:
+          path: |
+            ${{ steps.yarn-cache-dir.outputs.dir }}
+            node_modules
+          key: ${{ env.NODE_CACHE_KEY }}
+
+      - name: Update eslint cache
+        uses: ./.github/actions/update-cache
+        with:
+          path: ${{ github.workspace}}/.eslintcache
+          key: ${{ env.ESLINT_CACHE_KEY }}
+
+      - name: Update Cypress cache
+        uses: ./.github/actions/update-cache
+        with:
+          path: ~/.cache/Cypress
+          key: ${{ env.CYPRESS_CACHE_KEY }}

.github/workflows/chromatic.yml

@@ -40,8 +40,6 @@ jobs:
         uses: ./.github/actions/prepare-frontend
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "cljs"
       - name: Compile CLJS
         run: NODE_ENV=development yarn build-pure:cljs
       - name: Publish to Chromatic

.github/workflows/claude-code-review.yaml

@@ -0,0 +1,101 @@
+name: "Claude Code Review"
+
+on:
+  pull_request:
+    types:
+      - closed
+      - synchronize
+      - labeled
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}-claude-code-review
+  cancel-in-progress: true
+
+jobs:
+  review:
+    if: |
+      (
+        github.event.action == 'closed' &&
+        github.event.pull_request.merged == true
+      ) ||
+      contains(github.event.pull_request.labels.*.name, 'Claude Code Review')
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      pull-requests: write
+
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 1
+          # Make sure the value of GITHUB_TOKEN will not be persisted in repo's config
+          persist-credentials: false
+
+      # Apparently comments on specific lines and top-level comments about the PR as a whole are TWO TOTALLY DIFFERENT
+      # TYPES OF THINGS. So we have to fetch and delete separately.
+      - name: Delete old Claude PR review comments
+        env:
+          GH_TOKEN: ${{ github.token }}
+          REPO: ${{ github.repository }}
+          PR_NUMBER: ${{ github.event.pull_request.number }}
+        shell: bash
+        run: |
+          # Get old comment IDs from the bot
+          old_line_comment_ids=$(gh api "/repos/$REPO/pulls/$PR_NUMBER/comments" --jq '.[] | select(.user.login=="metabase-bot[bot]") | .id')
+          echo "Deleting line comments: $old_line_comment_ids"
+          # Delete each comment
+          for id in $old_line_comment_ids; do
+            gh api -X DELETE "/repos/$REPO/pulls/comments/$id"
+            echo "Deleted line comment $id"
+          done
+          # Get old top-level comment IDs (technically "issue" comments on the PR?)
+          old_top_level_comment_ids=$(gh api "/repos/$REPO/issues/$PR_NUMBER/comments" --jq '.[] | select(.user.login=="metabase-bot[bot]") | .id')
+          echo "Deleting top-level comments: $old_top_level_comment_ids"
+          # Delete each comment
+          for id in $old_top_level_comment_ids; do
+            gh api -X DELETE "/repos/$REPO/issues/comments/$id"
+            echo "Deleted top-level comment $id"
+          done
+
+      - name: Generate GitHub App Token for Claude
+        uses: actions/create-github-app-token@v1
+        id: claude-app-token
+        with:
+          app-id: ${{ secrets.METABASE_BOT_APP_ID }}
+          private-key: ${{ secrets.METABASE_BOT_APP_PRIVATE_KEY }}
+          permission-contents: read       # Required for reading repository files and PR diffs
+          permission-pull-requests: write # Required for creating inline comments and PR comments
+
+      - name: Fetch Community Clojure Style Guide
+        run: >-
+          curl https://raw.githubusercontent.com/bbatsov/clojure-style-guide/refs/heads/master/README.adoc -o CLOJURE_STYLE_GUIDE.adoc
+
+      - uses: anthropics/claude-code-action@v1
+        timeout-minutes: 10
+        with:
+          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
+          github_token: ${{ steps.claude-app-token.outputs.token }}
+          prompt: |
+            REPO: ${{ github.repository }}
+            PR NUMBER: ${{ github.event.pull_request.number }}
+
+            Review this pull request with a focus on:
+
+            - Compliance with project coding standards and conventions
+            - Code quality and best practices
+            - Clear and correct docstrings
+            - Clear and correct comments explaining what the code is doing
+
+            DO NOT post any comments confirming that something is correct or congratulating someone for following our
+            guidelines -- only post comments for style violations or concerns.
+
+            Follow additional guidelines in the "Code Review Guidelines" section of CLAUDE.md .
+
+            Note: The PR branch is already checked out in the current working directory.
+
+            Use `gh pr comment` for top-level feedback.
+            Use `mcp__github_inline_comment__create_inline_comment` to highlight specific code issues.
+            Only post GitHub comments - don't submit review text as messages.
+
+          claude_args: |
+            --allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*),ReadFile(CLOJURE_STYLE_GUIDE.adoc)"

.github/workflows/cljs.yml

@@ -38,7 +38,5 @@ jobs:
       uses: ./.github/actions/prepare-frontend
     - name: Prepare back-end environment
       uses: ./.github/actions/prepare-backend
-      with:
-        m2-cache-key: 'cljs'
     - name: Run Cljs tests
       run: yarn run test-cljs

.github/workflows/codeql.yml

@@ -30,9 +30,10 @@ jobs:
     - name: Checkout repository
       uses: actions/checkout@v4
     - name: Initialize CodeQL
-      uses: github/codeql-action/init@v2
+      uses: github/codeql-action/init@v4
       with:
         languages: javascript
         config-file: ./.github/codeql/codeql-config.yml
+        trap-caching: false
     - name: Perform CodeQL Analysis
-      uses: github/codeql-action/analyze@v2
+      uses: github/codeql-action/analyze@v4

.github/workflows/codespell.yml

@@ -0,0 +1,19 @@
+name: Codespell
+
+on:
+  push:
+    branches:
+      - 'master'
+      - 'release-**'
+  pull_request:
+
+jobs:
+  codespell:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: codespell-project/actions-codespell@v1
+        with:
+          # keep this list in sync with the list in `mage/src/mage/codespell.yml`
+          path: >-
+            docs

.github/workflows/coverage.yml

@@ -1,25 +1,28 @@
-name: Test coverage
+name: Frontend Test Coverage
 
 on:
+  workflow_dispatch:
   schedule:
     - cron: '40 1 * * *' # every day at 1:40
 
 jobs:
   fe-test-coverage:
     runs-on: ubuntu-22.04
-    timeout-minutes: 30
+    timeout-minutes: 60
     steps:
       - uses: actions/checkout@v4
       - name: Prepare front-end environment
         uses: ./.github/actions/prepare-frontend
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "cljs"
       - name: Run frontend unit tests
-        run: yarn run test-unit --silent --coverage
+        run: |
+          yarn run test-unit \
+            --silent \
+            --coverage \
+            --testTimeout=60000
       - name: Upload coverage to codecov.io
-        uses: codecov/codecov-action@v4
+        uses: codecov/codecov-action@v5
         with:
           files: ./coverage/lcov.info
           flags: front-end

.github/workflows/docs-generate-base.yml

@@ -9,10 +9,6 @@ on:
         description: "The branch to check out and create a PR against"
         required: true
         type: string
-      m2_cache_key_suffix:
-        description: "A unique suffix for the Maven cache key"
-        required: true
-        type: string
     secrets:
       automation_token:
         description: "A token with permissions to create branches and PRs"
@@ -37,8 +33,6 @@ jobs:
 
       - name: Prepare backend environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "auto-update-docs-${{ inputs.m2_cache_key_suffix }}"
 
       - name: Generate environment variables documentation
         id: generate_env_docs

.github/workflows/docs-master-generate.yml

@@ -11,6 +11,5 @@ jobs:
     uses: ./.github/workflows/docs-generate-base.yml
     with:
       target_branch: "master"
-      m2_cache_key_suffix: "master"
     secrets:
       automation_token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}

.github/workflows/docs-release-generate.yml

@@ -11,6 +11,5 @@ jobs:
     uses: ./.github/workflows/docs-generate-base.yml
     with:
       target_branch: "release-x.56.x"
-      m2_cache_key_suffix: "release"
     secrets:
       automation_token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}

.github/workflows/docs_bump_detected.yml

@@ -1,44 +0,0 @@
-name: Docs Bump Detected
-
-on:
-  pull_request_target:
-    types:
-      - opened
-      - synchronize
-      - reopened
-    branches:
-      - 'master'
-      - 'release-x.[0-9]+.x'
-    paths:
-      - 'docs/**'
-
-jobs:
-  push_workflow_to_docs_metabase_github_io_job:
-    runs-on: ubuntu-latest
-    env:
-      SOURCE_BRANCH: ${{ github.head_ref || github.ref  }}
-      TARGET_BRANCH: ${{ github.event.pull_request.base.ref }}
-      PR_NUMBER: ${{ github.event.pull_request.number }}
-    steps:
-      - name: Report changed docs files exist
-        run: |
-          echo "A PR with changes in the docs directory was detected."
-          echo "Sending workflow trigger to create/update the corresponding PR in docs repo."
-          echo "source branch: $SOURCE_BRANCH"
-          echo "target branch: $TARGET_BRANCH"
-          echo "pr number: $PR_NUMBER"
-
-      - name: Trigger repo dispatch
-        uses: actions/github-script@v7
-        with:
-          github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
-          script: | #js
-              github.rest.repos.createDispatchEvent({
-                owner: "${{ github.repository_owner }}",
-                repo: "docs.metabase.github.io",
-                event_type: "docs_update",
-                client_payload: {
-                  dispatch_type: "docs_update",
-                  source_branch: process.env.SOURCE_BRANCH,
-                  target_branch: process.env.TARGET_BRANCH,
-                  pr_number: process.env.PR_NUMBER}});

.github/workflows/docs_close_detected.yml

@@ -1,43 +0,0 @@
-name: Docs Close Detected
-
-on:
-  pull_request:
-    types:
-      - closed
-    branches:
-      - 'master'
-      - 'release-x.[0-9]+.x'
-    paths:
-      - 'docs/**'
-
-jobs:
-  push_and_merge_workflow_to_docs_metabase_github_io_job:
-    if: github.event.pull_request.merged == false
-    runs-on: ubuntu-latest
-    env:
-      SOURCE_BRANCH: ${{ github.head_ref || github.ref  }}
-      TARGET_BRANCH: ${{ github.event.pull_request.base.ref }}
-      PR_NUMBER: ${{ github.event.pull_request.number }}
-    steps:
-      - name: Report changed docs files exist
-        run: |
-          echo "A PR with changes to the docs directory was closed."
-          echo "Sending workflow trigger to close corresponding PR in docs repo."
-          echo "source branch: $SOURCE_BRANCH"
-          echo "target branch: $TARGET_BRANCH"
-          echo "pr number: $PR_NUMBER"
-
-      - name: Trigger repo dispatch
-        uses: actions/github-script@v7
-        with:
-          github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
-          script: | #js
-            github.rest.repos.createDispatchEvent({
-                owner: "${{ github.repository_owner }}",
-                repo: "docs.metabase.github.io",
-                event_type: "docs_close",
-                client_payload: {
-                  dispatch_type: "docs_close",
-                  source_branch: process.env.SOURCE_BRANCH,
-                  target_branch: process.env.TARGET_BRANCH,
-                  pr_number: process.env.PR_NUMBER}});

.github/workflows/docs_merge_detected.yml

@@ -1,43 +0,0 @@
-name: Docs Merge Detected
-
-on:
-  pull_request:
-    types:
-      - closed
-    branches:
-      - 'master'
-      - 'release-x.[0-9]+.x'
-    paths:
-      - 'docs/**'
-
-jobs:
-  push_and_merge_workflow_to_docs_metabase_github_io_job:
-    if: github.event.pull_request.merged == true
-    runs-on: ubuntu-latest
-    env:
-      SOURCE_BRANCH: ${{ github.head_ref || github.ref  }}
-      TARGET_BRANCH: ${{ github.event.pull_request.base.ref }}
-      PR_NUMBER: ${{ github.event.pull_request.number }}
-    steps:
-      - name: Report changed docs files exist
-        run: |
-          echo "A PR with changes to the docs directory was merged."
-          echo "Sending workflow trigger to merge corresponding PR in docs repo."
-          echo "source branch: $SOURCE_BRANCH"
-          echo "target branch: $TARGET_BRANCH"
-          echo "pr number: $PR_NUMBER"
-
-      - name: Trigger repo dispatch
-        uses: actions/github-script@v7
-        with:
-          github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
-          script: | #js
-            github.rest.repos.createDispatchEvent({
-                owner: "${{ github.repository_owner }}",
-                repo: "docs.metabase.github.io",
-                event_type: "docs_merge",
-                client_payload: {
-                  dispatch_type: "docs_merge",
-                  source_branch: process.env.SOURCE_BRANCH,
-                  target_branch: process.env.TARGET_BRANCH,
-                  pr_number: process.env.PR_NUMBER}});

.github/workflows/drivers.yml

@@ -85,6 +85,7 @@ jobs:
     steps:
     - uses: actions/checkout@v4
     - name: Test Athena driver
+      id: test-driver
       uses: ./.github/actions/test-driver
       with:
         junit-name: 'be-tests-athena-ee'
@@ -93,7 +94,7 @@ jobs:
           :exclude-tags [:mb/transforms-python-test]
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: always()
+      if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}
@@ -135,17 +136,19 @@ jobs:
     - uses: actions/checkout@v4
     - name: Setup python-runner
       if: ${{ matrix.job.needs-python-runner == true }}
-      env:
-        METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
-      run: ./.github/scripts/setup-python-runner.sh
+      uses: ./.github/actions/setup-python-runner
+      with:
+        metabase-automation-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
     - name: Test BigQuery Cloud SDK driver
+      id: test-driver
       uses: ./.github/actions/test-driver
       with:
         junit-name: 'be-tests-bigquery-cloud-sdk-ee'
         test-args: ${{ matrix.job.test-args }}
+        logs-artifact-suffix: ${{ matrix.job.name }}
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: always()
+      if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}
@@ -154,6 +157,12 @@ jobs:
         aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
         aws-region: ${{ vars.AWS_REGION }}
         trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
+    - name: Cleanup python-runner
+      if: ${{ always() && matrix.job.needs-python-runner == true }}
+      uses: ./.github/actions/cleanup-python-runner
+      with:
+        upload-logs: ${{ steps.test-driver.outcome == 'failure' }}
+        logs-artifact-suffix: ${{ matrix.job.name }} ${{ matrix.version.name }}
 
   be-tests-clickhouse-ee:
     if: ${{ !inputs.skip }}
@@ -183,9 +192,9 @@ jobs:
     - uses: actions/checkout@v4
     - name: Setup python-runner
       if: ${{ matrix.job.needs-python-runner == true }}
-      env:
-        METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
-      run: ./.github/scripts/setup-python-runner.sh
+      uses: ./.github/actions/setup-python-runner
+      with:
+        metabase-automation-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
     - name: Add ClickHouse TLS instance to /etc/hosts
       run: |
         sudo echo "127.0.0.1 server.clickhouseconnect.test" | sudo tee -a /etc/hosts
@@ -202,13 +211,15 @@ jobs:
           clickhouse_cluster_node2
           nginx
     - name: Test Clickhouse driver
+      id: test-driver
       uses: ./.github/actions/test-driver
       with:
         junit-name: 'be-tests-clickhouse-ee'
         test-args: ${{ matrix.job.test-args }}
+        logs-artifact-suffix: ${{ matrix.job.name }}
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: always()
+      if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}
@@ -217,6 +228,12 @@ jobs:
         aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
         aws-region: ${{ vars.AWS_REGION }}
         trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
+    - name: Cleanup python-runner
+      if: ${{ always() && matrix.job.needs-python-runner == true }}
+      uses: ./.github/actions/cleanup-python-runner
+      with:
+        upload-logs: ${{ steps.test-driver.outcome == 'failure' }}
+        logs-artifact-suffix: ${{ matrix.job.name }} ${{ matrix.version.name }}
 
   be-tests-druid-ee:
     needs: determine-driver-skips
@@ -237,6 +254,7 @@ jobs:
     steps:
     - uses: actions/checkout@v4
     - name: Test Druid driver
+      id: test-driver
       uses: ./.github/actions/test-driver
       with:
         junit-name: 'be-tests-druid-ee'
@@ -245,7 +263,7 @@ jobs:
           :exclude-tags [:mb/transforms-python-test]
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: always()
+      if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}
@@ -274,6 +292,7 @@ jobs:
     steps:
     - uses: actions/checkout@v4
     - name: Test Druid JDBC driver
+      id: test-driver
       uses: ./.github/actions/test-driver
       with:
         junit-name: 'be-tests-druid-jdbc-ee'
@@ -282,7 +301,7 @@ jobs:
           :exclude-tags [:mb/transforms-python-test]
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: always()
+      if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}
@@ -311,6 +330,7 @@ jobs:
     steps:
     - uses: actions/checkout@v4
     - name: Test Databricks driver
+      id: test-driver
       uses: ./.github/actions/test-driver
       with:
         junit-name: 'be-tests-databricks-ee'
@@ -319,7 +339,7 @@ jobs:
           :exclude-tags [:mb/transforms-python-test]
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: always()
+      if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}
@@ -349,6 +369,7 @@ jobs:
     steps:
     - uses: actions/checkout@v4
     - name: Test Databricks driver with multi-catalog
+      id: test-driver
       uses: ./.github/actions/test-driver
       with:
         junit-name: 'be-tests-databricks-multi-catalog-ee'
@@ -357,7 +378,7 @@ jobs:
           :exclude-tags [:mb/transforms-python-test]
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: always()
+      if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}
@@ -376,11 +397,8 @@ jobs:
     name: H2
     steps:
     - uses: actions/checkout@v4
-    - name: Setup python-runner
-      env:
-        METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
-      run: ./.github/scripts/setup-python-runner.sh
     - name: Test H2 driver
+      id: test-driver
       uses: ./.github/actions/test-driver
       with:
         junit-name: 'be-tests-h2'
@@ -389,7 +407,7 @@ jobs:
           :exclude-tags [:mb/transforms-python-test]
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: always()
+      if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}
@@ -431,7 +449,7 @@ jobs:
           - name: Driver Tests
             test-args: >-
               :only-tags [:mb/driver-tests]
-              :exclude-tags [:mb/transforms-python-test]
+            exclude-tag: ':mb/transforms-python-test'
           - name: Transforms Python Tests
             test-args: >-
               :only-tags [:mb/transforms-python-test]
@@ -491,20 +509,22 @@ jobs:
     steps:
       - uses: actions/checkout@v4
       - name: Setup python-runner
-      #if: ${{ matrix.job.needs-python-runner == true }}
-        env:
-          METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
-        run: ./.github/scripts/setup-python-runner.sh
+        if: ${{ matrix.job.needs-python-runner == true }}
+        uses: ./.github/actions/setup-python-runner
+        with:
+          metabase-automation-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
       - name: Test ${{ matrix.version.name }}
+        id: test-driver
         uses: ./.github/actions/test-driver
         with:
           junit-name: ${{ matrix.version.junit-name }}
+          logs-artifact-suffix: ${{ matrix.version.name }}
           test-args: >-
             ${{ matrix.job.test-args }}
             :exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
       - name: Upload Test Results
         uses: ./.github/actions/upload-test-results
-        if: always()
+        if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
         with:
           input-path: ./target/junit/
           output-name: ${{ github.job }}
@@ -513,6 +533,12 @@ jobs:
           aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
           aws-region: ${{ vars.AWS_REGION }}
           trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
+      - name: Cleanup python-runner
+        if: always()
+        uses: ./.github/actions/cleanup-python-runner
+        with:
+          upload-logs: ${{ steps.test-driver.outcome == 'failure' }}
+          logs-artifact-suffix: ${{ matrix.job.name }} ${{ matrix.version.name }}
 
   be-tests-mongo:
     needs: determine-driver-skips
@@ -556,17 +582,19 @@ jobs:
     - uses: actions/checkout@v4
     - name: Setup python-runner
       if: ${{ matrix.job.needs-python-runner == true }}
-      env:
-        METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
-      run: ./.github/scripts/setup-python-runner.sh
+      uses: ./.github/actions/setup-python-runner
+      with:
+        metabase-automation-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
     - name: Test ${{ matrix.version.name }}
+      id: test-driver
       uses: ./.github/actions/test-driver
       with:
         junit-name: ${{ matrix.version.junit-name }}
         test-args: ${{ matrix.job.test-args }}
+        logs-artifact-suffix: ${{ matrix.job.name }}
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: always()
+      if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}
@@ -575,6 +603,12 @@ jobs:
         aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
         aws-region: ${{ vars.AWS_REGION }}
         trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
+    - name: Cleanup python-runner
+      if: ${{ always() && matrix.job.needs-python-runner == true }}
+      uses: ./.github/actions/cleanup-python-runner
+      with:
+        upload-logs: ${{ steps.test-driver.outcome == 'failure' }}
+        logs-artifact-suffix: ${{ matrix.job.name }} ${{ matrix.version.name }}
 
   be-tests-mongo-ssl:
     needs: determine-driver-skips
@@ -623,6 +657,7 @@ jobs:
           curl https://raw.githubusercontent.com/metabase/metabase-qa/master/dbs/mongo/certificates/metaca.crt \
           -o ./test_resources/ssl/mongo/metaca.crt
       - name: Test ${{ matrix.version.name }}
+        id: test-driver
         uses: ./.github/actions/test-driver
         with:
           junit-name: ${{ matrix.version.junit-name }}
@@ -631,7 +666,7 @@ jobs:
             :only-tags [:mb/driver-tests]
       - name: Upload Test Results
         uses: ./.github/actions/upload-test-results
-        if: always()
+        if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
         with:
           input-path: ./target/junit/
           output-name: ${{ github.job }}
@@ -658,6 +693,7 @@ jobs:
     steps:
     - uses: actions/checkout@v4
     - name: Test MongoDB driver (Sharded cluster)
+      id: test-driver
       uses: ./.github/actions/test-driver
       with:
         junit-name: 'be-tests-mongo-sharded-cluster-ee'
@@ -665,7 +701,7 @@ jobs:
           :only metabase.driver.mongo.sharded-cluster-test
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: always()
+      if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}
@@ -730,15 +766,17 @@ jobs:
     steps:
     - uses: actions/checkout@v4
     - name: Test ${{ matrix.version.name }}
+      id: test-driver
       uses: ./.github/actions/test-driver
       with:
         junit-name: ${{ matrix.version.junit-name }}
+        logs-artifact-suffix: ${{ matrix.version.name }}
         test-args: >-
           :only-tags [:mb/driver-tests]
           :exclude-tags [:mb/transforms-python-test]
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: always()
+      if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}
@@ -770,7 +808,7 @@ jobs:
           - name: Driver Tests
             test-args: >-
               :only-tags [:mb/driver-tests]
-              :exclude-tags [:mb/transforms-python-test]
+            exclude-tag: ':mb/transforms-python-test'
           - name: Transforms Python Tests
             test-args: >-
               :only-tags [:mb/transforms-python-test]
@@ -817,11 +855,12 @@ jobs:
     steps:
       - uses: actions/checkout@v4
       - name: Setup python-runner
-      #if: ${{ matrix.job.needs-python-runner == true }}
-        env:
-          METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
-        run: ./.github/scripts/setup-python-runner.sh
+        if: ${{ matrix.job.needs-python-runner == true }}
+        uses: ./.github/actions/setup-python-runner
+        with:
+          metabase-automation-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
       - name: Test ${{ matrix.version.name }} (${{ matrix.job.name }})
+        id: test-driver
         uses: ./.github/actions/test-driver
         with:
           junit-name: 'be-tests-${{ matrix.version.junit-name }}'
@@ -830,7 +869,7 @@ jobs:
             :exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
       - name: Upload Test Results
         uses: ./.github/actions/upload-test-results
-        if: always()
+        if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
         with:
           input-path: ./target/junit/
           output-name: ${{ github.job }}
@@ -839,6 +878,12 @@ jobs:
           aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
           aws-region: ${{ vars.AWS_REGION }}
           trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
+      - name: Cleanup python-runner
+        if: always()
+        uses: ./.github/actions/cleanup-python-runner
+        with:
+          upload-logs: ${{ steps.test-driver.outcome == 'failure' }}
+          logs-artifact-suffix: ${{ matrix.job.name }} ${{ matrix.version.name }}
 
   be-tests-presto-jdbc-ee:
     needs: determine-driver-skips
@@ -892,6 +937,7 @@ jobs:
         -file /tmp/presto-ssl-ca.der
         -trustcacerts
     - name: Test Presto JDBC driver
+      id: test-driver
       uses: ./.github/actions/test-driver
       with:
         junit-name: 'be-tests-presto-jdbc-ee'
@@ -900,7 +946,7 @@ jobs:
           :exclude-tags [:mb/transforms-python-test]
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: always()
+      if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}
@@ -923,12 +969,22 @@ jobs:
             test-args: >-
               :only-tags [:mb/upload-tests]
               :fail-fast? true
-          - name: Redshift (Excluding Upload Tests)
+          - name: Redshift (Excluding Upload Tests) Part 1
             skip: false
             test-args: >-
               :only-tags [:mb/driver-tests]
               :exclude-tags '[:mb/upload-tests :mb/transforms-python-test]'
               :fail-fast? true
+              :partition/total 2
+              :partition/index 0
+          - name: Redshift (Excluding Upload Tests) Part 2
+            skip: false
+            test-args: >-
+              :only-tags [:mb/driver-tests]
+              :exclude-tags '[:mb/upload-tests :mb/transforms-python-test]'
+              :fail-fast? true
+              :partition/total 2
+              :partition/index 1
           - name: Redshift Transforms Python Tests
             skip: false
             test-args: >-
@@ -949,19 +1005,21 @@ jobs:
     - uses: actions/checkout@v4
     - name: Setup python-runner
       if: ${{ matrix.job.needs-python-runner == true }}
-      env:
-        METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
-      run: ./.github/scripts/setup-python-runner.sh
+      uses: ./.github/actions/setup-python-runner
+      with:
+        metabase-automation-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
     - name: Test ${{ matrix.job.name }}
+      id: test-driver
       if: ${{ !matrix.job.skip }}
       uses: ./.github/actions/test-driver
       with:
         junit-name: 'be-tests-redshift-ee'
         test-args: >-
           ${{ matrix.job.test-args }}
+        logs-artifact-suffix: ${{ matrix.job.name }}
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: ${{ !matrix.job.skip && !cancelled() }}
+      if: ${{ !matrix.job.skip && !cancelled() && steps.test-driver.outputs.quarantine-skipped != 'true' }}
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}
@@ -970,10 +1028,17 @@ jobs:
         aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
         aws-region: ${{ vars.AWS_REGION }}
         trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
+    - name: Cleanup python-runner
+      if: ${{ always() && matrix.job.needs-python-runner == true }}
+      uses: ./.github/actions/cleanup-python-runner
+      with:
+        upload-logs: ${{ steps.test-driver.outcome == 'failure' }}
+        logs-artifact-suffix: ${{ matrix.job.name }} ${{ matrix.version.name }}
 
   be-tests-snowflake-ee:
-    if: ${{ !inputs.skip }}
     runs-on: ubuntu-22.04
+    needs: determine-driver-skips
+    if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
     timeout-minutes: 60
     strategy:
       fail-fast: false
@@ -994,6 +1059,7 @@ jobs:
       MB_SNOWFLAKE_TEST_USER: METABASE CI
       MB_SNOWFLAKE_TEST_ACCOUNT: ${{ secrets.MB_SNOWFLAKE_TEST_ACCOUNT }}
       MB_SNOWFLAKE_TEST_PASSWORD: ${{ secrets.MB_SNOWFLAKE_TEST_PASSWORD }}
+      MB_SNOWFLAKE_TEST_PRIVATE_KEY: ${{ secrets.MB_SNOWFLAKE_TEST_PRIVATE_KEY }}
       MB_SNOWFLAKE_TEST_WAREHOUSE: ${{ secrets.MB_SNOWFLAKE_TEST_WAREHOUSE }}
       MB_SNOWFLAKE_TEST_PK_USER: METABASE PK
       MB_SNOWFLAKE_TEST_PK_PRIVATE_KEY: ${{ secrets.MB_SNOWFLAKE_TEST_PK_PRIVATE_KEY }}
@@ -1011,17 +1077,19 @@ jobs:
     - uses: actions/checkout@v4
     - name: Setup python-runner
       if: ${{ matrix.job.needs-python-runner == true }}
-      env:
-        METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
-      run: ./.github/scripts/setup-python-runner.sh
+      uses: ./.github/actions/setup-python-runner
+      with:
+        metabase-automation-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
     - name: Test Snowflake driver
+      id: test-driver
       uses: ./.github/actions/test-driver
       with:
         junit-name: 'be-tests-snowflake-ee'
         test-args: ${{ matrix.job.test-args }}
+        logs-artifact-suffix: ${{ matrix.job.name }}
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: always()
+      if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}
@@ -1030,6 +1098,12 @@ jobs:
         aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
         aws-region: ${{ vars.AWS_REGION }}
         trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
+    - name: Cleanup python-runner
+      if: ${{ always() && matrix.job.needs-python-runner == true }}
+      uses: ./.github/actions/cleanup-python-runner
+      with:
+        upload-logs: ${{ steps.test-driver.outcome == 'failure' }}
+        logs-artifact-suffix: ${{ matrix.job.name }} ${{ matrix.version.name }}
 
   be-tests-sparksql-ee:
     needs: determine-driver-skips
@@ -1048,6 +1122,7 @@ jobs:
     steps:
     - uses: actions/checkout@v4
     - name: Test Spark driver
+      id: test-driver
       uses: ./.github/actions/test-driver
       with:
         junit-name: 'be-tests-sparksql-ee'
@@ -1056,7 +1131,7 @@ jobs:
           :exclude-tags [:mb/transforms-python-test]
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: always()
+      if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}
@@ -1078,6 +1153,7 @@ jobs:
     steps:
     - uses: actions/checkout@v4
     - name: Test SQLite driver
+      id: test-driver
       uses: ./.github/actions/test-driver
       with:
         junit-name: 'be-tests-sqlite-ee'
@@ -1086,7 +1162,7 @@ jobs:
           :exclude-tags [:mb/transforms-python-test]
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: always()
+      if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}
@@ -1140,17 +1216,19 @@ jobs:
     - uses: actions/checkout@v4
     - name: Setup python-runner
       if: ${{ matrix.job.needs-python-runner == true }}
-      env:
-        METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
-      run: ./.github/scripts/setup-python-runner.sh
+      uses: ./.github/actions/setup-python-runner
+      with:
+        metabase-automation-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
     - name: Test ${{ matrix.version.name }}
+      id: test-driver
       uses: ./.github/actions/test-driver
       with:
         junit-name: ${{ matrix.version.junit-name }}
         test-args: ${{ matrix.job.test-args }}
+        logs-artifact-suffix: ${{ matrix.version.name }}
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: always()
+      if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}
@@ -1159,6 +1237,12 @@ jobs:
         aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
         aws-region: ${{ vars.AWS_REGION }}
         trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
+    - name: Cleanup python-runner
+      if: ${{ always() && matrix.job.needs-python-runner == true }}
+      uses: ./.github/actions/cleanup-python-runner
+      with:
+        upload-logs: ${{ steps.test-driver.outcome == 'failure' }}
+        logs-artifact-suffix: ${{ matrix.job.name }} ${{ matrix.version.name }}
 
   # TODO(dpsutton, 2025-08-20): Re-enable starburst when we figure out why it randomly dies in CI
   # be-tests-starburst-ee:
@@ -1251,6 +1335,7 @@ jobs:
     - name: Make plugins directory
       run: mkdir plugins
     - name: Test Vertica driver
+      id: test-driver
       uses: ./.github/actions/test-driver
       with:
         junit-name: 'be-tests-vertica-ee'
@@ -1259,7 +1344,7 @@ jobs:
           :exclude-tags [:mb/transforms-python-test]
     - name: Upload Test Results
       uses: ./.github/actions/upload-test-results
-      if: always()
+      if: always() && steps.test-driver.outputs.quarantine-skipped != 'true'
       with:
         input-path: ./target/junit/
         output-name: ${{ github.job }}

.github/workflows/e2e-component-tests-embedding-sdk-backward-compatibility.yml

@@ -1,185 +0,0 @@
-# Thi workflow does the backward compatibility tests
-# between the Embedding SDK package of version N and Embedding SDK Bundle of version N - 1
-# N - 1 is a target of a PR
-name: Embedding SDK backward-compatibility E2E Component Tests
-
-on:
-  workflow_call:
-    inputs:
-      cached-embedding-sdk-dist-artifact-name:
-        required: true
-        type: string
-
-jobs:
-  files-changed:
-    name: "SDK backward-compatibility E2E > Check which files changed"
-    runs-on: ubuntu-22.04
-    timeout-minutes: 3
-    outputs:
-      e2e_embedding_sdk: ${{ steps.changes.outputs.e2e_embedding_sdk }}
-    steps:
-      - uses: actions/checkout@v4
-      - name: Test which files changed
-        uses: dorny/paths-filter@v3.0.0
-        id: changes
-        with:
-          token: ${{ github.token }}
-          filters: .github/file-paths.yaml
-
-  decide-target-ref:
-    runs-on: ubuntu-latest
-    outputs:
-      target-ref: ${{ steps.get-target-ref.outputs.value }}
-    steps:
-      - uses: actions/checkout@v4
-        with:
-          fetch-depth: 2
-
-      - name: Decide ref to checkout
-        id: get-target-ref
-        run: |
-          if [[ "${{ github.event_name }}" == "pull_request" ]]; then
-            echo "value=${{ github.base_ref }}" >> "$GITHUB_OUTPUT"
-          else
-            echo "value=$(git rev-parse HEAD^1)" >> "$GITHUB_OUTPUT"
-          fi
-
-  # if this is a test on a release branch, we need to check the build requirements
-  get-build-requirements:
-    name: "SDK backward-compatibility E2E > [Backport only] Get build requirements"
-    needs: [decide-target-ref]
-    if: |
-      !cancelled() &&
-      contains(github.base_ref || github.ref, 'release-x')
-    runs-on: ubuntu-22.04
-    timeout-minutes: 10
-    outputs:
-      java_version: ${{ fromJson(steps.dependencies.outputs.result).java_version }}
-      node_version: ${{ fromJson(steps.dependencies.outputs.result).node_version }}
-    steps:
-      - uses: actions/checkout@v4
-        with:
-          ref: ${{ needs.decide-target-ref.outputs.target-ref }}
-          sparse-checkout: release
-      - name: Prepare build scripts
-        run: cd ${{ github.workspace }}/release && yarn && yarn build
-      - name: Get build dependencies
-        uses: actions/github-script@v7
-        id: dependencies
-        with:
-          script: | # js
-            const {
-              getBuildRequirements,
-              getVersionFromReleaseBranch,
-            } = require('${{ github.workspace }}/release/dist/index.cjs');
-            const targetBranchName = '${{ github.base_ref || github.ref }}';
-
-            const version = getVersionFromReleaseBranch(targetBranchName);
-            const requirements = getBuildRequirements(version);
-
-            return {
-              java_version: requirements.java,
-              node_version: requirements.node,
-            };
-
-  e2e-tests:
-    name: "SDK backward-compatibility E2E > Run e2e tests"
-    needs: [files-changed, decide-target-ref, get-build-requirements]
-    if: |
-      !cancelled() &&
-      needs.files-changed.outputs.e2e_embedding_sdk == 'true'
-    runs-on: ubuntu-22.04
-    timeout-minutes: 45
-    env:
-      MB_EDITION: ee
-      DISPLAY: ""
-      # Any env starting with `CYPRESS_` will be available to all Cypress tests via `Cypress.env()`
-      # Example: you can get `CYPRESS_FOO` with `Cypress.env("FOO")`
-      CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.MB_ALL_FEATURES_TOKEN }}
-      CYPRESS_MB_STARTER_CLOUD_TOKEN: ${{ secrets.MB_STARTER_CLOUD_TOKEN }}
-      CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.MB_PRO_CLOUD_TOKEN }}
-      CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.MB_PRO_SELF_HOSTED_TOKEN }}
-      CYPRESS_PULL_REQUEST_ID: ${{ github.event.pull_request.number || '' }}
-      COMMIT_INFO_MESSAGE: ${{ github.event.pull_request.title || github.event.head_commit.message || github.event.head.sha }}
-      TZ: US/Pacific # to make node match the instance tz
-      CYPRESS_CI: true
-
-    steps:
-      - uses: actions/checkout@v4
-        with:
-          ref: ${{ needs.decide-target-ref.outputs.target-ref }}
-
-      - name: Prepare back-end environment
-        uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: uberjar
-          java-version: "${{ needs.get-build-requirements.outputs.java_version || 21 }}"
-
-      - name: Prepare front-end environment
-        uses: ./.github/actions/prepare-frontend
-        with:
-          node-version: "${{ needs.get-build-requirements.outputs.node_version }}"
-
-      - name: Build uberjar with ./bin/build.sh
-        run: ./bin/build.sh
-
-      - name: Retrieve Embedding SDK dist artifact
-        uses: actions/download-artifact@v4
-        with:
-          name: ${{ inputs.cached-embedding-sdk-dist-artifact-name }}
-          path: ${{ github.workspace }}/resources/embedding-sdk
-
-      - name: Prepare Cypress environment
-        id: cypress-prep
-        uses: ./.github/actions/prepare-cypress
-
-      - name: Run Metabase
-        run: node e2e/runner/run_cypress_ci.js start
-
-      - name: Make app db snapshot
-        run: node e2e/runner/run_cypress_ci.js snapshot --browser ${{ steps.cypress-prep.outputs.chrome-path }}
-
-      - name: Run component tests for Embedding SDK
-        run: |
-          node e2e/runner/run_cypress_ci.js component \
-            --browser ${{ steps.cypress-prep.outputs.chrome-path }} \
-            --env grepTags="-@skip-backward-compatibility",grepOmitFiltered=true
-
-      - name: Upload Test Results
-        uses: ./.github/actions/upload-test-results
-        if: always()
-        with:
-          input-path: ./target/junit
-          output-name: e2e-embedding-sdk
-          bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
-          aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
-          aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
-          aws-region: ${{ vars.AWS_REGION }}
-          trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
-
-      - name: Upload Cypress Artifacts upon failure
-        uses: actions/upload-artifact@v4
-        if: failure()
-        with:
-          name: cypress-recording-embedding-sdk-cross-version
-          path: |
-            ./cypress
-            ./logs/test.log
-          if-no-files-found: ignore
-
-      - name: Publish Summary
-        if: failure()
-        uses: actions/github-script@v7
-        with:
-          script: | #js
-            const {
-               generateReport,
-               parseReport,
-               formatSummary
-             } = require("./.github/scripts/handle-mochawesome-report.js");
-
-             const report = await generateReport();
-             const results = parseReport(report);
-             const summary = formatSummary(results);
-
-             await core.summary.addRaw(summary).write();

.github/workflows/e2e-component-tests-embedding-sdk.yml

@@ -1,206 +0,0 @@
-name: E2E Component Tests for Embedding SDK
-
-on:
-  workflow_call:
-    inputs:
-      cached-embedding-sdk-dist-artifact-name:
-        required: true
-        type: string
-
-jobs:
-  files-changed:
-    name: "SDK e2e > Check which files changed"
-    runs-on: ubuntu-22.04
-    timeout-minutes: 3
-    outputs:
-      e2e_embedding_sdk: ${{ steps.changes.outputs.e2e_embedding_sdk }}
-    steps:
-      - uses: actions/checkout@v4
-      - name: Test which files changed
-        uses: dorny/paths-filter@v3.0.0
-        id: changes
-        with:
-          token: ${{ github.token }}
-          filters: .github/file-paths.yaml
-
-  # if this is a test on a release branch, we need to check the build requirements
-  get-build-requirements:
-    name: "SDK e2e > [Backport only] Get build requirements"
-    if: |
-      !cancelled() &&
-      contains(github.base_ref || github.ref, 'release-x')
-    runs-on: ubuntu-22.04
-    timeout-minutes: 10
-    outputs:
-      java_version: ${{ fromJson(steps.dependencies.outputs.result).java_version }}
-      node_version: ${{ fromJson(steps.dependencies.outputs.result).node_version }}
-    steps:
-      - uses: actions/checkout@v4
-        with:
-          sparse-checkout: release
-      - name: Prepare build scripts
-        run: cd ${{ github.workspace }}/release && yarn && yarn build
-      - name: Get build dependencies
-        uses: actions/github-script@v7
-        id: dependencies
-        with:
-          script: | # js
-            const {
-              getBuildRequirements,
-              getVersionFromReleaseBranch,
-            } = require('${{ github.workspace }}/release/dist/index.cjs');
-            const targetBranchName = '${{ github.base_ref || github.ref }}';
-
-            const version = getVersionFromReleaseBranch(targetBranchName);
-            const requirements = getBuildRequirements(version);
-
-            return {
-              java_version: requirements.java,
-              node_version: requirements.node,
-            };
-
-  build-uberjar:
-    name: "SDK e2e > Build uberjar"
-    needs: [files-changed, get-build-requirements]
-    if: |
-      !cancelled() &&
-      needs.files-changed.outputs.e2e_embedding_sdk == 'true'
-    runs-on: ubuntu-22.04
-    timeout-minutes: 25
-    env:
-      MB_EDITION: ee
-      INTERACTIVE: false
-      # make sure that builds on release branches get licenses, because we use them for releases
-      SKIP_LICENSES: ${{ github.event_name == 'pull_request' }}
-    steps:
-      - uses: actions/checkout@v4
-
-      - name: Prepare front-end environment
-        uses: ./.github/actions/prepare-frontend
-        with:
-          node-version: "${{ needs.get-build-requirements.outputs.node_version }}"
-
-      - name: Prepare back-end environment
-        uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: uberjar
-          java-version: "${{ needs.get-build-requirements.outputs.java_version || 21 }}"
-
-      - name: Build uberjar with ./bin/build.sh
-        run: ./bin/build.sh
-
-      - name: Prepare uberjar artifact
-        uses: ./.github/actions/prepare-uberjar-artifact
-        with:
-          name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar-sdk-tests
-
-  e2e-tests:
-    name: "SDK e2e > Run e2e tests (React ${{ matrix.react-version }})"
-    needs: [get-build-requirements, build-uberjar]
-    if: |
-      !cancelled() &&
-      needs.build-uberjar.result == 'success'
-    runs-on: ubuntu-22.04
-    timeout-minutes: 45
-    strategy:
-      matrix:
-        react-version: [18, 19]
-      fail-fast: false
-    env:
-      MB_EDITION: ee
-      DISPLAY: ""
-      # Any env starting with `CYPRESS_` will be available to all Cypress tests via `Cypress.env()`
-      # Example: you can get `CYPRESS_FOO` with `Cypress.env("FOO")`
-      CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.MB_ALL_FEATURES_TOKEN }}
-      CYPRESS_MB_STARTER_CLOUD_TOKEN: ${{ secrets.MB_STARTER_CLOUD_TOKEN }}
-      CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.MB_PRO_CLOUD_TOKEN }}
-      CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.MB_PRO_SELF_HOSTED_TOKEN }}
-      CYPRESS_PULL_REQUEST_ID: ${{ github.event.pull_request.number || '' }}
-      CYPRESS_IS_EMBEDDING_SDK: "true"
-      COMMIT_INFO_MESSAGE: ${{ github.event.pull_request.title || github.event.head_commit.message || github.event.head.sha }}
-      TZ: US/Pacific # to make node match the instance tz
-      CYPRESS_CI: true
-
-    steps:
-      - uses: actions/checkout@v4
-
-      - name: Retrieve uberjar artifact for ee
-        uses: actions/download-artifact@v4
-        with:
-          name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar-sdk-tests
-
-      - name: Prepare front-end environment
-        uses: ./.github/actions/prepare-frontend
-
-      - name: Prepare JDK ${{ needs.get-build-requirements.outputs.java_version || 21 }}
-        uses: actions/setup-java@v4
-        with:
-          java-version: ${{ needs.get-build-requirements.outputs.java_version || 21 }}
-          distribution: "temurin"
-
-      - name: Prepare Cypress environment
-        id: cypress-prep
-        uses: ./.github/actions/prepare-cypress
-
-      - name: Prepare back-end environment
-        uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "cljs"
-
-      - name: Retrieve Embedding SDK dist artifact
-        uses: actions/download-artifact@v4
-        with:
-          name: ${{ inputs.cached-embedding-sdk-dist-artifact-name }}
-          path: ${{ github.workspace }}/resources/embedding-sdk
-
-      - name: Run Metabase
-        run: node e2e/runner/run_cypress_ci.js start
-
-      - name: Make app db snapshot
-        run: node e2e/runner/run_cypress_ci.js snapshot --browser ${{ steps.cypress-prep.outputs.chrome-path }}
-
-      # Has to be run after the app db snapshot is made, as it is not compatible with React 19
-      - name: Change React version to ${{ matrix.react-version }}
-        run: ./bin/embedding-sdk/change-react-version.bash ${{ matrix.react-version }}
-
-      - name: Run component tests for Embedding SDK
-        run: node e2e/runner/run_cypress_ci.js component --browser ${{ steps.cypress-prep.outputs.chrome-path }}
-
-      - name: Upload Test Results
-        uses: ./.github/actions/upload-test-results
-        if: always()
-        with:
-          input-path: ./target/junit
-          output-name: e2e-embedding-sdk-react-${{ matrix.react-version }}
-          bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
-          aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
-          aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
-          aws-region: ${{ vars.AWS_REGION }}
-          trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
-
-      - name: "SDK e2e > Upload Cypress Artifacts upon failure"
-        uses: actions/upload-artifact@v4
-        if: failure()
-        with:
-          name: cypress-recording-embedding-sdk-react-${{ matrix.react-version }}
-          path: |
-            ./cypress
-            ./logs/test.log
-          if-no-files-found: ignore
-
-      - name: "SDK e2e > Publish Summary"
-        if: failure()
-        uses: actions/github-script@v7
-        with:
-          script: | #js
-            const {
-               generateReport,
-               parseReport,
-               formatSummary
-             } = require("./.github/scripts/handle-mochawesome-report.js");
-
-             const report = await generateReport();
-             const results = parseReport(report);
-             const summary = formatSummary(results);
-
-             await core.summary.addRaw(summary).write();

.github/workflows/e2e-cross-version.yml

@@ -1,171 +0,0 @@
-name: E2E Cross-version Tests
-
-on:
-  schedule:
-    - cron: '0 9 * * 1-5'
-  pull_request:
-    types: [opened, synchronize, reopened, ready_for_review]
-
-jobs:
-  files-changed:
-    name: Check which files changed
-    # don't run this workflow on a cron for forks
-    if: ${{ github.event_name != 'schedule' || github.repository == 'metabase/metabase' }}
-    runs-on: ubuntu-22.04
-    timeout-minutes: 3
-    outputs:
-      e2e_cross_version: ${{ steps.changes.outputs.e2e_cross_version }}
-    steps:
-      - uses: actions/checkout@v4
-      - name: Test which files changed
-        uses: dorny/paths-filter@v3.0.0
-        id: changes
-        with:
-          token: ${{ github.token }}
-          filters: .github/file-paths.yaml
-
-  cross-version-testing:
-    needs: files-changed
-    if: ${{ needs.files-changed.outputs.e2e_cross_version == 'true' || github.event_name == 'schedule'}}
-    runs-on: ubuntu-22.04
-    timeout-minutes: 20
-    strategy:
-      matrix:
-        version: [
-          # Major OSS upgrade
-          { source: v0.42.2, target: v0.48.7 },
-          # OSS upgrade
-          { source: v0.41.3.1, target: v0.42.2 },
-          # EE upgrade
-          { source: v1.43.4, target: v1.44.0 },
-          # EE downgrade
-          { source: v1.42.0, target: v1.41.3.1 },
-          # Cross-edition upgrade
-          { source: v0.41.3.1, target: v1.42.2 }
-        ]
-      fail-fast: false
-    env:
-      CROSS_VERSION_SOURCE: ${{ matrix.version.source }}
-      CROSS_VERSION_TARGET: ${{ matrix.version.target }}
-
-    steps:
-    - name: Checkout
-      uses: actions/checkout@v4
-    - name: Set source Docker image to OSS
-      run: |
-        echo "DOCKER_SOURCE_IMAGE=metabase/metabase:${{ matrix.version.source }}" >> $GITHUB_ENV
-      if: startsWith(matrix.version.source, 'v0')
-    - name: Set source Docker image to EE
-      run: |
-        echo "DOCKER_SOURCE_IMAGE=metabase/metabase-enterprise:${{ matrix.version.source }}" >> $GITHUB_ENV
-      if: startsWith(matrix.version.source, 'v1')
-    - name: Prepare front-end environment
-      uses: ./.github/actions/prepare-frontend
-    - name: Start Metabase ${{ matrix.version.source }}
-      run: |
-        docker run -d \
-        -v $PWD/my-metabase:/metabase.db \
-        -p 3000:3000 \
-        -e CYPRESS_MB_ALL_FEATURES_TOKEN=${{ secrets.MB_ALL_FEATURES_TOKEN }} \
-        -e CYPRESS_MB_STARTER_CLOUD_TOKEN=${{ secrets.MB_STARTER_CLOUD_TOKEN }} \
-        -e CYPRESS_MB_PRO_CLOUD_TOKEN=${{ secrets.MB_PRO_CLOUD_TOKEN }} \
-        -e CYPRESS_MB_PRO_SELF_HOSTED_TOKEN=${{ secrets.MB_PRO_SELF_HOSTED_TOKEN }} \
-        --name metabase-${{ matrix.version.source }} \
-        ${{ env.DOCKER_SOURCE_IMAGE }}
-    - name: Wait for Metabase to start on port 3000
-      run: while ! curl -s 'http://localhost:3000/api/health' | grep '{"status":"ok"}'; do sleep 1; done
-    - name: Prepare Cypress environment
-      id: cypress-prep
-      uses: ./.github/actions/prepare-cypress
-    - name: Run Cypress on the source
-      run: |
-        yarn cypress run \
-          --browser ${{ steps.cypress-prep.outputs.chrome-path }} \
-          --config-file e2e/test/scenarios/cross-version/source/shared/cross-version-source.config.js \
-          --spec e2e/test/scenarios/cross-version/source/**/*.cy.spec.js
-    - name: Stop Metabase ${{ matrix.version.source }}
-      run: docker stop metabase-${{ matrix.version.source }}
-
-    - name: Set target Docker image to OSS
-      run: |
-        echo "DOCKER_TARGET_IMAGE=metabase/metabase:${{ matrix.version.source }}" >> $GITHUB_ENV
-      if: startsWith(matrix.version.source, 'v0')
-    - name: Set target Docker image to EE
-      run: |
-        echo "DOCKER_TARGET_IMAGE=metabase/metabase-enterprise:${{ matrix.version.source }}" >> $GITHUB_ENV
-      if: startsWith(matrix.version.source, 'v1')
-    - name: Start Metabase ${{ matrix.version.target }}
-      run: |
-        docker run -d \
-          -v $PWD/my-metabase:/metabase.db \
-          -p 3001:3000 \
-          -e CYPRESS_MB_ALL_FEATURES_TOKEN=${{ secrets.MB_ALL_FEATURES_TOKEN }} \
-          -e CYPRESS_MB_STARTER_CLOUD_TOKEN=${{ secrets.MB_STARTER_CLOUD_TOKEN }} \
-          -e CYPRESS_MB_PRO_CLOUD_TOKEN=${{ secrets.MB_PRO_CLOUD_TOKEN }} \
-          -e CYPRESS_MB_PRO_SELF_HOSTED_TOKEN=${{ secrets.MB_PRO_SELF_HOSTED_TOKEN }} \
-          --name metabase-${{ matrix.version.target }} \
-          ${{ env.DOCKER_TARGET_IMAGE }}
-    - name: Wait for Metabase to start on port 3001
-      run: while ! curl -s 'http://localhost:3001/api/health' | grep '{"status":"ok"}'; do sleep 1; done
-    - name: Run Cypress on the target
-      run: |
-        yarn cypress run \
-          --browser ${{ steps.cypress-prep.outputs.chrome-path }} \
-          --config-file e2e/test/scenarios/cross-version/target/shared/cross-version-target.config.js \
-          --spec e2e/test/scenarios/cross-version/target/**/*.cy.spec.js
-    - name: Upload Cypress Artifacts upon failure
-      uses: actions/upload-artifact@v4
-      if: failure()
-      with:
-        name: cypress-artifacts-${{ matrix.version.source}}-to-${{ matrix.version.target }}
-        path: |
-          ./cypress
-        if-no-files-found: ignore
-  notify-on-failure:
-    needs: cross-version-testing
-    if: ${{ always() && contains(needs.*.result, 'failure') && github.event_name == 'schedule' }}
-    name: Cross-version test failure
-    runs-on: ubuntu-22.04
-    timeout-minutes: 5
-    steps:
-      - uses: actions/setup-node@v4
-        with:
-          node-version: lts/Jod # 22.x.x
-      - run: npm install @slack/web-api
-      - name: notify slack of failure
-        uses: actions/github-script@v7
-        with:
-          script: | # js
-            const FAILED_RUN_URL = "https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}";
-            const REF_NAME = "${{ github.ref_name }}";
-
-            // notify slack of failure
-            const { WebClient } = require('@slack/web-api');
-            const slack = new WebClient('${{ secrets.SLACK_BOT_TOKEN }}');
-
-            await slack.chat.postMessage({
-              channel: 'engineering-ci',
-              text: 'Cross-version Tests Have Failed',
-              blocks: [
-                {
-                  "type": "header",
-                  "text": {
-                    "type": "plain_text",
-                    "text": `:broken_heart:   Cross-version tests are failing on ${REF_NAME}`,
-                    "emoji": true,
-                  }
-                },
-              ],
-              attachments: [{
-                color: "#BF40BF",
-                blocks: [
-                  {
-                    "type": "section",
-                    "text": {
-                      "type": "mrkdwn",
-                      "text": `See <${FAILED_RUN_URL}|failing run details>.`
-                    }
-                  },
-                ]
-              }]
-            });

.github/workflows/e2e-stress-test-flake-fix.yml

@@ -37,8 +37,26 @@ on:
         default: true
 
 jobs:
+  determine-test-type:
+    name: Determine test type
+    runs-on: ubuntu-22.04
+    timeout-minutes: 5
+    outputs:
+      type: ${{ steps.detect.outputs.type }}
+    steps:
+      - name: Detect test type from spec path
+        id: detect
+        run: |
+          SPEC='${{ inputs.spec }}'
+          if [[ "$SPEC" == e2e/test-component/scenarios/embedding-sdk/* ]]; then
+            echo "type=component" >> $GITHUB_OUTPUT
+          else
+            echo "type=e2e" >> $GITHUB_OUTPUT
+          fi
+
   workflow-summary:
     name: Stress test inputs
+    needs: [determine-test-type]
     runs-on: ubuntu-22.04
     timeout-minutes: 5
     steps:
@@ -48,6 +66,7 @@ jobs:
           echo '' >> $GITHUB_STEP_SUMMARY
           echo '- `branch`: ${{ github.ref_name }}' >> $GITHUB_STEP_SUMMARY
           echo '- `spec`: ${{ inputs.spec }}' >> $GITHUB_STEP_SUMMARY
+          echo '- `test_type` (inferred): ${{ needs.determine-test-type.outputs.type }}' >> $GITHUB_STEP_SUMMARY
           echo '- `burn_in`: ${{ inputs.burn_in }}' >> $GITHUB_STEP_SUMMARY
           echo '- `grep`: "${{ inputs.grep }}"' >> $GITHUB_STEP_SUMMARY
           echo '- `qa_db`: "${{ inputs.qa_db }}"' >> $GITHUB_STEP_SUMMARY
@@ -56,6 +75,7 @@ jobs:
           echo 'triggered by: @${{ github.event.sender.login }}' >> $GITHUB_STEP_SUMMARY
 
   stress-test-flake-fix:
+    needs: [determine-test-type]
     runs-on: ubuntu-22.04
     timeout-minutes: 60
     name: Stress test E2E flake fix
@@ -63,10 +83,12 @@ jobs:
       DISPLAY: ""
       QA_DB_ENABLED: ${{ contains(fromJSON('["sql", "mongo"]'), inputs.qa_db) }}
       CYPRESS_QA_DB_MONGO: ${{ inputs.qa_db == 'mongo' }}
-      CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.MB_ALL_FEATURES_TOKEN }}
-      CYPRESS_MB_STARTER_CLOUD_TOKEN: ${{ secrets.MB_STARTER_CLOUD_TOKEN }}
-      CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.MB_PRO_CLOUD_TOKEN }}
-      CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.MB_PRO_SELF_HOSTED_TOKEN }}
+      MB_RUN_MODE: e2e
+      METASTORE_DEV_SERVER_URL: ${{ secrets.METASTORE_DEV_SERVER_URL }}
+      CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.STAGING_MB_ALL_FEATURES_TOKEN }}
+      CYPRESS_MB_STARTER_CLOUD_TOKEN: ${{ secrets.STAGING_MB_STARTER_CLOUD_TOKEN }}
+      CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.STAGING_MB_PRO_CLOUD_TOKEN }}
+      CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.STAGING_MB_PRO_SELF_HOSTED_TOKEN }}
       MB_SNOWPLOW_AVAILABLE: true
       MB_SNOWPLOW_URL: "http://localhost:9090" # Snowplow micro
       MB_EDITION: ${{ inputs.mb_edition }}
@@ -98,6 +120,16 @@ jobs:
 
       - name: Prepare front-end environment
         uses: ./.github/actions/prepare-frontend
+
+      - name: Prepare back-end environment
+        if: ${{ needs.determine-test-type.outputs.type == 'component' }}
+        uses: ./.github/actions/prepare-backend
+        with:
+          m2-cache-key: "cljs"
+      - name: Build Embedding SDK package
+        if: ${{ needs.determine-test-type.outputs.type == 'component' }}
+        run: yarn build-embedding-sdk-package
+
       - name: Prepare JDK 21
         uses: actions/setup-java@v4
         with:
@@ -116,12 +148,12 @@ jobs:
       - name: Stress-test ${{ github.event.inputs.spec }} ${{ github.event.inputs.burn_in }} times
         env:
           GREP: ${{ github.event.inputs.grep }}
+          CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
         run: |
-          node e2e/runner/run_cypress_ci.js e2e \
+          node e2e/runner/run_cypress_ci.js ${{ needs.determine-test-type.outputs.type }} \
           --spec '${{ github.event.inputs.spec }}' \
           --env burn=${{ github.event.inputs.burn_in }} \
-          --config-file e2e/support/cypress-stress-test.config.js \
-          --browser ${{ steps.cypress-prep.outputs.chrome-path }}
+          --config-file e2e/support/cypress-stress-test.config.js
 
       - name: Upload Cypress Artifacts upon failure
         uses: actions/upload-artifact@v4

.github/workflows/e2e-test.yml

@@ -42,12 +42,14 @@ jobs:
       GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       HASH: ${{ github.event.pull_request.head.sha || github.sha }}-${{ github.run_attempt }}
       JOB_NAME: ${{ inputs.name }}
+      MB_RUN_MODE: "e2e"
+      METASTORE_DEV_SERVER_URL: ${{ secrets.METASTORE_DEV_SERVER_URL }}
       # Any env starting with `CYPRESS_` will be available to all Cypress tests via `Cypress.env()`
       # Example: you can get `CYPRESS_FOO` with `Cypress.env("FOO")`
-      CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.MB_ALL_FEATURES_TOKEN }}
-      CYPRESS_MB_STARTER_CLOUD_TOKEN: ${{ secrets.MB_STARTER_CLOUD_TOKEN }}
-      CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.MB_PRO_CLOUD_TOKEN }}
-      CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.MB_PRO_SELF_HOSTED_TOKEN }}
+      CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.STAGING_MB_ALL_FEATURES_TOKEN }}
+      CYPRESS_MB_STARTER_CLOUD_TOKEN: ${{ secrets.STAGING_MB_STARTER_CLOUD_TOKEN }}
+      CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.STAGING_MB_PRO_CLOUD_TOKEN }}
+      CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.STAGING_MB_PRO_SELF_HOSTED_TOKEN }}
       CYPRESS_PULL_REQUEST_ID: ${{ github.event.pull_request.number || '' }}
       COMMIT_INFO_MESSAGE: ${{ github.event.pull_request.title || github.event.head_commit.message || github.event.head.sha }}
       MB_EXPERIMENTAL_SEARCH_BLOCK_ON_QUEUE: true
@@ -57,7 +59,6 @@ jobs:
       TZ: US/Pacific # to make node match the instance tz
       CYPRESS_CI: true
       CYPRESS_QA_DB_MONGO: ${{ inputs.name == 'mongo' }}
-      CYPRESS_IS_EMBEDDING_SDK: "${{ inputs.name == 'embedding-sdk' }}"
     steps:
       - uses: actions/checkout@v4
         with:
@@ -65,6 +66,7 @@ jobs:
           token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
 
       - name: Setup python-runner
+        if: ${{ inputs.name == 'python' }}
         env:
           METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
         run: ./.github/scripts/setup-python-runner.sh
@@ -82,10 +84,6 @@ jobs:
           mysql: ${{ inputs.name != 'mongo' }}
           mongo: ${{ inputs.name == 'mongo' }}
 
-      - name: Run Embedding SDK Storybook
-        if: inputs.name == 'embedding-sdk'
-        uses: ./.github/actions/run-embedding-sdk-storybook
-
       - name: Retrieve uberjar artifact for ${{ inputs.edition }}
         uses: ./.github/actions/fetch-artifact
         with:
@@ -122,9 +120,12 @@ jobs:
         run: node e2e/runner/run_cypress_ci.js start
 
       - name: Generate database snapshots
+        env:
+          CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
         run: node e2e/runner/run_cypress_ci.js snapshot
 
       - name: Run auto-split Cypress tests on ${{ inputs.name }}
+        id: split-e2e
         if: ${{ !inputs.specs }}
         env:
           SPLIT: ${{ inputs.total_chunks }}
@@ -135,42 +136,54 @@ jobs:
           # do not write cypress-split summary to github
           SPLIT_SUMMARY: false
           SPEC_PATH: ${{ env.specs || inputs.specs }}
+          CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
         shell: bash
         run: |
           echo "Running tests with specs: $SPEC_PATH"
           node e2e/runner/run_cypress_ci.js e2e \
-            --env grepTags="-@mongo+-@flaky+-@OSS+-@skip",grepOmitFiltered=true \
-            --spec "$SPEC_PATH" \
-            --browser ${{ steps.cypress-prep.outputs.chrome-path }}
+            --env grepTags="-@mongo+-@python+-@OSS+-@skip" \
+            --spec "$SPEC_PATH"
+        # Trunk Quarantine controls the final state of the run in the CI!
+        # Even if this step fails we let it pass, and then `trunk-io/analytics-uploader` determines
+        # whether to fail or to pass the whole job, based on the quarantine list.
+        continue-on-error: true
 
       - name: Run Tagged EE Cypress tests on ${{ inputs.name }}
+        id: tagged-e2e
         if: ${{ inputs.specs }}
         shell: bash
         env:
           SPEC_PATH: ${{ env.specs || inputs.specs }}
+          CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
         run: |
           echo "Running tests with specs: $SPEC_PATH"
           node e2e/runner/run_cypress_ci.js e2e \
-          --env grepTags="${{inputs.tags && format('{0}+-@skip', inputs.tags) || '-@skip'}}",grepOmitFiltered=true \
-          --spec "$SPEC_PATH" \
-          --browser ${{ steps.cypress-prep.outputs.chrome-path }}
+          --env grepTags="${{inputs.tags && format('{0}+-@skip', inputs.tags) || '-@skip'}}" \
+          --spec "$SPEC_PATH"
+        # Trunk Quarantine controls the final state of the run in the CI!
+        # Even if this step fails we let it pass, and then `trunk-io/analytics-uploader` determines
+        # whether to fail or to pass the whole job, based on the quarantine list.
+        continue-on-error: true
 
       - name: Upload Test Results
         uses: ./.github/actions/upload-test-results
         if: always()
         with:
           input-path: ./target/junit
+          # Related to our custom upload to S3
           output-name: e2e-${{ inputs.name }}
-          variant: e2e-tests
           bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
           aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
           aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
           aws-region: ${{ vars.AWS_REGION }}
+          # Related to Trunk uploader
+          variant: e2e-tests
           trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
 
       - name: Upload Cypress Artifacts upon failure
         uses: actions/upload-artifact@v4
-        if: failure()
+        if: |
+          !cancelled() && (steps.split-e2e.outcome == 'failure' || steps.tagged-e2e.outcome == 'failure')
         with:
           name: cypress-recording-${{ inputs.name }}-${{ inputs.edition }}
           path: |
@@ -180,10 +193,12 @@ jobs:
 
       - name: Upload Failed Tests
         uses: actions/upload-artifact@v4
-        if: failure()
+        if: |
+          !cancelled() && (steps.split-e2e.outcome == 'failure' || steps.tagged-e2e.outcome == 'failure')
         with:
           name: failed-tests-${{ inputs.name }}-${{ inputs.edition }}
           path: ./cypress/test-results
+          if-no-files-found: ignore
 
       - name: Extract chunk-specific changed timings
         id: extract-timings

.github/workflows/e2e-tests.yml

@@ -73,20 +73,6 @@ jobs:
       tags: ${{ matrix.tags }}
       runner: ${{ matrix.runner }}
 
-  # run this separately to make it non-blocking
-  flaky-e2e-tests:
-    name: Flaky e2e Tests
-    uses: ./.github/workflows/e2e-test.yml
-    needs:
-      - e2e-matrix-builder
-    # skip flaky test run if only e2e specs changed as an optimization
-    if: |
-      !cancelled() && !inputs.skip && needs.e2e-matrix-builder.outputs.isDefaultSpecPattern == 'true'
-    secrets: inherit
-    with:
-      name: Flaky
-      tags: "@flaky"
-
   collect-timings:
     name: Collect E2E Timings
     runs-on: ubuntu-22.04
@@ -118,7 +104,8 @@ jobs:
             const fs = require('fs');
             const path = require('path');
 
-            const chunkFiles = glob.sync('timing-files/*/chunkTimings.json');
+            const globber = await glob.create('timing-files/*/chunkTimings.json');
+            const chunkFiles = await globber.glob();
 
             const specs = new Map();
             chunkFiles.forEach(file => {
@@ -151,14 +138,12 @@ jobs:
   e2e-tests-result:
     needs:
       - e2e-tests
-      - flaky-e2e-tests
     runs-on: ubuntu-latest
     timeout-minutes: 5
     name: e2e-tests-result
     if: always() && !cancelled()
     env:
       E2E_TESTS_RESULT: ${{ needs.e2e-tests.result }}
-      FLAKY_E2E_TESTS_RESULT: ${{ needs.flaky-e2e-tests.result }}
     steps:
       - name: Stub
         if: ${{ needs.e2e-tests.result == 'skipped' }}
@@ -168,7 +153,6 @@ jobs:
         if: ${{ needs.e2e-tests.result != 'skipped' }}
         run: | # sh
           echo "E2E Tests: ${{ needs.e2e-tests.result }}"
-          echo "Flaky E2E Tests: ${{ needs.flaky-e2e-tests.result }}"
 
           if [ "${{ needs.e2e-tests.result }}" != "success" ]; then
             echo "E2E Tests failed"

.github/workflows/embedding-sdk-component-tests.yml

@@ -0,0 +1,250 @@
+name: Embedding SDK Component Tests
+
+on:
+  workflow_call:
+
+jobs:
+  sdk-component-tests:
+    name: "SDK component tests (React ${{ matrix.react-version }})"
+    runs-on: ubuntu-22.04
+    timeout-minutes: 45
+    strategy:
+      matrix:
+        react-version: [18, 19]
+      fail-fast: false
+    env:
+      MB_EDITION: ee
+      DISPLAY: ""
+      MB_RUN_MODE: "e2e"
+      METASTORE_DEV_SERVER_URL: ${{ secrets.METASTORE_DEV_SERVER_URL }}
+      # Any env starting with `CYPRESS_` will be available to all Cypress tests via `Cypress.env()`
+      # Example: you can get `CYPRESS_FOO` with `Cypress.env("FOO")`
+      CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.STAGING_MB_ALL_FEATURES_TOKEN }}
+      CYPRESS_MB_STARTER_CLOUD_TOKEN: ${{ secrets.STAGING_MB_STARTER_CLOUD_TOKEN }}
+      CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.STAGING_MB_PRO_CLOUD_TOKEN }}
+      CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.STAGING_MB_PRO_SELF_HOSTED_TOKEN }}
+      COMMIT_INFO_MESSAGE: ${{ github.event.pull_request.title || github.event.head_commit.message || github.event.head.sha }}
+      TZ: US/Pacific # to make node match the instance tz
+      CYPRESS_CI: true
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Retrieve uberjar artifact for ee
+        uses: ./.github/actions/fetch-artifact
+        with:
+          name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
+          extract-path: "target/uberjar"
+          output-name: "metabase.jar"
+
+      - name: Retrieve Embedding SDK dist artifact
+        uses: actions/download-artifact@v4
+        with:
+          name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
+          path: ${{ github.workspace }}/resources/embedding-sdk
+
+      - name: Prepare front-end environment
+        uses: ./.github/actions/prepare-frontend
+
+      - name: Prepare JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: 21
+          distribution: "temurin"
+
+      - name: Prepare Cypress environment
+        id: cypress-prep
+        uses: ./.github/actions/prepare-cypress
+
+      - name: Run Metabase
+        run: node e2e/runner/run_cypress_ci.js start
+
+      - name: Make app db snapshot
+        env:
+          CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
+        run: node e2e/runner/run_cypress_ci.js snapshot
+
+      # Has to be run after the app db snapshot is made, as it is not compatible with React 19
+      - name: Change React version to ${{ matrix.react-version }}
+        run: ./bin/embedding-sdk/change-react-version.bash ${{ matrix.react-version }}
+
+      - name: Run component tests for Embedding SDK
+        env:
+          CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
+        run: node e2e/runner/run_cypress_ci.js component
+
+      - name: Upload Test Results
+        uses: ./.github/actions/upload-test-results
+        if: always()
+        with:
+          input-path: ./target/junit
+          output-name: e2e-embedding-sdk-react-${{ matrix.react-version }}
+          bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
+          aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
+          aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
+          aws-region: ${{ vars.AWS_REGION }}
+          trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
+
+      - name: "SDK component tests > Upload Cypress Artifacts upon failure"
+        uses: actions/upload-artifact@v4
+        if: failure()
+        with:
+          name: cypress-recording-embedding-sdk-react-${{ matrix.react-version }}
+          path: |
+            ./cypress
+            ./logs/test.log
+          if-no-files-found: ignore
+
+      - name: "SDK component tests > Publish Summary"
+        if: failure()
+        uses: actions/github-script@v7
+        with:
+          script: | #js
+            const {
+               generateReport,
+               parseReport,
+               formatSummary
+             } = require("./.github/scripts/handle-mochawesome-report.js");
+
+             const report = await generateReport();
+             const results = parseReport(report);
+             const summary = formatSummary(results);
+
+             await core.summary.addRaw(summary).write();
+
+  decide-target-ref:
+    runs-on: ubuntu-latest
+    outputs:
+      target-ref: ${{ steps.get-target-ref.outputs.value }}
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 2
+
+      - name: Decide ref to checkout
+        id: get-target-ref
+        run: |
+          if [[ "${{ github.event_name }}" == "pull_request" ]]; then
+            echo "value=${{ github.base_ref }}" >> "$GITHUB_OUTPUT"
+          else
+            echo "value=$(git rev-parse HEAD^1)" >> "$GITHUB_OUTPUT"
+          fi
+
+  # This job ensures that the Embedding SDK package from the current commit (N) works with the
+  # Embedding SDK Bundle (Metabase JAR) from the previous commit (N-1).
+  # N - 1 is a target of a pull request, or a literal previous commit when this job is run on push in a branch.
+  sdk-component-tests-backward-compatibility:
+    name: "SDK component tests (Backward compatibility)"
+    needs: [decide-target-ref]
+    runs-on: ubuntu-22.04
+    timeout-minutes: 45
+    env:
+      MB_EDITION: ee
+      DISPLAY: ""
+      MB_RUN_MODE: "e2e"
+      METASTORE_DEV_SERVER_URL: ${{ secrets.METASTORE_DEV_SERVER_URL }}
+      # Any env starting with `CYPRESS_` will be available to all Cypress tests via `Cypress.env()`
+      # Example: you can get `CYPRESS_FOO` with `Cypress.env("FOO")`
+      CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.STAGING_MB_ALL_FEATURES_TOKEN }}
+      CYPRESS_MB_STARTER_CLOUD_TOKEN: ${{ secrets.STAGING_MB_STARTER_CLOUD_TOKEN }}
+      CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.STAGING_MB_PRO_CLOUD_TOKEN }}
+      CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.STAGING_MB_PRO_SELF_HOSTED_TOKEN }}
+      COMMIT_INFO_MESSAGE: ${{ github.event.pull_request.title || github.event.head_commit.message || github.event.head.sha }}
+      TZ: US/Pacific # to make node match the instance tz
+      CYPRESS_CI: true
+
+    steps:
+      - name: Check out the code from the previous commit
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ needs.decide-target-ref.outputs.target-ref }}
+
+      - name: Get the actual commit SHA from the checked out branch
+        id: get-commit-sha
+        run: echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
+
+      - name: Prepare back-end environment
+        uses: ./.github/actions/prepare-backend
+
+      - name: Prepare front-end environment
+        uses: ./.github/actions/prepare-frontend
+
+      - name: Attempt to retrieve the ee uberjar artifact from the previous commit
+        id: download-uberjar
+        uses: ./.github/actions/fetch-artifact
+        with:
+          name: metabase-ee-${{ steps.get-commit-sha.outputs.sha }}-uberjar
+          extract-path: "target/uberjar"
+          output-name: "metabase.jar"
+        continue-on-error: true # do not break the whole job if this step doesn't succeed
+
+      # Build the uberjar from the previous commit only if we couldn't fetch it.
+      # `step.foo.OUTCOME` is important because it looks at the step status *before* continue-on-error is applied.
+      # Do not confuse it with the `step.foo.conclusion`.
+      - name: Build MB uberjar from the commit N-1
+        if: steps.download-uberjar.outcome != 'success'
+        run: ./bin/build.sh
+
+      - name: Retrieve Embedding SDK dist artifact built from the current commit N
+        uses: actions/download-artifact@v4
+        with:
+          name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
+          path: ${{ github.workspace }}/resources/embedding-sdk
+
+      - name: Prepare Cypress environment
+        id: cypress-prep
+        uses: ./.github/actions/prepare-cypress
+
+      - name: Run Metabase
+        run: node e2e/runner/run_cypress_ci.js start
+
+      - name: Make app db snapshot
+        env:
+          CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
+        run: node e2e/runner/run_cypress_ci.js snapshot
+
+      - name: Run component tests for Embedding SDK
+        env:
+          CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
+        run: |
+          node e2e/runner/run_cypress_ci.js component \
+            --env grepTags="-@skip-backward-compatibility"
+
+      - name: Upload Test Results
+        uses: ./.github/actions/upload-test-results
+        if: always()
+        with:
+          input-path: ./target/junit
+          output-name: e2e-embedding-sdk
+          bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
+          aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
+          aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
+          aws-region: ${{ vars.AWS_REGION }}
+          trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
+
+      - name: Upload Cypress Artifacts upon failure
+        uses: actions/upload-artifact@v4
+        if: failure()
+        with:
+          name: cypress-recording-embedding-sdk-cross-version
+          path: |
+            ./cypress
+            ./logs/test.log
+          if-no-files-found: ignore
+
+      - name: Publish Summary
+        if: failure()
+        uses: actions/github-script@v7
+        with:
+          script: | #js
+            const {
+               generateReport,
+               parseReport,
+               formatSummary
+             } = require("./.github/scripts/handle-mochawesome-report.js");
+
+             const report = await generateReport();
+             const results = parseReport(report);
+             const summary = formatSummary(results);
+
+             await core.summary.addRaw(summary).write();

.github/workflows/embedding-sdk-host-sample-apps-tests.yml

@@ -1,78 +1,14 @@
 name: "[NOT REQUIRED] SDK Compatibility tests"
 
 on:
-  push:
-    branches:
-      - "master"
-      - "release-**"
-  pull_request:
-    types: [opened, synchronize, reopened]
-    branches:
-      - "master"
-      - "release-**"
-
-concurrency:
-  group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
-  cancel-in-progress: true
+  workflow_call:
 
 jobs:
-  files-changed:
-    name: Check which files changed
-    runs-on: ubuntu-22.04
-    timeout-minutes: 3
-    outputs:
-      e2e_embedding_sdk_compatibility: ${{ steps.changes.outputs.e2e_embedding_sdk_compatibility }}
-    steps:
-      - uses: actions/checkout@v4
-      - name: Test which files changed
-        uses: dorny/paths-filter@v3.0.0
-        id: changes
-        with:
-          token: ${{ github.token }}
-          filters: .github/file-paths.yaml
-
-  # if this is a test on a release branch, we need to check the build requirements
-  get-build-requirements:
-    if: |
-      !cancelled() &&
-      contains(github.base_ref || github.ref, 'release-x')
-    runs-on: ubuntu-22.04
-    timeout-minutes: 10
-    outputs:
-      java_version: ${{ fromJson(steps.dependencies.outputs.result).java_version }}
-      node_version: ${{ fromJson(steps.dependencies.outputs.result).node_version }}
-    steps:
-      - uses: actions/checkout@v4
-        with:
-          sparse-checkout: release
-      - name: Prepare build scripts
-        run: cd ${{ github.workspace }}/release && yarn && yarn build
-      - name: Get build dependencies
-        uses: actions/github-script@v7
-        id: dependencies
-        with:
-          script: | # js
-            const {
-              getBuildRequirements,
-              getVersionFromReleaseBranch,
-            } = require('${{ github.workspace }}/release/dist/index.cjs');
-            const targetBranchName = '${{ github.base_ref || github.ref }}';
-
-            const version = getVersionFromReleaseBranch(targetBranchName);
-            const requirements = getBuildRequirements(version);
-
-            return {
-              java_version: requirements.java,
-              node_version: requirements.node,
-            };
-
   get-sample-apps-data:
-    if: |
-      !cancelled()
     runs-on: ubuntu-22.04
     timeout-minutes: 10
     env:
-      # we must support last two stable releases
+      # we must support the last two stable releases
       SUPPORTED_RELEASES_COUNT: 2
       TEST_SUITES: |
         [
@@ -145,48 +81,6 @@ jobs:
               matrix: []
             };
 
-  build:
-    needs: [files-changed, get-build-requirements, get-sample-apps-data]
-    if: |
-      !cancelled() &&
-      needs.files-changed.outputs.e2e_embedding_sdk_compatibility == 'true' &&
-      fromJSON(needs.get-sample-apps-data.outputs.matrix)[0] != null
-    runs-on: ubuntu-22.04
-    timeout-minutes: 25
-    env:
-      MB_EDITION: ee
-      INTERACTIVE: false
-      # make sure that builds on release branches get licenses, because we use them for releases
-      SKIP_LICENSES: ${{ github.event_name == 'pull_request' }}
-    steps:
-      - uses: actions/checkout@v4
-      - name: Prepare front-end environment
-        uses: ./.github/actions/prepare-frontend
-        with:
-          node-version: "${{ needs.get-build-requirements.outputs.node_version }}"
-      - name: Prepare back-end environment
-        uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: uberjar
-          java-version: "${{ needs.get-build-requirements.outputs.java_version || 21 }}"
-
-      - name: Build uberjar with ./bin/build.sh
-        run: ./bin/build.sh
-
-      - name: Prepare uberjar artifact
-        uses: ./.github/actions/prepare-uberjar-artifact
-        with:
-          name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
-
-      - name: Build Embedding SDK package
-        run: yarn build-embedding-sdk-package
-
-      - name: Prepare Embedding SDK artifact
-        uses: actions/upload-artifact@v4
-        with:
-          name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
-          path: resources/embedding-sdk
-
   # In order to check compatibility between Sample Apps and Embedding SDK, the job for each sample app:
   # - pulls a Sample App
   # - starts it
@@ -194,13 +88,13 @@ jobs:
   # The job is not required, though if it's failing better to be sure that it's not caused by a breaking change.
   # For more details see `Sample Apps compatibility with Embedding SDK tests` in `enterprise/frontend/src/embedding-sdk-package/dev.md`
   e2e-sample-apps-tests:
-    needs: [get-build-requirements, get-sample-apps-data, build]
+    needs: [get-sample-apps-data]
     strategy:
       matrix:
         test_suite: ${{ fromJSON(needs.get-sample-apps-data.outputs.matrix) }}
         environment: ["production", "development"]
     if: |
-      !cancelled() && needs.build.result == 'success'
+      !cancelled() && needs.get-sample-apps-data.result == 'success'
     runs-on: ubuntu-22.04
     timeout-minutes: 45
     name: e2e-sample-app-${{ matrix.test_suite }}-tests-${{ matrix.environment }}-bundle
@@ -210,8 +104,9 @@ jobs:
       GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       PR_NUMBER: ${{ github.event.pull_request.number || '' }}
       HASH: ${{ github.event.pull_request.head.sha || github.sha }}-${{ github.run_attempt }}
-      CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.MB_ALL_FEATURES_TOKEN }}
-      CYPRESS_IS_EMBEDDING_SDK: "true"
+      MB_RUN_MODE: e2e
+      METASTORE_DEV_SERVER_URL: ${{ secrets.METASTORE_DEV_SERVER_URL }}
+      CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.STAGING_MB_ALL_FEATURES_TOKEN }}
       TZ: US/Pacific # to make node match the instance tz
       CYPRESS_CI: true
       SAMPLE_APP_BRANCH_NAME: ${{ needs.get-sample-apps-data.outputs.branch_name }}
@@ -222,6 +117,8 @@ jobs:
     steps:
       - uses: actions/checkout@v4
 
+      - uses: ./.github/actions/maximize-disk-space
+
       # We have to use the latest version that does not have this issue https://github.com/docker/compose/pull/12752
       - name: Set up Docker Compose
         uses: docker/setup-compose-action@v1
@@ -236,9 +133,11 @@ jobs:
           echo "sample_app_name=${test_suite%-e2e}" >> "$GITHUB_OUTPUT"
 
       - name: Retrieve uberjar artifact for ee
-        uses: actions/download-artifact@v4
+        uses: ./.github/actions/fetch-artifact
         with:
           name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
+          extract-path: "target/uberjar"
+          output-name: "metabase.jar"
 
       - name: Retrieve Embedding SDK dist artifact
         uses: actions/download-artifact@v4
@@ -290,11 +189,19 @@ jobs:
         if: ${{ steps.check-sample-app-branch.outputs.sample_app_branch_exists == 'true' && matrix.test_suite != 'shoppy-e2e' }}
         run: npx tsx ./e2e/runner/embedding-sdk/sample-apps/start-ci.ts ${{ matrix.test_suite }}
 
+      - name: Prune build cache
+        run: docker builder prune -af
+
+      - name: Log free disk space
+        run: df -h /
+
       - name: Run e2e tests for Sample App
         id: run-e2e-tests
         if: ${{ steps.check-sample-app-branch.outputs.sample_app_branch_exists == 'true' }}
         continue-on-error: true
-        run: node e2e/runner/run_cypress_ci.js ${{ matrix.test_suite }} --browser ${{ steps.cypress-prep.outputs.chrome-path }}
+        env:
+          CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
+        run: node e2e/runner/run_cypress_ci.js ${{ matrix.test_suite }}
 
       - name: Upload Cypress Artifacts upon failure
         uses: actions/upload-artifact@v4
@@ -318,7 +225,6 @@ jobs:
   # The job is not required, though if it's failing better to be sure that it's not caused by a breaking change.
   # For more details see `Embedding SDK integration tests with Host Apps` in `enterprise/frontend/src/embedding-sdk-package/dev.md`
   e2e-host-apps-tests:
-    needs: [get-build-requirements, build]
     strategy:
       matrix:
         test_suite:
@@ -329,8 +235,6 @@ jobs:
             "angular-20-host-app-e2e",
           ]
         environment: ["production", "development"]
-    if: |
-      !cancelled() && needs.build.result == 'success'
     runs-on: ubuntu-22.04
     timeout-minutes: 45
     name: e2e-host-app-${{ matrix.test_suite }}-tests-${{ matrix.environment }}-bundle
@@ -340,8 +244,9 @@ jobs:
       GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       PR_NUMBER: ${{ github.event.pull_request.number || '' }}
       HASH: ${{ github.event.pull_request.head.sha || github.sha }}-${{ github.run_attempt }}
-      CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.MB_ALL_FEATURES_TOKEN }}
-      CYPRESS_IS_EMBEDDING_SDK: "true"
+      MB_RUN_MODE: e2e
+      METASTORE_DEV_SERVER_URL: ${{ secrets.METASTORE_DEV_SERVER_URL }}
+      CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.STAGING_MB_ALL_FEATURES_TOKEN }}
       TZ: US/Pacific # to make node match the instance tz
       CYPRESS_CI: true
       HOST_APP_ENVIRONMENT: ${{ matrix.environment }}
@@ -352,9 +257,11 @@ jobs:
       - uses: actions/checkout@v4
 
       - name: Retrieve uberjar artifact for ee
-        uses: actions/download-artifact@v4
+        uses: ./.github/actions/fetch-artifact
         with:
           name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
+          extract-path: "target/uberjar"
+          output-name: "metabase.jar"
 
       - name: Retrieve Embedding SDK dist artifact
         uses: actions/download-artifact@v4
@@ -387,7 +294,9 @@ jobs:
       - name: Run e2e tests for Host App
         id: run-e2e-tests
         continue-on-error: true
-        run: node e2e/runner/run_cypress_ci.js ${{ matrix.test_suite }} --browser ${{ steps.cypress-prep.outputs.chrome-path }}
+        env:
+          CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
+        run: node e2e/runner/run_cypress_ci.js ${{ matrix.test_suite }}
 
       - name: Upload Cypress Artifacts upon failure
         uses: actions/upload-artifact@v4

.github/workflows/embedding-sdk-package-build.yml

@@ -0,0 +1,31 @@
+name: Embedding SDK Package Build
+
+on:
+  workflow_call:
+  workflow_dispatch:
+
+concurrency:
+  group: embedding-sdk-package-${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  build-sdk-package:
+    runs-on: ubuntu-22.04
+    timeout-minutes: 25
+    steps:
+      - uses: actions/checkout@v4
+      - name: Prepare front-end environment
+        uses: ./.github/actions/prepare-frontend
+      - name: Prepare back-end environment
+        uses: ./.github/actions/prepare-backend
+      - name: Build Embedding SDK package
+        run: |
+          # Remove the matchers to not report false positives on every PR
+          # unrelated to embedding (EMB-765)
+          echo "::remove-matcher owner=tsc::"
+          yarn build-embedding-sdk-package
+      - name: Prepare Embedding SDK artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
+          path: resources/embedding-sdk

.github/workflows/embedding-sdk.yml

@@ -2,59 +2,39 @@ name: SDK
 
 on:
   workflow_call:
+    inputs:
+      cli-snippets-type-check:
+        description: "Whether to run cli snippets type check."
+        required: true
+        type: boolean
+      docs-snippets-type-check:
+        description: "Whether to run docs snippets type check."
+        required: true
+        type: boolean
+      documentation-validation:
+        description: "Whether to run documentation generation validation check."
+        required: true
+        type: boolean
+      component-tests:
+        description: "Whether to run component tests."
+        required: true
+        type: boolean
+      host-sample-apps-tests:
+        description: "Whether to run host and sample apps compatibility tests."
+        required: true
+        type: boolean
+      skip:
+        description: Whether to skip running tests due to one of the artifacts failing to build.
+        required: true
+        type: boolean
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
   cancel-in-progress: true
 
 jobs:
-  files-changed:
-    name: Check which files changed
-    runs-on: ubuntu-22.04
-    timeout-minutes: 3
-    outputs:
-      e2e_embedding_sdk: ${{ steps.changes.outputs.e2e_embedding_sdk }}
-      frontend_embedding_sdk_sources: ${{ steps.changes.outputs.frontend_embedding_sdk_sources }}
-      embedding_documentation: ${{ steps.changes.outputs.embedding_documentation }}
-    steps:
-      - uses: actions/checkout@v4
-      - name: Test which files changed
-        uses: dorny/paths-filter@v3.0.0
-        id: changes
-        with:
-          token: ${{ github.token }}
-          filters: .github/file-paths.yaml
-
-  build:
-    needs: [files-changed]
-    if: |
-      !cancelled() &&
-      (needs.files-changed.outputs.e2e_embedding_sdk == 'true' ||
-      needs.files-changed.outputs.frontend_embedding_sdk_sources == 'true' ||
-      needs.files-changed.outputs.embedding_documentation == 'true')
-    runs-on: ubuntu-22.04
-    timeout-minutes: 25
-    steps:
-      - uses: actions/checkout@v4
-      - name: Prepare front-end environment
-        uses: ./.github/actions/prepare-frontend
-      - name: Prepare back-end environment
-        uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "cljs"
-      - name: Build Embedding SDK package
-        run: yarn build-embedding-sdk-package
-      - name: Prepare Embedding SDK artifact
-        uses: actions/upload-artifact@v4
-        with:
-          name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
-          path: resources/embedding-sdk
-
   embedding-sdk-cli-snippets-type-check:
-    needs: [files-changed, build]
-    if: |
-      needs.files-changed.outputs.frontend_embedding_sdk_sources == 'true' &&
-      needs.build.result == 'success'
+    if: inputs.cli-snippets-type-check && !inputs.skip
     runs-on: ubuntu-22.04
     timeout-minutes: 20
     steps:
@@ -70,10 +50,7 @@ jobs:
         run: yarn run embedding-sdk:cli-snippets:type-check
 
   embedding-sdk-docs-snippets-type-check:
-    needs: [files-changed, build]
-    if: |
-      needs.files-changed.outputs.embedding_documentation == 'true' &&
-      needs.build.result == 'success'
+    if: inputs.docs-snippets-type-check && !inputs.skip
     runs-on: ubuntu-22.04
     timeout-minutes: 20
     steps:
@@ -89,10 +66,7 @@ jobs:
         run: yarn run embedding-sdk:docs-snippets:type-check
 
   embedding-sdk-api-documentation-generation-validation:
-    needs: [files-changed, build]
-    if: |
-      needs.files-changed.outputs.frontend_embedding_sdk_sources == 'true' &&
-      needs.build.result == 'success'
+    if: inputs.documentation-validation && !inputs.skip
     runs-on: ubuntu-22.04
     timeout-minutes: 20
     steps:
@@ -138,29 +112,24 @@ jobs:
         run: exit 1
 
   # See `Component e2e tests` in `enterprise/frontend/src/embedding-sdk-package/dev.md`
-  sdk-e2e-tests:
-    name: "SDK e2e tests"
-    uses: ./.github/workflows/e2e-component-tests-embedding-sdk.yml
-    needs: [build]
+  embedding-sdk-component-tests:
+    name: "SDK component tests"
+    if: inputs.component-tests && !inputs.skip
+    uses: ./.github/workflows/embedding-sdk-component-tests.yml
     secrets: inherit
-    with:
-      cached-embedding-sdk-dist-artifact-name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
 
-  e2e-component-tests-embedding-sdk-backward-compatibility:
-    name: "SDK e2e backward compatibility tests"
-    uses: ./.github/workflows/e2e-component-tests-embedding-sdk-backward-compatibility.yml
-    needs: [build]
+  not-required-embedding-sdk-apps-compatibility-tests:
+    name: "SDK e2e tests for compatibility with (host/sample) apps"
+    if: inputs.host-sample-apps-tests && !inputs.skip
+    uses: ./.github/workflows/embedding-sdk-host-sample-apps-tests.yml
     secrets: inherit
-    with:
-      cached-embedding-sdk-dist-artifact-name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
 
   embedding-sdk-tests-result:
     needs:
       - embedding-sdk-cli-snippets-type-check
       - embedding-sdk-docs-snippets-type-check
       - embedding-sdk-api-documentation-generation-validation
-      - sdk-e2e-tests
-      - e2e-component-tests-embedding-sdk-backward-compatibility
+      - embedding-sdk-component-tests
     if: always() && !cancelled()
     runs-on: ubuntu-latest
     timeout-minutes: 5

.github/workflows/frontend-unit-test-stress-test-flake-fix.yml

@@ -43,8 +43,6 @@ jobs:
         uses: ./.github/actions/prepare-frontend
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "cljs"
       - name: Build cljs
         run: yarn build:cljs
       - name: Stress-test ${{ github.event.inputs.spec }} ${{ github.event.inputs.burn_in }} times

.github/workflows/frontend.yml

@@ -35,15 +35,16 @@ jobs:
         uses: ./.github/actions/prepare-frontend
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "cljs"
+
+      - name: Get Eslint cache key
+        uses: ./.github/actions/get-cache-keys
+        id: get-cache-keys
       - name: Restore ESLint cache
-        uses: actions/cache@v4
+        uses: actions/cache/restore@v4
         with:
-          path: .eslintcache
-          key: eslintcache-${{ hashFiles('yarn.lock', '.eslintrc.js') }}
-          restore-keys: |
-            eslintcache-
+          path: ${{ github.workspace}}/.eslintcache
+          key: ${{ steps.get-cache-keys.outputs.eslint-cache-key }}
+
       - name: Compile CLJS
         run: yarn build-pure:cljs
       - name: Find CSS variables that are used but never defined
@@ -65,8 +66,6 @@ jobs:
         uses: ./.github/actions/prepare-frontend
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "cljs"
       - run: yarn build-pure:cljs
         name: Compile CLJS
       - run: yarn type-check-pure
@@ -90,10 +89,12 @@ jobs:
         uses: ./.github/actions/prepare-frontend
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "cljs"
       - name: Run frontend unit tests
         run: yarn run test-unit --silent --shard=${{ matrix.shard }}/${{ env.SHARDS }}
+        # Trunk Quarantine controls the final state of the run in the CI!
+        # Even if this step fails we let it pass, and then `trunk-io/analytics-uploader` determines
+        # whether to fail or to pass the whole job, based on the quarantine list.
+        continue-on-error: true
       - name: Upload Test Results
         uses: ./.github/actions/upload-test-results
         if: always() && github.event_name != 'workflow_dispatch'
@@ -116,8 +117,6 @@ jobs:
         uses: ./.github/actions/prepare-frontend
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "cljs"
       - name: Run frontend timezones tests
         run: yarn run test-timezones
 

.github/workflows/fuzzing.yml

@@ -1,62 +0,0 @@
-name: Fuzzing
-
-on:
-  workflow_dispatch:
-  push:
-    branches:
-      - "master"
-      - "release-**"
-    paths:
-      - "frontend/src/metabase/querying/expressions/**"
-  pull_request:
-    paths:
-      - "frontend/src/metabase/querying/expressions/**"
-
-jobs:
-  fe-fuzz-lexifier:
-    runs-on: ubuntu-22.04
-    timeout-minutes: 15
-    steps:
-      - uses: actions/checkout@v4
-      - name: Prepare front-end environment
-        uses: ./.github/actions/prepare-frontend
-      - name: Prepare back-end environment
-        uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "cljs"
-      - run: yarn test-unit frontend/src/metabase/querying/expressions/pratt/fuzz.lexifier.unit.spec.ts
-        env:
-          MB_FUZZ: 1
-        name: Run fuzz testing on the lexifier
-
-  fe-fuzz-compiler:
-    runs-on: ubuntu-22.04
-    timeout-minutes: 15
-    steps:
-      - uses: actions/checkout@v4
-      - name: Prepare front-end environment
-        uses: ./.github/actions/prepare-frontend
-      - name: Prepare back-end environment
-        uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "cljs"
-      - run: yarn test-unit frontend/src/metabase/querying/expressions/fuzz.compiler.unit.spec.ts
-        env:
-          MB_FUZZ: 1
-        name: Run fuzz testing on the custom expression compiler
-
-  fe-fuzz-string-quoting:
-    runs-on: ubuntu-22.04
-    timeout-minutes: 15
-    steps:
-      - uses: actions/checkout@v4
-      - name: Prepare front-end environment
-        uses: ./.github/actions/prepare-frontend
-      - name: Prepare back-end environment
-        uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "cljs"
-      - run: yarn test-unit frontend/src/metabase/querying/expressions/fuzz.string.unit.spec.ts
-        env:
-          MB_FUZZ: 1
-        name: Run fuzz testing on the custom expression string quoter

.github/workflows/generative-query-test.yml

@@ -40,8 +40,6 @@ jobs:
           ref: ${{ inputs.ref }}
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: generative-query-test
       - name: Prepare front-end environment
         uses: ./.github/actions/prepare-frontend
 

.github/workflows/i18n.yml

@@ -38,8 +38,6 @@ jobs:
       uses: ./.github/actions/prepare-frontend
     - name: Prepare back-end environment
       uses: ./.github/actions/prepare-backend
-      with:
-        m2-cache-key: 'i18n'
     - name: Install xgettext if necessary
       run: |
         if ! command -v xgettext > /dev/null; then

.github/workflows/loki-update.yml

@@ -20,8 +20,6 @@ jobs:
 
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "cljs"
 
       - name: Compile CLJS
         run: yarn build-pure:cljs

.github/workflows/loki.yml

@@ -47,8 +47,6 @@ jobs:
 
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "cljs"
 
       - name: Compile CLJS
         run: NODE_ENV=development yarn build-pure:cljs

.github/workflows/mage.yml

@@ -2,7 +2,7 @@ name: Mage Tests
 
 on:
   pull_request:
-    paths: ["mage/**", "test/mage/**"]
+    paths: ["mage/**"]
 
 jobs:
   # run mage tests even if the rest of be isnt needed
@@ -13,8 +13,6 @@ jobs:
       - uses: actions/checkout@v4
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: 'kondo'
       - name: Setup Babashka
         uses: turtlequeue/setup-babashka@v1.7.0
         with:
@@ -24,4 +22,4 @@ jobs:
       - name: bb task list
         run: bb tasks
       - name: Run mage tests
-        run: bb ./test/mage/runtests.clj
+        run: bb ./mage/test/mage/runtests.clj

.github/workflows/migrations.yml

@@ -16,7 +16,5 @@ jobs:
       - uses: actions/checkout@v4
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: lint-migrations
       - name: Verify Liquibase Migrations
         run: ./bin/mage lint-migrations

.github/workflows/minimal-tests.yml

@@ -0,0 +1,227 @@
+name: Minimal Tests
+
+on:
+  pull_request:
+    types: [opened, synchronize, reopened, labeled]
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  # Only run if the PR has the minimal-tests label
+  check-label:
+    runs-on: ubuntu-latest
+    timeout-minutes: 1
+    outputs:
+      should-run: ${{ steps.check.outputs.result }}
+    steps:
+      - name: Check for minimal-tests label
+        id: check
+        run: |
+          if [[ "${{ contains(github.event.pull_request.labels.*.name, 'minimal-tests') }}" == "true" ]]; then
+            echo "result=true" >> $GITHUB_OUTPUT
+          else
+            echo "result=false" >> $GITHUB_OUTPUT
+          fi
+
+  # Postgres app-db tests - single version, single job
+  postgres-app-db-test:
+    needs: check-label
+    if: needs.check-label.outputs.should-run == 'true'
+    runs-on: ubuntu-22.04
+    timeout-minutes: 60
+    env:
+      CI: 'true'
+      DRIVERS: postgres
+      MB_DB_TYPE: postgres
+      MB_DB_PORT: 5432
+      MB_DB_HOST: localhost
+      MB_DB_DBNAME: mb_test
+      MB_DB_USER: mb_test
+      MB_POSTGRESQL_TEST_USER: mb_test
+    services:
+      postgres:
+        image: postgres:latest
+        ports:
+          - "5432:5432"
+        env:
+          POSTGRES_USER: mb_test
+          POSTGRES_DB: mb_test
+          POSTGRES_HOST_AUTH_METHOD: trust
+    name: Postgres Latest EE App DB Tests (Part 1)
+    steps:
+      - uses: actions/checkout@v4
+      - name: Test Postgres App DB
+        uses: ./.github/actions/test-driver
+        with:
+          build-static-viz: true
+          junit-name: 'minimal-be-tests-postgres-ee'
+          test-args: >-
+            :only '["test" "enterprise/backend/test"]'
+            :exclude-tags '[:mb/driver-tests :mb/transforms-python-test]'
+            :partition/total 2
+            :partition/index 0
+      - name: Upload Test Results
+        uses: ./.github/actions/upload-test-results
+        if: always()
+        with:
+          input-path: ./target/junit/
+          output-name: ${{ github.job }}
+          bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
+          aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
+          aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
+          aws-region: ${{ vars.AWS_REGION }}
+          trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
+
+  # Postgres driver tests
+  postgres-driver-test:
+    needs: check-label
+    if: needs.check-label.outputs.should-run == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+    env:
+      CI: 'true'
+      DRIVERS: postgres
+      MB_DB_TYPE: postgres
+      MB_DB_PORT: 5432
+      MB_DB_HOST: localhost
+      MB_DB_DBNAME: mb_test
+      MB_DB_USER: mb_test
+      MB_POSTGRESQL_TEST_USER: mb_test
+    services:
+      postgres:
+        image: postgres:latest
+        ports:
+          - "5432:5432"
+        env:
+          POSTGRES_USER: mb_test
+          POSTGRES_DB: mb_test
+          POSTGRES_HOST_AUTH_METHOD: trust
+    name: Postgres Latest Driver Tests
+    steps:
+      - uses: actions/checkout@v4
+      - name: Test Postgres Driver
+        uses: ./.github/actions/test-driver
+        with:
+          junit-name: 'minimal-be-tests-postgres-driver-ee'
+          test-args: >-
+            :only-tags [:mb/driver-tests]
+            :exclude-tags [:mb/transforms-python-test]
+      - name: Upload Test Results
+        uses: ./.github/actions/upload-test-results
+        if: always()
+        with:
+          input-path: ./target/junit/
+          output-name: ${{ github.job }}
+          bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
+          aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
+          aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
+          aws-region: ${{ vars.AWS_REGION }}
+          trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
+
+  # Frontend unit tests
+  frontend-unit-tests:
+    needs: check-label
+    if: needs.check-label.outputs.should-run == 'true'
+    runs-on: ubuntu-22.04
+    timeout-minutes: 30
+    env:
+      JEST_JUNIT_OUTPUT_DIR: ./target/junit
+      JEST_JUNIT_OUTPUT_NAME: test-report-frontend-unit.xml
+    name: Frontend Unit Tests
+    steps:
+      - uses: actions/checkout@v4
+      - name: Prepare front-end environment
+        uses: ./.github/actions/prepare-frontend
+      - name: Prepare back-end environment
+        uses: ./.github/actions/prepare-backend
+      - name: Run frontend unit tests
+        run: yarn run test-unit --silent
+        continue-on-error: true
+      - name: Upload Test Results
+        uses: ./.github/actions/upload-test-results
+        if: always()
+        with:
+          input-path: ./target/junit
+          output-name: frontend-unit
+          bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
+          aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
+          aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
+          aws-region: ${{ vars.AWS_REGION }}
+          trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
+
+  # Summary job
+  minimal-tests-result:
+    needs:
+      - postgres-app-db-test
+      - postgres-driver-test
+      - frontend-unit-tests
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    name: minimal-tests-result
+    if: always() && !cancelled()
+    steps:
+      - name: Check job status
+        uses: actions/github-script@v7
+        env:
+          needs: ${{ toJson(needs) }}
+        with:
+          script: | # js
+            const needs = JSON.parse(process.env.needs);
+
+            const jobs = Object.entries(needs).map(
+              ([jobName, jobValues]) => ({
+                name: jobName,
+                result: jobValues.result
+              }));
+
+            // Check if all jobs either succeeded or were skipped
+            // Skipped jobs are acceptable (they were intentionally skipped when the label is absent)
+            const allPassedOrSkipped = jobs.every(job =>
+              job.result === 'success' || job.result === 'skipped'
+            );
+
+            // Check if any jobs actually failed (not skipped, not cancelled)
+            const anyFailed = jobs.some(job =>
+              job.result === 'failure'
+            );
+
+            if (allPassedOrSkipped && !anyFailed) {
+              console.log("");
+              console.log("        _------.        ");
+              console.log("       /  ,     \_      ");
+              console.log("     /   /  /{}\ |o\_   ");
+              console.log("    /    \  `--' /-' \  ");
+              console.log("   |      \      \    | ");
+              console.log("  |              |`-, | ");
+              console.log("  /              /__/)/ ");
+              console.log(" |              |       ");
+              console.log("");
+              console.log("All minimal tests have passed. Fast feedback achieved!");
+              process.exit(0);
+            }
+
+            // otherwise, something failed
+            console.log("");
+            console.log("       .::::::::::.                          .::::::::::.       ");
+            console.log("     .::::''''''::::.                      .::::''''''::::.     ");
+            console.log("   .:::'          `::::....          ....::::'          `:::.   ");
+            console.log("  .::'             `:::::::|        |:::::::'             `::.  ");
+            console.log(" .::|               |::::::|_ ___ __|::::::|               |::. ");
+            console.log(" `--'               |::::::|_()__()_|::::::|               `--' ");
+            console.log("  :::               |::-o::|        |::o-::|               :::  ");
+            console.log("  `::.             .|::::::|        |::::::|.             .::'  ");
+            console.log("   `:::.          .::\-----'        `-----/::.          .:::'   ");
+            console.log("     `::::......::::'                      `::::......::::'     ");
+            console.log("       `::::::::::'                          `::::::::::'       ");
+            console.log("");
+            console.log("Minimal tests have failed.");
+
+            jobs.forEach((job) => {
+              if (job.result !== 'success' && job.result !== 'skipped') {
+                console.log(`${job.name} - ${job.result}`);
+              }
+            });
+
+            process.exit(1);

.github/workflows/openapi-check.yml

@@ -0,0 +1,44 @@
+name: OpenAPI Check
+
+on:
+  pull_request:
+    types: [opened, synchronize, reopened, labeled, unlabeled]
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
+  cancel-in-progress: true
+
+jobs:
+  check-openapi-specs:
+    runs-on: ubuntu-22.04
+    timeout-minutes: 5
+    if: contains(github.event.pull_request.labels.*.name, 'openapi-self-healing')
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          # we're intentionally using METABASE_AUTOMATION_USER_TOKEN to trigger the whole CI pipeline again
+          #  after the self-healing commit is created
+          #  if we used GITHUB_TOKEN, the self-healing commit would not trigger the CI pipeline again
+          token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
+          # fetch full git history to allow fetching the latest base branch commit and comparing changes
+          fetch-depth: 0
+      - name: Setup Node.js 22
+        uses: actions/setup-node@v4
+        with:
+          node-version: 22.x
+      - name: Prepare back-end environment
+        uses: ./.github/actions/prepare-backend
+      - name: Generate version.properties
+        run: ./bin/build.sh :steps [:version]
+      - name: Get latest base branch commit
+        run: |
+          git fetch origin ${{ github.event.pull_request.base.ref }}
+          echo "LATEST_BASE_SHA=$(git rev-parse origin/${{ github.event.pull_request.base.ref }})" >> $GITHUB_ENV
+      - name: Check OpenAPI specs are up to date
+        env:
+          AUTO_COMMIT: "true"
+          PR_BRANCH: ${{ github.event.pull_request.head.ref }}
+          BASE_SHA: ${{ env.LATEST_BASE_SHA }}
+          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
+          GITHUB_SHA: ${{ github.sha }}
+        run: ./bin/openapi-check.sh

.github/workflows/pr-env-close-unlabel.yml

@@ -10,6 +10,15 @@ concurrency:
 
 jobs:
   pr-env:
-    if: ${{ github.event.action == 'closed' && contains(github.event.pull_request.labels.*.name, 'PR-Env') || github.event.label.name == 'PR-Env' }}
+    if: ${{ (github.event.action == 'closed' && contains(github.event.pull_request.labels.*.name, 'PR-Env') && !contains(github.event.pull_request.labels.*.name, 'PR-Env-analytics-dev')) || github.event.label.name == 'PR-Env' }}
     uses: ./.github/workflows/pr-env-destroy.yml
     secrets: inherit
+    with:
+      analytics_dev_mode: false
+
+  pr-env-analytics-dev:
+    if: ${{ (github.event.action == 'closed' && contains(github.event.pull_request.labels.*.name, 'PR-Env-analytics-dev')) || github.event.label.name == 'PR-Env-analytics-dev' }}
+    uses: ./.github/workflows/pr-env-destroy.yml
+    secrets: inherit
+    with:
+      analytics_dev_mode: false

.github/workflows/pr-env-labeled.yml

@@ -10,7 +10,7 @@ concurrency:
 
 jobs:
   uberjar:
-    if: ${{ github.event.label.name == 'PR-Env' }}
+    if: ${{ github.event.label.name == 'PR-Env' || github.event.label.name == 'PR-Env-analytics-dev' }}
     uses: ./.github/workflows/uberjar.yml
     secrets: inherit
 
@@ -19,3 +19,13 @@ jobs:
     if: ${{ github.event.label.name == 'PR-Env' }}
     uses: ./.github/workflows/pr-env.yml
     secrets: inherit
+    with:
+      analytics_dev_mode: false
+
+  pr-env-analytics-dev:
+    needs: [uberjar]
+    if: ${{ github.event.label.name == 'PR-Env-analytics-dev' }}
+    uses: ./.github/workflows/pr-env.yml
+    secrets: inherit
+    with:
+      analytics_dev_mode: true

.github/workflows/pr-env.yml

@@ -3,6 +3,12 @@ run-name: Building Dynamic PR Environment for ${{ github.ref_name }} by @${{ git
 
 on:
   workflow_call:
+    inputs:
+      analytics_dev_mode:
+        description: 'Enable analytics development mode'
+        required: false
+        type: boolean
+        default: false
 jobs:
   build:
     runs-on: ubuntu-latest
@@ -124,6 +130,7 @@ jobs:
           PR_NUMBER: ${{ github.event.number }}
           MB_PREMIUM_EMBEDDING_TOKEN: ${{ secrets.PR_ENV_MB_PREMIUM_EMBEDDING_TOKEN }}
           SHA: ${{ github.event.pull_request.head.sha || github.sha }}
+          MB_ANALYTICS_DEV_MODE: ${{ inputs.analytics_dev_mode && 'true' || 'false' }}
       - name: Deploy PR Review ENV
         run: |
           kubectl apply -f ./metabase.yml
@@ -139,3 +146,4 @@ jobs:
             ✅ Preview:
                 https://pr${{ github.event.number }}.coredev.metabase.com
             🗒️ [login instructions](https://www.notion.so/metabase/Ephemeral-PR-Environments-c74a3710c87a460dbf4fc0007a001458)
+            📦 Deployment type: ${{ inputs.analytics_dev_mode && 'PR-Env-analytics-dev' || 'PR-Env' }}

.github/workflows/pre-release.yml

@@ -103,14 +103,16 @@ jobs:
     env:
       MB_EDITION: ${{ matrix.edition }}
       DISPLAY: ""
-      CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.MB_ALL_FEATURES_TOKEN }}
-      CYPRESS_MB_STARTER_CLOUD_TOKEN: ${{ secrets.MB_STARTER_CLOUD_TOKEN }}
-      CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.MB_PRO_CLOUD_TOKEN }}
-      CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.MB_PRO_SELF_HOSTED_TOKEN }}
+      MB_RUN_MODE: e2e
+      METASTORE_DEV_SERVER_URL: ${{ secrets.METASTORE_DEV_SERVER_URL }}
+      CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.STAGING_MB_ALL_FEATURES_TOKEN }}
+      CYPRESS_MB_STARTER_CLOUD_TOKEN: ${{ secrets.STAGING_MB_STARTER_CLOUD_TOKEN }}
+      CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.STAGING_MB_PRO_CLOUD_TOKEN }}
+      CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.STAGING_MB_PRO_SELF_HOSTED_TOKEN }}
       # these are needed for backwards compatibility with release branches earlier than v55, since
       # this workflow always runs from master, but we run tests from the release branch
-      CYPRESS_ALL_FEATURES_TOKEN: ${{ secrets.MB_ALL_FEATURES_TOKEN }}
-      CYPRESS_NO_FEATURES_TOKEN: ${{ secrets.MB_STARTER_CLOUD_TOKEN }}
+      CYPRESS_ALL_FEATURES_TOKEN: ${{ secrets.STAGING_MB_ALL_FEATURES_TOKEN }}
+      CYPRESS_NO_FEATURES_TOKEN: ${{ secrets.STAGING_MB_STARTER_CLOUD_TOKEN }}
       # disabled because of out of memory issues
       # probably related to https://github.com/cypress-io/cypress/issues/27415
       CYPRESS_NO_COMMAND_LOG: 1
@@ -159,19 +161,21 @@ jobs:
 
       - name: Run a few important OSS Cypress tests as sanity check
         if: ${{ matrix.edition == 'oss' }}
+        env:
+          CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
         run: |
           node e2e/runner/run_cypress_ci.js e2e \
-            --env grepTags="@smoke+-@EE",grepOmitFiltered=true \
-            --spec './e2e/test/scenarios/**/*.cy.spec.(js|ts)' \
-            --browser ${{ steps.cypress-prep.outputs.chrome-path }}
+            --env grepTags="@smoke+-@EE" \
+            --spec './e2e/test/scenarios/**/*.cy.spec.(js|ts)'
 
       - name: Run a few important EE Cypress tests as sanity check
         if: ${{ matrix.edition == 'ee' }}
+        env:
+          CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
         run: |
           node e2e/runner/run_cypress_ci.js e2e \
-            --env grepTags="@smoke+-@OSS",grepOmitFiltered=true \
-            --spec './e2e/test/scenarios/**/*.cy.spec.(js|ts)' \
-            --browser ${{ steps.cypress-prep.outputs.chrome-path }}
+            --env grepTags="@smoke+-@OSS" \
+            --spec './e2e/test/scenarios/**/*.cy.spec.(js|ts)'
 
       - name: Upload Cypress Artifacts upon failure
         uses: actions/upload-artifact@v4

.github/workflows/release-embedding-sdk.yml

@@ -8,9 +8,9 @@ on:
         description: "Branch we want to release the SDK from (to release from other branches use the workflow from those branches)"
         type: choice
         required: true
-        default: "master"
+        default: release-x.57.x
         options:
-          - master
+          - release-x.57.x
 
 concurrency:
   # We want to ensure only one job is running at a time per branch because
@@ -254,8 +254,6 @@ jobs:
 
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "release-sdk"
 
       - name: Run unit tests
         run: yarn embedding-sdk:test-unit
@@ -277,8 +275,6 @@ jobs:
 
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "release-sdk"
 
       - name: Bump published npm package version
         # NOTE: this should happen before "Build SDK bundle" as we inject SDK version into the code during build step
@@ -349,8 +345,9 @@ jobs:
           GH_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
 
       - name: Edit PR adding useful information (using Metabase bot app)
+        # metabase/embedding team is added automatically as code owner of updated `package.template.json` file
         run: |
-          gh pr edit --add-reviewer @metabase/embedding,albertoperdomo --add-label no-backport
+          gh pr edit --add-reviewer albertoperdomo --add-label no-backport
         env:
           GH_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
 
@@ -372,42 +369,16 @@ jobs:
           name: metabase-sdk
           path: sdk
 
-      - name: Compute master tag
-        id: compute-tag
-        run: |
-          if [ -n "${{ needs.determine-sdk-version.outputs.pre_release_label }}" ]; then
-            echo "masterTag=${{ needs.determine-sdk-version.outputs.pre_release_label }}" >> $GITHUB_OUTPUT
-          else
-            echo "masterTag=nightly" >> $GITHUB_OUTPUT
-          fi
-
       - name: Publish to NPM
         working-directory: sdk
         run: |
           echo //registry.npmjs.org/:_authToken=${{ secrets.NPM_RELEASE_TOKEN }} > .npmrc
+          npm publish --tag 57-stable
 
-          TAG=$(jq -r --null-input \
-            --arg branch "${{ inputs.branch }}" \
-            --arg masterTag "${{ steps.compute-tag.outputs.masterTag }}" '
-            {
-              "master": $masterTag,
-              "release-x.51.x": "51-stable",
-              "release-x.52.x": "52-stable",
-              "release-x.53.x": "53-stable",
-              "release-x.54.x": "54-stable",
-              "release-x.55.x": "55-stable",
-              "release-x.56.x": "56-stable"
-            }[$branch]'
-          )
-
-          echo "Resolved npm tag: $TAG"
-          npm publish --tag "$TAG"
-
-      - name: Add `latest` tag to the latest release branch (`release-x.55.x`) deployment
-        if: ${{ inputs.branch == 'release-x.56.x' }}
-        working-directory: sdk
-        run: |
-          npm dist-tag add @metabase/embedding-sdk-react@${{ env.sdk_version }} latest
+      # - name: Add `latest` tag to the latest release branch deployment
+      #   working-directory: sdk
+      #   run: |
+      #     npm dist-tag add @metabase/embedding-sdk-react@${{ env.sdk_version }} latest
 
   git-tag:
     needs: [publish-npm, determine-sdk-version]

.github/workflows/release-log.yml

@@ -47,7 +47,7 @@ jobs:
         with:
           script: | # js
             const {
-              getVersionFromReleaseBranch,
+              getMajorVersionFromRef,
               getMajorVersion,
             } = require('${{ github.workspace }}/release/dist/index.cjs');
 
@@ -60,11 +60,10 @@ jobs:
             }
 
             // otherwise, get the version from the release branch name
-            const branch = context.payload.ref;
-            const version = getVersionFromReleaseBranch(branch);
-            const majorVersion = getMajorVersion(version);
+            const ref = context.payload.ref;
+            const majorVersion = getMajorVersionFromRef(ref);
 
-            console.log({ branch, version, majorVersion });
+            console.log({ ref, majorVersion });
             core.exportVariable('VERSION', majorVersion);
 
       - name: generate release Log

.github/workflows/release-patch.yml

@@ -8,6 +8,11 @@ on:
         description: 'Major Metabase version (e.g. 45, 52, 68)'
         type: number
         required: true
+      override_sha:
+        description: DANGEROUSLY allow releasing from this commit sha, regardless of the state of CI (can be empty)
+        type: string
+        required: false
+        default: ''
   schedule:
     - cron: '0 21 * * 1-5' # every weekday at 4 pm EST/5 pm EDT
 
@@ -37,7 +42,7 @@ jobs:
             const lastRelease = currentRelease - 1;
             const AUTO_RELEASE_VERSIONS = [lastRelease, currentRelease];
 
-            async function releasePatchFor(majorVersion) {
+            async function releasePatchFor(majorVersion, overrideSha) {
               const nextPatch = await getNextPatchVersion({
                 github,
                 owner: context.repo.owner,
@@ -45,7 +50,7 @@ jobs:
                 majorVersion,
               });
 
-              const goodCommit = await getLatestGreenCommit({
+              const releaseCommit = overrideSha ?? await getLatestGreenCommit({
                 github,
                 owner: context.repo.owner,
                 repo: context.repo.repo,
@@ -56,25 +61,25 @@ jobs:
                 github,
                 owner: context.repo.owner,
                 repo: context.repo.repo,
-                ref: goodCommit,
+                ref: releaseCommit,
                 majorVersion: Number(majorVersion),
               });
 
-              if (nextPatch && goodCommit && !hasBeenReleased) {
+              if (nextPatch && releaseCommit && !hasBeenReleased) {
                 await github.rest.actions.createWorkflowDispatch({
                   owner: context.repo.owner,
                   repo: context.repo.repo,
                   workflow_id: 'tag-for-release.yml',
                   ref: 'refs/heads/master',
                   inputs: {
-                    commit: goodCommit,
+                    commit: releaseCommit,
                     version: nextPatch,
                     auto: true,
                   }
                 });
               } else {
                 console.log(
-                  { nextPatch, goodCommit, hasBeenReleased }
+                  { nextPatch, releaseCommit, hasBeenReleased }
                 );
                 console.error(`No new patch version or no green commit found for v${majorVersion}`);
               }
@@ -88,7 +93,15 @@ jobs:
                 throw new Error(`Invalid version number: ${inputVersion}`);
               }
 
-              await releasePatchFor(inputVersion);
+              // only allow overriding the commit SHA for workflow_dispatch events with a provided sha
+              const overrideSha = String(context.payload?.inputs?.override_sha ?? '').trim();
+              const isValidOverride = overrideSha.length === 40;
+
+              if (!isValidOverride && overrideSha.length > 0) {
+                throw new Error(`Invalid override SHA provided: ${overrideSha}`);
+              }
+
+              await releasePatchFor(inputVersion, isValidOverride ? overrideSha : undefined);
             } else { // scheduled release of AUTO_RELEASE_VERSIONS
               await Promise.all(AUTO_RELEASE_VERSIONS.map(releasePatchFor));
             }

.github/workflows/run-tests.yml

@@ -12,7 +12,7 @@ on:
 
 jobs:
   files-changed:
-    if: ${{ !contains(github.event.pull_request.labels.*.name,'ci skip') }}
+    if: ${{ !contains(github.event.pull_request.labels.*.name,'ci skip') && !contains(github.event.pull_request.labels.*.name,'minimal-tests') }}
     name: Check which files changed
     runs-on: ubuntu-22.04
     timeout-minutes: 5
@@ -25,6 +25,10 @@ jobs:
       frontend_all: ${{ steps.changes.outputs.frontend_all }}
       frontend_sources: ${{ steps.changes.outputs.frontend_sources }}
       documentation: ${{ steps.changes.outputs.documentation }}
+      embedding_documentation: ${{ steps.changes.outputs.embedding_documentation }}
+      embedding_sdk_components: ${{ steps.changes.outputs.embedding_sdk_components }}
+      embedding_sdk_host_sample_apps: ${{ steps.changes.outputs.embedding_sdk_host_sample_apps }}
+      embedding_sdk_ci: ${{ steps.changes.outputs.embedding_sdk_ci }}
     steps:
       - uses: actions/checkout@v4
       - name: Test which files changed
@@ -36,51 +40,19 @@ jobs:
           list-files: csv
           filters: .github/file-paths.yaml
 
-  static-viz-files-changed:
-    name: Check whether static-viz files changed
-    if: ${{ !contains(github.event.pull_request.labels.*.name,'ci skip') }}
-    runs-on: ubuntu-22.04
-    timeout-minutes: 10
-    outputs:
-      static_viz: ${{ steps.static_viz.outputs.static_viz }}
-    steps:
-      - uses: actions/checkout@v4
-      - name: Prepare front-end environment
-        uses: ./.github/actions/prepare-frontend
-      - name: Prepare back-end environment
-        uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "files-changed"
-      - name: Build static-viz frontend
-        run: yarn build-static-viz
-        env:
-          MB_EDITION: ee
-      - name: Upload Static Viz Bundle Artifact
-        uses: actions/upload-artifact@v4
-        with:
-          name: static-viz-bundle-${{ github.sha }}
-          path: resources/frontend_client/app/dist
-
-      - name: Check for static viz changes
-        uses: dorny/paths-filter@v3.0.0
-        id: static_viz
-        with:
-          token: ${{ github.token }}
-          filters: .github/static-viz-sources.yaml
-
   uberjar:
-    needs: [files-changed, static-viz-files-changed]
+    needs: [files-changed]
     if: ${{ !cancelled() && !(needs.files-changed.outputs.documentation == 'true' && needs.files-changed.outputs.backend_all == 'false' && needs.files-changed.outputs.frontend_all == 'false' && needs.files-changed.outputs.e2e_all == 'false') }}
     uses: ./.github/workflows/uberjar.yml
     secrets: inherit
 
   backend-tests:
     if: ${{ !cancelled() && needs.files-changed.result == 'success' }}
-    needs: [files-changed, static-viz-files-changed]
+    needs: [files-changed]
     uses: ./.github/workflows/backend.yml
     secrets: inherit
     with:
-      skip: ${{ needs.files-changed.outputs.backend_all != 'true' && needs.static-viz-files-changed.outputs.static_viz != 'true' }}
+      skip: ${{ needs.files-changed.outputs.backend_all != 'true' }}
 
   semantic-search-tests:
     needs: files-changed
@@ -124,18 +96,50 @@ jobs:
       # list of changed specs files, then only specs are changed and we can run only those specs
       specs: ${{ needs.files-changed.outputs.e2e_all_files == needs.files-changed.outputs.e2e_specs_files && needs.files-changed.outputs.e2e_specs_files || '' }}
 
+  embedding-sdk-package:
+    needs: [files-changed]
+    if: |
+      !cancelled() &&
+      (needs.files-changed.outputs.embedding_sdk_components == 'true' ||
+      needs.files-changed.outputs.frontend_sources == 'true' ||
+      needs.files-changed.outputs.embedding_documentation == 'true' ||
+      needs.files-changed.outputs.embedding_sdk_ci == 'true')
+    uses: ./.github/workflows/embedding-sdk-package-build.yml
+    secrets: inherit
+
   sdk-tests:
-    needs: files-changed
+    needs: [uberjar, embedding-sdk-package, files-changed]
+    if: always()
     uses: ./.github/workflows/embedding-sdk.yml
     secrets: inherit
+    with:
+      cli-snippets-type-check: ${{ needs.files-changed.outputs.frontend_sources == 'true' }}
+      docs-snippets-type-check: ${{ needs.files-changed.outputs.embedding_documentation == 'true' }}
+      documentation-validation: ${{ needs.files-changed.outputs.frontend_sources == 'true' }}
+      component-tests: ${{ needs.files-changed.outputs.embedding_sdk_components == 'true' }}
+      host-sample-apps-tests: ${{ needs.files-changed.outputs.embedding_sdk_host_sample_apps == 'true' }}
+      skip: ${{ needs.embedding-sdk-package.result != 'success' || needs.uberjar.result != 'success' }}
 
   pr-env:
     needs: [uberjar]
     if: |
       !cancelled() &&
-      contains(github.event.pull_request.labels.*.name, 'PR-Env')
+      contains(github.event.pull_request.labels.*.name, 'PR-Env') &&
+      !contains(github.event.pull_request.labels.*.name, 'PR-Env-analytics-dev')
     uses: ./.github/workflows/pr-env.yml
     secrets: inherit
+    with:
+      analytics_dev_mode: false
+
+  pr-env-analytics-dev:
+    needs: [uberjar]
+    if: |
+      !cancelled() &&
+      contains(github.event.pull_request.labels.*.name, 'PR-Env-analytics-dev')
+    uses: ./.github/workflows/pr-env.yml
+    secrets: inherit
+    with:
+      analytics_dev_mode: true
 
   containerize:
     needs: [uberjar]

.github/workflows/snyk.yml

@@ -23,8 +23,6 @@ jobs:
       - uses: actions/checkout@v4
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: 'snyk'
       - name: Generate all pom.xml
         run: .github/scripts/write-poms.sh
       - name: Upload pom files

.github/workflows/translation-update.yml

@@ -29,8 +29,6 @@ jobs:
         uses: ./.github/actions/prepare-frontend
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: manage-translations
       - name: install gettext
         run: sudo apt-get install gettext
 

.github/workflows/tusk-sanity-check.yml

@@ -46,8 +46,6 @@ jobs:
         uses: ./.github/actions/prepare-frontend
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: "cljs"
       - run: yarn build-pure:cljs
         name: Compile CLJS
 

.github/workflows/uberjar.yml

@@ -134,7 +134,6 @@ jobs:
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
         with:
-          m2-cache-key: uberjar
           java-version: "${{ needs.get-build-requirements.outputs.java_version || 21 }}"
       - name: Build
         run: ./bin/build.sh
@@ -147,7 +146,13 @@ jobs:
     runs-on: ubuntu-22.04
     name: Is ${{ matrix.edition }} (java ${{ matrix.java-version }}) healthy?
     needs: [build, check-existing-artifacts]
-    if: always() && !cancelled() && (needs.build.result == 'success' || (needs.check-existing-artifacts.outputs.should_build == 'false' && needs.check-existing-artifacts.outputs.ee_exists == 'true' && needs.check-existing-artifacts.outputs.oss_exists == 'true'))
+    # We only care about the explicit JAR health check on push to `master` and the `release-` branches.
+    # JARs created in random branches on PRs are 99% of the time treated as ephemeral, and used only in e2e tests.
+    # E2E tests already have a step where they wait for the instance health check.
+    if: |
+      always() && !cancelled() &&
+      (github.ref_name == 'master' || startsWith(github.ref_name, 'release-')) &&
+      (needs.build.result == 'success' || (needs.check-existing-artifacts.outputs.should_build == 'false' && needs.check-existing-artifacts.outputs.ee_exists == 'true' && needs.check-existing-artifacts.outputs.oss_exists == 'true'))
     timeout-minutes: 10
     strategy:
       matrix:

.github/workflows/upgrade-deps.yml

@@ -27,8 +27,6 @@ jobs:
           token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
       - name: Prepare back-end environment
         uses: ./.github/actions/prepare-backend
-        with:
-          m2-cache-key: 'update-be-deps'
       - name: Open PRs to Upgrade Backend Deps
         uses: ./.github/actions/upgrade-backend-deps
         with:

.github/workflows/whitespace.yml

@@ -16,7 +16,5 @@ jobs:
     - uses: actions/checkout@v4
     - name: Prepare back-end environment
       uses: ./.github/actions/prepare-backend
-      with:
-        m2-cache-key: 'whitespace-linter'
     - run: clojure -T:whitespace-linter lint
       name: Run Whitespace Linter

.gitignore

@@ -104,6 +104,7 @@ dev/serialization_deltas/
 *.tsbuildinfo
 /trace-output/
 /temp-types/
+/frontend/src/metabase-types/openapi
 
 **/cypress_sample_database.json
 **/cypress_sample_instance_data.json