Compare commits
4 Commits
python-sec
...
thin-metad
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
25982e240c | ||
|
|
3f0227b5d9 | ||
|
|
cd2b3ac3eb | ||
|
|
194695b1a9 |
@@ -22,6 +22,7 @@ You understand Metabase's driver architecture built on Clojure multimethods with
|
||||
:bigquery-cloud-sdk, :databricks, :clickhouse, :athena, :sparksql,
|
||||
:presto-jdbc / :starburst, :vertica, :sqlite, :h2
|
||||
:mongo (non-SQL, custom protocol)
|
||||
:druid (non-SQL, REST-based)
|
||||
```
|
||||
|
||||
Key driver extension points you know intimately:
|
||||
|
||||
1
.claude/commands/fix-flakey-test.md
Normal file
1
.claude/commands/fix-flakey-test.md
Normal file
@@ -0,0 +1 @@
|
||||
Fix flakey test $ARGUMENTS
|
||||
@@ -44,22 +44,6 @@ This guide covers Clojure and ClojureScript coding conventions for Metabase. See
|
||||
- Lines ≤ 120 characters
|
||||
- No blank non-comment lines within definition forms (except pairwise `let`/`cond`)
|
||||
|
||||
**Java interop: type-hint parameters in the arglist; prefer `ex-*` helpers:**
|
||||
|
||||
```clojure
|
||||
;; bad — reflection warning, calls Throwable methods directly
|
||||
(defn root-cause [e]
|
||||
(if-let [cause (.getCause e)] (recur cause) e))
|
||||
|
||||
;; good
|
||||
(defn- root-cause [^Throwable e]
|
||||
(if-let [cause (ex-cause e)]
|
||||
(recur cause)
|
||||
e))
|
||||
```
|
||||
|
||||
Use `ex-cause`/`ex-message`/`ex-data`/`ex-info` rather than `.getCause`/`.getMessage`/`(.getData ^ExceptionInfo …)` where possible.
|
||||
|
||||
## Style Conventions
|
||||
|
||||
**Keywords and Metadata:**
|
||||
@@ -67,37 +51,6 @@ Use `ex-cause`/`ex-message`/`ex-data`/`ex-info` rather than `.getCause`/`.getMes
|
||||
- Prefer namespaced keywords for internal use: `:query-type/normal` not `:normal`
|
||||
- Tag variables with `:arglists` metadata if they're functions but wouldn't otherwise have it
|
||||
|
||||
**Time/date: `java-time.api :as t` and `u.date`, not `java.time.*` direct interop:**
|
||||
|
||||
```clojure
|
||||
;; bad — Claude default, requires java.time interop
|
||||
(:import (java.time LocalDate))
|
||||
... (LocalDate/now) ...
|
||||
|
||||
;; good
|
||||
(:require
|
||||
[java-time.api :as t]
|
||||
[metabase.util.date-2 :as u.date])
|
||||
... (t/local-date) ...
|
||||
... (u.date/parse "2024-01-01") ...
|
||||
```
|
||||
|
||||
Only `(:import java.time.X)` when interop with a Java method genuinely needs the class.
|
||||
|
||||
**Check non-emptiness with `seq`, not `count`:**
|
||||
|
||||
```clojure
|
||||
;; bad — `count` realizes the whole seq into memory just to test emptiness
|
||||
(let [prompts (map build-prompt items)]
|
||||
(when (pos? (count prompts))
|
||||
(t2/insert! :model/Prompt prompts)))
|
||||
|
||||
;; good — `seq` realizes one cell; `insert!` streams the rest
|
||||
(let [prompts (map build-prompt items)]
|
||||
(when (seq prompts)
|
||||
(t2/insert! :model/Prompt prompts)))
|
||||
```
|
||||
|
||||
## Tests
|
||||
|
||||
**Organization:**
|
||||
@@ -109,33 +62,6 @@ Only `(:import java.time.X)` when interop with a Java method genuinely needs the
|
||||
|
||||
- Mark pure function tests `^:parallel`
|
||||
|
||||
**Redefining `defn`-vars under test:**
|
||||
|
||||
Use `mt/with-dynamic-fn-redefs` instead of `with-redefs` when stubbing `defn`/`defn-` vars — plain `with-redefs` is not parallel-test-safe and a clj-kondo hook (`:metabase/prefer-with-dynamic-fn-redefs`, see `.clj-kondo/config.edn`) will warn. `with-redefs` is still fine for `def` / dynamic var rebinding.
|
||||
|
||||
```clojure
|
||||
;; bad — flagged by the linter, not safe under :parallel tests
|
||||
(with-redefs [my.ns/some-fn (fn [_] :stub)] (do-test))
|
||||
|
||||
;; good
|
||||
(mt/with-dynamic-fn-redefs [my.ns/some-fn (fn [_] :stub)] (do-test))
|
||||
```
|
||||
|
||||
**Async tests: no bare `Thread/sleep` — use `deref` with a timeout sentinel:**
|
||||
|
||||
A fixed sleep is the #1 source of CI flake. Tests should fail fast and loud when a contract is violated, not hang or sleep through the bug.
|
||||
|
||||
```clojure
|
||||
;; bad — flaky on slow CI, silent on actual failure
|
||||
(Thread/sleep 300)
|
||||
(is (= :expected @seen))
|
||||
|
||||
;; good — fails fast with a clear message
|
||||
(let [v (deref some-promise 1000 ::timeout)]
|
||||
(is (not= ::timeout v) "promise never delivered")
|
||||
(is (= :expected v)))
|
||||
```
|
||||
|
||||
## Modules
|
||||
|
||||
**OSS Modules:**
|
||||
@@ -160,45 +86,6 @@ A fixed sleep is the #1 source of CI flake. Tests should fail fast and loud when
|
||||
|
||||
- Do not cheat module linters with `:clj-kondo/ignore [:metabase/modules]`
|
||||
|
||||
**Enterprise/OSS split: use `defenterprise`, not protocols or runtime feature checks:**
|
||||
|
||||
The OSS namespace defines the fallback body and names the EE namespace; the EE namespace overrides with the same fn name and a `:feature` clause naming the premium-features flag that gates the override. Both files use the same docstring shape — describe behavior on both sides.
|
||||
|
||||
```clojure
|
||||
;; src/metabase/remote_sync/core.clj (OSS)
|
||||
(defenterprise collection-editable?
|
||||
"Returns whether remote-synced collections are editable.
|
||||
Always true on OSS; EE consults the remote-sync settings."
|
||||
metabase-enterprise.remote-sync.core
|
||||
[_collection]
|
||||
true)
|
||||
|
||||
;; enterprise/backend/src/metabase_enterprise/remote_sync/core.clj (EE)
|
||||
(defenterprise collection-editable?
|
||||
"EE override — respects the synced-collections-editable? setting."
|
||||
:feature :remote-sync
|
||||
[collection]
|
||||
(or (synced-collections-editable?) (not (synced? collection))))
|
||||
```
|
||||
|
||||
`defenterprise-schema` is the malli-schema variant. Required from `[metabase.premium-features.core :refer [defenterprise defenterprise-schema]]`. The OSS body is what runs when the feature flag is off — never write `(when (premium/has-feature? ...) ...)` in user code; let `defenterprise` route.
|
||||
|
||||
**Security gates fail *open* under `:feature` — use `:feature :none` and check manually.** The router (`dynamic-ee-oss-fn`) calls the EE body only when both the EE namespace is loaded *and* the token currently grants the feature; otherwise it falls back to the OSS body. For a permissive default (`collection-editable?` → `true`) that's fine. But for a *restriction* — sandboxing, impersonation, row-level filtering — the OSS fallback is the unrestricted answer, so a temporarily lost/expired token silently downgrades a sandboxed user to full access. **Fail open.** When the EE override is what *decides whether to restrict*, gate it with `:feature :none` (which always routes to the EE body once EE code is loaded) and check the feature explicitly inside, so token loss fails *closed*:
|
||||
|
||||
```clojure
|
||||
;; enterprise/.../sandbox/api/util.clj — the gate that detects whether a user is sandboxed
|
||||
(defenterprise enforced-sandboxes-for-user
|
||||
"..."
|
||||
;; :none so we can decide whether sandboxing is *configured* even when the feature
|
||||
;; isn't currently available — letting us BLOCK requests that would otherwise be
|
||||
;; sandboxed, instead of returning the permissive OSS fallback. Fail closed.
|
||||
:feature :none
|
||||
[user-id]
|
||||
...)
|
||||
```
|
||||
|
||||
Reach for this only for the detection/enforcement decision point. Downstream functions that merely *act* on an already-made decision can keep their normal `:feature :sandboxes` gate.
|
||||
|
||||
## REST API Endpoints
|
||||
|
||||
**Required Elements:**
|
||||
@@ -207,20 +94,6 @@ Reach for this only for the detection/enforcement decision point. Downstream fun
|
||||
- All endpoints need Malli schemas for parameters (detailed and complete)
|
||||
- All new REST API endpoints MUST HAVE TESTS
|
||||
|
||||
**Endpoint shape: `api.macros/defendpoint`, four positional arglist:**
|
||||
|
||||
```clojure
|
||||
(api.macros/defendpoint :get "/:key/callback"
|
||||
"OIDC callback for a specific provider."
|
||||
[{provider-key :key} :- [:map [:key ProviderKey]]
|
||||
_query-params
|
||||
_body
|
||||
request]
|
||||
(oidc-integration/sso-callback provider-key request))
|
||||
```
|
||||
|
||||
The arglist is **always 4 positional bindings** in this order: `route-params`, `query-params`, `body`, `request`. Use `_`-prefixed names for unused ones. Each one gets a Malli schema via `:- <schema>` (omit when `_`). The HTTP verb is a keyword (`:get`/`:post`/`:put`/`:delete`), not a symbol. Required as `[metabase.api.macros :as api.macros]`.
|
||||
|
||||
**Naming Conventions:**
|
||||
|
||||
- Query parameters use kebab-case
|
||||
@@ -232,20 +105,6 @@ The arglist is **always 4 positional bindings** in this order: `route-params`, `
|
||||
- `GET` endpoints should not have side effects (except analytics)
|
||||
- `defendpoint` forms should be small wrappers around Toucan model code
|
||||
|
||||
**Closed response maps:**
|
||||
|
||||
Tag response Malli `:map` schemas with `{:closed true}` so the generated OpenAPI / TypeScript types reject extra keys. Open maps emit `additionalProperties: true`, which produces weak frontend types and makes discriminated unions ineffective. Tighten domain types in the same pass (`pos-int?` rather than `:int` when zero branches elsewhere).
|
||||
|
||||
```clojure
|
||||
;; weak
|
||||
{:status 200 :body [:map [:status [:= :generated]] [:prompt_count :int]]}
|
||||
|
||||
;; tight
|
||||
{:status 200 :body [:map {:closed true}
|
||||
[:status [:= :generated]]
|
||||
[:prompt_count pos-int?]]}
|
||||
```
|
||||
|
||||
## MBQL (Metabase Query Language)
|
||||
|
||||
**Restrictions:**
|
||||
@@ -285,215 +144,10 @@ Tag response Malli `:map` schemas with `{:closed true}` so the generated OpenAPI
|
||||
|
||||
**Linter Suppressions:**
|
||||
|
||||
The right shape is a **reader-discard map** placed directly above the form being silenced, listing the rule keywords explicitly:
|
||||
|
||||
```clojure
|
||||
#_{:clj-kondo/ignore [:metabase/validate-defendpoint-route-uses-kebab-case
|
||||
:metabase/validate-defendpoint-has-response-schema]}
|
||||
(api.macros/defendpoint :post "/api_key" ...)
|
||||
```
|
||||
|
||||
For a single require entry, hang it as metadata on that entry:
|
||||
|
||||
```clojure
|
||||
(:require
|
||||
^{:clj-kondo/ignore [:deprecated-namespace :discouraged-namespace]}
|
||||
[metabase.query-processor.store :as qp.store])
|
||||
```
|
||||
|
||||
Common Metabase-specific rules you'll see: `:metabase/modules`, `:metabase/validate-defendpoint-route-uses-kebab-case`, `:metabase/validate-defendpoint-has-response-schema`, `:metabase/prefer-with-dynamic-fn-redefs`, `:deprecated-namespace`, `:discouraged-namespace`. Don't use the keyword form (`#_:clj-kondo/ignore`); always the map form so it's grep-able by which rule(s) are suppressed.
|
||||
- Use proper format for kondo suppressions
|
||||
- No `#_:clj-kondo/ignore` (keyword form)
|
||||
|
||||
**Configurable Options:**
|
||||
|
||||
- Don't define configurable options that can only be set with environment variables
|
||||
- Use `:internal` `defsetting` instead
|
||||
|
||||
|
||||
## Metabase Helpers — Quick Reference
|
||||
|
||||
Metabase and its libraries have **many** helper functions to make code more readable. Don't hand-roll things that already exist! Reach for the right helper *the first time*.
|
||||
|
||||
Organized by **problem you're solving**, not by namespace.
|
||||
|
||||
## "I wrote this — use that instead" cheat sheet
|
||||
|
||||
*Authors: prefer the right column. Reviewers: flag the left column in PRs.*
|
||||
|
||||
| You wrote | Reach for | When NOT to |
|
||||
| -------------------------------------------------------------------- | ------------------------------------ | ----------------------------------------------------------------- |
|
||||
| `(vec (keep f xs))` | `(u/keepv f xs)` | — |
|
||||
| `(let [x (...)] (side-effect!) x)` | `(u/prog1 (...) (side-effect!))` | — (`<>` anaphor binds the result) |
|
||||
| `(into {} (map (juxt k identity)) xs)` | `(u/index-by k xs)` | — |
|
||||
| `(into {} (map (juxt k vf)) xs)` | `(u/index-by k vf xs)` | — |
|
||||
| `#(compare %2 %1)` | `u/reverse-compare` | — |
|
||||
| `(:col (t2/select-one [Model :col] …))` | `(t2/select-one-fn :col Model …)` | — |
|
||||
| `(or (:k m) default)` | `(:k m default)` | **The value can be present-but-nil** (hydrated DB null, etc.) |
|
||||
| `(let [x (...)] (if x then else))` | `(if-let [x (...)] then else)` | — |
|
||||
| `(filter #(= :a (:type %)) coll)` + same for `:b` … | `(group-by :type coll)` then dispatch | — |
|
||||
| `(->> xs (map f) (remove g) (mapv h))` | `(into [] (comp (map f) (remove g) (map h)) xs)` | when readability of the threading wins |
|
||||
| nested `let` / `if` / `if-let` | `b/cond` with `:let` | a 2-level nest may be clearer as-is |
|
||||
| `(let [{:keys [k]} arg] …)` as first form of body | destructure in the arg list | when the destructure also references something computed in `let` |
|
||||
| `(or (:k m) (get m (name k)))` per field | normalize once: `(update-keys m keyword)` then plain `:k` | when you genuinely need both shapes per-field |
|
||||
|
||||
## `metabase.util` (commonly aliased `[metabase.util :as u]`)
|
||||
|
||||
Sequences / collections:
|
||||
- `(u/keepv f coll)` — like `keep` but returns a vector (eager).
|
||||
- `(u/index-by kf coll)` / `(u/index-by kf vf coll)` — `{(kf x) (vf x)}` map.
|
||||
|
||||
Strings (locale-safe):
|
||||
- `(u/lower-case-en s)`, `(u/upper-case-en s)` — DO use these instead of `clojure.string/lower-case`/`upper-case` whenever the result feeds into matching/comparison; the locale-default versions break on Turkish `i`.
|
||||
- `(u/qualified-name x)` — keyword → `"ns/name"` or `"name"`; symbol → `(name x)`; string → identity.
|
||||
|
||||
Control flow:
|
||||
- `(u/prog1 first-form & body)` — anaphoric. Result of `first-form` is bound to `<>`. Returns first-form's value. Use for "do a thing, side-effect using its result, return the thing."
|
||||
- `(u/the-id x)` — get an entity id from an int or an instance.
|
||||
|
||||
Comparators:
|
||||
- `(u/reverse-compare x y)` — `(compare y x)`. Pass as a sort comparator: `(sort-by f u/reverse-compare coll)`.
|
||||
|
||||
Other:
|
||||
- `(u/conflicting-keys m1 m2)` — keys present in both with different values.
|
||||
|
||||
The full `metabase.util.cljc` is ~1300 lines — when in doubt, grep it.
|
||||
|
||||
## `metabase.util.namespaces` (aliased `shared.ns`)
|
||||
|
||||
- `(shared.ns/import-fns [other-ns f1 f2] [another-ns f3])` — re-export defns from another ns through the current ns. The `:require [other-ns :as other-ns]` alias **must stay** — clj-kondo may flag it as unused but it's needed at macro expansion. Add a `;; needed for import-fns` comment so it doesn't get auto-stripped.
|
||||
|
||||
## Toucan 2 (aliased `[toucan2.core :as t2]`)
|
||||
|
||||
Single-field reads (avoids fetching whole rows):
|
||||
- `(t2/select-one-fn :col Model :id 123)` — projection-friendly: when `f` is a keyword, only that column is fetched. Use instead of `(:col (t2/select-one [Model :col] …))`.
|
||||
- `(t2/select-fn-set f Model & conditions)` — set of `(f row)` values.
|
||||
- `(t2/select-fn-vec f Model & conditions)` — vector of `(f row)` values.
|
||||
- `(t2/select-pks-set Model & conditions)` — set of PKs.
|
||||
|
||||
Existence / counts:
|
||||
- `(t2/exists? Model & conditions)` → boolean.
|
||||
- `(t2/count Model & conditions)` → long.
|
||||
|
||||
Writes:
|
||||
- `(t2/insert-returning-instances! Model row-or-rows)` — returns the rows with PKs set.
|
||||
- `(t2/insert-returning-pks! Model row-or-rows)` — just the PKs.
|
||||
- `(t2/update! Model id-or-conditions changes)`.
|
||||
- `(t2/delete! Model id-or-conditions)`.
|
||||
|
||||
Transactions:
|
||||
- `(t2/with-transaction [t-conn] body…)` — opens a txn on the app DB conn.
|
||||
|
||||
Hydration:
|
||||
- `(t2/hydrate instances :key1 :key2)` — invokes `:batched-hydrate` methods.
|
||||
- Define via `(methodical/defmethod t2/batched-hydrate [:model/Foo :the-key] [_ k rows] …)`.
|
||||
|
||||
Define a model:
|
||||
- `(methodical/defmethod t2/table-name :model/Foo [_] :foo_table)`
|
||||
- `(t2/deftransforms :model/Foo {:col mi/transform-json})` — common: `transform-json`, `transform-secret-value` (encrypts), `transform-keyword`.
|
||||
|
||||
## Better-cond (`[better-cond.core :as b]`)
|
||||
|
||||
`(b/cond …)` like `cond` but supports interleaved `:let [bindings]` and `:when` clauses. Lets you flatten cascading `let`/`if`/`if-let`/`let` into a linear spine.
|
||||
|
||||
```clojure
|
||||
(b/cond
|
||||
:let [thread (load-thread id)]
|
||||
(nil? thread) :not-found
|
||||
|
||||
:let [exploration (load-exploration (:exploration-id thread))
|
||||
reason (gate-check exploration)]
|
||||
reason
|
||||
(skip! reason)
|
||||
|
||||
:let [pool (build-pool thread)]
|
||||
(empty? pool) :skip-no-data
|
||||
|
||||
:else
|
||||
(run! thread pool))
|
||||
```
|
||||
|
||||
Use it when a function would otherwise be a staircase of nested forms. Don't reach for it for a single `if`/`let`.
|
||||
|
||||
## Schemas — `mr/def` for shared, inline for single-use
|
||||
|
||||
When a schema is used in 2+ places or has a name worth documenting, extract it into the registry via `mr/def`:
|
||||
|
||||
```clojure
|
||||
(:require [metabase.util.malli.registry :as mr])
|
||||
|
||||
(mr/def ::options
|
||||
"Options for Pulse channel rendering."
|
||||
[:map
|
||||
[:channel.render/include-buttons? {:description "default: false" :optional true} :boolean]])
|
||||
```
|
||||
|
||||
`mr/def` keys are always `::namespaced` keywords; downstream callers reference via the fully-qualified keyword. Single-use schemas stay inline in the `mu/defn`'s `:-` annotation — no need to extract.
|
||||
|
||||
## Malli — `metabase.util.malli`
|
||||
|
||||
- `(mu/defn my-fn :- :string [x :- :int, y :- [:maybe :string]] …)` — inline schemas; runtime validation in dev, no-op in prod by default.
|
||||
- `(mu/disable-enforcement & body)` — wrap test/perf-sensitive blocks.
|
||||
- Common malli schemas: `:int`, `:string`, `:boolean`, `:keyword`, `:map`, `[:maybe T]`, `[:sequential T]`, `[:enum :a :b]`, `[:map [:k T]]`, `pos-int?`, `nat-int?`.
|
||||
|
||||
## i18n — `[metabase.util.i18n :refer [tru deferred-tru]]`
|
||||
|
||||
- `(tru "Hello {0}" name)` — translate at call time. Use in request-handling code.
|
||||
- `(deferred-tru "…")` — translate at read time. Use in static contexts: `defsetting :description`, `defn` metadata `:description`, top-level `def`s of user-visible labels.
|
||||
- ICU MessageFormat: doubled apostrophe for a literal: `(tru "it''s ready")`. Numbered placeholders: `{0}`, `{1}`.
|
||||
- **Don't translate** developer-only debug strings (REPL helper text, internal logs).
|
||||
|
||||
## Logging — `[metabase.util.log :as log]`
|
||||
|
||||
- Levels: `trace`, `debug`, `info`, `warn`, `error`.
|
||||
- Two flavors per level:
|
||||
- `(log/warn "msg")` or `(log/warn e "msg")` — exception first if present.
|
||||
- `(log/warnf "fmt %s" arg)` or `(log/warnf e "fmt %s" arg)` — printf-style.
|
||||
- Don't `(log/warn (format "fmt %s" arg))` — use `warnf`.
|
||||
|
||||
## Map / key normalization
|
||||
|
||||
When parsing loosely-typed external data (LLM JSON, request bodies) that may have **either** string or keyword keys:
|
||||
|
||||
- `(update-keys m keyword)` — top-level keys → keywords (Clojure 1.11+, in `clojure.core`).
|
||||
- `(clojure.walk/keywordize-keys m)` — recursive (use sparingly; can over-keywordize value strings if they happen to be in nested maps).
|
||||
- After normalizing once, downstream code uses plain `(:k m)` — don't write `(or (:k m) (get m (name k)))` at every use site.
|
||||
|
||||
Other useful map ops:
|
||||
- `(update-vals m f)` — Clojure 1.11+, in `clojure.core`.
|
||||
- `(medley.core/map-vals f m)`, `(m/map-keys f m)` — older alternatives. Medley pinned at **1.4** in this repo — `m/index-of` and some other newer fns are NOT available.
|
||||
|
||||
## Comparators & sorting
|
||||
|
||||
- `(sort-by f u/reverse-compare coll)` — descending.
|
||||
- Vector comparator: `(sort-by (juxt :primary :secondary) coll)`.
|
||||
- Use `(- x)` on a primary numeric key to invert: `(sort-by (juxt #(- (:score %)) :id) coll)` (handles ties on ascending `:id`).
|
||||
|
||||
## Anti-patterns to avoid
|
||||
|
||||
- **`(let [x (do-thing!)] (other-thing! x) x)`** — use `u/prog1`.
|
||||
- **`(vec (keep …))`, `(vec (map …))`** — use `u/keepv` or a transducer `(into [] (comp …) coll)`.
|
||||
- **Multiple `filter` passes over the same coll for different `:type`s** — one `(group-by :type coll)`.
|
||||
- **Per-field string/keyword dual-lookup** — normalize once with `update-keys keyword`.
|
||||
- **Reaching into another module's tables via raw `:left-join` SQL** — expose a function on the owning model namespace and call that.
|
||||
- **Snake-case keys on a massaged in-memory map** — appdb-row only. Convert at the parse boundary (e.g., when a json response becomes an internal record).
|
||||
- **`(double (:k m 0.0))` on a hydrated DB column** — present-but-nil NPEs. Use `(double (or (:k m) 0.0))`.
|
||||
- **`defmulti` with every impl on one page** — extension story is a fiction. Use a private lookup map + a private fn.
|
||||
- **`(pos? (count xs))` to test a lazy `xs` for non-emptiness** — `count` forces the whole seq into memory. Use `(seq xs)`. (Rebuilding the seq — `(map f xs)` twice — is the only thing that recomputes; a bound seq caches.)
|
||||
- **Plain `with-redefs` on a `defn` var in a test** — not parallel-safe; the kondo hook will warn. Use `mt/with-dynamic-fn-redefs`.
|
||||
- **`(Thread/sleep N)` in an async test** — flake source #1. `(deref p timeout-ms ::timeout)` and assert `not= ::timeout`.
|
||||
- **Hand-rolled `(str/split host #":")` for origin/host comparison** — mis-handles IPv6. Use `mw.security/parse-url` (or `try-parse-url` for client-controlled input).
|
||||
- **`(subs (pr-str huge) 0 N)` on a log path** — the multi-MB string is already allocated. Bind `*print-length*`/`*print-level*` around `pr-str` instead.
|
||||
- **New retry loop / new "find this column" helper / new transform-source-type predicate** — search first. `metabase.util.retry/with-retry`, `metabase.lib.equality/find-matching-column`, and `metabase.transforms.feature-gating/any-transforms-enabled?` already exist (along with many others). Reviewers ask "isn't there already an X for this?" more than any single idiom.
|
||||
- **Plain `defn` when the function has any real contract** — `mu/defn` (or `mu/defn-`) with `:- schema` on the return and each param is the house default; reserve plain `defn` for trivial dev/helper code.
|
||||
- **`(:import (java.time …))` + `(LocalDate/now)`** in new code — use `[java-time.api :as t]` + `[metabase.util.date-2 :as u.date]`. Direct `java.time` interop only when a Java API genuinely needs the class.
|
||||
- **Vanilla `defmethod`** — use `methodical/defmethod` (it takes a docstring; the codebase uses it uniformly).
|
||||
- **`(when (premium/has-feature? :flag) …)` in code** — gate via `defenterprise` so OSS/EE bodies live in their respective namespaces; never sprinkle runtime feature checks.
|
||||
- **No `(set! *warn-on-reflection* true)` after the `ns` form** in a new `.clj` file — every `.clj` source file in this repo has it (not `.cljc`).
|
||||
- **`(ns … (:require [a :as a] [b :as b]))` on one line; `(:import java.io.File java.net.URI)` bare** — both wrong shapes. Requires one per line under `:require`, imports as `(Package Class)` lists.
|
||||
- **Inventing a fresh `:as` alias for `metabase.util`, `metabase.util.malli`, `metabase.util.log`, `java-time.api`, etc.** — use the canonical alias (see the table in Style Conventions). Mixing aliases makes `grep`-based discovery worse.
|
||||
|
||||
## When in doubt
|
||||
|
||||
- Run `grep -n "defn " src/metabase/util.cljc | head -80` to scan helpers.
|
||||
- Run `grep -rn "defmacro\|defn-? [a-z-]*\?\b" src/metabase/util.cljc | head` for predicates.
|
||||
- Check `medley.core/` if it looks "collection-y" — but **Medley is pinned at 1.4** here; new fns aren't available.
|
||||
- Search the codebase for an existing use of the pattern you're about to write — if no one else does it, there's usually a helper.
|
||||
|
||||
@@ -1,69 +0,0 @@
|
||||
# React, Redux, and Data-Layer Patterns
|
||||
|
||||
## API access and RTK Query
|
||||
|
||||
- **Use RTK Query for server data.** Add typed endpoints under `metabase/api` or `metabase-enterprise/api`; don't add ad-hoc `fetch`, service wrappers, or new thunks for ordinary API access.
|
||||
- **Keep API files transport-focused.** Endpoint files should cover request/response typing, URLs, params, light response normalization, and cache tags — not feature business logic.
|
||||
- **Use canonical API types.** Prefer `metabase-types/api`; fix or derive missing types instead of casting endpoint results. Refer to API endpoint implementation in Clojure when building new endpoint connections.
|
||||
- **Tag caches deliberately.** Use `listTag`, `idTag`, `provide*Tags`, and `invalidateTags(error, tags)` where appropriate.
|
||||
- **Guard missing query inputs.** Use `skipToken` or `skip` until all required inputs exist. Build query requirements where the query is created, not in a downstream render path.
|
||||
- **Use lazy queries only for events.** Data needed to render a screen belongs in a regular query hook at the owner component/hook.
|
||||
- **Handle mutations with `unwrap()`.** Prefer `await trigger(args).unwrap()` in submit handlers; otherwise branch on `result.error` immediately.
|
||||
|
||||
## Where data should live
|
||||
|
||||
- **Server data:** RTK Query cache, not duplicated Redux state.
|
||||
- **Form state:** Form library or local component state, not global Redux.
|
||||
- **Transient UI state:** Component state or narrow context, not store-wide flags.
|
||||
- **Cross-route/global tasks:** Redux slice with public selectors.
|
||||
- **Derived values:** Selectors or pure helpers, not stored copies of computed data. Memoized if identity is important.
|
||||
|
||||
## Redux state modeling
|
||||
|
||||
- **Avoid using Redux**. Most problems can be solved with local component state or RTK Query.
|
||||
- **Use Redux only when needed.** Reach for Redux only when distant parts of the app need durable shared state.
|
||||
- **Model async status explicitly.** Use a discriminated union or explicit `status`/`isLoading`/`error`; don't infer loaded or failed state from truthy/falsy `data`.
|
||||
- **Store canonical state only.** Derive filtered lists, labels, permissions, booleans, and display variants in selectors.
|
||||
- **Read through selectors.** Name selectors after what they return (`getCurrentTask`, `getHasPendingMutation`), not how they compute it.
|
||||
- **Add missing public actions/selectors.** Don't reach into another feature's private state shape because an action or selector is missing.
|
||||
- **Use `createSelector` for derived state.** Keep selectors pure, cheap, and stable by reference when consumers depend on identity.
|
||||
|
||||
## React hooks and components
|
||||
|
||||
- **`useMemo` computes render values.** Side effects belong in `useEffect`; event-driven changes belong in event handlers or mutation callbacks.
|
||||
- **Keep effects narrow and idempotent.** Include all dependencies, guard inside the effect, and clean up timers/subscriptions. Complex logic should be split to multiple effects or custom hooks, also consider extracting functional pieces.
|
||||
- **Hooks can be used to encapsulate complex logic.** For example, a hook can normalize data, handle form's state and validation, or a hook can manage a table's pagination and sorting. If component's logic becomes too complex, consider extracting a hook.
|
||||
- **Hooks orchestrate behavior.** Custom hooks should return a small typed result object: data, loading/error state, and callbacks when relevant.
|
||||
- **Don't mirror props/query data into state** unless the consumer can edit that local copy. Keep derived render data as such.
|
||||
- **Use `useCallback` for stable identity needs, but deliberately.** Wrap callbacks passed to memoized children, dependency arrays, context values, or hooks that require stable identity — not every local handler.
|
||||
- **Memoize object/array props only when identity matters.** Prefer simpler plain values otherwise.
|
||||
- **Stories for presentational components**. Create Storybook stories for presentational components to test different states and props. Stories live next to the component.
|
||||
|
||||
## Performance
|
||||
|
||||
- **Optimize only where identity or scale matters.** Prefer clear code first, then add `memo`, `useMemo`, `useCallback`, selectors, virtualization, or split components when render cost, prop identity, or list size justifies it.
|
||||
- **Keep context values stable and avoid frequent updates.** Values updated often should be avoided in context consumed by many components, or stored by reference with a stable context value and explicit subscriptions/selectors.
|
||||
- **Read global state high in the tree.** Let parent components select context/Redux/API state, derive the view model, and pass narrow props top-down. But avoid passing props too deep (reconsider if more thatn 2 levels).
|
||||
- **Avoid unstable props in hot paths.** Don't pass fresh object/array/function literals to memoized children; memoize them or move them outside render when identity matters.
|
||||
- **Render large lists deliberately.** Use pagination, windowing/virtualization, stable keys, and row-level memoization for large or frequently updating lists.
|
||||
- **Consider memoization for leaf components.** If the source of data updates often, but the component doesn't need to re-render, consider memoizing it.
|
||||
|
||||
## Loading, error, and empty states
|
||||
|
||||
- **Every async operation needs loading UI.** Use `LoadingAndErrorWrapper` or `DelayedLoadingAndErrorWrapper` for full-page/panel loading; use disabled controls and inline progress for local actions.
|
||||
- **Every async operation needs an error branch.** Use wrappers or feature-specific alerts for query errors; use `getErrorMessage(error, fallback)` plus a toast/form error for mutation failures.
|
||||
- **Empty is its own state.** Render no-results, empty-list, and permission-limited states intentionally; don't let them look like unfinished loading.
|
||||
- **Combine multiple query states explicitly.** Example: `isLoading = a.isLoading || b.isLoading`; `error = a.error || b.error`.
|
||||
- **Distinguish first load from refresh.** Use `isLoading` for first-load blockers and `isFetching` for background refresh indicators.
|
||||
- **Test all async branches.** Cover loading, error, empty, and success states when adding or changing data access.
|
||||
|
||||
## Review checklist
|
||||
|
||||
- Server data uses a typed RTK Query endpoint with correct cache tags.
|
||||
- Query inputs are guarded with `skipToken`/`skip` until ready.
|
||||
- Global state is justified, minimal, and read through selectors.
|
||||
- Context values are stable; frequently changing values don't force broad re-renders.
|
||||
- Leaf components stay dumb and receive narrow props from parent containers.
|
||||
- Loading, error, empty, and success branches are visible and tested.
|
||||
- Hooks are pure during render; side effects are confined to effects or event handlers.
|
||||
- Mutation failures are surfaced to people using the UI.
|
||||
@@ -2,8 +2,8 @@
|
||||
|
||||
- **Lint:** `bun run lint-eslint-pure`
|
||||
- Run ESLint on the codebase
|
||||
- **Format:** `bun run format`
|
||||
- Format code with oxfmt (`oxfmt --write`); use `bun run lint-format-pure` to check without writing
|
||||
- **Format:** `bun run prettier`
|
||||
- Format code using Prettier
|
||||
- **Type Check:** `bun run type-check-pure`
|
||||
- Run TypeScript type checking
|
||||
|
||||
|
||||
@@ -102,7 +102,3 @@ breakage keeps happening, *that* is the signal a comment was warranted.
|
||||
- End all files with a newline
|
||||
- When editing tabular code, where the columns line up, try to keep them aligned
|
||||
- Spaces on a line with nothing after it is not allowed
|
||||
- After changing module boundaries (adding/removing/renaming a `src` namespace, a cross-module `require`
|
||||
or `:model/X` reference, or a new module), run `./bin/mage fix-modules-config` to regenerate
|
||||
`.clj-kondo/config/modules/config.edn` and keep `metabase.core.modules-test` green. No-op when nothing
|
||||
drifted; see "Module Boundaries" in the project `CLAUDE.md`.
|
||||
|
||||
@@ -22,11 +22,10 @@ Before starting, determine which mode to use:
|
||||
3. **(Conditional)** If after reading the spec a specific assertion or selector is **ambiguous** — you can't tell if it's the right anchor, or whether the test actually verifies what its title claims — briefly grep the related component for the test ID / role / text. **Don't read components by default.** Only do this when there's a real signal of confusion in the spec.
|
||||
4. Scan against the checklist + pattern table below.
|
||||
5. Number all issues sequentially. Skip nits — only flag what's worth fixing.
|
||||
6. **Always** finish with the honest e2e-vs-unit breakdown (see the section of that name below). It's part of every report, not an optional add-on.
|
||||
|
||||
### When to hand off instead of review
|
||||
|
||||
If the spec references an issue (`metabase#NNNNN`) and the user wants to **fix** flakiness or assess whether the test still reproduces the original bug, that's outside the review skill's scope. Point them at the dedicated flake-fixing workflow, which knows to fetch the issue and the resolving PR's diff. The review skill stays focused on "is this test well-written and conformant" — it doesn't fetch external issue context by default.
|
||||
If the spec references an issue (`metabase#NNNNN`) and the user wants to **fix** flakiness or assess whether the test still reproduces the original bug, that's outside the review skill's scope. Point them at `/fix-flakey-test` (or the dedicated flake-fixing workflow), which knows to fetch the issue and the resolving PR's diff. The review skill stays focused on "is this test well-written and conformant" — it doesn't fetch external issue context by default.
|
||||
|
||||
## Review checklist
|
||||
|
||||
@@ -195,41 +194,8 @@ Use the pending review workflow:
|
||||
|
||||
Each comment body starts with `**Issue N: [Brief title]**`.
|
||||
|
||||
## Honest e2e-vs-unit breakdown (always include)
|
||||
|
||||
Every report ends with this section. The numbered issues above are about whether the test is *well-written*; this section is about whether it should be an **e2e test at all**. They're orthogonal — a flawless spec can still be paying e2e prices for component-test value, and that's worth saying out loud.
|
||||
|
||||
The bar is simple: **an `it()` earns its place in e2e only if it exercises a layer you cannot fake without gutting the test's value.** In this codebase that means one or more of:
|
||||
|
||||
- a **real backend query against seeded data** (the assertion depends on the server actually filtering/sorting/bucketing rows, not on a mocked response);
|
||||
- **real rendering you then interact with** — clicking a charted ECharts SVG element at computed coordinates, drag/hit-testing, drill-through;
|
||||
- **routing across screens** — navigation, URL param round-trips that the backend then honors, browser back/forward;
|
||||
- **cross-screen traversal** — list → detail → back, where the value is the seam between screens.
|
||||
|
||||
Conversely, a test belongs in **Jest + RTL** (the cheaper layer this repo already uses for component/unit coverage) when its real subject is frontend-only:
|
||||
|
||||
- **pure frontend logic** — a bucketing/formatting/derivation decision (e.g. "single-day range → bucket by hour") that has one right answer independent of the backend;
|
||||
- **render assertions** — "the page mounts N titled cards and an SVG exists." A mocked dataset response + RTL verifies the titles; that an ECharts SVG renders is library behavior, not your logic;
|
||||
- **state/wiring** — tab → `aria-selected` → active-state, or select → querystring mapping. The *URL gets the param* half is component-level; only the *backend honors the param* half needs e2e.
|
||||
|
||||
Write the breakdown in three honest buckets. Ground each call in **which layers the test actually touches**, not in how the test is named:
|
||||
|
||||
1. **Genuinely must-be-e2e (keep)** — load-bearing. Name the layer(s) only a browser + backend can verify together.
|
||||
2. **Defensible-but-narrowable e2e** — justified as integration, but over-split or doing more boots than the coverage requires. Say what to merge.
|
||||
3. **Should be unit / component tests** — where the real savings are. Name the cheaper layer and what it would assert (often *more precisely* than the e2e test does indirectly).
|
||||
|
||||
Then a one-paragraph **bottom line**: what to keep as e2e, what to push down, and — if you pulled CI timing from `e2e/support/timings.json` for the Performance check — roughly how much of that wall time is *earned* by the must-be-e2e set versus paid by the rest.
|
||||
|
||||
Rules of engagement so this stays honest, not performative:
|
||||
|
||||
- **Don't inflate the keep bucket to look balanced.** If most of the spec is component-test value paying e2e prices, say exactly that. If every test is load-bearing, say that too — a spec where all 12 tests must be e2e is a legitimate outcome, not a failure to find candidates.
|
||||
- **Be specific.** Cite the `it()` titles and line ranges. "Several tests could be unit tests" is useless; "tests 1/17/18 assert only that N titled cards mount — RTL with a mocked dataset" is actionable.
|
||||
- **This is advice to the author, not a blocking issue.** It's a recommendation, separate from the numbered issues. Don't renumber it into the issue list.
|
||||
- **Stay rooted in what exists.** The cheaper layer here is Jest + RTL component/unit tests. Don't invent infrastructure the repo doesn't have, and don't recommend deleting backend-dependent coverage just because part of the test is frontend-only — split it, keep the integration half.
|
||||
|
||||
## Final check
|
||||
|
||||
1. Trim issues that won't materially help the author.
|
||||
2. Verify sequential numbering with no gaps.
|
||||
3. In PR mode, verify each issue was posted as a separate review comment.
|
||||
4. Verify the honest e2e-vs-unit breakdown is present — it's required in every report, even when the answer is "all of these need to be e2e."
|
||||
|
||||
@@ -20,8 +20,8 @@ For EACH styled component, go through every CSS property and ask: "Can this be a
|
||||
**Properties that ARE style props** (use these, not CSS modules):
|
||||
|
||||
- `display` → `display` prop
|
||||
- `color` → `c` prop (`c="core-brand"`, `c="text-primary"`)
|
||||
- `background-color` → `bg` prop (`bg="background_page-primary"`)
|
||||
- `color` → `c` prop (`c="brand"`, `c="text-primary"`)
|
||||
- `background-color` → `bg` prop (`bg="background-primary"`)
|
||||
- `font-size` → `fz` prop (`fz="md"`)
|
||||
- `font-weight` → `fw` prop (`fw="bold"`)
|
||||
- `line-height` → `lh` prop (`lh="md"`)
|
||||
@@ -57,7 +57,7 @@ For EACH styled component, go through every CSS property and ask: "Can this be a
|
||||
align="center" /* style prop */
|
||||
gap="sm" /* style prop */
|
||||
p="md" /* style prop */
|
||||
bg="background_page-primary" /* style prop */
|
||||
bg="background-primary" /* style prop */
|
||||
>
|
||||
```
|
||||
|
||||
@@ -149,19 +149,19 @@ Read the `.styled.tsx` file AND every component that imports from it. Understand
|
||||
|
||||
For each styled component, apply the Mantine-First Decision Gate above. Then determine the migration target:
|
||||
|
||||
| Emotion Pattern | Migration Target |
|
||||
| ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `styled.div` with only layout/spacing/color | `Box`, `Flex`, `Stack`, or `Group` with style props. **NO CSS module needed.** |
|
||||
| `styled.div` with flexbox column | `Stack` component with style props |
|
||||
| `styled.div` with flexbox row | `Flex` or `Group` component with style props |
|
||||
| `styled.span` / `styled.p` with color/weight/size | **`Text component="span"` with style props** (`c`, `fw`, `fz`). NO CSS module. |
|
||||
| `styled.div` with hover/focus/pseudo-selectors | **Hybrid**: Mantine component with style props for expressible properties + CSS module class for pseudo-selectors only |
|
||||
| `styled.div` with media queries (simple spacing/sizing) | **Responsive style props**: `p={{ base: "md", lg: "xl" }}`. NO CSS module. |
|
||||
| `styled.div` with media queries (complex/non-spacing) | CSS module for the media query parts, style props for the rest |
|
||||
| `styled.div` with animations/keyframes | CSS module for animation, style props for layout |
|
||||
| `styled(SomeComponent)` with only color/spacing/flex | **Wrap in Mantine component** with style props: `<Box c="core-brand" flex="0 0 auto"><Icon /></Box>`, or pass style props if the component accepts them |
|
||||
| `styled(SomeComponent)` with pseudo-selectors/complex styles | CSS module `className` on the component |
|
||||
| Dynamic props `styled.div<{ isActive: boolean }>` | Mantine style props for simple toggles (`c={active ? "core-brand" : "text-primary"}`), `cx()` with CSS module classes for complex state combinations involving pseudo-selectors |
|
||||
| Emotion Pattern | Migration Target |
|
||||
| ------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `styled.div` with only layout/spacing/color | `Box`, `Flex`, `Stack`, or `Group` with style props. **NO CSS module needed.** |
|
||||
| `styled.div` with flexbox column | `Stack` component with style props |
|
||||
| `styled.div` with flexbox row | `Flex` or `Group` component with style props |
|
||||
| `styled.span` / `styled.p` with color/weight/size | **`Text component="span"` with style props** (`c`, `fw`, `fz`). NO CSS module. |
|
||||
| `styled.div` with hover/focus/pseudo-selectors | **Hybrid**: Mantine component with style props for expressible properties + CSS module class for pseudo-selectors only |
|
||||
| `styled.div` with media queries (simple spacing/sizing) | **Responsive style props**: `p={{ base: "md", lg: "xl" }}`. NO CSS module. |
|
||||
| `styled.div` with media queries (complex/non-spacing) | CSS module for the media query parts, style props for the rest |
|
||||
| `styled.div` with animations/keyframes | CSS module for animation, style props for layout |
|
||||
| `styled(SomeComponent)` with only color/spacing/flex | **Wrap in Mantine component** with style props: `<Box c="brand" flex="0 0 auto"><Icon /></Box>`, or pass style props if the component accepts them |
|
||||
| `styled(SomeComponent)` with pseudo-selectors/complex styles | CSS module `className` on the component |
|
||||
| Dynamic props `styled.div<{ isActive: boolean }>` | Mantine style props for simple toggles (`c={active ? "brand" : "text-primary"}`), `cx()` with CSS module classes for complex state combinations involving pseudo-selectors |
|
||||
|
||||
### Step 3: Create CSS Module (if needed)
|
||||
|
||||
@@ -170,19 +170,19 @@ Create `ComponentName.module.css` alongside the component file:
|
||||
```css
|
||||
/* Use design system tokens — NEVER raw color/spacing values */
|
||||
.root {
|
||||
border: 1px solid var(--mb-color-border-neutral);
|
||||
border: 1px solid var(--mb-color-border);
|
||||
border-radius: var(--mantine-radius-md);
|
||||
background-color: var(--mb-color-background_page-primary);
|
||||
background-color: var(--mb-color-background-primary);
|
||||
}
|
||||
|
||||
/* Conditional states as separate classes, combined with cx() */
|
||||
.active {
|
||||
background-color: var(--mb-color-core-brand);
|
||||
background-color: var(--mb-color-brand);
|
||||
color: var(--mb-color-text-primary-inverse);
|
||||
}
|
||||
|
||||
.disabled {
|
||||
color: var(--mb-color-text-disabled);
|
||||
color: var(--mb-color-text-tertiary);
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
@@ -191,8 +191,8 @@ Create `ComponentName.module.css` alongside the component file:
|
||||
cursor: pointer;
|
||||
|
||||
&:hover {
|
||||
color: var(--mb-color-core-brand);
|
||||
background-color: var(--mb-color-background_surface-hover);
|
||||
color: var(--mb-color-brand);
|
||||
background-color: var(--mb-color-background-hover);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -255,9 +255,9 @@ Remove the old styled file entirely. Remove all imports of it from other files.
|
||||
|
||||
**Colors** (`c`, `bg`):
|
||||
|
||||
- Text: `"text-primary"`, `"text-secondary"`, `"text-disabled"`
|
||||
- Background: `"background_page-primary"`, `"background_page-secondary"`, `"background_surface-hover"`
|
||||
- Brand: `"core-brand"`, `"feedback-negative"`, `"feedback-positive"`, `"feedback-warning"`
|
||||
- Text: `"text-primary"`, `"text-secondary"`, `"text-tertiary"`
|
||||
- Background: `"background-primary"`, `"background-secondary"`, `"background-hover"`
|
||||
- Brand: `"brand"`, `"error"`, `"success"`, `"warning"`
|
||||
|
||||
**Typography** (`fz`, `fw`, `lh`, `ta`, `ff`):
|
||||
|
||||
@@ -279,11 +279,11 @@ Remove the old styled file entirely. Remove all imports of it from other files.
|
||||
|
||||
**Colors** — `var(--mb-color-<name>)`:
|
||||
|
||||
- `--mb-color-text-primary`, `--mb-color-text-secondary`, `--mb-color-text-disabled`
|
||||
- `--mb-color-background_page-primary`, `--mb-color-background_page-secondary`, `--mb-color-background_surface-hover`
|
||||
- `--mb-color-text-primary`, `--mb-color-text-secondary`, `--mb-color-text-tertiary`
|
||||
- `--mb-color-background-primary`, `--mb-color-background-secondary`, `--mb-color-background-hover`
|
||||
- `--mb-color-border`, `--mb-color-border-strong`, `--mb-color-border-subtle`
|
||||
- `--mb-color-core-brand`, `--mb-color-core-brand-hover`
|
||||
- `--mb-color-feedback-negative`, `--mb-color-feedback-positive`, `--mb-color-feedback-warning`
|
||||
- `--mb-color-brand`, `--mb-color-brand-hover`
|
||||
- `--mb-color-error`, `--mb-color-success`, `--mb-color-warning`
|
||||
- `--mb-color-shadow`, `--mb-color-focus`
|
||||
- `--mb-color-accent0` through `--mb-color-accent7`
|
||||
|
||||
@@ -398,13 +398,13 @@ export const Wrapper = styled.div`
|
||||
```tsx
|
||||
export const Item = styled.div<{ isSelected: boolean; disabled: boolean }>`
|
||||
color: ${(props) =>
|
||||
props.disabled ? color("text-disabled") : color("text-primary")};
|
||||
props.disabled ? color("text-tertiary") : color("text-primary")};
|
||||
background-color: ${(props) =>
|
||||
props.isSelected ? color("core-brand") : "transparent"};
|
||||
props.isSelected ? color("brand") : "transparent"};
|
||||
cursor: ${(props) => (props.disabled ? "default" : "pointer")};
|
||||
|
||||
&:hover {
|
||||
color: ${(props) => !props.disabled && color("core-brand")};
|
||||
color: ${(props) => !props.disabled && color("brand")};
|
||||
}
|
||||
`;
|
||||
```
|
||||
@@ -418,20 +418,20 @@ export const Item = styled.div<{ isSelected: boolean; disabled: boolean }>`
|
||||
cursor: pointer;
|
||||
|
||||
&:hover {
|
||||
color: var(--mb-color-core-brand);
|
||||
color: var(--mb-color-brand);
|
||||
}
|
||||
}
|
||||
|
||||
.selected {
|
||||
background-color: var(--mb-color-core-brand);
|
||||
background-color: var(--mb-color-brand);
|
||||
}
|
||||
|
||||
.disabled {
|
||||
color: var(--mb-color-text-disabled);
|
||||
color: var(--mb-color-text-tertiary);
|
||||
cursor: default;
|
||||
|
||||
&:hover {
|
||||
color: var(--mb-color-text-disabled);
|
||||
color: var(--mb-color-text-tertiary);
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -457,7 +457,7 @@ When a styled component wraps another component, first check if the styles can b
|
||||
export const CardIcon = styled(Icon)`
|
||||
display: block;
|
||||
flex: 0 0 auto;
|
||||
color: var(--mb-color-core-brand);
|
||||
color: var(--mb-color-brand);
|
||||
`;
|
||||
|
||||
export const CardTitle = styled(Ellipsified)`
|
||||
@@ -471,7 +471,7 @@ export const CardTitle = styled(Ellipsified)`
|
||||
**After — style props on Mantine wrapper (preferred when ALL properties are expressible):**
|
||||
|
||||
```tsx
|
||||
<Box display="block" flex="0 0 auto" c="core-brand">
|
||||
<Box display="block" flex="0 0 auto" c="brand">
|
||||
<Icon {...icon} />
|
||||
</Box>
|
||||
<Box c="text-primary" fz="md" fw="bold" ml="md">
|
||||
@@ -483,13 +483,13 @@ export const CardTitle = styled(Ellipsified)`
|
||||
|
||||
```css
|
||||
.icon {
|
||||
color: var(--mb-color-core-brand);
|
||||
color: var(--mb-color-brand);
|
||||
width: 1.5rem;
|
||||
height: 1.5rem;
|
||||
cursor: pointer;
|
||||
|
||||
&:hover {
|
||||
color: var(--mb-color-core-brand-hover);
|
||||
color: var(--mb-color-brand-hover);
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -570,12 +570,12 @@ export const Spinner = styled.div`
|
||||
|
||||
### Pattern 6: color() Function → CSS Variables
|
||||
|
||||
| Emotion (JS) | CSS Module | Mantine Style Prop |
|
||||
| -------------------------- | ----------------------------------------------------------------- | ------------------------------------- |
|
||||
| `color("core-brand")` | `var(--mb-color-core-brand)` | `c="core-brand"` or `bg="core-brand"` |
|
||||
| `color("text-primary")` | `var(--mb-color-text-primary)` | `c="text-primary"` |
|
||||
| `color("border-neutral")` | `var(--mb-color-border-neutral)` | N/A (use CSS module) |
|
||||
| `alpha("core-brand", 0.2)` | `color-mix(in srgb, var(--mb-color-core-brand), transparent 80%)` | N/A (use CSS module) |
|
||||
| Emotion (JS) | CSS Module | Mantine Style Prop |
|
||||
| ----------------------- | ------------------------------------------------------------ | --------------------------- |
|
||||
| `color("brand")` | `var(--mb-color-brand)` | `c="brand"` or `bg="brand"` |
|
||||
| `color("text-primary")` | `var(--mb-color-text-primary)` | `c="text-primary"` |
|
||||
| `color("border")` | `var(--mb-color-border)` | N/A (use CSS module) |
|
||||
| `alpha("brand", 0.2)` | `color-mix(in srgb, var(--mb-color-brand), transparent 80%)` | N/A (use CSS module) |
|
||||
|
||||
### Pattern 7: Responsive Spacing/Sizing → Responsive Style Props (NOT CSS Media Queries)
|
||||
|
||||
@@ -627,13 +627,13 @@ export const Bar = styled.div<{ width: number }>`
|
||||
|
||||
If existing code already uses `CS` classes and you're not migrating that specific code, leave them alone. But when migrating Emotion → Mantine, replace with the proper alternative:
|
||||
|
||||
| Instead of `CS.*` | Use |
|
||||
| ------------------- | ------------------------------------------------------------ |
|
||||
| `CS.overflowHidden` | CSS module: `overflow: hidden` |
|
||||
| `CS.cursorPointer` | CSS module: `cursor: pointer` |
|
||||
| `CS.flex1` | Style prop: `flex={1}` |
|
||||
| `CS.flexNoShrink` | Style prop: `flex="0 0 auto"` |
|
||||
| `CS.textBrandHover` | CSS module: `&:hover { color: var(--mb-color-core-brand); }` |
|
||||
| Instead of `CS.*` | Use |
|
||||
| ------------------- | ------------------------------------------------------- |
|
||||
| `CS.overflowHidden` | CSS module: `overflow: hidden` |
|
||||
| `CS.cursorPointer` | CSS module: `cursor: pointer` |
|
||||
| `CS.flex1` | Style prop: `flex={1}` |
|
||||
| `CS.flexNoShrink` | Style prop: `flex="0 0 auto"` |
|
||||
| `CS.textBrandHover` | CSS module: `&:hover { color: var(--mb-color-brand); }` |
|
||||
|
||||
### Pattern 10: Shared Emotion Styles Across Files → Shared CSS Module + cx()
|
||||
|
||||
@@ -759,7 +759,7 @@ import S from "./ComponentName.module.css"; // only if CSS module is needed
|
||||
- [ ] The `.styled.tsx` file is deleted
|
||||
- [ ] All imports of the deleted styled file are updated
|
||||
- [ ] No static inline styles — `style={{ }}` used only for truly dynamic runtime values
|
||||
- [ ] All colors use design tokens (`c="core-brand"`, `var(--mb-color-core-brand)`), not raw hex/rgb
|
||||
- [ ] All colors use design tokens (`c="brand"`, `var(--mb-color-brand)`), not raw hex/rgb
|
||||
- [ ] All spacing uses design tokens (`p="md"`, `var(--mantine-spacing-md)`), not arbitrary px values — hardcoded `rem`/`px` literals snapped to the nearest token (see mapping table above)
|
||||
- [ ] All border-radius values use radius tokens (`var(--mantine-radius-md)`), not hardcoded `0.5rem`
|
||||
- [ ] All font sizes use font-size tokens (`fz="md"`, `var(--mantine-font-size-md)`), not hardcoded `1rem`
|
||||
|
||||
@@ -1,244 +0,0 @@
|
||||
---
|
||||
name: metabase-ui-component-from-figma
|
||||
description: Update or build a Metabase design-system component (frontend/src/metabase/ui — Chip, Badge, Alert, Switch, etc.) to match a Figma spec described in a Linear issue. Use when a ticket asks to restyle an existing `metabase/ui` component or implement a new one from Figma. Covers: checking out the issue, mapping the component (usage/blast radius for an existing one, or scaffolding a new one), building a Storybook showcase matrix (before styling for existing components, after for new ones), styling from exact Figma tokens while iterating with the user, and committing + migrating call sites LAST.
|
||||
---
|
||||
|
||||
# Metabase UI component from Figma
|
||||
|
||||
End-to-end workflow for bringing a `metabase/ui` (Mantine) component in line with a Figma design, driven by a Linear ticket.
|
||||
|
||||
The golden rule (existing components): **build the Storybook showcase before changing any component styles.** Build it while the old styles are still in place — so the team can see the current component at a glance and get accustomed to it before the migration, then watch it light up as you restyle. You end up with a visual-regression (Loki) candidate either way. A **new** component has nothing to render until it's built, so this rule doesn't apply to create mode — there the order flips: implement and style the component first, then build its showcase to present the result.
|
||||
|
||||
This is a frontend task — also load **typescript-write** (and **typescript-review** before handing off).
|
||||
|
||||
## Two modes
|
||||
|
||||
The ticket is one of two shapes; identify which up front, because Phase 2 differs:
|
||||
|
||||
- **Update an existing component** — the component already exists with call sites, either in `metabase/ui` *or* as a legacy component elsewhere (e.g. `frontend/src/metabase/common/components/`, possibly Emotion styled-components) that this ticket also migrates into `metabase/ui`. The risk is *regression* (call sites, behavior), so you map usage and lock behavior with tests before touching it.
|
||||
- **Create a new component** — nothing equivalent exists anywhere yet. Rarer. The risk is *API design*, so you scaffold the component and its public surface first.
|
||||
|
||||
Phases 1, 2, 5 and 6 are shared. The middle two differ in **order**: update mode builds the showcase (Phase 3) *before* styling (Phase 4); create mode does the reverse — implement and style first, then build the showcase as the last step — because there's nothing to render until the component exists.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Check out the issue
|
||||
|
||||
1. Fetch the Linear issue with the Linear MCP `get_issue`.
|
||||
2. Pull from it:
|
||||
- **Figma link(s)** — extract the `node-id` (`?node-id=250-13588` → `250-13588`) and `fileKey` (the `/design/<fileKey>/…` segment).
|
||||
- **`gitBranchName`** — switch to it if not already there.
|
||||
- **The component's intended API** — variants, sizes, and states the design system supports for this component. This defines the axes of your showcase (Phase 3).
|
||||
3. Decide the mode (see **Two modes** above): search for an existing component — in `metabase/ui` or as a legacy component elsewhere. Found → **update**; nothing equivalent anywhere → **create**. Confirm with the user when unsure.
|
||||
4. **Verify a Figma desktop MCP is connected before doing the work** — this skill depends on it for exact tokens in Phase 4 (and styling can't proceed accurately without it). Do a quick read against the issue's node (e.g. `get_metadata`/`get_variable_defs`); if it errors with "nothing selected" or the server isn't connected, prompt the user to enable it (Switch to Dev Mode → MCP section in the right sidebar) and select the relevant layer before continuing.
|
||||
|
||||
## Phase 2 — Understand the component
|
||||
|
||||
### Mode: update an existing component
|
||||
|
||||
1. Locate the component dir (typically containing `Component.config.ts` or `Component.tsx`, `Component.module.css`, `Component.stories.tsx`, `index.ts`). A ticket may also be **migrating a legacy component** — if it still uses Emotion styled-components, reach for the **emotion-migrate** skill.
|
||||
2. Map every usage across `frontend/src` and `e2e` (JSX `<Component`, sub-components like `Component.Group`, and wrapper components).
|
||||
3. Write findings to **`local/<TICKET>-<component>-usage-findings.md`**, one entry per usage: code location (`file:line`), short context, and **non-standard usage** (excessive customization, inline styling, invalid/legacy props). Invalid props are a signal that call sites will need migrating in Phase 6. This doc **stays** — the user walks it at the end to verify each real usage still looks right.
|
||||
4. State the blast radius plainly: how many call sites, where, and whether they're confined.
|
||||
5. **Lock behavior with tests before changing anything — when warranted.** Only for components that have **custom interactive behavior** (their own event handlers, state, keyboard logic, controlled/uncontrolled wiring). Add unit tests covering that current API/behavior so a regression shows up as a failing test. **Skip tests for thin Mantine wrappers** — don't test built-in Mantine behavior, and a pure visual restyle is covered by the showcase/Loki, not unit tests.
|
||||
|
||||
### Mode: create a new component
|
||||
|
||||
1. Decide whether it wraps a Mantine primitive (most common — extend it via a `.config.ts` using `<Mantine>.extend({...})` and theme registration) or is bespoke. Prefer wrapping Mantine.
|
||||
2. Scaffold the component dir + its public surface: `Component.tsx`/`Component.config.ts`, `Component.module.css`, `Component.stories.tsx`, `index.ts`, and wire the export through the group barrel and the `metabase/ui` barrel.
|
||||
3. There are no call sites to map yet; instead note the intended usage from the ticket so the eventual showcase reflects real props.
|
||||
4. **Order note:** for a new component, implement and style it (Phase 4) first, *then* build the showcase (Phase 3) to present the finished result — there's nothing to render until it exists. The phases below are written update-first; just run 4 before 3 in create mode.
|
||||
|
||||
## Phase 3 — Build the Storybook showcase
|
||||
|
||||
Goal: one story that shows **every state at a glance**, themeable, suitable as a Loki visual-regression test. **Update mode: build this before Phase 4, while the old styles are still in place. Create mode: do this after Phase 4**, once the component exists to render.
|
||||
|
||||
### Reusable showcase helpers
|
||||
|
||||
Use the shared, Storybook-only primitives in **`frontend/src/metabase/ui/stories/showcase/`**: `StoryShowcase` (titled panel), `StorySection`, `StoryRow`, and `StoryJsx` (monospace JSX with light syntax highlighting via `tokenizeJsx`). Extend the set only when a new pattern is genuinely reusable.
|
||||
|
||||
### Derive the matrix from the component's full state space
|
||||
|
||||
Enumerate **all** the states the component can be in — every supported variant, size, and interaction/selection state (from its props/types and the design-system API, not merely the subset the issue happens to mention) — then cross-reference with the Figma spec and build the matrix on that. Don't omit a supported state because the ticket didn't list it; don't invent an axis Figma doesn't show (e.g. a static component has no hover/pressed).
|
||||
|
||||
**Exclude theme from the matrix.** Figma usually models light/dark as a component *property* (because it doesn't use variable modes), but theme is **not** a real component prop in the app — it's global. So never make theme a matrix axis or column; drive light/dark with the global Storybook `theme` toggle instead (see below).
|
||||
|
||||
Typical layout (mirrors the Figma component sheet): a `StoryShowcase` title, then a single **CSS-grid matrix** — state labels down the left column, one column per axis-combination (e.g. `variant × size`), each column headed by the real JSX usage via `StoryJsx`. Use `gridTemplateColumns: \`<labelWidth> repeat(N, max-content)\`` with a `React.Fragment` per row.
|
||||
|
||||
### Theme: use the global toggle, render ONE copy
|
||||
|
||||
Render the showcase once and let Storybook's global `theme` control (defined in `.storybook/preview.tsx`) switch light/dark. Do **not** render light and dark side by side: Metabase bakes per-scheme hex into `--mantine-color-*` at `:root`, and Mantine color tokens follow the global scheme — a nested scheme wrapper won't re-scope them.
|
||||
|
||||
### Forcing hover/pressed for the matrix
|
||||
|
||||
Use **`storybook-addon-pseudo-states`** (registered in `.storybook/main.ts`); don't add `data-force-*` hooks to component CSS. Set `parameters.pseudo` to force states per cell. Two things make or break it:
|
||||
|
||||
1. **Aim the selector at the element the pseudo-class actually sits on**, which for a Mantine component is often an inner slot, not the root. A string/array value applies `.pseudo-<state>` to the matched element, so it must match that slot. Import the CSS-module class (`S.<Slot>`) to target it.
|
||||
2. **Land your per-cell hook on the root via the factory's root-props slot** (e.g. `wrapperProps`), because Mantine forwards top-level `data-*`/rest props to an inner element, not the root. Check the component's `.d.ts` for the right slot.
|
||||
|
||||
The addon toolbar toggle keeps working independently (it forces every rule in the story).
|
||||
|
||||
### Example: a finalized matrix story
|
||||
|
||||
Illustrative — adapt the axes and states to the component you're working on. This is the shape the pieces above add up to (state labels on the left, one column per `variant × size`, forced hover/pressed via `parameters.pseudo`, the root hook on `wrapperProps`, controls scoped to the meaningful knobs):
|
||||
|
||||
```tsx
|
||||
import type { StoryFn } from "@storybook/react";
|
||||
import { Fragment } from "react";
|
||||
|
||||
import { Box, Chip, type ChipProps, Text } from "metabase/ui";
|
||||
import { StoryJsx, StoryShowcase } from "metabase/ui/stories/showcase";
|
||||
|
||||
import S from "./Chip.module.css";
|
||||
|
||||
// Axes derived from the component's API (here: variant × size).
|
||||
const COLUMNS = [
|
||||
{ variant: "light", size: "sm" },
|
||||
{ variant: "light", size: "md" },
|
||||
{ variant: "filled", size: "sm" },
|
||||
{ variant: "filled", size: "md" },
|
||||
] as const;
|
||||
|
||||
// Every state the component supports; `id` doubles as the pseudo-state hook.
|
||||
const STATES = [
|
||||
{ id: "default", label: "Default" },
|
||||
{ id: "hover", label: "Hover" },
|
||||
{ id: "pressed", label: "Pressed" },
|
||||
{ id: "selected", label: "Selected", checked: true },
|
||||
{ id: "hover-selected", label: "Hover selected", checked: true },
|
||||
{ id: "disabled", label: "Disabled", disabled: true },
|
||||
];
|
||||
|
||||
// Force hover/pressed by targeting the slot the pseudo-class sits on (.ChipLabel).
|
||||
const labelSelector = (id: string) => `[data-state-row="${id}"] .${S.ChipLabel}`;
|
||||
|
||||
const Overview: StoryFn<ChipProps> = ({ children }) => (
|
||||
<StoryShowcase title="Chip">
|
||||
<Box
|
||||
style={{
|
||||
display: "grid",
|
||||
gridTemplateColumns: `9rem repeat(${COLUMNS.length}, max-content)`,
|
||||
columnGap: "2rem",
|
||||
rowGap: "1rem",
|
||||
alignItems: "center",
|
||||
}}
|
||||
>
|
||||
<div />
|
||||
{COLUMNS.map(({ variant, size }) => (
|
||||
<StoryJsx
|
||||
key={`${variant}-${size}`}
|
||||
>{`<Chip variant="${variant}" size="${size}" />`}</StoryJsx>
|
||||
))}
|
||||
{STATES.map((state) => (
|
||||
<Fragment key={state.id}>
|
||||
<Text size="sm" c="text-secondary">
|
||||
{state.label}
|
||||
</Text>
|
||||
{COLUMNS.map(({ variant, size }) => (
|
||||
<Chip
|
||||
key={`${variant}-${size}`}
|
||||
wrapperProps={{ "data-state-row": state.id }} // hook on the ROOT
|
||||
variant={variant}
|
||||
size={size}
|
||||
checked={state.checked ?? false}
|
||||
disabled={state.disabled ?? false}
|
||||
onChange={() => {}}
|
||||
>
|
||||
{children}
|
||||
</Chip>
|
||||
))}
|
||||
</Fragment>
|
||||
))}
|
||||
</Box>
|
||||
</StoryShowcase>
|
||||
);
|
||||
|
||||
export const OverviewStory = {
|
||||
render: Overview,
|
||||
parameters: {
|
||||
pseudo: {
|
||||
hover: ["hover", "hover-selected"].map(labelSelector),
|
||||
active: ["pressed"].map(labelSelector),
|
||||
},
|
||||
controls: { include: ["children", "theme"] },
|
||||
},
|
||||
};
|
||||
```
|
||||
|
||||
### Story conventions
|
||||
|
||||
- Keep an interactive default story plus the matrix story.
|
||||
- Restrict `argTypes` to the supported API (e.g. only the sizes the DS uses).
|
||||
- **Scope controls per story.** A static matrix should hide knobs it ignores: `parameters.controls = { include: [...] }` (or `{ disable: true }`). Wire any kept knob (e.g. `children`) through to the cells so it does something.
|
||||
- `loki.config.js` has a `storiesFilter`; check it before renaming a story that's already covered.
|
||||
|
||||
Validate: `bun run lint-eslint-pure -- <files>` and `bun run type-check-pure`.
|
||||
|
||||
### Optional: verify in the browser yourself
|
||||
|
||||
If a browser-driving MCP is connected (e.g. **chrome-devtools-mcp** or **playwright-mcp**: `navigate_page`, `take_screenshot`, `take_snapshot`, `evaluate_script`, `resize_page`), use it to check your own work against the running Storybook (`http://localhost:6006/iframe.html?id=<story-id>`, the kebab-cased title + story). Screenshot for a visual read; `evaluate_script`/`take_snapshot` to confirm DOM facts (a hook landed on the root, pseudo classes applied, a resolved `--mb-color-*`). This is a **convenience, never a requirement** — without it, rely on the user's visual check (and you may suggest installing the plugin). Storybook is usually already running; don't start it. (Screenshot writes may be sandboxed to the repo root — save under `local/` then `mv`.)
|
||||
|
||||
Pause here and let the user look. Iterate on layout/spacing only — **no component style changes yet.**
|
||||
|
||||
## Phase 4 — Style the component from Figma
|
||||
|
||||
Now change the component's `.config.ts` / `.module.css`. Get **exact tokens from Figma** — never eyeball hex from a screenshot.
|
||||
|
||||
### Figma desktop MCP is required for tokens
|
||||
|
||||
`get_screenshot` works by `nodeId`, but `get_variable_defs`, `get_design_context`, and `get_metadata` read the **current selection in the Figma desktop app** via the **Dev Mode MCP server**. If they error with "nothing selected" or the server isn't connected, **prompt the user** to enable the Dev Mode MCP server (Switch to Dev Mode → MCP section in right sidebar) and select the relevant layer(s). Don't pixel-sample a screenshot as a substitute.
|
||||
|
||||
### Extract and map tokens
|
||||
|
||||
1. `get_metadata` on the spec frame for the node id + name of each variant/size/state symbol.
|
||||
2. `get_variable_defs` per symbol for exact colors, spacing, radius, font.
|
||||
3. `get_design_context` (forceCode) on one symbol for exact CSS dimensions (padding, gap, icon size). It may ask about Code Connect — skip that.
|
||||
4. **Use the semantic token names `get_variable_defs` returns.** An up-to-date spec binds each property to a semantic token (a role like `text/hover`, `background/selected`); match it to the corresponding `--mb-color-*` key (key list: `frontend/src/metabase/ui/colors/types/color-keys.ts`) and use it directly — one definition then works in both themes. Sanity-check the resolved shade against `colors/constants/themes/light.ts` / `dark.ts` and `base-colors.ts`. Caveat: returned *values* may be the wrong mode for dark symbols — trust the variable **names**, not the dark hex.
|
||||
|
||||
### Color rules — semantic tokens only, NO `color-mix`, NO primitives
|
||||
|
||||
Per `docs/developers-guide/frontend.md` ("Colors"): every color must be a **semantic** `--mb-color-*` token. No literal hex, and no `color-mix` in component CSS to fake a shade/alpha.
|
||||
|
||||
**Guardrail:** if `get_variable_defs` returns a **primitive ramp** (e.g. `Orion-Alpha/10`, `Ocean/60` — a ramp + number, not a role) bound directly to a property, do **not** translate it to a ramp, hardcode hex, or `color-mix` your way to it. **Stop and flag it to the user** with the specific property/state — it's a design-side gap (bind to a semantic token, or add one). Resume once the spec exposes a semantic token.
|
||||
|
||||
### Non-color tokens — flag scale mismatches too
|
||||
|
||||
The same discipline applies to **dimensional** values — radius, spacing/padding/gap, elevation/shadow, and typography (font size, line-height, weight). Prefer the codebase's existing scale variable over a literal: `--mantine-radius-*`, `--mantine-spacing-*`, `--mantine-shadow-*`, and the Mantine font tokens.
|
||||
|
||||
But Figma's scales don't always line up with the codebase's. When a Figma value **has no matching step** in the corresponding scale — e.g. Figma `radius/md = 12px` while `--mantine-radius-md` is `8px`, or a Figma `xxxs` spacing step with no Mantine equivalent — **don't silently bake in a literal.** Flag it to the user the same way you flag primitive colors: name the property, the Figma value, and the nearest codebase token, so it can be resolved with design / the scale owners (align the value, or add a scale step). Only fall back to a literal if the user confirms it's an accepted one-off, and leave a short comment in the code explaining why.
|
||||
|
||||
### Mantine implementation notes (reference)
|
||||
|
||||
General gotchas when extending Mantine components — adapt per component:
|
||||
|
||||
- **Know which slot each data-attribute lands on.** Mantine puts state attributes on specific slots (e.g. `data-variant` on the root, `data-checked`/`data-disabled` on an inner slot). Scope rules accordingly, usually through a root `classNames.root` class.
|
||||
- **Out-specify Mantine's built-in variant CSS.** It styles slots at `.m_*:not([data-disabled])` specificity; a descendant combinator through your root class wins. Avoid chained `:not()` (stylelint's `selector-not-notation: complex` rejects it, and combining lowers specificity) — gain specificity via the root class / attributes instead.
|
||||
- **Size via a `vars` resolver**, returning only Mantine's typed vars for that component (custom var names fail type-check). Map size → vars in `.config.ts`.
|
||||
- **Built-in icons use `currentColor`**, so they inherit the slot's text color.
|
||||
- For text on a brand fill, follow the existing convention (`text-primary-inverse`) unless the user wants pure white; flag the dark-theme implication.
|
||||
|
||||
After edits: `npx stylelint <css>`, `LINT_CSS_MODULES=true bun run lint-eslint-pure -- <files>`, `bun run type-check-pure`.
|
||||
|
||||
## Phase 5 — Iterate with the user
|
||||
|
||||
Show the result in the running Storybook and address feedback (color, spacing, radius, shadow, missing/extra state). Surface deliberate deviations explicitly so the user can veto them. If a browser MCP is connected, screenshot the matrix in both themes to catch issues first. **Do not commit until the user is satisfied.**
|
||||
|
||||
## Phase 6 — Commit, then migrate call sites
|
||||
|
||||
Only after sign-off:
|
||||
|
||||
1. Commit when the user explicitly asks; commit only relevant files (exclude build artifacts and the gitignored `local/` doc). Follow the user's commit conventions.
|
||||
2. **Update mode:** migrate the call sites flagged in Phase 2 if the restyle requires it (e.g. invalid/renamed props). Be surgical with bulk find/replace — a blanket prop rename can hit unrelated components; verify each hit is the target. **Create mode:** confirm the public export is wired through the barrels and used as the ticket intends.
|
||||
3. Leave the findings doc in place for the user's manual verification pass.
|
||||
|
||||
---
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] Linear issue fetched; Figma node/fileKey + `gitBranchName` extracted; on the right branch; mode (update vs create) decided.
|
||||
- [ ] **Update:** usage + blast radius in `local/<TICKET>-<component>-usage-findings.md`; behavior locked with unit tests **if** the component has custom interactive logic (skip for thin Mantine wrappers / pure restyles). **Create:** component scaffolded with its public surface and barrel exports.
|
||||
- [ ] Storybook showcase matrix built from the component's full state space (theme excluded — global toggle drives light/dark), single panel, hover/pressed forced via `storybook-addon-pseudo-states`, controls scoped per story; lint + type-check pass. (Update mode: built before styling, **component styles untouched**. Create mode: built after the component is implemented in Phase 4.)
|
||||
- [ ] Figma desktop MCP available; exact tokens extracted and mapped to semantic `--mb-color-*` (NO `color-mix`, NO primitives) and to existing scale vars for radius/spacing/elevation/type — any color *or* dimensional value with no matching token/scale step flagged to the user, not baked in as a literal.
|
||||
- [ ] Component styled; stylelint + eslint(+CSS modules) + type-check pass.
|
||||
- [ ] Iterated with the user until satisfied.
|
||||
- [ ] Committed (when asked); call sites migrated/verified (update) or export wired (create); findings doc handed back.
|
||||
@@ -1,33 +1,20 @@
|
||||
---
|
||||
name: typescript-review
|
||||
description: Review TypeScript and JavaScript code changes for compliance with Metabase coding standards, style violations, and code quality issues. Use when reviewing pull requests or diffs containing TypeScript/JavaScript code.
|
||||
allowed-tools: Read, Grep, Bash, Glob, Skill
|
||||
allowed-tools: Read, Grep, Bash, Glob
|
||||
---
|
||||
|
||||
# TypeScript/JavaScript Code Review Skill
|
||||
|
||||
@./../_shared/typescript-commands.md
|
||||
@./../_shared/react-redux-patterns.md
|
||||
|
||||
## Main Focus
|
||||
## Code Review Guidelines
|
||||
|
||||
**Primary standard: the [`typescript-write`](../typescript-write/SKILL.md) skill.** Load it first — it defines the authoring rules this review enforces, alongside `frontend/CLAUDE.md` and `docs/developers-guide/frontend.md`.
|
||||
Review pull requests with a focus on:
|
||||
|
||||
Adherence to `typescript-write` is the **highest-priority** review dimension: rank any violation of its provisions above all other findings. Treat its **no-`any` hard rule** (no explicit *or* implicit `any` in new code) as **blocking**, and verify it with the LSP rather than by eye.
|
||||
|
||||
Review in this priority order:
|
||||
|
||||
1. **Violations of [`typescript-write`](../typescript-write/SKILL.md) provisions** — no-`any`, type tightening, type modeling, null/undefined handling, naming, structure, comments. Highest priority; block on the no-`any` rule.
|
||||
2. Compliance with `frontend/CLAUDE.md`.
|
||||
3. Readability and maintainability.
|
||||
4. Appropriate test coverage.
|
||||
|
||||
## Blind spots — act as the missing reviewer
|
||||
|
||||
These rarely surface in team reviews, so this skill should raise them. They are **additive** — raise them, but rank them below `typescript-write` violations:
|
||||
|
||||
- **Accessibility.** Interactive elements need keyboard support, focus management, and accessible names. Flag missing `aria-label`/`aria-labelledby`, non-semantic click targets, modals without focus trap, icon-only buttons without labels, and form inputs without a linked label.
|
||||
- **Performance.** Flag areas that scale poorly and aren't memoized; inline object/array literals passed to memoized children; effects that fire on every batch of a progressive load; and new dependencies added to hot paths.
|
||||
- **Security.** Evaluate potential security issues in new code.
|
||||
- **Bundle size.** Flag new large dependencies, default imports from icon or util libs, and heavy modules imported at route-load time.
|
||||
- **Analytics.** User-facing flows should emit tracking events. If a PR adds a new flow (button, modal, navigation) without a tracking event, ask whether one is expected.
|
||||
- Readability and maintainability
|
||||
- Appropriate test coverage
|
||||
- Compliance with project coding standards and conventions
|
||||
- Code quality and best practices
|
||||
- Type safety and proper TypeScript usage
|
||||
- React best practices (when applicable)
|
||||
|
||||
@@ -7,77 +7,3 @@ description: Write TypeScript and JavaScript code following Metabase coding stan
|
||||
|
||||
@./../_shared/development-workflow.md
|
||||
@./../_shared/typescript-commands.md
|
||||
@./../_shared/react-redux-patterns.md
|
||||
|
||||
## No `any` — hard rule
|
||||
|
||||
- **New code must not introduce `any`, explicit or implicit.** No `any` annotations, no `as any` / `as unknown as`, no untyped parameters or returns that infer `any`, no implicitly-`any` destructures or array/object literals.
|
||||
- **Untyped third-party / boundary values** must be typed at the boundary (a declared type, `unknown` + type guard, or a small typed wrapper) — never let `any` propagate inward.
|
||||
- **Mandatory LSP verification.** After writing or editing any TS/TSX, inspect the changed symbols with the TypeScript Language Server (hover to read inferred types; `goToDefinition` to confirm sources) and run the project type-check.
|
||||
|
||||
## Type tightening
|
||||
|
||||
- **Avoid type casts and loose `unknown`** — fix the signature instead.
|
||||
- **If a function only needs one field of a wide object, accept that field** — not the wide object. The cast often disappears once the signature is right.
|
||||
- **Reach for `Partial<T>`, `Pick<T, K>`, `Record<K, V>`, and generics** before reaching for a cast.
|
||||
- **Prefer making props/components generic** (`<T>`) when a value flows through unchanged and the caller knows the type.
|
||||
- **Prefer `unknown` over loose typing** and narrow before use — an `unknown` value forces a guard at the point of use.
|
||||
- **`satisfies` for object literals** that must conform without widening (config objects, lookup maps, discriminated literals) — better than `: T` (widens) or `as T` (unsafe).
|
||||
- **Avoid non-null assertions (`!`)**. Prefer a guard, early return, or `?.`. Use `!` only when non-nullness is provably true and localized, with a comment.
|
||||
- **No redundant runtime coercion** — don't wrap already-typed values in `Number()` / `String()` / `Boolean()`.
|
||||
- **Type guards belong in `frontend/src/metabase-types/guards/`**. Do not redefine them locally.
|
||||
|
||||
## Type modeling
|
||||
|
||||
- **Reuse existing types; don't re-declare them.** Use canonical IDs and domain entity types from `metabase-types/api` (`FieldId`, `TableId`, `ConcreteTableId`, `SchemaName`, …) and key data structures by them (`new Map<ConcreteTableId, …>()`). Don't duplicate generated/API types — compose or derive (`Pick`, `Omit`, indexed access `SomeType["field"]`, `ReturnType`).
|
||||
- **Use generics to allow TypeScript to infer correct types** when creating functions and components that need to be reusable and type-safe. Don't hesitate to introduce complex generics if they allow to derive types automatically instead of manual narrowing.
|
||||
- **Model the actual data contract; keep types narrow.** Optional `field?: T` for a key that may be absent, `field: T | undefined` only when the key is always present but the value may be undefined, `| null` for explicit API nulls. Prefer domain unions over broad `string` / `number` / loose `Record`.
|
||||
- **Refer to API implementation** when defining or refining types to ensure they match the actual data structure. When considering a type cast, first consider if the type should be refined to match the actual data structure.
|
||||
- **Discriminated unions for variant state, with exhaustive checks.** Model "one of N shapes" as a union with a literal discriminant rather than a bag of optional fields, and exhaust it with ts-pattern's `.exhaustive()` so adding a variant becomes a compile error:
|
||||
```ts
|
||||
import { match } from "ts-pattern";
|
||||
|
||||
const result = match(status)
|
||||
.with({ type: "loading" }, () => <Spinner />)
|
||||
.with({ type: "error", error: P.select() }, (error) => <Error message={error.message} />)
|
||||
.with({ type: "success", data: P.select() }, (data) => <Content data={data} />)
|
||||
.exhaustive(); // Compile-time guarantee all cases handled
|
||||
```
|
||||
- **Derive union types from constants** (`as const` + `typeof`/`keyof`) so the type and the values can't drift.
|
||||
- **`readonly` / immutability where mutation isn't intended** — component props, shared constants, exported config. Prefer `readonly T[]` / `ReadonlyArray<T>` for inputs you don't mutate.
|
||||
- **Type async and error states explicitly** (a discriminated union or the data-layer's typed result) — never leave loading/error/empty implicit.
|
||||
|
||||
## Null and undefined
|
||||
|
||||
- **Narrow at the source**. If a value is optional only in a corner case, don't thread `undefined` through every layer — guard at the producer.
|
||||
- **Sensible defaults for optional values**. Use `?.` and `??` at the consumer.
|
||||
- **Lists should be filtered** before being used in a map or other iteration.
|
||||
- **Avoid non-strict null comparisons** (`X != null`) when `X` can never be `null` — use a strict check or narrow the type. Use `checkNotNull` where necessary.
|
||||
- **Check actual nullability against API implementation**. Find the API endpoint implementation and check if the field can actually be null.
|
||||
|
||||
## Naming
|
||||
|
||||
- **Names describe the entity, not the mechanism**. A name must reflect what the value holds.
|
||||
- **Align sibling concepts**: keep verb conventions consistent across a related API.
|
||||
- **No names that encode implementation history** rather than current meaning. Suffixes like `Base`, `New`, `Old`, `Initial` need a real semantic distinction, otherwise drop them.
|
||||
- **Avoid cryptic identifiers** (`v`, `n`, `$n`) for domain values; short names are fine only in tiny conventional contexts (loop index `i`, coordinates `x`/`y`, generic params `T`/`K`/`V`).
|
||||
|
||||
## Code structure and organization
|
||||
|
||||
- **Prioritize reusability over duplication**. The codebase already has many utility functions — leverage them. If you introduce duplicated logic, extract it to a shared utility.
|
||||
- **Generic helpers do not belong in feature folders**. Promote to a shared utility.
|
||||
- **Keep functions small and single-purpose**. A 100+ line function is hard to review — split into focused named helpers, each with one responsibility and a minimal dependency surface. When necessary, cover with unit tests.
|
||||
- **Extract distinct complex JSX into named components**. Choose same-file vs separate-file by reuse, coupling, testability, and readability.
|
||||
|
||||
## Comments
|
||||
|
||||
- **No comments by default**. Well-named identifiers carry the `what`.
|
||||
- **Comments should be concise**. Add a short, concise comment only when the `why` is non-obvious: a workaround, a hidden invariant, a subtle ordering constraint, a clever reduction. Never document the actual implementation, focus on the intent and the why.
|
||||
|
||||
## TypeScript Migration
|
||||
|
||||
**When touching existing JavaScript files, propose to convert them to TypeScript first**. Create a separate PR for the conversion, then implement the changes.
|
||||
|
||||
## Verify before done
|
||||
|
||||
- **Run the project type-check** when finished (see the shared TypeScript commands above).
|
||||
|
||||
@@ -61,7 +61,6 @@
|
||||
:metabase/check-def-check-not-uppercase-name {:level :warning}
|
||||
:metabase/check-def-no-underscores {:level :warning}
|
||||
:metabase/check-me-humanize {:level :warning}
|
||||
:metabase/check-testing-not-empty {:level :warning}
|
||||
:metabase/defsetting-must-specify-export {:level :warning}
|
||||
:metabase/defsetting-namespace {:level :warning}
|
||||
:metabase/discourage-millis-duration {:level :warning}
|
||||
@@ -179,26 +178,14 @@
|
||||
clojure.core.async/take!
|
||||
clojure.core.async/to-chan!
|
||||
clojure.core.async/to-chan!!
|
||||
metabase.ai-tracing.core/record!
|
||||
metabase.api.macros.defendpoint.tools-manifest/assert-optional-fields-nullable!
|
||||
metabase.channel.core/send!
|
||||
metabase.channel.render.pdf/draw-too-large!
|
||||
metabase.channel.render.pdf/draw-no-results!
|
||||
metabase.channel.render.pdf/render-card-cell!
|
||||
metabase.channel.render.pdf.font/load-fonts!
|
||||
metabase.warehouses.models.database/assert-not-h2!
|
||||
metabase.driver.sql-jdbc.execute/execute-prepared-statement!
|
||||
metabase.mcp.usage/record-mcp-session!
|
||||
metabase.mcp.usage/record-mcp-session-end!
|
||||
metabase.mcp.usage/record-mcp-tool-call!
|
||||
metabase.notification.models/create-notification!
|
||||
metabase.oauth-server.core/reset-provider!
|
||||
metabase.pulse.send/send-pulse!
|
||||
metabase-enterprise.transforms-python.python-runner-test/execute!
|
||||
;; MCP usage tests isolate their rows by unique keys (session id / tool name / duration /
|
||||
;; client version) and clean up only those, so these deletes are safe in parallel.
|
||||
metabase-enterprise.mcp.usage-test/cleanup!
|
||||
metabase-enterprise.mcp.usage-test/cleanup-calls!
|
||||
;; Workspace isolation functions are safe in parallel tests because they operate on
|
||||
;; isolated, test-specific workspaces and databases that don't affect other tests
|
||||
metabase-enterprise.workspaces.isolation/destroy-workspace-isolation!
|
||||
@@ -260,6 +247,9 @@
|
||||
;; misc config for built-in linters
|
||||
;;
|
||||
|
||||
:invalid-arity {:skip-args [metabase.util.match/match-one
|
||||
metabase.util.match/match-many
|
||||
metabase.util.match/replace]} ; TODO (cam): can we fix these?
|
||||
:refer-all {:level :warning, :exclude [clojure.test]}
|
||||
:unused-referred-var {:exclude {compojure.core [GET DELETE POST PUT]}}
|
||||
|
||||
@@ -279,6 +269,7 @@
|
||||
(metabase.util.match/match-many)
|
||||
(metabase.util.match/replace-in)
|
||||
(metabase.util.match/replace)
|
||||
(metabase.driver-api.core/match)
|
||||
(metabase.driver-api.core/match-one)
|
||||
(metabase.driver-api.core/match-many)
|
||||
(metabase.driver-api.core/replace)
|
||||
@@ -453,10 +444,6 @@
|
||||
instaparse.core/defparser clojure.core/def
|
||||
metabase-enterprise.action-v2.test-util/with-test-tables! clojure.core/let
|
||||
metabase-enterprise.content-translation.routes-test/with-new-secret-key! clojure.core/fn
|
||||
metabase-enterprise.entity-retrieval.reconcile-test/with-isolated-index clojure.core/fn
|
||||
metabase-enterprise.mfa.enrollment-flow-test/with-fresh-user-session! clojure.core/fn
|
||||
metabase-enterprise.mfa.login-flow-test/with-enrolled-rasta! clojure.core/fn
|
||||
metabase-enterprise.mfa.recovery-codes-test/with-confirmed-enrollment! clojure.core/fn
|
||||
metabase-enterprise.serialization.metadata-file-import.wire-format-test/with-tiny-fixture! clojure.core/fn
|
||||
metabase-enterprise.serialization.test-util/with-dbs clojure.core/fn
|
||||
metabase-enterprise.serialization.test-util/with-random-dump-dir clojure.core/let
|
||||
@@ -473,7 +460,6 @@
|
||||
metabase.channel.impl.http-test/with-captured-http-requests clojure.core/fn
|
||||
metabase.channel.impl.http-test/with-server clojure.core/let
|
||||
metabase.collections.models.collection-test/with-collection-in-location clojure.core/let
|
||||
metabase.collections.test-utils/with-library clojure.core/fn
|
||||
metabase.driver.mongo.connection/with-mongo-client clojure.core/let
|
||||
metabase.driver.mongo.connection/with-mongo-database clojure.core/let
|
||||
metabase.driver.mongo.query-processor/mongo-let clojure.core/let
|
||||
@@ -487,7 +473,6 @@
|
||||
metabase.notification.test-util/with-card-notification clojure.core/let
|
||||
metabase.notification.test-util/with-system-event-notification! clojure.core/let
|
||||
metabase.notification.test-util/with-temp-notification clojure.core/let
|
||||
metabase.osi.ai-context.api-test/with-test-entry clojure.core/let
|
||||
metabase.premium-features.core/defenterprise-schema clj-kondo.lint-as/def-catch-all
|
||||
metabase.premium-features.defenterprise/defenterprise-schema clj-kondo.lint-as/def-catch-all
|
||||
metabase.premium-features.settings/define-premium-feature clojure.core/def
|
||||
@@ -554,7 +539,6 @@
|
||||
{:analyze-call
|
||||
{cljs.test/deftest hooks.clojure.test/deftest
|
||||
cljs.test/is hooks.clojure.test/is
|
||||
cljs.test/testing hooks.clojure.test/testing
|
||||
cljs.test/use-fixtures hooks.clojure.test/use-fixtures
|
||||
clojure.core/def hooks.clojure.core.def/lint-def
|
||||
clojure.core/definterface hooks.clojure.core.def/lint-def
|
||||
@@ -573,7 +557,6 @@
|
||||
clojure.core/- hooks.metabase.util.timing/discourage-millis-duration
|
||||
clojure.test/deftest hooks.clojure.test/deftest
|
||||
clojure.test/is hooks.clojure.test/is
|
||||
clojure.test/testing hooks.clojure.test/testing
|
||||
clojure.test/use-fixtures hooks.clojure.test/use-fixtures
|
||||
malli.error/humanize hooks.malli.error/humanize
|
||||
metabase-enterprise.advanced-permissions.models.permissions.application-permissions-test/with-new-group-and-current-graph hooks.common/with-two-top-level-bindings
|
||||
@@ -589,7 +572,7 @@
|
||||
metabase.api.common.internal-test/with-jetty-server hooks.common/with-one-binding
|
||||
metabase.api.macros/defendpoint hooks.metabase.api.macros/defendpoint
|
||||
metabase.app-db.schema-migrations-test.impl/test-migrations hooks.metabase.app-db.schema-migrations-test.impl/test-migrations
|
||||
metabase.channel.render.js.pool/with-static-viz-context hooks.common/with-one-top-level-binding
|
||||
metabase.channel.render.js.svg/with-static-viz-context hooks.common/with-one-top-level-binding
|
||||
metabase.channel.template.handlebars-helper/defhelper hooks.metabase.channel.template.handlebars-helper/defhelper
|
||||
metabase.channel.template.handlebars-test/with-temp-template! hooks.common/with-vec-first-binding
|
||||
metabase.classloader.core/require hooks.clojure.core.require/lint-require
|
||||
@@ -625,9 +608,6 @@
|
||||
metabase.model-persistence.api-test/with-persistence-setup! hooks.common/with-one-top-level-binding
|
||||
metabase.model-persistence.api-test/with-setup! hooks.common/with-one-top-level-binding
|
||||
metabase.model-persistence.test-util/with-persistence-enabled! hooks.metabase.model-persistence.test-util/with-persistence-enabled!
|
||||
metabase.mq.core/with-queue hooks.common/let-second
|
||||
metabase.mq.test-util/with-test-mq hooks.common/with-one-binding
|
||||
metabase.mq.core/def-listener! hooks.metabase.mq.impl/def-listener
|
||||
metabase.models.interface/define-batched-hydration-method hooks.metabase.models.interface/define-hydration-method
|
||||
metabase.models.interface/define-simple-hydration-method hooks.metabase.models.interface/define-hydration-method
|
||||
metabase.permissions.core/allow-changing-all-users-group-members hooks.common/do*
|
||||
@@ -719,10 +699,6 @@
|
||||
metabase.util.log/warn hooks.metabase.util.log/info
|
||||
metabase.util.log/warnf hooks.metabase.util.log/infof
|
||||
metabase.util.malli.registry/def hooks.metabase.util.malli.registry/def
|
||||
metabase.util.match/match-one hooks.metabase.util.match/match-one
|
||||
metabase.util.match/match-many hooks.metabase.util.match/match-many
|
||||
metabase.util.match/replace hooks.metabase.util.match/replace
|
||||
metabase.util.match/replace-in hooks.metabase.util.match/replace-in
|
||||
metabase.util/case-enum hooks.metabase.util/case-enum
|
||||
metabase.util/format-color hooks.metabase.util/format-color
|
||||
metabase.xrays.api.automagic-dashboards-test/with-indexed-model! hooks.metabase.xrays.api.automagic-dashboards-test/with-indexed-model!
|
||||
@@ -766,7 +742,6 @@
|
||||
metabase.test/with-temporary-raw-setting-values macros.metabase.test.util/with-temporary-raw-setting-values
|
||||
metabase.users.models.user-test/with-groups! macros.metabase.users.models.user-test/with-groups!
|
||||
metabase.util.namespaces/import-fn macros.metabase.util.namespaces/import-fn
|
||||
metabase.util.namespaces/import-macro macros.metabase.util.namespaces/import-macro
|
||||
metabase.xrays.related-test/with-world macros.metabase.xrays.related-test/with-world}}
|
||||
|
||||
:config-in-comment
|
||||
@@ -824,8 +799,16 @@
|
||||
;;
|
||||
{:pattern "^metabase(?:((?:-enterprise\\.sandbox)?\\.query-processor)|(?:\\.driver\\.sql(?!server)(?!ite))).*-test$"
|
||||
:name general-qp-and-driver-test-namespaces}
|
||||
|
||||
{:pattern "(?:^mage.*)"
|
||||
:name mage-namespaces}]
|
||||
:name mage-namespaces}
|
||||
;;
|
||||
;; Namespaces where `with-redefs` is still the idiom because they haven't been audited for conversion
|
||||
;; to `with-dynamic-fn-redefs` yet (enterprise tests, driver tests). The linter's nudge is too noisy
|
||||
;; here — audit can happen in follow-up PRs.
|
||||
;;
|
||||
{:pattern "^(?:metabase-enterprise\\..+-test(?:\\..+)?|metabase\\.driver\\..+-test(?:\\..+)?)$"
|
||||
:name prefer-with-dynamic-fn-redefs-unaudited}]
|
||||
|
||||
:config-in-ns
|
||||
{test-namespaces
|
||||
@@ -849,6 +832,10 @@
|
||||
metabase.util.time-test
|
||||
{:linters {:metabase/prefer-with-dynamic-fn-redefs {:level :off}}}
|
||||
|
||||
;; Enterprise tests and driver tests haven't been audited yet — silence the linter so CI can ship.
|
||||
prefer-with-dynamic-fn-redefs-unaudited
|
||||
{:linters {:metabase/prefer-with-dynamic-fn-redefs {:level :off}}}
|
||||
|
||||
source-namespaces
|
||||
{:linters
|
||||
{:metabase/discourage-millis-duration {:level :warning}
|
||||
@@ -943,7 +930,10 @@
|
||||
|
||||
:deprecated-namespace
|
||||
{:exclude
|
||||
[metabase.legacy-mbql.normalize
|
||||
[metabase.driver.common.parameters.dates
|
||||
metabase.driver.common.parameters.operators
|
||||
metabase.driver.common.parameters.values
|
||||
metabase.legacy-mbql.normalize
|
||||
metabase.legacy-mbql.schema
|
||||
metabase.legacy-mbql.schema.helpers
|
||||
metabase.legacy-mbql.schema.macros
|
||||
|
||||
@@ -128,7 +128,6 @@
|
||||
analytics
|
||||
api
|
||||
collections
|
||||
eid-translation
|
||||
lib
|
||||
permissions
|
||||
public-sharing
|
||||
@@ -159,10 +158,8 @@
|
||||
{:team "Metabot"
|
||||
:api #{metabase.agent-api.api
|
||||
metabase.agent-api.init}
|
||||
:uses #{ai-tracing
|
||||
api
|
||||
:uses #{api
|
||||
auth-identity
|
||||
channel
|
||||
collections
|
||||
dashboards
|
||||
events
|
||||
@@ -178,11 +175,8 @@
|
||||
settings
|
||||
util}
|
||||
:model-imports #{:model/Card
|
||||
:model/Collection
|
||||
:model/Dashboard
|
||||
:model/DashboardCard
|
||||
:model/DashboardTab
|
||||
:model/Database
|
||||
:model/Table
|
||||
:model/User}}
|
||||
|
||||
@@ -195,19 +189,9 @@
|
||||
models
|
||||
util}}
|
||||
|
||||
ai-tracing
|
||||
{:team "Metabot"
|
||||
:api #{metabase.ai-tracing.api
|
||||
metabase.ai-tracing.core
|
||||
metabase.ai-tracing.init}
|
||||
:uses #{api
|
||||
settings
|
||||
util}}
|
||||
|
||||
analytics
|
||||
{:team "UX West"
|
||||
:api #{metabase.analytics.api
|
||||
metabase.analytics.api.proxy
|
||||
metabase.analytics.core
|
||||
metabase.analytics.event
|
||||
metabase.analytics.init
|
||||
@@ -264,6 +248,7 @@
|
||||
{:team "Graphy"
|
||||
:api #{metabase.analyze.core}
|
||||
:uses #{config
|
||||
driver
|
||||
lib
|
||||
models
|
||||
query-processor
|
||||
@@ -353,7 +338,8 @@
|
||||
connection-pool
|
||||
task
|
||||
util}
|
||||
:model-imports #{:model/Field
|
||||
:model-imports #{:model/Database
|
||||
:model/Field
|
||||
:model/QueryCache
|
||||
:model/Secret
|
||||
:model/Setting}}
|
||||
@@ -369,7 +355,6 @@
|
||||
{:team "UX West"
|
||||
:api #{metabase.audit-app.core
|
||||
metabase.audit-app.init}
|
||||
:friends #{enterprise/memoize-monitor}
|
||||
:uses #{api
|
||||
app-db
|
||||
events
|
||||
@@ -507,29 +492,23 @@
|
||||
driver
|
||||
events
|
||||
formatter
|
||||
geojson
|
||||
lib
|
||||
lib-be
|
||||
models
|
||||
notification
|
||||
parameters
|
||||
permissions
|
||||
pivot
|
||||
premium-features
|
||||
query-processor
|
||||
request
|
||||
settings
|
||||
slackbot
|
||||
system
|
||||
task
|
||||
tiles
|
||||
timeline
|
||||
types
|
||||
util}
|
||||
:model-exports #{:model/Channel :model/ChannelTemplate}
|
||||
:model-imports #{:model/Dashboard
|
||||
:model/DashboardCard
|
||||
:model/DashboardTab
|
||||
:model/Database
|
||||
:model/PulseChannel
|
||||
:model/User}}
|
||||
@@ -585,7 +564,6 @@
|
||||
collections
|
||||
{:team "UX West"
|
||||
:api #{metabase.collections.core
|
||||
metabase.collections.curation
|
||||
metabase.collections.models.collection
|
||||
metabase.collections.models.collection.root
|
||||
metabase.collections.schema}
|
||||
@@ -714,7 +692,6 @@
|
||||
audit-app
|
||||
collections
|
||||
config
|
||||
embedding
|
||||
events
|
||||
lib
|
||||
models
|
||||
@@ -725,9 +702,8 @@
|
||||
query-processor
|
||||
search
|
||||
settings
|
||||
staleness
|
||||
sync
|
||||
util}
|
||||
util
|
||||
warehouse-schema}
|
||||
:model-exports #{:model/Dashboard
|
||||
:model/DashboardCard
|
||||
:model/DashboardCardSeries
|
||||
@@ -840,6 +816,7 @@
|
||||
metabase.driver.common
|
||||
metabase.driver.connection
|
||||
metabase.driver.ddl.interface
|
||||
metabase.driver.h2
|
||||
metabase.driver.impl
|
||||
metabase.driver.init
|
||||
metabase.driver.mysql
|
||||
@@ -868,6 +845,7 @@
|
||||
config
|
||||
driver-api
|
||||
events
|
||||
legacy-mbql
|
||||
lib
|
||||
lib-be
|
||||
premium-features
|
||||
@@ -917,7 +895,6 @@
|
||||
metabase.eid-translation.core
|
||||
metabase.eid-translation.init}
|
||||
:uses #{api
|
||||
batch-processing
|
||||
settings
|
||||
util}
|
||||
:model-imports #{:model/Action
|
||||
@@ -974,14 +951,7 @@
|
||||
request
|
||||
tiles
|
||||
util}
|
||||
:model-imports #{:model/Card :model/Dashboard :model/DashboardCard :model/EmbeddingTheme}}
|
||||
|
||||
entity-retrieval
|
||||
{:team "Metabot"
|
||||
:api #{metabase.entity-retrieval.core
|
||||
metabase.entity-retrieval.mirror}
|
||||
:uses #{premium-features}
|
||||
:model-imports #{:model/OsiAiContext}}
|
||||
:model-imports #{:model/Card :model/Dashboard :model/EmbeddingTheme}}
|
||||
|
||||
events
|
||||
{:team "DevEx"
|
||||
@@ -1022,8 +992,7 @@
|
||||
geojson
|
||||
{:team "UX West"
|
||||
:api #{metabase.geojson.api
|
||||
metabase.geojson.init
|
||||
metabase.geojson.settings}
|
||||
metabase.geojson.init}
|
||||
:uses #{api
|
||||
config
|
||||
permissions
|
||||
@@ -1045,17 +1014,6 @@
|
||||
:api #{metabase.graph.core}
|
||||
:uses #{util}}
|
||||
|
||||
health-inspector
|
||||
{:team "Querying Platform"
|
||||
:api #{metabase.health-inspector.api
|
||||
metabase.health-inspector.init}
|
||||
:uses #{api
|
||||
lib
|
||||
lib-be
|
||||
settings
|
||||
task
|
||||
util}}
|
||||
|
||||
indexed-entities
|
||||
{:team "UX West"
|
||||
:api #{metabase.indexed-entities.api
|
||||
@@ -1066,7 +1024,6 @@
|
||||
legacy-mbql
|
||||
lib
|
||||
models
|
||||
permissions
|
||||
query-processor
|
||||
search
|
||||
sync
|
||||
@@ -1075,27 +1032,6 @@
|
||||
:model-exports #{:model/ModelIndex :model/ModelIndexValue}
|
||||
:model-imports #{:model/Card :model/Collection}}
|
||||
|
||||
indexes
|
||||
{:team "Gadget"
|
||||
:api #{metabase.indexes.models.table-index
|
||||
metabase.indexes.reconcile
|
||||
metabase.indexes.schema}
|
||||
:uses #{driver
|
||||
models
|
||||
util}
|
||||
:model-exports #{:model/TableIndex}}
|
||||
|
||||
indexes-rest
|
||||
{:team "Gadget"
|
||||
:api #{metabase.indexes-rest.api}
|
||||
:uses #{api
|
||||
indexes
|
||||
transforms-base
|
||||
util}
|
||||
:model-imports #{:model/Database
|
||||
:model/TableIndex
|
||||
:model/Transform}}
|
||||
|
||||
initialization-status
|
||||
{:team "UX West"
|
||||
:api #{metabase.initialization-status.core}
|
||||
@@ -1135,7 +1071,6 @@
|
||||
metabase.lib.metadata
|
||||
metabase.lib.metadata.protocols
|
||||
metabase.lib.options
|
||||
metabase.lib.pivot
|
||||
metabase.lib.schema
|
||||
metabase.lib.schema.actions
|
||||
metabase.lib.schema.aggregation
|
||||
@@ -1223,10 +1158,7 @@
|
||||
sync
|
||||
util
|
||||
warehouse-schema}
|
||||
:model-imports #{:model/Card
|
||||
:model/Database
|
||||
:model/Field
|
||||
:model/Table}}
|
||||
:model-imports #{:model/Card :model/Database :model/Field :model/Table}}
|
||||
|
||||
logger
|
||||
{:team "DevEx"
|
||||
@@ -1259,19 +1191,13 @@
|
||||
:api #{metabase.mcp.api
|
||||
metabase.mcp.callback-api
|
||||
metabase.mcp.core
|
||||
metabase.mcp.init
|
||||
metabase.mcp.usage}
|
||||
:model-exports #{:model/McpSessionLog
|
||||
:model/McpToolCallLog}
|
||||
metabase.mcp.init}
|
||||
:uses #{agent-api
|
||||
ai-tracing
|
||||
api
|
||||
app-db
|
||||
config
|
||||
llm
|
||||
metabot
|
||||
oauth-server
|
||||
premium-features
|
||||
request
|
||||
server
|
||||
session
|
||||
@@ -1310,7 +1236,6 @@
|
||||
slackbot}
|
||||
:uses #{activity-feed
|
||||
agent-lib
|
||||
ai-tracing
|
||||
analytics
|
||||
analytics-interface
|
||||
api
|
||||
@@ -1322,7 +1247,6 @@
|
||||
config
|
||||
documents
|
||||
driver
|
||||
entity-retrieval
|
||||
interestingness
|
||||
legacy-mbql
|
||||
lib
|
||||
@@ -1342,6 +1266,7 @@
|
||||
settings
|
||||
slackbot
|
||||
sql-tools
|
||||
store-api
|
||||
sync
|
||||
system
|
||||
task
|
||||
@@ -1357,8 +1282,6 @@
|
||||
:model-imports #{:model/Card
|
||||
:model/Collection
|
||||
:model/Dashboard
|
||||
:model/DashboardCard
|
||||
:model/DashboardTab
|
||||
:model/Database
|
||||
:model/Field
|
||||
:model/Glossary
|
||||
@@ -1448,17 +1371,6 @@
|
||||
:model/Table
|
||||
:model/User}}
|
||||
|
||||
mq
|
||||
{:team "UX West"
|
||||
:api #{metabase.mq.init}
|
||||
:uses #{analytics-interface
|
||||
app-db
|
||||
classloader
|
||||
settings
|
||||
startup
|
||||
task
|
||||
util}}
|
||||
|
||||
native-query-snippets
|
||||
{:team "Graphy"
|
||||
:api #{metabase.native-query-snippets.api
|
||||
@@ -1485,7 +1397,6 @@
|
||||
:uses #{analytics
|
||||
analytics-interface
|
||||
api
|
||||
app-db
|
||||
appearance
|
||||
auth-identity
|
||||
channel
|
||||
@@ -1516,18 +1427,14 @@
|
||||
:model/Dashboard
|
||||
:model/DashboardCard
|
||||
:model/DashboardTab
|
||||
:model/TaskHistory
|
||||
:model/TaskRun
|
||||
:model/User}}
|
||||
|
||||
oauth-server
|
||||
{:team "UX West"
|
||||
:api #{metabase.oauth-server.api ;; Special case OAuth standard endpoints under `/`
|
||||
metabase.oauth-server.api.admin ;; The usual Metabase internal API endpoints under `/api`
|
||||
:api #{metabase.oauth-server.api
|
||||
metabase.oauth-server.core
|
||||
metabase.oauth-server.init}
|
||||
:uses #{api
|
||||
api-scope
|
||||
appearance
|
||||
mcp
|
||||
models
|
||||
@@ -1538,20 +1445,6 @@
|
||||
tracing
|
||||
util}}
|
||||
|
||||
osi
|
||||
{:team "Metabot"
|
||||
:api #{metabase.osi.ai-context.api}
|
||||
:uses #{api
|
||||
app-db
|
||||
entity-retrieval
|
||||
models
|
||||
request
|
||||
util}
|
||||
:model-exports #{:model/OsiAiContext}
|
||||
:model-imports #{:model/Card
|
||||
:model/Measure
|
||||
:model/Segment}}
|
||||
|
||||
;; TODO -- too many API namespaces
|
||||
parameters
|
||||
{:team "Graphy"
|
||||
@@ -1573,7 +1466,8 @@
|
||||
models
|
||||
query-processor
|
||||
util
|
||||
warehouse-schema}
|
||||
warehouse-schema
|
||||
warehouses}
|
||||
:model-imports #{:model/Card
|
||||
:model/Dashboard
|
||||
:model/Field
|
||||
@@ -1632,8 +1526,7 @@
|
||||
|
||||
pivot
|
||||
{:team "UX West"
|
||||
:api #{metabase.pivot.core
|
||||
metabase.pivot.postprocess}
|
||||
:api #{metabase.pivot.core}
|
||||
:uses #{models
|
||||
util}}
|
||||
|
||||
@@ -1772,9 +1665,7 @@
|
||||
content-verification
|
||||
dashboards
|
||||
documents
|
||||
embedding
|
||||
events
|
||||
graph
|
||||
lib
|
||||
lib-be
|
||||
metrics
|
||||
@@ -1788,9 +1679,6 @@
|
||||
query-processor
|
||||
request
|
||||
search
|
||||
settings
|
||||
staleness
|
||||
sync
|
||||
util
|
||||
view-log
|
||||
warehouse-schema}
|
||||
@@ -1878,7 +1766,6 @@
|
||||
metabase.query-processor.middleware.permissions
|
||||
metabase.query-processor.parameters.dates
|
||||
metabase.query-processor.parameters.operators
|
||||
metabase.query-processor.parameters.values
|
||||
metabase.query-processor.pipeline
|
||||
metabase.query-processor.pivot
|
||||
metabase.query-processor.preprocess
|
||||
@@ -1902,7 +1789,6 @@
|
||||
cache
|
||||
config
|
||||
dashboards
|
||||
database-routing
|
||||
driver
|
||||
events
|
||||
formatter
|
||||
@@ -1917,7 +1803,6 @@
|
||||
premium-features
|
||||
queries
|
||||
query-permissions
|
||||
request
|
||||
server
|
||||
settings
|
||||
system
|
||||
@@ -1926,6 +1811,8 @@
|
||||
util
|
||||
workspaces}
|
||||
:model-imports #{:model/Card
|
||||
:model/Dashboard
|
||||
:model/DashboardCard
|
||||
:model/DashboardCardSeries
|
||||
:model/Database
|
||||
:model/QueryCache
|
||||
@@ -1984,24 +1871,15 @@
|
||||
:model/Segment
|
||||
:model/Transform}}
|
||||
|
||||
run-tracking
|
||||
{:team "Gadget"
|
||||
:api #{metabase.run-tracking.core}
|
||||
:uses #{analytics-interface
|
||||
app-db
|
||||
task
|
||||
util}}
|
||||
|
||||
sample-data
|
||||
{:team "UX West"
|
||||
:api #{metabase.sample-data.core
|
||||
metabase.sample-data.init}
|
||||
:uses #{driver
|
||||
plugins
|
||||
:uses #{plugins
|
||||
settings
|
||||
sync
|
||||
util}
|
||||
:model-imports #{:model/DataPermissions :model/Database :model/Table}}
|
||||
:model-imports #{:model/Database}}
|
||||
|
||||
search
|
||||
{:team "UX West"
|
||||
@@ -2099,7 +1977,6 @@
|
||||
tracing
|
||||
users
|
||||
util
|
||||
enterprise/custom-viz-plugin
|
||||
enterprise/sso}
|
||||
:model-imports #{:model/Session}}
|
||||
|
||||
@@ -2115,7 +1992,6 @@
|
||||
channel
|
||||
config
|
||||
events
|
||||
premium-features
|
||||
request
|
||||
settings
|
||||
sso
|
||||
@@ -2213,7 +2089,8 @@
|
||||
sql-parsing
|
||||
{:team "DevEx"
|
||||
:api #{metabase.sql-parsing.core}
|
||||
:uses #{analytics-interface util}}
|
||||
:uses #{analytics-interface
|
||||
util}}
|
||||
|
||||
sql-tools
|
||||
{:team "DevEx"
|
||||
@@ -2247,10 +2124,6 @@
|
||||
util}
|
||||
:model-imports #{:model/AuthIdentity :model/PermissionsGroupMembership}}
|
||||
|
||||
staleness
|
||||
{:team "UX West"
|
||||
:api #{metabase.staleness.core}}
|
||||
|
||||
startup
|
||||
{:team "DevEx"
|
||||
:api #{metabase.startup.core}
|
||||
@@ -2268,7 +2141,6 @@
|
||||
{:team "Graphy"
|
||||
:api #{metabase.sync.api
|
||||
metabase.sync.core
|
||||
metabase.sync.field-values
|
||||
metabase.sync.init
|
||||
metabase.sync.schedules
|
||||
metabase.sync.task.sync-databases
|
||||
@@ -2284,7 +2156,6 @@
|
||||
events
|
||||
interestingness
|
||||
lib
|
||||
lib-be
|
||||
models
|
||||
premium-features
|
||||
query-processor
|
||||
@@ -2322,17 +2193,17 @@
|
||||
metabase.task-history.core
|
||||
metabase.task-history.init}
|
||||
:uses #{api
|
||||
app-db
|
||||
config
|
||||
models
|
||||
permissions
|
||||
premium-features
|
||||
query-processor
|
||||
request
|
||||
run-tracking
|
||||
task
|
||||
tracing
|
||||
util}
|
||||
:model-exports #{:model/TaskHistory :model/TaskRun}
|
||||
:model-exports #{:model/TaskHistory}
|
||||
:model-imports #{:model/Card :model/Dashboard :model/Database}}
|
||||
|
||||
tenants
|
||||
@@ -2352,11 +2223,9 @@
|
||||
config
|
||||
lib
|
||||
lib-be
|
||||
mcp
|
||||
permissions
|
||||
premium-features
|
||||
search
|
||||
session
|
||||
util}
|
||||
:model-imports #{:model/AiUsageLog
|
||||
:model/Card
|
||||
@@ -2371,8 +2240,7 @@
|
||||
tiles
|
||||
{:team "UX West"
|
||||
:api #{metabase.tiles.api
|
||||
metabase.tiles.init
|
||||
metabase.tiles.settings}
|
||||
metabase.tiles.init}
|
||||
:uses #{api
|
||||
legacy-mbql
|
||||
lib
|
||||
@@ -2380,8 +2248,7 @@
|
||||
parameters
|
||||
query-processor
|
||||
settings
|
||||
util}
|
||||
:model-imports #{:model/Card :model/Dashboard :model/DashboardCard}}
|
||||
util}}
|
||||
|
||||
timeline
|
||||
{:team "UX West"
|
||||
@@ -2421,7 +2288,6 @@
|
||||
database-routing
|
||||
driver
|
||||
events
|
||||
indexes
|
||||
lib
|
||||
lib-be
|
||||
models
|
||||
@@ -2431,7 +2297,6 @@
|
||||
query-processor
|
||||
remote-sync
|
||||
revisions
|
||||
run-tracking
|
||||
search
|
||||
settings
|
||||
task
|
||||
@@ -2443,7 +2308,6 @@
|
||||
enterprise/transforms-python}
|
||||
:model-exports #{:model/Transform
|
||||
:model/TransformJob
|
||||
:model/TransformJobRun
|
||||
:model/TransformJobTransformTag
|
||||
:model/TransformRun
|
||||
:model/TransformTag}
|
||||
@@ -2451,7 +2315,6 @@
|
||||
:model/Database
|
||||
:model/Field
|
||||
:model/Table
|
||||
:model/TableIndex
|
||||
:model/User}}
|
||||
|
||||
transforms-base
|
||||
@@ -2463,15 +2326,12 @@
|
||||
;; :model/TransformRun directly. Remove once the module is agnostic to global
|
||||
;; versus workspace transforms. See https://linear.app/metabase/issue/BOT-1143
|
||||
:model-imports #{:model/Database
|
||||
:model/Field
|
||||
:model/Table
|
||||
:model/TableIndex
|
||||
:model/Transform
|
||||
:model/TransformRun}
|
||||
:uses #{database-routing
|
||||
driver
|
||||
events
|
||||
indexes
|
||||
lib
|
||||
lib-be
|
||||
premium-features
|
||||
@@ -2500,7 +2360,6 @@
|
||||
util}
|
||||
:model-imports #{:model/Transform
|
||||
:model/TransformJob
|
||||
:model/TransformJobRun
|
||||
:model/TransformJobTransformTag
|
||||
:model/TransformRun
|
||||
:model/TransformTag}}
|
||||
@@ -2538,23 +2397,6 @@
|
||||
:model/Field
|
||||
:model/Table}}
|
||||
|
||||
usage-metadata
|
||||
{:team "UX West"
|
||||
:api #{metabase.usage-metadata.init}
|
||||
:model-imports #{:model/Card
|
||||
:model/Field
|
||||
:model/Query
|
||||
:model/QueryExecution
|
||||
:model/Segment
|
||||
:model/Table}
|
||||
:uses #{app-db
|
||||
lib
|
||||
lib-be
|
||||
models
|
||||
settings
|
||||
task
|
||||
util}}
|
||||
|
||||
user-key-value
|
||||
{:team "UX West"
|
||||
:api #{metabase.user-key-value.api
|
||||
@@ -2660,7 +2502,6 @@
|
||||
warehouse-schema
|
||||
{:team "UX West"
|
||||
:api #{metabase.warehouse-schema.field
|
||||
metabase.warehouse-schema.field-values.distinct-batch
|
||||
metabase.warehouse-schema.metadata-from-qp
|
||||
metabase.warehouse-schema.metadata-queries
|
||||
metabase.warehouse-schema.models.field
|
||||
@@ -2835,11 +2676,10 @@
|
||||
driver
|
||||
events
|
||||
lib
|
||||
lib-be
|
||||
models
|
||||
query-processor
|
||||
sync
|
||||
util}
|
||||
util
|
||||
warehouse-schema}
|
||||
:model-imports #{:model/Database
|
||||
:model/Field
|
||||
:model/FieldValues
|
||||
@@ -2898,7 +2738,6 @@
|
||||
:uses #{driver
|
||||
premium-features
|
||||
enterprise/advanced-config
|
||||
enterprise/mfa
|
||||
enterprise/scim
|
||||
enterprise/semantic-search
|
||||
enterprise/sso}
|
||||
@@ -2924,6 +2763,7 @@
|
||||
app-db
|
||||
audit-app
|
||||
classloader
|
||||
config
|
||||
driver
|
||||
lib
|
||||
models
|
||||
@@ -3055,8 +2895,7 @@
|
||||
enterprise/custom-viz-plugin
|
||||
{:team "Gadget"
|
||||
:api #{metabase-enterprise.custom-viz-plugin.api
|
||||
metabase-enterprise.custom-viz-plugin.init
|
||||
metabase-enterprise.custom-viz-plugin.settings}
|
||||
metabase-enterprise.custom-viz-plugin.init}
|
||||
:uses #{api
|
||||
config
|
||||
events
|
||||
@@ -3132,6 +2971,7 @@
|
||||
{:team "UX West"
|
||||
:api #{metabase-enterprise.database-routing.api}
|
||||
:uses #{api
|
||||
config
|
||||
database-routing
|
||||
driver
|
||||
events
|
||||
@@ -3154,7 +2994,8 @@
|
||||
metabase-enterprise.dependencies.core
|
||||
metabase-enterprise.dependencies.init
|
||||
metabase-enterprise.dependencies.models.dependency}
|
||||
:uses #{api
|
||||
:uses #{analyze
|
||||
api
|
||||
app-db
|
||||
collections
|
||||
database-routing
|
||||
@@ -3165,6 +3006,7 @@
|
||||
lib
|
||||
lib-be
|
||||
models
|
||||
native-query-snippets
|
||||
permissions
|
||||
premium-features
|
||||
queries
|
||||
@@ -3227,29 +3069,12 @@
|
||||
;; make token tracking a hook). Drop this dep once the relocation lands.
|
||||
:uses #{enterprise/semantic-search}}
|
||||
|
||||
enterprise/entity-retrieval
|
||||
{:team "Metabot"
|
||||
:api #{metabase-enterprise.entity-retrieval.init}
|
||||
:uses #{analytics-interface
|
||||
collections
|
||||
entity-retrieval
|
||||
premium-features
|
||||
task
|
||||
util
|
||||
enterprise/semantic-search}
|
||||
:model-imports #{:model/Card
|
||||
:model/Measure
|
||||
:model/OsiAiContext
|
||||
:model/Segment
|
||||
:model/Table}}
|
||||
|
||||
enterprise/erd
|
||||
{:team "Gadget"
|
||||
:api #{metabase-enterprise.erd.api}
|
||||
:uses #{api
|
||||
models
|
||||
util
|
||||
warehouse-schema}
|
||||
util}
|
||||
:model-imports #{:model/Database
|
||||
:model/Field
|
||||
:model/Table}}
|
||||
@@ -3304,32 +3129,6 @@
|
||||
premium-features}
|
||||
:model-imports #{:model/Card :model/Collection :model/Table}}
|
||||
|
||||
enterprise/mcp
|
||||
{:team "Metabot"
|
||||
:api #{metabase-enterprise.mcp.init}
|
||||
:uses #{analytics
|
||||
mcp
|
||||
metabot
|
||||
premium-features
|
||||
task
|
||||
util}
|
||||
:model-imports #{:model/McpSessionLog
|
||||
:model/McpToolCallLog}}
|
||||
|
||||
enterprise/memoize-monitor
|
||||
{:team "Graphy"
|
||||
:api #{metabase-enterprise.memoize-monitor.init}
|
||||
:uses #{analytics
|
||||
analytics-interface
|
||||
audit-app
|
||||
collections
|
||||
driver
|
||||
models
|
||||
util
|
||||
warehouse-schema
|
||||
warehouses
|
||||
enterprise/serialization}}
|
||||
|
||||
enterprise/metabot
|
||||
{:team "Metabot"
|
||||
:api #{metabase-enterprise.metabot.api
|
||||
@@ -3369,21 +3168,19 @@
|
||||
util}
|
||||
:model-imports #{:model/MetabotConversation :model/MetabotFeedback :model/MetabotMessage}}
|
||||
|
||||
enterprise/mfa
|
||||
{:team "Graphy"
|
||||
:api #{metabase-enterprise.mfa.core
|
||||
metabase-enterprise.mfa.init
|
||||
metabase-enterprise.mfa.routes}
|
||||
enterprise/notification-admin
|
||||
{:team "Gadget"
|
||||
:api #{metabase-enterprise.notification-admin.api}
|
||||
:uses #{api
|
||||
appearance
|
||||
channel
|
||||
config
|
||||
events
|
||||
premium-features
|
||||
settings
|
||||
sso
|
||||
app-db
|
||||
models
|
||||
notification
|
||||
request
|
||||
util}
|
||||
:model-imports #{:model/AuthIdentity :model/User}}
|
||||
:model-imports #{:model/Notification
|
||||
:model/NotificationHandler
|
||||
:model/NotificationRecipient
|
||||
:model/TaskHistory}}
|
||||
|
||||
enterprise/permission-debug
|
||||
{:team "UX West"
|
||||
@@ -3414,7 +3211,6 @@
|
||||
events
|
||||
models
|
||||
premium-features
|
||||
search
|
||||
settings
|
||||
startup
|
||||
task
|
||||
@@ -3552,19 +3348,14 @@
|
||||
{:team "Metabot"
|
||||
:api #{metabase-enterprise.semantic-search.api
|
||||
metabase-enterprise.semantic-search.core
|
||||
metabase-enterprise.semantic-search.db.datasource
|
||||
metabase-enterprise.semantic-search.embedding
|
||||
metabase-enterprise.semantic-search.init}
|
||||
:uses #{activity-feed
|
||||
analytics
|
||||
analytics-interface
|
||||
api
|
||||
app-db
|
||||
collections
|
||||
config
|
||||
connection-pool
|
||||
events
|
||||
lib-be
|
||||
llm
|
||||
metabot
|
||||
models
|
||||
@@ -3575,9 +3366,7 @@
|
||||
task
|
||||
tracing
|
||||
util}
|
||||
:model-imports #{:model/Collection
|
||||
:model/Dashboard
|
||||
:model/Table}}
|
||||
:model-imports #{:model/Card :model/Collection :model/Dashboard}}
|
||||
|
||||
enterprise/serialization
|
||||
{:team "Graphy"
|
||||
@@ -3587,12 +3376,12 @@
|
||||
metabase-enterprise.serialization.init
|
||||
metabase-enterprise.serialization.schema
|
||||
metabase-enterprise.serialization.v2.backfill-ids}
|
||||
:friends #{enterprise/memoize-monitor}
|
||||
:uses #{analytics
|
||||
api
|
||||
app-db
|
||||
appearance
|
||||
collections
|
||||
config
|
||||
events
|
||||
lib
|
||||
logger
|
||||
@@ -3633,7 +3422,6 @@
|
||||
settings
|
||||
sso
|
||||
system
|
||||
task
|
||||
util
|
||||
enterprise/scim}
|
||||
:model-imports #{:model/PermissionsGroup :model/Session}}
|
||||
@@ -3645,11 +3433,11 @@
|
||||
:uses #{analytics
|
||||
api
|
||||
collections
|
||||
embedding
|
||||
premium-features
|
||||
queries
|
||||
request
|
||||
settings
|
||||
staleness
|
||||
util}
|
||||
:model-imports #{:model/Card :model/Collection :model/Dashboard}}
|
||||
|
||||
@@ -3688,9 +3476,7 @@
|
||||
{:team "Graphy"
|
||||
:api #{metabase-enterprise.transforms.api}
|
||||
:uses #{premium-features
|
||||
util
|
||||
enterprise/transforms-inspector}
|
||||
:model-imports #{:model/Transform}}
|
||||
enterprise/transforms-inspector}}
|
||||
|
||||
enterprise/transforms-inspector
|
||||
{:team "Gadget"
|
||||
@@ -3773,7 +3559,6 @@
|
||||
enterprise/serialization}
|
||||
:model-exports #{:model/Workspace}
|
||||
:model-imports #{:model/Database
|
||||
:model/Table
|
||||
:model/User}}}
|
||||
|
||||
;; namespaces matching these patterns (with `re-find`) are excluded from the module linter. Since regex literals
|
||||
|
||||
@@ -36,9 +36,6 @@
|
||||
;; Definitely thread safe
|
||||
metabase.test.util.dynamic-redefs/patch-vars!
|
||||
|
||||
;; test-only cleanup of in-memory throttle state (atom resets)
|
||||
metabase.session.api/reset-throttlers-for-testing!
|
||||
|
||||
;; TODO: most of these symbols shouldn't be here, we should go through them and
|
||||
;; find the functions/macros that use them and make sure their names end with !
|
||||
;; best way to do this is try remove each of these and rely on kondo output to find places where it's used
|
||||
|
||||
@@ -6,8 +6,3 @@
|
||||
`(def ~(-> sym name symbol) "docstring" ~sym))
|
||||
([sym name]
|
||||
`(def ~name "docstring" ~sym)))
|
||||
|
||||
(defmacro import-macro
|
||||
"Kondo macro replacement for [[metabase.util.namespaces/import-macro]]."
|
||||
[sym]
|
||||
`(def ~(-> sym name symbol) "docstring" ~sym))
|
||||
|
||||
@@ -224,38 +224,26 @@
|
||||
|
||||
nil)))))))
|
||||
|
||||
(defn is [{:keys [node lang], :as input}]
|
||||
(defn is [{:keys [node lang]}]
|
||||
(when (= lang :cljs)
|
||||
(warn-about-missing-test-expr-requires-in-cljs node))
|
||||
input)
|
||||
{:node node})
|
||||
|
||||
(defn use-fixtures
|
||||
"Flag `:parallel/unsafe` forms inside fixtures. Skipped when the surrounding
|
||||
namespace is marked `^:synchronous` -- the ns marker is the author's
|
||||
explicit opt-in to a single-threaded test ns where destructive setup is
|
||||
safe. Without that opt-in, kondo can't know whether sibling deftests will
|
||||
be `^:parallel`, so the conservative default is to flag.
|
||||
namespace is marked `^:synchronous` -- the ns marker is the author's
|
||||
explicit opt-in to a single-threaded test ns where destructive setup is
|
||||
safe. Without that opt-in, kondo can't know whether sibling deftests will
|
||||
be `^:parallel`, so the conservative default is to flag.
|
||||
|
||||
Sister to [[deftest-check-parallel]]: the deftest hook rejects `^:parallel`
|
||||
tests inside a `^:synchronous` ns, so the two checks together form a
|
||||
coherent opt-in: either the whole ns is synchronous (fixtures can be
|
||||
anything; no parallel tests allowed) or it isn't (fixtures must be
|
||||
parallel-safe)."
|
||||
[{:keys [node config], ns-sym :ns, :as input}]
|
||||
Sister to [[deftest-check-parallel]]: the deftest hook rejects `^:parallel`
|
||||
tests inside a `^:synchronous` ns, so the two checks together form a
|
||||
coherent opt-in: either the whole ns is synchronous (fixtures can be
|
||||
anything; no parallel tests allowed) or it isn't (fixtures must be
|
||||
parallel-safe)."
|
||||
[{:keys [node config], ns-sym :ns}]
|
||||
(when-not (:synchronous (meta ns-sym))
|
||||
(let [linter-config (get-in config [:linters :metabase/validate-deftest])]
|
||||
(doseq [form (rest (:children node))]
|
||||
(warn-about-disallowed-parallel-forms form linter-config))))
|
||||
input)
|
||||
|
||||
(defn testing
|
||||
"Check that we don't have an empty `testing` form like
|
||||
|
||||
(testing \"message\")"
|
||||
[{{[_testing _message & body] :children, :as node} :node, :as input}]
|
||||
(when (empty? body)
|
||||
(hooks/reg-finding!
|
||||
(assoc (meta node)
|
||||
:message "A `testing` form that doesn't wrap anything doesn't do anything"
|
||||
:type :metabase/check-testing-not-empty)))
|
||||
input)
|
||||
{:node node})
|
||||
|
||||
@@ -1,18 +0,0 @@
|
||||
(ns hooks.metabase.mq.impl
|
||||
(:require
|
||||
[clj-kondo.hooks-api :as hooks]))
|
||||
|
||||
(defn def-listener
|
||||
"Kondo hook for `mq/def-listener`. Rewrites to a `fn` form so kondo can analyze bindings.
|
||||
|
||||
Form:
|
||||
(mq/def-listener! :queue/bar [msgs] body...)"
|
||||
[{:keys [node]}]
|
||||
(let [[_ _channel-name bindings & body] (:children node)
|
||||
;; rewrite as: (fn [bindings...] body...)
|
||||
new-node (hooks/list-node
|
||||
(list*
|
||||
(hooks/token-node 'fn)
|
||||
bindings
|
||||
body))]
|
||||
{:node new-node}))
|
||||
@@ -1,6 +1,7 @@
|
||||
(ns hooks.metabase.util.i18n
|
||||
(:require
|
||||
[clj-kondo.hooks-api :as api]
|
||||
[clojure.string :as str]
|
||||
[hooks.common]))
|
||||
|
||||
(defn- valid-apostrophes?
|
||||
|
||||
@@ -1,132 +0,0 @@
|
||||
(ns hooks.metabase.util.match
|
||||
(:require [clj-kondo.hooks-api :as api]))
|
||||
|
||||
(defn- token-node? [node token-name]
|
||||
(and (api/token-node? node)
|
||||
(= (str (:value node)) token-name)))
|
||||
|
||||
(defn- keyword-node? [node k]
|
||||
(and (api/keyword-node? node)
|
||||
(= (:k node) k)))
|
||||
|
||||
(defn- binding-symbol?
|
||||
"True when the token is a plain symbol that should be treated as a binding
|
||||
(not a keyword, not a boolean literal, not nil, not a number, not `_`)."
|
||||
[node]
|
||||
(and (api/token-node? node)
|
||||
(let [v (:value node)]
|
||||
(and (symbol? v)
|
||||
(not (contains? '#{_ nil true false} v))))))
|
||||
|
||||
;; To establish fake let bindings, we need to bind them to something. This "something" has to be vague enough to trick
|
||||
;; kondo into not looking too much into it. E.g., if we bind everything to `nil`, Kondo linters will get fired up.
|
||||
(def opaque-node (api/list-node (list (api/token-node 'rand-nth) (api/list-node []))))
|
||||
|
||||
(defn collect-bindings
|
||||
"Return a list of token nodes (symbols) introduced as bindings by `pattern`."
|
||||
[pattern]
|
||||
(cond
|
||||
(binding-symbol? pattern)
|
||||
[[pattern opaque-node]]
|
||||
|
||||
;; Lists: (:or ...), (:and ...), (sym :guard pred ...), or a quoted symbol.
|
||||
(api/list-node? pattern)
|
||||
(let [[head & rst] (:children pattern)]
|
||||
(cond (or (keyword-node? head :or)
|
||||
(keyword-node? head :and))
|
||||
(mapcat collect-bindings rst)
|
||||
|
||||
(api/token-node? head)
|
||||
(cond-> []
|
||||
(binding-symbol? head) (conj [head opaque-node])
|
||||
(keyword-node? (first rst) :guard) (conj [(api/token-node '_) (second rst)]))
|
||||
|
||||
;; anything else (e.g. a function call used as predicate) — no bindings
|
||||
:else []))
|
||||
|
||||
(api/vector-node? pattern)
|
||||
(mapcat (fn [node]
|
||||
(when-not (token-node? node "&")
|
||||
(collect-bindings node)))
|
||||
(:children pattern))
|
||||
|
||||
;; Maps: keys are literal selectors, values are sub-patterns.
|
||||
;; Special value-symbols &truthy and _ are NOT user bindings.
|
||||
(api/map-node? pattern)
|
||||
(let [children (:children pattern)
|
||||
pairs (partition 2 children)]
|
||||
(mapcat (fn [[_k v]]
|
||||
(when-not (or (token-node? v "&truthy")
|
||||
(token-node? v "_"))
|
||||
(collect-bindings v)))
|
||||
pairs))
|
||||
|
||||
;; For sets, make sure to use them because they may reference some of the outer local bindings, so if we don't
|
||||
;; include them, Kondo will complain about those bindings being unused.
|
||||
(api/set-node? pattern)
|
||||
[[(api/token-node '_) pattern]]
|
||||
|
||||
;; Anything else — no bindings.
|
||||
:else []))
|
||||
|
||||
(defn- dedupe-bindings
|
||||
"Prevent Kondo complaining that we have multiple fake bindings."
|
||||
[pairs]
|
||||
(loop [[[sym v] & r] pairs, seen #{}, result []]
|
||||
(cond (nil? sym) result
|
||||
(and (seen sym) (not (token-node? sym "_"))) (recur r seen result)
|
||||
:else (recur r (conj seen sym) (conj result sym v)))))
|
||||
|
||||
(defn match-one
|
||||
"clj-kondo :analyze-call hook for match-one.
|
||||
|
||||
Rewrites
|
||||
|
||||
(match-one value
|
||||
pat1 ret1
|
||||
pat2 ret2
|
||||
...)
|
||||
|
||||
into a `let` form that declares all binding symbols introduced by the patterns, so clj-kondo knows they are
|
||||
legitimate and won't warn about unresolved/unused symbols:
|
||||
|
||||
(let [<all-bindings> (rand-nth []) ...]
|
||||
(or ret1 ret2 ...))"
|
||||
[{:keys [node]}]
|
||||
(let [[_ value & clauses] (:children node)]
|
||||
(when (odd? (count clauses))
|
||||
(api/reg-finding!
|
||||
{:message "match-one/match-many/replace requires an even number of pattern/result pairs"
|
||||
:type :metabase/match-odd-clauses
|
||||
:row (:row node)
|
||||
:col (:col node)
|
||||
:filename (-> node meta :filename)}))
|
||||
(let [pairs (partition 2 clauses)
|
||||
all-bindings (->> pairs
|
||||
(mapcat (fn [[pat _ret]] (collect-bindings pat)))
|
||||
dedupe-bindings)
|
||||
special-bindings [(api/token-node '_) value
|
||||
(api/token-node '&match) opaque-node
|
||||
(api/token-node '&parents) opaque-node
|
||||
(api/token-node '&recur) opaque-node
|
||||
;; Pretend to use &match, &parents, and &recur so that Kondo doesn't complain about them
|
||||
;; if they are not used by the user code.
|
||||
(api/token-node '_) (api/token-node '&match)
|
||||
(api/token-node '_) (api/token-node '&parents)
|
||||
(api/token-node '_) (api/token-node '&recur)]
|
||||
binding-nodes (into special-bindings all-bindings)
|
||||
result-exprs (map second pairs)
|
||||
new-node (api/list-node
|
||||
(list
|
||||
(api/token-node 'let)
|
||||
(api/vector-node binding-nodes)
|
||||
(api/list-node
|
||||
(concat [(api/token-node 'clojure.core/or)] result-exprs [opaque-node]))))]
|
||||
{:node new-node})))
|
||||
|
||||
(def match-many match-one)
|
||||
(def replace match-one)
|
||||
|
||||
(defn replace-in [context]
|
||||
;; Everything is the same as in match-one, just drop the middle argument.
|
||||
(match-one (update-in context [:node :children] (fn [[sym value path & clauses]] (list* sym value clauses)))))
|
||||
@@ -1,59 +0,0 @@
|
||||
(ns hooks.metabase.util.match-test
|
||||
(:require
|
||||
[clj-kondo.hooks-api :as hooks]
|
||||
[clj-kondo.impl.utils]
|
||||
[clojure.test :refer :all]
|
||||
[hooks.metabase.util.match :as sut]))
|
||||
|
||||
(defn- expand [src]
|
||||
(-> (sut/match-one {:node (hooks/parse-string (pr-str src))})
|
||||
:node
|
||||
hooks/sexpr))
|
||||
|
||||
(defn- lint [src]
|
||||
(binding [clj-kondo.impl.utils/*ctx* {:config {:linters {:metabase/match-odd-clauses {:level :warning}}}
|
||||
:ignores (atom nil)
|
||||
:findings (atom [])
|
||||
:namespaces (atom {})}]
|
||||
(expand src)
|
||||
@(:findings clj-kondo.impl.utils/*ctx*)))
|
||||
|
||||
(deftest odd-clauses-triggers-warning-test
|
||||
(is (=? [{:type :metabase/match-odd-clauses}]
|
||||
(lint '(metabase.util.match/match-one [] [a b c])))))
|
||||
|
||||
(deftest expansion-test
|
||||
(testing "simple expansion example"
|
||||
(is (= '(let [_ []
|
||||
&match (rand-nth ())
|
||||
&parents (rand-nth ())
|
||||
&recur (rand-nth ())
|
||||
_ &match
|
||||
_ &parents
|
||||
_ &recur
|
||||
a (rand-nth ())
|
||||
b (rand-nth ())
|
||||
c (rand-nth ())]
|
||||
(clojure.core/or (+ a b) (rand-nth ())))
|
||||
(expand '(match-one [] [a b c] (+ a b))))))
|
||||
(testing "complicated expansion example that uses most match-one macro features, and also binding deduplication by the rewriter"
|
||||
(is (= '(let [_ []
|
||||
&match (rand-nth ())
|
||||
&parents (rand-nth ())
|
||||
&recur (rand-nth ())
|
||||
_ &match
|
||||
_ &parents
|
||||
_ &recur
|
||||
val1 (rand-nth ())
|
||||
a (rand-nth ())
|
||||
_ #{maybe important}
|
||||
rst (rand-nth ())
|
||||
x (rand-nth ())
|
||||
_ (= x 3)
|
||||
b (rand-nth ())]
|
||||
(clojure.core/or (+ a val1) (rand-nth ())))
|
||||
(expand '(match-one []
|
||||
(:or [{:key1 val1, :key2 &truthy, :key3 _} _ a #{maybe important} & rst]
|
||||
(:and (x :guard (= x 3))
|
||||
[a b]))
|
||||
(+ a val1)))))))
|
||||
@@ -1,28 +0,0 @@
|
||||
# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json
|
||||
reviews:
|
||||
high_level_summary: false
|
||||
review_status: false
|
||||
|
||||
auto_review:
|
||||
enabled: false
|
||||
drafts: true
|
||||
labels:
|
||||
- "coderabbit"
|
||||
|
||||
path_filters:
|
||||
- "!**/*.snap"
|
||||
- "!**/*.po"
|
||||
- "!**/bun.lock"
|
||||
- "!**/package-lock.json"
|
||||
|
||||
tools:
|
||||
eslint:
|
||||
enabled: false
|
||||
|
||||
knowledge_base:
|
||||
code_guidelines:
|
||||
filePatterns:
|
||||
- ".claude/skills/clojure-review/SKILL.md"
|
||||
- ".claude/skills/typescript-review/SKILL.md"
|
||||
- ".claude/skills/e2e-test-review/SKILL.md"
|
||||
- ".claude/skills/docs-review/SKILL.md"
|
||||
@@ -1,32 +0,0 @@
|
||||
// dependency-cruiser config for the actual frontend import graph (what's
|
||||
// imported, vs the *allowed* edges in module-boundaries.mjs). Consumed by
|
||||
// .github/scripts/create-test-plan.ts.
|
||||
//
|
||||
// bunx depcruise frontend/src enterprise/frontend/src \
|
||||
// --config .dependency-cruiser.cjs --output-type json > dependency-graph.json
|
||||
|
||||
/** @type {import('dependency-cruiser').IConfiguration} */
|
||||
module.exports = {
|
||||
options: {
|
||||
// Keep only edges between our own source files (drops node_modules etc.),
|
||||
// which keeps the JSON small.
|
||||
includeOnly: "^(frontend/src|enterprise/frontend/src)/",
|
||||
|
||||
doNotFollow: { path: "node_modules" },
|
||||
|
||||
// Resolve the `"*": ["./frontend/src/*", ...]` aliases so bare imports like
|
||||
// `metabase/lib/foo` map back to a real file.
|
||||
tsConfig: { fileName: "tsconfig.json" },
|
||||
|
||||
// Keep type-only imports: a type-only dependent still breaks on a type change.
|
||||
tsPreCompilationDeps: true,
|
||||
|
||||
moduleSystems: ["es6", "cjs", "tsd"],
|
||||
|
||||
enhancedResolveOptions: {
|
||||
exportsFields: ["exports"],
|
||||
conditionNames: ["import", "require", "node", "default", "types"],
|
||||
extensions: [".js", ".jsx", ".ts", ".tsx", ".json"],
|
||||
},
|
||||
},
|
||||
};
|
||||
@@ -1,4 +1,4 @@
|
||||
FROM mcr.microsoft.com/vscode/devcontainers/java:25
|
||||
FROM mcr.microsoft.com/vscode/devcontainers/java:21
|
||||
|
||||
# Set up nodesource, install node, bun, fontconfig for static viz, rlwrap for dev ergonomics
|
||||
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
.git
|
||||
.babel_cache
|
||||
target
|
||||
target/*
|
||||
.circleci
|
||||
.cpcache
|
||||
.devcontainer
|
||||
@@ -16,7 +15,6 @@ test_config/*
|
||||
test_modules/*
|
||||
test_resources/*
|
||||
node_modules
|
||||
**/node_modules
|
||||
**metabase.jar
|
||||
|
||||
*.db
|
||||
|
||||
6
.gitattributes
vendored
6
.gitattributes
vendored
@@ -7,9 +7,3 @@
|
||||
#
|
||||
# TODO (Bryan 12/12/25) -- reenable this when we can completely verify correctness
|
||||
# resources/migrations/*_update_migrations.yaml merge=yaml-migrations
|
||||
|
||||
# Generated API documentation/specs — regenerated by the "Auto-update documentation for master"
|
||||
# CI automation, not edited by hand. Mark generated so tools/humans don't review or diff them as source.
|
||||
docs/api.json linguist-generated
|
||||
docs/api.html linguist-generated
|
||||
resources/openapi/openapi.json linguist-generated
|
||||
|
||||
20
.github/CODEOWNERS
vendored
20
.github/CODEOWNERS
vendored
@@ -12,27 +12,13 @@ enterprise/backend/src/metabase_enterprise/dependencies @metabase/graphy
|
||||
enterprise/frontend/src/metabase-enterprise/dependencies @metabase/graphy
|
||||
|
||||
# @metabase/embedding
|
||||
|
||||
frontend/src/metabase/embedding @metabase/embedding
|
||||
enterprise/frontend/src/embedding-sdk-package @metabase/embedding
|
||||
enterprise/frontend/src/embedding-sdk-bundle @metabase/embedding
|
||||
enterprise/frontend/src/embedding-sdk-shared @metabase/embedding
|
||||
frontend/src/metabase/embedding-sdk @metabase/embedding
|
||||
frontend/src/embedding-sdk-bundle @metabase/embedding
|
||||
frontend/src/embedding-sdk-shared @metabase/embedding
|
||||
frontend/src/metabase/public/containers/PublicOrEmbeddedQuestion @metabase/embedding
|
||||
frontend/src/metabase/public/containers/PublicOrEmbeddedDashboard @metabase/embedding
|
||||
|
||||
enterprise/frontend/src/embedding @metabase/embedding
|
||||
enterprise/frontend/src/embedding-sdk-ee @metabase/embedding
|
||||
enterprise/frontend/src/embedding-sdk-package @metabase/embedding
|
||||
|
||||
enterprise/frontend/src/metabase-enterprise/embedding @metabase/embedding
|
||||
enterprise/frontend/src/metabase-enterprise/embedding_iframe_sdk @metabase/embedding
|
||||
enterprise/frontend/src/metabase-enterprise/embedding_iframe_sdk_setup @metabase/embedding
|
||||
enterprise/frontend/src/metabase-enterprise/embedding-sdk @metabase/embedding
|
||||
|
||||
e2e/test-component @metabase/embedding
|
||||
e2e/test/scenarios/embedding @metabase/embedding
|
||||
e2e/embedding-sdk-host-apps @metabase/embedding
|
||||
|
||||
# @metabase/tech-writers
|
||||
docs @metabase/tech-writers
|
||||
docs/developers-guide # exclude from tech-writer ownership
|
||||
|
||||
2
.github/PULL_REQUEST_TEMPLATE.md
vendored
2
.github/PULL_REQUEST_TEMPLATE.md
vendored
@@ -1,5 +1,5 @@
|
||||
> [!IMPORTANT]
|
||||
> For those employed by Metabase: if you are fixing a P0 or P1 issue, add a `backport` label to this PR. No action is needed otherwise. Refer to the [Backporting](https://app.notion.com/p/metabase/Branching-Strategy-and-Backports-6eb577d5f61142aa960a626d6bbdfeb3?source=copy_link#e71f6e7de2f945d99fbb2d6b764e6f36) section in the Metabase Branching Strategy document for more details. If you're not employed by Metabase, this section does not apply to you, and the label will be taken care of by your reviewer.
|
||||
> For those employed by Metabase: if you are merging into master, please add either a `backport` or a `no-backport` label to this PR. You will not be able to merge until you do this step. Refer to the section [Do I need to backport this PR?](https://www.notion.so/metabase/Metabase-Branching-Strategy-6eb577d5f61142aa960a626d6bbdfeb3?pvs=4#89f80d6f17714a0198aeb66c0efd1b71) in the Metabase Branching Strategy document for more details. If you're not employed by Metabase, this section does not apply to you, and the label will be taken care of by your reviewer.
|
||||
|
||||
> **Warning**
|
||||
>
|
||||
|
||||
131
.github/actions/create-test-plan/action.yml
vendored
131
.github/actions/create-test-plan/action.yml
vendored
@@ -1,131 +0,0 @@
|
||||
name: Create test plan
|
||||
description: Cruise the frontend import graph and compute the affected-tests plan.
|
||||
|
||||
inputs:
|
||||
changed-files:
|
||||
description: All changed files (CSV).
|
||||
required: true
|
||||
fe-changed-files:
|
||||
description: Changed frontend_sources files (CSV).
|
||||
required: true
|
||||
be-changed-files:
|
||||
description: Changed backend_sources files (CSV).
|
||||
required: true
|
||||
unit-infra-touched:
|
||||
description: Whether FE unit infra changed.
|
||||
required: true
|
||||
loki-infra-touched:
|
||||
description: Whether Loki infra changed.
|
||||
required: true
|
||||
e2e-infra-touched:
|
||||
description: Whether e2e harness/support code changed.
|
||||
required: true
|
||||
shared-sources-touched:
|
||||
description: Whether cljc/cljs/package.json changed.
|
||||
required: true
|
||||
|
||||
outputs:
|
||||
stats:
|
||||
description: FE Affected Tests stats row (JSON).
|
||||
value: ${{ steps.compute.outputs.stats }}
|
||||
fe_unit_specs_to_run:
|
||||
description: Unit specs to run (JSON array).
|
||||
value: ${{ steps.compute.outputs.fe_unit_specs_to_run }}
|
||||
loki_stories_to_run:
|
||||
description: Loki stories to run (JSON array).
|
||||
value: ${{ steps.compute.outputs.loki_stories_to_run }}
|
||||
e2e_specs_to_run:
|
||||
description: E2E specs to run (JSON array).
|
||||
value: ${{ steps.compute.outputs.e2e_specs_to_run }}
|
||||
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Prepare Node + Bun
|
||||
uses: ./.github/actions/prepare-node-bun
|
||||
- name: Resolve closest-ancestor e2e coverage manifest
|
||||
id: manifest
|
||||
uses: actions/github-script@v9
|
||||
with:
|
||||
script: | # js
|
||||
const { pathToFileURL } = require("url");
|
||||
const { resolveCoverageManifest } = await import(
|
||||
pathToFileURL(
|
||||
`${process.env.GITHUB_WORKSPACE}/.github/scripts/resolve-coverage-manifest.mjs`,
|
||||
).href,
|
||||
);
|
||||
const result = await resolveCoverageManifest({
|
||||
github,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
headSha: context.payload.pull_request?.head?.sha ?? context.sha,
|
||||
});
|
||||
core.setOutput("path", result?.path ?? "");
|
||||
- name: Build dependency graph
|
||||
shell: bash
|
||||
run: |
|
||||
# Quiet on success; surface output only if install fails.
|
||||
bun install --frozen-lockfile > /tmp/bun-install.log 2>&1 \
|
||||
|| { echo "bun install issues (continuing):"; tail -20 /tmp/bun-install.log; }
|
||||
bunx depcruise frontend/src enterprise/frontend/src \
|
||||
--config .dependency-cruiser.cjs \
|
||||
--output-type json > dependency-graph.json || true
|
||||
|
||||
- name: Create test plan
|
||||
id: compute
|
||||
shell: bash
|
||||
env:
|
||||
CHANGED_FILES: ${{ inputs.changed-files }}
|
||||
FE_CHANGED_FILES: ${{ inputs.fe-changed-files }}
|
||||
BE_CHANGED_FILES: ${{ inputs.be-changed-files }}
|
||||
UNIT_INFRA_TOUCHED: ${{ inputs.unit-infra-touched }}
|
||||
LOKI_INFRA_TOUCHED: ${{ inputs.loki-infra-touched }}
|
||||
E2E_INFRA_TOUCHED: ${{ inputs.e2e-infra-touched }}
|
||||
SHARED_SOURCES_TOUCHED: ${{ inputs.shared-sources-touched }}
|
||||
DEP_GRAPH_JSON: dependency-graph.json
|
||||
E2E_SPEC_MANIFEST: ${{ steps.manifest.outputs.path }}
|
||||
run: |
|
||||
# stdout is the plan JSON; the planner's own info logs go to stderr.
|
||||
bun .github/scripts/create-test-plan.ts > test-plan.json
|
||||
echo "stats=$(jq -c '.stats' test-plan.json)" >> "$GITHUB_OUTPUT"
|
||||
echo "fe_unit_specs_to_run=$(jq -c '.fe_unit_specs_to_run' test-plan.json)" >> "$GITHUB_OUTPUT"
|
||||
echo "loki_stories_to_run=$(jq -c '.loki_stories_to_run' test-plan.json)" >> "$GITHUB_OUTPUT"
|
||||
echo "e2e_specs_to_run=$(jq -c '.e2e_specs_to_run' test-plan.json)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Report test plan
|
||||
shell: bash
|
||||
run: |
|
||||
plan=test-plan.json
|
||||
row() { # name, selected, total
|
||||
if [ "$2" -eq "$3" ]; then echo "| $1 | all ($3) | $3 |"; else echo "| $1 | $2 | $3 |"; fi
|
||||
}
|
||||
{
|
||||
echo "## Affected test plan"
|
||||
echo
|
||||
echo "| Suite | Selected | Total |"
|
||||
echo "|---|---:|---:|"
|
||||
row "Unit" "$(jq '.stats.unit_specs_to_run_usage' "$plan")" "$(jq '.stats.unit_specs_all' "$plan")"
|
||||
row "Loki" "$(jq '.stats.loki_stories_to_run_usage' "$plan")" "$(jq '.stats.loki_stories_all' "$plan")"
|
||||
row "E2E" "$(jq '.stats.e2e_specs_to_run_usage' "$plan")" "$(jq '.stats.e2e_specs_all' "$plan")"
|
||||
echo
|
||||
echo "Changed: $(jq '.stats.fe_files_changed' "$plan") FE / $(jq '.stats.be_files_changed' "$plan") BE file(s) · $(jq '.stats.fe_modules_affected_usage' "$plan") FE module(s) affected"
|
||||
echo
|
||||
e2e_run=$(jq '.stats.e2e_specs_to_run_usage' "$plan")
|
||||
e2e_all=$(jq '.stats.e2e_specs_all' "$plan")
|
||||
if [ "$e2e_run" -eq "$e2e_all" ]; then
|
||||
echo "**E2E: full run** — no manifest, or a change the coverage manifest can't scope (backend, e2e infra, cljc/cljs)."
|
||||
else
|
||||
echo "<details><summary><b>E2E specs to run ($e2e_run)</b></summary>"
|
||||
echo
|
||||
jq -r '.e2e_specs_to_run[]' "$plan" | sed 's/^/- /'
|
||||
echo
|
||||
echo "</details>"
|
||||
fi
|
||||
echo
|
||||
echo "<details><summary>Full stats (rules vs usage graph)</summary>"
|
||||
echo
|
||||
echo '```json'
|
||||
jq '.stats' "$plan"
|
||||
echo '```'
|
||||
echo "</details>"
|
||||
} | tee -a "${GITHUB_STEP_SUMMARY:-/dev/null}"
|
||||
@@ -40,24 +40,10 @@ runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Authenticate to prevent API rate limit
|
||||
shell: bash
|
||||
env:
|
||||
DOCKER_USERNAME: ${{ inputs.username }}
|
||||
DOCKER_PASSWORD: ${{ inputs.password }}
|
||||
run: |
|
||||
for attempt in 1 2 3; do
|
||||
echo "Docker login attempt $attempt of 3..."
|
||||
if echo "$DOCKER_PASSWORD" | docker login --username "$DOCKER_USERNAME" --password-stdin; then
|
||||
echo "Docker login succeeded on attempt $attempt"
|
||||
break
|
||||
fi
|
||||
if [ $attempt -eq 3 ]; then
|
||||
echo "::error::Docker login failed after 3 attempts"
|
||||
exit 1
|
||||
fi
|
||||
echo "::warning::Docker login attempt $attempt failed, retrying in 15s..."
|
||||
sleep 15
|
||||
done
|
||||
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
|
||||
with:
|
||||
username: ${{ inputs.username }}
|
||||
password: ${{ inputs.password }}
|
||||
|
||||
- name: Start Containers
|
||||
shell: bash
|
||||
@@ -67,29 +53,16 @@ runs:
|
||||
|
||||
cd e2e/test/scenarios/
|
||||
|
||||
SERVICES=()
|
||||
[ "${{ inputs.postgres }}" = true ] && SERVICES+=(postgres-sample)
|
||||
[ "${{ inputs.mysql }}" = true ] && SERVICES+=(mysql-sample)
|
||||
[ "${{ inputs.mongo }}" = true ] && SERVICES+=(mongo-sample)
|
||||
[ "${{ inputs.maildev }}" = true ] && SERVICES+=(maildev maildev-ssl)
|
||||
[ "${{ inputs.openldap }}" = true ] && SERVICES+=(ldap)
|
||||
[ "${{ inputs.webhook }}" = true ] && SERVICES+=(webhook-tester)
|
||||
[ "${{ inputs.snowplow }}" = true ] && SERVICES+=(snowplow-micro)
|
||||
|
||||
for attempt in 1 2 3; do
|
||||
echo "Docker compose up attempt $attempt of 3..."
|
||||
if docker compose up "${SERVICES[@]}" -d; then
|
||||
echo "Docker compose up succeeded on attempt $attempt"
|
||||
break
|
||||
fi
|
||||
if [ $attempt -eq 3 ]; then
|
||||
echo "::error::Docker compose up failed after 3 attempts"
|
||||
exit 1
|
||||
fi
|
||||
echo "::warning::Docker compose up attempt $attempt failed, retrying in 30s..."
|
||||
docker compose down 2>/dev/null || true
|
||||
sleep 30
|
||||
done
|
||||
docker compose up \
|
||||
$([ ${{ inputs.postgres }} = true ] && echo "postgres-sample") \
|
||||
$([ ${{ inputs.mysql }} = true ] && echo "mysql-sample") \
|
||||
$([ ${{ inputs.mongo }} = true ] && echo "mongo-sample") \
|
||||
$([ ${{ inputs.maildev }} = true ] && echo "maildev") \
|
||||
$([ ${{ inputs.maildev }} = true ] && echo "maildev-ssl") \
|
||||
$([ ${{ inputs.openldap }} = true ] && echo "ldap") \
|
||||
$([ ${{ inputs.webhook }} = true ] && echo "webhook-tester") \
|
||||
$([ ${{ inputs.snowplow }} = true ] && echo "snowplow-micro") \
|
||||
-d
|
||||
|
||||
if ${{ inputs.maildev }}; then
|
||||
echo -e "${Y}checking maildev container..." &&
|
||||
|
||||
2
.github/actions/prepare-backend/action.yml
vendored
2
.github/actions/prepare-backend/action.yml
vendored
@@ -4,7 +4,7 @@ description: |
|
||||
inputs:
|
||||
java-version:
|
||||
required: true
|
||||
default: "25"
|
||||
default: "21"
|
||||
clojure-version:
|
||||
required: true
|
||||
default: "1.12.0.1488"
|
||||
|
||||
52
.github/actions/prepare-ci-conductor/action.yml
vendored
52
.github/actions/prepare-ci-conductor/action.yml
vendored
@@ -1,52 +0,0 @@
|
||||
name: Prepare CI Conductor
|
||||
description: >-
|
||||
Make the ci-conductor sources runnable in the workspace.
|
||||
Unless the checkout is headed for master, they are overlaid from master, so every
|
||||
branch runs the latest scripts while its workflow wiring stays branch-historical.
|
||||
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Detect bun
|
||||
id: bun
|
||||
shell: bash
|
||||
run: |
|
||||
if command -v bun >/dev/null; then
|
||||
echo "present=true" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "present=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Install bun
|
||||
if: steps.bun.outputs.present != 'true'
|
||||
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
|
||||
with:
|
||||
bun-version-file: "package.json"
|
||||
token: ${{ github.token }}
|
||||
|
||||
# Anything headed for master runs master's own sources, so a PR can test its own changes.
|
||||
- name: Fetch ci-conductor sources from master
|
||||
if: ${{ github.base_ref != 'master' && github.ref_name != 'master' }}
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
repository: metabase/metabase
|
||||
ref: master
|
||||
path: .ci-conductor-master
|
||||
persist-credentials: false
|
||||
sparse-checkout-cone-mode: false
|
||||
sparse-checkout: |
|
||||
release/ci-conductor
|
||||
e2e/support/ci_conductor.ts
|
||||
|
||||
# e2e/support/config.js and e2e/support/ci_conductor.ts resolve these by relative
|
||||
# path at bundle time, so the sources have to land where the branch expects them.
|
||||
- name: Overlay ci-conductor sources from master
|
||||
if: ${{ github.base_ref != 'master' && github.ref_name != 'master' }}
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
rm -rf release/ci-conductor
|
||||
mv .ci-conductor-master/release/ci-conductor release/ci-conductor
|
||||
mv .ci-conductor-master/e2e/support/ci_conductor.ts e2e/support/ci_conductor.ts
|
||||
rm -rf .ci-conductor-master
|
||||
echo "Overlaid ci-conductor sources from master."
|
||||
4
.github/actions/prepare-cypress/action.yml
vendored
4
.github/actions/prepare-cypress/action.yml
vendored
@@ -15,10 +15,6 @@ outputs:
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
# e2e/support/config.js imports the reporter, so it has to be in place before Cypress bundles.
|
||||
- name: Prepare CI Conductor
|
||||
uses: ./.github/actions/prepare-ci-conductor
|
||||
|
||||
- name: Install Chrome v144
|
||||
uses: browser-actions/setup-chrome@2e1d749697dd1612b833dba4a722266286fbefcd # v2.1.2
|
||||
with:
|
||||
|
||||
2
.github/actions/prepare-node-bun/action.yml
vendored
2
.github/actions/prepare-node-bun/action.yml
vendored
@@ -16,5 +16,5 @@ runs:
|
||||
- name: Install bun
|
||||
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
|
||||
with:
|
||||
bun-version-file: 'package.json'
|
||||
bun-version: "1.3.7"
|
||||
token: ${{ github.token }}
|
||||
|
||||
@@ -6,7 +6,7 @@ runs:
|
||||
steps:
|
||||
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
|
||||
with:
|
||||
bun-version-file: 'package.json'
|
||||
bun-version: "1.3.7"
|
||||
token: ${{ github.token }}
|
||||
|
||||
- name: Build release scripts
|
||||
|
||||
38
.github/actions/quarantine-gate/action.yml
vendored
38
.github/actions/quarantine-gate/action.yml
vendored
@@ -1,38 +0,0 @@
|
||||
name: CI Conductor quarantine gate
|
||||
description: >-
|
||||
Check this job's test failures against ci-conductor's quarantine list.
|
||||
Either enforces the quarantine or only prints a verdict (dry run).
|
||||
Fails closed when enforcing but the `CI_CONDUCTOR_BASE_URL` is missing or unreachable.
|
||||
|
||||
inputs:
|
||||
adapter:
|
||||
description: Which gate to run — one of `backend`, `frontend`, `e2e`.
|
||||
required: true
|
||||
test-suite:
|
||||
description: >-
|
||||
Suite label to gate (CI_CONDUCTOR_TEST_SUITE). MUST match the label this
|
||||
job reports under, or the gate checks the wrong quarantine list.
|
||||
required: true
|
||||
enforce-quarantine:
|
||||
description: >-
|
||||
Whether the gate actually gates the job. When "false", it only prints a verdict.
|
||||
required: false
|
||||
default: "false"
|
||||
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Prepare CI Conductor
|
||||
uses: ./.github/actions/prepare-ci-conductor
|
||||
- name: Check failures against the quarantine list
|
||||
shell: bash
|
||||
env:
|
||||
CI_CONDUCTOR_TEST_SUITE: ${{ inputs.test-suite }}
|
||||
ADAPTER: ${{ inputs.adapter }}
|
||||
QUARANTINE_DRY_RUN: ${{ inputs.enforce-quarantine != 'true' }}
|
||||
run: |
|
||||
case "$ADAPTER" in
|
||||
backend|frontend|e2e) ;;
|
||||
*) echo "::error::quarantine-gate: unknown adapter '$ADAPTER' (expected backend|frontend|e2e)"; exit 1 ;;
|
||||
esac
|
||||
bun "release/ci-conductor/src/check-${ADAPTER}-quarantine.ts"
|
||||
@@ -1,34 +0,0 @@
|
||||
name: Report failures to CI Conductor
|
||||
description: >-
|
||||
Report this job's test failures to ci-conductor. Best-effort; no-ops unless
|
||||
CI_CONDUCTOR_BASE_URL is set. Gate it at the call site with continue-on-error
|
||||
and an `if:` on the test step's outcome.
|
||||
|
||||
inputs:
|
||||
test-suite:
|
||||
description: Per-job identity label (CI_CONDUCTOR_TEST_SUITE); unique per job/leg.
|
||||
required: true
|
||||
adapter:
|
||||
description: Which reporter to run; resolves to report-<adapter>.ts. backend or frontend.
|
||||
required: true
|
||||
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Prepare CI Conductor
|
||||
uses: ./.github/actions/prepare-ci-conductor
|
||||
- name: Report failed tests to ci-conductor
|
||||
shell: bash
|
||||
env:
|
||||
REPO_ID: ${{ github.event.repository.id }}
|
||||
COMMIT_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
|
||||
TARGET_BRANCH: ${{ github.event.pull_request.base.ref || github.ref_name }}
|
||||
CI_CONDUCTOR_TEST_SUITE: ${{ inputs.test-suite }}
|
||||
ADAPTER: ${{ inputs.adapter }}
|
||||
# Keep the path constrained to known entrypoints.
|
||||
run: |
|
||||
case "$ADAPTER" in
|
||||
backend|frontend) ;;
|
||||
*) echo "::error::Unknown ci-conductor adapter '$ADAPTER'"; exit 1 ;;
|
||||
esac
|
||||
bun "release/ci-conductor/src/report-${ADAPTER}.ts"
|
||||
66
.github/actions/resolve-job-id/action.yml
vendored
66
.github/actions/resolve-job-id/action.yml
vendored
@@ -1,66 +0,0 @@
|
||||
name: Resolve current job id
|
||||
description: >
|
||||
Resolve this job's GitHub workflow_jobs.id via the Actions API and export it
|
||||
as JOB_ID (and JOB_HTML_URL) to $GITHUB_ENV, so any later step can read the
|
||||
numeric job id without making its own API call. Best-effort: warns and
|
||||
proceeds on failure so it can never break the run.
|
||||
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Resolve JOB_ID
|
||||
shell: bash
|
||||
env:
|
||||
# Use the built-in per-job token, not a PAT: this lookup is best-effort
|
||||
# (a miss just leaves JOB_ID unset, never breaks the run) and other jobs
|
||||
# depend on the automation PAT's rate-limit budget — we don't want ~70
|
||||
# e2e shards drawing it down. Worst case under heavy CI load is the
|
||||
# occasional unresolved job_id, which is acceptable. DEV-2117.
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
RUNNER_NAME: ${{ runner.name }}
|
||||
run: |
|
||||
# GitHub doesn't expose the current job's numeric id, so we look it up
|
||||
# from the Actions API. A runner runs exactly one job at a time, so the
|
||||
# in-progress job with this runner_name within (run_id, run_attempt) is
|
||||
# unique. We require status == in_progress and no completed_at so a
|
||||
# reused runner can't match a job it already finished this attempt, with
|
||||
# max_by(.started_at) as a final tiebreaker.
|
||||
#
|
||||
# $GITHUB_RUN_ID is the whole CI run, whose total job count exceeds the
|
||||
# 100-per-page cap (~185 jobs), so this shard's job is frequently NOT on
|
||||
# the first page. --paginate --slurp pulls every page and slurps them
|
||||
# into one array so the filter sees all jobs, not just the first 100 —
|
||||
# without it the lookup returned null whenever this job sorted past
|
||||
# position 100 (DEV-2117).
|
||||
#
|
||||
# gh rejects `--slurp` together with `--jq` ("the --slurp option is not
|
||||
# supported with --jq"), so we slurp with gh and run the filter through a
|
||||
# standalone `jq`. The earlier `--slurp --jq` form errored on every
|
||||
# attempt — swallowed by 2>/dev/null — leaving JOB_ID permanently unset.
|
||||
set +e
|
||||
api_url="/repos/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID/attempts/$GITHUB_RUN_ATTEMPT/jobs?per_page=100"
|
||||
jq_filter="[.[].jobs[] | select(.runner_name == \"$RUNNER_NAME\" and .status == \"in_progress\" and .completed_at == null)] | max_by(.started_at) | \"\(.id) \(.html_url)\""
|
||||
|
||||
# The Actions jobs API is eventually consistent: after a runner picks
|
||||
# up a job, it can take a few seconds for runner_name to appear in the
|
||||
# response. Retry briefly to absorb that.
|
||||
out=""
|
||||
for attempt in 1 2 3 4 5; do
|
||||
out=$(gh api --paginate --slurp "$api_url" 2>/dev/null | jq -r "$jq_filter" 2>/dev/null)
|
||||
if [ -n "$out" ] && [ "$out" != "null null" ]; then
|
||||
break
|
||||
fi
|
||||
out=""
|
||||
echo "Attempt $attempt: job not yet visible in Actions API; retrying..."
|
||||
sleep 3
|
||||
done
|
||||
|
||||
if [ -n "$out" ]; then
|
||||
read -r job_id job_url <<<"$out"
|
||||
echo "JOB_ID=$job_id" >> "$GITHUB_ENV"
|
||||
echo "JOB_HTML_URL=$job_url" >> "$GITHUB_ENV"
|
||||
echo "Resolved JOB_ID=$job_id (runner=$RUNNER_NAME)"
|
||||
else
|
||||
echo "::warning::Could not resolve current job id after 5 attempts (runner=$RUNNER_NAME)"
|
||||
fi
|
||||
exit 0
|
||||
131
.github/actions/sdk-bundle-size-check/action.yml
vendored
131
.github/actions/sdk-bundle-size-check/action.yml
vendored
@@ -1,17 +1,15 @@
|
||||
name: Embedding SDK Bundle Size Check
|
||||
description: |
|
||||
Measures app + embedding SDK bundle sizes from the current and base-ref
|
||||
uberjars (reusing the builds that already ran) plus their bundle-stats
|
||||
artifacts, and flags changes to the chunked SDK's reachable, gzipped size
|
||||
beyond the threshold. Skips when the base-ref artifacts aren't available
|
||||
rather than rebuilding from source (which would compare across pipelines).
|
||||
Downloads the current branch's uberjar and builds the base ref's
|
||||
embedding-sdk.js from source, compares their sizes,
|
||||
and sets output if the size change exceeds the threshold.
|
||||
|
||||
inputs:
|
||||
edition:
|
||||
description: 'Metabase edition (ee or oss)'
|
||||
required: true
|
||||
base-ref-commit:
|
||||
description: 'Commit SHA of the base ref.'
|
||||
description: 'Commit SHA of the base ref (e.g. merge base).'
|
||||
required: true
|
||||
threshold-percent:
|
||||
description: 'Percent threshold for bundle size change'
|
||||
@@ -28,73 +26,86 @@ outputs:
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Prepare backend
|
||||
uses: ./.github/actions/prepare-backend
|
||||
- name: Prepare frontend
|
||||
uses: ./.github/actions/prepare-frontend
|
||||
- name: Download current uberjar
|
||||
uses: ./.github/actions/fetch-artifact
|
||||
with:
|
||||
commit: ${{ github.event.pull_request.head.sha || github.sha }}
|
||||
edition: ${{ inputs.edition }}
|
||||
extract-path: temp/current-uberjar
|
||||
- name: Download base-ref uberjar
|
||||
id: base-jar
|
||||
continue-on-error: true
|
||||
uses: ./.github/actions/fetch-artifact
|
||||
- name: Extract embedding-sdk bundles from current uberjar
|
||||
shell: bash
|
||||
run: |
|
||||
# Try new layout first (app/embedding-sdk/legacy/), fall back to old layout (app/embedding-sdk.js)
|
||||
unzip -j temp/current-uberjar/metabase.jar \
|
||||
"frontend_client/app/embedding-sdk/legacy/embedding-sdk.js" \
|
||||
-d temp/current-bundle 2>/dev/null || \
|
||||
unzip -j temp/current-uberjar/metabase.jar \
|
||||
"frontend_client/app/embedding-sdk.js" \
|
||||
-d temp/current-bundle
|
||||
|
||||
# Extract chunks directory if it exists (new layout only)
|
||||
mkdir -p temp/current-chunks
|
||||
unzip -j temp/current-uberjar/metabase.jar \
|
||||
"frontend_client/app/embedding-sdk/chunks/*" \
|
||||
-d temp/current-chunks 2>/dev/null || true
|
||||
- name: Checkout base ref
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
commit: ${{ inputs.base-ref-commit }}
|
||||
edition: ${{ inputs.edition }}
|
||||
extract-path: temp/base-uberjar
|
||||
- name: Measure and compare bundle sizes
|
||||
ref: ${{ inputs.base-ref-commit }}
|
||||
path: temp/base-ref-src
|
||||
- name: Build base-ref embedding-sdk.js
|
||||
shell: bash
|
||||
run: |
|
||||
cd temp/base-ref-src
|
||||
bun install
|
||||
MB_EDITION=${{ inputs.edition }} bun run build-release
|
||||
- name: Compare sizes and set output
|
||||
id: compare
|
||||
shell: bash
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
EDITION: ${{ inputs.edition }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
|
||||
BASE_SHA: ${{ inputs.base-ref-commit }}
|
||||
THRESHOLD: ${{ inputs.threshold-percent }}
|
||||
MEASURE_BUNDLES: app,embedding-sdk-legacy,embedding-sdk-chunked
|
||||
run: |
|
||||
set -uo pipefail
|
||||
# --- Legacy (monolithic) bundle comparison ---
|
||||
current_size=$(wc -c < temp/current-bundle/embedding-sdk.js) || { echo "::error::Failed to read current bundle"; exit 1; }
|
||||
|
||||
skip() {
|
||||
echo "::notice::Embedding SDK bundle-size check skipped: $1"
|
||||
echo "status=stable" >> "$GITHUB_OUTPUT"
|
||||
echo "size_change_percent=0" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
}
|
||||
# Base ref: try new layout first, fall back to old
|
||||
if [ -f "temp/base-ref-src/resources/frontend_client/app/embedding-sdk/legacy/embedding-sdk.js" ]; then
|
||||
base_ref_size=$(wc -c < "temp/base-ref-src/resources/frontend_client/app/embedding-sdk/legacy/embedding-sdk.js")
|
||||
else
|
||||
base_ref_size=$(wc -c < "temp/base-ref-src/resources/frontend_client/app/embedding-sdk.js")
|
||||
fi
|
||||
[ "$current_size" -gt 0 ] && [ "$base_ref_size" -gt 0 ] || { echo "::error::Bundle file is empty"; exit 1; }
|
||||
|
||||
# The base ref needs an already-built uberjar. Rebuilding it from source
|
||||
# would compare across two build pipelines (the bug this check used to
|
||||
# have), so skip instead when it — or its stats — isn't available.
|
||||
[ -f temp/base-uberjar/metabase.jar ] || skip "base-ref uberjar not found for ${BASE_SHA}"
|
||||
diff=$((current_size - base_ref_size))
|
||||
percent=$(( diff * 100 / base_ref_size ))
|
||||
|
||||
extract_js() { # $1=jar $2=dest-root
|
||||
mkdir -p "$2/resources"
|
||||
unzip -o "$1" \
|
||||
"frontend_client/app/dist/*" \
|
||||
"frontend_client/app/embed.js" \
|
||||
"frontend_client/app/embedding-sdk/*" \
|
||||
-d "$2/resources" >/dev/null || true
|
||||
}
|
||||
echo "=== Legacy (monolithic) bundle ==="
|
||||
echo "Current size: $current_size bytes"
|
||||
echo "Base ref size: $base_ref_size bytes"
|
||||
echo "Difference: $diff bytes ($percent%)"
|
||||
|
||||
fetch_stats() { # $1=sha $2=dest-dir
|
||||
local name="metabase-bundle-stats-${EDITION}-$1"
|
||||
local id
|
||||
id=$(gh api "/repos/${GITHUB_REPOSITORY}/actions/artifacts?name=${name}" --jq '.artifacts[0].id // empty' 2>/dev/null)
|
||||
[ -n "${id}" ] || return 1
|
||||
mkdir -p "$2"
|
||||
gh api "/repos/${GITHUB_REPOSITORY}/actions/artifacts/${id}/zip" > "$2.zip" 2>/dev/null || return 1
|
||||
unzip -o "$2.zip" -d "$2" >/dev/null || return 1
|
||||
}
|
||||
threshold="${{ inputs.threshold-percent }}"
|
||||
if [ "$percent" -gt "$threshold" ]; then
|
||||
status="increased"
|
||||
elif [ "$percent" -lt "$(( -threshold ))" ]; then
|
||||
status="decreased"
|
||||
else
|
||||
status="stable"
|
||||
fi
|
||||
|
||||
extract_js temp/current-uberjar/metabase.jar temp/current
|
||||
extract_js temp/base-uberjar/metabase.jar temp/base
|
||||
echo "status=$status" >> $GITHUB_OUTPUT
|
||||
echo "size_change_percent=$percent" >> $GITHUB_OUTPUT
|
||||
|
||||
fetch_stats "${HEAD_SHA}" temp/current/artifacts || skip "bundle stats not found for current ${HEAD_SHA}"
|
||||
fetch_stats "${BASE_SHA}" temp/base/artifacts || skip "bundle stats not found for base ${BASE_SHA}"
|
||||
|
||||
MEASURE_ROOT=temp/current node ./.github/scripts/measure-bundle-sizes.js > temp/current-sizes.json \
|
||||
|| { echo "::error::Failed to measure current bundle sizes"; exit 1; }
|
||||
MEASURE_ROOT=temp/base node ./.github/scripts/measure-bundle-sizes.js > temp/base-sizes.json \
|
||||
|| { echo "::error::Failed to measure base bundle sizes"; exit 1; }
|
||||
|
||||
node ./.github/actions/sdk-bundle-size-check/compare.js temp/current-sizes.json temp/base-sizes.json
|
||||
# --- Chunked bundle (informational) ---
|
||||
if ls temp/current-chunks/*.js 1>/dev/null 2>&1; then
|
||||
chunks_size=$(cat temp/current-chunks/*.js | wc -c)
|
||||
echo ""
|
||||
echo "=== Chunked bundle (bootstrap + chunks) ==="
|
||||
echo "Total chunked size: $chunks_size bytes"
|
||||
echo "chunks_size=$chunks_size" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo ""
|
||||
echo "No chunked bundle found (expected on master)"
|
||||
fi
|
||||
|
||||
110
.github/actions/sdk-bundle-size-check/compare.js
vendored
110
.github/actions/sdk-bundle-size-check/compare.js
vendored
@@ -1,110 +0,0 @@
|
||||
/* eslint-disable import/no-commonjs */
|
||||
// Diffs the bundle sizes produced by measure-bundle-sizes.js for the current
|
||||
// build and the base ref. Every (bundle, kind) is compared strictly against the
|
||||
// same (bundle, kind) — never one bundle type against another — using gzipped
|
||||
// (served) bytes. The gate is driven by the chunked SDK's reachable total, which
|
||||
// is what the SDK actually delivers.
|
||||
//
|
||||
// compareBundles is pure (data in, data out) so it can be unit-tested; main()
|
||||
// is the thin I/O wrapper that reads the size files, prints the report and sets
|
||||
// the step outputs / exit code.
|
||||
const fs = require("fs");
|
||||
|
||||
const pick = (rows, bundle, kind) => rows.find(row => row.bundle === bundle && row.kind === kind);
|
||||
const keyOf = row => `${row.bundle}/${row.kind}`;
|
||||
const mb = bytes => (bytes / 1024 / 1024).toFixed(2);
|
||||
const percentOf = (currentBytes, baseBytes) =>
|
||||
baseBytes ? Math.trunc(((currentBytes - baseBytes) * 100) / baseBytes) : 0;
|
||||
|
||||
/**
|
||||
* Compare two measurement sets and decide the gate. Returns a human-readable
|
||||
* `report` (always), and exactly one of:
|
||||
* - `gate` { status, percent }: the chunked SDK total moved by `percent`.
|
||||
* - `skip` reason: the gate is non-actionable (report stable, no comment).
|
||||
* - `error` reason: the gate inputs are missing (hard failure).
|
||||
*/
|
||||
function compareBundles({ current, base, threshold }) {
|
||||
const report = ["=== Bundle sizes (gzipped, current vs base) ==="];
|
||||
const keys = [...new Set([...current, ...base].map(keyOf))].sort();
|
||||
for (const key of keys) {
|
||||
const [bundle, kind] = key.split("/");
|
||||
const currentRow = pick(current, bundle, kind);
|
||||
const baseRow = pick(base, bundle, kind);
|
||||
if (!currentRow || !baseRow) {
|
||||
report.push(`${key}: present only in ${currentRow ? "current" : "base"} build`);
|
||||
continue;
|
||||
}
|
||||
const diff = currentRow.gzipBytes - baseRow.gzipBytes;
|
||||
const percent = percentOf(currentRow.gzipBytes, baseRow.gzipBytes);
|
||||
report.push(`${key}: ${mb(currentRow.gzipBytes)}MB vs ${mb(baseRow.gzipBytes)}MB (${percent}%, ${diff} bytes)`);
|
||||
}
|
||||
|
||||
// Gate on the chunked SDK's reachable total (gzipped) — what the SDK ships.
|
||||
const gateCurrent = pick(current, "embedding-sdk-chunked", "total");
|
||||
const gateBase = pick(base, "embedding-sdk-chunked", "total");
|
||||
if (!gateCurrent || !gateBase || !gateBase.gzipBytes) {
|
||||
return { report, error: "Could not find embedding-sdk-chunked total (gzip) in both builds" };
|
||||
}
|
||||
|
||||
// Both sides must measure "total" the same way. A base ref built before the
|
||||
// reachableAssets enrichment collapses its total to the initial set, so
|
||||
// comparing it against the current reachable total reports a phantom jump
|
||||
// (~30%). Skip until the base ref also carries reachable stats (i.e. once
|
||||
// this change is on the base).
|
||||
if (Boolean(gateCurrent.reachable) !== Boolean(gateBase.reachable)) {
|
||||
return {
|
||||
report,
|
||||
skip:
|
||||
`chunked "total" measured differently on each side ` +
|
||||
`(current reachable=${Boolean(gateCurrent.reachable)}, base reachable=${Boolean(gateBase.reachable)}); ` +
|
||||
`the base ref predates the reachable-chunk stats. Resolves once this change is on the base ref.`,
|
||||
};
|
||||
}
|
||||
|
||||
const percent = percentOf(gateCurrent.gzipBytes, gateBase.gzipBytes);
|
||||
const status = percent > threshold ? "increased" : percent < -threshold ? "decreased" : "stable";
|
||||
return { report, gate: { status, percent } };
|
||||
}
|
||||
|
||||
module.exports = { compareBundles };
|
||||
|
||||
const setOutput = (name, value) => {
|
||||
if (process.env.GITHUB_OUTPUT) {
|
||||
fs.appendFileSync(process.env.GITHUB_OUTPUT, `${name}=${value}\n`);
|
||||
}
|
||||
};
|
||||
|
||||
function main() {
|
||||
const [, , currentPath, basePath] = process.argv;
|
||||
const threshold = Number(process.env.THRESHOLD ?? 2);
|
||||
|
||||
const current = JSON.parse(fs.readFileSync(currentPath, "utf8"));
|
||||
const base = JSON.parse(fs.readFileSync(basePath, "utf8"));
|
||||
|
||||
const { report, gate, skip, error } = compareBundles({ current, base, threshold });
|
||||
console.log(report.join("\n"));
|
||||
|
||||
if (error) {
|
||||
console.error(`::error::${error}`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
if (skip) {
|
||||
// Treat the gate as non-actionable: report stable so no regression comment
|
||||
// is posted, mirroring the shell skip() for missing artifacts.
|
||||
console.log(`::notice::Embedding SDK bundle-size gate skipped: ${skip}`);
|
||||
setOutput("status", "stable");
|
||||
setOutput("size_change_percent", "0");
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
console.log("");
|
||||
console.log(`Gate: embedding-sdk-chunked total ${gate.percent}% (threshold ${threshold}%) → ${gate.status}`);
|
||||
|
||||
setOutput("status", gate.status);
|
||||
setOutput("size_change_percent", gate.percent);
|
||||
}
|
||||
|
||||
if (require.main === module) {
|
||||
main();
|
||||
}
|
||||
@@ -1,65 +0,0 @@
|
||||
const { compareBundles } = require("./compare");
|
||||
|
||||
const total = (gzipBytes, over = {}) => ({
|
||||
bundle: "embedding-sdk-chunked",
|
||||
kind: "total",
|
||||
gzipBytes,
|
||||
reachable: true,
|
||||
...over,
|
||||
});
|
||||
|
||||
describe("compareBundles", () => {
|
||||
it("gates on the chunked SDK total and flags an increase past the threshold", () => {
|
||||
const { gate, report } = compareBundles({
|
||||
current: [total(110)],
|
||||
base: [total(100)],
|
||||
threshold: 2,
|
||||
});
|
||||
|
||||
expect(gate).toEqual({ status: "increased", percent: 10 });
|
||||
expect(report[0]).toContain("current vs base");
|
||||
});
|
||||
|
||||
it("flags a decrease and stays stable within the threshold", () => {
|
||||
expect(compareBundles({ current: [total(90)], base: [total(100)], threshold: 2 }).gate).toEqual({
|
||||
status: "decreased",
|
||||
percent: -10,
|
||||
});
|
||||
expect(compareBundles({ current: [total(101)], base: [total(100)], threshold: 2 }).gate).toEqual({
|
||||
status: "stable",
|
||||
percent: 1,
|
||||
});
|
||||
});
|
||||
|
||||
it("skips when the two sides measured total differently (reachable mismatch)", () => {
|
||||
const result = compareBundles({
|
||||
current: [total(130, { reachable: true })],
|
||||
base: [total(100, { reachable: false })],
|
||||
threshold: 2,
|
||||
});
|
||||
|
||||
expect(result.gate).toBeUndefined();
|
||||
expect(result.skip).toContain("predates the reachable-chunk stats");
|
||||
});
|
||||
|
||||
it("errors when the gate bundle is missing on either side", () => {
|
||||
const result = compareBundles({
|
||||
current: [{ bundle: "app", kind: "total", gzipBytes: 1 }],
|
||||
base: [total(100)],
|
||||
threshold: 2,
|
||||
});
|
||||
|
||||
expect(result.gate).toBeUndefined();
|
||||
expect(result.error).toContain("embedding-sdk-chunked total");
|
||||
});
|
||||
|
||||
it("reports a bundle/kind present on only one side without erroring", () => {
|
||||
const { report } = compareBundles({
|
||||
current: [total(100), { bundle: "app", kind: "total", gzipBytes: 50 }],
|
||||
base: [total(100)],
|
||||
threshold: 2,
|
||||
});
|
||||
|
||||
expect(report.some(line => line.includes("present only in current build"))).toBe(true);
|
||||
});
|
||||
});
|
||||
49
.github/actions/test-driver/action.yml
vendored
49
.github/actions/test-driver/action.yml
vendored
@@ -14,15 +14,8 @@ inputs:
|
||||
description: Suffix to append to the test logs artifact name to avoid conflicts in matrix jobs
|
||||
edition:
|
||||
required: false
|
||||
default: "ee"
|
||||
default: 'ee'
|
||||
description: Edition to test (oss or ee)
|
||||
quarantine-engine:
|
||||
required: false
|
||||
driver-status:
|
||||
required: false
|
||||
default: ""
|
||||
description: >-
|
||||
Configured status for this driver ("info", "skip", "required", or empty), as reported by `mage driver-decisions`.
|
||||
|
||||
outputs:
|
||||
test-outcome:
|
||||
@@ -34,10 +27,6 @@ runs:
|
||||
steps:
|
||||
- name: Prepare back-end environment
|
||||
uses: ./.github/actions/prepare-backend
|
||||
- name: Resolve current job id
|
||||
# Resolve JOB_ID early (before the long test step) so the ci-conductor
|
||||
# reporter below can link failures to this exact job. Best-effort.
|
||||
uses: ./.github/actions/resolve-job-id
|
||||
- name: Prepare front-end environment
|
||||
if: ${{ inputs.build-static-viz == 'true' }}
|
||||
uses: ./.github/actions/prepare-frontend
|
||||
@@ -46,25 +35,15 @@ runs:
|
||||
run: bun run build-static-viz
|
||||
shell: bash
|
||||
env:
|
||||
MB_EDITION: ${{ inputs.edition }}
|
||||
|
||||
MB_EDITION: ee
|
||||
- name: Test database driver
|
||||
run: clojure -X:dev:ci:${{ inputs.edition }}:${{ inputs.edition }}-dev:drivers:drivers-dev:test ${{ inputs.test-args }}
|
||||
shell: bash
|
||||
id: run-test
|
||||
# Keep this step continue-on-error because the quarantine logic controls whether it ultimately fails.
|
||||
# Driver "info" status can affect this logic. See below.
|
||||
# Trunk Quarantine controls the final state of the run in the CI!
|
||||
# Even if this step fails we let it pass, and then `trunk-io/analytics-uploader` determines
|
||||
# whether to fail or to pass the whole job, based on the quarantine list.
|
||||
continue-on-error: true
|
||||
|
||||
- name: Report failed tests to ci-conductor
|
||||
if: ${{ !cancelled() && steps.run-test.outcome == 'failure' }}
|
||||
# This step should NEVER fail the job!
|
||||
continue-on-error: true
|
||||
uses: ./.github/actions/report-failures-to-ci-conductor
|
||||
with:
|
||||
test-suite: ${{ inputs.junit-name }}
|
||||
adapter: backend
|
||||
|
||||
- name: Publish Test Report (JUnit)
|
||||
uses: dorny/test-reporter@a43b3a5f7366b97d083190328d2c652e1a8b6aa2 # v3.0.0
|
||||
# As per the documentation: https://docs.github.com/en/actions/reference/workflows-and-actions/contexts#steps-context
|
||||
@@ -78,7 +57,6 @@ runs:
|
||||
list-suites: failed
|
||||
list-tests: failed
|
||||
fail-on-error: false
|
||||
|
||||
- name: Upload Logs on Failure
|
||||
if: always() && steps.run-test.outcome != 'success'
|
||||
uses: actions/upload-artifact@v7
|
||||
@@ -88,20 +66,3 @@ runs:
|
||||
retention-days: 1
|
||||
# Failing to upload the artifact should never affect the outcome of the whole test run.
|
||||
continue-on-error: true
|
||||
|
||||
# Last, because this is the only step allowed to fail the job: everything above it reports the
|
||||
# failure (JUnit report, logs, ci-conductor) and must run regardless of the verdict below.
|
||||
- name: Quarantine gate
|
||||
# Off master and release branches an "info" driver is data-collection only - the test always passes!
|
||||
# On master and release branches "info" carries no weight - the quarantine logic applies.
|
||||
if: |
|
||||
!cancelled()
|
||||
&& steps.run-test.outcome == 'failure'
|
||||
&& !(inputs.driver-status == 'info'
|
||||
&& !(github.ref_name == 'master' || startsWith(github.ref_name, 'release-x')))
|
||||
continue-on-error: ${{ inputs.quarantine-engine != 'ci-conductor' }}
|
||||
uses: ./.github/actions/quarantine-gate
|
||||
with:
|
||||
adapter: backend
|
||||
test-suite: ${{ inputs.junit-name }}
|
||||
enforce-quarantine: ${{ inputs.quarantine-engine == 'ci-conductor' }}
|
||||
|
||||
127
.github/actions/upload-test-results/action.yml
vendored
127
.github/actions/upload-test-results/action.yml
vendored
@@ -21,23 +21,6 @@ inputs:
|
||||
default: ""
|
||||
previous-step-outcome:
|
||||
required: true
|
||||
driver-status:
|
||||
required: false
|
||||
default: ""
|
||||
description: >-
|
||||
Configured status for this driver ("info", "skip", "required", or empty). When
|
||||
"info", the job is reported as passing regardless of the real test outcome, and
|
||||
that outcome is surfaced via a job summary and (on PRs) a comment instead.
|
||||
report-variant:
|
||||
required: false
|
||||
default: ""
|
||||
description: >-
|
||||
Optional label disambiguating matrix legs of the same job, so the info-driver
|
||||
job summary and PR comment stay distinct per leg.
|
||||
run-trunk:
|
||||
required: false
|
||||
default: "true"
|
||||
description: Whether to run the Trunk step at all.
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
@@ -64,12 +47,10 @@ runs:
|
||||
aws s3 cp ${OUTPUT_FILE}.zip s3://$BUCKET/$DATE/$GITHUB_RUN_ID/$GITHUB_RUN_ATTEMPT/
|
||||
|
||||
- name: Upload results to Trunk
|
||||
if: ${{ always() && github.repository_owner == 'metabase' && inputs.run-trunk == 'true' }}
|
||||
# Important!
|
||||
# For "info" drivers off master/release branches, our own status takes precedence: we let this
|
||||
# step pass so the job is reported as passing regardless of the real outcome (still uploaded).
|
||||
# On master and release branches, info status is ignored and the real outcome gates the job.
|
||||
continue-on-error: ${{ inputs.driver-status == 'info' && !(github.ref_name == 'master' || startsWith(github.ref_name, 'release-x')) }}
|
||||
# 1. This step must always run because it controls job's exit code.
|
||||
# 2. There's no point in running this outside the organization
|
||||
if: always() && github.repository_owner == 'metabase'
|
||||
uses: trunk-io/analytics-uploader@1198efcc0976501147b10a7e343c3fa069527526 # v2.0.9
|
||||
env:
|
||||
TRUNK_REPO_URL: git@github.com:metabase/metabase.git
|
||||
@@ -81,105 +62,3 @@ runs:
|
||||
token: ${{ inputs.trunk-api-token }}
|
||||
use-cache: true
|
||||
previous-step-outcome: ${{ inputs.previous-step-outcome }}
|
||||
|
||||
# For "info" drivers the job is forced green above, so surface the real outcome here —
|
||||
# but only when it did NOT pass, to avoid spamming summaries/comments for healthy runs.
|
||||
# Never include logs or stack traces — only the real status and a link to the run.
|
||||
- name: Report info-level driver result (job summary)
|
||||
if: ${{ always() && inputs.driver-status == 'info' && inputs.previous-step-outcome == 'failure' }}
|
||||
shell: bash
|
||||
env:
|
||||
OUTCOME: ${{ inputs.previous-step-outcome }}
|
||||
LABEL: ${{ inputs.output-name }}
|
||||
VARIANT: ${{ inputs.report-variant }}
|
||||
run: | # sh
|
||||
if [ "$OUTCOME" = "success" ]; then EMOJI="✅"; else EMOJI="❌"; fi
|
||||
TITLE="$LABEL"
|
||||
[ -n "$VARIANT" ] && TITLE="$LABEL ($VARIANT)"
|
||||
RUN_URL="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"
|
||||
{
|
||||
echo "### ℹ️ Info-level driver: $TITLE"
|
||||
echo ""
|
||||
echo "Reported as passing (non-blocking). Real result: $EMOJI \`$OUTCOME\`"
|
||||
echo ""
|
||||
echo "[View run]($RUN_URL)"
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
# Maintain ONE PR comment per commit for all info drivers. Each leg upserts its own line
|
||||
# (keyed by a hidden tag); a passing leg removes its line, and the comment is deleted once
|
||||
# no failing legs remain. Comments from older commits are cleaned up. Runs on pass and fail
|
||||
# so recoveries self-heal. Never includes logs or stack traces — only status and a run link.
|
||||
- name: Sync info-level driver result into the PR comment
|
||||
if: ${{ always() && inputs.driver-status == 'info' && github.event_name == 'pull_request' }}
|
||||
uses: actions/github-script@v9
|
||||
continue-on-error: true
|
||||
env:
|
||||
OUTCOME: ${{ inputs.previous-step-outcome }}
|
||||
LABEL: ${{ inputs.output-name }}
|
||||
VARIANT: ${{ inputs.report-variant }}
|
||||
with:
|
||||
script: | # js
|
||||
const { OUTCOME, LABEL, VARIANT, GITHUB_SERVER_URL, GITHUB_REPOSITORY, GITHUB_RUN_ID } = process.env;
|
||||
const pr = context.payload.pull_request;
|
||||
if (!pr) return;
|
||||
const { owner, repo } = context.repo;
|
||||
const sha = pr.head.sha;
|
||||
// PREFIX matches the current per-commit marker AND the older per-leg one, so legacy
|
||||
// comments from a previous version get cleaned up on the next sync.
|
||||
const PREFIX = "<!-- info-driver-result";
|
||||
const marker = `<!-- info-driver-results:${sha} -->`;
|
||||
const header = `${marker}\nℹ️ **Info-level drivers** failed (non-blocking) on \`${sha.slice(0, 7)}\` — these do not gate:`;
|
||||
const lineKey = `${LABEL}:${VARIANT}`;
|
||||
const title = VARIANT ? `${LABEL} (${VARIANT})` : LABEL;
|
||||
const runUrl = `${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}`;
|
||||
// Only real failures are reported; cancelled/skipped/success remove this leg's line.
|
||||
// List-item line with the hidden key tag at the END so markdown (links) renders.
|
||||
const ourLine = OUTCOME === "failure"
|
||||
? `- **${title}** — ❌ \`${OUTCOME}\` ([Logs](${runUrl})) <!--l:${lineKey}-->`
|
||||
: null;
|
||||
const linesOf = (body) => body.split("\n").filter((l) => l.includes("<!--l:"));
|
||||
|
||||
const sync = async () => {
|
||||
const { data: comments } = await github.rest.issues.listComments({
|
||||
owner, repo, issue_number: pr.number, per_page: 100,
|
||||
});
|
||||
const tagged = comments.filter((c) => c.body.includes(PREFIX));
|
||||
// Drop comments left over from older commits.
|
||||
for (const c of tagged.filter((c) => !c.body.includes(marker))) {
|
||||
await github.rest.issues.deleteComment({ owner, repo, comment_id: c.id }).catch(() => {});
|
||||
}
|
||||
const mine = tagged
|
||||
.filter((c) => c.body.includes(marker))
|
||||
.sort((a, b) => new Date(a.created_at) - new Date(b.created_at));
|
||||
// Merge any duplicates raced into existence; latest write of each line wins.
|
||||
const byKey = new Map();
|
||||
for (const c of mine) {
|
||||
for (const l of linesOf(c.body)) {
|
||||
const m = l.match(/<!--l:(.*?)-->/);
|
||||
if (m) byKey.set(m[1], l);
|
||||
}
|
||||
}
|
||||
if (ourLine) byKey.set(lineKey, ourLine);
|
||||
else byKey.delete(lineKey);
|
||||
const lines = [...byKey.values()].sort();
|
||||
const target = mine[0];
|
||||
for (const c of mine.slice(1)) {
|
||||
await github.rest.issues.deleteComment({ owner, repo, comment_id: c.id }).catch(() => {});
|
||||
}
|
||||
if (lines.length === 0) {
|
||||
if (target) await github.rest.issues.deleteComment({ owner, repo, comment_id: target.id }).catch(() => {});
|
||||
return;
|
||||
}
|
||||
const body = [header, "", ...lines].join("\n");
|
||||
if (target) {
|
||||
await github.rest.issues.updateComment({ owner, repo, comment_id: target.id, body });
|
||||
} else {
|
||||
await github.rest.issues.createComment({ owner, repo, issue_number: pr.number, body });
|
||||
}
|
||||
};
|
||||
|
||||
// Retry to shrink the window where parallel legs clobber each other's update.
|
||||
for (let i = 0; i < 4; i++) {
|
||||
try { await sync(); break; }
|
||||
catch (e) { if (i === 3) throw e; await new Promise((r) => setTimeout(r, 400 * (i + 1))); }
|
||||
}
|
||||
|
||||
13
.github/file-paths.yaml
vendored
13
.github/file-paths.yaml
vendored
@@ -7,10 +7,6 @@ default: &default
|
||||
- "locales/**"
|
||||
- "bin/**/!(*.md)"
|
||||
|
||||
actionlint:
|
||||
- ".github/workflows/*.yml"
|
||||
- ".github/workflows/*.yaml"
|
||||
|
||||
shared_sources: &shared_sources
|
||||
- "src/**/*.cljc"
|
||||
- "src/**/*.cljs"
|
||||
@@ -99,12 +95,6 @@ frontend_loki_infra: &frontend_loki_infra
|
||||
- *shared_ci
|
||||
- *frontend_loki_ci
|
||||
|
||||
# e2e harness/support code (commands, support file, runner). Not in the
|
||||
# instrumented bundle, so it maps to no covered module — a change here forces a
|
||||
# full e2e run (see create-test-plan / e2eInfraTouched).
|
||||
e2e_infra: &e2e_infra
|
||||
- *shared_e2e
|
||||
|
||||
backend_ci: &backend_ci
|
||||
- *shared_ci
|
||||
- ".github/actions/test-driver/**"
|
||||
@@ -205,9 +195,6 @@ documentation:
|
||||
- "docs/**"
|
||||
- "**/*.md"
|
||||
|
||||
docs:
|
||||
- "docs/**"
|
||||
|
||||
embedding_documentation:
|
||||
- ".typedoc/**"
|
||||
- "docs/embedding/**"
|
||||
|
||||
236
.github/scripts/affected-modules.ts
vendored
236
.github/scripts/affected-modules.ts
vendored
@@ -1,191 +1,32 @@
|
||||
import micromatch from "micromatch";
|
||||
|
||||
export type ModuleDef = { type: string; pattern: string };
|
||||
export type Rule = { from: string[]; allow: string[] };
|
||||
|
||||
export type ModuleNode = { type: string; regex: RegExp };
|
||||
|
||||
export type FileDependency = { source: string; dependencies: string[] };
|
||||
|
||||
/** First match wins, so element order matters for nested patterns. */
|
||||
export function buildNodes(elements: ModuleDef[]): ModuleNode[] {
|
||||
return elements.map((el) => ({
|
||||
type: el.type,
|
||||
// dot:true so files in dot-directories (e.g. .storybook) still match.
|
||||
regex: micromatch.makeRe(el.pattern, { dot: true }),
|
||||
}));
|
||||
}
|
||||
|
||||
export function mapFileToModule(
|
||||
nodes: ModuleNode[],
|
||||
path: string,
|
||||
): string | null {
|
||||
for (const node of nodes) {
|
||||
if (node.regex.test(path)) {
|
||||
return node.type;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export function getChangedModules(
|
||||
nodes: ModuleNode[],
|
||||
changedFiles: string[],
|
||||
): Set<string> {
|
||||
const direct = new Set<string>();
|
||||
for (const file of changedFiles) {
|
||||
const module = mapFileToModule(nodes, file);
|
||||
if (module) {
|
||||
direct.add(module);
|
||||
}
|
||||
}
|
||||
return direct;
|
||||
}
|
||||
|
||||
export type FileGraph = {
|
||||
nodes: ModuleNode[];
|
||||
fileDependents: Map<string, Set<string>>;
|
||||
};
|
||||
|
||||
/** Drops unresolvable imports (e.g. ambient types) that point at no real file. */
|
||||
export function parseCruiseModules(
|
||||
modules: Array<{
|
||||
source: string;
|
||||
dependencies?: Array<{ resolved: string; couldNotResolve?: boolean }>;
|
||||
}>,
|
||||
): FileDependency[] {
|
||||
return modules.map((module) => ({
|
||||
source: module.source,
|
||||
dependencies: (module.dependencies ?? [])
|
||||
.filter((dep) => !dep.couldNotResolve)
|
||||
.map((dep) => dep.resolved),
|
||||
}));
|
||||
}
|
||||
|
||||
/** Inverts the import edges into the `fileDependents` reverse graph. */
|
||||
export function buildFileGraph(
|
||||
elements: ModuleDef[],
|
||||
fileDependencies: FileDependency[],
|
||||
): FileGraph {
|
||||
const fileDependents = new Map<string, Set<string>>();
|
||||
for (const { source, dependencies } of fileDependencies) {
|
||||
for (const target of dependencies) {
|
||||
let importers = fileDependents.get(target);
|
||||
if (!importers) {
|
||||
importers = new Set();
|
||||
fileDependents.set(target, importers);
|
||||
}
|
||||
importers.add(source);
|
||||
}
|
||||
}
|
||||
return { nodes: buildNodes(elements), fileDependents };
|
||||
}
|
||||
|
||||
/** Changed files plus every file that transitively imports them. */
|
||||
export function getAffectedFiles(
|
||||
fileGraph: FileGraph,
|
||||
changedFiles: string[],
|
||||
): Set<string> {
|
||||
const affected = new Set(changedFiles);
|
||||
const queue = [...changedFiles];
|
||||
while (queue.length > 0) {
|
||||
const file = queue.shift()!;
|
||||
for (const importer of fileGraph.fileDependents.get(file) ?? []) {
|
||||
if (!affected.has(importer)) {
|
||||
affected.add(importer);
|
||||
queue.push(importer);
|
||||
}
|
||||
}
|
||||
}
|
||||
return affected;
|
||||
}
|
||||
type ModuleNode = { type: string; regex: RegExp };
|
||||
|
||||
export type ModuleGraph = {
|
||||
nodes: ModuleNode[];
|
||||
/**
|
||||
* dependents.get(M) is every module affected when M changes — the full
|
||||
* transitive closure, so getAffectedModules only needs a single hop.
|
||||
*/
|
||||
dependents: Map<string, Set<string>>;
|
||||
};
|
||||
|
||||
/** Expands a direct-dependents map into its transitive closure. */
|
||||
function transitiveClosure(
|
||||
direct: Map<string, Set<string>>,
|
||||
): Map<string, Set<string>> {
|
||||
const closure = new Map<string, Set<string>>();
|
||||
for (const start of direct.keys()) {
|
||||
const reached = new Set<string>();
|
||||
const queue = [...(direct.get(start) ?? [])];
|
||||
while (queue.length > 0) {
|
||||
const node = queue.shift()!;
|
||||
if (reached.has(node)) {
|
||||
continue;
|
||||
}
|
||||
reached.add(node);
|
||||
for (const next of direct.get(node) ?? []) {
|
||||
queue.push(next);
|
||||
}
|
||||
}
|
||||
reached.delete(start);
|
||||
closure.set(start, reached);
|
||||
}
|
||||
return closure;
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds a module graph from the actual import graph. Reachability is computed
|
||||
* per module at the file level and then collapsed to modules.
|
||||
* Compiles a glob pattern to an anchored RegExp.
|
||||
*/
|
||||
export function buildUsageModuleGraph(
|
||||
elements: ModuleDef[],
|
||||
fileDependencies: FileDependency[],
|
||||
): ModuleGraph {
|
||||
const nodes = buildNodes(elements);
|
||||
const fileGraph = buildFileGraph(elements, fileDependencies);
|
||||
|
||||
const filesByModule = new Map<string, string[]>();
|
||||
const allFiles = new Set<string>();
|
||||
for (const { source, dependencies } of fileDependencies) {
|
||||
allFiles.add(source);
|
||||
for (const target of dependencies) {
|
||||
allFiles.add(target);
|
||||
}
|
||||
}
|
||||
for (const file of allFiles) {
|
||||
const module = mapFileToModule(nodes, file);
|
||||
if (module) {
|
||||
const files = filesByModule.get(module);
|
||||
if (files) {
|
||||
files.push(file);
|
||||
} else {
|
||||
filesByModule.set(module, [file]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const dependents = new Map<string, Set<string>>(
|
||||
elements.map((el) => [el.type, new Set()]),
|
||||
);
|
||||
for (const [module, files] of filesByModule) {
|
||||
const affected = new Set<string>();
|
||||
for (const file of getAffectedFiles(fileGraph, files)) {
|
||||
const owner = mapFileToModule(nodes, file);
|
||||
if (owner && owner !== module) {
|
||||
affected.add(owner);
|
||||
}
|
||||
}
|
||||
dependents.set(module, affected);
|
||||
}
|
||||
|
||||
return { nodes, dependents };
|
||||
export function globToRegex(glob: string): RegExp {
|
||||
const escapeSegment = (seg: string) =>
|
||||
seg.replace(/[.+?^${}()|[\]\\]/g, "\\$&").replace(/\*/g, "[^/]*");
|
||||
const body = glob.split("**").map(escapeSegment).join(".*");
|
||||
return new RegExp(`^${body}$`);
|
||||
}
|
||||
|
||||
export function buildModuleGraph(
|
||||
elements: ModuleDef[],
|
||||
rules: Rule[],
|
||||
): ModuleGraph {
|
||||
const nodes = buildNodes(elements);
|
||||
const nodes = elements.map((el) => ({
|
||||
type: el.type,
|
||||
regex: globToRegex(el.pattern),
|
||||
}));
|
||||
|
||||
const allTypes = elements.map((e) => e.type);
|
||||
|
||||
function expandPattern(pattern: string): string[] {
|
||||
@@ -199,7 +40,9 @@ export function buildModuleGraph(
|
||||
return allTypes.includes(pattern) ? [pattern] : [];
|
||||
}
|
||||
|
||||
const directDependents = new Map<string, Set<string>>(
|
||||
// dependents.get(M) is the set of modules that depend on M
|
||||
// i.e. modules that are affected when M changes
|
||||
const dependents = new Map<string, Set<string>>(
|
||||
allTypes.map((type) => [type, new Set()]),
|
||||
);
|
||||
for (const rule of rules) {
|
||||
@@ -208,13 +51,46 @@ export function buildModuleGraph(
|
||||
for (const target of allowTypes) {
|
||||
for (const importer of fromTypes) {
|
||||
if (importer !== target) {
|
||||
directDependents.get(target)?.add(importer);
|
||||
dependents.get(target)?.add(importer);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return { nodes, dependents: transitiveClosure(directDependents) };
|
||||
return { nodes, dependents };
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the module type that owns the given file.
|
||||
* First match wins, so element ordering matters for nested patterns.
|
||||
*/
|
||||
export function mapFileToModule(
|
||||
moduleGraph: ModuleGraph,
|
||||
path: string,
|
||||
): string | null {
|
||||
for (const el of moduleGraph.nodes) {
|
||||
if (el.regex.test(path)) {
|
||||
return el.type;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the set of module types containing any of the changed files.
|
||||
*/
|
||||
export function getChangedModules(
|
||||
moduleGraph: ModuleGraph,
|
||||
changedFiles: string[],
|
||||
): Set<string> {
|
||||
const direct = new Set<string>();
|
||||
for (const file of changedFiles) {
|
||||
const module = mapFileToModule(moduleGraph, file);
|
||||
if (module) {
|
||||
direct.add(module);
|
||||
}
|
||||
}
|
||||
return direct;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -224,10 +100,16 @@ export function getAffectedModules(
|
||||
moduleGraph: ModuleGraph,
|
||||
changedFiles: string[],
|
||||
): Set<string> {
|
||||
const affected = getChangedModules(moduleGraph.nodes, changedFiles);
|
||||
for (const module of [...affected]) {
|
||||
const direct = getChangedModules(moduleGraph, changedFiles);
|
||||
const affected = new Set(direct);
|
||||
const queue = [...direct];
|
||||
while (queue.length > 0) {
|
||||
const module = queue.shift()!;
|
||||
for (const dep of moduleGraph.dependents.get(module) ?? []) {
|
||||
affected.add(dep);
|
||||
if (!affected.has(dep)) {
|
||||
affected.add(dep);
|
||||
queue.push(dep);
|
||||
}
|
||||
}
|
||||
}
|
||||
return affected;
|
||||
|
||||
90
.github/scripts/affected-modules.unit.spec.ts
vendored
90
.github/scripts/affected-modules.unit.spec.ts
vendored
@@ -1,13 +1,8 @@
|
||||
import {
|
||||
buildFileGraph,
|
||||
buildModuleGraph,
|
||||
buildNodes,
|
||||
buildUsageModuleGraph,
|
||||
getAffectedFiles,
|
||||
getAffectedModules,
|
||||
getChangedModules,
|
||||
mapFileToModule,
|
||||
parseCruiseModules,
|
||||
} from "./affected-modules";
|
||||
|
||||
const ELEMENTS = [
|
||||
@@ -36,7 +31,6 @@ const RULES = [
|
||||
{ from: ["app/*"], allow: ["lib/*", "feature/*", "app/*"] },
|
||||
];
|
||||
|
||||
const nodes = buildNodes(ELEMENTS);
|
||||
const graph = buildModuleGraph(ELEMENTS, RULES);
|
||||
|
||||
describe("affected modules", () => {
|
||||
@@ -55,19 +49,19 @@ describe("affected modules", () => {
|
||||
["external/lib/foo.ts", "feature/external"],
|
||||
["src/embed.tsx", "app/main"],
|
||||
])("should map %s to %s", (path, expected) => {
|
||||
expect(mapFileToModule(nodes, path)).toBe(expected);
|
||||
expect(mapFileToModule(graph, path)).toBe(expected);
|
||||
});
|
||||
|
||||
it("should return null for files outside any pattern", () => {
|
||||
expect(mapFileToModule(nodes, "docs/foo.md")).toBe(null);
|
||||
expect(mapFileToModule(nodes, "README.md")).toBe(null);
|
||||
expect(mapFileToModule(nodes, "external/notlib/foo.ts")).toBe(null);
|
||||
expect(mapFileToModule(graph, "docs/foo.md")).toBe(null);
|
||||
expect(mapFileToModule(graph, "README.md")).toBe(null);
|
||||
expect(mapFileToModule(graph, "external/notlib/foo.ts")).toBe(null);
|
||||
});
|
||||
});
|
||||
|
||||
describe("getChangedModules", () => {
|
||||
it("should return the unique set of modules whose files changed", () => {
|
||||
const result = getChangedModules(nodes, [
|
||||
const result = getChangedModules(graph, [
|
||||
"src/utils/colors.ts",
|
||||
"src/foo/x.ts",
|
||||
"src/foo/y.ts",
|
||||
@@ -77,81 +71,11 @@ describe("affected modules", () => {
|
||||
});
|
||||
|
||||
it("should return an empty set when no file maps to a module", () => {
|
||||
expect(getChangedModules(nodes, ["docs/foo.md"]).size).toBe(0);
|
||||
expect(getChangedModules(graph, ["docs/foo.md"]).size).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe("parseCruiseModules", () => {
|
||||
it("should flatten to source + resolved-paths and drop unresolved deps", () => {
|
||||
expect(
|
||||
parseCruiseModules([
|
||||
{
|
||||
source: "src/foo/foo.tsx",
|
||||
dependencies: [
|
||||
{ resolved: "src/utils/colors.ts" },
|
||||
{ resolved: "ambient/types", couldNotResolve: true },
|
||||
],
|
||||
},
|
||||
{ source: "src/bar/bar.tsx" }, // no dependencies key
|
||||
]),
|
||||
).toEqual([
|
||||
{ source: "src/foo/foo.tsx", dependencies: ["src/utils/colors.ts"] },
|
||||
{ source: "src/bar/bar.tsx", dependencies: [] },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
const FILE_DEPS = [
|
||||
{ source: "src/app.js", dependencies: ["src/foo/foo.tsx"] },
|
||||
{ source: "src/bar/bar.tsx", dependencies: ["src/embed.tsx"] },
|
||||
{ source: "src/foo/foo.tsx", dependencies: ["src/utils/colors.ts"] },
|
||||
];
|
||||
const fileGraph = buildFileGraph(ELEMENTS, FILE_DEPS);
|
||||
|
||||
describe("getAffectedFiles", () => {
|
||||
it("should return the changed file and everything that imports it", () => {
|
||||
expect(
|
||||
[...getAffectedFiles(fileGraph, ["src/utils/colors.ts"])].sort(),
|
||||
).toEqual(["src/app.js", "src/foo/foo.tsx", "src/utils/colors.ts"]);
|
||||
});
|
||||
|
||||
it("should include a changed file with no importers (itself only)", () => {
|
||||
// Nothing imports the app.js entry point.
|
||||
expect([...getAffectedFiles(fileGraph, ["src/app.js"])]).toEqual([
|
||||
"src/app.js",
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildUsageModuleGraph + getAffectedModules", () => {
|
||||
const usageGraph = buildUsageModuleGraph(ELEMENTS, FILE_DEPS);
|
||||
|
||||
it("should expand a changed module to its file-level affected modules", () => {
|
||||
expect(
|
||||
[...getAffectedModules(usageGraph, ["src/utils/colors.ts"])].sort(),
|
||||
).toEqual(["app/main", "feature/foo", "lib/utils"]);
|
||||
});
|
||||
|
||||
it("should NOT pull in feature/bar via the app/main hub", () => {
|
||||
const affected = getAffectedModules(usageGraph, ["src/foo/foo.tsx"]);
|
||||
expect([...affected].sort()).toEqual(["app/main", "feature/foo"]);
|
||||
expect(affected.has("feature/bar")).toBe(false);
|
||||
});
|
||||
|
||||
it("should treat the whole changed module as changed", () => {
|
||||
// app.js itself has no importers, but app/main also owns embed.tsx, which
|
||||
// feature/bar imports so changing any app/main file flags feature/bar.
|
||||
expect(
|
||||
[...getAffectedModules(usageGraph, ["src/app.js"])].sort(),
|
||||
).toEqual(["app/main", "feature/bar"]);
|
||||
});
|
||||
|
||||
it("should return empty when no file maps to a module", () => {
|
||||
expect(getAffectedModules(usageGraph, ["docs/foo.md"]).size).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe("getAffectedModules (rules graph)", () => {
|
||||
describe("getAffectedModules", () => {
|
||||
it("should affect feature/super and app/main but not feature/bar when feature/foo changes", () => {
|
||||
const result = getAffectedModules(graph, ["src/foo/x.ts"]);
|
||||
expect([...result].sort()).toEqual([
|
||||
|
||||
304
.github/scripts/affected-tests.ts
vendored
304
.github/scripts/affected-tests.ts
vendored
@@ -1,40 +1,36 @@
|
||||
// Builds the affected-tests plan from raw inputs: computes the rules and usage
|
||||
// affected-module sets and selects, per suite, which specs to run under each.
|
||||
import { getFeatureModules } from "../../frontend/lint/module-boundaries.mjs";
|
||||
// Per-suite affected-tests logic: combines module-affected closure with
|
||||
// per-suite "infra-touched" booleans (computed by dorny/paths-filter against
|
||||
// the `frontend_unit_infra` / `frontend_loki_infra` anchors in file-paths.yaml).
|
||||
// When infra files change, the whole suite runs in full.
|
||||
|
||||
import {
|
||||
type FileDependency,
|
||||
type ModuleDef,
|
||||
type ModuleNode,
|
||||
type ModuleGraph,
|
||||
type Rule,
|
||||
buildModuleGraph,
|
||||
buildUsageModuleGraph,
|
||||
getAffectedModules,
|
||||
getChangedModules,
|
||||
mapFileToModule,
|
||||
} from "./affected-modules";
|
||||
|
||||
export type TestSuiteDef = {
|
||||
statsPrefix: string;
|
||||
runAllTests?: boolean;
|
||||
};
|
||||
|
||||
export type TestPlanStats = {
|
||||
fe_files_changed: number;
|
||||
fe_files_total: number;
|
||||
be_files_changed: number;
|
||||
be_files_total: number;
|
||||
unit_infra_touched: boolean;
|
||||
loki_infra_touched: boolean;
|
||||
shared_sources_touched: boolean;
|
||||
fe_modules_total: number;
|
||||
fe_modules_changed: number;
|
||||
fe_modules_affected_rules: number;
|
||||
fe_modules_affected_usage: number;
|
||||
unit_specs_all: number;
|
||||
unit_specs_to_run_rules: number;
|
||||
unit_specs_to_run_usage: number;
|
||||
loki_stories_all: number;
|
||||
loki_stories_to_run_rules: number;
|
||||
loki_stories_to_run_usage: number;
|
||||
e2e_specs_all: number;
|
||||
e2e_specs_to_run_rules: number;
|
||||
e2e_specs_to_run_usage: number;
|
||||
modules_changed: number;
|
||||
modules_affected: number;
|
||||
modules_affected_list: string[];
|
||||
fe_unit_specs_total: number;
|
||||
fe_unit_specs_run: number;
|
||||
fe_unit_specs_skipped: number;
|
||||
loki_stories_total: number;
|
||||
loki_stories_run: number;
|
||||
loki_stories_skipped: number;
|
||||
e2e_specs_total: number;
|
||||
e2e_specs_run: number;
|
||||
e2e_specs_skipped: number;
|
||||
};
|
||||
|
||||
export type TestPlan = {
|
||||
@@ -44,187 +40,101 @@ export type TestPlan = {
|
||||
e2e_specs_to_run: string[];
|
||||
};
|
||||
|
||||
export type CreateTestPlanInput = {
|
||||
elements: ModuleDef[];
|
||||
rules: Rule[];
|
||||
changedFiles: string[];
|
||||
// Parsed dependency-cruiser edges, or null to fall back to the rules graph.
|
||||
fileDependencies: FileDependency[] | null;
|
||||
testFilesBySuite: { unit: string[]; loki: string[]; e2e: string[] };
|
||||
e2eSpecFiles: Record<string, string[]> | null;
|
||||
unitInfraTouched: boolean;
|
||||
lokiInfraTouched: boolean;
|
||||
e2eInfraTouched: boolean;
|
||||
sharedSourcesTouched: boolean;
|
||||
feFilesChanged: number;
|
||||
beFilesChanged: number;
|
||||
feFilesTotal: number;
|
||||
beFilesTotal: number;
|
||||
};
|
||||
|
||||
// Tests whose owning module is affected.
|
||||
export function filterAffectedTests(
|
||||
nodes: ModuleNode[],
|
||||
affected: Set<string>,
|
||||
testFiles: string[],
|
||||
): string[] {
|
||||
return testFiles.filter((file) => {
|
||||
const module = mapFileToModule(nodes, file);
|
||||
return module !== null && affected.has(module);
|
||||
});
|
||||
}
|
||||
|
||||
// Collapse each spec's covered files to the *feature* modules it exercises,
|
||||
// once, for reuse across the rules/usage selects. Non-feature tiers
|
||||
// (shared/lib/...) are intentionally dropped: the manifest is a stale nightly
|
||||
// artifact, so its coupling to the module graph stays minimal ("does this spec
|
||||
// touch feature X?"). Accuracy comes from the fresh `affected` set, which
|
||||
// already expands a shared/infra change into the feature modules that depend on
|
||||
// it.
|
||||
export function specFeatureModules(
|
||||
nodes: ModuleNode[],
|
||||
featureModules: Set<string>,
|
||||
specFiles: Record<string, string[]>,
|
||||
): Map<string, Set<string>> {
|
||||
const moduleCache = new Map<string, string | null>();
|
||||
const moduleOf = (file: string): string | null => {
|
||||
let module = moduleCache.get(file);
|
||||
if (module === undefined) {
|
||||
module = mapFileToModule(nodes, file);
|
||||
moduleCache.set(file, module);
|
||||
}
|
||||
return module;
|
||||
};
|
||||
|
||||
const result = new Map<string, Set<string>>();
|
||||
for (const [spec, files] of Object.entries(specFiles)) {
|
||||
const features = new Set<string>();
|
||||
for (const file of files) {
|
||||
const module = moduleOf(file);
|
||||
if (module !== null && featureModules.has(module)) {
|
||||
features.add(module);
|
||||
}
|
||||
}
|
||||
result.set(spec, features);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
// A spec runs if any feature module it covers is affected.
|
||||
// A spec doesn't map to any feature module always runs because its scope is unknown
|
||||
export function filterAffectedE2eSpecs(
|
||||
specFeatures: Map<string, Set<string>>,
|
||||
affected: Set<string>,
|
||||
e2eSpecs: string[],
|
||||
): string[] {
|
||||
return e2eSpecs.filter((spec) => {
|
||||
const features = specFeatures.get(spec);
|
||||
if (!features || features.size === 0) {
|
||||
return true;
|
||||
}
|
||||
return [...features].some((module) => affected.has(module));
|
||||
});
|
||||
}
|
||||
export const TEST_SUITES = {
|
||||
unit: {
|
||||
statsPrefix: "fe_unit_specs",
|
||||
},
|
||||
loki: {
|
||||
statsPrefix: "loki_stories",
|
||||
},
|
||||
e2e: {
|
||||
statsPrefix: "e2e_specs",
|
||||
// We don't map changed files to e2e tests yet, so for now just run them all
|
||||
runAllTests: true,
|
||||
},
|
||||
} satisfies Record<string, TestSuiteDef>;
|
||||
|
||||
export function createTestPlan({
|
||||
elements,
|
||||
rules,
|
||||
testSuites,
|
||||
changedFiles,
|
||||
fileDependencies,
|
||||
testFilesBySuite,
|
||||
e2eSpecFiles,
|
||||
unitInfraTouched,
|
||||
lokiInfraTouched,
|
||||
e2eInfraTouched,
|
||||
sharedSourcesTouched,
|
||||
feFilesChanged,
|
||||
beFilesChanged,
|
||||
feFilesTotal,
|
||||
beFilesTotal,
|
||||
}: CreateTestPlanInput): TestPlan {
|
||||
const rulesGraph = buildModuleGraph(elements, rules);
|
||||
const usageGraph = fileDependencies
|
||||
? buildUsageModuleGraph(elements, fileDependencies)
|
||||
: rulesGraph;
|
||||
infraTouchedBySuite,
|
||||
}: {
|
||||
elements: ModuleDef[];
|
||||
rules: Rule[];
|
||||
testSuites: Record<string, TestSuiteDef>;
|
||||
changedFiles: string[];
|
||||
testFilesBySuite: Record<string, string[]>;
|
||||
infraTouchedBySuite: Record<string, boolean>;
|
||||
}): TestPlan {
|
||||
const moduleGraph = buildModuleGraph(elements, rules);
|
||||
|
||||
const nodes = rulesGraph.nodes;
|
||||
// Distinct module types (an element type can span several patterns).
|
||||
const totalModules = new Set(elements.map((el) => el.type)).size;
|
||||
const changedModules = getChangedModules(nodes, changedFiles);
|
||||
const rulesAffected = getAffectedModules(rulesGraph, changedFiles);
|
||||
const usageAffected = getAffectedModules(usageGraph, changedFiles);
|
||||
const changedModules = getChangedModules(moduleGraph, changedFiles);
|
||||
const modulesAffected = getAffectedModules(moduleGraph, changedFiles);
|
||||
const modulesAffectedList = [...modulesAffected].sort();
|
||||
|
||||
// The coarse "feature" tier is the only set the e2e manifest is ever
|
||||
// collapsed to (see filterAffectedE2eSpecs).
|
||||
const featureModules = new Set(getFeatureModules(elements));
|
||||
|
||||
// cljc/cljs compile into the FE bundle, so they force a full run that module
|
||||
// selection can't narrow — same effect as a suite's own infra changing.
|
||||
const unitForceAll = unitInfraTouched || sharedSourcesTouched;
|
||||
const lokiForceAll = lokiInfraTouched || sharedSourcesTouched;
|
||||
// e2e is integration-level, so anything the FE-coverage manifest can't see
|
||||
// forces a full run: cljc/cljs in the bundle (sharedSourcesTouched), a backend
|
||||
// change that can break the UI (beFilesChanged), an e2e harness/support change
|
||||
// (e2eInfraTouched), or no manifest at all.
|
||||
const e2eForceAll =
|
||||
sharedSourcesTouched ||
|
||||
e2eInfraTouched ||
|
||||
beFilesChanged > 0 ||
|
||||
e2eSpecFiles === null;
|
||||
|
||||
const select = (forceAll: boolean, affected: Set<string>, files: string[]) =>
|
||||
forceAll ? files : filterAffectedTests(nodes, affected, files);
|
||||
|
||||
const { unit, loki, e2e } = testFilesBySuite;
|
||||
const unitRules = select(unitForceAll, rulesAffected, unit);
|
||||
const unitUsage = select(unitForceAll, usageAffected, unit);
|
||||
const lokiRules = select(lokiForceAll, rulesAffected, loki);
|
||||
const lokiUsage = select(lokiForceAll, usageAffected, loki);
|
||||
|
||||
// Precompute spec -> feature modules once (null when e2e runs in full).
|
||||
const specFeatures =
|
||||
e2eForceAll || e2eSpecFiles === null
|
||||
? null
|
||||
: specFeatureModules(nodes, featureModules, e2eSpecFiles);
|
||||
// A spec that was itself edited always runs, even when no app module changed.
|
||||
const changedSet = new Set(changedFiles);
|
||||
const selectE2e = (affected: Set<string>): string[] => {
|
||||
if (specFeatures === null) {
|
||||
return e2e;
|
||||
}
|
||||
const narrowed = new Set(
|
||||
filterAffectedE2eSpecs(specFeatures, affected, e2e),
|
||||
);
|
||||
return e2e.filter((spec) => narrowed.has(spec) || changedSet.has(spec));
|
||||
const specsToRun: Record<string, string[]> = {};
|
||||
const stats: Record<string, number | string[]> = {
|
||||
modules_changed: changedModules.size,
|
||||
modules_affected: modulesAffected.size,
|
||||
modules_affected_list: modulesAffectedList,
|
||||
};
|
||||
const e2eRules = selectE2e(rulesAffected);
|
||||
const e2eUsage = selectE2e(usageAffected);
|
||||
|
||||
for (const [name, testSuite] of Object.entries(testSuites)) {
|
||||
const files = testFilesBySuite[name] ?? [];
|
||||
const tests = createTestPlanForSuite(
|
||||
moduleGraph,
|
||||
testSuite,
|
||||
infraTouchedBySuite[name] ?? false,
|
||||
changedFiles,
|
||||
files,
|
||||
);
|
||||
specsToRun[name] = tests;
|
||||
stats[`${testSuite.statsPrefix}_total`] = files.length;
|
||||
stats[`${testSuite.statsPrefix}_run`] = tests.length;
|
||||
stats[`${testSuite.statsPrefix}_skipped`] = files.length - tests.length;
|
||||
}
|
||||
|
||||
return {
|
||||
stats: {
|
||||
fe_files_changed: feFilesChanged,
|
||||
fe_files_total: feFilesTotal,
|
||||
be_files_changed: beFilesChanged,
|
||||
be_files_total: beFilesTotal,
|
||||
unit_infra_touched: unitInfraTouched,
|
||||
loki_infra_touched: lokiInfraTouched,
|
||||
shared_sources_touched: sharedSourcesTouched,
|
||||
fe_modules_total: totalModules,
|
||||
fe_modules_changed: changedModules.size,
|
||||
fe_modules_affected_rules: rulesAffected.size,
|
||||
fe_modules_affected_usage: usageAffected.size,
|
||||
unit_specs_all: unit.length,
|
||||
unit_specs_to_run_rules: unitRules.length,
|
||||
unit_specs_to_run_usage: unitUsage.length,
|
||||
loki_stories_all: loki.length,
|
||||
loki_stories_to_run_rules: lokiRules.length,
|
||||
loki_stories_to_run_usage: lokiUsage.length,
|
||||
e2e_specs_all: e2e.length,
|
||||
e2e_specs_to_run_rules: e2eRules.length,
|
||||
e2e_specs_to_run_usage: e2eUsage.length,
|
||||
},
|
||||
fe_unit_specs_to_run: unitUsage,
|
||||
loki_stories_to_run: lokiUsage,
|
||||
e2e_specs_to_run: e2eUsage,
|
||||
stats: stats as TestPlanStats,
|
||||
fe_unit_specs_to_run: specsToRun.unit ?? [],
|
||||
loki_stories_to_run: specsToRun.loki ?? [],
|
||||
e2e_specs_to_run: specsToRun.e2e ?? [],
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Computes the test plan for one test suite
|
||||
*
|
||||
* Runs the full suite when `runAllTests` is set or `infraTouched` is true,
|
||||
* otherwise runs the tests based on affected-modules.
|
||||
*/
|
||||
export function createTestPlanForSuite(
|
||||
moduleGraph: ModuleGraph,
|
||||
testSuite: TestSuiteDef,
|
||||
infraTouched: boolean,
|
||||
changedFiles: string[],
|
||||
allTestFiles: string[],
|
||||
): string[] {
|
||||
if (testSuite.runAllTests || infraTouched) {
|
||||
return allTestFiles;
|
||||
}
|
||||
const affected = getAffectedModules(moduleGraph, changedFiles);
|
||||
return filterAffectedTests(moduleGraph, affected, allTestFiles);
|
||||
}
|
||||
|
||||
/**
|
||||
* Filters the test-file list down to tests whose owning module is affected.
|
||||
*/
|
||||
export function filterAffectedTests(
|
||||
moduleGraph: ModuleGraph,
|
||||
affected: Set<string>,
|
||||
testFiles: string[],
|
||||
): string[] {
|
||||
return testFiles.filter((file) => {
|
||||
const module = mapFileToModule(moduleGraph, file);
|
||||
return module !== null && affected.has(module);
|
||||
});
|
||||
}
|
||||
|
||||
353
.github/scripts/affected-tests.unit.spec.ts
vendored
353
.github/scripts/affected-tests.unit.spec.ts
vendored
@@ -1,5 +1,9 @@
|
||||
import { buildNodes } from "./affected-modules";
|
||||
import { createTestPlan, filterAffectedTests } from "./affected-tests";
|
||||
import { buildModuleGraph } from "./affected-modules";
|
||||
import {
|
||||
createTestPlan,
|
||||
createTestPlanForSuite,
|
||||
filterAffectedTests,
|
||||
} from "./affected-tests";
|
||||
|
||||
const ELEMENTS = [
|
||||
{ type: "lib/utils", pattern: "src/utils/**" },
|
||||
@@ -14,6 +18,12 @@ const RULES = [
|
||||
{ from: ["feature/*"], allow: ["lib/*"] },
|
||||
];
|
||||
|
||||
const TEST_SUITE_DEFS = {
|
||||
unit: { statsPrefix: "fe_unit_specs" },
|
||||
loki: { statsPrefix: "loki_stories" },
|
||||
e2e: { statsPrefix: "e2e_specs", runAllTests: true },
|
||||
};
|
||||
|
||||
const UNIT_FILES = [
|
||||
"src/foo/foo.unit.spec.ts",
|
||||
"src/foo/bar.unit.spec.ts",
|
||||
@@ -23,227 +33,158 @@ const UNIT_FILES = [
|
||||
const LOKI_FILES = ["src/foo/Foo.stories.tsx", "src/bar/Bar.stories.tsx"];
|
||||
const E2E_FILES = ["e2e/test/scenarios/a.cy.spec.ts"];
|
||||
|
||||
const baseInput = {
|
||||
elements: ELEMENTS,
|
||||
rules: RULES,
|
||||
fileDependencies: null,
|
||||
testFilesBySuite: { unit: UNIT_FILES, loki: LOKI_FILES, e2e: E2E_FILES },
|
||||
e2eSpecFiles: null,
|
||||
unitInfraTouched: false,
|
||||
lokiInfraTouched: false,
|
||||
e2eInfraTouched: false,
|
||||
sharedSourcesTouched: false,
|
||||
feFilesChanged: 0,
|
||||
beFilesChanged: 0,
|
||||
feFilesTotal: 0,
|
||||
beFilesTotal: 0,
|
||||
};
|
||||
const moduleGraph = buildModuleGraph(ELEMENTS, RULES);
|
||||
|
||||
describe("createTestPlan", () => {
|
||||
it("runs only specs in affected modules (rules graph)", () => {
|
||||
const plan = createTestPlan({ ...baseInput, changedFiles: ["src/foo/x.ts"] });
|
||||
|
||||
expect(plan.stats.fe_modules_changed).toBe(1);
|
||||
expect(plan.stats.fe_modules_affected_rules).toBe(1);
|
||||
expect(plan.fe_unit_specs_to_run.sort()).toEqual([
|
||||
"src/foo/bar.unit.spec.ts",
|
||||
"src/foo/foo.unit.spec.ts",
|
||||
]);
|
||||
expect(plan.stats.unit_specs_to_run_rules).toBe(2);
|
||||
expect(plan.stats.loki_stories_to_run_rules).toBe(1);
|
||||
// With no file deps, usage falls back to rules.
|
||||
expect(plan.stats.unit_specs_to_run_usage).toBe(2);
|
||||
});
|
||||
|
||||
it("narrows further with the usage graph than the rules graph", () => {
|
||||
const plan = createTestPlan({
|
||||
...baseInput,
|
||||
changedFiles: ["src/utils/colors.ts"],
|
||||
// Only feature/foo actually imports lib/utils.
|
||||
fileDependencies: [
|
||||
{ source: "src/foo/foo.tsx", dependencies: ["src/utils/colors.ts"] },
|
||||
],
|
||||
describe("affected tests", () => {
|
||||
describe("createTestPlanForSuite", () => {
|
||||
it("should run all tests when infra is touched", () => {
|
||||
const result = createTestPlanForSuite(
|
||||
moduleGraph,
|
||||
TEST_SUITE_DEFS.unit,
|
||||
true, // infraTouched
|
||||
["src/foo/x.ts"],
|
||||
UNIT_FILES,
|
||||
);
|
||||
expect(result).toEqual(UNIT_FILES);
|
||||
});
|
||||
|
||||
// Rules allow both features to import lib/utils, so both are "affected".
|
||||
expect(plan.stats.fe_modules_affected_rules).toBe(3);
|
||||
// Usage sees only feature/foo + lib/utils.
|
||||
expect(plan.stats.fe_modules_affected_usage).toBe(2);
|
||||
expect(plan.stats.unit_specs_to_run_rules).toBe(4);
|
||||
expect(plan.stats.unit_specs_to_run_usage).toBe(3);
|
||||
});
|
||||
|
||||
it("forces a full FE run on a shared-sources (cljc) change with no affected modules", () => {
|
||||
const plan = createTestPlan({
|
||||
...baseInput,
|
||||
changedFiles: ["docs/readme.md"],
|
||||
sharedSourcesTouched: true,
|
||||
it("should fall back to affected-tests filtering when infra not touched", () => {
|
||||
const result = createTestPlanForSuite(
|
||||
moduleGraph,
|
||||
TEST_SUITE_DEFS.unit,
|
||||
false, // infraTouched
|
||||
["src/foo/x.ts"],
|
||||
UNIT_FILES,
|
||||
);
|
||||
expect(result.sort()).toEqual([
|
||||
"src/foo/bar.unit.spec.ts",
|
||||
"src/foo/foo.unit.spec.ts",
|
||||
]);
|
||||
});
|
||||
|
||||
expect(plan.stats.fe_modules_affected_rules).toBe(0);
|
||||
expect(plan.stats.unit_specs_to_run_rules).toBe(UNIT_FILES.length);
|
||||
expect(plan.stats.unit_specs_to_run_usage).toBe(UNIT_FILES.length);
|
||||
expect(plan.stats.loki_stories_to_run_rules).toBe(LOKI_FILES.length);
|
||||
});
|
||||
|
||||
it("forces only the suite whose infra changed", () => {
|
||||
const plan = createTestPlan({
|
||||
...baseInput,
|
||||
changedFiles: ["docs/readme.md"],
|
||||
unitInfraTouched: true,
|
||||
it("should return nothing when infra not touched and no file maps to a module", () => {
|
||||
const result = createTestPlanForSuite(
|
||||
moduleGraph,
|
||||
TEST_SUITE_DEFS.unit,
|
||||
false,
|
||||
["docs/foo.md"],
|
||||
UNIT_FILES,
|
||||
);
|
||||
expect(result).toEqual([]);
|
||||
});
|
||||
|
||||
expect(plan.stats.unit_specs_to_run_rules).toBe(UNIT_FILES.length);
|
||||
expect(plan.stats.loki_stories_to_run_rules).toBe(0);
|
||||
it("should run all tests for runAllTests suites regardless of inputs", () => {
|
||||
expect(
|
||||
createTestPlanForSuite(
|
||||
moduleGraph,
|
||||
TEST_SUITE_DEFS.e2e,
|
||||
false,
|
||||
["docs/foo.md"],
|
||||
E2E_FILES,
|
||||
),
|
||||
).toEqual(E2E_FILES);
|
||||
expect(
|
||||
createTestPlanForSuite(
|
||||
moduleGraph,
|
||||
TEST_SUITE_DEFS.e2e,
|
||||
false,
|
||||
["src/foo/x.ts"],
|
||||
E2E_FILES,
|
||||
),
|
||||
).toEqual(E2E_FILES);
|
||||
});
|
||||
});
|
||||
|
||||
it("runs the full e2e suite when no coverage manifest is available", () => {
|
||||
const plan = createTestPlan({ ...baseInput, changedFiles: ["src/foo/x.ts"] });
|
||||
describe("createTestPlan", () => {
|
||||
it("should return run lists and stats together", () => {
|
||||
const result = createTestPlan({
|
||||
elements: ELEMENTS,
|
||||
rules: RULES,
|
||||
testSuites: TEST_SUITE_DEFS,
|
||||
changedFiles: ["src/foo/x.ts"],
|
||||
testFilesBySuite: {
|
||||
unit: UNIT_FILES,
|
||||
loki: LOKI_FILES,
|
||||
e2e: E2E_FILES,
|
||||
},
|
||||
infraTouchedBySuite: { unit: false, loki: false, e2e: false },
|
||||
});
|
||||
|
||||
expect(plan.stats.e2e_specs_to_run_rules).toBe(E2E_FILES.length);
|
||||
expect(plan.stats.e2e_specs_to_run_usage).toBe(E2E_FILES.length);
|
||||
expect(plan.e2e_specs_to_run).toEqual(E2E_FILES);
|
||||
});
|
||||
expect(result.fe_unit_specs_to_run.sort()).toEqual([
|
||||
"src/foo/bar.unit.spec.ts",
|
||||
"src/foo/foo.unit.spec.ts",
|
||||
]);
|
||||
expect(result.loki_stories_to_run).toEqual(["src/foo/Foo.stories.tsx"]);
|
||||
expect(result.e2e_specs_to_run).toEqual(E2E_FILES);
|
||||
|
||||
it("narrows e2e to specs covering affected feature modules", () => {
|
||||
const e2e = [
|
||||
"e2e/test/scenarios/foo.cy.spec.ts",
|
||||
"e2e/test/scenarios/bar.cy.spec.ts",
|
||||
"e2e/test/scenarios/new.cy.spec.ts",
|
||||
];
|
||||
const plan = createTestPlan({
|
||||
...baseInput,
|
||||
changedFiles: ["src/foo/x.ts"], // affects feature/foo only
|
||||
testFilesBySuite: { unit: UNIT_FILES, loki: LOKI_FILES, e2e },
|
||||
e2eSpecFiles: {
|
||||
"e2e/test/scenarios/foo.cy.spec.ts": ["src/foo/a.ts"], // feature/foo
|
||||
"e2e/test/scenarios/bar.cy.spec.ts": ["src/bar/b.ts"], // feature/bar
|
||||
// new.cy.spec.ts intentionally absent from the manifest
|
||||
},
|
||||
expect(result.stats).toEqual({
|
||||
modules_changed: 1,
|
||||
modules_affected: 1,
|
||||
modules_affected_list: ["feature/foo"],
|
||||
fe_unit_specs_total: 4,
|
||||
fe_unit_specs_run: 2,
|
||||
fe_unit_specs_skipped: 2,
|
||||
loki_stories_total: 2,
|
||||
loki_stories_run: 1,
|
||||
loki_stories_skipped: 1,
|
||||
e2e_specs_total: 1,
|
||||
e2e_specs_run: 1,
|
||||
e2e_specs_skipped: 0,
|
||||
});
|
||||
});
|
||||
|
||||
// foo's feature module is affected; bar's is not; the unmapped new spec
|
||||
// always runs.
|
||||
expect(plan.e2e_specs_to_run.sort()).toEqual([
|
||||
"e2e/test/scenarios/foo.cy.spec.ts",
|
||||
"e2e/test/scenarios/new.cy.spec.ts",
|
||||
]);
|
||||
expect(plan.stats.e2e_specs_all).toBe(3);
|
||||
});
|
||||
|
||||
it("runs a spec that maps to no feature module (unknown scope)", () => {
|
||||
const e2e = ["e2e/test/scenarios/lib-only.cy.spec.ts"];
|
||||
const plan = createTestPlan({
|
||||
...baseInput,
|
||||
changedFiles: ["src/foo/x.ts"], // affects feature/foo, NOT lib/utils
|
||||
testFilesBySuite: { unit: UNIT_FILES, loki: LOKI_FILES, e2e },
|
||||
e2eSpecFiles: {
|
||||
// Covers only a non-feature module (e.g. a home/auth spec whose feature
|
||||
// the baseline subtracted), so its feature scope is unknown.
|
||||
"e2e/test/scenarios/lib-only.cy.spec.ts": ["src/utils/colors.ts"],
|
||||
},
|
||||
it("should run all unit specs when unit infra is touched", () => {
|
||||
const result = createTestPlan({
|
||||
elements: ELEMENTS,
|
||||
rules: RULES,
|
||||
testSuites: TEST_SUITE_DEFS,
|
||||
changedFiles: ["docs/foo.md"],
|
||||
testFilesBySuite: {
|
||||
unit: UNIT_FILES,
|
||||
loki: LOKI_FILES,
|
||||
e2e: E2E_FILES,
|
||||
},
|
||||
infraTouchedBySuite: { unit: true, loki: false, e2e: false },
|
||||
});
|
||||
expect(result.stats.fe_unit_specs_run).toBe(UNIT_FILES.length);
|
||||
expect(result.stats.fe_unit_specs_skipped).toBe(0);
|
||||
// Only the unit suite was triggered.
|
||||
expect(result.stats.loki_stories_run).toBe(0);
|
||||
});
|
||||
|
||||
// No feature module to gate on -> can't bound scope -> always runs.
|
||||
expect(plan.e2e_specs_to_run).toEqual(e2e);
|
||||
it("should not run anything when no infra is touched and no modules affected", () => {
|
||||
const result = createTestPlan({
|
||||
elements: ELEMENTS,
|
||||
rules: RULES,
|
||||
testSuites: TEST_SUITE_DEFS,
|
||||
changedFiles: ["docs/foo.md"],
|
||||
testFilesBySuite: {
|
||||
unit: UNIT_FILES,
|
||||
loki: LOKI_FILES,
|
||||
e2e: E2E_FILES,
|
||||
},
|
||||
infraTouchedBySuite: { unit: false, loki: false, e2e: false },
|
||||
});
|
||||
expect(result.stats.fe_unit_specs_run).toBe(0);
|
||||
expect(result.stats.loki_stories_run).toBe(0);
|
||||
// e2e is runAllTests, so it still reports run = total.
|
||||
expect(result.stats.e2e_specs_run).toBe(E2E_FILES.length);
|
||||
});
|
||||
});
|
||||
|
||||
it("force-runs all e2e on a backend change (invisible to FE coverage)", () => {
|
||||
const e2e = ["e2e/test/scenarios/foo.cy.spec.ts"];
|
||||
const plan = createTestPlan({
|
||||
...baseInput,
|
||||
changedFiles: ["src/metabase/api/card.clj"],
|
||||
beFilesChanged: 1,
|
||||
testFilesBySuite: { unit: UNIT_FILES, loki: LOKI_FILES, e2e },
|
||||
e2eSpecFiles: { "e2e/test/scenarios/bar.cy.spec.ts": ["src/bar/b.ts"] },
|
||||
describe("filterAffectedTests", () => {
|
||||
it("should keep only tests inside affected modules", () => {
|
||||
const affected = new Set(["feature/foo", "lib/utils"]);
|
||||
const tests = [
|
||||
"src/foo/foo.unit.spec.ts",
|
||||
"src/utils/colors.unit.spec.ts",
|
||||
"src/bar/bar.unit.spec.ts",
|
||||
"docs/unrelated.unit.spec.ts",
|
||||
];
|
||||
expect(filterAffectedTests(moduleGraph, affected, tests)).toEqual([
|
||||
"src/foo/foo.unit.spec.ts",
|
||||
"src/utils/colors.unit.spec.ts",
|
||||
]);
|
||||
});
|
||||
|
||||
expect(plan.e2e_specs_to_run).toEqual(e2e);
|
||||
});
|
||||
|
||||
it("force-runs all e2e when e2e infra (support/runner) changes", () => {
|
||||
const e2e = ["e2e/test/scenarios/foo.cy.spec.ts"];
|
||||
const plan = createTestPlan({
|
||||
...baseInput,
|
||||
changedFiles: ["e2e/support/commands/api/createQuestion.js"],
|
||||
e2eInfraTouched: true,
|
||||
testFilesBySuite: { unit: UNIT_FILES, loki: LOKI_FILES, e2e },
|
||||
e2eSpecFiles: { "e2e/test/scenarios/bar.cy.spec.ts": ["src/bar/b.ts"] },
|
||||
});
|
||||
|
||||
expect(plan.e2e_specs_to_run).toEqual(e2e);
|
||||
});
|
||||
|
||||
it("runs an edited spec even when no app module changed", () => {
|
||||
const spec = "e2e/test/scenarios/foo.cy.spec.ts";
|
||||
const e2e = [spec, "e2e/test/scenarios/bar.cy.spec.ts"];
|
||||
const plan = createTestPlan({
|
||||
...baseInput,
|
||||
changedFiles: [spec], // only the spec itself changed
|
||||
testFilesBySuite: { unit: UNIT_FILES, loki: LOKI_FILES, e2e },
|
||||
e2eSpecFiles: {
|
||||
[spec]: ["src/bar/b.ts"], // covers feature/bar, which is NOT affected
|
||||
"e2e/test/scenarios/bar.cy.spec.ts": ["src/bar/b.ts"],
|
||||
},
|
||||
});
|
||||
|
||||
// No module is affected, but the edited spec must still run.
|
||||
expect(plan.e2e_specs_to_run).toEqual([spec]);
|
||||
});
|
||||
|
||||
it("runs a spec whose coverage was fully baseline-subtracted (empty files)", () => {
|
||||
const e2e = ["e2e/test/scenarios/empty.cy.spec.ts"];
|
||||
const plan = createTestPlan({
|
||||
...baseInput,
|
||||
changedFiles: ["src/foo/x.ts"], // affects feature/foo
|
||||
testFilesBySuite: { unit: UNIT_FILES, loki: LOKI_FILES, e2e },
|
||||
e2eSpecFiles: { "e2e/test/scenarios/empty.cy.spec.ts": [] },
|
||||
});
|
||||
|
||||
// Empty coverage means unknown scope, so the spec runs.
|
||||
expect(plan.e2e_specs_to_run).toEqual(e2e);
|
||||
});
|
||||
|
||||
it("forces a full e2e run when shared (cljc) sources change", () => {
|
||||
const e2e = ["e2e/test/scenarios/foo.cy.spec.ts"];
|
||||
const plan = createTestPlan({
|
||||
...baseInput,
|
||||
changedFiles: ["src/foo/x.ts"],
|
||||
sharedSourcesTouched: true,
|
||||
testFilesBySuite: { unit: UNIT_FILES, loki: LOKI_FILES, e2e },
|
||||
e2eSpecFiles: { "e2e/test/scenarios/bar.cy.spec.ts": ["src/bar/b.ts"] },
|
||||
});
|
||||
|
||||
// Manifest is present but cljc is invisible to JS coverage, so run all.
|
||||
expect(plan.e2e_specs_to_run).toEqual(e2e);
|
||||
});
|
||||
|
||||
it("passes the FE/BE file counts through to stats", () => {
|
||||
const plan = createTestPlan({
|
||||
...baseInput,
|
||||
changedFiles: [],
|
||||
feFilesChanged: 5,
|
||||
beFilesChanged: 3,
|
||||
});
|
||||
|
||||
expect(plan.stats.fe_files_changed).toBe(5);
|
||||
expect(plan.stats.be_files_changed).toBe(3);
|
||||
});
|
||||
});
|
||||
|
||||
describe("filterAffectedTests", () => {
|
||||
it("keeps only tests inside affected modules", () => {
|
||||
const nodes = buildNodes(ELEMENTS);
|
||||
const affected = new Set(["feature/foo", "lib/utils"]);
|
||||
const tests = [
|
||||
"src/foo/foo.unit.spec.ts",
|
||||
"src/utils/colors.unit.spec.ts",
|
||||
"src/bar/bar.unit.spec.ts",
|
||||
"docs/unrelated.unit.spec.ts",
|
||||
];
|
||||
expect(filterAffectedTests(nodes, affected, tests)).toEqual([
|
||||
"src/foo/foo.unit.spec.ts",
|
||||
"src/utils/colors.unit.spec.ts",
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
152
.github/scripts/bundle-size-stats-prepare.js
vendored
152
.github/scripts/bundle-size-stats-prepare.js
vendored
@@ -1,152 +0,0 @@
|
||||
/* eslint-disable import/no-commonjs */
|
||||
// Prepares the rows the bundle-size stats logger uploads to stats.metabase.com.
|
||||
//
|
||||
// For each measured (bundle, kind) it computes the delta — raw, gzipped and
|
||||
// brotli (as-served) bytes plus percent — against the previously *plotted* data
|
||||
// point (restored from a rolling cache), so the deltas can be charted directly.
|
||||
// It also decides whether this commit is worth recording at all: a point is kept
|
||||
// when the as-served size moved by at least MIN_DELTA_PERCENT, or when a new
|
||||
// bundle/kind appears. Only master builds run through here — releases form
|
||||
// non-linear lines whose deltas are meaningless, so they're not recorded (their
|
||||
// historical sizes are backfilled separately).
|
||||
//
|
||||
// buildStatsRows is pure (data in, data out) so it can be unit-tested; main()
|
||||
// is the thin I/O wrapper that reads env/files and writes the rows, cache and
|
||||
// step outputs.
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
|
||||
const deltaPercent = (current, base) =>
|
||||
base ? Math.round(((current - base) * 10000) / base) / 100 : "";
|
||||
|
||||
/**
|
||||
* Turn the current measurements + the last plotted point into upload rows, the
|
||||
* slim cache rows for the next diff, and the keep/skip decision.
|
||||
*
|
||||
* The significance threshold tracks the as-served size: brotli where the build
|
||||
* ships a precompressed .br, otherwise gzip. Current measurements always carry
|
||||
* brotliBytes (estimated when no .br shipped), but a cached base point recorded
|
||||
* before brotli logging may not, so the delta still falls back to gzip per row.
|
||||
*/
|
||||
function buildStatsRows({ measurements, previous, threshold, date, commit, commitMessage, version }) {
|
||||
const previousOf = (bundle, kind) => previous.find(row => row.bundle === bundle && row.kind === kind);
|
||||
|
||||
let maxServedDeltaPercent = 0;
|
||||
let hasNewSeries = false;
|
||||
|
||||
const rows = measurements.map(measurement => {
|
||||
const base = previousOf(measurement.bundle, measurement.kind);
|
||||
const hasBrotli = measurement.brotliBytes != null && base?.brotliBytes != null;
|
||||
const gzipDeltaPercent = deltaPercent(measurement.gzipBytes, base?.gzipBytes);
|
||||
const brotliDeltaPercent = hasBrotli ? deltaPercent(measurement.brotliBytes, base.brotliBytes) : "";
|
||||
if (!base) {
|
||||
hasNewSeries = true;
|
||||
} else {
|
||||
const servedDeltaPercent = hasBrotli ? brotliDeltaPercent : gzipDeltaPercent;
|
||||
maxServedDeltaPercent = Math.max(maxServedDeltaPercent, Math.abs(servedDeltaPercent));
|
||||
}
|
||||
return {
|
||||
"Date": date,
|
||||
"Version": version,
|
||||
"Commit": commit,
|
||||
"Bundle": measurement.bundle,
|
||||
"Kind": measurement.kind, // "initial" or "total"
|
||||
// The stats table carries a free-text Description column; append-csv
|
||||
// requires every table column to be present. Populate it with the commit
|
||||
// subject so the chart's points are self-describing.
|
||||
"Description": commitMessage,
|
||||
"Raw bytes": measurement.rawBytes,
|
||||
"Gzip bytes": measurement.gzipBytes,
|
||||
"Brotli bytes": measurement.brotliBytes, // as-served (shipped .br, else estimated)
|
||||
"File count": measurement.fileCount,
|
||||
"Raw bytes delta": base ? measurement.rawBytes - base.rawBytes : "",
|
||||
"Gzip bytes delta": base ? measurement.gzipBytes - base.gzipBytes : "",
|
||||
"Brotli bytes delta": hasBrotli ? measurement.brotliBytes - base.brotliBytes : "",
|
||||
"Raw delta %": deltaPercent(measurement.rawBytes, base?.rawBytes),
|
||||
"Gzip delta %": gzipDeltaPercent,
|
||||
"Brotli delta %": brotliDeltaPercent,
|
||||
};
|
||||
});
|
||||
|
||||
const firstPoint = previous.length === 0;
|
||||
const significant = firstPoint || hasNewSeries || maxServedDeltaPercent >= threshold;
|
||||
|
||||
// Slim rows the next run diffs against. The workflow only persists this to the
|
||||
// cache when we actually push, so the cached reference always stays the last
|
||||
// *plotted* point (cumulative drift is caught).
|
||||
const cacheRows = measurements.map(({ bundle, kind, rawBytes, gzipBytes, brotliBytes }) => ({
|
||||
bundle,
|
||||
kind,
|
||||
rawBytes,
|
||||
gzipBytes,
|
||||
brotliBytes,
|
||||
}));
|
||||
|
||||
const reason = firstPoint
|
||||
? "first point"
|
||||
: hasNewSeries
|
||||
? "new bundle/kind series"
|
||||
: `max served Δ ${maxServedDeltaPercent.toFixed(2)}% (threshold ${threshold}%)`;
|
||||
|
||||
return { rows, cacheRows, significant, reason, maxServedDeltaPercent, firstPoint, hasNewSeries };
|
||||
}
|
||||
|
||||
module.exports = { buildStatsRows };
|
||||
|
||||
const readJson = filePath => (filePath && fs.existsSync(filePath) ? JSON.parse(fs.readFileSync(filePath, "utf8")) : null);
|
||||
const writeJson = (filePath, value) => {
|
||||
fs.mkdirSync(path.dirname(filePath), { recursive: true });
|
||||
fs.writeFileSync(filePath, JSON.stringify(value));
|
||||
};
|
||||
const setOutput = (env, name, value) => {
|
||||
if (env.GITHUB_OUTPUT) {
|
||||
fs.appendFileSync(env.GITHUB_OUTPUT, `${name}=${value}\n`);
|
||||
}
|
||||
};
|
||||
|
||||
// version.properties (extracted from the uberjar) carries the build's tag.
|
||||
// Non-release (master) builds report tag=vUNKNOWN, so leave Version empty there.
|
||||
function readVersion(versionPropsPath) {
|
||||
const raw =
|
||||
versionPropsPath && fs.existsSync(versionPropsPath)
|
||||
? (fs.readFileSync(versionPropsPath, "utf8").match(/^tag=(.*)$/m)?.[1]?.trim() ?? "")
|
||||
: "";
|
||||
return raw === "vUNKNOWN" ? "" : raw;
|
||||
}
|
||||
|
||||
function main() {
|
||||
const env = process.env;
|
||||
|
||||
const measurements = readJson(env.CURRENT);
|
||||
if (!Array.isArray(measurements) || measurements.length === 0) {
|
||||
console.error(`::error::No bundle measurements found at ${env.CURRENT}`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const result = buildStatsRows({
|
||||
measurements,
|
||||
// The last plotted point, as slim rows: [{ bundle, kind, rawBytes, gzipBytes, brotliBytes }].
|
||||
previous: readJson(env.LAST) || [],
|
||||
threshold: Number(env.MIN_DELTA_PERCENT ?? 1),
|
||||
date: new Date().toISOString().slice(0, 10), // YYYY-MM-DD
|
||||
commit: (env.HEAD_SHA || "").slice(0, 12),
|
||||
commitMessage: (env.COMMIT_MESSAGE || "").split("\n")[0], // subject line only
|
||||
version: readVersion(env.VERSION_PROPS),
|
||||
});
|
||||
|
||||
writeJson(env.ROWS_OUT, result.rows);
|
||||
writeJson(env.CACHE_OUT, result.cacheRows);
|
||||
|
||||
console.log(`${result.significant ? "RECORD" : "SKIP"} — ${result.reason}`);
|
||||
|
||||
setOutput(env, "significant", result.significant ? "true" : "false");
|
||||
setOutput(
|
||||
env,
|
||||
"max_delta_percent",
|
||||
result.firstPoint || result.hasNewSeries ? "" : result.maxServedDeltaPercent.toFixed(2),
|
||||
);
|
||||
}
|
||||
|
||||
if (require.main === module) {
|
||||
main();
|
||||
}
|
||||
@@ -1,87 +0,0 @@
|
||||
const { buildStatsRows } = require("./bundle-size-stats-prepare");
|
||||
|
||||
const meta = { date: "2026-06-30", commit: "abc123", commitMessage: "Fix the thing (#123)", version: "" };
|
||||
const measurement = (over = {}) => ({
|
||||
bundle: "embedding-sdk-chunked",
|
||||
kind: "total",
|
||||
rawBytes: 1000,
|
||||
gzipBytes: 400,
|
||||
brotliBytes: 300,
|
||||
fileCount: 3,
|
||||
...over,
|
||||
});
|
||||
|
||||
describe("buildStatsRows", () => {
|
||||
it("records the first point and leaves every delta empty", () => {
|
||||
const result = buildStatsRows({
|
||||
...meta,
|
||||
measurements: [measurement()],
|
||||
previous: [],
|
||||
threshold: 1,
|
||||
});
|
||||
|
||||
expect(result.firstPoint).toBe(true);
|
||||
expect(result.significant).toBe(true);
|
||||
expect(result.reason).toBe("first point");
|
||||
expect(result.rows[0]["Gzip bytes delta"]).toBe("");
|
||||
expect(result.rows[0]["Brotli delta %"]).toBe("");
|
||||
expect(result.rows[0]["Description"]).toBe("Fix the thing (#123)");
|
||||
expect(result.cacheRows[0]).toEqual({
|
||||
bundle: "embedding-sdk-chunked",
|
||||
kind: "total",
|
||||
rawBytes: 1000,
|
||||
gzipBytes: 400,
|
||||
brotliBytes: 300,
|
||||
});
|
||||
});
|
||||
|
||||
it("records a never-seen bundle/kind as a new series", () => {
|
||||
const result = buildStatsRows({
|
||||
...meta,
|
||||
measurements: [measurement({ bundle: "app", kind: "initial" })],
|
||||
previous: [{ bundle: "app", kind: "total", rawBytes: 1, gzipBytes: 1, brotliBytes: 1 }],
|
||||
threshold: 99,
|
||||
});
|
||||
|
||||
expect(result.hasNewSeries).toBe(true);
|
||||
expect(result.significant).toBe(true);
|
||||
expect(result.reason).toBe("new bundle/kind series");
|
||||
});
|
||||
|
||||
it("gates significance on the brotli (as-served) delta when both sides have it", () => {
|
||||
const previous = [{ bundle: "embedding-sdk-chunked", kind: "total", rawBytes: 1000, gzipBytes: 400, brotliBytes: 300 }];
|
||||
|
||||
const big = buildStatsRows({
|
||||
...meta,
|
||||
measurements: [measurement({ brotliBytes: 315, gzipBytes: 400 })], // brotli +5%, gzip 0%
|
||||
previous,
|
||||
threshold: 1,
|
||||
});
|
||||
expect(big.maxServedDeltaPercent).toBeCloseTo(5);
|
||||
expect(big.significant).toBe(true);
|
||||
expect(big.rows[0]["Brotli delta %"]).toBeCloseTo(5);
|
||||
|
||||
const small = buildStatsRows({
|
||||
...meta,
|
||||
measurements: [measurement({ brotliBytes: 301, gzipBytes: 999 })], // brotli +0.33%, gzip ignored
|
||||
previous,
|
||||
threshold: 1,
|
||||
});
|
||||
expect(small.significant).toBe(false);
|
||||
expect(small.reason).toContain("threshold 1%");
|
||||
});
|
||||
|
||||
it("falls back to the gzip delta when the cached base predates brotli logging", () => {
|
||||
const result = buildStatsRows({
|
||||
...meta,
|
||||
measurements: [measurement({ gzipBytes: 440, brotliBytes: 300 })], // gzip +10%
|
||||
previous: [{ bundle: "embedding-sdk-chunked", kind: "total", rawBytes: 1000, gzipBytes: 400 }],
|
||||
threshold: 1,
|
||||
});
|
||||
|
||||
expect(result.maxServedDeltaPercent).toBeCloseTo(10);
|
||||
expect(result.significant).toBe(true);
|
||||
expect(result.rows[0]["Brotli bytes delta"]).toBe(""); // no base brotli to diff against
|
||||
expect(result.rows[0]["Gzip delta %"]).toBeCloseTo(10);
|
||||
});
|
||||
});
|
||||
127
.github/scripts/create-test-plan.ts
vendored
127
.github/scripts/create-test-plan.ts
vendored
@@ -1,112 +1,67 @@
|
||||
// I/O entrypoint: gather inputs (env vars, the cruise graph, the test-file
|
||||
// lists) and hand them to createTestPlan, which does the computing.
|
||||
// Changed files come from dorny/paths-filter's `all_changed_files` output
|
||||
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { existsSync, readFileSync } from "node:fs";
|
||||
|
||||
import micromatch from "micromatch";
|
||||
|
||||
import { elements, rules } from "../../frontend/lint/module-boundaries.mjs";
|
||||
|
||||
import { type FileDependency, parseCruiseModules } from "./affected-modules";
|
||||
import { createTestPlan } from "./affected-tests";
|
||||
import { listSpecFiles } from "./e2e-spec-globs.mjs";
|
||||
import { TEST_SUITES, createTestPlan } from "./affected-tests";
|
||||
|
||||
const UNIT_ROOTS = ["frontend/src", "enterprise/frontend/src"];
|
||||
const UNIT_GLOBS = [
|
||||
"frontend/src/**/*.unit.spec.{js,jsx,ts,tsx}",
|
||||
"enterprise/frontend/src/**/*.unit.spec.{js,jsx,ts,tsx}",
|
||||
"frontend/src/**/*.unit.spec.js",
|
||||
"frontend/src/**/*.unit.spec.jsx",
|
||||
"frontend/src/**/*.unit.spec.ts",
|
||||
"frontend/src/**/*.unit.spec.tsx",
|
||||
"enterprise/frontend/src/**/*.unit.spec.js",
|
||||
"enterprise/frontend/src/**/*.unit.spec.jsx",
|
||||
"enterprise/frontend/src/**/*.unit.spec.ts",
|
||||
"enterprise/frontend/src/**/*.unit.spec.tsx",
|
||||
];
|
||||
|
||||
const STORY_ROOTS = ["frontend", "enterprise/frontend"];
|
||||
const STORY_GLOBS = [
|
||||
"frontend/**/*.stories.{js,jsx,ts,tsx}",
|
||||
"enterprise/frontend/**/*.stories.{js,jsx,ts,tsx}",
|
||||
"frontend/**/*.stories.js",
|
||||
"frontend/**/*.stories.jsx",
|
||||
"frontend/**/*.stories.ts",
|
||||
"frontend/**/*.stories.tsx",
|
||||
"enterprise/frontend/**/*.stories.js",
|
||||
"enterprise/frontend/**/*.stories.jsx",
|
||||
"enterprise/frontend/**/*.stories.ts",
|
||||
"enterprise/frontend/**/*.stories.tsx",
|
||||
];
|
||||
|
||||
// Returns the tracked files under `roots` that match `globs`. The `dot: true`
|
||||
// option means files inside dot-directories such as `.storybook` are included.
|
||||
function listFiles(roots: string[], globs: string[]): string[] {
|
||||
const tracked = execFileSync("git", ["ls-files", "--", ...roots], {
|
||||
encoding: "utf8",
|
||||
})
|
||||
const E2E_GLOBS = [
|
||||
"e2e/test/scenarios/**/*.cy.spec.js",
|
||||
"e2e/test/scenarios/**/*.cy.spec.jsx",
|
||||
"e2e/test/scenarios/**/*.cy.spec.ts",
|
||||
"e2e/test/scenarios/**/*.cy.spec.tsx",
|
||||
];
|
||||
|
||||
function listFiles(globs: string[]): string[] {
|
||||
return execFileSync("git", ["ls-files", "--", ...globs], { encoding: "utf8" })
|
||||
.split("\n")
|
||||
.map((line) => line.trim())
|
||||
.filter(Boolean);
|
||||
return micromatch(tracked, globs, { dot: true });
|
||||
}
|
||||
|
||||
// dorny outputs comma-separated lists; CHANGED_FILES is `all_changed_files`.
|
||||
const csvToList = (csv: string | undefined) =>
|
||||
(csv ?? "")
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean);
|
||||
|
||||
// Reads the dependency-cruiser graph (DEP_GRAPH_JSON). Null falls back to the
|
||||
// rules graph, so a missing or unparseable file never breaks the plan.
|
||||
function readFileDependencies(): FileDependency[] | null {
|
||||
const path = process.env.DEP_GRAPH_JSON;
|
||||
if (path && existsSync(path)) {
|
||||
try {
|
||||
const { modules } = JSON.parse(readFileSync(path, "utf8"));
|
||||
process.stderr.write(`Using usage graph from ${path}.\n`);
|
||||
return parseCruiseModules(modules);
|
||||
} catch (error) {
|
||||
process.stderr.write(
|
||||
`Failed to read ${path}; falling back to rules graph: ${error}\n`,
|
||||
);
|
||||
}
|
||||
} else {
|
||||
process.stderr.write(
|
||||
"No DEP_GRAPH_JSON found; falling back to rules graph.\n",
|
||||
);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// Reads the nightly coverage manifest (E2E_SPEC_MANIFEST): { builtAt, specs:
|
||||
// { spec -> files } }. Returns the specs map, or null so e2e falls back to a
|
||||
// full run when the manifest is missing/unparseable.
|
||||
function readE2eSpecFiles(): Record<string, string[]> | null {
|
||||
const path = process.env.E2E_SPEC_MANIFEST;
|
||||
if (path && existsSync(path)) {
|
||||
try {
|
||||
const { specs } = JSON.parse(readFileSync(path, "utf8"));
|
||||
if (specs && typeof specs === "object") {
|
||||
process.stderr.write(`Using e2e coverage manifest from ${path}.\n`);
|
||||
return specs;
|
||||
}
|
||||
} catch (error) {
|
||||
process.stderr.write(
|
||||
`Failed to read ${path}; e2e will run in full: ${error}\n`,
|
||||
);
|
||||
}
|
||||
} else {
|
||||
process.stderr.write("No E2E_SPEC_MANIFEST found; e2e will run in full.\n");
|
||||
}
|
||||
return null;
|
||||
}
|
||||
const changedFiles = (process.env.CHANGED_FILES ?? "")
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean);
|
||||
|
||||
const testPlan = createTestPlan({
|
||||
elements,
|
||||
rules,
|
||||
changedFiles: csvToList(process.env.CHANGED_FILES),
|
||||
fileDependencies: readFileDependencies(),
|
||||
testSuites: TEST_SUITES,
|
||||
changedFiles,
|
||||
testFilesBySuite: {
|
||||
unit: listFiles(UNIT_ROOTS, UNIT_GLOBS),
|
||||
loki: listFiles(STORY_ROOTS, STORY_GLOBS),
|
||||
e2e: listSpecFiles(),
|
||||
unit: listFiles(UNIT_GLOBS),
|
||||
loki: listFiles(STORY_GLOBS),
|
||||
e2e: listFiles(E2E_GLOBS),
|
||||
},
|
||||
infraTouchedBySuite: {
|
||||
unit: process.env.UNIT_INFRA_TOUCHED === "true",
|
||||
loki: process.env.LOKI_INFRA_TOUCHED === "true",
|
||||
e2e: false,
|
||||
},
|
||||
e2eSpecFiles: readE2eSpecFiles(),
|
||||
unitInfraTouched: process.env.UNIT_INFRA_TOUCHED === "true",
|
||||
lokiInfraTouched: process.env.LOKI_INFRA_TOUCHED === "true",
|
||||
e2eInfraTouched: process.env.E2E_INFRA_TOUCHED === "true",
|
||||
sharedSourcesTouched: process.env.SHARED_SOURCES_TOUCHED === "true",
|
||||
feFilesChanged: csvToList(process.env.FE_CHANGED_FILES).length,
|
||||
beFilesChanged: csvToList(process.env.BE_CHANGED_FILES).length,
|
||||
feFilesTotal: listFiles(["frontend", "enterprise/frontend"], ["**"]).length,
|
||||
beFilesTotal: listFiles(["src", "enterprise/backend"], ["**"]).length,
|
||||
});
|
||||
|
||||
process.stdout.write(JSON.stringify(testPlan) + "\n");
|
||||
|
||||
78
.github/scripts/csv-to-mb.js
vendored
78
.github/scripts/csv-to-mb.js
vendored
@@ -19,73 +19,23 @@ function jsonToCsvFormData(jsonData) {
|
||||
return formData;
|
||||
}
|
||||
|
||||
// Transient failures worth retrying: throttling and server-side errors. A 4xx
|
||||
// (bad request, auth) won't succeed on retry, so those fail fast.
|
||||
const RETRYABLE_STATUS = new Set([429, 500, 502, 503, 504]);
|
||||
|
||||
class UploadError extends Error {
|
||||
constructor(message, { retryable }) {
|
||||
super(message);
|
||||
this.retryable = retryable;
|
||||
}
|
||||
}
|
||||
|
||||
async function attemptUpload(url, jsonData, timeoutMs) {
|
||||
let response;
|
||||
try {
|
||||
response = await fetch(url, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
"x-api-key": process.env.API_KEY
|
||||
},
|
||||
// Rebuilt per attempt: a consumed request body can't be reused.
|
||||
body: jsonToCsvFormData(jsonData),
|
||||
// Without this the request can hang indefinitely on a stalled server,
|
||||
// eating the whole job timeout. An abort surfaces below as a retryable
|
||||
// network error, so it retries with backoff instead.
|
||||
signal: AbortSignal.timeout(timeoutMs)
|
||||
});
|
||||
} catch (networkError) {
|
||||
throw new UploadError(networkError.message, { retryable: true });
|
||||
}
|
||||
|
||||
if (response.ok) {
|
||||
return;
|
||||
}
|
||||
|
||||
const body = await response.text().catch(() => '');
|
||||
throw new UploadError(
|
||||
`Upload failed: ${response.status} ${response.statusText} ${body}`.trim(),
|
||||
{ retryable: RETRYABLE_STATUS.has(response.status) }
|
||||
);
|
||||
}
|
||||
|
||||
const sleep = ms => new Promise(resolve => setTimeout(resolve, ms));
|
||||
|
||||
/**
|
||||
* Uploads jsonData as CSV, retrying transient API errors with exponential
|
||||
* backoff. `retries` is the number of extra attempts after the first (so the
|
||||
* default 2 means up to 3 tries). `timeoutMs` is a per-attempt total deadline
|
||||
* (default 60s) that aborts a stalled request. Throws the last error once
|
||||
* attempts are exhausted or the error is non-retryable.
|
||||
*/
|
||||
async function uploadCsvToMb({ baseUrl, tableId, jsonData, mode = 'append', retries = 2, retryDelayMs = 1000, timeoutMs = 60_000 }) {
|
||||
function uploadCsvToMb({ baseUrl, tableId, jsonData, mode = 'append' }) {
|
||||
const formData = jsonToCsvFormData(jsonData);
|
||||
const operation = mode === 'replace' ? 'replace-csv' : 'append-csv';
|
||||
const url = `${baseUrl}/api/table/${tableId}/${operation}`;
|
||||
|
||||
for (let attempt = 0; ; attempt++) {
|
||||
try {
|
||||
await attemptUpload(url, jsonData, timeoutMs);
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
if (!error.retryable || attempt >= retries) {
|
||||
throw error;
|
||||
}
|
||||
const delay = retryDelayMs * 2 ** attempt;
|
||||
console.warn(`Upload attempt ${attempt + 1} failed (${error.message}); retrying in ${delay}ms`);
|
||||
await sleep(delay);
|
||||
return fetch(`${baseUrl}/api/table/${tableId}/${operation}`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
"x-api-key": process.env.API_KEY
|
||||
},
|
||||
body: formData
|
||||
})
|
||||
.then(response => {
|
||||
if (!response.ok) {
|
||||
throw new Error(`Upload failed`);
|
||||
}
|
||||
}
|
||||
return { success: true };
|
||||
});
|
||||
}
|
||||
|
||||
module.exports = { uploadCsvToMb };
|
||||
|
||||
50
.github/scripts/csv-to-mb.unit.spec.js
vendored
50
.github/scripts/csv-to-mb.unit.spec.js
vendored
@@ -1,50 +0,0 @@
|
||||
const { uploadCsvToMb } = require("./csv-to-mb");
|
||||
|
||||
const ok = () => ({ ok: true });
|
||||
const fail = status => ({ ok: false, status, statusText: "err", text: async () => "" });
|
||||
|
||||
const upload = (over = {}) =>
|
||||
uploadCsvToMb({
|
||||
baseUrl: "https://stats.example.com",
|
||||
tableId: 1,
|
||||
jsonData: [{ a: 1 }],
|
||||
retryDelayMs: 1,
|
||||
...over,
|
||||
});
|
||||
|
||||
describe("uploadCsvToMb retries", () => {
|
||||
afterEach(() => {
|
||||
delete global.fetch;
|
||||
});
|
||||
|
||||
it("retries transient server errors then succeeds", async () => {
|
||||
global.fetch = jest.fn().mockResolvedValueOnce(fail(503)).mockResolvedValueOnce(fail(500)).mockResolvedValueOnce(ok());
|
||||
|
||||
await expect(upload({ retries: 2 })).resolves.toEqual({ success: true });
|
||||
expect(global.fetch).toHaveBeenCalledTimes(3);
|
||||
});
|
||||
|
||||
it("retries network errors", async () => {
|
||||
global.fetch = jest
|
||||
.fn()
|
||||
.mockRejectedValueOnce(new Error("ECONNRESET"))
|
||||
.mockResolvedValueOnce(ok());
|
||||
|
||||
await expect(upload({ retries: 2 })).resolves.toEqual({ success: true });
|
||||
expect(global.fetch).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it("gives up after exhausting retries and throws the last error", async () => {
|
||||
global.fetch = jest.fn().mockResolvedValue(fail(500));
|
||||
|
||||
await expect(upload({ retries: 2 })).rejects.toThrow("Upload failed: 500");
|
||||
expect(global.fetch).toHaveBeenCalledTimes(3); // 1 initial + 2 retries
|
||||
});
|
||||
|
||||
it("does not retry a non-transient 4xx", async () => {
|
||||
global.fetch = jest.fn().mockResolvedValue(fail(400));
|
||||
|
||||
await expect(upload({ retries: 2 })).rejects.toThrow("Upload failed: 400");
|
||||
expect(global.fetch).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
32
.github/scripts/e2e-spec-globs.mjs
vendored
32
.github/scripts/e2e-spec-globs.mjs
vendored
@@ -1,32 +0,0 @@
|
||||
import { execFileSync } from "node:child_process";
|
||||
|
||||
import micromatch from "micromatch";
|
||||
|
||||
const SPEC_DIR = "e2e/test/scenarios";
|
||||
|
||||
// This is the nightly-only baseline helper that runs in the instrumented pass to
|
||||
// capture boot-time coverage. It is not a product spec, so it is excluded from
|
||||
// the test plan universe and from the manifest backfill reconciliation.
|
||||
export const BASELINE_SPEC = `${SPEC_DIR}/coverage-baseline.cy.spec.js`;
|
||||
|
||||
// This glob set defines the spec "universe". The test planner
|
||||
// (create-test-plan.ts) selects which specs to run from it, and the coverage
|
||||
// manifest builder (build-coverage-manifest.mjs) reconciles against it when
|
||||
// backfilling specs that did not run.
|
||||
export const E2E_SPEC_GLOBS = [`${SPEC_DIR}/**/*.cy.spec.{js,jsx,ts,tsx}`];
|
||||
|
||||
// Returns the tracked e2e specs as repo-relative paths, excluding the baseline
|
||||
// helper.
|
||||
export function listSpecFiles(cwd = process.cwd()) {
|
||||
const tracked = execFileSync("git", ["ls-files", "--", SPEC_DIR], {
|
||||
cwd,
|
||||
encoding: "utf8",
|
||||
})
|
||||
.split("\n")
|
||||
.map((line) => line.trim())
|
||||
.filter(Boolean);
|
||||
return micromatch(tracked, E2E_SPEC_GLOBS, {
|
||||
ignore: [BASELINE_SPEC],
|
||||
dot: true,
|
||||
});
|
||||
}
|
||||
258
.github/scripts/measure-bundle-sizes.js
vendored
258
.github/scripts/measure-bundle-sizes.js
vendored
@@ -1,258 +0,0 @@
|
||||
/* eslint-disable import/no-commonjs */
|
||||
// Measures the front-end bundles emitted into an extracted uberjar tree.
|
||||
//
|
||||
// Every bundle reports one or both of:
|
||||
// - "initial": JS the browser downloads on first paint (the entry chunk plus
|
||||
// its initial/sync split chunks). Never excludes initial chunks.
|
||||
// - "total": all *reachable* JS (initial plus async chunks that are actually
|
||||
// loadable at runtime). Never includes unreachable/never-fetched chunks.
|
||||
// Reachability needs the per-entry asset list from the rspack stats file, so the
|
||||
// app and chunked SDK bundles require their stats file to be present.
|
||||
//
|
||||
// The pure pieces — the asset-name selection (initial vs reachable, runtime
|
||||
// filtering, the async-loadable flag) — are split out as select*Assets so they
|
||||
// can be unit-tested against a stats object. The measure* functions and main()
|
||||
// keep all the filesystem and env I/O, threading an explicit `config`.
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
const zlib = require("zlib");
|
||||
|
||||
const APP_DIST = "resources/frontend_client/app/dist";
|
||||
const SDK_ROOT = "resources/frontend_client/app/embedding-sdk";
|
||||
|
||||
// The chunk runtime is inlined into the bootstrap, never sent as a standalone file.
|
||||
const SDK_RUNTIME_RE = /embedding-sdk-chunk-runtime\./;
|
||||
const notRuntime = name => !SDK_RUNTIME_RE.test(name);
|
||||
|
||||
const assetFiles = (root, names) => names.map(name => path.join(root, name));
|
||||
const unique = files => [...new Set(files)];
|
||||
|
||||
// MEASURE_ROOT lets callers point at an extracted tree (e.g. the PR bundle-size
|
||||
// check measures a current and a base-ref tree); defaults to cwd for the stats
|
||||
// workflow. MEASURE_BUNDLES restricts which bundles are measured (comma-separated).
|
||||
// SDK_ASYNC_CHUNKS_LOADABLE (see selectChunkedSdkAssets) toggles whether async
|
||||
// SDK chunks count toward "total".
|
||||
function buildConfig(env) {
|
||||
const baseDir = env.MEASURE_ROOT ? path.resolve(env.MEASURE_ROOT) : process.cwd();
|
||||
const bundleFilter = env.MEASURE_BUNDLES
|
||||
? new Set(env.MEASURE_BUNDLES.split(",").map(bundle => bundle.trim()))
|
||||
: null;
|
||||
return {
|
||||
resolve: (...relativeParts) => path.resolve(baseDir, ...relativeParts),
|
||||
isBundleWanted: bundle => !bundleFilter || bundleFilter.has(bundle),
|
||||
// Async SDK chunks are reachable once output.publicPath is set so import()
|
||||
// chunks are actually fetched at runtime — which landed in master. Defaults
|
||||
// to true to match master; set SDK_ASYNC_CHUNKS_LOADABLE=false to measure an
|
||||
// older commit built before the publicPath fix (async unreachable, so
|
||||
// "total" collapses to "initial").
|
||||
sdkAsyncChunksLoadable: env.SDK_ASYNC_CHUNKS_LOADABLE !== "false",
|
||||
};
|
||||
}
|
||||
|
||||
function sizeOf(files) {
|
||||
let rawBytes = 0;
|
||||
let gzipBytes = 0;
|
||||
let brotliBytes = 0;
|
||||
for (const file of files) {
|
||||
const contents = fs.readFileSync(file);
|
||||
rawBytes += contents.length;
|
||||
// gzip: recompressed, a proxy for the on-the-fly gzip served on every version
|
||||
gzipBytes += zlib.gzipSync(contents).length;
|
||||
// brotli: as-served — count the precompressed .br the build ships; files
|
||||
// without one (static-viz, sub-threshold chunks) are served compressed on
|
||||
// the fly, so estimate with brotli. Always reported (like gzip).
|
||||
const brotliPath = `${file}.br`;
|
||||
brotliBytes += fs.existsSync(brotliPath)
|
||||
? fs.statSync(brotliPath).size
|
||||
: zlib.brotliCompressSync(contents).length;
|
||||
}
|
||||
return {
|
||||
fileCount: files.length,
|
||||
rawBytes,
|
||||
gzipBytes,
|
||||
brotliBytes,
|
||||
};
|
||||
}
|
||||
|
||||
function collectJsFiles(absolutePath) {
|
||||
if (fs.statSync(absolutePath).isFile()) {
|
||||
return [absolutePath];
|
||||
}
|
||||
return fs
|
||||
.readdirSync(absolutePath, { withFileTypes: true })
|
||||
.flatMap(entry => {
|
||||
const childPath = path.join(absolutePath, entry.name);
|
||||
if (entry.isDirectory()) {
|
||||
return collectJsFiles(childPath);
|
||||
}
|
||||
return entry.name.endsWith(".js") ? [childPath] : [];
|
||||
});
|
||||
}
|
||||
|
||||
function readStats(config, statsFile) {
|
||||
const statsPath = config.resolve(statsFile);
|
||||
return fs.existsSync(statsPath) ? JSON.parse(fs.readFileSync(statsPath, "utf8")) : null;
|
||||
}
|
||||
|
||||
// An entrypoint's .js assets for a given field: "assets" = its initial
|
||||
// (synchronous) load; "reachableAssets" = initial plus everything it can lazily
|
||||
// import (the chunk graph walked at build time in bundle-stats.js).
|
||||
function entrypointJsAssets(stats, name, field = "assets") {
|
||||
const entry = stats && stats.entrypoints && stats.entrypoints[name];
|
||||
if (!entry || !entry[field]) {
|
||||
return null;
|
||||
}
|
||||
return entry[field].map(asset => asset.name || asset).filter(assetName => assetName.endsWith(".js"));
|
||||
}
|
||||
|
||||
/**
|
||||
* Main app = the app-main entrypoint only (app/dist also holds other pages like
|
||||
* app-public, app-embed*, which are not the app). `reachableAssets` is null on
|
||||
* older stats built before the chunk-graph enrichment — the caller then falls
|
||||
* back to sizing the whole app/dist.
|
||||
*/
|
||||
function selectAppAssets(stats) {
|
||||
const initialAssets = entrypointJsAssets(stats, "app-main", "assets");
|
||||
if (!initialAssets) {
|
||||
throw new Error('app: stats-main.json / "app-main" entrypoint missing');
|
||||
}
|
||||
return { initialAssets, reachableAssets: entrypointJsAssets(stats, "app-main", "reachableAssets") };
|
||||
}
|
||||
|
||||
/**
|
||||
* Legacy SDK: the "embedding-sdk" entry, a single eager file in legacy/. It still
|
||||
* emits on-demand chunks (jspdf, echarts, leaflet, ...) into the shared chunks/
|
||||
* dir via dynamic import(). initial = the eager legacy file; total = initial plus
|
||||
* its reachable async chunks, but only when those are loadable at runtime
|
||||
* (sdkAsyncChunksLoadable), otherwise total collapses to initial. Returns null
|
||||
* when the stats lack the entry (older layout) so the caller can fall back.
|
||||
*/
|
||||
function selectLegacySdkAssets(stats, { sdkAsyncChunksLoadable }) {
|
||||
const initialNames = entrypointJsAssets(stats, "embedding-sdk", "assets");
|
||||
if (!initialNames) {
|
||||
return null;
|
||||
}
|
||||
const reachableNames = entrypointJsAssets(stats, "embedding-sdk", "reachableAssets") || initialNames;
|
||||
const initial = unique(initialNames).filter(notRuntime);
|
||||
return {
|
||||
initialNames: initial,
|
||||
reachableNames: sdkAsyncChunksLoadable ? unique(reachableNames).filter(notRuntime) : initial,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Chunked SDK delivery. initial = bootstrap (minus its inlined runtime) + the
|
||||
* chunked entry's initial chunks. total = the chunked entry's reachable set
|
||||
* (initial + async children, from the chunk graph). includesAsyncChunks is true
|
||||
* only when the stats carry the reachable chunk graph AND those chunks are
|
||||
* loadable at runtime (sdkAsyncChunksLoadable). A base ref built before the
|
||||
* reachableAssets enrichment has no graph, so its "total" must collapse to
|
||||
* "initial" — the flag lets the comparison avoid measuring a reachable total
|
||||
* against a collapsed one (an apples-to-oranges ~30% phantom).
|
||||
*/
|
||||
function selectChunkedSdkAssets(stats, { sdkAsyncChunksLoadable }) {
|
||||
const chunkedInitial = entrypointJsAssets(stats, "embedding-sdk-chunked", "assets");
|
||||
if (!chunkedInitial) {
|
||||
throw new Error('embedding-sdk-chunked: stats-embedding-sdk.json / "embedding-sdk-chunked" entrypoint missing');
|
||||
}
|
||||
const bootstrapInitial = entrypointJsAssets(stats, "embedding-sdk-bootstrap", "assets") || [];
|
||||
const chunkedReachableAssets = entrypointJsAssets(stats, "embedding-sdk-chunked", "reachableAssets");
|
||||
const chunkedReachable = chunkedReachableAssets || chunkedInitial;
|
||||
const bootstrapReachable = entrypointJsAssets(stats, "embedding-sdk-bootstrap", "reachableAssets") || bootstrapInitial;
|
||||
return {
|
||||
initialNames: unique([...bootstrapInitial, ...chunkedInitial]).filter(notRuntime),
|
||||
reachableNames: unique([...bootstrapReachable, ...chunkedReachable]).filter(notRuntime),
|
||||
includesAsyncChunks: chunkedReachableAssets != null && sdkAsyncChunksLoadable,
|
||||
};
|
||||
}
|
||||
|
||||
function measureApp(config) {
|
||||
const dist = config.resolve(APP_DIST);
|
||||
if (!fs.existsSync(dist)) {
|
||||
throw new Error(`Bundle output not found for "app": ${dist}`);
|
||||
}
|
||||
const { initialAssets, reachableAssets } = selectAppAssets(readStats(config, "artifacts/stats-main.json"));
|
||||
if (!reachableAssets) {
|
||||
console.warn("app: stats has no reachableAssets; app total falls back to the whole app/dist");
|
||||
}
|
||||
const totalFiles = reachableAssets ? assetFiles(dist, reachableAssets) : collectJsFiles(dist);
|
||||
return [
|
||||
{ bundle: "app", kind: "initial", ...sizeOf(assetFiles(dist, initialAssets)) },
|
||||
{ bundle: "app", kind: "total", ...sizeOf(totalFiles) },
|
||||
];
|
||||
}
|
||||
|
||||
function measureLegacySdk(config) {
|
||||
const absRoot = config.resolve(SDK_ROOT);
|
||||
const selected = selectLegacySdkAssets(readStats(config, "artifacts/stats-embedding-sdk.json"), config);
|
||||
if (!selected) {
|
||||
// No stats / older layout: fall back to whatever sits in the legacy dir.
|
||||
const candidates = [`${SDK_ROOT}/legacy`, `${SDK_ROOT}.js`];
|
||||
const chosen = candidates.find(candidate => fs.existsSync(config.resolve(candidate)));
|
||||
if (!chosen) {
|
||||
console.warn(`Skipping "embedding-sdk-legacy": output not found (${candidates.join(", ")})`);
|
||||
return [];
|
||||
}
|
||||
return [{ bundle: "embedding-sdk-legacy", kind: "total", ...sizeOf(collectJsFiles(config.resolve(chosen))) }];
|
||||
}
|
||||
return [
|
||||
{ bundle: "embedding-sdk-legacy", kind: "initial", ...sizeOf(assetFiles(absRoot, selected.initialNames)) },
|
||||
{ bundle: "embedding-sdk-legacy", kind: "total", ...sizeOf(assetFiles(absRoot, selected.reachableNames)) },
|
||||
];
|
||||
}
|
||||
|
||||
function measureChunkedSdk(config) {
|
||||
const chunksDir = config.resolve(SDK_ROOT, "chunks");
|
||||
if (!fs.existsSync(chunksDir)) {
|
||||
// pre-chunked layout (monolith only) — nothing to measure here
|
||||
return [];
|
||||
}
|
||||
const absRoot = config.resolve(SDK_ROOT);
|
||||
const { initialNames, reachableNames, includesAsyncChunks } = selectChunkedSdkAssets(
|
||||
readStats(config, "artifacts/stats-embedding-sdk.json"),
|
||||
config,
|
||||
);
|
||||
const initial = sizeOf(assetFiles(absRoot, initialNames));
|
||||
const total = includesAsyncChunks ? sizeOf(assetFiles(absRoot, reachableNames)) : initial;
|
||||
return [
|
||||
{ bundle: "embedding-sdk-chunked", kind: "initial", ...initial },
|
||||
{ bundle: "embedding-sdk-chunked", kind: "total", reachable: includesAsyncChunks, ...total },
|
||||
];
|
||||
}
|
||||
|
||||
// Single-file/dir bundles loaded in one shot — total only.
|
||||
function measureWholeDir(config, bundle, candidates) {
|
||||
const chosen = candidates.find(candidate => fs.existsSync(config.resolve(candidate)));
|
||||
if (!chosen) {
|
||||
console.warn(`Skipping "${bundle}": output not found (${candidates.join(", ")})`);
|
||||
return [];
|
||||
}
|
||||
return [{ bundle, kind: "total", ...sizeOf(collectJsFiles(config.resolve(chosen))) }];
|
||||
}
|
||||
|
||||
const bundles = {
|
||||
app: measureApp,
|
||||
"embedding-sdk-legacy": measureLegacySdk,
|
||||
"embedding-sdk-chunked": measureChunkedSdk,
|
||||
"embedding-sdk-package": config => measureWholeDir(config, "embedding-sdk-package", ["resources/embedding-sdk/dist"]),
|
||||
"embed-js": config => measureWholeDir(config, "embed-js", ["resources/frontend_client/app/embed.js"]),
|
||||
};
|
||||
|
||||
function measureBundleSizes(config) {
|
||||
return Object.entries(bundles)
|
||||
.filter(([bundle]) => config.isBundleWanted(bundle))
|
||||
.flatMap(([, measure]) => measure(config));
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
measureBundleSizes,
|
||||
buildConfig,
|
||||
entrypointJsAssets,
|
||||
selectAppAssets,
|
||||
selectLegacySdkAssets,
|
||||
selectChunkedSdkAssets,
|
||||
};
|
||||
|
||||
if (require.main === module) {
|
||||
console.log(JSON.stringify(measureBundleSizes(buildConfig(process.env)), null, 2));
|
||||
}
|
||||
@@ -1,87 +0,0 @@
|
||||
const {
|
||||
entrypointJsAssets,
|
||||
selectAppAssets,
|
||||
selectLegacySdkAssets,
|
||||
selectChunkedSdkAssets,
|
||||
} = require("./measure-bundle-sizes");
|
||||
|
||||
const RUNTIME = "embedding-sdk-chunk-runtime.deadbeef.js";
|
||||
|
||||
const stats = entrypoints => ({ entrypoints });
|
||||
|
||||
describe("entrypointJsAssets", () => {
|
||||
it("returns the .js asset names for a field, accepting strings or {name}", () => {
|
||||
const s = stats({ app: { assets: [{ name: "a.js" }, "b.js", { name: "c.css" }] } });
|
||||
expect(entrypointJsAssets(s, "app", "assets")).toEqual(["a.js", "b.js"]);
|
||||
});
|
||||
|
||||
it("returns null when the entrypoint or field is absent", () => {
|
||||
expect(entrypointJsAssets(stats({}), "missing")).toBeNull();
|
||||
expect(entrypointJsAssets(null, "app")).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("selectAppAssets", () => {
|
||||
it("returns the initial set and a null reachable set on pre-enrichment stats", () => {
|
||||
const s = stats({ "app-main": { assets: ["main.js"] } });
|
||||
expect(selectAppAssets(s)).toEqual({ initialAssets: ["main.js"], reachableAssets: null });
|
||||
});
|
||||
|
||||
it("throws when the app-main entrypoint is missing", () => {
|
||||
expect(() => selectAppAssets(stats({}))).toThrow("app-main");
|
||||
});
|
||||
});
|
||||
|
||||
describe("selectChunkedSdkAssets", () => {
|
||||
const base = {
|
||||
"embedding-sdk-chunked": { assets: ["a.js", RUNTIME], reachableAssets: ["a.js", "b.js", RUNTIME] },
|
||||
"embedding-sdk-bootstrap": { assets: ["boot.js"], reachableAssets: ["boot.js"] },
|
||||
};
|
||||
|
||||
it("merges bootstrap + chunked, drops the inlined runtime, and counts async chunks", () => {
|
||||
const result = selectChunkedSdkAssets(stats(base), { sdkAsyncChunksLoadable: true });
|
||||
expect(result.initialNames).toEqual(["boot.js", "a.js"]);
|
||||
expect(result.reachableNames).toEqual(["boot.js", "a.js", "b.js"]);
|
||||
expect(result.includesAsyncChunks).toBe(true);
|
||||
});
|
||||
|
||||
it("collapses total to initial when async chunks are not loadable", () => {
|
||||
const result = selectChunkedSdkAssets(stats(base), { sdkAsyncChunksLoadable: false });
|
||||
expect(result.includesAsyncChunks).toBe(false);
|
||||
});
|
||||
|
||||
it("collapses total to initial when the stats carry no reachable graph", () => {
|
||||
const noGraph = { "embedding-sdk-chunked": { assets: ["a.js"] } };
|
||||
const result = selectChunkedSdkAssets(stats(noGraph), { sdkAsyncChunksLoadable: true });
|
||||
expect(result.reachableNames).toEqual(["a.js"]);
|
||||
expect(result.includesAsyncChunks).toBe(false);
|
||||
});
|
||||
|
||||
it("throws when the chunked entrypoint is missing", () => {
|
||||
expect(() => selectChunkedSdkAssets(stats({}), { sdkAsyncChunksLoadable: true })).toThrow(
|
||||
"embedding-sdk-chunked",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("selectLegacySdkAssets", () => {
|
||||
it("returns null when the legacy entry is absent so the caller can fall back", () => {
|
||||
expect(selectLegacySdkAssets(stats({}), { sdkAsyncChunksLoadable: true })).toBeNull();
|
||||
});
|
||||
|
||||
it("expands total to the reachable async set when loadable", () => {
|
||||
const s = stats({ "embedding-sdk": { assets: ["legacy.js"], reachableAssets: ["legacy.js", "chunk.js"] } });
|
||||
expect(selectLegacySdkAssets(s, { sdkAsyncChunksLoadable: true })).toEqual({
|
||||
initialNames: ["legacy.js"],
|
||||
reachableNames: ["legacy.js", "chunk.js"],
|
||||
});
|
||||
});
|
||||
|
||||
it("collapses total to initial when async chunks are not loadable", () => {
|
||||
const s = stats({ "embedding-sdk": { assets: ["legacy.js"], reachableAssets: ["legacy.js", "chunk.js"] } });
|
||||
expect(selectLegacySdkAssets(s, { sdkAsyncChunksLoadable: false })).toEqual({
|
||||
initialNames: ["legacy.js"],
|
||||
reachableNames: ["legacy.js"],
|
||||
});
|
||||
});
|
||||
});
|
||||
123
.github/scripts/module-usage.ts
vendored
123
.github/scripts/module-usage.ts
vendored
@@ -1,123 +0,0 @@
|
||||
// Builds module-level "usage" maps: for each module, the set of modules that
|
||||
// depend on it, both directly and indirectly
|
||||
//
|
||||
// Both are computed at the FILE level off the real dependency-cruiser import
|
||||
// graph, then collapsed to module granularity at the very end.
|
||||
//
|
||||
// bunx depcruise frontend/src enterprise/frontend/src \
|
||||
// --config .dependency-cruiser.cjs --output-type json > dependency-graph.json
|
||||
// DEP_GRAPH_JSON=dependency-graph.json \
|
||||
// OUT_JSON=usage.json OUT_DIRECT_JSON=usage-direct.json \
|
||||
// bun .github/scripts/module-usage.ts
|
||||
|
||||
import { readFileSync, writeFileSync } from "node:fs";
|
||||
|
||||
import { elements } from "../../frontend/lint/module-boundaries.mjs";
|
||||
|
||||
import {
|
||||
buildFileGraph,
|
||||
getAffectedFiles,
|
||||
mapFileToModule,
|
||||
parseCruiseModules,
|
||||
} from "./affected-modules";
|
||||
|
||||
const depGraphPath = process.env.DEP_GRAPH_JSON ?? "dependency-graph.json";
|
||||
const { modules } = JSON.parse(readFileSync(depGraphPath, "utf8"));
|
||||
|
||||
const fileDependencies = parseCruiseModules(modules);
|
||||
const fileGraph = buildFileGraph(elements, fileDependencies);
|
||||
|
||||
// Group every known source file by its owning module. We seed from the cruise
|
||||
// output's `source` files plus everything that appears as a dependency target,
|
||||
// so files imported but never importing are still covered.
|
||||
const filesByModule = new Map<string, string[]>();
|
||||
const allFiles = new Set<string>();
|
||||
for (const { source, dependencies } of fileDependencies) {
|
||||
allFiles.add(source);
|
||||
for (const dep of dependencies) {
|
||||
allFiles.add(dep);
|
||||
}
|
||||
}
|
||||
for (const file of allFiles) {
|
||||
const module = mapFileToModule(fileGraph.nodes, file);
|
||||
if (!module) {
|
||||
continue;
|
||||
}
|
||||
const list = filesByModule.get(module);
|
||||
if (list) {
|
||||
list.push(file);
|
||||
} else {
|
||||
filesByModule.set(module, [file]);
|
||||
}
|
||||
}
|
||||
|
||||
// Ensure every module appears as a key, even those nothing depends on.
|
||||
const emptyDependents = (): Record<string, Set<string>> => {
|
||||
const map: Record<string, Set<string>> = {};
|
||||
for (const module of filesByModule.keys()) {
|
||||
map[module] = new Set();
|
||||
}
|
||||
return map;
|
||||
};
|
||||
|
||||
// DIRECT usage: module B directly depends on module A if any file in B imports
|
||||
// any file in A — a single edge in the import graph, collapsed to modules.
|
||||
const directDependents = emptyDependents();
|
||||
for (const { source, dependencies } of fileDependencies) {
|
||||
const importer = mapFileToModule(fileGraph.nodes, source);
|
||||
if (!importer) {
|
||||
continue;
|
||||
}
|
||||
for (const target of dependencies) {
|
||||
const owner = mapFileToModule(fileGraph.nodes, target);
|
||||
if (owner && owner !== importer) {
|
||||
directDependents[owner].add(importer);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TRANSITIVE usage: for each module, take all its files, walk the file-level
|
||||
// reverse graph to get every file that transitively imports them, collapse to
|
||||
// modules, and drop the module itself.
|
||||
const transitiveDependents = emptyDependents();
|
||||
for (const [module, files] of filesByModule) {
|
||||
const affectedFiles = getAffectedFiles(fileGraph, files);
|
||||
for (const file of affectedFiles) {
|
||||
const owner = mapFileToModule(fileGraph.nodes, file);
|
||||
if (owner && owner !== module) {
|
||||
transitiveDependents[module].add(owner);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Stable, sorted output for clean diffs.
|
||||
const toSorted = (
|
||||
map: Record<string, Set<string>>,
|
||||
): Record<string, string[]> => {
|
||||
const sorted: Record<string, string[]> = {};
|
||||
for (const module of Object.keys(map).sort()) {
|
||||
sorted[module] = [...map[module]].sort();
|
||||
}
|
||||
return sorted;
|
||||
};
|
||||
|
||||
const writeOut = (
|
||||
map: Record<string, Set<string>>,
|
||||
outVar: string,
|
||||
label: string,
|
||||
) => {
|
||||
const sorted = toSorted(map);
|
||||
const json = JSON.stringify(sorted, null, 2) + "\n";
|
||||
const outPath = process.env[outVar];
|
||||
if (outPath) {
|
||||
writeFileSync(outPath, json);
|
||||
process.stderr.write(
|
||||
`Wrote ${Object.keys(sorted).length} modules (${label}) to ${outPath}.\n`,
|
||||
);
|
||||
} else {
|
||||
process.stdout.write(json);
|
||||
}
|
||||
};
|
||||
|
||||
writeOut(transitiveDependents, "OUT_JSON", "transitive");
|
||||
writeOut(directDependents, "OUT_DIRECT_JSON", "direct");
|
||||
94
.github/scripts/resolve-coverage-manifest.mjs
vendored
94
.github/scripts/resolve-coverage-manifest.mjs
vendored
@@ -1,94 +0,0 @@
|
||||
/**
|
||||
* Resolves the nightly e2e coverage manifest closest to a commit's point in
|
||||
* history, so the test planner can narrow the e2e run.
|
||||
*
|
||||
* The whole flow lives here so callers "just get a manifest": list the retained
|
||||
* manifest artifacts, find the one whose nightly commit is the nearest ancestor
|
||||
* of `headSha` (via the GitHub compare API — no deep clone), download and unzip
|
||||
* just that one, and return its parsed contents.
|
||||
*
|
||||
* `github` is an authenticated Octokit, as injected by actions/github-script.
|
||||
* Returns { builtAt, specs, path, behindBy } or null when nothing is an
|
||||
* ancestor (caller then runs e2e in full). Never throws: any API/IO failure is
|
||||
* logged and resolves to null, because coverage resolution must not break the
|
||||
* test plan.
|
||||
*/
|
||||
|
||||
import { execFileSync } from "node:child_process";
|
||||
import fs from "node:fs";
|
||||
|
||||
const ARTIFACT_NAME = "spec-file-manifest";
|
||||
const EXTRACT_DIR = "e2e/coverage";
|
||||
const MANIFEST_PATH = `${EXTRACT_DIR}/${ARTIFACT_NAME}.json`;
|
||||
|
||||
export async function resolveCoverageManifest({
|
||||
github,
|
||||
owner,
|
||||
repo,
|
||||
headSha,
|
||||
log = console.log,
|
||||
}) {
|
||||
try {
|
||||
// Every retained manifest artifact, newest first. Each carries its
|
||||
// nightly's head_sha — the commit it was built from (== builtAt).
|
||||
const artifacts = await github.paginate(
|
||||
github.rest.actions.listArtifactsForRepo,
|
||||
{ owner, repo, name: ARTIFACT_NAME, per_page: 100 },
|
||||
);
|
||||
const candidates = artifacts
|
||||
.filter((artifact) => !artifact.expired && artifact.workflow_run?.head_sha)
|
||||
.sort(
|
||||
(a, b) =>
|
||||
new Date(b.created_at).getTime() - new Date(a.created_at).getTime(),
|
||||
);
|
||||
|
||||
// First candidate that is an ancestor of head is the closest in history
|
||||
// (master is linear, candidates are chronological). compare base...head:
|
||||
// behind_by === 0 ⇒ base is an ancestor of head; ahead_by is the distance.
|
||||
let chosen = null;
|
||||
for (const artifact of candidates) {
|
||||
const base = artifact.workflow_run.head_sha;
|
||||
// A stale candidate (force-pushed/GC'd commit) makes compare 404 — skip
|
||||
// it rather than lose the optimization for everyone.
|
||||
let comparison;
|
||||
try {
|
||||
({ data: comparison } =
|
||||
await github.rest.repos.compareCommitsWithBasehead({
|
||||
owner,
|
||||
repo,
|
||||
basehead: `${base}...${headSha}`,
|
||||
}));
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (comparison.behind_by === 0) {
|
||||
chosen = { artifact, base, behindBy: comparison.ahead_by };
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (!chosen) {
|
||||
log("No ancestor coverage manifest found; e2e will run in full.");
|
||||
return null;
|
||||
}
|
||||
|
||||
const download = await github.rest.actions.downloadArtifact({
|
||||
owner,
|
||||
repo,
|
||||
artifact_id: chosen.artifact.id,
|
||||
archive_format: "zip",
|
||||
});
|
||||
fs.writeFileSync(`${ARTIFACT_NAME}.zip`, Buffer.from(download.data));
|
||||
execFileSync("unzip", ["-o", `${ARTIFACT_NAME}.zip`, "-d", EXTRACT_DIR]);
|
||||
|
||||
const manifest = JSON.parse(fs.readFileSync(MANIFEST_PATH, "utf8"));
|
||||
log(
|
||||
`Using coverage manifest ${chosen.base.slice(0, 12)} ` +
|
||||
`(${chosen.behindBy} commits behind HEAD).`,
|
||||
);
|
||||
return { ...manifest, path: MANIFEST_PATH, behindBy: chosen.behindBy };
|
||||
} catch (error) {
|
||||
log(`Coverage manifest resolution failed; e2e will run in full: ${error}`);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
3
.github/scripts/setup-python-runner.sh
vendored
3
.github/scripts/setup-python-runner.sh
vendored
@@ -19,8 +19,7 @@ while [[ $# -gt 0 ]]; do
|
||||
done
|
||||
|
||||
echo "Cloning python-runner repository..."
|
||||
git clone --depth 1 https://${METABASE_AUTOMATION_USER_TOKEN}@github.com/metabase/python-runner-container.git --branch seccomp python-runner
|
||||
|
||||
git clone --depth 1 https://${METABASE_AUTOMATION_USER_TOKEN}@github.com/metabase/python-runner-container.git python-runner
|
||||
|
||||
if [ "$SKIP_START" = false ]; then
|
||||
echo "Starting python-runner..."
|
||||
|
||||
93
.github/scripts/upload-affected-tests-stats.js
vendored
93
.github/scripts/upload-affected-tests-stats.js
vendored
@@ -1,64 +1,41 @@
|
||||
// Appends the test-plan stats row produced by create-test-plan.ts to the
|
||||
// "FE Affected Tests" table on stats.metabase.com.
|
||||
// Reads STATS_JSON, TRIGGERED_BY, PR_NUMBER, HEAD_SHA, BASE_SHA, API_KEY from env.
|
||||
// This script is used in .github/workflows/run-tests.yml.
|
||||
// It uploads the test-plan stats row produced by create-test-plan.js to
|
||||
// the stats.metabase.com tracking table.
|
||||
// Reads STATS_JSON, PR_NUMBER, HEAD_SHA, BASE_SHA, API_KEY from env.
|
||||
|
||||
const { uploadCsvToMb } = require("./csv-to-mb.js");
|
||||
|
||||
// "FE Affected Tests" uploaded table on stats.metabase.com.
|
||||
const TABLE_ID = 81818;
|
||||
const stats = JSON.parse(process.env.STATS_JSON);
|
||||
console.log("stats", stats);
|
||||
|
||||
async function main() {
|
||||
// create-test-plan doesn't always produce stats (e.g. on release branches
|
||||
// running an older planner), leaving STATS_JSON empty. Nothing to upload.
|
||||
const raw = (process.env.STATS_JSON || "").trim();
|
||||
if (!raw || raw === "null") {
|
||||
console.log("No STATS_JSON to upload; skipping.");
|
||||
return;
|
||||
}
|
||||
const row = {
|
||||
Date: new Date().toISOString(),
|
||||
PR: Number(process.env.PR_NUMBER),
|
||||
"Head SHA": process.env.HEAD_SHA,
|
||||
"Base SHA": process.env.BASE_SHA,
|
||||
"Modules changed": stats.modules_changed,
|
||||
"Modules affected": stats.modules_affected,
|
||||
"FE Unit specs total": stats.fe_unit_specs_total,
|
||||
"FE Unit specs run": stats.fe_unit_specs_run,
|
||||
"FE Unit specs skipped": stats.fe_unit_specs_skipped,
|
||||
"Loki stories total": stats.loki_stories_total,
|
||||
"Loki stories run": stats.loki_stories_run,
|
||||
"Loki stories skipped": stats.loki_stories_skipped,
|
||||
"E2E specs total": stats.e2e_specs_total,
|
||||
"E2E specs run": stats.e2e_specs_run,
|
||||
"E2E specs skipped": stats.e2e_specs_skipped,
|
||||
};
|
||||
|
||||
const s = JSON.parse(raw);
|
||||
console.log("stats", s);
|
||||
|
||||
const row = {
|
||||
Date: new Date().toISOString(),
|
||||
// "pr_update" or "merge_to_master". ("Trigger" is a reserved SQL word.)
|
||||
"Triggered By": process.env.TRIGGERED_BY,
|
||||
PR: Number(process.env.PR_NUMBER),
|
||||
"Head SHA": process.env.HEAD_SHA,
|
||||
"Base SHA": process.env.BASE_SHA,
|
||||
"FE Files Changed": s.fe_files_changed,
|
||||
"FE Files Total": s.fe_files_total,
|
||||
"BE Files Changed": s.be_files_changed,
|
||||
"BE Files Total": s.be_files_total,
|
||||
"Unit Infra Touched": s.unit_infra_touched,
|
||||
"Loki Infra Touched": s.loki_infra_touched,
|
||||
"Shared Sources Touched": s.shared_sources_touched,
|
||||
"FE Modules Total": s.fe_modules_total,
|
||||
"FE Modules Changed": s.fe_modules_changed,
|
||||
"FE Modules Affected (Rules)": s.fe_modules_affected_rules,
|
||||
"FE Modules Affected (Usage)": s.fe_modules_affected_usage,
|
||||
"Unit Specs All": s.unit_specs_all,
|
||||
"Unit Specs To Run (Rules)": s.unit_specs_to_run_rules,
|
||||
"Unit Specs To Run (Usage)": s.unit_specs_to_run_usage,
|
||||
"Loki Stories All": s.loki_stories_all,
|
||||
"Loki Stories To Run (Rules)": s.loki_stories_to_run_rules,
|
||||
"Loki Stories To Run (Usage)": s.loki_stories_to_run_usage,
|
||||
"E2E Specs All": s.e2e_specs_all,
|
||||
"E2E Specs To Run (Rules)": s.e2e_specs_to_run_rules,
|
||||
"E2E Specs To Run (Usage)": s.e2e_specs_to_run_usage,
|
||||
};
|
||||
|
||||
await uploadCsvToMb({
|
||||
baseUrl: "https://stats.metabase.com",
|
||||
tableId: TABLE_ID,
|
||||
jsonData: [row],
|
||||
mode: "append",
|
||||
uploadCsvToMb({
|
||||
baseUrl: "https://stats.metabase.com",
|
||||
tableId: 77008, // "Affected Tests Stats" table on stats.metabase.com
|
||||
jsonData: [row],
|
||||
mode: "append",
|
||||
})
|
||||
.then(() => {
|
||||
console.log("Data uploaded successfully");
|
||||
})
|
||||
.catch((error) => {
|
||||
console.error("Error uploading data:", error);
|
||||
process.exit(1);
|
||||
});
|
||||
console.log("Data uploaded successfully");
|
||||
}
|
||||
|
||||
// Best-effort: never let a telemetry failure turn the job red.
|
||||
main().catch((error) => {
|
||||
console.error("Skipping stats upload after error:", error);
|
||||
process.exit(0);
|
||||
});
|
||||
|
||||
12
.github/team.json
vendored
12
.github/team.json
vendored
@@ -4,6 +4,7 @@
|
||||
"name": "Querying Platform",
|
||||
"label": ".Team/QueryingPlatform",
|
||||
"members": [
|
||||
"bshepherdson",
|
||||
"metamben",
|
||||
"rileythomp",
|
||||
"camsaul",
|
||||
@@ -47,8 +48,7 @@
|
||||
"members": [
|
||||
"fraserdrops",
|
||||
"nemanjaglumac",
|
||||
"romeovs",
|
||||
"ChrisVandoo"
|
||||
"romeovs"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -89,12 +89,11 @@
|
||||
"npfitz",
|
||||
"johnswanson",
|
||||
"nvoxland",
|
||||
"bshepherdson",
|
||||
"edpaget",
|
||||
"deniskaber",
|
||||
"andsalves",
|
||||
"kevinkelleher12",
|
||||
"AndreyChernykh",
|
||||
"bowbahdoe"
|
||||
"AndreyChernykh"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -121,8 +120,7 @@
|
||||
"iethree",
|
||||
"bpander",
|
||||
"gantoreno",
|
||||
"losrebellos",
|
||||
"tmashara"
|
||||
"losrebellos"
|
||||
]
|
||||
}
|
||||
]
|
||||
|
||||
42
.github/workflows/actionlint.yml
vendored
42
.github/workflows/actionlint.yml
vendored
@@ -1,42 +0,0 @@
|
||||
name: Actionlint
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'release-**'
|
||||
paths:
|
||||
- '.github/workflows/**'
|
||||
pull_request:
|
||||
paths:
|
||||
- '.github/workflows/**'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
actionlint:
|
||||
name: Lint Github Actions Workflows
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Get changed Github actions
|
||||
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
|
||||
id: changes
|
||||
with:
|
||||
token: ${{ github.token }}
|
||||
filters: .github/file-paths.yaml
|
||||
list-files: shell
|
||||
- name: Setup actionlint
|
||||
id: download
|
||||
shell: bash
|
||||
run: |
|
||||
if command -v actionlint; then
|
||||
echo "executable=actionlint" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
# script sets executable output, for details see: https://github.com/rhysd/actionlint/blob/main/docs/usage.md#use-actionlint-on-github-actions
|
||||
bash <(curl -fsSL https://raw.githubusercontent.com/rhysd/actionlint/v1.7.12/scripts/download-actionlint.bash)
|
||||
fi
|
||||
- name: Run actionlint
|
||||
run: ${{ steps.download.outputs.executable }} --color --verbose ${{ steps.changes.outputs.actionlint_files }}
|
||||
19
.github/workflows/app-db.yml
vendored
19
.github/workflows/app-db.yml
vendored
@@ -11,13 +11,6 @@ concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}-app-db
|
||||
cancel-in-progress: true
|
||||
|
||||
# Exposed to every job so the test-driver action can report failures to
|
||||
# ci-conductor (secrets: inherit is already set on the app-db call in
|
||||
# run-tests.yml). See DEV-2224.
|
||||
env:
|
||||
CI_CONDUCTOR_BASE_URL: ${{ secrets.CI_CONDUCTOR_BASE_URL }}
|
||||
CI_CONDUCTOR_WEBHOOK_SECRET: ${{ secrets.CI_CONDUCTOR_WEBHOOK_SECRET }}
|
||||
|
||||
jobs:
|
||||
be-tests-mariadb:
|
||||
if: ${{ !inputs.skip }}
|
||||
@@ -76,7 +69,7 @@ jobs:
|
||||
github.event_name == 'push' &&
|
||||
(
|
||||
github.ref_name == 'master' ||
|
||||
startsWith(github.ref_name, 'release-x')
|
||||
startsWith(github.ref_name, 'release-')
|
||||
) &&
|
||||
''
|
||||
) ||
|
||||
@@ -120,7 +113,6 @@ jobs:
|
||||
test-args: >-
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -132,7 +124,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
be-tests-mysql:
|
||||
if: ${{ !inputs.skip }}
|
||||
@@ -193,7 +184,7 @@ jobs:
|
||||
github.event_name == 'push' &&
|
||||
(
|
||||
github.ref_name == 'master' ||
|
||||
startsWith(github.ref_name, 'release-x')
|
||||
startsWith(github.ref_name, 'release-')
|
||||
) &&
|
||||
''
|
||||
) ||
|
||||
@@ -245,7 +236,6 @@ jobs:
|
||||
test-args: >-
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -257,7 +247,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
be-tests-postgres:
|
||||
if: ${{ !inputs.skip }}
|
||||
@@ -311,7 +300,7 @@ jobs:
|
||||
github.event_name == 'push' &&
|
||||
(
|
||||
github.ref_name == 'master' ||
|
||||
startsWith(github.ref_name, 'release-x')
|
||||
startsWith(github.ref_name, 'release-')
|
||||
) &&
|
||||
''
|
||||
) ||
|
||||
@@ -357,7 +346,6 @@ jobs:
|
||||
test-args: >-
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -369,7 +357,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
app-db-tests-result:
|
||||
needs:
|
||||
|
||||
85
.github/workflows/auto-backport.yml
vendored
85
.github/workflows/auto-backport.yml
vendored
@@ -5,94 +5,17 @@ on:
|
||||
types: [closed, labeled]
|
||||
|
||||
jobs:
|
||||
# --- Primary path: backport to every supported release line --------------
|
||||
# Triggered by the plain `backport` label. `contains(array, 'backport')` is an
|
||||
# exact element match, so this never fires for `double-backport`/`triple-backport`
|
||||
# — those are handled by the legacy escape hatch at the end of this file.
|
||||
prepare:
|
||||
create-backport:
|
||||
if:
|
||||
| # the PR must carry the `backport` label at merge, or be labeled `backport` after merge
|
||||
github.event.pull_request.merged == true && (
|
||||
github.event.action == 'closed' && contains(github.event.pull_request.labels.*.name, 'backport') ||
|
||||
github.event.action == 'labeled' && github.event.label.name == 'backport'
|
||||
)
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
outputs:
|
||||
commit: ${{ steps.find_commit.outputs.commit }}
|
||||
targets: ${{ steps.targets.outputs.result }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
fetch-depth: 0
|
||||
token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
|
||||
- uses: ./.github/actions/find-squashed-commit
|
||||
name: Find commit
|
||||
id: find_commit
|
||||
with:
|
||||
pull-request-number: ${{ github.event.pull_request.number }}
|
||||
base-ref: ${{ github.event.pull_request.base.ref }}
|
||||
|
||||
- name: Prepare release scripts
|
||||
uses: ./.github/actions/prepare-release-scripts
|
||||
|
||||
- name: Compute supported backport targets
|
||||
id: targets
|
||||
uses: actions/github-script@v9
|
||||
env:
|
||||
AWS_S3_STATIC_BUCKET: ${{ vars.AWS_S3_STATIC_BUCKET }}
|
||||
AWS_REGION: ${{ vars.AWS_REGION }}
|
||||
with:
|
||||
script: | # js
|
||||
// Backport targets are the supported major lines from version-info.json:
|
||||
// a line is in support — and a backport target — if and only if its EoL is
|
||||
// today or later. `getSupportedMajors` encapsulates that rule (and ignores
|
||||
// the display-only `lts` flag); it is shared with the auto-minor scheduler.
|
||||
const { getSupportedMajors } = require('${{ github.workspace }}/release/dist/index.cjs');
|
||||
|
||||
const targets = await getSupportedMajors();
|
||||
console.log('supported backport targets', targets);
|
||||
|
||||
return targets;
|
||||
|
||||
backport:
|
||||
needs: prepare
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
strategy:
|
||||
fail-fast: false # one conflicting branch must not block the others
|
||||
matrix:
|
||||
target: ${{ fromJson(needs.prepare.outputs.targets) }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
fetch-depth: 0
|
||||
token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
|
||||
|
||||
- name: Backport to v${{ matrix.target }}
|
||||
uses: ./.github/actions/create-backport
|
||||
with:
|
||||
target-version: ${{ matrix.target }}
|
||||
backport-commit: ${{ needs.prepare.outputs.commit }}
|
||||
github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
|
||||
github-token-2: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
# --- Legacy escape hatch: explicit depth backports -----------------------
|
||||
# `single-backport`/`double-backport`/`triple-backport` keep their original behavior —
|
||||
# backport to CURRENT_VERSION and the 0–2 lines below it — independent of
|
||||
# `major_version_support`. `single-backport` is a fallback for when the primary path
|
||||
# can't be used and only CURRENT_VERSION is wanted. Kept for the short term for one-off
|
||||
# manual depth control. If a branch from the primary path above already exists,
|
||||
# create-backport simply errors on the duplicate.
|
||||
legacy-backport:
|
||||
if:
|
||||
| # triggered only by the legacy depth labels, at merge or when added afterwards
|
||||
| # the PR must have a backport label when it's merged, or be labeled with a backport label after merge
|
||||
github.event.pull_request.merged == true && (
|
||||
github.event.action == 'closed' && (
|
||||
contains(github.event.pull_request.labels.*.name, 'single-backport') ||
|
||||
contains(github.event.pull_request.labels.*.name, 'backport') ||
|
||||
contains(github.event.pull_request.labels.*.name, 'double-backport') ||
|
||||
contains(github.event.pull_request.labels.*.name, 'triple-backport')
|
||||
) ||
|
||||
github.event.action == 'labeled' && (
|
||||
github.event.label.name == 'single-backport' ||
|
||||
github.event.label.name == 'backport' ||
|
||||
github.event.label.name == 'double-backport' ||
|
||||
github.event.label.name == 'triple-backport'
|
||||
)
|
||||
|
||||
48
.github/workflows/backend.yml
vendored
48
.github/workflows/backend.yml
vendored
@@ -11,13 +11,6 @@ concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}-backend
|
||||
cancel-in-progress: true
|
||||
|
||||
# Exposed to the test jobs so they can report failures to ci-conductor
|
||||
# (secrets: inherit is already set on the backend call in run-tests.yml).
|
||||
# See DEV-2224.
|
||||
env:
|
||||
CI_CONDUCTOR_BASE_URL: ${{ secrets.CI_CONDUCTOR_BASE_URL }}
|
||||
CI_CONDUCTOR_WEBHOOK_SECRET: ${{ secrets.CI_CONDUCTOR_WEBHOOK_SECRET }}
|
||||
|
||||
jobs:
|
||||
be-linter-clj-kondo:
|
||||
if: ${{ !inputs.skip }}
|
||||
@@ -45,9 +38,9 @@ jobs:
|
||||
|
||||
be-linter-eastwood-test:
|
||||
# Master / release safety net: lint every backend test namespace.
|
||||
if: ${{ !inputs.skip && github.event_name == 'push' && (github.ref_name == 'master' || startsWith(github.ref_name, 'release-x')) }}
|
||||
if: ${{ !inputs.skip && github.event_name == 'push' && (github.ref_name == 'master' || startsWith(github.ref_name, 'release-')) }}
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 45
|
||||
timeout-minutes: 30
|
||||
name: Eastwood (tests only)
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
@@ -121,7 +114,7 @@ jobs:
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
java-version: [25]
|
||||
java-version: [21]
|
||||
job:
|
||||
- name: Enterprise Tests
|
||||
edition: ee
|
||||
@@ -167,7 +160,7 @@ jobs:
|
||||
github.event_name == 'push' &&
|
||||
(
|
||||
github.ref_name == 'master' ||
|
||||
startsWith(github.ref_name, 'release-x')
|
||||
startsWith(github.ref_name, 'release-')
|
||||
) &&
|
||||
''
|
||||
) ||
|
||||
@@ -186,42 +179,20 @@ jobs:
|
||||
env:
|
||||
MB_EDITION: ${{ matrix.job.edition }}
|
||||
|
||||
- name: Resolve current job id
|
||||
# Resolve JOB_ID before the long test step so the ci-conductor reporter
|
||||
# below can link failures to this exact job. Best-effort.
|
||||
uses: ./.github/actions/resolve-job-id
|
||||
|
||||
- name: "Test Java ${{ matrix.java-version }} ${{ matrix.job.name }}"
|
||||
id: run-java-tests
|
||||
run: >-
|
||||
clojure -X:dev:ci:test:${{ matrix.job.edition }}:${{ matrix.job.edition }}-dev
|
||||
:exclude-tags '[:mb/driver-tests ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
|
||||
${{ matrix.job.test-args }}
|
||||
# Keep this step continue-on-error because the quarantine logic controls whether it ultimately fails.
|
||||
# Trunk Quarantine controls the final state of the run in the CI!
|
||||
# Even if this step fails we let it pass, and then `trunk-io/analytics-uploader` determines
|
||||
# whether to fail or to pass the whole job, based on the quarantine list.
|
||||
continue-on-error: true
|
||||
|
||||
- name: Report failed tests to ci-conductor
|
||||
if: ${{ !cancelled() && steps.run-java-tests.outcome == 'failure' }}
|
||||
# This step should NEVER fail the job!
|
||||
continue-on-error: true
|
||||
uses: ./.github/actions/report-failures-to-ci-conductor
|
||||
with:
|
||||
test-suite: be-tests-java-${{ matrix.java-version }}-${{ matrix.job.test-group-name }}
|
||||
adapter: backend
|
||||
|
||||
- name: Quarantine gate
|
||||
if: ${{ !cancelled() && steps.run-java-tests.outcome == 'failure' }}
|
||||
continue-on-error: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
timeout-minutes: 3
|
||||
uses: ./.github/actions/quarantine-gate
|
||||
with:
|
||||
adapter: backend
|
||||
test-suite: be-tests-java-${{ matrix.java-version }}-${{ matrix.job.test-group-name }}
|
||||
enforce-quarantine: ${{ vars.QUARANTINE_ENGINE == 'ci-conductor' }}
|
||||
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
if: always() && (steps.run-java-tests.conclusion != 'skipped' || steps.run-java-tests-21.conclusion != 'skipped')
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: be-tests-java-${{ matrix.java-version }}-${{ matrix.job.edition }}
|
||||
@@ -230,7 +201,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.run-java-tests.outcome }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
- name: Publish Test Report (JUnit)
|
||||
uses: dorny/test-reporter@a43b3a5f7366b97d083190328d2c652e1a8b6aa2 # v3.0.0
|
||||
@@ -259,7 +229,7 @@ jobs:
|
||||
timeout-minutes: 10
|
||||
strategy:
|
||||
matrix:
|
||||
java-version: [25]
|
||||
java-version: [21]
|
||||
name: "Check Java ${{ matrix.java-version }}"
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
166
.github/workflows/backport-reminder.yml
vendored
166
.github/workflows/backport-reminder.yml
vendored
@@ -2,173 +2,19 @@ name: Backport Reminder
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types: [opened, edited, auto_merge_enabled, review_requested, labeled, unlabeled, synchronize]
|
||||
types: [auto_merge_enabled, review_requested, labeled, unlabeled, synchronize]
|
||||
branches:
|
||||
- "master"
|
||||
|
||||
concurrency:
|
||||
group: "backport-reminder-${{ github.event.pull_request.number }}"
|
||||
cancel-in-progress: true
|
||||
merge_group:
|
||||
|
||||
jobs:
|
||||
flags:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
outputs:
|
||||
has_backport_label: ${{ steps.c.outputs.value }}
|
||||
steps:
|
||||
- id: c
|
||||
run: |
|
||||
echo "value=${{
|
||||
contains(github.event.pull_request.labels.*.name, 'no-backport') ||
|
||||
contains(github.event.pull_request.labels.*.name, 'backport') ||
|
||||
contains(github.event.pull_request.labels.*.name, 'single-backport') ||
|
||||
contains(github.event.pull_request.labels.*.name, 'double-backport') ||
|
||||
contains(github.event.pull_request.labels.*.name, 'triple-backport') }}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
files-changed:
|
||||
name: Check which files changed
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 3
|
||||
outputs:
|
||||
docs: ${{ steps.changes.outputs.docs }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Test which files changed
|
||||
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
|
||||
id: changes
|
||||
with:
|
||||
token: ${{ github.token }}
|
||||
filters: .github/file-paths.yaml
|
||||
|
||||
backport-reminder:
|
||||
needs: [ files-changed, flags ]
|
||||
if: ${{ !cancelled() }}
|
||||
name: Decide whether to backport or not
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
permissions: {}
|
||||
steps:
|
||||
- name: Fail if backport decision label is missing
|
||||
if: >
|
||||
github.event_name == 'pull_request' &&
|
||||
needs.files-changed.outputs.docs == 'true' &&
|
||||
needs.flags.outputs.has_backport_label == 'false'
|
||||
run: |
|
||||
echo "::error::PR has docs changes and is missing a backport decision label. Add one of: no-backport, backport, single-backport, double-backport, or triple-backport"
|
||||
exit 1
|
||||
|
||||
priority-backport-reminder:
|
||||
name: Remind about backporting P0 + P1 fixes
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
needs: [flags]
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: Add, update, or remove the high-priority backport reminder
|
||||
if: ${{ github.event_name == 'pull_request' }}
|
||||
uses: actions/github-script@v9
|
||||
env:
|
||||
HAS_BACKPORT_LABEL: ${{ needs.flags.outputs.has_backport_label }}
|
||||
- uses: jesusvasquez333/verify-pr-label-action@657d111bbbe13e22bbd55870f1813c699bde1401 # v1.4.0
|
||||
with:
|
||||
script: | # js
|
||||
const { owner, repo } = context.repo;
|
||||
const prNumber = context.payload.pull_request.number;
|
||||
|
||||
// A backport decision has already been made if a decision label is present.
|
||||
const alreadyDecided = process.env.HAS_BACKPORT_LABEL === "true";
|
||||
|
||||
// Issues linked to this PR via closing keywords (fixes/closes/resolves #N),
|
||||
// together with their labels.
|
||||
const query = `
|
||||
query($owner: String!, $repo: String!, $pr: Int!) {
|
||||
repository(owner: $owner, name: $repo) {
|
||||
pullRequest(number: $pr) {
|
||||
closingIssuesReferences(first: 50) {
|
||||
nodes {
|
||||
number
|
||||
title
|
||||
url
|
||||
labels(first: 50) { nodes { name } }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}`;
|
||||
|
||||
const data = await github.graphql(query, { owner, repo, pr: prNumber });
|
||||
const linkedIssues =
|
||||
data.repository.pullRequest.closingIssuesReferences.nodes ?? [];
|
||||
|
||||
const PRIORITY_LABELS = ["Priority:P0", "Priority:P1"];
|
||||
const priorityIssues = linkedIssues.filter((issue) =>
|
||||
issue.labels.nodes.some((l) => PRIORITY_LABELS.includes(l.name)),
|
||||
);
|
||||
|
||||
// Hidden marker so we can find/update/remove our own comment idempotently.
|
||||
const MARKER = "<!-- backport-priority-reminder -->";
|
||||
const comments = await github.paginate(
|
||||
github.rest.issues.listComments,
|
||||
{ owner, repo, issue_number: prNumber, per_page: 100 },
|
||||
);
|
||||
const existing = comments.find(
|
||||
(c) => c.body && c.body.includes(MARKER),
|
||||
);
|
||||
|
||||
// Remind only while a high-priority issue is closed AND no decision
|
||||
// has been made yet. Once either stops being true the comment is no
|
||||
// longer needed, so we remove any stale one below.
|
||||
const shouldComment = priorityIssues.length > 0 && !alreadyDecided;
|
||||
|
||||
if (!shouldComment) {
|
||||
if (existing) {
|
||||
await github.rest.issues.deleteComment({
|
||||
owner,
|
||||
repo,
|
||||
comment_id: existing.id,
|
||||
});
|
||||
console.log("Removed stale backport-priority reminder.");
|
||||
} else {
|
||||
console.log(
|
||||
"Reminder not needed (no high-priority linked issues, or a backport decision label is already applied).",
|
||||
);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
const list = priorityIssues
|
||||
.map((i) => {
|
||||
const tags = i.labels.nodes
|
||||
.map((l) => l.name)
|
||||
.filter((n) => PRIORITY_LABELS.includes(n))
|
||||
.join(", ");
|
||||
return `- #${i.number} ${i.title} (${tags})`;
|
||||
})
|
||||
.join("\n");
|
||||
|
||||
const body = [
|
||||
MARKER,
|
||||
"⚠️ This PR closes one or more high-priority issues:",
|
||||
"",
|
||||
list,
|
||||
"",
|
||||
"Please apply the `backport` label unless you have a very good reason not to.",
|
||||
].join("\n");
|
||||
|
||||
if (existing) {
|
||||
await github.rest.issues.updateComment({
|
||||
owner,
|
||||
repo,
|
||||
comment_id: existing.id,
|
||||
body,
|
||||
});
|
||||
} else {
|
||||
await github.rest.issues.createComment({
|
||||
owner,
|
||||
repo,
|
||||
issue_number: prNumber,
|
||||
body,
|
||||
});
|
||||
}
|
||||
github-token: '${{ secrets.GITHUB_TOKEN }}'
|
||||
valid-labels: 'no-backport, backport, double-backport, triple-backport'
|
||||
disable-reviews: true
|
||||
|
||||
148
.github/workflows/bundle-size-stats.yml
vendored
148
.github/workflows/bundle-size-stats.yml
vendored
@@ -1,148 +0,0 @@
|
||||
name: Bundle Size Stats
|
||||
|
||||
# Reuses the uberjar built on every master merge: downloads the jar (for the
|
||||
# emitted JS) plus the bundle-stats and embedding-sdk artifacts, measures
|
||||
# initial/total bundle sizes, and appends them to stats.metabase.com.
|
||||
|
||||
on:
|
||||
workflow_run:
|
||||
workflows: ["Build Uberjar"]
|
||||
types: [completed]
|
||||
# Master only: releases form non-linear lines whose deltas are meaningless,
|
||||
# so their sizes are backfilled separately rather than logged here.
|
||||
branches: [master]
|
||||
|
||||
concurrency:
|
||||
group: bundle-size-stats-${{ github.event.workflow_run.head_sha }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
bundle-size-stats:
|
||||
if: ${{ github.event.workflow_run.conclusion == 'success' && github.repository == 'metabase/metabase' }}
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 15
|
||||
env:
|
||||
HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
|
||||
# Only record a data point when a bundle's as-served size (brotli where the
|
||||
# build ships .br, else gzip) moved by at least this percent vs the last
|
||||
# plotted point, keeping the chart to real changes.
|
||||
MIN_DELTA_PERCENT: "0.5"
|
||||
steps:
|
||||
- name: Check out the code
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ github.event.workflow_run.head_sha }}
|
||||
- name: Download EE uberjar
|
||||
uses: ./.github/actions/fetch-artifact
|
||||
with:
|
||||
commit: ${{ github.event.workflow_run.head_sha }}
|
||||
edition: ee
|
||||
extract-path: temp/uberjar
|
||||
- name: Extract front-end bundles from uberjar
|
||||
run: |
|
||||
mkdir -p resources artifacts
|
||||
unzip -o "temp/uberjar/metabase.jar" \
|
||||
"frontend_client/app/dist/*" \
|
||||
"frontend_client/app/embed.js" \
|
||||
"frontend_client/app/embedding-sdk/*" \
|
||||
-d resources
|
||||
unzip -o -j "temp/uberjar/metabase.jar" version.properties -d artifacts
|
||||
- name: Download bundle stats and SDK package artifacts
|
||||
uses: actions/github-script@v9
|
||||
with:
|
||||
script: | # js
|
||||
const fs = require("fs");
|
||||
|
||||
async function fetchArtifact(name) {
|
||||
const res = await github.rest.actions.listArtifactsForRepo({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
name,
|
||||
per_page: 1,
|
||||
});
|
||||
const artifact = res.data?.artifacts?.[0];
|
||||
if (!artifact) {
|
||||
console.log(`Artifact not found: ${name}`);
|
||||
return "";
|
||||
}
|
||||
const download = await github.rest.actions.downloadArtifact({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
artifact_id: artifact.id,
|
||||
archive_format: "zip",
|
||||
});
|
||||
fs.writeFileSync(`${name}.zip`, Buffer.from(download.data));
|
||||
console.log(`Downloaded ${name}`);
|
||||
return `${name}.zip`;
|
||||
}
|
||||
|
||||
const sha = process.env.HEAD_SHA;
|
||||
core.exportVariable("STATS_ZIP", await fetchArtifact(`metabase-bundle-stats-ee-${sha}`));
|
||||
core.exportVariable("PKG_ZIP", await fetchArtifact(`embedding-sdk-${sha}`));
|
||||
- name: Extract stats and SDK package
|
||||
run: |
|
||||
mkdir -p artifacts resources/embedding-sdk
|
||||
if [ -n "$STATS_ZIP" ]; then unzip -o "$STATS_ZIP" -d artifacts; fi
|
||||
if [ -n "$PKG_ZIP" ]; then unzip -o "$PKG_ZIP" -d resources/embedding-sdk; fi
|
||||
- name: Restore last plotted bundle sizes
|
||||
uses: actions/cache/restore@v4
|
||||
with:
|
||||
path: bundle-size-cache
|
||||
key: bundle-size-last-${{ github.run_id }}
|
||||
restore-keys: |
|
||||
bundle-size-last-
|
||||
- name: Measure bundle sizes
|
||||
run: node ./.github/scripts/measure-bundle-sizes.js | tee artifacts/bundle-sizes.json
|
||||
- name: Compute deltas and decide whether to record this point
|
||||
id: prepare
|
||||
env:
|
||||
CURRENT: artifacts/bundle-sizes.json
|
||||
LAST: bundle-size-cache/last.json
|
||||
CACHE_OUT: bundle-size-cache/last.json
|
||||
ROWS_OUT: artifacts/upload-rows.json
|
||||
VERSION_PROPS: artifacts/version.properties
|
||||
# Passed via env (not interpolated into the shell) so the commit
|
||||
# message can't inject commands; the script keeps only the subject.
|
||||
COMMIT_MESSAGE: ${{ github.event.workflow_run.head_commit.message }}
|
||||
run: node ./.github/scripts/bundle-size-stats-prepare.js
|
||||
- name: Upload sizes (with deltas) to Stats
|
||||
id: upload
|
||||
if: steps.prepare.outputs.significant == 'true'
|
||||
uses: actions/github-script@v9
|
||||
env:
|
||||
API_KEY: ${{ secrets.STATS_API_KEY }}
|
||||
with:
|
||||
script: | # js
|
||||
const fs = require("fs");
|
||||
const { uploadCsvToMb } = require("${{ github.workspace }}/.github/scripts/csv-to-mb.js");
|
||||
|
||||
const data = JSON.parse(
|
||||
fs.readFileSync("${{ github.workspace }}/artifacts/upload-rows.json", "utf8"),
|
||||
);
|
||||
|
||||
console.table(data);
|
||||
|
||||
try {
|
||||
await uploadCsvToMb({
|
||||
baseUrl: "https://stats.metabase.com",
|
||||
tableId: 79702,
|
||||
jsonData: data,
|
||||
mode: "append",
|
||||
});
|
||||
console.log("Bundle sizes uploaded successfully");
|
||||
core.setOutput("uploaded", "true");
|
||||
} catch (error) {
|
||||
// Stats logging is best-effort: if stats.metabase.com is still
|
||||
// unreachable after the client's retries, keep the run green. The
|
||||
// last-plotted cache is only saved on a successful upload (below),
|
||||
// so this point is re-attempted on the next master build and
|
||||
// nothing is silently dropped.
|
||||
core.warning(`Bundle-size upload failed after retries; leaving run green: ${error.message}`);
|
||||
core.setOutput("uploaded", "false");
|
||||
}
|
||||
- name: Remember this as the last plotted point
|
||||
if: steps.upload.outputs.uploaded == 'true'
|
||||
uses: actions/cache/save@v4
|
||||
with:
|
||||
path: bundle-size-cache
|
||||
key: bundle-size-last-${{ github.run_id }}
|
||||
43
.github/workflows/cache-generator.yml
vendored
43
.github/workflows/cache-generator.yml
vendored
@@ -68,47 +68,10 @@ jobs:
|
||||
- name: Prepare backend
|
||||
uses: ./.github/actions/prepare-backend
|
||||
|
||||
# Pre-resolve every Clojure dependency classpath CI actually uses, so the
|
||||
# saved cache is a SUPERSET of what each job resolves at runtime.
|
||||
#
|
||||
# Why a single `-P` is not enough: tools.deps selects dependency *versions*
|
||||
# globally per classpath, so different alias combinations can resolve the
|
||||
# same transitive lib to different versions (e.g. `org.clojure/data.json`
|
||||
# 2.4.0 vs 2.5.x, `org.ow2.asm/asm` 9.2 vs newer). Resolving only one combo
|
||||
# leaves the others to fetch their version-deltas straight from Maven
|
||||
# Central / Clojars on every run -> intermittent 403s. See DEV-2094.
|
||||
#
|
||||
# Keep this list in sync with the `clojure` invocations across .github/.
|
||||
# Each entry notes its consumer. We resolve the FULL alias strings (not a
|
||||
# hand-reduced set) on purpose: if a currently dep-less alias later gains a
|
||||
# dependency, this still covers it instead of silently drifting back into a
|
||||
# cache gap.
|
||||
# Add or remove aliases to affect the cache size and its contents.
|
||||
# Adjust to taste, then redeploy this workflow.
|
||||
- name: Pre-resolve Clojure dependencies
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
combos=(
|
||||
":build:dev:ee:ee-dev:drivers:drivers-dev:cljs" # build + uberjar (combined classpath)
|
||||
":cljs" # shadow-cljs (build:cljs); resolves :cljs alone -> covers asm:9.2
|
||||
":dev:ee:ee-dev:drivers:drivers-dev:test:eastwood" # backend.yml (eastwood + eastwood/test)
|
||||
":dev:ci:ee:ee-dev:drivers:drivers-dev:test" # test-driver.yml (ee) -> covers data.json:2.4.0
|
||||
":dev:ci:oss:oss-dev:drivers:drivers-dev:test" # test-driver.yml (oss)
|
||||
":dev:ci:test:ee:ee-dev" # backend.yml matrix (ee) + generative-query-test.yml
|
||||
":dev:ci:test:oss:oss-dev" # backend.yml matrix (oss)
|
||||
":dev:ci:ee:ee-dev:test:cloverage" # backend-cloverage.yml
|
||||
":dev:drivers:drivers-dev:test" # reset-cloud-test-dataset.yml + presto script
|
||||
":dev:drivers:build:build-dev:build-test:ci" # build-scripts.yml
|
||||
":ee:drivers:check" # backend.yml (check; git dep athos/clj-check)
|
||||
":ee:doc" # docs-generate-base.yml
|
||||
":kondo" # resolve-backport-conflicts.yml (replace-deps)
|
||||
":run:ee" # cross-branch-migration-test
|
||||
":dev:ee:migrate" # cross-branch-migration-test
|
||||
)
|
||||
for combo in "${combos[@]}"; do
|
||||
echo "::group::clojure -P -A${combo}"
|
||||
clojure -P -A"${combo}"
|
||||
echo "::endgroup::"
|
||||
done
|
||||
run: clojure -A:build:dev:ee:ee-dev:drivers:drivers-dev:cljs -P
|
||||
|
||||
- name: Update M2 cache
|
||||
uses: ./.github/actions/update-cache
|
||||
|
||||
14
.github/workflows/cross-branch-migrations.yml
vendored
14
.github/workflows/cross-branch-migrations.yml
vendored
@@ -26,7 +26,7 @@ concurrency:
|
||||
jobs:
|
||||
migrations-h2-ee:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 15
|
||||
timeout-minutes: 10
|
||||
env:
|
||||
CI: 'true'
|
||||
MB_DB_TYPE: h2
|
||||
@@ -40,7 +40,7 @@ jobs:
|
||||
|
||||
migrations-postgres-14-ee:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 15
|
||||
timeout-minutes: 10
|
||||
env:
|
||||
CI: 'true'
|
||||
MB_DB_TYPE: postgres
|
||||
@@ -66,7 +66,7 @@ jobs:
|
||||
|
||||
migrations-postgres-latest-ee:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 15
|
||||
timeout-minutes: 10
|
||||
env:
|
||||
CI: 'true'
|
||||
MB_DB_TYPE: postgres
|
||||
@@ -92,7 +92,7 @@ jobs:
|
||||
|
||||
migrations-mysql-8-0-ee:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 15
|
||||
timeout-minutes: 10
|
||||
env:
|
||||
CI: 'true'
|
||||
MB_DB_TYPE: mysql
|
||||
@@ -117,7 +117,7 @@ jobs:
|
||||
|
||||
migrations-mysql-latest-ee:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 15
|
||||
timeout-minutes: 10
|
||||
env:
|
||||
CI: 'true'
|
||||
MB_DB_TYPE: mysql
|
||||
@@ -143,7 +143,7 @@ jobs:
|
||||
|
||||
migrations-mariadb-10-6-ee:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 15
|
||||
timeout-minutes: 10
|
||||
env:
|
||||
CI: 'true'
|
||||
MB_DB_TYPE: mysql
|
||||
@@ -168,7 +168,7 @@ jobs:
|
||||
|
||||
migrations-mariadb-latest-ee:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 15
|
||||
timeout-minutes: 10
|
||||
env:
|
||||
CI: 'true'
|
||||
MB_DB_TYPE: mysql
|
||||
|
||||
8
.github/workflows/cut-release-branch.yml
vendored
8
.github/workflows/cut-release-branch.yml
vendored
@@ -70,10 +70,6 @@ jobs:
|
||||
git push -u origin "$BRANCH"
|
||||
echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# CURRENT_VERSION is an org-level variable — that's the canonical one,
|
||||
# shared across repos. A repo-level variable of the same name would
|
||||
# shadow it (repo vars override org vars), so we deliberately update
|
||||
# the org-level value here and rely on no repo-level override existing.
|
||||
- name: Bump vars.CURRENT_VERSION
|
||||
id: bump
|
||||
continue-on-error: true
|
||||
@@ -82,7 +78,7 @@ jobs:
|
||||
run: |
|
||||
gh variable set CURRENT_VERSION \
|
||||
--body "${{ steps.preflight.outputs.next }}" \
|
||||
--org "${{ github.repository_owner }}"
|
||||
--repo "${{ github.repository }}"
|
||||
|
||||
# Pre-create the milestone for the version AFTER this cut, so it's
|
||||
# already open when next time's backfill needs somewhere to assign
|
||||
@@ -208,7 +204,7 @@ jobs:
|
||||
|
||||
const notes = [
|
||||
bumpFailed
|
||||
? `:warning: Auto-bump of \`CURRENT_VERSION\` to *${next}* failed — please update it manually: <https://github.com/organizations/${owner}/settings/variables/actions|Org Actions variables>`
|
||||
? `:warning: Auto-bump of \`CURRENT_VERSION\` to *${next}* failed — please update it manually: <https://github.com/${owner}/${repo}/settings/variables/actions|Actions variables>`
|
||||
: `\`CURRENT_VERSION\` bumped to *${next}*.`,
|
||||
milestoneFailed
|
||||
? `:warning: Pre-creating milestone \`0.${{ steps.preflight.outputs.nextMilestone }}\` (for the next cut) failed — please create it manually: <https://github.com/${owner}/${repo}/milestones|Repository milestones>`
|
||||
|
||||
@@ -30,23 +30,15 @@ jobs:
|
||||
const response = await fetch('https://raw.githubusercontent.com/metabase/ci-test-config/refs/heads/master/ci-test-config.json');
|
||||
const config = await response.json();
|
||||
|
||||
const drivers = config.drivers || [];
|
||||
const drivers = config.ignored?.drivers || [];
|
||||
const lastUpdated = config.metadata?.last_updated || 'unknown';
|
||||
const reason = config.metadata?.reason || 'No reason provided';
|
||||
|
||||
const skipped = drivers.filter(d => d.status === 'skip').map(d => d.name);
|
||||
const info = drivers.filter(d => d.status === 'info').map(d => d.name);
|
||||
|
||||
if (skipped.length === 0 && info.length === 0) {
|
||||
console.log('No drivers are disabled or collecting data. Skipping Slack notification.');
|
||||
if (drivers.length === 0) {
|
||||
console.log('No drivers are currently disabled. Skipping Slack notification.');
|
||||
return;
|
||||
}
|
||||
|
||||
const list = (names) => names.map(d => `\`${d}\``).join(' ');
|
||||
const lines = [];
|
||||
if (skipped.length) lines.push(`:no_entry: *Disabled (not run):* ${list(skipped)}`);
|
||||
if (info.length) lines.push(`:information_source: *Collecting data (non-blocking):* ${list(info)}`);
|
||||
|
||||
if (!process.env.SLACK_BOT_TOKEN) {
|
||||
throw new Error('SLACK_BOT_TOKEN is required');
|
||||
}
|
||||
@@ -55,13 +47,13 @@ jobs:
|
||||
|
||||
await slack.chat.postMessage({
|
||||
channel: 'engineering-ci',
|
||||
text: 'CI Driver Status Report',
|
||||
text: 'CI Disabled Drivers Report',
|
||||
blocks: [
|
||||
{
|
||||
type: 'section',
|
||||
text: {
|
||||
type: 'mrkdwn',
|
||||
text: `*CI Driver Status*\n${lines.join('\n')}`
|
||||
text: `:no_entry: *CI Disabled Drivers*\n${drivers.map(d => `\`${d}\``).join(' ')}`
|
||||
},
|
||||
accessory: {
|
||||
type: 'button',
|
||||
|
||||
22
.github/workflows/docs-generate-63.yml
vendored
22
.github/workflows/docs-generate-63.yml
vendored
@@ -1,22 +0,0 @@
|
||||
name: Auto-update documentation (release)
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: "0 23 * * 1-5" # Every Monday through Friday at 11 PM UTC
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
target_branch:
|
||||
description: "Target branch for documentation update"
|
||||
required: false
|
||||
default: "release-x.63.x"
|
||||
type: string
|
||||
|
||||
jobs:
|
||||
update-release-docs:
|
||||
if: ${{ github.event_name != 'schedule' || github.repository == 'metabase/metabase' }}
|
||||
name: Call documentation update for release branch
|
||||
uses: ./.github/workflows/docs-generate-base.yml
|
||||
with:
|
||||
target_branch: ${{ inputs.target_branch || 'release-x.63.x' }}
|
||||
secrets:
|
||||
automation_token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
|
||||
4
.github/workflows/docs-release-generate.yml
vendored
4
.github/workflows/docs-release-generate.yml
vendored
@@ -8,7 +8,7 @@ on:
|
||||
target_branch:
|
||||
description: "Target branch for documentation update"
|
||||
required: false
|
||||
default: "release-x.62.x"
|
||||
default: "release-x.60.x"
|
||||
type: string
|
||||
|
||||
jobs:
|
||||
@@ -17,6 +17,6 @@ jobs:
|
||||
name: Call documentation update for release branch
|
||||
uses: ./.github/workflows/docs-generate-base.yml
|
||||
with:
|
||||
target_branch: ${{ inputs.target_branch || 'release-x.62.x' }}
|
||||
target_branch: ${{ inputs.target_branch || 'release-x.60.x' }}
|
||||
secrets:
|
||||
automation_token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
|
||||
|
||||
15
.github/workflows/drivers-stress-test.yml
vendored
15
.github/workflows/drivers-stress-test.yml
vendored
@@ -4,10 +4,6 @@ on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
test:
|
||||
# Technically this accepts anything that is accepted by Hawk:
|
||||
# https://github.com/metabase/hawk/#leiningen-style-test-selection
|
||||
# For example, to run everything in the tests/ dir you can do
|
||||
# $ gh workflow run .github/workflows/drivers-stress-test.yml --field 'test=\"test/\" :only-tags [:mb/driver-tests]' --field times=2 --field driver=Snowflake
|
||||
description: 'The exact test name with its namespace.'
|
||||
type: string
|
||||
required: true
|
||||
@@ -83,7 +79,6 @@ jobs:
|
||||
MB_BIGQUERY_TEST_ACCESS_TOKEN: ${{ secrets.MB_BIGQUERY_TEST_ACCESS_TOKEN }}
|
||||
MB_BIGQUERY_TEST_REFRESH_TOKEN: ${{ secrets.MB_BIGQUERY_TEST_REFRESH_TOKEN }}
|
||||
MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON: ${{ secrets.MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON }}
|
||||
MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING: ${{ secrets.MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Test BigQuery Cloud SDK driver
|
||||
@@ -578,19 +573,9 @@ jobs:
|
||||
MB_SNOWFLAKE_TEST_USER: METABASE CI
|
||||
MB_SNOWFLAKE_TEST_ACCOUNT: ${{ secrets.MB_SNOWFLAKE_TEST_ACCOUNT }}
|
||||
MB_SNOWFLAKE_TEST_PASSWORD: ${{ secrets.MB_SNOWFLAKE_TEST_PASSWORD }}
|
||||
MB_SNOWFLAKE_TEST_PRIVATE_KEY: ${{ secrets.MB_SNOWFLAKE_TEST_PRIVATE_KEY }}
|
||||
MB_SNOWFLAKE_TEST_WAREHOUSE: ${{ secrets.MB_SNOWFLAKE_TEST_WAREHOUSE }}
|
||||
MB_SNOWFLAKE_TEST_PK_USER: METABASE PK
|
||||
MB_SNOWFLAKE_TEST_PK_PRIVATE_KEY: ${{ secrets.MB_SNOWFLAKE_TEST_PK_PRIVATE_KEY }}
|
||||
MB_SNOWFLAKE_TEST_PK_PRIVATE_KEY_2: ${{ secrets.MB_SNOWFLAKE_TEST_PK_PRIVATE_KEY_2 }}
|
||||
MB_SNOWFLAKE_TEST_PK_PUBLIC_KEY: ${{ secrets.MB_SNOWFLAKE_TEST_PK_PUBLIC_KEY }}
|
||||
MB_SNOWFLAKE_TEST_PK_PUBLIC_KEY_2: ${{ secrets.MB_SNOWFLAKE_TEST_PK_PUBLIC_KEY_2 }}
|
||||
|
||||
# RSA Role testing:
|
||||
MB_SNOWFLAKE_TEST_RSA_ROLE_TEST_DEFAULT_USER: RSA_ROLE_TEST_DEFAULT_USER
|
||||
MB_SNOWFLAKE_TEST_RSA_ROLE_TEST_CUSTOM_USER: RSA_ROLE_TEST_CUSTOM_USER
|
||||
MB_SNOWFLAKE_TEST_RSA_ROLE_TEST_ROLE: RSA_ROLE_TEST_ROLE
|
||||
MB_SNOWFLAKE_TEST_RSA_ROLE_TEST_DB: RSA_ROLE_TEST_DB
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Test Snowflake driver
|
||||
|
||||
323
.github/workflows/drivers.yml
vendored
323
.github/workflows/drivers.yml
vendored
@@ -11,13 +11,6 @@ concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}-drivers
|
||||
cancel-in-progress: true
|
||||
|
||||
# Exposed to every driver job so the test-driver action can report failures to
|
||||
# ci-conductor (secrets: inherit is already set on the drivers call in
|
||||
# run-tests.yml). See DEV-2224.
|
||||
env:
|
||||
CI_CONDUCTOR_BASE_URL: ${{ secrets.CI_CONDUCTOR_BASE_URL }}
|
||||
CI_CONDUCTOR_WEBHOOK_SECRET: ${{ secrets.CI_CONDUCTOR_WEBHOOK_SECRET }}
|
||||
|
||||
jobs:
|
||||
# This job determines which driver tests should run based on PR context.
|
||||
# All decision logic is consolidated in mage driver-decisions.
|
||||
@@ -45,26 +38,6 @@ jobs:
|
||||
sqlite-should-run: ${{ steps.driver-decisions.outputs.sqlite-should-run }}
|
||||
sqlserver-should-run: ${{ steps.driver-decisions.outputs.sqlserver-should-run }}
|
||||
vertica-should-run: ${{ steps.driver-decisions.outputs.vertica-should-run }}
|
||||
# Per-driver configured status from ci-test-config (skip | info | required)
|
||||
h2-status: ${{ steps.driver-decisions.outputs.h2-status }}
|
||||
athena-status: ${{ steps.driver-decisions.outputs.athena-status }}
|
||||
bigquery-status: ${{ steps.driver-decisions.outputs.bigquery-status }}
|
||||
clickhouse-status: ${{ steps.driver-decisions.outputs.clickhouse-status }}
|
||||
databricks-status: ${{ steps.driver-decisions.outputs.databricks-status }}
|
||||
druid-jdbc-status: ${{ steps.driver-decisions.outputs.druid-jdbc-status }}
|
||||
mongo-status: ${{ steps.driver-decisions.outputs.mongo-status }}
|
||||
mongo-ssl-status: ${{ steps.driver-decisions.outputs.mongo-ssl-status }}
|
||||
mongo-sharded-cluster-status: ${{ steps.driver-decisions.outputs.mongo-sharded-cluster-status }}
|
||||
mysql-mariadb-status: ${{ steps.driver-decisions.outputs.mysql-mariadb-status }}
|
||||
oracle-status: ${{ steps.driver-decisions.outputs.oracle-status }}
|
||||
postgres-status: ${{ steps.driver-decisions.outputs.postgres-status }}
|
||||
presto-jdbc-status: ${{ steps.driver-decisions.outputs.presto-jdbc-status }}
|
||||
redshift-status: ${{ steps.driver-decisions.outputs.redshift-status }}
|
||||
snowflake-status: ${{ steps.driver-decisions.outputs.snowflake-status }}
|
||||
sparksql-status: ${{ steps.driver-decisions.outputs.sparksql-status }}
|
||||
sqlite-status: ${{ steps.driver-decisions.outputs.sqlite-status }}
|
||||
sqlserver-status: ${{ steps.driver-decisions.outputs.sqlserver-status }}
|
||||
vertica-status: ${{ steps.driver-decisions.outputs.vertica-status }}
|
||||
# Per-driver quarantine conflict flags (set when driver is quarantined but has file changes)
|
||||
athena-quarantine-conflict: ${{ steps.driver-decisions.outputs.athena-quarantine-conflict }}
|
||||
bigquery-quarantine-conflict: ${{ steps.driver-decisions.outputs.bigquery-quarantine-conflict }}
|
||||
@@ -86,28 +59,6 @@ jobs:
|
||||
sqlserver-quarantine-conflict: ${{ steps.driver-decisions.outputs.sqlserver-quarantine-conflict }}
|
||||
vertica-quarantine-conflict: ${{ steps.driver-decisions.outputs.vertica-quarantine-conflict }}
|
||||
steps:
|
||||
# Up front (before any driver job), drop info-driver comments from older commits so a
|
||||
# superseded comment disappears immediately rather than waiting for this run's drivers.
|
||||
# Keeps this commit's comment (if any) so a re-run doesn't flicker.
|
||||
- name: Remove stale info-driver comments
|
||||
if: ${{ github.event_name == 'pull_request' }}
|
||||
uses: actions/github-script@v9
|
||||
continue-on-error: true
|
||||
with:
|
||||
script: | # js
|
||||
const pr = context.payload.pull_request;
|
||||
if (!pr) return;
|
||||
const { owner, repo } = context.repo;
|
||||
const PREFIX = "<!-- info-driver-result";
|
||||
const keep = `<!-- info-driver-results:${pr.head.sha} -->`;
|
||||
const { data: comments } = await github.rest.issues.listComments({
|
||||
owner, repo, issue_number: pr.number, per_page: 100,
|
||||
});
|
||||
for (const c of comments) {
|
||||
if (c.body.includes(PREFIX) && !c.body.includes(keep)) {
|
||||
await github.rest.issues.deleteComment({ owner, repo, comment_id: c.id }).catch(() => {});
|
||||
}
|
||||
}
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v6
|
||||
- name: Fetch base branch
|
||||
@@ -138,7 +89,7 @@ jobs:
|
||||
./bin/mage -driver-decisions "${MAGE_ARGS[@]}" --github-output-only >> $GITHUB_OUTPUT
|
||||
env:
|
||||
BASE_REF: ${{ github.event.pull_request.base.ref || github.ref_name }}
|
||||
IS_MASTER_OR_RELEASE: ${{ github.ref_name == 'master' || startsWith(github.ref_name, 'release-x') }}
|
||||
IS_MASTER_OR_RELEASE: ${{ github.ref_name == 'master' || startsWith(github.ref_name, 'release-') }}
|
||||
shell: bash
|
||||
|
||||
be-tests-athena-ee:
|
||||
@@ -179,8 +130,6 @@ jobs:
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.athena-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -192,9 +141,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.athena-status }}
|
||||
report-variant: ${{ matrix.job.name }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
be-tests-bigquery-cloud-sdk-ee:
|
||||
needs: determine-driver-skips
|
||||
@@ -205,20 +151,11 @@ jobs:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
job:
|
||||
- name: Driver Tests Part 1
|
||||
- name: Driver Tests
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
:fail-fast? true
|
||||
:partition/total 2
|
||||
:partition/index 0
|
||||
- name: Driver Tests Part 2
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
:fail-fast? true
|
||||
:partition/total 2
|
||||
:partition/index 1
|
||||
- name: Transforms Python Tests
|
||||
test-args: >-
|
||||
:only-tags [:mb/transforms-python-test]
|
||||
@@ -258,8 +195,6 @@ jobs:
|
||||
junit-name: 'be-tests-bigquery-cloud-sdk-ee'
|
||||
test-args: ${{ matrix.job.test-args }}
|
||||
logs-artifact-suffix: ${{ matrix.job.name }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.bigquery-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -271,9 +206,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.bigquery-status }}
|
||||
report-variant: ${{ matrix.job.name }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
- name: Cleanup python-runner
|
||||
if: ${{ always() && matrix.job.needs-python-runner == true }}
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -346,8 +278,6 @@ jobs:
|
||||
junit-name: 'be-tests-clickhouse-ee'
|
||||
test-args: ${{ matrix.job.test-args }}
|
||||
logs-artifact-suffix: ${{ matrix.job.name }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.clickhouse-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -359,8 +289,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.clickhouse-status }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
- name: Cleanup python-runner
|
||||
if: ${{ always() && matrix.job.needs-python-runner == true }}
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -368,6 +296,54 @@ jobs:
|
||||
upload-logs: ${{ steps.test-driver.outcome == 'failure' }}
|
||||
logs-artifact-suffix: ${{ matrix.job.name }} ${{ matrix.version.name }}
|
||||
|
||||
be-tests-druid-ee:
|
||||
needs: determine-driver-skips
|
||||
if: ${{ needs.determine-driver-skips.outputs.druid-should-run == 'true' || needs.determine-driver-skips.outputs.druid-quarantine-conflict == 'true' }}
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 40
|
||||
env:
|
||||
CI: 'true'
|
||||
DRIVERS: druid
|
||||
services:
|
||||
druid:
|
||||
image: metabase/druid:0.20.2
|
||||
ports:
|
||||
- "8082:8082"
|
||||
env:
|
||||
CLUSTER_SIZE: nano-quickstart
|
||||
name: Druid (Legacy)
|
||||
steps:
|
||||
- name: Fail on quarantine conflict
|
||||
if: ${{ needs.determine-driver-skips.outputs.druid-quarantine-conflict == 'true' }}
|
||||
run: |
|
||||
echo "::error::Druid driver is quarantined but you changed files in modules/drivers/druid/"
|
||||
echo ""
|
||||
echo "Your changes will NOT be tested because the driver is currently quarantined."
|
||||
echo "To run Druid tests, add the label: break-quarantine-druid"
|
||||
echo ""
|
||||
echo "If you merge without testing, you risk breaking the Druid driver!"
|
||||
exit 1
|
||||
- uses: actions/checkout@v6
|
||||
- name: Test Druid driver
|
||||
id: test-driver
|
||||
uses: ./.github/actions/test-driver
|
||||
with:
|
||||
junit-name: 'be-tests-druid-ee'
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
|
||||
be-tests-druid-jdbc-ee:
|
||||
needs: determine-driver-skips
|
||||
if: ${{ needs.determine-driver-skips.outputs.druid-jdbc-should-run == 'true' || needs.determine-driver-skips.outputs.druid-jdbc-quarantine-conflict == 'true' }}
|
||||
@@ -404,8 +380,6 @@ jobs:
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.druid-jdbc-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -417,39 +391,12 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.druid-jdbc-status }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
be-tests-databricks-ee:
|
||||
needs: determine-driver-skips
|
||||
if: ${{ needs.determine-driver-skips.outputs.databricks-should-run == 'true' || needs.determine-driver-skips.outputs.databricks-quarantine-conflict == 'true' }}
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 40
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
job:
|
||||
- name: Driver Tests Part 1
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
:fail-fast? true
|
||||
:partition/total 3
|
||||
:partition/index 0
|
||||
- name: Driver Tests Part 2
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
:fail-fast? true
|
||||
:partition/total 3
|
||||
:partition/index 1
|
||||
- name: Driver Tests Part 3
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
:fail-fast? true
|
||||
:partition/total 3
|
||||
:partition/index 2
|
||||
env:
|
||||
CI: 'true'
|
||||
DRIVERS: databricks
|
||||
@@ -478,9 +425,9 @@ jobs:
|
||||
uses: ./.github/actions/test-driver
|
||||
with:
|
||||
junit-name: 'be-tests-databricks-ee'
|
||||
test-args: ${{ matrix.job.test-args }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.databricks-status }}
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -492,40 +439,12 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.databricks-status }}
|
||||
report-variant: ${{ matrix.job.name }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
be-tests-databricks-multi-catalog-ee:
|
||||
needs: determine-driver-skips
|
||||
if: ${{ needs.determine-driver-skips.outputs.databricks-should-run == 'true' || needs.determine-driver-skips.outputs.databricks-quarantine-conflict == 'true' }}
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 40
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
job:
|
||||
- name: Driver Tests Part 1
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
:fail-fast? true
|
||||
:partition/total 3
|
||||
:partition/index 0
|
||||
- name: Driver Tests Part 2
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
:fail-fast? true
|
||||
:partition/total 3
|
||||
:partition/index 1
|
||||
- name: Driver Tests Part 3
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
:fail-fast? true
|
||||
:partition/total 3
|
||||
:partition/index 2
|
||||
env:
|
||||
CI: 'true'
|
||||
DRIVERS: databricks
|
||||
@@ -555,9 +474,9 @@ jobs:
|
||||
uses: ./.github/actions/test-driver
|
||||
with:
|
||||
junit-name: 'be-tests-databricks-multi-catalog-ee'
|
||||
test-args: ${{ matrix.job.test-args }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.databricks-status }}
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -569,9 +488,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.databricks-status }}
|
||||
report-variant: ${{ matrix.job.name }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
be-tests-h2:
|
||||
needs: determine-driver-skips
|
||||
@@ -591,8 +507,6 @@ jobs:
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.h2-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -604,8 +518,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.h2-status }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
be-tests-h2-oss:
|
||||
needs: determine-driver-skips
|
||||
@@ -626,8 +538,6 @@ jobs:
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.h2-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -639,8 +549,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.h2-status }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
be-tests-mysql-mariadb:
|
||||
needs: determine-driver-skips
|
||||
@@ -703,7 +611,7 @@ jobs:
|
||||
github.event_name == 'push' &&
|
||||
(
|
||||
github.ref_name == 'master' ||
|
||||
startsWith(github.ref_name, 'release-x')
|
||||
startsWith(github.ref_name, 'release-')
|
||||
) &&
|
||||
''
|
||||
) ||
|
||||
@@ -760,8 +668,6 @@ jobs:
|
||||
test-args: >-
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.mysql-mariadb-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -773,8 +679,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.mysql-mariadb-status }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
- name: Cleanup python-runner
|
||||
if: always()
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -844,8 +748,6 @@ jobs:
|
||||
junit-name: ${{ matrix.version.junit-name }}
|
||||
test-args: ${{ matrix.job.test-args }}
|
||||
logs-artifact-suffix: ${{ matrix.job.name }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.mongo-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -857,8 +759,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.mongo-status }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
- name: Cleanup python-runner
|
||||
if: ${{ always() && matrix.job.needs-python-runner == true }}
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -930,8 +830,6 @@ jobs:
|
||||
test-args: >-
|
||||
:exclude-tags '[:mongo-sharded-cluster-tests :mb/transforms-python-test]'
|
||||
:only-tags [:mb/driver-tests]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.mongo-ssl-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -943,8 +841,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.mongo-ssl-status }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
be-tests-mongo-sharded-cluster-ee:
|
||||
needs: determine-driver-skips
|
||||
@@ -979,8 +875,6 @@ jobs:
|
||||
junit-name: 'be-tests-mongo-sharded-cluster-ee'
|
||||
test-args: >-
|
||||
:only metabase.driver.mongo.sharded-cluster-test
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.mongo-sharded-cluster-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -992,8 +886,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.mongo-sharded-cluster-status }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
be-tests-oracle:
|
||||
needs: determine-driver-skips
|
||||
@@ -1068,8 +960,6 @@ jobs:
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.oracle-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -1081,8 +971,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.oracle-status }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
be-tests-postgres:
|
||||
needs: determine-driver-skips
|
||||
@@ -1125,7 +1013,7 @@ jobs:
|
||||
github.event_name == 'push' &&
|
||||
(
|
||||
github.ref_name == 'master' ||
|
||||
startsWith(github.ref_name, 'release-x')
|
||||
startsWith(github.ref_name, 'release-')
|
||||
) &&
|
||||
''
|
||||
) ||
|
||||
@@ -1176,8 +1064,6 @@ jobs:
|
||||
test-args: >-
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.postgres-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -1189,9 +1075,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.postgres-status }}
|
||||
report-variant: ${{ matrix.version.name }}-${{ matrix.job.name }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
- name: Cleanup python-runner
|
||||
if: always()
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -1268,8 +1151,6 @@ jobs:
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.presto-jdbc-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -1281,8 +1162,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.presto-jdbc-status }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
be-tests-redshift-ee:
|
||||
needs: determine-driver-skips
|
||||
@@ -1356,8 +1235,6 @@ jobs:
|
||||
test-args: >-
|
||||
${{ matrix.job.test-args }}
|
||||
logs-artifact-suffix: ${{ matrix.job.name }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.redshift-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: ${{ !matrix.job.skip && !cancelled() }}
|
||||
@@ -1369,9 +1246,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.redshift-status }}
|
||||
report-variant: ${{ matrix.job.name }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
- name: Cleanup python-runner
|
||||
if: ${{ always() && matrix.job.needs-python-runner == true }}
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -1391,11 +1265,8 @@ jobs:
|
||||
- name: Driver Tests
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags '[:mb/upload-tests :mb/transforms-python-test]'
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
:fail-fast? true
|
||||
- name: Upload Tests
|
||||
test-args: >-
|
||||
:only-tags [:mb/upload-tests]
|
||||
- name: Transforms Python Tests
|
||||
test-args: >-
|
||||
:only-tags [:mb/transforms-python-test]
|
||||
@@ -1444,8 +1315,6 @@ jobs:
|
||||
junit-name: 'be-tests-snowflake-ee'
|
||||
test-args: ${{ matrix.job.test-args }}
|
||||
logs-artifact-suffix: ${{ matrix.job.name }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.snowflake-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -1457,9 +1326,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.snowflake-status }}
|
||||
report-variant: ${{ matrix.job.name }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
- name: Cleanup python-runner
|
||||
if: ${{ always() && matrix.job.needs-python-runner == true }}
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -1501,8 +1367,6 @@ jobs:
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.sparksql-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -1514,8 +1378,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.sparksql-status }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
be-tests-sqlite-ee:
|
||||
needs: determine-driver-skips
|
||||
@@ -1530,7 +1392,7 @@ jobs:
|
||||
- name: Fail on quarantine conflict
|
||||
if: ${{ needs.determine-driver-skips.outputs.sqlite-quarantine-conflict == 'true' }}
|
||||
run: |
|
||||
echo "::error::SQLite driver is quarantined but you changed files that affect it"
|
||||
echo "::error::SQLite driver is quarantined but you changed files in modules/drivers/sqlite/"
|
||||
echo ""
|
||||
echo "Your changes will NOT be tested because the driver is currently quarantined."
|
||||
echo "To run SQLite tests, add the label: break-quarantine-sqlite"
|
||||
@@ -1546,8 +1408,6 @@ jobs:
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.sqlite-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -1559,8 +1419,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.sqlite-status }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
be-tests-sqlserver:
|
||||
needs: determine-driver-skips
|
||||
@@ -1626,8 +1484,6 @@ jobs:
|
||||
junit-name: ${{ matrix.version.junit-name }}
|
||||
test-args: ${{ matrix.job.test-args }}
|
||||
logs-artifact-suffix: ${{ matrix.version.name }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.sqlserver-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -1639,8 +1495,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.sqlserver-status }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
- name: Cleanup python-runner
|
||||
if: ${{ always() && matrix.job.needs-python-runner == true }}
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -1755,8 +1609,6 @@ jobs:
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.vertica-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -1768,24 +1620,16 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.vertica-status }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
drivers-tests-result:
|
||||
needs:
|
||||
# Supplies each driver's configured status (info/skip/required) so this gate can ignore
|
||||
# "info" drivers regardless of their result -- including cancelled or timed-out runs, which
|
||||
# the per-job continue-on-error cannot force green because the controlling step never
|
||||
# completes. Not a driver job; it is filtered out of the pass/fail check below.
|
||||
- determine-driver-skips
|
||||
# if you add a driver job, you must add it to this list for it to be a required check
|
||||
- be-tests-h2
|
||||
- be-tests-athena-ee
|
||||
# Listed unconditionally; ci-test-config controls their effect via per-driver status.
|
||||
# An "info" driver is reported as passing, so it never blocks this check; flip it to
|
||||
# "required" in the config to make a real failure gate, with no change here.
|
||||
- be-tests-snowflake-ee
|
||||
- be-tests-bigquery-cloud-sdk-ee
|
||||
# these are not reliable, but we want to run them anyway to gather data about failures
|
||||
# - be-tests-snowflake-ee
|
||||
# - be-tests-bigquery-cloud-sdk-ee
|
||||
- be-tests-druid-ee
|
||||
- be-tests-databricks-ee
|
||||
- be-tests-druid-jdbc-ee
|
||||
- be-tests-mysql-mariadb
|
||||
@@ -1812,46 +1656,15 @@ jobs:
|
||||
uses: actions/github-script@v9
|
||||
env:
|
||||
needs: ${{ toJson(needs) }}
|
||||
# Job name -> configured driver status, from determine-driver-skips. An "info" driver is
|
||||
# data-collection only and must never gate; a cancelled/timed-out info job lands here as
|
||||
# cancelled/failure (its own continue-on-error can't force it green), so we drop info jobs
|
||||
# from the gate entirely. Empty/unknown status falls back to "required" (gates as before).
|
||||
statuses: |
|
||||
{
|
||||
"be-tests-athena-ee": "${{ needs.determine-driver-skips.outputs.athena-status }}",
|
||||
"be-tests-bigquery-cloud-sdk-ee": "${{ needs.determine-driver-skips.outputs.bigquery-status }}",
|
||||
"be-tests-clickhouse-ee": "${{ needs.determine-driver-skips.outputs.clickhouse-status }}",
|
||||
"be-tests-databricks-ee": "${{ needs.determine-driver-skips.outputs.databricks-status }}",
|
||||
"be-tests-druid-jdbc-ee": "${{ needs.determine-driver-skips.outputs.druid-jdbc-status }}",
|
||||
"be-tests-h2": "${{ needs.determine-driver-skips.outputs.h2-status }}",
|
||||
"be-tests-mongo": "${{ needs.determine-driver-skips.outputs.mongo-status }}",
|
||||
"be-tests-mongo-ssl": "${{ needs.determine-driver-skips.outputs.mongo-ssl-status }}",
|
||||
"be-tests-mongo-sharded-cluster-ee": "${{ needs.determine-driver-skips.outputs.mongo-sharded-cluster-status }}",
|
||||
"be-tests-mysql-mariadb": "${{ needs.determine-driver-skips.outputs.mysql-mariadb-status }}",
|
||||
"be-tests-oracle": "${{ needs.determine-driver-skips.outputs.oracle-status }}",
|
||||
"be-tests-postgres": "${{ needs.determine-driver-skips.outputs.postgres-status }}",
|
||||
"be-tests-presto-jdbc-ee": "${{ needs.determine-driver-skips.outputs.presto-jdbc-status }}",
|
||||
"be-tests-redshift-ee": "${{ needs.determine-driver-skips.outputs.redshift-status }}",
|
||||
"be-tests-snowflake-ee": "${{ needs.determine-driver-skips.outputs.snowflake-status }}",
|
||||
"be-tests-sparksql-ee": "${{ needs.determine-driver-skips.outputs.sparksql-status }}",
|
||||
"be-tests-sqlite-ee": "${{ needs.determine-driver-skips.outputs.sqlite-status }}",
|
||||
"be-tests-sqlserver": "${{ needs.determine-driver-skips.outputs.sqlserver-status }}"
|
||||
}
|
||||
with:
|
||||
script: | # js
|
||||
const needs = JSON.parse(process.env.needs);
|
||||
const statuses = JSON.parse(process.env.statuses);
|
||||
|
||||
// Drop the determine-driver-skips bookkeeping job, then drop "info" drivers: they are
|
||||
// non-gating, so their result (including cancelled/timed-out) must not fail this check.
|
||||
const jobs = Object.entries(needs)
|
||||
.filter(([jobName]) => jobName !== 'determine-driver-skips')
|
||||
.map(([jobName, jobValues]) => ({
|
||||
const jobs = Object.entries(needs).map(
|
||||
([jobName, jobValues]) => ({
|
||||
name: jobName,
|
||||
result: jobValues.result,
|
||||
status: statuses[jobName] || 'required'
|
||||
}))
|
||||
.filter(job => job.status !== 'info');
|
||||
result: jobValues.result
|
||||
}));
|
||||
|
||||
// are all jobs skipped or successful?
|
||||
if (jobs.every(job => (job.result === 'skipped' || job.result === 'success'))) {
|
||||
|
||||
145
.github/workflows/e2e-coverage-manifest.yml
vendored
145
.github/workflows/e2e-coverage-manifest.yml
vendored
@@ -1,145 +0,0 @@
|
||||
name: E2E Coverage Manifest
|
||||
|
||||
# Nightly: builds an instrumented frontend, runs the e2e suite sharded to
|
||||
# record which source files each spec exercises, and publishes a spec -> files
|
||||
# manifest used for selective e2e runs on PRs.
|
||||
#
|
||||
# Isolated from the PR e2e path on purpose - the per-shard worker
|
||||
# (e2e-coverage-shard.yml) is a dedicated, trimmed copy so coverage runs can
|
||||
# never affect PR test results.
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# Nightly, after the day's merges have settled (UTC).
|
||||
- cron: "0 7 * * *"
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
edition:
|
||||
description: "Edition to instrument"
|
||||
type: choice
|
||||
options: ["ee", "oss"]
|
||||
default: "ee"
|
||||
|
||||
concurrency:
|
||||
group: e2e-coverage-manifest-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
build-instrumented:
|
||||
name: Build instrumented uberjar
|
||||
runs-on: ubuntu-22.04
|
||||
timeout-minutes: 60
|
||||
env:
|
||||
MB_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
INSTRUMENT_COVERAGE: "true"
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Prepare front-end environment
|
||||
uses: ./.github/actions/prepare-frontend
|
||||
- name: Prepare back-end environment
|
||||
uses: ./.github/actions/prepare-backend
|
||||
# build.clj forwards INSTRUMENT_COVERAGE to the rspack build, so only the
|
||||
# bundled FE is instrumented; everything else builds normally.
|
||||
- name: Build instrumented uberjar
|
||||
run: ./bin/build.sh
|
||||
- name: Upload instrumented uberjar
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: instrumented-uberjar
|
||||
path: target/uberjar/metabase.jar
|
||||
retention-days: 1
|
||||
|
||||
# Fixed-N sharding — coverage doesn't need the PR matrix-builder's tag/edition
|
||||
# groups, just an even split of the regular spec set. The shard count lives in
|
||||
# one place (SHARDS below); the matrix is generated from it.
|
||||
setup:
|
||||
name: Build shard matrix
|
||||
runs-on: ubuntu-22.04
|
||||
outputs:
|
||||
matrix: ${{ steps.gen.outputs.matrix }}
|
||||
count: ${{ steps.gen.outputs.count }}
|
||||
steps:
|
||||
- id: gen
|
||||
env:
|
||||
SHARDS: 100
|
||||
run: |
|
||||
matrix=$(node -e "console.log(JSON.stringify({split_index:[...Array(Number(process.env.SHARDS)).keys()]}))")
|
||||
echo "matrix=$matrix" >> "$GITHUB_OUTPUT"
|
||||
echo "count=$SHARDS" >> "$GITHUB_OUTPUT"
|
||||
|
||||
shards:
|
||||
name: Shard
|
||||
needs: [build-instrumented, setup]
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix: ${{ fromJson(needs.setup.outputs.matrix) }}
|
||||
uses: ./.github/workflows/e2e-coverage-shard.yml
|
||||
secrets: inherit
|
||||
with:
|
||||
edition: ${{ inputs.edition || 'ee' }}
|
||||
total_chunks: ${{ fromJson(needs.setup.outputs.count) }}
|
||||
split_index: ${{ matrix.split_index }}
|
||||
|
||||
merge:
|
||||
name: Merge and publish manifest
|
||||
needs: shards
|
||||
if: always()
|
||||
runs-on: ubuntu-22.04
|
||||
timeout-minutes: 15
|
||||
# github-script lists/downloads the previous manifest artifact.
|
||||
permissions:
|
||||
actions: read
|
||||
contents: read
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Prepare front-end environment
|
||||
uses: ./.github/actions/prepare-frontend
|
||||
# merge-multiple unions every shard's per-spec dir into one tree; the
|
||||
# per-spec file design means no conflict resolution is needed.
|
||||
- name: Download all shard coverage
|
||||
uses: actions/download-artifact@v8
|
||||
with:
|
||||
pattern: coverage-shard-*
|
||||
path: e2e/coverage-manifest-raw
|
||||
merge-multiple: true
|
||||
# Backfill source: the newest existing manifest is the previous nightly
|
||||
# (this run hasn't uploaded yet). build-coverage-manifest.mjs carries over
|
||||
# any still-existing spec that didn't run today.
|
||||
- name: Download previous manifest for backfill
|
||||
id: prev
|
||||
uses: actions/github-script@v9
|
||||
with:
|
||||
script: | # js
|
||||
const fs = require("fs");
|
||||
const { owner, repo } = context.repo;
|
||||
const { data } = await github.rest.actions.listArtifactsForRepo({
|
||||
owner, repo, name: "spec-file-manifest", per_page: 20,
|
||||
});
|
||||
const art = (data.artifacts ?? [])
|
||||
.filter((a) => !a.expired)
|
||||
.sort((a, b) => new Date(b.created_at) - new Date(a.created_at))[0];
|
||||
if (!art) {
|
||||
console.log("No previous manifest; nothing to backfill.");
|
||||
return;
|
||||
}
|
||||
const dl = await github.rest.actions.downloadArtifact({
|
||||
owner, repo, artifact_id: art.id, archive_format: "zip",
|
||||
});
|
||||
fs.writeFileSync("prev-manifest.zip", Buffer.from(dl.data));
|
||||
core.setOutput("found", "true");
|
||||
- name: Unzip previous manifest
|
||||
if: steps.prev.outputs.found == 'true'
|
||||
run: unzip -o prev-manifest.zip -d prev-manifest
|
||||
# GITHUB_SHA (the master commit this nightly built from) is stamped into
|
||||
# the manifest as `builtAt`, so a PR can later pick the manifest closest
|
||||
# to its own point in history. See resolve-coverage-manifest.mjs.
|
||||
- name: Build coverage manifest
|
||||
env:
|
||||
PREV_MANIFEST: ${{ steps.prev.outputs.found == 'true' && 'prev-manifest/spec-file-manifest.json' || '' }}
|
||||
run: node e2e/coverage/build-coverage-manifest.mjs
|
||||
- name: Upload manifest
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: spec-file-manifest
|
||||
path: e2e/coverage/spec-file-manifest.json
|
||||
retention-days: 30
|
||||
112
.github/workflows/e2e-coverage-shard.yml
vendored
112
.github/workflows/e2e-coverage-shard.yml
vendored
@@ -1,112 +0,0 @@
|
||||
name: E2E Coverage Shard
|
||||
|
||||
# Reusable per-shard worker for the coverage manifest run. Trimmed extract of
|
||||
# e2e-test.yml: runs one cypress-split shard against an instrumented build and
|
||||
# uploads that shard's per-spec coverage. Intentionally omits the PR-only
|
||||
# concerns (failed-spec retry, Trunk quarantine, S3 result upload, timing
|
||||
# output) since this is a nightly, non-blocking job.
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
edition:
|
||||
type: string
|
||||
default: ee
|
||||
total_chunks:
|
||||
type: number
|
||||
required: true
|
||||
split_index:
|
||||
type: number
|
||||
required: true
|
||||
|
||||
env:
|
||||
DISPLAY: ""
|
||||
TERM: xterm
|
||||
TZ: US/Pacific
|
||||
INSTRUMENT_COVERAGE: "true"
|
||||
|
||||
jobs:
|
||||
run:
|
||||
runs-on: ubuntu-22.04
|
||||
timeout-minutes: 60
|
||||
name: coverage-shard-${{ inputs.split_index }}
|
||||
env:
|
||||
MB_EDITION: ${{ inputs.edition }}
|
||||
CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.STAGING_MB_ALL_FEATURES_TOKEN }}
|
||||
CYPRESS_MB_STARTER_CLOUD_TOKEN: ${{ secrets.STAGING_MB_STARTER_CLOUD_TOKEN }}
|
||||
CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.STAGING_MB_PRO_CLOUD_TOKEN }}
|
||||
CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.STAGING_MB_PRO_SELF_HOSTED_TOKEN }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
submodules: true
|
||||
token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
|
||||
|
||||
- name: Prepare Docker containers
|
||||
uses: ./.github/actions/e2e-prepare-containers
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
maildev: true
|
||||
openldap: true
|
||||
webhook: true
|
||||
snowplow: true
|
||||
postgres: true
|
||||
mysql: true
|
||||
mongo: true
|
||||
|
||||
# The orchestrator built and uploaded this in an earlier job of the same run.
|
||||
- name: Download instrumented uberjar
|
||||
uses: actions/download-artifact@v8
|
||||
with:
|
||||
name: instrumented-uberjar
|
||||
path: target/uberjar
|
||||
|
||||
- name: Prepare front-end environment
|
||||
uses: ./.github/actions/prepare-frontend
|
||||
- name: Prepare back-end environment
|
||||
uses: ./.github/actions/prepare-backend
|
||||
- name: Prepare Cypress environment
|
||||
id: cypress-prep
|
||||
uses: ./.github/actions/prepare-cypress
|
||||
|
||||
- name: Compile CLJS
|
||||
run: bun run build-pure:cljs
|
||||
|
||||
- name: Run Metabase
|
||||
run: node e2e/runner/run_cypress_ci.js start
|
||||
|
||||
- name: Generate database snapshots
|
||||
env:
|
||||
CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
|
||||
run: node e2e/runner/run_cypress_ci.js snapshot
|
||||
|
||||
# cypress-split divides the spec set by timing data; INSTRUMENT_COVERAGE
|
||||
# makes the support hook drain __coverage__ into e2e/coverage-manifest-raw/.
|
||||
- name: Run shard ${{ inputs.split_index }}
|
||||
# No SPLIT_FILE: cypress-split shards evenly by spec count using
|
||||
# SPLIT/SPLIT_INDEX alone. Omitting it means no timings are read or
|
||||
# written, which avoids cypress-split's after:run writeback (it would
|
||||
# otherwise try to overwrite ./e2e/support/timings.json from a cwd where
|
||||
# that subpath isn't reachable, crashing the run). Even-by-count is fine
|
||||
# here — every spec runs exactly once across the shards.
|
||||
env:
|
||||
SPLIT: ${{ inputs.total_chunks }}
|
||||
SPLIT_INDEX: ${{ inputs.split_index }}
|
||||
SPLIT_SUMMARY: false
|
||||
CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
|
||||
run: |
|
||||
node e2e/runner/run_cypress_ci.js e2e \
|
||||
--expose grepTags="-@mongo+-@python+-@OSS+-@skip"
|
||||
# Don't let a single failing/flaky spec abort the shard — we still want
|
||||
# coverage from the specs that did run.
|
||||
continue-on-error: true
|
||||
|
||||
- name: Upload shard coverage
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: coverage-shard-${{ inputs.split_index }}
|
||||
path: e2e/coverage-manifest-raw/
|
||||
retention-days: 7
|
||||
if-no-files-found: warn
|
||||
2
.github/workflows/e2e-matrix-builder.yml
vendored
2
.github/workflows/e2e-matrix-builder.yml
vendored
@@ -46,7 +46,7 @@ jobs:
|
||||
const { buildMatrix } = require('${{ github.workspace }}/.github/scripts/build-e2e-matrix.js');
|
||||
const inputSpecs = "${{ inputs.specs }}";
|
||||
const inputChunks = Number(process.env.chunks);
|
||||
const java = 25;
|
||||
const java = 21;
|
||||
const defaultRunner = "${{ vars.DEFAULT_RUNNER_KEY }}";
|
||||
const options = {
|
||||
java,
|
||||
|
||||
69
.github/workflows/e2e-stress-test-flake-fix.yml
vendored
69
.github/workflows/e2e-stress-test-flake-fix.yml
vendored
@@ -36,54 +36,6 @@ on:
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
build_jar:
|
||||
description: "Build the uberjar from this branch (required to test frontend fixes) instead of downloading a prebuilt one"
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
# Reusable form so other workflows (e.g. the flaky-test-fix PR auto-runner) can fan a
|
||||
# matrix of changed specs into individual stress runs. Inputs mirror workflow_dispatch;
|
||||
# both triggers read them via the `inputs` context.
|
||||
workflow_call:
|
||||
inputs:
|
||||
spec:
|
||||
description: "Relative path of the target spec"
|
||||
type: string
|
||||
required: true
|
||||
burn_in:
|
||||
description: "Number of times to run the test (e.g. 20)"
|
||||
type: string
|
||||
required: false
|
||||
default: "20"
|
||||
grep:
|
||||
description: "Grep and filter tests to run in isolation"
|
||||
type: string
|
||||
required: false
|
||||
mb_edition:
|
||||
description: 'Set MB_EDITION env var to "ee" (enterprise) or "oss" (open-source)'
|
||||
type: string
|
||||
required: false
|
||||
default: "ee"
|
||||
qa_db:
|
||||
description: "Whether to set up and include external QA databases"
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
enable_network_throttling:
|
||||
description: "Enable network throttling simulation (4g slow like in chrome dev tools)"
|
||||
type: boolean
|
||||
required: false
|
||||
default: true
|
||||
fail_fast:
|
||||
description: "Stop running remaining tests after the first failure"
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
build_jar:
|
||||
description: "Build the uberjar from this branch (required to test frontend fixes) instead of downloading a prebuilt one"
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
|
||||
jobs:
|
||||
determine-test-type:
|
||||
@@ -121,7 +73,6 @@ jobs:
|
||||
echo '- `qa_db`: "${{ inputs.qa_db }}"' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `enable_network_throttling`: "${{ inputs.enable_network_throttling }}"' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `fail_fast`: "${{ inputs.fail_fast }}"' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `build_jar`: "${{ inputs.build_jar }}"' >> $GITHUB_STEP_SUMMARY
|
||||
echo '' >> $GITHUB_STEP_SUMMARY
|
||||
echo 'triggered by: @${{ github.event.sender.login }}' >> $GITHUB_STEP_SUMMARY
|
||||
|
||||
@@ -158,24 +109,16 @@ jobs:
|
||||
mysql: ${{ inputs.qa_db }}
|
||||
mongo: ${{ inputs.qa_db }}
|
||||
|
||||
- name: Download Metabase ${{ inputs.mb_edition }} uberjar
|
||||
if: ${{ !inputs.build_jar }}
|
||||
- name: Download Metabase ${{ matrix.edition }} uberjar
|
||||
uses: ./.github/actions/e2e-download-uberjar
|
||||
with:
|
||||
edition: ${{ inputs.mb_edition }}
|
||||
edition: "ee"
|
||||
|
||||
- name: Prepare front-end environment
|
||||
uses: ./.github/actions/prepare-frontend
|
||||
- name: Prepare back-end environment
|
||||
uses: ./.github/actions/prepare-backend
|
||||
|
||||
- name: Build Metabase ${{ inputs.mb_edition }} uberjar from this branch
|
||||
if: ${{ inputs.build_jar }}
|
||||
run: ./bin/build.sh
|
||||
env:
|
||||
# license check is irrelevant for ephemeral e2e jars and slows the build down
|
||||
SKIP_LICENSES: true
|
||||
|
||||
- name: Build Embedding SDK package
|
||||
if: ${{ needs.determine-test-type.outputs.type == 'component' }}
|
||||
run: bun run build-embedding-sdk-package
|
||||
@@ -203,13 +146,13 @@ jobs:
|
||||
|
||||
# For all options and fine-grained control, take a look at the documentation
|
||||
# https://github.com/cypress-io/cypress/tree/develop/npm/grep
|
||||
- name: Stress-test ${{ inputs.spec }} ${{ inputs.burn_in }} times
|
||||
- name: Stress-test ${{ github.event.inputs.spec }} ${{ github.event.inputs.burn_in }} times
|
||||
env:
|
||||
GREP: ${{ inputs.grep }}
|
||||
GREP: ${{ github.event.inputs.grep }}
|
||||
CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
|
||||
FAIL_FAST: ${{ inputs.fail_fast }}
|
||||
SPEC: ${{ inputs.spec }}
|
||||
BURN_IN: ${{ inputs.burn_in }}
|
||||
SPEC: ${{ github.event.inputs.spec }}
|
||||
BURN_IN: ${{ github.event.inputs.burn_in }}
|
||||
run: |
|
||||
node e2e/runner/run_cypress_ci.js ${{ needs.determine-test-type.outputs.type }} \
|
||||
--spec "${SPEC}" \
|
||||
|
||||
94
.github/workflows/e2e-stress-test-flaky-pr.yml
vendored
94
.github/workflows/e2e-stress-test-flaky-pr.yml
vendored
@@ -1,94 +0,0 @@
|
||||
name: E2E Stress Test (flaky-test-fix PR)
|
||||
|
||||
# Auto-runs the e2e stress test on every changed Cypress spec in a PR
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
# `opened`/`reopened` catch PRs created already labelled; `labeled` catches the label
|
||||
# being added later; `synchronize` re-burns after new commits while the label is set.
|
||||
types: [opened, reopened, synchronize, labeled]
|
||||
|
||||
concurrency:
|
||||
# One in-flight set of stress runs per PR branch — new commits cancel the previous burn.
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
detect-changed-specs:
|
||||
name: Detect changed e2e specs
|
||||
# The PR must carry the `flaky-test-fix` label (covers opened/reopened/synchronize), AND
|
||||
# when this is a `labeled` event we only proceed if the label *just added* is
|
||||
# `flaky-test-fix` — otherwise adding any unrelated label to an already-labelled
|
||||
# PR would needlessly re-burn the whole stress matrix.
|
||||
if: >-
|
||||
contains(github.event.pull_request.labels.*.name, 'flaky-test-fix') &&
|
||||
(github.event.action != 'labeled' || github.event.label.name == 'flaky-test-fix')
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
outputs:
|
||||
# JSON array of spec paths, consumed by the matrix below.
|
||||
specs: ${{ steps.detect.outputs.specs }}
|
||||
should_run: ${{ steps.detect.outputs.should_run }}
|
||||
# burn-50 / burn-100 labels bump the burn-in past 20
|
||||
burn_in: >-
|
||||
${{ contains(github.event.pull_request.labels.*.name, 'burn-100') && '100'
|
||||
|| contains(github.event.pull_request.labels.*.name, 'burn-50') && '50'
|
||||
|| '20' }}
|
||||
steps:
|
||||
- name: Detect changed Cypress spec files
|
||||
id: detect
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
REPO: ${{ github.repository }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
run: | # sh
|
||||
# List every file in the PR, drop deletions, then keep only regular e2e Cypress
|
||||
# specs: paths under e2e/ ending in .cy.spec.{js,ts,tsx}, excluding the
|
||||
# embedding-SDK component specs under e2e/test-component/ (those use a different
|
||||
# runner and are out of scope here). The greps are guarded with `|| true` so an
|
||||
# empty match doesn't trip `pipefail`.
|
||||
FILES=$(
|
||||
gh api "repos/${REPO}/pulls/${PR_NUMBER}/files" \
|
||||
--paginate \
|
||||
--jq '.[] | select(.status != "removed") | .filename' \
|
||||
| { grep -E '^e2e/.*\.cy\.spec\.(js|ts|tsx)$' || true; } \
|
||||
| { grep -v '^e2e/test-component/' || true; } \
|
||||
| sort -u
|
||||
)
|
||||
|
||||
if [[ -z "$FILES" ]]; then
|
||||
echo "No changed e2e Cypress spec files found — nothing to stress test."
|
||||
echo "should_run=false" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "Changed e2e Cypress specs:"
|
||||
echo "$FILES" | sed 's/^/ - /'
|
||||
|
||||
# Compact JSON array for the matrix, e.g. ["e2e/test/scenarios/a.cy.spec.js"].
|
||||
SPECS_JSON=$(echo "$FILES" | jq -R . | jq -cs .)
|
||||
echo "specs=${SPECS_JSON}" >> "$GITHUB_OUTPUT"
|
||||
echo "should_run=true" >> "$GITHUB_OUTPUT"
|
||||
|
||||
echo "Note: you can use the burn-50 or burn-100 labels to increase the burn-in count from the default of 20." >> $GITHUB_STEP_SUMMARY
|
||||
|
||||
stress-test:
|
||||
name: Stress test
|
||||
needs: detect-changed-specs
|
||||
if: needs.detect-changed-specs.outputs.should_run == 'true'
|
||||
strategy:
|
||||
# Don't let one flaky spec (or one throttling regime) cancel the burn-in of the others.
|
||||
fail-fast: false
|
||||
matrix:
|
||||
spec: ${{ fromJson(needs.detect-changed-specs.outputs.specs) }}
|
||||
# Burn each spec with and without throttle
|
||||
enable_network_throttling: [true, false]
|
||||
uses: ./.github/workflows/e2e-stress-test-flake-fix.yml
|
||||
secrets: inherit
|
||||
with:
|
||||
spec: ${{ matrix.spec }}
|
||||
burn_in: ${{ needs.detect-changed-specs.outputs.burn_in }}
|
||||
qa_db: true
|
||||
mb_edition: ee
|
||||
build_jar: true
|
||||
enable_network_throttling: ${{ matrix.enable_network_throttling }}
|
||||
56
.github/workflows/e2e-test.yml
vendored
56
.github/workflows/e2e-test.yml
vendored
@@ -37,20 +37,6 @@ env:
|
||||
HASH: ${{ github.event.pull_request.head.sha || github.sha }}-${{ github.run_attempt }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number || '' }}
|
||||
JOB_NAME: ${{ inputs.name }}
|
||||
# ci-conductor reporting (DEV-1999). Secret holds the base origin
|
||||
# (e.g. "https://conductor.<host>"); each consumer appends the path it needs
|
||||
# (the reporter POSTs to "/webhooks/failed-tests"). Callers pass
|
||||
# `secrets: inherit`; reporting no-ops when it's unset.
|
||||
CI_CONDUCTOR_BASE_URL: ${{ secrets.CI_CONDUCTOR_BASE_URL }}
|
||||
# Shared secret sent as the `x-internal-secret` header to authenticate.
|
||||
CI_CONDUCTOR_WEBHOOK_SECRET: ${{ secrets.CI_CONDUCTOR_WEBHOOK_SECRET }}
|
||||
REPO_ID: ${{ github.event.repository.id }}
|
||||
# Commit + PR target branch under test, so conductor can tie failures to the
|
||||
# right code. On PRs the ambient GITHUB_SHA is the synthetic merge commit and
|
||||
# GITHUB_BASE_REF the target, so we pass the PR head sha / base ref explicitly
|
||||
# (falling back to push-event values). The reporter reads these. DEV-1999.
|
||||
COMMIT_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
|
||||
TARGET_BRANCH: ${{ github.event.pull_request.base.ref || github.ref_name }}
|
||||
|
||||
jobs:
|
||||
e2e-tests:
|
||||
@@ -59,11 +45,6 @@ jobs:
|
||||
name: e2e-tests-${{ inputs.name }}-${{ inputs.edition }}
|
||||
env:
|
||||
MB_EDITION: ${{ inputs.edition }}
|
||||
# Absolute path for the quarantine failures file (DEV-2082). The after:spec
|
||||
# recorder runs in Cypress' plugins process, whose cwd differs from the
|
||||
# workspace where the gate step runs; an absolute path anchored to the
|
||||
# workspace makes both agree regardless of cwd.
|
||||
QUARANTINE_FAILURES_FILE: ${{ github.workspace }}/target/quarantine-failures.jsonl
|
||||
# Sensitive env vars (tokens, secrets) starting with `CYPRESS_` are accessed via async `cy.env()`
|
||||
# Example: `cy.env(["FOO"]).then(({ FOO }) => ...)`
|
||||
# For non-sensitive config, use `Cypress.expose()` in cypress config instead (synchronous)
|
||||
@@ -131,15 +112,6 @@ jobs:
|
||||
- name: Run Metabase
|
||||
run: node e2e/runner/run_cypress_ci.js start
|
||||
|
||||
# Export this job's numeric GitHub job id as $JOB_ID for the Cypress runs
|
||||
# below (ci-conductor failed-tests reporting reads it in after:spec).
|
||||
# Snapshot generation is itself a Cypress run that fires after:spec, so
|
||||
# this must precede it. Placed late, not after checkout: the Actions jobs
|
||||
# API is eventually consistent, so the longer the job has been alive the
|
||||
# more reliably the lookup resolves. JOB_ID exported here persists to the
|
||||
# steps below. DEV-1999.
|
||||
- uses: ./.github/actions/resolve-job-id
|
||||
|
||||
- name: Generate database snapshots
|
||||
env:
|
||||
CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
|
||||
@@ -164,7 +136,9 @@ jobs:
|
||||
node e2e/runner/run_cypress_ci.js e2e \
|
||||
--expose grepTags="-@mongo+-@python+-@OSS+-@skip" \
|
||||
--spec "$SPEC_PATH"
|
||||
# Keep this step continue-on-error because the quarantine logic controls whether it ultimately fails.
|
||||
# Trunk Quarantine controls the final state of the run in the CI!
|
||||
# Even if this step fails we let it pass, and then `trunk-io/analytics-uploader` determines
|
||||
# whether to fail or to pass the whole job, based on the quarantine list.
|
||||
continue-on-error: true
|
||||
|
||||
- name: Run Tagged EE Cypress tests on ${{ inputs.name }}
|
||||
@@ -177,20 +151,20 @@ jobs:
|
||||
run: |
|
||||
echo "Running tests with specs: $SPEC_PATH"
|
||||
node e2e/runner/run_cypress_ci.js e2e \
|
||||
--expose grepTags="${{inputs.tags && format('{0}+-@skip', inputs.tags) || '-@mongo+-@python+-@OSS+-@skip'}}" \
|
||||
--expose grepTags="${{inputs.tags && format('{0}+-@skip', inputs.tags) || '-@skip'}}" \
|
||||
--spec "$SPEC_PATH"
|
||||
# Keep this step continue-on-error because the quarantine logic controls whether it ultimately fails.
|
||||
# Trunk Quarantine controls the final state of the run in the CI!
|
||||
# Even if this step fails we let it pass, and then `trunk-io/analytics-uploader` determines
|
||||
# whether to fail or to pass the whole job, based on the quarantine list.
|
||||
continue-on-error: true
|
||||
|
||||
- name: Rewrite JUnit hook failures
|
||||
- name: Rewrite JUnit hook failures (dry-run)
|
||||
if: always()
|
||||
continue-on-error: true
|
||||
timeout-minutes: 1
|
||||
shell: bash
|
||||
run: |
|
||||
if [ -d ./target/junit ]; then
|
||||
echo "fix-junit-hooks: invoking bun"
|
||||
bun e2e/support/fix-junit-hooks.ts ./target/junit
|
||||
bun e2e/support/fix-junit-hooks.ts --dry-run ./target/junit
|
||||
else
|
||||
echo "No ./target/junit directory; nothing to process."
|
||||
fi
|
||||
@@ -209,18 +183,6 @@ jobs:
|
||||
variant: e2e-tests
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ inputs.specs && steps.tagged-e2e.outcome || steps.split-e2e.outcome }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
- name: Quarantine gate
|
||||
# Only one of the two run steps executes; the other reports `skipped`. Only gate on actual failure.
|
||||
if: ${{ !cancelled() && (steps.split-e2e.outcome == 'failure' || steps.tagged-e2e.outcome == 'failure') }}
|
||||
continue-on-error: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
timeout-minutes: 3
|
||||
uses: ./.github/actions/quarantine-gate
|
||||
with:
|
||||
adapter: e2e
|
||||
test-suite: e2e
|
||||
enforce-quarantine: ${{ vars.QUARANTINE_ENGINE == 'ci-conductor' }}
|
||||
|
||||
- name: Upload Cypress Artifacts upon failure
|
||||
uses: actions/upload-artifact@v7
|
||||
|
||||
@@ -8,12 +8,6 @@ env:
|
||||
TERM: xterm
|
||||
TZ: US/Pacific # to make Node match the instance tz
|
||||
MB_EDITION: "ee"
|
||||
CI_CONDUCTOR_BASE_URL: ${{ secrets.CI_CONDUCTOR_BASE_URL }}
|
||||
CI_CONDUCTOR_WEBHOOK_SECRET: ${{ secrets.CI_CONDUCTOR_WEBHOOK_SECRET }}
|
||||
REPO_ID: ${{ github.event.repository.id }}
|
||||
COMMIT_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
|
||||
TARGET_BRANCH: ${{ github.event.pull_request.base.ref || github.ref_name }}
|
||||
QUARANTINE_FAILURES_FILE: ${{ github.workspace }}/target/quarantine-failures.jsonl
|
||||
|
||||
jobs:
|
||||
sdk-component-tests:
|
||||
@@ -25,7 +19,6 @@ jobs:
|
||||
react-version: [18, 19]
|
||||
fail-fast: false
|
||||
env:
|
||||
CI_CONDUCTOR_TEST_SUITE: embedding-sdk-component-react-${{ matrix.react-version }}
|
||||
# Sensitive env vars (tokens, secrets) starting with `CYPRESS_` are accessed via async `cy.env()`
|
||||
# Example: `cy.env(["FOO"]).then(({ FOO }) => ...)`
|
||||
# For non-sensitive config, use `Cypress.expose()` in cypress config instead (synchronous)
|
||||
@@ -45,7 +38,7 @@ jobs:
|
||||
maildev: true
|
||||
openldap: false
|
||||
webhook: false
|
||||
snowplow: true
|
||||
snowplow: false
|
||||
postgres: false
|
||||
mysql: false
|
||||
mongo: false
|
||||
@@ -76,8 +69,6 @@ jobs:
|
||||
- name: Run Metabase
|
||||
run: node e2e/runner/run_cypress_ci.js start
|
||||
|
||||
- uses: ./.github/actions/resolve-job-id
|
||||
|
||||
- name: Make app db snapshot
|
||||
env:
|
||||
CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
|
||||
@@ -95,18 +86,14 @@ jobs:
|
||||
env:
|
||||
CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
|
||||
run: node e2e/runner/run_cypress_ci.js component
|
||||
# Keep this step continue-on-error because the quarantine logic controls whether it ultimately fails.
|
||||
continue-on-error: true
|
||||
|
||||
- name: Rewrite JUnit hook failures
|
||||
- name: Rewrite JUnit hook failures (dry-run)
|
||||
if: always()
|
||||
continue-on-error: true
|
||||
timeout-minutes: 1
|
||||
shell: bash
|
||||
run: |
|
||||
if [ -d ./target/junit ]; then
|
||||
echo "fix-junit-hooks: invoking bun"
|
||||
bun e2e/support/fix-junit-hooks.ts ./target/junit
|
||||
bun e2e/support/fix-junit-hooks.ts --dry-run ./target/junit
|
||||
else
|
||||
echo "No ./target/junit directory; nothing to process."
|
||||
fi
|
||||
@@ -122,21 +109,10 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.run-tests.outcome }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
- name: Quarantine gate
|
||||
if: ${{ !cancelled() && steps.run-tests.outcome == 'failure' }}
|
||||
continue-on-error: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
timeout-minutes: 3
|
||||
uses: ./.github/actions/quarantine-gate
|
||||
with:
|
||||
adapter: e2e
|
||||
test-suite: ${{ env.CI_CONDUCTOR_TEST_SUITE }}
|
||||
enforce-quarantine: ${{ vars.QUARANTINE_ENGINE == 'ci-conductor' }}
|
||||
|
||||
- name: "SDK component tests > Upload Cypress Artifacts upon failure"
|
||||
uses: actions/upload-artifact@v7
|
||||
if: ${{ !cancelled() && steps.run-tests.outcome == 'failure' }}
|
||||
if: failure()
|
||||
with:
|
||||
name: cypress-recording-embedding-sdk-react-${{ matrix.react-version }}
|
||||
path: |
|
||||
@@ -145,7 +121,7 @@ jobs:
|
||||
if-no-files-found: ignore
|
||||
|
||||
- name: "SDK component tests > Publish Summary"
|
||||
if: ${{ !cancelled() && steps.run-tests.outcome == 'failure' }}
|
||||
if: failure()
|
||||
uses: actions/github-script@v9
|
||||
with:
|
||||
script: | #js
|
||||
@@ -190,7 +166,6 @@ jobs:
|
||||
env:
|
||||
MB_RUN_MODE: "e2e"
|
||||
METASTORE_DEV_SERVER_URL: ${{ secrets.METASTORE_DEV_SERVER_URL }}
|
||||
CI_CONDUCTOR_TEST_SUITE: embedding-sdk-component-backward-compatibility
|
||||
# Sensitive env vars (tokens, secrets) starting with `CYPRESS_` are accessed via async `cy.env()`
|
||||
# Example: `cy.env(["FOO"]).then(({ FOO }) => ...)`
|
||||
# For non-sensitive config, use `Cypress.expose()` in cypress config instead (synchronous)
|
||||
@@ -213,7 +188,7 @@ jobs:
|
||||
maildev: true
|
||||
openldap: false
|
||||
webhook: false
|
||||
snowplow: true
|
||||
snowplow: false
|
||||
postgres: false
|
||||
mysql: false
|
||||
mongo: false
|
||||
@@ -260,8 +235,6 @@ jobs:
|
||||
- name: Run Metabase
|
||||
run: node e2e/runner/run_cypress_ci.js start
|
||||
|
||||
- uses: ./.github/actions/resolve-job-id
|
||||
|
||||
- name: Make app db snapshot
|
||||
env:
|
||||
CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
|
||||
@@ -277,18 +250,14 @@ jobs:
|
||||
run: |
|
||||
node e2e/runner/run_cypress_ci.js component \
|
||||
--expose grepTags="-@skip-backward-compatibility"
|
||||
# Keep this step continue-on-error because the quarantine logic controls whether it ultimately fails.
|
||||
continue-on-error: true
|
||||
|
||||
- name: Rewrite JUnit hook failures
|
||||
- name: Rewrite JUnit hook failures (dry-run)
|
||||
if: always()
|
||||
continue-on-error: true
|
||||
timeout-minutes: 1
|
||||
shell: bash
|
||||
run: |
|
||||
if [ -d ./target/junit ]; then
|
||||
echo "fix-junit-hooks: invoking bun"
|
||||
bun e2e/support/fix-junit-hooks.ts ./target/junit
|
||||
bun e2e/support/fix-junit-hooks.ts --dry-run ./target/junit
|
||||
else
|
||||
echo "No ./target/junit directory; nothing to process."
|
||||
fi
|
||||
@@ -304,21 +273,10 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.run-tests.outcome }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
- name: Quarantine gate
|
||||
if: ${{ !cancelled() && steps.run-tests.outcome == 'failure' }}
|
||||
continue-on-error: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
timeout-minutes: 3
|
||||
uses: ./.github/actions/quarantine-gate
|
||||
with:
|
||||
adapter: e2e
|
||||
test-suite: ${{ env.CI_CONDUCTOR_TEST_SUITE }}
|
||||
enforce-quarantine: ${{ vars.QUARANTINE_ENGINE == 'ci-conductor' }}
|
||||
|
||||
- name: Upload Cypress Artifacts upon failure
|
||||
uses: actions/upload-artifact@v7
|
||||
if: ${{ !cancelled() && steps.run-tests.outcome == 'failure' }}
|
||||
if: failure()
|
||||
with:
|
||||
name: cypress-recording-embedding-sdk-cross-version
|
||||
path: |
|
||||
@@ -327,7 +285,7 @@ jobs:
|
||||
if-no-files-found: ignore
|
||||
|
||||
- name: Publish Summary
|
||||
if: ${{ !cancelled() && steps.run-tests.outcome == 'failure' }}
|
||||
if: failure()
|
||||
uses: actions/github-script@v9
|
||||
with:
|
||||
script: | #js
|
||||
|
||||
@@ -171,8 +171,6 @@ jobs:
|
||||
- name: Log free disk space
|
||||
run: df -h /
|
||||
|
||||
- uses: ./.github/actions/resolve-job-id
|
||||
|
||||
- name: Run e2e tests for Sample App
|
||||
id: run-e2e-tests
|
||||
if: ${{ steps.check-sample-app-branch.outputs.sample_app_branch_exists == 'true' }}
|
||||
@@ -240,8 +238,6 @@ jobs:
|
||||
shoppy-datadog-application-id: ${{ secrets.SHOPPY_METABASE_APP_DATADOG_APPLICATION_ID }}
|
||||
shoppy-datadog-client-token: ${{ secrets.SHOPPY_METABASE_APP_DATADOG_CLIENT_TOKEN }}
|
||||
|
||||
- uses: ./.github/actions/resolve-job-id
|
||||
|
||||
- name: Run e2e tests for Shoppy
|
||||
id: run-e2e-tests
|
||||
if: ${{ steps.shoppy-prep.outputs.sample_app_branch_exists == 'true' }}
|
||||
@@ -320,8 +316,6 @@ jobs:
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- uses: ./.github/actions/resolve-job-id
|
||||
|
||||
- name: Run synthetic monitoring tests
|
||||
id: run-e2e-tests
|
||||
if: ${{ steps.shoppy-prep.outputs.sample_app_branch_exists == 'true' }}
|
||||
@@ -369,19 +363,6 @@ jobs:
|
||||
timeout-minutes: 45
|
||||
name: e2e-host-app-${{ matrix.test_suite }}-tests-${{ matrix.environment }}-bundle
|
||||
env:
|
||||
# Host apps are the only suite here running through Metabase's Cypress config,
|
||||
# whose `after:spec` hook reports failures and writes QUARANTINE_FAILURES_FILE.
|
||||
# Sample-app suites load their own config (no-op), so ci-conductor is scoped to this job.
|
||||
CI_CONDUCTOR_BASE_URL: ${{ secrets.CI_CONDUCTOR_BASE_URL }}
|
||||
CI_CONDUCTOR_WEBHOOK_SECRET: ${{ secrets.CI_CONDUCTOR_WEBHOOK_SECRET }}
|
||||
CI_CONDUCTOR_TEST_SUITE: embedding-sdk-host-app
|
||||
REPO_ID: ${{ github.event.repository.id }}
|
||||
COMMIT_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
|
||||
TARGET_BRANCH: ${{ github.event.pull_request.base.ref || github.ref_name }}
|
||||
QUARANTINE_FAILURES_FILE: ${{ github.workspace }}/target/quarantine-failures.jsonl
|
||||
# `Fail job if tests failed` is the arbiter until ci-conductor takes over.
|
||||
# Exactly one of the two gates the job.
|
||||
CI_CONDUCTOR_QUARANTINE_ACTIVE: ${{ vars.QUARANTINE_ENGINE == 'ci-conductor' }}
|
||||
CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.STAGING_MB_ALL_FEATURES_TOKEN }}
|
||||
HOST_APP_ENVIRONMENT: ${{ matrix.environment }}
|
||||
permissions:
|
||||
@@ -418,8 +399,6 @@ jobs:
|
||||
- name: Run Metabase
|
||||
run: node e2e/runner/run_cypress_ci.js start
|
||||
|
||||
- uses: ./.github/actions/resolve-job-id
|
||||
|
||||
- name: Generate database snapshots
|
||||
run: |
|
||||
node e2e/runner/run_cypress_ci.js snapshot \
|
||||
@@ -437,24 +416,14 @@ jobs:
|
||||
|
||||
- name: Upload Cypress Artifacts upon failure
|
||||
uses: actions/upload-artifact@v7
|
||||
if: ${{ !cancelled() && steps.run-e2e-tests.outcome == 'failure' }}
|
||||
if: ${{ steps.run-e2e-tests.outcome != 'success' }}
|
||||
with:
|
||||
name: cypress-recording-host-app-${{ matrix.test_suite }}-${{ matrix.environment }}-latest
|
||||
path: |
|
||||
./cypress
|
||||
if-no-files-found: ignore
|
||||
|
||||
- name: Quarantine gate
|
||||
if: ${{ !cancelled() && steps.run-e2e-tests.outcome == 'failure' }}
|
||||
continue-on-error: ${{ env.CI_CONDUCTOR_QUARANTINE_ACTIVE != 'true' }}
|
||||
timeout-minutes: 3
|
||||
uses: ./.github/actions/quarantine-gate
|
||||
with:
|
||||
adapter: e2e
|
||||
test-suite: ${{ env.CI_CONDUCTOR_TEST_SUITE }}
|
||||
enforce-quarantine: ${{ env.CI_CONDUCTOR_QUARANTINE_ACTIVE }}
|
||||
|
||||
- name: Fail job if tests failed
|
||||
if: ${{ !cancelled() && steps.run-e2e-tests.outcome == 'failure' && env.CI_CONDUCTOR_QUARANTINE_ACTIVE != 'true' }}
|
||||
if: ${{ steps.run-e2e-tests.outcome != 'success' }}
|
||||
run: |
|
||||
exit 1
|
||||
|
||||
45
.github/workflows/embedding-sdk-tag-latest.yml
vendored
45
.github/workflows/embedding-sdk-tag-latest.yml
vendored
@@ -1,45 +0,0 @@
|
||||
name: Tag Embedding SDK npm release as latest
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
version:
|
||||
description: "npm version to tag as latest, e.g. 0.61.5"
|
||||
type: string
|
||||
required: true
|
||||
workflow_call:
|
||||
inputs:
|
||||
version:
|
||||
description: "npm version to tag as latest, e.g. 0.61.5"
|
||||
type: string
|
||||
required: true
|
||||
|
||||
concurrency:
|
||||
group: npm-dist-tag-latest
|
||||
cancel-in-progress: false
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
tag-latest:
|
||||
name: Tag npm release as latest
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
env:
|
||||
VERSION: ${{ inputs.version }}
|
||||
steps:
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v6.4.0
|
||||
with:
|
||||
node-version: 24
|
||||
registry-url: https://registry.npmjs.org
|
||||
|
||||
- name: Verify version exists on npm
|
||||
run: npm view "@metabase/embedding-sdk-react@$VERSION" version
|
||||
|
||||
- name: Add `latest` dist-tag to npm release
|
||||
run: |
|
||||
npm dist-tag add "@metabase/embedding-sdk-react@$VERSION" latest
|
||||
echo "Tagged @metabase/embedding-sdk-react@$VERSION as latest" >> "$GITHUB_STEP_SUMMARY"
|
||||
env:
|
||||
NODE_AUTH_TOKEN: ${{ secrets.NPM_RELEASE_TOKEN }}
|
||||
6
.github/workflows/embedding-sdk.yml
vendored
6
.github/workflows/embedding-sdk.yml
vendored
@@ -150,7 +150,7 @@ jobs:
|
||||
with:
|
||||
remove-previous-comment: true
|
||||
add-new-comment: true
|
||||
message: "⚠️ Embedding SDK chunked bundle size (gzipped) increased by ${{ steps.compare.outputs.size_change_percent }}% compared to the ${{ github.base_ref }} branch (threshold: ${{ env.BUNDLE_SIZE_THRESHOLD_PERCENT }}%). Please contact Embedding team."
|
||||
message: "⚠️ Embedding SDK bundle size increased by ${{ steps.compare.outputs.size_change_percent }}% compared to the ${{ github.base_ref }} branch (threshold: ${{ env.BUNDLE_SIZE_THRESHOLD_PERCENT }}%). Please contact Embedding team."
|
||||
include-log: false
|
||||
|
||||
- name: Send notification to Slack via Zapier
|
||||
@@ -165,7 +165,7 @@ jobs:
|
||||
PR_URL: ${{ github.event.pull_request.html_url }}
|
||||
run: |
|
||||
curl -s -X POST "$ZAPIER_WEBHOOK_URL" \
|
||||
-d "⚠️ Embedding SDK chunked bundle size (gzipped) increased by ${SIZE_CHANGE_PERCENT}% (threshold: ${BUNDLE_SIZE_THRESHOLD_PERCENT}%)
|
||||
-d "⚠️ Embedding SDK bundle size increased by ${SIZE_CHANGE_PERCENT}% (threshold: ${BUNDLE_SIZE_THRESHOLD_PERCENT}%)
|
||||
|
||||
PR #${PR_NUMBER} ${PR_TITLE} (${PR_URL})"
|
||||
|
||||
@@ -175,7 +175,7 @@ jobs:
|
||||
with:
|
||||
remove-previous-comment: true
|
||||
add-new-comment: true
|
||||
message: "🎉 Embedding SDK chunked bundle size (gzipped) decreased by ${{ steps.compare.outputs.size_change_percent }}% compared to the ${{ github.base_ref }} branch. Nice work!"
|
||||
message: "🎉 Embedding SDK bundle size decreased by ${{ steps.compare.outputs.size_change_percent }}% compared to the ${{ github.base_ref }} branch. Nice work!"
|
||||
include-log: false
|
||||
|
||||
embedding-sdk-tests-result:
|
||||
|
||||
33
.github/workflows/frontend.yml
vendored
33
.github/workflows/frontend.yml
vendored
@@ -31,13 +31,6 @@ concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}-frontend
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
# Consumed by the report-failures-to-ci-conductor step in fe-tests-unit. Absent
|
||||
# (e.g. fork PRs without secret access) the reporter no-ops. Matches the other
|
||||
# suite workflows (backend, drivers, e2e, app-db, semantic-search).
|
||||
CI_CONDUCTOR_BASE_URL: ${{ secrets.CI_CONDUCTOR_BASE_URL }}
|
||||
CI_CONDUCTOR_WEBHOOK_SECRET: ${{ secrets.CI_CONDUCTOR_WEBHOOK_SECRET }}
|
||||
|
||||
jobs:
|
||||
fe-lint:
|
||||
if: ${{ !inputs.skip-lint }}
|
||||
@@ -103,32 +96,13 @@ jobs:
|
||||
uses: ./.github/actions/prepare-frontend
|
||||
- name: Prepare back-end environment
|
||||
uses: ./.github/actions/prepare-backend
|
||||
- name: Resolve current job id
|
||||
# Resolve JOB_ID before the long test step so the ci-conductor reporter
|
||||
# below can link failures to this exact job. Best-effort.
|
||||
uses: ./.github/actions/resolve-job-id
|
||||
- name: Run frontend unit tests
|
||||
id: fe-tests
|
||||
run: bun run test-unit --silent --shard=${{ matrix.shard }}/${{ env.SHARDS }}
|
||||
# Keep this step continue-on-error because the quarantine logic controls whether it ultimately fails.
|
||||
# Trunk Quarantine controls the final state of the run in the CI!
|
||||
# Even if this step fails we let it pass, and then `trunk-io/analytics-uploader` determines
|
||||
# whether to fail or to pass the whole job, based on the quarantine list.
|
||||
continue-on-error: true
|
||||
- name: Report failed tests to ci-conductor
|
||||
if: ${{ !cancelled() && steps.fe-tests.outcome == 'failure' }}
|
||||
# This step should NEVER fail the job!
|
||||
continue-on-error: true
|
||||
uses: ./.github/actions/report-failures-to-ci-conductor
|
||||
with:
|
||||
test-suite: fe-tests-unit
|
||||
adapter: frontend
|
||||
- name: Quarantine gate
|
||||
if: ${{ !cancelled() && steps.fe-tests.outcome == 'failure' }}
|
||||
continue-on-error: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
timeout-minutes: 3
|
||||
uses: ./.github/actions/quarantine-gate
|
||||
with:
|
||||
adapter: frontend
|
||||
test-suite: fe-tests-unit
|
||||
enforce-quarantine: ${{ vars.QUARANTINE_ENGINE == 'ci-conductor' }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always() && github.event_name != 'workflow_dispatch'
|
||||
@@ -140,7 +114,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.fe-tests.outcome }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
fe-tests-timezones:
|
||||
if: ${{ !inputs.skip }}
|
||||
|
||||
@@ -13,10 +13,10 @@ on:
|
||||
type: string
|
||||
required: true
|
||||
burn_in:
|
||||
description: "Number of times to run the test (e.g. 10)"
|
||||
description: "Number of times to run the test (e.g. 50)"
|
||||
type: string
|
||||
required: true
|
||||
default: "10"
|
||||
default: "50"
|
||||
|
||||
jobs:
|
||||
detect-changed-stories:
|
||||
@@ -93,7 +93,7 @@ jobs:
|
||||
env:
|
||||
BRANCH_NAME: ${{ github.head_ref || github.ref_name }}
|
||||
STORY_FILES: ${{ needs.detect-changed-stories.outputs.story_files }}
|
||||
BURN_IN: ${{ inputs.burn_in || 10 }}
|
||||
BURN_IN: ${{ inputs.burn_in || 50 }}
|
||||
EVENT_NAME: ${{ github.event_name }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
PR_ACTOR: ${{ github.event.pull_request.user.login }}
|
||||
@@ -154,7 +154,7 @@ jobs:
|
||||
|
||||
- name: Stress-test Loki
|
||||
run: |
|
||||
BURN_IN=${{ inputs.burn_in || '10' }}
|
||||
BURN_IN=${{ inputs.burn_in || '50' }}
|
||||
STORY_FILES='${{ needs.detect-changed-stories.outputs.story_files }}'
|
||||
|
||||
echo "Story files: $STORY_FILES"
|
||||
|
||||
2
.github/workflows/loki-update.yml
vendored
2
.github/workflows/loki-update.yml
vendored
@@ -42,7 +42,7 @@ jobs:
|
||||
git config --global user.name "Metabase Automation"
|
||||
git config --global user.email "github-automation@metabase.com"
|
||||
git add .loki/reference/
|
||||
git commit -m "Update Loki Snapshots" --no-verify
|
||||
git commit -m "Update Loki Snapshots"
|
||||
git push
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
|
||||
|
||||
@@ -13,8 +13,6 @@ jobs:
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
|
||||
with:
|
||||
bun-version-file: 'package.json'
|
||||
- name: Install dependencies
|
||||
run: bun install --frozen-lockfile
|
||||
- name: get module boundary stats
|
||||
|
||||
9
.github/workflows/pr-env-close-unlabel.yml
vendored
9
.github/workflows/pr-env-close-unlabel.yml
vendored
@@ -10,7 +10,7 @@ concurrency:
|
||||
|
||||
jobs:
|
||||
pr-env:
|
||||
if: ${{ (github.event.action == 'closed' && contains(github.event.pull_request.labels.*.name, 'PR-Env') && !contains(github.event.pull_request.labels.*.name, 'PR-Env-self-hosting') && !contains(github.event.pull_request.labels.*.name, 'PR-Env-analytics-dev') && !contains(github.event.pull_request.labels.*.name, 'PR-Env-Stats-Dump')) || github.event.label.name == 'PR-Env' }}
|
||||
if: ${{ (github.event.action == 'closed' && contains(github.event.pull_request.labels.*.name, 'PR-Env') && !contains(github.event.pull_request.labels.*.name, 'PR-Env-self-hosting') && !contains(github.event.pull_request.labels.*.name, 'PR-Env-analytics-dev')) || github.event.label.name == 'PR-Env' }}
|
||||
uses: ./.github/workflows/pr-env-destroy.yml
|
||||
secrets: inherit
|
||||
with:
|
||||
@@ -29,10 +29,3 @@ jobs:
|
||||
secrets: inherit
|
||||
with:
|
||||
self_hosting: false
|
||||
|
||||
pr-env-stats-dump:
|
||||
if: ${{ (github.event.action == 'closed' && contains(github.event.pull_request.labels.*.name, 'PR-Env-Stats-Dump')) || github.event.label.name == 'PR-Env-Stats-Dump' }}
|
||||
uses: ./.github/workflows/pr-env-destroy.yml
|
||||
secrets: inherit
|
||||
with:
|
||||
self_hosting: false
|
||||
|
||||
12
.github/workflows/pr-env-labeled.yml
vendored
12
.github/workflows/pr-env-labeled.yml
vendored
@@ -10,7 +10,7 @@ concurrency:
|
||||
|
||||
jobs:
|
||||
uberjar:
|
||||
if: ${{ github.event.label.name == 'PR-Env' || github.event.label.name == 'PR-Env-self-hosting' || github.event.label.name == 'PR-Env-analytics-dev' || github.event.label.name == 'PR-Env-Stats-Dump' }}
|
||||
if: ${{ github.event.label.name == 'PR-Env' || github.event.label.name == 'PR-Env-self-hosting' || github.event.label.name == 'PR-Env-analytics-dev' }}
|
||||
uses: ./.github/workflows/uberjar.yml
|
||||
secrets: inherit
|
||||
|
||||
@@ -40,13 +40,3 @@ jobs:
|
||||
with:
|
||||
self_hosting: false
|
||||
analytics_dev_mode: true
|
||||
|
||||
pr-env-stats-dump:
|
||||
needs: [uberjar]
|
||||
if: ${{ github.event.label.name == 'PR-Env-Stats-Dump' }}
|
||||
uses: ./.github/workflows/pr-env.yml
|
||||
secrets: inherit
|
||||
with:
|
||||
self_hosting: false
|
||||
analytics_dev_mode: false
|
||||
stats_dump: true
|
||||
|
||||
209
.github/workflows/pr-env.yml
vendored
209
.github/workflows/pr-env.yml
vendored
@@ -14,11 +14,6 @@ on:
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
stats_dump:
|
||||
description: "Load a recent app DB stats dump on first env creation"
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
@@ -79,169 +74,6 @@ jobs:
|
||||
tags: tag:ci
|
||||
version: 1.50.1
|
||||
sha256sum: d9fe6b480fb5078f0aa57dace686898dda7e2a768884271159faa74846bfb576
|
||||
- name: Configure AWS credentials
|
||||
uses: aws-actions/configure-aws-credentials@d979d5b3a71173a29b74b5b88418bfda9437d885 # v6.1.1
|
||||
with:
|
||||
role-to-assume: ${{ secrets.PR_ENV_IAM_ROLE }}
|
||||
role-session-name: GitHub_to_AWS_via_FederatedOIDC
|
||||
aws-region: us-east-1
|
||||
- name: Setup psql client
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y postgresql-client
|
||||
- name: Create app database if not exists
|
||||
id: create_app_db
|
||||
run: |
|
||||
export PGPASSWORD="${PR_ENV_DB_PASSWORD}"
|
||||
export PGUSER="${PR_ENV_DB_USER}"
|
||||
export PGHOST="${PR_ENV_DB_HOST}"
|
||||
export PGDATABASE="${PR_ENV_DB_NAME}"
|
||||
if psql -tc "SELECT 1 FROM pg_database WHERE datname = 'hosting_pr${{ github.event.number }}'" | grep -q 1; then
|
||||
echo "db_created=false" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
psql -c "CREATE DATABASE hosting_pr${{ github.event.number }}"
|
||||
echo "db_created=true" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
env:
|
||||
PR_ENV_DB_PASSWORD: ${{ secrets.PR_ENV_DB_PASSWORD }}
|
||||
PR_ENV_DB_USER: ${{ secrets.PR_ENV_DB_USER }}
|
||||
PR_ENV_DB_HOST: ${{ secrets.PR_ENV_DB_HOST }}
|
||||
PR_ENV_DB_NAME: ${{ secrets.PR_ENV_DB_NAME }}
|
||||
- name: Verify stats dump load status
|
||||
id: verify_stats_dump
|
||||
if: ${{ inputs.stats_dump }}
|
||||
run: |
|
||||
export PGPASSWORD="${PR_ENV_DB_PASSWORD}"
|
||||
ADMIN="psql -h ${PR_ENV_DB_HOST} -U ${PR_ENV_DB_USER} -d ${PR_ENV_DB_NAME}"
|
||||
MARKED=$($ADMIN -tAc "SELECT 1 FROM pg_database d WHERE d.datname='hosting_pr${{ github.event.number }}' AND shobj_description(d.oid, 'pg_database') = 'metabase-pr-env stats-dump loaded';")
|
||||
if [ "$MARKED" = "1" ]; then
|
||||
echo "Stats dump already loaded; skipping reload."
|
||||
echo "should_load=false" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "Stats dump not loaded (or previous load incomplete); recreating database."
|
||||
$ADMIN -c "ALTER DATABASE \"hosting_pr${{ github.event.number }}\" CONNECTION LIMIT 0;" || true
|
||||
$ADMIN -c "SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname='hosting_pr${{ github.event.number }}' AND pid <> pg_backend_pid();" || true
|
||||
$ADMIN -c "DROP DATABASE IF EXISTS \"hosting_pr${{ github.event.number }}\" WITH (FORCE);"
|
||||
$ADMIN -c "CREATE DATABASE \"hosting_pr${{ github.event.number }}\";"
|
||||
echo "should_load=true" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
env:
|
||||
PR_ENV_DB_PASSWORD: ${{ secrets.PR_ENV_DB_PASSWORD }}
|
||||
PR_ENV_DB_USER: ${{ secrets.PR_ENV_DB_USER }}
|
||||
PR_ENV_DB_HOST: ${{ secrets.PR_ENV_DB_HOST }}
|
||||
PR_ENV_DB_NAME: ${{ secrets.PR_ENV_DB_NAME }}
|
||||
- name: Locate latest stats dump in S3
|
||||
id: locate_stats_dump
|
||||
if: ${{ inputs.stats_dump && steps.verify_stats_dump.outputs.should_load == 'true' }}
|
||||
env:
|
||||
STATS_DUMP_BUCKET: metabase-cloud-stats-backups-coredev
|
||||
run: |
|
||||
LATEST_KEY=$(aws s3api list-objects-v2 \
|
||||
--bucket "${STATS_DUMP_BUCKET}" \
|
||||
--query "reverse(sort_by(Contents[?ends_with(Key, '.dump')], &LastModified))[0].Key" \
|
||||
--output text)
|
||||
if [ -z "$LATEST_KEY" ] || [ "$LATEST_KEY" = "None" ]; then
|
||||
echo "No *.dump found in s3://${STATS_DUMP_BUCKET}" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "Latest dump: $LATEST_KEY"
|
||||
echo "key=$LATEST_KEY" >> "$GITHUB_OUTPUT"
|
||||
- name: Download stats dump
|
||||
if: ${{ inputs.stats_dump && steps.verify_stats_dump.outputs.should_load == 'true' }}
|
||||
env:
|
||||
STATS_DUMP_BUCKET: metabase-cloud-stats-backups-coredev
|
||||
run: |
|
||||
aws s3 cp --no-progress \
|
||||
"s3://${STATS_DUMP_BUCKET}/${{ steps.locate_stats_dump.outputs.key }}" \
|
||||
/tmp/stats-dump.dump
|
||||
echo "Downloaded $(stat -c%s /tmp/stats-dump.dump) bytes"
|
||||
file /tmp/stats-dump.dump
|
||||
- name: Prepare app database schema
|
||||
if: ${{ inputs.stats_dump && steps.verify_stats_dump.outputs.should_load == 'true' }}
|
||||
run: |
|
||||
PGPASSWORD="${PR_ENV_DB_PASSWORD}" psql \
|
||||
-h "${PR_ENV_DB_HOST}" \
|
||||
-U "${PR_ENV_DB_USER}" \
|
||||
-d "hosting_pr${{ github.event.number }}" \
|
||||
--no-align --tuples-only --expanded \
|
||||
--command='DROP SCHEMA public CASCADE; CREATE SCHEMA public; CREATE EXTENSION IF NOT EXISTS citext WITH SCHEMA public;'
|
||||
env:
|
||||
PR_ENV_DB_PASSWORD: ${{ secrets.PR_ENV_DB_PASSWORD }}
|
||||
PR_ENV_DB_USER: ${{ secrets.PR_ENV_DB_USER }}
|
||||
PR_ENV_DB_HOST: ${{ secrets.PR_ENV_DB_HOST }}
|
||||
- name: Restore stats dump into app database
|
||||
if: ${{ inputs.stats_dump && steps.verify_stats_dump.outputs.should_load == 'true' }}
|
||||
run: |
|
||||
set +e
|
||||
echo "::group::pg_restore / pg client versions"
|
||||
pg_restore --version
|
||||
psql --version
|
||||
echo "::endgroup::"
|
||||
|
||||
echo "::group::Server version"
|
||||
PGPASSWORD="${PR_ENV_DB_PASSWORD}" psql \
|
||||
-h "${PR_ENV_DB_HOST}" -U "${PR_ENV_DB_USER}" -d "${PR_ENV_DB_NAME}" \
|
||||
-tAc 'SHOW server_version;'
|
||||
|
||||
echo "::group::pg_restore"
|
||||
PGPASSWORD="${PR_ENV_DB_PASSWORD}" pg_restore \
|
||||
-h "${PR_ENV_DB_HOST}" \
|
||||
-U "${PR_ENV_DB_USER}" \
|
||||
-d "hosting_pr${{ github.event.number }}" \
|
||||
--single-transaction \
|
||||
--no-acl \
|
||||
--no-owner \
|
||||
--schema=public \
|
||||
/tmp/stats-dump.dump
|
||||
RESTORE_EXIT=$?
|
||||
echo "::endgroup::"
|
||||
echo "pg_restore exit code: ${RESTORE_EXIT}"
|
||||
|
||||
if [ "${RESTORE_EXIT}" -ne 0 ]; then
|
||||
echo "Restore failed; the database will be rebuilt on the next run because it has not been marked as loaded."
|
||||
exit "${RESTORE_EXIT}"
|
||||
fi
|
||||
env:
|
||||
PR_ENV_DB_PASSWORD: ${{ secrets.PR_ENV_DB_PASSWORD }}
|
||||
PR_ENV_DB_USER: ${{ secrets.PR_ENV_DB_USER }}
|
||||
PR_ENV_DB_HOST: ${{ secrets.PR_ENV_DB_HOST }}
|
||||
PR_ENV_DB_NAME: ${{ secrets.PR_ENV_DB_NAME }}
|
||||
- name: Mark database as stats-dump loaded
|
||||
if: ${{ inputs.stats_dump && steps.verify_stats_dump.outputs.should_load == 'true' }}
|
||||
run: |
|
||||
PGPASSWORD="${PR_ENV_DB_PASSWORD}" psql \
|
||||
-h "${PR_ENV_DB_HOST}" \
|
||||
-U "${PR_ENV_DB_USER}" \
|
||||
-d "hosting_pr${{ github.event.number }}" \
|
||||
-c "COMMENT ON DATABASE \"hosting_pr${{ github.event.number }}\" IS 'metabase-pr-env stats-dump loaded';"
|
||||
env:
|
||||
PR_ENV_DB_PASSWORD: ${{ secrets.PR_ENV_DB_PASSWORD }}
|
||||
PR_ENV_DB_USER: ${{ secrets.PR_ENV_DB_USER }}
|
||||
PR_ENV_DB_HOST: ${{ secrets.PR_ENV_DB_HOST }}
|
||||
- name: Download Deployment YAML template
|
||||
if: ${{ !inputs.self_hosting }}
|
||||
run: aws s3 cp s3://metabase-pr-env/metabase.yml.tmpl ./metabase.yml.tmpl
|
||||
- name: Render Deployment YAML
|
||||
if: ${{ !inputs.self_hosting }}
|
||||
uses: nowactions/envsubst@c4b8e3977354d294659a7ec164279ce8fba5c2e1 # v1.0.1
|
||||
with:
|
||||
input: ./metabase.yml.tmpl
|
||||
output: ./metabase.yml
|
||||
env:
|
||||
IMAGE_TAG: pr${{ github.event.number }}
|
||||
PR_NUMBER: ${{ github.event.number }}
|
||||
MB_PREMIUM_EMBEDDING_TOKEN: ${{ secrets.PR_ENV_MB_PREMIUM_EMBEDDING_TOKEN }}
|
||||
SHA: ${{ github.event.pull_request.head.sha || github.sha }}
|
||||
MB_ANALYTICS_DEV_MODE: ${{ inputs.analytics_dev_mode && 'true' || 'false' }}
|
||||
- name: Inject stats-dump env vars into Deployment YAML
|
||||
if: ${{ !inputs.self_hosting && inputs.stats_dump }}
|
||||
env:
|
||||
MB_ENCRYPTION_SECRET_KEY: ${{ secrets.PR_ENV_STATS_DUMP_ENCRYPTION_KEY }}
|
||||
run: |
|
||||
yq -i 'select(.kind == "Metabase").spec.configuration.envVars += [
|
||||
{"name": "MB_ENCRYPTION_SECRET_KEY", "value": strenv(MB_ENCRYPTION_SECRET_KEY)},
|
||||
{"name": "MB_DISABLE_SCHEDULER", "value": "true"}
|
||||
]' ./metabase.yml
|
||||
- name: Create OIDC Token
|
||||
id: create-oidc-token
|
||||
shell: bash
|
||||
@@ -275,6 +107,45 @@ jobs:
|
||||
cluster: default
|
||||
namespace: default
|
||||
user: oidc-token
|
||||
- name: Configure AWS credentials
|
||||
uses: aws-actions/configure-aws-credentials@d979d5b3a71173a29b74b5b88418bfda9437d885 # v6.1.1
|
||||
with:
|
||||
role-to-assume: ${{ secrets.PR_ENV_IAM_ROLE }}
|
||||
role-session-name: GitHub_to_AWS_via_FederatedOIDC
|
||||
aws-region: us-east-1
|
||||
- name: Setup psql client
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y postgresql-client
|
||||
- name: Create app database if not exists
|
||||
run: |
|
||||
export PGPASSWORD="${PR_ENV_DB_PASSWORD}"
|
||||
export PGUSER="${PR_ENV_DB_USER}"
|
||||
export PGHOST="${PR_ENV_DB_HOST}"
|
||||
export PGDATABASE="${PR_ENV_DB_NAME}"
|
||||
psql -tc "SELECT 1 FROM pg_database WHERE datname = 'hosting_pr${{ github.event.number }}'" \
|
||||
| grep -q 1 \
|
||||
|| psql -c "CREATE DATABASE hosting_pr${{ github.event.number }}"
|
||||
env:
|
||||
PR_ENV_DB_PASSWORD: ${{ secrets.PR_ENV_DB_PASSWORD }}
|
||||
PR_ENV_DB_USER: ${{ secrets.PR_ENV_DB_USER }}
|
||||
PR_ENV_DB_HOST: ${{ secrets.PR_ENV_DB_HOST }}
|
||||
PR_ENV_DB_NAME: ${{ secrets.PR_ENV_DB_NAME }}
|
||||
- name: Download Deployment YAML template
|
||||
if: ${{ !inputs.self_hosting }}
|
||||
run: aws s3 cp s3://metabase-pr-env/metabase.yml.tmpl ./metabase.yml.tmpl
|
||||
- name: Render Deployment YAML
|
||||
if: ${{ !inputs.self_hosting }}
|
||||
uses: nowactions/envsubst@c4b8e3977354d294659a7ec164279ce8fba5c2e1 # v1.0.1
|
||||
with:
|
||||
input: ./metabase.yml.tmpl
|
||||
output: ./metabase.yml
|
||||
env:
|
||||
IMAGE_TAG: pr${{ github.event.number }}
|
||||
PR_NUMBER: ${{ github.event.number }}
|
||||
MB_PREMIUM_EMBEDDING_TOKEN: ${{ secrets.PR_ENV_MB_PREMIUM_EMBEDDING_TOKEN }}
|
||||
SHA: ${{ github.event.pull_request.head.sha || github.sha }}
|
||||
MB_ANALYTICS_DEV_MODE: ${{ inputs.analytics_dev_mode && 'true' || 'false' }}
|
||||
- name: Deploy PR Review ENV
|
||||
if: ${{ !inputs.self_hosting }}
|
||||
run: |
|
||||
@@ -338,4 +209,4 @@ jobs:
|
||||
✅ Preview:
|
||||
https://pr${{ github.event.number }}.coredev.metabase.com
|
||||
🗒️ [login instructions](https://www.notion.so/metabase/Ephemeral-PR-Environments-c74a3710c87a460dbf4fc0007a001458)
|
||||
📦 Deployment type: ${{ inputs.self_hosting && 'PR-Env-self-hosting' || (inputs.analytics_dev_mode && 'PR-Env-analytics-dev' || (inputs.stats_dump && 'PR-Env-Stats-Dump' || 'PR-Env')) }}
|
||||
📦 Deployment type: ${{ inputs.self_hosting && 'PR-Env-self-hosting' || (inputs.analytics_dev_mode && 'PR-Env-analytics-dev' || 'PR-Env') }}
|
||||
|
||||
2
.github/workflows/pre-release.yml
vendored
2
.github/workflows/pre-release.yml
vendored
@@ -122,7 +122,7 @@ jobs:
|
||||
strategy:
|
||||
matrix:
|
||||
edition: [oss, ee]
|
||||
java-version: [25]
|
||||
java-version: [21]
|
||||
steps:
|
||||
- name: Prepare JRE (Java Run-time Environment)
|
||||
uses: actions/setup-java@v5.2.0
|
||||
|
||||
18
.github/workflows/release-embedding-sdk.yml
vendored
18
.github/workflows/release-embedding-sdk.yml
vendored
@@ -94,7 +94,6 @@ jobs:
|
||||
"test",
|
||||
"build-sdk",
|
||||
"publish-npm",
|
||||
"tag-latest",
|
||||
"git-tag",
|
||||
]
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
@@ -399,17 +398,12 @@ jobs:
|
||||
TAG="${{ steps.compute-tag.outputs.masterTag }}"
|
||||
echo "Resolved npm tag: $TAG"
|
||||
npm publish --tag "$TAG"
|
||||
|
||||
# When a newer release branch goes gold, remove this entire job from this branch.
|
||||
tag-latest:
|
||||
name: Tag npm release as latest
|
||||
# Pre-gold: remove `if: false` once this branch becomes gold.
|
||||
if: false
|
||||
needs: [publish-npm, determine-sdk-version]
|
||||
uses: ./.github/workflows/embedding-sdk-tag-latest.yml
|
||||
with:
|
||||
version: ${{ needs.determine-sdk-version.outputs.sdk_version }}
|
||||
secrets: inherit
|
||||
# ⚠️ This step should only be executed on the latest stable (gold) release branch.
|
||||
# Only uncomment this when the new release branch becomes stable (gold).
|
||||
# - name: Add `latest` tag to the latest release branch (e.g. `release-x.61.x`) deployment
|
||||
# working-directory: sdk
|
||||
# run: |
|
||||
# npm dist-tag add @metabase/embedding-sdk-react@${{ env.sdk_version }} latest
|
||||
|
||||
git-tag:
|
||||
needs: [publish-npm, determine-sdk-version]
|
||||
|
||||
@@ -33,7 +33,7 @@ jobs:
|
||||
- name: Install bun
|
||||
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
|
||||
with:
|
||||
bun-version-file: 'package.json'
|
||||
bun-version: "1.3.7"
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
- name: Prepare build scripts
|
||||
run: bun install --cwd release --frozen-lockfile && bun run --cwd release build
|
||||
|
||||
27
.github/workflows/release-tests.yml
vendored
27
.github/workflows/release-tests.yml
vendored
@@ -39,7 +39,7 @@ jobs:
|
||||
- name: Install bun
|
||||
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
|
||||
with:
|
||||
bun-version-file: 'package.json'
|
||||
bun-version: "1.3.7"
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
- name: prepare dependencies in release directory
|
||||
run: bun install --cwd release --frozen-lockfile
|
||||
@@ -58,34 +58,13 @@ jobs:
|
||||
- name: Install bun
|
||||
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
|
||||
with:
|
||||
bun-version-file: 'package.json'
|
||||
bun-version: "1.3.7"
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
- name: prepare dependencies in release directory
|
||||
run: bun install --cwd release --frozen-lockfile
|
||||
- name: Run release unit tests
|
||||
run: bun run --cwd release test:unit
|
||||
|
||||
ci-conductor-tests:
|
||||
needs: files-changed
|
||||
if: needs.files-changed.outputs.release_source == 'true'
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
sparse-checkout: "release/ci-conductor"
|
||||
- name: Install bun
|
||||
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
|
||||
with:
|
||||
bun-version-file: 'package.json'
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
- name: Install ci-conductor dependencies
|
||||
run: bun install --cwd release/ci-conductor --frozen-lockfile
|
||||
- name: Type-check ci-conductor
|
||||
run: bun run --cwd release/ci-conductor type-check
|
||||
- name: Run ci-conductor unit tests
|
||||
run: bun run --cwd release/ci-conductor test
|
||||
|
||||
release-test-build:
|
||||
needs: files-changed
|
||||
if: needs.files-changed.outputs.release_source == 'true'
|
||||
@@ -98,7 +77,7 @@ jobs:
|
||||
- name: Install bun
|
||||
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
|
||||
with:
|
||||
bun-version-file: 'package.json'
|
||||
bun-version: "1.3.7"
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
- name: prepare dependencies in release directory
|
||||
run: bun install --cwd release --frozen-lockfile
|
||||
|
||||
53
.github/workflows/run-tests.yml
vendored
53
.github/workflows/run-tests.yml
vendored
@@ -44,6 +44,7 @@ jobs:
|
||||
e2e_specs_to_run: ${{ steps.compute.outputs.e2e_specs_to_run }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
|
||||
- name: Test which files changed
|
||||
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
|
||||
id: changes
|
||||
@@ -54,21 +55,23 @@ jobs:
|
||||
filters: .github/file-paths.yaml
|
||||
- name: Create test plan
|
||||
id: compute
|
||||
if: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && github.ref == 'refs/heads/master') }}
|
||||
uses: ./.github/actions/create-test-plan
|
||||
with:
|
||||
changed-files: ${{ steps.changes.outputs.all_changed_files }}
|
||||
fe-changed-files: ${{ steps.changes.outputs.frontend_sources_files }}
|
||||
be-changed-files: ${{ steps.changes.outputs.backend_sources_files }}
|
||||
unit-infra-touched: ${{ steps.changes.outputs.frontend_unit_infra }}
|
||||
loki-infra-touched: ${{ steps.changes.outputs.frontend_loki_infra }}
|
||||
e2e-infra-touched: ${{ steps.changes.outputs.e2e_infra }}
|
||||
shared-sources-touched: ${{ steps.changes.outputs.shared_sources }}
|
||||
if: ${{ github.event_name == 'pull_request' }}
|
||||
env:
|
||||
CHANGED_FILES: ${{ steps.changes.outputs.all_changed_files }}
|
||||
UNIT_INFRA_TOUCHED: ${{ steps.changes.outputs.frontend_unit_infra }}
|
||||
LOKI_INFRA_TOUCHED: ${{ steps.changes.outputs.frontend_loki_infra }}
|
||||
run: |
|
||||
OUTPUT=$(bun .github/scripts/create-test-plan.ts)
|
||||
echo "$OUTPUT" | jq '.stats'
|
||||
echo "stats=$(echo "$OUTPUT" | jq -c '.stats')" >> $GITHUB_OUTPUT
|
||||
echo "fe_unit_specs_to_run=$(echo "$OUTPUT" | jq -c '.fe_unit_specs_to_run')" >> $GITHUB_OUTPUT
|
||||
echo "loki_stories_to_run=$(echo "$OUTPUT" | jq -c '.loki_stories_to_run')" >> $GITHUB_OUTPUT
|
||||
echo "e2e_specs_to_run=$(echo "$OUTPUT" | jq -c '.e2e_specs_to_run')" >> $GITHUB_OUTPUT
|
||||
|
||||
upload-affected-tests-stats:
|
||||
needs: create-test-plan
|
||||
# Only land rows for the upstream repo so forks don't pollute the table.
|
||||
if: ${{ !cancelled() && needs.create-test-plan.result == 'success' && (github.event.pull_request.head.repo.full_name == 'metabase/metabase' || (github.event_name == 'push' && github.repository == 'metabase/metabase')) }}
|
||||
if: ${{ !cancelled() && needs.create-test-plan.result == 'success' && github.event.pull_request.head.repo.full_name == 'metabase/metabase' }}
|
||||
# Stats upload is best-effort — a failure here shouldn't fail the PR run.
|
||||
continue-on-error: true
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
@@ -76,14 +79,11 @@ jobs:
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Upload affected tests stats
|
||||
# On push (merge to master) there is no PR context, so fall back to the
|
||||
# commit SHAs and a 0 PR number.
|
||||
env:
|
||||
STATS_JSON: ${{ needs.create-test-plan.outputs.stats }}
|
||||
TRIGGERED_BY: ${{ github.event_name == 'pull_request' && 'pr_update' || 'merge_to_master' }}
|
||||
PR_NUMBER: ${{ github.event_name == 'pull_request' && github.event.pull_request.number || 0 }}
|
||||
HEAD_SHA: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
|
||||
BASE_SHA: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || github.event.before }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
API_KEY: ${{ secrets.STATS_API_KEY }}
|
||||
run: node .github/scripts/upload-affected-tests-stats.js
|
||||
|
||||
@@ -154,11 +154,8 @@ jobs:
|
||||
# here we pass the list of changed files
|
||||
# e.g. e2e/test/scenarios/onboarding/command-palette.cy.spec.js,e2e/test/scenarios/question/document-title.cy.spec.js
|
||||
# e2e_all_files includes all changes which trigger all e2e tests, if the list of changes files is included in the
|
||||
# list of changed specs files, then only specs are changed and we can run only those specs.
|
||||
# Only narrow to changed specs on pull requests. On push to master/release branches (and the
|
||||
# merge queue) the paths-filter still computes these file lists, so without this guard a push
|
||||
# that touched only specs would silently run a narrowed suite — we always want the full suite there.
|
||||
specs: ${{ github.event_name == 'pull_request' && needs.create-test-plan.outputs.e2e_all_files == needs.create-test-plan.outputs.e2e_specs_files && needs.create-test-plan.outputs.e2e_specs_files || '' }}
|
||||
# list of changed specs files, then only specs are changed and we can run only those specs
|
||||
specs: ${{ needs.create-test-plan.outputs.e2e_all_files == needs.create-test-plan.outputs.e2e_specs_files && needs.create-test-plan.outputs.e2e_specs_files || '' }}
|
||||
|
||||
embedding-sdk-package:
|
||||
needs: [create-test-plan]
|
||||
@@ -221,18 +218,6 @@ jobs:
|
||||
self_hosting: false
|
||||
analytics_dev_mode: true
|
||||
|
||||
pr-env-stats-dump:
|
||||
needs: [uberjar]
|
||||
if: |
|
||||
!cancelled() &&
|
||||
contains(github.event.pull_request.labels.*.name, 'PR-Env-Stats-Dump')
|
||||
uses: ./.github/workflows/pr-env.yml
|
||||
secrets: inherit
|
||||
with:
|
||||
self_hosting: false
|
||||
analytics_dev_mode: false
|
||||
stats_dump: true
|
||||
|
||||
containerize:
|
||||
needs: [uberjar]
|
||||
if: |
|
||||
|
||||
46
.github/workflows/schedule-minor.yml
vendored
46
.github/workflows/schedule-minor.yml
vendored
@@ -14,39 +14,21 @@ jobs:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
sparse-checkout: |
|
||||
release
|
||||
.github
|
||||
- name: Prepare release scripts
|
||||
uses: ./.github/actions/prepare-release-scripts
|
||||
- name: Dispatch a minor release for every supported major
|
||||
- name: Dispatch minor release for the current major
|
||||
uses: actions/github-script@v9
|
||||
env:
|
||||
AWS_S3_STATIC_BUCKET: ${{ vars.AWS_S3_STATIC_BUCKET }}
|
||||
AWS_REGION: ${{ vars.AWS_REGION }}
|
||||
with:
|
||||
script: | # js
|
||||
// Every supported major line gets regular minor releases so the
|
||||
// branches we backport into don't go stale. Support is read from
|
||||
// `major_version_support` in version-info.json via getSupportedMajors —
|
||||
// the same source the auto-backport workflow uses to pick its targets.
|
||||
const { getSupportedMajors } = require('${{ github.workspace }}/release/dist/index.cjs');
|
||||
// Minor releases only go out for the current major; the
|
||||
// previous major gets patches only.
|
||||
const currentRelease = Number('${{ vars.CURRENT_VERSION }}');
|
||||
|
||||
const majors = await getSupportedMajors();
|
||||
console.log('supported majors', majors);
|
||||
|
||||
// release-auto.yml releases one major per dispatch.
|
||||
await Promise.all(majors.map(version =>
|
||||
github.rest.actions.createWorkflowDispatch({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
workflow_id: 'release-auto.yml',
|
||||
ref: 'refs/heads/master',
|
||||
inputs: {
|
||||
version: String(version),
|
||||
kind: 'minor',
|
||||
},
|
||||
}).catch(console.error)
|
||||
));
|
||||
await github.rest.actions.createWorkflowDispatch({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
workflow_id: 'release-auto.yml',
|
||||
ref: 'refs/heads/master',
|
||||
inputs: {
|
||||
version: String(currentRelease),
|
||||
kind: 'minor',
|
||||
},
|
||||
}).catch(console.error);
|
||||
|
||||
90
.github/workflows/semantic-search.yml
vendored
90
.github/workflows/semantic-search.yml
vendored
@@ -11,13 +11,6 @@ concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}-semantic-search
|
||||
cancel-in-progress: true
|
||||
|
||||
# Exposed to every job so the test-driver action can report failures to
|
||||
# ci-conductor (secrets: inherit is already set on the semantic-search call in
|
||||
# run-tests.yml). See DEV-2224.
|
||||
env:
|
||||
CI_CONDUCTOR_BASE_URL: ${{ secrets.CI_CONDUCTOR_BASE_URL }}
|
||||
CI_CONDUCTOR_WEBHOOK_SECRET: ${{ secrets.CI_CONDUCTOR_WEBHOOK_SECRET }}
|
||||
|
||||
jobs:
|
||||
semantic-search-tests-h2:
|
||||
if: ${{ !inputs.skip }}
|
||||
@@ -34,7 +27,7 @@ jobs:
|
||||
- name: Semantic Search Tests
|
||||
build-static-viz: false
|
||||
test-args: >-
|
||||
:only '["enterprise/backend/test/metabase_enterprise/semantic_search" "enterprise/backend/test/metabase_enterprise/entity_retrieval" "enterprise/backend/test/metabase_enterprise/metabot/tools/semantic_search_test.clj"]'
|
||||
:only '["enterprise/backend/test/metabase_enterprise/semantic_search" "enterprise/backend/test/metabase_enterprise/metabot_v3/tools/semantic_search_test.clj"]'
|
||||
exclude-tag: ':mb/driver-tests'
|
||||
|
||||
services:
|
||||
@@ -61,7 +54,6 @@ jobs:
|
||||
test-args: >-
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -73,7 +65,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
semantic-search-tests-mariadb:
|
||||
if: ${{ !inputs.skip }}
|
||||
@@ -95,7 +86,7 @@ jobs:
|
||||
- name: Semantic Search Tests
|
||||
build-static-viz: false
|
||||
test-args: >-
|
||||
:only '["enterprise/backend/test/metabase_enterprise/semantic_search" "enterprise/backend/test/metabase_enterprise/entity_retrieval" "enterprise/backend/test/metabase_enterprise/metabot/tools/semantic_search_test.clj"]'
|
||||
:only '["enterprise/backend/test/metabase_enterprise/semantic_search" "enterprise/backend/test/metabase_enterprise/metabot_v3/tools/semantic_search_test.clj"]'
|
||||
exclude-tag: ':mb/driver-tests'
|
||||
|
||||
services:
|
||||
@@ -136,7 +127,6 @@ jobs:
|
||||
test-args: >-
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -148,7 +138,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
semantic-search-tests-mysql:
|
||||
if: ${{ !inputs.skip }}
|
||||
@@ -170,7 +159,7 @@ jobs:
|
||||
- name: Semantic Search Tests
|
||||
build-static-viz: false
|
||||
test-args: >-
|
||||
:only '["enterprise/backend/test/metabase_enterprise/semantic_search" "enterprise/backend/test/metabase_enterprise/entity_retrieval" "enterprise/backend/test/metabase_enterprise/metabot/tools/semantic_search_test.clj"]'
|
||||
:only '["enterprise/backend/test/metabase_enterprise/semantic_search" "enterprise/backend/test/metabase_enterprise/metabot_v3/tools/semantic_search_test.clj"]'
|
||||
exclude-tag: ':mb/driver-tests'
|
||||
|
||||
services:
|
||||
@@ -211,7 +200,6 @@ jobs:
|
||||
test-args: >-
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -223,7 +211,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
semantic-search-tests-postgres:
|
||||
if: ${{ !inputs.skip }}
|
||||
@@ -245,7 +232,7 @@ jobs:
|
||||
- name: Semantic Search Tests
|
||||
build-static-viz: false
|
||||
test-args: >-
|
||||
:only '["enterprise/backend/test/metabase_enterprise/semantic_search" "enterprise/backend/test/metabase_enterprise/entity_retrieval" "enterprise/backend/test/metabase_enterprise/metabot/tools/semantic_search_test.clj"]'
|
||||
:only '["enterprise/backend/test/metabase_enterprise/semantic_search" "enterprise/backend/test/metabase_enterprise/metabot_v3/tools/semantic_search_test.clj"]'
|
||||
exclude-tag: ':mb/driver-tests'
|
||||
|
||||
services:
|
||||
@@ -287,7 +274,6 @@ jobs:
|
||||
test-args: >-
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
@@ -299,73 +285,6 @@ jobs:
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
# pgvector-on-the-app-db mode: the app db itself is a pgvector-capable Postgres and
|
||||
# MB_PGVECTOR_DB_URL is deliberately unset. The dedicated-harness tests gate themselves off; the
|
||||
# appdb-pgvector-mode round trip (and the URL-independent unit tests) run for real.
|
||||
semantic-search-tests-appdb-mode:
|
||||
if: ${{ !inputs.skip }}
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 40
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
version:
|
||||
- name: pgvector pg17 as app db, no dedicated pgvector
|
||||
junit-name: semantic-search-tests-appdb-mode-pg17
|
||||
appdb-image: pgvector/pgvector:pg17
|
||||
job:
|
||||
- name: Semantic Search Tests
|
||||
build-static-viz: false
|
||||
test-args: >-
|
||||
:only '["enterprise/backend/test/metabase_enterprise/semantic_search" "enterprise/backend/test/metabase_enterprise/entity_retrieval" "enterprise/backend/test/metabase_enterprise/metabot/tools/semantic_search_test.clj"]'
|
||||
exclude-tag: ':mb/driver-tests'
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: ${{ matrix.version.appdb-image }}
|
||||
ports:
|
||||
- "5432:5432"
|
||||
env:
|
||||
POSTGRES_USER: mb_test
|
||||
POSTGRES_DB: mb_test
|
||||
POSTGRES_HOST_AUTH_METHOD: trust
|
||||
env:
|
||||
CI: 'true'
|
||||
MB_DB_TYPE: postgres
|
||||
MB_DB_PORT: 5432
|
||||
MB_DB_HOST: localhost
|
||||
MB_DB_DBNAME: mb_test
|
||||
MB_DB_USER: mb_test
|
||||
# opts this job into the destructive appdb-pgvector-mode round trip, which then FAILS (rather
|
||||
# than silently skipping) if the app db can't actually host pgvector
|
||||
MB_APPDB_PGVECTOR_MODE_TEST: 'true'
|
||||
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Test ${{ matrix.version.name }}
|
||||
id: test-driver
|
||||
uses: ./.github/actions/test-driver
|
||||
with:
|
||||
build-static-viz: ${{ matrix.job.build-static-viz }}
|
||||
junit-name: ${{ matrix.version.junit-name }}
|
||||
test-args: >-
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
run-trunk: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
semantic-search-tests-result:
|
||||
needs:
|
||||
@@ -373,7 +292,6 @@ jobs:
|
||||
- semantic-search-tests-mariadb
|
||||
- semantic-search-tests-mysql
|
||||
- semantic-search-tests-postgres
|
||||
- semantic-search-tests-appdb-mode
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
name: semantic-search-tests-result
|
||||
|
||||
2
.github/workflows/snowplow.yml
vendored
2
.github/workflows/snowplow.yml
vendored
@@ -32,7 +32,7 @@ jobs:
|
||||
uses: actions/setup-java@v5.2.0
|
||||
with:
|
||||
distribution: temurin
|
||||
java-version: 25
|
||||
java-version: 21
|
||||
- name: Set up Babashka
|
||||
uses: turtlequeue/setup-babashka@d78ec6570aea3b614bc695cafcceb82eb45c2af9 # v1.8.0
|
||||
with:
|
||||
|
||||
8
.github/workflows/snyk.yml
vendored
8
.github/workflows/snyk.yml
vendored
@@ -53,7 +53,7 @@ jobs:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
|
||||
with:
|
||||
bun-version-file: 'package.json'
|
||||
bun-version: "1.3.7"
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
# Snyk doesn't support bun.lock yet, so we generate yarn.lock and scan that.
|
||||
# Bun has a bug where it normalizes "< 3.0.0" to "< 3" in alias entries
|
||||
@@ -107,6 +107,9 @@ jobs:
|
||||
- name: Driver - Databricks
|
||||
file: modules/drivers/databricks/pom.xml
|
||||
category: driver-databricks
|
||||
- name: Driver - Druid
|
||||
file: modules/drivers/druid/pom.xml
|
||||
category: driver-druid
|
||||
- name: Driver - Druid JDBC
|
||||
file: modules/drivers/druid-jdbc/pom.xml
|
||||
category: driver-druid-jdbc
|
||||
@@ -131,6 +134,9 @@ jobs:
|
||||
- name: Driver - SparkSQL
|
||||
file: modules/drivers/sparksql/pom.xml
|
||||
category: driver-sparksql
|
||||
- name: Driver - SQLite
|
||||
file: modules/drivers/sqlite/pom.xml
|
||||
category: driver-sqlite
|
||||
- name: Driver - SQL Server
|
||||
file: modules/drivers/sqlserver/pom.xml
|
||||
category: driver-sqlserver
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user