Compare commits
93 Commits
module-plu
...
hackathon-
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f44310337f | ||
|
|
eda4dd42c2 | ||
|
|
b3dca3147f | ||
|
|
d83fa7119d | ||
|
|
fa28e69c73 | ||
|
|
9fb59a0050 | ||
|
|
acf76b2c84 | ||
|
|
be0d442838 | ||
|
|
a904bfc8bd | ||
|
|
9bd659825b | ||
|
|
b12e88dff6 | ||
|
|
c583ecdb11 | ||
|
|
df41ca699e | ||
|
|
0d5a95a6c2 | ||
|
|
a49cddac83 | ||
|
|
7e68e140ff | ||
|
|
e8f79b9981 | ||
|
|
9e8be3e3c2 | ||
|
|
95c4796aa1 | ||
|
|
ba0830c712 | ||
|
|
86e5e3106b | ||
|
|
8e94a3c000 | ||
|
|
281054e93c | ||
|
|
a7c68843f2 | ||
|
|
0bc3f71772 | ||
|
|
6703ffa2ec | ||
|
|
97164a087d | ||
|
|
5fb774b25f | ||
|
|
f0e0207ec4 | ||
|
|
d54991f137 | ||
|
|
c6ff597916 | ||
|
|
b5bc6e61e3 | ||
|
|
bce65b7a4c | ||
|
|
fc8db920f9 | ||
|
|
587692bffa | ||
|
|
b4e1213187 | ||
|
|
7e8e14e564 | ||
|
|
a2b7b0b0ae | ||
|
|
76899fcfa1 | ||
|
|
f4dba2bfaa | ||
|
|
6775c06bb7 | ||
|
|
a14ed883a9 | ||
|
|
56eba5d883 | ||
|
|
abfbf08801 | ||
|
|
c770223710 | ||
|
|
760a64747b | ||
|
|
02e68f7621 | ||
|
|
5f88da2c0b | ||
|
|
13cb229118 | ||
|
|
87531768c6 | ||
|
|
2dac32e4a6 | ||
|
|
d37db3c93f | ||
|
|
dff11c838e | ||
|
|
4ae0cd2408 | ||
|
|
1dd2d42b5f | ||
|
|
0e319d652f | ||
|
|
5eddb9f126 | ||
|
|
05d6b672a8 | ||
|
|
b19d32eb20 | ||
|
|
b0c91ca456 | ||
|
|
aeeac2b82b | ||
|
|
6c3c3aee9a | ||
|
|
e32f40f993 | ||
|
|
5894eb327c | ||
|
|
be93da347b | ||
|
|
c9a34d7043 | ||
|
|
33dd13b4f3 | ||
|
|
56dd904edd | ||
|
|
c8080065b3 | ||
|
|
883098d7cb | ||
|
|
a5f55aed2b | ||
|
|
62cc9fa5a8 | ||
|
|
be201fd978 | ||
|
|
d264df7bf9 | ||
|
|
17347f7ef5 | ||
|
|
d28132b8a9 | ||
|
|
7893398332 | ||
|
|
fcfc85cf91 | ||
|
|
3b25cb8c98 | ||
|
|
03e94693d9 | ||
|
|
5c0fbf2a12 | ||
|
|
f0290e1f5e | ||
|
|
b8c91393ce | ||
|
|
b6fb56858f | ||
|
|
5f84cbf87e | ||
|
|
c5c9fed24c | ||
|
|
2cde247d19 | ||
|
|
3bc97f6cdc | ||
|
|
4d752bacbe | ||
|
|
d1af0e2a16 | ||
|
|
87f9f999ae | ||
|
|
e89f68efae | ||
|
|
adbfc8916d |
@@ -22,6 +22,7 @@ You understand Metabase's driver architecture built on Clojure multimethods with
|
||||
:bigquery-cloud-sdk, :databricks, :clickhouse, :athena, :sparksql,
|
||||
:presto-jdbc / :starburst, :vertica, :sqlite, :h2
|
||||
:mongo (non-SQL, custom protocol)
|
||||
:druid (non-SQL, REST-based)
|
||||
```
|
||||
|
||||
Key driver extension points you know intimately:
|
||||
|
||||
1
.claude/commands/fix-flakey-test.md
Normal file
1
.claude/commands/fix-flakey-test.md
Normal file
@@ -0,0 +1 @@
|
||||
Fix flakey test $ARGUMENTS
|
||||
@@ -27,28 +27,9 @@ This guide covers Clojure and ClojureScript coding conventions for Metabase. See
|
||||
- Format using Markdown conventions
|
||||
- Reference other vars with `[[other-var]]` not backticks
|
||||
|
||||
A docstring states the *contract*: what it returns (including when it returns nil), what its
|
||||
params mean, what it throws versus swallows, and any precondition or invariant the caller must
|
||||
respect. A params block for non-obvious or optional args is good documentation — length is not
|
||||
the problem.
|
||||
|
||||
These seven things are, and each is a reason to cut or relocate:
|
||||
|
||||
| Anti-pattern | Why | Where it goes instead |
|
||||
| --- | --- | --- |
|
||||
| **Facts you don't own** — how a *callee*, another namespace, or another repo works internally | Drifts silently; nothing fails when it goes stale | Cut. Name the callee and let its own docstring say it |
|
||||
| **Said twice** — ns docstring restating its vars', a caller restating its callee's, sibling fns repeating boilerplate | Two copies means one is already wrong | Cut the copy. One fact, one owner |
|
||||
| **Design rationale** — why this approach, why it lives in this module, what the alternative was | The caller can't act on it | Commit message, or an inline comment on the specific line it constrains |
|
||||
| **Development history** — "matching master", "the X branch", "currently", "for now", "used to" | Stale by construction | Cut |
|
||||
| **Instructions to the reader** — "callers should trust this", "don't bother checking", "you should use X" | State the contract; the caller decides | Cut, keeping any actual guarantee as a plain statement |
|
||||
| **Invented precision** — score bands, thresholds, or tables no code enforces | Reads as spec, isn't | Cut, or make the code enforce it |
|
||||
| **A note on the wrong var** — describing how overrides behave, on the base/default impl | Nobody reading the override sees it | Move to the overrides it describes |
|
||||
|
||||
**Comments:**
|
||||
|
||||
- `TODO` format: `;; TODO (Name YYYY-MM-DD) -- description`
|
||||
- Prefer an inline comment over docstring prose for anything that explains *the body*. Put it on
|
||||
the line it explains, not at the top of the function.
|
||||
|
||||
## Code Organization
|
||||
|
||||
@@ -63,22 +44,6 @@ These seven things are, and each is a reason to cut or relocate:
|
||||
- Lines ≤ 120 characters
|
||||
- No blank non-comment lines within definition forms (except pairwise `let`/`cond`)
|
||||
|
||||
**Java interop: type-hint parameters in the arglist; prefer `ex-*` helpers:**
|
||||
|
||||
```clojure
|
||||
;; bad — reflection warning, calls Throwable methods directly
|
||||
(defn root-cause [e]
|
||||
(if-let [cause (.getCause e)] (recur cause) e))
|
||||
|
||||
;; good
|
||||
(defn- root-cause [^Throwable e]
|
||||
(if-let [cause (ex-cause e)]
|
||||
(recur cause)
|
||||
e))
|
||||
```
|
||||
|
||||
Use `ex-cause`/`ex-message`/`ex-data`/`ex-info` rather than `.getCause`/`.getMessage`/`(.getData ^ExceptionInfo …)` where possible.
|
||||
|
||||
## Style Conventions
|
||||
|
||||
**Keywords and Metadata:**
|
||||
@@ -86,37 +51,6 @@ Use `ex-cause`/`ex-message`/`ex-data`/`ex-info` rather than `.getCause`/`.getMes
|
||||
- Prefer namespaced keywords for internal use: `:query-type/normal` not `:normal`
|
||||
- Tag variables with `:arglists` metadata if they're functions but wouldn't otherwise have it
|
||||
|
||||
**Time/date: `java-time.api :as t` and `u.date`, not `java.time.*` direct interop:**
|
||||
|
||||
```clojure
|
||||
;; bad — Claude default, requires java.time interop
|
||||
(:import (java.time LocalDate))
|
||||
... (LocalDate/now) ...
|
||||
|
||||
;; good
|
||||
(:require
|
||||
[java-time.api :as t]
|
||||
[metabase.util.date-2 :as u.date])
|
||||
... (t/local-date) ...
|
||||
... (u.date/parse "2024-01-01") ...
|
||||
```
|
||||
|
||||
Only `(:import java.time.X)` when interop with a Java method genuinely needs the class.
|
||||
|
||||
**Check non-emptiness with `seq`, not `count`:**
|
||||
|
||||
```clojure
|
||||
;; bad — `count` realizes the whole seq into memory just to test emptiness
|
||||
(let [prompts (map build-prompt items)]
|
||||
(when (pos? (count prompts))
|
||||
(t2/insert! :model/Prompt prompts)))
|
||||
|
||||
;; good — `seq` realizes one cell; `insert!` streams the rest
|
||||
(let [prompts (map build-prompt items)]
|
||||
(when (seq prompts)
|
||||
(t2/insert! :model/Prompt prompts)))
|
||||
```
|
||||
|
||||
## Tests
|
||||
|
||||
**Organization:**
|
||||
@@ -128,33 +62,6 @@ Only `(:import java.time.X)` when interop with a Java method genuinely needs the
|
||||
|
||||
- Mark pure function tests `^:parallel`
|
||||
|
||||
**Redefining `defn`-vars under test:**
|
||||
|
||||
Use `mt/with-dynamic-fn-redefs` instead of `with-redefs` when stubbing `defn`/`defn-` vars — plain `with-redefs` is not parallel-test-safe and a clj-kondo hook (`:metabase/prefer-with-dynamic-fn-redefs`, see `.clj-kondo/config.edn`) will warn. `with-redefs` is still fine for `def` / dynamic var rebinding.
|
||||
|
||||
```clojure
|
||||
;; bad — flagged by the linter, not safe under :parallel tests
|
||||
(with-redefs [my.ns/some-fn (fn [_] :stub)] (do-test))
|
||||
|
||||
;; good
|
||||
(mt/with-dynamic-fn-redefs [my.ns/some-fn (fn [_] :stub)] (do-test))
|
||||
```
|
||||
|
||||
**Async tests: no bare `Thread/sleep` — use `deref` with a timeout sentinel:**
|
||||
|
||||
A fixed sleep is the #1 source of CI flake. Tests should fail fast and loud when a contract is violated, not hang or sleep through the bug.
|
||||
|
||||
```clojure
|
||||
;; bad — flaky on slow CI, silent on actual failure
|
||||
(Thread/sleep 300)
|
||||
(is (= :expected @seen))
|
||||
|
||||
;; good — fails fast with a clear message
|
||||
(let [v (deref some-promise 1000 ::timeout)]
|
||||
(is (not= ::timeout v) "promise never delivered")
|
||||
(is (= :expected v)))
|
||||
```
|
||||
|
||||
## Modules
|
||||
|
||||
**OSS Modules:**
|
||||
@@ -179,45 +86,6 @@ A fixed sleep is the #1 source of CI flake. Tests should fail fast and loud when
|
||||
|
||||
- Do not cheat module linters with `:clj-kondo/ignore [:metabase/modules]`
|
||||
|
||||
**Enterprise/OSS split: use `defenterprise`, not protocols or runtime feature checks:**
|
||||
|
||||
The OSS namespace defines the fallback body and names the EE namespace; the EE namespace overrides with the same fn name and a `:feature` clause naming the premium-features flag that gates the override. Both files use the same docstring shape — describe behavior on both sides.
|
||||
|
||||
```clojure
|
||||
;; src/metabase/remote_sync/core.clj (OSS)
|
||||
(defenterprise collection-editable?
|
||||
"Returns whether remote-synced collections are editable.
|
||||
Always true on OSS; EE consults the remote-sync settings."
|
||||
metabase-enterprise.remote-sync.core
|
||||
[_collection]
|
||||
true)
|
||||
|
||||
;; enterprise/backend/src/metabase_enterprise/remote_sync/core.clj (EE)
|
||||
(defenterprise collection-editable?
|
||||
"EE override — respects the synced-collections-editable? setting."
|
||||
:feature :remote-sync
|
||||
[collection]
|
||||
(or (synced-collections-editable?) (not (synced? collection))))
|
||||
```
|
||||
|
||||
`defenterprise-schema` is the malli-schema variant. Required from `[metabase.premium-features.core :refer [defenterprise defenterprise-schema]]`. The OSS body is what runs when the feature flag is off — never write `(when (premium/has-feature? ...) ...)` in user code; let `defenterprise` route.
|
||||
|
||||
**Security gates fail *open* under `:feature` — use `:feature :none` and check manually.** The router (`dynamic-ee-oss-fn`) calls the EE body only when both the EE namespace is loaded *and* the token currently grants the feature; otherwise it falls back to the OSS body. For a permissive default (`collection-editable?` → `true`) that's fine. But for a *restriction* — sandboxing, impersonation, row-level filtering — the OSS fallback is the unrestricted answer, so a temporarily lost/expired token silently downgrades a sandboxed user to full access. **Fail open.** When the EE override is what *decides whether to restrict*, gate it with `:feature :none` (which always routes to the EE body once EE code is loaded) and check the feature explicitly inside, so token loss fails *closed*:
|
||||
|
||||
```clojure
|
||||
;; enterprise/.../sandbox/api/util.clj — the gate that detects whether a user is sandboxed
|
||||
(defenterprise enforced-sandboxes-for-user
|
||||
"..."
|
||||
;; :none so we can decide whether sandboxing is *configured* even when the feature
|
||||
;; isn't currently available — letting us BLOCK requests that would otherwise be
|
||||
;; sandboxed, instead of returning the permissive OSS fallback. Fail closed.
|
||||
:feature :none
|
||||
[user-id]
|
||||
...)
|
||||
```
|
||||
|
||||
Reach for this only for the detection/enforcement decision point. Downstream functions that merely *act* on an already-made decision can keep their normal `:feature :sandboxes` gate.
|
||||
|
||||
## REST API Endpoints
|
||||
|
||||
**Required Elements:**
|
||||
@@ -226,20 +94,6 @@ Reach for this only for the detection/enforcement decision point. Downstream fun
|
||||
- All endpoints need Malli schemas for parameters (detailed and complete)
|
||||
- All new REST API endpoints MUST HAVE TESTS
|
||||
|
||||
**Endpoint shape: `api.macros/defendpoint`, four positional arglist:**
|
||||
|
||||
```clojure
|
||||
(api.macros/defendpoint :get "/:key/callback"
|
||||
"OIDC callback for a specific provider."
|
||||
[{provider-key :key} :- [:map [:key ProviderKey]]
|
||||
_query-params
|
||||
_body
|
||||
request]
|
||||
(oidc-integration/sso-callback provider-key request))
|
||||
```
|
||||
|
||||
The arglist is **always 4 positional bindings** in this order: `route-params`, `query-params`, `body`, `request`. Use `_`-prefixed names for unused ones. Each one gets a Malli schema via `:- <schema>` (omit when `_`). The HTTP verb is a keyword (`:get`/`:post`/`:put`/`:delete`), not a symbol. Required as `[metabase.api.macros :as api.macros]`.
|
||||
|
||||
**Naming Conventions:**
|
||||
|
||||
- Query parameters use kebab-case
|
||||
@@ -251,20 +105,6 @@ The arglist is **always 4 positional bindings** in this order: `route-params`, `
|
||||
- `GET` endpoints should not have side effects (except analytics)
|
||||
- `defendpoint` forms should be small wrappers around Toucan model code
|
||||
|
||||
**Closed response maps:**
|
||||
|
||||
Tag response Malli `:map` schemas with `{:closed true}` so the generated OpenAPI / TypeScript types reject extra keys. Open maps emit `additionalProperties: true`, which produces weak frontend types and makes discriminated unions ineffective. Tighten domain types in the same pass (`pos-int?` rather than `:int` when zero branches elsewhere).
|
||||
|
||||
```clojure
|
||||
;; weak
|
||||
{:status 200 :body [:map [:status [:= :generated]] [:prompt_count :int]]}
|
||||
|
||||
;; tight
|
||||
{:status 200 :body [:map {:closed true}
|
||||
[:status [:= :generated]]
|
||||
[:prompt_count pos-int?]]}
|
||||
```
|
||||
|
||||
## MBQL (Metabase Query Language)
|
||||
|
||||
**Restrictions:**
|
||||
@@ -304,215 +144,10 @@ Tag response Malli `:map` schemas with `{:closed true}` so the generated OpenAPI
|
||||
|
||||
**Linter Suppressions:**
|
||||
|
||||
The right shape is a **reader-discard map** placed directly above the form being silenced, listing the rule keywords explicitly:
|
||||
|
||||
```clojure
|
||||
#_{:clj-kondo/ignore [:metabase/validate-defendpoint-route-uses-kebab-case
|
||||
:metabase/validate-defendpoint-has-response-schema]}
|
||||
(api.macros/defendpoint :post "/api_key" ...)
|
||||
```
|
||||
|
||||
For a single require entry, hang it as metadata on that entry:
|
||||
|
||||
```clojure
|
||||
(:require
|
||||
^{:clj-kondo/ignore [:deprecated-namespace :discouraged-namespace]}
|
||||
[metabase.query-processor.store :as qp.store])
|
||||
```
|
||||
|
||||
Common Metabase-specific rules you'll see: `:metabase/modules`, `:metabase/validate-defendpoint-route-uses-kebab-case`, `:metabase/validate-defendpoint-has-response-schema`, `:metabase/prefer-with-dynamic-fn-redefs`, `:deprecated-namespace`, `:discouraged-namespace`. Don't use the keyword form (`#_:clj-kondo/ignore`); always the map form so it's grep-able by which rule(s) are suppressed.
|
||||
- Use proper format for kondo suppressions
|
||||
- No `#_:clj-kondo/ignore` (keyword form)
|
||||
|
||||
**Configurable Options:**
|
||||
|
||||
- Don't define configurable options that can only be set with environment variables
|
||||
- Use `:internal` `defsetting` instead
|
||||
|
||||
|
||||
## Metabase Helpers — Quick Reference
|
||||
|
||||
Metabase and its libraries have **many** helper functions to make code more readable. Don't hand-roll things that already exist! Reach for the right helper *the first time*.
|
||||
|
||||
Organized by **problem you're solving**, not by namespace.
|
||||
|
||||
## "I wrote this — use that instead" cheat sheet
|
||||
|
||||
*Authors: prefer the right column. Reviewers: flag the left column in PRs.*
|
||||
|
||||
| You wrote | Reach for | When NOT to |
|
||||
| -------------------------------------------------------------------- | ------------------------------------ | ----------------------------------------------------------------- |
|
||||
| `(vec (keep f xs))` | `(u/keepv f xs)` | — |
|
||||
| `(let [x (...)] (side-effect!) x)` | `(u/prog1 (...) (side-effect!))` | — (`<>` anaphor binds the result) |
|
||||
| `(into {} (map (juxt k identity)) xs)` | `(u/index-by k xs)` | — |
|
||||
| `(into {} (map (juxt k vf)) xs)` | `(u/index-by k vf xs)` | — |
|
||||
| `#(compare %2 %1)` | `u/reverse-compare` | — |
|
||||
| `(:col (t2/select-one [Model :col] …))` | `(t2/select-one-fn :col Model …)` | — |
|
||||
| `(or (:k m) default)` | `(:k m default)` | **The value can be present-but-nil** (hydrated DB null, etc.) |
|
||||
| `(let [x (...)] (if x then else))` | `(if-let [x (...)] then else)` | — |
|
||||
| `(filter #(= :a (:type %)) coll)` + same for `:b` … | `(group-by :type coll)` then dispatch | — |
|
||||
| `(->> xs (map f) (remove g) (mapv h))` | `(into [] (comp (map f) (remove g) (map h)) xs)` | when readability of the threading wins |
|
||||
| nested `let` / `if` / `if-let` | `b/cond` with `:let` | a 2-level nest may be clearer as-is |
|
||||
| `(let [{:keys [k]} arg] …)` as first form of body | destructure in the arg list | when the destructure also references something computed in `let` |
|
||||
| `(or (:k m) (get m (name k)))` per field | normalize once: `(update-keys m keyword)` then plain `:k` | when you genuinely need both shapes per-field |
|
||||
|
||||
## `metabase.util` (commonly aliased `[metabase.util :as u]`)
|
||||
|
||||
Sequences / collections:
|
||||
- `(u/keepv f coll)` — like `keep` but returns a vector (eager).
|
||||
- `(u/index-by kf coll)` / `(u/index-by kf vf coll)` — `{(kf x) (vf x)}` map.
|
||||
|
||||
Strings (locale-safe):
|
||||
- `(u/lower-case-en s)`, `(u/upper-case-en s)` — DO use these instead of `clojure.string/lower-case`/`upper-case` whenever the result feeds into matching/comparison; the locale-default versions break on Turkish `i`.
|
||||
- `(u/qualified-name x)` — keyword → `"ns/name"` or `"name"`; symbol → `(name x)`; string → identity.
|
||||
|
||||
Control flow:
|
||||
- `(u/prog1 first-form & body)` — anaphoric. Result of `first-form` is bound to `<>`. Returns first-form's value. Use for "do a thing, side-effect using its result, return the thing."
|
||||
- `(u/the-id x)` — get an entity id from an int or an instance.
|
||||
|
||||
Comparators:
|
||||
- `(u/reverse-compare x y)` — `(compare y x)`. Pass as a sort comparator: `(sort-by f u/reverse-compare coll)`.
|
||||
|
||||
Other:
|
||||
- `(u/conflicting-keys m1 m2)` — keys present in both with different values.
|
||||
|
||||
The full `metabase.util.cljc` is ~1300 lines — when in doubt, grep it.
|
||||
|
||||
## `metabase.util.namespaces` (aliased `shared.ns`)
|
||||
|
||||
- `(shared.ns/import-fns [other-ns f1 f2] [another-ns f3])` — re-export defns from another ns through the current ns. The `:require [other-ns :as other-ns]` alias **must stay** — clj-kondo may flag it as unused but it's needed at macro expansion. Add a `;; needed for import-fns` comment so it doesn't get auto-stripped.
|
||||
|
||||
## Toucan 2 (aliased `[toucan2.core :as t2]`)
|
||||
|
||||
Single-field reads (avoids fetching whole rows):
|
||||
- `(t2/select-one-fn :col Model :id 123)` — projection-friendly: when `f` is a keyword, only that column is fetched. Use instead of `(:col (t2/select-one [Model :col] …))`.
|
||||
- `(t2/select-fn-set f Model & conditions)` — set of `(f row)` values.
|
||||
- `(t2/select-fn-vec f Model & conditions)` — vector of `(f row)` values.
|
||||
- `(t2/select-pks-set Model & conditions)` — set of PKs.
|
||||
|
||||
Existence / counts:
|
||||
- `(t2/exists? Model & conditions)` → boolean.
|
||||
- `(t2/count Model & conditions)` → long.
|
||||
|
||||
Writes:
|
||||
- `(t2/insert-returning-instances! Model row-or-rows)` — returns the rows with PKs set.
|
||||
- `(t2/insert-returning-pks! Model row-or-rows)` — just the PKs.
|
||||
- `(t2/update! Model id-or-conditions changes)`.
|
||||
- `(t2/delete! Model id-or-conditions)`.
|
||||
|
||||
Transactions:
|
||||
- `(t2/with-transaction [t-conn] body…)` — opens a txn on the app DB conn.
|
||||
|
||||
Hydration:
|
||||
- `(t2/hydrate instances :key1 :key2)` — invokes `:batched-hydrate` methods.
|
||||
- Define via `(methodical/defmethod t2/batched-hydrate [:model/Foo :the-key] [_ k rows] …)`.
|
||||
|
||||
Define a model:
|
||||
- `(methodical/defmethod t2/table-name :model/Foo [_] :foo_table)`
|
||||
- `(t2/deftransforms :model/Foo {:col mi/transform-json})` — common: `transform-json`, `transform-secret-value` (encrypts), `transform-keyword`.
|
||||
|
||||
## Better-cond (`[better-cond.core :as b]`)
|
||||
|
||||
`(b/cond …)` like `cond` but supports interleaved `:let [bindings]` and `:when` clauses. Lets you flatten cascading `let`/`if`/`if-let`/`let` into a linear spine.
|
||||
|
||||
```clojure
|
||||
(b/cond
|
||||
:let [thread (load-thread id)]
|
||||
(nil? thread) :not-found
|
||||
|
||||
:let [exploration (load-exploration (:exploration-id thread))
|
||||
reason (gate-check exploration)]
|
||||
reason
|
||||
(skip! reason)
|
||||
|
||||
:let [pool (build-pool thread)]
|
||||
(empty? pool) :skip-no-data
|
||||
|
||||
:else
|
||||
(run! thread pool))
|
||||
```
|
||||
|
||||
Use it when a function would otherwise be a staircase of nested forms. Don't reach for it for a single `if`/`let`.
|
||||
|
||||
## Schemas — `mr/def` for shared, inline for single-use
|
||||
|
||||
When a schema is used in 2+ places or has a name worth documenting, extract it into the registry via `mr/def`:
|
||||
|
||||
```clojure
|
||||
(:require [metabase.util.malli.registry :as mr])
|
||||
|
||||
(mr/def ::options
|
||||
"Options for Pulse channel rendering."
|
||||
[:map
|
||||
[:channel.render/include-buttons? {:description "default: false" :optional true} :boolean]])
|
||||
```
|
||||
|
||||
`mr/def` keys are always `::namespaced` keywords; downstream callers reference via the fully-qualified keyword. Single-use schemas stay inline in the `mu/defn`'s `:-` annotation — no need to extract.
|
||||
|
||||
## Malli — `metabase.util.malli`
|
||||
|
||||
- `(mu/defn my-fn :- :string [x :- :int, y :- [:maybe :string]] …)` — inline schemas; runtime validation in dev, no-op in prod by default.
|
||||
- `(mu/disable-enforcement & body)` — wrap test/perf-sensitive blocks.
|
||||
- Common malli schemas: `:int`, `:string`, `:boolean`, `:keyword`, `:map`, `[:maybe T]`, `[:sequential T]`, `[:enum :a :b]`, `[:map [:k T]]`, `pos-int?`, `nat-int?`.
|
||||
|
||||
## i18n — `[metabase.util.i18n :refer [tru deferred-tru]]`
|
||||
|
||||
- `(tru "Hello {0}" name)` — translate at call time. Use in request-handling code.
|
||||
- `(deferred-tru "…")` — translate at read time. Use in static contexts: `defsetting :description`, `defn` metadata `:description`, top-level `def`s of user-visible labels.
|
||||
- ICU MessageFormat: doubled apostrophe for a literal: `(tru "it''s ready")`. Numbered placeholders: `{0}`, `{1}`.
|
||||
- **Don't translate** developer-only debug strings (REPL helper text, internal logs).
|
||||
|
||||
## Logging — `[metabase.util.log :as log]`
|
||||
|
||||
- Levels: `trace`, `debug`, `info`, `warn`, `error`.
|
||||
- Two flavors per level:
|
||||
- `(log/warn "msg")` or `(log/warn e "msg")` — exception first if present.
|
||||
- `(log/warnf "fmt %s" arg)` or `(log/warnf e "fmt %s" arg)` — printf-style.
|
||||
- Don't `(log/warn (format "fmt %s" arg))` — use `warnf`.
|
||||
|
||||
## Map / key normalization
|
||||
|
||||
When parsing loosely-typed external data (LLM JSON, request bodies) that may have **either** string or keyword keys:
|
||||
|
||||
- `(update-keys m keyword)` — top-level keys → keywords (Clojure 1.11+, in `clojure.core`).
|
||||
- `(clojure.walk/keywordize-keys m)` — recursive (use sparingly; can over-keywordize value strings if they happen to be in nested maps).
|
||||
- After normalizing once, downstream code uses plain `(:k m)` — don't write `(or (:k m) (get m (name k)))` at every use site.
|
||||
|
||||
Other useful map ops:
|
||||
- `(update-vals m f)` — Clojure 1.11+, in `clojure.core`.
|
||||
- `(medley.core/map-vals f m)`, `(m/map-keys f m)` — older alternatives. Medley pinned at **1.4** in this repo — `m/index-of` and some other newer fns are NOT available.
|
||||
|
||||
## Comparators & sorting
|
||||
|
||||
- `(sort-by f u/reverse-compare coll)` — descending.
|
||||
- Vector comparator: `(sort-by (juxt :primary :secondary) coll)`.
|
||||
- Use `(- x)` on a primary numeric key to invert: `(sort-by (juxt #(- (:score %)) :id) coll)` (handles ties on ascending `:id`).
|
||||
|
||||
## Anti-patterns to avoid
|
||||
|
||||
- **`(let [x (do-thing!)] (other-thing! x) x)`** — use `u/prog1`.
|
||||
- **`(vec (keep …))`, `(vec (map …))`** — use `u/keepv` or a transducer `(into [] (comp …) coll)`.
|
||||
- **Multiple `filter` passes over the same coll for different `:type`s** — one `(group-by :type coll)`.
|
||||
- **Per-field string/keyword dual-lookup** — normalize once with `update-keys keyword`.
|
||||
- **Reaching into another module's tables via raw `:left-join` SQL** — expose a function on the owning model namespace and call that.
|
||||
- **Snake-case keys on a massaged in-memory map** — appdb-row only. Convert at the parse boundary (e.g., when a json response becomes an internal record).
|
||||
- **`(double (:k m 0.0))` on a hydrated DB column** — present-but-nil NPEs. Use `(double (or (:k m) 0.0))`.
|
||||
- **`defmulti` with every impl on one page** — extension story is a fiction. Use a private lookup map + a private fn.
|
||||
- **`(pos? (count xs))` to test a lazy `xs` for non-emptiness** — `count` forces the whole seq into memory. Use `(seq xs)`. (Rebuilding the seq — `(map f xs)` twice — is the only thing that recomputes; a bound seq caches.)
|
||||
- **Plain `with-redefs` on a `defn` var in a test** — not parallel-safe; the kondo hook will warn. Use `mt/with-dynamic-fn-redefs`.
|
||||
- **`(Thread/sleep N)` in an async test** — flake source #1. `(deref p timeout-ms ::timeout)` and assert `not= ::timeout`.
|
||||
- **Hand-rolled `(str/split host #":")` for origin/host comparison** — mis-handles IPv6. Use `mw.security/parse-url` (or `try-parse-url` for client-controlled input).
|
||||
- **`(subs (pr-str huge) 0 N)` on a log path** — the multi-MB string is already allocated. Bind `*print-length*`/`*print-level*` around `pr-str` instead.
|
||||
- **New retry loop / new "find this column" helper / new transform-source-type predicate** — search first. `metabase.util.retry/with-retry`, `metabase.lib.equality/find-matching-column`, and `metabase.transforms.feature-gating/any-transforms-enabled?` already exist (along with many others). Reviewers ask "isn't there already an X for this?" more than any single idiom.
|
||||
- **Plain `defn` when the function has any real contract** — `mu/defn` (or `mu/defn-`) with `:- schema` on the return and each param is the house default; reserve plain `defn` for trivial dev/helper code.
|
||||
- **`(:import (java.time …))` + `(LocalDate/now)`** in new code — use `[java-time.api :as t]` + `[metabase.util.date-2 :as u.date]`. Direct `java.time` interop only when a Java API genuinely needs the class.
|
||||
- **Vanilla `defmethod`** — use `methodical/defmethod` (it takes a docstring; the codebase uses it uniformly).
|
||||
- **`(when (premium/has-feature? :flag) …)` in code** — gate via `defenterprise` so OSS/EE bodies live in their respective namespaces; never sprinkle runtime feature checks.
|
||||
- **No `(set! *warn-on-reflection* true)` after the `ns` form** in a new `.clj` file — every `.clj` source file in this repo has it (not `.cljc`).
|
||||
- **`(ns … (:require [a :as a] [b :as b]))` on one line; `(:import java.io.File java.net.URI)` bare** — both wrong shapes. Requires one per line under `:require`, imports as `(Package Class)` lists.
|
||||
- **Inventing a fresh `:as` alias for `metabase.util`, `metabase.util.malli`, `metabase.util.log`, `java-time.api`, etc.** — use the canonical alias (see the table in Style Conventions). Mixing aliases makes `grep`-based discovery worse.
|
||||
|
||||
## When in doubt
|
||||
|
||||
- Run `grep -n "defn " src/metabase/util.cljc | head -80` to scan helpers.
|
||||
- Run `grep -rn "defmacro\|defn-? [a-z-]*\?\b" src/metabase/util.cljc | head` for predicates.
|
||||
- Check `medley.core/` if it looks "collection-y" — but **Medley is pinned at 1.4** here; new fns aren't available.
|
||||
- Search the codebase for an existing use of the pattern you're about to write — if no one else does it, there's usually a helper.
|
||||
|
||||
@@ -6,4 +6,3 @@
|
||||
- Run targeted tests, and lint continuously during development
|
||||
- Prioritize understanding existing patterns before implementing
|
||||
- Don't commit changes, leave it for the user to review and make commits
|
||||
- Minimize comments. Document only non-obvious intent, trade-offs, or constraints that the code itself cannot convey. Do not narrate what the code does — keep code self-descriptive through naming and structure.
|
||||
|
||||
@@ -1,69 +0,0 @@
|
||||
# React, Redux, and Data-Layer Patterns
|
||||
|
||||
## API access and RTK Query
|
||||
|
||||
- **Use RTK Query for server data.** Add typed endpoints under `metabase/api` or `metabase-enterprise/api`; don't add ad-hoc `fetch`, service wrappers, or new thunks for ordinary API access.
|
||||
- **Keep API files transport-focused.** Endpoint files should cover request/response typing, URLs, params, light response normalization, and cache tags — not feature business logic.
|
||||
- **Use canonical API types.** Prefer `metabase-types/api`; fix or derive missing types instead of casting endpoint results. Refer to API endpoint implementation in Clojure when building new endpoint connections.
|
||||
- **Tag caches deliberately.** Use `listTag`, `idTag`, `provide*Tags`, and `invalidateTags(error, tags)` where appropriate.
|
||||
- **Guard missing query inputs.** Use `skipToken` or `skip` until all required inputs exist. Build query requirements where the query is created, not in a downstream render path.
|
||||
- **Use lazy queries only for events.** Data needed to render a screen belongs in a regular query hook at the owner component/hook.
|
||||
- **Handle mutations with `unwrap()`.** Prefer `await trigger(args).unwrap()` in submit handlers; otherwise branch on `result.error` immediately.
|
||||
|
||||
## Where data should live
|
||||
|
||||
- **Server data:** RTK Query cache, not duplicated Redux state.
|
||||
- **Form state:** Form library or local component state, not global Redux.
|
||||
- **Transient UI state:** Component state or narrow context, not store-wide flags.
|
||||
- **Cross-route/global tasks:** Redux slice with public selectors.
|
||||
- **Derived values:** Selectors or pure helpers, not stored copies of computed data. Memoized if identity is important.
|
||||
|
||||
## Redux state modeling
|
||||
|
||||
- **Avoid using Redux**. Most problems can be solved with local component state or RTK Query.
|
||||
- **Use Redux only when needed.** Reach for Redux only when distant parts of the app need durable shared state.
|
||||
- **Model async status explicitly.** Use a discriminated union or explicit `status`/`isLoading`/`error`; don't infer loaded or failed state from truthy/falsy `data`.
|
||||
- **Store canonical state only.** Derive filtered lists, labels, permissions, booleans, and display variants in selectors.
|
||||
- **Read through selectors.** Name selectors after what they return (`getCurrentTask`, `getHasPendingMutation`), not how they compute it.
|
||||
- **Add missing public actions/selectors.** Don't reach into another feature's private state shape because an action or selector is missing.
|
||||
- **Use `createSelector` for derived state.** Keep selectors pure, cheap, and stable by reference when consumers depend on identity.
|
||||
|
||||
## React hooks and components
|
||||
|
||||
- **`useMemo` computes render values.** Side effects belong in `useEffect`; event-driven changes belong in event handlers or mutation callbacks.
|
||||
- **Keep effects narrow and idempotent.** Include all dependencies, guard inside the effect, and clean up timers/subscriptions. Complex logic should be split to multiple effects or custom hooks, also consider extracting functional pieces.
|
||||
- **Hooks can be used to encapsulate complex logic.** For example, a hook can normalize data, handle form's state and validation, or a hook can manage a table's pagination and sorting. If component's logic becomes too complex, consider extracting a hook.
|
||||
- **Hooks orchestrate behavior.** Custom hooks should return a small typed result object: data, loading/error state, and callbacks when relevant.
|
||||
- **Don't mirror props/query data into state** unless the consumer can edit that local copy. Keep derived render data as such.
|
||||
- **Use `useCallback` for stable identity needs, but deliberately.** Wrap callbacks passed to memoized children, dependency arrays, context values, or hooks that require stable identity — not every local handler.
|
||||
- **Memoize object/array props only when identity matters.** Prefer simpler plain values otherwise.
|
||||
- **Stories for presentational components**. Create Storybook stories for presentational components to test different states and props. Stories live next to the component.
|
||||
|
||||
## Performance
|
||||
|
||||
- **Optimize only where identity or scale matters.** Prefer clear code first, then add `memo`, `useMemo`, `useCallback`, selectors, virtualization, or split components when render cost, prop identity, or list size justifies it.
|
||||
- **Keep context values stable and avoid frequent updates.** Values updated often should be avoided in context consumed by many components, or stored by reference with a stable context value and explicit subscriptions/selectors.
|
||||
- **Read global state high in the tree.** Let parent components select context/Redux/API state, derive the view model, and pass narrow props top-down. But avoid passing props too deep (reconsider if more thatn 2 levels).
|
||||
- **Avoid unstable props in hot paths.** Don't pass fresh object/array/function literals to memoized children; memoize them or move them outside render when identity matters.
|
||||
- **Render large lists deliberately.** Use pagination, windowing/virtualization, stable keys, and row-level memoization for large or frequently updating lists.
|
||||
- **Consider memoization for leaf components.** If the source of data updates often, but the component doesn't need to re-render, consider memoizing it.
|
||||
|
||||
## Loading, error, and empty states
|
||||
|
||||
- **Every async operation needs loading UI.** Use `LoadingAndErrorWrapper` or `DelayedLoadingAndErrorWrapper` for full-page/panel loading; use disabled controls and inline progress for local actions.
|
||||
- **Every async operation needs an error branch.** Use wrappers or feature-specific alerts for query errors; use `getErrorMessage(error, fallback)` plus a toast/form error for mutation failures.
|
||||
- **Empty is its own state.** Render no-results, empty-list, and permission-limited states intentionally; don't let them look like unfinished loading.
|
||||
- **Combine multiple query states explicitly.** Example: `isLoading = a.isLoading || b.isLoading`; `error = a.error || b.error`.
|
||||
- **Distinguish first load from refresh.** Use `isLoading` for first-load blockers and `isFetching` for background refresh indicators.
|
||||
- **Test all async branches.** Cover loading, error, empty, and success states when adding or changing data access.
|
||||
|
||||
## Review checklist
|
||||
|
||||
- Server data uses a typed RTK Query endpoint with correct cache tags.
|
||||
- Query inputs are guarded with `skipToken`/`skip` until ready.
|
||||
- Global state is justified, minimal, and read through selectors.
|
||||
- Context values are stable; frequently changing values don't force broad re-renders.
|
||||
- Leaf components stay dumb and receive narrow props from parent containers.
|
||||
- Loading, error, empty, and success branches are visible and tested.
|
||||
- Hooks are pure during render; side effects are confined to effects or event handlers.
|
||||
- Mutation failures are surfaced to people using the UI.
|
||||
@@ -2,8 +2,8 @@
|
||||
|
||||
- **Lint:** `bun run lint-eslint-pure`
|
||||
- Run ESLint on the codebase
|
||||
- **Format:** `bun run format`
|
||||
- Format code with oxfmt (`oxfmt --write`); use `bun run lint-format-pure` to check without writing
|
||||
- **Format:** `bun run prettier`
|
||||
- Format code using Prettier
|
||||
- **Type Check:** `bun run type-check-pure`
|
||||
- Run TypeScript type checking
|
||||
|
||||
|
||||
@@ -149,7 +149,8 @@ ms/NonBlankString ;; Non-empty string
|
||||
ms/BooleanValue ;; String "true"/"false" or boolean
|
||||
ms/MaybeBooleanValue ;; BooleanValue or nil
|
||||
ms/TemporalString ;; ISO-8601 date/time string (for REQUEST params only!)
|
||||
ms/OpaqueJSONObject ;; JSON object whose keys are not ours to declare; string-keyed
|
||||
ms/Map ;; Any map
|
||||
ms/JSONString ;; JSON-encoded string
|
||||
ms/PositiveNum ;; Positive number
|
||||
ms/IntGreaterThanOrEqualToZero ;; 0 or positive
|
||||
```
|
||||
|
||||
@@ -10,23 +10,23 @@ This skill helps you add product analytics (Snowplow) events to track user inter
|
||||
|
||||
## Quick Reference
|
||||
|
||||
Analytics events in Metabase use Snowplow with typed event schemas. Simple events are declared **where they are used** — `trackSimpleEvent` is generic and validates the payload at the call site.
|
||||
Analytics events in Metabase use Snowplow with typed event schemas. All events must be defined in TypeScript types before use.
|
||||
|
||||
**Key Files:**
|
||||
- `frontend/src/metabase/analytics/event.ts` - Core tracking functions, `trackSimpleEvent` / `trackSchemaEvent` (import from `metabase/analytics`)
|
||||
- `frontend/src/metabase-types/analytics/event.ts` - The shared `SimpleEventSchema` only. **Do not add event types here** (see below)
|
||||
- `frontend/src/metabase-types/analytics/schema.ts` - Schema registry (custom/legacy schemas only)
|
||||
- Feature-specific `analytics.ts` files - Where your tracking functions and any local types live
|
||||
- `frontend/src/metabase-types/analytics/event.ts` - Event type definitions
|
||||
- `frontend/src/metabase-types/analytics/schema.ts` - Schema registry
|
||||
- `frontend/src/metabase/analytics/` - Core tracking functions (import from `metabase/analytics`)
|
||||
- Feature-specific `analytics.ts` files - Tracking function wrappers
|
||||
|
||||
## Quick Checklist
|
||||
|
||||
When adding a new analytics event:
|
||||
|
||||
- [ ] Pick an event name (snake_case, past tense)
|
||||
- [ ] Add a tracking function to the feature's `analytics.ts` file, calling `trackSimpleEvent()`
|
||||
- [ ] Keep any field unions (e.g. `"success" | "failure"`) as local types in that same file
|
||||
- [ ] Import and call the tracking function at the interaction point
|
||||
- [ ] Do **not** add an event type to `metabase-types/analytics/event.ts` or to any union
|
||||
- [ ] Define event type in `frontend/src/metabase-types/analytics/event.ts`
|
||||
- [ ] Add event to appropriate union type (e.g., `DataStudioEvent`, `SimpleEvent`)
|
||||
- [ ] Create tracking function in feature's `analytics.ts` file
|
||||
- [ ] Import and call tracking function at the interaction point
|
||||
- [ ] Use `trackSimpleEvent()` for basic events (most common)
|
||||
|
||||
## Event Schema Types
|
||||
|
||||
@@ -47,25 +47,6 @@ type SimpleEventSchema = {
|
||||
|
||||
**When to use:** 90% of events fit this schema. Use for clicks, opens, closes, creates, deletes, etc.
|
||||
|
||||
`trackSimpleEvent` is generic and enforces this schema on the object literal you pass it:
|
||||
|
||||
```typescript
|
||||
// frontend/src/metabase/analytics/event.ts
|
||||
export function trackSimpleEvent<
|
||||
T extends SimpleEventSchema &
|
||||
Record<Exclude<keyof T, keyof SimpleEventSchema>, never>,
|
||||
>(event: T) {
|
||||
trackSchemaEvent("simple_event", event);
|
||||
}
|
||||
```
|
||||
|
||||
That means a missing `event` or any field outside `SimpleEventSchema` is a compile error at the call
|
||||
site. There is no separate event type to declare and no `satisfies` clause to add — the old
|
||||
`ValidateEvent<...>` helper is no longer exported and is not part of the workflow.
|
||||
|
||||
`trackSchemaEvent` is generic too: it correlates the schema name with the payload type, so you can't
|
||||
send a dashboard event under the `simple_event` schema.
|
||||
|
||||
### 2. Custom Schemas (legacy, no events are being added)
|
||||
|
||||
Consider adding new event schema only in very special cases.
|
||||
@@ -76,7 +57,36 @@ Consider adding new event schema only in very special cases.
|
||||
|
||||
### Example: Track when a user applies filters in a table picker
|
||||
|
||||
#### Step 1: Create Tracking Functions
|
||||
#### Step 1: Define Event Types
|
||||
|
||||
Add event type definitions to `frontend/src/metabase-types/analytics/event.ts`:
|
||||
|
||||
```typescript
|
||||
export type DataStudioTablePickerFiltersAppliedEvent = ValidateEvent<{
|
||||
event: "data_studio_table_picker_filters_applied";
|
||||
}>;
|
||||
|
||||
export type DataStudioTablePickerFiltersClearedEvent = ValidateEvent<{
|
||||
event: "data_studio_table_picker_filters_cleared";
|
||||
}>;
|
||||
```
|
||||
|
||||
#### Step 2: Add to Union Type
|
||||
|
||||
Find or create the appropriate union type and add your events:
|
||||
|
||||
```typescript
|
||||
export type DataStudioEvent =
|
||||
| DataStudioLibraryCreatedEvent
|
||||
| DataStudioTablePublishedEvent
|
||||
| DataStudioGlossaryCreatedEvent
|
||||
| DataStudioGlossaryEditedEvent
|
||||
| DataStudioGlossaryDeletedEvent
|
||||
| DataStudioTablePickerFiltersAppliedEvent // <- Add here
|
||||
| DataStudioTablePickerFiltersClearedEvent; // <- Add here
|
||||
```
|
||||
|
||||
#### Step 3: Create Tracking Functions
|
||||
|
||||
In your feature's `analytics.ts` file (e.g., `enterprise/frontend/src/metabase-enterprise/data-studio/analytics.ts`):
|
||||
|
||||
@@ -96,7 +106,7 @@ export const trackDataStudioTablePickerFiltersCleared = () => {
|
||||
};
|
||||
```
|
||||
|
||||
#### Step 2: Use in Components
|
||||
#### Step 4: Use in Components
|
||||
|
||||
Import and call the tracking function at the interaction point:
|
||||
|
||||
@@ -128,11 +138,16 @@ function FilterPopover({ filters, onSubmit }) {
|
||||
|
||||
## Using SimpleEventSchema Fields
|
||||
|
||||
All examples below live in the feature's own `analytics.ts` — nothing is registered centrally.
|
||||
|
||||
### Example: Event with target_id
|
||||
|
||||
```typescript
|
||||
// Type definition
|
||||
export type DataStudioLibraryCreatedEvent = ValidateEvent<{
|
||||
event: "data_studio_library_created";
|
||||
target_id: number | null;
|
||||
}>;
|
||||
|
||||
// Tracking function
|
||||
export const trackDataStudioLibraryCreated = (id: CollectionId) => {
|
||||
trackSimpleEvent({
|
||||
event: "data_studio_library_created",
|
||||
@@ -147,10 +162,14 @@ trackDataStudioLibraryCreated(newLibrary.id);
|
||||
### Example: Event with triggered_from
|
||||
|
||||
```typescript
|
||||
// Local union, exported only if another feature needs to pass the same value
|
||||
export type NewButtonLocation = "app-bar" | "empty-collection";
|
||||
// Type definition
|
||||
export type NewButtonClickedEvent = ValidateEvent<{
|
||||
event: "new_button_clicked";
|
||||
triggered_from: "app-bar" | "empty-collection";
|
||||
}>;
|
||||
|
||||
export const trackNewButtonClicked = (location: NewButtonLocation) => {
|
||||
// Tracking function
|
||||
export const trackNewButtonClicked = (location: "app-bar" | "empty-collection") => {
|
||||
trackSimpleEvent({
|
||||
event: "new_button_clicked",
|
||||
triggered_from: location,
|
||||
@@ -168,31 +187,44 @@ export const trackNewButtonClicked = (location: NewButtonLocation) => {
|
||||
|
||||
### Example: Event with event_detail
|
||||
|
||||
Real example — `frontend/src/metabase/metadata/pages/shared/analytics.ts`:
|
||||
|
||||
```typescript
|
||||
export type MetadataEditEventDetail =
|
||||
| "type_casting"
|
||||
| "semantic_type_change"
|
||||
| "visibility_change";
|
||||
// Type definition
|
||||
export type MetadataEditEvent = ValidateEvent<{
|
||||
event: "metadata_edited";
|
||||
event_detail: "type_casting" | "semantic_type_change" | "visibility_change";
|
||||
triggered_from: "admin" | "data_studio";
|
||||
}>;
|
||||
|
||||
export const trackMetadataChange = (detail: MetadataEditEventDetail) => {
|
||||
// Tracking function
|
||||
export const trackMetadataChange = (
|
||||
detail: "type_casting" | "semantic_type_change" | "visibility_change",
|
||||
location: "admin" | "data_studio"
|
||||
) => {
|
||||
trackSimpleEvent({
|
||||
event: "metadata_edited",
|
||||
event_detail: detail,
|
||||
triggered_from: "admin",
|
||||
triggered_from: location,
|
||||
});
|
||||
};
|
||||
|
||||
// Usage
|
||||
trackMetadataChange("semantic_type_change");
|
||||
trackMetadataChange("semantic_type_change", "data_studio");
|
||||
```
|
||||
|
||||
### Example: Event with result and duration
|
||||
|
||||
See `frontend/src/metabase/archive/analytics.ts` for the real version of this.
|
||||
|
||||
```typescript
|
||||
// Type definition
|
||||
export type MoveToTrashEvent = ValidateEvent<{
|
||||
event: "moved-to-trash";
|
||||
target_id: number | null;
|
||||
triggered_from: "collection" | "detail_page" | "cleanup_modal";
|
||||
duration_ms: number | null;
|
||||
result: "success" | "failure";
|
||||
event_detail: "question" | "model" | "metric" | "dashboard";
|
||||
}>;
|
||||
|
||||
// Tracking function
|
||||
export const trackMoveToTrash = (params: {
|
||||
targetId: number | null;
|
||||
triggeredFrom: "collection" | "detail_page" | "cleanup_modal";
|
||||
@@ -248,16 +280,17 @@ try {
|
||||
"filters-applied" // Use underscore, not hyphen
|
||||
```
|
||||
|
||||
### Local Field Types (PascalCase, named after the field)
|
||||
|
||||
There is usually no `...Event` type to name anymore. When you do need a union for a field, name it
|
||||
after the field it feeds:
|
||||
### Event Type Names (PascalCase with "Event" suffix)
|
||||
|
||||
```typescript
|
||||
// Good
|
||||
type MetricDimensionResult = "success" | "failure"; // -> result
|
||||
export type MetadataEditEventDetail = "type_casting"; // -> event_detail
|
||||
type NewButtonLocation = "app-bar" | "empty-collection"; // -> triggered_from
|
||||
DataStudioLibraryCreatedEvent
|
||||
TablePickerFiltersAppliedEvent
|
||||
MetabotChatOpenedEvent
|
||||
|
||||
// Bad
|
||||
dataStudioLibraryCreated // Wrong case
|
||||
DataStudioLibraryCreated // Missing "Event" suffix
|
||||
```
|
||||
|
||||
### Tracking Function Names (camelCase with "track" prefix)
|
||||
@@ -276,28 +309,29 @@ logLibraryCreated // Use "track" prefix
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Pattern 1: Sharing Field Types Across Features
|
||||
### Pattern 1: Feature-Specific Union Types
|
||||
|
||||
When two features send the same event with a different `triggered_from`, export the field union from
|
||||
the owning feature's `analytics.ts` and import it — don't hoist anything into `metabase-types`:
|
||||
Group related events together:
|
||||
|
||||
```typescript
|
||||
// frontend/src/metabase/data-studio/data-model/analytics.ts
|
||||
import { trackSimpleEvent } from "metabase/analytics";
|
||||
import type { MetadataEditEventDetail } from "metabase/metadata/pages/shared/analytics";
|
||||
export type DataStudioEvent =
|
||||
| DataStudioLibraryCreatedEvent
|
||||
| DataStudioTablePublishedEvent
|
||||
| DataStudioGlossaryCreatedEvent;
|
||||
|
||||
export function trackMetadataChange(detail: MetadataEditEventDetail) {
|
||||
trackSimpleEvent({
|
||||
event: "metadata_edited",
|
||||
event_detail: detail,
|
||||
triggered_from: "data_studio",
|
||||
});
|
||||
}
|
||||
export type MetabotEvent =
|
||||
| MetabotChatOpenedEvent
|
||||
| MetabotRequestSentEvent
|
||||
| MetabotFixQueryClickedEvent;
|
||||
|
||||
// Then add to SimpleEvent union
|
||||
export type SimpleEvent =
|
||||
| /* other events */
|
||||
| DataStudioEvent
|
||||
| MetabotEvent
|
||||
| /* more events */;
|
||||
```
|
||||
|
||||
This is the point of the extensible-events design: enterprise and feature-tier types stay in their
|
||||
own module instead of being imported down into a shared union.
|
||||
|
||||
### Pattern 2: Conditional Tracking
|
||||
|
||||
Track different events based on user action:
|
||||
@@ -316,10 +350,10 @@ const handleSave = async () => {
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
### Don't: Add custom fields to a simple event
|
||||
### Don't: Add custom fields to SimpleEvent
|
||||
|
||||
```typescript
|
||||
// WRONG - SimpleEventSchema doesn't support custom fields (this is a compile error)
|
||||
// WRONG - SimpleEvent doesn't support custom fields
|
||||
export const trackFiltersApplied = (filters: FilterState) => {
|
||||
trackSimpleEvent({
|
||||
event: "filters_applied",
|
||||
@@ -345,43 +379,23 @@ export const trackFilterApplied = (filterType: string) => {
|
||||
};
|
||||
```
|
||||
|
||||
### Don't: Add event types to `metabase-types/analytics/event.ts`
|
||||
|
||||
The central `SimpleEvent` union was removed — it forced feature-tier types to be imported down into
|
||||
shared code, causing module-boundary violations. `trackSimpleEvent` is generic now, so the type adds
|
||||
nothing but duplication.
|
||||
### Don't: Forget to add event to union type
|
||||
|
||||
```typescript
|
||||
// ❌ WRONG - central declaration + re-import for a `satisfies` clause
|
||||
// frontend/src/metabase-types/analytics/event.ts
|
||||
// Define the event
|
||||
export type NewFeatureClickedEvent = ValidateEvent<{
|
||||
event: "new_feature_clicked";
|
||||
target_id: number;
|
||||
}>;
|
||||
|
||||
// frontend/src/metabase/my-feature/analytics.ts
|
||||
import type { NewFeatureClickedEvent } from "metabase-types/analytics";
|
||||
// ❌ WRONG - Forgot to add to SimpleEvent union
|
||||
// Event won't be recognized by TypeScript
|
||||
|
||||
export const trackNewFeatureClicked = (id: number) => {
|
||||
trackSimpleEvent({
|
||||
event: "new_feature_clicked",
|
||||
target_id: id,
|
||||
} satisfies NewFeatureClickedEvent);
|
||||
};
|
||||
|
||||
// ✓ RIGHT - the object literal is already checked by the generic
|
||||
// frontend/src/metabase/my-feature/analytics.ts
|
||||
export const trackNewFeatureClicked = (id: number) => {
|
||||
trackSimpleEvent({
|
||||
event: "new_feature_clicked",
|
||||
target_id: id,
|
||||
});
|
||||
};
|
||||
// ✓ RIGHT - Add to appropriate union
|
||||
export type SimpleEvent =
|
||||
| /* other events */
|
||||
| NewFeatureClickedEvent;
|
||||
```
|
||||
|
||||
A few `...Event` types still sit in `metabase-types/analytics/event.ts`. They are leftovers from PRs
|
||||
that landed around the refactor — don't copy them, and don't add to them.
|
||||
|
||||
### Don't: Mix up event name formats
|
||||
|
||||
```typescript
|
||||
@@ -449,44 +463,41 @@ Example console output:
|
||||
### Where to put tracking functions:
|
||||
|
||||
```
|
||||
Tracking functions AND their local field types (this is where new events live):
|
||||
Feature-specific analytics functions:
|
||||
frontend/src/metabase/{feature}/analytics.ts
|
||||
enterprise/frontend/src/metabase-enterprise/{feature}/analytics.ts
|
||||
|
||||
Event type definitions (all in one place):
|
||||
frontend/src/metabase-types/analytics/event.ts
|
||||
|
||||
Core tracking utilities:
|
||||
frontend/src/metabase/analytics/ (import from `metabase/analytics`)
|
||||
|
||||
Shared SimpleEventSchema only — nothing new goes here:
|
||||
frontend/src/metabase-types/analytics/event.ts
|
||||
```
|
||||
|
||||
In embedding SDK code, use `trackSdkSimpleEvent`
|
||||
(`frontend/src/embedding-sdk-bundle/analytics/snowplow.ts`) instead — the main-app `"sp"` tracker
|
||||
isn't initialized in the customer's page, so `trackSimpleEvent`'s Snowplow leg is a no-op there.
|
||||
|
||||
## Real-World Examples
|
||||
|
||||
See these files for reference:
|
||||
|
||||
- **Simple events + local field union**: `frontend/src/metabase/metadata/pages/shared/analytics.ts`
|
||||
- **Reusing another feature's field type**: `frontend/src/metabase/data-studio/data-model/analytics.ts`
|
||||
- **Result + duration timing**: `frontend/src/metabase/archive/analytics.ts`
|
||||
- **Enterprise feature events**: `enterprise/frontend/src/metabase-enterprise/google_drive/analytics.ts`
|
||||
- **Simple events**: `enterprise/frontend/src/metabase-enterprise/data-studio/analytics.ts`
|
||||
- **Events with variants**: `frontend/src/metabase/dashboard/analytics.ts`
|
||||
- **Complex events**: `frontend/src/metabase/query_builder/analytics.js`
|
||||
- **Event type examples**: `frontend/src/metabase-types/analytics/event.ts`
|
||||
|
||||
## Workflow Summary
|
||||
|
||||
1. **Identify the user interaction** to track
|
||||
2. **Decide on event name** (snake_case, descriptive)
|
||||
3. **Create tracking function** in feature's `analytics.ts`, calling `trackSimpleEvent()`
|
||||
4. **Add local field unions** in that same file if a field has a fixed set of values
|
||||
5. **Import and call** at the interaction point
|
||||
6. **Test** that events fire correctly
|
||||
3. **Define event type** in `event.ts` using `ValidateEvent`
|
||||
4. **Add to union type** (create feature union if needed)
|
||||
5. **Create tracking function** in feature's `analytics.ts`
|
||||
6. **Import and call** at the interaction point
|
||||
7. **Test** that events fire correctly
|
||||
|
||||
## Tips
|
||||
|
||||
- **Be specific** - `filters_applied` is better than `action_performed`
|
||||
- **Use past tense** - `library_created` not `create_library`
|
||||
- **Group related events** - Keep a feature's tracking functions together in its `analytics.ts`
|
||||
- **Group related events** - Create feature-specific event union types
|
||||
- **Track meaningful actions** - Not every click needs tracking
|
||||
- **Consider the data** - What would you want to analyze later?
|
||||
- **Stay consistent** - Follow existing naming patterns in the codebase
|
||||
|
||||
@@ -56,11 +56,6 @@ Use this to scan through changes efficiently:
|
||||
- [ ] References use `[[other-var]]` not backticks
|
||||
- [ ] `TODO` comments include author and date: `;; TODO (Name 2025-01-01) -- description`
|
||||
|
||||
Docstring content — check every new or rewritten docstring against the anti-pattern table in the
|
||||
style guide above. Read the diff's docstrings on their own, separately from the code: prose is
|
||||
where review attention slides off, and a long docstring reads as diligence rather than as the
|
||||
liability it usually is.
|
||||
|
||||
### Code Organization
|
||||
|
||||
- [ ] Everything `^:private` unless used elsewhere
|
||||
@@ -140,17 +135,6 @@ Quick scan for common issues:
|
||||
|
||||
> This public var needs a docstring explaining its purpose, inputs, and outputs.
|
||||
|
||||
**For docstrings documenting code they don't own:**
|
||||
|
||||
> This describes how `lib` decides binning strategies internally. Nothing here fails when that
|
||||
> changes, so it will go stale silently — drop it and let `available-binning-strategies` answer
|
||||
> for its own behavior.
|
||||
|
||||
**For docstring content that belongs in the body:**
|
||||
|
||||
> This paragraph explains why the branches are ordered this way, which is what someone editing
|
||||
> the `cond` needs — but a caller can't act on it. Move it to an inline comment above the `cond`.
|
||||
|
||||
**For organization issues:**
|
||||
|
||||
> This function is only used in this namespace, so it should be marked `^:private`.
|
||||
|
||||
@@ -60,60 +60,6 @@ Only fall back to `./bin/mage` commands when clojure-mcp is not available.
|
||||
Feel free to copy these REPL session trials into actual test cases using `deftest` and `is`.
|
||||
4. Once you know these functions are good, return to 1, and compose them into the task that you need to build.
|
||||
|
||||
## Writing Docstrings
|
||||
|
||||
A docstring is a contract for the *caller*, not a diary for the
|
||||
implementer. It states what the function does, what it takes, returns,
|
||||
throws, and the preconditions/invariants the caller must respect. Those
|
||||
guarantees and requirements *belong* there — they are exactly what the
|
||||
caller needs surfaced in the IDE.
|
||||
|
||||
When you find implementation context in a docstring, the default is to
|
||||
**relocate it, not delete it** — move it to an inline comment at the
|
||||
point in the body where it is actually relevant. That context is often
|
||||
genuinely valuable; it is just in the wrong place (the caller should not
|
||||
have to read it; the implementer standing at that line should). Delete
|
||||
outright only when it is blather: self-congratulation, restating the
|
||||
obvious, or documenting a property that is the expected default.
|
||||
|
||||
On that last case — narrating properties like "portable across all
|
||||
supported appdbs" earns no sentence. If it were not portable, that is
|
||||
either a bug, or it means callers must handle each case themselves — and
|
||||
in *that* case it is the *absence* of the property that must be
|
||||
documented. Document deviations from expectation, not conformance to it.
|
||||
|
||||
Heuristic — a sentence earns its place in the docstring only if it
|
||||
passes both tests:
|
||||
|
||||
1. Rewrite test (necessary, not sufficient): if rewriting the body to a
|
||||
different implementation with an identical contract could make the
|
||||
sentence false, it describes the implementation, not the contract.
|
||||
Relocate it to an inline comment at the point in the body where it
|
||||
applies.
|
||||
|
||||
2. Ownership test: if the sentence states a fact about code this
|
||||
function does not own — a callee's behavior, a library's guarantee —
|
||||
it survives any rewrite trivially, which is exactly why the rewrite
|
||||
test alone is not enough. If this body *relies* on that fact, it is
|
||||
implementation context: relocate it to the call site. If nothing here
|
||||
relies on it, it is blather about someone else's code: delete it, and
|
||||
let the callee's own docstring answer for it.
|
||||
|
||||
Say each fact once. Duplicated prose is a stale comment in advance: when
|
||||
the fact changes, one copy gets updated and the rest quietly rot. This
|
||||
holds even within code you own — a namespace docstring that
|
||||
re-summarizes its own multimethods, three sibling functions carrying the
|
||||
same paragraph about retry behavior. Give the fact one home, and have
|
||||
the other places name it rather than restate it.
|
||||
|
||||
Multi-line docstrings are not banned — a genuinely non-obvious constraint
|
||||
the code had to deal with can be worth explaining. But be prudent; the
|
||||
failure mode is far too much detail. When tempted to write a
|
||||
multi-paragraph explanatory docstring, check with the user first. And
|
||||
prefer a *test* to prose: if a future reader thinks "that's a silly way
|
||||
to do it" and changes it, a test should fail and tell them why. If that
|
||||
breakage keeps happening, *that* is the signal a comment was warranted.
|
||||
|
||||
## Critical Rules for Editing
|
||||
|
||||
- Be careful with parentheses counts when editing Clojure code
|
||||
@@ -121,7 +67,3 @@ breakage keeps happening, *that* is the signal a comment was warranted.
|
||||
- End all files with a newline
|
||||
- When editing tabular code, where the columns line up, try to keep them aligned
|
||||
- Spaces on a line with nothing after it is not allowed
|
||||
- After changing module boundaries (adding/removing/renaming a `src` namespace, a cross-module `require`
|
||||
or `:model/X` reference, or a new module), run `./bin/mage fix-modules-config` to regenerate
|
||||
`.clj-kondo/config/modules/config.edn` and keep `metabase.core.modules-test` green. No-op when nothing
|
||||
drifted; see "Module Boundaries" in the project `CLAUDE.md`.
|
||||
|
||||
@@ -22,11 +22,10 @@ Before starting, determine which mode to use:
|
||||
3. **(Conditional)** If after reading the spec a specific assertion or selector is **ambiguous** — you can't tell if it's the right anchor, or whether the test actually verifies what its title claims — briefly grep the related component for the test ID / role / text. **Don't read components by default.** Only do this when there's a real signal of confusion in the spec.
|
||||
4. Scan against the checklist + pattern table below.
|
||||
5. Number all issues sequentially. Skip nits — only flag what's worth fixing.
|
||||
6. **Always** finish with the honest e2e-vs-unit breakdown (see the section of that name below). It's part of every report, not an optional add-on.
|
||||
|
||||
### When to hand off instead of review
|
||||
|
||||
If the spec references an issue (`metabase#NNNNN`) and the user wants to **fix** flakiness or assess whether the test still reproduces the original bug, that's outside the review skill's scope. Point them at the dedicated flake-fixing workflow, which knows to fetch the issue and the resolving PR's diff. The review skill stays focused on "is this test well-written and conformant" — it doesn't fetch external issue context by default.
|
||||
If the spec references an issue (`metabase#NNNNN`) and the user wants to **fix** flakiness or assess whether the test still reproduces the original bug, that's outside the review skill's scope. Point them at `/fix-flakey-test` (or the dedicated flake-fixing workflow), which knows to fetch the issue and the resolving PR's diff. The review skill stays focused on "is this test well-written and conformant" — it doesn't fetch external issue context by default.
|
||||
|
||||
## Review checklist
|
||||
|
||||
@@ -195,41 +194,8 @@ Use the pending review workflow:
|
||||
|
||||
Each comment body starts with `**Issue N: [Brief title]**`.
|
||||
|
||||
## Honest e2e-vs-unit breakdown (always include)
|
||||
|
||||
Every report ends with this section. The numbered issues above are about whether the test is *well-written*; this section is about whether it should be an **e2e test at all**. They're orthogonal — a flawless spec can still be paying e2e prices for component-test value, and that's worth saying out loud.
|
||||
|
||||
The bar is simple: **an `it()` earns its place in e2e only if it exercises a layer you cannot fake without gutting the test's value.** In this codebase that means one or more of:
|
||||
|
||||
- a **real backend query against seeded data** (the assertion depends on the server actually filtering/sorting/bucketing rows, not on a mocked response);
|
||||
- **real rendering you then interact with** — clicking a charted ECharts SVG element at computed coordinates, drag/hit-testing, drill-through;
|
||||
- **routing across screens** — navigation, URL param round-trips that the backend then honors, browser back/forward;
|
||||
- **cross-screen traversal** — list → detail → back, where the value is the seam between screens.
|
||||
|
||||
Conversely, a test belongs in **Jest + RTL** (the cheaper layer this repo already uses for component/unit coverage) when its real subject is frontend-only:
|
||||
|
||||
- **pure frontend logic** — a bucketing/formatting/derivation decision (e.g. "single-day range → bucket by hour") that has one right answer independent of the backend;
|
||||
- **render assertions** — "the page mounts N titled cards and an SVG exists." A mocked dataset response + RTL verifies the titles; that an ECharts SVG renders is library behavior, not your logic;
|
||||
- **state/wiring** — tab → `aria-selected` → active-state, or select → querystring mapping. The *URL gets the param* half is component-level; only the *backend honors the param* half needs e2e.
|
||||
|
||||
Write the breakdown in three honest buckets. Ground each call in **which layers the test actually touches**, not in how the test is named:
|
||||
|
||||
1. **Genuinely must-be-e2e (keep)** — load-bearing. Name the layer(s) only a browser + backend can verify together.
|
||||
2. **Defensible-but-narrowable e2e** — justified as integration, but over-split or doing more boots than the coverage requires. Say what to merge.
|
||||
3. **Should be unit / component tests** — where the real savings are. Name the cheaper layer and what it would assert (often *more precisely* than the e2e test does indirectly).
|
||||
|
||||
Then a one-paragraph **bottom line**: what to keep as e2e, what to push down, and — if you pulled CI timing from `e2e/support/timings.json` for the Performance check — roughly how much of that wall time is *earned* by the must-be-e2e set versus paid by the rest.
|
||||
|
||||
Rules of engagement so this stays honest, not performative:
|
||||
|
||||
- **Don't inflate the keep bucket to look balanced.** If most of the spec is component-test value paying e2e prices, say exactly that. If every test is load-bearing, say that too — a spec where all 12 tests must be e2e is a legitimate outcome, not a failure to find candidates.
|
||||
- **Be specific.** Cite the `it()` titles and line ranges. "Several tests could be unit tests" is useless; "tests 1/17/18 assert only that N titled cards mount — RTL with a mocked dataset" is actionable.
|
||||
- **This is advice to the author, not a blocking issue.** It's a recommendation, separate from the numbered issues. Don't renumber it into the issue list.
|
||||
- **Stay rooted in what exists.** The cheaper layer here is Jest + RTL component/unit tests. Don't invent infrastructure the repo doesn't have, and don't recommend deleting backend-dependent coverage just because part of the test is frontend-only — split it, keep the integration half.
|
||||
|
||||
## Final check
|
||||
|
||||
1. Trim issues that won't materially help the author.
|
||||
2. Verify sequential numbering with no gaps.
|
||||
3. In PR mode, verify each issue was posted as a separate review comment.
|
||||
4. Verify the honest e2e-vs-unit breakdown is present — it's required in every report, even when the answer is "all of these need to be e2e."
|
||||
|
||||
@@ -37,16 +37,14 @@ First check if snapshots exist:
|
||||
ls e2e/snapshots/default.sql 2>/dev/null && echo "snapshots exist" || echo "snapshots missing"
|
||||
```
|
||||
|
||||
Always pass the spec path via `--spec`. The runner feeds its arguments to Cypress's CLI parser (`cypress.cli.parseRunArguments`), which silently drops bare positional paths and then runs the entire suite. A leading `run` argument is harmless, but the path itself must follow `--spec`. Sanity-check that the first `Running:` line says `(1 of 1)`.
|
||||
|
||||
If snapshots exist, skip regeneration for speed:
|
||||
```bash
|
||||
MB_EDITION=oss CYPRESS_VIDEO=false CYPRESS_RETRIES=0 CYPRESS_GUI=false GENERATE_SNAPSHOTS=false bun test-cypress --spec path/to/spec.cy.spec.js
|
||||
MB_EDITION=oss CYPRESS_VIDEO=false CYPRESS_RETRIES=0 CYPRESS_GUI=false GENERATE_SNAPSHOTS=false bun test-cypress $ARGUMENTS
|
||||
```
|
||||
|
||||
If snapshots are missing (first run), let the runner generate them:
|
||||
```bash
|
||||
MB_EDITION=oss CYPRESS_VIDEO=false CYPRESS_RETRIES=0 CYPRESS_GUI=false bun test-cypress --spec path/to/spec.cy.spec.js
|
||||
MB_EDITION=oss CYPRESS_VIDEO=false CYPRESS_RETRIES=0 CYPRESS_GUI=false bun test-cypress $ARGUMENTS
|
||||
```
|
||||
|
||||
When running enterprise tests, replace `MB_EDITION=oss` with `MB_EDITION=ee` in the commands above.
|
||||
@@ -69,11 +67,9 @@ GREP="test name here" MB_EDITION=oss bun test-cypress --spec path/to/spec.js
|
||||
When the user asks to verify a test is not flaky:
|
||||
|
||||
```bash
|
||||
MB_EDITION=oss bin/e2e-stress-test --spec path/to/spec.cy.spec.js
|
||||
MB_EDITION=oss bin/e2e-stress-test $ARGUMENTS
|
||||
```
|
||||
|
||||
(The stress script forwards its arguments to `bun test-cypress`, so the same `--spec` rule applies.)
|
||||
|
||||
Set `E2E_STRESS_RUNS=N` for more iterations (default: 5). Stops on first failure and prints screenshot paths. Read those screenshots to analyze the failure.
|
||||
|
||||
## Analyzing failures
|
||||
|
||||
@@ -20,8 +20,8 @@ For EACH styled component, go through every CSS property and ask: "Can this be a
|
||||
**Properties that ARE style props** (use these, not CSS modules):
|
||||
|
||||
- `display` → `display` prop
|
||||
- `color` → `c` prop (`c="core-brand"`, `c="text-primary"`)
|
||||
- `background-color` → `bg` prop (`bg="background_page-primary"`)
|
||||
- `color` → `c` prop (`c="brand"`, `c="text-primary"`)
|
||||
- `background-color` → `bg` prop (`bg="background-primary"`)
|
||||
- `font-size` → `fz` prop (`fz="md"`)
|
||||
- `font-weight` → `fw` prop (`fw="bold"`)
|
||||
- `line-height` → `lh` prop (`lh="md"`)
|
||||
@@ -57,7 +57,7 @@ For EACH styled component, go through every CSS property and ask: "Can this be a
|
||||
align="center" /* style prop */
|
||||
gap="sm" /* style prop */
|
||||
p="md" /* style prop */
|
||||
bg="background_page-primary" /* style prop */
|
||||
bg="background-primary" /* style prop */
|
||||
>
|
||||
```
|
||||
|
||||
@@ -149,19 +149,19 @@ Read the `.styled.tsx` file AND every component that imports from it. Understand
|
||||
|
||||
For each styled component, apply the Mantine-First Decision Gate above. Then determine the migration target:
|
||||
|
||||
| Emotion Pattern | Migration Target |
|
||||
| ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `styled.div` with only layout/spacing/color | `Box`, `Flex`, `Stack`, or `Group` with style props. **NO CSS module needed.** |
|
||||
| `styled.div` with flexbox column | `Stack` component with style props |
|
||||
| `styled.div` with flexbox row | `Flex` or `Group` component with style props |
|
||||
| `styled.span` / `styled.p` with color/weight/size | **`Text component="span"` with style props** (`c`, `fw`, `fz`). NO CSS module. |
|
||||
| `styled.div` with hover/focus/pseudo-selectors | **Hybrid**: Mantine component with style props for expressible properties + CSS module class for pseudo-selectors only |
|
||||
| `styled.div` with media queries (simple spacing/sizing) | **Responsive style props**: `p={{ base: "md", lg: "xl" }}`. NO CSS module. |
|
||||
| `styled.div` with media queries (complex/non-spacing) | CSS module for the media query parts, style props for the rest |
|
||||
| `styled.div` with animations/keyframes | CSS module for animation, style props for layout |
|
||||
| `styled(SomeComponent)` with only color/spacing/flex | **Wrap in Mantine component** with style props: `<Box c="core-brand" flex="0 0 auto"><Icon /></Box>`, or pass style props if the component accepts them |
|
||||
| `styled(SomeComponent)` with pseudo-selectors/complex styles | CSS module `className` on the component |
|
||||
| Dynamic props `styled.div<{ isActive: boolean }>` | Mantine style props for simple toggles (`c={active ? "core-brand" : "text-primary"}`), `cx()` with CSS module classes for complex state combinations involving pseudo-selectors |
|
||||
| Emotion Pattern | Migration Target |
|
||||
| ------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `styled.div` with only layout/spacing/color | `Box`, `Flex`, `Stack`, or `Group` with style props. **NO CSS module needed.** |
|
||||
| `styled.div` with flexbox column | `Stack` component with style props |
|
||||
| `styled.div` with flexbox row | `Flex` or `Group` component with style props |
|
||||
| `styled.span` / `styled.p` with color/weight/size | **`Text component="span"` with style props** (`c`, `fw`, `fz`). NO CSS module. |
|
||||
| `styled.div` with hover/focus/pseudo-selectors | **Hybrid**: Mantine component with style props for expressible properties + CSS module class for pseudo-selectors only |
|
||||
| `styled.div` with media queries (simple spacing/sizing) | **Responsive style props**: `p={{ base: "md", lg: "xl" }}`. NO CSS module. |
|
||||
| `styled.div` with media queries (complex/non-spacing) | CSS module for the media query parts, style props for the rest |
|
||||
| `styled.div` with animations/keyframes | CSS module for animation, style props for layout |
|
||||
| `styled(SomeComponent)` with only color/spacing/flex | **Wrap in Mantine component** with style props: `<Box c="brand" flex="0 0 auto"><Icon /></Box>`, or pass style props if the component accepts them |
|
||||
| `styled(SomeComponent)` with pseudo-selectors/complex styles | CSS module `className` on the component |
|
||||
| Dynamic props `styled.div<{ isActive: boolean }>` | Mantine style props for simple toggles (`c={active ? "brand" : "text-primary"}`), `cx()` with CSS module classes for complex state combinations involving pseudo-selectors |
|
||||
|
||||
### Step 3: Create CSS Module (if needed)
|
||||
|
||||
@@ -170,19 +170,19 @@ Create `ComponentName.module.css` alongside the component file:
|
||||
```css
|
||||
/* Use design system tokens — NEVER raw color/spacing values */
|
||||
.root {
|
||||
border: 1px solid var(--mb-color-border-neutral);
|
||||
border: 1px solid var(--mb-color-border);
|
||||
border-radius: var(--mantine-radius-md);
|
||||
background-color: var(--mb-color-background_page-primary);
|
||||
background-color: var(--mb-color-background-primary);
|
||||
}
|
||||
|
||||
/* Conditional states as separate classes, combined with cx() */
|
||||
.active {
|
||||
background-color: var(--mb-color-core-brand);
|
||||
background-color: var(--mb-color-brand);
|
||||
color: var(--mb-color-text-primary-inverse);
|
||||
}
|
||||
|
||||
.disabled {
|
||||
color: var(--mb-color-text-disabled);
|
||||
color: var(--mb-color-text-tertiary);
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
@@ -191,8 +191,8 @@ Create `ComponentName.module.css` alongside the component file:
|
||||
cursor: pointer;
|
||||
|
||||
&:hover {
|
||||
color: var(--mb-color-core-brand);
|
||||
background-color: var(--mb-color-background_surface-hover);
|
||||
color: var(--mb-color-brand);
|
||||
background-color: var(--mb-color-background-hover);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -255,9 +255,9 @@ Remove the old styled file entirely. Remove all imports of it from other files.
|
||||
|
||||
**Colors** (`c`, `bg`):
|
||||
|
||||
- Text: `"text-primary"`, `"text-secondary"`, `"text-disabled"`
|
||||
- Background: `"background_page-primary"`, `"background_page-secondary"`, `"background_surface-hover"`
|
||||
- Brand: `"core-brand"`, `"feedback-negative"`, `"feedback-positive"`, `"feedback-warning"`
|
||||
- Text: `"text-primary"`, `"text-secondary"`, `"text-tertiary"`
|
||||
- Background: `"background-primary"`, `"background-secondary"`, `"background-hover"`
|
||||
- Brand: `"brand"`, `"error"`, `"success"`, `"warning"`
|
||||
|
||||
**Typography** (`fz`, `fw`, `lh`, `ta`, `ff`):
|
||||
|
||||
@@ -279,11 +279,11 @@ Remove the old styled file entirely. Remove all imports of it from other files.
|
||||
|
||||
**Colors** — `var(--mb-color-<name>)`:
|
||||
|
||||
- `--mb-color-text-primary`, `--mb-color-text-secondary`, `--mb-color-text-disabled`
|
||||
- `--mb-color-background_page-primary`, `--mb-color-background_page-secondary`, `--mb-color-background_surface-hover`
|
||||
- `--mb-color-text-primary`, `--mb-color-text-secondary`, `--mb-color-text-tertiary`
|
||||
- `--mb-color-background-primary`, `--mb-color-background-secondary`, `--mb-color-background-hover`
|
||||
- `--mb-color-border`, `--mb-color-border-strong`, `--mb-color-border-subtle`
|
||||
- `--mb-color-core-brand`, `--mb-color-core-brand-hover`
|
||||
- `--mb-color-feedback-negative`, `--mb-color-feedback-positive`, `--mb-color-feedback-warning`
|
||||
- `--mb-color-brand`, `--mb-color-brand-hover`
|
||||
- `--mb-color-error`, `--mb-color-success`, `--mb-color-warning`
|
||||
- `--mb-color-shadow`, `--mb-color-focus`
|
||||
- `--mb-color-accent0` through `--mb-color-accent7`
|
||||
|
||||
@@ -398,13 +398,13 @@ export const Wrapper = styled.div`
|
||||
```tsx
|
||||
export const Item = styled.div<{ isSelected: boolean; disabled: boolean }>`
|
||||
color: ${(props) =>
|
||||
props.disabled ? color("text-disabled") : color("text-primary")};
|
||||
props.disabled ? color("text-tertiary") : color("text-primary")};
|
||||
background-color: ${(props) =>
|
||||
props.isSelected ? color("core-brand") : "transparent"};
|
||||
props.isSelected ? color("brand") : "transparent"};
|
||||
cursor: ${(props) => (props.disabled ? "default" : "pointer")};
|
||||
|
||||
&:hover {
|
||||
color: ${(props) => !props.disabled && color("core-brand")};
|
||||
color: ${(props) => !props.disabled && color("brand")};
|
||||
}
|
||||
`;
|
||||
```
|
||||
@@ -418,20 +418,20 @@ export const Item = styled.div<{ isSelected: boolean; disabled: boolean }>`
|
||||
cursor: pointer;
|
||||
|
||||
&:hover {
|
||||
color: var(--mb-color-core-brand);
|
||||
color: var(--mb-color-brand);
|
||||
}
|
||||
}
|
||||
|
||||
.selected {
|
||||
background-color: var(--mb-color-core-brand);
|
||||
background-color: var(--mb-color-brand);
|
||||
}
|
||||
|
||||
.disabled {
|
||||
color: var(--mb-color-text-disabled);
|
||||
color: var(--mb-color-text-tertiary);
|
||||
cursor: default;
|
||||
|
||||
&:hover {
|
||||
color: var(--mb-color-text-disabled);
|
||||
color: var(--mb-color-text-tertiary);
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -457,7 +457,7 @@ When a styled component wraps another component, first check if the styles can b
|
||||
export const CardIcon = styled(Icon)`
|
||||
display: block;
|
||||
flex: 0 0 auto;
|
||||
color: var(--mb-color-core-brand);
|
||||
color: var(--mb-color-brand);
|
||||
`;
|
||||
|
||||
export const CardTitle = styled(Ellipsified)`
|
||||
@@ -471,7 +471,7 @@ export const CardTitle = styled(Ellipsified)`
|
||||
**After — style props on Mantine wrapper (preferred when ALL properties are expressible):**
|
||||
|
||||
```tsx
|
||||
<Box display="block" flex="0 0 auto" c="core-brand">
|
||||
<Box display="block" flex="0 0 auto" c="brand">
|
||||
<Icon {...icon} />
|
||||
</Box>
|
||||
<Box c="text-primary" fz="md" fw="bold" ml="md">
|
||||
@@ -483,13 +483,13 @@ export const CardTitle = styled(Ellipsified)`
|
||||
|
||||
```css
|
||||
.icon {
|
||||
color: var(--mb-color-core-brand);
|
||||
color: var(--mb-color-brand);
|
||||
width: 1.5rem;
|
||||
height: 1.5rem;
|
||||
cursor: pointer;
|
||||
|
||||
&:hover {
|
||||
color: var(--mb-color-core-brand-hover);
|
||||
color: var(--mb-color-brand-hover);
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -570,12 +570,12 @@ export const Spinner = styled.div`
|
||||
|
||||
### Pattern 6: color() Function → CSS Variables
|
||||
|
||||
| Emotion (JS) | CSS Module | Mantine Style Prop |
|
||||
| -------------------------- | ----------------------------------------------------------------- | ------------------------------------- |
|
||||
| `color("core-brand")` | `var(--mb-color-core-brand)` | `c="core-brand"` or `bg="core-brand"` |
|
||||
| `color("text-primary")` | `var(--mb-color-text-primary)` | `c="text-primary"` |
|
||||
| `color("border-neutral")` | `var(--mb-color-border-neutral)` | N/A (use CSS module) |
|
||||
| `alpha("core-brand", 0.2)` | `color-mix(in srgb, var(--mb-color-core-brand), transparent 80%)` | N/A (use CSS module) |
|
||||
| Emotion (JS) | CSS Module | Mantine Style Prop |
|
||||
| ----------------------- | ------------------------------------------------------------ | --------------------------- |
|
||||
| `color("brand")` | `var(--mb-color-brand)` | `c="brand"` or `bg="brand"` |
|
||||
| `color("text-primary")` | `var(--mb-color-text-primary)` | `c="text-primary"` |
|
||||
| `color("border")` | `var(--mb-color-border)` | N/A (use CSS module) |
|
||||
| `alpha("brand", 0.2)` | `color-mix(in srgb, var(--mb-color-brand), transparent 80%)` | N/A (use CSS module) |
|
||||
|
||||
### Pattern 7: Responsive Spacing/Sizing → Responsive Style Props (NOT CSS Media Queries)
|
||||
|
||||
@@ -627,13 +627,13 @@ export const Bar = styled.div<{ width: number }>`
|
||||
|
||||
If existing code already uses `CS` classes and you're not migrating that specific code, leave them alone. But when migrating Emotion → Mantine, replace with the proper alternative:
|
||||
|
||||
| Instead of `CS.*` | Use |
|
||||
| ------------------- | ------------------------------------------------------------ |
|
||||
| `CS.overflowHidden` | CSS module: `overflow: hidden` |
|
||||
| `CS.cursorPointer` | CSS module: `cursor: pointer` |
|
||||
| `CS.flex1` | Style prop: `flex={1}` |
|
||||
| `CS.flexNoShrink` | Style prop: `flex="0 0 auto"` |
|
||||
| `CS.textBrandHover` | CSS module: `&:hover { color: var(--mb-color-core-brand); }` |
|
||||
| Instead of `CS.*` | Use |
|
||||
| ------------------- | ------------------------------------------------------- |
|
||||
| `CS.overflowHidden` | CSS module: `overflow: hidden` |
|
||||
| `CS.cursorPointer` | CSS module: `cursor: pointer` |
|
||||
| `CS.flex1` | Style prop: `flex={1}` |
|
||||
| `CS.flexNoShrink` | Style prop: `flex="0 0 auto"` |
|
||||
| `CS.textBrandHover` | CSS module: `&:hover { color: var(--mb-color-brand); }` |
|
||||
|
||||
### Pattern 10: Shared Emotion Styles Across Files → Shared CSS Module + cx()
|
||||
|
||||
@@ -759,7 +759,7 @@ import S from "./ComponentName.module.css"; // only if CSS module is needed
|
||||
- [ ] The `.styled.tsx` file is deleted
|
||||
- [ ] All imports of the deleted styled file are updated
|
||||
- [ ] No static inline styles — `style={{ }}` used only for truly dynamic runtime values
|
||||
- [ ] All colors use design tokens (`c="core-brand"`, `var(--mb-color-core-brand)`), not raw hex/rgb
|
||||
- [ ] All colors use design tokens (`c="brand"`, `var(--mb-color-brand)`), not raw hex/rgb
|
||||
- [ ] All spacing uses design tokens (`p="md"`, `var(--mantine-spacing-md)`), not arbitrary px values — hardcoded `rem`/`px` literals snapped to the nearest token (see mapping table above)
|
||||
- [ ] All border-radius values use radius tokens (`var(--mantine-radius-md)`), not hardcoded `0.5rem`
|
||||
- [ ] All font sizes use font-size tokens (`fz="md"`, `var(--mantine-font-size-md)`), not hardcoded `1rem`
|
||||
|
||||
@@ -1,248 +0,0 @@
|
||||
---
|
||||
name: fe-modularization
|
||||
description: Deciding where frontend code lives — tier model, move mechanics, extension points, side effects, naming, and the traps. Use when moving code between modules, carving new modules, fixing boundary violations, adding a barrel or an endpoint, or reviewing module-shape decisions.
|
||||
---
|
||||
|
||||
# Frontend modularization
|
||||
|
||||
The module is the unit of blast radius: dependency direction, test selection,
|
||||
bundle cones, and review scope all follow module boundaries. Every decision
|
||||
here serves one goal: **make the file locations tell the truth about who owns
|
||||
what and who may depend on whom.**
|
||||
|
||||
## Hard rules (check these before anything else)
|
||||
|
||||
- NEVER add an `export … from` re-export at an old path, and NEVER `export *`
|
||||
from a barrel. Move the code, codemod every call site, delete the old path,
|
||||
all in the same PR.
|
||||
- NEVER import past a module's `index.ts` when that module has
|
||||
`enforcePublicApi: true`, and never past `metabase/ui`'s index at all. If
|
||||
the barrel is heavy, the fix is making the module side-effect free (below),
|
||||
not a deep import.
|
||||
- NEVER rewrite a consumer to a lower-level idiom to make the linter pass. If
|
||||
the fix makes the consumer worse, the plan is wrong.
|
||||
- NEVER give a new module `enforceSharedTiers: false`; new modules ship
|
||||
enforced with `enforcePublicApi: true` and an explicit `index.ts`.
|
||||
- NEVER move code without the consumer-tier check (procedure step 2).
|
||||
- ALWAYS delete a module's `enforceSharedTiers: false` line in the PR that
|
||||
takes it to zero violations.
|
||||
- ALWAYS measure `bun run module-boundaries` before and after, and report the
|
||||
numbers.
|
||||
- ALWAYS import `dayjs` from `metabase/dayjs`, routing from `metabase/router`,
|
||||
Mantine from `metabase/ui`, react-redux hooks from `metabase/redux`; the
|
||||
raw specifiers are lint-forbidden.
|
||||
|
||||
## Files that are ground truth
|
||||
|
||||
| what | where |
|
||||
|---|---|
|
||||
| module elements, tiers, `enforceSharedTiers`, `enforcePublicApi` | `frontend/lint/module-boundaries.mjs` |
|
||||
| shared sub-tiers and levels, cluster rules | `frontend/lint/shared-tiers.mjs` |
|
||||
| standalone boundaries lint (all violations, incl. grandfathered) | `bun run module-boundaries` (config `eslint.config.module-boundaries.mjs`) |
|
||||
| PR lint (only enforced modules fail CI) | `bun run lint-eslint-pure` |
|
||||
| side-effect-free directories and their exceptions | `frontend/build/shared/rspack/side-effect-free-modules.js` (`SIDE_EFFECT_FREE_PATHS`, `SIDE_EFFECT_PATHS`) |
|
||||
| the lint rules that enforce that promise | `metabase/no-module-side-effects`, `metabase/no-base-api-access` in `frontend/lint/eslint-plugin-metabase/rules/` |
|
||||
| public-api enforcement | `metabase/enforce-module-public-api` rule, driven by `getPublicApiModules()` |
|
||||
|
||||
## The model
|
||||
|
||||
Tiers: `lib < basic < shared < feature < app`. Imports point downward, never
|
||||
sideways at feature tier (features may not import features; app and EE may
|
||||
import anything). The shared tier is subdivided in `shared-tiers.mjs` into
|
||||
shared-utils (U0…), shared-platform (P0…), and shared-domain, each ordered
|
||||
into levels. A module imports only *strictly lower* levels of its sub-tier
|
||||
plus the sub-tiers below; same-level peers are deliberately forbidden, which
|
||||
is what makes cycles structurally impossible. A domain may hold two seats
|
||||
when consumers demand it: a low core and a high surface (metabase-lib below
|
||||
questions below query_builder; viz-core below visualizations).
|
||||
|
||||
Enforcement is per module: `enforceSharedTiers: false` on an element exempts
|
||||
it from the level rules (counted by `bun run module-boundaries`, invisible to
|
||||
PR lint). Modules with `enforcePublicApi: true` may only be imported from
|
||||
outside via `metabase/<module>`, and import relatively inside.
|
||||
|
||||
## Deciding where code lives (apply in order; first decisive test wins)
|
||||
|
||||
1. **Who owns the concept?** Code lives with the module that owns its
|
||||
concept, not the module that renders or calls it. A mode encoding dashboard
|
||||
click behaviour is dashboard code even if viz executes it. Metabot
|
||||
conversation state is metabot state even though the store registers it.
|
||||
2. **Consumer-tier check.** The destination must sit at or below the file's
|
||||
*lowest* consumer. A single consumer below the proposed home vetoes the
|
||||
move; report it, don't force it.
|
||||
3. **Only four legitimate fixes** for a bad edge: move the code to its
|
||||
terminal home; invert through a designed extension point (a prop, an
|
||||
injected component, a contract type owned by the socket); fix the tier
|
||||
placement when the declared level is wrong; delete a thin wrapper (a rename
|
||||
with no derivation isn't an abstraction — deleting it *is* the migration).
|
||||
4. **A bridge lives above the lower endpoint.** Code that needs both sides of
|
||||
a boundary belongs in or above the higher module.
|
||||
5. **"Genuinely shared" is proven by consumers.** A shared address needs two
|
||||
or more consumers the tier rules can't serve another way. One consumer
|
||||
pretending to be shared is a feature module at the wrong address.
|
||||
|
||||
## Extension points
|
||||
|
||||
The socket's owner defines the contract; every plug lives with its extender.
|
||||
Visualization owns the `mode` prop and click interfaces, each surface supplies
|
||||
its mode; the editor owns its extension contract, metabot supplies tiptap
|
||||
extensions; querying owns `TemplateTagsSidebarProps`, parameters implements
|
||||
it.
|
||||
|
||||
- Prefer plain props/injection at composition sites. `PLUGIN_*` registries are
|
||||
for edition gating; single-reader slots move out to their reading module;
|
||||
slots with many readers live in the module that owns the concept (every
|
||||
slot the whitelabel plugin fills lives in `whitelabel`), tier permitting.
|
||||
Keep contracts type-light so implementations can load lazily.
|
||||
- Identical injection at every callsite is acceptable until a fourth consumer
|
||||
or second slot appears; then the composition deserves its own module above
|
||||
both parts. Don't pre-build the wrapper.
|
||||
|
||||
## Store shape
|
||||
|
||||
A module that has redux state owns it:
|
||||
|
||||
- `store/` is private: reducer, plain creators/action types, and selectors,
|
||||
typed against the module's own state (`QueryBuilderStoreState`), never the
|
||||
global `State`. Thunks live in `actions/` and import `store/`, never the
|
||||
reverse.
|
||||
- The barrel is the state API: selectors for reads, creators/thunks for
|
||||
writes.
|
||||
- Store roots (`reducers-main`, `reducers-public`, the SDK store) import each
|
||||
slice from its module barrel and derive their own state type; no slice
|
||||
registry, no `declare module` merging.
|
||||
- Cross-module writes: a callback prop when a composition point exists;
|
||||
otherwise dispatch the owner's exported creator; an event only when the
|
||||
emitter must not know its listeners. Raw `dispatch({ type: "…" })` strings
|
||||
are banned. A reducer or listener matching another module's *internal*
|
||||
action is the coupling to remove.
|
||||
- Slices that mirror one endpoint get deleted (the RTK cache is the state); a
|
||||
fold over many sources with one writer (the metadata store) is kept as a
|
||||
module-private slice behind the barrel.
|
||||
|
||||
## Endpoints
|
||||
|
||||
Endpoints migrate to their owning module via `Api.injectEndpoints` from the
|
||||
module's `api/` file (settings, metabot, transforms precedents); `metabase/api`
|
||||
trends toward the client only. There is one `Api` object per backend because
|
||||
tag invalidation only works within one instance. Consumers reach endpoints
|
||||
only through the owner's exports (`useGetCardQuery`, `cardApi.endpoints.…`),
|
||||
never by name through the base object; `metabase/no-base-api-access` allows
|
||||
injection only in `metabase/api` and `**/api/**` / `**/api.ts`, and reaching by
|
||||
name only in `metabase/api` and test support (`invalidateTags` /
|
||||
`resetApiState` are fine anywhere). Each endpoint move carries its
|
||||
metadata-store hookup: the owner calls `entitiesFetched` from its own
|
||||
`onQueryStarted`.
|
||||
|
||||
## Import-time side effects
|
||||
|
||||
Directories in `SIDE_EFFECT_FREE_PATHS` are marked `sideEffects: false` for
|
||||
rspack, so importing one export from their barrel no longer drags the whole
|
||||
barrel into a bundle. The price: nothing in those directories may do work at
|
||||
import time, or production silently drops it. `metabase/no-module-side-effects`
|
||||
enforces this over every listed directory (module-scope calls, writes to
|
||||
imported objects or globals, bare imports, control flow at import; escape
|
||||
hatches are `/* #__PURE__ */` and the rule's pure-callee list). Files that
|
||||
must run at import are listed in `SIDE_EFFECT_PATHS` (files or directories).
|
||||
|
||||
- **A library the app configures gets a facade module** owning the vendor
|
||||
import, the registrations, and the types, with the raw specifier
|
||||
lint-forbidden outside it: `metabase/router`, `metabase/ui`,
|
||||
`metabase/redux`, `metabase/dayjs`. Facades are the effect, so they never
|
||||
enter `SIDE_EFFECT_FREE_PATHS`. Never re-register a vendor plugin locally
|
||||
(`dayjs.extend`, `echarts` `use`).
|
||||
- **A patch a component needs is an explicit registration module,
|
||||
bare-imported by the provider that needs it** and listed in
|
||||
`SIDE_EFFECT_PATHS` (Mantine's `Popover.Dropdown` replacement, imported by
|
||||
`ThemeProvider`).
|
||||
- **Effects with no value to import** (global CSS, polyfills, the CSP nonce,
|
||||
EE plugin bootstrap) live in the entry's side-effects file and nowhere else.
|
||||
- **Registries are filled from the composition root at boot**
|
||||
(`initializePlugins`, `registerVisualizations`), never by a module
|
||||
registering itself at import.
|
||||
- **RTK injection stays at import**; when a module with an `api/` folder is
|
||||
declared side-effect free, list `<module>/api/` in `SIDE_EFFECT_PATHS`.
|
||||
|
||||
## Naming
|
||||
|
||||
The feature keeps the plain product-surface name; a shared carve is named for
|
||||
what the shared part actually is: the concept when the concept is the shared
|
||||
thing (`settings`, `current-user`, `whitelabel`, `dayjs`), or a qualified
|
||||
capability when the plain noun is taken (`search-ui`, `metrics-ui`,
|
||||
`viz-core`, `data-studio-ui`). Never two modules distinguished only by tier
|
||||
prefix; never generic `-shared` / `-feature` suffixes.
|
||||
|
||||
## Procedure for a move
|
||||
|
||||
1. **Enumerate consumers honestly** (hook-name greps miss the last three):
|
||||
```
|
||||
grep -rn "metabase/<module>" frontend/src enterprise/frontend/src frontend/test e2e
|
||||
grep -rn "jest.mock(\"metabase/<module>" frontend enterprise/frontend
|
||||
grep -rn "addMatcher(.*<endpoint>" frontend/src enterprise/frontend/src
|
||||
```
|
||||
plus sibling files importing the moved file relatively. Classify each by
|
||||
module and tier from `module-boundaries.mjs` / `shared-tiers.mjs`.
|
||||
2. **Consumer-tier check**: destination at or below the lowest consumer. If
|
||||
not, stop and report the blocking consumer.
|
||||
3. **Measure before**: `bun run module-boundaries 2>&1 | tail -1` on the base
|
||||
(write the number down).
|
||||
4. **Move**: `git mv`; codemod every importer to `metabase/<module>` (outside)
|
||||
or a relative path (inside); add named exports to the destination barrel
|
||||
for every symbol an outside consumer needs (each must have a real
|
||||
consumer). Delete the old path. If the module didn't exist: add its element
|
||||
in `module-boundaries.mjs` (ordered before any element it's carved from,
|
||||
first match wins), its level in `shared-tiers.mjs`, `enforcePublicApi:
|
||||
true`, an `index.ts`. If the move empties a module's violations, delete
|
||||
its `enforceSharedTiers: false` line. If the module has an `api/` folder
|
||||
and is side-effect free, list `<module>/api/` in `SIDE_EFFECT_PATHS`.
|
||||
5. **Verify** (all of these, report the numbers):
|
||||
```
|
||||
grep -rn "<old path>" frontend enterprise/frontend e2e # must be empty
|
||||
bunx eslint --no-warn-ignored <touched files>
|
||||
bun run lint-eslint-pure # enforce-module-public-api included
|
||||
bun run type-check-pure
|
||||
bun run test-unit-keep-cljs <touched folders' specs>
|
||||
bun run module-boundaries 2>&1 | tail -1 # after; no violation may name the new files
|
||||
bun run build-release:static-viz # only if viz/static-viz/ui/dayjs touched; hard 3.5 MiB budget
|
||||
```
|
||||
6. **Report**: count before/after (state count-neutral explicitly), the edges
|
||||
removed by file, flags/rules deleted, barrel exports added, anything you
|
||||
skipped and why, and any consumer you found that vetoed part of the move.
|
||||
|
||||
## Traps (each has bitten at least once)
|
||||
|
||||
- **Late binding.** A bare function reference in a `createSelector` input
|
||||
array, a component identity captured for comparison, a `PLUGIN_*` object
|
||||
read at module scope: all capture at import time and dodge jest spies, EE
|
||||
overrides, and lazy loading. Read at call time (`(_state) => fn()`), or
|
||||
inject.
|
||||
- **Plans go stale.** Verify every recipe against current code before
|
||||
executing; things get merged, renamed, or turn out never to have existed.
|
||||
Skip-and-report beats force.
|
||||
- **The linter's blind spots**: `export … from` re-exports, bare-specifier
|
||||
packages (custom-viz), pre-seeded caches that never register tags. A grep
|
||||
for the old path is the stronger claim than a green lint run.
|
||||
- **Path-keyed baselines reset on rename**: a generated per-file ledger
|
||||
silently loses a file's history when it moves.
|
||||
- **A heavy barrel is a bundle hazard until its module is side-effect free.**
|
||||
One constant imported from the `metabase/ui` barrel pulled Mantine into the
|
||||
static-viz bundle and broke its budget; the fix was `SIDE_EFFECT_FREE_PATHS`,
|
||||
never a deep import.
|
||||
- **Type augmentation is program-wide.** `declare module "dayjs"` (and
|
||||
similar) makes plugin types available to any file in the same tsconfig, so
|
||||
a passing type-check never proves the setup ran; only importing the facade
|
||||
does.
|
||||
- **Environment before diagnosis**: after a dependency-touching merge,
|
||||
`bun install` before declaring a failure real; worktrees share staleness
|
||||
through the node_modules symlink, and a stale `target/cljs_dev` makes
|
||||
`type-check-pure` fail on `metabase-lib/metric/core.ts` (rebuild cljs in the
|
||||
worktree).
|
||||
|
||||
## PR shape
|
||||
|
||||
One logical change per PR; the body names the specific edges removed (files,
|
||||
imports, before/after count) and gives each move's ownership reason in plain
|
||||
language a reviewer outside the effort can follow, without campaign labels.
|
||||
Stack only for semantic dependency or same-hot-file serialisation, and say
|
||||
so. Consider codeowner surface when chunking. Open as a draft until measured
|
||||
and green.
|
||||
@@ -1,244 +0,0 @@
|
||||
---
|
||||
name: metabase-ui-component-from-figma
|
||||
description: Update or build a Metabase design-system component (frontend/src/metabase/ui — Chip, Badge, Alert, Switch, etc.) to match a Figma spec described in a Linear issue. Use when a ticket asks to restyle an existing `metabase/ui` component or implement a new one from Figma. Covers: checking out the issue, mapping the component (usage/blast radius for an existing one, or scaffolding a new one), building a Storybook showcase matrix (before styling for existing components, after for new ones), styling from exact Figma tokens while iterating with the user, and committing + migrating call sites LAST.
|
||||
---
|
||||
|
||||
# Metabase UI component from Figma
|
||||
|
||||
End-to-end workflow for bringing a `metabase/ui` (Mantine) component in line with a Figma design, driven by a Linear ticket.
|
||||
|
||||
The golden rule (existing components): **build the Storybook showcase before changing any component styles.** Build it while the old styles are still in place — so the team can see the current component at a glance and get accustomed to it before the migration, then watch it light up as you restyle. You end up with a visual-regression (Loki) candidate either way. A **new** component has nothing to render until it's built, so this rule doesn't apply to create mode — there the order flips: implement and style the component first, then build its showcase to present the result.
|
||||
|
||||
This is a frontend task — also load **typescript-write** (and **typescript-review** before handing off).
|
||||
|
||||
## Two modes
|
||||
|
||||
The ticket is one of two shapes; identify which up front, because Phase 2 differs:
|
||||
|
||||
- **Update an existing component** — the component already exists with call sites, either in `metabase/ui` *or* as a legacy component elsewhere (e.g. `frontend/src/metabase/common/components/`, possibly Emotion styled-components) that this ticket also migrates into `metabase/ui`. The risk is *regression* (call sites, behavior), so you map usage and lock behavior with tests before touching it.
|
||||
- **Create a new component** — nothing equivalent exists anywhere yet. Rarer. The risk is *API design*, so you scaffold the component and its public surface first.
|
||||
|
||||
Phases 1, 2, 5 and 6 are shared. The middle two differ in **order**: update mode builds the showcase (Phase 3) *before* styling (Phase 4); create mode does the reverse — implement and style first, then build the showcase as the last step — because there's nothing to render until the component exists.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Check out the issue
|
||||
|
||||
1. Fetch the Linear issue with the Linear MCP `get_issue`.
|
||||
2. Pull from it:
|
||||
- **Figma link(s)** — extract the `node-id` (`?node-id=250-13588` → `250-13588`) and `fileKey` (the `/design/<fileKey>/…` segment).
|
||||
- **`gitBranchName`** — switch to it if not already there.
|
||||
- **The component's intended API** — variants, sizes, and states the design system supports for this component. This defines the axes of your showcase (Phase 3).
|
||||
3. Decide the mode (see **Two modes** above): search for an existing component — in `metabase/ui` or as a legacy component elsewhere. Found → **update**; nothing equivalent anywhere → **create**. Confirm with the user when unsure.
|
||||
4. **Verify a Figma desktop MCP is connected before doing the work** — this skill depends on it for exact tokens in Phase 4 (and styling can't proceed accurately without it). Do a quick read against the issue's node (e.g. `get_metadata`/`get_variable_defs`); if it errors with "nothing selected" or the server isn't connected, prompt the user to enable it (Switch to Dev Mode → MCP section in the right sidebar) and select the relevant layer before continuing.
|
||||
|
||||
## Phase 2 — Understand the component
|
||||
|
||||
### Mode: update an existing component
|
||||
|
||||
1. Locate the component dir (typically containing `Component.config.ts` or `Component.tsx`, `Component.module.css`, `Component.stories.tsx`, `index.ts`). A ticket may also be **migrating a legacy component** — if it still uses Emotion styled-components, reach for the **emotion-migrate** skill.
|
||||
2. Map every usage across `frontend/src` and `e2e` (JSX `<Component`, sub-components like `Component.Group`, and wrapper components).
|
||||
3. Write findings to **`local/<TICKET>-<component>-usage-findings.md`**, one entry per usage: code location (`file:line`), short context, and **non-standard usage** (excessive customization, inline styling, invalid/legacy props). Invalid props are a signal that call sites will need migrating in Phase 6. This doc **stays** — the user walks it at the end to verify each real usage still looks right.
|
||||
4. State the blast radius plainly: how many call sites, where, and whether they're confined.
|
||||
5. **Lock behavior with tests before changing anything — when warranted.** Only for components that have **custom interactive behavior** (their own event handlers, state, keyboard logic, controlled/uncontrolled wiring). Add unit tests covering that current API/behavior so a regression shows up as a failing test. **Skip tests for thin Mantine wrappers** — don't test built-in Mantine behavior, and a pure visual restyle is covered by the showcase/Loki, not unit tests.
|
||||
|
||||
### Mode: create a new component
|
||||
|
||||
1. Decide whether it wraps a Mantine primitive (most common — extend it via a `.config.ts` using `<Mantine>.extend({...})` and theme registration) or is bespoke. Prefer wrapping Mantine.
|
||||
2. Scaffold the component dir + its public surface: `Component.tsx`/`Component.config.ts`, `Component.module.css`, `Component.stories.tsx`, `index.ts`, and wire the export through the group barrel and the `metabase/ui` barrel.
|
||||
3. There are no call sites to map yet; instead note the intended usage from the ticket so the eventual showcase reflects real props.
|
||||
4. **Order note:** for a new component, implement and style it (Phase 4) first, *then* build the showcase (Phase 3) to present the finished result — there's nothing to render until it exists. The phases below are written update-first; just run 4 before 3 in create mode.
|
||||
|
||||
## Phase 3 — Build the Storybook showcase
|
||||
|
||||
Goal: one story that shows **every state at a glance**, themeable, suitable as a Loki visual-regression test. **Update mode: build this before Phase 4, while the old styles are still in place. Create mode: do this after Phase 4**, once the component exists to render.
|
||||
|
||||
### Reusable showcase helpers
|
||||
|
||||
Use the shared, Storybook-only primitives in **`frontend/src/metabase/ui/stories/showcase/`**: `StoryShowcase` (titled panel), `StorySection`, `StoryRow`, and `StoryJsx` (monospace JSX with light syntax highlighting via `tokenizeJsx`). Extend the set only when a new pattern is genuinely reusable.
|
||||
|
||||
### Derive the matrix from the component's full state space
|
||||
|
||||
Enumerate **all** the states the component can be in — every supported variant, size, and interaction/selection state (from its props/types and the design-system API, not merely the subset the issue happens to mention) — then cross-reference with the Figma spec and build the matrix on that. Don't omit a supported state because the ticket didn't list it; don't invent an axis Figma doesn't show (e.g. a static component has no hover/pressed).
|
||||
|
||||
**Exclude theme from the matrix.** Figma usually models light/dark as a component *property* (because it doesn't use variable modes), but theme is **not** a real component prop in the app — it's global. So never make theme a matrix axis or column; drive light/dark with the global Storybook `theme` toggle instead (see below).
|
||||
|
||||
Typical layout (mirrors the Figma component sheet): a `StoryShowcase` title, then a single **CSS-grid matrix** — state labels down the left column, one column per axis-combination (e.g. `variant × size`), each column headed by the real JSX usage via `StoryJsx`. Use `gridTemplateColumns: \`<labelWidth> repeat(N, max-content)\`` with a `React.Fragment` per row.
|
||||
|
||||
### Theme: use the global toggle, render ONE copy
|
||||
|
||||
Render the showcase once and let Storybook's global `theme` control (defined in `.storybook/preview.tsx`) switch light/dark. Do **not** render light and dark side by side: Metabase bakes per-scheme hex into `--mantine-color-*` at `:root`, and Mantine color tokens follow the global scheme — a nested scheme wrapper won't re-scope them.
|
||||
|
||||
### Forcing hover/pressed for the matrix
|
||||
|
||||
Use **`storybook-addon-pseudo-states`** (registered in `.storybook/main.ts`); don't add `data-force-*` hooks to component CSS. Set `parameters.pseudo` to force states per cell. Two things make or break it:
|
||||
|
||||
1. **Aim the selector at the element the pseudo-class actually sits on**, which for a Mantine component is often an inner slot, not the root. A string/array value applies `.pseudo-<state>` to the matched element, so it must match that slot. Import the CSS-module class (`S.<Slot>`) to target it.
|
||||
2. **Land your per-cell hook on the root via the factory's root-props slot** (e.g. `wrapperProps`), because Mantine forwards top-level `data-*`/rest props to an inner element, not the root. Check the component's `.d.ts` for the right slot.
|
||||
|
||||
The addon toolbar toggle keeps working independently (it forces every rule in the story).
|
||||
|
||||
### Example: a finalized matrix story
|
||||
|
||||
Illustrative — adapt the axes and states to the component you're working on. This is the shape the pieces above add up to (state labels on the left, one column per `variant × size`, forced hover/pressed via `parameters.pseudo`, the root hook on `wrapperProps`, controls scoped to the meaningful knobs):
|
||||
|
||||
```tsx
|
||||
import type { StoryFn } from "@storybook/react";
|
||||
import { Fragment } from "react";
|
||||
|
||||
import { Box, Chip, type ChipProps, Text } from "metabase/ui";
|
||||
import { StoryJsx, StoryShowcase } from "metabase/ui/stories/showcase";
|
||||
|
||||
import S from "./Chip.module.css";
|
||||
|
||||
// Axes derived from the component's API (here: variant × size).
|
||||
const COLUMNS = [
|
||||
{ variant: "light", size: "sm" },
|
||||
{ variant: "light", size: "md" },
|
||||
{ variant: "filled", size: "sm" },
|
||||
{ variant: "filled", size: "md" },
|
||||
] as const;
|
||||
|
||||
// Every state the component supports; `id` doubles as the pseudo-state hook.
|
||||
const STATES = [
|
||||
{ id: "default", label: "Default" },
|
||||
{ id: "hover", label: "Hover" },
|
||||
{ id: "pressed", label: "Pressed" },
|
||||
{ id: "selected", label: "Selected", checked: true },
|
||||
{ id: "hover-selected", label: "Hover selected", checked: true },
|
||||
{ id: "disabled", label: "Disabled", disabled: true },
|
||||
];
|
||||
|
||||
// Force hover/pressed by targeting the slot the pseudo-class sits on (.ChipLabel).
|
||||
const labelSelector = (id: string) => `[data-state-row="${id}"] .${S.ChipLabel}`;
|
||||
|
||||
const Overview: StoryFn<ChipProps> = ({ children }) => (
|
||||
<StoryShowcase title="Chip">
|
||||
<Box
|
||||
style={{
|
||||
display: "grid",
|
||||
gridTemplateColumns: `9rem repeat(${COLUMNS.length}, max-content)`,
|
||||
columnGap: "2rem",
|
||||
rowGap: "1rem",
|
||||
alignItems: "center",
|
||||
}}
|
||||
>
|
||||
<div />
|
||||
{COLUMNS.map(({ variant, size }) => (
|
||||
<StoryJsx
|
||||
key={`${variant}-${size}`}
|
||||
>{`<Chip variant="${variant}" size="${size}" />`}</StoryJsx>
|
||||
))}
|
||||
{STATES.map((state) => (
|
||||
<Fragment key={state.id}>
|
||||
<Text size="sm" c="text-secondary">
|
||||
{state.label}
|
||||
</Text>
|
||||
{COLUMNS.map(({ variant, size }) => (
|
||||
<Chip
|
||||
key={`${variant}-${size}`}
|
||||
wrapperProps={{ "data-state-row": state.id }} // hook on the ROOT
|
||||
variant={variant}
|
||||
size={size}
|
||||
checked={state.checked ?? false}
|
||||
disabled={state.disabled ?? false}
|
||||
onChange={() => {}}
|
||||
>
|
||||
{children}
|
||||
</Chip>
|
||||
))}
|
||||
</Fragment>
|
||||
))}
|
||||
</Box>
|
||||
</StoryShowcase>
|
||||
);
|
||||
|
||||
export const OverviewStory = {
|
||||
render: Overview,
|
||||
parameters: {
|
||||
pseudo: {
|
||||
hover: ["hover", "hover-selected"].map(labelSelector),
|
||||
active: ["pressed"].map(labelSelector),
|
||||
},
|
||||
controls: { include: ["children", "theme"] },
|
||||
},
|
||||
};
|
||||
```
|
||||
|
||||
### Story conventions
|
||||
|
||||
- Keep an interactive default story plus the matrix story.
|
||||
- Restrict `argTypes` to the supported API (e.g. only the sizes the DS uses).
|
||||
- **Scope controls per story.** A static matrix should hide knobs it ignores: `parameters.controls = { include: [...] }` (or `{ disable: true }`). Wire any kept knob (e.g. `children`) through to the cells so it does something.
|
||||
- `loki.config.js` has a `storiesFilter`; check it before renaming a story that's already covered.
|
||||
|
||||
Validate: `bun run lint-eslint-pure -- <files>` and `bun run type-check-pure`.
|
||||
|
||||
### Optional: verify in the browser yourself
|
||||
|
||||
If a browser-driving MCP is connected (e.g. **chrome-devtools-mcp** or **playwright-mcp**: `navigate_page`, `take_screenshot`, `take_snapshot`, `evaluate_script`, `resize_page`), use it to check your own work against the running Storybook (`http://localhost:6006/iframe.html?id=<story-id>`, the kebab-cased title + story). Screenshot for a visual read; `evaluate_script`/`take_snapshot` to confirm DOM facts (a hook landed on the root, pseudo classes applied, a resolved `--mb-color-*`). This is a **convenience, never a requirement** — without it, rely on the user's visual check (and you may suggest installing the plugin). Storybook is usually already running; don't start it. (Screenshot writes may be sandboxed to the repo root — save under `local/` then `mv`.)
|
||||
|
||||
Pause here and let the user look. Iterate on layout/spacing only — **no component style changes yet.**
|
||||
|
||||
## Phase 4 — Style the component from Figma
|
||||
|
||||
Now change the component's `.config.ts` / `.module.css`. Get **exact tokens from Figma** — never eyeball hex from a screenshot.
|
||||
|
||||
### Figma desktop MCP is required for tokens
|
||||
|
||||
`get_screenshot` works by `nodeId`, but `get_variable_defs`, `get_design_context`, and `get_metadata` read the **current selection in the Figma desktop app** via the **Dev Mode MCP server**. If they error with "nothing selected" or the server isn't connected, **prompt the user** to enable the Dev Mode MCP server (Switch to Dev Mode → MCP section in right sidebar) and select the relevant layer(s). Don't pixel-sample a screenshot as a substitute.
|
||||
|
||||
### Extract and map tokens
|
||||
|
||||
1. `get_metadata` on the spec frame for the node id + name of each variant/size/state symbol.
|
||||
2. `get_variable_defs` per symbol for exact colors, spacing, radius, font.
|
||||
3. `get_design_context` (forceCode) on one symbol for exact CSS dimensions (padding, gap, icon size). It may ask about Code Connect — skip that.
|
||||
4. **Use the semantic token names `get_variable_defs` returns.** An up-to-date spec binds each property to a semantic token (a role like `text/hover`, `background/selected`); match it to the corresponding `--mb-color-*` key (key list: `frontend/src/metabase/ui/colors/types/color-keys.ts`) and use it directly — one definition then works in both themes. Sanity-check the resolved shade against `colors/constants/themes/light.ts` / `dark.ts` and `base-colors.ts`. Caveat: returned *values* may be the wrong mode for dark symbols — trust the variable **names**, not the dark hex.
|
||||
|
||||
### Color rules — semantic tokens only, NO `color-mix`, NO primitives
|
||||
|
||||
Per `docs/developers-guide/frontend.md` ("Colors"): every color must be a **semantic** `--mb-color-*` token. No literal hex, and no `color-mix` in component CSS to fake a shade/alpha.
|
||||
|
||||
**Guardrail:** if `get_variable_defs` returns a **primitive ramp** (e.g. `Orion-Alpha/10`, `Ocean/60` — a ramp + number, not a role) bound directly to a property, do **not** translate it to a ramp, hardcode hex, or `color-mix` your way to it. **Stop and flag it to the user** with the specific property/state — it's a design-side gap (bind to a semantic token, or add one). Resume once the spec exposes a semantic token.
|
||||
|
||||
### Non-color tokens — flag scale mismatches too
|
||||
|
||||
The same discipline applies to **dimensional** values — radius, spacing/padding/gap, elevation/shadow, and typography (font size, line-height, weight). Prefer the codebase's existing scale variable over a literal: `--mantine-radius-*`, `--mantine-spacing-*`, `--mantine-shadow-*`, and the Mantine font tokens.
|
||||
|
||||
But Figma's scales don't always line up with the codebase's. When a Figma value **has no matching step** in the corresponding scale — e.g. Figma `radius/md = 12px` while `--mantine-radius-md` is `8px`, or a Figma `xxxs` spacing step with no Mantine equivalent — **don't silently bake in a literal.** Flag it to the user the same way you flag primitive colors: name the property, the Figma value, and the nearest codebase token, so it can be resolved with design / the scale owners (align the value, or add a scale step). Only fall back to a literal if the user confirms it's an accepted one-off, and leave a short comment in the code explaining why.
|
||||
|
||||
### Mantine implementation notes (reference)
|
||||
|
||||
General gotchas when extending Mantine components — adapt per component:
|
||||
|
||||
- **Know which slot each data-attribute lands on.** Mantine puts state attributes on specific slots (e.g. `data-variant` on the root, `data-checked`/`data-disabled` on an inner slot). Scope rules accordingly, usually through a root `classNames.root` class.
|
||||
- **Out-specify Mantine's built-in variant CSS.** It styles slots at `.m_*:not([data-disabled])` specificity; a descendant combinator through your root class wins. Avoid chained `:not()` (stylelint's `selector-not-notation: complex` rejects it, and combining lowers specificity) — gain specificity via the root class / attributes instead.
|
||||
- **Size via a `vars` resolver**, returning only Mantine's typed vars for that component (custom var names fail type-check). Map size → vars in `.config.ts`.
|
||||
- **Built-in icons use `currentColor`**, so they inherit the slot's text color.
|
||||
- For text on a brand fill, follow the existing convention (`text-primary-inverse`) unless the user wants pure white; flag the dark-theme implication.
|
||||
|
||||
After edits: `npx stylelint <css>`, `LINT_CSS_MODULES=true bun run lint-eslint-pure -- <files>`, `bun run type-check-pure`.
|
||||
|
||||
## Phase 5 — Iterate with the user
|
||||
|
||||
Show the result in the running Storybook and address feedback (color, spacing, radius, shadow, missing/extra state). Surface deliberate deviations explicitly so the user can veto them. If a browser MCP is connected, screenshot the matrix in both themes to catch issues first. **Do not commit until the user is satisfied.**
|
||||
|
||||
## Phase 6 — Commit, then migrate call sites
|
||||
|
||||
Only after sign-off:
|
||||
|
||||
1. Commit when the user explicitly asks; commit only relevant files (exclude build artifacts and the gitignored `local/` doc). Follow the user's commit conventions.
|
||||
2. **Update mode:** migrate the call sites flagged in Phase 2 if the restyle requires it (e.g. invalid/renamed props). Be surgical with bulk find/replace — a blanket prop rename can hit unrelated components; verify each hit is the target. **Create mode:** confirm the public export is wired through the barrels and used as the ticket intends.
|
||||
3. Leave the findings doc in place for the user's manual verification pass.
|
||||
|
||||
---
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] Linear issue fetched; Figma node/fileKey + `gitBranchName` extracted; on the right branch; mode (update vs create) decided.
|
||||
- [ ] **Update:** usage + blast radius in `local/<TICKET>-<component>-usage-findings.md`; behavior locked with unit tests **if** the component has custom interactive logic (skip for thin Mantine wrappers / pure restyles). **Create:** component scaffolded with its public surface and barrel exports.
|
||||
- [ ] Storybook showcase matrix built from the component's full state space (theme excluded — global toggle drives light/dark), single panel, hover/pressed forced via `storybook-addon-pseudo-states`, controls scoped per story; lint + type-check pass. (Update mode: built before styling, **component styles untouched**. Create mode: built after the component is implemented in Phase 4.)
|
||||
- [ ] Figma desktop MCP available; exact tokens extracted and mapped to semantic `--mb-color-*` (NO `color-mix`, NO primitives) and to existing scale vars for radius/spacing/elevation/type — any color *or* dimensional value with no matching token/scale step flagged to the user, not baked in as a literal.
|
||||
- [ ] Component styled; stylelint + eslint(+CSS modules) + type-check pass.
|
||||
- [ ] Iterated with the user until satisfied.
|
||||
- [ ] Committed (when asked); call sites migrated/verified (update) or export wired (create); findings doc handed back.
|
||||
@@ -1,34 +1,20 @@
|
||||
---
|
||||
name: typescript-review
|
||||
description: Review TypeScript and JavaScript code changes for compliance with Metabase coding standards, style violations, and code quality issues. Use when reviewing pull requests or diffs containing TypeScript/JavaScript code.
|
||||
allowed-tools: Read, Grep, Bash, Glob, Skill
|
||||
allowed-tools: Read, Grep, Bash, Glob
|
||||
---
|
||||
|
||||
# TypeScript/JavaScript Code Review Skill
|
||||
|
||||
@./../_shared/typescript-commands.md
|
||||
@./../_shared/react-redux-patterns.md
|
||||
|
||||
## Main Focus
|
||||
## Code Review Guidelines
|
||||
|
||||
**Primary standard: the [`typescript-write`](../typescript-write/SKILL.md) skill.** Load it first — it defines the authoring rules this review enforces, alongside `frontend/CLAUDE.md` and `docs/developers-guide/frontend.md`.
|
||||
Review pull requests with a focus on:
|
||||
|
||||
Adherence to `typescript-write` is the **highest-priority** review dimension: rank any violation of its provisions above all other findings. Treat its **no-`any` hard rule** (no explicit *or* implicit `any` in new code) as **blocking**. Use TypeScript LSP tools to inspect inferred types when available; otherwise rely on type-checking and linting.
|
||||
|
||||
Review in this priority order:
|
||||
|
||||
1. **Violations of [`typescript-write`](../typescript-write/SKILL.md) provisions** — no-`any`, type tightening, type modeling, function signatures, null/undefined handling, naming, structure, comments. Highest priority; block on the no-`any` rule. Apply conditional guidance in context: explain the unsupported type guarantee or concrete readability problem, rather than treating every preference as a blanket ban.
|
||||
2. Compliance with `frontend/CLAUDE.md`.
|
||||
3. Readability and maintainability.
|
||||
4. Appropriate test coverage. For internal typed callers, avoid requesting tests solely for inputs the type system excludes. External API data, deserialised values, storage and JavaScript callers can violate annotations: test runtime validation and nontrivial assumptions at those boundaries. Types do not replace behavioural, security or data-integrity tests.
|
||||
|
||||
## Blind spots — act as the missing reviewer
|
||||
|
||||
These rarely surface in team reviews, so this skill should raise them. They are **additive** — raise them, but rank them below `typescript-write` violations:
|
||||
|
||||
- **Accessibility.** Interactive elements need keyboard support, focus management, and accessible names. Flag missing `aria-label`/`aria-labelledby`, non-semantic click targets, modals without focus trap, icon-only buttons without labels, and form inputs without a linked label.
|
||||
- **Performance.** Flag areas that scale poorly and aren't memoized; inline object/array literals passed to memoized children; effects that fire on every batch of a progressive load; and new dependencies added to hot paths.
|
||||
- **Security.** Evaluate potential security issues in new code.
|
||||
- **Bundle size.** Flag new large dependencies, default imports from icon or util libs, and heavy modules imported at route-load time.
|
||||
- **Analytics.** User-facing flows should emit tracking events. If a PR adds a new flow (button, modal, navigation) without a tracking event, ask whether one is expected.
|
||||
- **Public API surface** (embedding SDK). Consumers should be able to use public signatures and name types they need to import. Export those types deliberately and document public behaviour, including `@deprecated` for deprecated APIs; a referenced structural type does not automatically need its own named export.
|
||||
- Readability and maintainability
|
||||
- Appropriate test coverage
|
||||
- Compliance with project coding standards and conventions
|
||||
- Code quality and best practices
|
||||
- Type safety and proper TypeScript usage
|
||||
- React best practices (when applicable)
|
||||
|
||||
@@ -7,91 +7,3 @@ description: Write TypeScript and JavaScript code following Metabase coding stan
|
||||
|
||||
@./../_shared/development-workflow.md
|
||||
@./../_shared/typescript-commands.md
|
||||
@./../_shared/react-redux-patterns.md
|
||||
|
||||
## No `any` — hard rule
|
||||
|
||||
- **New code must not introduce `any`, explicit or implicit.** No `any` annotations, no `as any` / `as unknown as`, no untyped parameters or returns that infer `any`, no implicitly-`any` destructures or array/object literals.
|
||||
- **Untyped third-party / boundary values** must be typed at the boundary (a declared type, `unknown` + type guard, or a small typed wrapper) — never let `any` propagate inward.
|
||||
- **Mandatory type verification.** Before finishing a TS/TSX change, run `bun run type-check-pure`. If TypeScript LSP tools are available, also inspect changed symbols with hover and go-to-definition; otherwise skip LSP check.
|
||||
|
||||
## Type tightening
|
||||
|
||||
- **Avoid type casts and loose `unknown`** — fix the signature instead.
|
||||
- **If a function only needs one field of a wide object, accept that field** — not the wide object. The cast often disappears once the signature is right.
|
||||
- **Reach for `Partial<T>`, `Pick<T, K>`, `Record<K, V>`, and generics** before reaching for a cast.
|
||||
- **Match dictionary keys to the data.** Use a finite key union when the keys are known. Open dictionaries can use an index signature, `Record<string, T>`, or `Map`; index signatures still constrain values, and changing their spelling to `Record<string, T>` does not make missing-key access safe.
|
||||
- **Prefer making props/components generic** (`<T>`) when a value flows through unchanged and the caller knows the type.
|
||||
- **Prefer `unknown` over loose typing** and narrow before use — an `unknown` value forces a guard at the point of use.
|
||||
- **`satisfies` for object literals** that must conform without widening (config objects, lookup maps, discriminated literals) — better than `: T` (widens) or `as T` (unsafe).
|
||||
- **Avoid non-null assertions (`!`)**. Prefer a guard, early return, or `?.`. Use `!` only when non-nullness is provably true and localized, with a comment.
|
||||
- **Guard indexed lookups that may miss.** Arrays and dictionaries can return `undefined` even when the inferred type omits it. Use an iteration form that preserves the key/value relationship, and only assert `keyof` when the runtime keys are known to belong to the declared type.
|
||||
- **No redundant runtime coercion** — don't wrap already-typed values in `Number()` / `String()` / `Boolean()`.
|
||||
- **Type guards belong in `frontend/src/metabase-types/guards/`**. Do not redefine them locally.
|
||||
- **A cast you can't avoid needs a real justification comment.** The `metabase/no-unjustified-type-casts` rule accepts any preceding comment — state the actual reason the cast is safe. NEVER write the legacy `// Unjustified type cast. FIXME` placeholder; it exists only on casts that predated the rule, and copying it sneaks an unjustified cast past the linter. If you can't articulate why the cast is correct, the cast is wrong — fix the types.
|
||||
- **Keep unavoidable assertions local.** Isolate a repeated or complex assertion behind a helper when that makes its invariant easier to enforce. Test nontrivial runtime assumptions that justify it; a trivial assertion does not automatically need a new helper or test. Do not weaken a public signature just to silence implementation errors.
|
||||
|
||||
## Type modeling
|
||||
|
||||
- **Reuse existing types; don't re-declare them.** Use canonical IDs and domain entity types from `metabase-types/api` (`FieldId`, `TableId`, `ConcreteTableId`, `SchemaName`, …) and key data structures by them (`new Map<ConcreteTableId, …>()`). Don't duplicate generated/API types — compose or derive (`Pick`, `Omit`, indexed access `SomeType["field"]`, `ReturnType`).
|
||||
- **Generics must make promises the implementation can keep.** A caller-selected `get<T>(): T` must not disguise an unchecked assertion about external data. Return `unknown` and validate, or accept a validator that establishes `T`. A factory such as `function empty<T>(): T[] { return []; }` is valid; judge the implementation, not how often `T` appears in the signature.
|
||||
- **Prefer an honest type over false precision.** Use generics when they express a real relationship. If a complex type cannot model the behaviour accurately, choose a simpler type or `unknown` with narrowing instead of asserting an unsupported guarantee.
|
||||
- **Model the actual data contract; keep types narrow.** Optional `field?: T` for a key that may be absent, `field: T | undefined` only when the key is always present but the value may be undefined, `| null` for explicit API nulls. Prefer domain unions over broad `string` / `number` / loose `Record`.
|
||||
- **Refer to API implementation** when defining or refining types to ensure they match the actual data structure. When considering a type cast, first consider if the type should be refined to match the actual data structure.
|
||||
- **Optionality must reflect absence.** Keep required fields required and optional fields optional. Normalise input when the application has a meaningful default, not merely to remove a type error. Preserve the actual wire shape in API types.
|
||||
- **Represent related absence together when modelling internal state.** If several fields exist or disappear together, consider a nullable containing object or a discriminated union. Do not reshape a raw API declaration unless the data actually has that shape.
|
||||
- **Prefer explicit special states when designing a format.** Use nullability or named union variants when sentinel values such as `-1` hide meaning. Preserve established protocol sentinel values unless the behaviour is deliberately changed, or normalise them at an explicit boundary.
|
||||
- **Discriminated unions for variant state, with exhaustive checks.** Model "one of N shapes" as a union with a literal discriminant rather than a bag of optional fields, and exhaust it with ts-pattern's `.exhaustive()` so adding a variant becomes a compile error:
|
||||
```ts
|
||||
import { match } from "ts-pattern";
|
||||
|
||||
const result = match(status)
|
||||
.with({ type: "loading" }, () => <Spinner />)
|
||||
.with({ type: "error", error: P.select() }, (error) => <Error message={error.message} />)
|
||||
.with({ type: "success", data: P.select() }, (data) => <Content data={data} />)
|
||||
.exhaustive(); // Compile-time guarantee all cases handled
|
||||
```
|
||||
- **Derive union types from constants** (`as const` + `typeof`/`keyof`) so the type and the values can't drift.
|
||||
- **`readonly` / immutability where mutation isn't intended** — component props, shared constants, exported config, and unmutated parameters. Prefer `readonly T[]` / `ReadonlyArray<T>` for inputs you don't mutate. Functions that mutate caller-owned data should make that behaviour explicit.
|
||||
- **Construct complete, well-typed objects where practical.** Prefer an object expression when it avoids partially initialised state or assertions. Incremental construction is fine when it is clearer and maintains the type's invariants.
|
||||
- **Treat `metabase-lib` opaque types as black boxes.** Use `metabase-lib` functions to work with types such as `Lib.Query`, rather than casting into their internal representation.
|
||||
- **Type async and error states explicitly** (a discriminated union or the data-layer's typed result) — never leave loading/error/empty implicit.
|
||||
|
||||
## Function signatures
|
||||
|
||||
- **Make public contracts explicit where it improves stability and clarity.** Annotate parameters and return types at shared boundaries when useful; let local values infer. Use `satisfies` or an annotation when a declaration needs an explicit shape check.
|
||||
- **Accept the inputs the operation supports and return the most precise honest result.** Narrow avoidable uncertainty inside the function, but preserve meaningful nullability and union variants in its return type.
|
||||
- **Use named options when positional arguments are easy to confuse.** Consecutive arguments with the same type are a useful warning sign, not an automatic requirement to rewrite a clear API.
|
||||
- **Use `async`/`await` when it clarifies control flow or error handling.** Returning an existing promise directly is also valid; a Promise return type alone does not require adding `async`.
|
||||
|
||||
## Null and undefined
|
||||
|
||||
- **Narrow at the source**. If a value is optional only in a corner case, don't thread `undefined` through every layer — guard at the producer.
|
||||
- **Sensible defaults for optional values**. Use `?.` and `??` at the consumer.
|
||||
- **Narrow nullable list elements when the operation requires present values.** Filter with a type guard when missing entries should be discarded; preserve them when their absence or position carries meaning.
|
||||
- **Avoid non-strict null comparisons** (`X != null`) when `X` can never be `null` — use a strict check or narrow the type. Use `checkNotNull` where necessary.
|
||||
- **Check actual nullability against API implementation**. Find the API endpoint implementation and check if the field can actually be null.
|
||||
|
||||
## Naming
|
||||
|
||||
- **Names describe the entity, not the mechanism**. A name must reflect what the value holds.
|
||||
- **Use domain vocabulary and include units where useful** (`timeoutMs`, `widthPx`, `temperatureC`). Prefer a more specific name when `Info`, `Data`, or `Entity` obscures the meaning, while keeping established terminology when it is clear.
|
||||
- **Align sibling concepts**: keep verb conventions consistent across a related API.
|
||||
- **No names that encode implementation history** rather than current meaning. Suffixes like `Base`, `New`, `Old`, `Initial` need a real semantic distinction, otherwise drop them.
|
||||
- **Avoid cryptic identifiers** (`v`, `n`, `$n`) for domain values; short names are fine only in tiny conventional contexts (loop index `i`, coordinates `x`/`y`, generic params `T`/`K`/`V`).
|
||||
|
||||
## Code structure and organization
|
||||
|
||||
- **Prioritize reusability over duplication**. The codebase already has many utility functions — leverage them. If you introduce duplicated logic, extract it to a shared utility.
|
||||
- **Generic helpers do not belong in feature folders**. Promote to a shared utility.
|
||||
- **Keep functions small and single-purpose**. A 100+ line function is hard to review — split into focused named helpers, each with one responsibility and a minimal dependency surface. When necessary, cover with unit tests.
|
||||
- **Extract distinct complex JSX into named components**. Choose same-file vs separate-file by reuse, coupling, testability, and readability.
|
||||
|
||||
## Comments
|
||||
|
||||
- **No comments by default**. Well-named identifiers carry the `what`.
|
||||
- **Comments should be concise**. Add a short, concise comment only when the `why` is non-obvious: a workaround, a hidden invariant, a subtle ordering constraint, a clever reduction. Never document the actual implementation, focus on the intent and the why.
|
||||
|
||||
## Verify before done
|
||||
|
||||
- **Run the project type-check** when finished (see the shared TypeScript commands above).
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
|
||||
:aliased-namespace-symbol {:level :warning}
|
||||
:case-symbol-test {:level :warning}
|
||||
:condition-always-true {:level :warning}
|
||||
:def-fn {:level :warning}
|
||||
:dynamic-var-not-earmuffed {:level :warning}
|
||||
:equals-expected-position {:level :warning, :only-in-test-assertion true}
|
||||
@@ -49,11 +50,9 @@
|
||||
;; disabled linters
|
||||
;;
|
||||
|
||||
:constant-condition {:level :off} ; TODO (sashko): too confused by defenterprises, some hook/config is needed
|
||||
:redundant-ignore {:level :off} ; lots of false positives, see https://github.com/clj-kondo/clj-kondo/issues/2414
|
||||
:unexpected-recur {:level :off} ; TODO (cam): I think we just need to tell it how to handle MBQL match and we can enable this?
|
||||
:redundant-primitive-coercion {:level :off} ; there are many defensive boolean coercions, kondo looks too much into it
|
||||
:type-mismatch {:level :off} ; Still quite raw, can sometimes give useful insights, but sometimes is just confused
|
||||
|
||||
;;
|
||||
;; globally-enabled custom linters
|
||||
@@ -62,7 +61,6 @@
|
||||
:metabase/check-def-check-not-uppercase-name {:level :warning}
|
||||
:metabase/check-def-no-underscores {:level :warning}
|
||||
:metabase/check-me-humanize {:level :warning}
|
||||
:metabase/check-testing-not-empty {:level :warning}
|
||||
:metabase/defsetting-must-specify-export {:level :warning}
|
||||
:metabase/defsetting-namespace {:level :warning}
|
||||
:metabase/discourage-millis-duration {:level :warning}
|
||||
@@ -71,13 +69,6 @@
|
||||
:metabase/modules {:level :warning} ; more config in `.clj-kondo/config/modules/config.edn`
|
||||
:metabase/namespace-name {:level :warning}
|
||||
:metabase/no-jsqlparser-imports {:level :warning}
|
||||
:metabase/prefer-with-dynamic-fn-redefs {:level :warning}
|
||||
:metabase/t2-query-namespace {:level :warning}
|
||||
:metabase/table-or-field-query {:level :warning}
|
||||
;; App-DB query hygiene: a value reaching a value slot unmarked, and a query written as
|
||||
;; `:column value` pairs rather than a map. Off by default and enabled per namespace under
|
||||
;; `:config-in-ns`; flip it on globally once every `db.clj` has been converted.
|
||||
:metabase/unsafe-app-db-query {:level :off}
|
||||
:metabase/toucan-model-ns {:level :warning}
|
||||
:metabase/require-shape-checker {:level :warning}
|
||||
:metabase/tests-must-live-in-known-modules {:level :warning}
|
||||
@@ -186,70 +177,28 @@
|
||||
clojure.core.async/take!
|
||||
clojure.core.async/to-chan!
|
||||
clojure.core.async/to-chan!!
|
||||
;; writes the ratchet files from a :once fixture that finishes before the parallel tests
|
||||
dev.kondo-ratchet/fix!
|
||||
;; builds Babashka's classpath cache from a :once fixture, before the parallel tests start, so that
|
||||
;; they don't each race to create it; the side effect is process-external and idempotent
|
||||
metabase.core.modules-consistency-test/warm-babashka-classpath!
|
||||
metabase.ai-tracing.core/record!
|
||||
metabase.api.macros.defendpoint.tools-manifest/assert-optional-fields-nullable!
|
||||
;; MCP v2 tool tests: call-tool! dispatches a tool with no shared mutable state across tests
|
||||
metabase.mcp.v2.tools.alert-test/call-tool!
|
||||
metabase.mcp.v2.tools.definitions-test/call-tool!
|
||||
metabase.mcp.v2.tools.metric-test/call-tool!
|
||||
;; the `!` is for the throw (a teaching error), not a mutation; no shared mutable state
|
||||
metabase.mcp.v2.tools.browse/validate-args-for-action!
|
||||
metabase.channel.core/send!
|
||||
metabase.channel.render.pdf/draw-too-large!
|
||||
metabase.channel.render.pdf/draw-no-results!
|
||||
metabase.channel.render.pdf/render-card-cell!
|
||||
metabase.channel.render.pdf.font/load-fonts!
|
||||
metabase.warehouses.models.database/assert-not-h2!
|
||||
metabase.driver.sql-jdbc.execute/execute-prepared-statement!
|
||||
;; only inspects a details map and throws; the `!` is for the throw, not for a mutation
|
||||
metabase.driver/validate-db-details!
|
||||
metabase.mcp.usage/record-mcp-session!
|
||||
metabase.mcp.usage/record-mcp-session-end!
|
||||
metabase.mcp.usage/record-mcp-tool-call!
|
||||
metabase.notification.models/create-notification!
|
||||
metabase.oauth-server.core/reset-provider!
|
||||
metabase.pulse.send/send-pulse!
|
||||
metabase-enterprise.transforms-python.python-runner-test/execute!
|
||||
;; MCP usage tests isolate their rows by unique keys (session id / tool name / duration /
|
||||
;; client version) and clean up only those, so these deletes are safe in parallel.
|
||||
metabase-enterprise.mcp.usage-test/cleanup!
|
||||
metabase-enterprise.mcp.usage-test/cleanup-calls!
|
||||
;; Workspace isolation functions are safe in parallel tests because they operate on
|
||||
;; isolated, test-specific workspaces and databases that don't affect other tests
|
||||
metabase-enterprise.workspaces.isolation/destroy-workspace-isolation!
|
||||
metabase.lib-metric.ast.plan/validate-arithmetic-ast!
|
||||
metabase-enterprise.workspaces.isolation/ensure-database-isolation!
|
||||
metabase-enterprise.workspaces.test-util/create-ready-ws!
|
||||
metabase-enterprise.workspaces.core/clear-instance-workspace!
|
||||
metabase-enterprise.workspaces.core/set-instance-workspace!
|
||||
metabase.workspaces.core/clear-instance-workspace!
|
||||
metabase.workspaces.core/set-instance-workspace!
|
||||
;; Safe in tests, where we typically use a private, temporary index per thread.
|
||||
metabase.search.core/reindex!
|
||||
metabase.util/run-count!
|
||||
next.jdbc/execute!
|
||||
;; pure validator: parses a keyword and throws on a bad one, touches no state
|
||||
metabase.mcp.v2.tools.question/check-semantic-type!}}
|
||||
next.jdbc/execute!}}
|
||||
|
||||
:metabase/i-like-making-cams-eyes-bleed-with-horrifically-long-tests
|
||||
{:level :warning
|
||||
:max-length 150}
|
||||
|
||||
;; Warn on long tests (tests over `:min-length`) that look like
|
||||
;;
|
||||
;; (deftest my-test
|
||||
;; (testing ...)
|
||||
;; (testing ...)
|
||||
;; (testing ...))
|
||||
:metabase/validate-deftest-logically-separate-tests
|
||||
{:level :warning
|
||||
:min-length 60}
|
||||
|
||||
;; this is not enabled here, it is enabled below only for specific namespaces.
|
||||
:metabase/disallow-hardcoded-driver-names-in-tests
|
||||
;; don't include `:h2` since that will probably lead to a mountain of tests where we are specifically testing
|
||||
@@ -282,6 +231,9 @@
|
||||
;; misc config for built-in linters
|
||||
;;
|
||||
|
||||
:invalid-arity {:skip-args [metabase.util.match/match-one
|
||||
metabase.util.match/match-many
|
||||
metabase.util.match/replace]} ; TODO (cam): can we fix these?
|
||||
:refer-all {:level :warning, :exclude [clojure.test]}
|
||||
:unused-referred-var {:exclude {compojure.core [GET DELETE POST PUT]}}
|
||||
|
||||
@@ -297,13 +249,20 @@
|
||||
instaparse.core/transform
|
||||
(metabase.server.streaming-response/streaming-response)
|
||||
(metabase.driver.druid.query-processor-test/druid-query-returning-rows)
|
||||
(metabase.util.match/match-one)
|
||||
(metabase.util.match/match-many)
|
||||
(metabase.util.match/replace-in)
|
||||
(metabase.util.match/replace)
|
||||
(metabase.driver-api.core/match)
|
||||
(metabase.driver-api.core/match-one)
|
||||
(metabase.driver-api.core/match-many)
|
||||
(metabase.driver-api.core/replace)
|
||||
(metabase.driver-api.core/replace-in)
|
||||
(metabase.query-processor.middleware.cache-backend.interface/with-cached-results)
|
||||
(metabase.util.regex/rx [opt])
|
||||
(metabase.util/prog1 [<>])]}
|
||||
(metabase.util/prog1 [<>])
|
||||
(taoensso.nippy/extend-freeze)
|
||||
(taoensso.nippy/extend-thaw)]}
|
||||
|
||||
;; TODO (Cam 8/22/25) -- what's the point of marking stuff deprecated if we don't have Kondo enforce it? We should
|
||||
;; remove all of these and add individual ignores as needed.
|
||||
@@ -316,10 +275,7 @@
|
||||
metabase.query-processor.util/normalize-token {:namespaces ["metabase.*"]}}}
|
||||
|
||||
:discouraged-var
|
||||
{metabase-enterprise.entity-retrieval.core/search-unfiltered {:message "Permission bypass risk: library-index hits are unfiltered — resolve each entity and drop what the current user can't read"}
|
||||
metabase.entity-retrieval.core/search-unfiltered {:message "Permission bypass risk: library-index hits are unfiltered — resolve each entity and drop what the current user can't read"}
|
||||
metabase.entity-retrieval.mirror/search-unfiltered {:message "Permission bypass risk: library-index hits are unfiltered — resolve each entity and drop what the current user can't read"}
|
||||
clojure.core/print {:message "Use metabase.util.log instead of clojure.core/print"}
|
||||
{clojure.core/print {:message "Use metabase.util.log instead of clojure.core/print"}
|
||||
clojure.core/println {:message "Use metabase.util.log instead of clojure.core/println"}
|
||||
clojure.core/printf {:message "Use metabase.util.log instead of clojure.core/printf"}
|
||||
clojure.core/prn {:message "Use metabase.util.log instead of clojure.core/prn"}
|
||||
@@ -356,7 +312,8 @@
|
||||
metabase-enterprise.transforms.interface/execute! {:message "Use metabase-enterprise.transforms.execute/execute! instead of metabase-enterprise.transforms.interface/execute!"}
|
||||
next.jdbc/active-tx? {:message "Use metabase.app-db.core/in-transaction? to see if we are in a transaction"}
|
||||
toucan2.core/debug {:message "Don't commit code using t2/debug"}
|
||||
toucan2.tools.debug/debug {:message "Don't commit code using t2/debug"}}
|
||||
toucan2.tools.debug/debug {:message "Don't commit code using t2/debug"}
|
||||
}
|
||||
|
||||
:discouraged-namespace
|
||||
{camel-snake-kebab.core {:message "CSK is not Turkish-safe, use the versions in metabase.util instead."}
|
||||
@@ -418,7 +375,6 @@
|
||||
metabase.channel.render.core channel.render
|
||||
metabase.cloud-migration.core cloud-migration
|
||||
metabase.eid-translation.core eid-translation
|
||||
metabase.llm.test-util llm.tu
|
||||
metabase.login-history.core login-history
|
||||
metabase.model-persistence.core model-persistence
|
||||
metabase.notification.core notification
|
||||
@@ -475,21 +431,13 @@
|
||||
instaparse.core/defparser clojure.core/def
|
||||
metabase-enterprise.action-v2.test-util/with-test-tables! clojure.core/let
|
||||
metabase-enterprise.content-translation.routes-test/with-new-secret-key! clojure.core/fn
|
||||
metabase-enterprise.entity-retrieval.reconcile-test/with-isolated-index clojure.core/fn
|
||||
metabase-enterprise.mfa.enrollment-flow-test/with-fresh-user-session! clojure.core/fn
|
||||
metabase-enterprise.mfa.login-flow-test/with-enrolled-rasta! clojure.core/fn
|
||||
metabase-enterprise.mfa.recovery-codes-test/with-confirmed-enrollment! clojure.core/fn
|
||||
metabase-enterprise.serialization.metadata-file-import.wire-format-test/with-tiny-fixture! clojure.core/fn
|
||||
metabase-enterprise.serialization.test-util/with-dbs clojure.core/fn
|
||||
metabase-enterprise.serialization.test-util/with-random-dump-dir clojure.core/let
|
||||
metabase-enterprise.tenants.pulse-api-test/with-tenant-pulse-fixture! clojure.core/fn
|
||||
metabase-enterprise.workspaces.test-util/with-resources! clojure.core/let
|
||||
metabase-enterprise.workspaces.test-util/with-workspaces! clojure.core/let
|
||||
metabase.actions.test-util/with-actions clojure.core/let
|
||||
metabase.transforms.test-util/with-transform-cleanup! clojure.core/let
|
||||
metabase.api.common/let-404 clojure.core/let
|
||||
metabase.app-db.connection/with-unshared-connection clojure.core/fn
|
||||
metabase.app-db.core/with-unshared-connection clojure.core/fn
|
||||
metabase.app-db.custom-migrations.util/with-temp-scheduler! clojure.core/fn
|
||||
metabase.app-db.custom-migrations/define-migration clj-kondo.lint-as/def-catch-all
|
||||
metabase.app-db.custom-migrations/define-reversible-migration clj-kondo.lint-as/def-catch-all
|
||||
@@ -498,7 +446,6 @@
|
||||
metabase.channel.impl.http-test/with-captured-http-requests clojure.core/fn
|
||||
metabase.channel.impl.http-test/with-server clojure.core/let
|
||||
metabase.collections.models.collection-test/with-collection-in-location clojure.core/let
|
||||
metabase.collections.test-utils/with-library clojure.core/fn
|
||||
metabase.driver.mongo.connection/with-mongo-client clojure.core/let
|
||||
metabase.driver.mongo.connection/with-mongo-database clojure.core/let
|
||||
metabase.driver.mongo.query-processor/mongo-let clojure.core/let
|
||||
@@ -507,14 +454,11 @@
|
||||
metabase.driver.sql-jdbc.connection/with-connection-spec-for-testing-connection clojure.core/let
|
||||
metabase.driver.sql-jdbc.execute.diagnostic/capturing-diagnostic-info clojure.core/fn
|
||||
metabase.driver-api.core/defsetting hooks.metabase.settings.models.setting/defsetting
|
||||
metabase.mcp.v2.registry/deftool clojure.core/defn
|
||||
metabase.models.json-migration/def-json-migration clj-kondo.lint-as/def-catch-all
|
||||
metabase.models.util.spec-update/define-spec clojure.core/def
|
||||
metabase.notification.test-util/with-card-notification clojure.core/let
|
||||
metabase.notification.test-util/with-system-event-notification! clojure.core/let
|
||||
metabase.notification.test-util/with-temp-notification clojure.core/let
|
||||
metabase.oauth-server.test-util/with-oauth-client clojure.core/fn
|
||||
metabase.osi.ai-context.api-test/with-test-entry clojure.core/let
|
||||
metabase.premium-features.core/defenterprise-schema clj-kondo.lint-as/def-catch-all
|
||||
metabase.premium-features.defenterprise/defenterprise-schema clj-kondo.lint-as/def-catch-all
|
||||
metabase.premium-features.settings/define-premium-feature clojure.core/def
|
||||
@@ -523,9 +467,7 @@
|
||||
metabase.query-processor.pipeline-queries-test/mbql5-query clojure.core/->
|
||||
metabase.query-processor.pipeline-queries-test/run-mbql5-query clojure.core/->
|
||||
metabase.query-processor.error-type/deferror clojure.core/def
|
||||
metabase.query-processor.core/with-reducible-deserialized-results clojure.core/let
|
||||
metabase.query-processor.middleware.cache.impl/with-reducible-deserialized-results clojure.core/let
|
||||
metabase.query-processor.result-serialization/with-reducible-deserialized-results clojure.core/let
|
||||
metabase.query-processor.middleware.process-userland-query-test/with-query-execution! clojure.core/let
|
||||
metabase.query-processor.setup/with-qp-setup clojure.core/let
|
||||
metabase.search.api-test/do-test-users clojure.core/let
|
||||
@@ -542,6 +484,7 @@
|
||||
metabase.test.data.interface/defdataset clojure.core/def
|
||||
metabase.test.data.interface/defdataset-edn clojure.core/def
|
||||
metabase.test.embedded-postgres.core/with-system clojure.core/let
|
||||
metabase.test.gentest/defgentest clojure.test/deftest
|
||||
metabase.test/defdataset clojure.core/def
|
||||
metabase.test/let-url clojure.core/let
|
||||
metabase.test/with-actions clojure.core/let
|
||||
@@ -552,7 +495,6 @@
|
||||
metabase.test/with-temp-empty-app-db clojure.core/let
|
||||
metabase.test/with-temp-file clojure.core/let
|
||||
metabase.test/with-user-in-groups clojure.core/let
|
||||
metabase.transforms-rest.api.transform-runs-test/with-runs-fixture! clojure.core/fn
|
||||
metabase.upload.impl-test/with-upload-table! clojure.core/let
|
||||
metabase.util/for-map clojure.core/for
|
||||
metabase.util.files/with-open-path-to-resource clojure.core/let
|
||||
@@ -583,7 +525,6 @@
|
||||
{:analyze-call
|
||||
{cljs.test/deftest hooks.clojure.test/deftest
|
||||
cljs.test/is hooks.clojure.test/is
|
||||
cljs.test/testing hooks.clojure.test/testing
|
||||
cljs.test/use-fixtures hooks.clojure.test/use-fixtures
|
||||
clojure.core/def hooks.clojure.core.def/lint-def
|
||||
clojure.core/definterface hooks.clojure.core.def/lint-def
|
||||
@@ -595,15 +536,12 @@
|
||||
clojure.core/defprotocol hooks.clojure.core.def/lint-def
|
||||
clojure.core/defrecord hooks.clojure.core.def/lint-def
|
||||
clojure.core/deftype hooks.clojure.core.def/lint-def
|
||||
clojure.core/ex-info hooks.metabase.agent-message/lint-ex-info
|
||||
clojure.core/ns hooks.clojure.core.ns/lint-ns
|
||||
clojure.core/require hooks.clojure.core.require/lint-require
|
||||
clojure.core/requiring-resolve hooks.clojure.core.require/lint-requiring-resolve
|
||||
clojure.core/with-redefs hooks.clojure.core.with-redefs/lint-with-redefs
|
||||
clojure.core/- hooks.metabase.util.timing/discourage-millis-duration
|
||||
clojure.test/deftest hooks.clojure.test/deftest
|
||||
clojure.test/is hooks.clojure.test/is
|
||||
clojure.test/testing hooks.clojure.test/testing
|
||||
clojure.test/use-fixtures hooks.clojure.test/use-fixtures
|
||||
malli.error/humanize hooks.malli.error/humanize
|
||||
metabase-enterprise.advanced-permissions.models.permissions.application-permissions-test/with-new-group-and-current-graph hooks.common/with-two-top-level-bindings
|
||||
@@ -611,6 +549,7 @@
|
||||
metabase-enterprise.cache.config-test/with-temp-persist-models hooks.common/with-seven-bindings
|
||||
metabase-enterprise.dependencies.api-test/with-dependents-test! hooks.common/with-two-bindings
|
||||
metabase-enterprise.metabot-v3.tools.dependencies-test/with-dependent-transforms! hooks.common/with-two-bindings
|
||||
metabase-enterprise.metabot.tools.dependencies-test/with-dependent-transforms! hooks.common/with-two-bindings
|
||||
metabase-enterprise.semantic-search.db.connection/with-migrate-tx hooks.common/let-one-with-optional-value
|
||||
metabase-enterprise.serialization.api-test/with-serialization-test-data! hooks.common/with-three-bindings
|
||||
metabase-enterprise.serialization.test-util/with-temp-dpc hooks.toucan2.tools.with-temp/with-temp
|
||||
@@ -618,7 +557,7 @@
|
||||
metabase.api.common.internal-test/with-jetty-server hooks.common/with-one-binding
|
||||
metabase.api.macros/defendpoint hooks.metabase.api.macros/defendpoint
|
||||
metabase.app-db.schema-migrations-test.impl/test-migrations hooks.metabase.app-db.schema-migrations-test.impl/test-migrations
|
||||
metabase.channel.render.js.pool/with-static-viz-context hooks.common/with-one-top-level-binding
|
||||
metabase.channel.render.js.svg/with-static-viz-context hooks.common/with-one-top-level-binding
|
||||
metabase.channel.template.handlebars-helper/defhelper hooks.metabase.channel.template.handlebars-helper/defhelper
|
||||
metabase.channel.template.handlebars-test/with-temp-template! hooks.common/with-vec-first-binding
|
||||
metabase.classloader.core/require hooks.clojure.core.require/lint-require
|
||||
@@ -647,66 +586,13 @@
|
||||
metabase.lib.schema.mbql-clause/define-tuple-mbql-clause hooks.metabase.lib.schema.mbql-clause/define-mbql-clause
|
||||
methodical.core/defmethod hooks.metabase.toucan.table-name/lint-defmethod
|
||||
methodical.macros/defmethod hooks.metabase.toucan.table-name/lint-defmethod
|
||||
metabase.app-db.core/query hooks.metabase.toucan.db-ns/lint-query-call
|
||||
metabase.app-db.core/streaming-reducible-query hooks.metabase.toucan.db-ns/lint-query-call
|
||||
metabase.app-db.core/select-or-insert! hooks.metabase.toucan.db-ns/lint-query-call
|
||||
metabase.app-db.core/update-or-insert! hooks.metabase.toucan.db-ns/lint-query-call
|
||||
metabase.app-db.core/current-timestamp-string hooks.metabase.toucan.db-ns/lint-query-call
|
||||
metabase.app-db.query/query hooks.metabase.toucan.db-ns/lint-query-call
|
||||
metabase.app-db.query/streaming-reducible-query hooks.metabase.toucan.db-ns/lint-query-call
|
||||
metabase.app-db.query/select-or-insert! hooks.metabase.toucan.db-ns/lint-query-call
|
||||
metabase.app-db.query/update-or-insert! hooks.metabase.toucan.db-ns/lint-query-call
|
||||
metabase.app-db.query/current-timestamp-string hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/count hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/delete! hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/exists? hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/insert! hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/insert-returning-instance! hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/insert-returning-instances! hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/insert-returning-pk! hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/insert-returning-pks! hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/query hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/query-one hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/reducible-query hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/reducible-select hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/reducible-update hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/reducible-update-returning-pks hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/save! hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/select hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/select-fn->fn hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/select-fn->pk hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/select-fn-reducible hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/select-fn-set hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/select-fn-vec hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/select-one hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/select-one-fn hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/select-one-pk hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/select-pk->fn hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/select-pks-set hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/select-pks-vec hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/update! hooks.metabase.toucan.db-ns/lint-query-call
|
||||
toucan2.core/update-returning-pks! hooks.metabase.toucan.db-ns/lint-query-call
|
||||
metabase.lib.test-util.macros/$ids hooks.metabase.test.data/$ids
|
||||
metabase.lib.test-util.macros/mbql-query hooks.metabase.test.data/mbql-query
|
||||
metabase.lib.test-util.macros/mbql-5-query hooks.metabase.test.data/mbql-query
|
||||
metabase.lib.test-util.macros/with-testing-against-standard-queries hooks.metabase.lib.test-util.macros/with-testing-against-standard-queries
|
||||
metabase.mcp.transport/jsonrpc-error hooks.metabase.agent-message/lint-jsonrpc-error
|
||||
metabase.mcp.v2.common/ellipsize hooks.metabase.agent-message/lint-ellipsize
|
||||
metabase.mcp.v2.common/error-content hooks.metabase.agent-message/lint-teaching-exit
|
||||
metabase.mcp.v2.common/list-content hooks.metabase.agent-message/lint-list-content
|
||||
metabase.mcp.v2.common/success-content hooks.metabase.agent-message/lint-success-content
|
||||
metabase.mcp.v2.common/throw-teaching-error hooks.metabase.agent-message/lint-teaching-exit
|
||||
metabase.mcp.v2.dashboard-ops/op-error! hooks.metabase.agent-message/lint-op-error
|
||||
metabase.mcp.v2.message/msg hooks.metabase.agent-message/lint-msg
|
||||
metabase.mcp.v2.tools.browse/paged-list-content hooks.metabase.agent-message/lint-list-content
|
||||
metabase.mcp.v2.tools.parameters-test/with-fixtures hooks.common/with-one-binding
|
||||
metabase.metrics.api-dimension-test/with-seeded-metric hooks.common/with-one-binding
|
||||
metabase.model-persistence.api-test/with-persistence-setup! hooks.common/with-one-top-level-binding
|
||||
metabase.model-persistence.api-test/with-setup! hooks.common/with-one-top-level-binding
|
||||
metabase.model-persistence.test-util/with-persistence-enabled! hooks.metabase.model-persistence.test-util/with-persistence-enabled!
|
||||
metabase.mq.core/with-queue hooks.common/let-second
|
||||
metabase.mq.test-util/with-test-mq hooks.common/with-one-binding
|
||||
metabase.mq.core/def-listener! hooks.metabase.mq.impl/def-listener
|
||||
metabase.models.interface/define-batched-hydration-method hooks.metabase.models.interface/define-hydration-method
|
||||
metabase.models.interface/define-simple-hydration-method hooks.metabase.models.interface/define-hydration-method
|
||||
metabase.permissions.core/allow-changing-all-users-group-members hooks.common/do*
|
||||
@@ -749,6 +635,7 @@
|
||||
metabase.test.data/dataset hooks.metabase.test.data/dataset
|
||||
metabase.test.data/mbql-query hooks.metabase.test.data/mbql-query
|
||||
metabase.test.data/run-mbql-query hooks.metabase.test.data/mbql-query
|
||||
metabase.test.gentest/iterate hooks.metabase.test.gentest/iterate*
|
||||
metabase.test.util.async/with-open-channels hooks.common/let-with-optional-value-for-last-binding
|
||||
metabase.test.util.generators.jvm/with-random-cards hooks.metabase.test.util.generators.jvm/with-random-cards
|
||||
metabase.test.util.log/with-log-level hooks.common/with-ignored-first-arg
|
||||
@@ -777,7 +664,6 @@
|
||||
metabase.test/with-temp hooks.toucan2.tools.with-temp/with-temp
|
||||
metabase.test/with-temp-file hooks.metabase.test.util/with-temp-file
|
||||
metabase.test/with-temporary-setting-values hooks.metabase.test.util/with-temporary-setting-values
|
||||
metabase.tracing.test-util/with-span-exporter hooks.common/with-one-binding
|
||||
metabase.users-rest.api-test/with-temp-user-email! hooks.common/with-one-binding
|
||||
metabase.util.i18n/deferred-trs hooks.metabase.util.i18n/deferred-trs
|
||||
metabase.util.i18n/deferred-tru hooks.metabase.util.i18n/deferred-tru
|
||||
@@ -797,16 +683,9 @@
|
||||
metabase.util.log/warn hooks.metabase.util.log/info
|
||||
metabase.util.log/warnf hooks.metabase.util.log/infof
|
||||
metabase.util.malli.registry/def hooks.metabase.util.malli.registry/def
|
||||
metabase.util.match/match-one hooks.metabase.util.match/match-one
|
||||
metabase.util.match/match-many hooks.metabase.util.match/match-many
|
||||
metabase.util.match/matches? hooks.metabase.util.match/matches?
|
||||
metabase.util.match/replace hooks.metabase.util.match/replace
|
||||
metabase.util.match/replace-in hooks.metabase.util.match/replace-in
|
||||
metabase.util/case-enum hooks.metabase.util/case-enum
|
||||
metabase.util/format-color hooks.metabase.util/format-color
|
||||
metabase.xrays.api.automagic-dashboards-test/with-indexed-model! hooks.metabase.xrays.api.automagic-dashboards-test/with-indexed-model!
|
||||
taoensso.nippy/extend-freeze hooks.taoensso.nippy/extend-freeze
|
||||
taoensso.nippy/extend-thaw hooks.taoensso.nippy/extend-thaw}
|
||||
metabase.xrays.api.automagic-dashboards-test/with-indexed-model! hooks.metabase.xrays.api.automagic-dashboards-test/with-indexed-model!}
|
||||
|
||||
:macroexpand
|
||||
{clojurewerkz.quartzite.jobs/build macros.quartz/build-job
|
||||
@@ -815,7 +694,6 @@
|
||||
clojurewerkz.quartzite.schedule.simple/schedule macros.quartz/simple-schedule
|
||||
clojurewerkz.quartzite.triggers/build macros.quartz/build-trigger
|
||||
metabase-enterprise.database-routing.e2e-test/with-temp-dbs! macros.metabase-enterprise.database-routing.e2e-test/with-temp-dbs!
|
||||
metabase-enterprise.database-routing.e2e-test/with-routing-setup! macros.metabase-enterprise.database-routing.e2e-test/with-routing-setup!
|
||||
metabase-enterprise.impersonation.util-test/with-impersonations! macros.metabase-enterprise.impersonation.util-test/with-impersonations!
|
||||
metabase-enterprise.query-reference-validation.api-test/with-test-setup! macros.metabase-enterprise.query-reference-validation.api-test/with-test-setup!
|
||||
metabase-enterprise.sandbox.test-util/with-gtaps! macros.metabase-enterprise.sandbox.test-util/with-gtaps!
|
||||
@@ -846,7 +724,6 @@
|
||||
metabase.test/with-temporary-raw-setting-values macros.metabase.test.util/with-temporary-raw-setting-values
|
||||
metabase.users.models.user-test/with-groups! macros.metabase.users.models.user-test/with-groups!
|
||||
metabase.util.namespaces/import-fn macros.metabase.util.namespaces/import-fn
|
||||
metabase.util.namespaces/import-macro macros.metabase.util.namespaces/import-macro
|
||||
metabase.xrays.related-test/with-world macros.metabase.xrays.related-test/with-world}}
|
||||
|
||||
:config-in-comment
|
||||
@@ -868,7 +745,7 @@
|
||||
:name test-namespaces}
|
||||
{:pattern "^(?!.*-test(?:\\..*)?$)(?!.*test-util)(?!^metabase\\.test.*).*$"
|
||||
:name source-namespaces}
|
||||
{:pattern "(?:(?:^metabase\\.cmd.*)|(?:^build.*)|(?:^metabuild-common.*)|(?:^release.*)|(?:^i18n.*)|(?:^lint-migrations-file$))|(?:^mage.*)|(?:^load-namespaces.*)|(?:^dev\\..*)"
|
||||
{:pattern "(?:(?:^metabase\\.cmd.*)|(?:^build.*)|(?:^metabuild-common.*)|(?:^release.*)|(?:^i18n.*)|(?:^lint-migrations-file$))|(?:^mage.*)"
|
||||
:name printable-namespaces}
|
||||
{:pattern "^metabase\\.(?:(?:lib)|(?:legacy-mbql))\\..*"
|
||||
:name metabase-lib}
|
||||
@@ -904,41 +781,18 @@
|
||||
;;
|
||||
{:pattern "^metabase(?:((?:-enterprise\\.sandbox)?\\.query-processor)|(?:\\.driver\\.sql(?!server)(?!ite))).*-test$"
|
||||
:name general-qp-and-driver-test-namespaces}
|
||||
|
||||
{:pattern "(?:^mage.*)"
|
||||
:name mage-namespaces}
|
||||
;;
|
||||
;; the app-db module is the application database layer itself (connections, migrations, encryption), so it gets to
|
||||
;; run Toucan 2 queries anywhere.
|
||||
;;
|
||||
{:pattern "^metabase\\.app-db\\..*"
|
||||
:name app-db-namespaces}
|
||||
;;
|
||||
;; MCP source namespaces, OSS and enterprise, whose prose reaches an LLM agent, excluding test namespaces.
|
||||
;;
|
||||
{:pattern "^(?!.*-test(?:\\..*)?$)(?!.*test-util)metabase(?:-enterprise)?\\.mcp\\..*"
|
||||
:name agent-message-namespaces}
|
||||
;;
|
||||
;; MCP source namespaces whose caller-facing errors are MCP exits: `metabase.mcp.transport` and `metabase.mcp.v2.*`,
|
||||
;; and their enterprise counterparts. Other MCP namespaces' 4xx `ex-info`s are admin API and middleware errors, not
|
||||
;; MCP exits.
|
||||
;;
|
||||
{:pattern "^(?!.*-test(?:\\..*)?$)(?!.*test-util)metabase(?:-enterprise)?\\.mcp\\.(?:transport$|v2\\..*)"
|
||||
:name agent-exit-namespaces}]
|
||||
:name mage-namespaces}]
|
||||
|
||||
:config-in-ns
|
||||
{metabase.app-db.custom-migrations
|
||||
{:linters
|
||||
;; a migration reads the rows as they were when it ran; applying today's user settings would make it drift
|
||||
{:metabase/table-or-field-query {:level :off}}}
|
||||
|
||||
test-namespaces
|
||||
{test-namespaces
|
||||
{:linters
|
||||
{:inline-def {:level :off}
|
||||
:missing-docstring {:level :off}
|
||||
:private-call {:level :off}
|
||||
:metabase/defsetting-must-specify-export {:level :off}
|
||||
:metabase/defsetting-namespace {:level :off}
|
||||
:metabase/t2-query-namespace {:level :off}
|
||||
:metabase/test-helpers-use-non-thread-safe-functions {:level :warning} ; more config in `.clj-kondo/config/test-helpers/config.edn`
|
||||
|
||||
:discouraged-var
|
||||
@@ -946,38 +800,9 @@
|
||||
metabase.lib.convert/->legacy-MBQL {:level :off}
|
||||
metabase.lib.core/->legacy-MBQL {:level :off}}}}
|
||||
|
||||
app-db-namespaces
|
||||
{:linters {:metabase/t2-query-namespace {:level :off}}}
|
||||
|
||||
agent-message-namespaces
|
||||
{:linters {:metabase/agent-message-lines {:level :warning}}}
|
||||
|
||||
agent-exit-namespaces
|
||||
{:linters {:metabase/agent-message-exit {:level :warning}}}
|
||||
|
||||
;; Namespaces whose values are marked. Adding one here is what keeps it that way; see
|
||||
;; `metabase.app-db.value-guard`.
|
||||
metabase.content-translation.db
|
||||
{:linters {:metabase/unsafe-app-db-query {:level :warning}}}
|
||||
|
||||
;; .cljc tests run in both Clojure and ClojureScript — `with-dynamic-fn-redefs` is JVM-only,
|
||||
;; so leave these alone.
|
||||
metabase.lib.aggregation-test
|
||||
{:linters {:metabase/prefer-with-dynamic-fn-redefs {:level :off}}}
|
||||
metabase.util.time-test
|
||||
{:linters {:metabase/prefer-with-dynamic-fn-redefs {:level :off}}}
|
||||
metabase.lib.metadata.calculation-test
|
||||
{:linters {:metabase/prefer-with-dynamic-fn-redefs {:level :off}}}
|
||||
|
||||
;; classloader sits below `util` in the module graph, so it can't use `metabase.util.log` without creating
|
||||
;; a cycle; it gets to use tools.logging directly.
|
||||
metabase.classloader.impl
|
||||
{:linters {:discouraged-namespace {clojure.tools.logging {:level :off}}}}
|
||||
|
||||
source-namespaces
|
||||
{:linters
|
||||
{:metabase/discourage-millis-duration {:level :warning}
|
||||
:metabase/prefer-with-dynamic-fn-redefs {:level :off}
|
||||
:metabase/validate-defendpoint-has-response-schema {:level :warning}
|
||||
:metabase/validate-escaped-single-quotes-in-i18n {:level :warning}
|
||||
:metabase/validate-string-or-str-args-to-i18n {:level :warning}
|
||||
@@ -987,7 +812,9 @@
|
||||
clojure.core/eval {:message "Don't use eval outside of tests"}}
|
||||
|
||||
:discouraged-namespace
|
||||
{metabase.legacy-mbql.normalize {:message "Avoid legacy MBQL normalization, use MBQL 5 + lib/normalize instead"}
|
||||
{metabase.legacy-mbql.jvm-util {:message "Avoid legacy MBQL utils, use Lib instead"}
|
||||
metabase.legacy-mbql.normalize {:message "Avoid legacy MBQL normalization, use MBQL 5 + lib/normalize instead"}
|
||||
metabase.legacy-mbql.predicates {:message "Avoid legacy MBQL utils, use Lib instead"}
|
||||
metabase.legacy-mbql.schema {:message "Avoid the legacy MBQL schema, use MBQL 5 + lib.schema instead"}
|
||||
metabase.legacy-mbql.schema.helpers {:message "Avoid the legacy MBQL schema, use MBQL 5 + lib.schema instead"}
|
||||
metabase.legacy-mbql.schema.macros {:message "Avoid the legacy MBQL schema, use MBQL 5 + lib.schema instead"}
|
||||
@@ -1039,7 +866,9 @@
|
||||
legacy-mbql-namespaces
|
||||
{:linters
|
||||
{:discouraged-namespace
|
||||
{metabase.legacy-mbql.normalize {:level :off}
|
||||
{metabase.legacy-mbql.jvm-util {:level :off}
|
||||
metabase.legacy-mbql.normalize {:level :off}
|
||||
metabase.legacy-mbql.predicates {:level :off}
|
||||
metabase.legacy-mbql.schema {:level :off}
|
||||
metabase.legacy-mbql.schema.helpers {:level :off}
|
||||
metabase.legacy-mbql.schema.macros {:level :off}
|
||||
@@ -1059,7 +888,9 @@
|
||||
;; don't warn about the legacy MBQL namespaces being deprecated or discouraged in driver code now, we don't have a
|
||||
;; real alternative YET so it's not a helpful warning.
|
||||
:discouraged-namespace
|
||||
{metabase.legacy-mbql.normalize {:level :off}
|
||||
{metabase.legacy-mbql.jvm-util {:level :off}
|
||||
metabase.legacy-mbql.normalize {:level :off}
|
||||
metabase.legacy-mbql.predicates {:level :off}
|
||||
metabase.legacy-mbql.schema {:level :off}
|
||||
metabase.legacy-mbql.schema.helpers {:level :off}
|
||||
metabase.legacy-mbql.schema.macros {:level :off}
|
||||
@@ -1068,7 +899,14 @@
|
||||
|
||||
:deprecated-namespace
|
||||
{:exclude
|
||||
[metabase.legacy-mbql.normalize
|
||||
[metabase.driver.common.parameters
|
||||
metabase.driver.common.parameters.dates
|
||||
metabase.driver.common.parameters.operators
|
||||
metabase.driver.common.parameters.parse
|
||||
metabase.driver.common.parameters.values
|
||||
metabase.legacy-mbql.jvm-util
|
||||
metabase.legacy-mbql.normalize
|
||||
metabase.legacy-mbql.predicates
|
||||
metabase.legacy-mbql.schema
|
||||
metabase.legacy-mbql.schema.helpers
|
||||
metabase.legacy-mbql.schema.macros
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,7 +0,0 @@
|
||||
;; Budgets for escape hatches in config.edn. Counts may fall but not exceed these values.
|
||||
;; Leave reductions to the post-merge shrink workflow; explain increases in the PR.
|
||||
{:api-any 1
|
||||
:friend-edges 1
|
||||
:model-imports-bypass 2
|
||||
:ns-prefixes 81
|
||||
:uses-any 4}
|
||||
@@ -36,9 +36,6 @@
|
||||
;; Definitely thread safe
|
||||
metabase.test.util.dynamic-redefs/patch-vars!
|
||||
|
||||
;; test-only cleanup of in-memory throttle state (atom resets)
|
||||
metabase.session.api/reset-throttlers-for-testing!
|
||||
|
||||
;; TODO: most of these symbols shouldn't be here, we should go through them and
|
||||
;; find the functions/macros that use them and make sure their names end with !
|
||||
;; best way to do this is try remove each of these and rely on kondo output to find places where it's used
|
||||
@@ -58,7 +55,6 @@
|
||||
metabase.app-db.core/finish-db-setup!
|
||||
metabase.app-db.core/migrate!
|
||||
metabase.app-db.core/setup-db!
|
||||
metabase.app-db.encryption-test-util/do-with-encrypted-app-db!
|
||||
metabase.app-db.schema-migrations-test.impl/run-migrations-in-range!
|
||||
metabase.app-db.setup/migrate!
|
||||
metabase.app-db.setup/setup-db!
|
||||
|
||||
@@ -6,8 +6,3 @@
|
||||
`(def ~(-> sym name symbol) "docstring" ~sym))
|
||||
([sym name]
|
||||
`(def ~name "docstring" ~sym)))
|
||||
|
||||
(defmacro import-macro
|
||||
"Kondo macro replacement for [[metabase.util.namespaces/import-macro]]."
|
||||
[sym]
|
||||
`(def ~(-> sym name symbol) "docstring" ~sym))
|
||||
|
||||
@@ -3,8 +3,3 @@
|
||||
(defmacro with-temp-dbs! [bindings & body]
|
||||
`(let [~@(interleave bindings (repeat nil))]
|
||||
~@body))
|
||||
|
||||
(defmacro with-routing-setup! [[router-binding destinations] & body]
|
||||
`(let [~router-binding nil
|
||||
~@(interleave (map first destinations) (repeat nil))]
|
||||
~@body))
|
||||
|
||||
@@ -1,82 +0,0 @@
|
||||
;; Budgets for kondo suppressions: inline `:clj-kondo/ignore` forms per linter (:ignore-counts), and
|
||||
;; config-level waivers in .clj-kondo/config.edn (:config-counts -- :off switches and :exclude entries).
|
||||
;; Each :ignore-counts value is a non-negative integer budget, or :unlimited for no ceiling.
|
||||
;; Checks fail when a count exceeds its numeric budget; unused budget is allowed.
|
||||
;; Any ignore outside :comment-exempt needs an explanatory comment directly above or trailing on its line.
|
||||
;; `./bin/mage kondo-ratchets-shrink` lowers budgets unless `--seed` explicitly adds or raises one.
|
||||
;; The workflow runs it on master, so feature branches do not need to record reductions.
|
||||
;; Raising or adding a budget (`--seed` for inline ignores, a manual edit otherwise) or widening the
|
||||
;; exemptions must be explained in the PR.
|
||||
;; :all is the vector-less ignore form, which suppresses every linter on the next form.
|
||||
{:ignore-counts {:aliased-namespace-symbol 3
|
||||
:case-symbol-test 1
|
||||
:clojure-lsp/unused-public-var 9
|
||||
:consistent-alias 2
|
||||
:def-fn 1
|
||||
:deprecated-namespace 95
|
||||
:deprecated-var 95
|
||||
:discouraged-java-method 4
|
||||
:discouraged-namespace 69
|
||||
:discouraged-var 130
|
||||
:duplicate-require 1
|
||||
:equals-true 1
|
||||
:main-without-gen-class 1
|
||||
:metabase/disallow-hardcoded-driver-names-in-tests 34
|
||||
:metabase/discourage-millis-duration 1
|
||||
:metabase/i-like-making-cams-eyes-bleed-with-horrifically-long-tests 13
|
||||
:metabase/modules 24
|
||||
:metabase/namespace-name 4
|
||||
:metabase/prefer-with-dynamic-fn-redefs :unlimited
|
||||
:metabase/test-helpers-use-non-thread-safe-functions 18
|
||||
:metabase/validate-defendpoint-has-response-schema 427
|
||||
:metabase/validate-defendpoint-query-params-use-kebab-case 48
|
||||
:metabase/validate-defendpoint-route-uses-kebab-case 44
|
||||
:metabase/validate-deftest 9
|
||||
:missing-docstring 12
|
||||
:missing-protocol-method 3
|
||||
:non-arg-vec-return-type-hint 1
|
||||
:reduce-without-init 3
|
||||
:redundant-do 1
|
||||
:syntax 2
|
||||
:type-mismatch 8
|
||||
:uninitialized-var 1
|
||||
:unquote-not-syntax-quoted 1
|
||||
:unresolved-namespace 3
|
||||
:unresolved-symbol 11
|
||||
:unsorted-required-namespaces :unlimited
|
||||
:unused-alias 2
|
||||
:unused-binding 9
|
||||
:unused-excluded-var 2
|
||||
:unused-namespace 2
|
||||
:unused-private-var 8
|
||||
:unused-value 1}
|
||||
:config-counts {:constant-condition 1
|
||||
:deprecated-namespace 7
|
||||
:deprecated-var 16
|
||||
:discouraged-namespace 14
|
||||
:discouraged-var 14
|
||||
:inline-def 1
|
||||
:main-without-gen-class 1
|
||||
:metabase/defsetting-must-specify-export 1
|
||||
:metabase/defsetting-namespace 1
|
||||
:metabase/prefer-with-dynamic-fn-redefs 4
|
||||
:metabase/t2-query-namespace 2
|
||||
:metabase/table-or-field-query 1
|
||||
:metabase/unsafe-app-db-query 1
|
||||
:missing-docstring 1
|
||||
:private-call 1
|
||||
:redundant-ignore 1
|
||||
:redundant-primitive-coercion 1
|
||||
:refer-all 1
|
||||
:type-mismatch 1
|
||||
:unexpected-recur 1
|
||||
:unresolved-symbol 15
|
||||
:unresolved-var 1
|
||||
:unused-referred-var 4}
|
||||
:comment-exempt #{:deprecated-namespace
|
||||
:discouraged-namespace
|
||||
:discouraged-var
|
||||
:metabase/modules
|
||||
:metabase/validate-defendpoint-has-response-schema
|
||||
:metabase/validate-defendpoint-query-params-use-kebab-case
|
||||
:metabase/validate-defendpoint-route-uses-kebab-case}}
|
||||
@@ -71,28 +71,10 @@
|
||||
(some-> (ns-form-node->ns-symb-node ns-form-node)
|
||||
hooks/sexpr))
|
||||
|
||||
(defn- lint-test-parallel-metadata [ns-form-node]
|
||||
(when-let [ns-symb-node (ns-form-node->ns-symb-node ns-form-node)]
|
||||
(let [ns-metadata (meta (hooks/sexpr ns-symb-node))]
|
||||
(cond
|
||||
(:synchronous ns-metadata)
|
||||
(hooks/reg-finding!
|
||||
(assoc (meta ns-symb-node)
|
||||
:message (str "Use `^:synchronized` to mark this namespace explicitly non-parallel; "
|
||||
"`^:synchronous` is ignored by Hawk. [:metabase/validate-deftest]")
|
||||
:type :metabase/validate-deftest))
|
||||
|
||||
(and (contains? ns-metadata :parallel)
|
||||
(false? (:parallel ns-metadata)))
|
||||
(hooks/reg-finding!
|
||||
(assoc (meta ns-symb-node)
|
||||
:message "Use `^:synchronized` instead of `^{:parallel false}`. [:metabase/validate-deftest]"
|
||||
:type :metabase/validate-deftest))))))
|
||||
|
||||
(defn- lint-modules [ns-form-node config]
|
||||
(let [ns-symb (ns-form-node->ns-symb ns-form-node)]
|
||||
(when-not (modules/ignored-namespace? config ns-symb)
|
||||
(when-let [current-module (modules/module config ns-symb)]
|
||||
(when-let [current-module (modules/module ns-symb)]
|
||||
(let [required-namespace-symb-nodes (-> ns-form-node
|
||||
ns-form-node->require-node
|
||||
require-node->namespace-symb-nodes)]
|
||||
@@ -110,11 +92,11 @@
|
||||
(when-not (contains? (hooks.common/ignored-linters ns-symb-node) :metabase/namespace-name)
|
||||
(let [parts (str/split (str (hooks/sexpr ns-symb-node)) #"\.")]
|
||||
(when (and (<= (count parts) 2)
|
||||
(#{"metabase" "metabase-enterprise" "metabase-module"} (first parts))
|
||||
(#{"metabase" "metabase-enterprise"} (first parts))
|
||||
;; exclude test namespaces for now
|
||||
(not (str/ends-with? (last parts) "test")))
|
||||
(hooks/reg-finding! (assoc (meta ns-symb-node)
|
||||
:message "Metabase namespaces should have the form metabase[-enterprise|-module].<module>.* [:metabase/namespace-name]"
|
||||
:message "Metabase namespaces should have the form metabase[-enterprise].<module>.* [:metabase/namespace-name]"
|
||||
:type :metabase/namespace-name)))))))
|
||||
|
||||
(defn- ns-form-node->import-node [ns-form-node]
|
||||
@@ -158,7 +140,6 @@
|
||||
lint-requires-on-new-lines
|
||||
(lint-modules (modules/config x))
|
||||
lint-namespace-name
|
||||
lint-test-parallel-metadata
|
||||
lint-jsqlparser-imports)
|
||||
x)
|
||||
|
||||
|
||||
@@ -11,32 +11,15 @@
|
||||
(every? simple-symbol? required-namespaces)]}
|
||||
(when-not (modules/ignored-namespace? config current-ns)
|
||||
;; ignore namespaces outside of the module system
|
||||
(when-let [current-module (modules/module config current-ns)]
|
||||
(when-let [current-module (modules/module current-ns)]
|
||||
(doseq [required-namespace required-namespaces]
|
||||
;; ignore namespaces outside of the module system.
|
||||
(when (modules/module config required-namespace)
|
||||
(when (modules/module required-namespace)
|
||||
(when-let [error (modules/usage-error config current-module required-namespace)]
|
||||
(hooks/reg-finding! (assoc (meta node)
|
||||
:message error
|
||||
:type :metabase/modules))))))))
|
||||
|
||||
(defn- lint-dynamic-require* [node current-ns config]
|
||||
(when (and (not (modules/ignored-namespace? config current-ns))
|
||||
(modules/module config current-ns))
|
||||
(hooks/reg-finding! (assoc (meta node)
|
||||
:message "Module dependency cannot be statically determined from a dynamic require"
|
||||
:type :metabase/modules))))
|
||||
|
||||
(defn- quoted-require? [node]
|
||||
(or (= (hooks/tag node) :quote)
|
||||
(and (= (hooks/tag node) :list)
|
||||
(let [[x] (:children node)]
|
||||
(and (hooks/token-node? x)
|
||||
(= (hooks/sexpr x) 'quote))))))
|
||||
|
||||
(defn- keyword-node? [node]
|
||||
(keyword? (hooks/sexpr node)))
|
||||
|
||||
(defn- unwrap-require [node]
|
||||
(case (hooks/tag node)
|
||||
;; some.namespace of some.namespace/some-var
|
||||
@@ -59,11 +42,7 @@
|
||||
|
||||
;; something like [some.namespace :as whatever]
|
||||
:vector
|
||||
(recur (first (:children node)))
|
||||
|
||||
;; anything else -- e.g. a runtime expression like `(requiring-resolve (:failing-test-var opts))` --
|
||||
;; has no statically-known namespace
|
||||
nil))
|
||||
(recur (first (:children node)))))
|
||||
|
||||
(comment
|
||||
(unwrap-require (-> "(requiring-resolve 'metabase.notification.payload.execute/execute-dashboard)"
|
||||
@@ -72,24 +51,20 @@
|
||||
second)))
|
||||
|
||||
(defn- lint-require* [node current-ns config]
|
||||
(let [[_require & args] (:children node)
|
||||
;; Keywords such as `:reload` and `:reload-all` are options, not namespace dependencies.
|
||||
namespace-nodes (remove keyword-node? args)
|
||||
static-nodes (filter quoted-require? namespace-nodes)
|
||||
required-namespaces (->> (keep unwrap-require static-nodes)
|
||||
(filter simple-symbol?))]
|
||||
(lint* node current-ns config required-namespaces)
|
||||
(when (or (not= (count static-nodes) (count namespace-nodes))
|
||||
(not= (count required-namespaces) (count static-nodes)))
|
||||
(lint-dynamic-require* node current-ns config))))
|
||||
(let [[_require & args] (:children node)
|
||||
required-namespaces (->> (keep unwrap-require args)
|
||||
;; `require` runs through `cc/load-libs` which allows
|
||||
;; #{:as :reload :reload-all :require :use :verbose :refer :as-alias}
|
||||
;; as args. If you use a REPL tool that applies clj-kondo linting (e.g., Clojure MCP)
|
||||
;; you may wish to `:reload` or `:reload-all` a namespace. So we shouldn't assume all
|
||||
;; args are keywords:
|
||||
(remove keyword?))]
|
||||
(lint* node current-ns config required-namespaces)))
|
||||
|
||||
(defn- lint-requiring-resolve* [node current-ns config]
|
||||
(let [[_requiring-resolve symb-node] (:children node)
|
||||
required-namespace (unwrap-require symb-node)]
|
||||
(if (and (quoted-require? symb-node)
|
||||
(simple-symbol? required-namespace))
|
||||
(lint* node current-ns config [required-namespace])
|
||||
(lint-dynamic-require* node current-ns config))))
|
||||
(lint* node current-ns config [required-namespace])))
|
||||
|
||||
(defn lint-require [x]
|
||||
(lint-require* (:node x) (:ns x) (modules/config x))
|
||||
|
||||
@@ -1,72 +0,0 @@
|
||||
(ns hooks.clojure.core.with-redefs
|
||||
(:require
|
||||
[clj-kondo.hooks-api :as hooks]))
|
||||
|
||||
(defn- defn-arity?
|
||||
"Look up `var-sym` in `analysis` (the result of `hooks/ns-analysis`, keyed by language)
|
||||
and return true iff kondo recorded a non-empty arity for it. clj-kondo only emits
|
||||
arities for `defn`-style fns — `defmulti` and plain `def` have neither `:fixed-arities`
|
||||
nor `:varargs-min-arity`, so the presence of either is a clean, dynamic signal that
|
||||
the var is a regular function we can safely nudge.
|
||||
|
||||
We coerce to boolean and `seq`-check `:fixed-arities` so a hypothetical empty set
|
||||
doesn't leak through as truthy. The smoke test in `with-redefs-test` empirically
|
||||
verifies the \"arities iff `defn`\" invariant against a real `clj-kondo` run — if a
|
||||
future kondo release starts emitting arities for `defmulti` it will fail there
|
||||
rather than silently producing wrong nudges."
|
||||
[analysis var-sym]
|
||||
(boolean
|
||||
(some (fn [lang-vars]
|
||||
(when-let [v (get lang-vars var-sym)]
|
||||
(or (seq (:fixed-arities v))
|
||||
(:varargs-min-arity v))))
|
||||
(vals analysis))))
|
||||
|
||||
(defn- safely-nudgeable-lhs?
|
||||
"Is this LHS a regular function (defn) according to kondo's analysis?
|
||||
|
||||
Returns false for:
|
||||
- unresolved symbols (e.g. namespace alias not in scope, ns not yet analysed)
|
||||
- vars defined by `defmulti` (no arity recorded)
|
||||
- vars defined by `def` (no arity recorded)
|
||||
Returns true only when kondo has arity info for the var, which is the case for `defn`
|
||||
and `defn-`. This means the nudge fires only when we're sure the target is a plain
|
||||
function — no manual list of multimethod targets to maintain.
|
||||
|
||||
We deliberately bias toward false (skip the nudge) when uncertain. The cost of a missed
|
||||
nudge is small; the cost of a wrong nudge is a runtime error from
|
||||
`with-dynamic-fn-redefs` refusing to proxy a multimethod."
|
||||
[lhs]
|
||||
(and (hooks/token-node? lhs)
|
||||
(symbol? (hooks/sexpr lhs))
|
||||
(let [resolved (hooks/resolve {:name (hooks/sexpr lhs)})
|
||||
ns-sym (:ns resolved)]
|
||||
(and (symbol? ns-sym) ; not :clj-kondo/unknown-namespace
|
||||
(defn-arity? (hooks/ns-analysis ns-sym) (:name resolved))))))
|
||||
|
||||
(defn lint-with-redefs
|
||||
"Suggest `with-dynamic-fn-redefs` when every LHS is known to be a `defn`-style var.
|
||||
|
||||
We don't gate on the RHS — `with-dynamic-fn-redefs` accepts any `IFn` replacement
|
||||
(fns, keywords, colls), and the LHS check alone is enough to know the form is
|
||||
migratable as a whole. The `every?` keeps it whole-form: mixed bindings that include
|
||||
a non-defn LHS (defmulti, plain `def`, unresolved) can't be split usefully — the
|
||||
leftover `with-redefs` still does a global root-swap, so the form remains thread-unsafe.
|
||||
|
||||
The LHS check uses kondo's own analysis cache rather than a hand-maintained list of
|
||||
multimethod names — adding a new `defmulti` doesn't require touching this hook."
|
||||
[{:keys [node]}]
|
||||
(let [[_with-redefs bindings-vec] (:children node)]
|
||||
(when (hooks/vector-node? bindings-vec)
|
||||
(let [pairs (partition-all 2 (:children bindings-vec))]
|
||||
(when (and (seq pairs)
|
||||
(every? (fn [[lhs rhs]]
|
||||
(and rhs (safely-nudgeable-lhs? lhs)))
|
||||
pairs))
|
||||
(hooks/reg-finding!
|
||||
(assoc (meta node)
|
||||
:message (str "Every binding here redefines a defn-style var — prefer "
|
||||
"metabase.test/with-dynamic-fn-redefs for thread-safe "
|
||||
"redefs. [:metabase/prefer-with-dynamic-fn-redefs]")
|
||||
:type :metabase/prefer-with-dynamic-fn-redefs))))))
|
||||
{:node node})
|
||||
@@ -21,19 +21,15 @@
|
||||
(error! form (format "destructive functions like %s are not allowed inside a ^:parallel test or test fixture. If this should be allowed, add it to the whitelist in the Kondo config file [:metabase/validate-deftest]"
|
||||
qualified-symbol)))))
|
||||
(walk [form]
|
||||
(when-not (contains? (hooks.common/ignored-linters form) :metabase/validate-deftest)
|
||||
(f form)
|
||||
(doseq [child (:children form)]
|
||||
(walk child))))]
|
||||
(f form)
|
||||
(doseq [child (:children form)]
|
||||
(walk child)))]
|
||||
(walk form)))
|
||||
|
||||
(defn- deftest-check-parallel
|
||||
"1. Check if test is marked `^:parallel` / `^:synchronized` correctly.
|
||||
2. Make sure disallowed forms are not used in ^:parallel tests.
|
||||
3. Disallow `^:parallel` tests in namespaces marked `^:synchronized` --
|
||||
the ns marker is the author's blanket promise that nothing inside runs
|
||||
in parallel, so an individual `^:parallel` deftest contradicts it."
|
||||
[{[_ test-name & body] :children, :as _node} ns-sym config]
|
||||
"1. Check if test is marked ^:parallel / ^:synchronized correctly
|
||||
2. Make sure disallowed forms are not used in ^:parallel tests"
|
||||
[{[_ test-name & body] :children, :as _node} config]
|
||||
(let [test-metadata (:meta test-name)
|
||||
metadata-sexprs (map hooks/sexpr test-metadata)
|
||||
combined-metadata (transduce
|
||||
@@ -44,48 +40,19 @@
|
||||
(completing merge)
|
||||
{}
|
||||
metadata-sexprs)
|
||||
parallel? (true? (:parallel combined-metadata))
|
||||
synchronized? (true? (:synchronized combined-metadata))
|
||||
explicitly-nonparallel? (or synchronized?
|
||||
(and (contains? combined-metadata :parallel)
|
||||
(false? (:parallel combined-metadata))))
|
||||
ns-metadata (meta ns-sym)
|
||||
ns-explicitly-nonparallel? (or (true? (:synchronized ns-metadata))
|
||||
(and (contains? ns-metadata :parallel)
|
||||
(false? (:parallel ns-metadata))))]
|
||||
(when (and (contains? combined-metadata :parallel)
|
||||
(false? (:parallel combined-metadata)))
|
||||
(hooks/reg-finding! (assoc (meta test-name)
|
||||
:message (str "Use `^:synchronized` instead of `^{:parallel false}`. "
|
||||
"[:metabase/validate-deftest]")
|
||||
:type :metabase/validate-deftest)))
|
||||
(when (:sequential combined-metadata)
|
||||
(hooks/reg-finding! (assoc (meta test-name)
|
||||
:message (str "Use `^:synchronized` to mark this test explicitly non-parallel; "
|
||||
"`^:sequential` is ignored by Hawk. "
|
||||
"[:metabase/validate-deftest]")
|
||||
:type :metabase/validate-deftest)))
|
||||
parallel? (:parallel combined-metadata)
|
||||
synchronized? (:synchronized combined-metadata)]
|
||||
(when (and parallel? synchronized?)
|
||||
(hooks/reg-finding! (assoc (meta test-name)
|
||||
:message "Test should not be marked both `^:parallel` and `^:synchronized`"
|
||||
:message "Test should not be marked both ^:parallel and ^:synchronized"
|
||||
:type :metabase/validate-deftest)))
|
||||
(when (and parallel? ns-explicitly-nonparallel?)
|
||||
;; only when the custom `:metabase/deftest-not-marked-parallel-or-synchronized` is enabled: complain if tests are
|
||||
;; not explicitly marked `^:parallel` or `^:synchronized`. This is mostly to encourage people to mark everything
|
||||
;; `^:parallel` in places like `metabase.lib` tests unless there is a really good reason not to.
|
||||
(when-not (or parallel? synchronized?)
|
||||
(hooks/reg-finding!
|
||||
(assoc (meta test-name)
|
||||
:message (str "Test should not be marked `^:parallel` in a namespace marked "
|
||||
"`^:synchronized` -- "
|
||||
"the ns marker promises no parallel execution. "
|
||||
"Drop `^:parallel` from the deftest, or drop `^:synchronized` from the ns. "
|
||||
"[:metabase/validate-deftest]")
|
||||
:type :metabase/validate-deftest)))
|
||||
;; Only when the custom `:metabase/deftest-not-marked-parallel-or-synchronized` is enabled:
|
||||
;; complain if tests are not explicitly marked `^:parallel` or `^:synchronized`. This is
|
||||
;; mostly to encourage people to mark everything `^:parallel` in places like `metabase.lib`
|
||||
;; tests unless there is a really good reason not to.
|
||||
(when-not (or parallel? explicitly-nonparallel?)
|
||||
(hooks/reg-finding!
|
||||
(assoc (meta test-name)
|
||||
:message "Test should be marked either `^:parallel` or `^:synchronized`"
|
||||
:message "Test should be marked either ^:parallel or ^:synchronized"
|
||||
:type :metabase/deftest-not-marked-parallel-or-synchronized)))
|
||||
(when parallel?
|
||||
(doseq [form body]
|
||||
@@ -104,36 +71,6 @@
|
||||
"Split it up into smaller tests! 🥰")
|
||||
:type :metabase/i-like-making-cams-eyes-bleed-with-horrifically-long-tests)))))))
|
||||
|
||||
(defn- every-top-level-form-in-deftest-is-testing? [{[_deftest _test-name & forms] :children, :as _node}]
|
||||
(every? (fn [{[first-child] :children, :as form}]
|
||||
(and (hooks/list-node? form)
|
||||
(= (hooks.common/node->qualified-symbol first-child)
|
||||
'clojure.test/testing)))
|
||||
forms))
|
||||
|
||||
(defn- num-top-level-forms-in-deftest [{[_deftest _test-name & forms] :children, :as _node}]
|
||||
(count forms))
|
||||
|
||||
(defn- deftest-check-should-not-be-multiple-separate-tests
|
||||
"Warn on long tests (tests over `min-length`) that look like
|
||||
|
||||
(deftest my-test
|
||||
(testing ...)
|
||||
(testing ...)
|
||||
(testing ...))"
|
||||
[node {:keys [min-length], :as _config}]
|
||||
(let [{:keys [row end-row]} (meta node)
|
||||
test-length (- end-row row)]
|
||||
(when (and min-length
|
||||
(> test-length min-length)
|
||||
(every-top-level-form-in-deftest-is-testing? node)
|
||||
(> (num-top-level-forms-in-deftest node) 1))
|
||||
(hooks/reg-finding!
|
||||
(assoc (meta node)
|
||||
:message (str "This test looks like it contains several logically separate testing forms... break it"
|
||||
" out into separate deftests to make it easier to test and debug")
|
||||
:type :metabase/validate-deftest-logically-separate-tests)))))
|
||||
|
||||
(defn- ignore? [node error-type]
|
||||
(contains? (hooks.common/ignored-linters node) error-type))
|
||||
|
||||
@@ -171,11 +108,8 @@
|
||||
(str/starts-with? (name ns-symb) prefix))
|
||||
["build-drivers."
|
||||
"build."
|
||||
"hooks." ; clj-kondo hook tests, outside the module system
|
||||
"i18n." ; bin/i18n
|
||||
"lint-migrations-file-test"
|
||||
"load-namespaces." ; bin/load-namespaces
|
||||
"mage."
|
||||
"main-test" ; bin/release-list
|
||||
"metabase.deps-edn-test"
|
||||
"metabase.driver."
|
||||
@@ -194,21 +128,20 @@
|
||||
(not (contains? @warned-namespaces ns-symb)))
|
||||
(swap! warned-namespaces conj ns-symb)
|
||||
(let [known-modules (set (keys (:metabase/modules (modules/config input))))
|
||||
current-module (modules/module (modules/config input) ns-symb)]
|
||||
current-module (modules/module ns-symb)]
|
||||
(when (or (not current-module)
|
||||
(not (contains? known-modules current-module)))
|
||||
(hooks/reg-finding! (assoc (meta node)
|
||||
:message (format "All tests must live in a known module; %s is not a known module" (pr-str current-module))
|
||||
:type :metabase/tests-must-live-in-known-modules)))))))
|
||||
|
||||
(defn deftest [{:keys [node cljc lang config], ns-sym :ns, :as input}]
|
||||
(defn deftest [{:keys [node cljc lang config], :as input}]
|
||||
;; run [[deftest-check-parallel]] only once... if this is a `.cljc` file only run it for the `:clj` analysis, no point
|
||||
;; in running it twice.
|
||||
(when (or (not cljc)
|
||||
(= lang :clj))
|
||||
(deftest-check-parallel node ns-sym (get-in config [:linters :metabase/validate-deftest])))
|
||||
(deftest-check-parallel node (get-in config [:linters :metabase/validate-deftest])))
|
||||
(deftest-check-not-horrifically-long node (get-in config [:linters :metabase/i-like-making-cams-eyes-bleed-with-horrifically-long-tests]))
|
||||
(deftest-check-should-not-be-multiple-separate-tests node (get-in config [:linters :metabase/validate-deftest-logically-separate-tests]))
|
||||
(deftest-check-no-driver-keywords node (get-in config [:linters :metabase/disallow-hardcoded-driver-names-in-tests]))
|
||||
(deftest-check-in-valid-module input)
|
||||
input)
|
||||
@@ -247,40 +180,11 @@
|
||||
|
||||
nil)))))))
|
||||
|
||||
(defn is [{:keys [node lang], :as input}]
|
||||
(defn is [{:keys [node lang]}]
|
||||
(when (= lang :cljs)
|
||||
(warn-about-missing-test-expr-requires-in-cljs node))
|
||||
input)
|
||||
{:node node})
|
||||
|
||||
(defn use-fixtures
|
||||
"Flag `:parallel/unsafe` forms inside fixtures. Skipped when the surrounding
|
||||
namespace is marked `^:synchronized` -- the ns marker is the author's
|
||||
explicit opt-in to a single-threaded test ns where destructive setup is
|
||||
safe. Without that opt-in, kondo can't know whether sibling deftests will
|
||||
be `^:parallel`, so the conservative default is to flag.
|
||||
|
||||
Sister to [[deftest-check-parallel]]: the deftest hook rejects `^:parallel`
|
||||
tests inside a `^:synchronized` ns, so the two checks together form a
|
||||
coherent opt-in: either the whole ns is explicitly non-parallel (fixtures can be
|
||||
anything; no parallel tests allowed) or it isn't (fixtures must be
|
||||
parallel-safe)."
|
||||
[{:keys [node config], ns-sym :ns, :as input}]
|
||||
(when-not (or (true? (:synchronized (meta ns-sym)))
|
||||
(and (contains? (meta ns-sym) :parallel)
|
||||
(false? (:parallel (meta ns-sym)))))
|
||||
(let [linter-config (get-in config [:linters :metabase/validate-deftest])]
|
||||
(doseq [form (rest (:children node))]
|
||||
(warn-about-disallowed-parallel-forms form linter-config))))
|
||||
input)
|
||||
|
||||
(defn testing
|
||||
"Check that we don't have an empty `testing` form like
|
||||
|
||||
(testing \"message\")"
|
||||
[{{[_testing _message & body] :children, :as node} :node, :as input}]
|
||||
(when (empty? body)
|
||||
(hooks/reg-finding!
|
||||
(assoc (meta node)
|
||||
:message "A `testing` form that doesn't wrap anything doesn't do anything"
|
||||
:type :metabase/check-testing-not-empty)))
|
||||
input)
|
||||
(defn use-fixtures [{:keys [node config]}]
|
||||
(warn-about-disallowed-parallel-forms node config)
|
||||
{:node node})
|
||||
|
||||
@@ -500,8 +500,8 @@
|
||||
(and resolved (:ns resolved))
|
||||
(symbol (name (:ns resolved)) (name (:name resolved)))
|
||||
|
||||
;; if it wasn't resolved but is still qualified it's probably using the full namespace name rather than an
|
||||
;; alias.
|
||||
;; if it wasn't resolved but is still qualified it's probably using the full namespace name rather than an
|
||||
;; alias.
|
||||
(qualified-symbol? sexpr)
|
||||
sexpr)))))
|
||||
;; some symbols like `*count/Integer` aren't resolvable.
|
||||
|
||||
@@ -1,8 +1,4 @@
|
||||
(ns hooks.common.modules
|
||||
"Resolves namespaces to modules and enforces module boundaries.
|
||||
|
||||
The Kondo linter, config generator, and CI tooling share this namespace so
|
||||
they apply the same rules."
|
||||
(:require
|
||||
[clojure.string :as str]))
|
||||
|
||||
@@ -16,203 +12,30 @@
|
||||
(merge (get-in config [:linters :metabase/modules])
|
||||
(select-keys config [:metabase/modules])))
|
||||
|
||||
;;;; Module tree. These functions take the `:metabase/modules` map, or the prefix map built from it.
|
||||
;;;;
|
||||
;;;; Dots express nesting: `lib.schema` is a child of `lib`. A declared
|
||||
;;;; `enterprise/X` module is also a child of OSS module `X`. A namespace
|
||||
;;;; belongs to the module with the most specific matching `:ns-prefix`.
|
||||
;;;;
|
||||
;;;; A `module/X` module is a plugin built from `modules/X`, shipped as its own jar and possibly absent at
|
||||
;;;; runtime, so only its own subtree may use it.
|
||||
|
||||
;; TODO (Chris 2026-09-17) -- Driver plugins are not modules yet. The plan is `module/driver.X`, implying
|
||||
;; `metabase.driver.X` in `modules/drivers/X`, so drivers and the third-party plugins built like them keep
|
||||
;; their namespaces. With each driver a module, "should this change run driver tests" can come from the module
|
||||
;; graph instead of the override list in `mage.modules`. The drivers' `metabase.test.data.X` helpers will need
|
||||
;; resolving by directory, or a rename.
|
||||
|
||||
(defn plugin-module?
|
||||
"Whether `module` is a plugin, built from `modules/<name>` and shipped as its own jar."
|
||||
[module]
|
||||
(= "module" (namespace module)))
|
||||
|
||||
(defn default-ns-prefix
|
||||
"Namespace prefix a module owns unless it sets `:ns-prefix`: `metabase.lib.schema` for `lib.schema`,
|
||||
`metabase-enterprise.foo` for `enterprise/foo`, `metabase-module.embedder` for `module/embedder`."
|
||||
[module]
|
||||
(str (case (namespace module)
|
||||
"enterprise" "metabase-enterprise."
|
||||
"module" "metabase-module."
|
||||
"metabase.")
|
||||
(name module)))
|
||||
|
||||
(defn module-ns-prefix
|
||||
"Namespace prefix `module` owns: its `:ns-prefix`, else [[default-ns-prefix]]."
|
||||
[modules module]
|
||||
(or (get-in modules [module :ns-prefix])
|
||||
(default-ns-prefix module)))
|
||||
|
||||
(defn module-directory
|
||||
"Repo-relative directory holding `module`'s namespaces in `tree`, `\"src\"` or `\"test\"`, e.g.
|
||||
`src/metabase/lib/schema`, `enterprise/backend/test/metabase_enterprise/sso` or
|
||||
`modules/embedder/src/metabase_module/embedder`."
|
||||
[modules module tree]
|
||||
(let [ns-prefix (module-ns-prefix modules module)]
|
||||
(str (cond
|
||||
(plugin-module? module)
|
||||
(str "modules/" (first (str/split (name module) #"\.")) "/")
|
||||
|
||||
(str/starts-with? ns-prefix "metabase-enterprise.")
|
||||
"enterprise/backend/")
|
||||
tree "/"
|
||||
(-> ns-prefix (str/replace "." "/") (str/replace "-" "_")))))
|
||||
|
||||
(defn build-prefix->module
|
||||
"Map each declared namespace prefix to its module, for [[declared-module]]."
|
||||
[modules]
|
||||
(into {}
|
||||
(map (fn [module] [(module-ns-prefix modules module) module]))
|
||||
(keys modules)))
|
||||
|
||||
(defn declared-module
|
||||
"Resolve `ns-symb` to its declared module, or `nil`.
|
||||
|
||||
Chooses the most specific dotted prefix and ignores a trailing `-test`."
|
||||
[prefix->module ns-symb]
|
||||
(loop [candidate (str/replace (str ns-symb) #"-test$" "")]
|
||||
(or (get prefix->module candidate)
|
||||
(when-let [dot (str/last-index-of candidate ".")]
|
||||
(recur (subs candidate 0 dot))))))
|
||||
|
||||
(defn resolve-module
|
||||
(defn module
|
||||
"E.g.
|
||||
|
||||
(resolve-module prefix->module 'metabase.qp.middleware.wow) => 'qp
|
||||
(resolve-module prefix->module 'metabase-enterprise.whatever.core) => enterprise/whatever
|
||||
(resolve-module prefix->module 'metabase-module.whatever.core) => module/whatever
|
||||
|
||||
An unmatched Metabase namespace resolves to its first segment so the linter can report an undeclared module."
|
||||
[prefix->module ns-symb]
|
||||
(module 'metabase.qp.middleware.wow) => 'qp
|
||||
(module 'metabase-enterprise.whatever.core) => enterprise/whatever"
|
||||
[ns-symb]
|
||||
{:pre [(simple-symbol? ns-symb)]}
|
||||
;; treat something like `metabase.driver-test` (for a module that hasn't fully been updated to use `.core`
|
||||
;; namespaces) as being in the `driver` module
|
||||
(let [ns-symb (if (str/ends-with? (name ns-symb) "-test")
|
||||
(symbol (str/replace (name ns-symb) #"-test$" ""))
|
||||
ns-symb)]
|
||||
(or (declared-module prefix->module ns-symb)
|
||||
(some->> (re-find #"^metabase-enterprise\.([^.]+)" (str ns-symb))
|
||||
(or (some->> (re-find #"^metabase-enterprise\.([^.]+)" (str ns-symb))
|
||||
second
|
||||
(symbol "enterprise"))
|
||||
(some->> (re-find #"^metabase-module\.([^.]+)" (str ns-symb))
|
||||
second
|
||||
(symbol "module"))
|
||||
(some-> (re-find #"^metabase\.([^.]+)" (str ns-symb))
|
||||
second
|
||||
symbol))))
|
||||
|
||||
(defn parent-module
|
||||
"The direct parent of `module`, or `nil` when it is top-level."
|
||||
[modules module]
|
||||
(let [module-name (name module)]
|
||||
(if-let [dot (str/last-index-of module-name ".")]
|
||||
(symbol (namespace module) (subs module-name 0 dot))
|
||||
(when (= "enterprise" (namespace module))
|
||||
(let [oss (symbol module-name)]
|
||||
(when (contains? modules oss)
|
||||
oss))))))
|
||||
|
||||
(defn descendant-of?
|
||||
"Whether `module` is `ancestor` or one of its descendants."
|
||||
[modules module ancestor]
|
||||
(boolean (some #{ancestor} (take-while some? (iterate #(parent-module modules %) module)))))
|
||||
|
||||
(defn module-team
|
||||
"Team owning `module`: its own `:team`, else its nearest ancestor's."
|
||||
[modules module]
|
||||
(some #(get-in modules [% :team])
|
||||
(take-while some? (iterate #(parent-module modules %) module))))
|
||||
|
||||
(defn- rest-module? [module]
|
||||
(re-find #"[.-]rest$" (str module)))
|
||||
|
||||
(defn- exports-child?
|
||||
[modules parent child]
|
||||
(or (contains? (set (get-in modules [parent :module-exports])) child)
|
||||
;; A `.rest` child is its parent's public HTTP surface.
|
||||
(rest-module? child)
|
||||
;; OSS module `X` implicitly exports its `enterprise/X` companion.
|
||||
(and (not= (namespace parent) (namespace child))
|
||||
(contains? modules child))))
|
||||
|
||||
(defn- blocking-export
|
||||
"The first missing export from `module` toward the root.
|
||||
|
||||
Returns `{:ancestor parent, :child child}`, or `nil` when every link is
|
||||
exported."
|
||||
[modules module]
|
||||
(loop [child module]
|
||||
(when-let [ancestor (parent-module modules child)]
|
||||
(if (exports-child? modules ancestor child)
|
||||
(recur ancestor)
|
||||
{:ancestor ancestor, :child child}))))
|
||||
|
||||
(defn plugin-root
|
||||
"The top-level plugin `module` belongs to, e.g. `module/embedder` for `module/embedder.model`.
|
||||
Returns `nil` when `module` is no plugin."
|
||||
[modules module]
|
||||
(when (plugin-module? module)
|
||||
(last (take-while some? (iterate #(parent-module modules %) module)))))
|
||||
|
||||
(defn- plugin-error
|
||||
"Explain why `caller` may not use plugin `target`, or return `nil` when `target` is no plugin or `caller`
|
||||
sits inside the plugin."
|
||||
[modules caller target]
|
||||
(when-let [plugin (plugin-root modules target)]
|
||||
(when-not (descendant-of? modules caller plugin)
|
||||
(format (str "Module %s is a plugin that may not be installed; %s may not use it. Load it through "
|
||||
"metabase.plugins instead. [:metabase/modules %s :uses]")
|
||||
target caller caller))))
|
||||
|
||||
(defn namability-error
|
||||
"Explain why `caller` may not refer to `target` in `:uses`, or return `nil` if it may.
|
||||
|
||||
A nested module is private to the nearest ancestor with a missing export.
|
||||
Exporting every link makes it public.
|
||||
A plugin is private to itself, since it may be missing at runtime."
|
||||
[modules caller target]
|
||||
(or (plugin-error modules caller target)
|
||||
(when-let [{:keys [ancestor child]} (blocking-export modules target)]
|
||||
(when-not (descendant-of? modules caller ancestor)
|
||||
(format (str "Module %s is nested and not exported by its ancestors; %s may not use it. Add %s to "
|
||||
"%s's :module-exports, or move the caller into the %s subtree. "
|
||||
"[:metabase/modules %s :module-exports]")
|
||||
target caller child ancestor ancestor ancestor)))))
|
||||
|
||||
;;;; Lint rules. These functions take the full linter config.
|
||||
|
||||
(defonce ^:private prefix->module-cache
|
||||
(atom nil))
|
||||
|
||||
(defn- prefix->module
|
||||
"[[build-prefix->module]] for `config`, cached because Kondo hands every hook call the same `:metabase/modules` map."
|
||||
[config]
|
||||
(let [modules (:metabase/modules config)
|
||||
cached @prefix->module-cache]
|
||||
(if (identical? modules (:modules cached))
|
||||
(:prefix->module cached)
|
||||
(:prefix->module (reset! prefix->module-cache {:modules modules
|
||||
:prefix->module (build-prefix->module modules)})))))
|
||||
|
||||
(defn module
|
||||
"Resolve `ns-symb` to its module under `config`; see [[resolve-module]]."
|
||||
[config ns-symb]
|
||||
(resolve-module (prefix->module config) ns-symb))
|
||||
|
||||
(defn module-api-namespaces
|
||||
(defn- module-api-namespaces
|
||||
"Set of API namespace symbols for a given module. `:any` means you can use anything, there are no API namespaces for
|
||||
this module (yet). If unspecified, the defaults are `<prefix>.api`, `<prefix>.core`, and `<prefix>.init`."
|
||||
[modules module]
|
||||
(let [module-config (get-in modules [module :api])]
|
||||
this module (yet). If unspecified, the default is just the `<module>.core` namespace."
|
||||
[config module]
|
||||
(let [module-config (get-in config [:metabase/modules module :api])]
|
||||
(cond
|
||||
(= module-config :any)
|
||||
nil
|
||||
@@ -221,7 +44,9 @@
|
||||
module-config
|
||||
|
||||
:else
|
||||
(let [ns-prefix (module-ns-prefix modules module)]
|
||||
(let [ns-prefix (if (= (namespace module) "enterprise")
|
||||
(str "metabase-enterprise." (name module))
|
||||
(name module))]
|
||||
#{(symbol (str ns-prefix ".api"))
|
||||
(symbol (str ns-prefix ".core"))
|
||||
(symbol (str ns-prefix ".init"))}))))
|
||||
@@ -243,14 +68,15 @@
|
||||
(contains? (set allowed-modules) required-module))))
|
||||
|
||||
(defn- allowed-module-namespace? [config current-module ns-symb]
|
||||
(let [module (module config ns-symb)
|
||||
module-api-namespaces (module-api-namespaces (:metabase/modules config) module)
|
||||
(let [module (module ns-symb)
|
||||
module-api-namespaces (module-api-namespaces config module)
|
||||
module-friends (module-friends config module)]
|
||||
(or (nil? module-api-namespaces)
|
||||
(or (empty? module-api-namespaces)
|
||||
(contains? module-api-namespaces ns-symb)
|
||||
(contains? module-friends current-module)
|
||||
;; a child may use its ancestors' internals; a parent still goes through its child's `:api`
|
||||
(descendant-of? (:metabase/modules config) current-module module))))
|
||||
(contains? module-friends current-module))))
|
||||
|
||||
(defn- rest-module? [module]
|
||||
(str/ends-with? module "-rest"))
|
||||
|
||||
(defn- routes-module? [module]
|
||||
(str/ends-with? module "-routes"))
|
||||
@@ -263,7 +89,7 @@
|
||||
error type if there is one, otherwise `nil` if there are no errors."
|
||||
[config current-module required-namespace]
|
||||
;; ignore stuff not in a module i.e. non-Metabase stuff.
|
||||
(when-let [required-module (module config required-namespace)]
|
||||
(when-let [required-module (module required-namespace)]
|
||||
(when-not (= current-module required-module)
|
||||
(cond
|
||||
(not (allowed-module? config current-module required-module))
|
||||
@@ -287,8 +113,4 @@
|
||||
required-module
|
||||
current-module
|
||||
required-module
|
||||
(symbol (str/replace required-module #"[.-]rest$" "")))
|
||||
|
||||
;; Config tests check explicit `:uses`; wildcard callers are checked here.
|
||||
(= :any (allowed-modules config current-module))
|
||||
(namability-error (:metabase/modules config) current-module required-module)))))
|
||||
(symbol (str/replace required-module #"-rest$" "")))))))
|
||||
|
||||
@@ -1,375 +0,0 @@
|
||||
(ns hooks.metabase.agent-message
|
||||
"Hooks linting agent-facing prose built with `metabase.mcp.v2.message/msg` (`:metabase/agent-message-lines`) and the
|
||||
exits and helpers that carry it to the agent (`:metabase/agent-message-exit`). Each hook checks only when its
|
||||
linter's level is configured and not `:off`, and returns its input unchanged."
|
||||
(:require
|
||||
[clj-kondo.hooks-api :as hooks]
|
||||
[clojure.string :as str]))
|
||||
|
||||
(defn- level-on?
|
||||
[config linter-key]
|
||||
(not= :off (get-in config [:linters linter-key :level] :off)))
|
||||
|
||||
(defn- call-name
|
||||
"The symbol at the head of list `node`, or nil."
|
||||
[node]
|
||||
(when (hooks/list-node? node)
|
||||
(let [head (first (:children node))]
|
||||
(when (hooks/token-node? head)
|
||||
(let [v (hooks/sexpr head)]
|
||||
(when (symbol? v) v))))))
|
||||
|
||||
(defn- calls?
|
||||
"Whether `node` is a call to a var named `var-name` in one of `ns-names` (nil for an unqualified call)."
|
||||
[node ns-names var-name]
|
||||
(when-let [sym (call-name node)]
|
||||
(and (= var-name (name sym))
|
||||
(contains? ns-names (namespace sym)))))
|
||||
|
||||
(def ^:private core-ns #{nil "clojure.core"})
|
||||
(def ^:private string-ns #{"str" "string" "clojure.string"})
|
||||
|
||||
(defn- literal-string
|
||||
"The string value of `node` when it is a string literal or a `str` call over string literals, else nil."
|
||||
[node]
|
||||
(cond
|
||||
(hooks/string-node? node)
|
||||
(hooks/sexpr node)
|
||||
|
||||
(and (calls? node core-ns "str")
|
||||
(every? hooks/string-node? (rest (:children node))))
|
||||
(apply str (map hooks/sexpr (rest (:children node))))))
|
||||
|
||||
(defn- digits->long
|
||||
"The value of a string of decimal digits."
|
||||
[s]
|
||||
;; Kondo's hook interpreter has no `parse-long`.
|
||||
(reduce (fn [n c] (+ (* 10 n) (- (int c) (int \0)))) 0 s))
|
||||
|
||||
(def ^:private format-specifier
|
||||
"A `java.util.Formatter` specifier; groups are explicit index, flags, width, precision, and conversion. The index is
|
||||
capped at nine digits, so a longer one reads as malformed rather than a number too large to work with."
|
||||
#"%(?:(\d{1,9})\$)?([-#+ 0,(<]*)(\d+)?(?:\.(\d+))?([tT]?[a-zA-Z%])")
|
||||
|
||||
(defn- consumed-args
|
||||
"`[index conversion]` for each argument-consuming specifier in `fmt`, with zero-based argument indexes. An index of
|
||||
-1 marks a reference no argument can satisfy: an explicit `0$`, or a `<` with no specifier before it to repeat."
|
||||
[fmt]
|
||||
(loop [[[_ explicit flags _width _precision conversion] & more] (re-seq format-specifier fmt)
|
||||
next-index 0
|
||||
last-index nil
|
||||
acc []]
|
||||
(if-not conversion
|
||||
acc
|
||||
(let [consumes? (not (contains? #{"%" "n"} conversion))
|
||||
index (cond
|
||||
(not consumes?) nil
|
||||
explicit (dec (digits->long explicit))
|
||||
(str/includes? flags "<") (or last-index -1)
|
||||
:else next-index)
|
||||
implicit? (and consumes? (not explicit) (not (str/includes? flags "<")))]
|
||||
(recur more
|
||||
(cond-> next-index implicit? inc)
|
||||
(or index last-index)
|
||||
(cond-> acc
|
||||
index (conj [index conversion])))))))
|
||||
|
||||
(def ^:private msg-linter :metabase/agent-message-lines)
|
||||
|
||||
(defn- reg-msg-finding!
|
||||
[node message]
|
||||
(hooks/reg-finding! (assoc (meta node) :message message :type msg-linter)))
|
||||
|
||||
(defn- line-break?
|
||||
"Whether format string `line` contains a line-breaking, control, or private-use character, or a `%n` specifier."
|
||||
[line]
|
||||
;; Private-use characters delimit the argument markers `render` formats with, so a line holding one could fake one.
|
||||
(boolean (or (re-find #"[\p{Cc}\p{Cf}\p{Co}\p{Zl}\p{Zp}]" line)
|
||||
(some #(= "n" (nth % 5)) (re-seq format-specifier line)))))
|
||||
|
||||
(defn- allowed-specifier?
|
||||
"Whether specifier match `specifier` renders its value as `render` passes it: a bare `%%`, a `%s` with no flags but
|
||||
`<`, or a `%d` with any flags and width but no precision. An explicit argument index is allowed on each."
|
||||
[[_ explicit flags width precision conversion]]
|
||||
(case conversion
|
||||
"%" (not (or explicit (seq flags) width precision))
|
||||
"s" (not (or width precision (seq (str/replace flags "<" ""))))
|
||||
"d" (not precision)
|
||||
false))
|
||||
|
||||
(defn- disallowed-specifier
|
||||
"The first specifier in format string `line` that isn't [[allowed-specifier?]], as it is written, or nil."
|
||||
[line]
|
||||
(some #(when-not (allowed-specifier? %) (first %)) (re-seq format-specifier line)))
|
||||
|
||||
(def ^:private valid-conversion
|
||||
"The conversions `java.util.Formatter` accepts, with each date/time conversion's required suffix."
|
||||
#"[bBhHsScCdoxXeEfgGaA%n]|[tT][HIklMSLNpzZsQBbhAaCYyjmdeRTrDFc]")
|
||||
|
||||
(defn- malformed-specifier?
|
||||
"Whether format string `line` has a `%` that doesn't begin a valid `java.util.Formatter` specifier."
|
||||
[line]
|
||||
(let [valid-removed (str/replace line format-specifier
|
||||
(fn [[specifier & groups]]
|
||||
(if (re-matches valid-conversion (last groups)) "" specifier)))]
|
||||
(str/includes? valid-removed "%")))
|
||||
|
||||
(defn- line-problem
|
||||
"The finding message for the first problem with a `msg` line, given its literal string or nil; nil when it has none."
|
||||
[line]
|
||||
(cond
|
||||
(nil? line)
|
||||
(str "Each line of a `msg` must be a string literal or a `str` of string literals; "
|
||||
"pass values as arguments after the vector.")
|
||||
|
||||
(line-break? line)
|
||||
(str "A `msg` line can't contain a line break, `%n`, or a control or private-use character; "
|
||||
"put each line in its own string.")
|
||||
|
||||
(malformed-specifier? line)
|
||||
"A `%` in a `msg` line must begin a format specifier; write a literal percent sign as `%%`."
|
||||
|
||||
:else
|
||||
(when-let [specifier (disallowed-specifier line)]
|
||||
(str "A `msg` line may only use `%s`, `%d`, or `%%`; `" specifier "` would alter or hide the value."))))
|
||||
|
||||
(defn- lint-lines!
|
||||
"Flag each of `line-nodes` whose literal string, the matching element of `lines`, has a problem."
|
||||
[line-nodes lines]
|
||||
(doseq [[line-node line] (map vector line-nodes lines)]
|
||||
(when-let [problem (line-problem line)]
|
||||
(reg-msg-finding! line-node problem))))
|
||||
|
||||
(defn- plural
|
||||
[n word]
|
||||
(str n " " word (when-not (= 1 n) "s")))
|
||||
|
||||
(defn- lint-arg-count!
|
||||
"Flag `msg` call `node` when a literal line refers to an argument that can't exist, or when its `args` don't number
|
||||
the arguments its literal `lines` consume."
|
||||
[node lines args]
|
||||
(let [indexes (map first (consumed-args (str/join "\n" lines)))
|
||||
expected (if (seq indexes) (inc (apply max indexes)) 0)
|
||||
given (count args)]
|
||||
(cond
|
||||
;; `String/format` throws on these, so the message would render with every part quoted instead.
|
||||
(some neg? indexes)
|
||||
(reg-msg-finding! node (str "A `msg` line refers to an argument that can't exist; "
|
||||
"argument indexes start at `1$`, and `%<` repeats the specifier before it."))
|
||||
|
||||
(not= expected given)
|
||||
(reg-msg-finding! node (format "The `msg` lines take %s but %d %s given."
|
||||
(plural expected "argument") given (if (= 1 given) "is" "are"))))))
|
||||
|
||||
(defn lint-msg
|
||||
"Flag `msg` calls whose lines aren't a literal vector of single-line literal strings, that hold a malformed
|
||||
specifier or a conversion other than `%s`, `%d`, or `%%`, or whose arguments don't match the lines' specifiers."
|
||||
[{:keys [node config] :as input}]
|
||||
(when (level-on? config msg-linter)
|
||||
(let [[fn-node lines-node & args] (:children node)]
|
||||
(if-not (and lines-node (hooks/vector-node? lines-node))
|
||||
(reg-msg-finding! (or lines-node fn-node)
|
||||
"`msg` takes a literal vector of line strings, one string per line of the message.")
|
||||
(let [line-nodes (:children lines-node)
|
||||
lines (map literal-string line-nodes)]
|
||||
(lint-lines! line-nodes lines)
|
||||
;; A malformed specifier can't be counted, and its line already has a finding.
|
||||
(when (and (every? some? lines) (not-any? malformed-specifier? lines))
|
||||
(lint-arg-count! node lines args))))))
|
||||
input)
|
||||
|
||||
(def ^:private exit-linter :metabase/agent-message-exit)
|
||||
|
||||
(defn- exit-enabled?
|
||||
[{:keys [config]}]
|
||||
(level-on? config exit-linter))
|
||||
|
||||
(defn- reg-exit-finding!
|
||||
[node message]
|
||||
(hooks/reg-finding! (assoc (meta node) :message message :type exit-linter)))
|
||||
|
||||
(defn- calls-core?
|
||||
"Whether `node` is a call to a `clojure.core` var or special form named in `var-names`."
|
||||
[node var-names]
|
||||
(when-let [sym (call-name node)]
|
||||
(and (contains? var-names (name sym))
|
||||
(contains? core-ns (namespace sym)))))
|
||||
|
||||
(defn- calls-i18n?
|
||||
[node]
|
||||
(when-let [sym (call-name node)]
|
||||
(contains? #{"tru" "trs" "deferred-tru" "deferred-trs"} (name sym))))
|
||||
|
||||
(defn- calls-json-encode?
|
||||
"Whether `node` calls `encode` in a namespace or alias whose name ends in `json`."
|
||||
[node]
|
||||
(when-let [sym (call-name node)]
|
||||
(and (= "encode" (name sym))
|
||||
(boolean (some-> (namespace sym) (str/ends-with? "json"))))))
|
||||
|
||||
(defn- builds-string?
|
||||
"Whether `node` is a string literal or a call that builds a string."
|
||||
[node]
|
||||
(or (hooks/string-node? node)
|
||||
(calls-core? node #{"str" "format" "pr-str" "ex-message"})
|
||||
(calls-i18n? node)
|
||||
(calls? node string-ns "join")
|
||||
(calls-json-encode? node)))
|
||||
|
||||
(defn- clause-results
|
||||
"The result nodes of `clauses`, alternating test and result nodes, with a trailing default when their count is odd."
|
||||
[clauses]
|
||||
(concat (take-nth 2 (rest clauses))
|
||||
(when (odd? (count clauses)) [(last clauses)])))
|
||||
|
||||
(defn- result-fn-marker?
|
||||
[node]
|
||||
(and (hooks/keyword-node? node) (= :>> (hooks/sexpr node))))
|
||||
|
||||
(defn- condp-results
|
||||
"The result nodes of `condp` `clauses`: each test's result, skipping `:>>` result functions, and any default."
|
||||
[clauses]
|
||||
(loop [[_test result & more :as clauses] clauses
|
||||
acc []]
|
||||
(cond
|
||||
(empty? clauses) acc
|
||||
(nil? (next clauses)) (conj acc (first clauses))
|
||||
(result-fn-marker? result) (recur (next more) acc)
|
||||
:else (recur more (conj acc result)))))
|
||||
|
||||
(defn- branch-results
|
||||
"The nodes branching form `node` can evaluate to, or nil when it isn't one."
|
||||
[node]
|
||||
(let [args (rest (:children node))]
|
||||
(cond
|
||||
(calls-core? node #{"if" "if-not" "if-let"}) (rest args)
|
||||
(calls-core? node #{"or"}) args
|
||||
(calls-core? node #{"cond"}) (clause-results args)
|
||||
(calls-core? node #{"case"}) (clause-results (rest args))
|
||||
(calls-core? node #{"condp"}) (condp-results (drop 2 args)))))
|
||||
|
||||
(defn- step-call
|
||||
"Threading step `node` as the call it stands for: a list as is, anything else as a call with no written arguments."
|
||||
[node]
|
||||
(if (hooks/list-node? node) node (hooks/list-node [node])))
|
||||
|
||||
(defn- threading-results
|
||||
"The nodes whose text-ness decides what threading form `node` evaluates to, or nil when it isn't one."
|
||||
[node]
|
||||
(let [[init & steps] (rest (:children node))]
|
||||
(cond
|
||||
(calls-core? node #{"->" "->>" "some->" "some->>"}) [(if steps (step-call (last steps)) init)]
|
||||
(calls-core? node #{"cond->" "cond->>"}) (cons init (map step-call (take-nth 2 (rest steps))))
|
||||
(calls-core? node #{"as->"}) [(or (last (rest steps)) init)]
|
||||
(calls-core? node #{"doto"}) [init])))
|
||||
|
||||
(defn- tail-result
|
||||
"The node body form `node` evaluates to, or nil when it isn't one."
|
||||
[node]
|
||||
(let [args (rest (:children node))]
|
||||
(cond
|
||||
(calls-core? node #{"when" "when-not" "when-let" "let"}) (when (next args) (last args))
|
||||
(calls-core? node #{"do"}) (last args))))
|
||||
|
||||
(defn- stringy?
|
||||
"Whether `node` syntactically evaluates to text: a string literal, a call that builds a string, or a threading,
|
||||
branching, or body form whose result is one."
|
||||
[node]
|
||||
(boolean
|
||||
(or (builds-string? node)
|
||||
(some stringy? (branch-results node))
|
||||
(some stringy? (threading-results node))
|
||||
(some-> (tail-result node) stringy?))))
|
||||
|
||||
(defn- lint-exit-text!
|
||||
"Flag `arg`, the message passed to `target` (a description of where it goes), when it is text rather than a `msg`."
|
||||
[arg target]
|
||||
(when (and arg (stringy? arg))
|
||||
(reg-exit-finding! arg (format (str "`%s` passes text to %s; "
|
||||
"build agent-facing text with `msg` so interpolated values are cleaned.")
|
||||
(pr-str (hooks/sexpr arg))
|
||||
target))))
|
||||
|
||||
(defn- lint-exit-arg!
|
||||
"Flag the argument at child `index` of call `node` when it is text rather than a `msg`."
|
||||
[node index]
|
||||
(lint-exit-text! (nth (:children node) index nil) (str "`" (name (call-name node)) "`")))
|
||||
|
||||
(defn lint-teaching-exit
|
||||
"Flag a call throwing or returning a teaching error whose message, the first argument, is text rather than a `msg`."
|
||||
[{:keys [node] :as input}]
|
||||
(when (exit-enabled? input)
|
||||
(lint-exit-arg! node 1))
|
||||
input)
|
||||
|
||||
(defn lint-jsonrpc-error
|
||||
"Flag a `jsonrpc-error` whose message, the third argument, is text rather than a `msg`."
|
||||
[{:keys [node] :as input}]
|
||||
(when (exit-enabled? input)
|
||||
(lint-exit-arg! node 3))
|
||||
input)
|
||||
|
||||
(defn lint-op-error
|
||||
"Flag an `op-error!` whose detail message, the third argument, is text rather than a `msg`."
|
||||
[{:keys [node] :as input}]
|
||||
(when (exit-enabled? input)
|
||||
(lint-exit-arg! node 3))
|
||||
input)
|
||||
|
||||
(defn lint-ellipsize
|
||||
"Flag an `ellipsize` whose message, the first argument, is text rather than a `msg`."
|
||||
[{:keys [node] :as input}]
|
||||
(when (exit-enabled? input)
|
||||
(lint-exit-arg! node 1))
|
||||
input)
|
||||
|
||||
(defn- map-value
|
||||
"The value node of keyword `k` in map literal `node`, or nil."
|
||||
[node k]
|
||||
(when (hooks/map-node? node)
|
||||
(some (fn [[key-node value-node]]
|
||||
(when (and (hooks/keyword-node? key-node) (= k (hooks/sexpr key-node)))
|
||||
value-node))
|
||||
(partition 2 (:children node)))))
|
||||
|
||||
(defn lint-list-content
|
||||
"Flag a `list-content`, or a wrapper taking its options map as the third argument too, whose literal options map
|
||||
has an `:empty-hint` that is text rather than a `msg`."
|
||||
[{:keys [node] :as input}]
|
||||
(when (exit-enabled? input)
|
||||
(lint-exit-text! (map-value (nth (:children node) 3 nil) :empty-hint)
|
||||
(str "`" (name (call-name node)) "`'s `:empty-hint`")))
|
||||
input)
|
||||
|
||||
(defn lint-success-content
|
||||
"Flag a `success-content` whose first argument is text rather than a payload or a `msg`."
|
||||
[{:keys [node] :as input}]
|
||||
(when (exit-enabled? input)
|
||||
(let [text (second (:children node))]
|
||||
(when (and text (stringy? text))
|
||||
(reg-exit-finding! text (str "`success-content` takes data to JSON-encode or a `msg`; "
|
||||
"pass the payload itself, or build the text with `msg`.")))))
|
||||
input)
|
||||
|
||||
(defn- caller-facing-data?
|
||||
"Whether `node` is a map literal with a 4xx `:status-code` literal or an `error-code` key."
|
||||
[node]
|
||||
(and (hooks/map-node? node)
|
||||
(some (fn [[k v]]
|
||||
(when (hooks/keyword-node? k)
|
||||
(let [kw (hooks/sexpr k)]
|
||||
(or (= "error-code" (name kw))
|
||||
(and (= :status-code kw)
|
||||
(hooks/token-node? v)
|
||||
(let [status (hooks/sexpr v)]
|
||||
(and (number? status) (<= 400 status 499))))))))
|
||||
(partition 2 (:children node)))))
|
||||
|
||||
(defn lint-ex-info
|
||||
"Flag an `ex-info` whose data marks it as a caller-facing error."
|
||||
[{:keys [node] :as input}]
|
||||
(when (exit-enabled? input)
|
||||
(when (caller-facing-data? (nth (:children node) 2 nil))
|
||||
(reg-exit-finding! node (str "Caller-facing errors must be thrown with `throw-teaching-error` and a `msg`, "
|
||||
"so their text is rendered and cleaned."))))
|
||||
input)
|
||||
@@ -1,18 +0,0 @@
|
||||
(ns hooks.metabase.mq.impl
|
||||
(:require
|
||||
[clj-kondo.hooks-api :as hooks]))
|
||||
|
||||
(defn def-listener
|
||||
"Kondo hook for `mq/def-listener`. Rewrites to a `fn` form so kondo can analyze bindings.
|
||||
|
||||
Form:
|
||||
(mq/def-listener! :queue/bar [msgs] body...)"
|
||||
[{:keys [node]}]
|
||||
(let [[_ _channel-name bindings & body] (:children node)
|
||||
;; rewrite as: (fn [bindings...] body...)
|
||||
new-node (hooks/list-node
|
||||
(list*
|
||||
(hooks/token-node 'fn)
|
||||
bindings
|
||||
body))]
|
||||
{:node new-node}))
|
||||
@@ -4,9 +4,9 @@
|
||||
[hooks.common :as common]))
|
||||
|
||||
(defn lint-defsetting-namespace [node context]
|
||||
(when-not (re-matches #"^metabase(?:-enterprise)?\.([^\.]+\.)+settings$" (name (:ns context)))
|
||||
(when-not (re-matches #"^metabase(?:-enterprise)?\.[^\.]+\.settings$" (name (:ns context)))
|
||||
(hooks/reg-finding! (assoc (meta node)
|
||||
:message "All defsettings should live in a namespace ending in .settings under metabase[-enterprise]"
|
||||
:message "All defsettings should live in metabase[-enterprise].<module>.settings namespaces"
|
||||
:type :metabase/defsetting-namespace))))
|
||||
|
||||
;;; TODO -- move this into a Kondo config file in `.clj-kondo/config/`
|
||||
@@ -95,8 +95,6 @@
|
||||
metabot-feedback-url
|
||||
metabot-get-prompt-templates-url
|
||||
metabot-prompt-generator-token-limit
|
||||
metaplow-tracking-enabled
|
||||
metaplow-url
|
||||
multi-setting-read-only
|
||||
notebook-native-preview-sidebar-width
|
||||
notification-link-base-url
|
||||
@@ -286,7 +284,6 @@
|
||||
(update context :node update-node)))
|
||||
|
||||
(comment
|
||||
(require '[clojure.pprint])
|
||||
(defn- defsetting* [form]
|
||||
(hooks/sexpr
|
||||
(:node
|
||||
@@ -294,6 +291,7 @@
|
||||
{:node
|
||||
(hooks/parse-string
|
||||
(with-out-str
|
||||
#_{:clj-kondo/ignore [:unresolved-namespace]}
|
||||
(clojure.pprint/pprint
|
||||
form)))}))))
|
||||
|
||||
|
||||
14
.clj-kondo/src/hooks/metabase/test/gentest.clj
Normal file
14
.clj-kondo/src/hooks/metabase/test/gentest.clj
Normal file
@@ -0,0 +1,14 @@
|
||||
(ns hooks.metabase.test.gentest
|
||||
(:require
|
||||
[clj-kondo.hooks-api :as hooks]))
|
||||
|
||||
(defn iterate*
|
||||
"(iterate {:as limit-spec} [s1 val1 & more] body-form body-form...)"
|
||||
[{{:keys [children] :as node} :node :as orig}]
|
||||
(let [[token limit-spec bindings & body-nodes] children
|
||||
node* (-> (hooks/list-node
|
||||
(list token limit-spec
|
||||
(hooks/list-node (list* (hooks/token-node 'let) bindings
|
||||
body-nodes))))
|
||||
(with-meta (meta node)))]
|
||||
(assoc orig :node node*)))
|
||||
@@ -28,8 +28,8 @@
|
||||
(mapcat (fn [[setting-name v]]
|
||||
(concat
|
||||
[(with-meta (hooks/token-node '_) (meta setting-name)) v]
|
||||
;; if the setting name is namespace-qualified add a `_`
|
||||
;; entry for it too.
|
||||
;; if the setting name is namespace-qualified add a `_`
|
||||
;; entry for it too.
|
||||
(when (namespaced-symbol-node? setting-name)
|
||||
[(with-meta (hooks/token-node '_) (meta setting-name)) setting-name]))))
|
||||
(partition 2 (:children bindings))))
|
||||
|
||||
@@ -1,146 +0,0 @@
|
||||
(ns hooks.metabase.toucan.db-ns
|
||||
"Lint that application database query calls -- Toucan 2's `t2/select`, `t2/query`, `t2/insert!`, `t2/update!`,
|
||||
`t2/delete!` and friends, plus the `metabase.app-db.core` wrappers around them such as `mdb/query` and
|
||||
`mdb/update-or-insert!` -- live in a module's `db` namespace, i.e. `metabase[-enterprise].<module>.db` (or `metabase.driver.<driver>.db` for
|
||||
driver modules). Every other namespace in a module goes through those functions instead of talking to the
|
||||
application database directly.
|
||||
|
||||
Registered as an `:analyze-call` hook on each of those functions in `.clj-kondo/config.edn`. The hook
|
||||
returns its input unchanged so Kondo's normal analysis of the call (arity, var usage) still runs."
|
||||
(:require
|
||||
[clj-kondo.hooks-api :as hooks]
|
||||
[hooks.common.modules :as modules]))
|
||||
|
||||
(def ^:private driver-db-namespace
|
||||
"Matches `metabase.driver.<driver>.db`; individual drivers are not modules."
|
||||
#"^metabase\.driver\.[^.]+\.db$")
|
||||
|
||||
(defn- db-namespace? [config ns-sym]
|
||||
(let [ns-str (name ns-sym)]
|
||||
(boolean
|
||||
;; Resolve ownership first: a nested `.db` name alone does not make a module.
|
||||
(or (when-let [module (modules/module config ns-sym)]
|
||||
(= ns-str (str (modules/module-ns-prefix (:metabase/modules config) module) ".db")))
|
||||
(re-matches driver-db-namespace ns-str)))))
|
||||
|
||||
(defn- test-file?
|
||||
"Whether `filename` is in a test source tree. Test namespaces are exempt through the `test-namespaces` group in
|
||||
`config.edn`, but helper namespaces like `metabase.sso.test-helpers` don't match that group's pattern."
|
||||
[filename]
|
||||
(boolean (and filename (re-find #"(?:^|/)test/" filename))))
|
||||
|
||||
(def ^:private value-operators
|
||||
"Operators whose second argument is a value rather than a column."
|
||||
'#{= not= < > <= >= like not-like ilike not-ilike in not-in between})
|
||||
|
||||
(defn- marked?
|
||||
"Whether `node` is an `[:auto/param v]` marker."
|
||||
[node]
|
||||
(and (hooks/vector-node? node)
|
||||
(= :auto/param (some-> (first (:children node)) hooks/sexpr))))
|
||||
|
||||
(defn- value-nodes
|
||||
"The nodes sitting in a value slot of `node`, following the clause shapes a query map uses."
|
||||
[node]
|
||||
(cond
|
||||
;; A query map -- walk its clause values, not its keys.
|
||||
(hooks/map-node? node)
|
||||
(mapcat value-nodes (take-nth 2 (rest (:children node))))
|
||||
|
||||
(hooks/vector-node? node)
|
||||
(let [[head & args] (:children node)
|
||||
op (some-> head hooks/sexpr)]
|
||||
(cond
|
||||
(contains? #{:and :or :not} op)
|
||||
(mapcat value-nodes args)
|
||||
|
||||
;; `[:= col v]`, `[:in col vs]`, `[:between col lo hi]` -- the first argument is the
|
||||
;; column and the rest are values, whatever the arity.
|
||||
(and (keyword? op) (contains? value-operators (symbol (name op))))
|
||||
(rest args)
|
||||
|
||||
:else
|
||||
(mapcat value-nodes (:children node))))
|
||||
|
||||
:else nil))
|
||||
|
||||
(def ^:private write-fns
|
||||
"Calls whose values are written rather than filtered on. A value here is not a where-clause value
|
||||
an attacker can turn into SQL structure, and a `define-before-insert` hook may read it before the
|
||||
query compiles, so a marker would break it rather than protect anything."
|
||||
'#{insert! insert-returning-instance! insert-returning-instances! insert-returning-pk!
|
||||
insert-returning-pks! save!})
|
||||
|
||||
(defn- read-call?
|
||||
"Whether `node` is a query call whose values are filtered on rather than written."
|
||||
[node]
|
||||
(let [f (some-> (first (:children node)) hooks/sexpr)]
|
||||
(not (contains? write-fns (some-> f name symbol)))))
|
||||
|
||||
(defn- kv-arg-pair
|
||||
"The first `:column value` pair a query call passes after its model, if any.
|
||||
|
||||
`(t2/select :model/X :locale locale)`. Several of these fns take an argument before the model --
|
||||
`(t2/select-one-fn :value :model/X :key k)` -- so the pairs do not start at a fixed offset; they
|
||||
start after the `:model/...` keyword. A call that does not name a literal model is not checked."
|
||||
[args]
|
||||
(let [after-model (->> args
|
||||
(drop-while #(not (and (hooks/keyword-node? %)
|
||||
(= "model" (namespace (hooks/sexpr %))))))
|
||||
rest)
|
||||
[k v] after-model]
|
||||
(when (and k v (hooks/keyword-node? k))
|
||||
k)))
|
||||
|
||||
(defn- lint-kv-args!
|
||||
"Register a finding for a query written as `:column value` pairs.
|
||||
|
||||
Toucan builds those into the where clause itself, so the query's shape is not on the page: there
|
||||
is nothing for a reader, this linter, or the `.sql` extraction that follows this project to read.
|
||||
The same call takes a map -- `{:where [:= :locale v]}` for a select, `{:locale v}` for an
|
||||
`update!` -- which is the statement, near enough to read off."
|
||||
[node]
|
||||
(when-let [k (kv-arg-pair (rest (:children node)))]
|
||||
(hooks/reg-finding!
|
||||
(assoc (meta k)
|
||||
:message (format "Pass this query a map rather than `%s ...` pairs, so its shape is on the page."
|
||||
(hooks/sexpr k))
|
||||
:type :metabase/unsafe-app-db-query))))
|
||||
|
||||
(defn- lint-unmarked-values!
|
||||
"Register a finding for each argument of the enclosing function that reaches a value slot unmarked.
|
||||
|
||||
Only a symbol is reported. A literal cannot carry a request value, and a value built inside the
|
||||
function is out of reach of a check that does not follow it across a call."
|
||||
[node]
|
||||
(doseq [value (mapcat value-nodes (rest (:children node)))
|
||||
:when (and (hooks/token-node? value)
|
||||
(symbol? (hooks/sexpr value))
|
||||
(not (marked? value)))]
|
||||
(hooks/reg-finding!
|
||||
(assoc (meta value)
|
||||
:message (format "`%s` reaches a SQL value slot unmarked. Write it as [:auto/param %s] so it is bound as a parameter."
|
||||
(hooks/sexpr value) (hooks/sexpr value))
|
||||
:type :metabase/unsafe-app-db-query))))
|
||||
|
||||
(defn lint-query-call
|
||||
"Register a `:metabase/t2-query-namespace` finding when a Toucan 2 query call appears outside a `<module>.db`
|
||||
namespace."
|
||||
[{:keys [node ns filename] :as input}]
|
||||
(when (and ns
|
||||
(not (db-namespace? (modules/config input) ns))
|
||||
(not (test-file? filename)))
|
||||
(let [fn-node (first (:children node))]
|
||||
(hooks/reg-finding!
|
||||
(assoc (meta fn-node)
|
||||
:message (format "Application database query calls like `%s` must live in metabase[-enterprise].<module>.db (or metabase.driver.<driver>.db) namespaces"
|
||||
(hooks/sexpr fn-node))
|
||||
:type :metabase/t2-query-namespace))))
|
||||
((requiring-resolve 'hooks.metabase.warehouse-schema-overlay.table-or-field-query/lint-read) input)
|
||||
(when (and ns
|
||||
(db-namespace? (modules/config input) ns)
|
||||
(not (test-file? filename))
|
||||
(read-call? node))
|
||||
(lint-unmarked-values! node)
|
||||
(lint-kv-args! node))
|
||||
input)
|
||||
@@ -1,6 +1,7 @@
|
||||
(ns hooks.metabase.util.i18n
|
||||
(:require
|
||||
[clj-kondo.hooks-api :as api]
|
||||
[clojure.string :as str]
|
||||
[hooks.common]))
|
||||
|
||||
(defn- valid-apostrophes?
|
||||
@@ -12,23 +13,23 @@
|
||||
(let [c (first chars)]
|
||||
(if (= c \')
|
||||
(cond
|
||||
;; Doubled quotes: skip both
|
||||
;; Doubled quotes: skip both
|
||||
(and (next chars) (= (second chars) \'))
|
||||
(recur (nthrest chars 2))
|
||||
|
||||
;; Placeholder escape. We allow for intentional escaping, like either '{0}` or '{{`, but not 'arbitrary
|
||||
;; strings' So basically, just require the *next* character to be a `{`, find the next single-quote, and
|
||||
;; continue on from there.
|
||||
;;
|
||||
;; Technically a string like "foo bar's happy but i'm not" is totally fine, but we'll say it's invalid
|
||||
;; because... that inner string literal is probably not what we actually want. So since we've never done
|
||||
;; this intentionally as far as I can tell, let's just remove the possibility to avoid ambiguity.
|
||||
;; Placeholder escape. We allow for intentional escaping, like either '{0}` or '{{`, but not 'arbitrary
|
||||
;; strings' So basically, just require the *next* character to be a `{`, find the next single-quote, and
|
||||
;; continue on from there.
|
||||
;;
|
||||
;; Technically a string like "foo bar's happy but i'm not" is totally fine, but we'll say it's invalid
|
||||
;; because... that inner string literal is probably not what we actually want. So since we've never done
|
||||
;; this intentionally as far as I can tell, let's just remove the possibility to avoid ambiguity.
|
||||
(= (second chars) \{)
|
||||
(let [at-closing (drop-while #(not= \' %) (next chars))]
|
||||
(if (seq at-closing)
|
||||
(recur (next at-closing))
|
||||
false))
|
||||
;; Unpaired single quote: invalid
|
||||
;; Unpaired single quote: invalid
|
||||
:else
|
||||
false)
|
||||
(recur (rest chars)))))))
|
||||
|
||||
@@ -1,151 +0,0 @@
|
||||
(ns hooks.metabase.util.match
|
||||
(:require [clj-kondo.hooks-api :as api]))
|
||||
|
||||
(defn- token-node? [node token-name]
|
||||
(and (api/token-node? node)
|
||||
(= (str (:value node)) token-name)))
|
||||
|
||||
(defn- keyword-node? [node k]
|
||||
(and (api/keyword-node? node)
|
||||
(= (:k node) k)))
|
||||
|
||||
(defn- binding-symbol?
|
||||
"True when the token is a plain symbol that should be treated as a binding
|
||||
(not a keyword, not a boolean literal, not nil, not a number, not `_`)."
|
||||
[node]
|
||||
(and (api/token-node? node)
|
||||
(let [v (:value node)]
|
||||
(and (symbol? v)
|
||||
(not (contains? '#{_ nil true false} v))))))
|
||||
|
||||
;; To establish fake let bindings, we need to bind them to something. This "something" has to be vague enough to trick
|
||||
;; kondo into not looking too much into it. E.g., if we bind everything to `nil`, Kondo linters will get fired up.
|
||||
(def opaque-node (api/list-node (list (api/token-node 'rand-nth) (api/list-node []))))
|
||||
|
||||
(defn collect-bindings
|
||||
"Return a list of token nodes (symbols) introduced as bindings by `pattern`."
|
||||
[pattern]
|
||||
(cond
|
||||
(binding-symbol? pattern)
|
||||
[[pattern opaque-node]]
|
||||
|
||||
;; Lists: (:or ...), (:and ...), (sym :guard pred ...), or a quoted symbol.
|
||||
(api/list-node? pattern)
|
||||
(let [[head & rst] (:children pattern)]
|
||||
(cond (or (keyword-node? head :or)
|
||||
(keyword-node? head :and))
|
||||
(mapcat collect-bindings rst)
|
||||
|
||||
(api/token-node? head)
|
||||
(cond-> []
|
||||
(binding-symbol? head) (conj [head opaque-node])
|
||||
(keyword-node? (first rst) :guard) (conj [(api/token-node '_) (second rst)]))
|
||||
|
||||
;; anything else (e.g. a function call used as predicate) — no bindings
|
||||
:else []))
|
||||
|
||||
(api/vector-node? pattern)
|
||||
(mapcat (fn [node]
|
||||
(when-not (token-node? node "&")
|
||||
(collect-bindings node)))
|
||||
(:children pattern))
|
||||
|
||||
;; Maps: keys are literal selectors, values are sub-patterns.
|
||||
;; Special value-symbols &truthy and _ are NOT user bindings.
|
||||
(api/map-node? pattern)
|
||||
(let [children (:children pattern)
|
||||
pairs (partition 2 children)]
|
||||
(mapcat (fn [[_k v]]
|
||||
(when-not (or (token-node? v "&truthy")
|
||||
(token-node? v "_"))
|
||||
(collect-bindings v)))
|
||||
pairs))
|
||||
|
||||
;; For sets, make sure to use them because they may reference some of the outer local bindings, so if we don't
|
||||
;; include them, Kondo will complain about those bindings being unused.
|
||||
(api/set-node? pattern)
|
||||
[[(api/token-node '_) pattern]]
|
||||
|
||||
;; Anything else — no bindings.
|
||||
:else []))
|
||||
|
||||
(defn- dedupe-bindings
|
||||
"Prevent Kondo complaining that we have multiple fake bindings."
|
||||
[pairs]
|
||||
(loop [[[sym v] & r] pairs, seen #{}, result []]
|
||||
(cond (nil? sym) result
|
||||
(and (seen sym) (not (token-node? sym "_"))) (recur r seen result)
|
||||
:else (recur r (conj seen sym) (conj result sym v)))))
|
||||
|
||||
(defn match-one
|
||||
"clj-kondo :analyze-call hook for match-one.
|
||||
|
||||
Rewrites
|
||||
|
||||
(match-one value
|
||||
pat1 ret1
|
||||
pat2 ret2
|
||||
...)
|
||||
|
||||
into a `let` form that declares all binding symbols introduced by the patterns, so clj-kondo knows they are
|
||||
legitimate and won't warn about unresolved/unused symbols:
|
||||
|
||||
(let [<all-bindings> (rand-nth []) ...]
|
||||
(or ret1 ret2 ...))"
|
||||
[{:keys [node]}]
|
||||
(let [[_ value & clauses] (:children node)]
|
||||
(when (odd? (count clauses))
|
||||
(api/reg-finding!
|
||||
{:message "match-one/match-many/replace requires an even number of pattern/result pairs"
|
||||
:type :metabase/match-odd-clauses
|
||||
:row (:row node)
|
||||
:col (:col node)
|
||||
:filename (-> node meta :filename)}))
|
||||
(let [pairs (partition 2 clauses)
|
||||
all-bindings (->> pairs
|
||||
(mapcat (fn [[pat _ret]] (collect-bindings pat)))
|
||||
dedupe-bindings)
|
||||
special-bindings [(api/token-node '_) value
|
||||
(api/token-node '&match) opaque-node
|
||||
(api/token-node '&parents) opaque-node
|
||||
(api/token-node '&recur) opaque-node
|
||||
;; Pretend to use &match, &parents, and &recur so that Kondo doesn't complain about them
|
||||
;; if they are not used by the user code.
|
||||
(api/token-node '_) (api/token-node '&match)
|
||||
(api/token-node '_) (api/token-node '&parents)
|
||||
(api/token-node '_) (api/token-node '&recur)]
|
||||
binding-nodes (into special-bindings all-bindings)
|
||||
result-exprs (map second pairs)
|
||||
new-node (api/list-node
|
||||
(list
|
||||
(api/token-node 'let)
|
||||
(api/vector-node binding-nodes)
|
||||
(api/list-node
|
||||
(concat [(api/token-node 'clojure.core/or)] result-exprs [opaque-node]))))]
|
||||
{:node new-node})))
|
||||
|
||||
(defn matches?
|
||||
"clj-kondo :analyze-call hook for matches?. Similar to `match-one`, but supports just a single clause with no result."
|
||||
[{:keys [node]}]
|
||||
(let [[_ value pat] (:children node)]
|
||||
(let [all-bindings (->> (collect-bindings pat)
|
||||
dedupe-bindings)
|
||||
special-bindings [(api/token-node '_) value
|
||||
(api/token-node '&match) opaque-node
|
||||
;; Pretend to use &match so that Kondo doesn't complain about it.
|
||||
(api/token-node '_) (api/token-node '&match)]
|
||||
binding-nodes (into special-bindings all-bindings)
|
||||
new-node (api/list-node
|
||||
(list
|
||||
(api/token-node 'let)
|
||||
(api/vector-node binding-nodes)
|
||||
(api/list-node
|
||||
(list opaque-node))))]
|
||||
{:node new-node})))
|
||||
|
||||
(def match-many match-one)
|
||||
(def replace match-one)
|
||||
|
||||
(defn replace-in [context]
|
||||
;; Everything is the same as in match-one, just drop the middle argument.
|
||||
(match-one (update-in context [:node :children] (fn [[sym value path & clauses]] (list* sym value clauses)))))
|
||||
@@ -1,52 +0,0 @@
|
||||
(ns hooks.metabase.warehouse-schema-overlay.table-or-field-query
|
||||
"Lint that a query over `:model/Field` or `:model/Table` names its source with `field-query`/`table-query` from
|
||||
`metabase.warehouse-schema-overlay.core`, or reads sync's values with `{:user-settings? false}`."
|
||||
(:require
|
||||
[clj-kondo.hooks-api :as hooks]))
|
||||
|
||||
(def ^:private read-fn?
|
||||
"The Toucan 2 query functions that read rows. Writes are not affected: they go to `metabase_field` itself."
|
||||
#{"select" "select-one" "select-fn->fn" "select-fn->pk" "select-fn-reducible" "select-fn-set" "select-fn-vec"
|
||||
"select-one-fn" "select-one-pk" "select-pk->fn" "select-pks-set" "select-pks-vec" "select-reducible"
|
||||
"reducible-select" "reducible-query" "query" "query-one" "count" "exists?"})
|
||||
|
||||
(def ^:private model->helper
|
||||
{":model/Field" "field-query"
|
||||
":model/Table" "table-query"})
|
||||
|
||||
(defn- test-file? [filename]
|
||||
(boolean (and filename (re-find #"(?:^|/)test/" filename))))
|
||||
|
||||
(defn- tokens
|
||||
"Every token in `node`'s tree, as strings."
|
||||
[node]
|
||||
(tree-seq :children :children node))
|
||||
|
||||
(defn- mentions?
|
||||
[node pred]
|
||||
(boolean (some (fn [n] (when-let [s (some-> (hooks/sexpr n) pr-str)] (pred s)))
|
||||
(filter (complement :children) (tokens node)))))
|
||||
|
||||
(defn- writes?
|
||||
"Whether a `t2/query` form is really a write spelled as a query."
|
||||
[node]
|
||||
(mentions? node #{":update" ":delete-from" ":insert-into"}))
|
||||
|
||||
(defn lint-read
|
||||
"Register a `:metabase/table-or-field-query` finding when a read of `:model/Field` or `:model/Table` does not name
|
||||
its source with the matching helper."
|
||||
[{:keys [node filename]}]
|
||||
(let [fn-node (first (:children node))
|
||||
fn-name (some-> (hooks/sexpr fn-node) name)]
|
||||
(when (and fn-name
|
||||
(read-fn? fn-name)
|
||||
(not (test-file? filename))
|
||||
(not (writes? node)))
|
||||
(doseq [[model helper] model->helper
|
||||
:when (mentions? node #(= % model))
|
||||
:when (not (mentions? node #(re-find (re-pattern (str "\\b" helper "$")) %)))]
|
||||
(hooks/reg-finding!
|
||||
(assoc (meta fn-node)
|
||||
:message (format "A read of %s must name its source: add {:from [(%s)]} (or {:user-settings? false} for sync's own values)"
|
||||
model helper)
|
||||
:type :metabase/table-or-field-query))))))
|
||||
@@ -1,22 +0,0 @@
|
||||
(ns hooks.taoensso.nippy
|
||||
(:require [clj-kondo.hooks-api :as api]))
|
||||
|
||||
(defn extend-thaw
|
||||
[{:keys [node]}]
|
||||
(let [[_ _ arglist & body] (:children node)]
|
||||
{:node (with-meta (api/list-node
|
||||
(list*
|
||||
(api/token-node 'fn)
|
||||
arglist
|
||||
body))
|
||||
(meta node))}))
|
||||
|
||||
(defn extend-freeze
|
||||
[{:keys [node]}]
|
||||
(let [[_ _ _ arglist & body] (:children node)]
|
||||
{:node (with-meta (api/list-node
|
||||
(list*
|
||||
(api/token-node 'fn)
|
||||
arglist
|
||||
body))
|
||||
(meta node))}))
|
||||
@@ -19,19 +19,6 @@
|
||||
(defn- lint-modules [form config]
|
||||
(lint-ns form (assoc-in config [:linters :metabase/modules :level] :warning)))
|
||||
|
||||
(deftest ^:parallel explicit-nonparallel-namespace-metadata-test
|
||||
(let [config {:linters {:metabase/validate-deftest {:level :warning}}}]
|
||||
(testing "Hawk's supported namespace marker passes"
|
||||
(is (empty? (lint-ns '(ns ^:synchronized example.test) config))))
|
||||
(testing "the verbose Hawk spelling is not accepted as Metabase style"
|
||||
(is (=? [{:type :metabase/validate-deftest
|
||||
:message #(re-find #"Use `\^:synchronized` instead" %)}]
|
||||
(lint-ns '(ns ^{:parallel false} example.test) config))))
|
||||
(testing "the legacy marker explains that Hawk ignores it"
|
||||
(is (=? [{:type :metabase/validate-deftest
|
||||
:message #(re-find #"`\^:synchronous` is ignored by Hawk" %)}]
|
||||
(lint-ns '(ns ^:synchronous example.test) config))))))
|
||||
|
||||
(deftest ^:parallel module-checker-allowed-modules-test
|
||||
(are [task-namespace] (=? [{:message "Module task should not be used in the api module. [:metabase/modules api :uses]"
|
||||
:type :metabase/modules}]
|
||||
@@ -56,16 +43,6 @@
|
||||
'{:metabase/modules {api {:uses #{search}}
|
||||
search {:api #{metabase.search.core}}}}))))
|
||||
|
||||
(deftest ^:parallel module-checker-empty-api-test
|
||||
(is (=? [{:message "Namespace metabase.embeddings.core is not an allowed external API namespace for the embeddings module. [:metabase/modules embeddings :api]"
|
||||
:type :metabase/modules}]
|
||||
(lint-modules
|
||||
'(ns metabase.metabot.core
|
||||
(:require
|
||||
[metabase.embeddings.core :as embeddings]))
|
||||
'{:metabase/modules {metabot {:uses #{embeddings}}
|
||||
embeddings {:api #{}}}}))))
|
||||
|
||||
(deftest ^:parallel module-checker-friends-test
|
||||
(is (= []
|
||||
(lint-modules
|
||||
|
||||
@@ -1,65 +0,0 @@
|
||||
(ns hooks.clojure.core.require-test
|
||||
(:require
|
||||
[clj-kondo.hooks-api :as hooks]
|
||||
[clj-kondo.impl.utils]
|
||||
[clojure.test :refer :all]
|
||||
[hooks.clojure.core.require]))
|
||||
|
||||
(def ^:private config
|
||||
'{:linters {:metabase/modules {:level :warning}}
|
||||
:metabase/modules {search {:uses #{}}
|
||||
task {:api #{metabase.task.core}}}})
|
||||
|
||||
(defn- lint [hook form current-ns]
|
||||
(binding [clj-kondo.impl.utils/*ctx* {:config config
|
||||
:ignores (atom nil)
|
||||
:findings (atom [])
|
||||
:namespaces (atom {})}]
|
||||
(hook {:node (hooks/parse-string (pr-str form))
|
||||
:ns current-ns
|
||||
:config config})
|
||||
@(:findings clj-kondo.impl.utils/*ctx*)))
|
||||
|
||||
(defn- lint-require [form current-ns]
|
||||
(lint hooks.clojure.core.require/lint-require form current-ns))
|
||||
|
||||
(defn- lint-requiring-resolve [form current-ns]
|
||||
(lint hooks.clojure.core.require/lint-requiring-resolve form current-ns))
|
||||
|
||||
(def ^:private dynamic-require-finding
|
||||
{:message "Module dependency cannot be statically determined from a dynamic require"
|
||||
:type :metabase/modules})
|
||||
|
||||
(def ^:private cross-module-finding
|
||||
{:message "Module task should not be used in the search module. [:metabase/modules search :uses]"
|
||||
:type :metabase/modules})
|
||||
|
||||
(deftest ^:parallel dynamic-requiring-resolve-test
|
||||
(testing "outside the module system"
|
||||
(is (= [] (lint-requiring-resolve '(requiring-resolve (:failing-test-var opts)) 'dev))))
|
||||
(testing "inside a module"
|
||||
(are [form] (=? [dynamic-require-finding]
|
||||
(lint-requiring-resolve form 'metabase.search.core))
|
||||
'(requiring-resolve (:failing-test-var opts))
|
||||
'(requiring-resolve @some-atom)
|
||||
'(requiring-resolve some-local)
|
||||
'(requiring-resolve "not-a-symbol"))))
|
||||
|
||||
(deftest ^:parallel dynamic-require-test
|
||||
(testing "outside the module system"
|
||||
(is (= [] (lint-require '(require some-local) 'dev))))
|
||||
(testing "inside a module"
|
||||
(is (=? [dynamic-require-finding]
|
||||
(lint-require '(require some-local) 'metabase.search.core)))))
|
||||
|
||||
(deftest ^:parallel static-requiring-resolve-still-linted-test
|
||||
(is (=? [cross-module-finding]
|
||||
(lint-requiring-resolve '(requiring-resolve 'metabase.task.impl/go)
|
||||
'metabase.search.core))
|
||||
"a statically-known cross-module violation is still reported"))
|
||||
|
||||
(deftest ^:parallel static-require-still-linted-test
|
||||
(is (=? [cross-module-finding]
|
||||
(lint-require '(require 'metabase.task.impl :reload)
|
||||
'metabase.search.core))
|
||||
"a statically-known cross-module violation is still reported while keyword flags are ignored"))
|
||||
@@ -1,152 +0,0 @@
|
||||
(ns hooks.clojure.core.with-redefs-test
|
||||
(:require
|
||||
[clj-kondo.core :as kondo]
|
||||
[clj-kondo.hooks-api :as hooks]
|
||||
[clj-kondo.impl.utils]
|
||||
[clojure.java.io :as io]
|
||||
[clojure.test :refer :all]
|
||||
[hooks.clojure.core.with-redefs]))
|
||||
|
||||
;;; The new hook delegates the "is this LHS a regular defn?" decision to clj-kondo's own
|
||||
;;; analysis cache via [[hooks/resolve]] and [[hooks/ns-analysis]]. In unit tests we have
|
||||
;;; no cache, so we stub those calls. The mapping below describes what each stubbed name
|
||||
;;; should pretend to be.
|
||||
|
||||
(def ^:private stub-vars
|
||||
"Map of unqualified-name → kondo-style var-definition. Names not in the map are treated
|
||||
as `:clj-kondo/unknown-namespace` (unresolved). A `:fixed-arities` (or
|
||||
`:varargs-min-arity`) entry means \"this is a `defn`\" and is the only thing that lets
|
||||
the nudge fire."
|
||||
'{plain-fn {:ns example.ns :name plain-fn :fixed-arities #{1}}
|
||||
plain-fn-2 {:ns example.ns :name plain-fn-2 :fixed-arities #{0}}
|
||||
a-multimethod {:ns example.ns :name a-multimethod} ; defmulti — no arities
|
||||
can-read? {:ns example.ns :name can-read?} ; defmulti
|
||||
can-query? {:ns example.ns :name can-query?} ; defmulti
|
||||
a-value {:ns example.ns :name a-value}}) ; (def a-value 42) — no arities
|
||||
|
||||
(defn- stub-resolve [{nm :name}]
|
||||
(when (symbol? nm)
|
||||
(let [bare (symbol (name nm))]
|
||||
(if (contains? stub-vars bare)
|
||||
{:ns 'example.ns :name bare}
|
||||
{:ns :clj-kondo/unknown-namespace :name bare}))))
|
||||
|
||||
(defn- stub-ns-analysis [_ns-sym]
|
||||
;; All stubbed vars live in `example.ns` for simplicity.
|
||||
{:clj stub-vars})
|
||||
|
||||
(defn- lint
|
||||
"Run the hook on the given source. Pass a string to preserve reader-macro literals like
|
||||
`#(...)` (whose `:fn` node tag is distinct from a regular list); pass a quoted form
|
||||
for the common case where exact node shape doesn't matter. The optional
|
||||
`analysis-fn` stub overrides `hooks/ns-analysis` for tests that want to simulate a
|
||||
cache miss or other non-default cache state."
|
||||
([src] (lint src stub-ns-analysis))
|
||||
([src analysis-fn]
|
||||
(binding [clj-kondo.impl.utils/*ctx* {:config {:linters {:metabase/prefer-with-dynamic-fn-redefs {:level :warning}}}
|
||||
:ignores (atom nil)
|
||||
:findings (atom [])
|
||||
:namespaces (atom {})}]
|
||||
(with-redefs [hooks/resolve stub-resolve
|
||||
hooks/ns-analysis analysis-fn]
|
||||
(hooks.clojure.core.with-redefs/lint-with-redefs
|
||||
{:node (hooks/parse-string (if (string? src) src (pr-str src)))}))
|
||||
@(:findings clj-kondo.impl.utils/*ctx*))))
|
||||
|
||||
(deftest ^:synchronized flags-when-every-lhs-is-defn-test
|
||||
(testing "RHS shape doesn't matter — once every LHS resolves to a defn, the form is a
|
||||
migration candidate regardless of what's on the right. We deliberately stick
|
||||
to `IFn`-valued shapes here (fns, keywords, colls) — pairing a fn-var LHS
|
||||
with a non-`IFn` value (`42`) would be a weird shape callers wouldn't realistically
|
||||
write, and `with-dynamic-fn-redefs` would throw at proxy time anyway."
|
||||
(doseq [rhs ['(fn [x] x)
|
||||
'(constantly 42)
|
||||
'(partial + 1)
|
||||
'(comp inc dec)
|
||||
'identity ; bare symbol resolving to an IFn
|
||||
'(foo/constantly 42) ; namespaced lookalike — no longer special-cased
|
||||
:a-key ; keyword (IFn)
|
||||
{:a 1} ; map (IFn)
|
||||
[1 2 3]]] ; vector (IFn)
|
||||
(is (=? [{:type :metabase/prefer-with-dynamic-fn-redefs}]
|
||||
(lint (list 'with-redefs ['plain-fn rhs] :body)))
|
||||
(str "expected nudge for RHS: " (pr-str rhs)))))
|
||||
(testing "reader-macro `#(...)` literal — passed as a string so it parses as a
|
||||
`:fn`-tagged node, not the post-expansion `(fn* …)` list. Still works because
|
||||
it's just one more shape the LHS-only rule treats uniformly."
|
||||
(is (=? [{:type :metabase/prefer-with-dynamic-fn-redefs}]
|
||||
(lint "(with-redefs [plain-fn #(inc %)] (plain-fn 1))"))))
|
||||
(testing "multiple bindings, every LHS is a defn"
|
||||
(is (=? [{:type :metabase/prefer-with-dynamic-fn-redefs}]
|
||||
(lint '(with-redefs [plain-fn (constantly 1)
|
||||
plain-fn-2 (fn [] 2)]
|
||||
(plain-fn)))))))
|
||||
|
||||
(deftest ^:synchronized skips-multimethod-and-value-targets-test
|
||||
(testing "defmulti — no arity in analysis, so don't nudge"
|
||||
(is (= [] (lint '(with-redefs [a-multimethod (fn [& _] nil)] :body))))
|
||||
(is (= [] (lint '(with-redefs [can-read? (constantly true)] :body))))
|
||||
(is (= [] (lint '(with-redefs [can-query? (constantly false)] :body)))))
|
||||
(testing "plain `def` value also has no arity — don't nudge"
|
||||
(is (= [] (lint '(with-redefs [a-value (fn [] 1)] :body)))))
|
||||
(testing "any unresolved symbol skips the nudge — we never know what it is"
|
||||
(is (= [] (lint '(with-redefs [unknown.ns/something (fn [& _] nil)] :body)))))
|
||||
(testing "empty analysis (cache miss for the resolved ns) → skip the nudge"
|
||||
(is (= [] (lint '(with-redefs [plain-fn (fn [x] x)] :body)
|
||||
(constantly {})))))
|
||||
(testing "mixed bindings — even one non-defn LHS suppresses the nudge"
|
||||
(is (= [] (lint '(with-redefs [plain-fn (fn [x] x)
|
||||
a-multimethod (fn [& _] nil)]
|
||||
:body))))))
|
||||
|
||||
(defn- spit-fixture! [^java.io.File f content]
|
||||
(.mkdirs (.getParentFile f))
|
||||
(spit f content))
|
||||
|
||||
(defn- delete-tree! [^java.io.File f]
|
||||
(when (.isDirectory f)
|
||||
(run! delete-tree! (.listFiles f)))
|
||||
(.delete f))
|
||||
|
||||
(defn- run-kondo-twice
|
||||
"Lint `src` first to populate `cache-dir` (mirroring the real `kondo --lint src test`
|
||||
pipeline), then lint `test-file` against that cache and return its findings. The
|
||||
two-pass shape is required because the hook reads its decisions from the cache, which
|
||||
is only written *after* a file is analysed."
|
||||
[cache-dir src-file test-file]
|
||||
(kondo/run! {:lint [(.getPath src-file)] :cache-dir cache-dir :config-dir ".clj-kondo"})
|
||||
(:findings (kondo/run! {:lint [(.getPath test-file)] :cache-dir cache-dir :config-dir ".clj-kondo"})))
|
||||
|
||||
(deftest ^:synchronized integration-arities-iff-defn-smoke-test
|
||||
(testing "real kondo run validates the load-bearing invariant: only `defn`-style vars
|
||||
get arities recorded — `defmulti` and plain `def` do not. If a future kondo
|
||||
release breaks this, the smoke test fails here rather than the hook silently
|
||||
producing wrong nudges."
|
||||
(let [tmp-dir (.toFile (java.nio.file.Files/createTempDirectory
|
||||
"with-redefs-smoke" (into-array java.nio.file.attribute.FileAttribute [])))
|
||||
cache-dir (str tmp-dir "/cache")
|
||||
src (io/file tmp-dir "smoke_fixture.clj")
|
||||
tst (io/file tmp-dir "smoke_fixture_test.clj")]
|
||||
(try
|
||||
(spit-fixture! src "(ns smoke-fixture)
|
||||
(defmulti the-multi {:arglists '([x])} (fn [x] x))
|
||||
(defn the-defn [x] x)
|
||||
(def the-value 42)
|
||||
")
|
||||
(spit-fixture! tst "(ns smoke-fixture-test
|
||||
(:require [smoke-fixture :as f]))
|
||||
(with-redefs [f/the-multi (constantly nil)] :a)
|
||||
(with-redefs [f/the-defn (constantly nil)] :b)
|
||||
(with-redefs [f/the-value (constantly nil)] :c)
|
||||
")
|
||||
(let [findings (run-kondo-twice cache-dir src tst)
|
||||
nudges (filter #(= :metabase/prefer-with-dynamic-fn-redefs (:type %)) findings)
|
||||
nudge-rows (set (map :row nudges))]
|
||||
;; Row 3 = the-multi, row 4 = the-defn, row 5 = the-value (matches the
|
||||
;; with-redefs lines in the test fixture above).
|
||||
(testing "the defn binding gets nudged"
|
||||
(is (contains? nudge-rows 4)))
|
||||
(testing "the defmulti and def bindings do not get nudged"
|
||||
(is (not (contains? nudge-rows 3)))
|
||||
(is (not (contains? nudge-rows 5)))))
|
||||
(finally (delete-tree! tmp-dir))))))
|
||||
@@ -21,54 +21,6 @@
|
||||
#{:athena}}}}})
|
||||
(mapv :message @(:findings clj-kondo.impl.utils/*ctx*))))
|
||||
|
||||
(defn- parallel-metadata-findings
|
||||
([form]
|
||||
(parallel-metadata-findings form 'example.test))
|
||||
([form ns-sym]
|
||||
(let [linters {:metabase/deftest-not-marked-parallel-or-synchronized {:level :warning}
|
||||
:metabase/validate-deftest {:level :warning}}]
|
||||
(binding [clj-kondo.impl.utils/*ctx* {:config {:linters linters}
|
||||
:ignores (atom nil)
|
||||
:findings (atom [])
|
||||
:namespaces (atom {})}]
|
||||
(hooks.clojure.test/deftest {:node (api/parse-string form)
|
||||
:lang :clj
|
||||
:ns ns-sym
|
||||
:config {:linters {:metabase/validate-deftest {}}}})
|
||||
@(:findings clj-kondo.impl.utils/*ctx*)))))
|
||||
|
||||
(deftest ^:parallel explicit-nonparallel-metadata-test
|
||||
(testing "Hawk's supported test markers satisfy the explicit metadata linter"
|
||||
(are [metadata] (empty? (parallel-metadata-findings
|
||||
(format "(deftest %s example-test (is true))" metadata)))
|
||||
"^:parallel"
|
||||
"^:synchronized"))
|
||||
(testing "an unmarked test still gets the opt-in linter's guidance"
|
||||
(is (=? [{:type :metabase/deftest-not-marked-parallel-or-synchronized
|
||||
:message "Test should be marked either `^:parallel` or `^:synchronized`"}]
|
||||
(parallel-metadata-findings "(deftest example-test (is true))"))))
|
||||
(testing "the unsupported marker explains that Hawk ignores it"
|
||||
(is (some #(and (= (:type %) :metabase/validate-deftest)
|
||||
(re-find #"ignored by Hawk" (:message %)))
|
||||
(parallel-metadata-findings
|
||||
"(deftest ^:sequential example-test (is true))"))))
|
||||
(testing "the verbose Hawk spelling is not accepted as Metabase style"
|
||||
(is (some #(and (= (:type %) :metabase/validate-deftest)
|
||||
(re-find #"Use `\^:synchronized` instead" (:message %)))
|
||||
(parallel-metadata-findings
|
||||
"(deftest ^{:parallel false} example-test (is true))"))))
|
||||
(testing "a test cannot request both execution modes"
|
||||
(is (some #(and (= (:type %) :metabase/validate-deftest)
|
||||
(re-find #"both `\^:parallel` and `\^:synchronized`" (:message %)))
|
||||
(parallel-metadata-findings
|
||||
"(deftest ^:parallel ^:synchronized example-test (is true))"))))
|
||||
(testing "a namespace-level non-parallel promise rejects parallel tests"
|
||||
(is (=? [{:type :metabase/validate-deftest
|
||||
:message #(re-find #"namespace marked.*`\^:synchronized`" %)}]
|
||||
(parallel-metadata-findings
|
||||
"(deftest ^:parallel example-test (is true))"
|
||||
(with-meta 'example.test {:synchronized true}))))))
|
||||
|
||||
(deftest ^:parallel disallow-hardcoded-driver-names-in-tests-test
|
||||
(is (= []
|
||||
(deftest-warnings
|
||||
|
||||
@@ -5,6 +5,6 @@
|
||||
|
||||
(deftest ^:parallel module-test
|
||||
(are [symb expected] (= expected
|
||||
(hooks.common.modules/module {} symb))
|
||||
(hooks.common.modules/module symb))
|
||||
'metabase.qp.middleware.wow 'qp
|
||||
'metabase-enterprise.whatever.core 'enterprise/whatever))
|
||||
|
||||
@@ -1,334 +0,0 @@
|
||||
(ns hooks.metabase.agent-message-test
|
||||
(:require
|
||||
[clj-kondo.hooks-api :as hooks]
|
||||
[clj-kondo.impl.utils]
|
||||
[clojure.edn :as edn]
|
||||
[clojure.test :refer :all]
|
||||
[hooks.metabase.agent-message :as agent-message]))
|
||||
|
||||
(defn- findings
|
||||
"Run `hook-fn` over `form`, a form or its source string, with the linters at `level` (unconfigured when nil) in
|
||||
namespace `ns` of file `filename`; return the findings' messages."
|
||||
([hook-fn form]
|
||||
(findings hook-fn form :warning))
|
||||
([hook-fn form level]
|
||||
(findings hook-fn form level 'metabase.mcp.v2.tools.browse "src/metabase/mcp/v2/tools/browse.clj"))
|
||||
([hook-fn form level ns filename]
|
||||
(let [config (if level
|
||||
{:linters {:metabase/agent-message-lines {:level level}
|
||||
:metabase/agent-message-exit {:level level}}}
|
||||
{})]
|
||||
(binding [clj-kondo.impl.utils/*ctx* {:config config
|
||||
:ignores (atom nil)
|
||||
:findings (atom [])
|
||||
:namespaces (atom {})}]
|
||||
(let [input {:node (hooks/parse-string (if (string? form) form (pr-str form)))
|
||||
:ns ns
|
||||
:filename filename
|
||||
:config config}
|
||||
output (hook-fn input)]
|
||||
(is (identical? (:node input) (:node output))
|
||||
"the hook must return the node unchanged so Kondo's normal analysis still runs")
|
||||
(mapv :message @(:findings clj-kondo.impl.utils/*ctx*)))))))
|
||||
|
||||
(def ^:private hook-cases
|
||||
"`[hook form]` pairs that each report one finding when their linter is enabled."
|
||||
[[agent-message/lint-msg '(msg lines x)]
|
||||
[agent-message/lint-teaching-exit '(throw-teaching-error "x")]
|
||||
[agent-message/lint-jsonrpc-error '(jsonrpc-error id -32602 "x")]
|
||||
[agent-message/lint-success-content '(success-content "x")]
|
||||
[agent-message/lint-ex-info "(ex-info \"x\" {:status-code 404})"]
|
||||
[agent-message/lint-op-error '(op-error! idx "add_link" "x")]
|
||||
[agent-message/lint-ellipsize '(ellipsize "x" 300)]
|
||||
[agent-message/lint-list-content '(list-content data total {:empty-hint "x"})]])
|
||||
|
||||
(defn- linter-enabled-in-ns?
|
||||
"Whether `config` enables `linter` in namespace `ns-name` through an ns-group's `:config-in-ns` entry."
|
||||
[config linter ns-name]
|
||||
(boolean
|
||||
(some (fn [{:keys [pattern name]}]
|
||||
(and (string? pattern)
|
||||
(re-find (re-pattern pattern) ns-name)
|
||||
(not= :off (get-in config [:config-in-ns name :linters linter :level] :off))))
|
||||
(:ns-groups config))))
|
||||
|
||||
(deftest ^:parallel config-scope-test
|
||||
(let [config (edn/read-string (slurp ".clj-kondo/config.edn"))]
|
||||
(testing "GHY-4544: `msg` lines are checked in every OSS and enterprise MCP source namespace, and no test one"
|
||||
(doseq [[ns-name enabled?] {"metabase.mcp.ui-resource" true
|
||||
"metabase.mcp.transport" true
|
||||
"metabase.mcp.v2.tools.browse" true
|
||||
"metabase-enterprise.mcp.usage" true
|
||||
"metabase-enterprise.mcp.v2.tools.browse" true
|
||||
"metabase.mcp.ui-resource-test" false
|
||||
"metabase.mcp.v2.tools.browse-test" false
|
||||
"metabase.mcp.test-util" false
|
||||
"metabase-enterprise.mcp.usage-test" false
|
||||
"metabase-enterprise.mcp.v2.tools.browse-test" false
|
||||
"metabase.queries.models.card" false}]
|
||||
(testing ns-name
|
||||
(is (= enabled? (linter-enabled-in-ns? config :metabase/agent-message-lines ns-name))))))
|
||||
(testing "GHY-4544: exits are checked in MCP transport and v2 source namespaces, whose 4xx errors reach the agent"
|
||||
(doseq [[ns-name enabled?] {"metabase.mcp.transport" true
|
||||
"metabase.mcp.v2.common" true
|
||||
"metabase.mcp.v2.tools.browse" true
|
||||
"metabase-enterprise.mcp.v2.common" true
|
||||
"metabase.mcp.settings" false
|
||||
"metabase.mcp.validation" false
|
||||
"metabase-enterprise.mcp.usage" false
|
||||
"metabase.mcp.transport-test" false
|
||||
"metabase.mcp.v2.tools.browse-test" false
|
||||
"metabase-enterprise.mcp.v2.tools.browse-test" false
|
||||
"metabase.queries.models.card" false}]
|
||||
(testing ns-name
|
||||
(is (= enabled? (linter-enabled-in-ns? config :metabase/agent-message-exit ns-name))))))
|
||||
(testing "GHY-4544: `:empty-hint` is checked on `list-content` and on wrappers passing their options map to it"
|
||||
(doseq [fn-sym '[metabase.mcp.v2.common/list-content
|
||||
metabase.mcp.v2.tools.browse/paged-list-content]]
|
||||
(testing fn-sym
|
||||
(is (= 'hooks.metabase.agent-message/lint-list-content
|
||||
(get-in config [:hooks :analyze-call fn-sym]))))))
|
||||
(testing (str "GHY-4544: `check-execute-sql-enabled!` isn't checked, since it interpolates its subject into "
|
||||
"its own `msg` and a plain string is the intended way to name the tool")
|
||||
(is (nil? (get-in config [:hooks
|
||||
:analyze-call
|
||||
(symbol "metabase.mcp.v2.queries" "check-execute-sql-enabled!")]))))))
|
||||
|
||||
(deftest ^:parallel scope-test
|
||||
(testing "GHY-4544: the hooks leave scoping to config, reporting in any namespace where their linter is enabled"
|
||||
(doseq [[hook form] hook-cases
|
||||
[ns filename] [['metabase.mcp.v2.tools.browse "src/metabase/mcp/v2/tools/browse.clj"]
|
||||
['metabase.queries.models.card "src/metabase/queries/models/card.clj"]
|
||||
['metabase.mcp.v2.tools.browse-test "test/metabase/mcp/v2/tools/browse_test.clj"]]]
|
||||
(testing (str ns " " (pr-str form))
|
||||
(is (= 1 (count (findings hook form :warning ns filename)))))))
|
||||
(testing "nothing is reported when the linter is off or unconfigured"
|
||||
(doseq [[hook form] hook-cases
|
||||
level [:off nil]]
|
||||
(testing (str level " " (pr-str form))
|
||||
(is (empty? (findings hook form level)))))))
|
||||
|
||||
(deftest ^:parallel msg-test
|
||||
(testing "GHY-4544: a vector of single-line string literals with matching arguments is accepted"
|
||||
(is (empty? (findings agent-message/lint-msg
|
||||
'(msg ["Table %s: %d of %d fields." "Continue with `offset: %d`."] t n total next)))))
|
||||
(testing "specifiers that consume no argument aren't counted, and an explicit index can repeat an argument"
|
||||
(is (empty? (findings agent-message/lint-msg '(msg ["100%% done."]))))
|
||||
(is (empty? (findings agent-message/lint-msg '(msg ["%1$s, then %1$s again, then %<s"] x)))))
|
||||
(testing "the lines must be a literal vector"
|
||||
(is (=? [#".*vector.*"]
|
||||
(findings agent-message/lint-msg '(msg lines x))))
|
||||
(is (=? [#".*vector.*"]
|
||||
(findings agent-message/lint-msg '(msg "one line" x)))))
|
||||
(testing "a line may be split across a `str` of string literals"
|
||||
(is (empty? (findings agent-message/lint-msg '(msg [(str "Table %s has " "%d fields.")] t n))))
|
||||
(is (=? [#".*1 argument.*2.*"]
|
||||
(findings agent-message/lint-msg '(msg [(str "only " "%s")] a b)))))
|
||||
(testing "each line must be a string literal or a `str` of string literals"
|
||||
(is (=? [#".*string literal or a `str` of string literals.*"]
|
||||
(findings agent-message/lint-msg '(msg ["ok" line]))))
|
||||
(is (=? [#".*string literal or a `str` of string literals.*"]
|
||||
(findings agent-message/lint-msg '(msg [(str "split " x)])))))
|
||||
(testing "a line can't contain a line break, control, or private-use character; each line is its own string"
|
||||
(is (=? [#".*own string.*"]
|
||||
(findings agent-message/lint-msg '(msg ["first\nsecond"]))))
|
||||
(is (=? [#".*own string.*"]
|
||||
(findings agent-message/lint-msg '(msg [(str "first\n" "second")]))))
|
||||
(is (=? [#".*own string.*"]
|
||||
(findings agent-message/lint-msg (list 'msg [(str "first" (char 0x2028) "second")]))))
|
||||
(is (=? [#".*own string.*"]
|
||||
(findings agent-message/lint-msg '(msg ["tab\tseparated"]))))
|
||||
(is (=? [#".*own string.*"]
|
||||
(findings agent-message/lint-msg '(msg ["done.%n"]))))
|
||||
(testing "GHY-4544: a private-use character, which the truncation markers use, is flagged too"
|
||||
(is (=? [#".*own string.*"]
|
||||
(findings agent-message/lint-msg (list 'msg [(str "first" (char 0xE000) "second")]))))
|
||||
(is (=? [#".*own string.*"]
|
||||
(findings agent-message/lint-msg (list 'msg [(str "marked" (char 0xF8FF))]))))))
|
||||
(testing "the argument count must match the specifiers across all lines"
|
||||
(is (=? [#".*2 arguments.*1.*"]
|
||||
(findings agent-message/lint-msg '(msg ["%s and" "%s"] a))))
|
||||
(is (=? [#".*1 argument.*2.*"]
|
||||
(findings agent-message/lint-msg '(msg ["only %s"] a b)))))
|
||||
(testing (str "GHY-4544: a reference `String/format` can't resolve is flagged, since the runtime would fall back "
|
||||
"to rendering every part quoted")
|
||||
(testing "`%<` with no specifier before it to repeat, which counts as no argument at all"
|
||||
(is (=? [#".*can't exist.*"]
|
||||
(findings agent-message/lint-msg '(msg ["Value: %<s"]))))
|
||||
(is (=? [#".*can't exist.*"]
|
||||
(findings agent-message/lint-msg '(msg ["100%% done," "then %<d"])))))
|
||||
(testing "an explicit index of zero, which is below the first argument"
|
||||
(is (=? [#".*can't exist.*"]
|
||||
(findings agent-message/lint-msg '(msg ["Value: %0$s"]))))
|
||||
(is (=? [#".*can't exist.*"]
|
||||
(findings agent-message/lint-msg '(msg ["Value: %0$s"] x)))))
|
||||
(testing "a `%<` that repeats the first argument is still accepted"
|
||||
(is (empty? (findings agent-message/lint-msg '(msg ["%s then %<s"] x)))))))
|
||||
|
||||
(deftest ^:parallel msg-specifier-test
|
||||
(testing "GHY-4544: only `%s` with no flags but `<`, `%d` with flags and width, and `%%` are allowed"
|
||||
(doseq [line ["%s" "%d" "%1$s" "%,d" "%05d" "%-8d" "%(d" "%1$,10d"
|
||||
"%s again %<s" "%d again %<d" "%s (100%%)"]]
|
||||
(testing line
|
||||
(is (empty? (findings agent-message/lint-msg (list 'msg [(str "Value: " line)] 'x)))))))
|
||||
(testing "GHY-4544: any other conversion, or a `%s` or `%d` that reshapes its value, is flagged"
|
||||
(doseq [line ["%b" "%h" "%c" "%x" "%X" "%f" "%e" "%o" "%g" "%a" "%tY" "%TY" "%B"
|
||||
"%S" "%.20s" "%10s" "%-5s" "%#s" "%1$.3s" "%-10S" "%.1f" "%.2d" "%1$.2d"]]
|
||||
(testing line
|
||||
(is (=? [#"A `msg` line may only use `%s`, `%d`, or `%%`; .* would alter or hide the value\."]
|
||||
(findings agent-message/lint-msg (list 'msg [(str "Value: " line)] 'x)))))))
|
||||
(testing "GHY-4544: a `%` that doesn't begin a valid format specifier is flagged, since `String/format` rejects it"
|
||||
(doseq [line ["Braces %}" "Dot %." "Trailing %" "Space % " "Unknown %q" "Bad date %tq" "%" "%1$"
|
||||
;; GHY-4544: an argument index too large to be one is malformed, not an arithmetic overflow
|
||||
"%999999999999999999999999$s"]]
|
||||
(testing (pr-str line)
|
||||
(is (=? [#"A `%` in a `msg` line must begin a format specifier.*`%%`.*"]
|
||||
(findings agent-message/lint-msg (list 'msg [line]))))))
|
||||
(testing "an index at the digit cap still parses, and is reported as an argument count"
|
||||
(is (=? [#".*999999999 arguments.*"]
|
||||
(findings agent-message/lint-msg '(msg ["%999999999$s"] x))))))
|
||||
(testing "valid specifiers, including an escaped percent sign, aren't flagged as malformed"
|
||||
(doseq [form ['(msg ["100%% done"])
|
||||
'(msg ["%,d rows"] n)
|
||||
'(msg ["%1$s, then %<s"] x)
|
||||
'(msg [(str "50%" "% of %s")] x)]]
|
||||
(testing (pr-str form)
|
||||
(is (empty? (findings agent-message/lint-msg form)))))))
|
||||
|
||||
(def ^:private stringy-texts
|
||||
"Message arguments built as text rather than with `msg`."
|
||||
['"Query failed."
|
||||
'(str "Query failed: " err)
|
||||
'(format "Table %s not found" (pr-str t))
|
||||
'(cond-> (json/encode body) message (str "\n" message))
|
||||
'(if x "a" (str "b" y))
|
||||
'(if x (msg ["a"]) (str "b" y))
|
||||
'(when-let [e (:error r)] (str/join ", " e))
|
||||
'(or (:message r) (tru "Unknown error"))
|
||||
'(let [s (pr-str x)] (str "Bad value " s))
|
||||
'(cond (:a x) (msg ["a"]) :else "b")
|
||||
'(condp = x :a (str "a" y) (msg ["b"]))
|
||||
'(condp = x :a (msg ["a"]) "b")
|
||||
'(case x :a (msg ["a"]) "b")
|
||||
'(case x :a "a" :b (msg ["b"]))
|
||||
'(->> bad (map name) (str/join ", "))
|
||||
'(-> x (str " suffix"))
|
||||
'(-> x name str)
|
||||
'(some-> e ex-message)
|
||||
'(some->> xs (map name) (clojure.string/join ", "))
|
||||
'(cond->> xs x (map name) y (str/join ", "))
|
||||
'(cond-> "x" y (subs 1))
|
||||
'(as-> x v (assoc v :a 1) (json/encode v))
|
||||
'(doto (str "a" b) println)
|
||||
'(let [s x] (-> s (format y)))])
|
||||
|
||||
(def ^:private message-texts
|
||||
"Message arguments that aren't syntactically text."
|
||||
['(msg ["Table %s not found."] t)
|
||||
'message
|
||||
'(if x (msg ["a"]) (msg ["b"]))
|
||||
'(render-error e)
|
||||
'(cond (str x) (msg ["a"]) :else (msg ["b"]))
|
||||
'(condp = x "a" (msg ["a"]) (msg ["b"]))
|
||||
'(condp get x "a" :>> render-error (msg ["b"]))
|
||||
'(case x "a" (msg ["a"]) ("b" "c") (msg ["b"]))
|
||||
'(-> "x" count)
|
||||
'(->> "x" (map str) set)
|
||||
'(some-> (str "x") render-error)
|
||||
'(-> x)
|
||||
'(-> e :message)
|
||||
'(cond-> {:a 1} x (assoc :b "c"))
|
||||
'(as-> "x" v (count v))
|
||||
'(doto x (println "a"))])
|
||||
|
||||
(deftest ^:parallel teaching-exit-test
|
||||
(doseq [fn-name ["throw-teaching-error" "error-content"]]
|
||||
(testing (str "GHY-4544: text passed to `" fn-name "` is flagged")
|
||||
(doseq [text stringy-texts]
|
||||
(testing (pr-str text)
|
||||
(is (=? [(re-pattern (str ".*passes text to `" fn-name "`.*`msg`.*"))]
|
||||
(findings agent-message/lint-teaching-exit (list (symbol "common" fn-name) text)))))))
|
||||
(testing (str "`" fn-name "` accepts a `msg`, a symbol, and other calls")
|
||||
(doseq [text message-texts]
|
||||
(testing (pr-str text)
|
||||
(is (empty? (findings agent-message/lint-teaching-exit
|
||||
(list (symbol fn-name) text {:status-code 404}))))))))
|
||||
(testing "only the message argument is checked"
|
||||
(is (empty? (findings agent-message/lint-teaching-exit '(error-content (msg ["x"]) (str "code"))))))
|
||||
(testing "nothing is reported when the linter is off"
|
||||
(is (empty? (findings agent-message/lint-teaching-exit '(throw-teaching-error "x") :off)))))
|
||||
|
||||
(def ^:private helper-cases
|
||||
"`[hook fn-name form-fn]`: a message-taking helper's hook, the name its finding cites, and a function of a message
|
||||
returning a call that passes it to the helper."
|
||||
[[agent-message/lint-op-error "op-error!" #(list 'dashboard-ops/op-error! 'idx "add_link" %)]
|
||||
[agent-message/lint-ellipsize "ellipsize" #(list 'common/ellipsize % 300)]
|
||||
[agent-message/lint-list-content "list-content" #(list 'common/list-content 'rows 0 {:offset 0 :empty-hint %})]
|
||||
[agent-message/lint-list-content "paged-list-content"
|
||||
#(list 'paged-list-content 'args 'rows {:empty-hint %} 'identity)]])
|
||||
|
||||
(deftest ^:parallel message-helper-test
|
||||
(doseq [[hook fn-name form-fn] helper-cases]
|
||||
(testing (str "GHY-4544: text passed as `" fn-name "`'s message is flagged")
|
||||
(doseq [text stringy-texts]
|
||||
(testing (pr-str text)
|
||||
(is (=? [(re-pattern (str ".*passes text to `" fn-name "`.*`msg`.*"))]
|
||||
(findings hook (form-fn text)))))))
|
||||
(testing (str "`" fn-name "` accepts a `msg`, a symbol, and other calls as its message")
|
||||
(doseq [text message-texts]
|
||||
(testing (pr-str text)
|
||||
(is (empty? (findings hook (form-fn text))))))))
|
||||
(testing "only the message argument is checked"
|
||||
(is (empty? (findings agent-message/lint-op-error '(op-error! (str "0") (str "op") (msg ["x"])))))
|
||||
(is (empty? (findings agent-message/lint-ellipsize '(ellipsize (msg ["x"]) (str "300")))))
|
||||
(is (empty? (findings agent-message/lint-list-content '(list-content (str "data") 0 {:param (str "x")}))))
|
||||
(is (empty? (findings agent-message/lint-list-content '(list-content data 0 opts))))
|
||||
(is (empty? (findings agent-message/lint-list-content '(list-content data 0))))))
|
||||
|
||||
(deftest ^:parallel jsonrpc-error-test
|
||||
(testing "GHY-4544: text passed as the JSON-RPC error message is flagged"
|
||||
(doseq [text stringy-texts]
|
||||
(testing (pr-str text)
|
||||
(is (=? [#".*passes text to `jsonrpc-error`.*`msg`.*"]
|
||||
(findings agent-message/lint-jsonrpc-error (list 'transport/jsonrpc-error 'id -32602 text)))))))
|
||||
(testing "a `msg`, symbol, or other call as the message is accepted"
|
||||
(doseq [text message-texts]
|
||||
(testing (pr-str text)
|
||||
(is (empty? (findings agent-message/lint-jsonrpc-error (list 'jsonrpc-error 'id -32602 text)))))))
|
||||
(testing "only the third argument is the message"
|
||||
(is (empty? (findings agent-message/lint-jsonrpc-error '(jsonrpc-error "id" (str "code") (msg ["x"])))))))
|
||||
|
||||
(deftest ^:parallel success-content-test
|
||||
(testing "GHY-4544: text passed to `success-content` is flagged"
|
||||
(doseq [text stringy-texts]
|
||||
(testing (pr-str text)
|
||||
(is (=? [#"`success-content` takes data to JSON-encode or a `msg`.*"]
|
||||
(findings agent-message/lint-success-content (list 'common/success-content text)))))))
|
||||
(testing "a payload, a `msg`, or a symbol is accepted"
|
||||
(doseq [text (conj message-texts '{:data rows} '[a b])]
|
||||
(testing (pr-str text)
|
||||
(is (empty? (findings agent-message/lint-success-content (list 'success-content text))))))
|
||||
(is (empty? (findings agent-message/lint-success-content
|
||||
'(success-content {:data rows} (str "structured")))))))
|
||||
|
||||
(deftest ^:parallel ex-info-test
|
||||
(testing "GHY-4544: an ex-info carrying a 4xx status code or an error code is flagged"
|
||||
(doseq [form ["(ex-info \"Card not found\" {:status-code 404})"
|
||||
"(ex-info (str \"Bad \" x) {:status-code 400 :field f})"
|
||||
"(ex-info \"Nope\" {::common/error-code c})"
|
||||
"(ex-info \"Nope\" {:error-code c})"
|
||||
"(ex-info (msg [\"Nope\"]) {:error-code c})"]]
|
||||
(testing form
|
||||
(is (=? [#".*`throw-teaching-error` and a `msg`.*"]
|
||||
(findings agent-message/lint-ex-info form))))))
|
||||
(testing "other ex-info calls aren't flagged"
|
||||
(doseq [form ["(ex-info \"boom\" {:status-code 500})"
|
||||
"(ex-info \"boom\" {:status-code code})"
|
||||
"(ex-info \"boom\" {:foo 1})"
|
||||
"(ex-info \"boom\" data)"
|
||||
"(ex-info \"boom\" (merge {:status-code 400} data))"
|
||||
"(ex-info \"boom\")"]]
|
||||
(testing form
|
||||
(is (empty? (findings agent-message/lint-ex-info form))))))
|
||||
(testing "nothing is reported when the linter is off"
|
||||
(is (empty? (findings agent-message/lint-ex-info "(ex-info \"x\" {:status-code 404})" :off)))))
|
||||
@@ -1,138 +0,0 @@
|
||||
(ns hooks.metabase.toucan.db-ns-test
|
||||
(:require
|
||||
[clj-kondo.hooks-api :as hooks]
|
||||
[clj-kondo.impl.utils]
|
||||
[clojure.test :refer :all]
|
||||
[hooks.metabase.toucan.db-ns :as toucan.db-ns]))
|
||||
|
||||
(defn- confinement-findings
|
||||
"Only the `t2-query-namespace` findings, so these tests are not affected by the query-hygiene
|
||||
linter that shares this hook."
|
||||
[findings]
|
||||
(filter #(= :metabase/t2-query-namespace (:type %)) findings))
|
||||
|
||||
(defn- lint-query-call [form ns-sym & [filename modules]]
|
||||
(binding [clj-kondo.impl.utils/*ctx* {:config {:linters {:metabase/t2-query-namespace {:level :warning}
|
||||
:metabase/unsafe-app-db-query {:level :warning}}}
|
||||
:ignores (atom nil)
|
||||
:findings (atom [])
|
||||
:namespaces (atom {})}]
|
||||
(let [input {:node (hooks/parse-string (pr-str form))
|
||||
:ns ns-sym
|
||||
:filename (or filename "src/metabase/foo/bar.clj")
|
||||
:config {:metabase/modules modules}}
|
||||
output (toucan.db-ns/lint-query-call input)]
|
||||
(is (identical? (:node input) (:node output))
|
||||
"the hook must return the node unchanged so Kondo's normal analysis still runs")
|
||||
@(:findings clj-kondo.impl.utils/*ctx*))))
|
||||
|
||||
(deftest ^:parallel t2-query-must-live-in-db-namespace-test
|
||||
(testing "a query call outside a db namespace is flagged"
|
||||
(is (=? [{:type :metabase/t2-query-namespace
|
||||
:message #".*`t2/select-one`.*"}]
|
||||
(lint-query-call '(t2/select-one :model/Card :id 1) 'metabase.queries.models.card))))
|
||||
(testing "metabase.<module>.db is allowed"
|
||||
(is (empty? (confinement-findings (lint-query-call '(t2/select-one :model/Card :id 1) 'metabase.queries.db)))))
|
||||
(testing "metabase-enterprise.<module>.db is allowed"
|
||||
(is (empty? (confinement-findings (lint-query-call '(t2/select-one :model/Card :id 1) 'metabase-enterprise.sandbox.db)))))
|
||||
(testing "metabase.driver.<driver>.db is allowed for driver modules"
|
||||
(is (empty? (confinement-findings (lint-query-call '(t2/select-one :model/Database :id 1) 'metabase.driver.bigquery-cloud-sdk.db)))))
|
||||
(testing "a nested db namespace is not a module db namespace"
|
||||
(is (=? [{:type :metabase/t2-query-namespace}]
|
||||
(lint-query-call '(t2/query {:select [:*]}) 'metabase.queries.models.db))))
|
||||
(testing "a namespace merely ending in db is not a module db namespace"
|
||||
(is (=? [{:type :metabase/t2-query-namespace}]
|
||||
(lint-query-call '(t2/delete! :model/Card :id 1) 'metabase.queries.dashboards-db))))
|
||||
(testing "files in a test source tree are exempt even when the namespace name doesn't look like a test"
|
||||
(is (empty? (lint-query-call '(t2/update! :model/User 1 {:sso_source nil}) 'metabase.sso.test-helpers
|
||||
"test/metabase/sso/test_helpers.clj")))
|
||||
(is (empty? (lint-query-call '(t2/select :model/Card) 'metabase-enterprise.remote-sync.test-helpers
|
||||
"/repo/enterprise/backend/test/metabase_enterprise/remote_sync/test_helpers.clj"))))
|
||||
(testing "app-db wrappers around Toucan 2 are reported the same way"
|
||||
(is (=? [{:type :metabase/t2-query-namespace
|
||||
:message #".*`mdb/update-or-insert!`.*"}]
|
||||
(lint-query-call '(mdb/update-or-insert! :model/Foo {:id 1}) 'metabase.foo.models.foo))))
|
||||
(testing "fully-qualified calls are reported by their written name"
|
||||
(is (=? [{:type :metabase/t2-query-namespace
|
||||
:message #".*`toucan2.core/insert!`.*"}]
|
||||
(lint-query-call '(toucan2.core/insert! :model/Card {}) 'metabase.queries.models.card)))))
|
||||
|
||||
(deftest ^:parallel t2-query-namespace-follows-the-module-tree-test
|
||||
(let [modules '{queries {}
|
||||
metabot {}
|
||||
metabot.llm {}
|
||||
lib.be {:ns-prefix "metabase.lib-be"}
|
||||
enterprise/sandbox {}}]
|
||||
(testing "a nested module owns its own db namespace"
|
||||
(is (empty? (lint-query-call '(t2/select :model/Card) 'metabase.metabot.llm.db nil modules))))
|
||||
(testing "the parent's db namespace stays its own"
|
||||
(is (empty? (lint-query-call '(t2/select :model/Card) 'metabase.metabot.db nil modules))))
|
||||
(testing "a module with an explicit :ns-prefix is resolved through it, not its dotted name"
|
||||
(is (empty? (lint-query-call '(t2/select :model/Card) 'metabase.lib-be.db nil modules)))
|
||||
(is (=? [{:type :metabase/t2-query-namespace}]
|
||||
(lint-query-call '(t2/select :model/Card) 'metabase.lib.be.db nil modules))))
|
||||
(testing "a db namespace under a directory naming no module is still a finding"
|
||||
(is (=? [{:type :metabase/t2-query-namespace}]
|
||||
(lint-query-call '(t2/query {:select [:*]}) 'metabase.queries.models.db nil modules)))
|
||||
(is (=? [{:type :metabase/t2-query-namespace}]
|
||||
(lint-query-call '(t2/query {:select [:*]}) 'metabase.metabot.llm.models.db nil modules))))
|
||||
(testing "enterprise modules resolve through the metabase-enterprise root"
|
||||
(is (empty? (lint-query-call '(t2/select :model/Card) 'metabase-enterprise.sandbox.db nil modules))))))
|
||||
|
||||
(deftest ^:parallel unmarked-value-in-a-db-namespace-test
|
||||
(testing "a symbol reaching a value slot is flagged"
|
||||
(is (=? [{:type :metabase/unsafe-app-db-query
|
||||
:message #"`locale` reaches a SQL value slot unmarked.*"}]
|
||||
(lint-query-call '(t2/select :model/X {:where [:= :locale locale]}) 'metabase.foo.db))))
|
||||
(testing "a marked value is not flagged"
|
||||
(is (empty? (lint-query-call '(t2/select :model/X {:where [:= :locale [:auto/param locale]]})
|
||||
'metabase.foo.db))))
|
||||
(testing "a literal cannot carry a request value and is not flagged"
|
||||
(is (empty? (lint-query-call '(t2/select :model/X {:where [:= :locale "de"]}) 'metabase.foo.db))))
|
||||
(testing "a column reference is not a value"
|
||||
(is (empty? (lint-query-call '(t2/select :model/X {:where [:= :a.id :b.id]}) 'metabase.foo.db))))
|
||||
(testing "values nested under a boolean connective are reached"
|
||||
(is (=? [{:message #"`b`.*"}]
|
||||
(lint-query-call '(t2/select :model/X {:where [:and [:= :x [:auto/param a]] [:= :y b]]})
|
||||
'metabase.foo.db))))
|
||||
(testing "only db namespaces are linted for unmarked values"
|
||||
(is (empty? (filter #(= :metabase/unsafe-app-db-query (:type %))
|
||||
(lint-query-call '(t2/select :model/X {:where [:= :locale locale]})
|
||||
'metabase.foo.models.thing)))))
|
||||
(testing "a test source tree is exempt"
|
||||
(is (empty? (lint-query-call '(t2/select :model/X {:where [:= :locale locale]})
|
||||
'metabase.foo.db "test/metabase/foo/db_test.clj")))))
|
||||
|
||||
(deftest ^:parallel kv-arg-style-test
|
||||
(testing "a query written as :column value pairs is flagged whatever the values are"
|
||||
(are [form] (=? [{:type :metabase/unsafe-app-db-query, :message #"Pass this query a map.*"}]
|
||||
(lint-query-call form 'metabase.foo.db))
|
||||
'(t2/select :model/X :locale locale)
|
||||
'(t2/select :model/X :archived false)
|
||||
'(t2/select :model/X :locale [:auto/param locale])
|
||||
'(t2/select-one :model/X :id (long id))))
|
||||
(testing "one finding per call, not one per pair"
|
||||
(is (= 1 (count (lint-query-call '(t2/select :model/X :locale locale :msgid msgid)
|
||||
'metabase.foo.db)))))
|
||||
(testing "a fn with an argument before the model is still recognised"
|
||||
(is (=? [{:message #"Pass this query a map.*"}]
|
||||
(lint-query-call '(t2/select-one-fn :value :model/Setting :key k) 'metabase.foo.db))))
|
||||
(testing "a map query is not flagged"
|
||||
(are [form] (empty? (lint-query-call form 'metabase.foo.db))
|
||||
'(t2/select :model/X {:where [:= :locale [:auto/param locale]]})
|
||||
'(t2/update! :model/X {:locale [:auto/param locale]} {:msgstr "x"}))))
|
||||
|
||||
(deftest ^:parallel write-calls-are-not-linted-for-values-test
|
||||
(testing "an insert's values are written, not filtered on, so they are not flagged"
|
||||
(are [form] (empty? (filter #(= :metabase/unsafe-app-db-query (:type %))
|
||||
(lint-query-call form 'metabase.foo.db)))
|
||||
'(t2/insert! :model/X :key k :value v)
|
||||
'(t2/insert-returning-instances! :model/X :key k :value v)))
|
||||
(testing "a select's values are still flagged"
|
||||
(is (=? [{:type :metabase/unsafe-app-db-query}]
|
||||
(lint-query-call '(t2/select :model/X :key k) 'metabase.foo.db)))))
|
||||
|
||||
(deftest ^:parallel operator-arity-test
|
||||
(testing "a value operator with an unexpected arity still has its args examined"
|
||||
(is (seq (lint-query-call '(t2/select :model/X {:where [:= :a b c]}) 'metabase.foo.db)))
|
||||
(is (seq (lint-query-call '(t2/select :model/X {:where [:between :a lo hi]}) 'metabase.foo.db)))))
|
||||
@@ -1,59 +0,0 @@
|
||||
(ns hooks.metabase.util.match-test
|
||||
(:require
|
||||
[clj-kondo.hooks-api :as hooks]
|
||||
[clj-kondo.impl.utils]
|
||||
[clojure.test :refer :all]
|
||||
[hooks.metabase.util.match :as sut]))
|
||||
|
||||
(defn- expand [src]
|
||||
(-> (sut/match-one {:node (hooks/parse-string (pr-str src))})
|
||||
:node
|
||||
hooks/sexpr))
|
||||
|
||||
(defn- lint [src]
|
||||
(binding [clj-kondo.impl.utils/*ctx* {:config {:linters {:metabase/match-odd-clauses {:level :warning}}}
|
||||
:ignores (atom nil)
|
||||
:findings (atom [])
|
||||
:namespaces (atom {})}]
|
||||
(expand src)
|
||||
@(:findings clj-kondo.impl.utils/*ctx*)))
|
||||
|
||||
(deftest odd-clauses-triggers-warning-test
|
||||
(is (=? [{:type :metabase/match-odd-clauses}]
|
||||
(lint '(metabase.util.match/match-one [] [a b c])))))
|
||||
|
||||
(deftest expansion-test
|
||||
(testing "simple expansion example"
|
||||
(is (= '(let [_ []
|
||||
&match (rand-nth ())
|
||||
&parents (rand-nth ())
|
||||
&recur (rand-nth ())
|
||||
_ &match
|
||||
_ &parents
|
||||
_ &recur
|
||||
a (rand-nth ())
|
||||
b (rand-nth ())
|
||||
c (rand-nth ())]
|
||||
(clojure.core/or (+ a b) (rand-nth ())))
|
||||
(expand '(match-one [] [a b c] (+ a b))))))
|
||||
(testing "complicated expansion example that uses most match-one macro features, and also binding deduplication by the rewriter"
|
||||
(is (= '(let [_ []
|
||||
&match (rand-nth ())
|
||||
&parents (rand-nth ())
|
||||
&recur (rand-nth ())
|
||||
_ &match
|
||||
_ &parents
|
||||
_ &recur
|
||||
val1 (rand-nth ())
|
||||
a (rand-nth ())
|
||||
_ #{maybe important}
|
||||
rst (rand-nth ())
|
||||
x (rand-nth ())
|
||||
_ (= x 3)
|
||||
b (rand-nth ())]
|
||||
(clojure.core/or (+ a val1) (rand-nth ())))
|
||||
(expand '(match-one []
|
||||
(:or [{:key1 val1, :key2 &truthy, :key3 _} _ a #{maybe important} & rst]
|
||||
(:and (x :guard (= x 3))
|
||||
[a b]))
|
||||
(+ a val1)))))))
|
||||
@@ -1,27 +0,0 @@
|
||||
{:aliases
|
||||
{;; defendpoint shape: (api.macros/defendpoint METHOD URL DOCSTRING? [args] body)
|
||||
;; No name slot — synthesize :route [METHOD "path"] from first 2 args.
|
||||
"defendpoint"
|
||||
{:fields {:method (fn [z] (some-> z z/down z/right z/sexpr name str/upper-case))
|
||||
:path (fn [z] (-> z z/down z/right z/right z/sexpr))}}
|
||||
|
||||
;; defenterprise shape:
|
||||
;; (defenterprise NAME DOCSTRING? EE-NAMESPACE [args] body)
|
||||
"defenterprise"
|
||||
{:fields {:name ->defn-name
|
||||
:ee-namespace (fn [z]
|
||||
;; Skip name + optional docstring; the next
|
||||
;; symbol is the EE namespace.
|
||||
(let [name-z (-> z z/down z/right)
|
||||
after-doc (z/right name-z)
|
||||
candidate (if (string? (try (z/sexpr after-doc)
|
||||
(catch Exception _ nil)))
|
||||
(z/right after-doc)
|
||||
after-doc)]
|
||||
(try (let [v (z/sexpr candidate)]
|
||||
(when (symbol? v) v))
|
||||
(catch Exception _ nil))))
|
||||
:arglist ->defn-arg-list}}
|
||||
|
||||
"defsetting"
|
||||
{:fields {:name ->defn-name}}}}
|
||||
69
.cljfmt.edn
69
.cljfmt.edn
@@ -1,10 +1,12 @@
|
||||
;; Cljfmt config. See https://github.com/weavejester/cljfmt#formatting-options
|
||||
{:function-arguments-indentation :community
|
||||
:indent-line-comments? true
|
||||
:normalize-newlines-at-file-end? true
|
||||
:parallel? true
|
||||
:remove-blank-lines-in-forms? true
|
||||
:sort-ns-references? true
|
||||
{:sort-ns-references?
|
||||
true
|
||||
|
||||
:function-arguments-indentation
|
||||
:community
|
||||
|
||||
:parallel?
|
||||
true
|
||||
|
||||
:paths
|
||||
["src"
|
||||
@@ -12,65 +14,12 @@
|
||||
"enterprise/backend"
|
||||
"bin/build"
|
||||
"bin/lint-migrations-file"
|
||||
"bin/load-namespaces"
|
||||
"bin/release-list"
|
||||
#_"mage/src"
|
||||
#_"mage/test"
|
||||
"modules/drivers"
|
||||
".clj-kondo/src"
|
||||
".clj-kondo/test"
|
||||
#_"dev"]
|
||||
|
||||
;; a map of symbols that tell cljfmt which forms are allowed to have blank lines inside of them. The value may be
|
||||
;; either `:all`, which means blank lines are allowed between all elements in the form, e.g. `{cond :all}` to allow
|
||||
;; blank lines between any of the elements inside cond; or it may be a set of element indexes that are allowed to
|
||||
;; have blank lines, e.g. `{let #{0}}`, to allow blank lines in the binding of a `let` form.
|
||||
:extra-blank-line-forms
|
||||
{are :all
|
||||
case :all
|
||||
cond-> :all
|
||||
cond->> :all
|
||||
condp :all
|
||||
clojure.spec.alpha/or :all
|
||||
clojure.test/are :all
|
||||
metabase.driver-api.core/match-many :all
|
||||
metabase.driver-api.core/match-one :all
|
||||
metabase.driver-api.core/replace :all
|
||||
metabase.driver-api.core/replace-in :all
|
||||
metabase.lib.drill-thru.test-util/test-drill-variants-with-merged-args :all
|
||||
metabase.lib.drill-thru.test-util/test-returns-drill :all
|
||||
metabase.test/with-temp #{0}
|
||||
metabase.util.match/match-many :all
|
||||
metabase.util.match/match-one :all
|
||||
metabase.util.match/replace :all
|
||||
metabase.util.match/replace-in :all
|
||||
;; TODO (Cam 2026-05-21) I'm not really sure about `defprotocol`, `deftype`, `defrecord`, `proxy`, `reify`, etc., I
|
||||
;; think idiomatically these should probably not have blank lines but we can worry about that later
|
||||
defprotocol :all
|
||||
defrecord :all
|
||||
deftype :all
|
||||
extend :all
|
||||
extend-protocol :all
|
||||
potemkin.types/def-map-type :all
|
||||
potemkin.types/defprotocol+ :all
|
||||
potemkin.types/defrecord+ :all
|
||||
potemkin.types/deftype+ :all
|
||||
potemkin/def-map-type :all
|
||||
potemkin/defprotocol+ :all
|
||||
potemkin/defrecord+ :all
|
||||
potemkin/deftype+ :all
|
||||
proxy :all
|
||||
reify :all
|
||||
;; TODO (Cam 2026-05-21) -- save these for a follow-on PR
|
||||
defmacro :all
|
||||
defmethod :all
|
||||
defn :all
|
||||
defn- :all
|
||||
fn :all
|
||||
metabase.util.malli/defmethod :all
|
||||
metabase.util.malli/defn :all
|
||||
metabase.util.malli/defn- :all}
|
||||
|
||||
;; See https://github.com/weavejester/cljfmt/blob/master/cljfmt/resources/cljfmt/indents/clojure.clj for the default
|
||||
;; definitions
|
||||
;;
|
||||
@@ -88,7 +37,7 @@
|
||||
;; `:inner` indentation is like `:defn` indentation. `[[:inner 0]]` (default for `defn`) means all arguments at depth
|
||||
;; zero get indented 2 spaces if not on the first line regardless of number of elements on the first line.
|
||||
:extra-indents
|
||||
{ ;; clojure.core stuff
|
||||
{;; clojure.core stuff
|
||||
fn* [[:inner 0]]
|
||||
let* [[:block 1]]
|
||||
with-meta [[:default]]
|
||||
|
||||
@@ -1,28 +0,0 @@
|
||||
# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json
|
||||
reviews:
|
||||
high_level_summary: false
|
||||
review_status: false
|
||||
|
||||
auto_review:
|
||||
enabled: false
|
||||
drafts: true
|
||||
labels:
|
||||
- "coderabbit"
|
||||
|
||||
path_filters:
|
||||
- "!**/*.snap"
|
||||
- "!**/*.po"
|
||||
- "!**/bun.lock"
|
||||
- "!**/package-lock.json"
|
||||
|
||||
tools:
|
||||
eslint:
|
||||
enabled: false
|
||||
|
||||
knowledge_base:
|
||||
code_guidelines:
|
||||
filePatterns:
|
||||
- ".claude/skills/clojure-review/SKILL.md"
|
||||
- ".claude/skills/typescript-review/SKILL.md"
|
||||
- ".claude/skills/e2e-test-review/SKILL.md"
|
||||
- ".claude/skills/docs-review/SKILL.md"
|
||||
@@ -16,7 +16,7 @@ skip =
|
||||
frontend/src/metabase/querying/expressions/tokenizer/lezer.js,
|
||||
frontend/src/metabase/query_builder/components/NativeQueryEditor/CodeMirrorEditor/language.ts,
|
||||
frontend/src/metabase/static-viz/constants/char-sizes.ts,
|
||||
frontend/src/metabase/viz-core/lib/mapping_codes.ts,
|
||||
frontend/src/metabase/visualizations/lib/mapping_codes.ts,
|
||||
frontend/*/mocks/*,
|
||||
modules/drivers/mongo/src/metabase/driver/mongo/operators.clj,
|
||||
src/metabase/analyze/classifiers/name.clj,
|
||||
|
||||
10
.config/wt.toml
Normal file
10
.config/wt.toml
Normal file
@@ -0,0 +1,10 @@
|
||||
# worktrunk project config — https://worktrunk.dev
|
||||
# Hooks run when switching, creating, merging, or removing worktrees.
|
||||
|
||||
# Setup tasks for newly-created worktrees. Run once, blocking, in the
|
||||
# new worktree's cwd before post-start background tasks kick off.
|
||||
[pre-start]
|
||||
# Trust mise config so it loads the project's tool versions without prompting.
|
||||
trust-mise = "mise trust"
|
||||
# Install JS deps so husky git hooks get wired up in the new worktree.
|
||||
bun-install = "bun install"
|
||||
@@ -1,35 +0,0 @@
|
||||
// dependency-cruiser config for the actual frontend import graph (what's
|
||||
// imported, vs the *allowed* edges in module-boundaries.mjs). Consumed by
|
||||
// .github/scripts/create-test-plan.ts.
|
||||
//
|
||||
// bunx depcruise frontend/src enterprise/frontend/src .storybook/preview.tsx \
|
||||
// --config .dependency-cruiser.cjs --output-type json > dependency-graph.json
|
||||
|
||||
/** @type {import('dependency-cruiser').IConfiguration} */
|
||||
module.exports = {
|
||||
options: {
|
||||
// Keep only edges between our own source files (drops node_modules etc.),
|
||||
// which keeps the JSON small.
|
||||
includeOnly: "^(frontend/src|enterprise/frontend/src|\\.storybook)/",
|
||||
|
||||
doNotFollow: { path: "node_modules" },
|
||||
|
||||
// No rules here, so the cycle and orphan analysis has no consumer and only costs time.
|
||||
skipAnalysisNotInRules: true,
|
||||
|
||||
// Resolve the `"*": ["./frontend/src/*", ...]` aliases so bare imports like
|
||||
// `metabase/lib/foo` map back to a real file.
|
||||
tsConfig: { fileName: "tsconfig.json" },
|
||||
|
||||
// Keep type-only imports: a type-only dependent still breaks on a type change.
|
||||
tsPreCompilationDeps: true,
|
||||
|
||||
moduleSystems: ["es6", "cjs", "tsd"],
|
||||
|
||||
enhancedResolveOptions: {
|
||||
exportsFields: ["exports"],
|
||||
conditionNames: ["import", "require", "node", "default", "types"],
|
||||
extensions: [".js", ".jsx", ".ts", ".tsx", ".json"],
|
||||
},
|
||||
},
|
||||
};
|
||||
@@ -1,4 +1,4 @@
|
||||
FROM mcr.microsoft.com/vscode/devcontainers/java:25
|
||||
FROM mcr.microsoft.com/vscode/devcontainers/java:21
|
||||
|
||||
# Set up nodesource, install node, bun, fontconfig for static viz, rlwrap for dev ergonomics
|
||||
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
.git
|
||||
.babel_cache
|
||||
target
|
||||
target/*
|
||||
.circleci
|
||||
.cpcache
|
||||
.devcontainer
|
||||
@@ -16,7 +15,6 @@ test_config/*
|
||||
test_modules/*
|
||||
test_resources/*
|
||||
node_modules
|
||||
**/node_modules
|
||||
**metabase.jar
|
||||
|
||||
*.db
|
||||
|
||||
6
.gitattributes
vendored
6
.gitattributes
vendored
@@ -7,9 +7,3 @@
|
||||
#
|
||||
# TODO (Bryan 12/12/25) -- reenable this when we can completely verify correctness
|
||||
# resources/migrations/*_update_migrations.yaml merge=yaml-migrations
|
||||
|
||||
# Generated API documentation/specs — regenerated by the "Auto-update documentation for master"
|
||||
# CI automation, not edited by hand. Mark generated so tools/humans don't review or diff them as source.
|
||||
docs/api.json linguist-generated
|
||||
docs/api.html linguist-generated
|
||||
resources/openapi/openapi.json linguist-generated
|
||||
|
||||
38
.github/CODEOWNERS
vendored
38
.github/CODEOWNERS
vendored
@@ -10,30 +10,15 @@ src/metabase/query_processor @metabase/core-backend-querying-platform
|
||||
src/metabase/graph @metabase/graphy
|
||||
enterprise/backend/src/metabase_enterprise/dependencies @metabase/graphy
|
||||
enterprise/frontend/src/metabase-enterprise/dependencies @metabase/graphy
|
||||
enterprise/frontend/src/metabase-enterprise/monitor/dependency-diagnostics @metabase/graphy
|
||||
|
||||
# @metabase/embedding
|
||||
|
||||
frontend/src/metabase/embedding @metabase/embedding
|
||||
enterprise/frontend/src/embedding-sdk-package @metabase/embedding
|
||||
enterprise/frontend/src/embedding-sdk-bundle @metabase/embedding
|
||||
enterprise/frontend/src/embedding-sdk-shared @metabase/embedding
|
||||
frontend/src/metabase/embedding-sdk @metabase/embedding
|
||||
frontend/src/embedding-sdk-bundle @metabase/embedding
|
||||
frontend/src/embedding-sdk-shared @metabase/embedding
|
||||
frontend/src/metabase/public/containers/PublicOrEmbeddedQuestion @metabase/embedding
|
||||
frontend/src/metabase/public/containers/PublicOrEmbeddedDashboard @metabase/embedding
|
||||
|
||||
enterprise/frontend/src/embedding @metabase/embedding
|
||||
enterprise/frontend/src/embedding-sdk-ee @metabase/embedding
|
||||
enterprise/frontend/src/embedding-sdk-package @metabase/embedding
|
||||
|
||||
enterprise/frontend/src/metabase-enterprise/embedding @metabase/embedding
|
||||
enterprise/frontend/src/metabase-enterprise/embedding_iframe_sdk @metabase/embedding
|
||||
enterprise/frontend/src/metabase-enterprise/embedding_iframe_sdk_setup @metabase/embedding
|
||||
enterprise/frontend/src/metabase-enterprise/embedding-sdk @metabase/embedding
|
||||
|
||||
e2e/test-component @metabase/embedding
|
||||
e2e/test/scenarios/embedding @metabase/embedding
|
||||
e2e/embedding-sdk-host-apps @metabase/embedding
|
||||
|
||||
# @metabase/tech-writers
|
||||
docs @metabase/tech-writers
|
||||
docs/developers-guide # exclude from tech-writer ownership
|
||||
@@ -42,9 +27,6 @@ docs/api.json # auto-generated
|
||||
docs/configuring-metabase/environment-variables.md # auto-generated
|
||||
docs/configuring-metabase/config-template.md # auto-generated
|
||||
docs/installation-and-operation/commands.md # auto-generated
|
||||
docs/ai/providers.md # auto-generated
|
||||
docs/embedding/sdk/api # auto-generated
|
||||
docs/embedding/eajs/snippets # auto-generated
|
||||
|
||||
# @metabase/devexp
|
||||
.github @metabase/devexp
|
||||
@@ -53,17 +35,5 @@ docs/embedding/eajs/snippets # auto-generated
|
||||
src/metabase/analytics/prometheus.clj @metabase/cloud-ops
|
||||
|
||||
# @metabase/metabot-frontend
|
||||
enterprise/frontend/src/metabase-enterprise/metabot @metabase/metabot-frontend
|
||||
enterprise/frontend/src/metabase-enterprise/metabot @metabase/metabot-frontend
|
||||
frontend/src/metabase/metabot @metabase/metabot-frontend
|
||||
|
||||
# @metabase/core-frontend-gadget
|
||||
bin/custom-viz @metabase/core-frontend-gadget
|
||||
docs/developers-guide/custom-visualizations.md @metabase/core-frontend-gadget
|
||||
enterprise/frontend/src/custom-viz @metabase/core-frontend-gadget
|
||||
enterprise/frontend/src/metabase-enterprise/api/custom-viz-plugin.ts @metabase/core-frontend-gadget
|
||||
enterprise/frontend/src/metabase-enterprise/custom_viz @metabase/core-frontend-gadget
|
||||
frontend/src/metabase-types/api/custom-viz-plugin.ts @metabase/core-frontend-gadget
|
||||
frontend/src/metabase/plugins/oss/custom-viz.ts @metabase/core-frontend-gadget
|
||||
frontend/src/metabase/visualizations/custom-visualizations @metabase/core-frontend-gadget
|
||||
frontend/test/__support__/custom-viz-fixtures @metabase/core-frontend-gadget
|
||||
.github/workflows/release-custom-viz.yml @metabase/devexp @metabase/core-frontend-gadget
|
||||
|
||||
2
.github/PULL_REQUEST_TEMPLATE.md
vendored
2
.github/PULL_REQUEST_TEMPLATE.md
vendored
@@ -1,5 +1,5 @@
|
||||
> [!IMPORTANT]
|
||||
> For those employed by Metabase: if you are fixing a P0 or P1 issue, add a `backport` label to this PR. No action is needed otherwise. Refer to the [Backporting](https://app.notion.com/p/metabase/Branching-Strategy-and-Backports-6eb577d5f61142aa960a626d6bbdfeb3?source=copy_link#e71f6e7de2f945d99fbb2d6b764e6f36) section in the Metabase Branching Strategy document for more details. If you're not employed by Metabase, this section does not apply to you, and the label will be taken care of by your reviewer.
|
||||
> For those employed by Metabase: if you are merging into master, please add either a `backport` or a `no-backport` label to this PR. You will not be able to merge until you do this step. Refer to the section [Do I need to backport this PR?](https://www.notion.so/metabase/Metabase-Branching-Strategy-6eb577d5f61142aa960a626d6bbdfeb3?pvs=4#89f80d6f17714a0198aeb66c0efd1b71) in the Metabase Branching Strategy document for more details. If you're not employed by Metabase, this section does not apply to you, and the label will be taken care of by your reviewer.
|
||||
|
||||
> **Warning**
|
||||
>
|
||||
|
||||
19
.github/actionlint.yaml
vendored
19
.github/actionlint.yaml
vendored
@@ -1,19 +0,0 @@
|
||||
self-hosted-runner:
|
||||
# Custom runner labels that actionlint cannot know about. Everything else is
|
||||
# selected through `vars.DEFAULT_RUNNER_KEY` and is not checked.
|
||||
labels:
|
||||
- arm-ubuntu-24-2core
|
||||
|
||||
paths:
|
||||
# GitHub added parallel steps (`parallel`, `background`, `wait`, `wait-all`,
|
||||
# `cancel`) in June 2026, and actionlint 1.7.12 does not know the schema yet:
|
||||
# https://github.blog/changelog/2026-06-25-actions-steps-can-now-be-run-in-parallel/
|
||||
.github/workflows/backend.yml:
|
||||
ignore:
|
||||
- 'step must run script with "run" section or run action with "uses" section'
|
||||
.github/workflows/lint.yml:
|
||||
ignore:
|
||||
- 'step must run script with "run" section or run action with "uses" section'
|
||||
.github/workflows/pr.checks.yml:
|
||||
ignore:
|
||||
- 'step must run script with "run" section or run action with "uses" section'
|
||||
136
.github/actions/bundle-size-check/action.yml
vendored
136
.github/actions/bundle-size-check/action.yml
vendored
@@ -1,136 +0,0 @@
|
||||
name: Bundle Size Check
|
||||
description: |
|
||||
Measures app + embedding SDK bundle sizes from the current and base-ref
|
||||
uberjars (reusing the builds that already ran) plus their bundle-stats
|
||||
artifacts, and flags gzipped size changes beyond the threshold for the main
|
||||
app's initial bundle and the chunked SDK's reachable total. The base ref is
|
||||
read back off the current uberjar so that both sides differ only by the PR.
|
||||
Skips when the base-ref artifacts aren't available rather than rebuilding from
|
||||
source (which would compare across pipelines).
|
||||
|
||||
inputs:
|
||||
edition:
|
||||
description: 'Metabase edition (ee or oss)'
|
||||
required: true
|
||||
threshold-percent:
|
||||
description: 'Percent threshold for bundle size change'
|
||||
required: true
|
||||
|
||||
outputs:
|
||||
app_status:
|
||||
description: "App initial bundle status: increased, decreased, or stable"
|
||||
value: ${{ steps.compare.outputs.app_initial_status }}
|
||||
app_size_change_percent:
|
||||
description: "App initial bundle size change percent (positive = increase, negative = decrease)"
|
||||
value: ${{ steps.compare.outputs.app_initial_size_change_percent }}
|
||||
sdk_status:
|
||||
description: "Chunked SDK bundle status: increased, decreased, or stable"
|
||||
value: ${{ steps.compare.outputs.embedding_sdk_chunked_total_status }}
|
||||
sdk_size_change_percent:
|
||||
description: "Chunked SDK bundle size change percent (positive = increase, negative = decrease)"
|
||||
value: ${{ steps.compare.outputs.embedding_sdk_chunked_total_size_change_percent }}
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Download current uberjar
|
||||
uses: ./.github/actions/fetch-artifact
|
||||
with:
|
||||
commit: ${{ github.event.pull_request.head.sha || github.sha }}
|
||||
edition: ${{ inputs.edition }}
|
||||
extract-path: temp/current-uberjar
|
||||
- name: Resolve the base ref the current uberjar was built against
|
||||
id: base-ref
|
||||
shell: bash
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
set -uo pipefail
|
||||
|
||||
# The uberjar job checks out refs/pull/<n>/merge (the PR head already
|
||||
# merged into the base branch tip) but names its artifact after head.sha,
|
||||
# so the comparable base ref is that merge's first parent. Reading it back
|
||||
# off the jar describes the artifact actually being measured. Using
|
||||
# pull_request.base.sha instead compares against a stale snapshot of the
|
||||
# base branch, reporting everything that landed in between as this PR's
|
||||
# own bundle delta.
|
||||
unzip -o -j temp/current-uberjar/metabase.jar version.properties -d temp/current-version >/dev/null
|
||||
built=$(sed -n 's/^hash=//p' temp/current-version/version.properties)
|
||||
base=$(gh api "/repos/${GITHUB_REPOSITORY}/commits/${built}" \
|
||||
--jq 'if (.parents | length) == 2 then .parents[0].sha else empty end' 2>/dev/null) || base=""
|
||||
|
||||
echo "sha=${base}" >> "$GITHUB_OUTPUT"
|
||||
echo "The current uberjar was built from ${built}; its base ref is ${base:-unknown}"
|
||||
- name: Download base-ref uberjar
|
||||
id: base-jar
|
||||
if: steps.base-ref.outputs.sha != ''
|
||||
continue-on-error: true
|
||||
uses: ./.github/actions/fetch-artifact
|
||||
with:
|
||||
commit: ${{ steps.base-ref.outputs.sha }}
|
||||
edition: ${{ inputs.edition }}
|
||||
extract-path: temp/base-uberjar
|
||||
- name: Measure and compare bundle sizes
|
||||
id: compare
|
||||
shell: bash
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
EDITION: ${{ inputs.edition }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
|
||||
BASE_SHA: ${{ steps.base-ref.outputs.sha }}
|
||||
THRESHOLD: ${{ inputs.threshold-percent }}
|
||||
MEASURE_BUNDLES: app,embedding-sdk-legacy,embedding-sdk-chunked
|
||||
# The gated bundles, as "<bundle>/<kind>". Each one's step outputs are
|
||||
# named after its key (see outputPrefix in compare.js), which is what the
|
||||
# action outputs above read.
|
||||
GATES: app/initial,embedding-sdk-chunked/total
|
||||
run: |
|
||||
set -uo pipefail
|
||||
|
||||
skip() {
|
||||
echo "::notice::Bundle-size check skipped: $1"
|
||||
for gate in ${GATES//,/ }; do
|
||||
prefix=$(printf '%s' "$gate" | tr -c 'a-zA-Z0-9' '_')
|
||||
echo "${prefix}_status=stable" >> "$GITHUB_OUTPUT"
|
||||
echo "${prefix}_size_change_percent=0" >> "$GITHUB_OUTPUT"
|
||||
done
|
||||
exit 0
|
||||
}
|
||||
|
||||
# The base ref needs an already-built uberjar. Rebuilding it from source
|
||||
# would compare across two build pipelines (the bug this check used to
|
||||
# have), so skip instead when it — or its stats — isn't available.
|
||||
[ -n "${BASE_SHA}" ] || skip "no comparable base ref for the current uberjar"
|
||||
[ -f temp/base-uberjar/metabase.jar ] || skip "base-ref uberjar not found for ${BASE_SHA}"
|
||||
|
||||
extract_js() { # $1=jar $2=dest-root
|
||||
mkdir -p "$2/resources"
|
||||
unzip -o "$1" \
|
||||
"frontend_client/app/dist/*" \
|
||||
"frontend_client/app/embed.js" \
|
||||
"frontend_client/app/embedding-sdk/*" \
|
||||
-d "$2/resources" >/dev/null || true
|
||||
}
|
||||
|
||||
fetch_stats() { # $1=sha $2=dest-dir
|
||||
local name="metabase-bundle-stats-${EDITION}-$1"
|
||||
local id
|
||||
id=$(gh api "/repos/${GITHUB_REPOSITORY}/actions/artifacts?name=${name}" --jq '.artifacts[0].id // empty' 2>/dev/null)
|
||||
[ -n "${id}" ] || return 1
|
||||
mkdir -p "$2"
|
||||
gh api "/repos/${GITHUB_REPOSITORY}/actions/artifacts/${id}/zip" > "$2.zip" 2>/dev/null || return 1
|
||||
unzip -o "$2.zip" -d "$2" >/dev/null || return 1
|
||||
}
|
||||
|
||||
extract_js temp/current-uberjar/metabase.jar temp/current
|
||||
extract_js temp/base-uberjar/metabase.jar temp/base
|
||||
|
||||
fetch_stats "${HEAD_SHA}" temp/current/artifacts || skip "bundle stats not found for current ${HEAD_SHA}"
|
||||
fetch_stats "${BASE_SHA}" temp/base/artifacts || skip "bundle stats not found for base ${BASE_SHA}"
|
||||
|
||||
MEASURE_ROOT=temp/current node ./.github/scripts/measure-bundle-sizes.js > temp/current-sizes.json \
|
||||
|| { echo "::error::Failed to measure current bundle sizes"; exit 1; }
|
||||
MEASURE_ROOT=temp/base node ./.github/scripts/measure-bundle-sizes.js > temp/base-sizes.json \
|
||||
|| { echo "::error::Failed to measure base bundle sizes"; exit 1; }
|
||||
|
||||
node ./.github/actions/bundle-size-check/compare.js temp/current-sizes.json temp/base-sizes.json
|
||||
133
.github/actions/bundle-size-check/compare.js
vendored
133
.github/actions/bundle-size-check/compare.js
vendored
@@ -1,133 +0,0 @@
|
||||
/* eslint-disable import/no-commonjs */
|
||||
// Diffs the bundle sizes produced by measure-bundle-sizes.js for the current
|
||||
// build and the base ref. Every (bundle, kind) is compared strictly against the
|
||||
// same (bundle, kind) — never one bundle type against another — using gzipped
|
||||
// (served) bytes.
|
||||
//
|
||||
// The report covers every measured bundle; the gates are the subset that drives
|
||||
// PR comments, named as "<bundle>/<kind>" (e.g. "app/initial",
|
||||
// "embedding-sdk-chunked/total").
|
||||
//
|
||||
// compareBundles is pure (data in, data out) so it can be unit-tested; main()
|
||||
// is the thin I/O wrapper that reads the size files, prints the report and sets
|
||||
// the step outputs / exit code.
|
||||
const fs = require("fs");
|
||||
|
||||
const pick = (rows, bundle, kind) => rows.find(row => row.bundle === bundle && row.kind === kind);
|
||||
const keyOf = row => `${row.bundle}/${row.kind}`;
|
||||
const mb = bytes => (bytes / 1024 / 1024).toFixed(2);
|
||||
const percentOf = (currentBytes, baseBytes) =>
|
||||
baseBytes ? Math.trunc(((currentBytes - baseBytes) * 100) / baseBytes) : 0;
|
||||
|
||||
// Output names are derived from the gate key, so the workflow reads
|
||||
// `app_initial_status`, `embedding_sdk_chunked_total_status`, and so on.
|
||||
const outputPrefix = gateKey => gateKey.replace(/[^a-zA-Z0-9]+/g, "_");
|
||||
|
||||
/**
|
||||
* Evaluate one gate. Returns the key plus exactly one of:
|
||||
* - `status` / `percent`: the bundle moved by `percent`.
|
||||
* - `skip` reason: the gate is non-actionable (reported stable, no comment).
|
||||
* - `error` reason: the gate inputs are missing (hard failure).
|
||||
*/
|
||||
function evaluateGate({ current, base, threshold, gateKey }) {
|
||||
const [bundle, kind] = gateKey.split("/");
|
||||
const gateCurrent = pick(current, bundle, kind);
|
||||
const gateBase = pick(base, bundle, kind);
|
||||
if (!gateCurrent || !gateBase || !gateBase.gzipBytes) {
|
||||
return { gateKey, error: `Could not find ${bundle} ${kind} (gzip) in both builds` };
|
||||
}
|
||||
|
||||
// Both sides must measure the bundle the same way. A base ref built before the
|
||||
// reachableAssets enrichment collapses its total to the initial set, so
|
||||
// comparing it against the current reachable total reports a phantom jump
|
||||
// (~30%). Skip until the base ref also carries reachable stats.
|
||||
if (Boolean(gateCurrent.reachable) !== Boolean(gateBase.reachable)) {
|
||||
return {
|
||||
gateKey,
|
||||
skip:
|
||||
`"${gateKey}" measured differently on each side ` +
|
||||
`(current reachable=${Boolean(gateCurrent.reachable)}, base reachable=${Boolean(gateBase.reachable)}); ` +
|
||||
`the base ref predates the reachable-chunk stats. Resolves once this change is on the base ref.`,
|
||||
};
|
||||
}
|
||||
|
||||
const percent = percentOf(gateCurrent.gzipBytes, gateBase.gzipBytes);
|
||||
const status = percent > threshold ? "increased" : percent < -threshold ? "decreased" : "stable";
|
||||
return { gateKey, status, percent };
|
||||
}
|
||||
|
||||
/**
|
||||
* Compare two measurement sets and evaluate every gate. Returns a
|
||||
* human-readable `report` over all measured bundles plus one entry per gate.
|
||||
*/
|
||||
function compareBundles({ current, base, threshold, gateKeys }) {
|
||||
const report = ["=== Bundle sizes (gzipped, current vs base) ==="];
|
||||
const keys = [...new Set([...current, ...base].map(keyOf))].sort();
|
||||
for (const key of keys) {
|
||||
const [bundle, kind] = key.split("/");
|
||||
const currentRow = pick(current, bundle, kind);
|
||||
const baseRow = pick(base, bundle, kind);
|
||||
if (!currentRow || !baseRow) {
|
||||
report.push(`${key}: present only in ${currentRow ? "current" : "base"} build`);
|
||||
continue;
|
||||
}
|
||||
const diff = currentRow.gzipBytes - baseRow.gzipBytes;
|
||||
const percent = percentOf(currentRow.gzipBytes, baseRow.gzipBytes);
|
||||
report.push(`${key}: ${mb(currentRow.gzipBytes)}MB vs ${mb(baseRow.gzipBytes)}MB (${percent}%, ${diff} bytes)`);
|
||||
}
|
||||
|
||||
return { report, gates: gateKeys.map(gateKey => evaluateGate({ current, base, threshold, gateKey })) };
|
||||
}
|
||||
|
||||
module.exports = { compareBundles, outputPrefix };
|
||||
|
||||
const setOutput = (name, value) => {
|
||||
if (process.env.GITHUB_OUTPUT) {
|
||||
fs.appendFileSync(process.env.GITHUB_OUTPUT, `${name}=${value}\n`);
|
||||
}
|
||||
};
|
||||
|
||||
function main() {
|
||||
const [, , currentPath, basePath] = process.argv;
|
||||
const threshold = Number(process.env.THRESHOLD ?? 2);
|
||||
const gateKeys = (process.env.GATES ?? "embedding-sdk-chunked/total").split(",").map(key => key.trim());
|
||||
|
||||
const current = JSON.parse(fs.readFileSync(currentPath, "utf8"));
|
||||
const base = JSON.parse(fs.readFileSync(basePath, "utf8"));
|
||||
|
||||
const { report, gates } = compareBundles({ current, base, threshold, gateKeys });
|
||||
console.log(report.join("\n"));
|
||||
console.log("");
|
||||
|
||||
for (const gate of gates) {
|
||||
const prefix = outputPrefix(gate.gateKey);
|
||||
|
||||
if (gate.error) {
|
||||
// One broken gate must not hide the others, so report them all and fail
|
||||
// the step at the end.
|
||||
console.error(`::error::${gate.error}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (gate.skip) {
|
||||
// Treat the gate as non-actionable: report stable so no regression comment
|
||||
// is posted, mirroring the shell skip() for missing artifacts.
|
||||
console.log(`::notice::Bundle-size gate skipped: ${gate.skip}`);
|
||||
setOutput(`${prefix}_status`, "stable");
|
||||
setOutput(`${prefix}_size_change_percent`, "0");
|
||||
continue;
|
||||
}
|
||||
|
||||
console.log(`Gate: ${gate.gateKey} ${gate.percent}% (threshold ${threshold}%) → ${gate.status}`);
|
||||
setOutput(`${prefix}_status`, gate.status);
|
||||
setOutput(`${prefix}_size_change_percent`, gate.percent);
|
||||
}
|
||||
|
||||
if (gates.some(gate => gate.error)) {
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
if (require.main === module) {
|
||||
main();
|
||||
}
|
||||
@@ -1,84 +0,0 @@
|
||||
const { compareBundles, outputPrefix } = require("./compare");
|
||||
|
||||
const SDK_GATE = "embedding-sdk-chunked/total";
|
||||
const APP_GATE = "app/initial";
|
||||
|
||||
const total = (gzipBytes, over = {}) => ({
|
||||
bundle: "embedding-sdk-chunked",
|
||||
kind: "total",
|
||||
gzipBytes,
|
||||
reachable: true,
|
||||
...over,
|
||||
});
|
||||
|
||||
const appInitial = gzipBytes => ({ bundle: "app", kind: "initial", gzipBytes });
|
||||
|
||||
const compare = ({ current, base, threshold = 2, gateKeys = [SDK_GATE] }) =>
|
||||
compareBundles({ current, base, threshold, gateKeys });
|
||||
|
||||
describe("outputPrefix", () => {
|
||||
it("turns a gate key into a step-output prefix", () => {
|
||||
expect(outputPrefix(APP_GATE)).toBe("app_initial");
|
||||
expect(outputPrefix(SDK_GATE)).toBe("embedding_sdk_chunked_total");
|
||||
});
|
||||
});
|
||||
|
||||
describe("compareBundles", () => {
|
||||
it("gates on the chunked SDK total and flags an increase past the threshold", () => {
|
||||
const { gates, report } = compare({ current: [total(110)], base: [total(100)] });
|
||||
|
||||
expect(gates).toEqual([{ gateKey: SDK_GATE, status: "increased", percent: 10 }]);
|
||||
expect(report[0]).toContain("current vs base");
|
||||
});
|
||||
|
||||
it("flags a decrease and stays stable within the threshold", () => {
|
||||
expect(compare({ current: [total(90)], base: [total(100)] }).gates[0]).toMatchObject({
|
||||
status: "decreased",
|
||||
percent: -10,
|
||||
});
|
||||
expect(compare({ current: [total(101)], base: [total(100)] }).gates[0]).toMatchObject({
|
||||
status: "stable",
|
||||
percent: 1,
|
||||
});
|
||||
});
|
||||
|
||||
it("evaluates each gate against its own bundle", () => {
|
||||
const { gates } = compare({
|
||||
current: [total(100), appInitial(110)],
|
||||
base: [total(100), appInitial(100)],
|
||||
gateKeys: [APP_GATE, SDK_GATE],
|
||||
});
|
||||
|
||||
expect(gates).toEqual([
|
||||
{ gateKey: APP_GATE, status: "increased", percent: 10 },
|
||||
{ gateKey: SDK_GATE, status: "stable", percent: 0 },
|
||||
]);
|
||||
});
|
||||
|
||||
it("skips when the two sides measured a gate differently (reachable mismatch)", () => {
|
||||
const { gates } = compare({
|
||||
current: [total(130, { reachable: true })],
|
||||
base: [total(100, { reachable: false })],
|
||||
});
|
||||
|
||||
expect(gates[0].status).toBeUndefined();
|
||||
expect(gates[0].skip).toContain("predates the reachable-chunk stats");
|
||||
});
|
||||
|
||||
it("errors when a gate bundle is missing on either side, without hiding the others", () => {
|
||||
const { gates } = compare({
|
||||
current: [appInitial(100)],
|
||||
base: [appInitial(100)],
|
||||
gateKeys: [APP_GATE, SDK_GATE],
|
||||
});
|
||||
|
||||
expect(gates[0]).toEqual({ gateKey: APP_GATE, status: "stable", percent: 0 });
|
||||
expect(gates[1].error).toContain("embedding-sdk-chunked total");
|
||||
});
|
||||
|
||||
it("reports a bundle/kind present on only one side without erroring", () => {
|
||||
const { report } = compare({ current: [total(100), appInitial(50)], base: [total(100)] });
|
||||
|
||||
expect(report.some(line => line.includes("present only in current build"))).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -26,7 +26,7 @@ runs:
|
||||
|
||||
- name: Upload python-runner logs
|
||||
if: ${{inputs.upload-logs == 'true' && inputs.logs-artifact-suffix != ''}}
|
||||
uses: actions/upload-artifact@v7
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: python-runner-logs-${{inputs.logs-artifact-suffix}}
|
||||
path: |
|
||||
|
||||
61
.github/actions/collect-profile/action.yml
vendored
61
.github/actions/collect-profile/action.yml
vendored
@@ -1,61 +0,0 @@
|
||||
name: Collect profile
|
||||
description: >-
|
||||
Attach a flamegraph and per-test wall times to a job whose tests ran under clj-async-profiler.
|
||||
Does nothing when the run was not profiled.
|
||||
|
||||
inputs:
|
||||
profiled:
|
||||
required: true
|
||||
description: >-
|
||||
"true" when the test step ran under the profiler. Passed in rather than read from the
|
||||
environment so the decision stays recorded by the step that made it.
|
||||
artifact-name:
|
||||
required: true
|
||||
description: Name for the uploaded artifact. Must be unique across the jobs of one run.
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
# Unlike a JUnit failure report, these run on a green build too — a suite that passes slowly is
|
||||
# exactly what profiling is for.
|
||||
#
|
||||
# `awk 'NR<=25'` rather than `head -25`: the runner ignores SIGPIPE, so `head` exiting early
|
||||
# leaves `sort` with an EPIPE write error and `pipefail` fails the step. `|| true` covers `grep`
|
||||
# finding no junit files.
|
||||
- name: Summarize test wall times
|
||||
if: ${{ inputs.profiled == 'true' }}
|
||||
continue-on-error: true
|
||||
shell: bash
|
||||
run: |
|
||||
{
|
||||
echo "### Slowest test namespaces (seconds)"
|
||||
echo '```'
|
||||
grep -h '<testsuite' target/junit/*.xml 2>/dev/null \
|
||||
| sed -E 's|.*<testsuite name="([^"]*)" time="([^"]*)".*|\2 \1|' \
|
||||
| sort -rn | awk 'NR<=25' || true
|
||||
echo '```'
|
||||
echo "### Slowest tests (seconds)"
|
||||
echo '```'
|
||||
grep -h '<testcase' target/junit/*.xml 2>/dev/null \
|
||||
| sed -E 's|.*classname="([^"]*)" name="([^"]*)" time="([^"]*)".*|\3 \1/\2|' \
|
||||
| sort -rn | awk 'NR<=25' || true
|
||||
echo '```'
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
# Only the rendered flamegraph is kept: the raw collapsed stacks it was built from run to
|
||||
# hundreds of MB on a long suite, and are reproducible locally from the same test selection.
|
||||
#
|
||||
# The info-level test log rides along on a green profiled run too. The failure-only log artifact
|
||||
# the test jobs upload is not written when nothing failed, which is exactly the run being profiled.
|
||||
- name: Upload profile
|
||||
if: ${{ inputs.profiled == 'true' }}
|
||||
continue-on-error: true
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: ${{ inputs.artifact-name }}
|
||||
path: |
|
||||
target/clj-async-profiler/results/*flamegraph.html
|
||||
target/junit/*.xml
|
||||
logs/test-log.json
|
||||
retention-days: 14
|
||||
if-no-files-found: warn
|
||||
49
.github/actions/create-backport/action.yml
vendored
49
.github/actions/create-backport/action.yml
vendored
@@ -1,5 +1,4 @@
|
||||
name: Create Backport PR
|
||||
description: Cherry-picks a commit onto a release branch and opens the backport pull request.
|
||||
|
||||
inputs:
|
||||
target-version: # major version number like 34 or 86
|
||||
@@ -25,8 +24,6 @@ runs:
|
||||
COMMIT: ${{ inputs.backport-commit }}
|
||||
shell: bash
|
||||
run: | # bash
|
||||
source "${{ github.action_path }}/kondo-ratchets.sh"
|
||||
|
||||
git config --global user.email "metabase-bot@metabase.com"
|
||||
git config --global user.name "Metabase bot"
|
||||
|
||||
@@ -36,7 +33,7 @@ runs:
|
||||
git checkout ${TARGET_BRANCH}
|
||||
git fetch --all
|
||||
git checkout -b ${BACKPORT_BRANCH}
|
||||
cherry_pick_backport "${COMMIT}"
|
||||
git cherry-pick ${COMMIT} || true
|
||||
|
||||
CONFLICT_LIST=$(git ls-files -u)
|
||||
|
||||
@@ -45,10 +42,13 @@ runs:
|
||||
git cherry-pick --abort
|
||||
|
||||
echo "Could not cherry pick because of a conflict"
|
||||
echo "has-conflicts=true" >> "$GITHUB_OUTPUT"
|
||||
echo "has-conflicts=true" >> $GITHUB_OUTPUT
|
||||
|
||||
# Add a shell script for resolving conflicts
|
||||
write_backport_script "${COMMIT}"
|
||||
echo "git reset HEAD~1" > ./backport.sh
|
||||
echo "rm ./backport.sh" >> ./backport.sh
|
||||
echo "git cherry-pick ${COMMIT}" >> ./backport.sh
|
||||
echo "echo 'Resolve conflicts and force push this branch.\n\nTo backport translations run: bin/i18n/merge-translations <release-branch>'" >> ./backport.sh
|
||||
chmod +x ./backport.sh
|
||||
git add ./backport.sh
|
||||
git commit -m "Add backport resolution script"
|
||||
@@ -57,44 +57,45 @@ runs:
|
||||
#${ORIGINAL_PULL_REQUEST_NUMBER}
|
||||
> [!IMPORTANT]
|
||||
> Manual conflict resolution is required.
|
||||
> Checkout the branch and run \`./backport.sh\` script. Force push your changes after cherry-picking. Mark the PR as ready for review once conflicts have been addressed.
|
||||
> Checkout the branch and run \`./backport.sh\` script. Force push your changes after cherry-picking. Check the box below when done.
|
||||
|
||||
Conflicts:
|
||||
\`\`\`shell
|
||||
${CONFLICT_LIST}
|
||||
\`\`\`
|
||||
|
||||
- [ ] Conflicts resolved
|
||||
END
|
||||
)
|
||||
else
|
||||
echo "has-conflicts=false" >> "$GITHUB_OUTPUT"
|
||||
echo "has-conflicts=false" >> $GITHUB_OUTPUT
|
||||
PR_BODY="#${ORIGINAL_PULL_REQUEST_NUMBER}"
|
||||
fi
|
||||
|
||||
git push -u origin ${BACKPORT_BRANCH}
|
||||
|
||||
PR_ARGS=(--base "${TARGET_BRANCH}" --head "${BACKPORT_BRANCH}" --label "was-backported" --assignee "${{ github.event.pull_request.user.login }}" --title "🤖 v${TARGET_VERSION} backported \"${ORIGINAL_TITLE}\"" --body "${PR_BODY}")
|
||||
if [ "$CONFLICTS" -gt 0 ]; then
|
||||
PR_ARGS+=(--draft)
|
||||
fi
|
||||
|
||||
BACKPORT_PR_URL=$(gh pr create "${PR_ARGS[@]}")
|
||||
BACKPORT_PR_URL=$(gh pr create --base "${TARGET_BRANCH}" --head "${BACKPORT_BRANCH}" --label "was-backported" --assignee "${{ github.event.pull_request.user.login }}" --title "🤖 backported \"${ORIGINAL_TITLE}\"" --body "${PR_BODY}")
|
||||
BACKPORT_PR_NUMBER=${BACKPORT_PR_URL##*/}
|
||||
|
||||
echo "backport_pr_number=$BACKPORT_PR_NUMBER" >> "$GITHUB_OUTPUT"
|
||||
echo "backport_pr_number=$BACKPORT_PR_NUMBER" >> $GITHUB_OUTPUT
|
||||
echo "New PR has been created: #${BACKPORT_PR_NUMBER}"
|
||||
|
||||
git checkout ${{ github.sha }}
|
||||
|
||||
- name: Approve and Merge the PR
|
||||
- name: Auto approve backport PR
|
||||
if: ${{ steps.create_backport_pull_request.outputs.has-conflicts == 'false' }}
|
||||
run: |
|
||||
gh pr review --approve "$PR_NUMBER"
|
||||
gh pr merge --squash --auto "$PR_NUMBER"
|
||||
shell: bash
|
||||
env:
|
||||
# uses a different token than the token that created the PR so we can approve and merge it
|
||||
GH_TOKEN: ${{ inputs.github-token-2 }}
|
||||
PR_NUMBER: ${{ steps.create_backport_pull_request.outputs.backport_pr_number }}
|
||||
uses: juliangruber/approve-pull-request-action@v1
|
||||
with:
|
||||
github-token: ${{ inputs.github-token-2 }}
|
||||
number: ${{ steps.create_backport_pull_request.outputs.backport_pr_number }}
|
||||
|
||||
- name: Enable Pull Request Automerge
|
||||
if: ${{ steps.create_backport_pull_request.outputs.has-conflicts == 'false' }}
|
||||
uses: peter-evans/enable-pull-request-automerge@v2
|
||||
with:
|
||||
token: ${{ inputs.github-token }}
|
||||
pull-request-number: ${{ steps.create_backport_pull_request.outputs.backport_pr_number }}
|
||||
merge-method: squash
|
||||
|
||||
- uses: ./.github/actions/notify-pull-request
|
||||
if: ${{ steps.create_backport_pull_request.outputs.has-conflicts == 'true' }}
|
||||
|
||||
@@ -1,375 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
action_dir=$(cd "$(dirname "$0")" && pwd)
|
||||
cut_release_workflow="$action_dir/../../workflows/cut-release-branch.yml"
|
||||
|
||||
# shellcheck source=.github/actions/create-backport/kondo-ratchets.sh
|
||||
source "$action_dir/kondo-ratchets.sh"
|
||||
|
||||
# Any editor launch fails the run, so a continuation that waits on an editor cannot pass.
|
||||
export GIT_EDITOR=false
|
||||
|
||||
test_root=
|
||||
trap '[ -z "$test_root" ] || rm -rf -- "$test_root"' EXIT
|
||||
trap 'exit 129' HUP
|
||||
trap 'exit 130' INT
|
||||
trap 'exit 143' TERM
|
||||
test_root=$(mktemp -d)
|
||||
|
||||
active_ratchets='{:ignore-counts {:example 1}}'
|
||||
disabled_ratchets=';; Kondo ignore ratchets apply only to master; this release branch opts out.
|
||||
;; The test and fixer recognize this explicit opt-out.
|
||||
{:disabled true}'
|
||||
|
||||
# Coverage:
|
||||
#
|
||||
# Target state Incoming change Other conflict Expected result
|
||||
# ----------------- -------------------- --------------- ------------------------------------------
|
||||
# active ratchets unrelated file none leave ratchets unchanged
|
||||
# active ratchets ratchets none install the release opt-out
|
||||
# disabled ratchets ratchets none keep the opt-out; finish with an empty commit
|
||||
# divergent ratchets ratchets none resolve ratchets and finish the cherry-pick
|
||||
# divergent ratchets ratchets (on a pty) none finish without opening an editor
|
||||
# active ratchets unrelated file app.txt leave the conflict for manual resolution
|
||||
# divergent ratchets ratchets + app.txt app.txt resolve only ratchets
|
||||
# release cut ratchets none cut and backport commit the same opt-out
|
||||
# n/a cut-release workflow n/a the workflow calls the shared writer
|
||||
# any invalid commit n/a propagate the cherry-pick failure
|
||||
# divergent ratchets generated backport.sh app.txt resolve only ratchets through the manual script
|
||||
#
|
||||
# Known gaps: an older target with no ratchets file, and an incoming commit that
|
||||
# deletes the ratchets file.
|
||||
|
||||
test_unrelated_backport_preserves_target_ratchets() (
|
||||
local repo="$test_root/unrelated"
|
||||
init_repo "$repo"
|
||||
local commit
|
||||
commit=$(feature_commit "$repo" change_app)
|
||||
git -C "$repo" checkout -q release
|
||||
|
||||
cd "$repo"
|
||||
cherry_pick_backport "$commit"
|
||||
|
||||
assert_eq feature "$(cat app.txt)" "the unrelated change was not cherry-picked"
|
||||
assert_ratchets "$active_ratchets"
|
||||
assert_cherry_pick_completed
|
||||
assert_worktree_clean
|
||||
)
|
||||
|
||||
test_clean_ratchet_backport_installs_release_opt_out() (
|
||||
local repo="$test_root/clean-ratchet"
|
||||
init_repo "$repo"
|
||||
local commit
|
||||
commit=$(feature_commit "$repo" change_ratchets)
|
||||
git -C "$repo" checkout -q release
|
||||
|
||||
cd "$repo"
|
||||
cherry_pick_backport "$commit"
|
||||
|
||||
assert_ratchets_disabled
|
||||
assert_cherry_pick_completed
|
||||
assert_worktree_clean
|
||||
)
|
||||
|
||||
test_disabled_release_absorbs_ratchet_only_backport() (
|
||||
local repo="$test_root/disabled-release"
|
||||
init_repo "$repo"
|
||||
local commit
|
||||
commit=$(feature_commit "$repo" change_ratchets)
|
||||
git -C "$repo" checkout -q release
|
||||
commit_release_change "$repo" disable_release_ratchets "Disable release ratchets"
|
||||
|
||||
cd "$repo"
|
||||
cherry_pick_backport "$commit"
|
||||
|
||||
assert_ratchets_disabled
|
||||
assert_cherry_pick_completed
|
||||
assert_empty_commit
|
||||
assert_worktree_clean
|
||||
)
|
||||
|
||||
test_ratchet_only_conflict_is_resolved() (
|
||||
local repo="$test_root/ratchet-conflict"
|
||||
init_repo "$repo"
|
||||
local commit
|
||||
commit=$(conflicting_ratchet_commit "$repo")
|
||||
|
||||
cd "$repo"
|
||||
cherry_pick_backport "$commit"
|
||||
|
||||
assert_ratchets_disabled
|
||||
assert_cherry_pick_completed
|
||||
assert_worktree_clean
|
||||
)
|
||||
|
||||
test_continuation_never_opens_an_editor() (
|
||||
local repo="$test_root/editor"
|
||||
init_repo "$repo"
|
||||
local commit
|
||||
commit=$(conflicting_ratchet_commit "$repo")
|
||||
|
||||
cd "$repo"
|
||||
# Git only opens an editor for the continuation when stdin is a terminal, so run it on a pty. The
|
||||
# exported GIT_EDITOR=false turns any editor launch into a failure. The single quotes are deliberate:
|
||||
# $1 and $2 are for the inner bash, not this shell.
|
||||
# shellcheck disable=SC2016
|
||||
python3 -c 'import os, pty, sys; sys.exit(os.waitstatus_to_exitcode(pty.spawn(sys.argv[1:])))' \
|
||||
bash -c 'source "$1/kondo-ratchets.sh" && cherry_pick_backport "$2"' bash "$action_dir" "$commit" >/dev/null ||
|
||||
fail "the cherry-pick continuation opened an editor"
|
||||
|
||||
assert_ratchets_disabled
|
||||
assert_cherry_pick_completed
|
||||
)
|
||||
|
||||
test_non_ratchet_conflict_remains_manual() (
|
||||
local repo="$test_root/non-ratchet-conflict"
|
||||
init_repo "$repo"
|
||||
local commit
|
||||
commit=$(feature_commit "$repo" change_app)
|
||||
git -C "$repo" checkout -q release
|
||||
commit_release_change "$repo" change_release_app "Change release app"
|
||||
|
||||
cd "$repo"
|
||||
cherry_pick_backport "$commit"
|
||||
|
||||
assert_ratchets "$active_ratchets"
|
||||
assert_only_conflicts app.txt
|
||||
assert_cherry_pick_pending
|
||||
)
|
||||
|
||||
test_mixed_conflict_resolves_only_ratchets() (
|
||||
local repo="$test_root/mixed-conflict"
|
||||
init_repo "$repo"
|
||||
local commit
|
||||
commit=$(feature_commit "$repo" change_app_and_ratchets)
|
||||
git -C "$repo" checkout -q release
|
||||
commit_release_change "$repo" change_release_app_and_ratchets "Change release app and ratchets"
|
||||
|
||||
cd "$repo"
|
||||
cherry_pick_backport "$commit"
|
||||
|
||||
assert_ratchets_disabled
|
||||
assert_only_conflicts app.txt
|
||||
assert_cherry_pick_pending
|
||||
)
|
||||
|
||||
test_release_cut_and_backport_commit_the_same_opt_out() (
|
||||
local cut_repo="$test_root/release-cut"
|
||||
init_repo "$cut_repo"
|
||||
cd "$cut_repo"
|
||||
git checkout -qb release-x.99.x
|
||||
write_disabled_ratchets
|
||||
git add --sparse .clj-kondo/ratchets.edn
|
||||
git commit -q --allow-empty --no-verify -m "Cut release-x.99.x branch"
|
||||
git show HEAD:.clj-kondo/ratchets.edn > "$test_root/release-cut.edn"
|
||||
|
||||
local backport_repo="$test_root/release-cut-backport"
|
||||
init_repo "$backport_repo"
|
||||
local commit
|
||||
commit=$(feature_commit "$backport_repo" change_ratchets)
|
||||
git -C "$backport_repo" checkout -q release
|
||||
cd "$backport_repo"
|
||||
cherry_pick_backport "$commit"
|
||||
git show HEAD:.clj-kondo/ratchets.edn > "$test_root/backport.edn"
|
||||
|
||||
cmp -s "$test_root/release-cut.edn" "$test_root/backport.edn" ||
|
||||
fail "the release cut and the backport committed different opt-out files"
|
||||
assert_eq "$disabled_ratchets" "$(cat "$test_root/release-cut.edn")" \
|
||||
"the release cut did not install the expected opt-out"
|
||||
)
|
||||
|
||||
test_cut_release_workflow_uses_the_shared_writer() (
|
||||
grep -Fq 'source .github/actions/create-backport/kondo-ratchets.sh' "$cut_release_workflow" ||
|
||||
fail "the cut-release workflow does not source kondo-ratchets.sh"
|
||||
grep -Fq 'write_disabled_ratchets' "$cut_release_workflow" ||
|
||||
fail "the cut-release workflow does not call write_disabled_ratchets"
|
||||
! grep -Fq ':disabled' "$cut_release_workflow" ||
|
||||
fail "the cut-release workflow carries its own copy of the opt-out"
|
||||
)
|
||||
|
||||
test_invalid_commit_failure_is_propagated() (
|
||||
local repo="$test_root/invalid"
|
||||
init_repo "$repo"
|
||||
cd "$repo"
|
||||
|
||||
if cherry_pick_backport deadbeef; then
|
||||
fail "an invalid commit succeeded"
|
||||
fi
|
||||
)
|
||||
|
||||
test_generated_script_resolves_ratchets_end_to_end() (
|
||||
local repo="$test_root/generated-script"
|
||||
init_repo "$repo"
|
||||
local commit
|
||||
commit=$(feature_commit "$repo" change_app_and_ratchets)
|
||||
git -C "$repo" checkout -q release
|
||||
commit_release_change "$repo" change_release_app_and_ratchets "Change release app and ratchets"
|
||||
|
||||
cd "$repo"
|
||||
cherry_pick_backport "$commit"
|
||||
git cherry-pick --abort
|
||||
|
||||
local release_commit
|
||||
release_commit=$(git rev-parse HEAD)
|
||||
write_backport_script "$commit"
|
||||
chmod +x backport.sh
|
||||
git add backport.sh
|
||||
git commit -qm "Add manual backport script"
|
||||
|
||||
bash -n backport.sh
|
||||
grep -Fq 'write_disabled_ratchets ()' backport.sh || fail "the generated script omitted write_disabled_ratchets"
|
||||
./backport.sh
|
||||
|
||||
assert_ratchets_disabled
|
||||
assert_only_conflicts app.txt
|
||||
assert_cherry_pick_pending
|
||||
assert_eq "$release_commit" "$(git rev-parse HEAD)" "the generated script did not return HEAD to the release commit"
|
||||
[ ! -e backport.sh ] || fail "the generated script did not remove itself"
|
||||
! git ls-files --error-unmatch backport.sh >/dev/null 2>&1 || fail "the generated script is still tracked"
|
||||
! git status --porcelain | grep -q backport.sh || fail "the generated script left a status entry"
|
||||
)
|
||||
|
||||
# Test harness
|
||||
|
||||
fail() {
|
||||
echo "FAIL: $*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
assert_eq() {
|
||||
local expected=$1
|
||||
local actual=$2
|
||||
local message=$3
|
||||
[ "$expected" = "$actual" ] || fail "$message (expected '$expected', got '$actual')"
|
||||
}
|
||||
|
||||
assert_ratchets() {
|
||||
assert_eq "$1" "$(cat .clj-kondo/ratchets.edn)" "unexpected ratchet state"
|
||||
}
|
||||
|
||||
assert_ratchets_disabled() {
|
||||
assert_ratchets "$disabled_ratchets"
|
||||
}
|
||||
|
||||
assert_cherry_pick_completed() {
|
||||
! git rev-parse -q --verify CHERRY_PICK_HEAD >/dev/null || fail "the cherry-pick is still pending"
|
||||
}
|
||||
|
||||
assert_cherry_pick_pending() {
|
||||
git rev-parse -q --verify CHERRY_PICK_HEAD >/dev/null || fail "the cherry-pick state was lost"
|
||||
}
|
||||
|
||||
assert_only_conflicts() {
|
||||
local expected
|
||||
local actual
|
||||
expected=$(printf '%s\n' "$@" | sort)
|
||||
actual=$(git diff --name-only --diff-filter=U | sort)
|
||||
assert_eq "$expected" "$actual" "unexpected unresolved files"
|
||||
}
|
||||
|
||||
assert_empty_commit() {
|
||||
git diff --quiet HEAD^ HEAD || fail "the backport commit was not empty"
|
||||
}
|
||||
|
||||
assert_worktree_clean() {
|
||||
[ -z "$(git status --porcelain)" ] || fail "the backport left worktree changes"
|
||||
}
|
||||
|
||||
init_repo() {
|
||||
local repo=$1
|
||||
git init -q -b release "$repo"
|
||||
git -C "$repo" config user.email test@example.com
|
||||
git -C "$repo" config user.name "Backport test"
|
||||
printf 'base\n' > "$repo/app.txt"
|
||||
mkdir -p "$repo/.clj-kondo"
|
||||
printf '%s\n' "$active_ratchets" > "$repo/.clj-kondo/ratchets.edn"
|
||||
git -C "$repo" add .
|
||||
git -C "$repo" commit -qm "Base"
|
||||
}
|
||||
|
||||
feature_commit() {
|
||||
local repo=$1
|
||||
shift
|
||||
git -C "$repo" checkout -qb feature
|
||||
"$@" "$repo"
|
||||
git -C "$repo" add .
|
||||
git -C "$repo" commit -qm "Feature"
|
||||
git -C "$repo" rev-parse HEAD
|
||||
}
|
||||
|
||||
# Sets up a release branch whose ratchets conflict with the feature commit, and prints that commit.
|
||||
conflicting_ratchet_commit() {
|
||||
local repo=$1
|
||||
local commit
|
||||
commit=$(feature_commit "$repo" change_ratchets)
|
||||
git -C "$repo" checkout -q release
|
||||
commit_release_change "$repo" change_release_ratchets "Change release ratchets"
|
||||
echo "$commit"
|
||||
}
|
||||
|
||||
change_app() {
|
||||
printf 'feature\n' > "$1/app.txt"
|
||||
}
|
||||
|
||||
change_ratchets() {
|
||||
printf '{:ignore-counts {:example 2}}\n' > "$1/.clj-kondo/ratchets.edn"
|
||||
}
|
||||
|
||||
change_app_and_ratchets() {
|
||||
change_app "$1"
|
||||
change_ratchets "$1"
|
||||
}
|
||||
|
||||
change_release_app() {
|
||||
printf 'release\n' > "$1/app.txt"
|
||||
}
|
||||
|
||||
change_release_ratchets() {
|
||||
printf '{:ignore-counts {:release 1}}\n' > "$1/.clj-kondo/ratchets.edn"
|
||||
}
|
||||
|
||||
change_release_app_and_ratchets() {
|
||||
change_release_app "$1"
|
||||
change_release_ratchets "$1"
|
||||
}
|
||||
|
||||
# Setup writes the opt-out through the shared writer; the only copy is the expected text at the top.
|
||||
disable_release_ratchets() {
|
||||
(cd "$1" && write_disabled_ratchets)
|
||||
}
|
||||
|
||||
commit_release_change() {
|
||||
local repo=$1
|
||||
local change=$2
|
||||
local message=$3
|
||||
"$change" "$repo"
|
||||
git -C "$repo" add .
|
||||
git -C "$repo" commit -qm "$message"
|
||||
}
|
||||
|
||||
run_test() {
|
||||
local description=$1
|
||||
local test=${description// /_}
|
||||
test=${test//-/_}
|
||||
test="test_$test"
|
||||
|
||||
declare -F "$test" >/dev/null || fail "missing test function: $test"
|
||||
|
||||
printf 'TEST: %s\n' "$description"
|
||||
"$test"
|
||||
printf 'PASS: %s\n' "$description"
|
||||
}
|
||||
|
||||
run_test "unrelated backport preserves target ratchets"
|
||||
run_test "clean ratchet backport installs release opt-out"
|
||||
run_test "disabled release absorbs ratchet-only backport"
|
||||
run_test "ratchet-only conflict is resolved"
|
||||
run_test "continuation never opens an editor"
|
||||
run_test "non-ratchet conflict remains manual"
|
||||
run_test "mixed conflict resolves only ratchets"
|
||||
run_test "release cut and backport commit the same opt-out"
|
||||
run_test "cut-release workflow uses the shared writer"
|
||||
run_test "invalid commit failure is propagated"
|
||||
run_test "generated script resolves ratchets end-to-end"
|
||||
@@ -1,74 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# The only place that knows what a release branch's opt-out looks like. The cut-release workflow, the
|
||||
# backport helper, and the tests all go through this function.
|
||||
write_disabled_ratchets() {
|
||||
mkdir -p .clj-kondo
|
||||
printf '%s\n' \
|
||||
';; Kondo ignore ratchets apply only to master; this release branch opts out.' \
|
||||
';; The test and fixer recognize this explicit opt-out.' \
|
||||
'{:disabled true}' \
|
||||
> .clj-kondo/ratchets.edn
|
||||
}
|
||||
|
||||
disable_ratchets() {
|
||||
local commit=$1
|
||||
local file=.clj-kondo/ratchets.edn
|
||||
[ -n "$(git diff-tree --root --no-commit-id --name-only -r "$commit" -- "$file")" ] || return 0
|
||||
|
||||
echo "$file: disabling kondo ratchets on the release branch"
|
||||
write_disabled_ratchets
|
||||
git add -- "$file"
|
||||
|
||||
if ! git rev-parse -q --verify CHERRY_PICK_HEAD >/dev/null; then
|
||||
if ! git diff --cached --quiet HEAD; then
|
||||
git commit --amend --no-edit
|
||||
fi
|
||||
return 0
|
||||
fi
|
||||
|
||||
if [ -z "$(git ls-files -u)" ]; then
|
||||
echo "$file was the only conflict, finishing the cherry-pick"
|
||||
if git diff --cached --quiet HEAD; then
|
||||
echo "nothing left to apply, committing the backport empty"
|
||||
git commit --allow-empty --no-edit
|
||||
else
|
||||
# GIT_EDITOR outranks core.editor, VISUAL, and EDITOR, so the continuation can never wait on
|
||||
# an editor; the repository's own editor configuration is left alone.
|
||||
GIT_EDITOR=true git cherry-pick --continue
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
cherry_pick_backport() {
|
||||
local commit=$1
|
||||
local status=0
|
||||
git cherry-pick "$commit" || status=$?
|
||||
if [ "$status" -ne 0 ] && ! git rev-parse -q --verify CHERRY_PICK_HEAD >/dev/null; then
|
||||
echo "cherry-pick failed without entering a resolvable conflict state" >&2
|
||||
return "$status"
|
||||
fi
|
||||
|
||||
disable_ratchets "$commit"
|
||||
if [ "$status" -ne 0 ] &&
|
||||
git rev-parse -q --verify CHERRY_PICK_HEAD >/dev/null &&
|
||||
[ -z "$(git ls-files -u)" ]; then
|
||||
echo "cherry-pick stopped without a file conflict to resolve" >&2
|
||||
return "$status"
|
||||
fi
|
||||
}
|
||||
|
||||
write_backport_script() {
|
||||
local commit=$1
|
||||
{
|
||||
echo "#!/usr/bin/env bash"
|
||||
echo "set -euo pipefail"
|
||||
echo "git reset HEAD~1"
|
||||
echo "rm ./backport.sh"
|
||||
declare -f write_disabled_ratchets disable_ratchets cherry_pick_backport
|
||||
printf 'cherry_pick_backport %q\n' "$commit"
|
||||
printf '%s\n' "printf 'Resolve conflicts and force push this branch.\\n\\nTo backport translations run: bin/i18n/merge-translations <release-branch>\\n'"
|
||||
} > ./backport.sh
|
||||
}
|
||||
124
.github/actions/create-test-plan/action.yml
vendored
124
.github/actions/create-test-plan/action.yml
vendored
@@ -1,124 +0,0 @@
|
||||
name: Create test plan
|
||||
description: Compute the affected-tests plan for the changed files.
|
||||
|
||||
inputs:
|
||||
changed-files:
|
||||
description: All changed files (CSV).
|
||||
required: true
|
||||
fe-changed-files:
|
||||
description: Changed frontend_sources files (CSV).
|
||||
required: true
|
||||
be-changed-files:
|
||||
description: Changed backend_sources files (CSV).
|
||||
required: true
|
||||
unit-infra-touched:
|
||||
description: Whether FE unit infra changed.
|
||||
required: true
|
||||
loki-infra-touched:
|
||||
description: Whether Loki infra changed.
|
||||
required: true
|
||||
e2e-infra-touched:
|
||||
description: Whether e2e harness/support code changed.
|
||||
required: true
|
||||
shared-sources-touched:
|
||||
description: Whether cljc/cljs/package.json changed.
|
||||
required: true
|
||||
|
||||
outputs:
|
||||
stats:
|
||||
description: FE Affected Tests stats row (JSON).
|
||||
value: ${{ steps.compute.outputs.stats }}
|
||||
artifact-id:
|
||||
description: Artifact containing test-plan.json for this run.
|
||||
value: ${{ steps.upload.outputs.artifact-id }}
|
||||
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Prepare front-end environment
|
||||
uses: ./.github/actions/prepare-frontend
|
||||
- name: Resolve closest-ancestor e2e coverage manifest
|
||||
id: manifest
|
||||
uses: actions/github-script@v9
|
||||
with:
|
||||
script: | # js
|
||||
const { pathToFileURL } = require("url");
|
||||
const { resolveCoverageManifest } = await import(
|
||||
pathToFileURL(
|
||||
`${process.env.GITHUB_WORKSPACE}/.github/scripts/resolve-coverage-manifest.mjs`,
|
||||
).href,
|
||||
);
|
||||
const result = await resolveCoverageManifest({
|
||||
github,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
headSha: context.payload.pull_request?.head?.sha ?? context.sha,
|
||||
});
|
||||
core.setOutput("path", result?.path ?? "");
|
||||
- name: Create test plan
|
||||
id: compute
|
||||
shell: bash
|
||||
env:
|
||||
CHANGED_FILES: ${{ inputs.changed-files }}
|
||||
FE_CHANGED_FILES: ${{ inputs.fe-changed-files }}
|
||||
BE_CHANGED_FILES: ${{ inputs.be-changed-files }}
|
||||
UNIT_INFRA_TOUCHED: ${{ inputs.unit-infra-touched }}
|
||||
LOKI_INFRA_TOUCHED: ${{ inputs.loki-infra-touched }}
|
||||
E2E_INFRA_TOUCHED: ${{ inputs.e2e-infra-touched }}
|
||||
SHARED_SOURCES_TOUCHED: ${{ inputs.shared-sources-touched }}
|
||||
E2E_SPEC_MANIFEST: ${{ steps.manifest.outputs.path }}
|
||||
run: |
|
||||
# stdout is the plan JSON; the planner's own info logs go to stderr.
|
||||
# The planner writes this step's GITHUB_OUTPUT entries.
|
||||
bun .github/scripts/create-test-plan.ts > test-plan.json
|
||||
|
||||
- name: Upload test plan
|
||||
id: upload
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: test-plan-${{ github.run_id }}-${{ github.run_attempt }}
|
||||
path: test-plan.json
|
||||
if-no-files-found: error
|
||||
retention-days: 7
|
||||
|
||||
- name: Report test plan
|
||||
shell: bash
|
||||
run: |
|
||||
plan=test-plan.json
|
||||
row() { # name, selected, total
|
||||
if [ "$2" -eq "$3" ]; then echo "| $1 | all ($3) | $3 |"; else echo "| $1 | $2 | $3 |"; fi
|
||||
}
|
||||
{
|
||||
echo "## Affected test plan"
|
||||
echo
|
||||
echo "| Suite | Selected | Total |"
|
||||
echo "|---|---:|---:|"
|
||||
row "Unit" "$(jq '.stats.unit_specs_to_run_usage' "$plan")" "$(jq '.stats.unit_specs_all' "$plan")"
|
||||
row "Loki" "$(jq '.stats.loki_stories_to_run_usage' "$plan")" "$(jq '.stats.loki_stories_all' "$plan")"
|
||||
row "E2E" "$(jq '.stats.e2e_specs_to_run_usage' "$plan")" "$(jq '.stats.e2e_specs_all' "$plan")"
|
||||
echo
|
||||
echo "Changed: $(jq '.stats.fe_files_changed' "$plan") FE / $(jq '.stats.be_files_changed' "$plan") BE file(s) · $(jq '.stats.fe_modules_affected_usage' "$plan") FE module(s) affected"
|
||||
echo
|
||||
if [ "$(jq '.stats.loki_preview_touched' "$plan")" = "true" ]; then
|
||||
echo "**Loki: full run**: a file the Storybook preview imports changed."
|
||||
echo
|
||||
fi
|
||||
e2e_run=$(jq '.stats.e2e_specs_to_run_usage' "$plan")
|
||||
e2e_all=$(jq '.stats.e2e_specs_all' "$plan")
|
||||
if [ "$e2e_run" -eq "$e2e_all" ]; then
|
||||
echo "**E2E: full run** — no manifest, or a change the coverage manifest can't scope (backend, e2e infra, cljc/cljs)."
|
||||
else
|
||||
echo "<details><summary><b>E2E specs to run ($e2e_run)</b></summary>"
|
||||
echo
|
||||
jq -r '.e2e_specs_to_run[]' "$plan" | sed 's/^/- /'
|
||||
echo
|
||||
echo "</details>"
|
||||
fi
|
||||
echo
|
||||
echo "<details><summary>Full stats (rules vs usage graph)</summary>"
|
||||
echo
|
||||
echo '```json'
|
||||
jq '.stats' "$plan"
|
||||
echo '```'
|
||||
echo "</details>"
|
||||
} | tee -a "${GITHUB_STEP_SUMMARY:-/dev/null}"
|
||||
@@ -1,7 +1,4 @@
|
||||
name: Cross-branch migration test
|
||||
description: >-
|
||||
Boot Metabase on the base branch to migrate the app db, then boot the PR branch against the
|
||||
same db, verifying the PR's migrations apply cleanly on top of the base branch's schema.
|
||||
|
||||
inputs:
|
||||
base-ref: ## Base branch PR came from
|
||||
@@ -14,7 +11,7 @@ runs:
|
||||
steps:
|
||||
## First we deploy the base branch
|
||||
- name: Check out base branch
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ inputs.base-ref }}
|
||||
|
||||
@@ -45,7 +42,7 @@ runs:
|
||||
|
||||
## Then we deploy the PR branch to ensure it migrates up successfully
|
||||
- name: Check out pull request branch
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ inputs.head-ref }}
|
||||
|
||||
@@ -84,7 +81,7 @@ runs:
|
||||
shell: bash
|
||||
|
||||
- name: Re-check out base branch
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ inputs.base-ref }}
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Download previously stored uberjar
|
||||
uses: actions/github-script@v9
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: | # js
|
||||
const fs = require('fs');
|
||||
|
||||
@@ -15,10 +15,6 @@ inputs:
|
||||
description: OpenLDAP container
|
||||
required: true
|
||||
default: false
|
||||
mocksaml:
|
||||
description: Mock SAML container
|
||||
required: true
|
||||
default: false
|
||||
postgres:
|
||||
description: Postgres container
|
||||
required: true
|
||||
@@ -44,24 +40,10 @@ runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Authenticate to prevent API rate limit
|
||||
shell: bash
|
||||
env:
|
||||
DOCKER_USERNAME: ${{ inputs.username }}
|
||||
DOCKER_PASSWORD: ${{ inputs.password }}
|
||||
run: |
|
||||
for attempt in 1 2 3; do
|
||||
echo "Docker login attempt $attempt of 3..."
|
||||
if echo "$DOCKER_PASSWORD" | docker login --username "$DOCKER_USERNAME" --password-stdin; then
|
||||
echo "Docker login succeeded on attempt $attempt"
|
||||
break
|
||||
fi
|
||||
if [ $attempt -eq 3 ]; then
|
||||
echo "::error::Docker login failed after 3 attempts"
|
||||
exit 1
|
||||
fi
|
||||
echo "::warning::Docker login attempt $attempt failed, retrying in 15s..."
|
||||
sleep 15
|
||||
done
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
username: ${{ inputs.username }}
|
||||
password: ${{ inputs.password }}
|
||||
|
||||
- name: Start Containers
|
||||
shell: bash
|
||||
@@ -71,35 +53,21 @@ runs:
|
||||
|
||||
cd e2e/test/scenarios/
|
||||
|
||||
SERVICES=()
|
||||
[ "${{ inputs.postgres }}" = true ] && SERVICES+=(postgres-sample)
|
||||
[ "${{ inputs.mysql }}" = true ] && SERVICES+=(mysql-sample)
|
||||
[ "${{ inputs.mongo }}" = true ] && SERVICES+=(mongo-sample)
|
||||
[ "${{ inputs.maildev }}" = true ] && SERVICES+=(maildev maildev-ssl)
|
||||
[ "${{ inputs.openldap }}" = true ] && SERVICES+=(ldap)
|
||||
[ "${{ inputs.mocksaml }}" = true ] && SERVICES+=(saml-idp)
|
||||
[ "${{ inputs.webhook }}" = true ] && SERVICES+=(webhook-tester)
|
||||
[ "${{ inputs.snowplow }}" = true ] && SERVICES+=(snowplow-micro)
|
||||
|
||||
for attempt in 1 2 3; do
|
||||
echo "Docker compose up attempt $attempt of 3..."
|
||||
if docker compose up "${SERVICES[@]}" -d; then
|
||||
echo "Docker compose up succeeded on attempt $attempt"
|
||||
break
|
||||
fi
|
||||
if [ $attempt -eq 3 ]; then
|
||||
echo "::error::Docker compose up failed after 3 attempts"
|
||||
exit 1
|
||||
fi
|
||||
echo "::warning::Docker compose up attempt $attempt failed, retrying in 30s..."
|
||||
docker compose down 2>/dev/null || true
|
||||
sleep 30
|
||||
done
|
||||
docker compose up \
|
||||
$([ ${{ inputs.postgres }} = true ] && echo "postgres-sample") \
|
||||
$([ ${{ inputs.mysql }} = true ] && echo "mysql-sample") \
|
||||
$([ ${{ inputs.mongo }} = true ] && echo "mongo-sample") \
|
||||
$([ ${{ inputs.maildev }} = true ] && echo "maildev") \
|
||||
$([ ${{ inputs.maildev }} = true ] && echo "maildev-ssl") \
|
||||
$([ ${{ inputs.openldap }} = true ] && echo "ldap") \
|
||||
$([ ${{ inputs.webhook }} = true ] && echo "webhook-tester") \
|
||||
$([ ${{ inputs.snowplow }} = true ] && echo "snowplow-micro") \
|
||||
-d
|
||||
|
||||
if ${{ inputs.maildev }}; then
|
||||
echo -e "${Y}checking maildev container..." &&
|
||||
while ! nc -z localhost 1180; do sleep 1; done &&
|
||||
while ! nc -z localhost 1125; do sleep 1; done &&
|
||||
while ! nc -z localhost 1080; do sleep 1; done &&
|
||||
while ! nc -z localhost 1025; do sleep 1; done &&
|
||||
echo -e "${G}Maildev is up and running!"
|
||||
|
||||
echo -e "${Y}checking maildev-ssl container..." &&
|
||||
@@ -114,12 +82,6 @@ runs:
|
||||
echo -e "${G}Openldap is up and running!"
|
||||
fi
|
||||
|
||||
if ${{ inputs.mocksaml }}; then
|
||||
echo -e "${Y}checking mock saml container..." &&
|
||||
while ! nc -z localhost 8090; do sleep 1; done &&
|
||||
echo -e "${G}Mock SAML is up and running!"
|
||||
fi
|
||||
|
||||
if ${{ inputs.postgres }}; then
|
||||
echo -e "${Y}checking postgres container..." &&
|
||||
while ! nc -z localhost 5404; do sleep 1; done &&
|
||||
|
||||
35
.github/actions/fetch-artifact/action.yml
vendored
35
.github/actions/fetch-artifact/action.yml
vendored
@@ -20,29 +20,19 @@ inputs:
|
||||
description: Directory path where the jar should be extracted (defaults to current directory)
|
||||
required: false
|
||||
default: "."
|
||||
download:
|
||||
description: Default is true. Set to false to only look the artifact up and report `fetch-hit`.
|
||||
required: false
|
||||
default: "true"
|
||||
|
||||
outputs:
|
||||
fetch-hit:
|
||||
description: Boolean indicating if an artifact was successfully found and downloaded.
|
||||
value: ${{ steps.find_release_artifact.outputs.fetch-hit }}
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: find_release_artifact
|
||||
id: find_release_artifact
|
||||
uses: actions/github-script@v9
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
result-encoding: string
|
||||
script: | # js
|
||||
const fs = require('fs');
|
||||
|
||||
const isRelease = "${{ inputs.release }}" === "true";
|
||||
const failOnMiss = "${{ inputs.fail-on-fetch-miss }}" !== "false";
|
||||
const shouldDownload = "${{ inputs.download }}" !== "false";
|
||||
const prefix = isRelease ? "metabase-release" : "metabase";
|
||||
const artifactName = `${prefix}-${{ inputs.edition }}-${{ inputs.commit }}-uberjar`;
|
||||
console.log(`Looking for artifact: ${artifactName}`);
|
||||
@@ -54,19 +44,13 @@ runs:
|
||||
per_page: 1,
|
||||
});
|
||||
|
||||
const artifact_id = artifacts.data?.artifacts?.[0]?.id;
|
||||
|
||||
if (!artifact_id) {
|
||||
core.setOutput('fetch-hit', 'false');
|
||||
} else {
|
||||
core.setOutput('fetch-hit', 'true');
|
||||
}
|
||||
|
||||
if (!shouldDownload) {
|
||||
console.log(`skipping download (download: false)`);
|
||||
if (!artifacts.data?.artifacts?.[0]?.id) {
|
||||
core.setFailed(`No artifacts found for ${artifactName}`);
|
||||
return;
|
||||
}
|
||||
|
||||
const artifact_id = artifacts.data.artifacts[0].id;
|
||||
|
||||
let download;
|
||||
const maxAttempts = 3;
|
||||
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
|
||||
@@ -90,7 +74,6 @@ runs:
|
||||
fs.writeFileSync(`${process.env.GITHUB_WORKSPACE}/mb.zip`, Buffer.from(download.data));
|
||||
|
||||
- name: Extract and rename JAR
|
||||
if: ${{ steps.find_release_artifact.outputs.fetch-hit == 'true' && inputs.download != 'false' }}
|
||||
shell: bash
|
||||
run: | # bash
|
||||
OUTPUT_NAME="${{ inputs.output-name }}"
|
||||
@@ -130,20 +113,18 @@ runs:
|
||||
|
||||
# Set the JAR_PATH environment variable
|
||||
if [ "$EXTRACT_PATH" == "." ]; then
|
||||
echo "JAR_PATH=$OUTPUT_NAME" >> "$GITHUB_ENV"
|
||||
echo "JAR_PATH=$OUTPUT_NAME" >> $GITHUB_ENV
|
||||
else
|
||||
echo "JAR_PATH=$EXTRACT_PATH/$OUTPUT_NAME" >> "$GITHUB_ENV"
|
||||
echo "JAR_PATH=$EXTRACT_PATH/$OUTPUT_NAME" >> $GITHUB_ENV
|
||||
fi
|
||||
|
||||
- name: Verify that this is a valid JAR file
|
||||
if: ${{ steps.find_release_artifact.outputs.fetch-hit == 'true' && inputs.download != 'false' }}
|
||||
shell: bash
|
||||
run: |
|
||||
echo "Verifying JAR at: $JAR_PATH"
|
||||
file --mime-type "$JAR_PATH"
|
||||
|
||||
- name: Reveal its version.properties
|
||||
if: ${{ steps.find_release_artifact.outputs.fetch-hit == 'true' && inputs.download != 'false' }}
|
||||
shell: bash
|
||||
run: |
|
||||
echo "Extracting version.properties from: $JAR_PATH"
|
||||
|
||||
@@ -22,7 +22,7 @@ runs:
|
||||
COMMIT=$(env -i git log $BASE_REF --grep="(#$PULL_REQUEST_NUMBER)" --format="%H")
|
||||
echo "commit SHA $COMMIT"
|
||||
|
||||
echo "commit=$COMMIT" >> "$GITHUB_OUTPUT"
|
||||
echo "commit=$COMMIT" >> $GITHUB_OUTPUT
|
||||
id: find-squashed-commit
|
||||
shell: bash
|
||||
env:
|
||||
|
||||
45
.github/actions/get-cache-keys/action.yml
vendored
Normal file
45
.github/actions/get-cache-keys/action.yml
vendored
Normal file
@@ -0,0 +1,45 @@
|
||||
name: Get cache keys
|
||||
description: |
|
||||
Get deterministic cache keys for Maven and Node.js caches based on the centrally controlled cache prefix,
|
||||
and the suffix based on the runner OS and the current week of the year.
|
||||
|
||||
outputs:
|
||||
m2-cache-key:
|
||||
description: "The cache key for the Maven cache."
|
||||
value: ${{ steps.keys-factory.outputs.m2-cache-key }}
|
||||
m2-restore-key:
|
||||
description: "The fallback restore key prefix for the Maven cache."
|
||||
value: ${{ steps.keys-factory.outputs.m2-restore-key }}
|
||||
node-cache-key:
|
||||
description: "The cache key for the Node.js cache."
|
||||
value: ${{ steps.keys-factory.outputs.node-cache-key }}
|
||||
node-restore-key:
|
||||
description: "The fallback restore key prefix for the Node.js cache."
|
||||
value: ${{ steps.keys-factory.outputs.node-restore-key }}
|
||||
eslint-cache-key:
|
||||
description: "The cache key for Eslint cache"
|
||||
value: ${{ steps.keys-factory.outputs.eslint-cache-key }}
|
||||
cypress-cache-key:
|
||||
description: "The cache key for Cypress cache"
|
||||
value: ${{ steps.keys-factory.outputs.cypress-cache-key }}
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Generate and output cache keys
|
||||
id: keys-factory
|
||||
run: |
|
||||
M2_CACHE_PREFIX='M2'
|
||||
NODE_CACHE_PREFIX='node-modules'
|
||||
ESLINT_CACHE_PREFIX='eslint'
|
||||
CYPRESS_CACHE_PREFIX='cypress'
|
||||
WEEK_OF_YEAR=$(/bin/date "+%G-%V")
|
||||
CACHE_SUFFIX="${{ runner.os }}-$WEEK_OF_YEAR"
|
||||
|
||||
echo "m2-cache-key=$M2_CACHE_PREFIX-$CACHE_SUFFIX" >> $GITHUB_OUTPUT
|
||||
echo "m2-restore-key=$M2_CACHE_PREFIX-" >> $GITHUB_OUTPUT
|
||||
echo "node-cache-key=$NODE_CACHE_PREFIX-$CACHE_SUFFIX" >> $GITHUB_OUTPUT
|
||||
echo "node-restore-key=$NODE_CACHE_PREFIX-" >> $GITHUB_OUTPUT
|
||||
echo "eslint-cache-key=$ESLINT_CACHE_PREFIX-$CACHE_SUFFIX" >> $GITHUB_OUTPUT
|
||||
echo "cypress-cache-key=$CYPRESS_CACHE_PREFIX-$CACHE_SUFFIX" >> $GITHUB_OUTPUT
|
||||
shell: bash
|
||||
94
.github/actions/get-docker-info/action.yml
vendored
94
.github/actions/get-docker-info/action.yml
vendored
@@ -1,19 +1,7 @@
|
||||
name: Get Docker Info
|
||||
description: Determines the registry, repository, and tag a container image is pushed to.
|
||||
description: Determines Docker repository and tag based on branch and edition
|
||||
|
||||
inputs:
|
||||
registry:
|
||||
description: Registry the image is pushed to, either "docker" or "ecr".
|
||||
required: false
|
||||
default: docker
|
||||
repo:
|
||||
description: Repository name, e.g. "metabase-enterprise". Overrides a branch derived repo.
|
||||
required: false
|
||||
default: ''
|
||||
tag:
|
||||
description: Image tag, e.g. v1.87.2. Overrides a branch derived tag.
|
||||
required: false
|
||||
default: ''
|
||||
branch:
|
||||
description: Branch name (defaults to current branch)
|
||||
required: false
|
||||
@@ -22,22 +10,14 @@ inputs:
|
||||
description: Edition (ee or oss)
|
||||
required: false
|
||||
default: ee
|
||||
aws-account-id:
|
||||
description: AWS account id hosting the ECR registry. Required for the ecr registry.
|
||||
owner:
|
||||
description: Repository owner (defaults to current repo owner)
|
||||
required: false
|
||||
default: ''
|
||||
aws-region:
|
||||
description: Region of the ECR registry.
|
||||
required: false
|
||||
default: us-east-1
|
||||
dockerhub-owner:
|
||||
description: DockerHub owner.
|
||||
required: false
|
||||
default: metabase
|
||||
default: ${{ github.repository_owner }}
|
||||
|
||||
outputs:
|
||||
repo:
|
||||
description: Docker repository, including the registry host (ecr) or the owner (docker)
|
||||
description: Docker repository name
|
||||
value: ${{ steps.docker-info.outputs.repo }}
|
||||
tag:
|
||||
description: Docker tag
|
||||
@@ -45,56 +25,50 @@ outputs:
|
||||
full_image:
|
||||
description: Full Docker image name (repo:tag)
|
||||
value: ${{ steps.docker-info.outputs.full_image }}
|
||||
editions:
|
||||
description: JSON array of editions to build
|
||||
value: ${{ steps.docker-info.outputs.editions }}
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Determine Docker info
|
||||
id: docker-info
|
||||
uses: actions/github-script@v9
|
||||
env:
|
||||
REGISTRY: ${{ inputs.registry }}
|
||||
REPO: ${{ inputs.repo }}
|
||||
TAG: ${{ inputs.tag }}
|
||||
BRANCH: ${{ inputs.branch }}
|
||||
EDITION: ${{ inputs.edition }}
|
||||
AWS_ACCOUNT_ID: ${{ inputs.aws-account-id }}
|
||||
AWS_REGION: ${{ inputs.aws-region }}
|
||||
DOCKERHUB_OWNER: ${{ inputs.dockerhub-owner }}
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: | # js
|
||||
const { REGISTRY, OWNER, BRANCH, EDITION, AWS_ACCOUNT_ID, AWS_REGION, DOCKERHUB_OWNER } = process.env;
|
||||
const branch = '${{ inputs.branch }}';
|
||||
const edition = '${{ inputs.edition }}';
|
||||
const owner = '${{ inputs.owner }}';
|
||||
|
||||
let tag = process.env.TAG;
|
||||
if (!tag) {
|
||||
tag = BRANCH === 'master' ? 'latest' : `${BRANCH.replaceAll('/', '-')}-${EDITION}`;
|
||||
}
|
||||
console.log(`Determining Docker info for branch: ${branch}, edition: ${edition}`);
|
||||
|
||||
let name = process.env.REPO;
|
||||
if (!name) {
|
||||
if (BRANCH === 'master') {
|
||||
name = EDITION === 'ee' ? 'metabase-enterprise-head' : 'metabase-head';
|
||||
let repo, tag, editions;
|
||||
|
||||
if (branch === 'master') {
|
||||
// Master builds both editions with specific repos
|
||||
editions = ['ee', 'oss'];
|
||||
tag = 'latest';
|
||||
|
||||
if (edition === 'ee') {
|
||||
repo = `${owner}/metabase-enterprise-head`;
|
||||
} else {
|
||||
name = 'metabase-dev';
|
||||
repo = `${owner}/metabase-head`;
|
||||
}
|
||||
}
|
||||
|
||||
let repo;
|
||||
if (REGISTRY === 'ecr') {
|
||||
if (!AWS_ACCOUNT_ID) {
|
||||
core.setFailed('Pushing to AWS ECR requires aws-account-id to be set.');
|
||||
return;
|
||||
}
|
||||
repo = `${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/${name}`;
|
||||
} else if (REGISTRY === 'docker') {
|
||||
repo = `${DOCKERHUB_OWNER}/${name}`;
|
||||
} else {
|
||||
core.setFailed(`Unknown registry "${REGISTRY}", expected "docker" or "ecr".`);
|
||||
return;
|
||||
// All other branches only build EE to metabase-dev
|
||||
editions = ['ee'];
|
||||
tag = branch.replaceAll('/', '-'); // slashes make docker sad
|
||||
repo = `${owner}/metabase-dev`;
|
||||
}
|
||||
|
||||
const fullImage = `${repo}:${tag}`;
|
||||
|
||||
// Set outputs
|
||||
core.setOutput('repo', repo);
|
||||
core.setOutput('tag', tag);
|
||||
core.setOutput('full_image', `${repo}:${tag}`);
|
||||
core.setOutput('full_image', fullImage);
|
||||
core.setOutput('editions', JSON.stringify(editions));
|
||||
|
||||
console.log(`Resolved: registry = ${REGISTRY}, repo = ${repo}, tag = ${tag}`);
|
||||
console.log(`Docker image: ${fullImage}`);
|
||||
console.log(`Editions: ${JSON.stringify(editions)}`);
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
name: Notify Pull Request
|
||||
description: Post a comment on the pull request, optionally replacing the previous one with the same message.
|
||||
inputs:
|
||||
remove-previous-comment:
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
add-new-comment:
|
||||
required: false
|
||||
type: boolean
|
||||
default: true
|
||||
message:
|
||||
required: true
|
||||
@@ -17,7 +18,7 @@ runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Remove previous pull-request comment
|
||||
uses: actions/github-script@v9
|
||||
uses: actions/github-script@v7
|
||||
if: ${{ inputs.remove-previous-comment == 'true' && always() && github.event_name == 'pull_request' }}
|
||||
with:
|
||||
script: |
|
||||
@@ -52,7 +53,7 @@ runs:
|
||||
}
|
||||
|
||||
- name: Notify pull-request
|
||||
uses: actions/github-script@v9
|
||||
uses: actions/github-script@v7
|
||||
if: ${{ inputs.add-new-comment == 'true' && always() && github.event_name == 'pull_request' }}
|
||||
id: notify-pull-request
|
||||
with:
|
||||
|
||||
56
.github/actions/prepare-backend/action.yml
vendored
56
.github/actions/prepare-backend/action.yml
vendored
@@ -4,50 +4,25 @@ description: |
|
||||
inputs:
|
||||
java-version:
|
||||
required: true
|
||||
default: "25"
|
||||
default: "21"
|
||||
clojure-version:
|
||||
required: true
|
||||
default: "1.12.0.1488"
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Prepare JDK ${{ inputs.java-version }}
|
||||
uses: actions/setup-java@v5.2.0
|
||||
uses: actions/setup-java@v4
|
||||
with:
|
||||
java-version: ${{ inputs.java-version }}
|
||||
distribution: "temurin"
|
||||
- name: Cache spec
|
||||
id: cache-spec
|
||||
shell: bash
|
||||
run: .github/scripts/cache-keys.sh | tee -a "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Restore Clojure CLI
|
||||
uses: actions/cache/restore@v5
|
||||
with:
|
||||
path: ${{ steps.cache-spec.outputs.clojure-path }}
|
||||
key: ${{ steps.cache-spec.outputs.clojure-key }}
|
||||
|
||||
- name: Install Clojure CLI
|
||||
shell: bash
|
||||
env:
|
||||
CLOJURE_PREFIX: ${{ steps.cache-spec.outputs.clojure-path }}
|
||||
CLOJURE_VERSION: ${{ steps.cache-spec.outputs.clojure-version }}
|
||||
run: |
|
||||
set -uo pipefail
|
||||
echo "$CLOJURE_PREFIX/bin" >> "$GITHUB_PATH"
|
||||
export PATH="$CLOJURE_PREFIX/bin:$PATH"
|
||||
|
||||
# Whether the prefix already holds the right install is decided by running it, not by the
|
||||
# cache-hit flag. The installer bakes the prefix into the clj/clojure shims, so a tree restored
|
||||
# under a different $HOME points at a directory that is not there; and on a runner that is not
|
||||
# ephemeral, a previous version's prefix outlives the cache entry that stopped matching it.
|
||||
if clojure --version | grep -qF "$CLOJURE_VERSION"; then
|
||||
clojure --version
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cd "$RUNNER_TEMP"
|
||||
for attempt in 1 2 3; do
|
||||
curl --retry 3 --retry-delay 5 --fail -O "https://download.clojure.org/install/linux-install-$CLOJURE_VERSION.sh" \
|
||||
&& bash "./linux-install-$CLOJURE_VERSION.sh" --prefix "$CLOJURE_PREFIX" \
|
||||
curl --retry 3 --retry-delay 5 --fail -O https://download.clojure.org/install/linux-install-${{ inputs.clojure-version }}.sh \
|
||||
&& sudo bash ./linux-install-${{ inputs.clojure-version }}.sh \
|
||||
&& break
|
||||
echo "::warning::Clojure install attempt $attempt failed, retrying in 10s..."
|
||||
sleep 10
|
||||
@@ -57,16 +32,23 @@ runs:
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Get M2 cache key
|
||||
uses: ./.github/actions/get-cache-keys
|
||||
id: get-cache-keys
|
||||
|
||||
- name: Install uv (for Python package installation)
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
enable-cache: false
|
||||
version: "0.10.2" # keep in sync with mise.toml
|
||||
github-token: ${{ github.token }}
|
||||
|
||||
- name: Restore M2 cache
|
||||
uses: actions/cache/restore@v5
|
||||
uses: actions/cache/restore@v4
|
||||
with:
|
||||
path: ${{ steps.cache-spec.outputs.m2-path }}
|
||||
key: ${{ steps.cache-spec.outputs.m2-key }}
|
||||
restore-keys: ${{ steps.cache-spec.outputs.m2-restore-key }}
|
||||
path: |
|
||||
~/.m2
|
||||
~/.gitlibs
|
||||
key: ${{ steps.get-cache-keys.outputs.m2-cache-key }}
|
||||
restore-keys: |
|
||||
${{ steps.get-cache-keys.outputs.m2-restore-key }}
|
||||
|
||||
52
.github/actions/prepare-ci-conductor/action.yml
vendored
52
.github/actions/prepare-ci-conductor/action.yml
vendored
@@ -1,52 +0,0 @@
|
||||
name: Prepare CI Conductor
|
||||
description: >-
|
||||
Make the ci-conductor sources runnable in the workspace.
|
||||
Unless the checkout is headed for master, they are overlaid from master, so every
|
||||
branch runs the latest scripts while its workflow wiring stays branch-historical.
|
||||
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Detect bun
|
||||
id: bun
|
||||
shell: bash
|
||||
run: |
|
||||
if command -v bun >/dev/null; then
|
||||
echo "present=true" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "present=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Install bun
|
||||
if: steps.bun.outputs.present != 'true'
|
||||
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
|
||||
with:
|
||||
bun-version-file: "package.json"
|
||||
token: ${{ github.token }}
|
||||
|
||||
# Anything headed for master runs master's own sources, so a PR can test its own changes.
|
||||
- name: Fetch ci-conductor sources from master
|
||||
if: ${{ github.base_ref != 'master' && github.ref_name != 'master' }}
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
repository: metabase/metabase
|
||||
ref: master
|
||||
path: .ci-conductor-master
|
||||
persist-credentials: false
|
||||
sparse-checkout-cone-mode: false
|
||||
sparse-checkout: |
|
||||
release/ci-conductor
|
||||
e2e/support/ci_conductor.ts
|
||||
|
||||
# e2e/support/config.js and e2e/support/ci_conductor.ts resolve these by relative
|
||||
# path at bundle time, so the sources have to land where the branch expects them.
|
||||
- name: Overlay ci-conductor sources from master
|
||||
if: ${{ github.base_ref != 'master' && github.ref_name != 'master' }}
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
rm -rf release/ci-conductor
|
||||
mv .ci-conductor-master/release/ci-conductor release/ci-conductor
|
||||
mv .ci-conductor-master/e2e/support/ci_conductor.ts e2e/support/ci_conductor.ts
|
||||
rm -rf .ci-conductor-master
|
||||
echo "Overlaid ci-conductor sources from master."
|
||||
22
.github/actions/prepare-cypress/action.yml
vendored
22
.github/actions/prepare-cypress/action.yml
vendored
@@ -15,27 +15,21 @@ outputs:
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
# e2e/support/config.js imports the reporter, so it has to be in place before Cypress bundles.
|
||||
- name: Prepare CI Conductor
|
||||
uses: ./.github/actions/prepare-ci-conductor
|
||||
|
||||
- name: Install Chrome v144
|
||||
uses: browser-actions/setup-chrome@2e1d749697dd1612b833dba4a722266286fbefcd # v2.1.2
|
||||
uses: browser-actions/setup-chrome@v2
|
||||
with:
|
||||
chrome-version: 144
|
||||
id: setup-chrome
|
||||
|
||||
- name: Cache spec
|
||||
id: cache-spec
|
||||
shell: bash
|
||||
run: .github/scripts/cache-keys.sh | tee -a "$GITHUB_OUTPUT"
|
||||
- name: Get cache keys
|
||||
uses: ./.github/actions/get-cache-keys
|
||||
id: get-cache-keys
|
||||
|
||||
- name: Restore Cypress binary
|
||||
uses: actions/cache/restore@v5
|
||||
- name: Get Cypress cache
|
||||
uses: actions/cache/restore@v4
|
||||
with:
|
||||
path: ${{ steps.cache-spec.outputs.cypress-path }}
|
||||
key: ${{ steps.cache-spec.outputs.cypress-key }}
|
||||
restore-keys: ${{ steps.cache-spec.outputs.cypress-restore-key }}
|
||||
path: ~/.cache/Cypress
|
||||
key: ${{ steps.get-cache-keys.outputs.cypress-cache-key }}
|
||||
|
||||
- name: Ensure that Cypress executable is ready
|
||||
run: |
|
||||
|
||||
58
.github/actions/prepare-frontend/action.yml
vendored
58
.github/actions/prepare-frontend/action.yml
vendored
@@ -16,37 +16,43 @@ runs:
|
||||
with:
|
||||
node-version: ${{ inputs.node-version }}
|
||||
|
||||
- name: Cache spec
|
||||
id: cache-spec
|
||||
shell: bash
|
||||
run: .github/scripts/cache-keys.sh | tee -a "$GITHUB_OUTPUT"
|
||||
- name: Get cache keys
|
||||
uses: ./.github/actions/get-cache-keys
|
||||
id: get-cache-keys
|
||||
|
||||
# The spec names the store path rather than discovering it, so that a job can restore the cache
|
||||
# before bun exists. Exported job-wide rather than per-step because every later bun invocation has
|
||||
# to resolve against the same store. Confirm bun agreed: if the variable stopped taking effect, bun
|
||||
# would fill its default directory and the cache would silently save an empty tree.
|
||||
- name: Point bun at the cached store
|
||||
shell: bash
|
||||
env:
|
||||
STORE: ${{ steps.cache-spec.outputs.bun-store-path }}
|
||||
run: |
|
||||
echo "BUN_INSTALL_CACHE_DIR=$STORE" >> "$GITHUB_ENV"
|
||||
ACTUAL=$(BUN_INSTALL_CACHE_DIR="$STORE" bun pm cache)
|
||||
if [ "$ACTUAL" != "$STORE" ]; then
|
||||
echo "::error::bun reports its store at $ACTUAL, not $STORE; BUN_INSTALL_CACHE_DIR is not taking effect"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Restore bun store
|
||||
uses: actions/cache/restore@v5
|
||||
- name: Restore M2 cache
|
||||
uses: actions/cache/restore@v4
|
||||
with:
|
||||
path: ${{ steps.cache-spec.outputs.bun-store-path }}
|
||||
key: ${{ steps.cache-spec.outputs.bun-store-key }}
|
||||
restore-keys: ${{ steps.cache-spec.outputs.bun-store-restore-key }}
|
||||
path: |
|
||||
~/.m2
|
||||
~/.gitlibs
|
||||
key: ${{ steps.get-cache-keys.outputs.m2-cache-key }}
|
||||
restore-keys: |
|
||||
${{ steps.get-cache-keys.outputs.m2-restore-key }}
|
||||
|
||||
- name: Get bun cache directory
|
||||
id: bun-cache
|
||||
run: |
|
||||
BUN_CACHE_DIR=$(bun pm cache)
|
||||
echo "Bun cache directory: $BUN_CACHE_DIR"
|
||||
echo "dir=$BUN_CACHE_DIR" >> $GITHUB_OUTPUT
|
||||
shell: bash
|
||||
|
||||
- name: Restore node_modules cache
|
||||
uses: actions/cache/restore@v4
|
||||
id: node-modules-cache
|
||||
with:
|
||||
path: |
|
||||
${{ steps.bun-cache.outputs.dir }}
|
||||
node_modules
|
||||
key: ${{ steps.get-cache-keys.outputs.node-cache-key }}
|
||||
restore-keys: |
|
||||
${{ steps.get-cache-keys.outputs.node-restore-key }}
|
||||
|
||||
# postinstall lifecycle hook can be dangerous in CI so we only run it locally (see package.json)
|
||||
# If a job wants to patch packages, it does so explicitly in the CI. This setting is opt-out.
|
||||
- name: Install front-end dependencies
|
||||
# For example, we never want to cache node_modules with patches already applied so cache-generator.yml skips that step.
|
||||
- name: Clean install
|
||||
run: bun install --frozen-lockfile
|
||||
shell: bash
|
||||
- name: Patch frontend dependencies
|
||||
|
||||
6
.github/actions/prepare-node-bun/action.yml
vendored
6
.github/actions/prepare-node-bun/action.yml
vendored
@@ -8,13 +8,13 @@ runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Prepare Node.js
|
||||
uses: actions/setup-node@v6.4.0
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ inputs.node-version }}
|
||||
node-version-file: ".nvmrc"
|
||||
|
||||
- name: Install bun
|
||||
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
|
||||
uses: oven-sh/setup-bun@v2
|
||||
with:
|
||||
bun-version-file: 'package.json'
|
||||
bun-version: "1.3.7"
|
||||
token: ${{ github.token }}
|
||||
|
||||
@@ -4,9 +4,9 @@ description: Set up bun and build release scripts
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with:
|
||||
bun-version-file: 'package.json'
|
||||
bun-version: "1.3.7"
|
||||
token: ${{ github.token }}
|
||||
|
||||
- name: Build release scripts
|
||||
|
||||
39
.github/actions/prepare-shoppy-e2e/action.yml
vendored
39
.github/actions/prepare-shoppy-e2e/action.yml
vendored
@@ -14,6 +14,21 @@ inputs:
|
||||
staging-mb-all-features-token:
|
||||
description: "Staging MB all features token"
|
||||
required: true
|
||||
shoppy-iam-role:
|
||||
description: "Shoppy IAM role for AWS credentials"
|
||||
required: true
|
||||
shoppy-metabase-app-db-s3-bucket:
|
||||
description: "S3 bucket for Shoppy app DB dump"
|
||||
required: true
|
||||
shoppy-metabase-app-db-s3-key:
|
||||
description: "S3 key for Shoppy app DB dump"
|
||||
required: true
|
||||
shoppy-metabase-app-db-user:
|
||||
description: "Shoppy Metabase app DB user"
|
||||
required: true
|
||||
shoppy-metabase-app-db-database:
|
||||
description: "Shoppy Metabase app DB database"
|
||||
required: true
|
||||
metastore-dev-server-url:
|
||||
description: "Metastore dev server URL"
|
||||
required: true
|
||||
@@ -39,7 +54,7 @@ runs:
|
||||
|
||||
# We have to use the latest version that does not have this issue https://github.com/docker/compose/pull/12752
|
||||
- name: Set up Docker Compose
|
||||
uses: docker/setup-compose-action@8cccb8c14b6500aaffebff1aa49c502c34d2e5e6 # v2.1.0
|
||||
uses: docker/setup-compose-action@v1
|
||||
with:
|
||||
version: latest
|
||||
|
||||
@@ -52,7 +67,7 @@ runs:
|
||||
output-name: "metabase.jar"
|
||||
|
||||
- name: Retrieve Embedding SDK dist artifact
|
||||
uses: actions/download-artifact@v8
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: embedding-sdk-${{ inputs.commit }}
|
||||
path: ${{ github.workspace }}/resources/embedding-sdk
|
||||
@@ -73,11 +88,25 @@ runs:
|
||||
REPO_URL="https://github.com/metabase/shoppy.git"
|
||||
echo "Checking if branch '${SAMPLE_APP_BRANCH_NAME}' exists in ${REPO_URL}..."
|
||||
if git ls-remote --exit-code --heads "$REPO_URL" "${SAMPLE_APP_BRANCH_NAME}"; then
|
||||
echo "sample_app_branch_exists=true" >> "$GITHUB_OUTPUT"
|
||||
echo "sample_app_branch_exists=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "sample_app_branch_exists=false" >> "$GITHUB_OUTPUT"
|
||||
echo "sample_app_branch_exists=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Configure AWS credentials
|
||||
if: ${{ steps.check-sample-app-branch.outputs.sample_app_branch_exists == 'true' }}
|
||||
uses: aws-actions/configure-aws-credentials@v4
|
||||
with:
|
||||
role-to-assume: ${{ inputs.shoppy-iam-role }}
|
||||
role-session-name: GitHub_to_AWS_via_FederatedOIDC
|
||||
aws-region: us-east-1
|
||||
|
||||
- name: Download Shoppy's App DB PostgreSQL dump from S3
|
||||
if: ${{ steps.check-sample-app-branch.outputs.sample_app_branch_exists == 'true' }}
|
||||
shell: bash
|
||||
run: |
|
||||
aws s3 cp s3://${{ inputs.shoppy-metabase-app-db-s3-bucket }}/${{ inputs.shoppy-metabase-app-db-s3-key }} ./e2e/tmp/db_dumps/shoppy_metabase_app_db_dump.sql
|
||||
|
||||
- name: Prepare and launch Shoppy containers
|
||||
if: ${{ steps.check-sample-app-branch.outputs.sample_app_branch_exists == 'true' }}
|
||||
shell: bash
|
||||
@@ -86,6 +115,8 @@ runs:
|
||||
SAMPLE_APP_ENVIRONMENT: ${{ inputs.sample-app-environment }}
|
||||
CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ inputs.staging-mb-all-features-token }}
|
||||
METASTORE_DEV_SERVER_URL: ${{ inputs.metastore-dev-server-url }}
|
||||
METABASE_APP_DB_USER: ${{ inputs.shoppy-metabase-app-db-user }}
|
||||
METABASE_APP_DB: ${{ inputs.shoppy-metabase-app-db-database }}
|
||||
SHOPPY_DATADOG_APPLICATION_ID: ${{ inputs.shoppy-datadog-application-id }}
|
||||
SHOPPY_DATADOG_CLIENT_TOKEN: ${{ inputs.shoppy-datadog-client-token }}
|
||||
SHOPPY_DATADOG_SITE: "us5.datadoghq.com"
|
||||
|
||||
@@ -1,39 +0,0 @@
|
||||
name: Prepare test selection
|
||||
description: Download this run's test plan and write the suite's selected test paths.
|
||||
|
||||
inputs:
|
||||
suite:
|
||||
description: The suite to select tests for (unit, loki or e2e).
|
||||
required: true
|
||||
test-plan-artifact-id:
|
||||
description: ID of this run's test-plan.json artifact.
|
||||
required: true
|
||||
require-download:
|
||||
description: Fail the step when the test plan download fails, instead of running the full suite.
|
||||
required: false
|
||||
default: "false"
|
||||
|
||||
outputs:
|
||||
paths-file:
|
||||
description: JSON file listing the selected test paths, or '' when the suite runs in full.
|
||||
value: ${{ steps.selection.outputs.paths-file }}
|
||||
selection:
|
||||
description: full, narrowed or empty, or '' when the plan is unavailable and the suite runs in full.
|
||||
value: ${{ steps.selection.outputs.selection }}
|
||||
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Download test plan
|
||||
id: test-plan
|
||||
continue-on-error: ${{ inputs.require-download != 'true' }}
|
||||
uses: actions/download-artifact@v8
|
||||
with:
|
||||
artifact-ids: ${{ inputs.test-plan-artifact-id }}
|
||||
path: ${{ runner.temp }}/test-plan
|
||||
- name: Prepare test selection
|
||||
id: selection
|
||||
shell: bash
|
||||
env:
|
||||
PLAN_DOWNLOADED: ${{ steps.test-plan.outcome }}
|
||||
run: bun .github/scripts/prepare-test-selection.ts "${{ inputs.suite }}"
|
||||
26
.github/actions/prepare-uberjar-artifact/action.yml
vendored
Normal file
26
.github/actions/prepare-uberjar-artifact/action.yml
vendored
Normal file
@@ -0,0 +1,26 @@
|
||||
name: Prepare Uberjar Artifact
|
||||
description: Prepare and upload Metabase JAR as a GitHub artifact.
|
||||
|
||||
inputs:
|
||||
edition:
|
||||
required: false
|
||||
default: "ee"
|
||||
description: Metabase edition (ee or oss)
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Mark with the commit hash
|
||||
run: git rev-parse --short HEAD > COMMIT-ID
|
||||
shell: bash
|
||||
- name: Calculate SHA256 checksum
|
||||
run: sha256sum ./target/uberjar/metabase.jar > SHA256.sum
|
||||
shell: bash
|
||||
- name: Upload JARs as artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: metabase-${{ inputs.edition }}-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
|
||||
path: |
|
||||
./target/uberjar/metabase.jar
|
||||
./COMMIT-ID
|
||||
./SHA256.sum
|
||||
38
.github/actions/quarantine-gate/action.yml
vendored
38
.github/actions/quarantine-gate/action.yml
vendored
@@ -1,38 +0,0 @@
|
||||
name: CI Conductor quarantine gate
|
||||
description: >-
|
||||
Check this job's test failures against ci-conductor's quarantine list.
|
||||
Either enforces the quarantine or only prints a verdict (dry run).
|
||||
Fails closed when enforcing but the `CI_CONDUCTOR_BASE_URL` is missing or unreachable.
|
||||
|
||||
inputs:
|
||||
adapter:
|
||||
description: Which gate to run — one of `backend`, `frontend`, `e2e`.
|
||||
required: true
|
||||
test-suite:
|
||||
description: >-
|
||||
Suite label to gate (CI_CONDUCTOR_TEST_SUITE). MUST match the label this
|
||||
job reports under, or the gate checks the wrong quarantine list.
|
||||
required: true
|
||||
enforce-quarantine:
|
||||
description: >-
|
||||
Whether the gate actually gates the job. When "false", it only prints a verdict.
|
||||
required: false
|
||||
default: "false"
|
||||
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Prepare CI Conductor
|
||||
uses: ./.github/actions/prepare-ci-conductor
|
||||
- name: Check failures against the quarantine list
|
||||
shell: bash
|
||||
env:
|
||||
CI_CONDUCTOR_TEST_SUITE: ${{ inputs.test-suite }}
|
||||
ADAPTER: ${{ inputs.adapter }}
|
||||
QUARANTINE_DRY_RUN: ${{ inputs.enforce-quarantine != 'true' }}
|
||||
run: |
|
||||
case "$ADAPTER" in
|
||||
backend|frontend|e2e) ;;
|
||||
*) echo "::error::quarantine-gate: unknown adapter '$ADAPTER' (expected backend|frontend|e2e)"; exit 1 ;;
|
||||
esac
|
||||
bun "release/ci-conductor/src/check-${ADAPTER}-quarantine.ts"
|
||||
@@ -1,35 +0,0 @@
|
||||
name: Report failures to CI Conductor
|
||||
description: >-
|
||||
Report this job's test failures to ci-conductor. Best-effort; no-ops unless
|
||||
CI_CONDUCTOR_BASE_URL is set. Gate it at the call site with continue-on-error
|
||||
and an `if:` on the test step's outcome.
|
||||
|
||||
inputs:
|
||||
test-suite:
|
||||
description: Per-job identity label (CI_CONDUCTOR_TEST_SUITE); unique per job/leg.
|
||||
required: true
|
||||
adapter:
|
||||
description: Which reporter to run; resolves to report-<adapter>.ts. backend or frontend.
|
||||
required: true
|
||||
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Prepare CI Conductor
|
||||
uses: ./.github/actions/prepare-ci-conductor
|
||||
- name: Report failed tests to ci-conductor
|
||||
shell: bash
|
||||
env:
|
||||
REPO_ID: ${{ github.event.repository.id }}
|
||||
COMMIT_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
|
||||
TARGET_BRANCH: ${{ github.event.pull_request.base.ref || github.ref_name }}
|
||||
CI_CONDUCTOR_TEST_SUITE: ${{ inputs.test-suite }}
|
||||
ADAPTER: ${{ inputs.adapter }}
|
||||
IS_RERUN: ${{ github.run_attempt != '1' }}
|
||||
# Keep the path constrained to known entrypoints.
|
||||
run: |
|
||||
case "$ADAPTER" in
|
||||
backend|frontend) ;;
|
||||
*) echo "::error::Unknown ci-conductor adapter '$ADAPTER'"; exit 1 ;;
|
||||
esac
|
||||
bun "release/ci-conductor/src/report-${ADAPTER}.ts"
|
||||
66
.github/actions/resolve-job-id/action.yml
vendored
66
.github/actions/resolve-job-id/action.yml
vendored
@@ -1,66 +0,0 @@
|
||||
name: Resolve current job id
|
||||
description: >
|
||||
Resolve this job's GitHub workflow_jobs.id via the Actions API and export it
|
||||
as JOB_ID (and JOB_HTML_URL) to $GITHUB_ENV, so any later step can read the
|
||||
numeric job id without making its own API call. Best-effort: warns and
|
||||
proceeds on failure so it can never break the run.
|
||||
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Resolve JOB_ID
|
||||
shell: bash
|
||||
env:
|
||||
# Use the built-in per-job token, not a PAT: this lookup is best-effort
|
||||
# (a miss just leaves JOB_ID unset, never breaks the run) and other jobs
|
||||
# depend on the automation PAT's rate-limit budget — we don't want ~70
|
||||
# e2e shards drawing it down. Worst case under heavy CI load is the
|
||||
# occasional unresolved job_id, which is acceptable. DEV-2117.
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
RUNNER_NAME: ${{ runner.name }}
|
||||
run: |
|
||||
# GitHub doesn't expose the current job's numeric id, so we look it up
|
||||
# from the Actions API. A runner runs exactly one job at a time, so the
|
||||
# in-progress job with this runner_name within (run_id, run_attempt) is
|
||||
# unique. We require status == in_progress and no completed_at so a
|
||||
# reused runner can't match a job it already finished this attempt, with
|
||||
# max_by(.started_at) as a final tiebreaker.
|
||||
#
|
||||
# $GITHUB_RUN_ID is the whole CI run, whose total job count exceeds the
|
||||
# 100-per-page cap (~185 jobs), so this shard's job is frequently NOT on
|
||||
# the first page. --paginate --slurp pulls every page and slurps them
|
||||
# into one array so the filter sees all jobs, not just the first 100 —
|
||||
# without it the lookup returned null whenever this job sorted past
|
||||
# position 100 (DEV-2117).
|
||||
#
|
||||
# gh rejects `--slurp` together with `--jq` ("the --slurp option is not
|
||||
# supported with --jq"), so we slurp with gh and run the filter through a
|
||||
# standalone `jq`. The earlier `--slurp --jq` form errored on every
|
||||
# attempt — swallowed by 2>/dev/null — leaving JOB_ID permanently unset.
|
||||
set +e
|
||||
api_url="/repos/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID/attempts/$GITHUB_RUN_ATTEMPT/jobs?per_page=100"
|
||||
jq_filter="[.[].jobs[] | select(.runner_name == \"$RUNNER_NAME\" and .status == \"in_progress\" and .completed_at == null)] | max_by(.started_at) | \"\(.id) \(.html_url)\""
|
||||
|
||||
# The Actions jobs API is eventually consistent: after a runner picks
|
||||
# up a job, it can take a few seconds for runner_name to appear in the
|
||||
# response. Retry briefly to absorb that.
|
||||
out=""
|
||||
for attempt in 1 2 3 4 5; do
|
||||
out=$(gh api --paginate --slurp "$api_url" 2>/dev/null | jq -r "$jq_filter" 2>/dev/null)
|
||||
if [ -n "$out" ] && [ "$out" != "null null" ]; then
|
||||
break
|
||||
fi
|
||||
out=""
|
||||
echo "Attempt $attempt: job not yet visible in Actions API; retrying..."
|
||||
sleep 3
|
||||
done
|
||||
|
||||
if [ -n "$out" ]; then
|
||||
read -r job_id job_url <<<"$out"
|
||||
echo "JOB_ID=$job_id" >> "$GITHUB_ENV"
|
||||
echo "JOB_HTML_URL=$job_url" >> "$GITHUB_ENV"
|
||||
echo "Resolved JOB_ID=$job_id (runner=$RUNNER_NAME)"
|
||||
else
|
||||
echo "::warning::Could not resolve current job id after 5 attempts (runner=$RUNNER_NAME)"
|
||||
fi
|
||||
exit 0
|
||||
111
.github/actions/sdk-bundle-size-check/action.yml
vendored
Normal file
111
.github/actions/sdk-bundle-size-check/action.yml
vendored
Normal file
@@ -0,0 +1,111 @@
|
||||
name: Embedding SDK Bundle Size Check
|
||||
description: |
|
||||
Downloads the current branch's uberjar and builds the base ref's
|
||||
embedding-sdk.js from source, compares their sizes,
|
||||
and sets output if the size change exceeds the threshold.
|
||||
|
||||
inputs:
|
||||
edition:
|
||||
description: 'Metabase edition (ee or oss)'
|
||||
required: true
|
||||
base-ref-commit:
|
||||
description: 'Commit SHA of the base ref (e.g. merge base).'
|
||||
required: true
|
||||
threshold-percent:
|
||||
description: 'Percent threshold for bundle size change'
|
||||
required: true
|
||||
|
||||
outputs:
|
||||
status:
|
||||
description: "Bundle size status: increased, decreased, or stable"
|
||||
value: ${{ steps.compare.outputs.status }}
|
||||
size_change_percent:
|
||||
description: "size change percent (positive = increase, negative = decrease)"
|
||||
value: ${{ steps.compare.outputs.size_change_percent }}
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Prepare backend
|
||||
uses: ./.github/actions/prepare-backend
|
||||
- name: Prepare frontend
|
||||
uses: ./.github/actions/prepare-frontend
|
||||
- name: Download current uberjar
|
||||
uses: ./.github/actions/fetch-artifact
|
||||
with:
|
||||
commit: ${{ github.event.pull_request.head.sha || github.sha }}
|
||||
edition: ${{ inputs.edition }}
|
||||
extract-path: temp/current-uberjar
|
||||
- name: Extract embedding-sdk bundles from current uberjar
|
||||
shell: bash
|
||||
run: |
|
||||
# Try new layout first (app/embedding-sdk/legacy/), fall back to old layout (app/embedding-sdk.js)
|
||||
unzip -j temp/current-uberjar/metabase.jar \
|
||||
"frontend_client/app/embedding-sdk/legacy/embedding-sdk.js" \
|
||||
-d temp/current-bundle 2>/dev/null || \
|
||||
unzip -j temp/current-uberjar/metabase.jar \
|
||||
"frontend_client/app/embedding-sdk.js" \
|
||||
-d temp/current-bundle
|
||||
|
||||
# Extract chunks directory if it exists (new layout only)
|
||||
mkdir -p temp/current-chunks
|
||||
unzip -j temp/current-uberjar/metabase.jar \
|
||||
"frontend_client/app/embedding-sdk/chunks/*" \
|
||||
-d temp/current-chunks 2>/dev/null || true
|
||||
- name: Checkout base ref
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ inputs.base-ref-commit }}
|
||||
path: temp/base-ref-src
|
||||
- name: Build base-ref embedding-sdk.js
|
||||
shell: bash
|
||||
run: |
|
||||
cd temp/base-ref-src
|
||||
bun install
|
||||
MB_EDITION=${{ inputs.edition }} bun run build-release
|
||||
- name: Compare sizes and set output
|
||||
id: compare
|
||||
shell: bash
|
||||
run: |
|
||||
# --- Legacy (monolithic) bundle comparison ---
|
||||
current_size=$(wc -c < temp/current-bundle/embedding-sdk.js) || { echo "::error::Failed to read current bundle"; exit 1; }
|
||||
|
||||
# Base ref: try new layout first, fall back to old
|
||||
if [ -f "temp/base-ref-src/resources/frontend_client/app/embedding-sdk/legacy/embedding-sdk.js" ]; then
|
||||
base_ref_size=$(wc -c < "temp/base-ref-src/resources/frontend_client/app/embedding-sdk/legacy/embedding-sdk.js")
|
||||
else
|
||||
base_ref_size=$(wc -c < "temp/base-ref-src/resources/frontend_client/app/embedding-sdk.js")
|
||||
fi
|
||||
[ "$current_size" -gt 0 ] && [ "$base_ref_size" -gt 0 ] || { echo "::error::Bundle file is empty"; exit 1; }
|
||||
|
||||
diff=$((current_size - base_ref_size))
|
||||
percent=$(( diff * 100 / base_ref_size ))
|
||||
|
||||
echo "=== Legacy (monolithic) bundle ==="
|
||||
echo "Current size: $current_size bytes"
|
||||
echo "Base ref size: $base_ref_size bytes"
|
||||
echo "Difference: $diff bytes ($percent%)"
|
||||
|
||||
threshold="${{ inputs.threshold-percent }}"
|
||||
if [ "$percent" -gt "$threshold" ]; then
|
||||
status="increased"
|
||||
elif [ "$percent" -lt "$(( -threshold ))" ]; then
|
||||
status="decreased"
|
||||
else
|
||||
status="stable"
|
||||
fi
|
||||
|
||||
echo "status=$status" >> $GITHUB_OUTPUT
|
||||
echo "size_change_percent=$percent" >> $GITHUB_OUTPUT
|
||||
|
||||
# --- Chunked bundle (informational) ---
|
||||
if ls temp/current-chunks/*.js 1>/dev/null 2>&1; then
|
||||
chunks_size=$(cat temp/current-chunks/*.js | wc -c)
|
||||
echo ""
|
||||
echo "=== Chunked bundle (bootstrap + chunks) ==="
|
||||
echo "Total chunked size: $chunks_size bytes"
|
||||
echo "chunks_size=$chunks_size" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo ""
|
||||
echo "No chunked bundle found (expected on master)"
|
||||
fi
|
||||
190
.github/actions/test-driver/action.yml
vendored
190
.github/actions/test-driver/action.yml
vendored
@@ -9,48 +9,13 @@ inputs:
|
||||
build-static-viz:
|
||||
required: true
|
||||
default: false
|
||||
artifact-suffix:
|
||||
logs-artifact-suffix:
|
||||
required: false
|
||||
description: >-
|
||||
Discriminator appended to this job's artifact names (test logs, granular-rerun failed
|
||||
tests). A job whose matrix runs several concurrent legs MUST set a per-leg-unique value,
|
||||
or the legs collide on one artifact — for the rerun artifact that means a leg restores
|
||||
another leg's failed set. Must be stable across attempts of the same job.
|
||||
description: Suffix to append to the test logs artifact name to avoid conflicts in matrix jobs
|
||||
edition:
|
||||
required: false
|
||||
default: "ee"
|
||||
default: 'ee'
|
||||
description: Edition to test (oss or ee)
|
||||
runner-index:
|
||||
required: false
|
||||
description: >-
|
||||
The index of this runner among those sharing a partitioned suite. See
|
||||
`be-tests-bigquery-cloud-sdk-ee` in `.github/workflows/drivers.yml` for an example.
|
||||
runner-count:
|
||||
required: false
|
||||
description: >-
|
||||
How many runners share the suite. Recommended to set to `strategy.job-total` rather
|
||||
than a literal so that it doesn't need updating if the `runner-index` matrix changes.
|
||||
jvms-per-runner:
|
||||
required: false
|
||||
default: "1"
|
||||
description: >-
|
||||
Concurrent JVMs per runner. Recommended to raise above 1 for drivers whose tests
|
||||
spend a lot of time waiting on a remote warehouse.
|
||||
heap-per-jvm:
|
||||
required: false
|
||||
default: "4g"
|
||||
description: >-
|
||||
`-Xmx` per JVM, applied only when `jvms-per-runner` is more than 1 (otherwise the
|
||||
the `:ci` alias's `-Xmx12g` is used). To size it: reserve ~1G for the OS and ~2G
|
||||
of non-heap per JVM, then divide what's left. On a 16G runner that's ~4G each for
|
||||
two JVMs.
|
||||
granular-rerun:
|
||||
required: false
|
||||
default: "true"
|
||||
description: >-
|
||||
When "true" (default), a re-run of a failed job runs only the tests that failed on the
|
||||
previous attempt (hawk `:only`). Stress workflows set this to "false": they intentionally
|
||||
run a fixed test set `:times N`, and narrowing to failures would drop that repetition.
|
||||
|
||||
outputs:
|
||||
test-outcome:
|
||||
@@ -62,10 +27,6 @@ runs:
|
||||
steps:
|
||||
- name: Prepare back-end environment
|
||||
uses: ./.github/actions/prepare-backend
|
||||
- name: Resolve current job id
|
||||
# Resolve JOB_ID early (before the long test step) so the ci-conductor
|
||||
# reporter below can link failures to this exact job. Best-effort.
|
||||
uses: ./.github/actions/resolve-job-id
|
||||
- name: Prepare front-end environment
|
||||
if: ${{ inputs.build-static-viz == 'true' }}
|
||||
uses: ./.github/actions/prepare-frontend
|
||||
@@ -74,131 +35,17 @@ runs:
|
||||
run: bun run build-static-viz
|
||||
shell: bash
|
||||
env:
|
||||
MB_EDITION: ${{ inputs.edition }}
|
||||
|
||||
# Granular rerun: on a re-run of a failed job, restore the list
|
||||
# of tests that failed on the previous attempt (uploaded below) so the test
|
||||
# step can target only those instead of the whole suite. Best-effort: a
|
||||
# missing/empty artifact leaves no file and the run falls back to the full
|
||||
# suite. First attempts skip this entirely.
|
||||
#
|
||||
# Scoped to the IMMEDIATELY preceding attempt: an attempt that passed, or
|
||||
# that failed with nothing attributable to a var, uploads no artifact, and a
|
||||
# narrow set must not outlive it. Reading `attempt - 1` means "only what the
|
||||
# last attempt actually recorded" — anything else falls back to the full suite.
|
||||
# Workflow expressions have no arithmetic, so the shell computes it.
|
||||
#
|
||||
# run-tests.yml calls app-db.yml, backend.yml, drivers.yml and semantic-search.yml
|
||||
# as part of ONE run, so all four share an artifact namespace. `junit-name` alone is not
|
||||
# unique there — app-db.yml and drivers.yml both derive `be-tests-postgres-ee` — so
|
||||
# `artifact-suffix` carries the rest of the discriminator.
|
||||
- name: Resolve rerun artifact names
|
||||
if: ${{ inputs.granular-rerun == 'true' }}
|
||||
shell: bash
|
||||
env:
|
||||
RERUN_ARTIFACT_BASE: ${{ format('failed-tests-{0}{1}', inputs.junit-name, inputs.artifact-suffix && format('-{0}', inputs.artifact-suffix) || '') }}
|
||||
run: |
|
||||
echo "RERUN_ARTIFACT=$RERUN_ARTIFACT_BASE-attempt-$GITHUB_RUN_ATTEMPT" >> "$GITHUB_ENV"
|
||||
echo "RERUN_PREV_ARTIFACT=$RERUN_ARTIFACT_BASE-attempt-$((GITHUB_RUN_ATTEMPT - 1))" >> "$GITHUB_ENV"
|
||||
|
||||
# `pattern` (not `name`) because the artifact is legitimately absent whenever the
|
||||
# previous attempt recorded nothing: `name` treats that as an error, `pattern`
|
||||
# downloads nothing and moves on. The value is a literal — no glob metacharacters —
|
||||
# so it matches exactly one artifact, which lands at `target/failed-tests`. A pattern
|
||||
# that ever matched two would nest them under their artifact names instead, leaving
|
||||
# the file absent and falling back to the full suite.
|
||||
- name: Restore previously-failed tests
|
||||
if: ${{ inputs.granular-rerun == 'true' && github.run_attempt != '1' }}
|
||||
continue-on-error: true
|
||||
uses: actions/download-artifact@v8
|
||||
with:
|
||||
pattern: ${{ env.RERUN_PREV_ARTIFACT }}
|
||||
path: target
|
||||
|
||||
MB_EDITION: ee
|
||||
- name: Test database driver
|
||||
# On a rerun with a non-empty failed-tests file, target only those vars via
|
||||
# hawk `:only` (a var-level selector bypasses `:only-tags`, which is fine —
|
||||
# every listed var already passed that filter on the first attempt).
|
||||
# Otherwise run exactly as before. Quote the file so the whole `[ns/var ...]`
|
||||
# vector reaches clojure -X as a single argument.
|
||||
run: |
|
||||
ALIASES="dev:ci:${{ inputs.edition }}:${{ inputs.edition }}-dev:drivers:drivers-dev:test"
|
||||
BASE="clojure -X:dev:ci:${{ inputs.edition }}:${{ inputs.edition }}-dev:drivers:drivers-dev:test"
|
||||
# `:profile-tests` overrides `:test`'s exec-fn, so it has to come last. See dev.profile-tests.
|
||||
# The `profiled` output is what the collection steps below key off, so that the decision is
|
||||
# recorded by the step that acts on it rather than re-derived from the environment.
|
||||
if [ "$PROFILE_TESTS" = "true" ]; then
|
||||
echo "Profiling this run; a flamegraph will be attached to the job."
|
||||
echo "profiled=true" >> "$GITHUB_OUTPUT"
|
||||
BASE="$BASE:profile-tests"
|
||||
fi
|
||||
# An explicit selection wins over the rerun narrowing below: it was asked for by hand, on this
|
||||
# attempt. Names may be separated by commas or spaces; hawk wants them as one vector argument.
|
||||
if [ -n "$ONLY_TESTS" ]; then
|
||||
ONLY="[$(echo "$ONLY_TESTS" | tr ',' ' ')]"
|
||||
echo "Running only: $ONLY"
|
||||
$BASE :only "$ONLY"
|
||||
elif [ "${{ inputs.granular-rerun }}" = "true" ] && [ "${{ github.run_attempt }}" != "1" ] && [ -s target/failed-tests ]; then
|
||||
echo "Granular rerun of previously-failed tests: $(cat target/failed-tests)"
|
||||
clojure -X:"$ALIASES" :only "$(cat target/failed-tests)"
|
||||
elif [ -n "$TEST_RUNNER_INDEX" ]; then
|
||||
CLOJURE_ALIASES="$ALIASES" ./.github/scripts/run-backend-test-jvms.sh ${{ inputs.test-args }}
|
||||
else
|
||||
clojure -X:"$ALIASES" ${{ inputs.test-args }}
|
||||
fi
|
||||
run: clojure -X:dev:ci:${{ inputs.edition }}:${{ inputs.edition }}-dev:drivers:drivers-dev:test ${{ inputs.test-args }}
|
||||
shell: bash
|
||||
id: run-test
|
||||
env:
|
||||
TEST_RUNNER_INDEX: ${{ inputs.runner-index }}
|
||||
TEST_RUNNER_COUNT: ${{ inputs.runner-count }}
|
||||
TEST_JVMS_PER_RUNNER: ${{ inputs.jvms-per-runner }}
|
||||
TEST_HEAP_PER_JVM: ${{ inputs.heap-per-jvm }}
|
||||
# Keep this step continue-on-error because the quarantine gate below controls whether it
|
||||
# ultimately fails.
|
||||
# Trunk Quarantine controls the final state of the run in the CI!
|
||||
# Even if this step fails we let it pass, and then `trunk-io/analytics-uploader` determines
|
||||
# whether to fail or to pass the whole job, based on the quarantine list.
|
||||
continue-on-error: true
|
||||
|
||||
- name: Collect profile
|
||||
if: ${{ !cancelled() }}
|
||||
uses: ./.github/actions/collect-profile
|
||||
with:
|
||||
profiled: ${{ steps.run-test.outputs.profiled }}
|
||||
artifact-name: ${{ format('profile-{0}{1}', inputs.junit-name, inputs.artifact-suffix && format('-{0}', inputs.artifact-suffix) || '') }}
|
||||
|
||||
- name: Report failed tests to ci-conductor
|
||||
if: ${{ !cancelled() && ( steps.run-test.outcome == 'failure' || github.run_attempt != '1' ) }}
|
||||
# This step should NEVER fail the job!
|
||||
continue-on-error: true
|
||||
uses: ./.github/actions/report-failures-to-ci-conductor
|
||||
with:
|
||||
test-suite: ${{ inputs.junit-name }}
|
||||
adapter: backend
|
||||
|
||||
# Granular rerun: record which tests failed so a re-run of this
|
||||
# job targets only those (restored above). Reuses the backend JUnit adapter
|
||||
# from the ci-conductor sources the report step just prepared. Best-effort:
|
||||
# writes nothing unless every failure is attributable to a var, so an
|
||||
# unattributable failure falls back to a full rerun.
|
||||
- name: Collect failed tests for rerun
|
||||
if: ${{ inputs.granular-rerun == 'true' && !cancelled() && steps.run-test.outcome == 'failure' }}
|
||||
continue-on-error: true
|
||||
shell: bash
|
||||
# Clear the restored set first, so a collector that dies before deciding
|
||||
# can't leave the previous attempt's file to be uploaded as this one's.
|
||||
run: |
|
||||
rm -f target/failed-tests
|
||||
bun release/ci-conductor/src/collect-failed-backend.ts
|
||||
|
||||
- name: Upload failed tests for rerun
|
||||
if: ${{ inputs.granular-rerun == 'true' && !cancelled() && steps.run-test.outcome == 'failure' }}
|
||||
continue-on-error: true
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: ${{ env.RERUN_ARTIFACT }}
|
||||
path: target/failed-tests
|
||||
if-no-files-found: ignore
|
||||
|
||||
- name: Publish Test Report (JUnit)
|
||||
uses: dorny/test-reporter@a43b3a5f7366b97d083190328d2c652e1a8b6aa2 # v3.0.0
|
||||
uses: dorny/test-reporter@v1
|
||||
# As per the documentation: https://docs.github.com/en/actions/reference/workflows-and-actions/contexts#steps-context
|
||||
# test.outcome gives us access to the result before `continue-on-error` overrides it and sets it to 'success'.
|
||||
# We want test reports even if the test step gets cancelled (e.g. times out), hence we match anything that is not a 'success'.
|
||||
@@ -210,25 +57,12 @@ runs:
|
||||
list-suites: failed
|
||||
list-tests: failed
|
||||
fail-on-error: false
|
||||
|
||||
- name: Upload Logs on Failure
|
||||
if: always() && steps.run-test.outcome != 'success'
|
||||
uses: actions/upload-artifact@v7
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: ${{ format('test-logs-{0}{1}', github.job, inputs.artifact-suffix && format('-{0}', inputs.artifact-suffix) || '') }}
|
||||
# Glob: a single-JVM run writes `logs/test-log.json`; a multi-JVM run merges one
|
||||
# `logs/test-log-p<index>.json` per JVM.
|
||||
path: logs/test-log*.json
|
||||
name: ${{ format('test-logs-{0}{1}', github.job, inputs.logs-artifact-suffix && format('-{0}', inputs.logs-artifact-suffix) || '') }}
|
||||
path: logs/test-log.json
|
||||
retention-days: 1
|
||||
# Failing to upload the artifact should never affect the outcome of the whole test run.
|
||||
continue-on-error: true
|
||||
|
||||
# Last, because this is the only step allowed to fail the job: everything above it reports the
|
||||
# failure (JUnit report, logs, ci-conductor) and must run regardless of the verdict below.
|
||||
- name: Quarantine gate
|
||||
if: ${{ !cancelled() && steps.run-test.outcome == 'failure' }}
|
||||
uses: ./.github/actions/quarantine-gate
|
||||
with:
|
||||
adapter: backend
|
||||
test-suite: ${{ inputs.junit-name }}
|
||||
enforce-quarantine: "true"
|
||||
|
||||
41
.github/actions/update-cache/action.yml
vendored
Normal file
41
.github/actions/update-cache/action.yml
vendored
Normal file
@@ -0,0 +1,41 @@
|
||||
name: Update cache
|
||||
description: |
|
||||
Update a GitHub Actions cache based on a provided path and key.
|
||||
If a cache entry already exists for the given key, it is first removed.
|
||||
This ensures immutability of cache entries, and prevents runtime errors.
|
||||
|
||||
inputs:
|
||||
path:
|
||||
description: "The path(s) to cache, separated by newlines."
|
||||
required: true
|
||||
key:
|
||||
description: "The cache key to use."
|
||||
required: true
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Check if the cache entry exists
|
||||
id: cache-lookup
|
||||
uses: actions/cache/restore@v4
|
||||
with:
|
||||
path: ${{ inputs.path }}
|
||||
key: ${{ inputs.key }}
|
||||
lookup-only: true # Important bit: the action will not fetch anything, just check for the cache existence.
|
||||
- name: Delete the cache entry if it exists
|
||||
if: ${{ steps.cache-lookup.outputs.cache-hit == 'true' }}
|
||||
run: |
|
||||
echo "Cache entry found for key '${{ inputs.key }}'. Deleting it to ensure immutability."
|
||||
gh api \
|
||||
--method DELETE \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "X-GitHub-Api-Version: 2022-11-28" \
|
||||
/repos/${{ github.repository }}/actions/caches?key=${{ inputs.key }}
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
shell: bash
|
||||
- name: Save the new cache entry
|
||||
uses: actions/cache/save@v4
|
||||
with:
|
||||
path: ${{ inputs.path }}
|
||||
key: ${{ inputs.key }}
|
||||
@@ -1,5 +1,4 @@
|
||||
name: Create Backend Dep Bump PRs
|
||||
description: Opens a pull request for each backend dependency that has a newer version available.
|
||||
|
||||
inputs:
|
||||
github-token:
|
||||
|
||||
64
.github/actions/upload-test-results/action.yml
vendored
Normal file
64
.github/actions/upload-test-results/action.yml
vendored
Normal file
@@ -0,0 +1,64 @@
|
||||
name: Upload Test Results to s3
|
||||
description: Uploads test results files to s3
|
||||
|
||||
inputs:
|
||||
input-path:
|
||||
required: true
|
||||
output-name:
|
||||
required: true
|
||||
default: test
|
||||
role-to-assume:
|
||||
required: true
|
||||
aws-region:
|
||||
required: true
|
||||
default: us-east-1
|
||||
bucket:
|
||||
required: true
|
||||
trunk-api-token:
|
||||
required: true
|
||||
variant:
|
||||
required: false
|
||||
default: ""
|
||||
previous-step-outcome:
|
||||
required: true
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Configure AWS credentials
|
||||
uses: aws-actions/configure-aws-credentials@v4
|
||||
with:
|
||||
role-to-assume: ${{ inputs.role-to-assume }}
|
||||
aws-region: ${{ inputs.aws-region }}
|
||||
- name: zip test results
|
||||
env:
|
||||
INPUT_DIR: ${{ inputs.input-path }}
|
||||
OUTPUT_FILE: ${{ inputs.output-name }}
|
||||
shell: bash
|
||||
run: | # sh
|
||||
zip ${OUTPUT_FILE}.zip $INPUT_DIR/*.xml
|
||||
- name: Upload test results to S3
|
||||
env:
|
||||
BUCKET: ${{ inputs.bucket }}
|
||||
OUTPUT_FILE: ${{ inputs.output-name }}
|
||||
shell: bash
|
||||
run: | # sh
|
||||
DATE=$(date '+%Y-%m-%d')
|
||||
aws s3 cp ${OUTPUT_FILE}.zip s3://$BUCKET/$DATE/$GITHUB_RUN_ID/$GITHUB_RUN_ATTEMPT/
|
||||
|
||||
- name: Upload results to Trunk
|
||||
# Important!
|
||||
# 1. This step must always run because it controls job's exit code.
|
||||
# 2. There's no point in running this outside the organization
|
||||
if: always() && github.repository_owner == 'metabase'
|
||||
uses: trunk-io/analytics-uploader@main
|
||||
env:
|
||||
TRUNK_REPO_URL: git@github.com:metabase/metabase.git
|
||||
with:
|
||||
junit-paths: ${{ inputs.input-path }}
|
||||
variant: ${{ inputs.variant || inputs.output-name }}
|
||||
org-slug: metabase
|
||||
dry-run: ${{ (github.event_name == 'pull_request' && github.event.pull_request.draft == true) || github.repository != 'metabase/metabase' }} # quarantine, but do not upload results
|
||||
token: ${{ inputs.trunk-api-token }}
|
||||
use-cache: true
|
||||
previous-step-outcome: ${{ inputs.previous-step-outcome }}
|
||||
22
.github/actions/verify-membership/action.yml
vendored
Normal file
22
.github/actions/verify-membership/action.yml
vendored
Normal file
@@ -0,0 +1,22 @@
|
||||
name: 'Verify Organization Membership'
|
||||
description: 'Check if a user is a member of a GitHub organization'
|
||||
inputs:
|
||||
github_token:
|
||||
description: 'Token with access to the organization'
|
||||
required: true
|
||||
organization_name:
|
||||
description: 'Organization name'
|
||||
required: true
|
||||
username:
|
||||
description: 'Username'
|
||||
required: true
|
||||
outputs:
|
||||
is_member:
|
||||
value: ${{ steps.verify-membership.outputs.is_member }}
|
||||
description: "Is the user a member of the organization"
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- run: bash ${{ github.action_path }}/verify_membership.sh ${{ inputs.github_token }} ${{ inputs.organization_name }} ${{ inputs.username }}
|
||||
id: verify-membership
|
||||
shell: bash
|
||||
25
.github/actions/verify-membership/verify_membership.sh
vendored
Executable file
25
.github/actions/verify-membership/verify_membership.sh
vendored
Executable file
@@ -0,0 +1,25 @@
|
||||
#!/bin/bash
|
||||
|
||||
TOKEN=$1
|
||||
ORG_NAME=$2
|
||||
USERNAME=$3
|
||||
|
||||
response=$(curl -L -o /dev/null -w "%{http_code}" \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "X-GitHub-Api-Version: 2022-11-28" \
|
||||
"https://api.github.com/orgs/$ORG_NAME/members/$USERNAME")
|
||||
|
||||
# Github returns an empty response with 204 status if user is a member and 404 status otherwise
|
||||
# See: https://docs.github.com/en/rest/orgs/members?apiVersion=2022-11-28#check-organization-membership-for-a-user
|
||||
if [ "$response" == "204" ]; then
|
||||
echo "The user $USERNAME is a member of $ORG_NAME"
|
||||
echo "is_member=true" >> $GITHUB_OUTPUT
|
||||
elif [ "$response" == "404" ]; then
|
||||
echo "The user $USERNAME is not a member of $ORG_NAME"
|
||||
echo "is_member=false" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "Failed to determine the membership status for $USERNAME in $ORG_NAME"
|
||||
echo "is_member=false" >> $GITHUB_OUTPUT
|
||||
exit 1
|
||||
fi
|
||||
180
.github/file-paths.yaml
vendored
180
.github/file-paths.yaml
vendored
@@ -1,15 +1,7 @@
|
||||
# Catch-all filter — emits the full changed-files list as `all_changed_files`
|
||||
# so create-test-plan in run-tests.yml can reuse dorny's diff.
|
||||
all_changed:
|
||||
- "**"
|
||||
|
||||
default: &default
|
||||
- "locales/**"
|
||||
- "bin/**/!(*.md)"
|
||||
|
||||
workflows:
|
||||
- added|modified: ".github/workflows/*.{yml,yaml}"
|
||||
|
||||
shared_sources: &shared_sources
|
||||
- "src/**/*.cljc"
|
||||
- "src/**/*.cljs"
|
||||
@@ -53,6 +45,11 @@ frontend_sources: &frontend_sources
|
||||
- "enterprise/frontend/!(test)**/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
|
||||
- "frontend/!(test)**/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
|
||||
|
||||
# Used only in .github/workflows/conventional-commit-pr-title-reminder.yml
|
||||
frontend_embedding_sources:
|
||||
- "enterprise/frontend/src/{embedding,embedding-*}/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
|
||||
- "enterprise/frontend/src/metabase-enterprise/{embedding,embedding-*}**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
|
||||
|
||||
# Used only in .github/workflows/embedding-sdk-labels-reminder.yml
|
||||
frontend_embedding_sdk_package_sources:
|
||||
- "enterprise/frontend/src/embedding-sdk-package/**/!(*.spec).{js,jsx,ts,tsx,css}"
|
||||
@@ -63,10 +60,9 @@ frontend_specs: &frontend_specs
|
||||
- "frontend/**/tests/**"
|
||||
- "frontend/**/*.unit.*"
|
||||
- "enterprise/frontend/**/*.unit.*"
|
||||
- ".github/**/*.unit.spec.*"
|
||||
- "jest.config.js"
|
||||
- "jest.base.conf.js"
|
||||
- "jest.esm-packages.js"
|
||||
- "jest.tz.unit.conf.js"
|
||||
- "jest.tz.unit.conf.json"
|
||||
|
||||
release_source: &release_sources
|
||||
- "release/**"
|
||||
@@ -78,73 +74,22 @@ frontend_all: &frontend_all
|
||||
|
||||
frontend_loki_ci: &frontend_loki_ci
|
||||
- ".github/workflows/loki.yml"
|
||||
- "loki.config.js"
|
||||
- ".loki/**"
|
||||
- ".storybook/**"
|
||||
|
||||
frontend_unit_infra: &frontend_unit_infra
|
||||
- *frontend_configs
|
||||
- *frontend_ci
|
||||
- "jest.config.js"
|
||||
- "jest.base.conf.js"
|
||||
- "jest.esm-packages.js"
|
||||
- "jest.tz.unit.conf.js"
|
||||
- "frontend/test/**"
|
||||
- "frontend/**/__mocks__/**"
|
||||
- "frontend/src/embedding-sdk-shared/jest/**"
|
||||
|
||||
frontend_loki_infra: &frontend_loki_infra
|
||||
- *frontend_configs
|
||||
- *shared_ci
|
||||
- *frontend_loki_ci
|
||||
- "frontend/build/shared/**"
|
||||
- "frontend/test/__support__/**"
|
||||
- "patches/**"
|
||||
# dependency-cruiser doesn't follow CSS @import, so global styles must run every story.
|
||||
- "frontend/src/metabase/css/**/*.css"
|
||||
# Storybook serves resources/frontend_client as static files, and the stories bundle the app's SVGs.
|
||||
- "resources/frontend_client/**"
|
||||
- "frontend/**/*.svg"
|
||||
- "enterprise/frontend/**/*.svg"
|
||||
|
||||
frontend_loki_all:
|
||||
- *frontend_sources
|
||||
- *frontend_loki_infra
|
||||
|
||||
# Everything the ci-scripts jest project runs or depends on.
|
||||
# test-unit ignores that project, so this filter is the only thing that reruns it.
|
||||
ci_scripts:
|
||||
- *frontend_ci
|
||||
- ".nvmrc"
|
||||
# file-paths.unit.spec.ts asserts what this file's own patterns match, so a change to it has to
|
||||
# run the specs that guard it.
|
||||
- ".github/file-paths.yaml"
|
||||
- ".github/scripts/**"
|
||||
- ".github/actions/**"
|
||||
- ".github/workflows/loki.yml"
|
||||
- ".storybook/story-files.cjs"
|
||||
- "frontend/lint/module-boundaries.mjs"
|
||||
- "frontend/lint/shared-tiers.mjs"
|
||||
- "frontend/test/jest-test-paths-filter.ts"
|
||||
- ".dependency-cruiser.cjs"
|
||||
- "jest.config.js"
|
||||
- "jest.base.conf.js"
|
||||
- "jest.esm-packages.js"
|
||||
- "package.json"
|
||||
- "bun.lock"
|
||||
|
||||
# e2e harness/support code (commands, support file, runner). Not in the
|
||||
# instrumented bundle, so it maps to no covered module — a change here forces a
|
||||
# full e2e run (see create-test-plan / e2eInfraTouched).
|
||||
e2e_infra: &e2e_infra
|
||||
- *shared_e2e
|
||||
|
||||
backend_ci: &backend_ci
|
||||
- *shared_ci
|
||||
- ".github/actions/test-driver/**"
|
||||
- ".github/workflows/backend.yml"
|
||||
- ".github/workflows/drivers.yml"
|
||||
|
||||
backend_presto_kerberos:
|
||||
- "**/presto_jdbc/**"
|
||||
- "**/presto-jdbc/**" # the driver module has a hyphen. underscores are classpaths
|
||||
- "**/presto_jdbc.clj"
|
||||
- "**/presto_jdbc_test.clj"
|
||||
- ".github/workflows/presto-kerberos-integration-test.yml"
|
||||
|
||||
backend_sources: &backend_sources
|
||||
- *shared_sources
|
||||
- "**deps.edn"
|
||||
@@ -152,15 +97,11 @@ backend_sources: &backend_sources
|
||||
- "src/**"
|
||||
- "modules/drivers/{*,*/{*,!(test)/**}}"
|
||||
- "resources/**"
|
||||
# Ratchet files drive project_ratchet_checks alone, so automation PRs that lower budgets
|
||||
# run the Project tests job by themselves. Patterns are OR-ed, so a separate "!..." entry would match
|
||||
# every other path instead of excluding this one.
|
||||
- ".clj-kondo/**/!(ratchets.edn)"
|
||||
- ".clj-kondo/**"
|
||||
|
||||
backend_specs: &backend_specs
|
||||
- *shared_specs
|
||||
- "test/**"
|
||||
- "dev/test/**"
|
||||
- "enterprise/backend/test/**"
|
||||
- "test_modules/**"
|
||||
- "modules/drivers/*/test/**"
|
||||
@@ -173,48 +114,6 @@ backend_all: &backend_all
|
||||
- *backend_sources
|
||||
- *backend_specs
|
||||
|
||||
# Project checks have narrower triggers than the backend matrix.
|
||||
# Changes to their shared CI entry points exercise every suite; otherwise each
|
||||
# suite follows only the files that can affect its result.
|
||||
project_tests_infra: &project_tests_infra
|
||||
- ".github/actions/prepare-backend/**"
|
||||
- ".github/file-paths.yaml"
|
||||
- ".github/workflows/backend.yml"
|
||||
- ".github/workflows/run-tests.yml"
|
||||
- "bb.edn"
|
||||
- "bin/mage"
|
||||
- "bin/mage.bb"
|
||||
- "mage/src/mage/color.clj"
|
||||
- "mage/src/mage/project_tests.clj"
|
||||
- "mage/src/mage/shell.clj"
|
||||
- "mage/src/mage/util.clj"
|
||||
|
||||
project_backend_checks:
|
||||
- *project_tests_infra
|
||||
- "{src,test,enterprise,dev,bin,mage}/**/*.{clj,cljc,cljs}"
|
||||
- "modules/**/*.clj"
|
||||
- "modules/**/*.cljc"
|
||||
- "modules/**/*.cljs"
|
||||
- "**deps.edn"
|
||||
- ".clj-kondo/{src,test}/**/*.{clj,cljc,cljs}"
|
||||
- "{resources,test_resources,test_config}/**/*.{clj,cljc,cljs}"
|
||||
- ".clj-kondo/config.edn"
|
||||
- ".clj-kondo/config/modules/config.edn"
|
||||
- ".github/team.json"
|
||||
- "dev/resources/dev/modules_config_test/**"
|
||||
|
||||
# Only the ratchet check reads the budgets, so a diff that touches nothing else runs that suite alone.
|
||||
project_ratchet_checks:
|
||||
- ".clj-kondo/ratchets.edn"
|
||||
- ".clj-kondo/config/modules/ratchets.edn"
|
||||
|
||||
project_migration_checks:
|
||||
- *project_tests_infra
|
||||
- "bin/lint-migrations-file/**"
|
||||
- "resources/migrations/**/*.{yaml,yml}"
|
||||
- "src/metabase/app_db/custom_migrations.clj"
|
||||
- "src/metabase/app_db/custom_migrations/**"
|
||||
|
||||
sources: &sources
|
||||
- *shared_sources
|
||||
- *frontend_sources
|
||||
@@ -228,7 +127,7 @@ custom_viz_all:
|
||||
|
||||
e2e_ci: &e2e_ci
|
||||
- *shared_ci
|
||||
- ".github/workflows/uberjar.yml"
|
||||
- ".github/actions/prepare-uberjar-artifact/**"
|
||||
- ".github/actions/fetch-artifact/action.yml"
|
||||
- ".github/actions/e2e-prepare-containers/**"
|
||||
- ".github/actions/prepare-cypress/**"
|
||||
@@ -239,7 +138,7 @@ e2e_specs: &e2e_specs
|
||||
|
||||
embedding_sdk_ci: &embedding_sdk_ci
|
||||
- *shared_ci
|
||||
- ".github/workflows/uberjar.yml"
|
||||
- ".github/actions/prepare-uberjar-artifact/**"
|
||||
- ".github/actions/fetch-artifact/action.yml"
|
||||
- ".github/actions/prepare-cypress/**"
|
||||
- ".github/workflows/embedding-sdk*.yml"
|
||||
@@ -261,7 +160,7 @@ embedding_sdk_host_sample_apps:
|
||||
- "e2e/test-host-app/**/*.cy.*.{js,ts}"
|
||||
|
||||
snowplow: &snowplow
|
||||
- ".github/workflows/lint.yml"
|
||||
- ".github/workflows/snowplow.yml"
|
||||
- "snowplow/**"
|
||||
- "bin/lint-snowplow-schemas.bb"
|
||||
- "bin/lint-snowplow-schemas-test.bb"
|
||||
@@ -278,9 +177,6 @@ documentation:
|
||||
- "docs/**"
|
||||
- "**/*.md"
|
||||
|
||||
docs:
|
||||
- "docs/**"
|
||||
|
||||
embedding_documentation:
|
||||
- ".typedoc/**"
|
||||
- "docs/embedding/**"
|
||||
@@ -290,7 +186,7 @@ yaml:
|
||||
- "**/*.yaml"
|
||||
|
||||
codeql:
|
||||
- ".github/workflows/lint.yml"
|
||||
- ".github/workflows/codeql.yml"
|
||||
- "frontend/src/**"
|
||||
- "enterprise/frontend/src/**"
|
||||
|
||||
@@ -298,50 +194,26 @@ i18n:
|
||||
- *default
|
||||
- ".github/actions/prepare-frontend/**"
|
||||
- ".github/actions/prepare-backend/**"
|
||||
- ".github/workflows/lint.yml"
|
||||
- ".github/workflows/i18n.yml"
|
||||
- *sources
|
||||
|
||||
visualizations:
|
||||
- "frontend/src/metabase/visualizations/**"
|
||||
- "frontend/src/metabase/static-viz/**"
|
||||
- "src/metabase/pulse/**"
|
||||
|
||||
cljs:
|
||||
- *shared_sources
|
||||
- *shared_specs
|
||||
- ".github/workflows/cljs.yml"
|
||||
- "shadow-cljs.edn"
|
||||
|
||||
build_scripts:
|
||||
- ".github/actions/prepare-backend/**"
|
||||
- ".github/file-paths.yaml"
|
||||
- ".github/workflows/build-scripts.yml"
|
||||
- "bin/**"
|
||||
- "deps.edn"
|
||||
|
||||
embedder_plugin:
|
||||
- ".github/actions/prepare-backend/**"
|
||||
- ".github/file-paths.yaml"
|
||||
- ".github/workflows/build-scripts.yml"
|
||||
- "bin/build-embedder-plugin.sh"
|
||||
- "bin/build/src/build/embedder_model.clj"
|
||||
- "bin/build/src/build/plugin_uberjar.clj"
|
||||
- "bin/build/src/build_embedder_plugin.clj"
|
||||
- "deps.edn"
|
||||
- "modules/embedder/**"
|
||||
- "enterprise/backend/src/metabase_enterprise/data_complexity_score/complexity_embedders.clj"
|
||||
- "enterprise/backend/src/metabase_enterprise/data_complexity_score/settings.clj"
|
||||
- "enterprise/backend/src/metabase_enterprise/data_complexity_score/synonym_source.clj"
|
||||
- "enterprise/backend/src/metabase_enterprise/semantic_search/embedding.clj"
|
||||
- "enterprise/backend/src/metabase_enterprise/semantic_search/settings.clj"
|
||||
- "enterprise/backend/test/metabase_enterprise/semantic_search/appdb_pgvector_mode_test.clj"
|
||||
- "enterprise/backend/test/metabase_enterprise/semantic_search/embedding_test.clj"
|
||||
- "src/metabase/embeddings/provider.clj"
|
||||
- "src/metabase/embeddings/startup.clj"
|
||||
- "src/metabase/plugins/**"
|
||||
- "test/metabase/embeddings/embedder_plugin_artifact_test.clj"
|
||||
|
||||
serialization_baseline:
|
||||
- ".github/workflows/serialization.yml"
|
||||
- "test_resources/serialization_baseline/**"
|
||||
|
||||
openapi:
|
||||
- ".github/workflows/lint.yml"
|
||||
- ".github/workflows/openapi.yml"
|
||||
- "src/metabase/api/**"
|
||||
- "src/metabase/api_routes/**"
|
||||
- "src/metabase/*_rest/api.clj"
|
||||
|
||||
12
.github/instructions/e2e.instructions.md
vendored
12
.github/instructions/e2e.instructions.md
vendored
@@ -1,12 +0,0 @@
|
||||
---
|
||||
applyTo: "e2e/**/*.cy.spec.{js,ts}"
|
||||
---
|
||||
|
||||
# Cypress E2E Spec Review
|
||||
|
||||
When reviewing Cypress E2E specs, apply the guidelines in these files:
|
||||
|
||||
- `.claude/skills/_shared/cypress-conventions.md`
|
||||
- `.claude/skills/e2e-test-review/SKILL.md`
|
||||
|
||||
These are in addition to the TypeScript/JavaScript instructions, which also match these files.
|
||||
264
.github/scripts/affected-modules.ts
vendored
264
.github/scripts/affected-modules.ts
vendored
@@ -1,264 +0,0 @@
|
||||
import micromatch from "micromatch";
|
||||
|
||||
export type ModuleDef = { type: string; pattern: string };
|
||||
|
||||
export type Rule = {
|
||||
from: string[];
|
||||
allow?: string[];
|
||||
disallow?: string[];
|
||||
};
|
||||
|
||||
export type ModuleNode = { type: string; regex: RegExp };
|
||||
|
||||
export type FileDependency = { source: string; dependencies: string[] };
|
||||
|
||||
/** First match wins, so element order matters for nested patterns. */
|
||||
export function buildNodes(elements: ModuleDef[]): ModuleNode[] {
|
||||
return elements.map((el) => ({
|
||||
type: el.type,
|
||||
// dot:true so files in dot-directories (e.g. .storybook) still match.
|
||||
regex: micromatch.makeRe(el.pattern, { dot: true }),
|
||||
}));
|
||||
}
|
||||
|
||||
export function mapFileToModule(
|
||||
nodes: ModuleNode[],
|
||||
path: string,
|
||||
): string | null {
|
||||
for (const node of nodes) {
|
||||
if (node.regex.test(path)) {
|
||||
return node.type;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export function getChangedModules(
|
||||
nodes: ModuleNode[],
|
||||
changedFiles: string[],
|
||||
): Set<string> {
|
||||
const direct = new Set<string>();
|
||||
for (const file of changedFiles) {
|
||||
const module = mapFileToModule(nodes, file);
|
||||
if (module) {
|
||||
direct.add(module);
|
||||
}
|
||||
}
|
||||
return direct;
|
||||
}
|
||||
|
||||
export type FileGraph = {
|
||||
nodes: ModuleNode[];
|
||||
fileDependents: Map<string, Set<string>>;
|
||||
};
|
||||
|
||||
/** Drops unresolvable imports (e.g. ambient types) that point at no real file. */
|
||||
export function parseCruiseModules(
|
||||
modules: Array<{
|
||||
source: string;
|
||||
dependencies?: Array<{ resolved: string; couldNotResolve?: boolean }>;
|
||||
}>,
|
||||
): FileDependency[] {
|
||||
return modules.map((module) => ({
|
||||
source: module.source,
|
||||
dependencies: (module.dependencies ?? [])
|
||||
.filter((dep) => !dep.couldNotResolve)
|
||||
.map((dep) => dep.resolved),
|
||||
}));
|
||||
}
|
||||
|
||||
/** Inverts the import edges into the `fileDependents` reverse graph. */
|
||||
export function buildFileGraph(
|
||||
elements: ModuleDef[],
|
||||
fileDependencies: FileDependency[],
|
||||
): FileGraph {
|
||||
const fileDependents = new Map<string, Set<string>>();
|
||||
for (const { source, dependencies } of fileDependencies) {
|
||||
for (const target of dependencies) {
|
||||
let importers = fileDependents.get(target);
|
||||
if (!importers) {
|
||||
importers = new Set();
|
||||
fileDependents.set(target, importers);
|
||||
}
|
||||
importers.add(source);
|
||||
}
|
||||
}
|
||||
return { nodes: buildNodes(elements), fileDependents };
|
||||
}
|
||||
|
||||
/** Changed files plus every file that transitively imports them. */
|
||||
export function getAffectedFiles(
|
||||
fileGraph: FileGraph,
|
||||
changedFiles: string[],
|
||||
): Set<string> {
|
||||
const affected = new Set(changedFiles);
|
||||
const queue = [...changedFiles];
|
||||
while (queue.length > 0) {
|
||||
const file = queue.shift()!;
|
||||
for (const importer of fileGraph.fileDependents.get(file) ?? []) {
|
||||
if (!affected.has(importer)) {
|
||||
affected.add(importer);
|
||||
queue.push(importer);
|
||||
}
|
||||
}
|
||||
}
|
||||
return affected;
|
||||
}
|
||||
|
||||
/** The file plus every file it transitively imports. */
|
||||
export function getTransitiveDependencies(
|
||||
fileDependencies: FileDependency[],
|
||||
file: string,
|
||||
): Set<string> {
|
||||
const dependenciesOf = new Map(
|
||||
fileDependencies.map(({ source, dependencies }) => [source, dependencies]),
|
||||
);
|
||||
const reached = new Set([file]);
|
||||
const queue = [file];
|
||||
while (queue.length > 0) {
|
||||
const current = queue.shift()!;
|
||||
for (const dependency of dependenciesOf.get(current) ?? []) {
|
||||
if (!reached.has(dependency)) {
|
||||
reached.add(dependency);
|
||||
queue.push(dependency);
|
||||
}
|
||||
}
|
||||
}
|
||||
return reached;
|
||||
}
|
||||
|
||||
export type ModuleGraph = {
|
||||
nodes: ModuleNode[];
|
||||
/**
|
||||
* dependents.get(M) is every module affected when M changes — the full
|
||||
* transitive closure, so getAffectedModules only needs a single hop.
|
||||
*/
|
||||
dependents: Map<string, Set<string>>;
|
||||
};
|
||||
|
||||
/** Expands a direct-dependents map into its transitive closure. */
|
||||
function transitiveClosure(
|
||||
direct: Map<string, Set<string>>,
|
||||
): Map<string, Set<string>> {
|
||||
const closure = new Map<string, Set<string>>();
|
||||
for (const start of direct.keys()) {
|
||||
const reached = new Set<string>();
|
||||
const queue = [...(direct.get(start) ?? [])];
|
||||
while (queue.length > 0) {
|
||||
const node = queue.shift()!;
|
||||
if (reached.has(node)) {
|
||||
continue;
|
||||
}
|
||||
reached.add(node);
|
||||
for (const next of direct.get(node) ?? []) {
|
||||
queue.push(next);
|
||||
}
|
||||
}
|
||||
reached.delete(start);
|
||||
closure.set(start, reached);
|
||||
}
|
||||
return closure;
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds a module graph from the actual import graph. Reachability is computed
|
||||
* per module at the file level and then collapsed to modules.
|
||||
*/
|
||||
export function buildUsageModuleGraph(
|
||||
elements: ModuleDef[],
|
||||
fileDependencies: FileDependency[],
|
||||
): ModuleGraph {
|
||||
const nodes = buildNodes(elements);
|
||||
const fileGraph = buildFileGraph(elements, fileDependencies);
|
||||
|
||||
const filesByModule = new Map<string, string[]>();
|
||||
const allFiles = new Set<string>();
|
||||
for (const { source, dependencies } of fileDependencies) {
|
||||
allFiles.add(source);
|
||||
for (const target of dependencies) {
|
||||
allFiles.add(target);
|
||||
}
|
||||
}
|
||||
for (const file of allFiles) {
|
||||
const module = mapFileToModule(nodes, file);
|
||||
if (module) {
|
||||
const files = filesByModule.get(module);
|
||||
if (files) {
|
||||
files.push(file);
|
||||
} else {
|
||||
filesByModule.set(module, [file]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const dependents = new Map<string, Set<string>>(
|
||||
elements.map((el) => [el.type, new Set()]),
|
||||
);
|
||||
for (const [module, files] of filesByModule) {
|
||||
const affected = new Set<string>();
|
||||
for (const file of getAffectedFiles(fileGraph, files)) {
|
||||
const owner = mapFileToModule(nodes, file);
|
||||
if (owner && owner !== module) {
|
||||
affected.add(owner);
|
||||
}
|
||||
}
|
||||
dependents.set(module, affected);
|
||||
}
|
||||
|
||||
return { nodes, dependents };
|
||||
}
|
||||
|
||||
export function buildModuleGraph(
|
||||
elements: ModuleDef[],
|
||||
rules: Rule[],
|
||||
): ModuleGraph {
|
||||
const nodes = buildNodes(elements);
|
||||
const allTypes = elements.map((e) => e.type);
|
||||
|
||||
function expandPattern(pattern: string): string[] {
|
||||
if (pattern === "*") {
|
||||
return [...allTypes];
|
||||
}
|
||||
if (pattern.endsWith("/*")) {
|
||||
const prefix = pattern.slice(0, -1);
|
||||
return allTypes.filter((type) => type.startsWith(prefix));
|
||||
}
|
||||
return allTypes.includes(pattern) ? [pattern] : [];
|
||||
}
|
||||
|
||||
const directDependents = new Map<string, Set<string>>(
|
||||
allTypes.map((type) => [type, new Set()]),
|
||||
);
|
||||
for (const rule of rules) {
|
||||
// Grandfathered imports that break the disallow rules still exist in the code,
|
||||
// and selection must cover every import that actually exists.
|
||||
// So disallow rules are ignored, at the cost of a looser graph.
|
||||
const fromTypes = rule.from.flatMap(expandPattern);
|
||||
const allowTypes = (rule.allow ?? []).flatMap(expandPattern);
|
||||
for (const target of allowTypes) {
|
||||
for (const importer of fromTypes) {
|
||||
if (importer !== target) {
|
||||
directDependents.get(target)?.add(importer);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return { nodes, dependents: transitiveClosure(directDependents) };
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the changed modules and all modules that directly or indirectly depend on them
|
||||
*/
|
||||
export function getAffectedModules(
|
||||
moduleGraph: ModuleGraph,
|
||||
changedFiles: string[],
|
||||
): Set<string> {
|
||||
const affected = getChangedModules(moduleGraph.nodes, changedFiles);
|
||||
for (const module of [...affected]) {
|
||||
for (const dep of moduleGraph.dependents.get(module) ?? []) {
|
||||
affected.add(dep);
|
||||
}
|
||||
}
|
||||
return affected;
|
||||
}
|
||||
203
.github/scripts/affected-modules.unit.spec.ts
vendored
203
.github/scripts/affected-modules.unit.spec.ts
vendored
@@ -1,203 +0,0 @@
|
||||
import {
|
||||
buildFileGraph,
|
||||
buildModuleGraph,
|
||||
buildNodes,
|
||||
buildUsageModuleGraph,
|
||||
getAffectedFiles,
|
||||
getAffectedModules,
|
||||
getChangedModules,
|
||||
getTransitiveDependencies,
|
||||
mapFileToModule,
|
||||
parseCruiseModules,
|
||||
} from "./affected-modules";
|
||||
|
||||
const ELEMENTS = [
|
||||
{ type: "lib/utils", pattern: "src/utils/**" },
|
||||
{ type: "lib/types", pattern: "src/types/**" },
|
||||
// Nested-pattern pair (mirrors mlv1-inside-mlv2): inner must come first
|
||||
// so first-match-wins picks it up before the outer pattern.
|
||||
{ type: "lib/inner", pattern: "src/shared/v1/**" },
|
||||
{ type: "lib/outer", pattern: "src/shared/**" },
|
||||
{ type: "feature/foo", pattern: "src/foo/**" },
|
||||
{ type: "feature/bar", pattern: "src/bar/**" },
|
||||
{ type: "feature/super", pattern: "src/super/**" },
|
||||
// Different folder root (mirrors feature/enterprise's enterprise/ path).
|
||||
{ type: "feature/external", pattern: "external/lib/**" },
|
||||
// Same type, two patterns (mirrors app/misc per-entry-point declarations).
|
||||
{ type: "app/main", pattern: "src/app.js" },
|
||||
{ type: "app/main", pattern: "src/embed.tsx" },
|
||||
{ type: "shared/other", pattern: "src/*/**" }, // catch-all, must be last
|
||||
];
|
||||
|
||||
const RULES = [
|
||||
...ELEMENTS.map((el) => ({ from: [el.type], allow: [el.type] })),
|
||||
{ from: ["lib/*"], allow: ["lib/*"] },
|
||||
{ from: ["feature/*"], allow: ["lib/*"] },
|
||||
{ from: ["feature/super"], allow: ["feature/*"] },
|
||||
{ from: ["app/*"], allow: ["lib/*", "feature/*", "app/*"] },
|
||||
];
|
||||
|
||||
const nodes = buildNodes(ELEMENTS);
|
||||
const graph = buildModuleGraph(ELEMENTS, RULES);
|
||||
|
||||
describe("affected modules", () => {
|
||||
describe("mapFileToModule", () => {
|
||||
it.each([
|
||||
["src/utils/colors.ts", "lib/utils"],
|
||||
["src/types/api.ts", "lib/types"],
|
||||
["src/foo/foo.tsx", "feature/foo"],
|
||||
["src/bar/bar.tsx", "feature/bar"],
|
||||
["src/super/index.ts", "feature/super"],
|
||||
["src/app.js", "app/main"],
|
||||
["src/randomthing/file.ts", "shared/other"],
|
||||
["src/shared/v1/expr.ts", "lib/inner"],
|
||||
["src/shared/v2/expr.ts", "lib/outer"],
|
||||
["src/shared/index.ts", "lib/outer"],
|
||||
["external/lib/foo.ts", "feature/external"],
|
||||
["src/embed.tsx", "app/main"],
|
||||
])("should map %s to %s", (path, expected) => {
|
||||
expect(mapFileToModule(nodes, path)).toBe(expected);
|
||||
});
|
||||
|
||||
it("should return null for files outside any pattern", () => {
|
||||
expect(mapFileToModule(nodes, "docs/foo.md")).toBe(null);
|
||||
expect(mapFileToModule(nodes, "README.md")).toBe(null);
|
||||
expect(mapFileToModule(nodes, "external/notlib/foo.ts")).toBe(null);
|
||||
});
|
||||
});
|
||||
|
||||
describe("getChangedModules", () => {
|
||||
it("should return the unique set of modules whose files changed", () => {
|
||||
const result = getChangedModules(nodes, [
|
||||
"src/utils/colors.ts",
|
||||
"src/foo/x.ts",
|
||||
"src/foo/y.ts",
|
||||
"docs/unrelated.md",
|
||||
]);
|
||||
expect([...result].sort()).toEqual(["feature/foo", "lib/utils"]);
|
||||
});
|
||||
|
||||
it("should return an empty set when no file maps to a module", () => {
|
||||
expect(getChangedModules(nodes, ["docs/foo.md"]).size).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe("parseCruiseModules", () => {
|
||||
it("should flatten to source + resolved-paths and drop unresolved deps", () => {
|
||||
expect(
|
||||
parseCruiseModules([
|
||||
{
|
||||
source: "src/foo/foo.tsx",
|
||||
dependencies: [
|
||||
{ resolved: "src/utils/colors.ts" },
|
||||
{ resolved: "ambient/types", couldNotResolve: true },
|
||||
],
|
||||
},
|
||||
{ source: "src/bar/bar.tsx" }, // no dependencies key
|
||||
]),
|
||||
).toEqual([
|
||||
{ source: "src/foo/foo.tsx", dependencies: ["src/utils/colors.ts"] },
|
||||
{ source: "src/bar/bar.tsx", dependencies: [] },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
const FILE_DEPS = [
|
||||
{ source: "src/app.js", dependencies: ["src/foo/foo.tsx"] },
|
||||
{ source: "src/bar/bar.tsx", dependencies: ["src/embed.tsx"] },
|
||||
{ source: "src/foo/foo.tsx", dependencies: ["src/utils/colors.ts"] },
|
||||
];
|
||||
const fileGraph = buildFileGraph(ELEMENTS, FILE_DEPS);
|
||||
|
||||
describe("getAffectedFiles", () => {
|
||||
it("should return the changed file and everything that imports it", () => {
|
||||
expect(
|
||||
[...getAffectedFiles(fileGraph, ["src/utils/colors.ts"])].sort(),
|
||||
).toEqual(["src/app.js", "src/foo/foo.tsx", "src/utils/colors.ts"]);
|
||||
});
|
||||
|
||||
it("should include a changed file with no importers (itself only)", () => {
|
||||
// Nothing imports the app.js entry point.
|
||||
expect([...getAffectedFiles(fileGraph, ["src/app.js"])]).toEqual([
|
||||
"src/app.js",
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("getTransitiveDependencies", () => {
|
||||
it("should return the file and everything it transitively imports", () => {
|
||||
expect(
|
||||
[...getTransitiveDependencies(FILE_DEPS, "src/app.js")].sort(),
|
||||
).toEqual(["src/app.js", "src/foo/foo.tsx", "src/utils/colors.ts"]);
|
||||
});
|
||||
|
||||
it("should return only the file when it imports nothing", () => {
|
||||
expect([
|
||||
...getTransitiveDependencies(FILE_DEPS, "src/utils/colors.ts"),
|
||||
]).toEqual(["src/utils/colors.ts"]);
|
||||
});
|
||||
|
||||
it("should stop at an import cycle", () => {
|
||||
const cyclic = [
|
||||
{ source: "src/a.ts", dependencies: ["src/b.ts"] },
|
||||
{ source: "src/b.ts", dependencies: ["src/a.ts"] },
|
||||
];
|
||||
|
||||
expect([...getTransitiveDependencies(cyclic, "src/a.ts")].sort()).toEqual(
|
||||
["src/a.ts", "src/b.ts"],
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildUsageModuleGraph + getAffectedModules", () => {
|
||||
const usageGraph = buildUsageModuleGraph(ELEMENTS, FILE_DEPS);
|
||||
|
||||
it("should expand a changed module to its file-level affected modules", () => {
|
||||
expect(
|
||||
[...getAffectedModules(usageGraph, ["src/utils/colors.ts"])].sort(),
|
||||
).toEqual(["app/main", "feature/foo", "lib/utils"]);
|
||||
});
|
||||
|
||||
it("should NOT pull in feature/bar via the app/main hub", () => {
|
||||
const affected = getAffectedModules(usageGraph, ["src/foo/foo.tsx"]);
|
||||
expect([...affected].sort()).toEqual(["app/main", "feature/foo"]);
|
||||
expect(affected.has("feature/bar")).toBe(false);
|
||||
});
|
||||
|
||||
it("should treat the whole changed module as changed", () => {
|
||||
// app.js itself has no importers, but app/main also owns embed.tsx, which
|
||||
// feature/bar imports so changing any app/main file flags feature/bar.
|
||||
expect(
|
||||
[...getAffectedModules(usageGraph, ["src/app.js"])].sort(),
|
||||
).toEqual(["app/main", "feature/bar"]);
|
||||
});
|
||||
|
||||
it("should return empty when no file maps to a module", () => {
|
||||
expect(getAffectedModules(usageGraph, ["docs/foo.md"]).size).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe("getAffectedModules (rules graph)", () => {
|
||||
it("should affect feature/super and app/main but not feature/bar when feature/foo changes", () => {
|
||||
const result = getAffectedModules(graph, ["src/foo/x.ts"]);
|
||||
expect([...result].sort()).toEqual([
|
||||
"app/main",
|
||||
"feature/foo",
|
||||
"feature/super",
|
||||
]);
|
||||
});
|
||||
|
||||
it("should affect many modules when a lib module changes", () => {
|
||||
const result = getAffectedModules(graph, ["src/utils/colors.ts"]);
|
||||
expect(result.has("lib/utils")).toBe(true);
|
||||
expect(result.has("feature/foo")).toBe(true);
|
||||
expect(result.has("feature/bar")).toBe(true);
|
||||
expect(result.has("feature/super")).toBe(true);
|
||||
expect(result.has("app/main")).toBe(true);
|
||||
});
|
||||
|
||||
it("should return empty when no file maps to a module", () => {
|
||||
expect(getAffectedModules(graph, ["docs/foo.md"]).size).toBe(0);
|
||||
});
|
||||
});
|
||||
});
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user