Compare commits
262 Commits
stack-test
...
pr/engine
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2db6e9b6f1 | ||
|
|
b5c05a0708 | ||
|
|
403f6dadb3 | ||
|
|
906aebf3d3 | ||
|
|
7735051b1b | ||
|
|
e091d8d3c2 | ||
|
|
4818565f36 | ||
|
|
dde4a0092e | ||
|
|
651cec42ba | ||
|
|
2566752bc9 | ||
|
|
efaf23cc89 | ||
|
|
b5a9b92e8a | ||
|
|
3d70c0cfa4 | ||
|
|
16d23aa517 | ||
|
|
7d97df904c | ||
|
|
294fe63325 | ||
|
|
fa7e19d9a4 | ||
|
|
e420118c97 | ||
|
|
566300ec08 | ||
|
|
64906c88c0 | ||
|
|
01174b002d | ||
|
|
6ab4126f78 | ||
|
|
7887841730 | ||
|
|
cb0a754b4c | ||
|
|
0f3dee138e | ||
|
|
2def4914d2 | ||
|
|
451156caea | ||
|
|
7b85662b6c | ||
|
|
2961a56e9d | ||
|
|
6a841f5c45 | ||
|
|
8724e1794c | ||
|
|
94d7dd520c | ||
|
|
e4cf63ff23 | ||
|
|
c73e309b13 | ||
|
|
8196701636 | ||
|
|
042ec24fe9 | ||
|
|
ff335470e7 | ||
|
|
75bce3e619 | ||
|
|
9ddee1f434 | ||
|
|
0337de20c1 | ||
|
|
c71cf2a379 | ||
|
|
06462d0005 | ||
|
|
e6aa23212c | ||
|
|
79041e2487 | ||
|
|
9d216767d0 | ||
|
|
a9a062731b | ||
|
|
a7090f30ba | ||
|
|
8a7375b368 | ||
|
|
a55b6b6b8a | ||
|
|
58819b5ea2 | ||
|
|
0c06e94821 | ||
|
|
e34e6190c6 | ||
|
|
eeba22c5fe | ||
|
|
a06e24c8bd | ||
|
|
af1c635d4e | ||
|
|
30f9f25ba9 | ||
|
|
89da67a9f3 | ||
|
|
ae10677d61 | ||
|
|
3bcaab2e3b | ||
|
|
2921ecdcf9 | ||
|
|
8819a72555 | ||
|
|
22e8ddba1f | ||
|
|
8dbfef0fe6 | ||
|
|
de7cedfeb6 | ||
|
|
5885d30ad0 | ||
|
|
0b4bbf0427 | ||
|
|
74e847104b | ||
|
|
bf5778bc50 | ||
|
|
386fe3b12a | ||
|
|
dc3f2134cb | ||
|
|
30be3958a2 | ||
|
|
58ad1ba564 | ||
|
|
0da75ad4a3 | ||
|
|
b3365881d0 | ||
|
|
663a04c231 | ||
|
|
fc3927c030 | ||
|
|
7a20107f87 | ||
|
|
577f242c41 | ||
|
|
de202b144a | ||
|
|
9bf3b136af | ||
|
|
84223a0d4d | ||
|
|
e24150083f | ||
|
|
70f9234106 | ||
|
|
540817adfe | ||
|
|
7f73a7464c | ||
|
|
85f1e975a5 | ||
|
|
bc0200ece0 | ||
|
|
b4f84f1736 | ||
|
|
52ba78451d | ||
|
|
8e4e18f6d5 | ||
|
|
3c5d1111d7 | ||
|
|
af549487df | ||
|
|
9e2fa97aa1 | ||
|
|
1d598c7af4 | ||
|
|
b244f3c7ca | ||
|
|
3fd1f63cfc | ||
|
|
2459fe20ba | ||
|
|
0fec254d6d | ||
|
|
ebad52172a | ||
|
|
d7a519ab60 | ||
|
|
0e0e6ac82b | ||
|
|
4348d9373e | ||
|
|
3e094bd928 | ||
|
|
880bb31835 | ||
|
|
698fd4f1ee | ||
|
|
4dda44859b | ||
|
|
bd7048574b | ||
|
|
7c8d735d20 | ||
|
|
8b29969f1c | ||
|
|
90b3db6dec | ||
|
|
39ad1d5167 | ||
|
|
ba06454a57 | ||
|
|
a1059fff57 | ||
|
|
3d5011e100 | ||
|
|
08b90a304c | ||
|
|
49af71fd44 | ||
|
|
e34951e8b7 | ||
|
|
8d618a8620 | ||
|
|
72edd1917c | ||
|
|
bb4b24c65f | ||
|
|
baa2f30a5d | ||
|
|
2742202af1 | ||
|
|
2bd8150f2c | ||
|
|
bd77074097 | ||
|
|
a93af09ff5 | ||
|
|
c5e99eb320 | ||
|
|
4b8fca4eb0 | ||
|
|
0b035e3f44 | ||
|
|
1b7226c27a | ||
|
|
fd707ac9a3 | ||
|
|
4f81af5691 | ||
|
|
f2cef9e46c | ||
|
|
596e0c0f71 | ||
|
|
8c3ea6f40b | ||
|
|
72987c24bc | ||
|
|
105262e022 | ||
|
|
883ff5c9ac | ||
|
|
fa7e1e598a | ||
|
|
94f0aec2e9 | ||
|
|
2da893377b | ||
|
|
e815f913d1 | ||
|
|
63057ee485 | ||
|
|
38cc09d133 | ||
|
|
ba823a5488 | ||
|
|
b7d74aab6d | ||
|
|
e8b8ad1a88 | ||
|
|
a1c217c51b | ||
|
|
629fd1fee0 | ||
|
|
f332a4e4b2 | ||
|
|
1c7ac88006 | ||
|
|
7d46650cad | ||
|
|
24f4aec505 | ||
|
|
c839a31539 | ||
|
|
3866917fba | ||
|
|
1866aaccd9 | ||
|
|
52fa5626a2 | ||
|
|
ad47b4094b | ||
|
|
d568f38f1a | ||
|
|
0c54590106 | ||
|
|
840faeb408 | ||
|
|
172bc22044 | ||
|
|
c5878f2509 | ||
|
|
1e78c186c0 | ||
|
|
6b9bcaca8e | ||
|
|
418fbf1bf9 | ||
|
|
7293988a96 | ||
|
|
e925a8df94 | ||
|
|
a0d5c5a0c7 | ||
|
|
441cfa66ef | ||
|
|
01add7e6e2 | ||
|
|
e023152e90 | ||
|
|
e9235e2983 | ||
|
|
5bf02e30d7 | ||
|
|
cb7c71d1ba | ||
|
|
2bfd318907 | ||
|
|
4c62275732 | ||
|
|
2971a5b944 | ||
|
|
439b8e2af5 | ||
|
|
8861e84d3f | ||
|
|
0cad99631a | ||
|
|
393086c44a | ||
|
|
7e3ab7a682 | ||
|
|
e56d3de794 | ||
|
|
c06c3b8f16 | ||
|
|
190dcf1bd0 | ||
|
|
4fc305fa64 | ||
|
|
daf31c8bc3 | ||
|
|
cfd6896c17 | ||
|
|
847666c6d1 | ||
|
|
bb2b9786f3 | ||
|
|
57ac36ff71 | ||
|
|
8b42f49288 | ||
|
|
3ef8df323a | ||
|
|
8b478a76ab | ||
|
|
ed6247eb24 | ||
|
|
53a31166db | ||
|
|
72d6cfe9cd | ||
|
|
f9008f7884 | ||
|
|
e6e11ee362 | ||
|
|
41cd2ab4f7 | ||
|
|
f73eb6d00e | ||
|
|
21024ec984 | ||
|
|
a7815e9354 | ||
|
|
e819ade161 | ||
|
|
0163ebfd7a | ||
|
|
27852dcc4c | ||
|
|
9eb4ce8524 | ||
|
|
a9337e6046 | ||
|
|
96faa6d511 | ||
|
|
dff2fd94ce | ||
|
|
a25cc1f7be | ||
|
|
5c63c45970 | ||
|
|
cb3d067c6e | ||
|
|
3ac0958c50 | ||
|
|
05300912fb | ||
|
|
3ecdab5ea7 | ||
|
|
d18a2df374 | ||
|
|
32ff8a7c7c | ||
|
|
0be2972872 | ||
|
|
13dde0bba3 | ||
|
|
c24f084ae6 | ||
|
|
e1fbadec0b | ||
|
|
ad48fed046 | ||
|
|
7c08b5d95c | ||
|
|
25ed6aac2b | ||
|
|
5c169f142a | ||
|
|
42cf92479a | ||
|
|
fc487f06c9 | ||
|
|
6d2c060ef1 | ||
|
|
f8ce01fcca | ||
|
|
e433af3f90 | ||
|
|
57f6f37864 | ||
|
|
2ef0e4d264 | ||
|
|
c090eb292a | ||
|
|
9e329ce6fe | ||
|
|
0d36304bf5 | ||
|
|
dd1cb0f0ba | ||
|
|
ec3b24fadd | ||
|
|
7eba2629f2 | ||
|
|
c433278e06 | ||
|
|
11be7cb3e7 | ||
|
|
cad622a164 | ||
|
|
92fdc53695 | ||
|
|
c3dddcfcc6 | ||
|
|
31e8463f0f | ||
|
|
afa13457ae | ||
|
|
7692d3f479 | ||
|
|
0a601f07c6 | ||
|
|
cf3200aa96 | ||
|
|
f5b9b156e8 | ||
|
|
518f75d5f5 | ||
|
|
cc1d5a93d1 | ||
|
|
3a824943f0 | ||
|
|
3d7eed5d4a | ||
|
|
cb7114ff8a | ||
|
|
af1df1432c | ||
|
|
d35b91c92c | ||
|
|
bceebfaee4 | ||
|
|
55d7df2c9c | ||
|
|
a135d86c9a | ||
|
|
1f807106ab | ||
|
|
245862781f |
@@ -81,7 +81,7 @@ Everything else in the module is **internal** and not allowed to be referenced f
|
||||
|
||||
`dev/src/dev/modularization_help.clj` — `potemkin-ns!` helper that generates `.core` re-export boilerplate from a target namespace.
|
||||
|
||||
`test/metabase/core/modules_test.clj` — runs on config changes; validates structure, team assignments, API and model boundaries.
|
||||
`dev/test/metabase/core/modules_test.clj` — runs on config changes; validates structure, team assignments, API and model boundaries.
|
||||
|
||||
### API Design Inside a Module
|
||||
|
||||
@@ -119,7 +119,7 @@ When the linter complains, the answer is **always refactoring**, never suppressi
|
||||
- `dev/src/dev/module_score.clj` — quality scoring
|
||||
- `dev/src/dev/modularization_help.clj` — Potemkin re-export helper
|
||||
- `dev/src/dev/model_boundary_config.clj` + `dev/src/dev/model_tracking.clj` — model-level enforcement
|
||||
- `test/metabase/core/modules_test.clj` — config validation tests
|
||||
- `dev/test/metabase/core/modules_test.clj` — config validation tests
|
||||
- `src/metabase/core/init.clj` — OSS module init aggregation
|
||||
- `enterprise/backend/src/metabase_enterprise/core/init.clj` — EE module init aggregation
|
||||
- `src/metabase/models/resolution.clj` — `:model/X` keyword → namespace map
|
||||
@@ -198,6 +198,6 @@ Use the `clojure-eval` skill (preferred) or `clj-nrepl-eval` to drive the dev to
|
||||
(dev.module-score/scores (dev.module-score/deps) (dev.module-score/config))
|
||||
```
|
||||
|
||||
For tests of the config itself, use `./bin/test-agent :only '[metabase.core.modules-test]'`. After editing Clojure files, run `clj-paren-repair`. After editing `config.edn`, re-run the modules-test.
|
||||
For tests of the config itself, use `./bin/mage project-tests modules`. After editing Clojure files, run `clj-paren-repair`. After editing `config.edn`, re-run the modules-test.
|
||||
|
||||
**Update your agent memory** as you discover idioms: which modules use `:friends` and why, which models are most-imported (suggesting they belong in their own module), which settings are accessed cross-module via `(setting/get …)` instead of `:uses` deps, and which refactors the team is actively running.
|
||||
|
||||
@@ -304,12 +304,13 @@ Tag response Malli `:map` schemas with `{:closed true}` so the generated OpenAPI
|
||||
|
||||
**Linter Suppressions:**
|
||||
|
||||
The right shape is a **reader-discard map** placed directly above the form being silenced, listing the rule keywords explicitly:
|
||||
The right shape is a **reader-discard map** placed directly above the form being silenced, listing the
|
||||
rule keywords explicitly and including the required explanation:
|
||||
|
||||
```clojure
|
||||
#_{:clj-kondo/ignore [:metabase/validate-defendpoint-route-uses-kebab-case
|
||||
:metabase/validate-defendpoint-has-response-schema]}
|
||||
(api.macros/defendpoint :post "/api_key" ...)
|
||||
;; Loaded by name from the plugin registry.
|
||||
#_{:clj-kondo/ignore [:clojure-lsp/unused-public-var]}
|
||||
(defn plugin-entrypoint ...)
|
||||
```
|
||||
|
||||
For a single require entry, hang it as metadata on that entry:
|
||||
@@ -320,7 +321,12 @@ For a single require entry, hang it as metadata on that entry:
|
||||
[metabase.query-processor.store :as qp.store])
|
||||
```
|
||||
|
||||
Common Metabase-specific rules you'll see: `:metabase/modules`, `:metabase/validate-defendpoint-route-uses-kebab-case`, `:metabase/validate-defendpoint-has-response-schema`, `:metabase/prefer-with-dynamic-fn-redefs`, `:deprecated-namespace`, `:discouraged-namespace`. Don't use the keyword form (`#_:clj-kondo/ignore`); always the map form so it's grep-able by which rule(s) are suppressed.
|
||||
Use the map form, not the keyword form (`#_:clj-kondo/ignore`), so searches can identify the suppressed
|
||||
rules. Keep `:clj-kondo/ignore` as the map's first key; the ratchet scanner rejects other orderings.
|
||||
|
||||
Fix the warning when possible. Adding a suppression is a last resort and requires approval. If one is
|
||||
necessary, add an explanatory `;;` comment directly above it or at the end of the same line. CI enforces
|
||||
these requirements with `./bin/mage kondo-ratchets`.
|
||||
|
||||
**Configurable Options:**
|
||||
|
||||
|
||||
@@ -111,7 +111,12 @@ liability it usually is.
|
||||
### Miscellaneous
|
||||
|
||||
- [ ] Example data is bird-themed when possible
|
||||
- [ ] Kondo linter suppressions use proper format (not `#_:clj-kondo/ignore` keyword form)
|
||||
- [ ] Kondo suppressions use a reader-discard map, not `#_:clj-kondo/ignore`, with
|
||||
`:clj-kondo/ignore` as the first key
|
||||
- [ ] Each new suppression is approved and has a `;;` comment explaining why the warning cannot be fixed
|
||||
- [ ] Any ratchet budget increase is explained in the PR
|
||||
- [ ] PRs that remove ignores leave ratchet budgets unchanged; the shrink workflow records reductions on
|
||||
`master`
|
||||
|
||||
## Pattern matching table
|
||||
|
||||
|
||||
@@ -123,5 +123,12 @@ breakage keeps happening, *that* is the signal a comment was warranted.
|
||||
- Spaces on a line with nothing after it is not allowed
|
||||
- After changing module boundaries (adding/removing/renaming a `src` namespace, a cross-module `require`
|
||||
or `:model/X` reference, or a new module), run `./bin/mage fix-modules-config` to regenerate
|
||||
`.clj-kondo/config/modules/config.edn` and keep `metabase.core.modules-test` green. No-op when nothing
|
||||
drifted; see "Module Boundaries" in the project `CLAUDE.md`.
|
||||
`.clj-kondo/config/modules/config.edn`, then run `./bin/mage project-tests modules`. The fixer is a no-op when
|
||||
nothing drifted; see "Module Boundaries" in the project `CLAUDE.md`.
|
||||
- Fix kondo warnings instead of suppressing them. Add a suppression only as a last resort, with approval
|
||||
and a `;;` comment explaining why it is necessary.
|
||||
- Do not lower budgets in `.clj-kondo/ratchets.edn` after removing ignores. The shrink workflow records
|
||||
reductions on `master`.
|
||||
- If `.clj-kondo/ratchets.edn` conflicts during a merge, rebase, or restack, run
|
||||
`./bin/merge-kondo-ratchets`. It preserves one-sided changes, chooses the policy that allows fewer
|
||||
suppressions when both sides change the same entry, and stops when the conflict needs a human decision.
|
||||
|
||||
@@ -37,14 +37,16 @@ First check if snapshots exist:
|
||||
ls e2e/snapshots/default.sql 2>/dev/null && echo "snapshots exist" || echo "snapshots missing"
|
||||
```
|
||||
|
||||
Always pass the spec path via `--spec`. The runner feeds its arguments to Cypress's CLI parser (`cypress.cli.parseRunArguments`), which silently drops bare positional paths and then runs the entire suite. A leading `run` argument is harmless, but the path itself must follow `--spec`. Sanity-check that the first `Running:` line says `(1 of 1)`.
|
||||
|
||||
If snapshots exist, skip regeneration for speed:
|
||||
```bash
|
||||
MB_EDITION=oss CYPRESS_VIDEO=false CYPRESS_RETRIES=0 CYPRESS_GUI=false GENERATE_SNAPSHOTS=false bun test-cypress $ARGUMENTS
|
||||
MB_EDITION=oss CYPRESS_VIDEO=false CYPRESS_RETRIES=0 CYPRESS_GUI=false GENERATE_SNAPSHOTS=false bun test-cypress --spec path/to/spec.cy.spec.js
|
||||
```
|
||||
|
||||
If snapshots are missing (first run), let the runner generate them:
|
||||
```bash
|
||||
MB_EDITION=oss CYPRESS_VIDEO=false CYPRESS_RETRIES=0 CYPRESS_GUI=false bun test-cypress $ARGUMENTS
|
||||
MB_EDITION=oss CYPRESS_VIDEO=false CYPRESS_RETRIES=0 CYPRESS_GUI=false bun test-cypress --spec path/to/spec.cy.spec.js
|
||||
```
|
||||
|
||||
When running enterprise tests, replace `MB_EDITION=oss` with `MB_EDITION=ee` in the commands above.
|
||||
@@ -67,9 +69,11 @@ GREP="test name here" MB_EDITION=oss bun test-cypress --spec path/to/spec.js
|
||||
When the user asks to verify a test is not flaky:
|
||||
|
||||
```bash
|
||||
MB_EDITION=oss bin/e2e-stress-test $ARGUMENTS
|
||||
MB_EDITION=oss bin/e2e-stress-test --spec path/to/spec.cy.spec.js
|
||||
```
|
||||
|
||||
(The stress script forwards its arguments to `bun test-cypress`, so the same `--spec` rule applies.)
|
||||
|
||||
Set `E2E_STRESS_RUNS=N` for more iterations (default: 5). Stops on first failure and prints screenshot paths. Read those screenshots to analyze the failure.
|
||||
|
||||
## Analyzing failures
|
||||
|
||||
@@ -171,7 +171,7 @@ Create `ComponentName.module.css` alongside the component file:
|
||||
/* Use design system tokens — NEVER raw color/spacing values */
|
||||
.root {
|
||||
border: 1px solid var(--mb-color-border-neutral);
|
||||
border-radius: var(--mantine-radius-md);
|
||||
border-radius: var(--mantine-radius-sm);
|
||||
background-color: var(--mb-color-background_page-primary);
|
||||
}
|
||||
|
||||
@@ -246,7 +246,8 @@ Remove the old styled file entirely. Remove all imports of it from other files.
|
||||
|
||||
**Spacing** (`p`, `px`, `py`, `pt`, `pb`, `pl`, `pr`, `m`, `mx`, `my`, `mt`, `mb`, `ml`, `mr`):
|
||||
|
||||
- `"xs"` = 4px, `"sm"` = 8px, `"md"` = 16px, `"lg"` = 24px, `"xl"` = 32px
|
||||
- `"xxxs"` = 0.125rem, `"xxs"` = 0.25rem, `"xs"` = 0.375rem, `"sm"` = 0.5rem, `"md"` = 0.75rem, `"lg"` = 1rem, `"xl"` = 1.5rem, `"xxl"` = 2rem, `"xxxl"` = 2.5rem
|
||||
- Use numeric `0` for zero spacing.
|
||||
- Custom: `rem(48)` for non-standard values (import `rem` from `metabase/ui`)
|
||||
|
||||
**Dimensions** (`w`, `h`, `maw`, `mah`, `miw`, `mih`):
|
||||
@@ -289,11 +290,16 @@ Remove the old styled file entirely. Remove all imports of it from other files.
|
||||
|
||||
**Mantine spacing** — `var(--mantine-spacing-<size>)`:
|
||||
|
||||
- `--mantine-spacing-xs` (4px), `--mantine-spacing-sm` (8px), `--mantine-spacing-md` (16px), `--mantine-spacing-lg` (24px), `--mantine-spacing-xl` (32px)
|
||||
- `--mantine-spacing-xxxs` (0.125rem), `--mantine-spacing-xxs` (0.25rem), `--mantine-spacing-xs` (0.375rem), `--mantine-spacing-sm` (0.5rem), `--mantine-spacing-md` (0.75rem), `--mantine-spacing-lg` (1rem), `--mantine-spacing-xl` (1.5rem), `--mantine-spacing-xxl` (2rem), `--mantine-spacing-xxxl` (2.5rem)
|
||||
|
||||
**Mantine radius** — `var(--mantine-radius-<size>)`:
|
||||
|
||||
- `--mantine-radius-xs` (4px), `--mantine-radius-sm` (6px), `--mantine-radius-md` (8px), `--mantine-radius-xl` (40px)
|
||||
- `--mantine-radius-xxxs` (0.125rem), `--mantine-radius-xxs` (0.25rem), `--mantine-radius-xs` (0.375rem), `--mantine-radius-sm` (0.5rem), `--mantine-radius-md` (0.75rem), `--mantine-radius-lg` (1rem), `--mantine-radius-xl` (1.5rem)
|
||||
|
||||
**Mantine shadows (elevation)** — `var(--mantine-shadow-<size>)`:
|
||||
|
||||
- `--mantine-shadow-xs`, `--mantine-shadow-xs_outline`, `--mantine-shadow-sm`, `--mantine-shadow-sm_outline`, `--mantine-shadow-lg_outline`
|
||||
- Light and dark modes use different shadow values (defined in `frontend/src/metabase/ui/theme.ts`)
|
||||
|
||||
**Mantine font sizes** — `var(--mantine-font-size-<size>)`:
|
||||
|
||||
@@ -313,24 +319,36 @@ When migrating, **never carry over hardcoded `rem`/`px` values** from the origin
|
||||
|
||||
**Spacing** — snap to the nearest Mantine spacing token:
|
||||
|
||||
| Hardcoded value | Nearest token | Style prop | CSS variable |
|
||||
| --------------------------------- | ------------- | ------------ | ------------------------------ |
|
||||
| `0.125rem` (2px) | `2px` | `2` (number) | `2px` (keep literal, no token) |
|
||||
| `0.2rem` (3.2px), `0.25rem` (4px) | **xs** (4px) | `"xs"` | `var(--mantine-spacing-xs)` |
|
||||
| `0.5rem` (8px) | **sm** (8px) | `"sm"` | `var(--mantine-spacing-sm)` |
|
||||
| `0.75rem` (12px) | between sm/md | `rem(12)` | `0.75rem` (no exact token) |
|
||||
| `1rem` (16px) | **md** (16px) | `"md"` | `var(--mantine-spacing-md)` |
|
||||
| `1.5rem` (24px) | **lg** (24px) | `"lg"` | `var(--mantine-spacing-lg)` |
|
||||
| `2rem` (32px) | **xl** (32px) | `"xl"` | `var(--mantine-spacing-xl)` |
|
||||
| Hardcoded value | Nearest token | Style prop | CSS variable |
|
||||
| --------------------------------- | ------------- | ---------- | ------------------------------- |
|
||||
| `0` (0px) | — | `0` | `0` |
|
||||
| `0.125rem` (2px) | **xxxs** | `"xxxs"` | `var(--mantine-spacing-xxxs)` |
|
||||
| `0.2rem` (3.2px), `0.25rem` (4px) | **xxs** | `"xxs"` | `var(--mantine-spacing-xxs)` |
|
||||
| `0.375rem` (6px) | **xs** | `"xs"` | `var(--mantine-spacing-xs)` |
|
||||
| `0.5rem` (8px) | **sm** | `"sm"` | `var(--mantine-spacing-sm)` |
|
||||
| `0.75rem` (12px) | **md** | `"md"` | `var(--mantine-spacing-md)` |
|
||||
| `1rem` (16px) | **lg** | `"lg"` | `var(--mantine-spacing-lg)` |
|
||||
| `1.5rem` (24px) | **xl** | `"xl"` | `var(--mantine-spacing-xl)` |
|
||||
| `2rem` (32px) | **xxl** | `"xxl"` | `var(--mantine-spacing-xxl)` |
|
||||
| `2.5rem` (40px) | **xxxl** | `"xxxl"` | `var(--mantine-spacing-xxxl)` |
|
||||
|
||||
Values between steps: keep the literal (e.g. `rem(10)`), don't round to a token.
|
||||
|
||||
**Border radius** — snap to the nearest Mantine radius token:
|
||||
|
||||
| Hardcoded value | Nearest token | CSS variable |
|
||||
| ---------------- | ----------------------------- | -------------------------- |
|
||||
| `0.25rem` (4px) | **xs** (4px) | `var(--mantine-radius-xs)` |
|
||||
| `0.375rem` (6px) | **sm** (6px) | `var(--mantine-radius-sm)` |
|
||||
| `0.5rem` (8px) | **md** (8px) | `var(--mantine-radius-md)` |
|
||||
| `1rem`+ (16px+) | **xl** (40px) or keep literal | `var(--mantine-radius-xl)` |
|
||||
| Hardcoded value | Nearest token | CSS variable |
|
||||
| ---------------- | ------------- | ----------------------------- |
|
||||
| `0` (0px) | — | `0` |
|
||||
| `0.125rem` (2px) | **xxxs** | `var(--mantine-radius-xxxs)` |
|
||||
| `0.25rem` (4px) | **xxs** | `var(--mantine-radius-xxs)` |
|
||||
| `0.375rem` (6px) | **xs** | `var(--mantine-radius-xs)` |
|
||||
| `0.5rem` (8px) | **sm** | `var(--mantine-radius-sm)` |
|
||||
| `0.75rem` (12px) | **md** | `var(--mantine-radius-md)` |
|
||||
| `1rem` (16px) | **lg** | `var(--mantine-radius-lg)` |
|
||||
| `1.5rem` (24px) | **xl** | `var(--mantine-radius-xl)` |
|
||||
|
||||
Fully-round pills/badges (radius ≥ half the element height): keep a literal
|
||||
such as `9999px`.
|
||||
|
||||
**Font sizes** — snap to the nearest Mantine font size token:
|
||||
|
||||
|
||||
@@ -205,7 +205,7 @@ Per `docs/developers-guide/frontend.md` ("Colors"): every color must be a **sema
|
||||
|
||||
The same discipline applies to **dimensional** values — radius, spacing/padding/gap, elevation/shadow, and typography (font size, line-height, weight). Prefer the codebase's existing scale variable over a literal: `--mantine-radius-*`, `--mantine-spacing-*`, `--mantine-shadow-*`, and the Mantine font tokens.
|
||||
|
||||
But Figma's scales don't always line up with the codebase's. When a Figma value **has no matching step** in the corresponding scale — e.g. Figma `radius/md = 12px` while `--mantine-radius-md` is `8px`, or a Figma `xxxs` spacing step with no Mantine equivalent — **don't silently bake in a literal.** Flag it to the user the same way you flag primitive colors: name the property, the Figma value, and the nearest codebase token, so it can be resolved with design / the scale owners (align the value, or add a scale step). Only fall back to a literal if the user confirms it's an accepted one-off, and leave a short comment in the code explaining why.
|
||||
But Figma's scales don't always line up with the codebase's. When a Figma value **has no matching step** in the corresponding scale — e.g. Figma `radius/md = 16px` while `--mantine-radius-md` is `0.75rem`, or a Figma spacing step that has no Mantine equivalent — **don't silently bake in a literal.** Flag it to the user the same way you flag primitive colors: name the property, the Figma value, and the nearest codebase token, so it can be resolved with design / the scale owners (align the value, or add a scale step). Only fall back to a literal if the user confirms it's an accepted one-off, and leave a short comment in the code explaining why. The codebase scales are defined in `frontend/src/metabase/ui/theme.ts` (`SPACING_SCALE`, `RADIUS_SCALE`, `LIGHT_SHADOWS`/`DARK_SHADOWS`).
|
||||
|
||||
### Mantine implementation notes (reference)
|
||||
|
||||
|
||||
@@ -191,6 +191,8 @@
|
||||
metabase.channel.render.pdf.font/load-fonts!
|
||||
metabase.warehouses.models.database/assert-not-h2!
|
||||
metabase.driver.sql-jdbc.execute/execute-prepared-statement!
|
||||
;; only inspects a details map and throws; the `!` is for the throw, not for a mutation
|
||||
metabase.driver/validate-db-details!
|
||||
metabase.mcp.usage/record-mcp-session!
|
||||
metabase.mcp.usage/record-mcp-session-end!
|
||||
metabase.mcp.usage/record-mcp-tool-call!
|
||||
@@ -496,6 +498,7 @@
|
||||
metabase.notification.test-util/with-card-notification clojure.core/let
|
||||
metabase.notification.test-util/with-system-event-notification! clojure.core/let
|
||||
metabase.notification.test-util/with-temp-notification clojure.core/let
|
||||
metabase.oauth-server.test-util/with-oauth-client clojure.core/fn
|
||||
metabase.osi.ai-context.api-test/with-test-entry clojure.core/let
|
||||
metabase.premium-features.core/defenterprise-schema clj-kondo.lint-as/def-catch-all
|
||||
metabase.premium-features.defenterprise/defenterprise-schema clj-kondo.lint-as/def-catch-all
|
||||
|
||||
@@ -104,6 +104,7 @@
|
||||
model-persistence
|
||||
models
|
||||
parameters
|
||||
public-sharing
|
||||
queries
|
||||
query-processor
|
||||
search
|
||||
@@ -141,7 +142,7 @@
|
||||
metabase.activity-feed.core
|
||||
metabase.activity-feed.init}
|
||||
:uses #{api
|
||||
app-db
|
||||
batch-processing
|
||||
collections
|
||||
config
|
||||
events
|
||||
@@ -347,6 +348,7 @@
|
||||
:api #{metabase.app-db.cluster-lock ; TODO FIXME
|
||||
metabase.app-db.core
|
||||
metabase.app-db.init
|
||||
metabase.app-db.setting
|
||||
metabase.app-db.setup
|
||||
metabase.app-db.transient-error}
|
||||
:uses #{auth-provider
|
||||
@@ -355,10 +357,7 @@
|
||||
connection-pool
|
||||
task
|
||||
util}
|
||||
:model-imports #{:model/Field
|
||||
:model/QueryCache
|
||||
:model/Secret
|
||||
:model/Setting}}
|
||||
:model-imports #{:model/Field}}
|
||||
|
||||
appearance
|
||||
{:team "UX West"
|
||||
@@ -440,6 +439,7 @@
|
||||
:uses #{api
|
||||
app-db
|
||||
collections
|
||||
permissions
|
||||
queries
|
||||
util}
|
||||
:model-exports #{:model/CardBookmark :model/CollectionBookmark :model/DashboardBookmark}
|
||||
@@ -579,6 +579,8 @@
|
||||
auth-identity
|
||||
classloader
|
||||
config
|
||||
llm
|
||||
metabot
|
||||
models
|
||||
query-processor
|
||||
search
|
||||
@@ -590,6 +592,7 @@
|
||||
{:team "UX West"
|
||||
:api #{metabase.collections.core
|
||||
metabase.collections.curation
|
||||
metabase.collections.init
|
||||
metabase.collections.models.collection
|
||||
metabase.collections.models.collection.root
|
||||
metabase.collections.schema}
|
||||
@@ -628,6 +631,7 @@
|
||||
:uses #{api
|
||||
app-db
|
||||
collections
|
||||
documents
|
||||
eid-translation
|
||||
events
|
||||
lib-be
|
||||
@@ -656,6 +660,7 @@
|
||||
comments
|
||||
{:team "UX West"
|
||||
:api #{metabase.comments.api
|
||||
metabase.comments.core
|
||||
metabase.comments.init}
|
||||
:uses #{analytics
|
||||
api
|
||||
@@ -696,7 +701,7 @@
|
||||
models
|
||||
util}
|
||||
:model-exports #{:model/ModerationReview}
|
||||
:model-imports #{:model/Card :model/Dashboard}}
|
||||
:model-imports #{:model/Card :model/Dashboard :model/User}}
|
||||
|
||||
contextual-interestingness
|
||||
{:team "UX West"
|
||||
@@ -999,9 +1004,14 @@
|
||||
entity-retrieval
|
||||
{:team "Metabot"
|
||||
:api #{metabase.entity-retrieval.core
|
||||
metabase.entity-retrieval.mirror}
|
||||
:uses #{premium-features}
|
||||
:model-imports #{:model/OsiAiContext}}
|
||||
metabase.entity-retrieval.mirror
|
||||
metabase.entity-retrieval.spec}
|
||||
:uses #{collections premium-features util}
|
||||
:model-imports #{:model/Card
|
||||
:model/Measure
|
||||
:model/OsiAiContext
|
||||
:model/Segment
|
||||
:model/Table}}
|
||||
|
||||
events
|
||||
{:team "DevEx"
|
||||
@@ -1032,7 +1042,9 @@
|
||||
api
|
||||
app-db
|
||||
collections
|
||||
comments
|
||||
contextual-interestingness
|
||||
documents
|
||||
events
|
||||
interestingness
|
||||
lib
|
||||
@@ -1057,6 +1069,7 @@
|
||||
:model-imports #{:model/Card
|
||||
:model/Collection
|
||||
:model/Comment
|
||||
:model/Document
|
||||
:model/Segment
|
||||
:model/StoredResult
|
||||
:model/StoredResultUse
|
||||
@@ -1350,6 +1363,7 @@
|
||||
{:team "Graphy"
|
||||
:api #{metabase.measures.api}
|
||||
:uses #{api
|
||||
entity-retrieval
|
||||
events
|
||||
lib
|
||||
lib-be
|
||||
@@ -1368,6 +1382,7 @@
|
||||
metabase.metabot.config
|
||||
metabase.metabot.core
|
||||
metabase.metabot.init
|
||||
metabase.metabot.metadata-perms
|
||||
metabase.metabot.self
|
||||
metabase.metabot.settings
|
||||
metabase.metabot.tools.entity-details}
|
||||
@@ -1448,6 +1463,7 @@
|
||||
:api #{metabase.metrics.api
|
||||
metabase.metrics.core}
|
||||
:uses #{api
|
||||
app-db
|
||||
events
|
||||
lib
|
||||
lib-be
|
||||
@@ -1510,6 +1526,7 @@
|
||||
:model/Dashboard
|
||||
:model/DashboardTab
|
||||
:model/Database
|
||||
:model/Document
|
||||
:model/Field
|
||||
:model/Measure
|
||||
:model/NativeQuerySnippet
|
||||
@@ -1600,6 +1617,7 @@
|
||||
:uses #{api
|
||||
api-scope
|
||||
appearance
|
||||
events
|
||||
mcp
|
||||
models
|
||||
request
|
||||
@@ -1607,7 +1625,8 @@
|
||||
system
|
||||
task
|
||||
tracing
|
||||
util}}
|
||||
util}
|
||||
:model-imports #{:model/User}}
|
||||
|
||||
osi
|
||||
{:team "Metabot"
|
||||
@@ -1765,7 +1784,11 @@
|
||||
metabase.public-sharing.validation}
|
||||
:uses #{api
|
||||
settings
|
||||
util}}
|
||||
util}
|
||||
:model-imports #{:model/Action
|
||||
:model/Card
|
||||
:model/Dashboard
|
||||
:model/Document}}
|
||||
|
||||
public-sharing-rest
|
||||
{:team "UX West"
|
||||
@@ -1786,7 +1809,8 @@
|
||||
request
|
||||
tiles
|
||||
util}
|
||||
:model-imports #{:model/Card
|
||||
:model-imports #{:model/Action
|
||||
:model/Card
|
||||
:model/Dashboard
|
||||
:model/DashboardCard
|
||||
:model/Document}}
|
||||
@@ -1851,6 +1875,7 @@
|
||||
dashboards
|
||||
documents
|
||||
embedding
|
||||
entity-retrieval
|
||||
events
|
||||
graph
|
||||
lib
|
||||
@@ -1920,6 +1945,8 @@
|
||||
:model/Collection
|
||||
:model/Dashboard
|
||||
:model/Database
|
||||
:model/StoredResult
|
||||
:model/StoredResultUse
|
||||
:model/Table}}
|
||||
|
||||
query-permissions
|
||||
@@ -1961,6 +1988,7 @@
|
||||
metabase.query-processor.pipeline
|
||||
metabase.query-processor.pivot
|
||||
metabase.query-processor.preprocess
|
||||
metabase.query-processor.result-serialization
|
||||
metabase.query-processor.schema
|
||||
metabase.query-processor.setup
|
||||
metabase.query-processor.store
|
||||
@@ -2140,14 +2168,14 @@
|
||||
secrets
|
||||
{:team "Graphy"
|
||||
:api #{metabase.secrets.core}
|
||||
:uses #{api driver models premium-features util}
|
||||
:model-exports #{:model/Secret}}
|
||||
:uses #{api driver models premium-features util}}
|
||||
|
||||
segments
|
||||
{:team "Graphy"
|
||||
:api #{metabase.segments.api
|
||||
metabase.segments.schema}
|
||||
:uses #{api
|
||||
entity-retrieval
|
||||
events
|
||||
lib
|
||||
lib-be
|
||||
@@ -2187,7 +2215,6 @@
|
||||
request
|
||||
session
|
||||
settings
|
||||
setup
|
||||
sso
|
||||
system
|
||||
tracing
|
||||
@@ -2222,8 +2249,7 @@
|
||||
|
||||
settings
|
||||
{:team "DevEx"
|
||||
:api #{metabase.settings.core
|
||||
metabase.settings.init}
|
||||
:api #{metabase.settings.core metabase.settings.init}
|
||||
:uses #{api
|
||||
app-db
|
||||
config
|
||||
@@ -2232,7 +2258,8 @@
|
||||
premium-features
|
||||
util
|
||||
enterprise/advanced-permissions}
|
||||
:model-exports #{:model/Setting}}
|
||||
:model-exports #{:model/Setting}
|
||||
:model-imports #{:model/User}}
|
||||
|
||||
settings-rest
|
||||
{:team "DevEx"
|
||||
@@ -2408,6 +2435,7 @@
|
||||
metabase.task.core}
|
||||
:uses #{app-db
|
||||
classloader
|
||||
config
|
||||
tracing
|
||||
util}}
|
||||
|
||||
@@ -2473,6 +2501,7 @@
|
||||
lib
|
||||
lib-be
|
||||
parameters
|
||||
premium-features
|
||||
query-processor
|
||||
settings
|
||||
util}
|
||||
@@ -2717,6 +2746,7 @@
|
||||
:model-imports #{:model/Dashboard
|
||||
:model/Database
|
||||
:model/PermissionsGroupMembership
|
||||
:model/PulseChannelRecipient
|
||||
:model/Tenant}}
|
||||
|
||||
users-rest
|
||||
@@ -2776,7 +2806,10 @@
|
||||
query-permissions
|
||||
util}
|
||||
:model-exports #{:model/ViewLog}
|
||||
:model-imports #{:model/Card :model/Dashboard :model/Table}}
|
||||
:model-imports #{:model/Card
|
||||
:model/Dashboard
|
||||
:model/Document
|
||||
:model/Table}}
|
||||
|
||||
warehouse-schema
|
||||
{:team "UX West"
|
||||
@@ -2795,6 +2828,7 @@
|
||||
audit-app
|
||||
collections
|
||||
driver
|
||||
entity-retrieval
|
||||
lib
|
||||
lib-be
|
||||
models
|
||||
@@ -3161,7 +3195,8 @@
|
||||
:uses #{api
|
||||
content-verification
|
||||
util
|
||||
enterprise/api}}
|
||||
enterprise/api}
|
||||
:model-imports #{:model/Card :model/Dashboard}}
|
||||
|
||||
enterprise/core
|
||||
{:team "DevEx"
|
||||
@@ -3364,7 +3399,6 @@
|
||||
:api #{metabase-enterprise.entity-retrieval.init}
|
||||
:uses #{analytics-interface
|
||||
app-db
|
||||
collections
|
||||
entity-retrieval
|
||||
health-inspector
|
||||
premium-features
|
||||
@@ -3524,6 +3558,23 @@
|
||||
util}
|
||||
:model-imports #{:model/AuthIdentity :model/User}}
|
||||
|
||||
enterprise/osi-generation
|
||||
{:team "Metabot"
|
||||
:uses #{analytics-interface
|
||||
api
|
||||
app-db
|
||||
entity-retrieval
|
||||
llm
|
||||
metabot
|
||||
osi
|
||||
premium-features
|
||||
settings
|
||||
task
|
||||
util
|
||||
enterprise/semantic-search}
|
||||
:api #{metabase-enterprise.osi-generation.api metabase-enterprise.osi-generation.init}
|
||||
:model-imports #{:model/OsiAiContext}}
|
||||
|
||||
enterprise/permission-debug
|
||||
{:team "UX West"
|
||||
:api #{metabase-enterprise.permission-debug.api}
|
||||
@@ -3801,12 +3852,14 @@
|
||||
:api #{metabase-enterprise.support-access-grants.api
|
||||
metabase-enterprise.support-access-grants.init}
|
||||
:uses #{api
|
||||
app-db
|
||||
auth-identity
|
||||
events
|
||||
models
|
||||
request
|
||||
settings
|
||||
system
|
||||
task
|
||||
util}
|
||||
:model-imports #{:model/AuthIdentity :model/Session :model/User}}
|
||||
|
||||
|
||||
@@ -58,6 +58,7 @@
|
||||
metabase.app-db.core/finish-db-setup!
|
||||
metabase.app-db.core/migrate!
|
||||
metabase.app-db.core/setup-db!
|
||||
metabase.app-db.encryption-test-util/do-with-encrypted-app-db!
|
||||
metabase.app-db.schema-migrations-test.impl/run-migrations-in-range!
|
||||
metabase.app-db.setup/migrate!
|
||||
metabase.app-db.setup/setup-db!
|
||||
|
||||
@@ -1,47 +1,80 @@
|
||||
;; Per-linter budgets for inline `:clj-kondo/ignore` forms.
|
||||
;; metabase.core.kondo-ratchet-test fails when the budgets drift from the actual counts.
|
||||
;; `./bin/mage fix-kondo-ratchets` lowers budgets to match the tree; local test runs do it
|
||||
;; automatically. Raising a budget, or adding one for a new linter (`--seed`), is a hand edit
|
||||
;; to defend in your PR.
|
||||
;; Budgets for kondo suppressions: inline `:clj-kondo/ignore` forms per linter (:ignore-counts), and
|
||||
;; config-level waivers in .clj-kondo/config.edn (:config-counts -- :off switches and :exclude entries).
|
||||
;; Each :ignore-counts value is a non-negative integer budget, or :unlimited for no ceiling.
|
||||
;; Checks fail when a count exceeds its numeric budget; unused budget is allowed.
|
||||
;; Any ignore outside :comment-exempt needs an explanatory comment directly above or trailing on its line.
|
||||
;; `./bin/mage kondo-ratchets-shrink` lowers budgets unless `--seed` explicitly adds or raises one.
|
||||
;; The workflow runs it on master, so feature branches do not need to record reductions.
|
||||
;; Raising or adding a budget (`--seed` for inline ignores, a manual edit for config) or widening the
|
||||
;; exemptions must be explained in the PR.
|
||||
;; :all is the vector-less ignore form, which suppresses every linter on the next form.
|
||||
{:ignore-counts {:aliased-namespace-symbol 3
|
||||
:all 48
|
||||
:case-symbol-test 1
|
||||
:clojure-lsp/unused-public-var 9
|
||||
:def-fn 1
|
||||
:deprecated-namespace 95
|
||||
:deprecated-var 84
|
||||
:discouraged-java-method 3
|
||||
:discouraged-namespace 81
|
||||
:discouraged-var 111
|
||||
:equals-true 1
|
||||
:main-without-gen-class 1
|
||||
:metabase/defsetting-namespace 2
|
||||
:metabase/disallow-hardcoded-driver-names-in-tests 34
|
||||
:metabase/discourage-millis-duration 1
|
||||
:metabase/i-like-making-cams-eyes-bleed-with-horrifically-long-tests 13
|
||||
:metabase/modules 7
|
||||
:metabase/namespace-name 4
|
||||
:metabase/prefer-with-dynamic-fn-redefs 28
|
||||
:metabase/test-helpers-use-non-thread-safe-functions 17
|
||||
:metabase/validate-defendpoint-has-response-schema 440
|
||||
:metabase/validate-defendpoint-query-params-use-kebab-case 50
|
||||
:metabase/validate-defendpoint-route-uses-kebab-case 44
|
||||
:metabase/validate-deftest 10
|
||||
:missing-docstring 10
|
||||
:missing-protocol-method 3
|
||||
:non-arg-vec-return-type-hint 1
|
||||
:reduce-without-init 3
|
||||
:syntax 2
|
||||
:uninitialized-var 1
|
||||
:unquote-not-syntax-quoted 1
|
||||
:unresolved-namespace 3
|
||||
:unresolved-require 1
|
||||
:unresolved-symbol 4
|
||||
:unsorted-required-namespaces 1
|
||||
:unused-alias 1
|
||||
:unused-binding 9
|
||||
:unused-excluded-var 2
|
||||
:unused-namespace 1
|
||||
:unused-private-var 9
|
||||
:unused-value 1}}
|
||||
{:ignore-counts {:aliased-namespace-symbol 3
|
||||
:case-symbol-test 1
|
||||
:clojure-lsp/unused-public-var 9
|
||||
:consistent-alias 2
|
||||
:def-fn 1
|
||||
:deprecated-namespace 95
|
||||
:deprecated-var 86
|
||||
:discouraged-java-method 4
|
||||
:discouraged-namespace 71
|
||||
:discouraged-var 130
|
||||
:duplicate-require 1
|
||||
:equals-true 1
|
||||
:main-without-gen-class 1
|
||||
:metabase/defsetting-namespace 2
|
||||
:metabase/disallow-hardcoded-driver-names-in-tests 34
|
||||
:metabase/discourage-millis-duration 1
|
||||
:metabase/i-like-making-cams-eyes-bleed-with-horrifically-long-tests 13
|
||||
:metabase/modules 23
|
||||
:metabase/namespace-name 4
|
||||
:metabase/prefer-with-dynamic-fn-redefs 28
|
||||
:metabase/test-helpers-use-non-thread-safe-functions 18
|
||||
:metabase/validate-defendpoint-has-response-schema 439
|
||||
:metabase/validate-defendpoint-query-params-use-kebab-case 50
|
||||
:metabase/validate-defendpoint-route-uses-kebab-case 44
|
||||
:metabase/validate-deftest 10
|
||||
:missing-docstring 12
|
||||
:missing-protocol-method 3
|
||||
:non-arg-vec-return-type-hint 1
|
||||
:reduce-without-init 3
|
||||
:redundant-do 1
|
||||
:syntax 2
|
||||
:type-mismatch 8
|
||||
:uninitialized-var 1
|
||||
:unquote-not-syntax-quoted 1
|
||||
:unresolved-namespace 3
|
||||
:unresolved-symbol 11
|
||||
:unsorted-required-namespaces :unlimited
|
||||
:unused-alias 2
|
||||
:unused-binding 9
|
||||
:unused-excluded-var 2
|
||||
:unused-namespace 2
|
||||
:unused-private-var 8
|
||||
:unused-value 1}
|
||||
:config-counts {:constant-condition 1
|
||||
:deprecated-namespace 7
|
||||
:deprecated-var 16
|
||||
:discouraged-namespace 14
|
||||
:discouraged-var 14
|
||||
:inline-def 1
|
||||
:main-without-gen-class 1
|
||||
:metabase/defsetting-must-specify-export 1
|
||||
:metabase/defsetting-namespace 1
|
||||
:metabase/prefer-with-dynamic-fn-redefs 3
|
||||
:missing-docstring 1
|
||||
:private-call 1
|
||||
:redundant-ignore 1
|
||||
:redundant-primitive-coercion 1
|
||||
:refer-all 1
|
||||
:type-mismatch 1
|
||||
:unexpected-recur 1
|
||||
:unresolved-symbol 19
|
||||
:unresolved-var 1
|
||||
:unused-referred-var 4}
|
||||
:comment-exempt #{:deprecated-namespace
|
||||
:discouraged-namespace
|
||||
:discouraged-var
|
||||
:metabase/modules
|
||||
:metabase/validate-defendpoint-has-response-schema
|
||||
:metabase/validate-defendpoint-query-params-use-kebab-case
|
||||
:metabase/validate-defendpoint-route-uses-kebab-case}}
|
||||
|
||||
@@ -20,6 +20,23 @@
|
||||
:message error
|
||||
:type :metabase/modules))))))))
|
||||
|
||||
(defn- lint-dynamic-require* [node current-ns config]
|
||||
(when (and (not (modules/ignored-namespace? config current-ns))
|
||||
(modules/module current-ns))
|
||||
(hooks/reg-finding! (assoc (meta node)
|
||||
:message "Module dependency cannot be statically determined from a dynamic require"
|
||||
:type :metabase/modules))))
|
||||
|
||||
(defn- quoted-require? [node]
|
||||
(or (= (hooks/tag node) :quote)
|
||||
(and (= (hooks/tag node) :list)
|
||||
(let [[x] (:children node)]
|
||||
(and (hooks/token-node? x)
|
||||
(= (hooks/sexpr x) 'quote))))))
|
||||
|
||||
(defn- keyword-node? [node]
|
||||
(keyword? (hooks/sexpr node)))
|
||||
|
||||
(defn- unwrap-require [node]
|
||||
(case (hooks/tag node)
|
||||
;; some.namespace of some.namespace/some-var
|
||||
@@ -42,7 +59,11 @@
|
||||
|
||||
;; something like [some.namespace :as whatever]
|
||||
:vector
|
||||
(recur (first (:children node)))))
|
||||
(recur (first (:children node)))
|
||||
|
||||
;; anything else -- e.g. a runtime expression like `(requiring-resolve (:failing-test-var opts))` --
|
||||
;; has no statically-known namespace
|
||||
nil))
|
||||
|
||||
(comment
|
||||
(unwrap-require (-> "(requiring-resolve 'metabase.notification.payload.execute/execute-dashboard)"
|
||||
@@ -51,20 +72,24 @@
|
||||
second)))
|
||||
|
||||
(defn- lint-require* [node current-ns config]
|
||||
(let [[_require & args] (:children node)
|
||||
required-namespaces (->> (keep unwrap-require args)
|
||||
;; `require` runs through `cc/load-libs` which allows
|
||||
;; #{:as :reload :reload-all :require :use :verbose :refer :as-alias}
|
||||
;; as args. If you use a REPL tool that applies clj-kondo linting (e.g., Clojure MCP)
|
||||
;; you may wish to `:reload` or `:reload-all` a namespace. So we shouldn't assume all
|
||||
;; args are keywords:
|
||||
(remove keyword?))]
|
||||
(lint* node current-ns config required-namespaces)))
|
||||
(let [[_require & args] (:children node)
|
||||
;; Keywords such as `:reload` and `:reload-all` are options, not namespace dependencies.
|
||||
namespace-nodes (remove keyword-node? args)
|
||||
static-nodes (filter quoted-require? namespace-nodes)
|
||||
required-namespaces (->> (keep unwrap-require static-nodes)
|
||||
(filter simple-symbol?))]
|
||||
(lint* node current-ns config required-namespaces)
|
||||
(when (or (not= (count static-nodes) (count namespace-nodes))
|
||||
(not= (count required-namespaces) (count static-nodes)))
|
||||
(lint-dynamic-require* node current-ns config))))
|
||||
|
||||
(defn- lint-requiring-resolve* [node current-ns config]
|
||||
(let [[_requiring-resolve symb-node] (:children node)
|
||||
required-namespace (unwrap-require symb-node)]
|
||||
(lint* node current-ns config [required-namespace])))
|
||||
(if (and (quoted-require? symb-node)
|
||||
(simple-symbol? required-namespace))
|
||||
(lint* node current-ns config [required-namespace])
|
||||
(lint-dynamic-require* node current-ns config))))
|
||||
|
||||
(defn lint-require [x]
|
||||
(lint-require* (:node x) (:ns x) (modules/config x))
|
||||
|
||||
@@ -171,6 +171,7 @@
|
||||
(str/starts-with? (name ns-symb) prefix))
|
||||
["build-drivers."
|
||||
"build."
|
||||
"hooks." ; clj-kondo hook tests, outside the module system
|
||||
"i18n." ; bin/i18n
|
||||
"lint-migrations-file-test"
|
||||
"load-namespaces." ; bin/load-namespaces
|
||||
|
||||
@@ -286,6 +286,7 @@
|
||||
(update context :node update-node)))
|
||||
|
||||
(comment
|
||||
(require '[clojure.pprint])
|
||||
(defn- defsetting* [form]
|
||||
(hooks/sexpr
|
||||
(:node
|
||||
@@ -293,7 +294,6 @@
|
||||
{:node
|
||||
(hooks/parse-string
|
||||
(with-out-str
|
||||
#_{:clj-kondo/ignore [:unresolved-namespace]}
|
||||
(clojure.pprint/pprint
|
||||
form)))}))))
|
||||
|
||||
|
||||
65
.clj-kondo/test/hooks/clojure/core/require_test.clj
Normal file
@@ -0,0 +1,65 @@
|
||||
(ns hooks.clojure.core.require-test
|
||||
(:require
|
||||
[clj-kondo.hooks-api :as hooks]
|
||||
[clj-kondo.impl.utils]
|
||||
[clojure.test :refer :all]
|
||||
[hooks.clojure.core.require]))
|
||||
|
||||
(def ^:private config
|
||||
'{:linters {:metabase/modules {:level :warning}}
|
||||
:metabase/modules {search {:uses #{}}
|
||||
task {:api #{metabase.task.core}}}})
|
||||
|
||||
(defn- lint [hook form current-ns]
|
||||
(binding [clj-kondo.impl.utils/*ctx* {:config config
|
||||
:ignores (atom nil)
|
||||
:findings (atom [])
|
||||
:namespaces (atom {})}]
|
||||
(hook {:node (hooks/parse-string (pr-str form))
|
||||
:ns current-ns
|
||||
:config config})
|
||||
@(:findings clj-kondo.impl.utils/*ctx*)))
|
||||
|
||||
(defn- lint-require [form current-ns]
|
||||
(lint hooks.clojure.core.require/lint-require form current-ns))
|
||||
|
||||
(defn- lint-requiring-resolve [form current-ns]
|
||||
(lint hooks.clojure.core.require/lint-requiring-resolve form current-ns))
|
||||
|
||||
(def ^:private dynamic-require-finding
|
||||
{:message "Module dependency cannot be statically determined from a dynamic require"
|
||||
:type :metabase/modules})
|
||||
|
||||
(def ^:private cross-module-finding
|
||||
{:message "Module task should not be used in the search module. [:metabase/modules search :uses]"
|
||||
:type :metabase/modules})
|
||||
|
||||
(deftest ^:parallel dynamic-requiring-resolve-test
|
||||
(testing "outside the module system"
|
||||
(is (= [] (lint-requiring-resolve '(requiring-resolve (:failing-test-var opts)) 'dev))))
|
||||
(testing "inside a module"
|
||||
(are [form] (=? [dynamic-require-finding]
|
||||
(lint-requiring-resolve form 'metabase.search.core))
|
||||
'(requiring-resolve (:failing-test-var opts))
|
||||
'(requiring-resolve @some-atom)
|
||||
'(requiring-resolve some-local)
|
||||
'(requiring-resolve "not-a-symbol"))))
|
||||
|
||||
(deftest ^:parallel dynamic-require-test
|
||||
(testing "outside the module system"
|
||||
(is (= [] (lint-require '(require some-local) 'dev))))
|
||||
(testing "inside a module"
|
||||
(is (=? [dynamic-require-finding]
|
||||
(lint-require '(require some-local) 'metabase.search.core)))))
|
||||
|
||||
(deftest ^:parallel static-requiring-resolve-still-linted-test
|
||||
(is (=? [cross-module-finding]
|
||||
(lint-requiring-resolve '(requiring-resolve 'metabase.task.impl/go)
|
||||
'metabase.search.core))
|
||||
"a statically-known cross-module violation is still reported"))
|
||||
|
||||
(deftest ^:parallel static-require-still-linted-test
|
||||
(is (=? [cross-module-finding]
|
||||
(lint-require '(require 'metabase.task.impl :reload)
|
||||
'metabase.search.core))
|
||||
"a statically-known cross-module violation is still reported while keyword flags are ignored"))
|
||||
@@ -16,7 +16,7 @@ skip =
|
||||
frontend/src/metabase/querying/expressions/tokenizer/lezer.js,
|
||||
frontend/src/metabase/query_builder/components/NativeQueryEditor/CodeMirrorEditor/language.ts,
|
||||
frontend/src/metabase/static-viz/constants/char-sizes.ts,
|
||||
frontend/src/metabase/visualizations/lib/mapping_codes.ts,
|
||||
frontend/src/metabase/viz-core/lib/mapping_codes.ts,
|
||||
frontend/*/mocks/*,
|
||||
modules/drivers/mongo/src/metabase/driver/mongo/operators.clj,
|
||||
src/metabase/analyze/classifiers/name.clj,
|
||||
|
||||
13
.github/CODEOWNERS
vendored
@@ -42,6 +42,7 @@ docs/api.json # auto-generated
|
||||
docs/configuring-metabase/environment-variables.md # auto-generated
|
||||
docs/configuring-metabase/config-template.md # auto-generated
|
||||
docs/installation-and-operation/commands.md # auto-generated
|
||||
docs/ai/providers.md # auto-generated
|
||||
docs/embedding/sdk/api # auto-generated
|
||||
docs/embedding/eajs/snippets # auto-generated
|
||||
|
||||
@@ -54,3 +55,15 @@ src/metabase/analytics/prometheus.clj @metabase/cloud-ops
|
||||
# @metabase/metabot-frontend
|
||||
enterprise/frontend/src/metabase-enterprise/metabot @metabase/metabot-frontend
|
||||
frontend/src/metabase/metabot @metabase/metabot-frontend
|
||||
|
||||
# @metabase/core-frontend-gadget
|
||||
bin/custom-viz @metabase/core-frontend-gadget
|
||||
docs/developers-guide/custom-visualizations.md @metabase/core-frontend-gadget
|
||||
enterprise/frontend/src/custom-viz @metabase/core-frontend-gadget
|
||||
enterprise/frontend/src/metabase-enterprise/api/custom-viz-plugin.ts @metabase/core-frontend-gadget
|
||||
enterprise/frontend/src/metabase-enterprise/custom_viz @metabase/core-frontend-gadget
|
||||
frontend/src/metabase-types/api/custom-viz-plugin.ts @metabase/core-frontend-gadget
|
||||
frontend/src/metabase/plugins/oss/custom-viz.ts @metabase/core-frontend-gadget
|
||||
frontend/src/metabase/visualizations/custom-visualizations @metabase/core-frontend-gadget
|
||||
frontend/test/__support__/custom-viz-fixtures @metabase/core-frontend-gadget
|
||||
.github/workflows/release-custom-viz.yml @metabase/devexp @metabase/core-frontend-gadget
|
||||
|
||||
3
.github/actionlint.yaml
vendored
@@ -8,6 +8,9 @@ paths:
|
||||
# GitHub added parallel steps (`parallel`, `background`, `wait`, `wait-all`,
|
||||
# `cancel`) in June 2026, and actionlint 1.7.12 does not know the schema yet:
|
||||
# https://github.blog/changelog/2026-06-25-actions-steps-can-now-be-run-in-parallel/
|
||||
.github/workflows/backend.yml:
|
||||
ignore:
|
||||
- 'step must run script with "run" section or run action with "uses" section'
|
||||
.github/workflows/lint.yml:
|
||||
ignore:
|
||||
- 'step must run script with "run" section or run action with "uses" section'
|
||||
|
||||
9
.github/actions/create-backport/action.yml
vendored
@@ -25,6 +25,8 @@ runs:
|
||||
COMMIT: ${{ inputs.backport-commit }}
|
||||
shell: bash
|
||||
run: | # bash
|
||||
source "${{ github.action_path }}/kondo-ratchets.sh"
|
||||
|
||||
git config --global user.email "metabase-bot@metabase.com"
|
||||
git config --global user.name "Metabase bot"
|
||||
|
||||
@@ -34,7 +36,7 @@ runs:
|
||||
git checkout ${TARGET_BRANCH}
|
||||
git fetch --all
|
||||
git checkout -b ${BACKPORT_BRANCH}
|
||||
git cherry-pick ${COMMIT} || true
|
||||
cherry_pick_backport "${COMMIT}"
|
||||
|
||||
CONFLICT_LIST=$(git ls-files -u)
|
||||
|
||||
@@ -46,10 +48,7 @@ runs:
|
||||
echo "has-conflicts=true" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# Add a shell script for resolving conflicts
|
||||
echo "git reset HEAD~1" > ./backport.sh
|
||||
echo "rm ./backport.sh" >> ./backport.sh
|
||||
echo "git cherry-pick ${COMMIT}" >> ./backport.sh
|
||||
echo "echo 'Resolve conflicts and force push this branch.\n\nTo backport translations run: bin/i18n/merge-translations <release-branch>'" >> ./backport.sh
|
||||
write_backport_script "${COMMIT}"
|
||||
chmod +x ./backport.sh
|
||||
git add ./backport.sh
|
||||
git commit -m "Add backport resolution script"
|
||||
|
||||
375
.github/actions/create-backport/create-backport-test.sh
vendored
Executable file
@@ -0,0 +1,375 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
action_dir=$(cd "$(dirname "$0")" && pwd)
|
||||
cut_release_workflow="$action_dir/../../workflows/cut-release-branch.yml"
|
||||
|
||||
# shellcheck source=.github/actions/create-backport/kondo-ratchets.sh
|
||||
source "$action_dir/kondo-ratchets.sh"
|
||||
|
||||
# Any editor launch fails the run, so a continuation that waits on an editor cannot pass.
|
||||
export GIT_EDITOR=false
|
||||
|
||||
test_root=
|
||||
trap '[ -z "$test_root" ] || rm -rf -- "$test_root"' EXIT
|
||||
trap 'exit 129' HUP
|
||||
trap 'exit 130' INT
|
||||
trap 'exit 143' TERM
|
||||
test_root=$(mktemp -d)
|
||||
|
||||
active_ratchets='{:ignore-counts {:example 1}}'
|
||||
disabled_ratchets=';; Kondo ignore ratchets apply only to master; this release branch opts out.
|
||||
;; The test and fixer recognize this explicit opt-out.
|
||||
{:disabled true}'
|
||||
|
||||
# Coverage:
|
||||
#
|
||||
# Target state Incoming change Other conflict Expected result
|
||||
# ----------------- -------------------- --------------- ------------------------------------------
|
||||
# active ratchets unrelated file none leave ratchets unchanged
|
||||
# active ratchets ratchets none install the release opt-out
|
||||
# disabled ratchets ratchets none keep the opt-out; finish with an empty commit
|
||||
# divergent ratchets ratchets none resolve ratchets and finish the cherry-pick
|
||||
# divergent ratchets ratchets (on a pty) none finish without opening an editor
|
||||
# active ratchets unrelated file app.txt leave the conflict for manual resolution
|
||||
# divergent ratchets ratchets + app.txt app.txt resolve only ratchets
|
||||
# release cut ratchets none cut and backport commit the same opt-out
|
||||
# n/a cut-release workflow n/a the workflow calls the shared writer
|
||||
# any invalid commit n/a propagate the cherry-pick failure
|
||||
# divergent ratchets generated backport.sh app.txt resolve only ratchets through the manual script
|
||||
#
|
||||
# Known gaps: an older target with no ratchets file, and an incoming commit that
|
||||
# deletes the ratchets file.
|
||||
|
||||
test_unrelated_backport_preserves_target_ratchets() (
|
||||
local repo="$test_root/unrelated"
|
||||
init_repo "$repo"
|
||||
local commit
|
||||
commit=$(feature_commit "$repo" change_app)
|
||||
git -C "$repo" checkout -q release
|
||||
|
||||
cd "$repo"
|
||||
cherry_pick_backport "$commit"
|
||||
|
||||
assert_eq feature "$(cat app.txt)" "the unrelated change was not cherry-picked"
|
||||
assert_ratchets "$active_ratchets"
|
||||
assert_cherry_pick_completed
|
||||
assert_worktree_clean
|
||||
)
|
||||
|
||||
test_clean_ratchet_backport_installs_release_opt_out() (
|
||||
local repo="$test_root/clean-ratchet"
|
||||
init_repo "$repo"
|
||||
local commit
|
||||
commit=$(feature_commit "$repo" change_ratchets)
|
||||
git -C "$repo" checkout -q release
|
||||
|
||||
cd "$repo"
|
||||
cherry_pick_backport "$commit"
|
||||
|
||||
assert_ratchets_disabled
|
||||
assert_cherry_pick_completed
|
||||
assert_worktree_clean
|
||||
)
|
||||
|
||||
test_disabled_release_absorbs_ratchet_only_backport() (
|
||||
local repo="$test_root/disabled-release"
|
||||
init_repo "$repo"
|
||||
local commit
|
||||
commit=$(feature_commit "$repo" change_ratchets)
|
||||
git -C "$repo" checkout -q release
|
||||
commit_release_change "$repo" disable_release_ratchets "Disable release ratchets"
|
||||
|
||||
cd "$repo"
|
||||
cherry_pick_backport "$commit"
|
||||
|
||||
assert_ratchets_disabled
|
||||
assert_cherry_pick_completed
|
||||
assert_empty_commit
|
||||
assert_worktree_clean
|
||||
)
|
||||
|
||||
test_ratchet_only_conflict_is_resolved() (
|
||||
local repo="$test_root/ratchet-conflict"
|
||||
init_repo "$repo"
|
||||
local commit
|
||||
commit=$(conflicting_ratchet_commit "$repo")
|
||||
|
||||
cd "$repo"
|
||||
cherry_pick_backport "$commit"
|
||||
|
||||
assert_ratchets_disabled
|
||||
assert_cherry_pick_completed
|
||||
assert_worktree_clean
|
||||
)
|
||||
|
||||
test_continuation_never_opens_an_editor() (
|
||||
local repo="$test_root/editor"
|
||||
init_repo "$repo"
|
||||
local commit
|
||||
commit=$(conflicting_ratchet_commit "$repo")
|
||||
|
||||
cd "$repo"
|
||||
# Git only opens an editor for the continuation when stdin is a terminal, so run it on a pty. The
|
||||
# exported GIT_EDITOR=false turns any editor launch into a failure. The single quotes are deliberate:
|
||||
# $1 and $2 are for the inner bash, not this shell.
|
||||
# shellcheck disable=SC2016
|
||||
python3 -c 'import os, pty, sys; sys.exit(os.waitstatus_to_exitcode(pty.spawn(sys.argv[1:])))' \
|
||||
bash -c 'source "$1/kondo-ratchets.sh" && cherry_pick_backport "$2"' bash "$action_dir" "$commit" >/dev/null ||
|
||||
fail "the cherry-pick continuation opened an editor"
|
||||
|
||||
assert_ratchets_disabled
|
||||
assert_cherry_pick_completed
|
||||
)
|
||||
|
||||
test_non_ratchet_conflict_remains_manual() (
|
||||
local repo="$test_root/non-ratchet-conflict"
|
||||
init_repo "$repo"
|
||||
local commit
|
||||
commit=$(feature_commit "$repo" change_app)
|
||||
git -C "$repo" checkout -q release
|
||||
commit_release_change "$repo" change_release_app "Change release app"
|
||||
|
||||
cd "$repo"
|
||||
cherry_pick_backport "$commit"
|
||||
|
||||
assert_ratchets "$active_ratchets"
|
||||
assert_only_conflicts app.txt
|
||||
assert_cherry_pick_pending
|
||||
)
|
||||
|
||||
test_mixed_conflict_resolves_only_ratchets() (
|
||||
local repo="$test_root/mixed-conflict"
|
||||
init_repo "$repo"
|
||||
local commit
|
||||
commit=$(feature_commit "$repo" change_app_and_ratchets)
|
||||
git -C "$repo" checkout -q release
|
||||
commit_release_change "$repo" change_release_app_and_ratchets "Change release app and ratchets"
|
||||
|
||||
cd "$repo"
|
||||
cherry_pick_backport "$commit"
|
||||
|
||||
assert_ratchets_disabled
|
||||
assert_only_conflicts app.txt
|
||||
assert_cherry_pick_pending
|
||||
)
|
||||
|
||||
test_release_cut_and_backport_commit_the_same_opt_out() (
|
||||
local cut_repo="$test_root/release-cut"
|
||||
init_repo "$cut_repo"
|
||||
cd "$cut_repo"
|
||||
git checkout -qb release-x.99.x
|
||||
write_disabled_ratchets
|
||||
git add --sparse .clj-kondo/ratchets.edn
|
||||
git commit -q --allow-empty --no-verify -m "Cut release-x.99.x branch"
|
||||
git show HEAD:.clj-kondo/ratchets.edn > "$test_root/release-cut.edn"
|
||||
|
||||
local backport_repo="$test_root/release-cut-backport"
|
||||
init_repo "$backport_repo"
|
||||
local commit
|
||||
commit=$(feature_commit "$backport_repo" change_ratchets)
|
||||
git -C "$backport_repo" checkout -q release
|
||||
cd "$backport_repo"
|
||||
cherry_pick_backport "$commit"
|
||||
git show HEAD:.clj-kondo/ratchets.edn > "$test_root/backport.edn"
|
||||
|
||||
cmp -s "$test_root/release-cut.edn" "$test_root/backport.edn" ||
|
||||
fail "the release cut and the backport committed different opt-out files"
|
||||
assert_eq "$disabled_ratchets" "$(cat "$test_root/release-cut.edn")" \
|
||||
"the release cut did not install the expected opt-out"
|
||||
)
|
||||
|
||||
test_cut_release_workflow_uses_the_shared_writer() (
|
||||
grep -Fq 'source .github/actions/create-backport/kondo-ratchets.sh' "$cut_release_workflow" ||
|
||||
fail "the cut-release workflow does not source kondo-ratchets.sh"
|
||||
grep -Fq 'write_disabled_ratchets' "$cut_release_workflow" ||
|
||||
fail "the cut-release workflow does not call write_disabled_ratchets"
|
||||
! grep -Fq ':disabled' "$cut_release_workflow" ||
|
||||
fail "the cut-release workflow carries its own copy of the opt-out"
|
||||
)
|
||||
|
||||
test_invalid_commit_failure_is_propagated() (
|
||||
local repo="$test_root/invalid"
|
||||
init_repo "$repo"
|
||||
cd "$repo"
|
||||
|
||||
if cherry_pick_backport deadbeef; then
|
||||
fail "an invalid commit succeeded"
|
||||
fi
|
||||
)
|
||||
|
||||
test_generated_script_resolves_ratchets_end_to_end() (
|
||||
local repo="$test_root/generated-script"
|
||||
init_repo "$repo"
|
||||
local commit
|
||||
commit=$(feature_commit "$repo" change_app_and_ratchets)
|
||||
git -C "$repo" checkout -q release
|
||||
commit_release_change "$repo" change_release_app_and_ratchets "Change release app and ratchets"
|
||||
|
||||
cd "$repo"
|
||||
cherry_pick_backport "$commit"
|
||||
git cherry-pick --abort
|
||||
|
||||
local release_commit
|
||||
release_commit=$(git rev-parse HEAD)
|
||||
write_backport_script "$commit"
|
||||
chmod +x backport.sh
|
||||
git add backport.sh
|
||||
git commit -qm "Add manual backport script"
|
||||
|
||||
bash -n backport.sh
|
||||
grep -Fq 'write_disabled_ratchets ()' backport.sh || fail "the generated script omitted write_disabled_ratchets"
|
||||
./backport.sh
|
||||
|
||||
assert_ratchets_disabled
|
||||
assert_only_conflicts app.txt
|
||||
assert_cherry_pick_pending
|
||||
assert_eq "$release_commit" "$(git rev-parse HEAD)" "the generated script did not return HEAD to the release commit"
|
||||
[ ! -e backport.sh ] || fail "the generated script did not remove itself"
|
||||
! git ls-files --error-unmatch backport.sh >/dev/null 2>&1 || fail "the generated script is still tracked"
|
||||
! git status --porcelain | grep -q backport.sh || fail "the generated script left a status entry"
|
||||
)
|
||||
|
||||
# Test harness
|
||||
|
||||
fail() {
|
||||
echo "FAIL: $*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
assert_eq() {
|
||||
local expected=$1
|
||||
local actual=$2
|
||||
local message=$3
|
||||
[ "$expected" = "$actual" ] || fail "$message (expected '$expected', got '$actual')"
|
||||
}
|
||||
|
||||
assert_ratchets() {
|
||||
assert_eq "$1" "$(cat .clj-kondo/ratchets.edn)" "unexpected ratchet state"
|
||||
}
|
||||
|
||||
assert_ratchets_disabled() {
|
||||
assert_ratchets "$disabled_ratchets"
|
||||
}
|
||||
|
||||
assert_cherry_pick_completed() {
|
||||
! git rev-parse -q --verify CHERRY_PICK_HEAD >/dev/null || fail "the cherry-pick is still pending"
|
||||
}
|
||||
|
||||
assert_cherry_pick_pending() {
|
||||
git rev-parse -q --verify CHERRY_PICK_HEAD >/dev/null || fail "the cherry-pick state was lost"
|
||||
}
|
||||
|
||||
assert_only_conflicts() {
|
||||
local expected
|
||||
local actual
|
||||
expected=$(printf '%s\n' "$@" | sort)
|
||||
actual=$(git diff --name-only --diff-filter=U | sort)
|
||||
assert_eq "$expected" "$actual" "unexpected unresolved files"
|
||||
}
|
||||
|
||||
assert_empty_commit() {
|
||||
git diff --quiet HEAD^ HEAD || fail "the backport commit was not empty"
|
||||
}
|
||||
|
||||
assert_worktree_clean() {
|
||||
[ -z "$(git status --porcelain)" ] || fail "the backport left worktree changes"
|
||||
}
|
||||
|
||||
init_repo() {
|
||||
local repo=$1
|
||||
git init -q -b release "$repo"
|
||||
git -C "$repo" config user.email test@example.com
|
||||
git -C "$repo" config user.name "Backport test"
|
||||
printf 'base\n' > "$repo/app.txt"
|
||||
mkdir -p "$repo/.clj-kondo"
|
||||
printf '%s\n' "$active_ratchets" > "$repo/.clj-kondo/ratchets.edn"
|
||||
git -C "$repo" add .
|
||||
git -C "$repo" commit -qm "Base"
|
||||
}
|
||||
|
||||
feature_commit() {
|
||||
local repo=$1
|
||||
shift
|
||||
git -C "$repo" checkout -qb feature
|
||||
"$@" "$repo"
|
||||
git -C "$repo" add .
|
||||
git -C "$repo" commit -qm "Feature"
|
||||
git -C "$repo" rev-parse HEAD
|
||||
}
|
||||
|
||||
# Sets up a release branch whose ratchets conflict with the feature commit, and prints that commit.
|
||||
conflicting_ratchet_commit() {
|
||||
local repo=$1
|
||||
local commit
|
||||
commit=$(feature_commit "$repo" change_ratchets)
|
||||
git -C "$repo" checkout -q release
|
||||
commit_release_change "$repo" change_release_ratchets "Change release ratchets"
|
||||
echo "$commit"
|
||||
}
|
||||
|
||||
change_app() {
|
||||
printf 'feature\n' > "$1/app.txt"
|
||||
}
|
||||
|
||||
change_ratchets() {
|
||||
printf '{:ignore-counts {:example 2}}\n' > "$1/.clj-kondo/ratchets.edn"
|
||||
}
|
||||
|
||||
change_app_and_ratchets() {
|
||||
change_app "$1"
|
||||
change_ratchets "$1"
|
||||
}
|
||||
|
||||
change_release_app() {
|
||||
printf 'release\n' > "$1/app.txt"
|
||||
}
|
||||
|
||||
change_release_ratchets() {
|
||||
printf '{:ignore-counts {:release 1}}\n' > "$1/.clj-kondo/ratchets.edn"
|
||||
}
|
||||
|
||||
change_release_app_and_ratchets() {
|
||||
change_release_app "$1"
|
||||
change_release_ratchets "$1"
|
||||
}
|
||||
|
||||
# Setup writes the opt-out through the shared writer; the only copy is the expected text at the top.
|
||||
disable_release_ratchets() {
|
||||
(cd "$1" && write_disabled_ratchets)
|
||||
}
|
||||
|
||||
commit_release_change() {
|
||||
local repo=$1
|
||||
local change=$2
|
||||
local message=$3
|
||||
"$change" "$repo"
|
||||
git -C "$repo" add .
|
||||
git -C "$repo" commit -qm "$message"
|
||||
}
|
||||
|
||||
run_test() {
|
||||
local description=$1
|
||||
local test=${description// /_}
|
||||
test=${test//-/_}
|
||||
test="test_$test"
|
||||
|
||||
declare -F "$test" >/dev/null || fail "missing test function: $test"
|
||||
|
||||
printf 'TEST: %s\n' "$description"
|
||||
"$test"
|
||||
printf 'PASS: %s\n' "$description"
|
||||
}
|
||||
|
||||
run_test "unrelated backport preserves target ratchets"
|
||||
run_test "clean ratchet backport installs release opt-out"
|
||||
run_test "disabled release absorbs ratchet-only backport"
|
||||
run_test "ratchet-only conflict is resolved"
|
||||
run_test "continuation never opens an editor"
|
||||
run_test "non-ratchet conflict remains manual"
|
||||
run_test "mixed conflict resolves only ratchets"
|
||||
run_test "release cut and backport commit the same opt-out"
|
||||
run_test "cut-release workflow uses the shared writer"
|
||||
run_test "invalid commit failure is propagated"
|
||||
run_test "generated script resolves ratchets end-to-end"
|
||||
74
.github/actions/create-backport/kondo-ratchets.sh
vendored
Normal file
@@ -0,0 +1,74 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# The only place that knows what a release branch's opt-out looks like. The cut-release workflow, the
|
||||
# backport helper, and the tests all go through this function.
|
||||
write_disabled_ratchets() {
|
||||
mkdir -p .clj-kondo
|
||||
printf '%s\n' \
|
||||
';; Kondo ignore ratchets apply only to master; this release branch opts out.' \
|
||||
';; The test and fixer recognize this explicit opt-out.' \
|
||||
'{:disabled true}' \
|
||||
> .clj-kondo/ratchets.edn
|
||||
}
|
||||
|
||||
disable_ratchets() {
|
||||
local commit=$1
|
||||
local file=.clj-kondo/ratchets.edn
|
||||
[ -n "$(git diff-tree --root --no-commit-id --name-only -r "$commit" -- "$file")" ] || return 0
|
||||
|
||||
echo "$file: disabling kondo ratchets on the release branch"
|
||||
write_disabled_ratchets
|
||||
git add -- "$file"
|
||||
|
||||
if ! git rev-parse -q --verify CHERRY_PICK_HEAD >/dev/null; then
|
||||
if ! git diff --cached --quiet HEAD; then
|
||||
git commit --amend --no-edit
|
||||
fi
|
||||
return 0
|
||||
fi
|
||||
|
||||
if [ -z "$(git ls-files -u)" ]; then
|
||||
echo "$file was the only conflict, finishing the cherry-pick"
|
||||
if git diff --cached --quiet HEAD; then
|
||||
echo "nothing left to apply, committing the backport empty"
|
||||
git commit --allow-empty --no-edit
|
||||
else
|
||||
# GIT_EDITOR outranks core.editor, VISUAL, and EDITOR, so the continuation can never wait on
|
||||
# an editor; the repository's own editor configuration is left alone.
|
||||
GIT_EDITOR=true git cherry-pick --continue
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
cherry_pick_backport() {
|
||||
local commit=$1
|
||||
local status=0
|
||||
git cherry-pick "$commit" || status=$?
|
||||
if [ "$status" -ne 0 ] && ! git rev-parse -q --verify CHERRY_PICK_HEAD >/dev/null; then
|
||||
echo "cherry-pick failed without entering a resolvable conflict state" >&2
|
||||
return "$status"
|
||||
fi
|
||||
|
||||
disable_ratchets "$commit"
|
||||
if [ "$status" -ne 0 ] &&
|
||||
git rev-parse -q --verify CHERRY_PICK_HEAD >/dev/null &&
|
||||
[ -z "$(git ls-files -u)" ]; then
|
||||
echo "cherry-pick stopped without a file conflict to resolve" >&2
|
||||
return "$status"
|
||||
fi
|
||||
}
|
||||
|
||||
write_backport_script() {
|
||||
local commit=$1
|
||||
{
|
||||
echo "#!/usr/bin/env bash"
|
||||
echo "set -euo pipefail"
|
||||
echo "git reset HEAD~1"
|
||||
echo "rm ./backport.sh"
|
||||
declare -f write_disabled_ratchets disable_ratchets cherry_pick_backport
|
||||
printf 'cherry_pick_backport %q\n' "$commit"
|
||||
printf '%s\n' "printf 'Resolve conflicts and force push this branch.\\n\\nTo backport translations run: bin/i18n/merge-translations <release-branch>\\n'"
|
||||
} > ./backport.sh
|
||||
}
|
||||
@@ -93,8 +93,8 @@ runs:
|
||||
|
||||
if ${{ inputs.maildev }}; then
|
||||
echo -e "${Y}checking maildev container..." &&
|
||||
while ! nc -z localhost 1080; do sleep 1; done &&
|
||||
while ! nc -z localhost 1025; do sleep 1; done &&
|
||||
while ! nc -z localhost 1180; do sleep 1; done &&
|
||||
while ! nc -z localhost 1125; do sleep 1; done &&
|
||||
echo -e "${G}Maildev is up and running!"
|
||||
|
||||
echo -e "${Y}checking maildev-ssl container..." &&
|
||||
|
||||
40
.github/actions/prepare-backend/action.yml
vendored
@@ -5,9 +5,6 @@ inputs:
|
||||
java-version:
|
||||
required: true
|
||||
default: "25"
|
||||
clojure-version:
|
||||
required: true
|
||||
default: "1.12.0.1488"
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
@@ -17,12 +14,40 @@ runs:
|
||||
with:
|
||||
java-version: ${{ inputs.java-version }}
|
||||
distribution: "temurin"
|
||||
- name: Cache spec
|
||||
id: cache-spec
|
||||
shell: bash
|
||||
run: .github/scripts/cache-keys.sh | tee -a "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Restore Clojure CLI
|
||||
uses: actions/cache/restore@v5
|
||||
with:
|
||||
path: ${{ steps.cache-spec.outputs.clojure-path }}
|
||||
key: ${{ steps.cache-spec.outputs.clojure-key }}
|
||||
|
||||
- name: Install Clojure CLI
|
||||
shell: bash
|
||||
env:
|
||||
CLOJURE_PREFIX: ${{ steps.cache-spec.outputs.clojure-path }}
|
||||
CLOJURE_VERSION: ${{ steps.cache-spec.outputs.clojure-version }}
|
||||
run: |
|
||||
set -uo pipefail
|
||||
echo "$CLOJURE_PREFIX/bin" >> "$GITHUB_PATH"
|
||||
export PATH="$CLOJURE_PREFIX/bin:$PATH"
|
||||
|
||||
# Whether the prefix already holds the right install is decided by running it, not by the
|
||||
# cache-hit flag. The installer bakes the prefix into the clj/clojure shims, so a tree restored
|
||||
# under a different $HOME points at a directory that is not there; and on a runner that is not
|
||||
# ephemeral, a previous version's prefix outlives the cache entry that stopped matching it.
|
||||
if clojure --version | grep -qF "$CLOJURE_VERSION"; then
|
||||
clojure --version
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cd "$RUNNER_TEMP"
|
||||
for attempt in 1 2 3; do
|
||||
curl --retry 3 --retry-delay 5 --fail -O https://download.clojure.org/install/linux-install-${{ inputs.clojure-version }}.sh \
|
||||
&& sudo bash ./linux-install-${{ inputs.clojure-version }}.sh \
|
||||
curl --retry 3 --retry-delay 5 --fail -O "https://download.clojure.org/install/linux-install-$CLOJURE_VERSION.sh" \
|
||||
&& bash "./linux-install-$CLOJURE_VERSION.sh" --prefix "$CLOJURE_PREFIX" \
|
||||
&& break
|
||||
echo "::warning::Clojure install attempt $attempt failed, retrying in 10s..."
|
||||
sleep 10
|
||||
@@ -39,11 +64,6 @@ runs:
|
||||
version: "0.10.2" # keep in sync with mise.toml
|
||||
github-token: ${{ github.token }}
|
||||
|
||||
- name: Cache spec
|
||||
id: cache-spec
|
||||
shell: bash
|
||||
run: .github/scripts/cache-keys.sh | tee -a "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Restore M2 cache
|
||||
uses: actions/cache/restore@v5
|
||||
with:
|
||||
|
||||
53
.github/file-paths.yaml
vendored
@@ -7,9 +7,8 @@ default: &default
|
||||
- "locales/**"
|
||||
- "bin/**/!(*.md)"
|
||||
|
||||
actionlint:
|
||||
- added|modified: ".github/workflows/*.yml"
|
||||
- added|modified: ".github/workflows/*.yaml"
|
||||
workflows:
|
||||
- added|modified: ".github/workflows/*.{yml,yaml}"
|
||||
|
||||
shared_sources: &shared_sources
|
||||
- "src/**/*.cljc"
|
||||
@@ -65,6 +64,9 @@ frontend_specs: &frontend_specs
|
||||
- "frontend/**/*.unit.*"
|
||||
- "enterprise/frontend/**/*.unit.*"
|
||||
- ".github/**/*.unit.spec.*"
|
||||
# file-paths.unit.spec.ts asserts what this file's own patterns match, so a change to it has to
|
||||
# run the specs that guard it.
|
||||
- ".github/file-paths.yaml"
|
||||
- "jest.config.js"
|
||||
- "jest.tz.unit.conf.json"
|
||||
|
||||
@@ -113,11 +115,15 @@ backend_sources: &backend_sources
|
||||
- "src/**"
|
||||
- "modules/drivers/{*,*/{*,!(test)/**}}"
|
||||
- "resources/**"
|
||||
- ".clj-kondo/**"
|
||||
# The ratchets file drives project_ratchet_checks alone, so the automation PR that lowers the budgets
|
||||
# runs the Project tests job by itself. Patterns are OR-ed, so a separate "!..." entry would match
|
||||
# every other path instead of excluding this one.
|
||||
- ".clj-kondo/{!(ratchets.edn),*/**}"
|
||||
|
||||
backend_specs: &backend_specs
|
||||
- *shared_specs
|
||||
- "test/**"
|
||||
- "dev/test/**"
|
||||
- "enterprise/backend/test/**"
|
||||
- "test_modules/**"
|
||||
- "modules/drivers/*/test/**"
|
||||
@@ -130,6 +136,45 @@ backend_all: &backend_all
|
||||
- *backend_sources
|
||||
- *backend_specs
|
||||
|
||||
# Project checks have narrower triggers than the backend matrix.
|
||||
# Changes to their shared CI entry points exercise every suite; otherwise each
|
||||
# suite follows only the files that can affect its result.
|
||||
project_tests_infra: &project_tests_infra
|
||||
- ".github/actions/prepare-backend/**"
|
||||
- ".github/file-paths.yaml"
|
||||
- ".github/workflows/backend.yml"
|
||||
- ".github/workflows/run-tests.yml"
|
||||
- "bb.edn"
|
||||
- "bin/mage"
|
||||
- "bin/mage.bb"
|
||||
- "mage/src/mage/color.clj"
|
||||
- "mage/src/mage/project_tests.clj"
|
||||
- "mage/src/mage/shell.clj"
|
||||
- "mage/src/mage/util.clj"
|
||||
|
||||
project_backend_checks:
|
||||
- *project_tests_infra
|
||||
- "{src,test,enterprise,dev,bin,mage}/**/*.{clj,cljc,cljs}"
|
||||
- "modules/**/*.clj"
|
||||
- "modules/**/*.cljc"
|
||||
- "modules/**/*.cljs"
|
||||
- "**deps.edn"
|
||||
- ".clj-kondo/config.edn"
|
||||
- ".clj-kondo/config/modules/config.edn"
|
||||
- ".github/team.json"
|
||||
- "dev/resources/dev/modules_config_test/**"
|
||||
|
||||
# Only the ratchet check reads the budgets, so a diff that touches nothing else runs that suite alone.
|
||||
project_ratchet_checks:
|
||||
- ".clj-kondo/ratchets.edn"
|
||||
|
||||
project_migration_checks:
|
||||
- *project_tests_infra
|
||||
- "bin/lint-migrations-file/**"
|
||||
- "resources/migrations/**/*.{yaml,yml}"
|
||||
- "src/metabase/app_db/custom_migrations.clj"
|
||||
- "src/metabase/app_db/custom_migrations/**"
|
||||
|
||||
sources: &sources
|
||||
- *shared_sources
|
||||
- *frontend_sources
|
||||
|
||||
19
.github/scripts/cache-keys.sh
vendored
@@ -50,7 +50,11 @@ digest() {
|
||||
}
|
||||
|
||||
shopt -s nullglob
|
||||
DEPS_FILES=(deps.edn modules/drivers/deps.edn modules/drivers/*/deps.edn)
|
||||
DEPS_FILES=(deps.edn
|
||||
bin/lint-migrations-file/deps.edn
|
||||
bin/load-namespaces/deps.edn
|
||||
modules/drivers/deps.edn
|
||||
modules/drivers/*/deps.edn)
|
||||
shopt -u nullglob
|
||||
|
||||
DEPS_HASH="$(digest "${DEPS_FILES[@]}")"
|
||||
@@ -70,6 +74,10 @@ if [ "$CYPRESS_MATCHES" != "1" ]; then
|
||||
fi
|
||||
CYPRESS_VERSION="$CYPRESS_VERSIONS"
|
||||
|
||||
# The Clojure CLI version CI installs. It lives here rather than in prepare-backend so that the version
|
||||
# and the key that identifies its cache entry cannot drift apart.
|
||||
CLOJURE_VERSION="1.12.0.1488"
|
||||
|
||||
# An incremental lint cache is only ever an accelerator, so it keys on the commit and always falls back
|
||||
# to the newest entry on the prefix. The exact key never pre-exists, which keeps the entry current.
|
||||
ESLINT_SHA="${GITHUB_SHA:-$(git rev-parse HEAD)}"
|
||||
@@ -96,6 +104,15 @@ spec() {
|
||||
echo "bun-store-key=bun-store-$OS-$LOCK_HASH"
|
||||
echo "bun-store-restore-key=bun-store-$OS-"
|
||||
|
||||
# No restore-key: an entry from another version installs a working `clojure` of the wrong version, and
|
||||
# the freshness check in prepare-backend - running the binary - would accept it.
|
||||
#
|
||||
# The installer bakes this prefix into the clj/clojure shims, so the tree only works when restored to
|
||||
# the same absolute path it was built at; prepare-backend reinstalls when it is not.
|
||||
echo "clojure-path=$HOME/.clojure-cli"
|
||||
echo "clojure-key=clojure-cli-$OS-$CLOJURE_VERSION"
|
||||
echo "clojure-version=$CLOJURE_VERSION"
|
||||
|
||||
echo "cypress-path=$HOME/.cache/Cypress"
|
||||
echo "cypress-key=cypress-$OS-$CYPRESS_VERSION"
|
||||
echo "cypress-restore-key=cypress-$OS-"
|
||||
|
||||
71
.github/scripts/check-job-timeouts.sh
vendored
Executable file
@@ -0,0 +1,71 @@
|
||||
#!/usr/bin/env bash
|
||||
# Fails when a job in .github/workflows/ does not set `timeout-minutes`.
|
||||
#
|
||||
# Without it a job inherits GitHub's 6-hour default, so a hung job burns six hours of runner time -
|
||||
# and holds up everything queued behind it - before anyone notices. actionlint cannot check this: its
|
||||
# rule set is compiled in and .github/actionlint.yaml only takes `self-hosted-runner`,
|
||||
# `config-variables`, and `paths` (which suppresses findings rather than adding them). Hence this
|
||||
# script.
|
||||
#
|
||||
# The rule is per job: a `timeout-minutes` on a step does not satisfy it, and steps are not required
|
||||
# to carry one of their own.
|
||||
#
|
||||
# Jobs that call a reusable workflow (`uses:` at the job level) are skipped - GitHub rejects
|
||||
# `timeout-minutes` there. The called workflow's own jobs are checked when it lives in this repo.
|
||||
#
|
||||
# .github/scripts/check-job-timeouts.sh [workflow-file ...]
|
||||
#
|
||||
# With no arguments it checks every workflow; CI passes the files that changed, like actionlint does.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# Arguments are used as given, so the paths in the annotations match what the caller passed. Without
|
||||
# them the full scan has to run from the repo root.
|
||||
if [ $# -gt 0 ]; then
|
||||
workflows=("$@")
|
||||
else
|
||||
cd "$(git rev-parse --show-toplevel)"
|
||||
|
||||
shopt -s nullglob
|
||||
workflows=(.github/workflows/*.yml .github/workflows/*.yaml)
|
||||
|
||||
if [ ${#workflows[@]} -eq 0 ]; then
|
||||
echo "::error::check-job-timeouts.sh: no workflow files found under .github/workflows/" >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
status=0
|
||||
|
||||
for file in "${workflows[@]}"; do
|
||||
# `awk NF` drops the blank line yq emits when a file has nothing to report.
|
||||
missing="$(
|
||||
# --header-preprocess=false: yq otherwise slurps any leading comment block before parsing, which
|
||||
# shifts every line number it reports by the size of that block.
|
||||
yq --header-preprocess=false '
|
||||
.jobs
|
||||
| to_entries
|
||||
| .[]
|
||||
| select(((.value | has("timeout-minutes")) or (.value | has("uses"))) | not)
|
||||
| [(.key | line), .key]
|
||||
| @tsv
|
||||
' "$file" | awk NF
|
||||
)"
|
||||
|
||||
[ -n "$missing" ] || continue
|
||||
|
||||
while IFS=$'\t' read -r line job; do
|
||||
echo "::error file=${file},line=${line}::Job \"${job}\" does not set a job-level timeout-minutes" >&2
|
||||
done <<< "$missing"
|
||||
|
||||
status=1
|
||||
done
|
||||
|
||||
if [ "$status" -ne 0 ]; then
|
||||
echo >&2
|
||||
echo "Every job must set an explicit timeout-minutes; without one it inherits GitHub's 6-hour default." >&2
|
||||
echo "Pick a value with headroom over the job's normal runtime." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "All jobs in the ${#workflows[@]} workflow file(s) checked set timeout-minutes."
|
||||
68
.github/scripts/check-preresolve-aliases.sh
vendored
@@ -1,6 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# Fails when a `clojure` invocation under .github/ uses an alias combination that cache-warm does not
|
||||
# pre-resolve.
|
||||
# Fails when a Clojure classpath used by CI is not pre-resolved by cache-warm.
|
||||
#
|
||||
# tools.deps selects dependency versions globally per classpath, so two alias combinations can resolve
|
||||
# the same transitive library to different versions. A combination the warm job never resolved is
|
||||
@@ -30,10 +29,18 @@ EOF
|
||||
# :deps - tools.deps built-in used by write-poms.sh; contributes no dependencies of its own.
|
||||
# :outdated - scheduled dependency-bump workflow only; a cold resolve there costs nothing anyone waits on.
|
||||
|
||||
# Alias combinations the warm job pre-resolves.
|
||||
declared="$(sed -n '/combos=(/,/^ *)/p' "$WARM" | grep -oE '":[^"]*"' | tr -d '"' | sort -u)"
|
||||
# Dependency roots and alias combinations the warm job pre-resolves.
|
||||
declared="$({
|
||||
sed -n '/^[[:space:]]*combos=(/,/^ *)/p' "$WARM" \
|
||||
| grep -oE '":[^"]*"' \
|
||||
| tr -d '"' \
|
||||
| sed 's#^#.|#'
|
||||
sed -n '/^[[:space:]]*nested_combos=(/,/^ *)/p' "$WARM" \
|
||||
| grep -oE '"[^"]+\|:[^"]+"' \
|
||||
| tr -d '"'
|
||||
} | sort -u)"
|
||||
if [ -z "$declared" ]; then
|
||||
echo "::error::check-preresolve-aliases.sh: found no combos=( ... ) list in $WARM" >&2
|
||||
echo "::error::check-preresolve-aliases.sh: found no pre-resolved classpaths in $WARM" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -43,27 +50,60 @@ unknown="$(grep -rhoE '\$\{\{[^}]*\}\}' --include='*' \
|
||||
--exclude='*.png' --exclude='*.jpg' .github/ 2>/dev/null \
|
||||
| sort -u | grep -v 'edition' || true)"
|
||||
|
||||
scan() { # scan <edition>: alias strings used by clojure invocations, with expressions substituted
|
||||
scan() { # scan <edition>: dependency root and aliases used by .github Clojure invocations
|
||||
grep -rhE 'clojure[[:space:]]+(-P[[:space:]]+)?-[XMATP]?:' \
|
||||
--include='*' --exclude='*.png' --exclude='*.jpg' .github/ 2>/dev/null \
|
||||
| sed "s/\\\${{[^}]*edition[^}]*}}/$1/g" \
|
||||
| grep -oE 'clojure[[:space:]]+(-P[[:space:]]+)?-[XMATP]?:[A-Za-z0-9:_.-]+' \
|
||||
| grep -oE ':[A-Za-z0-9:_.-]+' \
|
||||
| sed 's/:$//'
|
||||
| grep -oE 'clojure[[:space:]]+(-P[[:space:]]+)?-[XMATP]?:[A-Za-z0-9:_./-]+' \
|
||||
| grep -oE ':[A-Za-z0-9:_./-]+' \
|
||||
| sed -e 's/:$//' -e 's#^#.|#'
|
||||
}
|
||||
|
||||
found="$( { scan ee; scan oss; } | sort -u )"
|
||||
scan_project_tests() { # scan_project_tests <function> <dependency-root>
|
||||
# Mage shell commands are vectors split across lines, so flatten each function body before
|
||||
# extracting its alias.
|
||||
sed -n "/(defn- $1 /,/^(defn/p" mage/src/mage/project_tests.clj \
|
||||
| tr '\n' ' ' \
|
||||
| grep -oE '"clojure"[[:space:]]+("-P"[[:space:]]+)?"-[XMATP]?:[A-Za-z0-9:_./-]+' \
|
||||
| grep -oE ':[A-Za-z0-9:_./-]+' \
|
||||
| sed -e 's/:$//' -e "s#^#$2|#"
|
||||
}
|
||||
|
||||
covered="$(printf '%s\n%s\n' "$declared" "$WAIVED" | sort -u)"
|
||||
# Check each scan separately. One scan still finding commands would otherwise conceal that another has
|
||||
# stopped matching, which silently drops everything it used to cover.
|
||||
require_scan() { # require_scan <what> <result>
|
||||
[ -n "$2" ] && return
|
||||
echo "::error::check-preresolve-aliases.sh: found no Clojure invocation in $1; the scan pattern no longer matches" >&2
|
||||
exit 1
|
||||
}
|
||||
workflow_checks=$( { scan ee; scan oss; } || true)
|
||||
clojure_checks=$(scan_project_tests run-clojure-checks! . || true)
|
||||
migration_checks=$(scan_project_tests run-migration-checks! bin/lint-migrations-file || true)
|
||||
require_scan ".github/" "$workflow_checks"
|
||||
require_scan "run-clojure-checks! (mage/src/mage/project_tests.clj)" "$clojure_checks"
|
||||
require_scan "run-migration-checks! (mage/src/mage/project_tests.clj)" "$migration_checks"
|
||||
|
||||
scanned="$(printf '%s\n' "$workflow_checks" "$clojure_checks" "$migration_checks" | sort -u)"
|
||||
|
||||
found="$( {
|
||||
printf '%s\n' "$scanned"
|
||||
# build-scripts.yml runs this classpath from its own dependency root.
|
||||
printf '%s\n' 'bin/load-namespaces|:test'
|
||||
} | sort -u )"
|
||||
|
||||
covered="$( {
|
||||
printf '%s\n' "$declared"
|
||||
printf '%s\n' "$WAIVED" | sed 's#^#.|#'
|
||||
} | sort -u)"
|
||||
missing="$(comm -23 <(printf '%s\n' "$found") <(printf '%s\n' "$covered") || true)"
|
||||
|
||||
if [ -n "$missing" ]; then
|
||||
echo "::error::Alias combinations used under .github/ that cache-warm does not pre-resolve:" >&2
|
||||
echo "::error::Clojure classpaths used by CI that cache-warm does not pre-resolve:" >&2
|
||||
printf '%s\n' "$missing" | sed 's/^/ /' >&2
|
||||
echo >&2
|
||||
echo "Add each to the combos list in $WARM, or waive it in this script with a reason." >&2
|
||||
echo "Add each to the matching root or nested combo list in $WARM, or waive it here with a reason." >&2
|
||||
echo "Leaving it uncovered means those jobs re-fetch their version deltas from Maven on every run." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "All $(printf '%s\n' "$found" | grep -c .) clojure alias combinations under .github/ are pre-resolved."
|
||||
echo "All $(printf '%s\n' "$found" | grep -c .) Clojure classpaths used by CI are pre-resolved."
|
||||
|
||||
57
.github/scripts/file-paths.unit.spec.ts
vendored
Normal file
@@ -0,0 +1,57 @@
|
||||
import { readFileSync } from "fs";
|
||||
import { join } from "path";
|
||||
|
||||
import { load } from "js-yaml";
|
||||
import { isMatch } from "micromatch";
|
||||
|
||||
const FILTERS = join(__dirname, "..", "file-paths.yaml");
|
||||
|
||||
type Filter = string | Filter[] | { [changeType: string]: Filter };
|
||||
|
||||
// A filter is a list of patterns, and an entry may name the change types it applies to as
|
||||
// { "added|modified": pattern }. Either way the patterns are what matters here.
|
||||
const patterns = (value: Filter): string[] => {
|
||||
if (typeof value === "string") return [value];
|
||||
if (Array.isArray(value)) return value.flatMap(patterns);
|
||||
return Object.values(value).flatMap(patterns);
|
||||
};
|
||||
|
||||
describe("file-paths.yaml", () => {
|
||||
const filters = load(readFileSync(FILTERS, "utf8")) as Record<string, Filter>;
|
||||
|
||||
// dorny/paths-filter matches with picomatch, which micromatch wraps, so this asks the same
|
||||
// question CI asks of a changed path.
|
||||
const matches = (name: string, path: string) =>
|
||||
patterns(filters[name]).some((pattern) =>
|
||||
isMatch(path, pattern, { dot: true }),
|
||||
);
|
||||
|
||||
// dorny/paths-filter ORs the patterns in a filter, so a "!..." entry does not subtract from the
|
||||
// other entries - it matches every path they don't, turning the filter permanently on. Excluding
|
||||
// a path means writing the exclusion inside one pattern, the way backend_sources keeps
|
||||
// ratchets.edn out with ".clj-kondo/{!(ratchets.edn),*/**}".
|
||||
it("never excludes a path with a standalone negation", () => {
|
||||
const negated = Object.entries(filters).flatMap(([name, filter]) =>
|
||||
patterns(filter)
|
||||
.filter((pattern) => pattern.startsWith("!"))
|
||||
.map((pattern) => `${name}: ${pattern}`),
|
||||
);
|
||||
|
||||
expect(negated).toEqual([]);
|
||||
});
|
||||
|
||||
it.each(["backend_sources", "backend_all"])(
|
||||
"%s skips a ratchets-only change and still follows the rest of .clj-kondo",
|
||||
(name) => {
|
||||
expect(matches(name, ".clj-kondo/ratchets.edn")).toBe(false);
|
||||
expect(matches(name, ".clj-kondo/config.edn")).toBe(true);
|
||||
expect(matches(name, ".clj-kondo/config/modules/config.edn")).toBe(true);
|
||||
},
|
||||
);
|
||||
|
||||
it("runs the ratchet check on the ratchets file", () => {
|
||||
expect(matches("project_ratchet_checks", ".clj-kondo/ratchets.edn")).toBe(
|
||||
true,
|
||||
);
|
||||
});
|
||||
});
|
||||
4
.github/team.json
vendored
@@ -110,7 +110,9 @@
|
||||
"uladzimirdev",
|
||||
"egorgrushin",
|
||||
"metatyb",
|
||||
"olehk-mb"
|
||||
"olehk-mb",
|
||||
"annatomka",
|
||||
"brybacki"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
165
.github/workflows/ai-benchmark.yml
vendored
@@ -1,5 +1,5 @@
|
||||
name: AI Benchmark
|
||||
run-name: "dispatch evals · ${{ github.event_name == 'pull_request' && format('smoke PR#{0}', github.event.pull_request.number) || (inputs.ref != '' && inputs.ref || (inputs.pr != '' && format('PR#{0}', inputs.pr) || 'master')) }}"
|
||||
run-name: "Evals · ${{ github.event_name == 'pull_request' && 'label' || 'manual' }} · ${{ github.event.pull_request.number || inputs.pr || inputs.ref || github.sha }}"
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
@@ -33,23 +33,59 @@ concurrency:
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
unsupported:
|
||||
prepare:
|
||||
if: >-
|
||||
github.event_name == 'pull_request'
|
||||
github.event.pull_request.head.repo.full_name == github.repository
|
||||
&& github.event_name == 'pull_request'
|
||||
&& github.event.label.name == 'evals/smoke'
|
||||
&& github.event.pull_request.head.repo.full_name != github.repository
|
||||
|| github.event_name == 'workflow_dispatch'
|
||||
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
outputs:
|
||||
pr: ${{ steps.prepare.outputs.pr }}
|
||||
ref: ${{ steps.prepare.outputs.ref }}
|
||||
scope: ${{ steps.prepare.outputs.scope }}
|
||||
sample: ${{ steps.prepare.outputs.sample }}
|
||||
evals_branch: ${{ steps.prepare.outputs.evals_branch }}
|
||||
steps:
|
||||
- run: |
|
||||
echo "::error::evals/smoke only runs on same-repo PRs; this PR's head is on a fork"
|
||||
exit 1
|
||||
- name: Prepare
|
||||
id: prepare
|
||||
env:
|
||||
EVENT_NAME: ${{ github.event_name }}
|
||||
LABEL_PR: ${{ github.event.pull_request.number }}
|
||||
INPUT_PR: ${{ inputs.pr }}
|
||||
INPUT_REF: ${{ inputs.ref }}
|
||||
INPUT_SCOPE: ${{ inputs.scope }}
|
||||
INPUT_SAMPLE: ${{ inputs.sample }}
|
||||
INPUT_EVALS_BRANCH: ${{ inputs.evals_branch }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
# The two paths are mutually exclusive, and `on:` admits no other trigger.
|
||||
if [[ "$EVENT_NAME" == 'pull_request' ]]; then
|
||||
PR="$LABEL_PR"; REF=''; SCOPE='smoke'
|
||||
SAMPLE='5'; EVALS_BRANCH='main'
|
||||
else
|
||||
PR="$INPUT_PR"; REF="$INPUT_REF"; SCOPE="$INPUT_SCOPE"
|
||||
SAMPLE="${INPUT_SAMPLE:-5}"; EVALS_BRANCH="$INPUT_EVALS_BRANCH"
|
||||
fi
|
||||
# None of these is ever multi-line; a newline would forge extra output entries.
|
||||
for v in PR REF SCOPE SAMPLE EVALS_BRANCH; do
|
||||
if [[ "${!v}" == *$'\n'* ]]; then
|
||||
echo "::error::${v} must be a single line"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
{
|
||||
echo "pr=$PR"
|
||||
echo "ref=$REF"
|
||||
echo "scope=$SCOPE"
|
||||
echo "sample=$SAMPLE"
|
||||
echo "evals_branch=$EVALS_BRANCH"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
|
||||
announce:
|
||||
if: >-
|
||||
github.event_name == 'pull_request'
|
||||
&& github.event.label.name == 'evals/smoke'
|
||||
&& github.event.pull_request.head.repo.full_name == github.repository
|
||||
needs: prepare
|
||||
if: github.event_name == 'pull_request'
|
||||
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
outputs:
|
||||
@@ -75,11 +111,7 @@ jobs:
|
||||
core.setOutput('comment_id', String(data.id));
|
||||
|
||||
resolve:
|
||||
# Collapse the dispatch inputs into the one full sha everything downstream needs: it keys the
|
||||
# uberjar artifact, tags the ECR image, and tells eval-run which commit was measured. `pr` was
|
||||
# not consulted at all before, so `pr: 12345` built the tip of whatever branch the run was
|
||||
# launched from and then reported the result against the PR.
|
||||
if: github.event_name == 'workflow_dispatch'
|
||||
needs: prepare
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
permissions:
|
||||
@@ -92,8 +124,8 @@ jobs:
|
||||
id: sha
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
PR: ${{ inputs.pr }}
|
||||
REF: ${{ inputs.ref }}
|
||||
PR: ${{ needs.prepare.outputs.pr }}
|
||||
REF: ${{ needs.prepare.outputs.ref }}
|
||||
DEFAULT_SHA: ${{ github.sha }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
@@ -119,75 +151,53 @@ jobs:
|
||||
echo "sha=${SHA}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
image:
|
||||
needs: [resolve]
|
||||
# `resolve` is workflow_dispatch-only, so tolerate it being skipped. The label guard is not
|
||||
# optional: the trigger is `pull_request: [labeled]` with no filter of its own, so without it
|
||||
# any label on any same-repo PR builds ee+oss and pushes to ECR — and since announce/smoke/
|
||||
# finalize all skip in that case, a failure shows up as a bare red check with nothing
|
||||
# explaining it.
|
||||
if: >-
|
||||
!cancelled()
|
||||
&& (needs.resolve.result == 'success' || needs.resolve.result == 'skipped')
|
||||
&& (github.event_name == 'workflow_dispatch'
|
||||
|| (github.event_name == 'pull_request'
|
||||
&& github.event.label.name == 'evals/smoke'
|
||||
&& github.event.pull_request.head.repo.full_name == github.repository))
|
||||
needs: resolve
|
||||
uses: ./.github/workflows/build-private-ee-docker-image.yml
|
||||
secrets:
|
||||
PR_ENV_IAM_ROLE: ${{ secrets.PR_ENV_IAM_ROLE }}
|
||||
PR_ENV_AWS_ACCOUNT_ID: ${{ secrets.PR_ENV_AWS_ACCOUNT_ID }}
|
||||
with:
|
||||
# Empty on the label path on purpose — the image workflow then falls back to the PR head,
|
||||
# which is the artifact run-tests has already built and cached under that sha.
|
||||
commit: ${{ needs.resolve.outputs.sha }}
|
||||
|
||||
smoke:
|
||||
needs: [announce, image]
|
||||
# `!cancelled()` is load-bearing, not decoration: on the label path `resolve` is skipped by
|
||||
# design, and `image` only runs at all because it opts back in with its own `!cancelled()`.
|
||||
# GitHub propagates that skip *transitively*, so a plain boolean `if` here — carrying the
|
||||
# implicit `success()` — skips smoke even though image succeeded and the image is in ECR.
|
||||
# Opting out of the implicit check means the needs' results have to be asserted by hand.
|
||||
dispatch:
|
||||
needs: [prepare, announce, resolve, image]
|
||||
if: >-
|
||||
!cancelled()
|
||||
&& needs.announce.result == 'success'
|
||||
&& needs.prepare.result == 'success'
|
||||
&& (needs.announce.result == 'success' || needs.announce.result == 'skipped')
|
||||
&& needs.resolve.result == 'success'
|
||||
&& needs.image.result == 'success'
|
||||
&& github.event_name == 'pull_request'
|
||||
&& github.event.label.name == 'evals/smoke'
|
||||
&& github.event.pull_request.head.repo.full_name == github.repository
|
||||
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
env:
|
||||
COMMENT_ID: ${{ needs.announce.outputs.comment_id }}
|
||||
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
|
||||
steps:
|
||||
- name: Dispatch smoke eval-run
|
||||
- name: Dispatch eval-run
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
|
||||
PR: ${{ github.event.pull_request.number }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
EVALS_BRANCH: main
|
||||
EVALS_BRANCH: ${{ needs.prepare.outputs.evals_branch }}
|
||||
PR: ${{ needs.prepare.outputs.pr }}
|
||||
SHA: ${{ needs.resolve.outputs.sha }}
|
||||
SCOPE: ${{ needs.prepare.outputs.scope }}
|
||||
SAMPLE: ${{ needs.prepare.outputs.sample }}
|
||||
COMMENT_ID: ${{ needs.announce.outputs.comment_id }}
|
||||
run: |
|
||||
gh api repos/metabase/evals/actions/workflows/eval-run.yml/dispatches \
|
||||
-f ref="$EVALS_BRANCH" \
|
||||
-f "inputs[pr]=$PR" \
|
||||
-f "inputs[ref]=$HEAD_SHA" \
|
||||
-f "inputs[scope]=smoke" \
|
||||
-f "inputs[ref]=$SHA" \
|
||||
-f "inputs[scope]=$SCOPE" \
|
||||
-f "inputs[surfaces]=all" \
|
||||
-f "inputs[sample]=5" \
|
||||
-f "inputs[sample]=$SAMPLE" \
|
||||
-f "inputs[evals_branch]=$EVALS_BRANCH" \
|
||||
-f "inputs[comment_id]=$COMMENT_ID"
|
||||
|
||||
finalize:
|
||||
# If the build or dispatch fell over, eval-run never took over the comment — leave a
|
||||
# failure verdict instead of a comment stuck at "building the image…".
|
||||
needs: [announce, image, smoke]
|
||||
if: always() && needs.announce.result == 'success' && needs.smoke.result != 'success'
|
||||
needs: [announce, resolve, image, dispatch]
|
||||
if: always() && needs.announce.result == 'success' && needs.dispatch.result != 'success'
|
||||
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
env:
|
||||
COMMENT_ID: ${{ needs.announce.outputs.comment_id }}
|
||||
SMOKE_RESULT: ${{ needs.smoke.result }}
|
||||
IMAGE_RESULT: ${{ needs.image.result }}
|
||||
NEEDS_JSON: ${{ toJSON(needs) }}
|
||||
steps:
|
||||
- name: Comment (build/dispatch failed)
|
||||
uses: actions/github-script@v9
|
||||
@@ -195,11 +205,11 @@ jobs:
|
||||
github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
|
||||
script: |
|
||||
const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
|
||||
// A concurrency-cancelled retrigger lands here too; don't stamp it as an infra failure.
|
||||
const cancelled = [process.env.SMOKE_RESULT, process.env.IMAGE_RESULT].includes('cancelled');
|
||||
const cancelled = Object.values(JSON.parse(process.env.NEEDS_JSON))
|
||||
.some(j => j.result === 'cancelled');
|
||||
const body = cancelled
|
||||
? `🤖 **Evals smoke** — ⚪ cancelled before evals started ([logs](${runUrl}))`
|
||||
: `🤖 **Evals smoke** — ❌ build or dispatch failed before evals started ([logs](${runUrl}))`;
|
||||
? `🤖 **Evals** — ⚪ cancelled before evals started ([logs](${runUrl}))`
|
||||
: `🤖 **Evals** — ❌ build or dispatch failed before evals started ([logs](${runUrl}))`;
|
||||
// The replacement run deletes this comment on its way in, so a concurrency cancel can
|
||||
// land here with nothing to patch. Losing the stamp is fine; failing the job isn't.
|
||||
await github.rest.issues.updateComment({
|
||||
@@ -214,30 +224,3 @@ jobs:
|
||||
issue_number: context.payload.pull_request.number,
|
||||
name: 'evals/smoke',
|
||||
}).catch(() => {});
|
||||
|
||||
dispatch:
|
||||
needs: [resolve, image]
|
||||
# Deliberately no `always()`: benchmarking without the image this run was supposed to produce
|
||||
# would measure a stale build or nothing at all. A failed `image` fails the run, which is the
|
||||
# signal here — unlike the PR path there is no comment for `finalize` to stamp.
|
||||
if: github.event_name == 'workflow_dispatch'
|
||||
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- name: Dispatch eval-run
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
|
||||
EVALS_BRANCH: ${{ github.event.inputs.evals_branch }}
|
||||
PR: ${{ inputs.pr }}
|
||||
# The sha the image was actually built and tagged with, not the raw `ref` input.
|
||||
REF: ${{ needs.resolve.outputs.sha }}
|
||||
SCOPE: ${{ inputs.scope }}
|
||||
SAMPLE: ${{ inputs.sample }}
|
||||
run: |
|
||||
gh api repos/metabase/evals/actions/workflows/eval-run.yml/dispatches \
|
||||
-f ref="$EVALS_BRANCH" \
|
||||
-f "inputs[pr]=$PR" \
|
||||
-f "inputs[ref]=$REF" \
|
||||
-f "inputs[scope]=$SCOPE" \
|
||||
-f "inputs[sample]=${SAMPLE:-5}" \
|
||||
-f "inputs[evals_branch]=$EVALS_BRANCH"
|
||||
|
||||
1
.github/workflows/app-db.yml
vendored
@@ -332,7 +332,6 @@ jobs:
|
||||
build-static-viz: true
|
||||
test-args: >-
|
||||
:only '"test"'
|
||||
:exclude-tags [:mb/driver-tests]
|
||||
:partition/total 2
|
||||
:partition/index 0
|
||||
exclude-tag: ':mb/driver-tests :mb/transforms-python-test'
|
||||
|
||||
118
.github/workflows/backend.yml
vendored
@@ -10,6 +10,18 @@ on:
|
||||
description: Run the full suite regardless of the diff (master / release branches).
|
||||
type: boolean
|
||||
default: false
|
||||
run-project-backend-checks:
|
||||
description: Run project-level backend checks.
|
||||
type: boolean
|
||||
default: false
|
||||
run-project-migration-checks:
|
||||
description: Run project-level migration checks.
|
||||
type: boolean
|
||||
default: false
|
||||
run-project-ratchet-checks:
|
||||
description: Run the ratchet check for a diff that touches nothing but the ratchets file.
|
||||
type: boolean
|
||||
default: false
|
||||
|
||||
# Run the backend test suite on demand, against the ref you dispatch on, optionally under a
|
||||
# profiler. Everything else about the jobs -- matrix legs, tag filters, quarantine gating -- is
|
||||
@@ -46,6 +58,95 @@ env:
|
||||
ONLY_TESTS: ${{ inputs.tests }}
|
||||
|
||||
jobs:
|
||||
project-checks:
|
||||
name: 'Project tests: Suites'
|
||||
if: >-
|
||||
${{ inputs.force-run ||
|
||||
inputs.run-project-backend-checks ||
|
||||
inputs.run-project-migration-checks ||
|
||||
inputs.run-project-ratchet-checks }}
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Prepare back-end environment
|
||||
uses: ./.github/actions/prepare-backend
|
||||
# Bootstrap the shared Mage runtime before concurrent commands try to install it themselves.
|
||||
- name: Prepare Mage
|
||||
run: ./bin/mage
|
||||
- parallel:
|
||||
- name: Run backend checks
|
||||
if: ${{ inputs.force-run || inputs.run-project-backend-checks }}
|
||||
run: ./bin/mage project-tests backend
|
||||
- name: Run migration checks
|
||||
if: ${{ inputs.force-run || inputs.run-project-migration-checks }}
|
||||
run: ./bin/mage project-tests migrations
|
||||
# Babashka, not a JVM test run, so it is cheap enough to run alongside the backend checks.
|
||||
# It still needs the JDK prepare-backend installs, to resolve clj-kondo on a cold cache.
|
||||
- name: Run ratchet checks
|
||||
if: ${{ inputs.force-run || inputs.run-project-backend-checks || inputs.run-project-ratchet-checks }}
|
||||
run: ./bin/mage kondo-ratchets
|
||||
|
||||
project-tests:
|
||||
name: Project tests
|
||||
needs: project-checks
|
||||
if: ${{ always() && !cancelled() }}
|
||||
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- name: Check Project tests status
|
||||
uses: actions/github-script@v9
|
||||
env:
|
||||
needs: ${{ toJson(needs) }}
|
||||
with:
|
||||
script: | # js
|
||||
const needs = JSON.parse(process.env.needs);
|
||||
const jobs = Object.entries(needs).map(
|
||||
([jobName, jobValues]) => ({
|
||||
name: jobName,
|
||||
result: jobValues.result
|
||||
}));
|
||||
|
||||
if (jobs.every(job => job.result === 'skipped' || job.result === 'success')) {
|
||||
console.log('');
|
||||
console.log(' |>>> |>>> |>>>');
|
||||
console.log(' | | |');
|
||||
console.log(' _ _|_ _ _ _|_ _ _ _|_ _');
|
||||
console.log(' | |_| |_| |______| |_| |_| |______| |_| |_| |');
|
||||
console.log(' | | | | | |');
|
||||
console.log(' | [] [] | | [] [] | | [] [] |');
|
||||
console.log(' | |______| |______| |');
|
||||
console.log(' | ___ |');
|
||||
console.log(' |___________________| |___________________|');
|
||||
console.log(' |___|');
|
||||
console.log('');
|
||||
console.log('Everything is up to code and structurally sound.');
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
console.log('');
|
||||
console.log(' .-===========-.____');
|
||||
console.log(' /_______________\\ \\____');
|
||||
console.log(' | .-----------. | \\___');
|
||||
console.log(' | | x x | | \\__');
|
||||
console.log(' | | ~ | | _\\_');
|
||||
console.log(" | `-----------' | /___\\");
|
||||
console.log(' | o o |');
|
||||
console.log(" `-----------------'");
|
||||
console.log(' /=============\\');
|
||||
console.log(' / O O O O O O O \\');
|
||||
console.log(' \\_______________/');
|
||||
console.log('');
|
||||
console.log('One or more project checks are off the blueprint. Hard hats required.');
|
||||
|
||||
jobs.forEach((job) => {
|
||||
if (job.result !== 'success' && job.result !== 'skipped') {
|
||||
console.log(`${job.name} - ${job.result}`);
|
||||
}
|
||||
});
|
||||
|
||||
process.exit(1);
|
||||
|
||||
be-linter-clj-kondo:
|
||||
if: ${{ !inputs.skip }}
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
@@ -80,7 +181,7 @@ jobs:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Prepare back-end environment
|
||||
uses: ./.github/actions/prepare-backend
|
||||
- run: clojure -X:dev:ee:ee-dev:drivers:drivers-dev:test:eastwood/test
|
||||
- run: clojure -X:dev:dev/test:ee:ee-dev:drivers:drivers-dev:test:eastwood/test
|
||||
name: Run Eastwood linter on our backend tests
|
||||
|
||||
be-linter-eastwood-test-changed:
|
||||
@@ -108,13 +209,14 @@ jobs:
|
||||
# last synced don't widen our scope. Falls back to BASE_SHA if older than fetched depth.
|
||||
git fetch --no-tags --depth=200 origin "$BASE_SHA"
|
||||
DIFF_BASE=$(git merge-base "$BASE_SHA" "$HEAD_SHA" 2>/dev/null || echo "$BASE_SHA")
|
||||
# Test roots we lint: core, EE, and each driver module's test tree.
|
||||
# Test roots we lint: core, EE, dev/test, and each driver module's test tree.
|
||||
# :eastwood/test derives matching :source-paths from the classpath, so any
|
||||
# namespace we name here is reachable. Include renames (R) so a test that
|
||||
# was just renamed still gets linted under its new name.
|
||||
FILES=$(git diff --name-only --diff-filter=AMR "$DIFF_BASE" "$HEAD_SHA" \
|
||||
-- 'test/**/*.clj' 'test/**/*.cljc' \
|
||||
'enterprise/backend/test/**/*.clj' 'enterprise/backend/test/**/*.cljc' \
|
||||
'dev/test/**/*.clj' 'dev/test/**/*.cljc' \
|
||||
'modules/drivers/**/test/**/*.clj' 'modules/drivers/**/test/**/*.cljc')
|
||||
|
||||
if [ -z "$FILES" ]; then
|
||||
@@ -130,7 +232,14 @@ jobs:
|
||||
exit 1
|
||||
fi
|
||||
|
||||
NS=$(sed -E 's#^(test/|enterprise/backend/test/|modules/drivers/[^/]+/test/)##; s#\.cljc?$##; s#/#.#g; s#_#-#g' <<<"$FILES" \
|
||||
# Convert each test path to a Clojure namespace: strip its known test root and
|
||||
# extension, then translate path separators and underscores. Keep these roots
|
||||
# aligned with FILES above and the Eastwood classpath aliases below.
|
||||
NS=$(sed -E \
|
||||
-e 's#^(test/|enterprise/backend/test/|dev/test/|modules/drivers/[^/]+/test/)##' \
|
||||
-e 's#\.cljc?$##' \
|
||||
-e 's#/#.#g' \
|
||||
-e 's#_#-#g' <<<"$FILES" \
|
||||
| sort -u \
|
||||
| paste -sd' ' -)
|
||||
echo "Changed test namespaces: $NS"
|
||||
@@ -139,7 +248,7 @@ jobs:
|
||||
if: steps.changed.outputs.namespaces != ''
|
||||
env:
|
||||
NAMESPACES: ${{ steps.changed.outputs.namespaces }}
|
||||
run: clojure -X:dev:ee:ee-dev:drivers:drivers-dev:test:eastwood/test :namespaces "[$NAMESPACES]"
|
||||
run: clojure -X:dev:dev/test:ee:ee-dev:drivers:drivers-dev:test:eastwood/test :namespaces "[$NAMESPACES]"
|
||||
|
||||
be-tests:
|
||||
if: ${{ !inputs.skip }}
|
||||
@@ -368,6 +477,7 @@ jobs:
|
||||
|
||||
be-tests-result:
|
||||
needs:
|
||||
- project-tests
|
||||
- be-linter-eastwood
|
||||
- be-linter-eastwood-test
|
||||
- be-linter-eastwood-test-changed
|
||||
|
||||
@@ -136,5 +136,7 @@ jobs:
|
||||
context: bin/docker/
|
||||
platforms: linux/amd64
|
||||
network: host
|
||||
build-args: |
|
||||
GIT_COMMIT_SHA=${{ steps.meta.outputs.tag }}
|
||||
tags: ${{ secrets.PR_ENV_AWS_ACCOUNT_ID }}.dkr.ecr.us-east-1.amazonaws.com/metabase-enterprise:${{ steps.meta.outputs.tag }}
|
||||
push: true
|
||||
|
||||
5
.github/workflows/build-scripts.yml
vendored
@@ -79,6 +79,7 @@ jobs:
|
||||
|
||||
test-build-scripts:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 40
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Prepare back-end environment
|
||||
@@ -88,10 +89,6 @@ jobs:
|
||||
- name: Run build and release script tests
|
||||
run: clojure -X:dev:drivers:build:build-dev:build-test:ci
|
||||
timeout-minutes: 15
|
||||
- name: Run Liquibase migrations linter tests
|
||||
run: clojure -M:test
|
||||
working-directory: bin/lint-migrations-file
|
||||
timeout-minutes: 15
|
||||
- name: Run load-namespaces tests
|
||||
run: clojure -M:test
|
||||
working-directory: bin/load-namespaces
|
||||
|
||||
40
.github/workflows/cache-warm.yml
vendored
@@ -1,4 +1,4 @@
|
||||
# Writes the three large dependency caches. Every other job restores them and none of them writes.
|
||||
# Writes the shared dependency caches. Every other job restores them and none of them writes.
|
||||
#
|
||||
# A single writer rather than the usual let-every-job-save arrangement, because a pull request here fans
|
||||
# out to roughly 200 jobs, ~75 of which install Cypress. Letting each of those save would have them all
|
||||
@@ -13,7 +13,7 @@
|
||||
# at it, and master's copy backs up every ref that has none of its own.
|
||||
#
|
||||
# Keys are content-derived (see .github/scripts/cache-keys.sh), so a push whose dependencies already have
|
||||
# an entry does nothing but three lookups. There is no cron: nothing here is keyed by date.
|
||||
# an entry does nothing but four lookups. There is no cron: nothing here is keyed by date.
|
||||
#
|
||||
#
|
||||
# PRINCIPLES:
|
||||
@@ -69,6 +69,13 @@ jobs:
|
||||
path: ${{ steps.cache-spec.outputs.bun-store-path }}
|
||||
key: ${{ steps.cache-spec.outputs.bun-store-key }}
|
||||
lookup-only: true
|
||||
- name: Look up Clojure CLI
|
||||
id: clojure-lookup
|
||||
uses: actions/cache/restore@v5
|
||||
with:
|
||||
path: ${{ steps.cache-spec.outputs.clojure-path }}
|
||||
key: ${{ steps.cache-spec.outputs.clojure-key }}
|
||||
lookup-only: true
|
||||
- name: Look up Cypress binary
|
||||
id: cypress-lookup
|
||||
uses: actions/cache/restore@v5
|
||||
@@ -77,10 +84,19 @@ jobs:
|
||||
key: ${{ steps.cache-spec.outputs.cypress-key }}
|
||||
lookup-only: true
|
||||
|
||||
# Also runs when only the Clojure CLI entry is missing: installing it is what there is to save,
|
||||
# and the pre-resolve below needs it either way.
|
||||
- name: Prepare backend
|
||||
if: steps.m2-lookup.outputs.cache-hit != 'true'
|
||||
if: steps.m2-lookup.outputs.cache-hit != 'true' || steps.clojure-lookup.outputs.cache-hit != 'true'
|
||||
uses: ./.github/actions/prepare-backend
|
||||
|
||||
- name: Save Clojure CLI
|
||||
if: steps.clojure-lookup.outputs.cache-hit != 'true'
|
||||
uses: actions/cache/save@v5
|
||||
with:
|
||||
path: ${{ steps.cache-spec.outputs.clojure-path }}
|
||||
key: ${{ steps.cache-spec.outputs.clojure-key }}
|
||||
|
||||
# Pre-resolve every Clojure dependency classpath CI actually uses, so the saved cache is a SUPERSET
|
||||
# of what each job resolves at runtime.
|
||||
#
|
||||
@@ -102,7 +118,9 @@ jobs:
|
||||
combos=(
|
||||
":build:dev:ee:ee-dev:drivers:drivers-dev:cljs" # build + uberjar (combined classpath)
|
||||
":cljs" # shadow-cljs (build:cljs); resolves :cljs alone -> covers asm:9.2
|
||||
":dev:ee:ee-dev:drivers:drivers-dev:test:eastwood" # backend.yml (eastwood + eastwood/test)
|
||||
":dev:ee:ee-dev:drivers:drivers-dev:test:eastwood" # backend.yml (eastwood)
|
||||
":dev:dev/test:ee:ee-dev:drivers:drivers-dev:test:eastwood/test" # backend.yml (eastwood tests)
|
||||
":dev:dev/test:ee:ee-dev:drivers:drivers-dev:test:ci" # run-tests.yml (project tests)
|
||||
":dev:ci:ee:ee-dev:drivers:drivers-dev:test" # test-driver.yml (ee) -> covers data.json:2.4.0
|
||||
":dev:ci:oss:oss-dev:drivers:drivers-dev:test" # test-driver.yml (oss)
|
||||
":dev:ci:test:ee:ee-dev" # backend.yml matrix (ee)
|
||||
@@ -124,6 +142,20 @@ jobs:
|
||||
echo "::endgroup::"
|
||||
done
|
||||
|
||||
# These commands use nested tools.deps projects; resolving the same alias from the
|
||||
# repository root does not warm their dependencies.
|
||||
nested_combos=(
|
||||
"bin/lint-migrations-file|:test" # project migration checks
|
||||
"bin/load-namespaces|:test" # build-scripts.yml
|
||||
)
|
||||
for spec in "${nested_combos[@]}"; do
|
||||
project=${spec%%|*}
|
||||
combo=${spec#*|}
|
||||
echo "::group::${project}: clojure -P -M${combo}"
|
||||
(cd "$project" && clojure -P -M"${combo}")
|
||||
echo "::endgroup::"
|
||||
done
|
||||
|
||||
echo "::group::./bin/mage"
|
||||
./bin/mage
|
||||
echo "::endgroup::"
|
||||
|
||||
9
.github/workflows/cut-release-branch.yml
vendored
@@ -59,6 +59,12 @@ jobs:
|
||||
# Pushing the new branch triggers set-milestone.yml, which does the
|
||||
# full milestone backfill and then dispatches release-notes-preview
|
||||
# to upload the markdown to #writing.
|
||||
#
|
||||
# Kondo ignore ratchets apply only to master. The cut commit replaces the
|
||||
# budgets with an explicit opt-out so future budget changes are not applied
|
||||
# to the release branch. Keeping the opt-out in the canonical file avoids
|
||||
# introducing a separate marker that could drift out of sync, and the
|
||||
# backport helper writes the same opt-out so both paths stay identical.
|
||||
- name: Cut the branch
|
||||
id: cut
|
||||
run: |
|
||||
@@ -66,6 +72,9 @@ jobs:
|
||||
git config user.name github-actions
|
||||
git config user.email github-actions@github.com
|
||||
git checkout -b "$BRANCH"
|
||||
source .github/actions/create-backport/kondo-ratchets.sh
|
||||
write_disabled_ratchets
|
||||
git add --sparse .clj-kondo/ratchets.edn
|
||||
git commit --allow-empty --no-verify -m "Cut $BRANCH branch"
|
||||
git push -u origin "$BRANCH"
|
||||
echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
9
.github/workflows/docs-generate-base.yml
vendored
@@ -75,6 +75,14 @@ jobs:
|
||||
clojure -M:ee:doc command-documentation
|
||||
continue-on-error: false
|
||||
|
||||
- name: Generate AI provider documentation
|
||||
id: generate_ai_provider_docs
|
||||
run: |
|
||||
set -e # Exit on any error
|
||||
echo "Generating AI provider documentation..."
|
||||
clojure -M:ee:doc ai-providers-documentation
|
||||
continue-on-error: false
|
||||
|
||||
- name: Generate usage analytics documentation
|
||||
run: ./bin/generate-usage-analytics-docs.bb
|
||||
|
||||
@@ -164,6 +172,7 @@ jobs:
|
||||
- \`clojure -M:ee:doc config-template\`
|
||||
- \`clojure -M:ee:doc api-documentation\`
|
||||
- \`clojure -M:ee:doc command-documentation\`
|
||||
- \`clojure -M:ee:doc ai-providers-documentation\`
|
||||
- \`./bin/generate-usage-analytics-docs.bb\`
|
||||
- \`bun run embedding-sdk:docs:generate\`
|
||||
- \`bun run embedding-eajs:docs:generate\`
|
||||
|
||||
102
.github/workflows/drivers.yml
vendored
@@ -329,34 +329,45 @@ jobs:
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
:fail-fast? true
|
||||
:partition/total 3
|
||||
:partition/total 2
|
||||
:partition/index 0
|
||||
- name: Driver Tests Part 2
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
:fail-fast? true
|
||||
:partition/total 3
|
||||
:partition/total 2
|
||||
:partition/index 1
|
||||
- name: Driver Tests Part 3
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
:fail-fast? true
|
||||
:partition/total 3
|
||||
:partition/index 2
|
||||
env:
|
||||
CI: 'true'
|
||||
DRIVERS: databricks
|
||||
MB_DATABRICKS_TEST_HOST: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HOST }}
|
||||
MB_DATABRICKS_TEST_HTTP_PATH: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HTTP_PATH }}
|
||||
MB_DATABRICKS_TEST_TOKEN: ${{ secrets.MB_DATABRICKS_JDBC_TEST_TOKEN }}
|
||||
MB_DATABRICKS_TEST_CATALOG: 'metabase_ci'
|
||||
MB_DATABRICKS_TEST_CATALOG_ROUTING: 'metabase_routing_catalog'
|
||||
MB_DATABRICKS_TEST_CLIENT_ID: ${{ secrets.MB_DATABRICKS_JDBC_TEST_CLIENT_ID }}
|
||||
MB_DATABRICKS_TEST_OAUTH_SECRET: ${{ secrets.MB_DATABRICKS_JDBC_TEST_OAUTH_SECRET }}
|
||||
name: Databricks
|
||||
steps:
|
||||
- name: Select Databricks workspace
|
||||
shell: bash
|
||||
env:
|
||||
TOTAL_SHARDS: '2'
|
||||
HOST_0: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HOST }}
|
||||
TOKEN_0: ${{ secrets.MB_DATABRICKS_JDBC_TEST_TOKEN }}
|
||||
HTTP_PATH_0: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HTTP_PATH }}
|
||||
HOST_1: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HOST_2 }}
|
||||
TOKEN_1: ${{ secrets.MB_DATABRICKS_JDBC_TEST_TOKEN_2 }}
|
||||
HTTP_PATH_1: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HTTP_PATH_2 }}
|
||||
run: | # sh
|
||||
shard=$(( GITHUB_RUN_NUMBER % TOTAL_SHARDS ))
|
||||
echo "Using Databricks workspace shard $(( shard + 1 )) of $TOTAL_SHARDS"
|
||||
for name in HOST HTTP_PATH TOKEN; do
|
||||
var="${name}_${shard}"
|
||||
if [ -z "${!var}" ]; then
|
||||
echo "::error::Secret $var is empty - is the shard $shard Databricks workspace configured?"
|
||||
exit 1
|
||||
fi
|
||||
echo "MB_DATABRICKS_TEST_${name}=${!var}" >> "$GITHUB_ENV"
|
||||
done
|
||||
- uses: actions/checkout@v6
|
||||
- name: Test Databricks driver
|
||||
id: test-driver
|
||||
@@ -367,59 +378,6 @@ jobs:
|
||||
test-args: ${{ matrix.job.test-args }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
|
||||
be-tests-databricks-multi-catalog-ee:
|
||||
needs: determine-driver-skips
|
||||
if: ${{ needs.determine-driver-skips.outputs.databricks-should-run == 'true' }}
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 50
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
job:
|
||||
- name: Driver Tests Part 1
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
:fail-fast? true
|
||||
:partition/total 3
|
||||
:partition/index 0
|
||||
- name: Driver Tests Part 2
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
:fail-fast? true
|
||||
:partition/total 3
|
||||
:partition/index 1
|
||||
- name: Driver Tests Part 3
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
:fail-fast? true
|
||||
:partition/total 3
|
||||
:partition/index 2
|
||||
env:
|
||||
CI: 'true'
|
||||
DRIVERS: databricks
|
||||
MB_DATABRICKS_TEST_MULTI_LEVEL_SCHEMA: true
|
||||
MB_DATABRICKS_TEST_HOST: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HOST }}
|
||||
MB_DATABRICKS_TEST_HTTP_PATH: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HTTP_PATH }}
|
||||
MB_DATABRICKS_TEST_TOKEN: ${{ secrets.MB_DATABRICKS_JDBC_TEST_TOKEN }}
|
||||
MB_DATABRICKS_TEST_CATALOG: 'metabase_ci'
|
||||
MB_DATABRICKS_TEST_CATALOG_ROUTING: 'metabase_routing_catalog'
|
||||
MB_DATABRICKS_TEST_CLIENT_ID: ${{ secrets.MB_DATABRICKS_JDBC_TEST_CLIENT_ID }}
|
||||
MB_DATABRICKS_TEST_OAUTH_SECRET: ${{ secrets.MB_DATABRICKS_JDBC_TEST_OAUTH_SECRET }}
|
||||
name: Databricks (Multi-Catalog)
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Test Databricks driver with multi-catalog
|
||||
id: test-driver
|
||||
uses: ./.github/actions/test-driver
|
||||
with:
|
||||
junit-name: 'be-tests-databricks-multi-catalog-ee'
|
||||
artifact-suffix: ${{ matrix.job.name }}
|
||||
test-args: ${{ matrix.job.test-args }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
|
||||
be-tests-h2:
|
||||
needs: determine-driver-skips
|
||||
if: ${{ needs.determine-driver-skips.outputs.h2-should-run == 'true' }}
|
||||
@@ -814,9 +772,20 @@ jobs:
|
||||
enable-ssl-tests: 'true'
|
||||
enable-aws-iam-tests: 'true'
|
||||
job:
|
||||
- name: Driver Tests
|
||||
# Split because the unpartitioned Driver Tests job was running ~40 minutes of tests against
|
||||
# `timeout-minutes: 50` (checkout and prepare-backend take the rest), and Postgres Latest was
|
||||
# timing out.
|
||||
- name: Driver Tests Part 1
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:partition/total 2
|
||||
:partition/index 0
|
||||
exclude-tag: ':mb/transforms-python-test'
|
||||
- name: Driver Tests Part 2
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:partition/total 2
|
||||
:partition/index 1
|
||||
exclude-tag: ':mb/transforms-python-test'
|
||||
- name: Transforms Python Tests
|
||||
test-args: >-
|
||||
@@ -1309,7 +1278,6 @@ jobs:
|
||||
- be-tests-bigquery-cloud-sdk-ee
|
||||
- be-tests-clickhouse-ee
|
||||
- be-tests-databricks-ee
|
||||
- be-tests-databricks-multi-catalog-ee
|
||||
- be-tests-druid-jdbc-ee
|
||||
- be-tests-h2
|
||||
- be-tests-h2-oss
|
||||
|
||||
@@ -73,6 +73,21 @@ jobs:
|
||||
id: cypress-prep
|
||||
uses: ./.github/actions/prepare-cypress
|
||||
|
||||
# esbuild resolves every import in the spec bundle's graph, even the ones tree shaking then drops.
|
||||
# The cljs imports resolve into target/cljs_dev, so the compiled CLJS has to exist before Cypress runs.
|
||||
# run-tests.yml compiles it in parallel with the uberjar build, so downloading is ~10x faster than compiling here.
|
||||
- name: Download pre-built CLJS
|
||||
id: prebuilt-cljs
|
||||
uses: actions/download-artifact@v8
|
||||
continue-on-error: true
|
||||
with:
|
||||
name: e2e-cljs-${{ github.event.pull_request.head.sha || github.sha }}
|
||||
path: target/cljs_dev
|
||||
|
||||
- name: Compile CLJS
|
||||
if: steps.prebuilt-cljs.outcome != 'success'
|
||||
run: bun run build-pure:cljs
|
||||
|
||||
- name: Run Metabase
|
||||
run: node e2e/runner/run_cypress_ci.js start
|
||||
|
||||
@@ -246,6 +261,13 @@ jobs:
|
||||
id: cypress-prep
|
||||
uses: ./.github/actions/prepare-cypress
|
||||
|
||||
# esbuild resolves every import in the spec bundle's graph, even the ones tree shaking then drops.
|
||||
# The cljs imports resolve into target/cljs_dev, so the compiled CLJS has to exist before Cypress runs.
|
||||
# The checked-out tree is the previous commit, which the current run's CLJS artifact was not built from,
|
||||
# so compile from source instead of downloading.
|
||||
- name: Compile CLJS
|
||||
run: bun run build-pure:cljs
|
||||
|
||||
- name: Run Metabase
|
||||
run: node e2e/runner/run_cypress_ci.js start
|
||||
|
||||
|
||||
@@ -203,7 +203,9 @@ jobs:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
environment: ["production", "development"]
|
||||
# TEMPORARILY DISABLED: re-enable by removing the `false &&` below.
|
||||
if: |
|
||||
false &&
|
||||
!cancelled() && needs.get-sample-apps-data.result == 'success' &&
|
||||
(github.ref == 'refs/heads/master' || github.base_ref == 'master')
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
@@ -270,7 +272,9 @@ jobs:
|
||||
# They use the shoppy production environment and include performance measurement tests.
|
||||
e2e-shoppy-synthetic-tests:
|
||||
needs: [get-sample-apps-data]
|
||||
# TEMPORARILY DISABLED: re-enable by removing the `false &&` below.
|
||||
if: |
|
||||
false &&
|
||||
!cancelled() && needs.get-sample-apps-data.result == 'success' &&
|
||||
github.event_name == 'push' &&
|
||||
github.ref == 'refs/heads/master'
|
||||
@@ -419,6 +423,21 @@ jobs:
|
||||
id: cypress-prep
|
||||
uses: ./.github/actions/prepare-cypress
|
||||
|
||||
# esbuild resolves every import in the spec bundle's graph, even the ones tree shaking then drops.
|
||||
# The cljs imports resolve into target/cljs_dev, so the compiled CLJS has to exist before Cypress runs.
|
||||
# run-tests.yml compiles it in parallel with the uberjar build, so downloading is ~10x faster than compiling here.
|
||||
- name: Download pre-built CLJS
|
||||
id: prebuilt-cljs
|
||||
uses: actions/download-artifact@v8
|
||||
continue-on-error: true
|
||||
with:
|
||||
name: e2e-cljs-${{ github.event.pull_request.head.sha || github.sha }}
|
||||
path: target/cljs_dev
|
||||
|
||||
- name: Compile CLJS
|
||||
if: steps.prebuilt-cljs.outcome != 'success'
|
||||
run: bun run build-pure:cljs
|
||||
|
||||
- name: Run Metabase
|
||||
run: node e2e/runner/run_cypress_ci.js start
|
||||
|
||||
|
||||
87
.github/workflows/kondo-ratchets-self-heal.yml
vendored
@@ -1,87 +0,0 @@
|
||||
name: Kondo Ratchets Self-Heal
|
||||
|
||||
# Stale budgets in .clj-kondo/ratchets.edn (above the actual ignore counts) fail CI. On PRs
|
||||
# labelled kondo-ratchets-self-healing, this commits the lowered budgets to the branch;
|
||||
# unlabelled PRs rely on local test runs to tighten the file.
|
||||
#
|
||||
# Two jobs so the automation token never shares an environment with PR-controlled code: the
|
||||
# ratchet-down job runs ./bin/mage from the PR branch and only exports the fixed file as an
|
||||
# artifact; the push job starts from a clean runner, executes nothing from the PR, and pushes.
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types: [opened, synchronize, reopened, labeled, unlabeled]
|
||||
paths:
|
||||
- "**.clj"
|
||||
- "**.cljc"
|
||||
- "**.cljs"
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
ratchet-down:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 10
|
||||
# fork PRs don't get the automation token, so we couldn't push to them anyway
|
||||
if: >-
|
||||
contains(github.event.pull_request.labels.*.name, 'kondo-ratchets-self-healing')
|
||||
&& github.event.pull_request.head.repo.full_name == github.repository
|
||||
outputs:
|
||||
changed: ${{ steps.diff.outputs.changed }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.head.ref }}
|
||||
persist-credentials: false
|
||||
- name: Prepare back-end environment
|
||||
uses: ./.github/actions/prepare-backend
|
||||
- name: Lower ignore budgets to match the source tree
|
||||
run: ./bin/mage fix-kondo-ratchets
|
||||
- name: Check for changes
|
||||
id: diff
|
||||
run: |
|
||||
if git diff --quiet .clj-kondo/ratchets.edn; then
|
||||
echo "budgets already tight"
|
||||
echo "changed=false" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "changed=true" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
- uses: actions/upload-artifact@v7
|
||||
if: steps.diff.outputs.changed == 'true'
|
||||
with:
|
||||
name: ratchets-edn
|
||||
path: .clj-kondo/ratchets.edn
|
||||
# dot-directories are excluded by default, which would ship an empty artifact
|
||||
include-hidden-files: true
|
||||
|
||||
push:
|
||||
needs: ratchet-down
|
||||
if: needs.ratchet-down.outputs.changed == 'true'
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.head.ref }}
|
||||
persist-credentials: false
|
||||
- uses: actions/download-artifact@v8
|
||||
with:
|
||||
name: ratchets-edn
|
||||
path: .clj-kondo/
|
||||
- name: Commit to the PR branch
|
||||
env:
|
||||
PR_BRANCH: ${{ github.event.pull_request.head.ref }}
|
||||
REPO: ${{ github.repository }}
|
||||
# METABASE_AUTOMATION_USER_TOKEN so the self-healing commit triggers CI
|
||||
# (a GITHUB_TOKEN push would not)
|
||||
AUTOMATION_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
|
||||
run: |
|
||||
git config --global user.name "Metabase Automation"
|
||||
git config --global user.email "github-automation@metabase.com"
|
||||
git add .clj-kondo/ratchets.edn
|
||||
git commit -m "Ratchet down kondo ignore budgets" --no-verify
|
||||
git push "https://x-access-token:${AUTOMATION_TOKEN}@github.com/${REPO}.git" "HEAD:$PR_BRANCH"
|
||||
166
.github/workflows/kondo-ratchets-shrink.yml
vendored
Normal file
@@ -0,0 +1,166 @@
|
||||
name: Shrink ratchets
|
||||
|
||||
# Lowers the budgets in .clj-kondo/ratchets.edn to match master and delivers the result through one
|
||||
# automation PR, because the rulesets require a pull request for every change to master. The PR is
|
||||
# approved by github-actions[bot] and merges itself once the ratchet check passes. A merge of that PR
|
||||
# touches only ratchets.edn, which matches `.clj-kondo/**` below, so it triggers one more run that finds
|
||||
# nothing to lower and stops: a single no-op, not a loop.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [master]
|
||||
paths:
|
||||
- "**.clj"
|
||||
- "**.cljc"
|
||||
- "**.cljs"
|
||||
- ".clj-kondo/**"
|
||||
- "bb.edn"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
# checkout and the fetch of the automation branch
|
||||
contents: read
|
||||
# the compare step reads whether the automation PR is still open, and the approve step reviews it as
|
||||
# github-actions[bot]; pushes and the PR itself use the automation user's token
|
||||
pull-requests: write
|
||||
|
||||
# When master is pushed again, the newest run cancels the older one, checks out the latest master it can
|
||||
# see, and force-updates the automation branch. The PR therefore contains one commit on top of master.
|
||||
concurrency:
|
||||
group: kondo-ratchets-shrink
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
AUTOMATION_BRANCH: automation/shrink-ratchets
|
||||
RATCHETS_FILE: .clj-kondo/ratchets.edn
|
||||
|
||||
jobs:
|
||||
shrink:
|
||||
name: Shrink ratchets
|
||||
# A fork with Actions enabled has no automation token, so this would only ever fail at the push.
|
||||
if: github.repository == 'metabase/metabase'
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
# `ref: master` checks out the branch head at checkout time rather than the pushed commit, so a run
|
||||
# that queued behind another one starts from the newest master it can see.
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
ref: master
|
||||
# The persisted GITHUB_TOKEN header would override the automation token in the push URL and
|
||||
# get the force-push rejected. Nothing here needs it: the repo is public, so the fetch of the
|
||||
# automation branch is anonymous, and every gh call carries its own token.
|
||||
persist-credentials: false
|
||||
- name: Set up Java for cold dependency resolution
|
||||
uses: actions/setup-java@v5.2.0
|
||||
with:
|
||||
distribution: temurin
|
||||
java-version: 25
|
||||
- name: Shrink ignore limits
|
||||
run: |
|
||||
cp "$RATCHETS_FILE" "$RUNNER_TEMP/ratchets-before.edn"
|
||||
./bin/mage kondo-ratchets-shrink
|
||||
# Two independent outputs: `push` says the branch needs a new commit, `pr` names the open automation
|
||||
# PR if there is one. Approval and auto-merge key on `pr` alone, so a run that failed after opening
|
||||
# the PR is repaired by the next run instead of leaving the PR stranded.
|
||||
- name: Compare with master and the automation PR
|
||||
id: compare
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
# The REST `head=owner:branch` filter is applied server-side and only matches this repository's
|
||||
# branch, so a fork carrying a branch of the same name can never hand its PR number to the
|
||||
# approval and auto-merge steps, however many such forks exist. Only an open PR counts; a branch
|
||||
# left behind by a PR closed without merging is overwritten below and gets a new PR, so a closed
|
||||
# one never stops the budgets from being lowered.
|
||||
open_pr=$(gh api "repos/${GITHUB_REPOSITORY}/pulls?state=open&base=master&head=${GITHUB_REPOSITORY_OWNER}:${AUTOMATION_BRANCH}" \
|
||||
--jq '.[0].number // empty')
|
||||
echo "pr=$open_pr" >> "$GITHUB_OUTPUT"
|
||||
|
||||
if git diff --quiet -- "$RATCHETS_FILE"; then
|
||||
echo "Ratchet budgets already match master $(git rev-parse --short HEAD)"
|
||||
[ -z "$open_pr" ] || echo "The open automation PR #$open_pr is re-approved below"
|
||||
echo "push=false" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# The ratchet file is the whole payload, so an open PR that already carries this result is left
|
||||
# alone. Rebasing it onto every master push would restart its CI and churn the diff without
|
||||
# changing what it delivers. Compare the open PR with the working tree, which contains the
|
||||
# shrunk ratchets file produced above, to determine whether the PR would deliver the same file.
|
||||
#
|
||||
# A conflicted PR is the exception: only a fresh commit clears it. Mergeability is computed
|
||||
# lazily, so a pending UNKNOWN reads as not conflicted and the next run settles it.
|
||||
if [ -n "$open_pr" ] \
|
||||
&& git fetch --quiet --depth=1 origin "refs/heads/$AUTOMATION_BRANCH" 2>/dev/null \
|
||||
&& git diff --quiet FETCH_HEAD -- "$RATCHETS_FILE" \
|
||||
&& [ "$(gh pr view "$open_pr" --repo "$GITHUB_REPOSITORY" \
|
||||
--json mergeable --jq .mergeable)" != CONFLICTING ]; then
|
||||
echo "The automation PR #$open_pr already carries this result"
|
||||
echo "push=false" >> "$GITHUB_OUTPUT"
|
||||
echo "head=$(git rev-parse FETCH_HEAD)" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
git diff --stat -- "$RATCHETS_FILE"
|
||||
echo "push=true" >> "$GITHUB_OUTPUT"
|
||||
- name: Update the automation PR
|
||||
id: deliver
|
||||
if: steps.compare.outputs.push == 'true'
|
||||
# The automation user's token pushes the branch and owns the PR: a branch pushed with
|
||||
# GITHUB_TOKEN starts no pull_request workflows, so the PR would never get its checks. The commit
|
||||
# identity is the docs generator's, which merges with one approval under the unattributed-changes
|
||||
# rule. Nothing raises a budget.
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
|
||||
OPEN_PR: ${{ steps.compare.outputs.pr }}
|
||||
run: |
|
||||
./bin/mage -kondo-ratchets-pr-body \
|
||||
"$RUNNER_TEMP/ratchets-before.edn" \
|
||||
"$RATCHETS_FILE" \
|
||||
"${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}" \
|
||||
> "$RUNNER_TEMP/ratchets-pr-body.md"
|
||||
|
||||
git config user.name github-actions
|
||||
git config user.email github-actions@github.com
|
||||
git add -- "$RATCHETS_FILE"
|
||||
git commit --quiet -m "Tighten ratchets"
|
||||
git push --force --quiet \
|
||||
"https://x-access-token:${GH_TOKEN}@github.com/${GITHUB_REPOSITORY}.git" \
|
||||
"HEAD:refs/heads/$AUTOMATION_BRANCH"
|
||||
echo "head=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
if [ -n "$OPEN_PR" ]; then
|
||||
gh pr edit "$OPEN_PR" --body-file "$RUNNER_TEMP/ratchets-pr-body.md"
|
||||
echo "Updated the automation PR #$OPEN_PR"
|
||||
echo "pr=$OPEN_PR" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
url=$(gh pr create \
|
||||
--base master \
|
||||
--head "$AUTOMATION_BRANCH" \
|
||||
--title "Tighten ratchets" \
|
||||
--label "no-backport" \
|
||||
--body-file "$RUNNER_TEMP/ratchets-pr-body.md")
|
||||
echo "Opened the automation PR $url"
|
||||
echo "pr=${url##*/}" >> "$GITHUB_OUTPUT"
|
||||
- name: Approve the automation PR
|
||||
if: (steps.deliver.outputs.pr || steps.compare.outputs.pr) != ''
|
||||
# GITHUB_TOKEN reviews as github-actions[bot], a different identity from the automation user that
|
||||
# owns the PR, so the approval counts; a user cannot approve their own PR. Reviews survive a push,
|
||||
# so approving an already-open PR again is harmless and keeps a reopened one covered.
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
PR_NUMBER: ${{ steps.deliver.outputs.pr || steps.compare.outputs.pr }}
|
||||
run: gh pr review --approve "$PR_NUMBER"
|
||||
# Armed only against the commit this run pushed or compared: the branch name is fixed, and anyone
|
||||
# with push access could otherwise collect this run's approval for their own commit.
|
||||
- name: Enable auto-merge
|
||||
if: (steps.deliver.outputs.pr || steps.compare.outputs.pr) != ''
|
||||
&& (steps.deliver.outputs.head || steps.compare.outputs.head) != ''
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
|
||||
PR_NUMBER: ${{ steps.deliver.outputs.pr || steps.compare.outputs.pr }}
|
||||
HEAD_SHA: ${{ steps.deliver.outputs.head || steps.compare.outputs.head }}
|
||||
run: gh pr merge --squash --auto --match-head-commit "$HEAD_SHA" "$PR_NUMBER"
|
||||
15
.github/workflows/lint.yml
vendored
@@ -16,12 +16,12 @@ jobs:
|
||||
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
|
||||
timeout-minutes: 10
|
||||
outputs:
|
||||
actionlint: ${{ steps.changes.outputs.actionlint }}
|
||||
actionlint_files: ${{ steps.changes.outputs.actionlint_files }}
|
||||
codeql: ${{ steps.changes.outputs.codeql }}
|
||||
i18n: ${{ steps.changes.outputs.i18n }}
|
||||
openapi: ${{ steps.changes.outputs.openapi }}
|
||||
snowplow: ${{ steps.changes.outputs.snowplow }}
|
||||
workflows: ${{ steps.changes.outputs.workflows }}
|
||||
workflows_files: ${{ steps.changes.outputs.workflows_files }}
|
||||
yaml: ${{ steps.changes.outputs.yaml }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
@@ -52,7 +52,7 @@ jobs:
|
||||
uses: ./.github/actions/prepare-backend
|
||||
|
||||
- name: Setup actionlint
|
||||
if: needs.files-changed.outputs.actionlint == 'true'
|
||||
if: needs.files-changed.outputs.workflows == 'true'
|
||||
id: download
|
||||
shell: bash
|
||||
run: |
|
||||
@@ -81,8 +81,12 @@ jobs:
|
||||
|
||||
# Lint Github Actions
|
||||
- name: Run actionlint
|
||||
if: needs.files-changed.outputs.actionlint == 'true'
|
||||
run: ${{ steps.download.outputs.executable }} --color --verbose ${{ needs.files-changed.outputs.actionlint_files }}
|
||||
if: needs.files-changed.outputs.workflows == 'true'
|
||||
run: ${{ steps.download.outputs.executable }} --color --verbose ${{ needs.files-changed.outputs.workflows_files }}
|
||||
|
||||
- name: Check every workflow job sets a timeout
|
||||
if: needs.files-changed.outputs.workflows == 'true'
|
||||
run: .github/scripts/check-job-timeouts.sh ${{ needs.files-changed.outputs.workflows_files }}
|
||||
|
||||
# Generate and Lint OpenAPI spec
|
||||
- name: Generate OpenAPI spec
|
||||
@@ -123,6 +127,7 @@ jobs:
|
||||
needs: files-changed
|
||||
if: needs.files-changed.outputs.snowplow == 'true'
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v6
|
||||
|
||||
4
.github/workflows/mage.yml
vendored
@@ -16,3 +16,7 @@ jobs:
|
||||
run: ./bin/mage -h
|
||||
- name: Run mage tests
|
||||
run: ./bin/mage -test
|
||||
- name: Lint create-backport scripts
|
||||
run: shellcheck .github/actions/create-backport/{kondo-ratchets,create-backport-test}.sh
|
||||
- name: Run create-backport tests
|
||||
run: ./.github/actions/create-backport/create-backport-test.sh
|
||||
|
||||
26
.github/workflows/release-log.yml
vendored
@@ -45,40 +45,36 @@ jobs:
|
||||
- name: Install tsx
|
||||
run: npm i -g tsx
|
||||
- name: Get version number
|
||||
id: get-version
|
||||
uses: actions/github-script@v9
|
||||
env:
|
||||
VERSION_INPUT: ${{ inputs.version }}
|
||||
with:
|
||||
script: | # js
|
||||
const {
|
||||
getMajorVersionFromRef,
|
||||
getMajorVersion,
|
||||
} = require('${{ github.workspace }}/release/dist/index.cjs');
|
||||
const { getMajorVersionFromRef } = require('${{ github.workspace }}/release/dist/index.cjs');
|
||||
|
||||
console.log({ context });
|
||||
|
||||
// if the version is explicitly set, use that
|
||||
if (context.payload?.inputs?.version) {
|
||||
core.exportVariable('VERSION', context.payload.inputs.version);
|
||||
return;
|
||||
}
|
||||
const version = process.env.VERSION_INPUT || getMajorVersionFromRef(context.payload.ref);
|
||||
|
||||
// otherwise, get the version from the release branch name
|
||||
const ref = context.payload.ref;
|
||||
const majorVersion = getMajorVersionFromRef(ref);
|
||||
|
||||
console.log({ ref, majorVersion });
|
||||
core.exportVariable('VERSION', majorVersion);
|
||||
console.log({ version });
|
||||
core.exportVariable('VERSION', version);
|
||||
core.setOutput('version', version);
|
||||
|
||||
- name: generate release Log
|
||||
if: ${{ steps.get-version.outputs.version != '' }}
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: cd release && tsx ./src/release-log-run.ts "$VERSION" > "v$VERSION.html"
|
||||
- name: upload release log to the web
|
||||
if: ${{ steps.get-version.outputs.version != '' }}
|
||||
run: |
|
||||
aws s3 cp \
|
||||
"release/v$VERSION.html" \
|
||||
"s3://${{ vars.AWS_S3_STATIC_BUCKET }}/release-log/v$VERSION.html"
|
||||
|
||||
- name: Create cloudfront invalidation
|
||||
if: ${{ steps.get-version.outputs.version != '' }}
|
||||
run: |
|
||||
aws cloudfront create-invalidation \
|
||||
--distribution-id ${{ vars.AWS_CLOUDFRONT_STATIC_ID }} \
|
||||
|
||||
63
.github/workflows/reset-cloud-test-dataset.yml
vendored
@@ -1,63 +0,0 @@
|
||||
name: Reset Cloud Test Dataset
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
driver:
|
||||
description: 'Driver to reset dataset for'
|
||||
required: true
|
||||
default: 'snowflake'
|
||||
type: choice
|
||||
options:
|
||||
- snowflake
|
||||
- bigquery-cloud-sdk
|
||||
- redshift
|
||||
- databricks
|
||||
dataset-name:
|
||||
description: 'Dataset name to drop (e.g., test-data)'
|
||||
required: true
|
||||
default: 'test-data'
|
||||
type: string
|
||||
|
||||
jobs:
|
||||
reset-dataset:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 10
|
||||
env:
|
||||
CI: 'true'
|
||||
DRIVERS: ${{ inputs.driver }}
|
||||
# Snowflake
|
||||
MB_SNOWFLAKE_TEST_USER: METABASE CI
|
||||
MB_SNOWFLAKE_TEST_ACCOUNT: ${{ secrets.MB_SNOWFLAKE_TEST_ACCOUNT }}
|
||||
MB_SNOWFLAKE_TEST_PASSWORD: ${{ secrets.MB_SNOWFLAKE_TEST_PASSWORD }}
|
||||
MB_SNOWFLAKE_TEST_PRIVATE_KEY: ${{ secrets.MB_SNOWFLAKE_TEST_PRIVATE_KEY }}
|
||||
MB_SNOWFLAKE_TEST_WAREHOUSE: ${{ secrets.MB_SNOWFLAKE_TEST_WAREHOUSE }}
|
||||
# BigQuery
|
||||
MB_BIGQUERY_TEST_PROJECT_ID: ${{ secrets.BIGQUERY_TEST_PROJECT_ID }}
|
||||
MB_BIGQUERY_TEST_CLIENT_ID: ${{ secrets.MB_BIGQUERY_TEST_CLIENT_ID }}
|
||||
MB_BIGQUERY_TEST_CLIENT_SECRET: ${{ secrets.MB_BIGQUERY_TEST_CLIENT_SECRET }}
|
||||
MB_BIGQUERY_TEST_ACCESS_TOKEN: ${{ secrets.MB_BIGQUERY_TEST_ACCESS_TOKEN }}
|
||||
MB_BIGQUERY_TEST_REFRESH_TOKEN: ${{ secrets.MB_BIGQUERY_TEST_REFRESH_TOKEN }}
|
||||
MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON: ${{ secrets.MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON }}
|
||||
MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING: ${{ secrets.MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING }}
|
||||
# Redshift
|
||||
MB_REDSHIFT_TEST_USER: metabase_ci
|
||||
MB_REDSHIFT_TEST_DB: testdb
|
||||
MB_REDSHIFT_TEST_HOST: ${{ secrets.MB_REDSHIFT_TEST_HOST }}
|
||||
MB_REDSHIFT_TEST_PASSWORD: ${{ secrets.MB_REDSHIFT_TEST_PASSWORD }}
|
||||
# Databricks
|
||||
MB_DATABRICKS_TEST_HOST: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HOST }}
|
||||
MB_DATABRICKS_TEST_HTTP_PATH: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HTTP_PATH }}
|
||||
MB_DATABRICKS_TEST_TOKEN: ${{ secrets.MB_DATABRICKS_JDBC_TEST_TOKEN }}
|
||||
MB_DATABRICKS_TEST_CATALOG: metabase_ci
|
||||
MB_DATABRICKS_TEST_CLIENT_ID: ${{ secrets.MB_DATABRICKS_JDBC_TEST_CLIENT_ID }}
|
||||
MB_DATABRICKS_TEST_OAUTH_SECRET: ${{ secrets.MB_DATABRICKS_JDBC_TEST_OAUTH_SECRET }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: ./.github/actions/prepare-backend
|
||||
- name: Drop dataset
|
||||
run: >-
|
||||
clojure -X:dev:drivers:drivers-dev:test
|
||||
metabase.test.data.impl/drop-dataset!
|
||||
:driver '"${{ inputs.driver }}"'
|
||||
:dataset-name '"${{ inputs.dataset-name }}"'
|
||||
15
.github/workflows/run-tests.yml
vendored
@@ -20,6 +20,10 @@ permissions:
|
||||
jobs:
|
||||
# `force-run` and `force-skip` are global overrides; `defer` means this
|
||||
# job has no opinion and every downstream gate behaves as it always has.
|
||||
#
|
||||
# Only the lowest unmerged PR of a stack runs CI: it is the one whose base
|
||||
# ref is the stack base. Every PR above it is force-skipped and gets tested
|
||||
# once it reaches the bottom of the stack.
|
||||
should-run:
|
||||
name: Decide which jobs should run
|
||||
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
|
||||
@@ -35,6 +39,7 @@ jobs:
|
||||
${{ (github.ref_name == 'master' || startsWith(github.ref_name, 'release-x.')) && 'force-run'
|
||||
|| contains(github.event.pull_request.labels.*.name, 'ci:run-all') && 'force-run'
|
||||
|| contains(github.event.pull_request.labels.*.name, 'ci:skip') && 'force-skip'
|
||||
|| (github.event.pull_request.stack != null && github.event.pull_request.stack.base.ref != github.event.pull_request.base.ref) && 'force-skip'
|
||||
|| 'defer' }}
|
||||
run: echo "verdict=$VERDICT" | tee -a "$GITHUB_OUTPUT"
|
||||
|
||||
@@ -60,8 +65,13 @@ jobs:
|
||||
embedding_sdk_host_sample_apps: ${{ steps.changes.outputs.embedding_sdk_host_sample_apps }}
|
||||
embedding_sdk_ci: ${{ steps.changes.outputs.embedding_sdk_ci }}
|
||||
custom_viz_all: ${{ steps.changes.outputs.custom_viz_all }}
|
||||
project_backend_checks: ${{ steps.changes.outputs.project_backend_checks }}
|
||||
project_migration_checks: ${{ steps.changes.outputs.project_migration_checks }}
|
||||
project_ratchet_checks: ${{ steps.changes.outputs.project_ratchet_checks }}
|
||||
# A ratchets-only change (the shrink automation PR) needs the ratchet check, not an uberjar.
|
||||
no_source_code_changes: >-
|
||||
${{ steps.changes.outputs.documentation == 'true'
|
||||
${{ (steps.changes.outputs.documentation == 'true'
|
||||
|| steps.changes.outputs.project_ratchet_checks == 'true')
|
||||
&& steps.changes.outputs.backend_all == 'false'
|
||||
&& steps.changes.outputs.frontend_all == 'false'
|
||||
&& steps.changes.outputs.e2e_all == 'false' }}
|
||||
@@ -210,6 +220,9 @@ jobs:
|
||||
needs.files-changed.outputs.backend_all != 'true' &&
|
||||
needs.static-viz-files-changed.outputs.static_viz != 'true' }}
|
||||
force-run: ${{ needs.should-run.outputs.verdict == 'force-run' }}
|
||||
run-project-backend-checks: ${{ needs.files-changed.outputs.project_backend_checks == 'true' }}
|
||||
run-project-migration-checks: ${{ needs.files-changed.outputs.project_migration_checks == 'true' }}
|
||||
run-project-ratchet-checks: ${{ needs.files-changed.outputs.project_ratchet_checks == 'true' }}
|
||||
|
||||
semantic-search-tests:
|
||||
if: |
|
||||
|
||||
81
.github/workflows/test.cleanup-dwh-data.yml
vendored
Normal file
@@ -0,0 +1,81 @@
|
||||
name: Cleanup DWH Test Data
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 7 * * *'
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
drivers:
|
||||
description: 'Drivers to sweep (comma-separated)'
|
||||
required: false
|
||||
default: 'snowflake,bigquery-cloud-sdk,redshift'
|
||||
type: string
|
||||
temp-data-hours:
|
||||
description: 'TTL in hours for per-run garbage (minimum 2)'
|
||||
required: false
|
||||
default: '2'
|
||||
type: string
|
||||
fixture-hours:
|
||||
description: 'TTL in hours for datasets runs share (minimum 24)'
|
||||
required: false
|
||||
default: '96'
|
||||
type: string
|
||||
|
||||
# `inputs.*` is empty on the scheduled run, so each falls back to the nightly value.
|
||||
env:
|
||||
GC_TEMP_HOURS: ${{ inputs.temp-data-hours || 2 }} # for temp data
|
||||
GC_FIXTURE_HOURS: ${{ inputs.fixture-hours || 96 }} # for reusable data
|
||||
GC_DRIVERS: ${{ inputs.drivers || 'snowflake,bigquery-cloud-sdk,redshift' }}
|
||||
|
||||
concurrency:
|
||||
group: test-cleanup-dwh-data
|
||||
cancel-in-progress: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
gc:
|
||||
name: Sweep orphaned test data
|
||||
if: ${{ github.repository == 'metabase/metabase' }}
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 60
|
||||
env:
|
||||
CI: 'true'
|
||||
# Snowflake
|
||||
MB_SNOWFLAKE_TEST_USER: METABASE CI
|
||||
MB_SNOWFLAKE_TEST_ACCOUNT: ${{ secrets.MB_SNOWFLAKE_TEST_ACCOUNT }}
|
||||
MB_SNOWFLAKE_TEST_PASSWORD: ${{ secrets.MB_SNOWFLAKE_TEST_PASSWORD }}
|
||||
MB_SNOWFLAKE_TEST_PRIVATE_KEY: ${{ secrets.MB_SNOWFLAKE_TEST_PRIVATE_KEY }}
|
||||
MB_SNOWFLAKE_TEST_WAREHOUSE: ${{ secrets.MB_SNOWFLAKE_TEST_WAREHOUSE }}
|
||||
# BigQuery
|
||||
MB_BIGQUERY_TEST_PROJECT_ID: ${{ secrets.BIGQUERY_TEST_PROJECT_ID }}
|
||||
MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON: ${{ secrets.MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON }}
|
||||
MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING: ${{ secrets.MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING }}
|
||||
# Redshift
|
||||
MB_REDSHIFT_TEST_USER: metabase_ci
|
||||
MB_REDSHIFT_TEST_DB: testdb
|
||||
MB_REDSHIFT_TEST_DB_ROUTING: dev
|
||||
MB_REDSHIFT_TEST_HOSTS: ${{ secrets.MB_REDSHIFT_TEST_HOSTS }}
|
||||
MB_REDSHIFT_TEST_HOST: ${{ secrets.MB_REDSHIFT_TEST_HOST }}
|
||||
MB_REDSHIFT_TEST_PASSWORD: ${{ secrets.MB_REDSHIFT_TEST_PASSWORD }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: ./.github/actions/prepare-backend
|
||||
- name: Sweep orphaned test data
|
||||
run: |
|
||||
clojure -X:dev:drivers:drivers-dev:test metabase.test.data.gc/gc-orphans! \
|
||||
:drivers "\"${{ env.GC_DRIVERS }}\"" \
|
||||
:temp-data-hours "${{ env.GC_TEMP_HOURS }}" \
|
||||
:fixture-hours "${{ env.GC_FIXTURE_HOURS }}" \
|
||||
:report-dir '"target/gc-report"'
|
||||
|
||||
# The report is most wanted on the runs that fail, so upload it regardless of the sweep's outcome.
|
||||
- name: Upload sweep report
|
||||
if: ${{ !cancelled() }}
|
||||
continue-on-error: true
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: dwh-gc-report-${{ github.run_id }}-attempt-${{ github.run_attempt }}
|
||||
path: target/gc-report
|
||||
if-no-files-found: warn
|
||||
|
After Width: | Height: | Size: 26 KiB |
|
Before Width: | Height: | Size: 40 KiB After Width: | Height: | Size: 40 KiB |
|
Before Width: | Height: | Size: 40 KiB After Width: | Height: | Size: 40 KiB |
|
Before Width: | Height: | Size: 7.2 KiB After Width: | Height: | Size: 8.0 KiB |
|
Before Width: | Height: | Size: 67 KiB After Width: | Height: | Size: 68 KiB |
|
Before Width: | Height: | Size: 67 KiB After Width: | Height: | Size: 68 KiB |
|
Before Width: | Height: | Size: 60 KiB After Width: | Height: | Size: 61 KiB |
|
Before Width: | Height: | Size: 60 KiB After Width: | Height: | Size: 61 KiB |
|
Before Width: | Height: | Size: 66 KiB After Width: | Height: | Size: 66 KiB |
|
Before Width: | Height: | Size: 66 KiB After Width: | Height: | Size: 67 KiB |
|
Before Width: | Height: | Size: 59 KiB After Width: | Height: | Size: 60 KiB |
|
Before Width: | Height: | Size: 59 KiB After Width: | Height: | Size: 60 KiB |
|
Before Width: | Height: | Size: 66 KiB After Width: | Height: | Size: 66 KiB |
|
Before Width: | Height: | Size: 66 KiB After Width: | Height: | Size: 66 KiB |
|
Before Width: | Height: | Size: 59 KiB After Width: | Height: | Size: 60 KiB |
|
Before Width: | Height: | Size: 59 KiB After Width: | Height: | Size: 60 KiB |
|
Before Width: | Height: | Size: 29 KiB After Width: | Height: | Size: 29 KiB |
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 26 KiB |
|
Before Width: | Height: | Size: 29 KiB After Width: | Height: | Size: 29 KiB |
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 27 KiB |
|
Before Width: | Height: | Size: 24 KiB After Width: | Height: | Size: 25 KiB |
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 26 KiB |
|
Before Width: | Height: | Size: 23 KiB After Width: | Height: | Size: 24 KiB |
|
Before Width: | Height: | Size: 31 KiB After Width: | Height: | Size: 32 KiB |
|
Before Width: | Height: | Size: 25 KiB After Width: | Height: | Size: 26 KiB |
|
Before Width: | Height: | Size: 25 KiB After Width: | Height: | Size: 25 KiB |
|
Before Width: | Height: | Size: 24 KiB After Width: | Height: | Size: 25 KiB |
|
Before Width: | Height: | Size: 32 KiB After Width: | Height: | Size: 32 KiB |
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 27 KiB |
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 27 KiB |
|
Before Width: | Height: | Size: 83 KiB After Width: | Height: | Size: 84 KiB |
|
Before Width: | Height: | Size: 76 KiB After Width: | Height: | Size: 78 KiB |
|
Before Width: | Height: | Size: 72 KiB After Width: | Height: | Size: 73 KiB |
|
Before Width: | Height: | Size: 74 KiB After Width: | Height: | Size: 76 KiB |
|
Before Width: | Height: | Size: 89 KiB After Width: | Height: | Size: 90 KiB |
|
Before Width: | Height: | Size: 78 KiB After Width: | Height: | Size: 79 KiB |
|
Before Width: | Height: | Size: 85 KiB After Width: | Height: | Size: 87 KiB |
|
Before Width: | Height: | Size: 71 KiB After Width: | Height: | Size: 72 KiB |
|
Before Width: | Height: | Size: 76 KiB After Width: | Height: | Size: 78 KiB |
|
Before Width: | Height: | Size: 74 KiB After Width: | Height: | Size: 75 KiB |
|
Before Width: | Height: | Size: 75 KiB After Width: | Height: | Size: 76 KiB |
|
Before Width: | Height: | Size: 76 KiB After Width: | Height: | Size: 78 KiB |
|
Before Width: | Height: | Size: 75 KiB After Width: | Height: | Size: 75 KiB |
|
Before Width: | Height: | Size: 71 KiB After Width: | Height: | Size: 72 KiB |
|
Before Width: | Height: | Size: 71 KiB After Width: | Height: | Size: 72 KiB |
|
Before Width: | Height: | Size: 74 KiB After Width: | Height: | Size: 75 KiB |
|
Before Width: | Height: | Size: 82 KiB After Width: | Height: | Size: 83 KiB |
|
Before Width: | Height: | Size: 75 KiB After Width: | Height: | Size: 76 KiB |
|
Before Width: | Height: | Size: 72 KiB After Width: | Height: | Size: 73 KiB |
|
Before Width: | Height: | Size: 74 KiB After Width: | Height: | Size: 74 KiB |
|
Before Width: | Height: | Size: 88 KiB After Width: | Height: | Size: 89 KiB |
|
Before Width: | Height: | Size: 78 KiB After Width: | Height: | Size: 79 KiB |