Compare commits
2 Commits
split-api-
...
debug-push
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d3c9e3c2b1 | ||
|
|
695bc9bbe1 |
@@ -27,28 +27,9 @@ This guide covers Clojure and ClojureScript coding conventions for Metabase. See
|
||||
- Format using Markdown conventions
|
||||
- Reference other vars with `[[other-var]]` not backticks
|
||||
|
||||
A docstring states the *contract*: what it returns (including when it returns nil), what its
|
||||
params mean, what it throws versus swallows, and any precondition or invariant the caller must
|
||||
respect. A params block for non-obvious or optional args is good documentation — length is not
|
||||
the problem.
|
||||
|
||||
These seven things are, and each is a reason to cut or relocate:
|
||||
|
||||
| Anti-pattern | Why | Where it goes instead |
|
||||
| --- | --- | --- |
|
||||
| **Facts you don't own** — how a *callee*, another namespace, or another repo works internally | Drifts silently; nothing fails when it goes stale | Cut. Name the callee and let its own docstring say it |
|
||||
| **Said twice** — ns docstring restating its vars', a caller restating its callee's, sibling fns repeating boilerplate | Two copies means one is already wrong | Cut the copy. One fact, one owner |
|
||||
| **Design rationale** — why this approach, why it lives in this module, what the alternative was | The caller can't act on it | Commit message, or an inline comment on the specific line it constrains |
|
||||
| **Development history** — "matching master", "the X branch", "currently", "for now", "used to" | Stale by construction | Cut |
|
||||
| **Instructions to the reader** — "callers should trust this", "don't bother checking", "you should use X" | State the contract; the caller decides | Cut, keeping any actual guarantee as a plain statement |
|
||||
| **Invented precision** — score bands, thresholds, or tables no code enforces | Reads as spec, isn't | Cut, or make the code enforce it |
|
||||
| **A note on the wrong var** — describing how overrides behave, on the base/default impl | Nobody reading the override sees it | Move to the overrides it describes |
|
||||
|
||||
**Comments:**
|
||||
|
||||
- `TODO` format: `;; TODO (Name YYYY-MM-DD) -- description`
|
||||
- Prefer an inline comment over docstring prose for anything that explains *the body*. Put it on
|
||||
the line it explains, not at the top of the function.
|
||||
|
||||
## Code Organization
|
||||
|
||||
|
||||
@@ -6,4 +6,3 @@
|
||||
- Run targeted tests, and lint continuously during development
|
||||
- Prioritize understanding existing patterns before implementing
|
||||
- Don't commit changes, leave it for the user to review and make commits
|
||||
- Minimize comments. Document only non-obvious intent, trade-offs, or constraints that the code itself cannot convey. Do not narrate what the code does — keep code self-descriptive through naming and structure.
|
||||
|
||||
@@ -56,11 +56,6 @@ Use this to scan through changes efficiently:
|
||||
- [ ] References use `[[other-var]]` not backticks
|
||||
- [ ] `TODO` comments include author and date: `;; TODO (Name 2025-01-01) -- description`
|
||||
|
||||
Docstring content — check every new or rewritten docstring against the anti-pattern table in the
|
||||
style guide above. Read the diff's docstrings on their own, separately from the code: prose is
|
||||
where review attention slides off, and a long docstring reads as diligence rather than as the
|
||||
liability it usually is.
|
||||
|
||||
### Code Organization
|
||||
|
||||
- [ ] Everything `^:private` unless used elsewhere
|
||||
@@ -140,17 +135,6 @@ Quick scan for common issues:
|
||||
|
||||
> This public var needs a docstring explaining its purpose, inputs, and outputs.
|
||||
|
||||
**For docstrings documenting code they don't own:**
|
||||
|
||||
> This describes how `lib` decides binning strategies internally. Nothing here fails when that
|
||||
> changes, so it will go stale silently — drop it and let `available-binning-strategies` answer
|
||||
> for its own behavior.
|
||||
|
||||
**For docstring content that belongs in the body:**
|
||||
|
||||
> This paragraph explains why the branches are ordered this way, which is what someone editing
|
||||
> the `cond` needs — but a caller can't act on it. Move it to an inline comment above the `cond`.
|
||||
|
||||
**For organization issues:**
|
||||
|
||||
> This function is only used in this namespace, so it should be marked `^:private`.
|
||||
|
||||
@@ -82,29 +82,10 @@ either a bug, or it means callers must handle each case themselves — and
|
||||
in *that* case it is the *absence* of the property that must be
|
||||
documented. Document deviations from expectation, not conformance to it.
|
||||
|
||||
Heuristic — a sentence earns its place in the docstring only if it
|
||||
passes both tests:
|
||||
|
||||
1. Rewrite test (necessary, not sufficient): if rewriting the body to a
|
||||
different implementation with an identical contract could make the
|
||||
sentence false, it describes the implementation, not the contract.
|
||||
Relocate it to an inline comment at the point in the body where it
|
||||
applies.
|
||||
|
||||
2. Ownership test: if the sentence states a fact about code this
|
||||
function does not own — a callee's behavior, a library's guarantee —
|
||||
it survives any rewrite trivially, which is exactly why the rewrite
|
||||
test alone is not enough. If this body *relies* on that fact, it is
|
||||
implementation context: relocate it to the call site. If nothing here
|
||||
relies on it, it is blather about someone else's code: delete it, and
|
||||
let the callee's own docstring answer for it.
|
||||
|
||||
Say each fact once. Duplicated prose is a stale comment in advance: when
|
||||
the fact changes, one copy gets updated and the rest quietly rot. This
|
||||
holds even within code you own — a namespace docstring that
|
||||
re-summarizes its own multimethods, three sibling functions carrying the
|
||||
same paragraph about retry behavior. Give the fact one home, and have
|
||||
the other places name it rather than restate it.
|
||||
Heuristic: if a sentence would still be true after a full rewrite of the
|
||||
body, it may belong in the docstring. If it describes *how the current
|
||||
body works*, it belongs in the body — as an inline comment, if it is
|
||||
non-obvious.
|
||||
|
||||
Multi-line docstrings are not banned — a genuinely non-obvious constraint
|
||||
the code had to deal with can be worth explaining. But be prudent; the
|
||||
|
||||
@@ -504,9 +504,7 @@
|
||||
metabase.query-processor.pipeline-queries-test/mbql5-query clojure.core/->
|
||||
metabase.query-processor.pipeline-queries-test/run-mbql5-query clojure.core/->
|
||||
metabase.query-processor.error-type/deferror clojure.core/def
|
||||
metabase.query-processor.core/with-reducible-deserialized-results clojure.core/let
|
||||
metabase.query-processor.middleware.cache.impl/with-reducible-deserialized-results clojure.core/let
|
||||
metabase.query-processor.result-serialization/with-reducible-deserialized-results clojure.core/let
|
||||
metabase.query-processor.middleware.process-userland-query-test/with-query-execution! clojure.core/let
|
||||
metabase.query-processor.setup/with-qp-setup clojure.core/let
|
||||
metabase.search.api-test/do-test-users clojure.core/let
|
||||
@@ -523,6 +521,7 @@
|
||||
metabase.test.data.interface/defdataset clojure.core/def
|
||||
metabase.test.data.interface/defdataset-edn clojure.core/def
|
||||
metabase.test.embedded-postgres.core/with-system clojure.core/let
|
||||
metabase.test.gentest/defgentest clojure.test/deftest
|
||||
metabase.test/defdataset clojure.core/def
|
||||
metabase.test/let-url clojure.core/let
|
||||
metabase.test/with-actions clojure.core/let
|
||||
@@ -681,6 +680,7 @@
|
||||
metabase.test.data/dataset hooks.metabase.test.data/dataset
|
||||
metabase.test.data/mbql-query hooks.metabase.test.data/mbql-query
|
||||
metabase.test.data/run-mbql-query hooks.metabase.test.data/mbql-query
|
||||
metabase.test.gentest/iterate hooks.metabase.test.gentest/iterate*
|
||||
metabase.test.util.async/with-open-channels hooks.common/let-with-optional-value-for-last-binding
|
||||
metabase.test.util.generators.jvm/with-random-cards hooks.metabase.test.util.generators.jvm/with-random-cards
|
||||
metabase.test.util.log/with-log-level hooks.common/with-ignored-first-arg
|
||||
@@ -860,11 +860,6 @@
|
||||
metabase.util.time-test
|
||||
{:linters {:metabase/prefer-with-dynamic-fn-redefs {:level :off}}}
|
||||
|
||||
;; classloader sits below `util` in the module graph, so it can't use `metabase.util.log` without creating
|
||||
;; a cycle; it gets to use tools.logging directly.
|
||||
metabase.classloader.impl
|
||||
{:linters {:discouraged-namespace {clojure.tools.logging {:level :off}}}}
|
||||
|
||||
source-namespaces
|
||||
{:linters
|
||||
{:metabase/discourage-millis-duration {:level :warning}
|
||||
|
||||
@@ -302,7 +302,6 @@
|
||||
:model/Database
|
||||
:model/Dimension
|
||||
:model/Document
|
||||
:model/Exploration
|
||||
:model/Measure
|
||||
:model/ModelIndexValue
|
||||
:model/NativeQuerySnippet
|
||||
@@ -446,8 +445,7 @@
|
||||
:model-imports #{:model/Card
|
||||
:model/Collection
|
||||
:model/Dashboard
|
||||
:model/Document
|
||||
:model/Exploration}}
|
||||
:model/Document}}
|
||||
|
||||
bug-reporting
|
||||
{:team "UX West"
|
||||
@@ -541,7 +539,7 @@
|
||||
{:team "DevEx"
|
||||
:api #{metabase.classloader.core
|
||||
metabase.classloader.init}
|
||||
:uses #{}}
|
||||
:uses #{util}}
|
||||
|
||||
cloud-migration
|
||||
{:team "UX West"
|
||||
@@ -612,7 +610,6 @@
|
||||
:model/DashboardCard
|
||||
:model/DashboardCardSeries
|
||||
:model/Document
|
||||
:model/Exploration
|
||||
:model/NativeQuerySnippet
|
||||
:model/Permissions
|
||||
:model/Pulse
|
||||
@@ -647,7 +644,6 @@
|
||||
:model/Collection
|
||||
:model/Dashboard
|
||||
:model/Document
|
||||
:model/Exploration
|
||||
:model/NativeQuerySnippet
|
||||
:model/Pulse
|
||||
:model/Table
|
||||
@@ -667,7 +663,7 @@
|
||||
users
|
||||
util}
|
||||
:model-exports #{:model/Comment}
|
||||
:model-imports #{:model/Document :model/Exploration :model/User}}
|
||||
:model-imports #{:model/Document :model/User}}
|
||||
|
||||
;; technically this 'uses' `enterprise/core` and `test` since it tries to load them to see if they exists so we know
|
||||
;; if EE/test code is available; however we can ignore them since they're not real usages.
|
||||
@@ -699,14 +695,6 @@
|
||||
:model-exports #{:model/ModerationReview}
|
||||
:model-imports #{:model/Card :model/Dashboard}}
|
||||
|
||||
contextual-interestingness
|
||||
{:team "UX West"
|
||||
:api #{metabase.contextual-interestingness.core}
|
||||
:uses #{interestingness
|
||||
metabot
|
||||
query-processor
|
||||
util}}
|
||||
|
||||
core
|
||||
{:team "UX West"
|
||||
:api #{}
|
||||
@@ -936,7 +924,6 @@
|
||||
:model/DashboardTab
|
||||
:model/Dimension
|
||||
:model/Document
|
||||
:model/Exploration
|
||||
:model/Measure
|
||||
:model/NativeQuerySnippet
|
||||
:model/PermissionsGroup
|
||||
@@ -1011,47 +998,6 @@
|
||||
:model/Table
|
||||
:model/User}}
|
||||
|
||||
explorations
|
||||
{:team "UX West"
|
||||
:api #{metabase.explorations.api
|
||||
metabase.explorations.core
|
||||
metabase.explorations.init}
|
||||
:uses #{analytics
|
||||
analytics-interface
|
||||
api
|
||||
app-db
|
||||
collections
|
||||
contextual-interestingness
|
||||
events
|
||||
interestingness
|
||||
lib
|
||||
lib-be
|
||||
lib-metric
|
||||
metrics
|
||||
models
|
||||
mq
|
||||
permissions
|
||||
queries
|
||||
query-permissions
|
||||
query-processor
|
||||
request
|
||||
search
|
||||
settings
|
||||
task
|
||||
task-history
|
||||
timeline
|
||||
types
|
||||
util}
|
||||
:model-exports #{:model/Exploration}
|
||||
:model-imports #{:model/Card
|
||||
:model/Collection
|
||||
:model/Comment
|
||||
:model/Segment
|
||||
:model/StoredResult
|
||||
:model/StoredResultUse
|
||||
:model/Timeline
|
||||
:model/User}}
|
||||
|
||||
formatter
|
||||
{:team "UX West"
|
||||
:api #{metabase.formatter.core}
|
||||
@@ -1206,6 +1152,7 @@
|
||||
metabase.lib.schema.ref
|
||||
metabase.lib.schema.template-tag
|
||||
metabase.lib.schema.temporal-bucketing
|
||||
metabase.lib.schema.test-spec
|
||||
metabase.lib.schema.util
|
||||
metabase.lib.schema.validate
|
||||
metabase.lib.types.isa
|
||||
@@ -1261,7 +1208,6 @@
|
||||
metabase.llm.startup}
|
||||
:uses #{analytics
|
||||
api
|
||||
config
|
||||
driver
|
||||
lib
|
||||
lib-be
|
||||
@@ -1377,8 +1323,8 @@
|
||||
driver
|
||||
entity-retrieval
|
||||
events
|
||||
explorations
|
||||
interestingness
|
||||
legacy-mbql
|
||||
lib
|
||||
lib-be
|
||||
llm
|
||||
@@ -1402,7 +1348,6 @@
|
||||
sync
|
||||
system
|
||||
task
|
||||
timeline
|
||||
transforms
|
||||
transforms-base
|
||||
util
|
||||
@@ -1434,6 +1379,7 @@
|
||||
:api #{metabase.metrics.api
|
||||
metabase.metrics.core}
|
||||
:uses #{api
|
||||
collections
|
||||
events
|
||||
lib
|
||||
lib-be
|
||||
@@ -1441,7 +1387,6 @@
|
||||
models
|
||||
parameters
|
||||
permissions
|
||||
queries
|
||||
query-processor
|
||||
request
|
||||
server
|
||||
@@ -1505,8 +1450,7 @@
|
||||
|
||||
mq
|
||||
{:team "UX West"
|
||||
:api #{metabase.mq.core
|
||||
metabase.mq.init}
|
||||
:api #{metabase.mq.init}
|
||||
:uses #{analytics-interface
|
||||
app-db
|
||||
classloader
|
||||
@@ -1791,7 +1735,6 @@
|
||||
lib
|
||||
models
|
||||
notification
|
||||
parameters
|
||||
permissions
|
||||
premium-features
|
||||
query-processor
|
||||
@@ -1858,9 +1801,7 @@
|
||||
:model-exports #{:model/Card
|
||||
:model/Query
|
||||
:model/QueryExecution
|
||||
:model/QueryTable
|
||||
:model/StoredResult
|
||||
:model/StoredResultUse}
|
||||
:model/QueryTable}
|
||||
:model-imports #{:model/Action
|
||||
:model/CardBookmark
|
||||
:model/Collection
|
||||
@@ -2044,7 +1985,6 @@
|
||||
:model/DashboardCard
|
||||
:model/DashboardTab
|
||||
:model/Document
|
||||
:model/Exploration
|
||||
:model/Measure
|
||||
:model/Segment
|
||||
:model/Transform}}
|
||||
@@ -2577,8 +2517,9 @@
|
||||
|
||||
typed-schemas
|
||||
{:team "Embedding"
|
||||
:api #{metabase.typed-schemas.core}
|
||||
:api #{metabase.typed-schemas.api}
|
||||
:uses #{actions
|
||||
api
|
||||
collections
|
||||
lib
|
||||
lib-be
|
||||
@@ -2597,13 +2538,6 @@
|
||||
:model/Measure
|
||||
:model/Table}}
|
||||
|
||||
typed-schemas-rest
|
||||
{:team "Embedding"
|
||||
:api #{metabase.typed-schemas-rest.api}
|
||||
:uses #{api
|
||||
typed-schemas
|
||||
util}}
|
||||
|
||||
types
|
||||
{:team "Querying Platform"
|
||||
:api #{metabase.types.core
|
||||
@@ -2725,7 +2659,8 @@
|
||||
:api :any
|
||||
:uses #{classloader
|
||||
config
|
||||
settings}}
|
||||
settings
|
||||
system}}
|
||||
|
||||
version
|
||||
{:team "UX West"
|
||||
|
||||
@@ -5,27 +5,27 @@
|
||||
;; to defend in your PR.
|
||||
;; :all is the vector-less ignore form, which suppresses every linter on the next form.
|
||||
{:ignore-counts {:aliased-namespace-symbol 3
|
||||
:all 49
|
||||
:all 50
|
||||
:case-symbol-test 1
|
||||
:clojure-lsp/unused-public-var 9
|
||||
:def-fn 1
|
||||
:deprecated-namespace 95
|
||||
:deprecated-var 79
|
||||
:discouraged-java-method 3
|
||||
:discouraged-namespace 79
|
||||
:discouraged-var 107
|
||||
:discouraged-namespace 80
|
||||
:discouraged-var 106
|
||||
:equals-true 1
|
||||
:main-without-gen-class 1
|
||||
:metabase/defsetting-namespace 2
|
||||
:metabase/defsetting-namespace 7
|
||||
:metabase/disallow-hardcoded-driver-names-in-tests 34
|
||||
:metabase/discourage-millis-duration 1
|
||||
:metabase/i-like-making-cams-eyes-bleed-with-horrifically-long-tests 13
|
||||
:metabase/modules 7
|
||||
:metabase/modules 8
|
||||
:metabase/namespace-name 4
|
||||
:metabase/prefer-with-dynamic-fn-redefs 28
|
||||
:metabase/test-helpers-use-non-thread-safe-functions 17
|
||||
:metabase/validate-defendpoint-has-response-schema 441
|
||||
:metabase/validate-defendpoint-query-params-use-kebab-case 47
|
||||
:metabase/validate-defendpoint-query-params-use-kebab-case 40
|
||||
:metabase/validate-defendpoint-route-uses-kebab-case 44
|
||||
:metabase/validate-deftest 9
|
||||
:missing-docstring 10
|
||||
|
||||
14
.clj-kondo/src/hooks/metabase/test/gentest.clj
Normal file
14
.clj-kondo/src/hooks/metabase/test/gentest.clj
Normal file
@@ -0,0 +1,14 @@
|
||||
(ns hooks.metabase.test.gentest
|
||||
(:require
|
||||
[clj-kondo.hooks-api :as hooks]))
|
||||
|
||||
(defn iterate*
|
||||
"(iterate {:as limit-spec} [s1 val1 & more] body-form body-form...)"
|
||||
[{{:keys [children] :as node} :node :as orig}]
|
||||
(let [[token limit-spec bindings & body-nodes] children
|
||||
node* (-> (hooks/list-node
|
||||
(list token limit-spec
|
||||
(hooks/list-node (list* (hooks/token-node 'let) bindings
|
||||
body-nodes))))
|
||||
(with-meta (meta node)))]
|
||||
(assoc orig :node node*)))
|
||||
13
.github/actionlint.yaml
vendored
13
.github/actionlint.yaml
vendored
@@ -1,13 +0,0 @@
|
||||
self-hosted-runner:
|
||||
# Custom runner labels that actionlint cannot know about. Everything else is
|
||||
# selected through `vars.DEFAULT_RUNNER_KEY` and is not checked.
|
||||
labels:
|
||||
- arm-ubuntu-24-2core
|
||||
|
||||
paths:
|
||||
# GitHub added parallel steps (`parallel`, `background`, `wait`, `wait-all`,
|
||||
# `cancel`) in June 2026, and actionlint 1.7.12 does not know the schema yet:
|
||||
# https://github.blog/changelog/2026-06-25-actions-steps-can-now-be-run-in-parallel/
|
||||
.github/workflows/lint.yml:
|
||||
ignore:
|
||||
- 'step must run script with "run" section or run action with "uses" section'
|
||||
7
.github/actions/create-backport/action.yml
vendored
7
.github/actions/create-backport/action.yml
vendored
@@ -1,5 +1,4 @@
|
||||
name: Create Backport PR
|
||||
description: Cherry-picks a commit onto a release branch and opens the backport pull request.
|
||||
|
||||
inputs:
|
||||
target-version: # major version number like 34 or 86
|
||||
@@ -43,7 +42,7 @@ runs:
|
||||
git cherry-pick --abort
|
||||
|
||||
echo "Could not cherry pick because of a conflict"
|
||||
echo "has-conflicts=true" >> "$GITHUB_OUTPUT"
|
||||
echo "has-conflicts=true" >> $GITHUB_OUTPUT
|
||||
|
||||
# Add a shell script for resolving conflicts
|
||||
echo "git reset HEAD~1" > ./backport.sh
|
||||
@@ -67,7 +66,7 @@ runs:
|
||||
END
|
||||
)
|
||||
else
|
||||
echo "has-conflicts=false" >> "$GITHUB_OUTPUT"
|
||||
echo "has-conflicts=false" >> $GITHUB_OUTPUT
|
||||
PR_BODY="#${ORIGINAL_PULL_REQUEST_NUMBER}"
|
||||
fi
|
||||
|
||||
@@ -81,7 +80,7 @@ runs:
|
||||
BACKPORT_PR_URL=$(gh pr create "${PR_ARGS[@]}")
|
||||
BACKPORT_PR_NUMBER=${BACKPORT_PR_URL##*/}
|
||||
|
||||
echo "backport_pr_number=$BACKPORT_PR_NUMBER" >> "$GITHUB_OUTPUT"
|
||||
echo "backport_pr_number=$BACKPORT_PR_NUMBER" >> $GITHUB_OUTPUT
|
||||
echo "New PR has been created: #${BACKPORT_PR_NUMBER}"
|
||||
|
||||
git checkout ${{ github.sha }}
|
||||
|
||||
4
.github/actions/fetch-artifact/action.yml
vendored
4
.github/actions/fetch-artifact/action.yml
vendored
@@ -113,9 +113,9 @@ runs:
|
||||
|
||||
# Set the JAR_PATH environment variable
|
||||
if [ "$EXTRACT_PATH" == "." ]; then
|
||||
echo "JAR_PATH=$OUTPUT_NAME" >> "$GITHUB_ENV"
|
||||
echo "JAR_PATH=$OUTPUT_NAME" >> $GITHUB_ENV
|
||||
else
|
||||
echo "JAR_PATH=$EXTRACT_PATH/$OUTPUT_NAME" >> "$GITHUB_ENV"
|
||||
echo "JAR_PATH=$EXTRACT_PATH/$OUTPUT_NAME" >> $GITHUB_ENV
|
||||
fi
|
||||
|
||||
- name: Verify that this is a valid JAR file
|
||||
|
||||
@@ -22,7 +22,7 @@ runs:
|
||||
COMMIT=$(env -i git log $BASE_REF --grep="(#$PULL_REQUEST_NUMBER)" --format="%H")
|
||||
echo "commit SHA $COMMIT"
|
||||
|
||||
echo "commit=$COMMIT" >> "$GITHUB_OUTPUT"
|
||||
echo "commit=$COMMIT" >> $GITHUB_OUTPUT
|
||||
id: find-squashed-commit
|
||||
shell: bash
|
||||
env:
|
||||
|
||||
12
.github/actions/get-cache-keys/action.yml
vendored
12
.github/actions/get-cache-keys/action.yml
vendored
@@ -36,10 +36,10 @@ runs:
|
||||
WEEK_OF_YEAR=$(/bin/date "+%G-%V")
|
||||
CACHE_SUFFIX="${{ runner.os }}-$WEEK_OF_YEAR"
|
||||
|
||||
echo "m2-cache-key=$M2_CACHE_PREFIX-$CACHE_SUFFIX" >> "$GITHUB_OUTPUT"
|
||||
echo "m2-restore-key=$M2_CACHE_PREFIX-" >> "$GITHUB_OUTPUT"
|
||||
echo "node-cache-key=$NODE_CACHE_PREFIX-$CACHE_SUFFIX" >> "$GITHUB_OUTPUT"
|
||||
echo "node-restore-key=$NODE_CACHE_PREFIX-" >> "$GITHUB_OUTPUT"
|
||||
echo "eslint-cache-key=$ESLINT_CACHE_PREFIX-$CACHE_SUFFIX" >> "$GITHUB_OUTPUT"
|
||||
echo "cypress-cache-key=$CYPRESS_CACHE_PREFIX-$CACHE_SUFFIX" >> "$GITHUB_OUTPUT"
|
||||
echo "m2-cache-key=$M2_CACHE_PREFIX-$CACHE_SUFFIX" >> $GITHUB_OUTPUT
|
||||
echo "m2-restore-key=$M2_CACHE_PREFIX-" >> $GITHUB_OUTPUT
|
||||
echo "node-cache-key=$NODE_CACHE_PREFIX-$CACHE_SUFFIX" >> $GITHUB_OUTPUT
|
||||
echo "node-restore-key=$NODE_CACHE_PREFIX-" >> $GITHUB_OUTPUT
|
||||
echo "eslint-cache-key=$ESLINT_CACHE_PREFIX-$CACHE_SUFFIX" >> $GITHUB_OUTPUT
|
||||
echo "cypress-cache-key=$CYPRESS_CACHE_PREFIX-$CACHE_SUFFIX" >> $GITHUB_OUTPUT
|
||||
shell: bash
|
||||
|
||||
10
.github/actions/prepare-frontend/action.yml
vendored
10
.github/actions/prepare-frontend/action.yml
vendored
@@ -35,7 +35,7 @@ runs:
|
||||
run: |
|
||||
BUN_CACHE_DIR=$(bun pm cache)
|
||||
echo "Bun cache directory: $BUN_CACHE_DIR"
|
||||
echo "dir=$BUN_CACHE_DIR" >> "$GITHUB_OUTPUT"
|
||||
echo "dir=$BUN_CACHE_DIR" >> $GITHUB_OUTPUT
|
||||
shell: bash
|
||||
|
||||
- name: Restore node_modules cache
|
||||
@@ -53,13 +53,7 @@ runs:
|
||||
# If a job wants to patch packages, it does so explicitly in the CI. This setting is opt-out.
|
||||
# For example, we never want to cache node_modules with patches already applied so cache-generator.yml skips that step.
|
||||
- name: Clean install
|
||||
# Remove the restored node_modules before installing. The os-week cache can
|
||||
# carry a stale tree (for example a different React major), and
|
||||
# `--frozen-lockfile` alone does not prune it. Rebuilding from the cached bun
|
||||
# store gives an exact, lockfile-correct node_modules with no extra download.
|
||||
run: |
|
||||
rm -rf node_modules
|
||||
bun install --frozen-lockfile
|
||||
run: bun install --frozen-lockfile
|
||||
shell: bash
|
||||
- name: Patch frontend dependencies
|
||||
if: inputs.apply-patches == 'true'
|
||||
|
||||
@@ -88,9 +88,9 @@ runs:
|
||||
REPO_URL="https://github.com/metabase/shoppy.git"
|
||||
echo "Checking if branch '${SAMPLE_APP_BRANCH_NAME}' exists in ${REPO_URL}..."
|
||||
if git ls-remote --exit-code --heads "$REPO_URL" "${SAMPLE_APP_BRANCH_NAME}"; then
|
||||
echo "sample_app_branch_exists=true" >> "$GITHUB_OUTPUT"
|
||||
echo "sample_app_branch_exists=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "sample_app_branch_exists=false" >> "$GITHUB_OUTPUT"
|
||||
echo "sample_app_branch_exists=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Configure AWS credentials
|
||||
|
||||
26
.github/actions/prepare-uberjar-artifact/action.yml
vendored
Normal file
26
.github/actions/prepare-uberjar-artifact/action.yml
vendored
Normal file
@@ -0,0 +1,26 @@
|
||||
name: Prepare Uberjar Artifact
|
||||
description: Prepare and upload Metabase JAR as a GitHub artifact.
|
||||
|
||||
inputs:
|
||||
edition:
|
||||
required: false
|
||||
default: "ee"
|
||||
description: Metabase edition (ee or oss)
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Mark with the commit hash
|
||||
run: git rev-parse --short HEAD > COMMIT-ID
|
||||
shell: bash
|
||||
- name: Calculate SHA256 checksum
|
||||
run: sha256sum ./target/uberjar/metabase.jar > SHA256.sum
|
||||
shell: bash
|
||||
- name: Upload JARs as artifact
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: metabase-${{ inputs.edition }}-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
|
||||
path: |
|
||||
./target/uberjar/metabase.jar
|
||||
./COMMIT-ID
|
||||
./SHA256.sum
|
||||
34
.github/actions/test-driver/action.yml
vendored
34
.github/actions/test-driver/action.yml
vendored
@@ -22,6 +22,11 @@ inputs:
|
||||
description: Edition to test (oss or ee)
|
||||
quarantine-engine:
|
||||
required: false
|
||||
driver-status:
|
||||
required: false
|
||||
default: ""
|
||||
description: >-
|
||||
Configured status for this driver ("info", "skip", "required", or empty), as reported by `mage driver-decisions`.
|
||||
granular-rerun:
|
||||
required: false
|
||||
default: "true"
|
||||
@@ -67,30 +72,25 @@ runs:
|
||||
# Workflow expressions have no arithmetic, so the shell computes it.
|
||||
#
|
||||
# run-tests.yml calls app-db.yml, backend.yml, drivers.yml and semantic-search.yml
|
||||
# as part of ONE run, so all four share an artifact namespace. `junit-name` alone is not
|
||||
# unique there — app-db.yml and drivers.yml both derive `be-tests-postgres-ee` — so
|
||||
# `artifact-suffix` carries the rest of the discriminator.
|
||||
# into ONE run, so all four share an artifact namespace. Neither `github.job`
|
||||
# (`be-tests-postgres` exists in app-db.yml and drivers.yml) nor `junit-name`
|
||||
# (both derive `be-tests-postgres-ee`) is unique there on its own; together with
|
||||
# `artifact-suffix` they are.
|
||||
- name: Resolve rerun artifact names
|
||||
if: ${{ inputs.granular-rerun == 'true' }}
|
||||
shell: bash
|
||||
env:
|
||||
RERUN_ARTIFACT_BASE: ${{ format('failed-tests-{0}{1}', inputs.junit-name, inputs.artifact-suffix && format('-{0}', inputs.artifact-suffix) || '') }}
|
||||
RERUN_ARTIFACT_BASE: ${{ format('failed-tests-{0}-{1}{2}', github.job, inputs.junit-name, inputs.artifact-suffix && format('-{0}', inputs.artifact-suffix) || '') }}
|
||||
run: |
|
||||
echo "RERUN_ARTIFACT=$RERUN_ARTIFACT_BASE-attempt-$GITHUB_RUN_ATTEMPT" >> "$GITHUB_ENV"
|
||||
echo "RERUN_PREV_ARTIFACT=$RERUN_ARTIFACT_BASE-attempt-$((GITHUB_RUN_ATTEMPT - 1))" >> "$GITHUB_ENV"
|
||||
|
||||
# `pattern` (not `name`) because the artifact is legitimately absent whenever the
|
||||
# previous attempt recorded nothing: `name` treats that as an error, `pattern`
|
||||
# downloads nothing and moves on. The value is a literal — no glob metacharacters —
|
||||
# so it matches exactly one artifact, which lands at `target/failed-tests`. A pattern
|
||||
# that ever matched two would nest them under their artifact names instead, leaving
|
||||
# the file absent and falling back to the full suite.
|
||||
- name: Restore previously-failed tests
|
||||
if: ${{ inputs.granular-rerun == 'true' && github.run_attempt != '1' }}
|
||||
continue-on-error: true
|
||||
uses: actions/download-artifact@v8
|
||||
with:
|
||||
pattern: ${{ env.RERUN_PREV_ARTIFACT }}
|
||||
name: ${{ env.RERUN_PREV_ARTIFACT }}
|
||||
path: target
|
||||
|
||||
- name: Test database driver
|
||||
@@ -109,8 +109,8 @@ runs:
|
||||
fi
|
||||
shell: bash
|
||||
id: run-test
|
||||
# Keep this step continue-on-error because the quarantine gate below controls whether it
|
||||
# ultimately fails.
|
||||
# Keep this step continue-on-error because the quarantine logic controls whether it ultimately fails.
|
||||
# Driver "info" status can affect this logic. See below.
|
||||
continue-on-error: true
|
||||
|
||||
- name: Report failed tests to ci-conductor
|
||||
@@ -173,7 +173,13 @@ runs:
|
||||
# Last, because this is the only step allowed to fail the job: everything above it reports the
|
||||
# failure (JUnit report, logs, ci-conductor) and must run regardless of the verdict below.
|
||||
- name: Quarantine gate
|
||||
if: ${{ !cancelled() && steps.run-test.outcome == 'failure' }}
|
||||
# Off master and release branches an "info" driver is data-collection only - the test always passes!
|
||||
# On master and release branches "info" carries no weight - the quarantine logic applies.
|
||||
if: |
|
||||
!cancelled()
|
||||
&& steps.run-test.outcome == 'failure'
|
||||
&& !(inputs.driver-status == 'info'
|
||||
&& !(github.ref_name == 'master' || startsWith(github.ref_name, 'release-x')))
|
||||
uses: ./.github/actions/quarantine-gate
|
||||
with:
|
||||
adapter: backend
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
name: Create Backend Dep Bump PRs
|
||||
description: Opens a pull request for each backend dependency that has a newer version available.
|
||||
|
||||
inputs:
|
||||
github-token:
|
||||
|
||||
161
.github/actions/upload-test-results/action.yml
vendored
Normal file
161
.github/actions/upload-test-results/action.yml
vendored
Normal file
@@ -0,0 +1,161 @@
|
||||
name: Upload Test Results to s3
|
||||
description: Uploads test results files to s3
|
||||
|
||||
inputs:
|
||||
input-path:
|
||||
required: true
|
||||
output-name:
|
||||
required: true
|
||||
default: test
|
||||
role-to-assume:
|
||||
required: true
|
||||
aws-region:
|
||||
required: true
|
||||
default: us-east-1
|
||||
bucket:
|
||||
required: true
|
||||
previous-step-outcome:
|
||||
required: false
|
||||
default: ""
|
||||
description: >-
|
||||
Outcome of the test step. Only read when `driver-status` is "info", to report the
|
||||
real result of a job that was forced green.
|
||||
driver-status:
|
||||
required: false
|
||||
default: ""
|
||||
description: >-
|
||||
Configured status for this driver ("info", "skip", "required", or empty). When
|
||||
"info", the job is reported as passing regardless of the real test outcome, and
|
||||
that outcome is surfaced via a job summary and (on PRs) a comment instead.
|
||||
report-variant:
|
||||
required: false
|
||||
default: ""
|
||||
description: >-
|
||||
Optional label disambiguating matrix legs of the same job, so the info-driver
|
||||
job summary and PR comment stay distinct per leg.
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Configure AWS credentials
|
||||
uses: aws-actions/configure-aws-credentials@d979d5b3a71173a29b74b5b88418bfda9437d885
|
||||
with:
|
||||
role-to-assume: ${{ inputs.role-to-assume }}
|
||||
aws-region: ${{ inputs.aws-region }}
|
||||
- name: zip test results
|
||||
env:
|
||||
INPUT_DIR: ${{ inputs.input-path }}
|
||||
OUTPUT_FILE: ${{ inputs.output-name }}
|
||||
shell: bash
|
||||
run: | # sh
|
||||
zip ${OUTPUT_FILE}.zip $INPUT_DIR/*.xml
|
||||
- name: Upload test results to S3
|
||||
env:
|
||||
BUCKET: ${{ inputs.bucket }}
|
||||
OUTPUT_FILE: ${{ inputs.output-name }}
|
||||
shell: bash
|
||||
run: | # sh
|
||||
DATE=$(date '+%Y-%m-%d')
|
||||
aws s3 cp ${OUTPUT_FILE}.zip s3://$BUCKET/$DATE/$GITHUB_RUN_ID/$GITHUB_RUN_ATTEMPT/
|
||||
|
||||
# For "info" drivers the job is forced green above, so surface the real outcome here —
|
||||
# but only when it did NOT pass, to avoid spamming summaries/comments for healthy runs.
|
||||
# Never include logs or stack traces — only the real status and a link to the run.
|
||||
- name: Report info-level driver result (job summary)
|
||||
if: ${{ always() && inputs.driver-status == 'info' && inputs.previous-step-outcome == 'failure' }}
|
||||
shell: bash
|
||||
env:
|
||||
OUTCOME: ${{ inputs.previous-step-outcome }}
|
||||
LABEL: ${{ inputs.output-name }}
|
||||
VARIANT: ${{ inputs.report-variant }}
|
||||
run: | # sh
|
||||
if [ "$OUTCOME" = "success" ]; then EMOJI="✅"; else EMOJI="❌"; fi
|
||||
TITLE="$LABEL"
|
||||
[ -n "$VARIANT" ] && TITLE="$LABEL ($VARIANT)"
|
||||
RUN_URL="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"
|
||||
{
|
||||
echo "### ℹ️ Info-level driver: $TITLE"
|
||||
echo ""
|
||||
echo "Reported as passing (non-blocking). Real result: $EMOJI \`$OUTCOME\`"
|
||||
echo ""
|
||||
echo "[View run]($RUN_URL)"
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
# Maintain ONE PR comment per commit for all info drivers. Each leg upserts its own line
|
||||
# (keyed by a hidden tag); a passing leg removes its line, and the comment is deleted once
|
||||
# no failing legs remain. Comments from older commits are cleaned up. Runs on pass and fail
|
||||
# so recoveries self-heal. Never includes logs or stack traces — only status and a run link.
|
||||
- name: Sync info-level driver result into the PR comment
|
||||
if: ${{ always() && inputs.driver-status == 'info' && github.event_name == 'pull_request' }}
|
||||
uses: actions/github-script@v9
|
||||
continue-on-error: true
|
||||
env:
|
||||
OUTCOME: ${{ inputs.previous-step-outcome }}
|
||||
LABEL: ${{ inputs.output-name }}
|
||||
VARIANT: ${{ inputs.report-variant }}
|
||||
with:
|
||||
script: | # js
|
||||
const { OUTCOME, LABEL, VARIANT, GITHUB_SERVER_URL, GITHUB_REPOSITORY, GITHUB_RUN_ID } = process.env;
|
||||
const pr = context.payload.pull_request;
|
||||
if (!pr) return;
|
||||
const { owner, repo } = context.repo;
|
||||
const sha = pr.head.sha;
|
||||
// PREFIX matches the current per-commit marker AND the older per-leg one, so legacy
|
||||
// comments from a previous version get cleaned up on the next sync.
|
||||
const PREFIX = "<!-- info-driver-result";
|
||||
const marker = `<!-- info-driver-results:${sha} -->`;
|
||||
const header = `${marker}\nℹ️ **Info-level drivers** failed (non-blocking) on \`${sha.slice(0, 7)}\` — these do not gate:`;
|
||||
const lineKey = `${LABEL}:${VARIANT}`;
|
||||
const title = VARIANT ? `${LABEL} (${VARIANT})` : LABEL;
|
||||
const runUrl = `${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}`;
|
||||
// Only real failures are reported; cancelled/skipped/success remove this leg's line.
|
||||
// List-item line with the hidden key tag at the END so markdown (links) renders.
|
||||
const ourLine = OUTCOME === "failure"
|
||||
? `- **${title}** — ❌ \`${OUTCOME}\` ([Logs](${runUrl})) <!--l:${lineKey}-->`
|
||||
: null;
|
||||
const linesOf = (body) => body.split("\n").filter((l) => l.includes("<!--l:"));
|
||||
|
||||
const sync = async () => {
|
||||
const { data: comments } = await github.rest.issues.listComments({
|
||||
owner, repo, issue_number: pr.number, per_page: 100,
|
||||
});
|
||||
const tagged = comments.filter((c) => c.body.includes(PREFIX));
|
||||
// Drop comments left over from older commits.
|
||||
for (const c of tagged.filter((c) => !c.body.includes(marker))) {
|
||||
await github.rest.issues.deleteComment({ owner, repo, comment_id: c.id }).catch(() => {});
|
||||
}
|
||||
const mine = tagged
|
||||
.filter((c) => c.body.includes(marker))
|
||||
.sort((a, b) => new Date(a.created_at) - new Date(b.created_at));
|
||||
// Merge any duplicates raced into existence; latest write of each line wins.
|
||||
const byKey = new Map();
|
||||
for (const c of mine) {
|
||||
for (const l of linesOf(c.body)) {
|
||||
const m = l.match(/<!--l:(.*?)-->/);
|
||||
if (m) byKey.set(m[1], l);
|
||||
}
|
||||
}
|
||||
if (ourLine) byKey.set(lineKey, ourLine);
|
||||
else byKey.delete(lineKey);
|
||||
const lines = [...byKey.values()].sort();
|
||||
const target = mine[0];
|
||||
for (const c of mine.slice(1)) {
|
||||
await github.rest.issues.deleteComment({ owner, repo, comment_id: c.id }).catch(() => {});
|
||||
}
|
||||
if (lines.length === 0) {
|
||||
if (target) await github.rest.issues.deleteComment({ owner, repo, comment_id: target.id }).catch(() => {});
|
||||
return;
|
||||
}
|
||||
const body = [header, "", ...lines].join("\n");
|
||||
if (target) {
|
||||
await github.rest.issues.updateComment({ owner, repo, comment_id: target.id, body });
|
||||
} else {
|
||||
await github.rest.issues.createComment({ owner, repo, issue_number: pr.number, body });
|
||||
}
|
||||
};
|
||||
|
||||
// Retry to shrink the window where parallel legs clobber each other's update.
|
||||
for (let i = 0; i < 4; i++) {
|
||||
try { await sync(); break; }
|
||||
catch (e) { if (i === 3) throw e; await new Promise((r) => setTimeout(r, 400 * (i + 1))); }
|
||||
}
|
||||
22
.github/actions/verify-membership/action.yml
vendored
Normal file
22
.github/actions/verify-membership/action.yml
vendored
Normal file
@@ -0,0 +1,22 @@
|
||||
name: 'Verify Organization Membership'
|
||||
description: 'Check if a user is a member of a GitHub organization'
|
||||
inputs:
|
||||
github_token:
|
||||
description: 'Token with access to the organization'
|
||||
required: true
|
||||
organization_name:
|
||||
description: 'Organization name'
|
||||
required: true
|
||||
username:
|
||||
description: 'Username'
|
||||
required: true
|
||||
outputs:
|
||||
is_member:
|
||||
value: ${{ steps.verify-membership.outputs.is_member }}
|
||||
description: "Is the user a member of the organization"
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- run: bash ${{ github.action_path }}/verify_membership.sh ${{ inputs.github_token }} ${{ inputs.organization_name }} ${{ inputs.username }}
|
||||
id: verify-membership
|
||||
shell: bash
|
||||
25
.github/actions/verify-membership/verify_membership.sh
vendored
Executable file
25
.github/actions/verify-membership/verify_membership.sh
vendored
Executable file
@@ -0,0 +1,25 @@
|
||||
#!/bin/bash
|
||||
|
||||
TOKEN=$1
|
||||
ORG_NAME=$2
|
||||
USERNAME=$3
|
||||
|
||||
response=$(curl -L -o /dev/null -w "%{http_code}" \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "X-GitHub-Api-Version: 2022-11-28" \
|
||||
"https://api.github.com/orgs/$ORG_NAME/members/$USERNAME")
|
||||
|
||||
# Github returns an empty response with 204 status if user is a member and 404 status otherwise
|
||||
# See: https://docs.github.com/en/rest/orgs/members?apiVersion=2022-11-28#check-organization-membership-for-a-user
|
||||
if [ "$response" == "204" ]; then
|
||||
echo "The user $USERNAME is a member of $ORG_NAME"
|
||||
echo "is_member=true" >> $GITHUB_OUTPUT
|
||||
elif [ "$response" == "404" ]; then
|
||||
echo "The user $USERNAME is not a member of $ORG_NAME"
|
||||
echo "is_member=false" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "Failed to determine the membership status for $USERNAME in $ORG_NAME"
|
||||
echo "is_member=false" >> $GITHUB_OUTPUT
|
||||
exit 1
|
||||
fi
|
||||
24
.github/file-paths.yaml
vendored
24
.github/file-paths.yaml
vendored
@@ -106,6 +106,13 @@ backend_ci: &backend_ci
|
||||
- ".github/workflows/backend.yml"
|
||||
- ".github/workflows/drivers.yml"
|
||||
|
||||
backend_presto_kerberos:
|
||||
- "**/presto_jdbc/**"
|
||||
- "**/presto-jdbc/**" # the driver module has a hyphen. underscores are classpaths
|
||||
- "**/presto_jdbc.clj"
|
||||
- "**/presto_jdbc_test.clj"
|
||||
- ".github/workflows/presto-kerberos-integration-test.yml"
|
||||
|
||||
backend_sources: &backend_sources
|
||||
- *shared_sources
|
||||
- "**deps.edn"
|
||||
@@ -143,7 +150,7 @@ custom_viz_all:
|
||||
|
||||
e2e_ci: &e2e_ci
|
||||
- *shared_ci
|
||||
- ".github/workflows/uberjar.yml"
|
||||
- ".github/actions/prepare-uberjar-artifact/**"
|
||||
- ".github/actions/fetch-artifact/action.yml"
|
||||
- ".github/actions/e2e-prepare-containers/**"
|
||||
- ".github/actions/prepare-cypress/**"
|
||||
@@ -154,7 +161,7 @@ e2e_specs: &e2e_specs
|
||||
|
||||
embedding_sdk_ci: &embedding_sdk_ci
|
||||
- *shared_ci
|
||||
- ".github/workflows/uberjar.yml"
|
||||
- ".github/actions/prepare-uberjar-artifact/**"
|
||||
- ".github/actions/fetch-artifact/action.yml"
|
||||
- ".github/actions/prepare-cypress/**"
|
||||
- ".github/workflows/embedding-sdk*.yml"
|
||||
@@ -176,7 +183,7 @@ embedding_sdk_host_sample_apps:
|
||||
- "e2e/test-host-app/**/*.cy.*.{js,ts}"
|
||||
|
||||
snowplow: &snowplow
|
||||
- ".github/workflows/lint.yml"
|
||||
- ".github/workflows/snowplow.yml"
|
||||
- "snowplow/**"
|
||||
- "bin/lint-snowplow-schemas.bb"
|
||||
- "bin/lint-snowplow-schemas-test.bb"
|
||||
@@ -205,7 +212,7 @@ yaml:
|
||||
- "**/*.yaml"
|
||||
|
||||
codeql:
|
||||
- ".github/workflows/lint.yml"
|
||||
- ".github/workflows/codeql.yml"
|
||||
- "frontend/src/**"
|
||||
- "enterprise/frontend/src/**"
|
||||
|
||||
@@ -213,9 +220,14 @@ i18n:
|
||||
- *default
|
||||
- ".github/actions/prepare-frontend/**"
|
||||
- ".github/actions/prepare-backend/**"
|
||||
- ".github/workflows/lint.yml"
|
||||
- ".github/workflows/i18n.yml"
|
||||
- *sources
|
||||
|
||||
visualizations:
|
||||
- "frontend/src/metabase/visualizations/**"
|
||||
- "frontend/src/metabase/static-viz/**"
|
||||
- "src/metabase/pulse/**"
|
||||
|
||||
cljs:
|
||||
- *shared_sources
|
||||
- *shared_specs
|
||||
@@ -227,7 +239,7 @@ serialization_baseline:
|
||||
- "test_resources/serialization_baseline/**"
|
||||
|
||||
openapi:
|
||||
- ".github/workflows/lint.yml"
|
||||
- ".github/workflows/openapi.yml"
|
||||
- "src/metabase/api/**"
|
||||
- "src/metabase/api_routes/**"
|
||||
- "src/metabase/*_rest/api.clj"
|
||||
|
||||
66
.github/scripts/fix-yarn-lock-aliases.ts
vendored
66
.github/scripts/fix-yarn-lock-aliases.ts
vendored
@@ -1,66 +0,0 @@
|
||||
/**
|
||||
* Renames aliased dependency keys in a bun-generated yarn.lock.
|
||||
*
|
||||
* Yarn names an alias entry after the alias, for example
|
||||
* `"typescript7@npm:typescript@7.0.2"`. Bun names it after the aliased package
|
||||
* instead, for example `"typescript@npm:typescript@7.0.2"`. Snyk follows the
|
||||
* yarn convention and fails to resolve the dependency.
|
||||
*/
|
||||
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
|
||||
async function main() {
|
||||
const packageJsonPath = `${process.cwd()}/package.json`;
|
||||
const lockPath = `${process.cwd()}/yarn.lock`;
|
||||
|
||||
const packageJson = JSON.parse(await readFile(packageJsonPath, "utf8"));
|
||||
|
||||
const dependencies = Object.entries<string>(
|
||||
Object.assign(
|
||||
{},
|
||||
packageJson.dependencies,
|
||||
packageJson.devDependencies,
|
||||
packageJson.optionalDependencies,
|
||||
),
|
||||
);
|
||||
|
||||
let lock = await readFile(lockPath, "utf8");
|
||||
const renamedKeys: string[] = [];
|
||||
|
||||
for (const [alias, specifier] of dependencies) {
|
||||
if (!specifier.startsWith("npm:")) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const aliasedPackage = specifier
|
||||
.slice("npm:".length)
|
||||
.replace(/@[^@]*$/, "");
|
||||
|
||||
if (aliasedPackage === alias) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const bunKey = `"${aliasedPackage}@${specifier}"`;
|
||||
const yarnKey = `"${alias}@${specifier}"`;
|
||||
|
||||
if (lock.includes(bunKey)) {
|
||||
lock = lock.replaceAll(bunKey, yarnKey);
|
||||
renamedKeys.push(`${bunKey} -> ${yarnKey}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (renamedKeys.length > 0) {
|
||||
await writeFile(lockPath, lock);
|
||||
}
|
||||
|
||||
process.stdout.write(
|
||||
renamedKeys.length > 0
|
||||
? `Renamed alias keys in yarn.lock:\n${renamedKeys.join("\n")}\n`
|
||||
: "No alias keys to rename in yarn.lock\n",
|
||||
);
|
||||
}
|
||||
|
||||
main().catch((error) => {
|
||||
process.stderr.write(`Failed to rename alias keys in yarn.lock: ${error}\n`);
|
||||
process.exit(1);
|
||||
});
|
||||
84
.github/scripts/get-issue-data.js
vendored
Normal file
84
.github/scripts/get-issue-data.js
vendored
Normal file
@@ -0,0 +1,84 @@
|
||||
const githubSlackMap = require("../../release/github-slack-map.json");
|
||||
|
||||
async function getIssueData({ context, github, manualIssueNumber }) {
|
||||
let issueNumber;
|
||||
let issueData;
|
||||
let prReviews = [];
|
||||
const triggerType = context.eventName;
|
||||
|
||||
if (context.eventName === "issues") {
|
||||
issueNumber = context.payload.issue.number;
|
||||
issueData = context.payload.issue;
|
||||
} else if (context.eventName === "workflow_dispatch") {
|
||||
issueNumber = parseInt(manualIssueNumber);
|
||||
const { data } = await github.rest.issues.get({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: issueNumber,
|
||||
});
|
||||
issueData = data;
|
||||
} else if (context.eventName === "pull_request_review") {
|
||||
const prNumber = context.payload.pull_request.number;
|
||||
const { data: prData } = await github.rest.pulls.get({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
pull_number: prNumber,
|
||||
});
|
||||
|
||||
// Extract issue number from branch name (e.g., "claude-fix/issue-12345-some-title")
|
||||
const branchMatch = prData.head.ref.match(/claude-fix\/issue-(\d+)/);
|
||||
if (!branchMatch) {
|
||||
throw new Error(
|
||||
`Could not extract issue number from branch name: ${prData.head.ref}`,
|
||||
);
|
||||
}
|
||||
|
||||
issueNumber = parseInt(branchMatch[1]);
|
||||
|
||||
const { data } = await github.rest.issues.get({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: issueNumber,
|
||||
});
|
||||
issueData = data;
|
||||
|
||||
const { data: reviews } = await github.rest.pulls.listReviews({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
pull_number: prNumber,
|
||||
});
|
||||
|
||||
prReviews = reviews.filter(
|
||||
(review) =>
|
||||
review.state === "CHANGES_REQUESTED" &&
|
||||
isMetabaseMember(review.user.login),
|
||||
);
|
||||
}
|
||||
|
||||
const { data: comments } = await github.rest.issues.listComments({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: issueNumber,
|
||||
});
|
||||
|
||||
const employeeComments = comments.filter((comment) =>
|
||||
isMetabaseMember(comment.user.login),
|
||||
);
|
||||
|
||||
return {
|
||||
issueNumber,
|
||||
issueData,
|
||||
employeeComments,
|
||||
prReviews,
|
||||
triggerType,
|
||||
isIssueAuthorEmployee: isMetabaseMember(issueData.user.login),
|
||||
};
|
||||
}
|
||||
|
||||
function isMetabaseMember(username) {
|
||||
return username in githubSlackMap;
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
getIssueData,
|
||||
};
|
||||
65
.github/scripts/run-presto-kerberos-integration-test.sh
vendored
Executable file
65
.github/scripts/run-presto-kerberos-integration-test.sh
vendored
Executable file
@@ -0,0 +1,65 @@
|
||||
#! /usr/bin/env bash
|
||||
# runs one or more Metabase test(s) against a Kerberized Presto instance
|
||||
set -eo pipefail
|
||||
|
||||
# Need Java commands on $PATH, which apparently is not yet the case
|
||||
export JAVA_HOME="/usr/lib/jvm/msopenjdk-current"
|
||||
export PATH="$PATH:$JAVA_HOME/bin"
|
||||
|
||||
# ensure java commmand is available
|
||||
which java
|
||||
|
||||
# install clojure version needed for Metabase
|
||||
curl -O https://download.clojure.org/install/linux-install-1.12.0.1488.sh
|
||||
chmod +x linux-install-1.12.0.1488.sh
|
||||
./linux-install-1.12.0.1488.sh
|
||||
|
||||
RESOURCES_DIR=/app/source/resources
|
||||
|
||||
# ensure the expected files are in place, in the resources dir
|
||||
if [ ! -f "$RESOURCES_DIR/ssl_keystore.jks" ]; then
|
||||
echo "$RESOURCES_DIR/ssl_keystore.jks does not exist; cannot run test" >&2
|
||||
exit 11
|
||||
fi
|
||||
|
||||
if [ ! -f "$RESOURCES_DIR/krb5.conf" ]; then
|
||||
echo "$RESOURCES_DIR/krb5.conf does not exist; cannot run test" >&2
|
||||
exit 12
|
||||
fi
|
||||
|
||||
if [ ! -f "$RESOURCES_DIR/client.keytab" ]; then
|
||||
echo "$RESOURCES_DIR/client.keytab does not exist; cannot run test" >&2
|
||||
exit 13
|
||||
fi
|
||||
|
||||
# Copy the JDK cacerts file to our resources
|
||||
cp $JAVA_HOME/lib/security/cacerts $RESOURCES_DIR/cacerts-with-presto-ca.jks
|
||||
|
||||
# Capture the Presto server self signed CA in PEM format
|
||||
openssl s_client -showcerts -connect presto-kerberos:7778 </dev/null \
|
||||
| openssl x509 -outform PEM >$RESOURCES_DIR/presto-ssl-root-ca.pem
|
||||
|
||||
# Convert the Presto server self signed CA to DER format
|
||||
openssl x509 -outform der -in $RESOURCES_DIR/presto-ssl-root-ca.pem -out $RESOURCES_DIR/presto-ssl-root-ca.der
|
||||
|
||||
# Add Presto's self signed CA to the truststore
|
||||
keytool -noprompt -import -alias presto-kerberos -keystore $RESOURCES_DIR/cacerts-with-presto-ca.jks \
|
||||
-storepass changeit -file $RESOURCES_DIR/presto-ssl-root-ca.der -trustcacerts
|
||||
|
||||
# Set up the environment variables pointing to all of this, and run some tests
|
||||
DRIVERS=presto-jdbc \
|
||||
MB_ENABLE_PRESTO_JDBC_DRIVER=true \
|
||||
MB_PRESTO_JDBC_TEST_HOST=presto-kerberos \
|
||||
MB_PRESTO_JDBC_TEST_PORT=7778 \
|
||||
MB_PRESTO_JDBC_TEST_SSL=true \
|
||||
MB_PRESTO_JDBC_TEST_SSL_KEYSTORE_PATH=$RESOURCES_DIR/ssl_keystore.jks \
|
||||
MB_PRESTO_JDBC_TEST_SSL_KEYSTORE_PASSWORD=presto \
|
||||
MB_PRESTO_JDBC_TEST_SSL_TRUSTSTORE_PATH=$RESOURCES_DIR/cacerts-with-presto-ca.jks \
|
||||
MB_PRESTO_JDBC_TEST_SSL_TRUSTSTORE_PASSWORD=changeit \
|
||||
MB_PRESTO_JDBC_TEST_KERBEROS=true \
|
||||
MB_PRESTO_JDBC_TEST_USER=bob@EXAMPLE.COM \
|
||||
MB_PRESTO_JDBC_TEST_KERBEROS_PRINCIPAL=bob@EXAMPLE.COM \
|
||||
MB_PRESTO_JDBC_TEST_KERBEROS_REMOTE_SERVICE_NAME=HTTP \
|
||||
MB_PRESTO_JDBC_TEST_KERBEROS_KEYTAB_PATH=$RESOURCES_DIR/client.keytab \
|
||||
MB_PRESTO_JDBC_TEST_KERBEROS_CONFIG_PATH=$RESOURCES_DIR/krb5.conf \
|
||||
clojure -X:dev:test:drivers:drivers-dev :only metabase.driver.presto-jdbc-test
|
||||
3
.github/team.json
vendored
3
.github/team.json
vendored
@@ -65,8 +65,7 @@
|
||||
"piranha",
|
||||
"crisptrutski",
|
||||
"undrfined",
|
||||
"andreis",
|
||||
"prascuna"
|
||||
"andreis"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
44
.github/workflows/actionlint.yml
vendored
Normal file
44
.github/workflows/actionlint.yml
vendored
Normal file
@@ -0,0 +1,44 @@
|
||||
name: Actionlint
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'release-**'
|
||||
paths:
|
||||
- '.github/workflows/**'
|
||||
pull_request:
|
||||
paths:
|
||||
- '.github/workflows/**'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
actionlint:
|
||||
name: Lint Github Actions Workflows
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Get changed Github actions
|
||||
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
|
||||
id: changes
|
||||
with:
|
||||
token: ${{ github.token }}
|
||||
filters: .github/file-paths.yaml
|
||||
list-files: shell
|
||||
- name: Setup actionlint
|
||||
if: steps.changes.outputs.actionlint == 'true'
|
||||
id: download
|
||||
shell: bash
|
||||
run: |
|
||||
if command -v actionlint; then
|
||||
echo "executable=actionlint" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
# script sets executable output, for details see: https://github.com/rhysd/actionlint/blob/main/docs/usage.md#use-actionlint-on-github-actions
|
||||
bash <(curl -fsSL https://raw.githubusercontent.com/rhysd/actionlint/v1.7.12/scripts/download-actionlint.bash)
|
||||
fi
|
||||
- name: Run actionlint
|
||||
if: steps.changes.outputs.actionlint == 'true'
|
||||
run: ${{ steps.download.outputs.executable }} --color --verbose ${{ steps.changes.outputs.actionlint_files }}
|
||||
30
.github/workflows/app-db.yml
vendored
30
.github/workflows/app-db.yml
vendored
@@ -125,6 +125,16 @@ jobs:
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
|
||||
be-tests-mysql:
|
||||
if: ${{ !inputs.skip }}
|
||||
@@ -251,6 +261,16 @@ jobs:
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
|
||||
be-tests-postgres:
|
||||
if: ${{ !inputs.skip }}
|
||||
@@ -352,6 +372,16 @@ jobs:
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
|
||||
app-db-tests-result:
|
||||
needs:
|
||||
|
||||
71
.github/workflows/auto-backport.yml
vendored
71
.github/workflows/auto-backport.yml
vendored
@@ -99,36 +99,6 @@ jobs:
|
||||
)
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
steps:
|
||||
- name: "!! double-backport and triple-backport label deprecation warning !!"
|
||||
if:
|
||||
|
|
||||
github.event.pull_request.merged == true && (
|
||||
github.event.action == 'closed' && (
|
||||
contains(github.event.pull_request.labels.*.name, 'double-backport') ||
|
||||
contains(github.event.pull_request.labels.*.name, 'triple-backport')
|
||||
) ||
|
||||
github.event.action == 'labeled' && (
|
||||
github.event.label.name == 'double-backport' ||
|
||||
github.event.label.name == 'triple-backport'
|
||||
)
|
||||
)
|
||||
env:
|
||||
REPO: ${{ github.repository }}
|
||||
PR: ${{ github.event.pull_request.number }}
|
||||
GH_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
|
||||
run: |
|
||||
msg="🚨🚨 Backport failed because \`double-backport\` and \`triple-backport\` labels are **deprecated**! Use the \`backport\` label to port changes to all supported releases instead. 🚨🚨"
|
||||
|
||||
gh api \
|
||||
--silent \
|
||||
--method POST \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "X-GitHub-Api-Version: 2026-03-10" \
|
||||
/repos/"$REPO"/issues/"$PR"/comments \
|
||||
-f "body=$msg"
|
||||
|
||||
echo "::error::$msg"
|
||||
exit 1
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
fetch-depth: 0
|
||||
@@ -140,6 +110,29 @@ jobs:
|
||||
pull-request-number: ${{ github.event.pull_request.number }}
|
||||
base-ref: ${{ github.event.pull_request.base.ref }}
|
||||
|
||||
- name: how many backports?
|
||||
id: how-many-backports
|
||||
uses: actions/github-script@v9
|
||||
with:
|
||||
script: | # js
|
||||
const labels = [
|
||||
...${{ toJson(github.event.pull_request.labels) }},
|
||||
{ name:"${{ github.event.label.name }}" }
|
||||
];
|
||||
|
||||
const backportLabelNames = labels.filter(label => label.name.includes('backport')).map(label => label.name);
|
||||
|
||||
let result = 1;
|
||||
|
||||
if (backportLabelNames.includes("triple-backport")) {
|
||||
result = 3;
|
||||
} else if (backportLabelNames.includes("double-backport")) {
|
||||
result = 2;
|
||||
}
|
||||
|
||||
console.log("backports", result);
|
||||
return result;
|
||||
|
||||
- name: get release versions
|
||||
id: get-latest-release-version
|
||||
uses: actions/github-script@v9
|
||||
@@ -164,3 +157,21 @@ jobs:
|
||||
backport-commit: ${{ steps.find_commit.outputs.commit }}
|
||||
github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
|
||||
github-token-2: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Double Backport
|
||||
if: ${{ always() && fromJson(steps.how-many-backports.outputs.result) >= 2 }}
|
||||
uses: ./.github/actions/create-backport
|
||||
with:
|
||||
target-version: ${{ fromJson(steps.get-latest-release-version.outputs.result).secondLatest }}
|
||||
backport-commit: ${{ steps.find_commit.outputs.commit }}
|
||||
github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
|
||||
github-token-2: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Triple Backport
|
||||
if: ${{ always() && fromJson(steps.how-many-backports.outputs.result) >= 3 }}
|
||||
uses: ./.github/actions/create-backport
|
||||
with:
|
||||
target-version: ${{ fromJson(steps.get-latest-release-version.outputs.result).thirdLatest }}
|
||||
backport-commit: ${{ steps.find_commit.outputs.commit }}
|
||||
github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
|
||||
github-token-2: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
8
.github/workflows/backend-cloverage.yml
vendored
8
.github/workflows/backend-cloverage.yml
vendored
@@ -24,8 +24,8 @@ jobs:
|
||||
- uses: actions/checkout@v6
|
||||
- id: get-modules
|
||||
run: |
|
||||
modules=$(for d in src/metabase/*/; do basename "$d"; done | tr '_' '-' | jq -R -s -c 'split("\n") | map(select(length > 0))')
|
||||
echo "modules=$modules" >> "$GITHUB_OUTPUT"
|
||||
modules=$(ls -1d src/metabase/*/ | xargs -n1 basename | tr '_' '-' | jq -R -s -c 'split("\n") | map(select(length > 0))')
|
||||
echo "modules=$modules" >> $GITHUB_OUTPUT
|
||||
|
||||
be-linter-cloverage:
|
||||
needs: setup-oss
|
||||
@@ -71,8 +71,8 @@ jobs:
|
||||
- uses: actions/checkout@v6
|
||||
- id: get-modules
|
||||
run: |
|
||||
modules=$(for d in enterprise/backend/src/metabase_enterprise/*/; do basename "$d"; done | tr '_' '-' | jq -R -s -c 'split("\n") | map(select(length > 0))')
|
||||
echo "modules=$modules" >> "$GITHUB_OUTPUT"
|
||||
modules=$(ls -1d enterprise/backend/src/metabase_enterprise/*/ | xargs -n1 basename | tr '_' '-' | jq -R -s -c 'split("\n") | map(select(length > 0))')
|
||||
echo "modules=$modules" >> $GITHUB_OUTPUT
|
||||
|
||||
be-linter-cloverage-enterprise:
|
||||
needs: setup-ee
|
||||
|
||||
19
.github/workflows/backend.yml
vendored
19
.github/workflows/backend.yml
vendored
@@ -207,18 +207,12 @@ jobs:
|
||||
if: ${{ github.run_attempt != '1' }}
|
||||
run: echo "RERUN_PREV_ATTEMPT=$((GITHUB_RUN_ATTEMPT - 1))" >> "$GITHUB_ENV"
|
||||
|
||||
# `pattern` (not `name`) because the artifact is legitimately absent whenever the
|
||||
# previous attempt recorded nothing: `name` treats that as an error, `pattern`
|
||||
# downloads nothing and moves on. The value is a literal — no glob metacharacters —
|
||||
# so it matches exactly one artifact, which lands at `target/failed-tests`. A pattern
|
||||
# that ever matched two would nest them under their artifact names instead, leaving
|
||||
# the file absent and falling back to the full partition.
|
||||
- name: Restore previously-failed tests
|
||||
if: ${{ github.run_attempt != '1' }}
|
||||
continue-on-error: true
|
||||
uses: actions/download-artifact@v8
|
||||
with:
|
||||
pattern: failed-tests-be-tests-java-${{ matrix.java-version }}-${{ matrix.job.test-group-name }}-attempt-${{ env.RERUN_PREV_ATTEMPT }}
|
||||
name: failed-tests-be-tests-java-${{ matrix.java-version }}-${{ matrix.job.test-group-name }}-attempt-${{ env.RERUN_PREV_ATTEMPT }}
|
||||
path: target
|
||||
|
||||
- name: "Test Java ${{ matrix.java-version }} ${{ matrix.job.name }}"
|
||||
@@ -281,6 +275,17 @@ jobs:
|
||||
test-suite: be-tests-java-${{ matrix.java-version }}-${{ matrix.job.test-group-name }}
|
||||
enforce-quarantine: ${{ vars.QUARANTINE_ENGINE == 'ci-conductor' }}
|
||||
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: be-tests-java-${{ matrix.java-version }}-${{ matrix.job.edition }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.run-java-tests.outcome }}
|
||||
|
||||
- name: Publish Test Report (JUnit)
|
||||
uses: dorny/test-reporter@a43b3a5f7366b97d083190328d2c652e1a8b6aa2 # v3.0.0
|
||||
if: failure()
|
||||
|
||||
@@ -5,10 +5,10 @@ on:
|
||||
|
||||
jobs:
|
||||
check-source-branch:
|
||||
if: github.head_ref == 'master'
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
steps:
|
||||
- name: Block master branch PRs
|
||||
if: github.head_ref == 'master'
|
||||
run: |
|
||||
echo "PRs from \`master\` branches are not allowed, please use a new branch name for your PR."
|
||||
echo "PRs from `master` branches are not allowed, please use a new branch name for your PR."
|
||||
exit 1
|
||||
|
||||
270
.github/workflows/build-for-release.yml
vendored
Normal file
270
.github/workflows/build-for-release.yml
vendored
Normal file
@@ -0,0 +1,270 @@
|
||||
# Builds a fresh release JAR from scratch at a given commit.
|
||||
#
|
||||
# You should almost never have to use this. The normal release flow relies on a
|
||||
# commit that has already been built and tested on its branch — see
|
||||
# `tag-for-release.yml`, which tags that existing JAR. Building from scratch
|
||||
# here is only for exceptional cases.
|
||||
name: Release 0 - Build Release Artifact
|
||||
run-name: Build ${{ inputs.version }}
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
version:
|
||||
description: 'Metabase version (e.g. v0.46.3)'
|
||||
type: string
|
||||
required: true
|
||||
commit:
|
||||
description: 'A full-length commit SHA-1 hash'
|
||||
required: true
|
||||
|
||||
jobs:
|
||||
start-message:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
sparse-checkout: |
|
||||
release
|
||||
.github
|
||||
- name: Prepare release scripts
|
||||
uses: ./.github/actions/prepare-release-scripts
|
||||
- name: Send build start message
|
||||
uses: actions/github-script@v9
|
||||
env:
|
||||
SLACK_RELEASE_CHANNEL: ${{ vars.SLACK_RELEASE_CHANNEL }}
|
||||
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
with:
|
||||
script: | # js
|
||||
const { sendPreReleaseMessage } = require('${{ github.workspace }}/release/dist/index.cjs');
|
||||
|
||||
await sendPreReleaseMessage({
|
||||
github,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
version: '${{ inputs.version }}',
|
||||
runId: '${{ github.run_id }}',
|
||||
releaseSha: '${{ inputs.commit }}',
|
||||
userName: '${{ github.actor }}',
|
||||
channelName: '${{ vars.SLACK_RELEASE_CHANNEL }}',
|
||||
}).catch(console.error);
|
||||
|
||||
trigger-milestone-check:
|
||||
# Posts a Slack FYI about issues that still need review for this release.
|
||||
# Informational only — nothing depends on it, so it never blocks the release.
|
||||
needs: [check-version, check-commit]
|
||||
permissions:
|
||||
contents: read
|
||||
issues: read
|
||||
uses: ./.github/workflows/release-milestone-check.yml
|
||||
secrets: inherit
|
||||
with:
|
||||
version: ${{ inputs.version }}
|
||||
commit: ${{ inputs.commit }}
|
||||
|
||||
check-version:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 10
|
||||
outputs:
|
||||
ee: ${{ fromJson(steps.canonical_version.outputs.result).ee }}
|
||||
oss: ${{ fromJson(steps.canonical_version.outputs.result).oss }}
|
||||
steps:
|
||||
- name: Fail early on the incorrect version format
|
||||
if: ${{ !(startsWith(inputs.version,'v0.') || startsWith(inputs.version,'v1.')) }}
|
||||
run: |
|
||||
echo "The version format is invalid!"
|
||||
echo "It must start with either 'v0.' or 'v1.'."
|
||||
echo "Please, try again."
|
||||
exit 1
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
sparse-checkout: |
|
||||
release
|
||||
.github
|
||||
- name: Prepare release scripts
|
||||
uses: ./.github/actions/prepare-release-scripts
|
||||
- name: Get Release Version
|
||||
uses: actions/github-script@v9
|
||||
id: canonical_version
|
||||
with:
|
||||
script: | # js
|
||||
const { isValidVersionString, getCanonicalVersion, hasBeenReleased } = require('${{ github.workspace }}/release/dist/index.cjs');
|
||||
|
||||
const version = '${{ inputs.version }}';
|
||||
|
||||
if (!isValidVersionString(version)) {
|
||||
throw new Error("The version format is invalid! It must start with either 'v0.' or 'v1.'.");
|
||||
}
|
||||
|
||||
const versions = {
|
||||
ee: getCanonicalVersion(version, 'ee'),
|
||||
oss: getCanonicalVersion(version, 'oss'),
|
||||
};
|
||||
|
||||
const released = await hasBeenReleased({
|
||||
github,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
version: versions.oss,
|
||||
});
|
||||
|
||||
if (released) {
|
||||
throw new Error("This version has already been released!", version);
|
||||
}
|
||||
|
||||
return versions;
|
||||
|
||||
check-commit:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- name: Check out the code to verify the release branch
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
fetch-depth: 0 # IMPORTANT! to get all the branches
|
||||
sparse-checkout: |
|
||||
release
|
||||
.github
|
||||
- name: Prepare release scripts
|
||||
uses: ./.github/actions/prepare-release-scripts
|
||||
- name: Get Release Branch
|
||||
uses: actions/github-script@v9
|
||||
id: release_branch
|
||||
with:
|
||||
result-encoding: string
|
||||
script: | # js
|
||||
const { getReleaseBranch } = require('${{ github.workspace }}/release/dist/index.cjs');
|
||||
const version = '${{ inputs.version }}';
|
||||
const releaseBranch = getReleaseBranch(version);
|
||||
return releaseBranch;
|
||||
- name: Ensure that the specified commit exists in the ${{ steps.release_branch.outputs.result }} release branch
|
||||
run: | # bash
|
||||
RELEASE_BRANCH=${{ steps.release_branch.outputs.result }}
|
||||
git checkout $RELEASE_BRANCH
|
||||
git branch --contains ${{ inputs.commit }} | grep -q $RELEASE_BRANCH \
|
||||
&& echo "Commit found in correct release branch" \
|
||||
|| (echo "Commit not found in correct release branch" && exit 1)
|
||||
git checkout -
|
||||
|
||||
get-build-requirements:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 10
|
||||
outputs:
|
||||
java_version: ${{ fromJson(steps.dependencies.outputs.result).java_version }}
|
||||
node_version: ${{ fromJson(steps.dependencies.outputs.result).node_version }}
|
||||
build_process: ${{ fromJson(steps.dependencies.outputs.result).build_process }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
sparse-checkout: |
|
||||
release
|
||||
.github
|
||||
- name: Prepare release scripts
|
||||
uses: ./.github/actions/prepare-release-scripts
|
||||
- name: Get build dependencies
|
||||
uses: actions/github-script@v9
|
||||
id: dependencies
|
||||
with:
|
||||
script: | # js
|
||||
const { getBuildRequirements, getMajorVersion } = require('${{ github.workspace }}/release/dist/index.cjs');
|
||||
|
||||
const version = '${{ inputs.version }}';
|
||||
|
||||
const requirements = getBuildRequirements(version);
|
||||
const build_process = getMajorVersion(version) < 46 ? 'legacy' : 'new';
|
||||
console.log('Build process:', build_process);
|
||||
|
||||
return {
|
||||
java_version: requirements.java,
|
||||
node_version: requirements.node,
|
||||
build_process,
|
||||
};
|
||||
|
||||
manage-milestones:
|
||||
# Close the milestone we're about to ship and open the next one BEFORE building,
|
||||
# so any PRs that merge while the build is in flight are tagged with the next
|
||||
# milestone (not the one about to be released).
|
||||
#
|
||||
# Sequenced after trigger-milestone-check so that the check sees the milestone
|
||||
# in its open state. milestone-check is informational and must never block, so
|
||||
# we run regardless of its outcome; only preflight failures suppress this.
|
||||
needs: [check-version, check-commit, trigger-milestone-check]
|
||||
if: ${{ !cancelled() && needs.check-version.result == 'success' && needs.check-commit.result == 'success' }}
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
uses: ./.github/workflows/release-milestone-manage.yml
|
||||
with:
|
||||
version: ${{ inputs.version }}
|
||||
|
||||
build-uberjar-for-release:
|
||||
needs: [check-version, check-commit, get-build-requirements]
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 50
|
||||
strategy:
|
||||
matrix:
|
||||
edition: [oss, ee]
|
||||
env:
|
||||
INTERACTIVE: false
|
||||
steps:
|
||||
- name: Get correct version number
|
||||
run: | # bash
|
||||
if [ '${{matrix.edition}}' == 'ee' ]; then
|
||||
version='${{ needs.check-version.outputs.ee }}'
|
||||
else
|
||||
version='${{ needs.check-version.outputs.oss }}'
|
||||
fi
|
||||
|
||||
echo "VERSION=$version" >> $GITHUB_ENV
|
||||
- name: Setup Java ${{ needs.get-build-requirements.outputs.java_version }}
|
||||
uses: actions/setup-java@v5.2.0
|
||||
with:
|
||||
distribution: 'temurin'
|
||||
java-version: ${{ needs.get-build-requirements.outputs.java_version }}
|
||||
- name: Setup Node.js ${{ needs.get-build-requirements.outputs.node_version }}
|
||||
uses: actions/setup-node@v6.4.0
|
||||
with:
|
||||
node-version: ${{ needs.get-build-requirements.outputs.node_version }}.x
|
||||
- name: Check out the code
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ github.event.inputs.commit }}
|
||||
- name: Prepare front-end environment
|
||||
uses: ./.github/actions/prepare-frontend
|
||||
|
||||
- name: Prepare back-end environment
|
||||
uses: ./.github/actions/prepare-backend
|
||||
|
||||
- name: Build Metabase
|
||||
if: ${{ needs.get-build-requirements.outputs.build_process != 'legacy' }}
|
||||
run: ./bin/build.sh :edition :${{ matrix.edition }} :version $VERSION
|
||||
|
||||
- name: Build Metabase (legacy)
|
||||
if: ${{ needs.get-build-requirements.outputs.build_process == 'legacy' }}
|
||||
run: cd ./bin/build-mb && clojure -X build/build! :edition :${{ matrix.edition }} :version $VERSION
|
||||
|
||||
- name: Store commit's SHA-1 hash
|
||||
run: echo ${{ inputs.commit }} > COMMIT-ID
|
||||
shell: bash
|
||||
- run: mv ./target/uberjar/metabase.jar ./metabase.jar
|
||||
- name: Calculate SHA256 checksum
|
||||
run: sha256sum ./metabase.jar > SHA256.sum
|
||||
shell: bash
|
||||
- name: Upload JARs as artifact
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: metabase-release-${{ matrix.edition }}-${{ inputs.commit }}-uberjar
|
||||
path: |
|
||||
./metabase.jar
|
||||
./COMMIT-ID
|
||||
./SHA256.sum
|
||||
|
||||
run-pre-release-tests:
|
||||
name: Run pre-release tests for ${{ inputs.version }}
|
||||
needs: build-uberjar-for-release
|
||||
uses: ./.github/workflows/pre-release.yml
|
||||
secrets: inherit
|
||||
with:
|
||||
commit: ${{ inputs.commit }}
|
||||
version: ${{ inputs.version }}
|
||||
6
.github/workflows/cache-generator.yml
vendored
6
.github/workflows/cache-generator.yml
vendored
@@ -93,10 +93,10 @@ jobs:
|
||||
":dev:ee:ee-dev:drivers:drivers-dev:test:eastwood" # backend.yml (eastwood + eastwood/test)
|
||||
":dev:ci:ee:ee-dev:drivers:drivers-dev:test" # test-driver.yml (ee) -> covers data.json:2.4.0
|
||||
":dev:ci:oss:oss-dev:drivers:drivers-dev:test" # test-driver.yml (oss)
|
||||
":dev:ci:test:ee:ee-dev" # backend.yml matrix (ee)
|
||||
":dev:ci:test:ee:ee-dev" # backend.yml matrix (ee) + generative-query-test.yml
|
||||
":dev:ci:test:oss:oss-dev" # backend.yml matrix (oss)
|
||||
":dev:ci:ee:ee-dev:test:cloverage" # backend-cloverage.yml
|
||||
":dev:drivers:drivers-dev:test" # reset-cloud-test-dataset.yml
|
||||
":dev:drivers:drivers-dev:test" # reset-cloud-test-dataset.yml + presto script
|
||||
":dev:drivers:build:build-dev:build-test:ci" # build-scripts.yml
|
||||
":ee:drivers:check" # backend.yml (check; git dep athos/clj-check)
|
||||
":ee:doc" # docs-generate-base.yml
|
||||
@@ -135,7 +135,7 @@ jobs:
|
||||
run: |
|
||||
BUN_CACHE_DIR=$(bun pm cache)
|
||||
echo "Bun cache directory: $BUN_CACHE_DIR"
|
||||
echo "dir=$BUN_CACHE_DIR" >> "$GITHUB_OUTPUT"
|
||||
echo "dir=$BUN_CACHE_DIR" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Ensure that Cypress executable is ready
|
||||
run: |
|
||||
|
||||
39
.github/workflows/codeql.yml
vendored
Normal file
39
.github/workflows/codeql.yml
vendored
Normal file
@@ -0,0 +1,39 @@
|
||||
name: "CodeQL"
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'release-**'
|
||||
pull_request:
|
||||
|
||||
jobs:
|
||||
files-changed:
|
||||
name: Check which files changed
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 3
|
||||
outputs:
|
||||
codeql: ${{ steps.changes.outputs.codeql }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Test which files changed
|
||||
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
|
||||
id: changes
|
||||
with:
|
||||
token: ${{ github.token }}
|
||||
filters: .github/file-paths.yaml
|
||||
analyze:
|
||||
needs: files-changed
|
||||
if: needs.files-changed.outputs.codeql == 'true'
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v6
|
||||
- name: Initialize CodeQL
|
||||
uses: github/codeql-action/init@v4
|
||||
with:
|
||||
languages: javascript
|
||||
config-file: ./.github/codeql/codeql-config.yml
|
||||
trap-caching: false
|
||||
- name: Perform CodeQL Analysis
|
||||
uses: github/codeql-action/analyze@v4
|
||||
2
.github/workflows/codespell.yml
vendored
2
.github/workflows/codespell.yml
vendored
@@ -27,7 +27,7 @@ jobs:
|
||||
|
||||
- name: Get current date
|
||||
id: date
|
||||
run: echo "date=$(date +'%Y-%m-%d')" >> "$GITHUB_OUTPUT"
|
||||
run: echo "date=$(date +'%Y-%m-%d')" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Run codespell check
|
||||
id: codespell_check
|
||||
|
||||
23
.github/workflows/containerize-jar.yml
vendored
23
.github/workflows/containerize-jar.yml
vendored
@@ -16,8 +16,8 @@ on:
|
||||
commit:
|
||||
type: string
|
||||
release:
|
||||
type: boolean
|
||||
default: false
|
||||
type: string
|
||||
default: 'false'
|
||||
build-args:
|
||||
type: string
|
||||
workflow_dispatch:
|
||||
@@ -40,8 +40,11 @@ on:
|
||||
type: string
|
||||
release:
|
||||
description: Whether this is a release artifact (uses metabase-release- prefix)
|
||||
type: boolean
|
||||
default: false
|
||||
type: choice
|
||||
options:
|
||||
- "false"
|
||||
- "true"
|
||||
default: "false"
|
||||
|
||||
jobs:
|
||||
containerize:
|
||||
@@ -85,8 +88,8 @@ jobs:
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
|
||||
with:
|
||||
username: ${{ inputs.release && secrets.DOCKERHUB_RELEASE_USERNAME || secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ inputs.release && secrets.DOCKERHUB_RELEASE_TOKEN || secrets.DOCKERHUB_TOKEN }}
|
||||
username: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_USERNAME || secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_TOKEN || secrets.DOCKERHUB_TOKEN }}
|
||||
- name: Set up QEMU
|
||||
if: contains(steps.platforms.outputs.result, 'arm')
|
||||
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4.0.0
|
||||
@@ -123,8 +126,8 @@ jobs:
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
|
||||
with:
|
||||
username: ${{ inputs.release && secrets.DOCKERHUB_RELEASE_USERNAME || secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ inputs.release && secrets.DOCKERHUB_RELEASE_TOKEN || secrets.DOCKERHUB_TOKEN }}
|
||||
username: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_USERNAME || secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_TOKEN || secrets.DOCKERHUB_TOKEN }}
|
||||
- name: Pull the container image
|
||||
run: docker pull ${{ inputs.repo }}:${{ inputs.tag }}
|
||||
- name: Launch container
|
||||
@@ -143,8 +146,8 @@ jobs:
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
|
||||
with:
|
||||
username: ${{ inputs.release && secrets.DOCKERHUB_RELEASE_USERNAME || secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ inputs.release && secrets.DOCKERHUB_RELEASE_TOKEN || secrets.DOCKERHUB_TOKEN }}
|
||||
username: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_USERNAME || secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_TOKEN || secrets.DOCKERHUB_TOKEN }}
|
||||
- name: Pull the container image
|
||||
run: docker pull ${{ inputs.repo }}:${{ inputs.tag }}
|
||||
- name: Launch container
|
||||
|
||||
6
.github/workflows/containerize-uberjar.yml
vendored
6
.github/workflows/containerize-uberjar.yml
vendored
@@ -3,10 +3,6 @@ name: Containerize Uberjar
|
||||
on:
|
||||
workflow_call:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
commit:
|
||||
description: the full commit hash to containerize (defaults to the triggering commit)
|
||||
type: string
|
||||
|
||||
jobs:
|
||||
get-container-info:
|
||||
@@ -63,7 +59,7 @@ jobs:
|
||||
commit: ${{ github.event.inputs.commit || github.event.pull_request.head.sha || github.sha }}
|
||||
repo: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).repo }}
|
||||
tag: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).tag }}
|
||||
release: false
|
||||
release: "false"
|
||||
build-args: |
|
||||
GIT_COMMIT_SHA=${{ github.event.inputs.commit || github.event.pull_request.head.sha || github.sha }}
|
||||
|
||||
|
||||
24
.github/workflows/debug-push-event.yml
vendored
Normal file
24
.github/workflows/debug-push-event.yml
vendored
Normal file
@@ -0,0 +1,24 @@
|
||||
name: Debug push event payload
|
||||
|
||||
# Throwaway workflow that prints the `push` webhook payload.
|
||||
#
|
||||
# Push a branch named `debug-push-event*` to see the payload for a branch
|
||||
# creation; push again to that branch to see the payload for a normal commit.
|
||||
#
|
||||
# Delete this file once you're done.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- debug-push-event**
|
||||
|
||||
jobs:
|
||||
dump-event:
|
||||
name: Print push event payload
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- name: Print push event payload
|
||||
run: |
|
||||
jq '.' "$GITHUB_EVENT_PATH"
|
||||
echo "${{github.repository}}"
|
||||
@@ -34,18 +34,18 @@ jobs:
|
||||
const lastUpdated = config.metadata?.last_updated || 'unknown';
|
||||
const reason = config.metadata?.reason || 'No reason provided';
|
||||
|
||||
// "skip" is the only status CI reads: the driver is not run at all. A driver that
|
||||
// runs but must not gate is expressed as a suite-level rule in CI Conductor, which
|
||||
// reports it in its own UI.
|
||||
const skipped = drivers.filter(d => d.status === 'skip').map(d => d.name);
|
||||
const info = drivers.filter(d => d.status === 'info').map(d => d.name);
|
||||
|
||||
if (skipped.length === 0) {
|
||||
console.log('No drivers are disabled. Skipping Slack notification.');
|
||||
if (skipped.length === 0 && info.length === 0) {
|
||||
console.log('No drivers are disabled or collecting data. Skipping Slack notification.');
|
||||
return;
|
||||
}
|
||||
|
||||
const list = (names) => names.map(d => `\`${d}\``).join(' ');
|
||||
const lines = [`:no_entry: *Disabled (not run):* ${list(skipped)}`];
|
||||
const lines = [];
|
||||
if (skipped.length) lines.push(`:no_entry: *Disabled (not run):* ${list(skipped)}`);
|
||||
if (info.length) lines.push(`:information_source: *Collecting data (non-blocking):* ${list(info)}`);
|
||||
|
||||
if (!process.env.SLACK_BOT_TOKEN) {
|
||||
throw new Error('SLACK_BOT_TOKEN is required');
|
||||
|
||||
40
.github/workflows/docs-generate-base.yml
vendored
40
.github/workflows/docs-generate-base.yml
vendored
@@ -79,10 +79,10 @@ jobs:
|
||||
id: check_changes
|
||||
run: |
|
||||
if git diff --quiet docs/; then
|
||||
echo "changes=false" >> "$GITHUB_OUTPUT"
|
||||
echo "changes=false" >> $GITHUB_OUTPUT
|
||||
echo "No changes detected in docs directory"
|
||||
else
|
||||
echo "changes=true" >> "$GITHUB_OUTPUT"
|
||||
echo "changes=true" >> $GITHUB_OUTPUT
|
||||
echo "Changes detected in docs directory"
|
||||
fi
|
||||
|
||||
@@ -109,7 +109,7 @@ jobs:
|
||||
git commit -m "Update generated docs (${{ inputs.target_branch }})" --no-verify
|
||||
git push -u origin "$BRANCH_NAME"
|
||||
|
||||
echo "branch_name=$BRANCH_NAME" >> "$GITHUB_OUTPUT"
|
||||
echo "branch_name=$BRANCH_NAME" >> $GITHUB_OUTPUT
|
||||
echo "Successfully created and pushed branch"
|
||||
|
||||
- name: Create pull request
|
||||
@@ -155,7 +155,7 @@ jobs:
|
||||
break
|
||||
else
|
||||
echo "PR creation failed on attempt $i"
|
||||
if [ "$i" -eq 3 ]; then
|
||||
if [ $i -eq 3 ]; then
|
||||
echo "All attempts failed. Exiting with error."
|
||||
exit 1
|
||||
fi
|
||||
@@ -172,24 +172,22 @@ jobs:
|
||||
- name: Workflow summary
|
||||
if: always()
|
||||
run: |
|
||||
{
|
||||
if [ "${{ steps.check_changes.outputs.changes }}" == "true" ]; then
|
||||
if [ "${{ steps.commit_and_push.outcome }}" == "success" ]; then
|
||||
if [ "${{ steps.create_pr.outcome }}" == "success" ]; then
|
||||
echo "## ✅ Documentation Updated Successfully"
|
||||
echo "- Branch: \`${{ steps.commit_and_push.outputs.branch_name }}\`"
|
||||
echo "- PR created targeting **${{ inputs.target_branch }}**"
|
||||
else
|
||||
echo "## ⚠️ Branch Created but PR Failed"
|
||||
echo "- Branch: \`${{ steps.commit_and_push.outputs.branch_name }}\`"
|
||||
echo "- Manual PR creation required"
|
||||
fi
|
||||
if [ "${{ steps.check_changes.outputs.changes }}" == "true" ]; then
|
||||
if [ "${{ steps.commit_and_push.outcome }}" == "success" ]; then
|
||||
if [ "${{ steps.create_pr.outcome }}" == "success" ]; then
|
||||
echo "## ✅ Documentation Updated Successfully" >> $GITHUB_STEP_SUMMARY
|
||||
echo "- Branch: \`${{ steps.commit_and_push.outputs.branch_name }}\`" >> $GITHUB_STEP_SUMMARY
|
||||
echo "- PR created targeting **${{ inputs.target_branch }}**" >> $GITHUB_STEP_SUMMARY
|
||||
else
|
||||
echo "## ❌ Failed to Create Branch"
|
||||
echo "- Check workflow logs for details"
|
||||
echo "## ⚠️ Branch Created but PR Failed" >> $GITHUB_STEP_SUMMARY
|
||||
echo "- Branch: \`${{ steps.commit_and_push.outputs.branch_name }}\`" >> $GITHUB_STEP_SUMMARY
|
||||
echo "- Manual PR creation required" >> $GITHUB_STEP_SUMMARY
|
||||
fi
|
||||
else
|
||||
echo "## ℹ️ No Changes Detected"
|
||||
echo "- Documentation is up to date on **${{ inputs.target_branch }}**"
|
||||
echo "## ❌ Failed to Create Branch" >> $GITHUB_STEP_SUMMARY
|
||||
echo "- Check workflow logs for details" >> $GITHUB_STEP_SUMMARY
|
||||
fi
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
else
|
||||
echo "## ℹ️ No Changes Detected" >> $GITHUB_STEP_SUMMARY
|
||||
echo "- Documentation is up to date on **${{ inputs.target_branch }}**" >> $GITHUB_STEP_SUMMARY
|
||||
fi
|
||||
|
||||
352
.github/workflows/drivers.yml
vendored
352
.github/workflows/drivers.yml
vendored
@@ -45,6 +45,26 @@ jobs:
|
||||
sqlite-should-run: ${{ steps.driver-decisions.outputs.sqlite-should-run }}
|
||||
sqlserver-should-run: ${{ steps.driver-decisions.outputs.sqlserver-should-run }}
|
||||
vertica-should-run: ${{ steps.driver-decisions.outputs.vertica-should-run }}
|
||||
# Per-driver configured status from ci-test-config (skip | info | required)
|
||||
h2-status: ${{ steps.driver-decisions.outputs.h2-status }}
|
||||
athena-status: ${{ steps.driver-decisions.outputs.athena-status }}
|
||||
bigquery-status: ${{ steps.driver-decisions.outputs.bigquery-status }}
|
||||
clickhouse-status: ${{ steps.driver-decisions.outputs.clickhouse-status }}
|
||||
databricks-status: ${{ steps.driver-decisions.outputs.databricks-status }}
|
||||
druid-jdbc-status: ${{ steps.driver-decisions.outputs.druid-jdbc-status }}
|
||||
mongo-status: ${{ steps.driver-decisions.outputs.mongo-status }}
|
||||
mongo-ssl-status: ${{ steps.driver-decisions.outputs.mongo-ssl-status }}
|
||||
mongo-sharded-cluster-status: ${{ steps.driver-decisions.outputs.mongo-sharded-cluster-status }}
|
||||
mysql-mariadb-status: ${{ steps.driver-decisions.outputs.mysql-mariadb-status }}
|
||||
oracle-status: ${{ steps.driver-decisions.outputs.oracle-status }}
|
||||
postgres-status: ${{ steps.driver-decisions.outputs.postgres-status }}
|
||||
presto-jdbc-status: ${{ steps.driver-decisions.outputs.presto-jdbc-status }}
|
||||
redshift-status: ${{ steps.driver-decisions.outputs.redshift-status }}
|
||||
snowflake-status: ${{ steps.driver-decisions.outputs.snowflake-status }}
|
||||
sparksql-status: ${{ steps.driver-decisions.outputs.sparksql-status }}
|
||||
sqlite-status: ${{ steps.driver-decisions.outputs.sqlite-status }}
|
||||
sqlserver-status: ${{ steps.driver-decisions.outputs.sqlserver-status }}
|
||||
vertica-status: ${{ steps.driver-decisions.outputs.vertica-status }}
|
||||
# Per-driver quarantine conflict flags (set when driver is quarantined but has file changes)
|
||||
athena-quarantine-conflict: ${{ steps.driver-decisions.outputs.athena-quarantine-conflict }}
|
||||
bigquery-quarantine-conflict: ${{ steps.driver-decisions.outputs.bigquery-quarantine-conflict }}
|
||||
@@ -66,6 +86,28 @@ jobs:
|
||||
sqlserver-quarantine-conflict: ${{ steps.driver-decisions.outputs.sqlserver-quarantine-conflict }}
|
||||
vertica-quarantine-conflict: ${{ steps.driver-decisions.outputs.vertica-quarantine-conflict }}
|
||||
steps:
|
||||
# Up front (before any driver job), drop info-driver comments from older commits so a
|
||||
# superseded comment disappears immediately rather than waiting for this run's drivers.
|
||||
# Keeps this commit's comment (if any) so a re-run doesn't flicker.
|
||||
- name: Remove stale info-driver comments
|
||||
if: ${{ github.event_name == 'pull_request' }}
|
||||
uses: actions/github-script@v9
|
||||
continue-on-error: true
|
||||
with:
|
||||
script: | # js
|
||||
const pr = context.payload.pull_request;
|
||||
if (!pr) return;
|
||||
const { owner, repo } = context.repo;
|
||||
const PREFIX = "<!-- info-driver-result";
|
||||
const keep = `<!-- info-driver-results:${pr.head.sha} -->`;
|
||||
const { data: comments } = await github.rest.issues.listComments({
|
||||
owner, repo, issue_number: pr.number, per_page: 100,
|
||||
});
|
||||
for (const c of comments) {
|
||||
if (c.body.includes(PREFIX) && !c.body.includes(keep)) {
|
||||
await github.rest.issues.deleteComment({ owner, repo, comment_id: c.id }).catch(() => {});
|
||||
}
|
||||
}
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v6
|
||||
- name: Fetch base branch
|
||||
@@ -138,6 +180,18 @@ jobs:
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.athena-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.athena-status }}
|
||||
|
||||
be-tests-bigquery-cloud-sdk-ee:
|
||||
needs: determine-driver-skips
|
||||
@@ -202,6 +256,19 @@ jobs:
|
||||
test-args: ${{ matrix.job.test-args }}
|
||||
artifact-suffix: ${{ matrix.job.name }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.bigquery-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.bigquery-status }}
|
||||
report-variant: ${{ matrix.job.name }}
|
||||
- name: Cleanup python-runner
|
||||
if: ${{ always() && matrix.job.needs-python-runner == true }}
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -275,6 +342,18 @@ jobs:
|
||||
test-args: ${{ matrix.job.test-args }}
|
||||
artifact-suffix: ${{ matrix.job.name }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.clickhouse-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.clickhouse-status }}
|
||||
- name: Cleanup python-runner
|
||||
if: ${{ always() && matrix.job.needs-python-runner == true }}
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -319,6 +398,18 @@ jobs:
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.druid-jdbc-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.druid-jdbc-status }}
|
||||
|
||||
be-tests-databricks-ee:
|
||||
needs: determine-driver-skips
|
||||
@@ -381,6 +472,19 @@ jobs:
|
||||
artifact-suffix: ${{ matrix.job.name }}
|
||||
test-args: ${{ matrix.job.test-args }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.databricks-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.databricks-status }}
|
||||
report-variant: ${{ matrix.job.name }}
|
||||
|
||||
be-tests-databricks-multi-catalog-ee:
|
||||
needs: determine-driver-skips
|
||||
@@ -444,6 +548,19 @@ jobs:
|
||||
artifact-suffix: ${{ matrix.job.name }}
|
||||
test-args: ${{ matrix.job.test-args }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.databricks-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.databricks-status }}
|
||||
report-variant: ${{ matrix.job.name }}
|
||||
|
||||
be-tests-h2:
|
||||
needs: determine-driver-skips
|
||||
@@ -464,6 +581,18 @@ jobs:
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.h2-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.h2-status }}
|
||||
|
||||
be-tests-h2-oss:
|
||||
needs: determine-driver-skips
|
||||
@@ -485,6 +614,18 @@ jobs:
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.h2-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.h2-status }}
|
||||
|
||||
be-tests-mysql-mariadb:
|
||||
needs: determine-driver-skips
|
||||
@@ -616,6 +757,18 @@ jobs:
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.mysql-mariadb-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.mysql-mariadb-status }}
|
||||
- name: Cleanup python-runner
|
||||
if: always()
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -686,6 +839,18 @@ jobs:
|
||||
test-args: ${{ matrix.job.test-args }}
|
||||
artifact-suffix: ${{ matrix.version.name }}-${{ matrix.job.name }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.mongo-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.mongo-status }}
|
||||
- name: Cleanup python-runner
|
||||
if: ${{ always() && matrix.job.needs-python-runner == true }}
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -758,6 +923,18 @@ jobs:
|
||||
:exclude-tags '[:mongo-sharded-cluster-tests :mb/transforms-python-test]'
|
||||
:only-tags [:mb/driver-tests]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.mongo-ssl-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.mongo-ssl-status }}
|
||||
|
||||
be-tests-mongo-sharded-cluster-ee:
|
||||
needs: determine-driver-skips
|
||||
@@ -793,6 +970,18 @@ jobs:
|
||||
test-args: >-
|
||||
:only metabase.driver.mongo.sharded-cluster-test
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.mongo-sharded-cluster-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.mongo-sharded-cluster-status }}
|
||||
|
||||
be-tests-oracle:
|
||||
needs: determine-driver-skips
|
||||
@@ -868,6 +1057,18 @@ jobs:
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.oracle-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.oracle-status }}
|
||||
|
||||
be-tests-postgres:
|
||||
needs: determine-driver-skips
|
||||
@@ -962,6 +1163,19 @@ jobs:
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.postgres-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.postgres-status }}
|
||||
report-variant: ${{ matrix.version.name }}-${{ matrix.job.name }}
|
||||
- name: Cleanup python-runner
|
||||
if: always()
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -1039,6 +1253,18 @@ jobs:
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.presto-jdbc-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.presto-jdbc-status }}
|
||||
|
||||
be-tests-redshift-ee:
|
||||
needs: determine-driver-skips
|
||||
@@ -1113,6 +1339,19 @@ jobs:
|
||||
${{ matrix.job.test-args }}
|
||||
artifact-suffix: ${{ matrix.job.name }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.redshift-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: ${{ !matrix.job.skip && !cancelled() }}
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.redshift-status }}
|
||||
report-variant: ${{ matrix.job.name }}
|
||||
- name: Cleanup python-runner
|
||||
if: ${{ always() && matrix.job.needs-python-runner == true }}
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -1186,6 +1425,19 @@ jobs:
|
||||
test-args: ${{ matrix.job.test-args }}
|
||||
artifact-suffix: ${{ matrix.job.name }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.snowflake-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.snowflake-status }}
|
||||
report-variant: ${{ matrix.job.name }}
|
||||
- name: Cleanup python-runner
|
||||
if: ${{ always() && matrix.job.needs-python-runner == true }}
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -1228,6 +1480,18 @@ jobs:
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.sparksql-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.sparksql-status }}
|
||||
|
||||
be-tests-sqlite-ee:
|
||||
needs: determine-driver-skips
|
||||
@@ -1259,6 +1523,18 @@ jobs:
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.sqlite-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.sqlite-status }}
|
||||
|
||||
be-tests-sqlserver:
|
||||
needs: determine-driver-skips
|
||||
@@ -1325,6 +1601,18 @@ jobs:
|
||||
test-args: ${{ matrix.job.test-args }}
|
||||
artifact-suffix: ${{ matrix.version.name }}-${{ matrix.job.name }}
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.sqlserver-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.sqlserver-status }}
|
||||
- name: Cleanup python-runner
|
||||
if: ${{ always() && matrix.job.needs-python-runner == true }}
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -1359,6 +1647,15 @@ jobs:
|
||||
# junit-name: 'be-tests-starburst-ee'
|
||||
# test-args: >-
|
||||
# :only-tags [:mb/driver-tests]
|
||||
# - name: Upload Test Results
|
||||
# uses: ./.github/actions/upload-test-results
|
||||
# if: always()
|
||||
# with:
|
||||
# input-path: ./target/junit/
|
||||
# output-name: ${{ github.job }}
|
||||
# bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
# role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
# aws-region: ${{ vars.AWS_REGION }}
|
||||
|
||||
login-to-ecr:
|
||||
needs: determine-driver-skips
|
||||
@@ -1432,13 +1729,32 @@ jobs:
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.vertica-status }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
driver-status: ${{ needs.determine-driver-skips.outputs.vertica-status }}
|
||||
|
||||
drivers-tests-result:
|
||||
needs:
|
||||
# Supplies each driver's configured status (info/skip/required) so this gate can ignore
|
||||
# "info" drivers regardless of their result -- including cancelled or timed-out runs, which
|
||||
# the per-job continue-on-error cannot force green because the controlling step never
|
||||
# completes. Not a driver job; it is filtered out of the pass/fail check below.
|
||||
- determine-driver-skips
|
||||
# if you add a driver job, you must add it to this list for it to be a required check
|
||||
- be-tests-h2
|
||||
- be-tests-athena-ee
|
||||
# Listed unconditionally; ci-test-config controls their effect via per-driver status.
|
||||
# An "info" driver is reported as passing, so it never blocks this check; flip it to
|
||||
# "required" in the config to make a real failure gate, with no change here.
|
||||
- be-tests-snowflake-ee
|
||||
- be-tests-bigquery-cloud-sdk-ee
|
||||
- be-tests-databricks-ee
|
||||
@@ -1467,18 +1783,46 @@ jobs:
|
||||
uses: actions/github-script@v9
|
||||
env:
|
||||
needs: ${{ toJson(needs) }}
|
||||
# Job name -> configured driver status, from determine-driver-skips. An "info" driver is
|
||||
# data-collection only and must never gate; a cancelled/timed-out info job lands here as
|
||||
# cancelled/failure (its own continue-on-error can't force it green), so we drop info jobs
|
||||
# from the gate entirely. Empty/unknown status falls back to "required" (gates as before).
|
||||
statuses: |
|
||||
{
|
||||
"be-tests-athena-ee": "${{ needs.determine-driver-skips.outputs.athena-status }}",
|
||||
"be-tests-bigquery-cloud-sdk-ee": "${{ needs.determine-driver-skips.outputs.bigquery-status }}",
|
||||
"be-tests-clickhouse-ee": "${{ needs.determine-driver-skips.outputs.clickhouse-status }}",
|
||||
"be-tests-databricks-ee": "${{ needs.determine-driver-skips.outputs.databricks-status }}",
|
||||
"be-tests-druid-jdbc-ee": "${{ needs.determine-driver-skips.outputs.druid-jdbc-status }}",
|
||||
"be-tests-h2": "${{ needs.determine-driver-skips.outputs.h2-status }}",
|
||||
"be-tests-mongo": "${{ needs.determine-driver-skips.outputs.mongo-status }}",
|
||||
"be-tests-mongo-ssl": "${{ needs.determine-driver-skips.outputs.mongo-ssl-status }}",
|
||||
"be-tests-mongo-sharded-cluster-ee": "${{ needs.determine-driver-skips.outputs.mongo-sharded-cluster-status }}",
|
||||
"be-tests-mysql-mariadb": "${{ needs.determine-driver-skips.outputs.mysql-mariadb-status }}",
|
||||
"be-tests-oracle": "${{ needs.determine-driver-skips.outputs.oracle-status }}",
|
||||
"be-tests-postgres": "${{ needs.determine-driver-skips.outputs.postgres-status }}",
|
||||
"be-tests-presto-jdbc-ee": "${{ needs.determine-driver-skips.outputs.presto-jdbc-status }}",
|
||||
"be-tests-redshift-ee": "${{ needs.determine-driver-skips.outputs.redshift-status }}",
|
||||
"be-tests-snowflake-ee": "${{ needs.determine-driver-skips.outputs.snowflake-status }}",
|
||||
"be-tests-sparksql-ee": "${{ needs.determine-driver-skips.outputs.sparksql-status }}",
|
||||
"be-tests-sqlite-ee": "${{ needs.determine-driver-skips.outputs.sqlite-status }}",
|
||||
"be-tests-sqlserver": "${{ needs.determine-driver-skips.outputs.sqlserver-status }}"
|
||||
}
|
||||
with:
|
||||
script: | # js
|
||||
const needs = JSON.parse(process.env.needs);
|
||||
const statuses = JSON.parse(process.env.statuses);
|
||||
|
||||
// Drop the determine-driver-skips bookkeeping job; every remaining job is a driver
|
||||
// job whose result gates this check.
|
||||
// Drop the determine-driver-skips bookkeeping job, then drop "info" drivers: they are
|
||||
// non-gating, so their result (including cancelled/timed-out) must not fail this check.
|
||||
const jobs = Object.entries(needs)
|
||||
.filter(([jobName]) => jobName !== 'determine-driver-skips')
|
||||
.map(([jobName, jobValues]) => ({
|
||||
name: jobName,
|
||||
result: jobValues.result
|
||||
}));
|
||||
result: jobValues.result,
|
||||
status: statuses[jobName] || 'required'
|
||||
}))
|
||||
.filter(job => job.status !== 'info');
|
||||
|
||||
// are all jobs skipped or successful?
|
||||
if (jobs.every(job => (job.result === 'skipped' || job.result === 'success'))) {
|
||||
|
||||
2
.github/workflows/e2e-matrix-builder.yml
vendored
2
.github/workflows/e2e-matrix-builder.yml
vendored
@@ -4,7 +4,7 @@ on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
chunks:
|
||||
required: false
|
||||
required: true
|
||||
type: number
|
||||
default: 30
|
||||
specs:
|
||||
|
||||
43
.github/workflows/e2e-stress-test-flake-fix.yml
vendored
43
.github/workflows/e2e-stress-test-flake-fix.yml
vendored
@@ -98,9 +98,9 @@ jobs:
|
||||
run: |
|
||||
SPEC='${{ inputs.spec }}'
|
||||
if [[ "$SPEC" == e2e/test-component/scenarios/embedding-sdk/* ]]; then
|
||||
echo "type=component" >> "$GITHUB_OUTPUT"
|
||||
echo "type=component" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "type=e2e" >> "$GITHUB_OUTPUT"
|
||||
echo "type=e2e" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
workflow-summary:
|
||||
@@ -110,33 +110,20 @@ jobs:
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- name: Generate workflow summary
|
||||
env:
|
||||
BRANCH: ${{ github.ref_name }}
|
||||
SPEC: ${{ inputs.spec }}
|
||||
TEST_TYPE: ${{ needs.determine-test-type.outputs.type }}
|
||||
BURN_IN: ${{ inputs.burn_in }}
|
||||
GREP: ${{ inputs.grep }}
|
||||
QA_DB: ${{ inputs.qa_db }}
|
||||
ENABLE_NETWORK_THROTTLING: ${{ inputs.enable_network_throttling }}
|
||||
FAIL_FAST: ${{ inputs.fail_fast }}
|
||||
BUILD_JAR: ${{ inputs.build_jar }}
|
||||
SENDER: ${{ github.event.sender.login }}
|
||||
run: |
|
||||
{
|
||||
echo '**Inputs:**'
|
||||
echo ''
|
||||
echo "- \`branch\`: $BRANCH"
|
||||
echo "- \`spec\`: $SPEC"
|
||||
echo "- \`test_type\` (inferred): $TEST_TYPE"
|
||||
echo "- \`burn_in\`: $BURN_IN"
|
||||
echo "- \`grep\`: \"$GREP\""
|
||||
echo "- \`qa_db\`: \"$QA_DB\""
|
||||
echo "- \`enable_network_throttling\`: \"$ENABLE_NETWORK_THROTTLING\""
|
||||
echo "- \`fail_fast\`: \"$FAIL_FAST\""
|
||||
echo "- \`build_jar\`: \"$BUILD_JAR\""
|
||||
echo ''
|
||||
echo "triggered by: @$SENDER"
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
echo '**Inputs:**' >> $GITHUB_STEP_SUMMARY
|
||||
echo '' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `branch`: ${{ github.ref_name }}' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `spec`: ${{ inputs.spec }}' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `test_type` (inferred): ${{ needs.determine-test-type.outputs.type }}' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `burn_in`: ${{ inputs.burn_in }}' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `grep`: "${{ inputs.grep }}"' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `qa_db`: "${{ inputs.qa_db }}"' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `enable_network_throttling`: "${{ inputs.enable_network_throttling }}"' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `fail_fast`: "${{ inputs.fail_fast }}"' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `build_jar`: "${{ inputs.build_jar }}"' >> $GITHUB_STEP_SUMMARY
|
||||
echo '' >> $GITHUB_STEP_SUMMARY
|
||||
echo 'triggered by: @${{ github.event.sender.login }}' >> $GITHUB_STEP_SUMMARY
|
||||
|
||||
stress-test-flake-fix:
|
||||
needs: [determine-test-type]
|
||||
|
||||
@@ -63,14 +63,14 @@ jobs:
|
||||
fi
|
||||
|
||||
echo "Changed e2e Cypress specs:"
|
||||
echo "$FILES" | awk '{ print " - " $0 }'
|
||||
echo "$FILES" | sed 's/^/ - /'
|
||||
|
||||
# Compact JSON array for the matrix, e.g. ["e2e/test/scenarios/a.cy.spec.js"].
|
||||
SPECS_JSON=$(echo "$FILES" | jq -R . | jq -cs .)
|
||||
echo "specs=${SPECS_JSON}" >> "$GITHUB_OUTPUT"
|
||||
echo "should_run=true" >> "$GITHUB_OUTPUT"
|
||||
|
||||
echo "Note: you can use the burn-50 or burn-100 labels to increase the burn-in count from the default of 20." >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "Note: you can use the burn-50 or burn-100 labels to increase the burn-in count from the default of 20." >> $GITHUB_STEP_SUMMARY
|
||||
|
||||
stress-test:
|
||||
name: Stress test
|
||||
|
||||
12
.github/workflows/e2e-test.yml
vendored
12
.github/workflows/e2e-test.yml
vendored
@@ -216,6 +216,18 @@ jobs:
|
||||
echo "No ./target/junit directory; nothing to process."
|
||||
fi
|
||||
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit
|
||||
# Related to our custom upload to S3
|
||||
output-name: e2e-${{ inputs.name }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ inputs.specs && steps.tagged-e2e.outcome || steps.split-e2e.outcome }}
|
||||
|
||||
- name: Quarantine gate
|
||||
# Only one of the two run steps executes; the other reports `skipped`. Only gate on actual failure.
|
||||
if: ${{ !cancelled() && (steps.split-e2e.outcome == 'failure' || steps.tagged-e2e.outcome == 'failure') }}
|
||||
|
||||
@@ -111,6 +111,17 @@ jobs:
|
||||
echo "No ./target/junit directory; nothing to process."
|
||||
fi
|
||||
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit
|
||||
output-name: e2e-embedding-sdk-react-${{ matrix.react-version }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.run-tests.outcome }}
|
||||
|
||||
- name: Quarantine gate
|
||||
if: ${{ !cancelled() && steps.run-tests.outcome == 'failure' }}
|
||||
continue-on-error: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
@@ -280,6 +291,17 @@ jobs:
|
||||
echo "No ./target/junit directory; nothing to process."
|
||||
fi
|
||||
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit
|
||||
output-name: e2e-embedding-sdk
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.run-tests.outcome }}
|
||||
|
||||
- name: Quarantine gate
|
||||
if: ${{ !cancelled() && steps.run-tests.outcome == 'failure' }}
|
||||
continue-on-error: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
|
||||
|
||||
4
.github/workflows/emotion-stats-update.yml
vendored
4
.github/workflows/emotion-stats-update.yml
vendored
@@ -40,9 +40,9 @@ jobs:
|
||||
| wc -l || true
|
||||
)
|
||||
|
||||
echo "$((oss + ee)) files importing @emotion/*"
|
||||
echo "$(($oss+$ee)) files importing @emotion/*"
|
||||
|
||||
echo "EMOTION_FILES=$((oss + ee))" >> "$GITHUB_OUTPUT"
|
||||
echo "EMOTION_FILES=$(($oss+$ee))" >> $GITHUB_OUTPUT
|
||||
- name: Upload count to Stats
|
||||
uses: actions/github-script@v9
|
||||
env:
|
||||
|
||||
36
.github/workflows/escalation-slack-notification.yml
vendored
Normal file
36
.github/workflows/escalation-slack-notification.yml
vendored
Normal file
@@ -0,0 +1,36 @@
|
||||
name: Escalation Slack Notification
|
||||
|
||||
on:
|
||||
issues:
|
||||
types:
|
||||
- labeled
|
||||
jobs:
|
||||
on-escalation-label:
|
||||
if: github.event.label.name == '.Escalation'
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- name: Format issue title (escape quotes)
|
||||
uses: actions/github-script@v9
|
||||
id: vars
|
||||
with:
|
||||
script: |
|
||||
core.setOutput('issue_title', ${{ toJson(github.event.issue.title) }}.replaceAll(/"/g, '\\"'));
|
||||
- name: Send escalated issue ${{ github.event.issue.number }} to Slack
|
||||
id: slack
|
||||
uses: slackapi/slack-github-action@45a88b9581bfab2566dc881e2cd66d334e621e2c # v3.0.3
|
||||
with:
|
||||
webhook: ${{ secrets.SLACK_ESCALATIONS_WEBHOOK_URL }}
|
||||
webhook-type: incoming-webhook
|
||||
payload: |
|
||||
{
|
||||
"blocks": [
|
||||
{
|
||||
"type": "section",
|
||||
"text": {
|
||||
"type": "mrkdwn",
|
||||
"text": "🔥🔥🔥 ${{ steps.vars.outputs.issue_title }}\n${{ github.event.issue.html_url }}"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -23,23 +23,15 @@ jobs:
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- name: Generate workflow summary
|
||||
env:
|
||||
BRANCH: ${{ github.ref_name }}
|
||||
SPEC: ${{ inputs.spec }}
|
||||
BURN_IN: ${{ inputs.burn_in }}
|
||||
GREP: ${{ inputs.grep }}
|
||||
SENDER: ${{ github.event.sender.login }}
|
||||
run: |
|
||||
{
|
||||
echo '**Inputs:**'
|
||||
echo ''
|
||||
echo "- \`branch\`: $BRANCH"
|
||||
echo "- \`spec\`: $SPEC"
|
||||
echo "- \`burn_in\`: $BURN_IN"
|
||||
echo "- \`grep\`: \"$GREP\""
|
||||
echo ''
|
||||
echo "triggered by: @$SENDER"
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
echo '**Inputs:**' >> $GITHUB_STEP_SUMMARY
|
||||
echo '' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `branch`: ${{ github.ref_name }}' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `spec`: ${{ inputs.spec }}' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `burn_in`: ${{ inputs.burn_in }}' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `grep`: "${{ inputs.grep }}"' >> $GITHUB_STEP_SUMMARY
|
||||
echo '' >> $GITHUB_STEP_SUMMARY
|
||||
echo 'triggered by: @${{ github.event.sender.login }}' >> $GITHUB_STEP_SUMMARY
|
||||
|
||||
frontend-unit-test-stress-test-flake-fix:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
@@ -56,9 +48,10 @@ jobs:
|
||||
- name: Stress-test ${{ github.event.inputs.spec }} ${{ github.event.inputs.burn_in }} times
|
||||
run: |
|
||||
for ((i=1; i<=BURN_IN; i++)); do
|
||||
if ! bun run jest --runInBand --ci --silent --no-cache \
|
||||
bun run jest --runInBand --ci --silent --no-cache \
|
||||
--testPathPatterns "${SPEC}" \
|
||||
--testNamePattern "${GREP}"; then
|
||||
--testNamePattern "${GREP}"
|
||||
if [[ "$?" != 0 ]]; then
|
||||
echo "Failed after $i attempts" && break;
|
||||
fi
|
||||
done
|
||||
|
||||
10
.github/workflows/frontend.yml
vendored
10
.github/workflows/frontend.yml
vendored
@@ -129,6 +129,16 @@ jobs:
|
||||
adapter: frontend
|
||||
test-suite: fe-tests-unit
|
||||
enforce-quarantine: ${{ vars.QUARANTINE_ENGINE == 'ci-conductor' }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always() && github.event_name != 'workflow_dispatch'
|
||||
with:
|
||||
input-path: ./target/junit
|
||||
output-name: frontend-unit
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.fe-tests.outcome }}
|
||||
|
||||
fe-tests-timezones:
|
||||
if: ${{ !inputs.skip }}
|
||||
|
||||
58
.github/workflows/generative-query-test.yml
vendored
Normal file
58
.github/workflows/generative-query-test.yml
vendored
Normal file
@@ -0,0 +1,58 @@
|
||||
name: Generative Query Test
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
context-seed:
|
||||
description: 'Context seed (empty = random)'
|
||||
type: number
|
||||
iteration-seed:
|
||||
description: 'Iteration seed (for iteration replay, empty = dependent on Context seed)'
|
||||
type: number
|
||||
iterations:
|
||||
description: 'Iterations (number of test iterations, overrides seconds)'
|
||||
type: number
|
||||
seconds:
|
||||
description: 'Seconds (Number of seconds after which next iteration will not start)'
|
||||
test:
|
||||
description: 'Test or namespace to run'
|
||||
default: metabase.query-processor.generative-test/execution-with-cards-test
|
||||
ref:
|
||||
description: 'Ref to run generative tests on'
|
||||
default: qgen-ms2
|
||||
|
||||
# TODO: Consider renaming the job in next milestones
|
||||
jobs:
|
||||
generative-query-test:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 90
|
||||
name: Generative Query Test
|
||||
env:
|
||||
CI: 'true'
|
||||
MB_GENTEST_CONTEXT_SEED: ${{ inputs.context-seed }}
|
||||
MB_GENTEST_ITERATION_SEED: ${{ inputs.iteration-seed }}
|
||||
MB_GENTEST_LIMIT_ITERATIONS: ${{ inputs.iterations }}
|
||||
MB_GENTEST_LIMIT_SECONDS: ${{ inputs.seconds }}
|
||||
MB_GENTEST_RUN: true
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ inputs.ref }}
|
||||
- name: Prepare back-end environment
|
||||
uses: ./.github/actions/prepare-backend
|
||||
- name: Prepare front-end environment
|
||||
uses: ./.github/actions/prepare-frontend
|
||||
|
||||
- name: Run test
|
||||
run: clojure -X:dev:ci:test:ee:ee-dev :only '${{ inputs.test }}'
|
||||
|
||||
- name: Publish Test Report (JUnit)
|
||||
uses: dorny/test-reporter@a43b3a5f7366b97d083190328d2c652e1a8b6aa2 # v3.0.0
|
||||
if: failure()
|
||||
with:
|
||||
path: "target/junit/**/*_test.xml"
|
||||
name: JUnit Test Report (Generative Query test)
|
||||
reporter: java-junit
|
||||
list-suites: failed
|
||||
list-tests: failed
|
||||
fail-on-error: false
|
||||
51
.github/workflows/i18n.yml
vendored
Normal file
51
.github/workflows/i18n.yml
vendored
Normal file
@@ -0,0 +1,51 @@
|
||||
name: i18n
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'release-**'
|
||||
pull_request:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
files-changed:
|
||||
name: Check which files changed
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 3
|
||||
outputs:
|
||||
i18n: ${{ steps.changes.outputs.i18n }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Test which files changed
|
||||
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
|
||||
id: changes
|
||||
with:
|
||||
token: ${{ github.token }}
|
||||
filters: .github/file-paths.yaml
|
||||
|
||||
verify-i18n-files:
|
||||
needs: files-changed
|
||||
if: needs.files-changed.outputs.i18n == 'true'
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Prepare front-end environment
|
||||
uses: ./.github/actions/prepare-frontend
|
||||
- name: Prepare back-end environment
|
||||
uses: ./.github/actions/prepare-backend
|
||||
- name: Install xgettext if necessary
|
||||
run: |
|
||||
if ! command -v xgettext > /dev/null; then
|
||||
sudo apt install gettext
|
||||
fi
|
||||
- name: Check xgettext version
|
||||
run: xgettext --version
|
||||
- name: Check i18n tags/make sure template can be built
|
||||
run: ./bin/i18n/update-translation-template
|
||||
- name: Verify i18n translations (.po files)
|
||||
run: ./bin/i18n/build-translation-resources
|
||||
4
.github/workflows/js-stats-update.yml
vendored
4
.github/workflows/js-stats-update.yml
vendored
@@ -18,9 +18,9 @@ jobs:
|
||||
oss=$(find ./frontend/src/metabase -type f -name "*.js" -o -name "*.jsx" | wc -l)
|
||||
ee=$(find ./enterprise/frontend/src/metabase-enterprise -type f -name "*.js" -o -name "*.jsx" | wc -l)
|
||||
|
||||
echo $((oss + ee)) javascript files
|
||||
echo $(($oss+$ee)) javascript files
|
||||
|
||||
echo "JS_FILES=$((oss + ee))" >> "$GITHUB_OUTPUT"
|
||||
echo "JS_FILES=$(($oss+$ee))" >> $GITHUB_OUTPUT
|
||||
- name: Upload count to Stats
|
||||
uses: actions/github-script@v9
|
||||
env:
|
||||
|
||||
184
.github/workflows/lint.yml
vendored
184
.github/workflows/lint.yml
vendored
@@ -1,184 +0,0 @@
|
||||
name: Lint
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'release-**'
|
||||
pull_request:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
files-changed:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 10
|
||||
outputs:
|
||||
actionlint: ${{ steps.changes.outputs.actionlint }}
|
||||
actionlint_files: ${{ steps.changes.outputs.actionlint_files }}
|
||||
codeql: ${{ steps.changes.outputs.codeql }}
|
||||
i18n: ${{ steps.changes.outputs.i18n }}
|
||||
openapi: ${{ steps.changes.outputs.openapi }}
|
||||
snowplow: ${{ steps.changes.outputs.snowplow }}
|
||||
yaml: ${{ steps.changes.outputs.yaml }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Test which files changed
|
||||
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
|
||||
id: changes
|
||||
with:
|
||||
token: ${{ github.token }}
|
||||
filters: .github/file-paths.yaml
|
||||
list-files: shell
|
||||
|
||||
lint:
|
||||
needs: files-changed
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup frontend env
|
||||
if: |
|
||||
needs.files-changed.outputs.yaml == 'true' ||
|
||||
needs.files-changed.outputs.openapi == 'true'
|
||||
uses: ./.github/actions/prepare-frontend
|
||||
|
||||
# Always required by whitespace linter, also used by OpenAPI spec validation
|
||||
- name: Setup backend env
|
||||
uses: ./.github/actions/prepare-backend
|
||||
|
||||
- name: Setup actionlint
|
||||
if: needs.files-changed.outputs.actionlint == 'true'
|
||||
id: download
|
||||
shell: bash
|
||||
run: |
|
||||
if command -v actionlint; then
|
||||
echo "executable=actionlint" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
# script sets executable output, for details see: https://github.com/rhysd/actionlint/blob/main/docs/usage.md#use-actionlint-on-github-actions
|
||||
bash <(curl -fsSL https://raw.githubusercontent.com/rhysd/actionlint/v1.7.12/scripts/download-actionlint.bash)
|
||||
fi
|
||||
|
||||
- parallel:
|
||||
# Lint whitespace, always runs
|
||||
- name: Run Whitespace Linter
|
||||
run: clojure -T:whitespace-linter lint :whitespace-linter/progress-bar false
|
||||
|
||||
# Lint YAML
|
||||
- name: Lint YAML files
|
||||
if: needs.files-changed.outputs.yaml == 'true'
|
||||
run: bun run lint-yaml
|
||||
|
||||
# Lint Github Actions
|
||||
- name: Run actionlint
|
||||
if: needs.files-changed.outputs.actionlint == 'true'
|
||||
run: ${{ steps.download.outputs.executable }} --color --verbose ${{ needs.files-changed.outputs.actionlint_files }}
|
||||
|
||||
# Generate and Lint OpenAPI spec
|
||||
- name: Generate OpenAPI spec
|
||||
if: needs.files-changed.outputs.openapi == 'true'
|
||||
run: bun run generate-openapi
|
||||
|
||||
# Parallel steps run asynchronously, and explicitly waiting for a parallel step means "if" statements are ignored.
|
||||
# Move OpenAPI validation outside the parallel block so that spec generation happens asynchronously without requiring
|
||||
# an explicit "wait".
|
||||
- name: Validate OpenAPI spec
|
||||
if: needs.files-changed.outputs.openapi == 'true'
|
||||
run: bun run lint-openapi
|
||||
|
||||
codeql:
|
||||
if: needs.files-changed.outputs.codeql == 'true'
|
||||
needs: files-changed
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Initialize CodeQL
|
||||
uses: github/codeql-action/init@v4
|
||||
with:
|
||||
languages: javascript
|
||||
config-file: ./.github/codeql/codeql-config.yml
|
||||
trap-caching: false
|
||||
- name: Perform CodeQL Analysis
|
||||
uses: github/codeql-action/analyze@v4
|
||||
|
||||
i18n:
|
||||
needs: files-changed
|
||||
if: needs.files-changed.outputs.i18n == 'true'
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Prepare front-end environment
|
||||
uses: ./.github/actions/prepare-frontend
|
||||
- name: Prepare back-end environment
|
||||
uses: ./.github/actions/prepare-backend
|
||||
- name: Install xgettext if necessary
|
||||
run: |
|
||||
if ! command -v xgettext > /dev/null; then
|
||||
sudo apt install gettext
|
||||
fi
|
||||
- name: Check xgettext version
|
||||
run: xgettext --version
|
||||
- name: Check i18n tags/make sure template can be built
|
||||
run: ./bin/i18n/update-translation-template
|
||||
- name: Verify i18n translations (.po files)
|
||||
run: ./bin/i18n/build-translation-resources
|
||||
|
||||
snowplow:
|
||||
needs: files-changed
|
||||
if: needs.files-changed.outputs.snowplow == 'true'
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v6
|
||||
- name: Set up Java
|
||||
uses: actions/setup-java@v5.2.0
|
||||
with:
|
||||
distribution: temurin
|
||||
java-version: 25
|
||||
- name: Set up Babashka
|
||||
uses: turtlequeue/setup-babashka@d78ec6570aea3b614bc695cafcceb82eb45c2af9 # v1.8.0
|
||||
with:
|
||||
babashka-version: 1.12.197
|
||||
- name: Install igluctl
|
||||
run: |
|
||||
set -euo pipefail
|
||||
curl -fsSL -o "$RUNNER_TEMP/igluctl.zip" https://github.com/snowplow/igluctl/releases/download/0.13.0/igluctl_0.13.0.zip
|
||||
echo "0eac19121fca1431dec44738ada8eacc4242811a95d015ee71e00deec2b4d748 $RUNNER_TEMP/igluctl.zip" | sha256sum -c -
|
||||
unzip -q "$RUNNER_TEMP/igluctl.zip" -d "$RUNNER_TEMP/igluctl"
|
||||
chmod +x "$RUNNER_TEMP/igluctl/igluctl"
|
||||
echo "$RUNNER_TEMP/igluctl" >> "$GITHUB_PATH"
|
||||
- name: Lint (igluctl)
|
||||
# igluctl recursively parses every file as JSON, so it chokes on colocated
|
||||
# *.md docs and on two schemas whose internal `self.version` doesn't match
|
||||
# the filename version (pending Snowcat audit). Stage a filtered copy and
|
||||
# patch only the `self.version` mismatch so igluctl still lints the rest
|
||||
# of those schemas.
|
||||
run: |
|
||||
set -euo pipefail
|
||||
LINT_ROOT="$RUNNER_TEMP/iglu-lint"
|
||||
rm -rf "$LINT_ROOT"
|
||||
mkdir -p "$LINT_ROOT"
|
||||
cp -r snowplow/iglu-client-embedded/schemas "$LINT_ROOT/"
|
||||
find "$LINT_ROOT" -name '*.md' -delete
|
||||
# TODO(BOT-1403): drop these patches once the Snowcat audit fixes the
|
||||
# internal `self.version` mismatch on these two schemas. We rewrite
|
||||
# only `self.version` in the staged copy so the file's path version
|
||||
# matches what igluctl expects; every other lint rule still runs.
|
||||
# Each `sed` invocation uses the schema's wrong/right pair, so it
|
||||
# silently no-ops once the underlying schema is fixed.
|
||||
sed -i 's/"version": "1-0-1"/"version": "1-0-2"/' \
|
||||
"$LINT_ROOT/schemas/com.metabase/embed_share/jsonschema/1-0-2"
|
||||
sed -i 's/"version": "1-0-0"/"version": "1-0-1"/' \
|
||||
"$LINT_ROOT/schemas/com.metabase/serialization/jsonschema/1-0-1"
|
||||
igluctl lint "$LINT_ROOT/schemas"
|
||||
- name: Lint (Metabase schema rules)
|
||||
run: ./bin/lint-snowplow-schemas.bb
|
||||
- name: Test linter rules
|
||||
run: ./bin/lint-snowplow-schemas-test.bb
|
||||
48
.github/workflows/loki-stress-test-flake-fix.yml
vendored
48
.github/workflows/loki-stress-test-flake-fix.yml
vendored
@@ -53,8 +53,8 @@ jobs:
|
||||
if ! validate_paths "$FILES"; then
|
||||
exit 1
|
||||
fi
|
||||
echo "story_files=$FILES" >> "$GITHUB_OUTPUT"
|
||||
echo "should_run=true" >> "$GITHUB_OUTPUT"
|
||||
echo "story_files=$FILES" >> $GITHUB_OUTPUT
|
||||
echo "should_run=true" >> $GITHUB_OUTPUT
|
||||
exit 0
|
||||
fi
|
||||
|
||||
@@ -70,7 +70,7 @@ jobs:
|
||||
|
||||
if [[ -z "$FILES" ]]; then
|
||||
echo "No changed story files found"
|
||||
echo "should_run=false" >> "$GITHUB_OUTPUT"
|
||||
echo "should_run=false" >> $GITHUB_OUTPUT
|
||||
exit 0
|
||||
fi
|
||||
|
||||
@@ -79,8 +79,8 @@ jobs:
|
||||
fi
|
||||
|
||||
echo "Changed story files: $FILES"
|
||||
echo "story_files=$FILES" >> "$GITHUB_OUTPUT"
|
||||
echo "should_run=true" >> "$GITHUB_OUTPUT"
|
||||
echo "story_files=$FILES" >> $GITHUB_OUTPUT
|
||||
echo "should_run=true" >> $GITHUB_OUTPUT
|
||||
|
||||
workflow-summary:
|
||||
name: Summary
|
||||
@@ -99,24 +99,22 @@ jobs:
|
||||
PR_ACTOR: ${{ github.event.pull_request.user.login }}
|
||||
ACTOR: ${{ github.actor }}
|
||||
run: |
|
||||
{
|
||||
echo '## Loki Stress Test'
|
||||
echo ''
|
||||
echo "**Branch:** \`${BRANCH_NAME}\`"
|
||||
echo ''
|
||||
echo '**Story files:**'
|
||||
echo "${STORY_FILES}" | tr ',' '\n' | while read -r file; do
|
||||
echo "- \`$file\`"
|
||||
done
|
||||
echo ''
|
||||
echo "**Burn-in:** ${BURN_IN} runs"
|
||||
echo ''
|
||||
if [[ "${EVENT_NAME}" == "pull_request" ]]; then
|
||||
echo "Triggered by PR #${PR_NUMBER} from @${PR_ACTOR}"
|
||||
else
|
||||
echo "Triggered manually by @${ACTOR}"
|
||||
fi
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
echo '## Loki Stress Test' >> $GITHUB_STEP_SUMMARY
|
||||
echo '' >> $GITHUB_STEP_SUMMARY
|
||||
echo "**Branch:** \`${BRANCH_NAME}\`" >> $GITHUB_STEP_SUMMARY
|
||||
echo '' >> $GITHUB_STEP_SUMMARY
|
||||
echo '**Story files:**' >> $GITHUB_STEP_SUMMARY
|
||||
echo "${STORY_FILES}" | tr ',' '\n' | while read -r file; do
|
||||
echo "- \`$file\`" >> $GITHUB_STEP_SUMMARY
|
||||
done
|
||||
echo '' >> $GITHUB_STEP_SUMMARY
|
||||
echo "**Burn-in:** ${BURN_IN} runs" >> $GITHUB_STEP_SUMMARY
|
||||
echo '' >> $GITHUB_STEP_SUMMARY
|
||||
if [[ "${EVENT_NAME}" == "pull_request" ]]; then
|
||||
echo "Triggered by PR #${PR_NUMBER} from @${PR_ACTOR}" >> $GITHUB_STEP_SUMMARY
|
||||
else
|
||||
echo "Triggered manually by @${ACTOR}" >> $GITHUB_STEP_SUMMARY
|
||||
fi
|
||||
|
||||
stress-test-loki:
|
||||
needs: detect-changed-stories
|
||||
@@ -177,8 +175,8 @@ jobs:
|
||||
echo "❌ Run $i FAILED"
|
||||
FAILED=$((FAILED + 1))
|
||||
|
||||
mkdir -p ".loki/failures/run-$i"
|
||||
cp -r .loki/difference/* ".loki/failures/run-$i/" 2>/dev/null || true
|
||||
mkdir -p .loki/failures/run-$i
|
||||
cp -r .loki/difference/* .loki/failures/run-$i/ 2>/dev/null || true
|
||||
fi
|
||||
echo ""
|
||||
done
|
||||
|
||||
4
.github/workflows/loki-update.yml
vendored
4
.github/workflows/loki-update.yml
vendored
@@ -31,9 +31,9 @@ jobs:
|
||||
id: check-changes
|
||||
run: |
|
||||
if [[ -n "$(git status --porcelain .loki/reference/)" ]]; then
|
||||
echo "has_changes=true" >> "$GITHUB_OUTPUT"
|
||||
echo "has_changes=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "has_changes=false" >> "$GITHUB_OUTPUT"
|
||||
echo "has_changes=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Commit and Push Changes
|
||||
|
||||
37
.github/workflows/main.yml
vendored
Normal file
37
.github/workflows/main.yml
vendored
Normal file
@@ -0,0 +1,37 @@
|
||||
name: Slack Notification for New Feature Requests
|
||||
|
||||
on:
|
||||
issues:
|
||||
types:
|
||||
- labeled
|
||||
jobs:
|
||||
on-issue-labeled:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
if: |
|
||||
contains(github.event.issue.labels.*.name, 'Type:New Feature')
|
||||
steps:
|
||||
- name: Setting title
|
||||
uses: actions/github-script@v9
|
||||
id: vars
|
||||
with:
|
||||
script: |
|
||||
core.setOutput('issue_title', ${{ toJson(github.event.issue.title) }}.replaceAll(/"/g, '\\"'));
|
||||
- name: Send the issue ${{ github.event.issue.number }} to Slack
|
||||
id: slack
|
||||
uses: slackapi/slack-github-action@45a88b9581bfab2566dc881e2cd66d334e621e2c # v3.0.3
|
||||
with:
|
||||
webhook: ${{ secrets.SLACK_FEATURE_REQUEST_TRIAGE_WEBHOOK_URL }}
|
||||
webhook-type: incoming-webhook
|
||||
payload: |
|
||||
{
|
||||
"blocks": [
|
||||
{
|
||||
"type": "section",
|
||||
"text": {
|
||||
"type": "mrkdwn",
|
||||
"text": "New incoming feature request: ${{ steps.vars.outputs.issue_title }}\n${{ github.event.issue.html_url }}"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
2
.github/workflows/mbql-library-changelog.yml
vendored
2
.github/workflows/mbql-library-changelog.yml
vendored
@@ -27,6 +27,6 @@ jobs:
|
||||
else
|
||||
echo "metabase.lib.js namespace changed but docs/developers/guide/mbql-library-changelog.md did not"
|
||||
echo "Update the changelog with the changes made to the library's API."
|
||||
echo "If the edits did not change the API, add the \`.metabase-lib/no-change\` label."
|
||||
echo "If the edits did not change the API, add the `.metabase-lib/no-change` label."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
31
.github/workflows/minimal-tests.yml
vendored
31
.github/workflows/minimal-tests.yml
vendored
@@ -9,6 +9,7 @@ concurrency:
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
id-token: write
|
||||
contents: read
|
||||
checks: write
|
||||
actions: read
|
||||
@@ -69,6 +70,16 @@ jobs:
|
||||
:exclude-tags '[:mb/driver-tests :mb/transforms-python-test]'
|
||||
:partition/total 2
|
||||
:partition/index 0
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
|
||||
# Postgres driver tests
|
||||
postgres-driver-test:
|
||||
@@ -106,6 +117,16 @@ jobs:
|
||||
test-args: >-
|
||||
:only-tags [:mb/driver-tests]
|
||||
:exclude-tags [:mb/transforms-python-test]
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
|
||||
# Frontend unit tests
|
||||
frontend-unit-tests:
|
||||
@@ -149,6 +170,16 @@ jobs:
|
||||
adapter: frontend
|
||||
test-suite: fe-tests-unit
|
||||
enforce-quarantine: ${{ vars.QUARANTINE_ENGINE == 'ci-conductor' }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit
|
||||
output-name: frontend-unit
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.run-tests.outcome }}
|
||||
|
||||
# Summary job
|
||||
minimal-tests-result:
|
||||
|
||||
208
.github/workflows/notify.issue-labeled.yml
vendored
208
.github/workflows/notify.issue-labeled.yml
vendored
@@ -1,208 +0,0 @@
|
||||
# Send Slack and/or PagerDuty notifications when labels are added to a Github issue.
|
||||
name: Notify on Issue Labeled
|
||||
|
||||
on:
|
||||
issues:
|
||||
types:
|
||||
- labeled
|
||||
|
||||
jobs:
|
||||
on-escalation-label:
|
||||
if: github.event.label.name == '.Escalation'
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- name: Format issue title (escape quotes)
|
||||
uses: actions/github-script@v9
|
||||
id: vars
|
||||
env:
|
||||
ISSUE_TITLE: ${{ github.event.issue.title }}
|
||||
with:
|
||||
script: |
|
||||
core.setOutput('issue_title', process.env.ISSUE_TITLE.replaceAll(/"/g, '\\"'));
|
||||
- name: Send escalated issue ${{ github.event.issue.number }} to Slack
|
||||
id: slack
|
||||
uses: slackapi/slack-github-action@45a88b9581bfab2566dc881e2cd66d334e621e2c # v3.0.3
|
||||
with:
|
||||
webhook: ${{ secrets.SLACK_ESCALATIONS_WEBHOOK_URL }}
|
||||
webhook-type: incoming-webhook
|
||||
payload: |
|
||||
{
|
||||
"blocks": [
|
||||
{
|
||||
"type": "section",
|
||||
"text": {
|
||||
"type": "mrkdwn",
|
||||
"text": "🔥🔥🔥 ${{ steps.vars.outputs.issue_title }}\n${{ github.event.issue.html_url }}"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
on-security-label:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
if: github.event.label.name == 'SECURITY'
|
||||
steps:
|
||||
- name: Setting title
|
||||
uses: actions/github-script@v9
|
||||
id: vars
|
||||
env:
|
||||
ISSUE_TITLE: ${{ github.event.issue.title }}
|
||||
with:
|
||||
script: |
|
||||
core.setOutput('issue_title', process.env.ISSUE_TITLE.replaceAll(/"/g, '\\"'));
|
||||
- name: Send the issue ${{ github.event.issue.number }} to Slack
|
||||
id: slack
|
||||
uses: slackapi/slack-github-action@45a88b9581bfab2566dc881e2cd66d334e621e2c # v3.0.3
|
||||
with:
|
||||
webhook: ${{ secrets.SLACK_SECURITY_WEBHOOK_URL }}
|
||||
webhook-type: incoming-webhook
|
||||
payload: |
|
||||
{
|
||||
"blocks": [
|
||||
{
|
||||
"type": "section",
|
||||
"text": {
|
||||
"type": "mrkdwn",
|
||||
"text": "${{ steps.vars.outputs.issue_title }}\n${{ github.event.issue.html_url }}"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
# Note: this only triggers if a issue is already labeled with `Type:Bug` and `flaky-test-fix` and then a team label is added.
|
||||
on-team-label:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
if: |
|
||||
contains(github.event.label.name, '.Team/') &&
|
||||
( contains(github.event.issue.labels.*.name, 'Type:Bug') ||
|
||||
contains(github.event.issue.labels.*.name, 'flaky-test-fix') )
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
sparse-checkout: .github/scripts
|
||||
|
||||
- name: Build Slack message (with Linear link if available)
|
||||
uses: actions/github-script@v9
|
||||
id: vars
|
||||
env:
|
||||
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
|
||||
with:
|
||||
script: |
|
||||
const { findLinearIssue } = require('./.github/scripts/find-linear-issue.js');
|
||||
|
||||
const title = context.payload.issue.title;
|
||||
const issueUrl = context.payload.issue.html_url;
|
||||
|
||||
let text = title + '\n' + issueUrl;
|
||||
|
||||
// The Linear issue is usually already synced by the time the
|
||||
// team label lands at triage, but retry briefly just in case.
|
||||
try {
|
||||
const delaysSeconds = [0, 15, 30];
|
||||
let result = null;
|
||||
|
||||
for (const delay of delaysSeconds) {
|
||||
if (delay) await new Promise((r) => setTimeout(r, delay * 1000));
|
||||
result = await findLinearIssue({
|
||||
githubIssueUrl: issueUrl,
|
||||
linearApiKey: process.env.LINEAR_API_KEY,
|
||||
});
|
||||
if (result.success) break;
|
||||
}
|
||||
|
||||
if (result && result.success) {
|
||||
const id = result.issue; // e.g. "GIT-1234"
|
||||
text += `\nLinear: <https://linear.app/metabase/issue/${id}|${id}>`;
|
||||
} else {
|
||||
core.info(`No Linear issue found for ${issueUrl}; posting without link.`);
|
||||
}
|
||||
} catch (e) {
|
||||
// Never block the notification on a Linear lookup failure
|
||||
core.warning(`Linear lookup failed: ${e.message}`);
|
||||
}
|
||||
|
||||
core.setOutput('message', JSON.stringify(text));
|
||||
|
||||
- name: Extracting team name
|
||||
id: team_name
|
||||
env:
|
||||
LABEL_NAME: ${{ github.event.label.name }}
|
||||
run: |
|
||||
team_label="${LABEL_NAME//&/$}"
|
||||
echo "$team_label"
|
||||
team_name=${team_label#*.Team/}
|
||||
echo "$team_name"
|
||||
echo "team_name=${team_name^^}_SLACK_ISSUES_WEBHOOK_URL" >> "$GITHUB_OUTPUT"
|
||||
- name: Check webhook secret exists
|
||||
id: check_secret
|
||||
env:
|
||||
WEBHOOK_URL: ${{ secrets[steps.team_name.outputs.team_name] }}
|
||||
run: |
|
||||
if [ -z "$WEBHOOK_URL" ]; then
|
||||
echo "Secret ${{ steps.team_name.outputs.team_name }} is not set, skipping Slack notification"
|
||||
echo "exists=false" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "exists=true" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
- name: Send the issue ${{ github.event.issue.number }} to Slack
|
||||
if: steps.check_secret.outputs.exists == 'true'
|
||||
id: slack
|
||||
uses: slackapi/slack-github-action@45a88b9581bfab2566dc881e2cd66d334e621e2c # v3.0.3
|
||||
with:
|
||||
webhook: ${{ secrets[steps.team_name.outputs.team_name] }}
|
||||
webhook-type: incoming-webhook
|
||||
payload: |
|
||||
{
|
||||
"blocks": [
|
||||
{
|
||||
"type": "section",
|
||||
"text": {
|
||||
"type": "mrkdwn",
|
||||
"text": ${{ steps.vars.outputs.message }}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
on-escalation-and-team-querying-label:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
if: |
|
||||
(github.event.label.name == 'Priority:P0' || github.event.label.name == 'Priority:P1' || github.event.label.name == '.Escalation') &&
|
||||
contains(github.event.issue.labels.*.name, '.Team/Querying')
|
||||
steps:
|
||||
- name: Trigger PagerDuty Incident
|
||||
env:
|
||||
ROUTING_KEY: ${{ secrets.QUERYING_PAGERDUTY_INTEGRATION_KEY }}
|
||||
ISSUE_TITLE: ${{ github.event.issue.title }}
|
||||
LABEL_NAME: ${{ github.event.label.name }}
|
||||
ISSUE_NUMBER: ${{ github.event.issue.number }}
|
||||
ISSUE_URL: ${{ github.event.issue.html_url }}
|
||||
REPO: ${{ github.repository }}
|
||||
ALL_LABELS: ${{ join(github.event.issue.labels.*.name, ', ') }}
|
||||
run: |
|
||||
payload=$(jq -n \
|
||||
--arg rk "$ROUTING_KEY" \
|
||||
--arg dk "github-issue-$ISSUE_NUMBER" \
|
||||
--arg sum "[$LABEL_NAME] $ISSUE_TITLE" \
|
||||
--arg src "GitHub Issues - $REPO" \
|
||||
--arg lbl "$ALL_LABELS" \
|
||||
--arg url "$ISSUE_URL" \
|
||||
'{
|
||||
routing_key: $rk,
|
||||
event_action: "trigger",
|
||||
dedup_key: $dk,
|
||||
payload: {
|
||||
summary: $sum,
|
||||
source: $src,
|
||||
severity: "critical",
|
||||
custom_details: { Labels: $lbl }
|
||||
},
|
||||
links: [{ href: $url, text: "View GitHub Issue" }]
|
||||
}')
|
||||
curl -X POST https://events.pagerduty.com/v2/enqueue \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d "$payload"
|
||||
2
.github/workflows/openapi-check.yml
vendored
2
.github/workflows/openapi-check.yml
vendored
@@ -31,7 +31,7 @@ jobs:
|
||||
- name: Get latest base branch commit
|
||||
run: |
|
||||
git fetch origin ${{ github.event.pull_request.base.ref }}
|
||||
echo "LATEST_BASE_SHA=$(git rev-parse origin/${{ github.event.pull_request.base.ref }})" >> "$GITHUB_ENV"
|
||||
echo "LATEST_BASE_SHA=$(git rev-parse origin/${{ github.event.pull_request.base.ref }})" >> $GITHUB_ENV
|
||||
- name: Check OpenAPI specs are up to date
|
||||
env:
|
||||
AUTO_COMMIT: "true"
|
||||
|
||||
44
.github/workflows/openapi.yml
vendored
Normal file
44
.github/workflows/openapi.yml
vendored
Normal file
@@ -0,0 +1,44 @@
|
||||
name: Validate OpenAPI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'release-**'
|
||||
pull_request:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
files-changed:
|
||||
name: Check which files changed
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 3
|
||||
outputs:
|
||||
openapi: ${{ steps.changes.outputs.openapi }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Test which files changed
|
||||
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
|
||||
id: changes
|
||||
with:
|
||||
token: ${{ github.token }}
|
||||
filters: .github/file-paths.yaml
|
||||
|
||||
validate-openapi:
|
||||
needs: files-changed
|
||||
if: needs.files-changed.outputs.openapi == 'true'
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Prepare front-end environment
|
||||
uses: ./.github/actions/prepare-frontend
|
||||
- name: Prepare back-end environment
|
||||
uses: ./.github/actions/prepare-backend
|
||||
- name: Generate OpenAPI spec
|
||||
run: bun run generate-openapi
|
||||
- name: Validate OpenAPI spec
|
||||
run: bun run lint-openapi
|
||||
10
.github/workflows/optimize-images.yml
vendored
10
.github/workflows/optimize-images.yml
vendored
@@ -63,12 +63,10 @@ jobs:
|
||||
total_saved=$((total_before - total_after))
|
||||
|
||||
# Output results for use in commit message
|
||||
{
|
||||
echo "files_optimized=${total_files}"
|
||||
echo "bytes_saved=${total_saved}"
|
||||
echo "total_before=${total_before}"
|
||||
echo "total_after=${total_after}"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
echo "files_optimized=${total_files}" >> "$GITHUB_OUTPUT"
|
||||
echo "bytes_saved=${total_saved}" >> "$GITHUB_OUTPUT"
|
||||
echo "total_before=${total_before}" >> "$GITHUB_OUTPUT"
|
||||
echo "total_after=${total_after}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
echo "Optimization complete"
|
||||
echo "Files processed: ${total_files}"
|
||||
|
||||
4
.github/workflows/perf-test.yml
vendored
4
.github/workflows/perf-test.yml
vendored
@@ -70,7 +70,7 @@ jobs:
|
||||
export OIDC_URL_WITH_AUDIENCE="$ACTIONS_ID_TOKEN_REQUEST_URL&audience=${PR_ENV_K8S_AUDIENCE}"
|
||||
IDTOKEN=$(curl -H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" -H "Accept: application/json; api-version=2.0" "$OIDC_URL_WITH_AUDIENCE" | jq -r .value)
|
||||
echo "::add-mask::${IDTOKEN}"
|
||||
echo "idToken=${IDTOKEN}" >>"$GITHUB_OUTPUT"
|
||||
echo "idToken=${IDTOKEN}" >>$GITHUB_OUTPUT
|
||||
env:
|
||||
PR_ENV_K8S_AUDIENCE: ${{ secrets.PR_ENV_K8S_AUDIENCE }}
|
||||
- name: Setup Kube Context
|
||||
@@ -108,7 +108,7 @@ jobs:
|
||||
env:
|
||||
SHA: ${{ github.event.pull_request.head.sha || github.sha }}
|
||||
id: split
|
||||
run: echo "fragment=${SHA:5}" >> "$GITHUB_OUTPUT"
|
||||
run: echo "fragment=${SHA:5}" >> $GITHUB_OUTPUT
|
||||
- name: Render Deployment YAML
|
||||
uses: nowactions/envsubst@c4b8e3977354d294659a7ec164279ce8fba5c2e1 # v1.0.1
|
||||
with:
|
||||
|
||||
5
.github/workflows/pr-env-destroy-manual.yml
vendored
5
.github/workflows/pr-env-destroy-manual.yml
vendored
@@ -18,10 +18,7 @@ jobs:
|
||||
environment: pr-env-admin
|
||||
steps:
|
||||
- name: Approved
|
||||
env:
|
||||
PR_NUMBER: ${{ inputs.pr_number }}
|
||||
run: |
|
||||
echo "Destroying PR environment for PR #${PR_NUMBER}"
|
||||
run: echo "Destroying PR environment for PR #${{ inputs.pr_number }}"
|
||||
|
||||
destroy:
|
||||
needs: approve
|
||||
|
||||
2
.github/workflows/pr-env-destroy.yml
vendored
2
.github/workflows/pr-env-destroy.yml
vendored
@@ -42,7 +42,7 @@ jobs:
|
||||
export OIDC_URL_WITH_AUDIENCE="$ACTIONS_ID_TOKEN_REQUEST_URL&audience=${PR_ENV_K8S_AUDIENCE}"
|
||||
IDTOKEN=$(curl -H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" -H "Accept: application/json; api-version=2.0" "$OIDC_URL_WITH_AUDIENCE" | jq -r .value)
|
||||
echo "::add-mask::${IDTOKEN}"
|
||||
echo "idToken=${IDTOKEN}" >>"$GITHUB_OUTPUT"
|
||||
echo "idToken=${IDTOKEN}" >>$GITHUB_OUTPUT
|
||||
env:
|
||||
PR_ENV_K8S_AUDIENCE: ${{ secrets.PR_ENV_K8S_AUDIENCE }}
|
||||
- name: Setup Kube Context
|
||||
|
||||
2
.github/workflows/pr-env.yml
vendored
2
.github/workflows/pr-env.yml
vendored
@@ -249,7 +249,7 @@ jobs:
|
||||
export OIDC_URL_WITH_AUDIENCE="$ACTIONS_ID_TOKEN_REQUEST_URL&audience=${PR_ENV_K8S_AUDIENCE}"
|
||||
IDTOKEN=$(curl -H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" -H "Accept: application/json; api-version=2.0" "$OIDC_URL_WITH_AUDIENCE" | jq -r .value)
|
||||
echo "::add-mask::${IDTOKEN}"
|
||||
echo "idToken=${IDTOKEN}" >>"$GITHUB_OUTPUT"
|
||||
echo "idToken=${IDTOKEN}" >>$GITHUB_OUTPUT
|
||||
env:
|
||||
PR_ENV_K8S_AUDIENCE: ${{ secrets.PR_ENV_K8S_AUDIENCE }}
|
||||
- name: Setup Kube Context
|
||||
|
||||
14
.github/workflows/pre-release.yml
vendored
14
.github/workflows/pre-release.yml
vendored
@@ -95,14 +95,14 @@ jobs:
|
||||
id: version-properties
|
||||
run: |
|
||||
cat version.properties
|
||||
echo "commit=$(cat ./COMMIT-ID)" >> "$GITHUB_OUTPUT"
|
||||
echo "commit=$(cat ./COMMIT-ID)" >> $GITHUB_OUTPUT
|
||||
|
||||
version=$(grep -o '^tag=.*' version.properties | cut -d'=' -f2)
|
||||
|
||||
if [[ "${{ matrix.edition }}" == "ee" ]]; then
|
||||
echo "ee_version=$version" >> "$GITHUB_OUTPUT"
|
||||
echo "ee_version=$version" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "oss_version=$version" >> "$GITHUB_OUTPUT"
|
||||
echo "oss_version=$version" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
shell: bash
|
||||
- name: Upload Metabase ${{ matrix.edition }} JAR as artifact
|
||||
@@ -223,7 +223,7 @@ jobs:
|
||||
CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
|
||||
run: |
|
||||
node e2e/runner/run_cypress_ci.js e2e \
|
||||
"$CYPRESS_GREP_FLAG" grepTags="@prerelease+-@EE" \
|
||||
$CYPRESS_GREP_FLAG grepTags="@prerelease+-@EE" \
|
||||
--spec './e2e/test/scenarios/**/*.cy.spec.(js|ts)'
|
||||
|
||||
- name: Run a few important EE Cypress tests as sanity check
|
||||
@@ -232,7 +232,7 @@ jobs:
|
||||
CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
|
||||
run: |
|
||||
node e2e/runner/run_cypress_ci.js e2e \
|
||||
"$CYPRESS_GREP_FLAG" grepTags="@prerelease+-@OSS" \
|
||||
$CYPRESS_GREP_FLAG grepTags="@prerelease+-@OSS" \
|
||||
--spec './e2e/test/scenarios/**/*.cy.spec.(js|ts)'
|
||||
|
||||
- name: Upload Cypress Artifacts upon failure
|
||||
@@ -254,7 +254,7 @@ jobs:
|
||||
commit: ${{ inputs.commit }}
|
||||
repo: ${{ vars.DOCKERHUB_OWNER }}/${{ vars.DOCKERHUB_STAGING_REPO }}
|
||||
tag: ${{ needs.release-artifact.outputs.oss_version }}-${{ inputs.commit }}
|
||||
release: true
|
||||
release: "true"
|
||||
|
||||
containerize-ee:
|
||||
needs: release-artifact
|
||||
@@ -265,7 +265,7 @@ jobs:
|
||||
commit: ${{ inputs.commit }}
|
||||
repo: ${{ vars.DOCKERHUB_OWNER }}/${{ vars.DOCKERHUB_STAGING_REPO }}
|
||||
tag: ${{ needs.release-artifact.outputs.ee_version }}-${{ inputs.commit }}
|
||||
release: true
|
||||
release: "true"
|
||||
|
||||
tests-complete-message:
|
||||
if: always()
|
||||
|
||||
109
.github/workflows/presto-kerberos-integration-test.yml
vendored
Normal file
109
.github/workflows/presto-kerberos-integration-test.yml
vendored
Normal file
@@ -0,0 +1,109 @@
|
||||
# this file uses MBA https://github.com/metabase/mba to spin up a Metabase
|
||||
# instance then runs the test script in the MBA instance sharing the same
|
||||
# network to interact with presto container
|
||||
# TODO: get rid of MBA and use actions/prepare-backend instead
|
||||
|
||||
name: Kerberized Presto Integration Test
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
- "release**"
|
||||
- "feature**"
|
||||
tags:
|
||||
- "**"
|
||||
|
||||
jobs:
|
||||
files-changed:
|
||||
name: Check which files changed
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 3
|
||||
outputs:
|
||||
backend_presto_kerberos: ${{ steps.changes.outputs.backend_presto_kerberos }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Test which files changed
|
||||
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
|
||||
id: changes
|
||||
with:
|
||||
token: ${{ github.token }}
|
||||
filters: .github/file-paths.yaml
|
||||
|
||||
run-presto-kerberos-test:
|
||||
needs: files-changed
|
||||
if: needs.files-changed.outputs.backend_presto_kerberos == 'true'
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 40
|
||||
steps:
|
||||
- name: Install babashka
|
||||
run: >
|
||||
mkdir -p /tmp/babashka-install \
|
||||
&& cd /tmp/babashka-install \
|
||||
&& curl -sLO https://raw.githubusercontent.com/babashka/babashka/master/install \
|
||||
&& chmod +x install \
|
||||
&& sudo ./install \
|
||||
&& cd -
|
||||
- name: Checkout Metabase repository
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
- name: Check out Presto Kerberos Docker Compose
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
repository: metabase/presto-kerberos-docker
|
||||
ref: master
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
path: presto-kerberos-docker
|
||||
- name: Bring up Presto+Kerberos cluster
|
||||
run: cd presto-kerberos-docker && docker compose up -d && cd ..
|
||||
- name: Run Presto test query from command line (sanity check)
|
||||
run: cd presto-kerberos-docker && ./test.sh && cd ..
|
||||
# Since we are managing the Docker containers from the GitHub action container, we need to copy all the
|
||||
# relevant resources now, into the resources dir for later consumption by the app
|
||||
- name: Copy Presto SSL keystore to resources
|
||||
run: docker cp presto-kerberos:/tmp/ssl_keystore.jks resources
|
||||
- name: Copy krb5.conf file to resources
|
||||
run: docker cp presto-kerberos:/etc/krb5.conf resources
|
||||
- name: Copy client.keytab file to resources
|
||||
run: docker cp presto-kerberos:/home/presto/client.keytab resources
|
||||
|
||||
- name: Checkout mba
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
repository: metabase/mba
|
||||
ref: master
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
path: mba-src
|
||||
- name: ls mba
|
||||
run: ls -latr mba-src
|
||||
- name: Symlink mba
|
||||
run: cd mba-src && sudo ln -s $(pwd)/src/main.clj /usr/local/bin/mba && chmod +x /usr/local/bin/mba && cd ..
|
||||
|
||||
- name: Ensure mba
|
||||
run: which mba
|
||||
|
||||
- name: Run Metabase via MBA
|
||||
run: mba --mb . --data-db postgres-data -n example.com compose config > mba-compose.yml
|
||||
|
||||
- name: Adjust generated docker-compose config
|
||||
uses: actions/github-script@v9
|
||||
with:
|
||||
script: | #js
|
||||
const fs = require("fs");
|
||||
const config = fs.readFileSync("mba-compose.yml", "utf8");
|
||||
const newConfig = config.replace(
|
||||
/name:\s+example.com/,
|
||||
"name: example.com\n external: true",
|
||||
);
|
||||
fs.writeFileSync("mba-compose.yml", newConfig);
|
||||
|
||||
- name: Print compose config
|
||||
run: cat mba-compose.yml
|
||||
|
||||
- name: Run mba
|
||||
run: docker compose -f mba-compose.yml up -d
|
||||
|
||||
- name: Run test script in MBA instance
|
||||
run: docker compose -f mba-compose.yml exec metabase .github/scripts/run-presto-kerberos-integration-test.sh
|
||||
47
.github/workflows/querying-team-pagerduty.yml
vendored
Normal file
47
.github/workflows/querying-team-pagerduty.yml
vendored
Normal file
@@ -0,0 +1,47 @@
|
||||
name: Querying On-call PagerDuty Alert
|
||||
|
||||
on:
|
||||
issues:
|
||||
types:
|
||||
- labeled
|
||||
|
||||
jobs:
|
||||
notify-querying-on-call:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
if: |
|
||||
(github.event.label.name == 'Priority:P0' || github.event.label.name == 'Priority:P1' || github.event.label.name == '.Escalation') &&
|
||||
contains(github.event.issue.labels.*.name, '.Team/Querying')
|
||||
steps:
|
||||
- name: Trigger PagerDuty Incident
|
||||
env:
|
||||
ROUTING_KEY: ${{ secrets.QUERYING_PAGERDUTY_INTEGRATION_KEY }}
|
||||
ISSUE_TITLE: ${{ github.event.issue.title }}
|
||||
LABEL_NAME: ${{ github.event.label.name }}
|
||||
ISSUE_NUMBER: ${{ github.event.issue.number }}
|
||||
ISSUE_URL: ${{ github.event.issue.html_url }}
|
||||
REPO: ${{ github.repository }}
|
||||
ALL_LABELS: ${{ join(github.event.issue.labels.*.name, ', ') }}
|
||||
run: |
|
||||
payload=$(jq -n \
|
||||
--arg rk "$ROUTING_KEY" \
|
||||
--arg dk "github-issue-$ISSUE_NUMBER" \
|
||||
--arg sum "[$LABEL_NAME] $ISSUE_TITLE" \
|
||||
--arg src "GitHub Issues - $REPO" \
|
||||
--arg lbl "$ALL_LABELS" \
|
||||
--arg url "$ISSUE_URL" \
|
||||
'{
|
||||
routing_key: $rk,
|
||||
event_action: "trigger",
|
||||
dedup_key: $dk,
|
||||
payload: {
|
||||
summary: $sum,
|
||||
source: $src,
|
||||
severity: "critical",
|
||||
custom_details: { Labels: $lbl }
|
||||
},
|
||||
links: [{ href: $url, text: "View GitHub Issue" }]
|
||||
}')
|
||||
curl -X POST https://events.pagerduty.com/v2/enqueue \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d "$payload"
|
||||
6
.github/workflows/release-log.yml
vendored
6
.github/workflows/release-log.yml
vendored
@@ -71,12 +71,12 @@ jobs:
|
||||
- name: generate release Log
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: cd release && tsx ./src/release-log-run.ts "$VERSION" > "v$VERSION.html"
|
||||
run: cd release && tsx ./src/release-log-run.ts $VERSION > v$VERSION.html
|
||||
- name: upload release log to the web
|
||||
run: |
|
||||
aws s3 cp \
|
||||
"release/v$VERSION.html" \
|
||||
"s3://${{ vars.AWS_S3_STATIC_BUCKET }}/release-log/v$VERSION.html"
|
||||
release/v$VERSION.html \
|
||||
s3://${{ vars.AWS_S3_STATIC_BUCKET }}/release-log/v$VERSION.html
|
||||
|
||||
- name: Create cloudfront invalidation
|
||||
run: |
|
||||
|
||||
12
.github/workflows/release-tag.yml
vendored
12
.github/workflows/release-tag.yml
vendored
@@ -238,12 +238,12 @@ jobs:
|
||||
run: | # sh
|
||||
if [[ "${{ matrix.edition }}" == "ee" ]]; then
|
||||
echo "Metabase EE: image is going to be pushed to ${{ github.repository_owner }}/metabase-enterprise"
|
||||
echo "DOCKERHUB_REPO=${{ github.repository_owner }}/metabase-enterprise" >> "$GITHUB_ENV"
|
||||
echo "DOCKERHUB_VERSION=${{ needs.check-version.outputs.ee }}" >> "$GITHUB_ENV"
|
||||
echo "DOCKERHUB_REPO=${{ github.repository_owner }}/metabase-enterprise" >> $GITHUB_ENV
|
||||
echo "DOCKERHUB_VERSION=${{ needs.check-version.outputs.ee }}" >> $GITHUB_ENV
|
||||
else
|
||||
echo "Metabase OSS: image is going to be pushed to ${{ github.repository_owner }}/metabase"
|
||||
echo "DOCKERHUB_REPO=${{ github.repository_owner }}/metabase" >> "$GITHUB_ENV"
|
||||
echo "DOCKERHUB_VERSION=${{ needs.check-version.outputs.oss }}" >> "$GITHUB_ENV"
|
||||
echo "DOCKERHUB_REPO=${{ github.repository_owner }}/metabase" >> $GITHUB_ENV
|
||||
echo "DOCKERHUB_VERSION=${{ needs.check-version.outputs.oss }}" >> $GITHUB_ENV
|
||||
fi
|
||||
|
||||
- name: Login to Docker Hub
|
||||
@@ -255,8 +255,8 @@ jobs:
|
||||
- name: Tag the docker image
|
||||
run: | # sh
|
||||
docker buildx imagetools create --tag \
|
||||
"$DOCKERHUB_REPO:$TAG_NAME" \
|
||||
"$DOCKERHUB_REPO:$DOCKERHUB_VERSION"
|
||||
$DOCKERHUB_REPO\:$TAG_NAME \
|
||||
$DOCKERHUB_REPO\:$DOCKERHUB_VERSION
|
||||
|
||||
push-git-tags:
|
||||
permissions:
|
||||
|
||||
7
.github/workflows/release.yml
vendored
7
.github/workflows/release.yml
vendored
@@ -281,7 +281,7 @@ jobs:
|
||||
commit: ${{ inputs.commit }}
|
||||
repo: ${{ vars.DOCKERHUB_OWNER }}/${{ vars.DOCKERHUB_REPO }}
|
||||
tag: ${{ needs.check-version.outputs.oss }}
|
||||
release: true
|
||||
release: ${{ 'true' }}
|
||||
|
||||
containerize-ee:
|
||||
needs: check-version
|
||||
@@ -292,7 +292,7 @@ jobs:
|
||||
commit: ${{ inputs.commit }}
|
||||
repo: ${{ vars.DOCKERHUB_OWNER }}/${{ vars.DOCKERHUB_REPO }}-enterprise
|
||||
tag: ${{ needs.check-version.outputs.ee }}
|
||||
release: true
|
||||
release: ${{ 'true' }}
|
||||
|
||||
push-tags:
|
||||
needs: [verify-s3-download, containerize-oss, containerize-ee, check-version]
|
||||
@@ -381,8 +381,7 @@ jobs:
|
||||
dot-x-tag: ${{ startsWith(matrix.tag, 'v') }}
|
||||
|
||||
trigger-cloud-issues:
|
||||
# Only create issues in the cloud ops repository if this release happens in https://github.com/metabase/metabase.
|
||||
if: ${{ github.repository == 'metabase/metabase' && (needs.check-version.outputs.kind == 'minor' || needs.check-version.outputs.kind == 'major') }}
|
||||
if: ${{ needs.check-version.outputs.kind == 'minor' || needs.check-version.outputs.kind == 'major' }}
|
||||
needs: [push-tags, check-version]
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 10
|
||||
|
||||
6
.github/workflows/repro-bot.yml
vendored
6
.github/workflows/repro-bot.yml
vendored
@@ -37,7 +37,7 @@ jobs:
|
||||
else
|
||||
ISSUE_NUMBER="${{ github.event.issue.number }}"
|
||||
fi
|
||||
echo "number=$ISSUE_NUMBER" >> "$GITHUB_OUTPUT"
|
||||
echo "number=$ISSUE_NUMBER" >> $GITHUB_OUTPUT
|
||||
echo "Issue: https://github.com/${{ github.repository }}/issues/$ISSUE_NUMBER"
|
||||
|
||||
env:
|
||||
@@ -58,7 +58,7 @@ jobs:
|
||||
run: git clone --bare "https://github.com/${{ github.repository }}.git" repro-bot/git/metabase
|
||||
|
||||
- name: Capture Node.js path for MCP subprocesses
|
||||
run: echo "NODE_BIN_DIR=$(dirname "$(which node)")" >> "$GITHUB_ENV"
|
||||
run: echo "NODE_BIN_DIR=$(dirname $(which node))" >> $GITHUB_ENV
|
||||
|
||||
- name: Pre-install MCP server dependencies
|
||||
run: uv sync --directory repro-bot/mcp
|
||||
@@ -90,7 +90,7 @@ jobs:
|
||||
--main-opts '["-m" "clojure-mcp-light.nrepl-eval"]'
|
||||
|
||||
# Add to PATH for the rest of the workflow
|
||||
echo "$HOME/.bbin/bin" >> "$GITHUB_PATH"
|
||||
echo "$HOME/.bbin/bin" >> $GITHUB_PATH
|
||||
|
||||
# Verify
|
||||
clj-nrepl-eval --help | head -3
|
||||
|
||||
32
.github/workflows/resolve-backport-conflicts.yml
vendored
32
.github/workflows/resolve-backport-conflicts.yml
vendored
@@ -63,9 +63,9 @@ jobs:
|
||||
id: pkg
|
||||
run: |
|
||||
if [ -f "bun.lock" ] || [ -f "bun.lockb" ]; then
|
||||
echo "mgr=bun" >> "$GITHUB_OUTPUT"
|
||||
echo "mgr=bun" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "mgr=yarn" >> "$GITHUB_OUTPUT"
|
||||
echo "mgr=yarn" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Validate backport PR
|
||||
@@ -92,15 +92,13 @@ jobs:
|
||||
# Check if we're in a cherry-pick state with conflicts
|
||||
CONFLICTED=$(git diff --name-only --diff-filter=U || true)
|
||||
if [ -n "$CONFLICTED" ]; then
|
||||
{
|
||||
echo "has_conflicts=true"
|
||||
echo "conflicted_files<<EOF"
|
||||
echo "$CONFLICTED"
|
||||
echo "EOF"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
echo "has_conflicts=true" >> $GITHUB_OUTPUT
|
||||
echo "conflicted_files<<EOF" >> $GITHUB_OUTPUT
|
||||
echo "$CONFLICTED" >> $GITHUB_OUTPUT
|
||||
echo "EOF" >> $GITHUB_OUTPUT
|
||||
echo "::notice::Found conflicts in: $CONFLICTED"
|
||||
else
|
||||
echo "has_conflicts=false" >> "$GITHUB_OUTPUT"
|
||||
echo "has_conflicts=false" >> $GITHUB_OUTPUT
|
||||
echo "::notice::Cherry-pick applied cleanly, no conflicts to resolve"
|
||||
fi
|
||||
|
||||
@@ -210,25 +208,23 @@ jobs:
|
||||
if [ -f "${{ steps.claude.outputs.execution_file }}" ]; then
|
||||
SUBTYPE=$(jq -r '.[] | select(.type == "result") | .subtype' "${{ steps.claude.outputs.execution_file }}" | tail -1)
|
||||
if [ "$SUBTYPE" = "error_max_turns" ]; then
|
||||
echo "max_turns_exceeded=true" >> "$GITHUB_OUTPUT"
|
||||
echo "max_turns_exceeded=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "max_turns_exceeded=false" >> "$GITHUB_OUTPUT"
|
||||
echo "max_turns_exceeded=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
else
|
||||
echo "max_turns_exceeded=false" >> "$GITHUB_OUTPUT"
|
||||
echo "max_turns_exceeded=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Get commit details
|
||||
if: always() && steps.claude.outcome == 'success' && steps.check-failure.outputs.max_turns_exceeded != 'true'
|
||||
id: commit-info
|
||||
run: |
|
||||
echo "sha=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT
|
||||
FILES=$(git diff --name-only HEAD~1)
|
||||
{
|
||||
echo "sha=$(git rev-parse HEAD)"
|
||||
echo "changed_files<<EOF"
|
||||
echo "$FILES"
|
||||
echo "EOF"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
echo "changed_files<<EOF" >> $GITHUB_OUTPUT
|
||||
echo "$FILES" >> $GITHUB_OUTPUT
|
||||
echo "EOF" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Comment on PR (success)
|
||||
if: always() && steps.claude.outcome == 'success' && steps.check-failure.outputs.max_turns_exceeded != 'true'
|
||||
|
||||
37
.github/workflows/security-slack-notification.yml
vendored
Normal file
37
.github/workflows/security-slack-notification.yml
vendored
Normal file
@@ -0,0 +1,37 @@
|
||||
name: Security Issue Slack Notification
|
||||
|
||||
on:
|
||||
issues:
|
||||
types:
|
||||
- labeled
|
||||
|
||||
jobs:
|
||||
on-issue-opened:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
if: github.event.label.name == 'SECURITY'
|
||||
steps:
|
||||
- name: Setting title
|
||||
uses: actions/github-script@v9
|
||||
id: vars
|
||||
with:
|
||||
script: |
|
||||
core.setOutput('issue_title', ${{ toJson(github.event.issue.title) }}.replaceAll(/"/g, '\\"'));
|
||||
- name: Send the issue ${{ github.event.issue.number }} to Slack
|
||||
id: slack
|
||||
uses: slackapi/slack-github-action@45a88b9581bfab2566dc881e2cd66d334e621e2c # v3.0.3
|
||||
with:
|
||||
webhook: ${{ secrets.SLACK_SECURITY_WEBHOOK_URL }}
|
||||
webhook-type: incoming-webhook
|
||||
payload: |
|
||||
{
|
||||
"blocks": [
|
||||
{
|
||||
"type": "section",
|
||||
"text": {
|
||||
"type": "mrkdwn",
|
||||
"text": "${{ steps.vars.outputs.issue_title }}\n${{ github.event.issue.html_url }}"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
50
.github/workflows/semantic-search.yml
vendored
50
.github/workflows/semantic-search.yml
vendored
@@ -63,6 +63,16 @@ jobs:
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
|
||||
semantic-search-tests-mariadb:
|
||||
if: ${{ !inputs.skip }}
|
||||
@@ -129,6 +139,16 @@ jobs:
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
|
||||
semantic-search-tests-mysql:
|
||||
if: ${{ !inputs.skip }}
|
||||
@@ -202,6 +222,16 @@ jobs:
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
|
||||
semantic-search-tests-postgres:
|
||||
if: ${{ !inputs.skip }}
|
||||
@@ -267,6 +297,16 @@ jobs:
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
|
||||
# pgvector-on-the-app-db mode: the app db itself is a pgvector-capable Postgres and
|
||||
# MB_PGVECTOR_DB_URL is deliberately unset. The dedicated-harness tests gate themselves off; the
|
||||
@@ -322,6 +362,16 @@ jobs:
|
||||
${{ matrix.job.test-args }}
|
||||
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
|
||||
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
|
||||
|
||||
semantic-search-tests-result:
|
||||
needs:
|
||||
|
||||
75
.github/workflows/snowplow.yml
vendored
Normal file
75
.github/workflows/snowplow.yml
vendored
Normal file
@@ -0,0 +1,75 @@
|
||||
name: Snowplow
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches:
|
||||
- '**'
|
||||
|
||||
jobs:
|
||||
files-changed:
|
||||
name: Check which files changed
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 3
|
||||
outputs:
|
||||
snowplow: ${{ steps.changes.outputs.snowplow }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Test which files changed
|
||||
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
|
||||
id: changes
|
||||
with:
|
||||
token: ${{ github.token }}
|
||||
filters: .github/file-paths.yaml
|
||||
|
||||
lint:
|
||||
needs: files-changed
|
||||
if: needs.files-changed.outputs.snowplow == 'true'
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v6
|
||||
- name: Set up Java
|
||||
uses: actions/setup-java@v5.2.0
|
||||
with:
|
||||
distribution: temurin
|
||||
java-version: 25
|
||||
- name: Set up Babashka
|
||||
uses: turtlequeue/setup-babashka@d78ec6570aea3b614bc695cafcceb82eb45c2af9 # v1.8.0
|
||||
with:
|
||||
babashka-version: 1.12.197
|
||||
- name: Install igluctl
|
||||
run: |
|
||||
set -euo pipefail
|
||||
curl -fsSL -o "$RUNNER_TEMP/igluctl.zip" https://github.com/snowplow/igluctl/releases/download/0.13.0/igluctl_0.13.0.zip
|
||||
echo "0eac19121fca1431dec44738ada8eacc4242811a95d015ee71e00deec2b4d748 $RUNNER_TEMP/igluctl.zip" | sha256sum -c -
|
||||
unzip -q "$RUNNER_TEMP/igluctl.zip" -d "$RUNNER_TEMP/igluctl"
|
||||
chmod +x "$RUNNER_TEMP/igluctl/igluctl"
|
||||
echo "$RUNNER_TEMP/igluctl" >> "$GITHUB_PATH"
|
||||
- name: Lint (igluctl)
|
||||
# igluctl recursively parses every file as JSON, so it chokes on colocated
|
||||
# *.md docs and on two schemas whose internal `self.version` doesn't match
|
||||
# the filename version (pending Snowcat audit). Stage a filtered copy and
|
||||
# patch only the `self.version` mismatch so igluctl still lints the rest
|
||||
# of those schemas.
|
||||
run: |
|
||||
set -euo pipefail
|
||||
LINT_ROOT="$RUNNER_TEMP/iglu-lint"
|
||||
rm -rf "$LINT_ROOT"
|
||||
mkdir -p "$LINT_ROOT"
|
||||
cp -r snowplow/iglu-client-embedded/schemas "$LINT_ROOT/"
|
||||
find "$LINT_ROOT" -name '*.md' -delete
|
||||
# TODO(BOT-1403): drop these patches once the Snowcat audit fixes the
|
||||
# internal `self.version` mismatch on these two schemas. We rewrite
|
||||
# only `self.version` in the staged copy so the file's path version
|
||||
# matches what igluctl expects; every other lint rule still runs.
|
||||
# Each `sed` invocation uses the schema's wrong/right pair, so it
|
||||
# silently no-ops once the underlying schema is fixed.
|
||||
sed -i 's/"version": "1-0-1"/"version": "1-0-2"/' \
|
||||
"$LINT_ROOT/schemas/com.metabase/embed_share/jsonschema/1-0-2"
|
||||
sed -i 's/"version": "1-0-0"/"version": "1-0-1"/' \
|
||||
"$LINT_ROOT/schemas/com.metabase/serialization/jsonschema/1-0-1"
|
||||
igluctl lint "$LINT_ROOT/schemas"
|
||||
- name: Lint (Metabase schema rules)
|
||||
run: ./bin/lint-snowplow-schemas.bb
|
||||
- name: Test linter rules
|
||||
run: ./bin/lint-snowplow-schemas-test.bb
|
||||
9
.github/workflows/snyk.yml
vendored
9
.github/workflows/snyk.yml
vendored
@@ -10,7 +10,6 @@ on:
|
||||
- "**/package.json"
|
||||
- ".github/workflows/snyk.yml"
|
||||
- ".github/scripts/write-poms.sh"
|
||||
- ".github/scripts/fix-yarn-lock-aliases.ts"
|
||||
schedule:
|
||||
- cron: "0 5 * * *"
|
||||
|
||||
@@ -59,22 +58,18 @@ jobs:
|
||||
# Snyk doesn't support bun.lock yet, so we generate yarn.lock and scan that.
|
||||
# Bun has a bug where it normalizes "< 3.0.0" to "< 3" in alias entries
|
||||
# but not in dependency specs, causing Snyk to fail matching dependencies.
|
||||
# The sed commands fix this inconsistency. Bun also names aliased entries
|
||||
# after the aliased package instead of the alias, which the script fixes.
|
||||
# Both continue on error to let Snyk report parsing issues directly rather
|
||||
# than failing on the workaround.
|
||||
# The sed commands fix this inconsistency. They continue on error to let
|
||||
# Snyk report parsing issues directly rather than failing on the workaround.
|
||||
- name: Generate yarn.lock
|
||||
if: matrix.project.file == 'yarn.lock'
|
||||
run: |
|
||||
bun install --yarn
|
||||
sed -i -E 's/< ([0-9]+)\.0\.0"/< \1"/g; s/< ([0-9]+)\.0"/< \1"/g' yarn.lock || true
|
||||
bun ${{ github.workspace }}/.github/scripts/fix-yarn-lock-aliases.ts || true
|
||||
- name: Generate release/yarn.lock
|
||||
if: matrix.project.file == 'release/yarn.lock'
|
||||
run: |
|
||||
bun install --yarn
|
||||
sed -i -E 's/< ([0-9]+)\.0\.0"/< \1"/g; s/< ([0-9]+)\.0"/< \1"/g' yarn.lock || true
|
||||
bun ${{ github.workspace }}/.github/scripts/fix-yarn-lock-aliases.ts || true
|
||||
working-directory: release
|
||||
- uses: snyk/actions/setup@9adf32b1121593767fc3c057af55b55db032dc04 # v1.0.0
|
||||
- name: Run snyk test
|
||||
|
||||
37
.github/workflows/sync-fork.yml
vendored
37
.github/workflows/sync-fork.yml
vendored
@@ -24,25 +24,18 @@ jobs:
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- name: Generate workflow summary
|
||||
env:
|
||||
RELEASE_BRANCH_COUNT: ${{ inputs.release_branch_count || 5 }}
|
||||
FORCE: ${{ inputs.force || false }}
|
||||
EVENT_NAME: ${{ github.event_name }}
|
||||
SENDER: ${{ github.event.sender.login }}
|
||||
run: |
|
||||
{
|
||||
echo '**Inputs:**'
|
||||
echo ''
|
||||
echo "- \`release_branch_count\`: $RELEASE_BRANCH_COUNT"
|
||||
echo "- \`force\`: $FORCE"
|
||||
echo "- \`trigger\`: $EVENT_NAME"
|
||||
echo ''
|
||||
if [ "$EVENT_NAME" = "workflow_dispatch" ]; then
|
||||
echo "triggered by: @$SENDER"
|
||||
else
|
||||
echo 'triggered by: schedule'
|
||||
fi
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
echo '**Inputs:**' >> $GITHUB_STEP_SUMMARY
|
||||
echo '' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `release_branch_count`: ${{ inputs.release_branch_count || 5 }}' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `force`: ${{ inputs.force || false }}' >> $GITHUB_STEP_SUMMARY
|
||||
echo '- `trigger`: ${{ github.event_name }}' >> $GITHUB_STEP_SUMMARY
|
||||
echo '' >> $GITHUB_STEP_SUMMARY
|
||||
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
|
||||
echo 'triggered by: @${{ github.event.sender.login }}' >> $GITHUB_STEP_SUMMARY
|
||||
else
|
||||
echo 'triggered by: schedule' >> $GITHUB_STEP_SUMMARY
|
||||
fi
|
||||
|
||||
sync-master:
|
||||
# ONLY run on internal metabase org forks — NEVER on the public repo or external forks
|
||||
@@ -96,9 +89,9 @@ jobs:
|
||||
REPO: ${{ github.repository }}
|
||||
run: |
|
||||
if [ "${{ steps.sync.outcome }}" = "success" ]; then
|
||||
echo "| [\`master\`](https://github.com/${REPO}/tree/master) | ✅ ${{ steps.sync.outputs.detail }} |" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "| [\`master\`](https://github.com/${REPO}/tree/master) | ✅ ${{ steps.sync.outputs.detail }} |" >> $GITHUB_STEP_SUMMARY
|
||||
else
|
||||
echo "| [\`master\`](https://github.com/${REPO}/tree/master) | ❌ failed |" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "| [\`master\`](https://github.com/${REPO}/tree/master) | ❌ failed |" >> $GITHUB_STEP_SUMMARY
|
||||
fi
|
||||
|
||||
list-release-branches:
|
||||
@@ -197,7 +190,7 @@ jobs:
|
||||
run: |
|
||||
branch="${{ matrix.branch }}"
|
||||
if [ "${{ steps.sync.outcome }}" = "success" ]; then
|
||||
echo "| [\`${branch}\`](https://github.com/${REPO}/tree/${branch}) | ✅ ${{ steps.sync.outputs.detail }} |" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "| [\`${branch}\`](https://github.com/${REPO}/tree/${branch}) | ✅ ${{ steps.sync.outputs.detail }} |" >> $GITHUB_STEP_SUMMARY
|
||||
else
|
||||
echo "| [\`${branch}\`](https://github.com/${REPO}/tree/${branch}) | ❌ failed |" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "| [\`${branch}\`](https://github.com/${REPO}/tree/${branch}) | ❌ failed |" >> $GITHUB_STEP_SUMMARY
|
||||
fi
|
||||
|
||||
114
.github/workflows/team-issues-slack-notification.yml
vendored
Normal file
114
.github/workflows/team-issues-slack-notification.yml
vendored
Normal file
@@ -0,0 +1,114 @@
|
||||
# .github/workflows/team-issues-slack-notification.yml
|
||||
#
|
||||
# Posts newly team-labeled bugs to the team's Slack channel (e.g.
|
||||
# #team-graphy-bugs), now including a link to the synced Linear issue.
|
||||
#
|
||||
# Changes from the current version:
|
||||
# - Checks out .github/scripts so we can reuse find-linear-issue.js
|
||||
# - The message-building step now queries Linear (via attachmentsForURL)
|
||||
# with a short retry, and appends "Linear: <link|ID>" when found
|
||||
# - Degrades gracefully: if no Linear issue is attached (or the API is
|
||||
# down), the message posts without the Linear issue link
|
||||
|
||||
name: Team Issues Slack Notification
|
||||
|
||||
on:
|
||||
issues:
|
||||
types:
|
||||
- labeled
|
||||
|
||||
jobs:
|
||||
on-team-label-added:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
if: |
|
||||
contains(github.event.label.name, '.Team/') &&
|
||||
( contains(github.event.issue.labels.*.name, 'Type:Bug') ||
|
||||
contains(github.event.issue.labels.*.name, 'flaky-test-fix') )
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
sparse-checkout: .github/scripts
|
||||
|
||||
- name: Build Slack message (with Linear link if available)
|
||||
uses: actions/github-script@v9
|
||||
id: vars
|
||||
env:
|
||||
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
|
||||
with:
|
||||
script: |
|
||||
const { findLinearIssue } = require('./.github/scripts/find-linear-issue.js');
|
||||
|
||||
const title = context.payload.issue.title;
|
||||
const issueUrl = context.payload.issue.html_url;
|
||||
|
||||
let text = title + '\n' + issueUrl;
|
||||
|
||||
// The Linear issue is usually already synced by the time the
|
||||
// team label lands at triage, but retry briefly just in case.
|
||||
try {
|
||||
const delaysSeconds = [0, 15, 30];
|
||||
let result = null;
|
||||
|
||||
for (const delay of delaysSeconds) {
|
||||
if (delay) await new Promise((r) => setTimeout(r, delay * 1000));
|
||||
result = await findLinearIssue({
|
||||
githubIssueUrl: issueUrl,
|
||||
linearApiKey: process.env.LINEAR_API_KEY,
|
||||
});
|
||||
if (result.success) break;
|
||||
}
|
||||
|
||||
if (result && result.success) {
|
||||
const id = result.issue; // e.g. "GIT-1234"
|
||||
text += `\nLinear: <https://linear.app/metabase/issue/${id}|${id}>`;
|
||||
} else {
|
||||
core.info(`No Linear issue found for ${issueUrl}; posting without link.`);
|
||||
}
|
||||
} catch (e) {
|
||||
// Never block the notification on a Linear lookup failure
|
||||
core.warning(`Linear lookup failed: ${e.message}`);
|
||||
}
|
||||
|
||||
core.setOutput('message', JSON.stringify(text));
|
||||
|
||||
- name: Extracting team name
|
||||
id: team_name
|
||||
env:
|
||||
LABEL_NAME: ${{ github.event.label.name }}
|
||||
run: |
|
||||
team_label="${LABEL_NAME//&/$}"
|
||||
echo "$team_label"
|
||||
team_name=${team_label#*.Team/}
|
||||
echo "$team_name"
|
||||
echo "team_name=${team_name^^}_SLACK_ISSUES_WEBHOOK_URL" >> "$GITHUB_OUTPUT"
|
||||
- name: Check webhook secret exists
|
||||
id: check_secret
|
||||
env:
|
||||
WEBHOOK_URL: ${{ secrets[steps.team_name.outputs.team_name] }}
|
||||
run: |
|
||||
if [ -z "$WEBHOOK_URL" ]; then
|
||||
echo "Secret ${{ steps.team_name.outputs.team_name }} is not set, skipping Slack notification"
|
||||
echo "exists=false" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "exists=true" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
- name: Send the issue ${{ github.event.issue.number }} to Slack
|
||||
if: steps.check_secret.outputs.exists == 'true'
|
||||
id: slack
|
||||
uses: slackapi/slack-github-action@45a88b9581bfab2566dc881e2cd66d334e621e2c # v3.0.3
|
||||
with:
|
||||
webhook: ${{ secrets[steps.team_name.outputs.team_name] }}
|
||||
webhook-type: incoming-webhook
|
||||
payload: |
|
||||
{
|
||||
"blocks": [
|
||||
{
|
||||
"type": "section",
|
||||
"text": {
|
||||
"type": "mrkdwn",
|
||||
"text": ${{ steps.vars.outputs.message }}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -25,6 +25,7 @@ on:
|
||||
- sqlserver
|
||||
|
||||
permissions:
|
||||
id-token: write
|
||||
contents: read
|
||||
checks: write
|
||||
actions: read
|
||||
@@ -63,6 +64,15 @@ jobs:
|
||||
granular-rerun: 'false'
|
||||
test-args: >-
|
||||
:only-tags [:mb/transforms-python-test]
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
- name: Cleanup python-runner
|
||||
if: always()
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -114,6 +124,15 @@ jobs:
|
||||
granular-rerun: 'false'
|
||||
test-args: >-
|
||||
:only-tags [:mb/transforms-python-test]
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
- name: Cleanup python-runner
|
||||
if: always()
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -155,6 +174,15 @@ jobs:
|
||||
granular-rerun: 'false'
|
||||
test-args: >-
|
||||
:only-tags [:mb/transforms-python-test]
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
- name: Cleanup python-runner
|
||||
if: always()
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -200,6 +228,15 @@ jobs:
|
||||
granular-rerun: 'false'
|
||||
test-args: >-
|
||||
:only-tags [:mb/transforms-python-test]
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
- name: Cleanup python-runner
|
||||
if: always()
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -246,6 +283,15 @@ jobs:
|
||||
granular-rerun: 'false'
|
||||
test-args: >-
|
||||
:only-tags [:mb/transforms-python-test]
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
- name: Cleanup python-runner
|
||||
if: always()
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -282,6 +328,15 @@ jobs:
|
||||
granular-rerun: 'false'
|
||||
test-args: >-
|
||||
:only-tags [:mb/transforms-python-test]
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
- name: Cleanup python-runner
|
||||
if: always()
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -327,6 +382,15 @@ jobs:
|
||||
granular-rerun: 'false'
|
||||
test-args: >-
|
||||
:only-tags [:mb/transforms-python-test]
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
- name: Cleanup python-runner
|
||||
if: always()
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
@@ -370,6 +434,15 @@ jobs:
|
||||
granular-rerun: 'false'
|
||||
test-args: >-
|
||||
:only-tags [:mb/transforms-python-test]
|
||||
- name: Upload Test Results
|
||||
uses: ./.github/actions/upload-test-results
|
||||
if: always()
|
||||
with:
|
||||
input-path: ./target/junit/
|
||||
output-name: ${{ github.job }}
|
||||
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
|
||||
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
|
||||
aws-region: ${{ vars.AWS_REGION }}
|
||||
- name: Cleanup python-runner
|
||||
if: always()
|
||||
uses: ./.github/actions/cleanup-python-runner
|
||||
|
||||
8
.github/workflows/translation-update.yml
vendored
8
.github/workflows/translation-update.yml
vendored
@@ -59,7 +59,7 @@ jobs:
|
||||
run: | # sh
|
||||
git config user.name github-actions
|
||||
git config user.email github-actions@github.com
|
||||
git checkout -b "$BRANCH_NAME"
|
||||
git checkout -b $BRANCH_NAME
|
||||
|
||||
DATE=$(date +'%Y-%m-%d')
|
||||
|
||||
@@ -72,7 +72,7 @@ jobs:
|
||||
|
||||
git add locales/
|
||||
git commit -a -m "Update translations $Date" --allow-empty --no-verify
|
||||
git push -u origin "$BRANCH_NAME"
|
||||
git push -u origin $BRANCH_NAME
|
||||
|
||||
- name: Upload i18n violations report
|
||||
id: upload-violations
|
||||
@@ -105,7 +105,7 @@ jobs:
|
||||
|
||||
URL=$(gh pr create \
|
||||
--base master \
|
||||
--head "$BRANCH_NAME" \
|
||||
--head $BRANCH_NAME \
|
||||
--title "Update translations $DATE" \
|
||||
--body "Synchronize translations with Crowdin" \
|
||||
--label "no-backport" \
|
||||
@@ -113,7 +113,7 @@ jobs:
|
||||
|
||||
PR_NUMBER=${URL##*/}
|
||||
|
||||
echo "pr_number=$PR_NUMBER" >> "$GITHUB_OUTPUT"
|
||||
echo "pr_number=$PR_NUMBER" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Comment violations summary on PR
|
||||
if: ${{ steps.create_pr.outputs.pr_number && hashFiles('target/i18n-violations.csv') != '' }}
|
||||
|
||||
20
.github/workflows/uberjar.yml
vendored
20
.github/workflows/uberjar.yml
vendored
@@ -78,7 +78,7 @@ jobs:
|
||||
echo "Skipping build: artifacts exist for this commit hash"
|
||||
fi
|
||||
|
||||
echo "should_build=$SHOULD_BUILD" >> "$GITHUB_OUTPUT"
|
||||
echo "should_build=$SHOULD_BUILD" >> $GITHUB_OUTPUT
|
||||
|
||||
# if this is on a release branch, we need to check the build requirements
|
||||
get-build-requirements:
|
||||
@@ -154,22 +154,10 @@ jobs:
|
||||
name: metabase-bundle-stats-${{ matrix.edition }}-${{ github.event.inputs.commit || github.event.pull_request.head.sha || github.sha }}
|
||||
path: target/bundle-stats
|
||||
if-no-files-found: warn
|
||||
- name: Mark with the commit hash
|
||||
run: git rev-parse --short HEAD > COMMIT-ID
|
||||
shell: bash
|
||||
- name: Calculate SHA256 checksum
|
||||
run: sha256sum ./target/uberjar/metabase.jar > SHA256.sum
|
||||
shell: bash
|
||||
- name: Upload JARs as artifact
|
||||
uses: actions/upload-artifact@v7
|
||||
- name: Prepare uberjar artifact
|
||||
uses: ./.github/actions/prepare-uberjar-artifact
|
||||
with:
|
||||
# Same key the artifact lookups use, so a build for an explicitly dispatched commit
|
||||
# uploads where `check-existing-artifacts` and `check-jar-health` go looking for it.
|
||||
name: metabase-${{ matrix.edition }}-${{ github.event.inputs.commit || github.event.pull_request.head.sha || github.sha }}-uberjar
|
||||
path: |
|
||||
./target/uberjar/metabase.jar
|
||||
./COMMIT-ID
|
||||
./SHA256.sum
|
||||
edition: ${{ matrix.edition }}
|
||||
|
||||
check-jar-health:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
|
||||
2
.github/workflows/update-e2e-timings.yml
vendored
2
.github/workflows/update-e2e-timings.yml
vendored
@@ -61,7 +61,7 @@ jobs:
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
BRANCH_NAME="update-e2e-timings-$(date +%Y%m%d-%H%M%S)"
|
||||
echo "BRANCH_NAME=$BRANCH_NAME" >> "$GITHUB_ENV"
|
||||
echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV
|
||||
echo "Branch name will be: $BRANCH_NAME"
|
||||
|
||||
- name: Create Pull Request
|
||||
|
||||
20
.github/workflows/whitespace.yml
vendored
Normal file
20
.github/workflows/whitespace.yml
vendored
Normal file
@@ -0,0 +1,20 @@
|
||||
name: Whitespace
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'release-**'
|
||||
pull_request:
|
||||
merge_group:
|
||||
|
||||
jobs:
|
||||
whitespace-linter:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Prepare back-end environment
|
||||
uses: ./.github/actions/prepare-backend
|
||||
- run: clojure -T:whitespace-linter lint :whitespace-linter/progress-bar false
|
||||
name: Run Whitespace Linter
|
||||
34
.github/workflows/yaml.yml
vendored
Normal file
34
.github/workflows/yaml.yml
vendored
Normal file
@@ -0,0 +1,34 @@
|
||||
name: YAML
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'release-**'
|
||||
pull_request:
|
||||
|
||||
jobs:
|
||||
files-changed:
|
||||
name: Check which files changed
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
timeout-minutes: 3
|
||||
outputs:
|
||||
yaml: ${{ steps.changes.outputs.yaml }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Test which files changed
|
||||
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
|
||||
id: changes
|
||||
with:
|
||||
token: ${{ github.token }}
|
||||
filters: .github/file-paths.yaml
|
||||
|
||||
yaml-linter:
|
||||
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
|
||||
if: needs.files-changed.outputs.yaml == 'true'
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Prepare front-end environment
|
||||
uses: ./.github/actions/prepare-frontend
|
||||
- run: bun run lint-yaml
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 24 KiB After Width: | Height: | Size: 25 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 22 KiB After Width: | Height: | Size: 23 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 57 KiB After Width: | Height: | Size: 59 KiB |
15
bb.edn
15
bb.edn
@@ -425,8 +425,8 @@
|
||||
"RUN"
|
||||
(str "any of " (pr-str driver-triggers) " affected by module changes")]
|
||||
["12" "Self-hosted driver, not affected" "SKIP" "driver module not affected"]]))
|
||||
"\nThis table decides only WHETHER a driver runs. Whether its failures GATE is a separate\n"
|
||||
"question, answered at the end of the job by ci-conductor's quarantine rules.\n"
|
||||
"\nThis table decides only WHETHER a driver runs. A driver's configured status then\n"
|
||||
"applies to that decision (see 'Driver Status' below) and never overrides it.\n"
|
||||
"\nNote: the remote quarantine list (status = skip) is ignored entirely on master and\n"
|
||||
"release-* branches -- every driver runs and gates there (Priority 4 is bypassed).\n"
|
||||
"\n\n## Labels of Interest\n\n"
|
||||
@@ -436,9 +436,14 @@
|
||||
" ci:all-cloud-drivers Run all cloud drivers (athena, bigquery, databricks, redshift, snowflake)\n"
|
||||
" break-quarantine-DRIVER Un-quarantine a specific driver, e.g. break-quarantine-mysql\n"
|
||||
"\n## Driver Status (from the CI driver config)\n\n"
|
||||
" skip Listed with status \"skip\" in ci-test-config.json: not run at all (quarantined;\n"
|
||||
" break-quarantine-DRIVER overrides). Every other driver runs and gates as usual.\n"
|
||||
"\nTo let a driver run but not gate, add a suite-quarantine rule in CI Conductor instead.\n"
|
||||
"Each driver also gets a configured status, emitted as DRIVER-status=... for the result job. The\n"
|
||||
"run/skip decision above always takes precedence; status only shapes the outcome of a run mage\n"
|
||||
"decides on:\n\n"
|
||||
" skip Listed with status \"skip\": not run at all (quarantined; break-quarantine-DRIVER overrides)\n"
|
||||
" info Listed with status \"info\": if mage decides to run it, the run proceeds but its result\n"
|
||||
" must NOT gate (data collection). info NEVER forces a run -- if mage wouldn't run the\n"
|
||||
" driver, info does not change that.\n"
|
||||
" required No configured status: runs and gates as usual (the default)\n"
|
||||
"\n## All Drivers\n\n"
|
||||
(str/join ", " (map name (sort mage.modules/all-drivers)))
|
||||
"\n")))
|
||||
|
||||
@@ -107,11 +107,11 @@
|
||||
[artifact-id version]
|
||||
(let [pom-name (pom-file-name artifact-id version)
|
||||
xml (x/element :project {}
|
||||
(x/element :modelVersion {} "4.0.0")
|
||||
(x/element :groupId {} "com.metabase")
|
||||
(x/element :artifactId {} artifact-id)
|
||||
(x/element :version {} version)
|
||||
(x/element :packaging {} "jar"))]
|
||||
(x/element :modelVersion {} "4.0.0")
|
||||
(x/element :groupId {} "com.metabase")
|
||||
(x/element :artifactId {} artifact-id)
|
||||
(x/element :version {} version)
|
||||
(x/element :packaging {} "jar"))]
|
||||
(println "pom =>" pom-name)
|
||||
(with-open [w (io/writer pom-name)]
|
||||
(x/emit xml w))
|
||||
|
||||
11
deps.edn
11
deps.edn
@@ -353,9 +353,6 @@
|
||||
"-Dmb.jetty.join=false"
|
||||
"-Dmb.field.filter.operators.enabled=true"
|
||||
"-Dmb.api.key=test-api-key"
|
||||
;; Test warehouses live on localhost, so the network policy has to be off by default or every test
|
||||
;; that creates one breaks.
|
||||
"-Dmb.warehouse.allowed.networks=allow-all"
|
||||
;; Port 0 tells the OS to assign a random free port, allowing parallel test runs.
|
||||
;; The actual port is read from the running Jetty server via server.instance/server-port.
|
||||
"-Dmb.jetty.port=0"]}
|
||||
@@ -390,14 +387,6 @@
|
||||
"-Duser.timezone=US/Pacific"
|
||||
"-Dmb.db.type=h2"
|
||||
"-Dmb.enable.test.endpoints=true"
|
||||
;; Use the in-memory queue backend in e2e. The persistent Quartz backend keeps its
|
||||
;; durable-job state in the app DB's QRTZ_* tables, but e2e restores app-db snapshots
|
||||
;; between tests (`H.restore`), which drops those tables out from under the running
|
||||
;; scheduler while the per-process `ensured-jobs` cache still thinks the jobs exist —
|
||||
;; so post-restore publishes target a non-existent job and never deliver. e2e is a
|
||||
;; single node, so the in-memory backend (no app-db state, unaffected by restore) is
|
||||
;; the correct fit.
|
||||
"-Dmb.queue.backend=memory"
|
||||
"-Dmb.dangerous.unsafe.enable.testing.h2.connections.do.not.enable=true"
|
||||
;; Ignore connection URI in favor of the MB_DB_FILE
|
||||
"-Dmb.db.connection.uri="
|
||||
|
||||
@@ -10,6 +10,7 @@
|
||||
[clojure.walk :as walk]
|
||||
[lambdaisland.deep-diff2 :as ddiff]
|
||||
[metabase.util.malli :as mu]
|
||||
[metabase.util.malli.registry :as mr]
|
||||
[metabase.util.malli.schema :as ms]
|
||||
[rewrite-clj.node :as n]
|
||||
[rewrite-clj.parser :as r.parser]
|
||||
|
||||
@@ -13,7 +13,6 @@
|
||||
[metabase.channel.render.style :as style]
|
||||
[metabase.notification.payload.core :as notification.payload]
|
||||
[metabase.query-processor.core :as qp]
|
||||
[metabase.system.core :as system]
|
||||
[metabase.test :as mt]
|
||||
[metabase.util.markdown :as markdown]
|
||||
[toucan2.core :as t2])
|
||||
@@ -86,7 +85,7 @@
|
||||
:font-style "normal"
|
||||
:color "#4c5773"
|
||||
:-moz-osx-font-smoothing "grayscale"})}
|
||||
(markdown/process-markdown (:text dashboard-result) :html (system/site-url))]
|
||||
(markdown/process-markdown (:text dashboard-result) :html)]
|
||||
:attachments nil})
|
||||
png-bytes (-> render (png/render-html-to-png 1000))
|
||||
tmp-file (java.io.File/createTempFile "card-png" ".png")]
|
||||
@@ -151,7 +150,7 @@
|
||||
:font-style "normal"
|
||||
:color "#4c5773"
|
||||
:-moz-osx-font-smoothing "grayscale"})}
|
||||
(markdown/process-markdown (:text dashboard-result) :html (system/site-url))])
|
||||
(markdown/process-markdown (:text dashboard-result) :html)])
|
||||
(cellfn nil)])))
|
||||
|
||||
(defn render-dashboard-to-hiccup
|
||||
|
||||
@@ -223,7 +223,6 @@ Metabase's reference documentation.
|
||||
- [SSO](./embedding/modular-embedding.md)
|
||||
- [Guest](./embedding/guest-embedding.md)
|
||||
- [SDK](./embedding/sdk/introduction.md)
|
||||
- [Custom visualizations](./embedding/custom-visualizations.md)
|
||||
- [Full app embedding](./embedding/full-app-embedding.md)
|
||||
- [Securing embeds](./embedding/securing-embeds.md)
|
||||
- [AI agent resources](./embedding/ai-agent-resources.md)
|
||||
|
||||
7960
docs/api.json
generated
7960
docs/api.json
generated
File diff suppressed because it is too large
Load Diff
@@ -38,7 +38,7 @@ These policies determine how long cached results will remain valid.
|
||||
|
||||
{% include plans-blockquote.html feature="Duration caching policy" %}
|
||||
|
||||
Invalidate and clear the cache after a certain amount of time (seconds, minutes, hours, or days). When someone runs a query, Metabase will first check whether it has cached the results, and whether those results are still valid. If not, Metabase runs the query against the database and caches the results. These cached results will remain valid for the duration you've set.
|
||||
Invalidate and clear the cache after a certain number of hours. When someone runs a query, Metabase will first check whether it has cached the results, and whether those results are still valid. If not, Metabase runs the query against the database and caches the results. These cached results will remain valid for the duration you've set.
|
||||
|
||||
### Schedule caching policy
|
||||
|
||||
|
||||
@@ -155,7 +155,6 @@ config:
|
||||
enable-pivoted-exports: true
|
||||
enable-public-sharing: true
|
||||
enable-xrays: true
|
||||
explorations-worker-count: 2
|
||||
fingerprint-max-fields-per-table: 10000
|
||||
follow-up-email-sent: false
|
||||
google-auth-auto-create-accounts-domain: null
|
||||
@@ -217,13 +216,11 @@ config:
|
||||
llm-bedrock-region: us-east-1
|
||||
llm-bedrock-secret-access-key: null
|
||||
llm-bedrock-session-token: null
|
||||
llm-connection-timeout-ms: 10000
|
||||
llm-connection-timeout-ms: 5000
|
||||
llm-max-tokens: 4096
|
||||
llm-metabot-provider: anthropic/claude-sonnet-4-6
|
||||
llm-mistral-api-base-url: https://api.mistral.ai/v1
|
||||
llm-mistral-api-key: null
|
||||
llm-moonshot-api-base-url: https://api.moonshot.ai/v1
|
||||
llm-moonshot-api-key: null
|
||||
llm-openai-api-base-url: https://api.openai.com
|
||||
llm-openai-api-key: null
|
||||
llm-openai-model: gpt-5.4
|
||||
@@ -231,7 +228,7 @@ config:
|
||||
llm-openrouter-api-key: null
|
||||
llm-rate-limit-per-ip: 100
|
||||
llm-rate-limit-per-user: 20
|
||||
llm-request-timeout-ms: 120000
|
||||
llm-request-timeout-ms: 60000
|
||||
llm-zai-api-base-url: https://api.z.ai/api/paas/v4
|
||||
llm-zai-api-key: null
|
||||
loading-message: doing-science
|
||||
@@ -311,7 +308,7 @@ config:
|
||||
search-typeahead-enabled: true
|
||||
send-new-sso-user-admin-email: null
|
||||
session-cookie-samesite: lax
|
||||
session-cookies: false
|
||||
session-cookies: null
|
||||
session-timeout: null
|
||||
setup-embedding-autoenabled: false
|
||||
setup-license-active-at-setup: false
|
||||
@@ -355,5 +352,4 @@ config:
|
||||
usage-metadata-schedule: 0 0 2 * * ? *
|
||||
use-native-pivot-tables: false
|
||||
user-visibility: all
|
||||
warehouse-allowed-networks: null
|
||||
```
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user