Compare commits

...

25 Commits

Author SHA1 Message Date
Bryan Maass
599979fe73 Regenerate bun.lock to match package.json with @redocly/cli
The previous merge used master's bun.lock directly, but our branch
added @redocly/cli to package.json. Regenerated the lockfile to
include all dependencies while keeping patch-compatible versions.
2026-02-18 16:57:25 -07:00
Bryan Maass
b98b5f7d87 Merge master to fix CI - properly resolve bun.lock conflicts 2026-02-18 14:17:45 -07:00
Bryan Maass
88c9768141 Merge master into fix-open-api-spec
Resolved conflicts:
- GitHub Actions: Took master's Bun migration changes
- package.json: Merged lint-openapi script with master's bun changes
- bun.lock: Regenerated
- Clojure lib/metadata: Accepted master's :source-alias deprecation
- clj-kondo config: Merged both module dependency additions
- Migrations: Accepted master's new changesets
- Frontend: Kept master's new files (snippet tree hooks, BoxPlot tests)
2026-02-17 15:39:26 -07:00
bryan
24df4b2f45 Merge branch 'master' into fix-open-api-spec 2026-02-11 16:37:20 -07:00
Bryan Maass
33d431c68c Merge remote-tracking branch 'origin/master' into fix-open-api-spec
# Conflicts:
2026-02-11 15:37:14 -07:00
bryan
61c1b458ee Merge branch 'master' into fix-open-api-spec 2026-02-10 11:35:01 -07:00
Bryan Maass
adff051520 Update bun.lock to include @redocly/cli dependency 2026-02-10 10:57:30 -07:00
Bryan Maass
0802d4caa7 Merge master into fix-open-api-spec
Resolved conflicts:
- package.json: kept both @redocly/cli and @lezer/common dependencies,
  retained lint-openapi script while using bun run syntax
- yarn.lock: deleted (Metabase migrated to Bun)
2026-02-10 10:24:02 -07:00
Bryan Maass
6e422e8632 Merge remote-tracking branch 'origin/master' into fix-open-api-spec 2026-02-04 14:43:40 -07:00
Bryan Maass
1755a4717a Merge branch 'master' into fix-open-api-spec
Resolved yarn.lock conflict by regenerating lockfile.
2026-02-04 14:32:21 -07:00
bryan
a0a2cc4d60 Merge branch 'master' into fix-open-api-spec 2026-01-29 10:21:38 -07:00
Bryan Maass
44ec6b42d9 Merge remote-tracking branch 'origin/master' into fix-open-api-spec 2026-01-16 11:06:28 -07:00
bryan
4639013bd2 Merge branch 'master' into fix-open-api-spec 2026-01-12 10:03:42 -07:00
bryan
c45ba52582 Merge branch 'master' into fix-open-api-spec 2026-01-09 12:04:32 -07:00
bryan
c7ac7eb329 Merge branch 'master' into fix-open-api-spec 2026-01-09 10:08:28 -07:00
Bryan Maass
f5319f4a09 kondo 2026-01-08 16:12:16 -07:00
Bryan Maass
aeb067d0f1 Fix doc-test to use path without trailing slash
The strip-trailing-slash change in open_api.clj now removes trailing
slashes from OpenAPI paths, so update the test to use /api/card instead
of /api/card/.
2026-01-08 11:33:50 -07:00
Bryan Maass
31cb4f0d13 clojure review skill should ignore comment lines
Got 3 in a row like:

```
    (let [schema (-> schema
    ;; Helper to recursively fix nested schemas and strip :optional (which is only
    ;; meaningful at the top level for parameter detection, not inside oneOf/anyOf/allOf)

```
> metabase-bot: According to the Clojure style guide, there should be no
blank lines within definition forms (except for pairwise constructs in
let/cond). This blank line on line 76 should be removed.

But: that is not a blank line!!
2026-01-07 16:55:59 -07:00
Bryan Maass
62f2bd0a80 Add CI workflow to validate OpenAPI spec
- Create .github/workflows/openapi.yml to generate and lint OpenAPI spec
- Add openapi filter to file-paths.yaml for conditional execution
- Workflow triggers on changes to API endpoints (src/metabase/**/api*.clj,
  enterprise/backend/src/metabase_enterprise/**/api*.clj)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 16:39:21 -07:00
Bryan Maass
cfec194940 Refactor: extract sanitize-ref and path->operation-id helpers
Move inline functions to top-level for better readability:
- sanitize-ref: Sanitize $ref paths to use sanitized schema names
- path->operation-id: Generate operationId from method + path
2026-01-07 15:43:44 -07:00
Bryan Maass
13d835bd76 Fix additional OpenAPI validation rules
This commit addresses several OpenAPI 3.1.0 validation rules that were
previously skipped:

- security-defined: Add API key auth security scheme and global security
- operation-operationId: Generate operationId from method + path
- info-license: Add AGPL-3.0 license info
- no-path-trailing-slash: Strip trailing slashes from paths
- spec-components-invalid-map-name: Sanitize schema names for OpenAPI

The following rules remain skipped (documented in PR description):
- no-unused-components: 66 unused schemas (needs broader cleanup)
- no-identical-paths / no-ambiguous-paths: 6 routes with different param names
- path-parameters-defined: 11 endpoints missing path param definitions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 15:23:02 -07:00
Bryan Maass
0ab5bf7b08 kondo 2026-01-07 14:27:31 -07:00
Bryan Maass
c7a686e002 Move OpenAPI linting from mage to yarn script
- Add @redocly/cli ^2.14.3 as devDependency
- Add `yarn lint-openapi` script with skip-rules for pre-existing issues
- Remove mage -lint-openapi task and mage/src/mage/openapi.clj
- Remove redundant openapi-spec-valid-test (validation now handled by redocly)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 13:24:24 -07:00
Bryan Maass
ea442820f6 Address PR review feedback
- Fix exception handling to print actual error message instead of parsed options
- Pin @redocly/cli to version 1.34.2 for supply chain security
- Update error message to use yarn generate-openapi

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 12:44:01 -07:00
Bryan Maass
bae0325828 Fix OpenAPI 3.1.0 validation errors (#67748)
- Add required `info.version` field to OpenAPI spec using Metabase version tag
- Fix Malli `optional` field leaking into nested schemas (oneOf/anyOf/allOf)
- Add tests for info.version and spec validity
- Add mage task `-lint-openapi` for validation with Redocly CLI
- Enhance mage `-repl` to wrap code in `do` block for multi-form evaluation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 11:54:51 -07:00
336 changed files with 3444 additions and 24656 deletions

View File

@@ -410,6 +410,7 @@
models
request
users
users-rest
util}}
;; technically this 'uses' `enterprise/core` and `test` since it tries to load them to see if they exists so we know
@@ -1024,7 +1025,6 @@
events
internal-stats
settings
startup
task
util}}
@@ -1316,7 +1316,8 @@
startup
task
transforms
util}}
util
warehouses}}
secrets
{:team "Querying"
@@ -1608,6 +1609,7 @@
enterprise/transforms-python
enterprise/workspaces}}
types
{:team "Querying"
:api #{metabase.types.core
@@ -1646,42 +1648,41 @@
users
{:team "UX West"
:api #{metabase.users.core
metabase.users.init
:api #{metabase.users.init
metabase.users.models.user
metabase.users.models.user-parameter-value
metabase.users.schema
metabase.users.settings}
:uses #{analytics
api
:uses #{api
batch-processing
channel
config
events
models
notification
permissions
premium-features
settings
setup
system
tenants
util
enterprise/advanced-permissions}}
util}}
users-rest
{:team "UX West"
:api #{metabase.users-rest.api}
:uses #{api
:uses #{analytics
api
appearance
auth-identity
collections
config
events
models
notification
permissions
premium-features
request
settings
sso
tenants
users
@@ -1972,7 +1973,7 @@
setup
startup
sync
users
users-rest
util
enterprise/serialization}}
@@ -2087,7 +2088,7 @@
settings
util
warehouse-schema
warehouses}}
warehouses-rest}}
enterprise/dependencies
{:team "Querying"
@@ -2233,9 +2234,11 @@
system
task
transforms
users
users-rest
util
warehouse-schema
warehouses
warehouses-rest
enterprise/dependencies
enterprise/transforms-python}}
@@ -2271,8 +2274,7 @@
task
task-history
util
enterprise/serialization
enterprise/transforms-python}}
enterprise/serialization}}
enterprise/sandbox
{:team "UX West"
@@ -2397,6 +2399,7 @@
:uses #{analytics
api
collections
collections-rest
embedding
premium-features
queries
@@ -2440,7 +2443,6 @@
enterprise/transforms-python
{:team "Querying"
:api #{metabase-enterprise.transforms-python.api
metabase-enterprise.transforms-python.core
metabase-enterprise.transforms-python.init
metabase-enterprise.transforms-python.python-runner}
:uses #{analytics

View File

@@ -75,15 +75,6 @@
(contains? module-api-namespaces ns-symb)
(contains? module-friends current-module))))
(defn- rest-module? [module]
(str/ends-with? module "-rest"))
(defn- routes-module? [module]
(str/ends-with? module "-routes"))
(defn- core-module? [module]
(str/ends-with? module "core"))
(defn usage-error
"Find usage errors when a `required-namespace` is required in the `current-module`. Returns a string describing the
error type if there is one, otherwise `nil` if there are no errors."
@@ -102,15 +93,4 @@
(format "Namespace %s is not an allowed external API namespace for the %s module. [:metabase/modules %s :api]"
required-namespace
required-module
required-module)
;; (for now) rest modules are allowed to use one another; `routes` is ok because it collects routes together
;; and `core` is ok because [[metabase.core.init]] might need to init some of the `-routes` modules'
;; namespaces
(and (not ((some-fn rest-module? routes-module? core-module?) current-module))
(rest-module? required-module))
(format "Do not use -rest modules (%s) in non-rest modules (%s) -- move things from %s to %s if needed"
required-module
current-module
required-module
(symbol (str/replace required-module #"-rest$" "")))))))
required-module)))))

View File

@@ -199,3 +199,15 @@ cljs:
- *shared_specs
- ".github/workflows/cljs.yml"
- "shadow-cljs.edn"
openapi:
- ".github/workflows/openapi.yml"
- "src/metabase/api/**"
- "src/metabase/api_routes/**"
- "src/metabase/*_rest/api.clj"
- "src/metabase/*/api.clj"
- "src/metabase/*/api/*.clj"
- "enterprise/backend/src/metabase_enterprise/**/api.clj"
- "enterprise/backend/src/metabase_enterprise/**/api/*.clj"
- "resources/openapi/**"
- "package.json"

View File

@@ -72,7 +72,7 @@ jobs:
# All decision logic is in mage - it analyzes module dependencies,
# branch context, labels, and file changes to decide what runs.
MAGE_ARGS=(
--git-ref="${{ github.event.pull_request.base.ref || github.ref_name }}"
--git-ref="${{ github.event.pull_request.base.ref || 'master' }}"
--is-master-or-release="${{ github.ref_name == 'master' || startsWith(github.ref_name, 'release-') }}"
--pr-labels="${{ join(github.event.pull_request.labels.*.name, ',') }}"
--skip="${{ inputs.skip }}"

44
.github/workflows/openapi.yml vendored Normal file
View File

@@ -0,0 +1,44 @@
name: Validate OpenAPI
on:
push:
branches:
- 'master'
- 'release-**'
pull_request:
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
jobs:
files-changed:
name: Check which files changed
runs-on: ubuntu-22.04
timeout-minutes: 3
outputs:
openapi: ${{ steps.changes.outputs.openapi }}
steps:
- uses: actions/checkout@v4
- name: Test which files changed
uses: dorny/paths-filter@v3.0.0
id: changes
with:
token: ${{ github.token }}
filters: .github/file-paths.yaml
validate-openapi:
needs: files-changed
if: needs.files-changed.outputs.openapi == 'true'
runs-on: ubuntu-22.04
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- name: Generate OpenAPI spec
run: yarn generate-openapi
- name: Validate OpenAPI spec
run: yarn lint-openapi

View File

@@ -69,8 +69,7 @@ jobs:
if: matrix.project.file == 'release/yarn.lock'
run: |
bun install --yarn
sed -i -E 's/< ([0-9]+)\.0\.0"/< \1"/g; s/< ([0-9]+)\.0"/< \1"/g' yarn.lock || true
working-directory: release
cd release && bun install --yarn
- uses: snyk/actions/setup@0.4.0
- name: Run snyk test
env:

Binary file not shown.

Before

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 48 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 9.0 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 44 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 50 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 12 KiB

8
bb.edn
View File

@@ -430,9 +430,13 @@
["mage -repl --port 59498 --namespace metabase.app-db.connection '(read-string \"::hello\")'"
"Evaluate a keyword inside a namespace"]]
:options [["-n" "--namespace NAMESPACE" "Namespace to use for the task"]
["-p" "--port PORT" "Port to use for the task, defaults to value in .nrepl-port"]]
["-p" "--port PORT" "Port to use for the task, defaults to value in .nrepl-port"]
["-f" "--file FILE" "A file with clojure code to evaluate"]]
:task (let [parsed (cli/parse! (current-task))
code (first (:arguments parsed))
code (or
(try (some-> parsed :options :file slurp)
(catch Exception e (println "No file found at given path:" (ex-message e))))
(first (:arguments parsed)))
port (or (:port (:options parsed))
(slurp ".nrepl-port")
(throw (ex-info (c/red "No port specified, and no .nrepl-port file found.") {:babashka/exit 1})))

View File

@@ -62,173 +62,6 @@ is_rosetta_installed() {
arch -x86_64 /usr/bin/true 2>/dev/null
}
# =============================================================================
# Build Tools Detection and Installation
# =============================================================================
# Check if Xcode Command Line Tools are installed (macOS)
is_xcode_clt_installed() {
# xcode-select -p returns the path if CLT is installed, fails otherwise
xcode-select -p &>/dev/null
}
# Check if build-essential (or equivalent) is installed (Linux)
is_build_tools_installed_linux() {
# Check for gcc and make as proxies for build tools
has_tool gcc && has_tool make
}
# Detect Linux distribution family
detect_linux_distro() {
if [ -f /etc/os-release ]; then
. /etc/os-release
case "$ID" in
ubuntu|debian|linuxmint|pop) echo "debian" ;;
fedora|rhel|centos|rocky|alma) echo "redhat" ;;
arch|manjaro|endeavouros) echo "arch" ;;
opensuse*|sles) echo "suse" ;;
*) echo "unknown" ;;
esac
elif [ -f /etc/debian_version ]; then
echo "debian"
elif [ -f /etc/redhat-release ]; then
echo "redhat"
else
echo "unknown"
fi
}
# Install Xcode Command Line Tools (macOS)
install_xcode_clt() {
if is_xcode_clt_installed; then
print_success "Xcode Command Line Tools are installed"
return 0
fi
print_warning "Xcode Command Line Tools are not installed"
echo ""
echo " These tools provide compilers (clang, gcc) and build utilities"
echo " required for compiling native extensions in development tools."
echo ""
if ask_yes_no "Install Xcode Command Line Tools?" "y"; then
print_info "Installing Xcode Command Line Tools..."
echo " A system dialog will appear. Please follow the prompts."
echo ""
# Trigger the CLT installer - this opens a GUI dialog
xcode-select --install 2>/dev/null || true
# Wait for the user to complete the installation
echo ""
print_info "Waiting for installation to complete..."
echo " Please complete the installation in the dialog that appeared."
echo ""
# Poll until installation completes or user gives up
local max_attempts=60 # 10 minutes max (60 * 10 seconds)
local attempt=0
printf " "
while ! is_xcode_clt_installed; do
attempt=$((attempt + 1))
if [ $attempt -ge $max_attempts ]; then
echo ""
print_error "Xcode Command Line Tools installation timed out"
echo " Please install manually with: xcode-select --install"
return 1
fi
# Show progress dot every 10 seconds
printf "."
sleep 10
done
echo ""
print_success "Xcode Command Line Tools installed successfully"
else
print_warning "Skipped Xcode Command Line Tools installation"
echo " Some tools may fail to compile without build tools."
echo " Install manually with: xcode-select --install"
fi
}
# Install build tools on Linux
install_build_tools_linux() {
if is_build_tools_installed_linux; then
print_success "Build tools (gcc, make) are installed"
return 0
fi
print_warning "Build tools are not installed"
echo ""
echo " Build tools (gcc, make, etc.) are required for compiling"
echo " native extensions in development tools."
echo ""
local distro=$(detect_linux_distro)
local install_cmd=""
local package_name=""
case "$distro" in
debian)
package_name="build-essential"
install_cmd="sudo apt-get update && sudo apt-get install -y build-essential"
;;
redhat)
package_name="Development Tools"
if has_tool dnf; then
install_cmd="sudo dnf groupinstall -y 'Development Tools'"
else
install_cmd="sudo yum groupinstall -y 'Development Tools'"
fi
;;
arch)
package_name="base-devel"
install_cmd="sudo pacman -S --needed --noconfirm base-devel"
;;
suse)
package_name="devel_basis"
install_cmd="sudo zypper install -y -t pattern devel_basis"
;;
*)
print_warning "Unknown Linux distribution"
echo " Please install build tools (gcc, make, etc.) manually."
return 0
;;
esac
echo " Package: $package_name"
echo " Command: $install_cmd"
echo ""
if ask_yes_no "Install build tools?" "y"; then
print_info "Installing build tools..."
if eval "$install_cmd"; then
print_success "Build tools installed successfully"
else
print_error "Failed to install build tools"
echo " Please install manually with: $install_cmd"
return 1
fi
else
print_warning "Skipped build tools installation"
echo " Some tools may fail to compile without build tools."
fi
}
# Main function to install build tools based on OS
install_build_tools() {
local os=$(detect_os)
case "$os" in
macos)
install_xcode_clt
;;
linux)
install_build_tools_linux
;;
esac
}
# Install Rosetta 2 for running x86_64 binaries on Apple Silicon
install_rosetta() {
local os=$(detect_os)
@@ -593,8 +426,8 @@ main() {
print_header "Metabase Development Environment Setup"
echo "This script will:"
echo " 1. Check prerequisites and install build tools"
echo -e "${DIM} - Checks OS, shell, curl; installs Xcode CLT (Mac) or build-essential (Linux)${NC}"
echo " 1. Check prerequisites"
echo -e "${DIM} - Checks your OS, shell, curl installed, etc. ${NC}"
echo " 2. Install/update mise"
echo -e "${DIM} - Installs if needed or checks for updates${NC}"
echo " 3. Configure shell integration (with your permission)"
@@ -609,7 +442,7 @@ main() {
ask_yes_no "Are you ready?" "y" || exit 1
# Check prerequisites
print_step "1. Check prerequisites and build tools"
print_step "1. Check prerequisites"
local os=$(detect_os)
local arch=$(detect_arch)
@@ -636,9 +469,6 @@ main() {
fi
print_success "curl is available"
# Install build tools (Xcode CLT on macOS, build-essential on Linux)
install_build_tools
# Install Rosetta 2 if needed (macOS ARM64 only)
install_rosetta

499
bun.lock

File diff suppressed because it is too large Load Diff

View File

@@ -5600,42 +5600,60 @@
"required" : [ "type", "name" ],
"additionalProperties" : false
},
"metabase-enterprise.workspaces.types.ancestors-result" : {
"description" : "Result of running stale ancestor transforms.",
"type" : "object",
"properties" : {
"failed" : {
"type" : "array",
"items" : {
"$ref" : "#/components/schemas/metabase-enterprise.workspaces.types.ref-id"
}
},
"not_run" : {
"type" : "array",
"items" : {
"$ref" : "#/components/schemas/metabase-enterprise.workspaces.types.ref-id"
}
},
"succeeded" : {
"type" : "array",
"items" : {
"$ref" : "#/components/schemas/metabase-enterprise.workspaces.types.ref-id"
}
}
},
"required" : [ "succeeded", "failed", "not_run" ]
},
"metabase-enterprise.workspaces.types.appdb-id" : {
"description" : "value must be an integer greater than zero.",
"type" : "integer",
"minimum" : 1
},
"metabase-enterprise.workspaces.types.dry-run-result" : {
"description" : "Result of a transform dry-run (preview without persisting).\n Data is nested under :data to match /api/dataset response format.",
"type" : "object",
"properties" : {
"data" : {
"type" : "object",
"properties" : {
"cols" : {
"type" : "array",
"items" : {
"type" : "object",
"properties" : { }
}
},
"results_metadata" : {
"type" : "object",
"properties" : {
"columns" : {
"type" : "array",
"items" : {
"type" : "object",
"properties" : { }
}
}
}
},
"rows" : {
"type" : "array",
"items" : { }
}
}
},
"message" : {
"oneOf" : [ {
"type" : "string"
}, {
"type" : "null"
} ]
},
"status" : {
"type" : "string",
"enum" : [ "succeeded", "failed" ]
}
},
"required" : [ "status" ]
},
"metabase-enterprise.workspaces.types.execution-result" : {
"type" : "object",
"properties" : {
"ancestors" : {
"$ref" : "#/components/schemas/metabase-enterprise.workspaces.types.ancestors-result"
},
"end_time" : {
"oneOf" : [ { }, {
"type" : "null"
@@ -5676,14 +5694,6 @@
},
"required" : [ "status", "table" ]
},
"metabase-enterprise.workspaces.types.flag" : {
"description" : "Boolean flag that accepts true, false, or 1 (for query param compatibility).",
"anyOf" : [ {
"const" : 1
}, {
"type" : "boolean"
} ]
},
"metabase-enterprise.workspaces.types.problem" : {
"description" : "A problem detected during workspace validation.",
"type" : "object",
@@ -5715,67 +5725,6 @@
"type" : "string",
"enum" : [ "target-conflict", "not-run", "stale", "cycle", "failed", "removed-table", "removed-field" ]
},
"metabase-enterprise.workspaces.types.query-result" : {
"description" : "Result of a query preview (dry-run or ad-hoc query).\n Data is nested under :data to match /api/dataset response format.",
"type" : "object",
"properties" : {
"ancestors" : {
"$ref" : "#/components/schemas/metabase-enterprise.workspaces.types.ancestors-result"
},
"data" : {
"type" : "object",
"properties" : {
"cols" : {
"type" : "array",
"items" : {
"type" : "object",
"properties" : { }
}
},
"results_metadata" : {
"type" : "object",
"properties" : {
"columns" : {
"type" : "array",
"items" : {
"type" : "object",
"properties" : { }
}
}
}
},
"rows" : {
"type" : "array",
"items" : { }
}
}
},
"message" : {
"oneOf" : [ {
"type" : "string"
}, {
"type" : "null"
} ]
},
"running_time" : {
"oneOf" : [ {
"type" : "integer"
}, {
"type" : "null"
} ]
},
"started_at" : {
"oneOf" : [ { }, {
"type" : "null"
} ]
},
"status" : {
"type" : "string",
"enum" : [ "succeeded", "failed" ]
}
},
"required" : [ "status" ]
},
"metabase-enterprise.workspaces.types.ref-id" : {
"type" : "string",
"minLength" : 1
@@ -14122,7 +14071,7 @@
"$ref" : "#/components/schemas/metabase.transforms.schema.table-incremental-target"
} ]
},
"metabase.users.schema.user-group-membership" : {
"metabase.users-rest.api.user-group-membership" : {
"description" : "Group Membership info of a User.\n In which :is_group_manager is only included if `advanced-permissions` is enabled.",
"type" : "object",
"properties" : {
@@ -27857,7 +27806,8 @@
} ]
},
"id" : {
"type" : "integer"
"type" : "integer",
"minimum" : 1
},
"name" : {
"oneOf" : [ {
@@ -30027,9 +29977,6 @@
"schema" : {
"type" : "object",
"properties" : {
"api_key" : {
"type" : "string"
},
"collection_id" : {
"$ref" : "#/components/schemas/metabase-enterprise.workspaces.types.appdb-id"
},
@@ -30326,9 +30273,6 @@
"schema" : {
"type" : "object",
"properties" : {
"api_key" : {
"type" : "string"
},
"collection_id" : {
"$ref" : "#/components/schemas/metabase-enterprise.workspaces.types.appdb-id"
},
@@ -30388,9 +30332,6 @@
"schema" : {
"type" : "object",
"properties" : {
"api_key" : {
"type" : "string"
},
"collection_id" : {
"$ref" : "#/components/schemas/metabase-enterprise.workspaces.types.appdb-id"
},
@@ -30510,9 +30451,6 @@
"schema" : {
"type" : "object",
"properties" : {
"api_key" : {
"type" : "string"
},
"collection_id" : {
"$ref" : "#/components/schemas/metabase-enterprise.workspaces.types.appdb-id"
},
@@ -31096,55 +31034,6 @@
"tags" : [ "/api/ee/workspace" ]
}
},
"/api/ee/workspace/{ws-id}/query" : {
"post" : {
"summary" : "POST /api/ee/workspace/{ws-id}/query",
"description" : "Execute an arbitrary SQL query in the workspace's isolated database context.\n Table references are remapped to isolated workspace tables.\n Returns the first 2000 rows of query results.",
"parameters" : [ {
"in" : "path",
"name" : "ws-id",
"required" : true,
"schema" : {
"$ref" : "#/components/schemas/metabase-enterprise.workspaces.types.appdb-id"
}
} ],
"responses" : {
"2XX" : {
"description" : "Successful response",
"content" : {
"application/json" : {
"schema" : {
"$ref" : "#/components/schemas/metabase-enterprise.workspaces.types.query-result"
}
}
}
},
"4XX" : {
"description" : "Client error response"
},
"5XX" : {
"description" : "Server error response"
}
},
"requestBody" : {
"content" : {
"application/json" : {
"schema" : {
"type" : "object",
"properties" : {
"sql" : {
"type" : "string",
"minLength" : 1
}
},
"required" : [ "sql" ]
}
}
}
},
"tags" : [ "/api/ee/workspace" ]
}
},
"/api/ee/workspace/{ws-id}/run" : {
"post" : {
"summary" : "POST /api/ee/workspace/{ws-id}/run",
@@ -31203,7 +31092,11 @@
"type" : "object",
"properties" : {
"stale_only" : {
"$ref" : "#/components/schemas/metabase-enterprise.workspaces.types.flag"
"anyOf" : [ {
"const" : 1
}, {
"type" : "boolean"
} ]
}
}
}
@@ -31656,7 +31549,7 @@
},
"put" : {
"summary" : "PUT /api/ee/workspace/{ws-id}/transform/{tx-id}",
"description" : "Update or create a transform in a workspace.\n If the transform exists, updates it. If it doesn't exist, creates a new transform with the provided ref_id.\n For creation, name, source, and target are required.",
"description" : "Update a transform in a workspace.",
"parameters" : [ {
"in" : "path",
"name" : "ws-id",
@@ -31867,7 +31760,7 @@
"/api/ee/workspace/{ws-id}/transform/{tx-id}/dry-run" : {
"post" : {
"summary" : "POST /api/ee/workspace/{ws-id}/transform/{tx-id}/dry-run",
"description" : "Dry-run a transform in a workspace without persisting to the target table.\n\n Returns the first 2000 rows of transform output for preview purposes.\n Does not update last_run_at or create any database tables.\n\n When run_stale_ancestors is true, any stale ancestor transforms will be executed first\n in dependency order (these ARE persisted). If any ancestor fails, execution stops and\n the dry-run will not proceed.",
"description" : "Dry-run a transform in a workspace without persisting to the target table.\n\n Returns the first 2000 rows of transform output for preview purposes.\n Does not update last_run_at or create any database tables.",
"parameters" : [ {
"in" : "path",
"name" : "ws-id",
@@ -31889,7 +31782,7 @@
"content" : {
"application/json" : {
"schema" : {
"$ref" : "#/components/schemas/metabase-enterprise.workspaces.types.query-result"
"$ref" : "#/components/schemas/metabase-enterprise.workspaces.types.dry-run-result"
}
}
}
@@ -31901,20 +31794,6 @@
"description" : "Server error response"
}
},
"requestBody" : {
"content" : {
"application/json" : {
"schema" : {
"type" : "object",
"properties" : {
"run_stale_ancestors" : {
"$ref" : "#/components/schemas/metabase-enterprise.workspaces.types.flag"
}
}
}
}
}
},
"tags" : [ "/api/ee/workspace" ]
}
},
@@ -31996,7 +31875,7 @@
"/api/ee/workspace/{ws-id}/transform/{tx-id}/run" : {
"post" : {
"summary" : "POST /api/ee/workspace/{ws-id}/transform/{tx-id}/run",
"description" : "Run a transform in a workspace.\n\n App DB changes are rolled back. Warehouse DB changes persist.\n\n When run_stale_ancestors is true, any stale ancestor transforms will be executed first\n in dependency order. If any ancestor fails, execution stops and remaining ancestors\n are marked as not_run. The target transform will not run if any ancestor failed.",
"description" : "Run a transform in a workspace.\n\n App DB changes are rolled back. Warehouse DB changes persist.",
"parameters" : [ {
"in" : "path",
"name" : "ws-id",
@@ -32030,20 +31909,6 @@
"description" : "Server error response"
}
},
"requestBody" : {
"content" : {
"application/json" : {
"schema" : {
"type" : "object",
"properties" : {
"run_stale_ancestors" : {
"$ref" : "#/components/schemas/metabase-enterprise.workspaces.types.flag"
}
}
}
}
}
},
"tags" : [ "/api/ee/workspace" ]
}
},
@@ -32109,9 +31974,6 @@
"schema" : {
"type" : "object",
"properties" : {
"api_key" : {
"type" : "string"
},
"collection_id" : {
"$ref" : "#/components/schemas/metabase-enterprise.workspaces.types.appdb-id"
},
@@ -45894,7 +45756,8 @@
"source" : {
"default" : "admin",
"oneOf" : [ {
"type" : "string"
"type" : "string",
"minLength" : 1
}, {
"type" : "null"
} ]
@@ -45912,7 +45775,7 @@
"oneOf" : [ {
"type" : "array",
"items" : {
"$ref" : "#/components/schemas/metabase.users.schema.user-group-membership"
"$ref" : "#/components/schemas/metabase.users-rest.api.user-group-membership"
}
}, {
"type" : "null"
@@ -46034,7 +45897,7 @@
"oneOf" : [ {
"type" : "array",
"items" : {
"$ref" : "#/components/schemas/metabase.users.schema.user-group-membership"
"$ref" : "#/components/schemas/metabase.users-rest.api.user-group-membership"
}
}, {
"type" : "null"

View File

@@ -288,7 +288,6 @@ config:
surveys-enabled: true
sync-leaf-fields-limit: 1000
synchronous-batch-updates: false
thread-interrupt-escalation-timeout-ms: 0
transforms-enabled: false
unaggregated-query-row-limit: null
uploads-settings: null

View File

@@ -1997,16 +1997,6 @@ Maximum number of leaf fields synced per collection of document database. Curren
Process batches updates synchronously. If true, all `submit!` calls will be processed immediately. Default is false.
### `MB_THREAD_INTERRUPT_ESCALATION_TIMEOUT_MS`
- Type: integer
- Default: `0`
By default, this is 0 and the thread interrupt escalation does not run.
Timeout in milliseconds to wait after query cancellation before escalating to thread interruption.
This is used to free up threads that are stuck waiting for a DB response after a query has been cancelled.
### `MB_TRANSFORMS_ENABLED`
- Type: boolean
@@ -2402,20 +2392,6 @@ Default: True
Whether to include the Sample Database in your Metabase. To exclude the Sample Database, set `MB_LOAD_SAMPLE_CONTENT=false`.
### `MB_MONITOR_PERFORMANCE`
Type: string<br>
Default: `""`
When set, starts a Java Flight Recorder (JFR) recording at startup that can be analyzed with JDK Mission Control or other JFR tools.
- `"true"` generates a timestamped output file like `metabase-2026_01_15.jfr`
- Any other non-empty value is used as the output filename (`.jfr` extension is appended if missing)
- `""` or `"false"` disables monitoring (the default)
The performance recording stores only method signature calls and other code execution metrics.
It does not store any sensitive information such as environment variables, system properties, or other machine information.
### `MB_NO_SURVEYS`
Type: boolean<br>

View File

@@ -1,23 +0,0 @@
collection_id: null
created_at: '2026-01-30T15:33:24.179945Z'
description: null
entity_id: BssO7GsawP2WszjcnIjMw
name: Imported Simple SQL transform
source:
query:
database: Writable Postgres12
native:
query: select 1;
type: native
type: query
source_database_id: Writable Postgres12
tags: []
target:
database: Writable Postgres12
name: imported_transform
schema: public
type: table
serdes/meta:
- id: BssO7GsawP2WszjcnIjMw
label: imported_transform
model: Transform

View File

@@ -3,7 +3,6 @@ export { addQuestionToDashboard } from "./addQuestionToDashboard";
export { archiveCollection } from "./archiveCollection";
export { archiveDashboard } from "./archiveDashboard";
export { archiveQuestion } from "./archiveQuestion";
export { updateSnippet } from "./updateSnippet";
export { createApiKey } from "./createApiKey";
export { createCard } from "./createCard";
export { createCollection } from "./createCollection";

View File

@@ -1,12 +0,0 @@
import type {
NativeQuerySnippet,
NativeQuerySnippetId,
} from "metabase-types/api";
export const updateSnippet = (
id: NativeQuerySnippetId,
data: Partial<NativeQuerySnippet>,
) => {
cy.log(`Updating a snippet with id: ${id}`);
return cy.request("PUT", `/api/native-query-snippet/${id}`, data);
};

View File

@@ -12,7 +12,6 @@ const { H } = cy;
const libraryPage = () => cy.findByTestId("library-page");
const newSnippetPage = () => cy.findByTestId("new-snippet-page");
const archivedSnippetsPage = () => cy.findByTestId("archived-snippets-page");
const editSnippetPage = () => cy.findByTestId("edit-snippet-page");
const metricOverviewPage = () => cy.findByTestId("metric-overview-page");
const metricQueryEditor = () => cy.findByTestId("metric-query-editor");
@@ -83,7 +82,6 @@ export const DataStudio = {
Snippets: {
newPage: newSnippetPage,
editPage: editSnippetPage,
archivedPage: archivedSnippetsPage,
nameInput: () => newSnippetPage().findByDisplayValue("New SQL snippet"),
descriptionInput: () => cy.findByPlaceholderText("No description"),
saveButton: () => cy.findByRole("button", { name: "Save" }),

View File

@@ -16,9 +16,6 @@ export const LOCAL_GIT_PATH =
export const SYNCED_COLLECTION_FIXTURE_PATH =
Cypress.config("projectRoot") +
"/e2e/support/assets/example_synced_collection";
export const SYNCED_TRANSFORMS_COLLECTION_FIXTURE_PATH =
Cypress.config("projectRoot") +
"/e2e/support/assets/example_synced_transforms_collection";
// Copy the sample synced collection from the fixture folder to the working directory
export const copySyncedCollectionFixture = () => {
@@ -27,13 +24,6 @@ export const copySyncedCollectionFixture = () => {
destination: LOCAL_GIT_PATH,
});
};
// Copy the sample synced transforms collection from the fixture folder to the working directory
export const copySyncedTransformsCollectionFixture = () => {
cy.task("copyDirectory", {
source: SYNCED_TRANSFORMS_COLLECTION_FIXTURE_PATH,
destination: LOCAL_GIT_PATH,
});
};
export const checkoutSyncedCollectionBranch = (branch: string) => {
cy.exec("git -C " + LOCAL_GIT_PATH + ` checkout -b '${branch}'`);
@@ -258,7 +248,7 @@ export const interceptTask = () =>
export const waitForTask = (
{ taskName }: { taskName: "import" | "export" },
retries = 0,
): Cypress.Chainable => {
) => {
if (retries > 3) {
throw Error(`Too many retries waiting for ${taskName}`);
}
@@ -277,7 +267,7 @@ export const waitForTask = (
export const pollForTask = (
{ taskName }: { taskName: "import" | "export" },
retries = 0,
): Cypress.Chainable => {
) => {
if (retries > 30) {
throw Error(`Too many retries waiting for ${taskName}`);
}
@@ -307,13 +297,6 @@ export const pollForTask = (
`Task ${taskName} failed: ${body.error_message || "Unknown error"}`,
);
}
if (body.status === "conflict") {
throw Error(
`Task ${taskName} returned conflict: ${body.error_message || "Unknown error"}`,
);
}
cy.wait(500);
return pollForTask({ taskName }, retries + 1);
}

View File

@@ -127,9 +127,7 @@ export function createSqlTransform({
tagIds,
visitTransform,
sourceCheckpointStrategy,
name = "SQL transform",
}: {
name?: string;
sourceQuery: string;
targetTable: string;
targetSchema: string;
@@ -139,7 +137,7 @@ export function createSqlTransform({
}) {
return createTransform(
{
name,
name: "SQL transform",
source: {
type: "query",
query: {

View File

@@ -86,7 +86,6 @@ export function chartPathsWithFillColors(colors) {
}
const CIRCLE_PATH = "M1 0A1 1 0 1 1 1 -0.0001";
const DIAMOND_PATH = "M0 -1L1 0L0 1L-1 0Z";
export function cartesianChartCircle() {
return echartsContainer()
.find(`path[d="${CIRCLE_PATH}"]`)
@@ -262,17 +261,3 @@ export function assertEChartsTooltipNotContain(rows) {
});
});
}
export const BoxPlot = {
getBoxes() {
return echartsContainer().find('path[fill-opacity="0.15"][stroke]');
},
getPoints() {
return echartsContainer().find(`path[d="${CIRCLE_PATH}"]`);
},
getMeanMarkers() {
return echartsContainer().find(`path[d="${DIAMOND_PATH}"]`);
},
};

View File

@@ -68,35 +68,6 @@ describe("scenarios > embedding-sdk > interactive-question", () => {
});
});
it("should not show the expand button (metabase#68975)", () => {
mountInteractiveQuestion();
getSdkRoot().within(() => {
cy.findByText("Product ID").should("be.visible");
cy.findByText("Max of Quantity").should("be.visible");
cy.findByTestId("viz-settings-button").click();
H.popover().within(() => {
cy.findByText("Show row index").click();
});
// close the popover
cy.findByTestId("viz-settings-button").click();
cy.findByText("#").should("be.visible");
cy.findByTestId("table-body")
.get("[data-index='0']")
.within(() => {
cy.get("[data-column-id$='_INDEX']")
.realHover({ scrollBehavior: false })
.findByTestId("detail-shortcut")
.should("not.exist");
});
});
});
it("should show a watermark in development mode", () => {
cy.intercept("/api/session/properties", (req) => {
req.continue((res) => {

View File

@@ -1,4 +1,3 @@
import { WRITABLE_DB_ID } from "e2e/support/cypress_data";
import { SAMPLE_DATABASE } from "e2e/support/cypress_sample_database";
import { ORDERS_DASHBOARD_ID } from "e2e/support/cypress_sample_instance_data";
import type { Collection } from "metabase-types/api";
@@ -13,7 +12,7 @@ const REMOTE_QUESTION_NAME = "Remote Sync Test Question";
describe("Remote Sync", () => {
beforeEach(() => {
H.restore("postgres-writable");
H.restore();
H.resetSnowplow();
cy.signInAsAdmin();
H.activateToken("bleeding-edge");
@@ -872,99 +871,6 @@ describe("Remote Sync", () => {
});
});
});
describe("initial pull conflict handling", () => {
beforeEach(() => {
// Create a local transform that could be overwritten by the remote
H.createSqlTransform({
sourceQuery: "SELECT 1",
targetTable: "existing_transform",
targetSchema: "public",
name: "Batman's Existing Transform",
});
// Create the target table for the imported transform
H.queryWritableDB(
"CREATE TABLE IF NOT EXISTS imported_transform (column1 INT);",
);
// Resync the database so Metabase knows about the new table
H.resyncDatabase({
dbId: WRITABLE_DB_ID,
tableName: "imported_transform",
});
// Add collection to remote repository
H.copySyncedTransformsCollectionFixture();
H.commitToRepo();
// Set up in read-write mode without marking anything as synced and pull changes
H.configureGit("read-write");
});
it("shows conflict modal with available options when remote would override local", () => {
H.DataStudio.Transforms.visit();
cy.findByRole("treegrid").within(() => {
cy.findByText("Batman's Existing Transform").should("be.visible");
});
H.getPullOption().click();
cy.log("make sure conflict modal is displayed");
H.modal().within(() => {
cy.findByRole("heading", {
name: /Your local data will be overwritten by the remote branch/,
}).should("be.visible");
});
cy.findAllByRole("radio").should("have.length", 2);
cy.findByLabelText(/Create a new branch and push changes there/).should(
"be.visible",
);
cy.log("choose the delete option and pull");
cy.findByLabelText(/Delete unsynced changes/)
.should("be.visible")
.click();
cy.findByRole("button", { name: "Delete unsynced changes" }).click();
cy.findByRole("treegrid").within(() => {
cy.log(
"check existing transform was removed after pulling from remote",
);
cy.findByText("Batman's Existing Transform").should("not.exist");
cy.log("check remote transform was pulled in");
cy.findByText("Imported Simple SQL transform").should("be.visible");
});
});
it("can push to a new branch", () => {
cy.intercept("POST", "/api/ee/remote-sync/export").as("exportChanges");
H.DataStudio.Transforms.visit();
H.getPullOption().click();
cy.log("choose the new branch option and push");
cy.findByLabelText(/Create a new branch and push changes there/)
.should("be.visible")
.click();
cy.findByLabelText("Name for your new branch")
.should("be.visible")
.type("new-branch");
cy.findByRole("button", { name: "Push changes" }).click();
cy.wait("@exportChanges");
H.waitForTask({ taskName: "export" });
H.getGitSyncControls().should("have.text", "new-branch");
cy.findByRole("treegrid").within(() => {
cy.findByText("Batman's Existing Transform").should("be.visible");
});
});
});
});
const ensureSyncedCollectionIsVisible = () => {

View File

@@ -145,7 +145,9 @@ describe("scenarios > data studio > library", () => {
cy.log("Verify Data section empty state");
H.DataStudio.Library.libraryPage()
.findByText("Cleaned, pre-transformed data sources ready for exploring")
.findByText(
"Cleaned, pre-transformed data sources ready for exploring.",
)
.should("be.visible");
H.DataStudio.Library.libraryPage()
.findByRole("button", { name: "Publish a table" })
@@ -153,7 +155,7 @@ describe("scenarios > data studio > library", () => {
cy.log("Verify Metrics section empty state");
H.DataStudio.Library.libraryPage()
.findByText("Standardized calculations with known dimensions")
.findByText("Standardized calculations with known dimensions.")
.should("be.visible");
H.DataStudio.Library.libraryPage()
.findByRole("link", { name: "New metric" })
@@ -161,7 +163,7 @@ describe("scenarios > data studio > library", () => {
cy.log("Verify SQL snippets section empty state");
H.DataStudio.Library.libraryPage()
.findByText("Reusable bits of code that save your time")
.findByText("Reusable bits of code that save your time.")
.should("be.visible");
H.DataStudio.Library.libraryPage()
.findByRole("link", { name: "New snippet" })
@@ -191,7 +193,9 @@ describe("scenarios > data studio > library", () => {
cy.log("Verify Data empty state is visible initially");
H.DataStudio.Library.libraryPage()
.findByText("Cleaned, pre-transformed data sources ready for exploring")
.findByText(
"Cleaned, pre-transformed data sources ready for exploring.",
)
.should("be.visible");
cy.log("Publish a table via the +New menu");
@@ -216,10 +220,10 @@ describe("scenarios > data studio > library", () => {
"Verify Metrics and SQL snippets still show empty states (always expanded behavior)",
);
H.DataStudio.Library.libraryPage()
.findByText("Standardized calculations with known dimensions")
.findByText("Standardized calculations with known dimensions.")
.should("be.visible");
H.DataStudio.Library.libraryPage()
.findByText("Reusable bits of code that save your time")
.findByText("Reusable bits of code that save your time.")
.should("be.visible");
});
});

View File

@@ -177,8 +177,8 @@ describe("scenarios > data studio > snippets", () => {
});
});
describe("archiving", () => {
it("should be able to archive a snippet", () => {
describe("deletion", () => {
it("should be able to delete a snippet", () => {
H.createSnippet({
name: "Test snippet",
content: "SELECT * FROM orders",
@@ -191,11 +191,11 @@ describe("scenarios > data studio > snippets", () => {
cy.findByTestId("snippet-header")
.findByRole("button", { name: /Snippet menu options/ })
.click();
H.popover().findByText("Archive").click();
H.popover().findByText("Delete").click();
H.modal().within(() => {
cy.findByText("Archive snippet?").should("be.visible");
cy.button("Archive").click();
cy.findByText("Delete snippet?").should("be.visible");
cy.button("Delete").click();
cy.wait("@updateSnippet");
});
@@ -203,48 +203,6 @@ describe("scenarios > data studio > snippets", () => {
.findByText("Test snippet")
.should("not.exist");
});
it("should be able to unarchive a snippet", () => {
H.createSnippet({
name: "Test snippet",
content: "SELECT * FROM orders",
}).then((response) => {
cy.log("Archive snippet");
return H.updateSnippet(response.body.id, { archived: true });
});
H.DataStudio.Library.visit();
H.DataStudio.Library.libraryPage()
.findByRole("button", { name: "Snippet collection options" })
.click();
H.popover()
.findByText(/View archived snippets/)
.click();
cy.url().should("include", "/snippets/archived");
H.DataStudio.Snippets.archivedPage()
.findByText("Test snippet")
.should("be.visible");
H.DataStudio.Snippets.archivedPage()
.findByRole("button", { name: "Unarchive snippet" })
.click();
cy.wait("@updateSnippet");
H.DataStudio.Snippets.archivedPage()
.findByText("Test snippet")
.should("not.exist");
H.DataStudio.Library.visit();
H.DataStudio.Library.libraryPage()
.findByText("Test snippet")
.should("be.visible");
});
});
describe("snippet folders", () => {

View File

@@ -1570,7 +1570,7 @@ describe("scenarios > data studio > workspaces", () => {
});
});
describe("check out transform into a workspace", () => {
describe("transform -> workspace > ", () => {
it("should check out transform into a new workspace from the transform page", () => {
cy.log("Create 2 workspaces, add transform to the second one");
createTransforms();
@@ -2265,7 +2265,7 @@ describe("scenarios > data studio > workspaces", () => {
});
});
describe("repros", () => {
describe("repros > ", () => {
it("should not show error when editing a new transform in a workspace (GDGT-1445)", () => {
Workspaces.visitTransformListPage();
cy.findByLabelText("Create a transform").click();
@@ -2441,30 +2441,6 @@ describe("scenarios > data studio > workspaces", () => {
"Last ran a few seconds ago successfully.",
);
});
it("should immediately open transform tab and show a loading state (GDGT-1531)", () => {
createTransforms();
Workspaces.visitWorkspaces();
createWorkspace();
cy.intercept("GET", "/api/transform/*", (request) => {
request.reply((response) => {
response.setDelay(2000);
response.send();
});
}).as("getTransform");
cy.log("Create first SQL transform that will succeed");
Workspaces.getMainlandTransforms().findByText("SQL transform").click();
Workspaces.getWorkspaceContent().within(() => {
H.tabsShouldBe("SQL transform", ["Setup", "Graph", "SQL transform"]);
cy.findByTestId("loading-indicator").should("be.visible");
});
cy.wait("@getTransform");
cy.findByTestId("loading-indicator").should("not.exist");
});
});
});

View File

@@ -1,479 +0,0 @@
const { H } = cy;
import { SAMPLE_DB_ID } from "e2e/support/cypress_data";
import { SAMPLE_DATABASE } from "e2e/support/cypress_sample_database";
const { ORDERS, ORDERS_ID, PRODUCTS, PRODUCTS_ID } = SAMPLE_DATABASE;
const testQuery = {
type: "query",
database: SAMPLE_DB_ID,
query: {
"source-table": ORDERS_ID,
aggregation: [["count"]],
breakout: [
["field", ORDERS.CREATED_AT, { "temporal-unit": "year" }],
[
"field",
ORDERS.TOTAL,
{ binning: { strategy: "num-bins", "num-bins": 50 } },
],
],
},
};
const singleSeriesQuestion = {
dataset_query: testQuery,
display: "boxplot",
visualization_settings: {
"graph.dimensions": ["CREATED_AT"],
},
};
describe("scenarios > visualizations > boxplot", () => {
beforeEach(() => {
H.restore();
cy.signInAsNormalUser();
});
it("should render boxplot and update chart on display settings changes", () => {
H.visitQuestionAdhoc(singleSeriesQuestion);
// 5 Boxes: 2022-2026
H.BoxPlot.getBoxes().should("have.length", 5);
// By default: Tukey whiskers, outliers shown, mean shown
H.BoxPlot.getPoints().should("have.length", 1); // Only one outlier
H.BoxPlot.getMeanMarkers().should("have.length", 5);
// Open settings and change whisker type to Min/Max
H.openVizSettingsSidebar();
H.leftSidebar().within(() => {
cy.findByText("Display").click();
cy.findByText("Whiskers extend to").should("exist");
cy.findByText("1.5 × interquartile range").should("exist");
cy.findByText("Min/Max").click();
});
// With Min/Max whiskers, there should be no outliers
H.BoxPlot.getPoints().should("not.exist");
// "Outliers only" option should be hidden when Min/Max is selected
H.leftSidebar().findByText("Outliers only").should("not.exist");
// Change points mode to show all points
H.leftSidebar().findByText("All points").click();
H.BoxPlot.getPoints().should("have.length.at.least", 5);
// Hide points entirely
H.leftSidebar().findByText("None").click();
H.BoxPlot.getPoints().should("not.exist");
// Toggle mean off
H.leftSidebar().findByText("Show mean").click();
H.BoxPlot.getMeanMarkers().should("not.exist");
// Toggle mean back on
H.leftSidebar().findByText("Show mean").click();
H.BoxPlot.getMeanMarkers().should("have.length", 5);
});
it("should show and configure data labels", () => {
H.visitQuestionAdhoc(singleSeriesQuestion);
H.openVizSettingsSidebar();
H.leftSidebar().within(() => {
cy.findByText("Display").click();
cy.findByText("Show values on data points").click();
});
// After enabling, "Values to display" option appears with segmented buttons
H.leftSidebar().within(() => {
cy.findByText("Values to display").should("exist");
cy.findByRole("button", { name: "Median only" }).should("exist");
cy.findByRole("button", { name: "All" }).click();
});
// Verify label value appears
H.echartsContainer().findByText("412").should("exist");
// Disable "Hide overlapping labels" to show more labels
H.leftSidebar().findByText("Hide overlapping labels").click();
H.echartsContainer().findByText("91.75").should("exist");
// Test "Auto formatting" segmented button
H.leftSidebar().within(() => {
cy.findByText("Auto formatting").should("exist");
cy.findByRole("button", { name: "Full" }).click();
});
});
it("should display tooltips on hover", () => {
H.visitQuestionAdhoc(singleSeriesQuestion);
// Hover over a box element from the left side to avoid mean marker overlap
H.BoxPlot.getBoxes().first().trigger("mousemove", "left");
H.assertEChartsTooltip({
header: "2022",
rows: [
{ name: "Upper whisker", value: "84" },
{ name: "Q3 (75th percentile)", value: "59" },
{ name: "Median", value: "35" },
{ name: "Mean", value: "39.16" },
{ name: "Q1 (25th percentile)", value: "15.5" },
{ name: "Lower whisker", value: "1" },
],
});
// Hover over the outlier point
H.BoxPlot.getPoints().first().trigger("mousemove");
H.assertEChartsTooltip({
header: "2026 (outlier)",
rows: [
{ name: "Count", value: "189" },
{ name: "Total: 50 bins", value: "70 – 75" },
],
});
});
it("should support axis customization", () => {
H.visitQuestionAdhoc(singleSeriesQuestion);
H.openVizSettingsSidebar();
H.leftSidebar().findByText("Axes").click();
// Add y-axis label
H.leftSidebar().within(() => {
cy.findByDisplayValue("Count").clear().type("Count Label");
});
H.echartsContainer().findByText("Count Label").should("exist");
// Add x-axis label
H.leftSidebar().within(() => {
cy.findByDisplayValue("Created At: Year").clear().type("Year Label");
});
H.echartsContainer().findByText("Year Label").should("exist");
// Toggle auto y-axis range
// Before toggling auto y-axis range, the y-axis labels contains 600
H.echartsContainer().findByText("600");
H.leftSidebar().findByText("Auto y-axis range").click();
H.echartsContainer().within(() => {
cy.findByText("100"); // Y-axis label since default non-auto range is [0, 100]
cy.findByText("600").should("not.exist");
});
});
it("should display goal line when configured", () => {
H.visitQuestionAdhoc(singleSeriesQuestion);
H.openVizSettingsSidebar();
H.leftSidebar().findByText("Display").click();
H.leftSidebar().findByText("Goal line").click();
H.leftSidebar().within(() => {
cy.findByLabelText("Goal value").clear().type("100");
cy.findByLabelText("Goal label").clear().type("Target");
});
H.echartsContainer().findByText("Target").should("exist");
H.goalLine().should("exist");
});
it("should render in dashboard and support drill-through on boxes and outliers", () => {
H.createQuestionAndDashboard({
questionDetails: {
name: "BoxPlot Dashboard Test",
query: singleSeriesQuestion.dataset_query.query,
display: "boxplot",
visualization_settings: singleSeriesQuestion.visualization_settings,
},
}).then(({ body: dashcard }) => {
H.visitDashboard(dashcard.dashboard_id);
});
H.echartsContainer().should("be.visible");
// Verify boxplot renders in dashboard context (5 boxes: 2022-2026)
H.BoxPlot.getBoxes().should("have.length", 5);
H.BoxPlot.getMeanMarkers().should("have.length", 5);
// Click on a box to trigger drill-through (click left to avoid mean marker overlap)
H.BoxPlot.getBoxes().first().click("left");
// Click "See these Orders" to drill down
H.popover().findByText("See these Orders").click();
// Should navigate to filtered table view with dimension filter applied
cy.findByTestId("filter-pill").should(
"have.text",
"Created At: Year is Jan 1 – Dec 31, 2022",
);
// Verify we're viewing a table with results
H.tableInteractiveBody().contains("79.37").should("exist");
// Go back to dashboard
cy.findByLabelText("Back to Test Dashboard").click();
// Click on an outlier point
H.BoxPlot.getPoints().first().click();
// Should show drill options
H.popover().within(() => {
cy.findByText("See these Orders").should("exist");
});
H.popover().findByText("See these Orders").click();
// Should filter to specific dimension and show table
cy.findByTestId("filter-pill").should(
"have.text",
"Created At: Year is Jan 1 – Dec 31, 2026",
);
// Verify we're viewing a table with results
H.tableInteractiveBody().contains("97.44").should("exist");
});
describe("multi-series", () => {
const breakoutQuery = {
type: "query",
database: SAMPLE_DB_ID,
query: {
"source-table": ORDERS_ID,
aggregation: [["count"]],
breakout: [
["field", ORDERS.CREATED_AT, { "temporal-unit": "year" }],
[
"field",
ORDERS.TOTAL,
{ binning: { strategy: "num-bins", "num-bins": 50 } },
],
["field", PRODUCTS.CATEGORY, { "source-field": ORDERS.PRODUCT_ID }],
],
},
};
const twoMetricsQuery = {
type: "query",
database: SAMPLE_DB_ID,
query: {
"source-table": ORDERS_ID,
aggregation: [["count"], ["sum", ["field", ORDERS.QUANTITY, null]]],
breakout: [
["field", ORDERS.CREATED_AT, { "temporal-unit": "year" }],
[
"field",
ORDERS.TOTAL,
{ binning: { strategy: "num-bins", "num-bins": 50 } },
],
],
},
};
it("should render boxplot with breakout (two dimensions, one metric)", () => {
H.visitQuestionAdhoc({
dataset_query: breakoutQuery,
display: "boxplot",
});
// Select dimensions via UI
H.openVizSettingsSidebar();
H.leftSidebar().findByText("Data").click();
// Configure dimensions
H.leftSidebar().findAllByPlaceholderText("Select a field").eq(0).click();
H.popover().findByText("Created At: Year").click();
H.leftSidebar().findByText("Add series breakout").click();
H.popover().findByText("Product → Category").click();
// Configure metrics
H.leftSidebar().findAllByPlaceholderText("Select a field").eq(2).click();
H.popover().findByText("Count").click();
// Should have legend items for each category
cy.findAllByTestId("legend-item").should("have.length", 4);
cy.findAllByTestId("legend-item").should("contain", "Doohickey");
cy.findAllByTestId("legend-item").should("contain", "Gadget");
cy.findAllByTestId("legend-item").should("contain", "Gizmo");
cy.findAllByTestId("legend-item").should("contain", "Widget");
// Should have 5 boxes per category (5 years × 4 categories = 20 boxes)
H.BoxPlot.getBoxes().should("have.length", 20);
// Hover over a box and verify tooltip shows breakout value
H.BoxPlot.getBoxes().first().trigger("mousemove", "left");
H.assertEChartsTooltip({
header: "2022",
rows: [
{ name: "Product → Category", value: "Gadget" },
{ name: "Upper whisker", value: "25" },
{ name: "Q3 (75th percentile)", value: "18.25" },
{ name: "Median", value: "12" },
{ name: "Mean", value: "12.44" },
{ name: "Q1 (25th percentile)", value: "6.5" },
{ name: "Lower whisker", value: "1" },
],
});
// Hide first series (Doohickey)
cy.findAllByTestId("legend-item")
.eq(0)
.findByLabelText("Hide series")
.click();
H.BoxPlot.getBoxes().should("have.length", 15);
// Show it back
cy.findAllByTestId("legend-item")
.eq(0)
.findByLabelText("Show series")
.click();
// Move mouse away to reset focus/blur state
H.echartsContainer().realHover({ position: "top" });
H.BoxPlot.getBoxes().should("have.length", 20);
// Verify drill-through includes both dimension and breakout filters
H.BoxPlot.getBoxes().first().click("left");
H.popover().findByText("See these Orders").click();
cy.findAllByTestId("filter-pill").should("have.length", 2);
cy.findAllByTestId("filter-pill")
.eq(0)
.should("have.text", "Created At: Year is Jan 1 – Dec 31, 2022");
cy.findAllByTestId("filter-pill")
.eq(1)
.should("have.text", "Product → Category is Doohickey");
H.tableInteractiveBody().should("exist");
});
it("should render boxplot with two metrics", () => {
H.visitQuestionAdhoc({
dataset_query: twoMetricsQuery,
display: "boxplot",
});
// Select via UI
H.openVizSettingsSidebar();
H.leftSidebar().findByText("Data").click();
// Configure dimension
H.leftSidebar().findAllByPlaceholderText("Select a field").eq(0).click();
H.popover().findByText("Created At: Year").click();
// Should have legend items for each metric
cy.findAllByTestId("legend-item").should("have.length", 2);
cy.findAllByTestId("legend-item").should("contain", "Count");
cy.findAllByTestId("legend-item").should("contain", "Sum of Quantity");
// Should have 5 boxes per metric (5 years × 2 metrics = 10 boxes)
H.BoxPlot.getBoxes().should("have.length", 10);
// Hover over a box from the first metric and verify tooltip
H.BoxPlot.getBoxes().first().trigger("mousemove", "left");
H.assertEChartsTooltip({
header: "2022",
rows: [
{ name: "Upper whisker", value: "84" },
{ name: "Q3 (75th percentile)", value: "59" },
{ name: "Median", value: "35" },
{ name: "Mean", value: "39.16" },
{ name: "Q1 (25th percentile)", value: "15.5" },
{ name: "Lower whisker", value: "1" },
],
});
// Hide first series (Count)
cy.findAllByTestId("legend-item")
.eq(0)
.findByLabelText("Hide series")
.click();
H.BoxPlot.getBoxes().should("have.length", 5);
// Show it back
cy.findAllByTestId("legend-item")
.eq(0)
.findByLabelText("Show series")
.click();
// Move mouse away to reset focus/blur state
H.echartsContainer().realHover({ position: "top" });
H.BoxPlot.getBoxes().should("have.length", 10);
// Verify drill-through applies dimension filter
H.BoxPlot.getBoxes().first().click("left");
H.popover().findByText("See these Orders").click();
cy.findByTestId("filter-pill").should(
"have.text",
"Created At: Year is Jan 1 – Dec 31, 2022",
);
H.tableInteractiveBody().should("exist");
});
});
it("should support value formatting, axis settings, and series customization", () => {
// Use Products table: Average of Price by Category and Created at (year)
const priceByYearQuery = {
type: "query",
database: SAMPLE_DB_ID,
query: {
"source-table": PRODUCTS_ID,
aggregation: [["avg", ["field", PRODUCTS.PRICE, null]]],
breakout: [
["field", PRODUCTS.CREATED_AT, { "temporal-unit": "year" }],
["field", PRODUCTS.CATEGORY, null],
],
},
};
H.visitQuestionAdhoc({
dataset_query: priceByYearQuery,
display: "boxplot",
visualization_settings: {
"graph.dimensions": ["CREATED_AT"],
"graph.metrics": ["avg"],
"graph.show_values": true,
},
});
H.BoxPlot.getBoxes().should("have.length", 4);
// Set currency formatting via UI using series settings
H.openVizSettingsSidebar();
H.openSeriesSettings("Average of Price");
H.popover().within(() => {
cy.findByText("Formatting").click();
cy.findByLabelText("Style").click();
cy.findByText("Currency").click();
});
// Verify formatted value appears in labels (should have $ prefix)
H.echartsContainer().findByText("$52.13").should("be.visible");
cy.button("Done").click();
// Verify formatted tooltip with all currency values
H.BoxPlot.getBoxes().first().trigger("mousemove", "left");
H.assertEChartsTooltip({
header: "2022",
rows: [
{ name: "Upper whisker", value: "$53.93" },
{ name: "Q3 (75th percentile)", value: "$53.93" },
{ name: "Median", value: "$52.13" },
{ name: "Mean", value: "$52.94" },
{ name: "Q1 (25th percentile)", value: "$51.13" },
{ name: "Lower whisker", value: "$48.62" },
],
});
H.openVizSettingsSidebar();
H.leftSidebar().findByText("Axes").click();
// Default: pinned to zero, y-axis should include 0
H.echartsContainer().findByText("$0").should("be.visible");
H.leftSidebar().findByText("Unpin from zero").click();
// After unpinning, 0 should not be visible (y-axis starts higher since prices are ~$40-80)
H.echartsContainer().findByText("$0").should("not.exist");
});
});

View File

@@ -6,7 +6,7 @@
[metabase.api.macros :as api.macros]
[metabase.audit-app.core :as audit]
[metabase.models.interface :as mi]
[metabase.users.core :as users]
[metabase.users-rest.api :as api.user]
[metabase.util :as u]
[metabase.util.malli.schema :as ms]
[toucan2.core :as t2]))
@@ -39,7 +39,7 @@
Archive all Alerts and DashboardSubscriptions created by the User. Only allowed for admins or for the current user."
[{:keys [id]} :- [:map
[:id ms/PositiveInt]]]
(users/check-self-or-superuser id)
(api.user/check-self-or-superuser id)
;; delete all `PulseChannelRecipient` rows for this User, which means they will no longer receive any
;; Alerts/DashboardSubscriptions
(t2/delete! :model/PulseChannelRecipient :user_id id)

View File

@@ -8,7 +8,6 @@
[metabase.api.routes.common :refer [+auth]]
[metabase.collections.core :as collection]
[metabase.events.core :as events]
[metabase.models.interface :as mi]
[metabase.util.i18n :refer [tru]]
[metabase.util.malli :as mu]
[metabase.util.malli.registry :as mr]
@@ -114,19 +113,6 @@
[:map
[:target_collection [:maybe (ms/InstanceOf :model/Collection)]]])
(defn- can-publish?
"Publishing a table means that it's now query-able by a new set of people. So we should not allow you to publish a
table if you don't *already* have permissions to query it - otherwise, maybe you can just publish it to circumvent your
lack of query permissions."
[table]
(and (mi/can-write? table) (mi/can-query? table)))
(defn- can-publish-all-tables?
"This function returns `true` iff you have permission to publish every table passed."
[table-ids]
(every? can-publish? (when (seq table-ids)
(t2/select :model/Table :id [:in table-ids]))))
(api.macros/defendpoint :post "/publish-tables" :- ::publish-tables-response
"Set collection for each of selected tables and all upstream dependencies recursively."
[_route-params
@@ -147,7 +133,6 @@
where)
;; Get table IDs before update for event publishing
table-ids-to-update (t2/select-pks-set :model/Table {:where update-where})]
(api/check-403 (can-publish-all-tables? table-ids-to-update))
(t2/query {:update (t2/table-name :model/Table)
:set {:collection_id (:id target-collection)
:is_published true}
@@ -173,7 +158,6 @@
where)
;; Get table IDs before update for event publishing
table-ids-to-update (t2/select-pks-set :model/Table {:where update-where})]
(api/check-403 (can-publish-all-tables? table-ids-to-update))
(t2/query {:update (t2/table-name :model/Table)
:set {:collection_id nil
:is_published false}

View File

@@ -46,18 +46,3 @@
[table]
(when (:is_published table)
(mi/current-user-has-full-permissions? (perms/perms-objects-set-for-parent-collection table :read))))
(defenterprise published-table-visible-clause
"Returns a HoneySQL clause matching published tables that are readable via collection permissions."
:feature :library
[table-id-column {:keys [user-id is-superuser?]}]
[:in table-id-column
{:select [:id]
:from [:metabase_table]
:where [:and
[:= :is_published true]
(collection/visible-collection-filter-clause
:collection_id
{}
{:current-user-id user-id
:is-superuser? is-superuser?})]}])

View File

@@ -8,7 +8,7 @@
[metabase.settings.core :as setting]
[metabase.util :as u]
[metabase.util.malli.schema :as ms]
[metabase.warehouses.core :as warehouses]
[metabase.warehouses-rest.api :as api.database]
[toucan2.core :as t2]))
;; TODO (Cam 10/28/25) -- fix this endpoint so it uses kebab-case for query parameters for consistency with the rest
@@ -42,7 +42,7 @@
(if-let [invalid-destinations (and check_connection_details
(->> destinations
(keep (fn [{details :details n :name}]
(let [details-or-error (warehouses/test-connection-details engine details)
(let [details-or-error (api.database/test-connection-details (name engine) details)
valid? (not= (:valid details-or-error) false)]
(when-not valid?
[n (dissoc details-or-error :valid)]))))

View File

@@ -27,12 +27,11 @@
becomes a serious enough issue, at which point we will have to redesign version marking and
analysis triggering."
{:style/indent 0}
[entity-type entity-id & body]
[& body]
`(try
~@body
(catch Throwable e#
(log/error e# "Dependency calculation failed" {:entity-type ~entity-type
:entity-id ~entity-id})
(log/error e# "Dependency calculation failed")
nil)))
;; ### Cards
@@ -47,8 +46,8 @@
(when (premium-features/has-feature? :dependencies)
(t2/with-transaction [_conn]
(models.dependency/replace-dependencies! :card (:id object)
(ignore-errors :card (:id object)
(deps.calculation/upstream-deps:card object)))
(ignore-errors
(deps.calculation/upstream-deps:card object)))
(when (not= (:dependency_analysis_version object) models.dependency/current-dependency-analysis-version)
(t2/update! :model/Card (:id object)
{:dependency_analysis_version models.dependency/current-dependency-analysis-version})))))
@@ -71,8 +70,8 @@
(when (premium-features/has-feature? :dependencies)
(t2/with-transaction [_conn]
(models.dependency/replace-dependencies! :snippet (:id object)
(ignore-errors :snippet (:id object)
(deps.calculation/upstream-deps:snippet object)))
(ignore-errors
(deps.calculation/upstream-deps:snippet object)))
(when (not= (:dependency_analysis_version object) models.dependency/current-dependency-analysis-version)
(t2/update! :model/NativeQuerySnippet (:id object)
{:dependency_analysis_version models.dependency/current-dependency-analysis-version})))))
@@ -120,8 +119,8 @@
(when (premium-features/has-feature? :dependencies)
(t2/with-transaction [_conn]
(models.dependency/replace-dependencies! :transform (:id object)
(ignore-errors :transform (:id object)
(deps.calculation/upstream-deps:transform object)))
(ignore-errors
(deps.calculation/upstream-deps:transform object)))
(when (not= (:dependency_analysis_version object) models.dependency/current-dependency-analysis-version)
(t2/update! :model/Transform (:id object) {:dependency_analysis_version models.dependency/current-dependency-analysis-version}))
(drop-outdated-target-dep! object))))
@@ -170,8 +169,8 @@
:dashboardcard_id [:in (map :id dashcards)]))
dashboard (assoc object :dashcards dashcards :series-card-ids series-card-ids)]
(models.dependency/replace-dependencies! :dashboard dashboard-id
(ignore-errors :dashboard dashboard-id
(deps.calculation/upstream-deps:dashboard dashboard))))
(ignore-errors
(deps.calculation/upstream-deps:dashboard dashboard))))
(when (not= (:dependency_analysis_version object) models.dependency/current-dependency-analysis-version)
(t2/update! :model/Dashboard (:id object)
{:dependency_analysis_version models.dependency/current-dependency-analysis-version})))))
@@ -194,8 +193,8 @@
(when (premium-features/has-feature? :dependencies)
(t2/with-transaction [_conn]
(models.dependency/replace-dependencies! :document (:id object)
(ignore-errors :document (:id object)
(deps.calculation/upstream-deps:document object)))
(ignore-errors
(deps.calculation/upstream-deps:document object)))
(when (not= (:dependency_analysis_version object) models.dependency/current-dependency-analysis-version)
(t2/update! :model/Document (:id object)
{:dependency_analysis_version models.dependency/current-dependency-analysis-version})))))
@@ -217,8 +216,8 @@
[_ {:keys [object]}]
(when (premium-features/has-feature? :dependencies)
(t2/with-transaction [_conn]
(models.dependency/replace-dependencies! :sandbox (:id object) (ignore-errors :sandbox (:id object)
(deps.calculation/upstream-deps:sandbox object)))
(models.dependency/replace-dependencies! :sandbox (:id object) (ignore-errors
(deps.calculation/upstream-deps:sandbox object)))
(when (not= (:dependency_analysis_version object) models.dependency/current-dependency-analysis-version)
(t2/update! :model/Sandbox (:id object)
{:dependency_analysis_version models.dependency/current-dependency-analysis-version})))))
@@ -241,8 +240,8 @@
(when (premium-features/has-feature? :dependencies)
(t2/with-transaction [_conn]
(models.dependency/replace-dependencies! :segment (:id object)
(ignore-errors :segment (:id object)
(deps.calculation/upstream-deps:segment object)))
(ignore-errors
(deps.calculation/upstream-deps:segment object)))
(when (not= (:dependency_analysis_version object) models.dependency/current-dependency-analysis-version)
(t2/update! :model/Segment (:id object)
{:dependency_analysis_version models.dependency/current-dependency-analysis-version})))))
@@ -264,8 +263,8 @@
(when (premium-features/has-feature? :dependencies)
(t2/with-transaction [_conn]
(models.dependency/replace-dependencies! :measure (:id object)
(ignore-errors :measure (:id object)
(deps.calculation/upstream-deps:measure object)))
(ignore-errors
(deps.calculation/upstream-deps:measure object)))
(when (not= (:dependency_analysis_version object) models.dependency/current-dependency-analysis-version)
(t2/update! :model/Measure (:id object)
{:dependency_analysis_version models.dependency/current-dependency-analysis-version})))))

View File

@@ -9,7 +9,7 @@
[metabase.api.routes.common :refer [+auth]]
[metabase.query-processor :as qp]
[metabase.util.malli.schema :as ms]
[metabase.warehouses.core :as warehouses]))
[metabase.warehouses-rest.api :as warehouses]))
(set! *warn-on-reflection* true)

View File

@@ -1,12 +1,12 @@
(ns metabase-enterprise.metabot-v3.tools.invite-user
(:require
[metabase.users.core :as users]))
[metabase.users-rest.api :as api.user]))
(defn invite-user
"Send a Metabase invitation to the address `email`."
[{:keys [email]}]
(try
(users/invite-user! {:email email})
(api.user/invite-user {:email email})
{:output (format "%s has been invited to Metabase." email)}
(catch clojure.lang.ExceptionInfo e
(let [d (ex-data e)]

View File

@@ -23,15 +23,20 @@
[toucan2.core :as t2])
(:import (metabase_enterprise.remote_sync.source.protocol SourceSnapshot)))
(def ^:private transform-models
"Models that indicate transforms content in a snapshot."
#{"Transform" "TransformTag" "PythonLibrary"})
(defn- snapshot-has-transforms?
"Checks if the snapshot contains any Transform, TransformTag, or PythonLibrary entities.
Used to auto-enable remote-sync-transforms setting during import.
Uses the ingestable to list all entities and checks their :model metadata."
[ingestable]
(let [serdes-paths (serialization/ingest-list ingestable)
models-present (spec/models-in-import serdes-paths)]
(some spec/transform-models models-present)))
(let [serdes-paths (serialization/ingest-list ingestable)]
(some (fn [path]
(some #(transform-models (:model %)) path))
serdes-paths)))
(defn- sync-objects!
"Populates the remote-sync-object table with imported entities. Deletes all existing RemoteSyncObject records and
@@ -46,19 +51,6 @@
(when (seq all-inserts)
(t2/insert! :model/RemoteSyncObject all-inserts))))
(defn- build-entity-id-where-clause
"Builds a HoneySQL WHERE clause for entity_id filtering.
Combines the imported entity-ids exclusion with any spec-level entity_id condition."
[entity-ids spec-entity-id-condition]
(let [imported-condition (when (seq entity-ids)
[:not-in :entity_id entity-ids])
spec-condition (when spec-entity-id-condition
(let [[op value] spec-entity-id-condition]
[op :entity_id value]))
conditions (filterv some? [imported-condition spec-condition])]
(when (pos? (count conditions))
(into [:and] conditions))))
(defn- remove-unsynced!
"Deletes any remote sync content that was NOT part of the import.
@@ -76,32 +68,22 @@
(doseq [[model-key model-spec] (spec/specs-for-deletion)
:let [serdes-model (:model-type model-spec)
entity-ids (get by-entity-id serdes-model [])
spec-entity-id-condition (get-in model-spec [:conditions :entity_id])
entity-id-where (build-entity-id-where-clause entity-ids spec-entity-id-condition)
entity-id-clause (if (seq entity-ids)
[:not-in entity-ids]
:entity_id)
scope-key (get-in model-spec [:removal :scope-key])
;; Get non-entity_id conditions from spec
other-conditions (into [] cat (dissoc (:conditions model-spec) :entity_id))]]
extra-conditions (into [] cat (:conditions model-spec))]]
(if scope-key
;; Collection-scoped: delete only within synced collections
(when (seq synced-collection-ids)
(if entity-id-where
(apply t2/delete! model-key
{:where [:and
[:in scope-key synced-collection-ids]
entity-id-where]}
other-conditions)
(apply t2/delete! model-key
scope-key [:in synced-collection-ids]
other-conditions)))
;; Global: delete by entity_id
(if entity-id-where
(apply t2/delete! model-key
{:where entity-id-where}
other-conditions)
;; No entity_id conditions - delete all (respecting other spec conditions if any)
(if (seq other-conditions)
(apply t2/delete! model-key other-conditions)
(t2/delete! model-key))))))
scope-key [:in synced-collection-ids]
:entity_id entity-id-clause
extra-conditions))
;; Global: delete by entity_id only
(apply t2/delete! model-key
:entity_id entity-id-clause
extra-conditions))))
(defn source-error-message
"Constructs user-friendly error messages from remote sync source exceptions.
@@ -157,37 +139,6 @@
:details {:error-type (type e)}})))
(defn- get-conflicts
"Detects conflicts that would prevent or complicate import.
Returns a map with :conflicts (detailed list) and :summary (set of category names).
Conflict types detected:
- :entity-id-conflict - Items with existing entity IDs that are NOT already synced
- :library-conflict - First import only, local Library exists, import has Library
- :transforms-not-enabled - Import has Transform/TransformTag/PythonLibrary, setting disabled
- :snippets-without-library - Import has NativeQuerySnippet, Library not remote-synced"
[ingestable first-import?]
(let [ingest-list (serialization/ingest-list ingestable)
imported-data (spec/extract-imported-entities ingest-list)
models-present (spec/models-in-import ingest-list)
;; TODO (epaget 2026-02-02) -- entity-id conflict checking (detect unsynced local entities with matching entity_ids)
feature-conflicts (spec/check-feature-conflicts models-present)
library-conflict (when-let [local-library (t2/select-one :model/Collection :type collection/library-collection-type)]
(when (and first-import?
(contains? (get-in imported-data [:by-entity-id "Collection"] #{})
collection/library-entity-id)
(not (t2/exists? :model/RemoteSyncObject
:model_type "Collection"
:model_id (:id local-library))))
{:type :library-conflict
:category "Library"
:message "Import contains Library but local instance has an unsynced Library collection"}))
all-conflicts (concat
feature-conflicts
(when library-conflict [library-conflict]))]
{:conflicts (vec all-conflicts)
:summary (into #{} (map :category) all-conflicts)}))
(defn import!
"Imports and reloads Metabase entities from a remote snapshot.
@@ -206,32 +157,18 @@
(if snapshot
(try
(let [snapshot-version (source.p/version snapshot)
last-imported-version (remote-sync.task/last-version)
first-import? (nil? last-imported-version)
path-filters [#"collections/.*" #"databases/.*" #"actions/.*"
#"transforms/.*" #"python-libraries/.*" #"snippets/.*"]
base-ingestable (source.p/->ingestable snapshot {:path-filters path-filters})
has-transforms? (snapshot-has-transforms? base-ingestable)
{:keys [conflicts summary]} (get-conflicts base-ingestable first-import?)
ingestable-snapshot (source.ingestable/wrap-progress-ingestable task-id 0.7 base-ingestable)]
(cond
(and first-import? (not force?) (seq conflicts))
(u/prog1 {:status :conflict
:version (source.p/version snapshot)
:conflicts summary ; Keep backward compatibility: return set of category names
:conflict-details conflicts ; New: detailed conflict info
:message (format "Skipping import: snapshot version %s contains conflicts use force to override" snapshot-version)}
(log/infof (:message <>)))
(and (not force?) (= last-imported-version snapshot-version))
last-imported-version (remote-sync.task/last-version)]
(if (and (not force?) (= last-imported-version snapshot-version))
(u/prog1 {:status :success
:version (source.p/version snapshot)
:message (format "Skipping import: snapshot version %s matches last imported version" snapshot-version)}
(log/infof (:message <>)))
:else
(let [load-result (serdes/with-cache
(let [path-filters [#"collections/.*" #"databases/.*" #"actions/.*"
#"transforms/.*" #"python-libraries/.*" #"snippets/.*"]
base-ingestable (source.p/->ingestable snapshot {:path-filters path-filters})
has-transforms? (snapshot-has-transforms? base-ingestable)
ingestable-snapshot (source.ingestable/wrap-progress-ingestable task-id 0.7 base-ingestable)
load-result (serdes/with-cache
(serialization/load-metabase! ingestable-snapshot))
seen-paths (:seen load-result)
imported-data (spec/extract-imported-entities seen-paths)]
@@ -379,11 +316,10 @@
(defn handle-task-result!
"Handles the outcome of running import! or export! by updating the RemoteSyncTask record.
Takes a result map with a :status key (either :success, :conflict, or :error) and optional :message key, a
RemoteSyncTask ID, and an optional branch name. On success, updates the remote-sync-branch setting (if branch
provided), marks the task complete, and invalidates the remote changes cache. On conflict, sets the version and
stores the conflicts. On error, marks the task as failed with the error message. For any other status, marks the
task as failed with 'Unexpected Error'."
Takes a result map with a :status key (either :success or :error) and optional :message key, a RemoteSyncTask ID,
and an optional branch name. On success, updates the remote-sync-branch setting (if branch provided), marks the
task complete, and invalidates the remote changes cache. On error, marks the task as failed with the error message.
For any other status, marks the task as failed with 'Unexpected Error'."
[result task-id & [branch]]
(case (:status result)
:success (do
@@ -392,9 +328,6 @@
(settings/remote-sync-branch! branch))
(remote-sync.task/complete-sync-task! task-id))
(invalidate-remote-changes-cache!))
:conflict (do
(remote-sync.task/set-version! task-id (:version result))
(remote-sync.task/conflict-sync-task! task-id (:conflicts result)))
:error (remote-sync.task/fail-sync-task! task-id (:message result))
(remote-sync.task/fail-sync-task! task-id "Unexpected Error")))
@@ -425,8 +358,8 @@
(defn async-import!
"Imports remote-synced collections from a remote source repository asynchronously.
Takes a branch name to import from, a force? boolean (if true, imports even if there are unsaved changes or conflicts),
and an import-args map of additional arguments to pass to the import function. Checks for dirty changes and throws an
Takes a branch name to import from, a force? boolean (if true, imports even if there are unsaved changes), and an
import-args map of additional arguments to pass to the import function. Checks for dirty changes and throws an
exception if force? is false and changes exist.
Returns a RemoteSyncTask. Throws ExceptionInfo with status 400 and :conflicts true if there

View File

@@ -20,9 +20,6 @@
(derive :model/RemoteSyncTask :metabase/model)
(t2/deftransforms :model/RemoteSyncTask
{:conflicts mi/transform-json})
(declare current-task)
(t2/define-before-insert :model/RemoteSyncTask
@@ -211,30 +208,6 @@
(t/< (:last_progress_report_at task)
(t/minus (t/offset-date-time) (t/millis (settings/remote-sync-task-time-limit-ms))))))
(defn conflict?
"Checks if a task ended with conflicts.
Takes a RemoteSyncTask instance.
Returns true if the task has conflicts stored, false otherwise."
[task]
(and (some? (:ended_at task))
(false? (:cancelled task))
(nil? (:error_message task))
(some? (:conflicts task))))
(defn conflict-sync-task!
"Marks a sync task as having conflicts.
Takes the ID of the sync task and a collection of conflicts (vector of strings).
Conflicts are automatically serialized to JSON via the model transform.
Returns the number of rows updated (should be 1 if successful)."
[task-id conflicts]
(t2/update! :model/RemoteSyncTask task-id
{:ended_at (mi/now)
:conflicts conflicts}))
;;; ------------------------------------------- Hydration -------------------------------------------
(methodical/defmethod t2/batched-hydrate [:model/RemoteSyncTask :initiated_by_user]
@@ -249,7 +222,6 @@
(for [task tasks]
(assoc task :status (cond
(failed? task) :errored
(conflict? task) :conflict
(successful? task) :successful
(cancelled? task) :cancelled
(timed-out? task) :timed-out

View File

@@ -5,7 +5,7 @@
(def TaskStatus
"Status of a remote sync task."
[:enum :running :successful :errored :cancelled :timed-out :conflict])
[:enum :running :successful :errored :cancelled :timed-out])
(def TaskType
"Type of remote sync task."
@@ -24,38 +24,8 @@
[:version {:optional true} [:maybe :string]]
[:cancelled {:optional true} [:maybe :boolean]]
[:error_message {:optional true} [:maybe :string]]
[:conflicts {:optional true} [:maybe [:sequential :string]]]
[:status TaskStatus]])
;;; ------------------------------------------- Conflict Schemas -------------------------------------------
(def ConflictType
"Type of import conflict detected during pre-flight check."
[:enum
:entity-id-conflict ; Local entity exists with same entity_id but not in RemoteSyncObject
:library-conflict ; First import, local Library exists, import has Library
:transforms-conflict ; Local has transforms AND import has transforms
:snippets-conflict ; Local has snippets AND import has snippets
:dirty]) ; RemoteSyncObject has items with status != "synced"
(def ConflictDetail
"Schema for detailed conflict information."
[:map
[:type ConflictType]
[:category {:optional true} :string]
[:count {:optional true} pos-int?]
[:entity-ids {:optional true} [:set :string]]
[:message {:optional true} :string]])
(def ConflictResponse
"Schema for conflict error response from async-import!."
[:map
[:status-code [:= 400]]
[:conflicts [:= true]]
[:conflict-type {:optional true} ConflictType]
[:conflict-details {:optional true} [:sequential ConflictDetail]]
[:conflict-summary {:optional true} [:set :string]]])
;;; ------------------------------------------- Dirty Item Schemas -------------------------------------------
(def DirtyItem

View File

@@ -11,13 +11,10 @@
To add a new syncable model, add a spec entry to `remote-sync-specs` and
optionally implement custom multimethods if the default behavior doesn't fit."
(:require
[clojure.set :as set]
[clojure.string :as str]
[clojure.walk :as walk]
[metabase-enterprise.remote-sync.settings :as rs-settings]
[metabase-enterprise.transforms-python.core :as transforms-python]
[metabase.collections.core :as collections]
[metabase.collections.models.collection :as collection]
[metabase.models.serialization :as serdes]
[metabase.settings.core :as setting]
[metabase.util :as u]
@@ -313,8 +310,7 @@
:archived-key nil ; no archived field
:tracking {:select-fields [:path]
:field-mappings {:model_name :path}}
:conditions {:entity_id [:not= transforms-python/builtin-entity-id]} ; exclude built-in common.py from sync
:removal {:statuses #{"removed" "delete"} ; no scope-key = global deletion
:removal {:statuses #{"removed" "delete"} ; no scope-key = global deletion
:all-on-setting-disable :remote-sync-transforms}
:export-scope :all ; query for all instances
:enabled? :remote-sync-transforms}})
@@ -424,111 +420,6 @@
[]
(into #{} (map :model-type (vals remote-sync-specs))))
;;; ---------------------------------------- Import Conflict Detection -----------------------------------------------
(defn optional-feature-specs
"Returns specs for optional features, grouped by their enabling setting keyword.
Optional features are those with a keyword `:enabled?` value (not `true`).
Returns map of {setting-keyword {model-key spec}}."
[]
(-> remote-sync-specs
(->> (filter (fn [[_ spec]] (keyword? (:enabled? spec))))
(group-by (fn [[_ spec]] (:enabled? spec))))
(update-vals #(into {} %))))
(defn- setting->category
"Converts a setting keyword to a human-readable conflict category name."
[setting-kw]
(case setting-kw
:remote-sync-transforms "Transforms"
:library-synced "Snippets"
(str/capitalize (name setting-kw))))
(defn models-for-setting
"Returns set of model-type strings for specs with the given `:enabled?` setting."
[setting-kw]
(->> remote-sync-specs
(filter (fn [[_ spec]] (= (:enabled? spec) setting-kw)))
(map (fn [[_ spec]] (:model-type spec)))
(into #{})))
(def transform-models
"Models that indicate transforms content in a snapshot.
Derived from specs with `:enabled? :remote-sync-transforms`."
(disj (models-for-setting :remote-sync-transforms) :model/TransformTag))
(defn models-in-import
"Returns set of model-type strings present in the import.
Takes an ingest-list (sequence of serdes paths from serialization/ingest-list)."
[ingest-list]
(->> ingest-list
(map (fn [path] (:model (last path))))
(into #{})))
(defn check-entity-id-conflicts
"Checks if imported entity_ids exist locally but are NOT in RemoteSyncObject.
Returns map of {model-type #{conflicting-entity-ids}}.
Excludes the Library collection entity_id since that's handled by the library-conflict check."
[imported-entity-ids-by-model]
(into {}
(for [[model-type entity-ids] imported-entity-ids-by-model
:when (seq entity-ids)
:let [spec (spec-for-model-type model-type)
model-key (:model-key spec)]
:when (and spec model-key (#{:entity-id :hybrid} (:identity spec)))
:let [local-entity-ids (t2/select-fn-set :entity_id model-key :entity_id [:in entity-ids])
tracked-entity-ids (when (seq local-entity-ids)
(let [pks (t2/select-pks-vec model-key
:entity_id [:in local-entity-ids])]
(t2/select-fn-set
(fn [rso]
(:entity_id (t2/select-one model-key :id (:model_id rso))))
:model/RemoteSyncObject
:model_type model-type
:model_id [:in pks])))
conflicting-entity-ids (set/difference local-entity-ids (or tracked-entity-ids #{}))
conflicting-entity-ids (if (= model-type "Collection")
(disj conflicting-entity-ids collection/library-entity-id)
conflicting-entity-ids)]
:when (seq conflicting-entity-ids)]
[model-type conflicting-entity-ids])))
(defn- has-unsynced-entities-for-feature?
"Returns true if any model in the feature group has local entities not tracked in RemoteSyncObject.
Excludes built-in TransformTags from the count since they are system-created and not user data."
[specs-for-feature]
(some (fn [[_ spec]]
(let [model-key (:model-key spec)
model-type (:model-type spec)
;; Exclude built-in entities from count (they are system-created, not user data)
local-count (case model-key
:model/TransformTag (t2/count model-key :built_in_type nil)
:model/PythonLibrary (t2/count model-key :entity_id [:not= transforms-python/builtin-entity-id])
(t2/count model-key))
synced-count (t2/count :model/RemoteSyncObject :model_type model-type)]
(and (pos? local-count)
(> local-count synced-count))))
specs-for-feature))
(defn check-feature-conflicts
"Checks if import contains models that conflict with existing local entities that are NOT already remote synced.
Derives conflict categories from specs with keyword `:enabled?` values (optional features).
Only triggers conflict when local has unsynced entities AND import has matching entities.
If local entities are already synced, dirty tracking handles conflicts instead."
[models-present]
(let [feature-groups (optional-feature-specs)]
(into []
(for [[setting-kw specs-for-feature] feature-groups
:let [feature-model-types (into #{} (map (fn [[_ s]] (:model-type s))) specs-for-feature)]
:when (some feature-model-types models-present)
:when (has-unsynced-entities-for-feature? specs-for-feature)
:let [category (setting->category setting-kw)]]
{:type (keyword (str (u/lower-case-en category) "-conflict"))
:category category
:message (format "Import contains %s but local instance has unsynced %s"
category category)}))))
;;; ------------------------------------------------ Eligibility Checking ----------------------------------------------
(defn transforms-namespace-collection?

View File

@@ -97,7 +97,6 @@
"PermissionsGroupMembership"
"PermissionsRevision"
"PersistedInfo"
"PremiumFeaturesCache"
"Pulse"
"PulseCard"
"PulseChannel"

View File

@@ -8,7 +8,7 @@
[metabase.api.common :as api]
[metabase.api.macros :as api.macros]
[metabase.api.routes.common :refer [+auth]]
[metabase.collections.core :as collections]
[metabase.collections-rest.api :as api.collection]
[metabase.collections.models.collection :as collection]
[metabase.premium-features.core :as premium-features]
[metabase.queries.core :as queries]
@@ -94,7 +94,7 @@
"For dashboards, we want `here` and `location` since they can contain cards as children."
[dashboards]
(for [{parent-coll :collection
:as dashboard} (collections/annotate-dashboards dashboards)]
:as dashboard} (api.collection/annotate-dashboards dashboards)]
(assoc dashboard
:location (or (some-> parent-coll collection/children-location)
"/")

View File

@@ -1,10 +0,0 @@
(ns metabase-enterprise.transforms-python.core
"Core namespace for Python transforms functionality.
Re-exports commonly used vars from internal namespaces."
(:require
[metabase-enterprise.transforms-python.models.python-library :as python-library]
[potemkin :as p]))
(p/import-vars
[python-library
builtin-entity-id])

View File

@@ -50,11 +50,6 @@
"Set of allowed library paths. Currently only 'common' is supported."
#{"common.py"})
(def builtin-entity-id
"The entity_id of the built-in common.py PythonLibrary created by migration.
Used to protect it from deletion during remote-sync import."
"cWWH9qJPvHNB3rP2vLZrK")
(defn- validate-path!
"Validates that the given path is allowed. Throws an exception if not."
[path]

View File

@@ -52,7 +52,7 @@
(deftest collection-items-table-permissions-test
(mt/with-premium-features #{:library}
(testing "GET /api/collection/:id/items - published tables require view-data plus query access"
(testing "GET /api/collection/:id/items - published tables only require collection read permissions"
(mt/with-no-data-perms-for-all-users!
(mt/with-temp [:model/Collection collection {}
:model/Table {table-id :id :as table} {:collection_id (u/the-id collection)
@@ -61,15 +61,9 @@
(perms/add-user-to-group! (mt/user->id :rasta) group-id)
(t2/delete! :model/DataPermissions :db_id (mt/id))
(perms/grant-collection-read-permissions! (perms/all-users-group) collection)
(testing "with collection read but no view-data, user should not see published table"
(testing "with collection read but NO data permissions, user should see published table"
(data-perms/set-database-permission! group-id (mt/id) :perms/view-data :blocked)
(data-perms/set-database-permission! group-id (mt/id) :perms/create-queries :no)
(let [items (:data (mt/user-http-request :rasta :get 200
(str "collection/" (u/the-id collection) "/items")))]
(is (empty? (filter #(= "table" (:model %)) items))
"User without view-data should not see published tables")))
(testing "with collection read and view-data, user should see published table"
(data-perms/set-table-permission! group-id table :perms/view-data :unrestricted)
(let [items (:data (mt/user-http-request :rasta :get 200
(str "collection/" (u/the-id collection) "/items")))]
(is (=? [{:id table-id
@@ -78,8 +72,8 @@
:database_id (mt/id)
:archived false}]
(filter #(= "table" (:model %)) items))
"User with view-data and collection access should see published tables")))
(testing "with collection read and direct data query perms, user should still see published table"
"User with collection read permissions should see published tables (even without data permissions)")))
(testing "with collection read AND data permissions, user should still see published table"
(data-perms/set-table-permission! group-id table :perms/view-data :unrestricted)
(data-perms/set-table-permission! group-id table :perms/create-queries :query-builder)
(let [items (:data (mt/user-http-request :rasta :get 200
@@ -104,21 +98,16 @@
(data-perms/set-table-permission! group-id table :perms/create-queries :query-builder)
(testing "user cannot access collection at all without collection permissions"
(mt/user-http-request :rasta :get 403 (str "collection/" (u/the-id collection) "/items"))))))
(testing "GET /api/collection/root/items - published tables in root still require view-data"
(testing "GET /api/collection/root/items - published tables in root do NOT require data permissions"
(mt/with-no-data-perms-for-all-users!
(mt/with-temp [:model/Table {table-id :id :as table} {:collection_id nil
:is_published true}
:model/PermissionsGroup {group-id :id} {}]
(perms/add-user-to-group! (mt/user->id :rasta) group-id)
(t2/delete! :model/DataPermissions :db_id (mt/id))
(testing "with no view-data, user should not see published table in root"
(testing "with NO data permissions, user should see published table in root"
(data-perms/set-database-permission! group-id (mt/id) :perms/view-data :blocked)
(data-perms/set-database-permission! group-id (mt/id) :perms/create-queries :no)
(let [items (:data (mt/user-http-request :rasta :get 200 "collection/root/items"))]
(is (empty? (filter #(= table-id (:id %)) (filter #(= "table" (:model %)) items)))
"Users without view-data should not see published tables in root")))
(testing "with view-data permissions, user should see published table in root"
(data-perms/set-table-permission! group-id table :perms/view-data :unrestricted)
(let [items (:data (mt/user-http-request :rasta :get 200 "collection/root/items"))]
(is (=? [{:id table-id
:name (:display_name table)
@@ -126,8 +115,8 @@
:database_id (mt/id)
:archived false}]
(filter #(= table-id (:id %)) (filter #(= "table" (:model %)) items)))
"User with view-data should see published tables in root")))
(testing "with direct query permissions, user should still see published table in root"
"Users should see published tables in root collection (even without data permissions)")))
(testing "with data permissions, user should still see published table in root"
(data-perms/set-table-permission! group-id table :perms/view-data :unrestricted)
(data-perms/set-table-permission! group-id table :perms/create-queries :query-builder)
(let [items (:data (mt/user-http-request :rasta :get 200 "collection/root/items"))]

View File

@@ -76,7 +76,7 @@
(mt/user-http-request :rasta :get 200 (format "table/card__%d/query_metadata" (u/the-id card))))))))))
(deftest card-query-metadata-published-table-collection-perms-test
(testing "GET /api/card/:id/query_metadata should include published tables that are queryable"
(testing "GET /api/card/:id/query_metadata should include published tables accessible via collection permissions"
(mt/with-premium-features #{:library}
(mt/with-temp [:model/Collection table-coll {}
:model/Database db {}
@@ -90,7 +90,7 @@
:model/PermissionsGroup {group-id :id} {}
:model/PermissionsGroupMembership _ {:user_id (mt/user->id :rasta) :group_id group-id}]
(mt/with-no-data-perms-for-all-users!
(data-perms/set-database-permission! group-id db :perms/view-data :unrestricted)
(data-perms/set-database-permission! group-id db :perms/view-data :blocked)
(data-perms/set-database-permission! group-id db :perms/create-queries :no)
(perms/grant-collection-read-permissions! group-id (u/the-id table-coll))
(perms/grant-collection-read-permissions! group-id (u/the-id card-coll))
@@ -118,7 +118,7 @@
:model/PermissionsGroup {group-id :id} {}
:model/PermissionsGroupMembership _ {:user_id (mt/user->id :rasta) :group_id group-id}]
(mt/with-no-data-perms-for-all-users!
(data-perms/set-database-permission! group-id db :perms/view-data :unrestricted)
(data-perms/set-database-permission! group-id db :perms/view-data :blocked)
(data-perms/set-database-permission! group-id db :perms/create-queries :no)
(perms/grant-collection-read-permissions! group-id (u/the-id table-coll))
(perms/grant-collection-read-permissions! group-id (u/the-id card-coll))

View File

@@ -4,8 +4,6 @@
[clojure.test :refer :all]
[metabase.collections.models.collection :as collection]
[metabase.collections.test-utils :refer [without-library]]
[metabase.permissions.core :as perms]
[metabase.permissions.models.data-permissions :as data-perms]
[metabase.permissions.models.permissions-group :as perms-group]
[metabase.test :as mt]
[metabase.test.fixtures :as fixtures]
@@ -76,7 +74,7 @@
{:table_ids [(mt/id :users)]}))))))
(deftest data-analyst-can-access-endpoints-test
(mt/with-premium-features #{:library :advanced-permissions}
(mt/with-premium-features #{:library}
(testing "Data analysts (members of Data Analysts group) can access library endpoints"
(let [data-analyst-group-id (:id (perms-group/data-analyst))]
(mt/with-temp [:model/User {analyst-id :id} {:first_name "Data"
@@ -87,8 +85,6 @@
:model/Database {db-id :id} {}
:model/Table {table-id :id} {:db_id db-id}
:model/Collection _ {:type collection/library-data-collection-type}]
;; Grant data analyst group view-data permission on this database
(data-perms/set-database-permission! data-analyst-group-id db-id :perms/view-data :unrestricted)
(testing "data analyst can publish tables"
(is (map? (mt/user-http-request analyst-id :post 200 "ee/data-studio/table/publish-tables"
{:table_ids [table-id]}))))
@@ -250,46 +246,3 @@
{:table_ids [customers-id]})
(are [table-id] (false? (t2/select-one-fn :is_published :model/Table table-id))
customers-id orders-id items-id))))))
;;; ------------------------------------------ Publish/Unpublish requires write and query perms ------------------------------------------
(deftest publish-tables-requires-write-and-query-perms-test
(mt/with-premium-features #{:library :advanced-permissions}
(testing "POST /api/ee/data-studio/table/publish-tables requires write and query permission on all tables"
(mt/with-temp [:model/Collection _ {:type collection/library-data-collection-type}
:model/Database {db-id :id} {}
:model/Table {table-id :id} {:db_id db-id :name "restricted_table"}
:model/User {analyst-id :id} {:first_name "Data"
:last_name "Analyst"
:email "analyst-no-perms@metabase.com"
:is_data_analyst true}
:model/PermissionsGroupMembership _ {:user_id analyst-id
:group_id (:id (perms-group/data-analyst))}]
(mt/with-no-data-perms-for-all-users!
(data-perms/set-database-permission! (perms-group/data-analyst) db-id :perms/view-data :blocked)
(data-perms/set-database-permission! (perms-group/data-analyst) db-id :perms/create-queries :no)
;; Analyst has data analyst role but no data perms on the table
(is (= "You don't have permissions to do that."
(mt/user-http-request analyst-id :post 403 "ee/data-studio/table/publish-tables"
{:table_ids [table-id]}))))))))
(deftest unpublish-tables-requires-write-and-query-perms-test
(mt/with-premium-features #{:library :advanced-permissions}
(testing "POST /api/ee/data-studio/table/unpublish-tables requires write and query permission on all tables"
(mt/with-temp [:model/Collection {coll-id :id} {:type collection/library-data-collection-type}
:model/Database {db-id :id} {}
:model/Table {table-id :id} {:db_id db-id :name "restricted_table"
:is_published true :collection_id coll-id}
:model/User {analyst-id :id} {:first_name "Data"
:last_name "Analyst"
:email "analyst-no-perms@metabase.com"
:is_data_analyst true}
:model/PermissionsGroupMembership _ {:user_id analyst-id
:group_id (:id (perms-group/data-analyst))}]
(mt/with-no-data-perms-for-all-users!
(perms/revoke-collection-permissions! (perms-group/all-users) coll-id)
(perms/revoke-collection-permissions! (perms-group/data-analyst) coll-id)
;; Analyst has data analyst role but no data perms on the table
(is (= "You don't have permissions to do that."
(mt/user-http-request analyst-id :post 403 "ee/data-studio/table/unpublish-tables"
{:table_ids [table-id]}))))))))

View File

@@ -5,8 +5,7 @@
[metabase.permissions.core :as perms]
[metabase.permissions.published-tables :as published-tables]
[metabase.test :as mt]
[metabase.test.fixtures :as fixtures]
[toucan2.core :as t2]))
[metabase.test.fixtures :as fixtures]))
(use-fixtures :once (fixtures/initialize :db))
@@ -59,25 +58,3 @@
(perms/grant-collection-read-permissions! group-id (:id collection))
(mt/with-current-user user-id
(is (false? (boolean (published-tables/can-access-via-collection? table)))))))))
(deftest published-table-visible-clause-filters-by-collection-perms-test
(testing "Returns clause that only includes published tables in readable collections"
(mt/with-premium-features #{:library}
(mt/with-temp [:model/Collection {allowed-coll-id :id} {}
:model/Collection {blocked-coll-id :id} {}
:model/PermissionsGroup {group-id :id} {}
:model/User {user-id :id} {}
:model/PermissionsGroupMembership _ {:user_id user-id :group_id group-id}
:model/Table {allowed-table-id :id} {:is_published true :collection_id allowed-coll-id}
:model/Table {blocked-table-id :id} {:is_published true :collection_id blocked-coll-id}
:model/Table _ {:is_published false :collection_id allowed-coll-id}]
(mt/with-no-data-perms-for-all-users!
(perms/grant-collection-read-permissions! group-id allowed-coll-id)
(perms/revoke-collection-permissions! group-id blocked-coll-id)
(perms/revoke-collection-permissions! (perms/all-users-group) blocked-coll-id)
(let [clause (published-tables/published-table-visible-clause
:id
{:user-id user-id
:is-superuser? false})]
(is (= #{allowed-table-id}
(t2/select-pks-set :model/Table {:where [:and clause [:in :id [allowed-table-id blocked-table-id]]]})))))))))

View File

@@ -4,8 +4,6 @@
[clojure.test :refer :all]
[medley.core :as m]
[metabase.collections.models.collection :as collection]
[metabase.permissions.core :as perms]
[metabase.permissions.models.data-permissions :as data-perms]
[metabase.search.core :as search]
[metabase.test :as mt]
[metabase.util.random :refer [random-name]]))
@@ -117,44 +115,20 @@
:collection {:id nil}
:database_name "Context Test DB"}]
our-result))))))))
(testing "Users without query permissions cannot discover tables in search"
(testing "Collection does not matter for permissions for unpublished tables"
(let [search-term (random-name)
name-1 (str search-term " 1")
name-2 (str search-term " 2")]
(mt/with-no-data-perms-for-all-users!
(mt/with-temp [:model/Table {unpub-table :id} {:collection_id nil :is_published false :name name-1}
:model/Table {pub-table :id} {:collection_id nil :is_published true :name name-2}]
;; Explicitly deny both view and query perms so this assertion doesn't depend on global defaults.
(data-perms/set-database-permission! (perms/all-users-group) (mt/id) :perms/view-data :blocked)
(data-perms/set-database-permission! (perms/all-users-group) (mt/id) :perms/create-queries :no)
(doseq [engine ["in-place" "appdb"]]
(search/reindex! {:async? false :in-place? true})
(mt/with-non-admin-groups-no-root-collection-perms
(let [results (mt/user-http-request :rasta :get 200 "search"
:q search-term :models "table" :search_engine engine)
our-result (->> (filter (comp #{pub-table unpub-table} :id) (:data results))
(sort-by :id)
(map #(select-keys % [:id])))]
(testing "no tables appear"
(is (= []
our-result))))))))))))
(deftest unpublished-table-visible-with-data-perms-test
(testing "Unpublished tables are discoverable when the user has data/query permissions"
(let [search-term (random-name)
table-name (str search-term " unpublished")]
(mt/with-no-data-perms-for-all-users!
(mt/with-temp [:model/Table {unpub-table :id} {:collection_id nil :is_published false :name table-name}]
;; Explicitly grant data/query perms so this does not rely on global test defaults.
(data-perms/set-database-permission! (perms/all-users-group) (mt/id) :perms/view-data :unrestricted)
(data-perms/set-database-permission! (perms/all-users-group) (mt/id) :perms/create-queries :query-builder)
(mt/with-temp [:model/Table {unpub-table :id} {:collection_id nil :is_published false :name name-1}
:model/Table {pub-table :id} {:collection_id nil :is_published true :name name-2}]
(doseq [engine ["in-place" "appdb"]]
(search/reindex! {:async? false :in-place? true})
(mt/with-non-admin-groups-no-root-collection-perms
(let [result-ids (->> (mt/user-http-request :rasta :get 200 "search"
:q search-term :models "table" :search_engine engine)
:data
(map :id)
(into #{}))]
(is (contains? result-ids unpub-table)
(format "Unpublished table should be visible with data permissions (engine=%s)" engine))))))))))
(let [results (mt/user-http-request :rasta :get 200 "search"
:q search-term :models "table" :search_engine engine)
our-result (->> (filter (comp #{pub-table unpub-table} :id) (:data results))
(sort-by :id)
(map #(select-keys % [:id])))]
(testing "only the unpublished table appears"
(is (= [{:id unpub-table}]
our-result)))))))))))

View File

@@ -1,6 +1,5 @@
(ns metabase-enterprise.dependencies.events-test
(:require
[clojure.string :as str]
[clojure.test :refer [deftest is testing]]
[metabase-enterprise.dependencies.calculation :as deps.calculation]
[metabase-enterprise.dependencies.events]
@@ -240,9 +239,7 @@
(testing "on create event"
(events/publish-event! :event/card-create {:object card :user-id api/*current-user-id*})
(is (some #(and (= "Dependency calculation failed" (ex-message (:e %)))
(= :error (:level %))
(str/includes? (:message %) "{:entity-type :card")
(str/includes? (:message %) (str card-id)))
(= :error (:level %)))
(messages)))
(is (= models.dependency/current-dependency-analysis-version
(t2/select-one-fn :dependency_analysis_version :model/Card :id card-id)))
@@ -250,9 +247,7 @@
(testing "on update event"
(events/publish-event! :event/card-update {:object card :previous-object card :user-id api/*current-user-id*})
(is (some #(and (= "Dependency calculation failed" (ex-message (:e %)))
(= :error (:level %))
(str/includes? (:message %) "{:entity-type :card")
(str/includes? (:message %) (str card-id)))
(= :error (:level %)))
(messages)))
(is (= models.dependency/current-dependency-analysis-version
(t2/select-one-fn :dependency_analysis_version :model/Card :id card-id)))

View File

@@ -1155,166 +1155,3 @@ serdes/meta:
"python-libraries path should be included in filters")
(is (some #(str/includes? % "snippets") filter-strs)
"snippets path should be included in filters"))))))))))))
(deftest import!-blocks-if-it-encounters-library-conflicts
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})]
(t2/delete! :model/Collection :entity_id collection/library-entity-id)
(mt/with-temp [:model/Collection _ {:name "Test Library Collection"
:type "library"
:entity_id collection/library-entity-id
:location "/"}]
(let [test-files {"main" {"collections/test-collection-1xxxx-_/test-collection-1xxxx.yaml"
(test-helpers/generate-collection-yaml collection/library-entity-id "Another Library")}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :conflict (:status result)))
(is (= #{"Library"} (:conflicts result)))))))
(deftest import!-blocks-if-it-encounters-snippet-conflicts
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})]
(mt/with-temp [:model/NativeQuerySnippet _ {:name "Test Snippet"}]
(let [test-files {"main" {"collections/test-collection-1xxxx-_/test-collection-1xxxx.yaml"
(test-helpers/generate-snippet-yaml "blahblahblah" "A Snippet" "select 123")}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :conflict (:status result)))
(is (= #{"Snippets"} (:conflicts result)))))))
(deftest import!-blocks-if-it-encounters-transform-conflicts
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})]
(mt/with-temp [:model/Transform _ {:name "Test Transform"}]
(let [test-files {"main" {"collections/test-collection-1xxxx-_/test-collection-1xxxx.yaml"
(test-helpers/generate-transform-yaml "blahblahblah" "A Transform")}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :conflict (:status result)))
(is (= #{"Transforms"} (:conflicts result)))))))
(deftest import!-reports-multiple-conflicts
(testing "when multiple conflict types exist, all are reported in the :conflicts set"
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})]
(t2/delete! :model/Collection :entity_id collection/library-entity-id)
(mt/with-temp [:model/Collection _ {:name "Test Library Collection"
:type "library"
:entity_id collection/library-entity-id
:location "/"}
:model/NativeQuerySnippet _ {:name "Test Snippet"}
:model/Transform _ {:name "Test Transform"}]
(let [test-files {"main" {"collections/lib-_/lib.yaml"
(test-helpers/generate-collection-yaml collection/library-entity-id "Remote Library")
"collections/snip-_/snip.yaml"
(test-helpers/generate-snippet-yaml "snip-entity-id" "Remote Snippet" "select 1")
"collections/trans-_/trans.yaml"
(test-helpers/generate-transform-yaml "trans-entity-id" "Remote Transform")}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :conflict (:status result)))
(is (= #{"Library" "Transforms" "Snippets"} (:conflicts result))))))))
(deftest import!-force-bypasses-conflicts
(testing "force?: true allows import to proceed despite conflicts"
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})]
(t2/delete! :model/Collection :entity_id collection/library-entity-id)
(mt/with-temp [:model/Collection _ {:name "Test Library Collection"
:type "library"
:entity_id collection/library-entity-id
:location "/"}]
(let [test-files {"main" {"collections/lib-_/lib.yaml"
(test-helpers/generate-collection-yaml collection/library-entity-id "Remote Library")}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id :force? true)]
(is (= :success (:status result))))))))
(deftest import!-conflicts-only-block-initial-import
(testing "when last-imported-version is set, conflicts do not block subsequent imports"
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})]
(t2/delete! :model/Collection :entity_id collection/library-entity-id)
(mt/with-temp [:model/Collection _ {:name "Local Library"
:type "library"
:entity_id collection/library-entity-id
:location "/"}]
(with-redefs [remote-sync.task/last-version (constantly "previous-version")]
(let [test-files {"main" {"collections/lib-_/lib.yaml"
(test-helpers/generate-collection-yaml collection/library-entity-id "Remote Library")}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :success (:status result))
"Conflicts should not block when last-imported-version is set")))))))
(deftest import!-no-conflict-when-local-only
(testing "having local entities but no matching remote entities is not a conflict"
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})]
(mt/with-temp [:model/NativeQuerySnippet _ {:name "Local Snippet Only"}]
(let [result (impl/import! (source.p/snapshot (test-helpers/create-mock-source)) task-id)]
(is (= :success (:status result))))))))
(deftest import!-no-conflict-when-remote-only
(testing "having remote entities but no matching local entities is not a conflict"
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})
result (impl/import! (source.p/snapshot (test-helpers/create-mock-source)) task-id)]
(is (= :success (:status result))
"Should not trigger conflict when only remote has entities"))))
(deftest import!-library-conflict-requires-correct-type
(testing "a Collection with library entity_id but type!=library does not trigger conflict"
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})]
(t2/delete! :model/Collection :entity_id collection/library-entity-id)
(mt/with-temp [:model/Collection _ {:name "Not A Library"
:type nil
:entity_id collection/library-entity-id
:location "/"}]
(let [test-files {"main" {"collections/lib-_/lib.yaml"
(test-helpers/generate-collection-yaml collection/library-entity-id "Remote Library")}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :success (:status result))))))))
;; --- Spec-driven conflict detection tests ---
(deftest import!-returns-conflict-details-test
(testing "import! returns detailed conflict information in :conflict-details"
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})]
(t2/delete! :model/Collection :entity_id collection/library-entity-id)
(mt/with-temp [:model/Collection _ {:name "Test Library Collection"
:type "library"
:entity_id collection/library-entity-id
:location "/"}]
(let [test-files {"main" {"collections/test-collection-1xxxx-_/test-collection-1xxxx.yaml"
(test-helpers/generate-collection-yaml collection/library-entity-id "Another Library")}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :conflict (:status result)))
(is (= #{"Library"} (:conflicts result))
"Should return backward-compatible :conflicts set")
(is (seq (:conflict-details result))
"Should include detailed conflict information")
(is (some #(= :library-conflict (:type %)) (:conflict-details result))
"Should have library-conflict type in details"))))))
(deftest import!-transforms-conflict-test
(testing "import! detects transforms conflict when local has transforms and import has transforms"
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})]
(mt/with-temp [:model/Transform _ {:name "Local Transform"}]
(let [test-files {"main" {"collections/test-coll-_/test-coll.yaml"
(test-helpers/generate-collection-yaml "test-collection-1xxxx" "Test Collection")
"transforms/test-transform-_/test-transform.yaml"
(test-helpers/generate-transform-yaml "test-transform-xxxxx" "Remote Transform")}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :conflict (:status result)))
(is (contains? (:conflicts result) "Transforms"))
(is (some #(= :transforms-conflict (:type %)) (:conflict-details result))))))))
(deftest import!-snippets-conflict-test
(testing "import! detects snippets conflict when local has snippets and import has snippets"
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})]
(mt/with-temp [:model/NativeQuerySnippet _ {:name "Local Snippet"}]
(let [test-files {"main" {"collections/test-coll-_/test-coll.yaml"
(test-helpers/generate-collection-yaml "test-collection-1xxxx" "Test Collection")
"snippets/test-snippet-_/test-snippet.yaml"
(test-helpers/generate-snippet-yaml "test-snippet-xxxxxxx" "Remote Snippet" "SELECT 1")}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :conflict (:status result)))
(is (contains? (:conflicts result) "Snippets"))
(is (some #(= :snippets-conflict (:type %)) (:conflict-details result))))))))

View File

@@ -276,6 +276,26 @@ is_sample: false
name entity-id (str/replace (u/lower-case-en name) #"\s+" "_")
entity-id (str/replace (u/lower-case-en name) #"\s+" "_")))
(defn- generate-snippet-yaml
"Generates YAML content for a NativeQuerySnippet."
[entity-id name content & {:keys [collection-id]}]
(format "name: %s
description: null
entity_id: %s
content: '%s'
archived: false
template_tags: null
created_at: '2024-08-28T09:46:18.671622Z'
creator_id: rasta@metabase.com
collection_id: %s
serdes/meta:
- id: %s
label: %s
model: NativeQuerySnippet
"
name entity-id content (or collection-id "null")
entity-id (str/replace (u/lower-case-en name) #"\s+" "_")))
(deftest export-includes-snippets-when-library-synced-test
(testing "Export includes snippets when Library is synced"
(collections.tu/with-library-synced
@@ -325,7 +345,7 @@ is_sample: false
test-files {"main" {(str "collections/" coll-entity-id "_snippets_collection/" coll-entity-id "_snippets_collection.yaml")
(generate-snippets-namespace-collection-yaml coll-entity-id "Snippets Collection")
(str "snippets/" coll-entity-id "_snippets_collection/" snippet-entity-id "_test_snippet.yaml")
(test-helpers/generate-snippet-yaml snippet-entity-id "Test Snippet" "SELECT 42" :collection-id coll-entity-id)}}
(generate-snippet-yaml snippet-entity-id "Test Snippet" "SELECT 42" :collection-id coll-entity-id)}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :success (:status result))
@@ -410,7 +430,7 @@ is_sample: false
snippet-removal-path (first (filter #(str/includes? % snippet-eid) removal-paths))
_ (is (some? snippet-removal-path) "Should have a removal path for the archived snippet")
initial-files {"main" {(str snippet-removal-path ".yaml")
(test-helpers/generate-snippet-yaml snippet-eid "Archived Snippet" "SELECT 1" :collection-id coll-eid)}}
(generate-snippet-yaml snippet-eid "Archived Snippet" "SELECT 1" :collection-id coll-eid)}}
mock-source (test-helpers/create-mock-source :initial-files initial-files)
result (impl/export! (source.p/snapshot mock-source) task-id "Test export")]
(is (= :success (:status result))

View File

@@ -4,7 +4,6 @@
[clojure.string :as str]
[clojure.test :as t]
[metabase-enterprise.remote-sync.source.protocol :as source.p]
[metabase-enterprise.transforms-python.core :as transforms-python]
[metabase.util :as u]
[toucan2.core :as t2]))
@@ -285,42 +284,9 @@ width: fixed
(when (seq old-models)
(t2/insert! :model/RemoteSyncTask old-models))))))
(def ^:private builtin-python-library
"The built-in PythonLibrary created by migration. Recreated by fixture after cleanup."
{:path "common.py" :source "" :entity_id transforms-python/builtin-entity-id})
(defn- ensure-builtin-python-library!
"Ensures the built-in common.py PythonLibrary exists, creating it if missing."
[]
(when-not (t2/exists? :model/PythonLibrary :path (:path builtin-python-library))
(t2/insert! :model/PythonLibrary builtin-python-library)))
(defn clean-optional-feature-models
"Test fixture that cleans Transform, TransformTag, and PythonLibrary tables to prevent
conflict detection during first-import tests. Preserves built-in TransformTags and
recreates the built-in common.py PythonLibrary after cleanup."
[f]
(let [old-transforms (t2/select :model/Transform)
old-tags (t2/select :model/TransformTag :built_in_type nil)
old-libs (t2/select :model/PythonLibrary)]
(try
(t2/delete! :model/TransformTag :built_in_type nil)
(t2/delete! :model/Transform)
(t2/delete! :model/PythonLibrary)
(f)
(finally
(t2/delete! :model/TransformTag :built_in_type nil)
(t2/delete! :model/Transform)
(t2/delete! :model/PythonLibrary)
(when (seq old-transforms) (t2/insert! :model/Transform old-transforms))
(when (seq old-tags) (t2/insert! :model/TransformTag old-tags))
(when (seq old-libs) (t2/insert! :model/PythonLibrary old-libs))
(ensure-builtin-python-library!)))))
(def clean-remote-sync-state
"Composed test fixture that ensures RemoteSyncObject, RemoteSyncTask, and optional feature
model tables (Transform, TransformTag, PythonLibrary) are clean."
(t/compose-fixtures clean-object (t/compose-fixtures clean-task-table clean-optional-feature-models)))
"Composed test fixture that ensures both RemoteSyncObject and RemoteSyncTask tables are clean."
(t/compose-fixtures clean-object clean-task-table))
(defn generate-table-yaml
"Generate YAML content for a table with the given `table-name` and `db-name`.
@@ -616,23 +582,3 @@ serdes/meta:
table-name
eid
(str/replace (u/lower-case-en measure-name) #"\s+" "_"))))
(defn generate-snippet-yaml
"Generates YAML content for a NativeQuerySnippet."
[entity-id name content & {:keys [collection-id]}]
(format "name: %s
description: null
entity_id: %s
content: '%s'
archived: false
template_tags: null
created_at: '2024-08-28T09:46:18.671622Z'
creator_id: rasta@metabase.com
collection_id: %s
serdes/meta:
- id: %s
label: %s
model: NativeQuerySnippet
"
name entity-id content (or collection-id "null")
entity-id (str/replace (u/lower-case-en name) #"\s+" "_")))

View File

@@ -10,7 +10,6 @@
[metabase-enterprise.remote-sync.source.protocol :as source.p]
[metabase-enterprise.remote-sync.spec :as spec]
[metabase-enterprise.remote-sync.test-helpers :as test-helpers]
[metabase-enterprise.transforms-python.core :as transforms-python]
[metabase.collections.models.collection :as collection]
[metabase.events.core :as events]
[metabase.search.core :as search]
@@ -455,11 +454,10 @@ is_sample: false
(generate-transforms-namespace-collection-yaml coll-entity-id "Transforms Collection")
(str "collections/" coll-entity-id "_transforms/transforms/" remote-transform-entity-id "_remote_transform.yaml")
(test-helpers/generate-transform-yaml remote-transform-entity-id "Remote Transform" :collection-id coll-entity-id)}}
mock-source (test-helpers/create-mock-source :initial-files test-files)]
(testing "fails with `conflict` status because local transforms will be deleted"
(is (= :conflict (:status (impl/import! (source.p/snapshot mock-source) task-id)))))
(testing "passing `force? true` allows overriding the conflict."
(is (= :success (:status (impl/import! (source.p/snapshot mock-source) task-id :force? true)))))
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :success (:status result))
(str "Import should succeed. Result: " result))
(is (t2/exists? :model/Collection :id coll-id)
"Transforms collection should still exist")
(is (not (t2/exists? :model/Transform :id local-transform-id))
@@ -489,7 +487,7 @@ is_sample: false
(let [test-files {"main" {(str "collections/" remote-coll-entity-id "_remote_transforms/" remote-coll-entity-id "_remote_transforms.yaml")
(generate-transforms-namespace-collection-yaml remote-coll-entity-id "Remote Transforms")}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id :force? true)]
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :success (:status result))
(str "Import should succeed. Result: " result))
(is (not (t2/exists? :model/Transform :id local-transform-id))
@@ -518,144 +516,148 @@ serdes/meta:
entity-id
entity-id))
(defn- with-clean-python-library
"Ensures PythonLibrary table is clean before and after running the test function.
Saves and restores any existing entries."
[f]
(let [existing (t2/select :model/PythonLibrary)]
(try
(t2/delete! :model/PythonLibrary)
(f)
(finally
(t2/delete! :model/PythonLibrary)
(when (seq existing)
(t2/insert! :model/PythonLibrary existing))))))
(deftest python-library-event-creates-sync-object-when-setting-enabled-test
(testing "Creating a PythonLibrary creates a RemoteSyncObject entry when remote-sync-transforms is enabled"
(mt/with-premium-features #{:transforms}
(mt/with-temporary-setting-values [remote-sync-transforms true
remote-sync-enabled true]
(let [library (t2/insert-returning-instance! :model/PythonLibrary {:path "common.py" :source "# test"})]
(is (t2/exists? :model/RemoteSyncObject
:model_type "PythonLibrary"
:model_id (:id library))
"PythonLibrary should be tracked when remote-sync-transforms is enabled"))))))
(with-clean-python-library
(fn []
(mt/with-premium-features #{:transforms}
(mt/with-temporary-setting-values [remote-sync-transforms true
remote-sync-enabled true]
(let [library (t2/insert-returning-instance! :model/PythonLibrary {:path "common.py" :source "# test"})]
(is (t2/exists? :model/RemoteSyncObject
:model_type "PythonLibrary"
:model_id (:id library))
"PythonLibrary should be tracked when remote-sync-transforms is enabled"))))))))
(deftest python-library-event-ignored-when-setting-disabled-test
(testing "PythonLibrary events are ignored when remote-sync-transforms is disabled"
(mt/with-premium-features #{:transforms}
(mt/with-temporary-setting-values [remote-sync-transforms false
remote-sync-enabled true]
(let [library (t2/insert-returning-instance! :model/PythonLibrary {:path "common.py" :source "# test"})]
(is (not (t2/exists? :model/RemoteSyncObject
:model_type "PythonLibrary"
:model_id (:id library)))
"PythonLibrary should NOT be tracked when remote-sync-transforms is disabled"))))))
(with-clean-python-library
(fn []
(mt/with-premium-features #{:transforms}
(mt/with-temporary-setting-values [remote-sync-transforms false
remote-sync-enabled true]
(let [library (t2/insert-returning-instance! :model/PythonLibrary {:path "common.py" :source "# test"})]
(is (not (t2/exists? :model/RemoteSyncObject
:model_type "PythonLibrary"
:model_id (:id library)))
"PythonLibrary should NOT be tracked when remote-sync-transforms is disabled"))))))))
(deftest python-library-event-updates-sync-object-test
(testing "Updating a PythonLibrary updates the RemoteSyncObject entry when setting is enabled"
(mt/with-premium-features #{:transforms}
(mt/with-temporary-setting-values [remote-sync-transforms true
remote-sync-enabled true]
(let [library (t2/insert-returning-instance! :model/PythonLibrary {:path "common.py" :source "# test"})]
(t2/update! :model/RemoteSyncObject {:model_type "PythonLibrary" :model_id (:id library)}
{:status "synced"})
(t2/update! :model/PythonLibrary (:id library) {:source "# updated"})
(let [entry (t2/select-one :model/RemoteSyncObject
:model_type "PythonLibrary"
:model_id (:id library))]
(is (= "update" (:status entry))
"PythonLibrary should have 'update' status after modification")))))))
(with-clean-python-library
(fn []
(mt/with-premium-features #{:transforms}
(mt/with-temporary-setting-values [remote-sync-transforms true
remote-sync-enabled true]
(let [library (t2/insert-returning-instance! :model/PythonLibrary {:path "common.py" :source "# test"})]
(t2/update! :model/RemoteSyncObject {:model_type "PythonLibrary" :model_id (:id library)}
{:status "synced"})
(t2/update! :model/PythonLibrary (:id library) {:source "# updated"})
(let [entry (t2/select-one :model/RemoteSyncObject
:model_type "PythonLibrary"
:model_id (:id library))]
(is (= "update" (:status entry))
"PythonLibrary should have 'update' status after modification")))))))))
(deftest export-includes-python-library-when-setting-enabled-test
(testing "Export includes PythonLibrary when remote-sync-transforms is enabled"
(mt/with-premium-features #{:transforms}
(mt/with-temporary-setting-values [remote-sync-type :read-write
remote-sync-transforms true
remote-sync-enabled true]
(mt/with-model-cleanup [:model/RemoteSyncTask]
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "export" :initiated_by (mt/user->id :rasta)})
library (t2/insert-returning-instance! :model/PythonLibrary {:path "common.py" :source "# shared code"})]
(t2/update! :model/RemoteSyncObject {:model_type "PythonLibrary" :model_id (:id library)}
{:status "create"})
(let [mock-source (test-helpers/create-mock-source)
result (impl/export! (source.p/snapshot mock-source) task-id "Test export")]
(is (= :success (:status result))
(str "Export should succeed. Result: " result))
(let [files-after-export (get @(:files-atom mock-source) "main")]
(is (some #(str/includes? % "python-libraries/") (keys files-after-export))
"Export should include the PythonLibrary file")))))))))
(with-clean-python-library
(fn []
(mt/with-premium-features #{:transforms}
(mt/with-temporary-setting-values [remote-sync-type :read-write
remote-sync-transforms true
remote-sync-enabled true]
(mt/with-model-cleanup [:model/RemoteSyncTask]
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "export" :initiated_by (mt/user->id :rasta)})
library (t2/insert-returning-instance! :model/PythonLibrary {:path "common.py" :source "# shared code"})]
(t2/update! :model/RemoteSyncObject {:model_type "PythonLibrary" :model_id (:id library)}
{:status "create"})
(let [mock-source (test-helpers/create-mock-source)
result (impl/export! (source.p/snapshot mock-source) task-id "Test export")]
(is (= :success (:status result))
(str "Export should succeed. Result: " result))
(let [files-after-export (get @(:files-atom mock-source) "main")]
(is (some #(str/includes? % "python-libraries/") (keys files-after-export))
"Export should include the PythonLibrary file")))))))))))
(deftest import-python-library-from-yaml-test
(testing "Import brings in PythonLibrary from YAML files"
(mt/with-premium-features #{:transforms}
(mt/with-temporary-setting-values [remote-sync-transforms true
remote-sync-enabled true]
(mt/with-model-cleanup [:model/RemoteSyncTask]
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})
lib-entity-id (u/generate-nano-id)
test-files {"main" {(str "python-libraries/" lib-entity-id ".yaml")
(generate-python-library-yaml lib-entity-id "common.py" "def shared_func():\n return 42")}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
snapshot (source.p/snapshot mock-source)
result (impl/import! snapshot task-id)]
(is (= :success (:status result))
(str "Import should succeed. Result: " result))
(is (t2/exists? :model/PythonLibrary :path "common.py")
"PythonLibrary should be imported")
(when-let [library (t2/select-one :model/PythonLibrary :path "common.py")]
(is (str/includes? (:source library) "def shared_func()")
"PythonLibrary source should be imported correctly"))))))))
(with-clean-python-library
(fn []
(mt/with-premium-features #{:transforms}
(mt/with-temporary-setting-values [remote-sync-transforms true
remote-sync-enabled true]
(mt/with-model-cleanup [:model/RemoteSyncTask]
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})
lib-entity-id (u/generate-nano-id)
test-files {"main" {(str "python-libraries/" lib-entity-id ".yaml")
(generate-python-library-yaml lib-entity-id "common.py" "def shared_func():\n return 42")}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :success (:status result))
(str "Import should succeed. Result: " result))
(is (t2/exists? :model/PythonLibrary :path "common.py")
"PythonLibrary should be imported")
(when-let [library (t2/select-one :model/PythonLibrary :path "common.py")]
(is (str/includes? (:source library) "def shared_func()")
"PythonLibrary source should be imported correctly"))))))))))
(deftest import-removes-python-library-not-on-remote-test
(testing "Import removes local PythonLibrary that doesn't exist on the remote"
(mt/with-premium-features #{:transforms}
(mt/with-temporary-setting-values [remote-sync-transforms true
remote-sync-enabled true]
(mt/with-model-cleanup [:model/RemoteSyncTask]
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})
local-library (t2/insert-returning-instance! :model/PythonLibrary {:path "common.py" :source "# local only"})]
(is (t2/exists? :model/PythonLibrary :id (:id local-library))
"Local PythonLibrary should exist before import")
(let [test-files {"main" {}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :success (:status result))
(str "Import should succeed. Result: " result))
(is (not (t2/exists? :model/PythonLibrary :id (:id local-library)))
"Local PythonLibrary should be deleted after import since it wasn't on remote"))))))))
(deftest import-preserves-builtin-python-library-test
(testing "Import does NOT delete the built-in PythonLibrary even when it's not on remote"
(mt/with-premium-features #{:transforms}
(mt/with-temporary-setting-values [remote-sync-transforms true
remote-sync-enabled true]
(mt/with-model-cleanup [:model/RemoteSyncTask]
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})]
;; Create the builtin library explicitly (fixture cleans PythonLibrary before test)
(t2/insert-returning-instance! :model/PythonLibrary
{:path "common.py"
:source "# builtin"
:entity_id transforms-python/builtin-entity-id})
(is (t2/exists? :model/PythonLibrary :entity_id transforms-python/builtin-entity-id)
"Built-in PythonLibrary should exist before import")
(let [test-files {"main" {}} ;; Empty remote - no python libraries
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :success (:status result))
(str "Import should succeed. Result: " result))
(is (t2/exists? :model/PythonLibrary :entity_id transforms-python/builtin-entity-id)
"Built-in PythonLibrary should NOT be deleted after import"))))))))
(with-clean-python-library
(fn []
(mt/with-premium-features #{:transforms}
(mt/with-temporary-setting-values [remote-sync-transforms true
remote-sync-enabled true]
(mt/with-model-cleanup [:model/RemoteSyncTask]
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})
local-library (t2/insert-returning-instance! :model/PythonLibrary {:path "common.py" :source "# local only"})]
(is (t2/exists? :model/PythonLibrary :id (:id local-library))
"Local PythonLibrary should exist before import")
(let [test-files {"main" {}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :success (:status result))
(str "Import should succeed. Result: " result))
(is (not (t2/exists? :model/PythonLibrary :id (:id local-library)))
"Local PythonLibrary should be deleted after import since it wasn't on remote"))))))))))
(deftest import-replaces-python-library-with-remote-version-test
(testing "Import updates local PythonLibrary when remote has same entity_id"
(mt/with-premium-features #{:transforms}
(mt/with-temporary-setting-values [remote-sync-transforms true
remote-sync-enabled true]
(mt/with-model-cleanup [:model/RemoteSyncTask]
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})
local-library (t2/insert-returning-instance! :model/PythonLibrary {:path "common.py" :source "# local version"})
local-entity-id (:entity_id local-library)]
(is (t2/exists? :model/PythonLibrary :path "common.py")
"Local PythonLibrary should exist before import")
(let [test-files {"main" {(str "python-libraries/" local-entity-id ".yaml")
(generate-python-library-yaml local-entity-id "common.py" "# remote version\ndef new_func():\n pass")}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :success (:status result))
(str "Import should succeed. Result: " result))
(let [library (t2/select-one :model/PythonLibrary :path "common.py")]
(is (some? library) "PythonLibrary should still exist")
(is (str/includes? (:source library) "# remote version")
"PythonLibrary source should be updated with remote version")))))))))
(with-clean-python-library
(fn []
(mt/with-premium-features #{:transforms}
(mt/with-temporary-setting-values [remote-sync-transforms true
remote-sync-enabled true]
(mt/with-model-cleanup [:model/RemoteSyncTask]
(let [task-id (t2/insert-returning-pk! :model/RemoteSyncTask {:sync_task_type "import" :initiated_by (mt/user->id :rasta)})
local-library (t2/insert-returning-instance! :model/PythonLibrary {:path "common.py" :source "# local version"})
local-entity-id (:entity_id local-library)]
(is (t2/exists? :model/PythonLibrary :path "common.py")
"Local PythonLibrary should exist before import")
(let [test-files {"main" {(str "python-libraries/" local-entity-id ".yaml")
(generate-python-library-yaml local-entity-id "common.py" "# remote version\ndef new_func():\n pass")}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :success (:status result))
(str "Import should succeed. Result: " result))
(let [library (t2/select-one :model/PythonLibrary :path "common.py")]
(is (some? library) "PythonLibrary should still exist")
(is (str/includes? (:source library) "# remote version")
"PythonLibrary source should be updated with remote version")))))))))))
;;; ------------------------------------------- Transform Sync Behavior Tests -------------------------------------------
@@ -724,9 +726,7 @@ serdes/meta:
(str "collections/" remote-coll-entity-id "_transforms/transforms/" remote-transform-entity-id "_remote_transform.yaml")
(test-helpers/generate-transform-yaml remote-transform-entity-id "Remote Transform" :collection-id remote-coll-entity-id)}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result-without-force (impl/import! (source.p/snapshot mock-source) task-id)
result (impl/import! (source.p/snapshot mock-source) task-id {:force? true})]
(is (= :conflict (:status result-without-force)))
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :success (:status result))
(str "Import should succeed. Result: " result))
(is (settings/remote-sync-transforms)
@@ -805,7 +805,7 @@ serdes/meta:
(str "collections/" remote-coll-entity-id "_transforms/transforms/" remote-transform-entity-id "_remote_transform.yaml")
(test-helpers/generate-transform-yaml remote-transform-entity-id "Remote Transform" :collection-id remote-coll-entity-id)}}
mock-source (test-helpers/create-mock-source :initial-files test-files)
result (impl/import! (source.p/snapshot mock-source) task-id :force? true)]
result (impl/import! (source.p/snapshot mock-source) task-id)]
(is (= :success (:status result))
(str "Import should succeed. Result: " result))
(is (settings/remote-sync-transforms)
@@ -819,38 +819,40 @@ serdes/meta:
(deftest build-all-removal-paths-includes-all-transforms-on-setting-disable-test
(testing "build-all-removal-paths returns paths for all transforms content when sentinel RSO has 'delete' status"
(mt/with-premium-features #{:transforms}
(mt/with-temporary-setting-values [remote-sync-transforms true
remote-sync-enabled true]
(mt/with-temp [:model/Collection {coll-id :id} {:name "Transforms Collection"
:namespace collection/transforms-ns
:location "/"}
:model/Transform {transform-id :id transform-eid :entity_id} {:name "Test Transform"
:collection_id coll-id}
:model/TransformTag {tag-id :id tag-eid :entity_id} {:name "Test Tag"
:built_in_type nil}]
(let [library (t2/insert-returning-instance! :model/PythonLibrary {:path "common.py" :source "# test"})
lib-eid (:entity_id library)]
(is (t2/exists? :model/Transform :id transform-id))
(is (t2/exists? :model/TransformTag :id tag-id))
(is (t2/exists? :model/PythonLibrary :id (:id library)))
(let [paths-before (spec/build-all-removal-paths)]
(is (not (some #(str/includes? % transform-eid) paths-before))
"Transform should not be in removal paths before setting is disabled"))
(settings/sync-transform-tracking! true)
(settings/sync-transform-tracking! false)
(is (t2/exists? :model/RemoteSyncObject
:model_type "Collection"
:model_id settings/transforms-root-id
:status "delete")
"Sentinel RSO should exist with 'delete' status")
(let [paths-after (spec/build-all-removal-paths)]
(is (some #(str/includes? % transform-eid) paths-after)
"Transform should be in removal paths after setting is disabled")
(is (some #(str/includes? % tag-eid) paths-after)
"TransformTag should be in removal paths after setting is disabled")
(is (some #(str/includes? % lib-eid) paths-after)
"PythonLibrary should be in removal paths after setting is disabled"))))))))
(with-clean-python-library
(fn []
(mt/with-premium-features #{:transforms}
(mt/with-temporary-setting-values [remote-sync-transforms true
remote-sync-enabled true]
(mt/with-temp [:model/Collection {coll-id :id} {:name "Transforms Collection"
:namespace collection/transforms-ns
:location "/"}
:model/Transform {transform-id :id transform-eid :entity_id} {:name "Test Transform"
:collection_id coll-id}
:model/TransformTag {tag-id :id tag-eid :entity_id} {:name "Test Tag"
:built_in_type nil}]
(let [library (t2/insert-returning-instance! :model/PythonLibrary {:path "common.py" :source "# test"})
lib-eid (:entity_id library)]
(is (t2/exists? :model/Transform :id transform-id))
(is (t2/exists? :model/TransformTag :id tag-id))
(is (t2/exists? :model/PythonLibrary :id (:id library)))
(let [paths-before (spec/build-all-removal-paths)]
(is (not (some #(str/includes? % transform-eid) paths-before))
"Transform should not be in removal paths before setting is disabled"))
(settings/sync-transform-tracking! true)
(settings/sync-transform-tracking! false)
(is (t2/exists? :model/RemoteSyncObject
:model_type "Collection"
:model_id settings/transforms-root-id
:status "delete")
"Sentinel RSO should exist with 'delete' status")
(let [paths-after (spec/build-all-removal-paths)]
(is (some #(str/includes? % transform-eid) paths-after)
"Transform should be in removal paths after setting is disabled")
(is (some #(str/includes? % tag-eid) paths-after)
"TransformTag should be in removal paths after setting is disabled")
(is (some #(str/includes? % lib-eid) paths-after)
"PythonLibrary should be in removal paths after setting is disabled"))))))))))
(deftest build-all-removal-paths-excludes-builtin-transform-tags-test
(testing "build-all-removal-paths respects :conditions and excludes built-in tags"

View File

@@ -221,7 +221,7 @@
io/input-stream)
ba (#'api.serialization/ba-copy res)]
(testing "Archive contains correct number of files with proper log entries"
(is (= 13
(is (= 12
(with-open [tar (open-tar ba)]
(count
(for [^TarArchiveEntry e (u.compress/entries tar)
@@ -229,7 +229,7 @@
(do
(condp re-find (.getName e)
#"/export.log$" (testing "Log contains extract and store entries"
(is (= (+ #_extract 12 #_store 12)
(is (= (+ #_extract 11 #_store 11)
(count (line-seq (io/reader tar))))))
nil)
(.getName e))))))))
@@ -243,7 +243,7 @@
"settings" false
"field_values" false
"duration_ms" (every-pred number? pos?)
"count" 12
"count" 11
"error_count" 0
"source" "api"
"secrets" false
@@ -273,7 +273,7 @@
{:request-options {:headers {"content-type" "multipart/form-data"}}}
{:file ba}))]
(testing "Log contains imported entity types"
(is (= #{"Collection" "Dashboard" "Card" "PythonLibrary" "TransformJob" "TransformTag"}
(is (= #{"Collection" "Dashboard" "Card" "TransformJob" "TransformTag"}
(log-types (line-seq (io/reader (io/input-stream res)))))))
(testing "Entities are restored in the database"
@@ -285,8 +285,8 @@
"direction" "import"
"duration_ms" pos?
"source" "api"
"models" "Card,Collection,Dashboard,PythonLibrary,TransformJob,TransformTag"
"count" 12
"models" "Card,Collection,Dashboard,TransformJob,TransformTag"
"count" 11
"error_count" 0
"success" true
"error_message" nil}
@@ -349,7 +349,7 @@
:continue_on_error true)
log (slurp (io/input-stream res))]
(testing "Log shows loaded entities and error"
(is (= #{"Dashboard" "Card" "Collection" "TransformJob" "TransformTag" "PythonLibrary"}
(is (= #{"Dashboard" "Card" "Collection" "TransformJob" "TransformTag"}
(log-types (str/split-lines log))))
(is (re-find #"Collection 'DoesNotExist' was not found" log)))
@@ -359,9 +359,9 @@
"direction" "import"
"source" "api"
"duration_ms" int?
"count" 11
"count" 10
"error_count" 1
"models" "Collection,Dashboard,PythonLibrary,TransformJob,TransformTag"}
"models" "Collection,Dashboard,TransformJob,TransformTag"}
(-> (snowplow-test/pop-event-data-and-user-id!) last :data))))))))))
(deftest import-invalid-archive-test
@@ -429,7 +429,7 @@
(doseq [^TarArchiveEntry e (u.compress/entries tar)]
(condp re-find (.getName e)
#"/export.log$" (testing "Log shows extract entries, error, and store entries"
(is (= (+ #_extract 12 #_error 1 #_store 11)
(is (= (+ #_extract 11 #_error 1 #_store 10)
(count (line-seq (io/reader tar))))))
nil))))
@@ -442,7 +442,7 @@
"settings" false
"field_values" false
"duration_ms" pos?
"count" 11
"count" 10
"error_count" 1
"source" "api"
"secrets" false

View File

@@ -72,7 +72,6 @@
:model/PermissionsGroupMembership
:model/PermissionsRevision
:model/PersistedInfo
:model/PremiumFeaturesCache
:model/Pulse
:model/PulseCard
:model/PulseChannel

View File

@@ -2586,72 +2586,3 @@
(testing "has no dependencies"
(is (empty? (serdes/dependencies ser)))))))))
(deftest ^:parallel export-parameters-sorts-by-id-test
(let [params [{:id "zebra" :name "Z param" :type :category}
{:id "alpha" :name "A param" :type :category}
{:id "middle" :name "M param" :type :category}]
result (serdes/export-parameters params)]
(is (= ["alpha" "middle" "zebra"]
(map :id result)))))
(deftest ^:parallel export-parameters-empty-input-test
(is (= [] (serdes/export-parameters []))))
(deftest ^:parallel export-parameters-nil-input-test
(is (= [] (serdes/export-parameters nil))))
(deftest ^:parallel export-parameters-single-element-test
(let [params [{:id "only" :name "Only param"}]
result (serdes/export-parameters params)]
(is (= ["only"] (map :id result)))))
(deftest ^:parallel export-parameters-nil-id-sorts-first-test
(let [params [{:id "beta" :name "B"}
{:id nil :name "Nil"}
{:id "alpha" :name "A"}]
result (serdes/export-parameters params)]
(is (= [nil "alpha" "beta"] (map :id result)))))
(deftest ^:parallel export-parameter-mappings-sorts-by-parameter-id-test
(let [mappings [{:parameter_id "z-param" :target [:dimension [:field 1 nil]]}
{:parameter_id "a-param" :target [:dimension [:field 2 nil]]}
{:parameter_id "m-param" :target [:dimension [:field 3 nil]]}]
result (serdes/export-parameter-mappings mappings)]
(is (= ["a-param" "m-param" "z-param"]
(map :parameter_id result)))))
(deftest ^:parallel export-parameter-mappings-empty-input-test
(is (= [] (serdes/export-parameter-mappings []))))
(deftest ^:parallel export-parameter-mappings-nil-input-test
(is (= [] (serdes/export-parameter-mappings nil))))
(deftest ^:parallel export-parameter-mappings-single-element-test
(let [mappings [{:parameter_id "only" :target [:dimension [:field 1 nil]]}]
result (serdes/export-parameter-mappings mappings)]
(is (= ["only"] (map :parameter_id result)))))
(deftest dashboard-parameters-sorted-extraction-test
(mt/with-empty-h2-app-db!
(ts/with-temp-dpc [:model/Collection {coll-id :id} {:name "Test Collection"}
:model/Card {card-id :id} {:name "Source Card"
:collection_id coll-id}
:model/Dashboard {dash-id :id} {:name "Test Dashboard"
:collection_id coll-id
:parameters [{:id "zebra" :name "Z" :type :category}
{:id "alpha" :name "A" :type :category}
{:id "beta" :name "B" :type :category}]}
:model/DashboardCard _ {:dashboard_id dash-id
:card_id card-id
:parameter_mappings [{:parameter_id "zebra"
:card_id card-id
:target [:dimension [:field 1 nil]]}
{:parameter_id "alpha"
:card_id card-id
:target [:dimension [:field 2 nil]]}]}]
(let [ser (ts/extract-one "Dashboard" dash-id)]
(is (= ["alpha" "beta" "zebra"]
(map :id (:parameters ser))))
(is (= ["alpha" "zebra"]
(map :parameter_id (-> ser :dashcards first :parameter_mappings))))))))

View File

@@ -34,23 +34,6 @@ export * from "./main.bundle.js";
writeToFile("nextjs.cjs", nextjs_cjs);
writeToFile("nextjs.js", nextjs_js);
// Development mode entry point.
// When the host app bundler resolves the "development" exports condition,
// this file sets a window global so the SDK bundle can detect dev mode.
const devMode_js = `
try {
if (typeof window !== "undefined") {
window.METABASE_EMBEDDING_SDK_IS_HOST_APP_IN_DEV_MODE = true;
}
} catch (e) {
console.warn("Metabase SDK: Failed to set dev mode flag", e);
}
module.exports = require("./main.bundle");
`.trim();
writeToFile("main.development.js", devMode_js);
writeToFile(
"nextjs.d.ts",
`

View File

@@ -11,7 +11,6 @@
"exports": {
".": {
"types": "./dist/index.d.ts",
"development": "./dist/main.development.js",
"default": "./dist/main.bundle.js"
},
"./nextjs": {

View File

@@ -4,7 +4,6 @@ const CHART_ANALYSIS_ENABLED = {
// enabled
area: true,
bar: true,
boxplot: true,
combo: true,
funnel: true,
gauge: true,
@@ -45,7 +44,6 @@ type EnabledChartTypes = {
export const CHART_ANALYSIS_RENDER_FORMATS = {
area: "svg",
bar: "svg",
boxplot: "svg",
combo: "svg",
funnel: "png",
gauge: "svg",

View File

@@ -393,6 +393,7 @@ export const {
useGetWorkspaceTransformsQuery,
useGetExternalTransformsQuery,
useGetWorkspaceTransformQuery,
useLazyGetWorkspaceTransformQuery,
useCreateWorkspaceMutation,
useCreateWorkspaceTransformMutation,
useUpdateWorkspaceTransformMutation,

View File

@@ -10,15 +10,6 @@ import { useHasTokenFeature } from "metabase/common/hooks";
import { SectionLayout } from "metabase/data-studio/app/components/SectionLayout";
import { DataStudioBreadcrumbs } from "metabase/data-studio/common/components/DataStudioBreadcrumbs";
import { PaneHeader } from "metabase/data-studio/common/components/PaneHeader";
import { useBuildSnippetTree } from "metabase/data-studio/common/hooks/use-build-snippet-tree";
import type {
EmptyStateData,
TreeItem,
} from "metabase/data-studio/common/types";
import {
isCollection,
isEmptyStateData,
} from "metabase/data-studio/common/utils";
import type { ExpandedState } from "metabase/data-studio/data-model/components/TablePicker/types";
import { LibraryUpsellPage } from "metabase/data-studio/upsells/pages";
import { usePageTitle } from "metabase/hooks/use-page-title";
@@ -47,7 +38,17 @@ import { LibraryEmptyState } from "../components/LibraryEmptyState";
import { CreateMenu } from "./CreateMenu";
import { PublishTableModal } from "./PublishTableModal";
import { RootSnippetsCollectionMenu } from "./RootSnippetsCollectionMenu";
import { useBuildTreeForCollection, useErrorHandling } from "./hooks";
import {
useBuildSnippetTree,
useBuildTreeForCollection,
useErrorHandling,
} from "./hooks";
import {
type EmptyStateData,
type TreeItem,
isCollection,
isEmptyStateData,
} from "./types";
import { getAccessibleCollection, getWritableCollection } from "./utils";
interface EmptyStateActionProps {

View File

@@ -1,8 +1,6 @@
import { Link } from "react-router";
import { t } from "ttag";
import { useSelector } from "metabase/lib/redux";
import { dataStudioArchivedSnippets } from "metabase/lib/urls";
import { getUserIsAdmin } from "metabase/selectors/user";
import { ActionIcon, FixedSizeIcon, Menu, Tooltip } from "metabase/ui";
import { getIsRemoteSyncReadOnly } from "metabase-enterprise/remote_sync/selectors";
@@ -15,7 +13,10 @@ export const RootSnippetsCollectionMenu = ({
}) => {
const isAdmin = useSelector(getUserIsAdmin);
const remoteSyncReadOnly = useSelector(getIsRemoteSyncReadOnly);
const isReadOnly = remoteSyncReadOnly || !isAdmin;
if (!isAdmin || remoteSyncReadOnly) {
return null;
}
const optionsLabel = t`Snippet collection options`;
@@ -33,23 +34,14 @@ export const RootSnippetsCollectionMenu = ({
</Tooltip>
</Menu.Target>
<Menu.Dropdown>
{!isReadOnly && (
<Menu.Item
leftSection={<FixedSizeIcon name="lock" />}
onClick={(e) => {
e.stopPropagation();
setPermissionsCollectionId("root");
}}
>
{t`Change permissions`}
</Menu.Item>
)}
<Menu.Item
component={Link}
leftSection={<FixedSizeIcon name="view_archive" />}
to={dataStudioArchivedSnippets()}
leftSection={<FixedSizeIcon name="lock" />}
onClick={(e) => {
e.stopPropagation();
setPermissionsCollectionId("root");
}}
>
{t`View archived snippets`}
{t`Change permissions`}
</Menu.Item>
</Menu.Dropdown>
</Menu>

View File

@@ -1,9 +1,7 @@
import userEvent from "@testing-library/user-event";
import { Route } from "react-router";
import { mockSettings } from "__support__/settings";
import { renderWithProviders, screen } from "__support__/ui";
import { dataStudioArchivedSnippets } from "metabase/lib/urls";
import type { EnterpriseSettings } from "metabase-types/api";
import { createMockUser } from "metabase-types/api/mocks";
import { createMockState } from "metabase-types/store/mocks/state";
@@ -26,23 +24,11 @@ const setup = ({ isSuperuser = true, remoteSyncType }: SetupOptions = {}) => {
currentUser: createMockUser({ is_superuser: isSuperuser }),
});
return renderWithProviders(
<>
<Route
path="/"
component={() => (
<RootSnippetsCollectionMenu
setPermissionsCollectionId={setPermissionsCollectionId}
/>
)}
/>
<Route
path={dataStudioArchivedSnippets()}
component={() => <div data-testid="archived-snippets" />}
/>
</>,
<RootSnippetsCollectionMenu
setPermissionsCollectionId={setPermissionsCollectionId}
/>,
{
storeInitialState: state,
withRouter: true,
},
);
};
@@ -59,29 +45,21 @@ describe("RootSnippetsCollectionMenu", () => {
).toBeInTheDocument();
});
it("does not render 'change permissions' option if user is not a superuser", async () => {
it("renders nothing if user is not a superuser", () => {
setup({ isSuperuser: false });
await userEvent.click(
screen.getByRole("button", { name: "Snippet collection options" }),
);
expect(
screen.queryByRole("menuitem", { name: /Change permissions/ }),
screen.queryByRole("button", { name: "Snippet collection options" }),
).not.toBeInTheDocument();
});
it("does not render 'change permissions' option if remote sync is set to read-only", async () => {
it("renders nothing if remote sync is set to read-only", () => {
setup({ remoteSyncType: "read-only" });
await userEvent.click(
screen.getByRole("button", { name: "Snippet collection options" }),
);
expect(
screen.queryByRole("menuitem", { name: /Change permissions/ }),
screen.queryByRole("button", { name: "Snippet collection options" }),
).not.toBeInTheDocument();
});
describe("show permissions option", () => {
describe("show permissions options", () => {
it("is rendered on menu click", async () => {
setup();
await userEvent.click(
@@ -103,19 +81,4 @@ describe("RootSnippetsCollectionMenu", () => {
expect(setPermissionsCollectionId).toHaveBeenCalledWith("root");
});
});
describe("view archived snippets option", () => {
it("navigates to archived snippets page", async () => {
const { history } = setup();
await userEvent.click(
screen.getByRole("button", { name: "Snippet collection options" }),
);
await userEvent.click(
screen.getByRole("menuitem", { name: /View archived snippets/ }),
);
expect(history?.getCurrentLocation()?.pathname).toMatch(
/\/snippets\/archived/,
);
});
});
});

View File

@@ -2,17 +2,20 @@ import { useEffect, useMemo } from "react";
import { t } from "ttag";
import _ from "underscore";
import { skipToken, useListCollectionItemsQuery } from "metabase/api";
import {
skipToken,
useListCollectionItemsQuery,
useListCollectionsQuery,
useListSnippetsQuery,
} from "metabase/api";
import { useDebouncedValue } from "metabase/common/hooks/use-debounced-value";
import type {
LibrarySectionType,
TreeItem,
} from "metabase/data-studio/common/types";
import { createEmptyStateItem } from "metabase/data-studio/common/utils";
import { getIcon } from "metabase/lib/icon";
import { useMetadataToasts } from "metabase/metadata/hooks";
import type { Collection, CollectionId } from "metabase-types/api";
import type { LibrarySectionType, TreeItem } from "./types";
import { buildSnippetTree, createEmptyStateItem } from "./utils";
export const useBuildTreeForCollection = (
collection: Collection | undefined,
sectionType: LibrarySectionType,
@@ -68,6 +71,49 @@ export const useBuildTreeForCollection = (
}, [isLoading, items, collection, error, sectionType, metricCollectionId]);
};
export const useBuildSnippetTree = (): {
isLoading: boolean;
tree: TreeItem[];
error?: unknown;
} => {
const {
data: snippets,
isLoading: loadingSnippets,
error,
} = useListSnippetsQuery();
const { data: snippetCollections, isLoading: loadingCollections } =
useListCollectionsQuery({
namespace: "snippets",
});
return useMemo(() => {
if (
loadingSnippets ||
loadingCollections ||
!snippets ||
!snippetCollections
) {
return {
isLoading: true,
tree: [],
error,
};
}
return {
isLoading: false,
error,
tree: buildSnippetTree(snippetCollections, snippets),
};
}, [
loadingSnippets,
loadingCollections,
snippets,
snippetCollections,
error,
]);
};
export const useErrorHandling = (_error: unknown) => {
const error = useDebouncedValue(_error, 1000);
const { sendErrorToast } = useMetadataToasts();

View File

@@ -32,3 +32,15 @@ export type TreeItem = {
| EmptyStateData;
children?: TreeItem[];
};
export const isCollection = (
c: Collection | Omit<CollectionItem, "getUrl">,
): c is Collection => {
return Object.keys(c).includes("namespace");
};
export const isEmptyStateData = (
data: TreeItem["data"],
): data is EmptyStateData => {
return data.model === "empty-state";
};

View File

@@ -1,4 +1,86 @@
import type { Collection, CollectionType } from "metabase-types/api";
import { t } from "ttag";
import { isRootCollection } from "metabase/collections/utils";
import * as Urls from "metabase/lib/urls";
import type {
Collection,
CollectionId,
CollectionType,
NativeQuerySnippet,
} from "metabase-types/api";
import type { LibrarySectionType, TreeItem } from "./types";
function createSnippetNode(snippet: NativeQuerySnippet): TreeItem {
return {
id: `snippet:${snippet.id}`,
name: snippet.name,
icon: "snippet",
model: "snippet",
data: { ...snippet, model: "snippet" },
updatedAt: snippet.updated_at,
};
}
function buildCollectionNode(
collection: Collection,
allCollections: Collection[],
allSnippets: NativeQuerySnippet[],
): TreeItem {
const isRoot = isRootCollection(collection);
const parentIdToMatch = isRoot ? null : collection.id;
const childCollections = allCollections.filter(
(c) => c.parent_id === parentIdToMatch,
);
const childSnippets = allSnippets.filter(
(s) => s.collection_id === parentIdToMatch,
);
const children = [
...childCollections.map((child) =>
buildCollectionNode(child, allCollections, allSnippets),
),
...childSnippets.map(createSnippetNode),
];
return {
id: `collection:${collection.id}`,
name: collection.name,
model: "collection",
icon: isRoot ? "snippet" : "folder",
data: { ...collection, model: "collection" },
children: children.length > 0 ? children : undefined,
};
}
export function buildSnippetTree(
snippetCollections: Collection[],
snippets: NativeQuerySnippet[],
): TreeItem[] {
const collections = snippetCollections.filter((c) => !c.archived);
const activeSnippets = snippets.filter((s) => !s.archived);
const rootCollection = collections.find(isRootCollection);
if (!rootCollection) {
return [];
}
const nonRootCollections = collections.filter((c) => !isRootCollection(c));
const rootNode = buildCollectionNode(
rootCollection,
nonRootCollections,
activeSnippets,
);
// If the root has no children (no snippets or subfolders), add an empty state
const hasContent = activeSnippets.length > 0 || nonRootCollections.length > 0;
const children = hasContent
? rootNode.children
: [createEmptyStateItem("snippets")];
return [{ ...rootNode, name: t`SQL snippets`, children }];
}
export function getAccessibleCollection(
rootCollection: Collection,
@@ -16,3 +98,63 @@ export function getWritableCollection(
const collection = getAccessibleCollection(rootCollection, type);
return collection?.can_write ? collection : undefined;
}
type EmptyStateConfig = {
sectionType: LibrarySectionType;
description: string;
actionLabel: string;
actionUrl?: string;
};
function getEmptyStateConfig(
sectionType: LibrarySectionType,
): Omit<EmptyStateConfig, "sectionType" | "actionUrl"> {
const config: Record<
LibrarySectionType,
Omit<EmptyStateConfig, "sectionType" | "actionUrl">
> = {
data: {
description: t`Cleaned, pre-transformed data sources ready for exploring.`,
actionLabel: t`Publish a table`,
},
metrics: {
description: t`Standardized calculations with known dimensions.`,
actionLabel: t`New metric`,
},
snippets: {
description: t`Reusable bits of code that save your time.`,
actionLabel: t`New snippet`,
},
};
return config[sectionType];
}
export function createEmptyStateItem(
sectionType: LibrarySectionType,
metricCollectionId?: CollectionId,
): TreeItem {
const config = getEmptyStateConfig(sectionType);
let actionUrl: string | undefined;
if (sectionType === "metrics" && metricCollectionId) {
actionUrl = Urls.newDataStudioMetric({ collectionId: metricCollectionId });
} else if (sectionType === "snippets") {
actionUrl = Urls.newDataStudioSnippet();
}
// "data" section opens a modal, so no actionUrl
return {
id: `empty-state:${sectionType}`,
name: "",
icon: "empty",
model: "empty-state",
data: {
model: "empty-state",
sectionType,
description: config.description,
actionLabel: config.actionLabel,
actionUrl,
},
};
}

View File

@@ -6,48 +6,56 @@ import { useToast } from "metabase/common/hooks";
import { Button, Group, Modal, Stack, Text } from "metabase/ui";
import type { NativeQuerySnippet } from "metabase-types/api";
type ArchiveSnippetModalProps = {
snippet: Pick<NativeQuerySnippet, "id" | "name">;
type DeleteSnippetModalProps = {
snippet: NativeQuerySnippet;
onDelete?: () => void;
onClose: () => void;
};
export function UnarchiveSnippetModal(props: ArchiveSnippetModalProps) {
const { snippet, onClose } = props;
export function DeleteSnippetModal({
snippet,
onDelete,
onClose,
}: DeleteSnippetModalProps) {
const [updateSnippet, { isLoading }] = useUpdateSnippetMutation();
const [sendToast] = useToast();
const handleUnarchive = async () => {
const handleDelete = async () => {
const { error } = await updateSnippet({
id: snippet.id,
archived: false,
archived: true,
});
if (error) {
sendToast({
message: getErrorMessage(error, t`Failed to unarchive snippet`),
message: getErrorMessage(error, t`Failed to delete snippet`),
icon: "warning",
});
} else {
sendToast({
message: t`"${snippet.name}" unarchived`,
message: t`Snippet deleted`,
icon: "check",
});
onDelete?.();
onClose();
}
};
return (
<Modal opened onClose={onClose} title={t`Unarchive snippet?`}>
<Modal opened onClose={onClose} title={t`Delete snippet?`}>
<Stack>
<Text>{t`Are you sure you want to unarchive "${snippet.name}"?`}</Text>
<Text>
{t`Are you sure you want to delete "${snippet.name}"? This action cannot be undone.`}
</Text>
<Group gap="sm" justify="flex-end">
<Button onClick={onClose}>{t`Cancel`}</Button>
<Button
variant="filled"
onClick={handleUnarchive}
color="error"
onClick={handleDelete}
loading={isLoading}
>
{t`Unarchive`}
{t`Delete`}
</Button>
</Group>
</Stack>

View File

@@ -0,0 +1 @@
export { DeleteSnippetModal } from "./DeleteSnippetModal";

View File

@@ -1,5 +1,6 @@
import type { ReactNode } from "react";
import { Link } from "react-router";
import { push } from "react-router-redux";
import { t } from "ttag";
import { useUpdateSnippetMutation } from "metabase/api";
@@ -14,7 +15,7 @@ import {
PaneHeaderTabs,
} from "metabase/data-studio/common/components/PaneHeader";
import { useCollectionPath } from "metabase/data-studio/common/hooks/use-collection-path/useCollectionPath";
import { useSelector } from "metabase/lib/redux";
import { useDispatch, useSelector } from "metabase/lib/redux";
import * as Urls from "metabase/lib/urls";
import { PLUGIN_DEPENDENCIES, PLUGIN_REMOTE_SYNC } from "metabase/plugins";
import type { NativeQuerySnippet } from "metabase-types/api";
@@ -33,10 +34,15 @@ export function SnippetHeader({
actions,
...rest
}: SnippetHeaderProps & Omit<PaneHeaderProps, "breadcrumbs">) {
const dispatch = useDispatch();
const remoteSyncReadOnly = useSelector(
PLUGIN_REMOTE_SYNC.getIsRemoteSyncReadOnly,
);
const handleDelete = () => {
dispatch(push(Urls.dataStudioLibrary()));
};
const { path, isLoadingPath } = useCollectionPath({
collectionId: snippet.collection_id,
});
@@ -44,12 +50,13 @@ export function SnippetHeader({
return (
<PaneHeader
title={
<SnippetNameInput
snippet={snippet}
readOnly={remoteSyncReadOnly || !!snippet?.archived}
/>
<SnippetNameInput snippet={snippet} readOnly={remoteSyncReadOnly} />
}
menu={
remoteSyncReadOnly ? null : (
<SnippetMoreMenu snippet={snippet} onDelete={handleDelete} />
)
}
menu={remoteSyncReadOnly ? null : <SnippetMoreMenu snippet={snippet} />}
tabs={<SnippetTabs snippet={snippet} />}
actions={actions}
data-testid="snippet-header"
@@ -62,20 +69,11 @@ export function SnippetHeader({
{path?.map((collection, i) => (
<Link
key={collection.id}
to={
collection.type === "trash" || collection.archived
? Urls.dataStudioArchivedSnippets()
: Urls.dataStudioLibrary({
expandedIds: [
"root",
...path.slice(0, i + 1).map((c) => c.id),
],
})
}
to={Urls.dataStudioLibrary({
expandedIds: ["root", ...path.slice(0, i + 1).map((c) => c.id)],
})}
>
{collection.type === "trash"
? t`Archived snippets`
: collection.name}
{collection.name}
</Link>
))}
<span>{snippet.name}</span>

View File

@@ -1,75 +0,0 @@
import { Link } from "react-router";
import { push } from "react-router-redux";
import { t } from "ttag";
import { useUpdateSnippetMutation } from "metabase/api";
import { getErrorMessage } from "metabase/api/utils";
import { useToast } from "metabase/common/hooks";
import { useDispatch } from "metabase/lib/redux";
import * as Urls from "metabase/lib/urls";
import { Button, Group, Modal, Stack, Text } from "metabase/ui";
import type { NativeQuerySnippet } from "metabase-types/api";
type ArchiveSnippetModalProps = {
snippet: NativeQuerySnippet;
onClose: () => void;
};
export function ArchiveSnippetModal(props: ArchiveSnippetModalProps) {
const { snippet, onClose } = props;
const dispatch = useDispatch();
const [updateSnippet, { isLoading }] = useUpdateSnippetMutation();
const [sendToast] = useToast();
const handleArchive = async () => {
const { error } = await updateSnippet({
id: snippet.id,
archived: true,
});
if (error) {
sendToast({
message: getErrorMessage(error, t`Failed to archive snippet`),
icon: "warning",
});
} else {
sendToast({
message: (
<>
{t`"${snippet.name}" archived.`}{" "}
<Text
c="inherit"
component={Link}
td="underline"
to={Urls.dataStudioArchivedSnippets()}
>
{t`View archived snippets`}
</Text>
</>
),
icon: "check",
});
dispatch(push(Urls.dataStudioLibrary()));
onClose();
}
};
return (
<Modal opened onClose={onClose} title={t`Archive snippet?`}>
<Stack>
<Text>{t`Are you sure you want to archive "${snippet.name}"?`}</Text>
<Group gap="sm" justify="flex-end">
<Button onClick={onClose}>{t`Cancel`}</Button>
<Button
color="error"
loading={isLoading}
onClick={handleArchive}
variant="filled"
>
{t`Archive`}
</Button>
</Group>
</Stack>
</Modal>
);
}

View File

@@ -1,35 +0,0 @@
import { PLUGIN_SNIPPET_FOLDERS } from "metabase/plugins";
import type { NativeQuerySnippet } from "metabase-types/api";
import { ArchiveSnippetModal } from "./ArchiveSnippetModal";
import { UnarchiveSnippetModal } from "./UnarchiveSnippetModal";
export type SnippetModalType = "move" | "archive" | "unarchive";
type SnippetModalProps = {
snippet: NativeQuerySnippet;
modalType: SnippetModalType;
onClose: () => void;
};
export function SnippetModal({
snippet,
modalType,
onClose,
}: SnippetModalProps) {
switch (modalType) {
case "move":
return (
<PLUGIN_SNIPPET_FOLDERS.MoveSnippetModal
snippet={snippet}
onClose={onClose}
/>
);
case "archive":
return <ArchiveSnippetModal snippet={snippet} onClose={onClose} />;
case "unarchive":
return <UnarchiveSnippetModal snippet={snippet} onClose={onClose} />;
default:
return null;
}
}

View File

@@ -1 +0,0 @@
export { SnippetModal, type SnippetModalType } from "./SnippetModal";

View File

@@ -5,22 +5,26 @@ import { PLUGIN_SNIPPET_FOLDERS } from "metabase/plugins";
import { ActionIcon, Icon, Menu } from "metabase/ui";
import type { NativeQuerySnippet } from "metabase-types/api";
import { SnippetModal, type SnippetModalType } from "./SnippetModal";
import { DeleteSnippetModal } from "../DeleteSnippetModal";
type SnippetModalType = "move" | "delete";
type SnippetMoreMenuProps = {
snippet: NativeQuerySnippet;
onDelete?: () => void;
};
export function SnippetMoreMenu({ snippet }: SnippetMoreMenuProps) {
export function SnippetMoreMenu({ snippet, onDelete }: SnippetMoreMenuProps) {
const [modalType, setModalType] = useState<SnippetModalType>();
return (
<>
<SnippetMenu onOpenModal={setModalType} snippet={snippet} />
{!!modalType && (
<SnippetMenu onOpenModal={setModalType} />
{modalType != null && (
<SnippetModal
snippet={snippet}
modalType={modalType}
onDelete={onDelete}
onClose={() => setModalType(undefined)}
/>
)}
@@ -30,13 +34,12 @@ export function SnippetMoreMenu({ snippet }: SnippetMoreMenuProps) {
type SnippetMenuProps = {
onOpenModal: (modalType: SnippetModalType) => void;
snippet: NativeQuerySnippet;
};
function SnippetMenu({ onOpenModal, snippet }: SnippetMenuProps) {
function SnippetMenu({ onOpenModal }: SnippetMenuProps) {
const menuItems = [];
if (PLUGIN_SNIPPET_FOLDERS.isEnabled && !snippet.archived) {
if (PLUGIN_SNIPPET_FOLDERS.isEnabled) {
menuItems.push(
<Menu.Item
key="move"
@@ -48,27 +51,15 @@ function SnippetMenu({ onOpenModal, snippet }: SnippetMenuProps) {
);
}
if (snippet.archived) {
menuItems.push(
<Menu.Item
key="archive"
leftSection={<Icon name="unarchive" />}
onClick={() => onOpenModal("unarchive")}
>
{t`Unarchive`}
</Menu.Item>,
);
} else {
menuItems.push(
<Menu.Item
key="archive"
leftSection={<Icon name="archive" />}
onClick={() => onOpenModal("archive")}
>
{t`Archive`}
</Menu.Item>,
);
}
menuItems.push(
<Menu.Item
key="delete"
leftSection={<Icon name="trash" />}
onClick={() => onOpenModal("delete")}
>
{t`Delete`}
</Menu.Item>,
);
if (menuItems.length === 0) {
return null;
@@ -85,3 +76,37 @@ function SnippetMenu({ onOpenModal, snippet }: SnippetMenuProps) {
</Menu>
);
}
type SnippetModalProps = {
snippet: NativeQuerySnippet;
modalType: SnippetModalType;
onDelete?: () => void;
onClose: () => void;
};
function SnippetModal({
snippet,
modalType,
onDelete,
onClose,
}: SnippetModalProps) {
switch (modalType) {
case "move":
return (
<PLUGIN_SNIPPET_FOLDERS.MoveSnippetModal
snippet={snippet}
onClose={onClose}
/>
);
case "delete":
return (
<DeleteSnippetModal
snippet={snippet}
onDelete={onDelete}
onClose={onClose}
/>
);
default:
return null;
}
}

View File

@@ -1,135 +0,0 @@
import { useCallback, useState } from "react";
import { Link } from "react-router";
import { t } from "ttag";
import { useUpdateSnippetMutation } from "metabase/api";
import { getErrorMessage } from "metabase/api/utils";
import { LoadingAndErrorWrapper } from "metabase/common/components/LoadingAndErrorWrapper";
import { SectionLayout } from "metabase/data-studio/app/components/SectionLayout";
import { DataStudioBreadcrumbs } from "metabase/data-studio/common/components/DataStudioBreadcrumbs";
import { PaneHeader } from "metabase/data-studio/common/components/PaneHeader";
import { useBuildSnippetTree } from "metabase/data-studio/common/hooks/use-build-snippet-tree";
import * as Urls from "metabase/lib/urls";
import { useMetadataToasts } from "metabase/metadata/hooks";
import {
Card,
Center,
Icon,
Stack,
TextInput,
TreeTable,
TreeTableSkeleton,
useTreeTableInstance,
} from "metabase/ui";
import type { CollectionItem } from "metabase-types/api";
import { useColumnDef } from "./hooks/useColumnDef";
export function ArchivedSnippetsPage() {
const [searchQuery, setSearchQuery] = useState("");
const {
tree: snippetTree,
isLoading,
error,
} = useBuildSnippetTree({ archived: true });
const { sendSuccessToast, sendErrorToast } = useMetadataToasts();
const [updateSnippet] = useUpdateSnippetMutation();
const handleUnarchiveClick = useCallback(
async (item: CollectionItem) => {
try {
await updateSnippet({
id: item.id,
archived: false,
}).unwrap();
sendSuccessToast(t`"${item.name}" unarchived`);
} catch (error) {
sendErrorToast(getErrorMessage(error, t`Failed to unarchive snippet`));
}
},
[sendErrorToast, sendSuccessToast, updateSnippet],
);
const columns = useColumnDef({ handleUnarchiveClick });
const treeTableInstance = useTreeTableInstance({
data: snippetTree,
columns,
getSubRows: (node) => node.children,
getNodeId: (node) => node.id,
globalFilter: searchQuery,
onGlobalFilterChange: setSearchQuery,
defaultExpanded: true,
});
if (isLoading || error) {
return (
<Center h="100%">
<LoadingAndErrorWrapper loading={isLoading} error={error} />
</Center>
);
}
return (
<SectionLayout>
<PaneHeader
breadcrumbs={
<DataStudioBreadcrumbs>
<Link to={Urls.dataStudioLibrary()}>{t`SQL snippets`}</Link>
{t`Archived snippets`}
</DataStudioBreadcrumbs>
}
px="3.5rem"
py={0}
/>
<Stack
bg="background-secondary"
data-testid="archived-snippets-page"
pb="2rem"
px="3.5rem"
style={{ overflow: "hidden" }}
>
<TextInput
placeholder={t`Search...`}
leftSection={<Icon name="search" />}
bdrs="md"
value={searchQuery}
onChange={(e) => setSearchQuery(e.target.value)}
/>
<Card withBorder p={0}>
{isLoading ? (
<TreeTableSkeleton columnWidths={[0.6, 0.2, 0.05]} />
) : (
<TreeTable
instance={treeTableInstance}
emptyState={t`No archived snippets`}
onRowClick={(row) => {
const { data } = row.original;
if (data.model === "empty-state") {
return;
}
if (row.getCanExpand()) {
row.toggleExpanded();
return;
}
}}
getRowHref={(row) => {
const { data } = row.original;
if (data.model === "snippet") {
const snippetId = Number(data.id);
return Urls.dataStudioSnippet(snippetId);
}
return null;
}}
/>
)}
</Card>
</Stack>
</SectionLayout>
);
}

View File

@@ -1,78 +0,0 @@
import { useMemo } from "react";
import { t } from "ttag";
import type { TreeItem } from "metabase/data-studio/common/types";
import {
isCollection,
isEmptyStateData,
} from "metabase/data-studio/common/utils";
import { useSelector } from "metabase/lib/redux";
import { PLUGIN_SNIPPET_FOLDERS } from "metabase/plugins";
import {
ActionIcon,
EntityNameCell,
FixedSizeIcon,
Tooltip,
type TreeTableColumnDef,
} from "metabase/ui";
import { getIsRemoteSyncReadOnly } from "metabase-enterprise/remote_sync/selectors";
import type { CollectionItem } from "metabase-types/api";
type ColumnDefProps = {
handleUnarchiveClick: (item: CollectionItem) => void;
};
export const useColumnDef = ({ handleUnarchiveClick }: ColumnDefProps) => {
const remoteSyncReadOnly = useSelector(getIsRemoteSyncReadOnly);
return useMemo<TreeTableColumnDef<TreeItem>[]>(
() => [
{
id: "name",
header: t`Name`,
enableSorting: true,
accessorKey: "name",
minWidth: 200,
cell: ({ row }) => (
<EntityNameCell
data-testid={`${row.original.model}-name`}
icon={row.original.icon}
name={row.original.name}
/>
),
},
{
id: "actions",
width: 48,
cell: ({ row }) => {
const { data } = row.original;
if (remoteSyncReadOnly || isEmptyStateData(data)) {
return null;
}
if (isCollection(data)) {
return <PLUGIN_SNIPPET_FOLDERS.CollectionMenu collection={data} />;
}
return (
<Tooltip label={t`Unarchive snippet`}>
<ActionIcon
aria-label={t`Unarchive snippet`}
size="md"
onClick={async (event) => {
event.stopPropagation();
event.preventDefault();
handleUnarchiveClick(data);
}}
>
<FixedSizeIcon name={"unarchive"} c="text-primary" />
</ActionIcon>
</Tooltip>
);
},
},
],
[handleUnarchiveClick, remoteSyncReadOnly],
);
};

View File

@@ -1 +0,0 @@
export { ArchivedSnippetsPage } from "./ArchivedSnippetsPage";

View File

@@ -11,7 +11,3 @@
flex-shrink: 0;
border-left: 1px solid var(--mb-color-border);
}
.flexStart {
align-self: flex-start;
}

View File

@@ -12,7 +12,6 @@ import {
import { getErrorMessage } from "metabase/api/utils";
import { CodeMirror } from "metabase/common/components/CodeMirror";
import { EntityCreationInfo } from "metabase/common/components/EntityCreationInfo";
import { NotFound } from "metabase/common/components/ErrorPages";
import { LeaveRouteConfirmModal } from "metabase/common/components/LeaveConfirmModal";
import { LoadingAndErrorWrapper } from "metabase/common/components/LoadingAndErrorWrapper";
import { useToast } from "metabase/common/hooks";
@@ -21,7 +20,7 @@ import { PaneHeaderActions } from "metabase/data-studio/common/components/PaneHe
import { useSelector } from "metabase/lib/redux";
import * as Urls from "metabase/lib/urls";
import { PLUGIN_REMOTE_SYNC } from "metabase/plugins";
import { Alert, Card, Center, Flex, Stack } from "metabase/ui";
import { Card, Center, Flex, Stack } from "metabase/ui";
import { SnippetDescriptionSection } from "../../components/SnippetDescriptionSection";
import { SnippetHeader } from "../../components/SnippetHeader";
@@ -52,7 +51,6 @@ export function EditSnippetPage({ params, route }: EditSnippetPageProps) {
const [content, setContent] = useState(snippet?.content ?? "");
const [updateSnippet, { isLoading: isSaving }] = useUpdateSnippetMutation();
const isReadOnly = remoteSyncReadOnly || !!snippet?.archived;
const isDirty = useMemo(
() => snippet != null && content !== snippet.content,
@@ -97,10 +95,6 @@ export function EditSnippetPage({ params, route }: EditSnippetPageProps) {
const extensions = useMemo(() => [sql()], []);
if (!snippetId) {
return <NotFound />;
}
if (isLoading || error || !snippet) {
return (
<Center h="100%">
@@ -111,7 +105,7 @@ export function EditSnippetPage({ params, route }: EditSnippetPageProps) {
return (
<>
<PageContainer pos="relative" data-testid="edit-snippet-page" gap="md">
<PageContainer pos="relative" data-testid="edit-snippet-page">
<SnippetHeader
snippet={snippet}
actions={
@@ -124,21 +118,7 @@ export function EditSnippetPage({ params, route }: EditSnippetPageProps) {
/>
}
/>
{isReadOnly && (
<Alert
className={S.flexStart}
color="warning"
p="0.75rem"
title={
snippet?.archived
? t`This snippet is archived and cannot be edited. Unarchive it to edit.`
: t`This snippet is not editable because Remote Sync is in read-only mode.`
}
variant="outline"
w="auto"
/>
)}
<Flex flex={1} w="100%" gap="sm" mt="md">
<Flex flex={1} w="100%" gap="sm">
<Card
withBorder
p={0}
@@ -155,8 +135,7 @@ export function EditSnippetPage({ params, route }: EditSnippetPageProps) {
height="100%"
className={S.editor}
data-testid="snippet-editor"
editable={!isReadOnly}
readOnly={isReadOnly}
editable={!remoteSyncReadOnly}
basicSetup={{
lineNumbers: true,
foldGutter: true,
@@ -168,7 +147,7 @@ export function EditSnippetPage({ params, route }: EditSnippetPageProps) {
<Stack p="md" gap="lg" flex="0 0 20rem">
<SnippetDescriptionSection
snippet={snippet}
isDisabled={isReadOnly}
isDisabled={remoteSyncReadOnly}
/>
<EntityCreationInfo
createdAt={snippet.created_at}

View File

@@ -2,7 +2,6 @@ import { IndexRoute, Route } from "react-router";
import { PLUGIN_DEPENDENCIES } from "metabase/plugins";
import { ArchivedSnippetsPage } from "./pages/ArchivedSnippetsPage";
import { EditSnippetPage } from "./pages/EditSnippetPage";
import { NewSnippetPage } from "./pages/NewSnippetPage";
import { SnippetDependenciesPage } from "./pages/SnippetDependenciesPage";
@@ -11,7 +10,6 @@ export function getDataStudioSnippetRoutes() {
return (
<>
<Route path="snippets/new" component={NewSnippetPage} />
<Route path="snippets/archived" component={ArchivedSnippetsPage} />
<Route path="snippets/:snippetId" component={EditSnippetPage} />
{PLUGIN_DEPENDENCIES.isEnabled && (
<Route

View File

@@ -6,6 +6,7 @@ import {
} from "metabase/transforms/pages/NewTransformPage/utils";
import { ActionIcon, Icon, Menu, Tooltip } from "metabase/ui";
import type { DatabaseId, TransformSource } from "metabase-types/api";
import { createMockTransform } from "metabase-types/api/mocks/transform";
import { useWorkspace } from "./WorkspaceProvider";
@@ -53,7 +54,11 @@ export const AddTransformMenu = ({ databaseId, disabled }: Props) => {
<Menu.Item
leftSection={<Icon name="sql" />}
onClick={() => {
addUnsavedTransform(getSource("sql"));
const mockTransform = createMockTransform({
source: getSource("sql"),
});
addUnsavedTransform(mockTransform.source);
}}
>
{t`SQL Transform`}
@@ -61,7 +66,11 @@ export const AddTransformMenu = ({ databaseId, disabled }: Props) => {
<Menu.Item
leftSection={<Icon name="code_block" />}
onClick={() => {
addUnsavedTransform(getSource("python"));
const mockTransform = createMockTransform({
source: getSource("python"),
});
addUnsavedTransform(mockTransform.source);
}}
>
{t`Python Script`}

View File

@@ -1,7 +1,7 @@
import { useCallback } from "react";
import { t } from "ttag";
import { skipToken } from "metabase/api";
import { skipToken, useLazyGetTransformQuery } from "metabase/api";
import { EmptyState } from "metabase/common/components/EmptyState";
import { Flex, Skeleton, Stack, Text } from "metabase/ui";
import { useGetExternalTransformsQuery } from "metabase-enterprise/api";
@@ -9,16 +9,15 @@ import { getCheckoutDisabledMessage } from "metabase-enterprise/data-studio/work
import type {
DatabaseId,
ExternalTransform,
TaggedTransform,
UnsavedTransform,
WorkspaceId,
WorkspaceTransform,
WorkspaceTransformListItem,
} from "metabase-types/api";
import { isUnsavedTransform } from "metabase-types/api";
import {
type AnyWorkspaceTransformRef,
useWorkspace,
} from "../WorkspaceProvider";
import { useWorkspace } from "../WorkspaceProvider";
import { TransformListItem } from "./TransformListItem";
import { TransformListItemMenu } from "./TransformListItemMenu";
@@ -42,7 +41,9 @@ type CodeTabProps = {
workspaceTransforms: WorkspaceTransformItem[];
isLoadingWorkspaceTransforms?: boolean;
readOnly: boolean;
onTransformClick: (transformRef: AnyWorkspaceTransformRef) => void;
onTransformClick: (
transform: TaggedTransform | WorkspaceTransform | UnsavedTransform,
) => void;
};
export const CodeTab = ({
@@ -55,6 +56,7 @@ export const CodeTab = ({
onTransformClick,
}: CodeTabProps) => {
const { editedTransforms } = useWorkspace();
const { sendErrorToast } = useMetadataToasts();
const {
data: availableTransforms,
@@ -64,38 +66,59 @@ export const CodeTab = ({
databaseId ? { workspaceId, databaseId } : skipToken,
);
const [fetchWorkspaceTransform] = useLazyGetWorkspaceTransformQuery();
const [fetchTransform] = useLazyGetTransformQuery();
const handleExternalTransformClick = useCallback(
async (externalTransform: ExternalTransform) => {
if (externalTransform.checkout_disabled) {
return;
}
onTransformClick({
type: "transform",
id: externalTransform.id,
name: externalTransform.name,
});
const { data: transform, error } = await fetchTransform(
externalTransform.id,
true,
);
if (error) {
sendErrorToast(t`Failed to fetch transform`);
} else if (transform) {
const taggedTransform: TaggedTransform = {
...transform,
type: "transform",
};
onTransformClick(taggedTransform);
}
},
[onTransformClick],
[fetchTransform, onTransformClick, sendErrorToast],
);
const handleWorkspaceTransformItemClick = useCallback(
async (item: WorkspaceTransformItem) => {
// Unsaved transforms are stored locally.
// They should open directly without API fetch.
if (isUnsavedTransform(item)) {
onTransformClick({
type: "unsaved-transform",
id: item.id,
name: item.name,
});
} else {
onTransformClick({
type: "workspace-transform",
ref_id: item.ref_id,
name: item.name,
});
// Unsaved transforms should already be opened via the provider
// This path shouldn't normally be hit, but if it is, we can't
// call onTransformClick because we don't have a full transform
return onTransformClick(item);
}
const { data: transform, error } = await fetchWorkspaceTransform(
{
workspaceId,
transformId: item.ref_id,
},
true,
);
if (error) {
sendErrorToast(t`Failed to fetch transform`);
} else if (transform) {
onTransformClick(transform);
}
},
[onTransformClick],
[fetchWorkspaceTransform, workspaceId, onTransformClick, sendErrorToast],
);
return (

Some files were not shown because too many files have changed in this diff Show More