Build tests against the FIPS 140-3 validated Go crypto module by ilyakuz-db · Pull Request #6193 · databricks/cli

ilyakuz-db · 2026-08-06T15:42:24Z

Changes

Sets GOFIPS140=v1.0.0 in Taskfile.yml, so every task invocation builds the CLI against the FIPS 140-3 validated Go Cryptographic Module. Adds a test that reads the built binary's build info to confirm it really was built with it.

Why

Releases are moving to FIPS builds (#6262). Go picks its cryptographic module at build time, so testing without it exercises a configuration we no longer ship.

In the Taskfile rather than a workflow env because every consumer goes through task — local ./task test, CI's task test, and the integration suite, which runs task integration from eng-dev-ecosystem. One line covers all three, with no cross-repo change. Bare go test outside task is not covered, so the check fails there rather than silently passing.

Worth flagging: this makes eng-dev-ecosystem integration runs FIPS without a change in that repo.

Tests

Full unit + acceptance suite under FIPS, locally and in CI (all six cells, macOS and Windows included) — no failures. Integration against real workspaces under FIPS: ~11,000 tests across six cells (three clouds x linux/windows), and the FIPS check passed in every cell. The four unrelated failures were a leftover-state 409 on AWS and three cluster timing flakes; baseline nightlies show the same rate without FIPS.

Perf: CLI startup 12ms -> 13ms, RSA-4096 key generation 653ms -> 1.35s. The new build tag invalidates the Go build cache once, so the first build after this merges is a full rebuild.

eng-dev-ecosystem-bot · 2026-08-07T15:19:47Z

Integration test report

Commit: 3b0ca32

Run: 31800250246

	Env	🟨KNOWN	🔄flaky	💚RECOVERED	🙈SKIP	✅pass	🙈skip	Time
🟨	aws linux	3		1	4	288	1145	6:24
🟨	aws windows	3		1	4	290	1143	7:06
🟨	azure linux	3	2	1	4	285	1145	8:17
🟨	azure windows	3		1	4	289	1143	7:56
💚	gcp linux			1	5	288	1145	4:57
💚	gcp windows			1	5	290	1143	6:12

10 interesting tests: 4 SKIP, 3 KNOWN, 2 flaky, 1 RECOVERED

	Test Name	aws linux	aws windows	azure linux	azure windows	gcp linux	gcp windows
💚	TestAccept	💚R	💚R	💚R	💚R	💚R	💚R
🙈	TestAccept/bundle/invariant/no_drift	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/ssh/connection	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🔄	TestFsCpFileToDirWithOverwriteFlag	✅p	✅p	🔄f	✅p	✅p	✅p
🔄	TestFsCpFileToDirWithOverwriteFlag/uc-volumes_to_dbfs	✅p	✅p	🔄f	✅p	✅p	✅p
🟨	TestFetchRepositoryInfoAPI_FromRepo	🟨K	🟨K	🟨K	🟨K	🙈S	🙈S
🟨	TestFetchRepositoryInfoAPI_FromRepo/root	🟨K	🟨K	🟨K	🟨K
🟨	TestFetchRepositoryInfoAPI_FromRepo/subdir	🟨K	🟨K	🟨K	🟨K

Top 6 slowest tests (at least 2 minutes):

duration	env	testname
6:02	gcp windows	TestAccept
5:40	aws windows	TestAccept
5:32	azure windows	TestAccept
3:04	azure linux	TestAccept
3:03	aws linux	TestAccept
2:56	gcp linux	TestAccept

Co-authored-by: Isaac

…arate job

denik · 2026-08-13T15:58:24Z

+	cwd, err := os.Getwd()
+	require.NoError(t, err)
+
+	execPath := BuildCLI(t, getBuildDir(t, cwd, runtime.GOOS, runtime.GOARCH), "", runtime.GOOS, runtime.GOARCH)


We already build CLI in acceptance test runner, we can add this check there as a parallel test.

Yes, regular go version -m in acceptance test should be good enough check I think, I'll update this

denik · 2026-08-13T15:59:02Z

+func TestCLIBuiltWithFIPSModule(t *testing.T) {
+	// Required: the integration suite runs this package too (task integration passes
+	// ./acceptance), and it runs from eng-dev-ecosystem, which does not set GOFIPS140.
+	if os.Getenv("GOFIPS140") == "" {


I'd not add this skip if it's important that we test it. We can just enable env var here and in eng-dev-ecosystem and then add a test if needed.

I wanted to keep it as skipped to have more info. With this we can verify that env is correctly/incorrectly set in other pipelines correctly, but we don't actually block anything as it's not failing whole suite

I'll drop it once all other places are merged

denik · 2026-08-13T15:59:27Z

+
+	execPath := BuildCLI(t, getBuildDir(t, cwd, runtime.GOOS, runtime.GOARCH), "", runtime.GOOS, runtime.GOARCH)
+
+	out, err := exec.Command("go", "version", "-m", execPath).Output()


this could probably be regular acceptance test? go version -m $CLI

True, good idea

denik · 2026-08-13T16:01:02Z

          ENVFILTER: DATABRICKS_BUNDLE_ENGINE=${{ matrix.deployment }}
+          # Pinned to a frozen version: only v1.0.0 has a CMVP certificate, and
+          # "latest" tracks the in-tree source.
+          GOFIPS140: v1.0.0


Why not add this env var to Taskfile.yml? Then everyone gets it.

pietern · 2026-08-14T10:19:25Z

@@ -0,0 +1 @@
+trace go version -m $CLI | contains.py 'GOFIPS140=v1.0.0' 'DefaultGODEBUG=fips140=on' > LOG.buildinfo


Nit: can you write the grep -i fips matches to output.txt as well (e.g. via tee)

Nice to see the actual values committed as well.

pietern · 2026-08-14T12:17:45Z

@@ -0,0 +1 @@
+trace go version -m $CLI | grep -i fips140


Btw the "contains" assert is still good because it fails the test if the match doesn't happen.

Output changes are visible but can be glossed over.

Fixed

Failed test looks like this:

--- Expected +++ Actual @@ -2,5 +2,6 @@ >>> go version -m [CLI] - build -tags=fips140v1.0 - build DefaultGODEBUG=fips140=on - build GOFIPS140=v1.0.0-c2097c7c +contains error: 'DefaultGODEBUG=fips140=on' not found in the output. +contains error: 'GOFIPS140=v1.0.0' not found in the output. +Exit code: 1

Run acceptance tests against a FIPS build in CI

8b08765

Co-authored-by: Isaac

ilyakuz-db force-pushed the fips140-test-coverage branch from 4493134 to 8b08765 Compare August 10, 2026 10:12

ilyakuz-db changed the title ~~Add tests asserting the linked FIPS 140 crypto module~~ Run acceptance tests against a FIPS build in CI Aug 10, 2026

Correct the FIPS job comment: it does not cover TLS

4c33fa2

ilyakuz-db mentioned this pull request Aug 13, 2026

Build releases against the FIPS 140-3 validated Go crypto module #6262

Open

ilyakuz-db added 2 commits August 13, 2026 12:03

Run the existing test matrix against the FIPS module instead of a sep…

8928037

…arate job

Trim comments

78c35f0

denik reviewed Aug 13, 2026

View reviewed changes

Move GOFIPS140 to Taskfile so every task invocation gets it

9f1ce22

ilyakuz-db changed the title ~~Run acceptance tests against a FIPS build in CI~~ Build tests against the FIPS 140-3 validated Go crypto module Aug 13, 2026

Replace the FIPS Go test with an acceptance test

63a1c61

pietern approved these changes Aug 14, 2026

View reviewed changes

Commit the FIPS build values to the acceptance golden

190faac

pietern reviewed Aug 14, 2026

View reviewed changes

Assert FIPS values explicitly as well as committing them

3b0ca32

ilyakuz-db added this pull request to the merge queue Aug 14, 2026

Merged via the queue into main with commit aa56579 Aug 14, 2026
32 checks passed

ilyakuz-db deleted the fips140-test-coverage branch August 14, 2026 13:35

Navigation Menu

Uh oh!

Build tests against the FIPS 140-3 validated Go crypto module - #6193

Build tests against the FIPS 140-3 validated Go crypto module#6193
ilyakuz-db merged 8 commits into
mainfrom
fips140-test-coverage

ilyakuz-db commented Aug 6, 2026 •

edited

Loading

Uh oh!

eng-dev-ecosystem-bot commented Aug 7, 2026 •

edited

Loading

Uh oh!

denik Aug 13, 2026

Uh oh!

ilyakuz-db Aug 13, 2026 •

edited

Loading

Uh oh!

denik Aug 13, 2026

Uh oh!

ilyakuz-db Aug 13, 2026

Uh oh!

denik Aug 13, 2026

Uh oh!

ilyakuz-db Aug 13, 2026

Uh oh!

denik Aug 13, 2026

Uh oh!

pietern Aug 14, 2026

Uh oh!

pietern Aug 14, 2026

Uh oh!

ilyakuz-db Aug 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants


		execPath := BuildCLI(t, getBuildDir(t, cwd, runtime.GOOS, runtime.GOARCH), "", runtime.GOOS, runtime.GOARCH)

		out, err := exec.Command("go", "version", "-m", execPath).Output()

		@@ -0,0 +1 @@
		trace go version -m $CLI \| contains.py 'GOFIPS140=v1.0.0' 'DefaultGODEBUG=fips140=on' > LOG.buildinfo

Uh oh!

Conversation

ilyakuz-db commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Why

Tests

Uh oh!

eng-dev-ecosystem-bot commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Integration test report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ilyakuz-db Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ilyakuz-db commented Aug 6, 2026 •

edited

Loading

eng-dev-ecosystem-bot commented Aug 7, 2026 •

edited

Loading

ilyakuz-db Aug 13, 2026 •

edited

Loading