Skip to content

Add fuzz testing with CI and fix snapshot ID panic#1165

Open
arska wants to merge 2 commits intomasterfrom
add-fuzz-tests
Open

Add fuzz testing with CI and fix snapshot ID panic#1165
arska wants to merge 2 commits intomasterfrom
add-fuzz-tests

Conversation

@arska
Copy link
Copy Markdown
Member

@arska arska commented Mar 22, 2026

Summary

  • Add Go native fuzz tests for 3 targets that process untrusted input
  • Fix a real panic found by fuzzing: snapshot.ID[:8] crashes on IDs shorter than 8 chars
  • Add CI workflow that runs each fuzz target for 60 seconds on PRs and pushes

Bug found and fixed

filterAndConvert() in restic/kubernetes/snapshots.go:103 used snapshot.ID[:8] without checking length. If restic ever returns a snapshot with an ID shorter than 8 characters, the operator panics. Fixed by checking length before slicing.

Fuzz targets

Target Package What it tests
FuzzFilterAndConvert restic/kubernetes Snapshot ID processing from restic output
FuzzBackupOutputParser restic/logging Restic JSON backup output parsing
FuzzJsonArgsArrayUnmarshalJSON operator/utils Custom JSON unmarshaling of string-or-array

CI integration

Each fuzz target runs for 60 seconds in parallel via matrix strategy. Short runs catch regressions; longer runs can be done locally (go test -fuzz=... -fuzztime=5m) or via oss-fuzz.

Running locally

# Run a specific fuzz target for 5 minutes
go test ./restic/kubernetes/ -fuzz=FuzzFilterAndConvert -fuzztime=5m

# Run all fuzz targets briefly
go test ./restic/kubernetes/ -fuzz=FuzzFilterAndConvert -fuzztime=30s
go test ./restic/logging/ -fuzz=FuzzBackupOutputParser -fuzztime=30s
go test ./operator/utils/ -fuzz=FuzzJsonArgsArrayUnmarshalJSON -fuzztime=30s

Test plan

  • Fuzz tests find the snapshot ID panic (seed corpus)
  • Fix passes 15+ seconds of fuzzing with no crashes
  • All three fuzz targets run cleanly locally
  • CI workflow runs successfully

🤖 Generated with Claude Code

@arska arska requested a review from a team as a code owner March 22, 2026 15:38
@arska arska requested review from Kidswiss and lieneluksika and removed request for a team March 22, 2026 15:38
@arska arska added bug Something isn't working enhancement New feature or request area:operator labels Mar 22, 2026
@arska arska self-assigned this Mar 22, 2026
@tobru
Copy link
Copy Markdown
Contributor

tobru commented Mar 23, 2026

@Kidswiss mind giving this a review? I think it's OK, but I need another eye.

What I'm also unsure of is if the commit history is fine in this PR, because it mixes adding fuzzy testing and a fix found in one commit. It might be fine, but I want to have another opinion on that.

@arska arska force-pushed the add-fuzz-tests branch 2 times, most recently from 2687880 to 9e17f48 Compare March 23, 2026 13:41
arska and others added 2 commits March 23, 2026 14:57
Add Go native fuzz tests for:
- filterAndConvert: snapshot ID processing
- BackupOutputParser.out: restic JSON output parsing
- JsonArgsArray.UnmarshalJSON: custom JSON unmarshaling

The fuzzer immediately found a panic in filterAndConvert() where
snapshot.ID[:8] crashes on IDs shorter than 8 characters. Fixed
by checking length before slicing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Aarno Aukia <aarno.aukia@vshn.ch>
Run each fuzz target for 60 seconds on PRs and pushes to master.
Uses a matrix strategy so targets run in parallel. Short fuzz
runs in CI catch regressions; longer runs can be done locally
or via oss-fuzz.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Aarno Aukia <aarno.aukia@vshn.ch>
Copy link
Copy Markdown
Contributor

@Kidswiss Kidswiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're introducing a completely new testing paradigm here. I'm not convinced if the additional complexity outweighs the benefits.

As for the found issue: Restic's ID's are well defined SHA-256 hashes. So if Restic ever returns something shorter than 8 characters, something is really, really wrong. IMHO I'd rather have the runner pod crash and burn instead of silently discarding it. At least then it should pop up in the backup monitoring.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:operator bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants