Skip to content

Make engine health metrics granular on chunk and result - #5147

Open
mcastorina wants to merge 1 commit into
mainfrom
chore-metrics-granularity
Open

Make engine health metrics granular on chunk and result#5147
mcastorina wants to merge 1 commit into
mainfrom
chore-metrics-granularity

Conversation

@mcastorina

@mcastorina mcastorina commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Description:

Some metrics were emitting multiple times per chunk or result, so this adds logic to track it on a per chunk and result basis. This will help us describe the percentage of items getting filtered out in each stage.

Checklist:

  • Tests passing (make test-community)?
  • Lint passing (make lint this requires golangci-lint)?

Note

Low Risk
Metrics-only changes; scanning, detection, and verification behavior are unchanged.

Overview
Aligns chunks_dropped_total with chunks_entered_stage_total so drop reasons are counted once per work item at each stage, instead of firing multiple times inside loops.

At the scanner, no_matching_detectors is emitted only when no decoded variant gets any Aho–Corasick detector match (not once per empty decoded slice). In verification overlap and detect, detector_error is recorded at most once per chunk when any match/FromData call fails, rather than on every failing iteration.

The chunks_dropped_total metric help text now states it uses the same granularity as chunks_entered_stage_total for the same stage, so drop ratios per stage are meaningful.

Reviewed by Cursor Bugbot for commit 6927e58. Bugbot is set up for automated code reviews on this repo. Configure here.

Some metrics were emitting multiple times per chunk or result, so this
adds logic to track it on a per chunk and result basis. This will help
us describe the percentage of items getting filtered out in each stage.
@mcastorina
mcastorina requested review from a team July 21, 2026 20:21

@MuneebUllahKhan222 MuneebUllahKhan222 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants