[Parquet] Skip bloom filter predicate evaluation when no bloom filters are loaded - #24101
Open
rgehan wants to merge 1 commit into
Open
[Parquet] Skip bloom filter predicate evaluation when no bloom filters are loaded#24101rgehan wants to merge 1 commit into
rgehan wants to merge 1 commit into
Conversation
Evaluating the pruning predicate runs once per row group and is expensive for wide predicates, yet it can never prune a row group that has no bloom filters to consult. Short-circuit the per-row-group evaluation, and skip the pass entirely when no bloom filters were loaded for any row group (files written without them).
rgehan
commented
Aug 5, 2026
| assert_pruned(row_groups, ExpectedPruning::Some(vec![0])); | ||
| assert_eq!(metrics.row_groups_pruned_bloom_filter.pruned(), 1); | ||
| assert_eq!(metrics.row_groups_pruned_bloom_filter.matched(), 1); | ||
| } |
Contributor
Author
There was a problem hiding this comment.
Not a big fan of this test, as it would pass even without the fix, and doesn't really distinguish between "skipped predicate" and "evaluated predicate that matches".
An alternative would be to test with an expensive predicate, and assert on the timer, but that sounds fragile.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #24101 +/- ##
=======================================
Coverage 80.91% 80.91%
=======================================
Files 1102 1102
Lines 377102 377129 +27
Branches 377102 377129 +27
=======================================
+ Hits 305127 305156 +29
+ Misses 53784 53780 -4
- Partials 18191 18193 +2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
rgehan
marked this pull request as ready for review
August 5, 2026 12:30
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #24107
Rationale for this change
Bloom filter pruning codepath evaluates pruning predicates even when no bloom filters are defined for the row group.
For wide / expensive predicates, this can add visible latency.
This PR suggests shunting the evaluation of the predicate altogether when a row group provably doesn't contain bloom filters.
What changes are included in this PR?
Short-circuits the pruning predicate evaluation in bloom filter pruning codepath when there are no bloom filters.
Are these changes tested?
Yes, but the assertions are weak and cannot differentiate between a skipped pruning predicate (no bloom filter case) vs a pruning predicate that was evaluated and matches the row group.
I suppose we could build a test with an expensive predicate, and rely on the timer, but I find time-based tests to be generally weak/flaky. Let me know if you prefer this solution / if you see a better approach.
Are there any user-facing changes?
None