Skip to content

Conversation

@KKould
Copy link
Member

@KKould KKould commented Nov 24, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

root@localhost:8000/default/default> EXPLAIN ANALYZE SELECT * FROM article where article_id < 3;

EXPLAIN ANALYZE
SELECT
  *
FROM
  article
where
  article_id < 3

-[ EXPLAIN ]-----------------------------------
Filter
├── output columns: [article.article_id (#0), article.author_id (#1), article.viewer_id (#2), article.view_date (#3)]
├── filters: [is_true(article.article_id (#0) < 3)]
├── estimated rows: 3.00
└── TableScan
    ├── cpu time: 80.847814ms
    ├── wait time: 423.744107ms
    ├── output rows: 4
    ├── output bytes: 80.00 B
    ├── bytes scanned: 80.00 B
    ├── runtime filter inlist/min-max time: 2.413µs
    ├── table: default.default.article
    ├── scan id: 0
    ├── output columns: [article_id (#0), author_id (#1), viewer_id (#2), view_date (#3)]
    ├── read rows: 4
    ├── read size: < 1 KiB
    ├── partitions total: 6
    ├── partitions scanned: 4
    ├── pruning stats: [segments: <range pruning: 6 to 4 cost: 6 ms>, blocks: <range pruning: 4 to 4 cost: 1 ms>]
    ├── push downs: [filters: [is_true(article.article_id (#0) < 3)], limit: NONE]
    └── estimated rows: 6.00

21 rows explain in 14.880 sec. Processed 0 rows, 0 B (0 row/s, 0 B/s)

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - TODO: Cost is dynamic and difficult to test in logic tests.

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@KKould KKould requested review from b41sh and Copilot and removed request for b41sh November 24, 2025 09:18
@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Nov 24, 2025
@KKould KKould requested a review from BohuTANG November 24, 2025 09:18
Copilot finished reviewing on behalf of KKould November 24, 2025 09:20
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds cost tracking to the explain pruning statistics feature, controlled by a new session setting explain_pruner_cost. The implementation measures the time spent in different pruning phases (segment range, block range, bloom filter, inverted index, and vector index) and displays this information in EXPLAIN ANALYZE output.

  • Introduces a PruningCostController that uses RAII guards to measure pruning costs
  • Adds timing measurements for all pruning operations (segments and blocks)
  • Extends the PruningStatistics struct to include cost fields (in microseconds)
  • Adds formatting support to display costs in explain output

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/query/storages/fuse/src/pruning/pruning_statistics.rs Adds PruningCostController, PruningCostGuard, and cost tracking fields to FusePruningStatistics
src/query/storages/fuse/src/pruning/segment_pruner.rs Integrates cost measurement for segment range pruning
src/query/storages/fuse/src/pruning/block_pruner.rs Integrates cost measurement for block range, bloom, and inverted index pruning
src/query/storages/fuse/src/pruning/vector_index_pruner.rs Integrates cost measurement for vector index pruning
src/query/storages/fuse/src/pruning/fuse_pruner.rs Collects cost statistics and populates the PruningStatistics structure
src/query/storages/fuse/src/pruning/mod.rs Exports new cost controller types
src/query/settings/src/settings_default.rs Adds explain_pruner_cost setting (default: 0)
src/query/settings/src/settings_getter_setter.rs Adds getter for explain_pruner_cost setting
src/query/sql/src/planner/planner.rs Extends QueryKind::Explain to include ExplainAnalyze
src/query/service/src/physical_plans/format/common.rs Adds cost suffix formatting for pruning statistics display
src/query/catalog/src/plan/pruning_statistics.rs Extends PruningStatistics with cost fields and backward-compatible serialization

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@KKould KKould marked this pull request as ready for review November 25, 2025 07:20
@BohuTANG
Copy link
Member

@codex review

@chatgpt-codex-connector
Copy link

Codex Review: Didn't find any major issues. What shall we delve into next?

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@KKould KKould force-pushed the feat/cost_explain_pruning_stats branch from 86c741a to 2474668 Compare November 26, 2025 17:20
@KKould KKould force-pushed the feat/cost_explain_pruning_stats branch from 2474668 to d9a46e9 Compare November 27, 2025 01:43
@BohuTANG BohuTANG merged commit 72faddb into databendlabs:main Nov 27, 2025
87 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-feature this PR introduces a new feature to the codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: add cost to explain pruning stats

3 participants