Assess the impact of AST-aware file reads, search, and mapping #22745
Description
This EPIC tracks a series of investigations as to whether there is value in AST aware file read, search, or codebase mapping.
AST aware tools can be used to 1) more precisely read method bounds with a single tool call, reducing turns from misaligned reads, and reducing noise in tokens, 2) navigate to usages and definitions of syntax elements via syntax aware search (AST grep), and 3) compress high level context about the codebase, such as the location of specific classes and their members by collapsing code down to just top level classes and types and their children's signatures.
1. tilth (Rust)
tilth is built specifically to generate a structural outline of a codebase. It uses Tree-sitter under the hood but ships as a single binary. It gives you an
immediate map of the repository's symbols.
* Installation: cargo install tilth (or download the binary from their releases)
* Usage:
* tilth map . (Prints a tree-like overview of the whole codebase structure)
* tilth read <file> (Prints the structural outline of a specific file)
2. glyph (Go)
glyph is another single-binary tool designed to generate clean, multi-file symbol maps. It's excellent if you want to quickly see the skeleton of a
directory of source code.
* Installation: go install github.com/benmyles/glyph@latest
* Usage: glyph cli './**/*.go' (Prints a tree of functions, structs, and methods)
3. ast-grep (sg) (Rust)
While primarily known as a structural search-and-replace tool, ast-grep is the industry standard for distributing Tree-sitter as a fast, single binary. You
can use it to extract landmarks via the command line.
* Installation: brew install ast-grep or cargo install ast-grep
* Usage: You can write quick one-liners to extract specific symbols (like all function declarations) and format the output.
4. The Classic: Universal Ctags (C)
If you truly want zero modern dependencies, u-ctags is the classic UNIX approach. It uses a hybrid regex/parser approach rather than full AST generation, so
it's incredibly fast and supports over 100 languages out of the box.
* Installation: brew install universal-ctags
* Usage: ctags -R --output-format=json .
* Note: It outputs a flat list of symbols (or JSON) rather than a visual tree, but it is the most robust and portable way to extract codebase landmarks.