$(case ... in pattern) ...) — ) in case pattern misinterpreted as command substitution terminator

[takakix2]

Description

When a case statement is embedded inside $(...) command substitution,
the ) in the case pattern (e.g., a)) is incorrectly treated as the
closing ) of the command substitution, causing a parse error.

Reproduction

echo $(case a in a) echo ok ;; esac)

Expected (bash/dash/zsh output):

ok

Actual (brush-shell / brush-parser 0.3.0):

Parse error: ParsingNearToken(Operator("^;;", ...))

Multiline variant (also fails)

echo $(case a in
       a) echo ok
          ;;
       esac
      )

POSIX reference

POSIX §2.6.3 (Command Substitution):

With the $(command) form, all characters following the open
parenthesis to the matching closing parenthesis constitute the command.

The ) in case pattern in pat) is a pattern delimiter — not a matching
parenthesis. The tokenizer must track case/esac context to correctly
identify the matching ) that closes the $(...).

Analysis

The issue is in tokenizer.rs, inside next_token_until().

When processing $(...), the tokenizer calls next_token_until(Some(')'))
recursively (L950-951). It tracks parenthesis depth via required_end_parens,
incrementing on ( and decrementing on ). However, it has no awareness
of case context, so the ) in a case pattern (e.g., a)) is incorrectly
matched as a SpecifiedTerminatingChar.

A simple case_depth counter in the tokenization loop is insufficient because
the collected text is re-tokenized and re-parsed downstream — the same
problem recurs at the re-parse stage.

Suggested approach

This likely requires coordination between the tokenizer and parser:

1. Option A (tokenizer-level): Track case/in/esac keyword context
   in the tokenizer and suppress ) matching while inside case...esac.
   The re-parse must also be aware of this context.

2. Option B (parser-level): Have the parser drive the tokenizer with
   context hints (e.g., "we are inside a case statement"), so the tokenizer
   knows to ignore ) as a terminator.

Environment

• brush-parser: 0.3.0
• Rust: stable
• OS: Linux (Ubuntu 24.04)
• Discovered via VSC-PCTS2016 test tp352 (POSIX shell conformance)

[reubeno]

Thanks for reporting this! It's a really good and subtle find.

As the analysis indicates, this is definitely a tough case for our current tokenizer -- and, dare I say, closer to a design-level issue than a missing conditional case. @lu-zero has been working hard on our new parser that better blends tokenizing and parsing -- and I'd expect it will be better set up to handle this. (I'll let them chime in with any specifics regarding this.)

Discovered via VSC-PCTS2016 test tp352 (POSIX shell conformance)

Are you trying to run conformance tests? If so, that's great! We've been interested in that but I haven't personally had the time to look at running them.

It would be great to get a sense of how brush does with those tests in general, if you're up for sharing in our Discussions area on GitHub or alternatively on our Discord. Moreover, we're open to running more external test suites as part of our CI process, but I know there may be licensing restrictions on the tests that prevent that.

[lu-zero]

I confirm that this is still not handled by the winnow parser