[eval] memstore: speed up LoadNextMsg when using wildcard filter

[Uzay-G]

Mirror of nats-io#7840 (MERGED) for Orpheus review evaluation.

Upstream: nats-io#7840

---

Original PR description:

This commit speeds up wildcard based filtering:

• Avoid expanding the bounds for matching fss entries that are past our search
• Avoid unnecessary creation of a list with matching subjects
• Introduce MatchUntil, this allows early stop if we find a match with first <= start

Signed-off-by: Daniele Sciascia daniele@nats.io

[orpheus-by-fulcrum-dev-uzay-g]

Review started.

[orpheus-by-fulcrum-dev-uzay-g]

All checks passed.

Expected behavior

• LoadNextMsg with wildcard filters returns the correct next message at or after the requested start sequence, with no missing or out-of-order messages
• LoadNextMsg with literal (non-wildcard) filters returns the correct message via a single subject tree lookup
• The new MatchUntil method on SubjectTree allows early termination of wildcard matching, and Match continues to enumerate all matches
• The shouldLinearScan heuristic is preserved: linear scan for >, for small message ranges relative to subject count, and for wildcard filters with >256 unique subjects
• Message deletion and purge do not break wildcard/literal filter correctness (via recalculateForSubj)
• No regressions in existing memstore or stree test suites

What happens

• ✅ Happy path works: wildcard (events.*), full wildcard (events.>), and literal (data.signal) consumers all receive the correct messages in order across 5 subjects and 50 messages
• ✅ Sparse match works: 5 data.signal messages scattered among 300 noise messages on 300+ distinct subjects are all found by a literal consumer
• ✅ Early-stop correctness verified: consumer starting from seq 7 on a stream with 3 subjects (alpha/beta/gamma) correctly delivers all 19 remaining messages, confirming nextWildcardMatchLocked bounds are correct even when early-stop triggers
• ✅ Edge cases pass: message deletion (seqs 1-5 deleted, wildcard consumer still finds remaining 15), subject purge (20 purged, 40 remaining found), multi-depth wildcards (app.* vs app.*.* vs app.>), and no-match filter all behave correctly
• ✅ All 37 memstore tests, 56 stree tests, and cross-store LoadNextMsg tests pass with no regressions

Detailed evidence

Setup

export PATH="/usr/local/go/bin:$HOME/go/bin:$PATH"
go build -o /tmp/nats-server .
go install github.com/nats-io/natscli/nats@latest

cat > /tmp/nats-demo.conf <<'EOF'
listen: 0.0.0.0:4222
jetstream {
  store_dir: /tmp/nats-js-data
  max_mem: 256M
  max_file: 256M
}
EOF

rm -rf /tmp/nats-js-data
/tmp/nats-server -c /tmp/nats-demo.conf &
nats account info  # Confirms JetStream enabled, v2.14.0-dev

Demo 1: Happy path — wildcard and literal consumers

nats stream add DEMO1 --subjects "events.>" --storage memory --defaults
for i in $(seq 1 10); do
  nats pub events.orders "order-$i"
  nats pub events.payments "payment-$i"
  nats pub events.shipping "shipping-$i"
  nats pub events.returns "return-$i"
  nats pub events.notifications "notif-$i"
done
# Stream: 50 messages, 5 subjects

# Literal consumer: events.orders
nats consumer add DEMO1 WC_ORDERS --filter "events.orders" --pull --deliver all --defaults
nats consumer next DEMO1 WC_ORDERS --count 10 --no-ack
# Result: 10 messages, str seq 1,6,11,16,21,26,31,36,41,46 — all events.orders, correct order

# Wildcard consumer: events.*
nats consumer add DEMO1 WC_ALL --filter "events.*" --pull --deliver all --defaults
nats consumer next DEMO1 WC_ALL --count 50 --no-ack
# Result: 50 messages, str seq 1-50, all subjects interleaved correctly

# FWC consumer: events.>
nats consumer add DEMO1 FWC_ALL --filter "events.>" --pull --deliver all --defaults
nats consumer next DEMO1 FWC_ALL --count 50 --no-ack
# Result: 50 messages, identical order to events.*

Demo 2: Sparse literal match (301 subjects, only 5 matching)

nats stream add SPARSE --subjects "data.>" --storage memory --defaults
for i in $(seq 1 200); do nats pub "data.noise.$i" "noise-$i"; done
nats pub "data.signal" "signal-1"  # seq 201
nats pub "data.signal" "signal-2"  # seq 202
nats pub "data.signal" "signal-3"  # seq 203
for i in $(seq 201 300); do nats pub "data.noise.$i" "noise-$i"; done
nats pub "data.signal" "signal-4"  # seq 304
nats pub "data.signal" "signal-5"  # seq 305
# Stream: 305 messages, 301 subjects

nats consumer add SPARSE LITERAL_SIGNAL --filter "data.signal" --pull --deliver all --defaults
nats consumer next SPARSE LITERAL_SIGNAL --count 5 --no-ack
# Result: 5 messages at str seq 201,202,203,304,305 — all signal-1 through signal-5
nats consumer next SPARSE LITERAL_SIGNAL --count 1 --no-ack --timeout 2s
# Result: timeout (correct, no more messages)

Demo 3: Wildcard start-before-first-match (mirrors TestStoreLoadNextMsgWildcardStartBeforeFirstMatch)

nats stream add STARTAFTER --subjects "bar.*,foo.*" --storage memory --defaults
for i in $(seq 0 99); do nats pub "bar.$i" "bar-$i"; done  # seqs 1-100
nats pub "foo.1" "foo-value-1"  # seq 101
# Stream: 101 messages, 101 subjects

nats consumer add STARTAFTER FOO_WC --filter "foo.*" --pull --deliver all --defaults
nats consumer next STARTAFTER FOO_WC --count 1 --no-ack
# Result: 1 message at str seq 101, subject foo.1 — correct
nats consumer next STARTAFTER FOO_WC --count 1 --no-ack --timeout 2s
# Result: 408 Request Timeout (correct, no more messages)

Demo 7: Early-stop correctness with interleaved subjects

nats stream add EARLYSTOP --subjects "evt.>" --storage memory --defaults
for i in $(seq 1 5); do nats pub "evt.alpha" "alpha-$i"; done    # seqs 1-5
for i in $(seq 1 5); do nats pub "evt.beta" "beta-$i"; done      # seqs 6-10
for i in $(seq 1 5); do nats pub "evt.gamma" "gamma-$i"; done    # seqs 11-15
for i in $(seq 6 10); do nats pub "evt.alpha" "alpha-$i"; done   # seqs 16-20
for i in $(seq 6 10); do nats pub "evt.beta" "beta-$i"; done     # seqs 21-25
# Stream: 25 messages, 3 subjects

# Consumer from seq 7 exercises early-stop:
# evt.alpha has First=1 < start=7, triggering early stop in nextWildcardMatchLocked
nats consumer add EARLYSTOP FROM_7 --filter "evt.*" --pull --deliver "7" --defaults
nats consumer next EARLYSTOP FROM_7 --count 19 --no-ack
# Result: 19 messages starting at str seq 7:
#   beta-2..beta-5 (7-10), gamma-1..gamma-5 (11-15),
#   alpha-6..alpha-10 (16-20), beta-6..beta-10 (21-25)
# All messages present, none skipped despite early-stop optimization

Demo 8: Message deletion + wildcard filter

nats stream add DELTEST --subjects "del.>" --storage memory --defaults
for i in $(seq 1 10); do nats pub "del.a" "a-$i"; done    # seqs 1-10
for i in $(seq 1 10); do nats pub "del.b" "b-$i"; done    # seqs 11-20
for i in $(seq 1 5); do nats stream rmm DELTEST $i -f; done  # delete seqs 1-5

nats consumer add DELTEST DEL_WC --filter "del.*" --pull --deliver all --defaults
nats consumer next DELTEST DEL_WC --count 15 --no-ack
# Result: 15 messages: a-6..a-10 (6-10), b-1..b-10 (11-20) — correct after deletion

Demo 9: Purge by subject + wildcard filter

nats stream add PURGETEST --subjects "p.>" --storage memory --defaults
for i in $(seq 1 20); do nats pub "p.keep" "keep-$i"; done      # seqs 1-20
for i in $(seq 1 20); do nats pub "p.remove" "remove-$i"; done  # seqs 21-40
for i in $(seq 1 20); do nats pub "p.keep" "keep2-$i"; done     # seqs 41-60
nats stream purge PURGETEST --subject "p.remove" -f
# Purged 20 messages

nats consumer add PURGETEST AFTER_PURGE --filter "p.*" --pull --deliver all --defaults
nats consumer next PURGETEST AFTER_PURGE --count 40 --no-ack
# Result: 40 messages: keep-1..keep-20 (1-20), keep2-1..keep2-20 (41-60)
# Correctly skips purged range 21-40

Demo 5: Multi-level wildcard depth filtering

nats stream add MULTILEVEL --subjects "app.>" --storage memory --defaults
nats pub "app.a" "depth-1"          # seq 1
nats pub "app.a.b" "depth-2"        # seq 2
nats pub "app.a.b.c" "depth-3"      # seq 3
nats pub "app.x" "depth-1x"         # seq 4
nats pub "app.x.y" "depth-2x"       # seq 5
nats pub "app.x.y.z" "depth-3x"     # seq 6

# app.* — single level
nats consumer next MULTILEVEL SINGLE_WC --count 2 --no-ack
# Result: app.a (seq 1), app.x (seq 4) — correct, no deeper subjects

# app.> — all levels
nats consumer next MULTILEVEL FWC_ALL2 --count 6 --no-ack
# Result: all 6 messages in order

# app.*.* — exactly two levels
nats consumer next MULTILEVEL TWO_WC --count 2 --no-ack
# Result: app.a.b (seq 2), app.x.y (seq 5) — correct

Unit tests

go test ./server/ -run "TestMemStore" -v -count=1
# PASS: all 37 tests pass (7.239s)

go test ./server/stree/ -v -count=1
# PASS: all 56 tests pass (0.109s), including TestSubjectTreeMatchUntil

go test ./server/ -run "TestStore.*LoadNextMsg|TestStoreMsgLoadNextMsg" -v -count=1
# PASS: TestStoreMsgLoadNextMsgMulti (Memory + File), TestStoreLoadNextMsgWildcardStartBeforeFirstMatch (Memory + File)