index: add hybrid go-re2 engine for large file content matching (#1024)

Adds an optional hybrid regex engine (internal/hybridre2) that transparently
switches between grafana/regexp and wasilibs/go-re2 (RE2 via WebAssembly)
based on file content size. Disabled by default — no behaviour change
without opt-in via ZOEKT_RE2_THRESHOLD_BYTES.

## Motivation

Issue #323 identified regex as the dominant CPU consumer in zoekt's
webserver profile. Go's regexp engine (including the grafana/regexp fork
already in use) lacks a lazy DFA. RE2's lazy DFA provides linear-time
matching with much better constant factors for alternations, character
classes, and complex patterns on large inputs.

The tradeoff: go-re2 uses WebAssembly (~600ns per-call overhead), making
it slower than grafana/regexp for small inputs (<4KB) but dramatically
faster above the threshold. A full engine swap would regress small-file
searches, so a threshold-based hybrid is the pragmatic approach.

## Implementation

### New package: internal/hybridre2

hybridre2.Regexp compiles both engines once at query-parse time and
dispatches FindAllIndex based on len(input) >= Threshold():

func (re *Regexp) FindAllIndex(b []byte, n int) [][]int {
if useRE2(len(b)) {
return re.re2.FindAllIndex(b, n)
}
return re.grafana.FindAllIndex(b, n)
}

### Change to index/matchtree.go

regexpMatchTree gains a hybridRegexp field used for file content matching;
filename matching keeps using grafana/regexp directly (filenames are always
short, so WASM overhead dominates there).

### Configuration

ZOEKT_RE2_THRESHOLD_BYTES env var, read once at startup:

-1 (default): disabled — always grafana/regexp, zero behaviour change
0: always use go-re2 (useful for evaluation/testing)
32768: use go-re2 for files >= 32KB (recommended starting point)

## Benchmarks

Hardware: AMD EPYC 9B14, go-re2 v1.10.0 (WASM, no CGO).

Alternations — `func|var|const|type|import`:
32KB: grafana 2505µs go-re2 467µs 5.4x speedup
128KB: grafana 9900µs go-re2 1699µs 5.8x speedup
512KB: grafana 40.7ms go-re2 6.8ms 6.0x speedup

Complex — `(func|var)\s+[A-Z]\w*\s*(`:
32KB: grafana 1237µs go-re2 230µs 5.4x speedup
128KB: grafana 4935µs go-re2 911µs 5.4x speedup
512KB: grafana 19.9ms go-re2 3.8ms 5.3x speedup

Literal — `main` (grafana wins; threshold protects this case):
32KB: grafana 33.2µs go-re2 59.8µs

## Testing

go test ./internal/hybridre2/ # unit + correctness matrix
go test ./index/ -short # full existing suite: passes
go test ./... -short # full suite: passes

Correctness verified by asserting identical match offsets between grafana
and go-re2 for 9 patterns x 5 sizes (64B-256KB).

## Notes

- Binary/non-UTF-8 content: go-re2 stops at invalid UTF-8 (vs. grafana
which replaces with the replacement character). The default threshold of
-1 ensures zero behaviour change. Operators enabling the threshold should
be aware; future work could detect non-UTF-8 and force the grafana path.
- Dependency: github.com/wasilibs/go-re2 v1.10.0 — pure Go WASM, no system
deps. Binary size increase: ~2MB (the embedded RE2 WASM module).
- Rollout plan: enable in GitLab via feature flag starting at 32KB, compare
p95 regex latency before/after using per-shard timing in search responses.