zoekt/build/ at 44eab0c512ef7ff25b09b2c133d4cba3d9850519 · boltless.me/zoekt · Tangled

boltless.me / zoekt

0

fork of https://github.com/sourcegraph/zoekt

0

zoekt / build /

at 44eab0c512ef7ff25b09b2c133d4cba3d9850519 1 folder 7 files

Stefan Hengl index: enable shard merging by default (#798) 2y ago

Add benchmark for ctags conversion (#679) This change adds a benchmark for the conversion from ctags output to Zoekt document data, plus a tiny optimization to presize the symbol slices.

2 years ago

index: enable shard merging by default (#798) This enables shard merging by default for zoekt-sourcegraph-indexserver. Sourcegraph has been using shard merging in production for several years. We have recently confirmed significant performance improvements for queries which are bound by matchTree construction. I also remove -merge_max_priority because we have stopped using it. Use SRC_DISABLE_SHARD_MERGING to disable shard merging. Test plan: mostly CI, I did some manual testing to confirm that shard merging is enabled by default for zoekt-sourcegraph-indexserver.

2 years ago

builder_test.go

all: gofumpt -l -w . gofumpt is a stricter gofmt. I took a look at the changes and in general they are nice. I don't think we need to enforce the use of gofumpt, but I like the idea of running it every once in a while. Test Plan: go test ./...

2 years ago

builder_unix.go

Allow wasm compilation (#786)

2 years ago

all: use stdlib slices package (#735) Noticed we weren't using this yet and that the API signatures had changed. Test Plan: go test

2 years ago

Indexing: clean up ctags parser wrapper (#708) This change cleans up the Go ctags parser wrapper as a follow-up to #702. Specific changes: * Remove synchronization in `lockedParser` and rename it to `CTagsParser` * Push delegation to universal vs. SCIP ctags into parser wrapper * Simplify document timeout logic * Rename some files

2 years ago

all: gofumpt -l -w . gofumpt is a stricter gofmt. I took a look at the changes and in general they are nice. I don't think we need to enforce the use of gofumpt, but I like the idea of running it every once in a while. Test Plan: go test ./...

2 years ago

scoring_test.go

ranking: add IDF to BM25 score calculation (#788) So far, we didn't include IDF in our BM25 score function. Zoekt uses a trigram index and hence doesn't compute document frequency during indexing. We could add this information to the index, but it is not immediately obvious how to tokenize code in a way that is compatible with tokens from a natural language query. Here we calulate the document frequency at query time under the assumption that we visit all documents containing any of the query terms. Notes: Also fixed an off-by-1 bug with how we count documents. Test plan: - Updated unit test - Context evaluation results are slightly worse with a decrease from 64/89 to 63/89

2 years ago