zoekt/build/ at a6e3dc263315f1248f002aa7dc9d34072db0d0ce · boltless.me/zoekt · Tangled

boltless.me / zoekt

0

fork of https://github.com/sourcegraph/zoekt

0

zoekt / build /

at a6e3dc263315f1248f002aa7dc9d34072db0d0ce 1 folder 7 files

Keegan Carruthers-Smith build: use enry to detect low priority files (#829) 2y ago

Add benchmark for ctags conversion (#679) This change adds a benchmark for the conversion from ctags output to Zoekt document data, plus a tiny optimization to presize the symbol slices.

2 years ago

build: use enry to detect low priority files (#829) This is a much more robust detection mechanism. Additionally we have these signals we can also add in: func IsConfiguration(path string) bool func IsDocumentation(path string) bool func IsDotFile(path string) bool func IsImage(path string) bool My main concern with this change is generated file detection on content using up RAM or CPU. Will monitor this impact on pprof in production. Test Plan: go test.

2 years ago

builder_test.go

build: use enry to detect low priority files (#829) This is a much more robust detection mechanism. Additionally we have these signals we can also add in: func IsConfiguration(path string) bool func IsDocumentation(path string) bool func IsDotFile(path string) bool func IsImage(path string) bool My main concern with this change is generated file detection on content using up RAM or CPU. Will monitor this impact on pprof in production. Test Plan: go test.

2 years ago

builder_unix.go

Allow wasm compilation (#786)

2 years ago

all: use stdlib slices package (#735) Noticed we weren't using this yet and that the API signatures had changed. Test Plan: go test

2 years ago

Indexing: clean up ctags parser wrapper (#708) This change cleans up the Go ctags parser wrapper as a follow-up to #702. Specific changes: * Remove synchronization in `lockedParser` and rename it to `CTagsParser` * Push delegation to universal vs. SCIP ctags into parser wrapper * Simplify document timeout logic * Rename some files

2 years ago

build: use enry to detect low priority files (#829) This is a much more robust detection mechanism. Additionally we have these signals we can also add in: func IsConfiguration(path string) bool func IsDocumentation(path string) bool func IsDotFile(path string) bool func IsImage(path string) bool My main concern with this change is generated file detection on content using up RAM or CPU. Will monitor this impact on pprof in production. Test Plan: go test.

2 years ago

scoring_test.go

ranking: add IDF to BM25 score calculation (#788) So far, we didn't include IDF in our BM25 score function. Zoekt uses a trigram index and hence doesn't compute document frequency during indexing. We could add this information to the index, but it is not immediately obvious how to tokenize code in a way that is compatible with tokens from a natural language query. Here we calulate the document frequency at query time under the assumption that we visit all documents containing any of the query terms. Notes: Also fixed an off-by-1 bug with how we count documents. Test plan: - Updated unit test - Context evaluation results are slightly worse with a decrease from 64/89 to 63/89

2 years ago