zoekt/shards/ at aea6871d0b9c75c5717f6679d53ea4999a0cc4be · boltless.me/zoekt · Tangled

boltless.me / zoekt

0

fork of https://github.com/sourcegraph/zoekt

0

zoekt / shards /

at aea6871d0b9c75c5717f6679d53ea4999a0cc4be 9 files

Keegan Carruthers-Smith shards: only trigger rescan on .zoekt files changing (#801) 2y ago

shards: only trigger rescan on .zoekt files changing (#801) Any write to the index dir triggered a scan. This means on busy instances we are constantly rescanning, leading to an over-representation in CPU profiles around watch. The events are normally writes to our temporary files. By only considering events for .zoekt files (which is what scan reads) we can avoid the constant scan calls. Just in case we also introduce a re-scan every minute in case we miss an event. There is error handling around this, but I thought it is just more reliable to call scan every once in a while. Note: this doesn't represent significant CPU use, but it does muddy the CPU profiler output. So this makes it easier to understand trends in our continuous cpu profiling. Test Plan: CI

2 years ago

all: remove gob and SSE rpc endpoints (#758) We now only consume zoekt via gRPC at Sourcegraph and I doubt anyone uses the old endpoints. This will have one required update in sourcegraph, and that is to use SenderFunc from the main zoekt package rather than from the now deleted stream package.

2 years ago

all: rename module to github.com/sourcegraph/zoekt We are a hard enough for now that we might as well have our own module name. This will also make it possible to more directly use go tooling.

3 years ago

scheduler: add some clarifying comments (#508)

3 years ago

all: fix errcheck lints in test files

4 years ago

Revert "shards: respect scheduler and use smarter synchronization for List (#750)" (#752) This reverts commit 931974dd953e07ce8fbe28e3b615323c2d156e40. After monitoring production we are seeing increased latencies in List calls. Additionally we are seeing many more scheduler transitions into interactive queued. The only realistic cause of this was this commit, so we are reverting for now until further investigation. Test Plan: go test

2 years ago

Rename UseKeywordScoring to mention BM25 (#778) It's confusing to call this `UseKeywordScoring`, since we do not use it for the `keyword` patterntype in Sourcegraph. This commit clarifies the name to mention BM25.

2 years ago

shards: only trigger rescan on .zoekt files changing (#801) Any write to the index dir triggered a scan. This means on busy instances we are constantly rescanning, leading to an over-representation in CPU profiles around watch. The events are normally writes to our temporary files. By only considering events for .zoekt files (which is what scan reads) we can avoid the constant scan calls. Just in case we also introduce a re-scan every minute in case we miss an event. There is error handling around this, but I thought it is just more reliable to call scan every once in a while. Note: this doesn't represent significant CPU use, but it does muddy the CPU profiler output. So this makes it easier to understand trends in our continuous cpu profiling. Test Plan: CI

2 years ago

watcher_test.go

all: gofumpt -l -w . gofumpt is a stricter gofmt. I took a look at the changes and in general they are nice. I don't think we need to enforce the use of gofumpt, but I like the idea of running it every once in a while. Test Plan: go test ./...

2 years ago