zoekt/query/ at 647e86c96a227411ae6823879f6bc43e607a6449 · boltless.me/zoekt · Tangled

boltless.me / zoekt

0

fork of https://github.com/sourcegraph/zoekt

0

zoekt / query /

at 647e86c96a227411ae6823879f6bc43e607a6449 12 files

Keegan Carruthers-Smith all: remove gob and SSE rpc endpoints (#758) 2y ago

all: remove gob and SSE rpc endpoints (#758) We now only consume zoekt via gRPC at Sourcegraph and I doubt anyone uses the old endpoints. This will have one required update in sourcegraph, and that is to use SenderFunc from the main zoekt package rather than from the now deleted stream package.

2 years ago

score: introduce query.Boost to scale score (#728) This commit introduces a new primitive Boost to our query language. It allows boosting (or dampening) the contribution to the score a query atoms will match contribute. To achieve this we introduce boostMatchTree which records this weight. We then adjust the visitMatches to take an initial score weight (1.0), and then each time we recurse through a boostMatchTree the score weight is multiplied by the boost weight. Additionally candidateMatch now has a new field, scoreWeight, which records the weight at time of candidate collection. Without boosting in the query this value will always be 1. Finally when scoring a candidateMatch we take the final score for it and multiply it by scoreWeight. Note: we do not expose a way to set this in the query language, only the query API. Test Plan: Manual testing against webserver via the new phrase-boost URL param. Additionally updated ranking tests to use the phrase booster.

2 years ago

query: introduce FileNameSet (#482) Sourcegraph's new hybrid searcher sends around large ORs of filenames to search for. FileNameSet provides an efficient implementation to transport and query these sets of filenames. We include an efficient gob implementation based on previous work around encoding a more complicated map of strings. Test Plan: Added unit tests. Additionally, updated sourcegraph to use this implementation and ran its tests.

3 years ago

marshal_test.go

query: introduce FileNameSet (#482) Sourcegraph's new hybrid searcher sends around large ORs of filenames to search for. FileNameSet provides an efficient implementation to transport and query these sets of filenames. We include an efficient gob implementation based on previous work around encoding a more complicated map of strings. Test Plan: Added unit tests. Additionally, updated sourcegraph to use this implementation and ran its tests.

3 years ago

all: gofumpt -l -w . gofumpt is a stricter gofmt. I took a look at the changes and in general they are nice. I don't think we need to enforce the use of gofumpt, but I like the idea of running it every once in a while. Test Plan: go test ./...

2 years ago

query: parse support for `fork:` and `public:` (#550) Add support for parsing queries for zoekt.fork and zoekt.public.

3 years ago

all: remove gob and SSE rpc endpoints (#758) We now only consume zoekt via gRPC at Sourcegraph and I doubt anyone uses the old endpoints. This will have one required update in sourcegraph, and that is to use SenderFunc from the main zoekt package rather than from the now deleted stream package.

2 years ago

all: remove gob and SSE rpc endpoints (#758) We now only consume zoekt via gRPC at Sourcegraph and I doubt anyone uses the old endpoints. This will have one required update in sourcegraph, and that is to use SenderFunc from the main zoekt package rather than from the now deleted stream package.

2 years ago

query_proto_test.go

all: gofumpt -l -w . gofumpt is a stricter gofmt. I took a look at the changes and in general they are nice. I don't think we need to enforce the use of gofumpt, but I like the idea of running it every once in a while. Test Plan: go test ./...

2 years ago

query: fix expandFileContent (#483) This fixes a bug where if s.FileName and s.Content were TRUE, we didn't expand and the resulting matchTree was implicitly matching filenames only. Now we match content and filenames if s.Filename == s.Content.

3 years ago

all: use a faster vendored regexp/syntax/Regexp.String (#753) We replace all calls to Regexp.String with a vendored version which is faster. go1.22 introduced a commit which "minimizes" the string returned by Regexp.String(). Part of what it does is run enumerate through literals runes in your string to see calculate flags related to unicode and case sensitivity. This can be quite slow, but is made worse by the fact we call it per shard per regexp in your query.Q to construct the matchtree. Currently Regexp.String() represents 40% of CPU time on sourcegraph.com. Before go1.22 it was ~0%. Note: This is a temporary change to resolve the issue. I have a deeper change to make this less clumsy. Note: In one place we remove the use of string by relying on Regexp.Equal instead. Test Plan: go test

2 years ago

all: use a faster vendored regexp/syntax/Regexp.String (#753) We replace all calls to Regexp.String with a vendored version which is faster. go1.22 introduced a commit which "minimizes" the string returned by Regexp.String(). Part of what it does is run enumerate through literals runes in your string to see calculate flags related to unicode and case sensitivity. This can be quite slow, but is made worse by the fact we call it per shard per regexp in your query.Q to construct the matchtree. Currently Regexp.String() represents 40% of CPU time on sourcegraph.com. Before go1.22 it was ~0%. Note: This is a temporary change to resolve the issue. I have a deeper change to make this less clumsy. Note: In one place we remove the use of string by relying on Regexp.Equal instead. Test Plan: go test

2 years ago