zoekt/gitindex/ at 27effbadc957cb77a48098ce69455f4caeaaa7e2 · boltless.me/zoekt · Tangled

boltless.me / zoekt

0

fork of https://github.com/sourcegraph/zoekt

0

zoekt / gitindex /

at 27effbadc957cb77a48098ce69455f4caeaaa7e2 14 files

Julie Tibshirani Ranking: handle files with missing ranks (#555) 3y ago

Ranking: handle files with missing ranks (#555) Even when a repo has ranking data, certain files will not have ranks, like Markdown or yaml files. Currently these have rank 0, which puts them at a big disadvantage and means they're usually ranked last. This PR proposes to use the mean repo rank instead of 0. The rules: * If we have a concrete rank for the file, always use it * If there's no rank, and it's a low priority file like a test, then use rank 0 * Otherwise use the mean rank for the repository We don't attempt to handle the case where an entire repo is missing ranks because it doesn't have precise code intel.

3 years ago

tests: replace ioutil.TempDir with testing.T.TempDir (#359) testing.T.TempDir was introduced in Go 1.15 (August 2020) and the majority of tests had been written earlier.

4 years ago

zoekt-mirror-gerrit: delete stale repos (#282) This change adds an option on gerrit mirror to delete repos that don't match mirror criteria anymore. It also changes the DeleteRepos to handle repository names with multiple "/".

4 years ago

tests: replace ioutil.TempDir with testing.T.TempDir (#359) testing.T.TempDir was introduced in Go 1.15 (August 2020) and the majority of tests had been written earlier.

4 years ago

all: use grafana/regexp fork for faster regexp (#270) grafana/regexp's speedup branch contains the stdlib regexp with multiple patches applied to speed up regexp. The benchmark improvements are quite impressive across the board. The series of patches are being considered for inclusion in golang. I took a look over them, and they look legit enough for us to start using it.

4 years ago

all: rename module to github.com/sourcegraph/zoekt We are a hard enough for now that we might as well have our own module name. This will also make it possible to more directly use go tooling.

3 years ago

Ranking: handle files with missing ranks (#555) Even when a repo has ranking data, certain files will not have ranks, like Markdown or yaml files. Currently these have rank 0, which puts them at a big disadvantage and means they're usually ranked last. This PR proposes to use the mean repo rank instead of 0. The rules: * If we have a concrete rank for the file, always use it * If there's no rank, and it's a low priority file like a test, then use rank 0 * Otherwise use the mean rank for the repository We don't attempt to handle the case where an entire repo is missing ranks because it doesn't have precise code intel.

3 years ago

Ranking: handle files with missing ranks (#555) Even when a repo has ranking data, certain files will not have ranks, like Markdown or yaml files. Currently these have rank 0, which puts them at a big disadvantage and means they're usually ranked last. This PR proposes to use the mean repo rank instead of 0. The rules: * If we have a concrete rank for the file, always use it * If there's no rank, and it's a low priority file like a test, then use rank 0 * Otherwise use the mean rank for the repository We don't attempt to handle the case where an entire repo is missing ranks because it doesn't have precise code intel.

3 years ago

all: migrate go-git to new package go-git now lives at github.com/go-git/go-git. v5 exists to allow the migration to the new location and is API compatible with v4 [1]. [1]: https://github.com/go-git/go-git/releases/tag/v5.0.0 Change-Id: If68ba927e38ae8ef81447f995e449994928f0cfa

6 years ago

repocache_test.go

tests: replace ioutil.TempDir with testing.T.TempDir (#359) testing.T.TempDir was introduced in Go 1.15 (August 2020) and the majority of tests had been written earlier.

4 years ago

gitindex: correctly skip UTF-8 BOM (#230) Previously we called buf.ReadRune to detect if we had a BOM. However, buf.ReadRune on the BOM just consumes the first byte and returns "\uFFFD". So this code accidently worked. In the case of the BOM the UnreadRune call actually returned an error which we didn't check. This updates the code to correctly detect a BOM. Additionally it doesn't have to rely on reading then unreading. Instead we can peak at what is remaining since this is a bytes.Buffer. There is a risk that there are submodule files that don't have the exact BOM we are detecting and somehow we are skipping over and working on them due to the code before. So when this code rolls out we should monitor production.

4 years ago

submodule_test.go

zoekt-git-index: fix ParseGitModules BOM scanner to work with empty files

5 years ago

gitindex: index symbolic links (#237) Up to now we skipped symbolic links when compiling the list of files during indexing. With this change we index symbolic links with the relative path of their target file as content.

4 years ago

ranking: add document ranks to shards (#449) We persist document ranks in the shards and sort file matches based on the rankings determined by the document ranks and match scores. Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com>

3 years ago