zoekt/gitindex/ at 2fb345a6c2e58e54fe090028d0bd58bcd9288ea6 · boltless.me/zoekt

catfile.go

gitindex: disable git cat-file optimization by default (#1038) In our production sourcegraph.com cluster we are regularly seeing git grow to over 9GB in size leading to OOMs. Once we disabled this flag the OOMs went away. For now lets default this feature to off until it is resolved.

2 months ago

catfile_bench_test.go

gitindex: set filter for cat-file (#1026) At Sourcegraph we do a sparse clone which excludes files based on max file size. However, cat-file will hydrate in missing objects. So we pass in the same filter to avoid hydrating in those files.

3 months ago

catfile_hardening_test.go

gitindex: speed up tests (#1031)

2 months ago

catfile_test.go

gitindex: speed up tests (#1031)

2 months ago

clone.go

gitindex: diff config updates for existing clones (#1019) This supersedes #635 by porting its selective config-sync idea onto current main with a smaller, easier-to-read shape. Clone orchestration stays in gitindex/clone.go, while config argument generation and existing-clone sync logic now live in gitindex/clone_config.go. For existing clones, we now diff each zoekt.* setting before writing. Unchanged values are skipped, changed values are updated, and settings that disappear are removed. CloneRepo returns the repo destination only when a setting change actually happened so the caller can trigger reindexing only when needed. cmd/zoekt-mirror-github was also cleaned up so optional integer metadata keys are only added to the config map when present, which avoids pushing empty config values downstream. Note: On #635 review it mentioned using go-git. This commit initially explored that but it ended up being a _lot_ more code due to missing utilities around easily setting values based on a git config string.

3 months ago

clone_config.go

gitindex: diff config updates for existing clones (#1019) This supersedes #635 by porting its selective config-sync idea onto current main with a smaller, easier-to-read shape. Clone orchestration stays in gitindex/clone.go, while config argument generation and existing-clone sync logic now live in gitindex/clone_config.go. For existing clones, we now diff each zoekt.* setting before writing. Unchanged values are skipped, changed values are updated, and settings that disappear are removed. CloneRepo returns the repo destination only when a setting change actually happened so the caller can trigger reindexing only when needed. cmd/zoekt-mirror-github was also cleaned up so optional integer metadata keys are only added to the config map when present, which avoids pushing empty config values downstream. Note: On #635 review it mentioned using go-git. This commit initially explored that but it ended up being a _lot_ more code due to missing utilities around easily setting values based on a git config string.

3 months ago

clone_test.go

gitindex: speed up tests (#1031)

2 months ago

delete.go

gitindex: move package from internal (#981) Co-authored-by: ARyzhikov <ARyzhikov@nota.tech>

9 months ago

delete_test.go

gitindex: speed up tests (#1031)

2 months ago

filter.go

gitindex: move package from internal (#981) Co-authored-by: ARyzhikov <ARyzhikov@nota.tech>

9 months ago

ignore_test.go

gitindex: speed up tests (#1031)

2 months ago

index.go

gitindex: disable git cat-file optimization by default (#1038) In our production sourcegraph.com cluster we are regularly seeing git grow to over 9GB in size leading to OOMs. Once we disabled this flag the OOMs went away. For now lets default this feature to off until it is resolved.

2 months ago

index_test.go

gitindex: speed up tests (#1031)

2 months ago

main_test.go

gitindex: speed up tests (#1031)

2 months ago

repocache.go

gitindex: move package from internal (#981) Co-authored-by: ARyzhikov <ARyzhikov@nota.tech>

9 months ago

repocache_test.go

gitindex: speed up tests (#1031)

2 months ago

submodule.go

gitindex: move package from internal (#981) Co-authored-by: ARyzhikov <ARyzhikov@nota.tech>

9 months ago

submodule_test.go

gitindex: speed up tests (#1031)

2 months ago

tree.go

Add support for indexing submodules without repocache (#985) * Fix git.Open() fs argument for .git dir instead of worktree The filesystem argument submitted to git.Open() is of the .git directory, it should be for the worktree, as in go-git this logic is copied from: https://github.com/go-git/go-git/blob/145daf2492dd86b1d7e7787555ebd9837b93bff8/repository.go#L373-L375 One side effect of this bug was that go-git would refuse to fetch repo submodules * Fix missing RepoURLs and LineFragments for result subrepositories RepoURLs for subrepositories are currently filtered out when results are streamed per-repo from shards. * Add submodule indexing without repo cache When not using repo cache, index git submodules recursively into subrepos using the go-git API * Adjust subrepo naming conventions - Revert repo-cache behavior to unchanged - Add support for naming repos and modules with no remote (use the dirname for the former and subrepopath for the latter) - Add test for submodule no-repo-cache behavior * Fix overallocation in repoURL and lineFragment filtering * Revert basename treatment from 08f67597 Missed the fact that it is already done in zoekt-git-index command, the test was improperly configured https://github.com/sourcegraph/zoekt/blob/4e4a529c3b63c7d4c7897ba736f1cd52cc163134/cmd/zoekt-git-index/main.go#L93-L100

8 months ago

tree_test.go

gitindex: speed up tests (#1031)

2 months ago

Configure Feed

Configure Feed