zoekt/cmd/ at 6e162c732f8a291ef990b809bcc71ebb1fde7df0 · boltless.me/zoekt · Tangled

boltless.me / zoekt

0

fork of https://github.com/sourcegraph/zoekt

0

zoekt / cmd /

at 6e162c732f8a291ef990b809bcc71ebb1fde7df0 18 folders 1 file

Keegan Carruthers-Smith all: consistently use cpu_profile as flag name (#895) 1y ago

cmd/zoekt: trim '\n' from match (#867) Before, we'd typically have two newlines, one coming from the corpus, and one terminating the fmt.Printf call.

2 years ago

zoekt-archive-index

zoekt-archive-index: split out ranking tests and archive indexing (#712) We had ranking e2e tests living in the zoekt-archive-index cmd for convenience since that contained useful functions for indexing a remote tarball from the GitHub API. This commit splits the archive functionality into a new internal/archive package and the ranking tests into a new internal/e2e package. The zoekt-archive-index code is now quite minimal. This is similiar to how zoekt-git-index mostly just calls out to the gitindex package. What is different is that archive package is marked internal, unlike gitindex. gitindex should also be internal, but the code predates go's support for internal. I suspect more of our e2e tests will end up in this package. Test Plan: go test ./...

2 years ago

zoekt-dynamic-indexserver

all: gofumpt -l -w . gofumpt is a stricter gofmt. I took a look at the changes and in general they are nice. I don't think we need to enforce the use of gofumpt, but I like the idea of running it every once in a while. Test Plan: go test ./...

2 years ago

zoekt-git-clone

remove bazel (#634)

2 years ago

zoekt-git-index

all: consistently use cpu_profile as flag name (#895) This is more a workaround since a transitive dependency has introduced a global flag "cpuprofile", leading to a panic due to registring the flag twice. To make ourselves immune to this issue we can refactor our usages to use a FlagSet, even for "main". This is a bigger and frankly inconvenient change for a somewhat rare occurance. Instead we just rename our flag. I feel comfortable renaming since this flag should only really be used by Zoekt developers. There will be the issue that the flag will be shown twice for commands, but I will report to the upstream repo about this problem. Test Plan: go get -u ./... && go run ./cmd/zoekt-git-index works

1 year ago

zoekt-index: USAGE message if no arguments supplied (#827) Test Plan: go run ./cmd/zoekt-index prints out useful instructions

2 years ago

zoekt-indexserver

feat(cmd/zoekt-mirror-gitlab): add no_archived Option (#881) Add support to the GitLab mirror for excluding archived repos.

1 year ago

zoekt-merge-index

indexserver: log name of freshly minted compound shard (#849) This updates zoekt-merge-index to print the name of a new compound shard to stdout. Indexserver picks it up and logs it. This has the nice property that indexserver now has all the info. If we want to log this to a file in the future, we don't have to worry as much about competing writes to the file. Together with a new log line in vacuum we can now follow the full lifecycle of a compound shard in the logs. Test plan: updated unit tests

2 years ago

zoekt-mirror-bitbucket-server

remove bazel (#634)

2 years ago

zoekt-mirror-gerrit

gomod: go get -u ./... (#820) Test Plan: go test ./...

2 years ago

zoekt-mirror-gitea

add zoekt-mirror-gitea (#844) * add zoekt-mirror-gitea * * Clean up setting the default * update note about topic filtering not being implemented as topics are missing from the API * cleanup some pointers * cleanup some code syntax

2 years ago

zoekt-mirror-github

remove bazel (#634)

2 years ago

zoekt-mirror-gitiles

remove bazel (#634)

2 years ago

zoekt-mirror-gitlab

feat(cmd/zoekt-mirror-gitlab): add no_archived Option (#881) Add support to the GitLab mirror for excluding archived repos.

1 year ago

zoekt-repo-index

Use single map for collecting files across branches (#839) When looking at large profiles for `inuse_space` on dot com, I noticed the filename maps in `prepareNormalBuild` taking a bunch of memory. This PR avoids allocating a separate map per branch, instead having `RepoWalker` collect all the entries in a single instance variable.

2 years ago

zoekt-sourcegraph-indexserver

Add metric for total number of shards (#891) Currently, we only record the number of compound shards. This PR adds a metric for the total number of index shards. This metric is helpful for alerting, as it's important to catch a significant decrease in the number of indexed data.

1 year ago

all: consistently use cpu_profile as flag name (#895) This is more a workaround since a transitive dependency has introduced a global flag "cpuprofile", leading to a panic due to registring the flag twice. To make ourselves immune to this issue we can refactor our usages to use a FlagSet, even for "main". This is a bigger and frankly inconvenient change for a somewhat rare occurance. Instead we just rename our flag. I feel comfortable renaming since this flag should only really be used by Zoekt developers. There will be the issue that the flag will be shown twice for commands, but I will report to the upstream repo about this problem. Test Plan: go get -u ./... && go run ./cmd/zoekt-git-index works

1 year ago

zoekt-webserver

sourcegraph: multi-tenant Zoekt (#859) This updates webserver and sourcegraph-indexserver to support multi-tenancy. The change is behind an ENV feature-flag. Key changes: - tenant ID is now part of the index (repo metadata) - GRPC: IndexOption and Repository have a new field TenantId - If multi-tenancy is enabled, webserver checks if tenant in context matches the tenant id in the shard - zoekt-git-index has a new parameter "-shard_prefix ". If set, the value will be used instead of repository name as prefix for the name of the shard. For Sourcegraph we use "<tenant id>_<repository id>" as prefix if multi-tenancy is enabled Assumption: All calls to Sourcegraph are privileged Test plan: - New tests - Ran this together with Sourcegraph (with and without MT enabled)

2 years ago

all: consistently use cpu_profile as flag name (#895) This is more a workaround since a transitive dependency has introduced a global flag "cpuprofile", leading to a panic due to registring the flag twice. To make ourselves immune to this issue we can refactor our usages to use a FlagSet, even for "main". This is a bigger and frankly inconvenient change for a somewhat rare occurance. Instead we just rename our flag. I feel comfortable renaming since this flag should only really be used by Zoekt developers. There will be the issue that the flag will be shown twice for commands, but I will report to the upstream repo about this problem. Test Plan: go get -u ./... && go run ./cmd/zoekt-git-index works

1 year ago