alpha
Login
or
Join now
boltless.me
/
zoekt
Star
0
Fork
0
Atom
Configure Feed
Issues
Pull Requests
Commits
Tags
Feed URL
Select the types of activity you want to include in your feed.
fork of https://github.com/sourcegraph/zoekt
Star
0
Fork
0
Atom
Configure Feed
Issues
Pull Requests
Commits
Tags
Feed URL
Select the types of activity you want to include in your feed.
Overview
Issues
Pulls
Pipelines
zoekt
/
cmd
/
zoekt-sourcegraph-indexserver
/
at
70647bad55921645285ce3776d8c29928eb205bc
2 folders
21 files
Julie Tibshirani
Fully revert windows support (#909)
1y ago
70647bad
json_schemas
zoekt-sourcegraph-indexserver: add gRPC support to sourcegraph client (#551) Co-authored-by: Camden Cheek <camden@ccheek.com>
3 years ago
protos
sourcegraph: multi-tenant Zoekt (#859) This updates webserver and sourcegraph-indexserver to support multi-tenancy. The change is behind an ENV feature-flag. Key changes: - tenant ID is now part of the index (repo metadata) - GRPC: IndexOption and Repository have a new field TenantId - If multi-tenancy is enabled, webserver checks if tenant in context matches the tenant id in the shard - zoekt-git-index has a new parameter "-shard_prefix ". If set, the value will be used instead of repository name as prefix for the name of the shard. For Sourcegraph we use "<tenant id>_<repository id>" as prefix if multi-tenancy is enabled Assumption: All calls to Sourcegraph are privileged Test plan: - New tests - Ran this together with Sourcegraph (with and without MT enabled)
2 years ago
backoff.go
Fully revert windows support (#909) We added support for Windows in https://github.com/sourcegraph/zoekt/pull/535. We then partially reverted the change, specifically the parts related to mmap (#706). This was okay because we no longer build for Windows. This PR fully removes support to avoid being in a partially-implemented state.
1 year ago
backoff_test.go
Add metric for indexing delay (#804) Our dashboards currently report indexing delay using the `index_queue_age_seconds` metric, which tracks the duration indexing jobs wait in the queue before being "popped" for indexing. For dot com, we regularly see the median for this metric at 2 days. However, this metric may be misleading because it includes "noop" indexing jobs where the index is already up to date. These jobs can stay in the queue for a long time since they are not high priority. This PR adds a new metric `index_indexing_delay_seconds`, which reports the duration from when an indexing job enters the queue, to when it completes. It uses 'state' as a label, so by setting `state='success'` we can (roughly) see the delay from when Zoekt learns about a new version, to when that version is available to search.
2 years ago
cleanup.go
Move root-level index code to index package (#902) In the repo root, we have a bunch of low level logic around index building and searching. So we end up exposing internal logic through the main public `zoekt` package, for example `zoekt.Merge(...)`. This PR moves it into the `build` package, so all code related to index building lives together. It then renames `build` to `index` to reflect the broader focus on indexing and searching the index.
1 year ago
cleanup_test.go
Rename IndexBuilder -> ShardBuilder (#908) When navigating the code, I've often forgotten the difference between `NewBuilder` and `NewIndexBuilder`. This rename clarifies that one of these indexes a whole repo, while the other builds individual shards. Also `index.NewShardBuilder` sounds better.
1 year ago
debug.go
all: set severity hint in most log output (#871) I was inspecting logs in GKE and it incorrectly categorized the severity of nearly all logs from zoekt-webserver and zoekt-indexserver. This is a hack to make it work better without putting in the bigger work of migrating us to structured logging. Test Plan: go test
2 years ago
default_grpc_service_configuration.json
zoekt-sourcegraph-indexserver: add gRPC support to sourcegraph client (#551) Co-authored-by: Camden Cheek <camden@ccheek.com>
3 years ago
index.go
Move root-level index code to index package (#902) In the repo root, we have a bunch of low level logic around index building and searching. So we end up exposing internal logic through the main public `zoekt` package, for example `zoekt.Merge(...)`. This PR moves it into the `build` package, so all code related to index building lives together. It then renames `build` to `index` to reflect the broader focus on indexing and searching the index.
1 year ago
index_mutex.go
indexserver: only log if someone else is running (#395) We inverted the condition around alreadyRunning. Luckily the only side-effect was extra debug logs. I noticed this while testing with debug logs on. Test Plan: Run with debug logs and notice the already running messages disappear. Add in a sleep for 2 minutes in indexing then check already running messages pop up. SRC_LOG_LEVEL=dbug go run ./cmd/zoekt-sourcegraph-indexserver \ -listen 127.0.0.1:6072 \ -index_concurrency 4 \ -sourcegraph_url ~/src \ -index /tmp/zoekt-test plz-review-url: https://plz.review/review/6899
4 years ago
index_test.go
Move several packages to internal/ (#901) This PR moves the following packages to `internal` to avoid exposing them in the API: * `ctags` * `debugserver` * `gitindex` * `shards` * `trace`
1 year ago
main.go
Document all the commands + packages (#904) This PR adds doc comments for all packages/ commands.
1 year ago
main_test.go
all: set severity hint in most log output (#871) I was inspecting logs in GKE and it incorrectly categorized the severity of nearly all logs from zoekt-webserver and zoekt-indexserver. This is a hack to make it work better without putting in the bigger work of migrating us to structured logging. Test Plan: go test
2 years ago
merge.go
Move root-level index code to index package (#902) In the repo root, we have a bunch of low level logic around index building and searching. So we end up exposing internal logic through the main public `zoekt` package, for example `zoekt.Merge(...)`. This PR moves it into the `build` package, so all code related to index building lives together. It then renames `build` to `index` to reflect the broader focus on indexing and searching the index.
1 year ago
merge_test.go
Move root-level index code to index package (#902) In the repo root, we have a bunch of low level logic around index building and searching. So we end up exposing internal logic through the main public `zoekt` package, for example `zoekt.Merge(...)`. This PR moves it into the `build` package, so all code related to index building lives together. It then renames `build` to `index` to reflect the broader focus on indexing and searching the index.
1 year ago
meta.go
Fully revert windows support (#909) We added support for Windows in https://github.com/sourcegraph/zoekt/pull/535. We then partially reverted the change, specifically the parts related to mmap (#706). This was okay because we no longer build for Windows. This PR fully removes support to avoid being in a partially-implemented state.
1 year ago
meta_test.go
Move root-level index code to index package (#902) In the repo root, we have a bunch of low level logic around index building and searching. So we end up exposing internal logic through the main public `zoekt` package, for example `zoekt.Merge(...)`. This PR moves it into the `build` package, so all code related to index building lives together. It then renames `build` to `index` to reflect the broader focus on indexing and searching the index.
1 year ago
owner.go
all: set severity hint in most log output (#871) I was inspecting logs in GKE and it incorrectly categorized the severity of nearly all logs from zoekt-webserver and zoekt-indexserver. This is a hack to make it work better without putting in the bigger work of migrating us to structured logging. Test Plan: go test
2 years ago
owner_test.go
all: gofumpt -l -w . gofumpt is a stricter gofmt. I took a look at the changes and in general they are nice. I don't think we need to enforce the use of gofumpt, but I like the idea of running it every once in a while. Test Plan: go test ./...
2 years ago
queue.go
all: set severity hint in most log output (#871) I was inspecting logs in GKE and it incorrectly categorized the severity of nearly all logs from zoekt-webserver and zoekt-indexserver. This is a hack to make it work better without putting in the bigger work of migrating us to structured logging. Test Plan: go test
2 years ago
queue_test.go
Add metric for indexing delay (#804) Our dashboards currently report indexing delay using the `index_queue_age_seconds` metric, which tracks the duration indexing jobs wait in the queue before being "popped" for indexing. For dot com, we regularly see the median for this metric at 2 days. However, this metric may be misleading because it includes "noop" indexing jobs where the index is already up to date. These jobs can stay in the queue for a long time since they are not high priority. This PR adds a new metric `index_indexing_delay_seconds`, which reports the duration from when an indexing job enters the queue, to when it completes. It uses 'state' as a label, so by setting `state='success'` we can (roughly) see the delay from when Zoekt learns about a new version, to when that version is available to search.
2 years ago
sg.go
Move several packages to internal/ (#901) This PR moves the following packages to `internal` to avoid exposing them in the API: * `ctags` * `debugserver` * `gitindex` * `shards` * `trace`
1 year ago
sg_test.go
ranking: removing document ranks (#853) Document ranks was an experimental feature of Sourcegraph. We have already removed the code in Sourcegraph in the last release. This is the corresponding cleanup for Zoekt. Test plan: updated tests
2 years ago