alpha
Login
or
Join now
boltless.me
/
zoekt
Star
0
Fork
0
Atom
Configure Feed
Issues
Pull Requests
Commits
Tags
Feed URL
Select the types of activity you want to include in your feed.
fork of https://github.com/sourcegraph/zoekt
Star
0
Fork
0
Atom
Configure Feed
Issues
Pull Requests
Commits
Tags
Feed URL
Select the types of activity you want to include in your feed.
Overview
Issues
Pulls
Pipelines
zoekt
/
cmd
/
zoekt-webserver
/
at
6e162c732f8a291ef990b809bcc71ebb1fde7df0
1 folder
5 files
Stefan Hengl
sourcegraph: multi-tenant Zoekt (#859)
2y ago
c52a9cde
grpc
sourcegraph: multi-tenant Zoekt (#859) This updates webserver and sourcegraph-indexserver to support multi-tenancy. The change is behind an ENV feature-flag. Key changes: - tenant ID is now part of the index (repo metadata) - GRPC: IndexOption and Repository have a new field TenantId - If multi-tenancy is enabled, webserver checks if tenant in context matches the tenant id in the shard - zoekt-git-index has a new parameter "-shard_prefix ". If set, the value will be used instead of repository name as prefix for the name of the shard. For Sourcegraph we use "<tenant id>_<repository id>" as prefix if multi-tenancy is enabled Assumption: All calls to Sourcegraph are privileged Test plan: - New tests - Ran this together with Sourcegraph (with and without MT enabled)
2 years ago
main.go
sourcegraph: multi-tenant Zoekt (#859) This updates webserver and sourcegraph-indexserver to support multi-tenancy. The change is behind an ENV feature-flag. Key changes: - tenant ID is now part of the index (repo metadata) - GRPC: IndexOption and Repository have a new field TenantId - If multi-tenancy is enabled, webserver checks if tenant in context matches the tenant id in the shard - zoekt-git-index has a new parameter "-shard_prefix ". If set, the value will be used instead of repository name as prefix for the name of the shard. For Sourcegraph we use "<tenant id>_<repository id>" as prefix if multi-tenancy is enabled Assumption: All calls to Sourcegraph are privileged Test plan: - New tests - Ran this together with Sourcegraph (with and without MT enabled)
2 years ago
main_linux.go
all: gofumpt -l -w . gofumpt is a stricter gofmt. I took a look at the changes and in general they are nice. I don't think we need to enforce the use of gofumpt, but I like the idea of running it every once in a while. Test Plan: go test ./...
2 years ago
main_unix.go
peterguy/cross platform build (#535) * Replace unix/syscall `mmap` with `mmap-go` `mmap-go` is a cross-platform memory-mapping package. It uses `Mmap` on *Nix systems and `CreateFileMapping` + `MapViewOfFile` on Windows. `mmap-go` benchmarks the same as `unix.Mmap` (see [https://github.com/peterguy/benchmark-mmap](https://github.com/peterguy/benchmark-mmap)). BONUS! `indexfile_other.go` used straight file reads (`os.File::ReadAt`) to support the same kind of behavior as mmap on non-*Nix platforms; `mmap-go` uses Windows APIs to do real memory mapping on Windows, which is faster by several magnitudes. Also change the rounding/padding of the memory buffer being allocated. It was hard-coded to round up to the nearest 4096 bytes in order to page align for mmap, but not all operating systems use 4k pages. Instead of the hard coded value, use `os.Getpagesize()`. * Build-constrain usages of `unix.Umask` * build-constrain zoekt webserver for cross-platform * Fix buffer size in Windows and refacter to a method that calculates the correct buffer size for the platform * use gopsutil v3 for disk space metrics instead of unix/syscall Statfs * Build-constrain the Prometheus memory map metrics Windows does not have the concept of a count of mapped memory areas. Non-linux operating systems do not use the /sys pseudo-filesystem. Constrain `mustRegisterMemoryMapMetrics` to linux-only builds. Other platforms use a no-op function. * cleanup - go get -u -t ./.. - remove unused import * Update cmd/zoekt-webserver/main.go Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com> --------- Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com>
3 years ago
main_unsupported.go
peterguy/cross platform build (#535) * Replace unix/syscall `mmap` with `mmap-go` `mmap-go` is a cross-platform memory-mapping package. It uses `Mmap` on *Nix systems and `CreateFileMapping` + `MapViewOfFile` on Windows. `mmap-go` benchmarks the same as `unix.Mmap` (see [https://github.com/peterguy/benchmark-mmap](https://github.com/peterguy/benchmark-mmap)). BONUS! `indexfile_other.go` used straight file reads (`os.File::ReadAt`) to support the same kind of behavior as mmap on non-*Nix platforms; `mmap-go` uses Windows APIs to do real memory mapping on Windows, which is faster by several magnitudes. Also change the rounding/padding of the memory buffer being allocated. It was hard-coded to round up to the nearest 4096 bytes in order to page align for mmap, but not all operating systems use 4k pages. Instead of the hard coded value, use `os.Getpagesize()`. * Build-constrain usages of `unix.Umask` * build-constrain zoekt webserver for cross-platform * Fix buffer size in Windows and refacter to a method that calculates the correct buffer size for the platform * use gopsutil v3 for disk space metrics instead of unix/syscall Statfs * Build-constrain the Prometheus memory map metrics Windows does not have the concept of a count of mapped memory areas. Non-linux operating systems do not use the /sys pseudo-filesystem. Constrain `mustRegisterMemoryMapMetrics` to linux-only builds. Other platforms use a no-op function. * cleanup - go get -u -t ./.. - remove unused import * Update cmd/zoekt-webserver/main.go Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com> --------- Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com>
3 years ago
main_windows.go
peterguy/cross platform build (#535) * Replace unix/syscall `mmap` with `mmap-go` `mmap-go` is a cross-platform memory-mapping package. It uses `Mmap` on *Nix systems and `CreateFileMapping` + `MapViewOfFile` on Windows. `mmap-go` benchmarks the same as `unix.Mmap` (see [https://github.com/peterguy/benchmark-mmap](https://github.com/peterguy/benchmark-mmap)). BONUS! `indexfile_other.go` used straight file reads (`os.File::ReadAt`) to support the same kind of behavior as mmap on non-*Nix platforms; `mmap-go` uses Windows APIs to do real memory mapping on Windows, which is faster by several magnitudes. Also change the rounding/padding of the memory buffer being allocated. It was hard-coded to round up to the nearest 4096 bytes in order to page align for mmap, but not all operating systems use 4k pages. Instead of the hard coded value, use `os.Getpagesize()`. * Build-constrain usages of `unix.Umask` * build-constrain zoekt webserver for cross-platform * Fix buffer size in Windows and refacter to a method that calculates the correct buffer size for the platform * use gopsutil v3 for disk space metrics instead of unix/syscall Statfs * Build-constrain the Prometheus memory map metrics Windows does not have the concept of a count of mapped memory areas. Non-linux operating systems do not use the /sys pseudo-filesystem. Constrain `mustRegisterMemoryMapMetrics` to linux-only builds. Other platforms use a no-op function. * cleanup - go get -u -t ./.. - remove unused import * Update cmd/zoekt-webserver/main.go Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com> --------- Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com>
3 years ago