zoekt/cmd/zoekt-webserver/ at d4c60f21a65f0118f2cada13c0ae9187f8f06010 · boltless.me/zoekt · Tangled

boltless.me / zoekt

0

fork of https://github.com/sourcegraph/zoekt

0

zoekt / cmd / zoekt-webserver /

at d4c60f21a65f0118f2cada13c0ae9187f8f06010 1 folder 5 files

Julie Tibshirani Move root-level index code to index package (#902) 1y ago

Move root-level index code to index package (#902) In the repo root, we have a bunch of low level logic around index building and searching. So we end up exposing internal logic through the main public `zoekt` package, for example `zoekt.Merge(...)`. This PR moves it into the `build` package, so all code related to index building lives together. It then renames `build` to `index` to reflect the broader focus on indexing and searching the index.

1 year ago

Move root-level index code to index package (#902) In the repo root, we have a bunch of low level logic around index building and searching. So we end up exposing internal logic through the main public `zoekt` package, for example `zoekt.Merge(...)`. This PR moves it into the `build` package, so all code related to index building lives together. It then renames `build` to `index` to reflect the broader focus on indexing and searching the index.

1 year ago

all: gofumpt -l -w . gofumpt is a stricter gofmt. I took a look at the changes and in general they are nice. I don't think we need to enforce the use of gofumpt, but I like the idea of running it every once in a while. Test Plan: go test ./...

2 years ago

peterguy/cross platform build (#535) * Replace unix/syscall `mmap` with `mmap-go` `mmap-go` is a cross-platform memory-mapping package. It uses `Mmap` on *Nix systems and `CreateFileMapping` + `MapViewOfFile` on Windows. `mmap-go` benchmarks the same as `unix.Mmap` (see [https://github.com/peterguy/benchmark-mmap](https://github.com/peterguy/benchmark-mmap)). BONUS! `indexfile_other.go` used straight file reads (`os.File::ReadAt`) to support the same kind of behavior as mmap on non-*Nix platforms; `mmap-go` uses Windows APIs to do real memory mapping on Windows, which is faster by several magnitudes. Also change the rounding/padding of the memory buffer being allocated. It was hard-coded to round up to the nearest 4096 bytes in order to page align for mmap, but not all operating systems use 4k pages. Instead of the hard coded value, use `os.Getpagesize()`. * Build-constrain usages of `unix.Umask` * build-constrain zoekt webserver for cross-platform * Fix buffer size in Windows and refacter to a method that calculates the correct buffer size for the platform * use gopsutil v3 for disk space metrics instead of unix/syscall Statfs * Build-constrain the Prometheus memory map metrics Windows does not have the concept of a count of mapped memory areas. Non-linux operating systems do not use the /sys pseudo-filesystem. Constrain `mustRegisterMemoryMapMetrics` to linux-only builds. Other platforms use a no-op function. * cleanup - go get -u -t ./.. - remove unused import * Update cmd/zoekt-webserver/main.go Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com> --------- Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com>

3 years ago

main_unsupported.go

peterguy/cross platform build (#535) * Replace unix/syscall `mmap` with `mmap-go` `mmap-go` is a cross-platform memory-mapping package. It uses `Mmap` on *Nix systems and `CreateFileMapping` + `MapViewOfFile` on Windows. `mmap-go` benchmarks the same as `unix.Mmap` (see [https://github.com/peterguy/benchmark-mmap](https://github.com/peterguy/benchmark-mmap)). BONUS! `indexfile_other.go` used straight file reads (`os.File::ReadAt`) to support the same kind of behavior as mmap on non-*Nix platforms; `mmap-go` uses Windows APIs to do real memory mapping on Windows, which is faster by several magnitudes. Also change the rounding/padding of the memory buffer being allocated. It was hard-coded to round up to the nearest 4096 bytes in order to page align for mmap, but not all operating systems use 4k pages. Instead of the hard coded value, use `os.Getpagesize()`. * Build-constrain usages of `unix.Umask` * build-constrain zoekt webserver for cross-platform * Fix buffer size in Windows and refacter to a method that calculates the correct buffer size for the platform * use gopsutil v3 for disk space metrics instead of unix/syscall Statfs * Build-constrain the Prometheus memory map metrics Windows does not have the concept of a count of mapped memory areas. Non-linux operating systems do not use the /sys pseudo-filesystem. Constrain `mustRegisterMemoryMapMetrics` to linux-only builds. Other platforms use a no-op function. * cleanup - go get -u -t ./.. - remove unused import * Update cmd/zoekt-webserver/main.go Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com> --------- Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com>

3 years ago

main_windows.go

peterguy/cross platform build (#535) * Replace unix/syscall `mmap` with `mmap-go` `mmap-go` is a cross-platform memory-mapping package. It uses `Mmap` on *Nix systems and `CreateFileMapping` + `MapViewOfFile` on Windows. `mmap-go` benchmarks the same as `unix.Mmap` (see [https://github.com/peterguy/benchmark-mmap](https://github.com/peterguy/benchmark-mmap)). BONUS! `indexfile_other.go` used straight file reads (`os.File::ReadAt`) to support the same kind of behavior as mmap on non-*Nix platforms; `mmap-go` uses Windows APIs to do real memory mapping on Windows, which is faster by several magnitudes. Also change the rounding/padding of the memory buffer being allocated. It was hard-coded to round up to the nearest 4096 bytes in order to page align for mmap, but not all operating systems use 4k pages. Instead of the hard coded value, use `os.Getpagesize()`. * Build-constrain usages of `unix.Umask` * build-constrain zoekt webserver for cross-platform * Fix buffer size in Windows and refacter to a method that calculates the correct buffer size for the platform * use gopsutil v3 for disk space metrics instead of unix/syscall Statfs * Build-constrain the Prometheus memory map metrics Windows does not have the concept of a count of mapped memory areas. Non-linux operating systems do not use the /sys pseudo-filesystem. Constrain `mustRegisterMemoryMapMetrics` to linux-only builds. Other platforms use a no-op function. * cleanup - go get -u -t ./.. - remove unused import * Update cmd/zoekt-webserver/main.go Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com> --------- Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com>

3 years ago