boltless.me / zoekt
fork of https://github.com/sourcegraph/zoekt
0
Fork 0

Configure Feed

Select the types of activity you want to include in your feed.

shards: handle failed cached List for selectRepoSet (#864)

If we failed to List the repositories when loading a shard we would
never search it due to selectRepoSet optimization. In practice this
feels very rare to happen (only for logic error or disk corruption).
However, in those cases we should surface these crashes searches by
attempting to search the shard.

Additionally I add logging so we can notice when this happens. I didn't
add a metric since this is the sort of thing that I think is so rare we
would never think to check the metric (but may notice logs).

Note: I used the slightly tricky invariant that repos being nil means
error. If the shard is actually empty (eg all repos tombstoned) then we
still correctly apply the optimization. In practice having an empty
shard shouldn't really happen so I'm open to just treating empty repos
list as something we have to search.

author
Keegan Carruthers-Smith committer
GitHub date (Nov 21, 2024, 1:38 PM +0200) commit c4b0bc47 parent c1295f90
+10 -4
+10 -4
shards/shards.go
··· 180 180 // We have out of band ranking on compound shards which can change even if 181 181 // the shard file does not. So we compute a rank in getShards. We store 182 182 // repos here to avoid the cost of List in the search request path. 183 + // 184 + // repos is nil only if that call failed. 183 185 repos []*zoekt.Repository 184 186 } 185 187 ··· 447 449 filteredAll := true 448 450 449 451 for _, s := range shards { 450 - if any, all := hasRepos(s.repos); any { 452 + if s.repos == nil { 453 + // repos is nil if we failed to List the shard. This shouldn't 454 + // happen, but if it does we don't know what is in it and must search 455 + // it without simplifying the query. 456 + filtered = append(filtered, s) 457 + filteredAll = false 458 + } else if any, all := hasRepos(s.repos); any { 451 459 filtered = append(filtered, s) 452 460 filteredAll = filteredAll && all 453 461 } ··· 1066 1074 q := query.Const{Value: true} 1067 1075 result, err := s.List(context.Background(), &q, nil) 1068 1076 if err != nil { 1069 - return &rankedShard{Searcher: s} 1070 - } 1071 - if len(result.Repos) == 0 { 1077 + log.Printf("mkRankedShard(%s): failed to cache repository list: %v", s, err) 1072 1078 return &rankedShard{Searcher: s} 1073 1079 } 1074 1080