build: bump keepcurrent for sync memory fix#671
Merged
Conversation
Pulls in getlantern/keepcurrent#9, which pre-sizes the reads used to sync the MaxMind GeoLite2 mmdb (~75MB) through geo's FromTarGz(FromWeb(...)) pipeline. The old io.ReadAll grew its buffer by repeated reallocation, churning through a chain of ever-larger backing arrays per sync; on 1GB hosts that transient spike dominated RSS and tripped the host-memory>85% alert. http-proxy's keepcurrent was pinned to a 2022 commit, so the bump also carries the transitive archiver/v3 -> mholt/archives migration and the minor golang.org/x/* upgrades it brings in. keepcurrent stays indirect; no first-party code changes.
This was referenced Jun 16, 2026
reflog
added a commit
that referenced
this pull request
Jun 16, 2026
* build: bump keepcurrent for fileSource pre-size fix Pulls in getlantern/keepcurrent#10, which makes fileSource size-aware so geo's startup InitFrom(FromFile) read of the cached ~75MB mmdb stops churning through io.ReadAll's realloc staircase. Completes the memory fix started in #671 (which covered the web-sync path). NOTE: pinned to the keepcurrent PR-head commit; re-pin to the squashed main commit after keepcurrent#10 merges (go get keepcurrent@<main> && go mod tidy). * build: re-pin keepcurrent to merged main (fileSource pre-size fix #10) --------- Co-authored-by: Ilya Yakelzon <reflog@getlantern.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bumps
github.com/getlantern/keepcurrentto include keepcurrent#9.Why
Several 1 GB proxy hosts were tripping the SigNoz "Host memory usage high" alert (10 m avg
system.memory.usage> 85%). Heap pprof from thehttp-proxybinary showed the dominant live allocation wasio.ReadAll— 94.9 MB / 146 MB ≈ 65% of the Go heap — all from:That's
getlantern/geosyncing the MaxMind GeoLite2 mmdb (~75 MB) viakeepcurrent.FromTarGz(FromWeb(...)). The payload was buffered withio.ReadAll, which grows by repeated reallocation — churning through a chain of ever-larger backing arrays (the 75/60/48/38… MB generations seen in the heap) on every sync. On a 1 GB box that transient churn dominated RSS.What
keepcurrent#9 pre-sizes those reads from the reader's known length (HTTP
Content-Length, archive entry size, in-memoryLen()), turning the multi-realloc read into a single allocation, with an overflow/OOM cap that falls back toio.ReadAllfor bogus sizes. No public API change.Note on diff size
This module's keepcurrent was pinned to a 2022 commit, so the bump also carries keepcurrent's
mholt/archiver/v3→mholt/archivesmigration and the minorgolang.org/x/*upgrades MVS pulls in with it — hence the larger go.mod/go.sum churn. No first-party code changes; keepcurrent stays an indirect dependency.Build
go build ./...produces no new errors — all geo-consuming packages (incl. thehttp-proxymain command) compile clean. The only failure is the pre-existinganacrolix/go-libutpcgo issue (typedef uint8 bool), which also fails on cleanmainand is unrelated to this change.