fix: delete heap dump file when attachment exceeds size limit - #5481
fix: delete heap dump file when attachment exceeds size limit#5481XAN9xXx wants to merge 2 commits into
Conversation
jamescrosswell
left a comment
There was a problem hiding this comment.
Thanks for the PR @XAN9xXx . I think it might not have been clear from the issue what we were trying to achieve though.
Your current PR would only delete heap dumps if they exceeded the maximum size and we were unable to send these to Sentry. Other heap dumps that are successfully sent to Sentry would stay on disk forever.
We originally wanted to remove any heap dumps that had been successfully sent to Sentry - these are no longer required as they're now available in the Sentry dashboard.
Bruno's comment was that we should also remove heap dumps that we failed to send, since (and especially since these are large) these could easily exhaust available disk space on some servers.
BackgroundWorker.DoWorkAsync does using var _ = envelope around the send, so the envelope — and the FileStream — is disposed once the background worker has finished with it... using FileOptions.DeleteOnClose (only for heap dumps) might be a good way to deal with this.
Using FileOptions.DeleteOnClose also has the advantage that it would allow cleaning up gcdump files even when CacheDirectoryPath is set:
- Worker awaits _transport.SendEnvelopeAsync(envelope) → CachingTransport.StoreToCacheAsync
- That awaits envelope.SerializeAsync(stream, …) (CachingTransport.cs:486), which copies the .gcdump bytes into the cache file
- Returns → worker's using disposes the envelope → FileStream closes → dump deleted
So the original is only removed once a copy exists in the cache directory. The CachingTransport send reads from that copy, and InnerProcessCacheAsync deletes the cache file afterwards.
|
Thanks for the explanation @jamescrosswell , that clarifies things a lot — makes sense to use My laptop is currently being repaired, so I likely won't get back to this for a few days. Will follow up once I'm back! |
|
@jamescrosswell Reworked this using |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 5d51697. Configure here.
| FileAccess.Read, | ||
| FileShare.ReadWrite, | ||
| bufferSize: 4096, | ||
| options); |
There was a problem hiding this comment.
DeleteOnClose races with Spotlight
Medium Severity
With EnableSpotlight, SpotlightHttpTransport starts the inner send and then runs ProcessEnvelope on the same envelope, disposing shared attachment streams before await sentryTask. Heap dumps open with FileOptions.DeleteOnClose, so that dispose deletes the file and closes the stream while the inner transport (cache write or HTTP send) may still be reading it, causing failed or empty sends and lost heap dumps.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit 5d51697. Configure here.
There was a problem hiding this comment.
ok, I'll revisit the ownership/disposal here and add a Spotlight-specific regression test before adjusting the implementation.
There was a problem hiding this comment.
@jamescrosswell just a quick update on the Spotlight issue.
I dug into it and added a few local regression tests before changing anything else.
Two findings:
The item.Dispose() I added can close the heap-dump stream while the inner transport is still using it. Removing it fixes that case — cleanup can happen when the original envelope is disposed, which is what DeleteOnClose is meant to do.
I also ran into a broader issue that seems to predate this PR: ProcessEnvelope reuses the same EnvelopeItem instances, so Spotlight and inner share the same attachment stream. Spotlight reading it to EOF can leave inner with an empty attachment. DeleteOnClose makes it more visible, but it doesn't look like this PR introduced it.
Before I go any further: should I keep this PR focused on heap-dump cleanup and handle the Spotlight stream-sharing issue separately, or fix both here?
There was a problem hiding this comment.
Thanks @XAN9xXx - I've added a separate issue to track that:
Let's address that separately then and keep this PR focused on the heap dumps.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #5481 +/- ##
==========================================
- Coverage 74.73% 74.73% -0.01%
==========================================
Files 513 513
Lines 18744 18755 +11
Branches 3666 3668 +2
==========================================
+ Hits 14009 14016 +7
- Misses 3863 3864 +1
- Partials 872 875 +3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|


Description
When a heap dump (
.gcdump) attachment exceedsMaxAttachmentSizeand is dropped before sending, the local file was never deleted. Over time this could cause heap dump files to accumulate on disk.This PR adds cleanup logic: when a heap dump attachment is dropped for being too large, the underlying local file is now deleted, with the outcome logged.
Changes
AttachmentType.HeapDumpto distinguish heap dump attachments from other attachment typesEnvelopeItemnow tracks the originating local file path (LocalFilePath) for file-based attachmentsHttpTransportBase.ProcessEnvelopeItemdeletes the local file when a heap dump attachment is dropped for exceeding the size limitEnvelopeItem's underlyingFileStreamwas never disposed when an item was dropped, which would otherwise prevent the file from being deleted on WindowsTesting
Added two tests in
HttpTransportTests.cs:SendEnvelopeAsync_HeapDumpAttachmentTooLarge_DeleteFilesAndLogsDebug— verifies the file is deleted and a debug log is recordedSendEnvelopeAsync_NonHeapDumpAttachmentTooLarge_DoesNotDeleteFile— verifies non-heap-dump attachments are left untouchedFixes #4009