CUDA illegal memory access in clamp_max/bucketize after culling small number of Gaussians (RTX 5090 / sm_120)

[richiepic]

## Description
Training crashes deterministically with a CUDA illegal memory access error shortly after a densification/refinement step that culls a very small number of Gaussians (5-6 in my reproductions). The crash occurs inside `at::cuda::where_outf` / `at::cuda::bucketize_outf` / `clamp_max`, immediately following the "Culled N gaussians, remaining M" log message.

## Environment
- OS: Windows 11
- GPU: NVIDIA RTX 5090 (Blackwell, compute capability sm_120)
- CUDA Toolkit: 13.1
- LibTorch: 2.8.0+cu129 (Windows, shared-with-deps)
- Build: from latest `main` branch (commit as of 2026-07-06), CMAKE_CUDA_ARCHITECTURES=120
- Compiler: MSVC 19.42 (VS 2022)

## Steps to reproduce
1. Build OpenSplat from source with the above configuration.
2. Download the sample "banana" dataset from the README.
3. Run: 

opensplat.exe path\to\banana -n 30000 -o out.ply

4. Training proceeds normally until ~step 11200-11500 (varies slightly between runs), where a refinement step culls a small number of Gaussians (observed: 5 and 6 in two separate runs), then immediately crashes.

## Observed behavior

Added 0 gaussians, new count 4566
Culled 5 gaussians, remaining 4561
CUDA error: an illegal memory access was encountered

Full stack trace (CUDA_LAUNCH_BLOCKING=1):

Step 11500: 0.291828 (38%)
Added 0 gaussians, new count 4566
Culled 5 gaussians, remaining 4561
CUDA error: an illegal memory access was encountered
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

Exception raised from c10_cuda_check_implementation at C:\actions-runner_work\pytorch\pytorch\pytorch\c10\cuda\CUDAException.cpp:42 (most recent call first):
00007FFF7955199400007FFF795518F0 c10.dll!c10::Error::Error [ @ ]
00007FFF794F3BE800007FFF794F3BA0 c10.dll!c10::AcceleratorError::AcceleratorError [ @ ]
00007FFF7949819200007FFF79498050 c10_cuda.dll!c10::cuda::c10_cuda_check_implementation [ @ ]
00007FFF79485C5300007FFF79484790 c10_cuda.dll!c10::cuda::MemPool::allocator [ @ ]
00007FFF7948398900007FFF794822A0 c10_cuda.dll!c10::cuda::CUDAKernelLaunchRegistry::insert [ @ ]
00007FFF7948BB2A00007FFF7948B780 c10_cuda.dll!c10::cuda::MemPool::is_symmetric [ @ ]
00007FFF7948CB2000007FFF7948B780 c10_cuda.dll!c10::cuda::MemPool::is_symmetric [ @ ]
00007FFF794846A600007FFF794822A0 c10_cuda.dll!c10::cuda::CUDAKernelLaunchRegistry::insert [ @ ]
00007FFF79513DA300007FFF79513D40 c10.dll!c10::StorageImpl::StorageImpl [ @ ]
00007FFF240D32B100007FFF240D2B70 torch_cpu.dll!at::DynamicLibrary::sym [ @ ]
00007FFF240D55E400007FFF240D55B0 torch_cpu.dll!at::detail::empty_generic [ @ ]
00007FFED5EF1F3500007FFED5EF1E50 torch_cuda.dll!at::detail::empty_cuda [ @ ]
00007FFED5EF1D4900007FFED5EF1C10 torch_cuda.dll!at::detail::empty_cuda [ @ ]
00007FFED7D2680900007FFED7D18D80 torch_cuda.dll!at::cuda::set__symint [ @ ]
00007FFF2413DBE500007FFF2413D6D0 torch_cpu.dll!at::TensorIteratorBase::fast_set_up [ @ ]
00007FFF24139BD700007FFF24139B50 torch_cpu.dll!at::TensorIteratorBase::build [ @ ]
00007FFF2413A90300007FFF2413A860 torch_cpu.dll!at::TensorIteratorBase::build_borrowing_unary_op [ @ ]
00007FFF2462D3AF00007FFF2462D330 torch_cpu.dll!at::meta::structured_clamp_max::meta [ @ ]
00007FFED7DF499500007FFED7DB5950 torch_cuda.dll!at::cuda::where_outf [ @ ]
00007FFED7D00BC600007FFED7C94830 torch_cuda.dll!at::cuda::bucketize_outf [ @ ]
00007FFF24DE12A500007FFF24DE1220 torch_cpu.dll!at::_ops::clamp_max::redispatch [ @ ]
00007FFF2680E02400007FFF26618760 torch_cpu.dll!torch::autograd::UndefinedGradBackward::apply_with_saved [ @ ]
00007FFF2680723C00007FFF26618760 torch_cpu.dll!torch::autograd::UndefinedGradBackward::apply_with_saved [ @ ]
00007FFF24D522F500007FFF24D52170 torch_cpu.dll!at::_ops::clamp_max::call [ @ ]
00007FF6184B938000007FF61847E400 opensplat.exe!c10::ivalue::Future::waitAndThrow [ @ ]
00007FF618481C9100007FF61847E400 opensplat.exe!c10::ivalue::Future::waitAndThrow [ @ ]
00007FF6184F6C7800007FF61847E400 opensplat.exe!c10::ivalue::Future::waitAndThrow [ @ ]
00007FF8C8CDE95700007FF8C8CDE940 KERNEL32.DLL!BaseThreadInitThunk [ @ ]
00007FF8C9D87C1C00007FF8C9D87BF0 ntdll.dll!RtlUserThreadStart [ @ ]

## Workaround found
Setting `--warmup-length` higher than `--num-iters` (so densification/pruning never triggers) allows training to complete without any crash. This strongly suggests the bug is isolated to the pruning/culling code path, specifically when a very small number of Gaussians is removed in a single refinement step.

## Additional notes
- Reproduced twice, independently, both times crashing right after a small culling event (not necessarily the same step number, but always shortly after a "Culled N gaussians" log with a low N).
- Not yet tested on non-Blackwell GPUs — happy to help test if useful, but wanted to flag it given how clean the repro is.

[pierotofy]

Mm, I unfortunately don't have an RTX 5090, so if it's hardware dependent I won't have a way to reproduce this. Open a PR if you find a solution? 🙏