Skip to content

Feat/image pickup animated urp#938

Draft
Toys0125 wants to merge 4 commits into
BasisVR:developerfrom
Toys0125:feat/image-pickup-animated-urp
Draft

Feat/image pickup animated urp#938
Toys0125 wants to merge 4 commits into
BasisVR:developerfrom
Toys0125:feat/image-pickup-animated-urp

Conversation

@Toys0125

@Toys0125 Toys0125 commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Add end-to-end animated image pickup support for local file drops, synchronized networking, Burst decoding with packet building, GPU/CPU compositing, visibility-based resource management.

  • New animation types: BasisAnimatedImageData, BasisNativeAnimationPayload for frame-based animations
  • GIF decoding: BasisBurstGifDecoder, BasisGifDecoder using Burst-compiled jobs for performant GIF parsing
  • CPU/GPU canvas rendering: BasisAnimatedImageCpuCanvas, BasisAnimatedImageGpuCanvas for animated texture display
  • Network sync: BasisAnimatedImageNetworkCodec, BasisAnimationPacketJobRequest for synchronized playback across clients.
  • Scheduling & playback: BasisAnimatedImageScheduler, BasisAnimatedImagePlayer for coordinated multi-client animation with epoch-based sync
  • Depth visibility: BasisImageDepthVisibilityFeature, BasisAnimatedImageDepthVisibility for occlusion-aware rendering
  • Frame atlas: BasisAnimationFrameAtlas for texture atlas management
  • Burst codec: BasisBurstAnimationCodec for efficient frame compression/decompression
  • Security: Updated BasisImageSecurity with animation-specific validation
  • Localization: 17 language support for animated pickup UI

Supporting changes:

  • Refactored BasisEventDriver with improved per-frame scheduling and new animation job integration
  • Updated BasisNetworkModeration with new moderation categories
  • New BasisGuid128 type for deterministic 128-bit IDs (with tests)
  • Editor drop hook updates (BasisDesktopImageDropHook, BasisImagePickupEditorDrop) for animated image import
  • Rejection popup for invalid animated files
  • Comprehensive editor tests (BasisAnimatedImageTests, BasisGifDecoderTests, BasisGuid128Tests, BasisImageSecurityTests)

Required checks

All boxes below must be ticked before this PR can merge. If a check is genuinely N/A, tick it anyway and explain under Notes.

  • Tested — I built and ran this locally. The change works in the editor and (where relevant) in a built player.
  • Transform access is combined and limited — In hot paths, transform reads/writes go through TransformAccessArray or are otherwise batched. I have not added per-frame transform.position / transform.rotation / transform.localPosition calls inside loops. Whenever I need both position and rotation, I use the combined APIs — SetPositionAndRotation / SetLocalPositionAndRotation for writes, GetPositionAndRotation / GetLocalPositionAndRotation for reads — instead of two separate property accesses; the combined call does one local-to-world matrix traversal instead of two.
  • Addressables used for asset/memory loading — Any new asset loads go through Addressables. No new Resources.Load, no direct asset references that pull large content into memory on scene load.
  • No new GetComponent / AddComponent where avoidable — Where unavoidable, the result is cached on a field, and any GetComponent<T> is replaced with TryGetComponent<T>(out var x) — bare GetComponent will be denied. TryGetComponent is the modern API (Unity 2019.2+) and skips the Editor-only GC allocation GetComponent causes when a component is missing: Unity wraps the null return in a managed "fake null" object so its overloaded == operator can still detect destroyed C++ objects, and constructing that wrapper allocates; TryGetComponent returns a bool plus out parameter and never builds the wrapper. None of these calls run inside Update, LateUpdate, FixedUpdate, jobs, or other per-frame code paths.
  • Per-frame work is scheduled through BasisEventDriver — Any new per-frame work hooks into BasisEventDriver rather than adding standalone Update / LateUpdate / FixedUpdate callbacks on a MonoBehaviour.
  • Anything added to BasisEventDriver is bulletproof, or guarded by try/catchBasisEventDriver runs the single per-frame tick that drives the whole framework (network apply, local player sim, blendshapes, JigglePhysics, nameplates, and more) as one sequential chain. An unhandled exception anywhere in that chain aborts the rest of the tick, so every step after the throwing one is silently skipped for that frame. New work added to the driver must either be guaranteed not to throw, or be wrapped in a try/catch that contains the failure and surfaces it through BasisDebug — logged once / rate-limited, never every frame (see the existing HVRBasisBuiltInAddresses.Simulate() guard for the pattern). Expect this to be scrutinized closely in review.
  • Considered jobification — I asked whether this work can be moved to a Unity Job (Burst-compiled where possible). If it can, it is. If it cannot, the reason is in Notes.
  • No needless { get; set; } properties or access lockdowns — Public fields are fine; Basis is meant to be read and modified freely, so don't wall things off private/internal without a real reason. Don't wrap a field in { get; set; } when the accessors do nothing — property accessors have a real performance cost vs direct field access, and the lead maintainer prefers plain fields (or a method / setter-only property when only the setter needs logic) over a noop-getter pair. For .Instance singletons, callers reassigning Type.Instance is allowed; if that would break your code, log a warning or throw — don't block the assignment. Locking down access is not your call.
  • Camera access goes through BasisLocalCameraDriver — Code that needs the local camera (transform, projection, rig data, etc.) pulls it from BasisLocalCameraDriver rather than looking one up itself. Don't roll a separate camera discovery path.
  • Logging uses BasisDebug — All new logging calls go through BasisDebug.Log / BasisDebug.LogWarning / BasisDebug.LogError (with an appropriate LogTag) instead of UnityEngine.Debug.Log / Debug.LogWarning / Debug.LogError. BasisDebug routes through Basis's tagged, color-coded logger and respects the project-wide LoggingDisabled toggle so logging can be killed at runtime; bare Debug.Log calls bypass that and will be denied.
  • No scene-wide discovery for dependencies — New code is architected so it does not need FindObjectOfType / FindObjectsOfType / GameObject.Find / FindGameObjectsWithTag to locate what it depends on. References are wired in — registered through an existing manager/driver, injected at init, or passed in by the caller — rather than discovered by scanning the scene at runtime. If a scene scan is genuinely unavoidable, justify it under Notes.
  • No allocations in hot paths — Per-frame code (Update / LateUpdate / FixedUpdate, simulation loops, jobs, anything called once per frame or more) does not allocate. No new on reference types, no LINQ, no string concatenation/interpolation, no boxing, no foreach over interface-typed collections. Allocate once at init and reuse the buffer.
  • No debugging in hot paths — No log calls of any kind on per-frame paths, including BasisDebug. Hot-path logging floods the console and incurs cost on every frame regardless of whether the message is filtered out downstream. If a hot-path log is needed while iterating, gate it behind #if UNITY_EDITOR and remove (or leave gated) before merge.
  • Hot-path collection access is optimized — Cache .Count (lists) / .Length (arrays) into a local int before the loop instead of re-reading the property each iteration. Prefer T[] (with a separate length int when the array is over-sized) over List<T> where the data is hot — Unity's mono BCL doesn't expose CollectionsMarshal.AsSpan(List<T>), so a list can't be fed into Span<T> / unsafe paths cleanly. Where the perf justifies it, drop into Span<T> / ref locals / Unsafe.As / unsafe pointer code to skip bounds checks and copies, and call out the invariants you're relying on under Notes so reviewers can sanity-check them.

Testing details

Tick the platforms you actually tested on. Leave the rest unticked — these are informational and do not block merge.

  • Windows
  • Linux
  • Android
  • iOS
  • macOS

Input / control mode coverage:

  • Tested in VR (note headset under Notes)
  • Tested in desktop / non-VR mode
  • Tested with phone controls (mobile touch input)
  • N/A — change does not touch player/XR/input code

Where applicable, confirm these flows still work after your changes:

  • Hot-switching (desktop ↔ VR mode swap at runtime)
  • Avatar swapping
  • Server swapping (joining / leaving / changing servers)
  • N/A — change does not touch any of the above

Notes

  • Attempt to use the compressed payload in a GPU compute shader wasn't working out and was holding up the entire GPU time, and likely need another attempt.
  • When memory budget is overflown it should only play animations that are closer in view, and unloads atlas textures out of VRAM. Images keep their compressed payloads for unloading and loading.

Toys0125 added 2 commits July 11, 2026 21:09
Add end-to-end animated image pickup support for local file drops, synchronized networking, Burst decoding with packet building, GPU/CPU compositing, visibility-based resource management, and hardened IL2CPP behavior.

Features:

* Add GIF support alongside existing PNG and JPEG image pickup handling.
* Support Windows player file drops and Unity editor drag-and-drop workflows.
* Queue dropped image batches so large drops are processed without blocking the main thread.
* Display detailed rejection popups containing the filename, validation reason, configured limits, and batch processing information.
* Add stable 128-bit image identifiers for networked image pickup state.
* Synchronize animated image playback using a shared UTC playback epoch instead of sending frame-by-frame updates.
* Serialize animated image metadata and frame pixels into a compressed Burst-friendly network payload.
* Chunk large animation payloads across network messages and rebuild them on receiving clients.
* Limit concurrent encoding, packet building, transmission, and inbound decoding work.
* Add per-sender animation count, byte, canvas, decoded-pixel, and pending-job budgets.
* Preserve static poster images while animation data is loading, offscreen, or unavailable.

Animated image runtime:

* A Burst GIF parser and decoder with support for:

  * global and local color tables
  * transparency
  * interlaced frames
  * loop metadata
  * Source and Over blending
  * None, Background, and Previous disposal modes
  * truncated or invalid data detection
  * bounded frame, duration, dimension, source-byte, and decoded-pixel limits
* Store frame metadata, timing data, and decoded pixels in persistent native arrays.
* A Burst animation network codec using a compact binary representation and LZ4 compression.
* Asynchronous animation encoding, decoding, poster generation, and packet construction jobs.
* Transfer ownership explicitly when asynchronous results are returned so disposing a request cannot invalidate a claimed animation or payload.

Compositing:

* Add a persistent GPU compositor backed by:

  * a packed frame atlas
  * a premultiplied-alpha render texture canvas
  * Source and Over blend passes
  * Background and Previous frame disposal
  * optional previous-canvas storage
* Add a Burst CPU compositor as a correctness fallback when GPU allocation or rendering is unavailable.
* Use a consistent Unity bottom-left coordinate convention across decoded frame rectangles, CPU composition, atlas storage, GPU viewports, texture copies, and UV sampling.
* Keep a sanitized static poster available when no compositor is allocated.

Scheduling and visibility:

* Add a global animated image scheduler with configurable per-frame transition and composited-pixel budgets.
* Allow one oversized animation update when needed to prevent permanent starvation.
* Suspend animation compositor resources after an image remains offscreen for the configured grace period.
* Restore animation playback from the synchronized epoch when the image becomes visible again.
* Add front-face to avoid animating images that cannot contribute to the current view *Primary on mobile*.
* Add default URP depth-buffer visibility checks using a compute shader and asynchronous GPU readback.
* Grow depth visibility GPU buffers beyond the initial 64-image capacity when additional graphics-buffer memory is available.
* Preserve the current buffer capacity if growth fails and rotate excess images through later batches using physics visibility fallback.
* Track GPU readback generation, camera, and frame state so cancelled or stale readbacks cannot overwrite reset visibility results.

File and platform handling:

* Keep GIF disk reads on a managed background task
* Isolate failures between dropped-file batches and always clear processed batch state.
* Avoid respawning previously processed batches after an exception.

Validation and security:

* Validate file extensions, signatures, dimensions, byte sizes, and decoded output before spawning pickups.
* Apply animation-specific canvas limits to GIF headers before scheduling decode work.
* Reject malformed network payload headers before allocating decoded buffers.
* Validate animation frame bounds, timing, blend modes, disposal modes, pixel ranges, reserved fields, and aggregate limits.
* Re-encode static inputs to sanitized PNG data before networking.
* Provide descriptive limit errors including actual size, configured maximum, and exceeded amount.

Lifecycle and robustness:

* Clear static BasisEventDriver update and late-update callbacks during teardown so stale subscriptions cannot survive play sessions when Domain Reload is disabled.
* Invoke EventDriver subscribers independently so one failing callback does not prevent later subscribers from running.
* Preserve callback snapshots without allocating invocation lists every frame.
* Harden native-array ownership on constructor failure, successful result transfer, cancellation, disposal, and error paths.
* Ensure decoded animations, compressed payloads, textures, render targets, and native buffers have a single explicit owner.
* Prevent stale asynchronous operations from applying results after their associated camera, frame, player, or request has been invalidated.

Tests:

* Add editor tests covering GIF decoding, transparency, interlacing, disposal modes, truncated data, and invalid headers.
* Add animation codec encode/decode round-trip tests.
* Verify claimed GIF, encode, and decode results remain valid after request disposal.
* Verify caller-owned native arrays remain valid when request construction fails.
* Test animation dimension and security limits.
* Test CPU composition and Previous disposal behavior.
* Test scheduler bookkeeping, GPU visibility capacity growth, and graphics-buffer size calculations.
* Test stable image identifier serialization and equality behavior.
@Toys0125 Toys0125 marked this pull request as draft July 12, 2026 05:45
Improve the synchronized animated image system introduced by the previous commit:

- bound decoded-frame, native-memory, payload, transfer, and per-sender resource usage
- simplify GIF decode, network codec, reload, and animation lifecycle internals
- make disposable animation and payload ownership transfers explicit and validated
- retain the resident full-frame GPU atlas compositor with a CPU compatibility fallback
- discard back-facing and out-of-frustum cards on the CPU before depth, raycast, decode, or composition work
- harden spawn metadata parsing, transfer validation, batch image drops, and rejection handling
- improve D3D11 compositor compatibility, alpha validation, and previous-frame bounds safety
- compile edit-mode destruction only for Unity Editor and rely on BasisLocalization for English fallback
- register image-pickup language tables in the shared Addressables localization group
- cache collection and array bounds in added for-loops and use fields where accessors add no behavior
- expand editor coverage for playback, memory limits, visibility, ownership, codecs, and malformed inputs
@Toys0125 Toys0125 force-pushed the feat/image-pickup-animated-urp branch from 31461fd to a8ce876 Compare July 12, 2026 05:46
Release player-owned native data and pending reload jobs synchronously when pickups are removed. Notify the persistent manager when scene-bound pickups are destroyed so manager-owned animation payloads and transfer state do not accumulate across scene changes. Destroy tracked pickups during manager shutdown before Unity leak validation and add editor regressions for native data, pending decode, and payload ownership cleanup.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant