Skip to content

fix(mediaplayer): crop the coded macroblock pad on Windows so 1080p has no top strip#948

Merged
dooly123 merged 2 commits into
BasisVR:developerfrom
towneh:fix/mediaplayer-video-geometry
Jul 13, 2026
Merged

fix(mediaplayer): crop the coded macroblock pad on Windows so 1080p has no top strip#948
dooly123 merged 2 commits into
BasisVR:developerfrom
towneh:fix/mediaplayer-video-geometry

Conversation

@towneh

@towneh towneh commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes a thin black strip along the top edge of the video on Windows at non-16-aligned heights (1080p being the common case).

The Windows decoder sized its output from the coded NV12 surface. H.264/H.265 round coded height up to a multiple of 16 (1080 to 1088), and the VideoProcessorBlt copied the whole coded picture with no source rect, so the ~8 pad rows carried through. The decode-time vertical mirror then moved them from the bottom of the coded picture to the top of the displayed frame.

The backend now reads MF_MT_MINIMUM_DISPLAY_APERTURE from the MFT output type (at configure and on the first-frame stream change, where most decoders first populate it) and crops the video processor to it: the shared/output texture is sized to the visible region and the stream source rect is set to the aperture. The pad never reaches Unity, and Unity receives the true visible dimensions. 720p and other 16-aligned heights are unaffected.

Android is unaffected: it decodes through an AImageReader Surface with render=true, and MediaCodec applies the crop on Surface render, so the delivered frame is already display-sized.

Rebuilds the RIST-enabled Windows x64 plugin DLL.

Scope note: an earlier revision of this branch also flipped the default aspect mode to letterbox non-16:9 content. That turned out to need a deeper fix (the aspect transform is applied as a texture scale/offset, which can't produce black bars on the current shader), so it's been pulled out and is tracked separately in #949. This PR is the top-strip crop only.

Required checks

All boxes below must be ticked before this PR can merge. If a check is genuinely N/A, tick it anyway and explain under Notes.

  • Tested — I built and ran this locally. The change works in the editor and (where relevant) in a built player.
  • Transform access is combined and limited — In hot paths, transform reads/writes go through TransformAccessArray or are otherwise batched. I have not added per-frame transform.position / transform.rotation / transform.localPosition calls inside loops. Whenever I need both position and rotation, I use the combined APIs — SetPositionAndRotation / SetLocalPositionAndRotation for writes, GetPositionAndRotation / GetLocalPositionAndRotation for reads — instead of two separate property accesses; the combined call does one local-to-world matrix traversal instead of two.
  • Addressables used for asset/memory loading — Any new asset loads go through Addressables. No new Resources.Load, no direct asset references that pull large content into memory on scene load.
  • No new GetComponent / AddComponent where avoidable — Where unavoidable, the result is cached on a field, and any GetComponent<T> is replaced with TryGetComponent<T>(out var x) — bare GetComponent will be denied. TryGetComponent is the modern API (Unity 2019.2+) and skips the Editor-only GC allocation GetComponent causes when a component is missing: Unity wraps the null return in a managed "fake null" object so its overloaded == operator can still detect destroyed C++ objects, and constructing that wrapper allocates; TryGetComponent returns a bool plus out parameter and never builds the wrapper. None of these calls run inside Update, LateUpdate, FixedUpdate, jobs, or other per-frame code paths.
  • Per-frame work is scheduled through BasisEventDriver — Any new per-frame work hooks into BasisEventDriver rather than adding standalone Update / LateUpdate / FixedUpdate callbacks on a MonoBehaviour.
  • Anything added to BasisEventDriver is bulletproof, or guarded by try/catchBasisEventDriver runs the single per-frame tick that drives the whole framework (network apply, local player sim, blendshapes, JigglePhysics, nameplates, and more) as one sequential chain. An unhandled exception anywhere in that chain aborts the rest of the tick, so every step after the throwing one is silently skipped for that frame. New work added to the driver must either be guaranteed not to throw, or be wrapped in a try/catch that contains the failure and surfaces it through BasisDebug — logged once / rate-limited, never every frame (see the existing HVRBasisBuiltInAddresses.Simulate() guard for the pattern). Expect this to be scrutinized closely in review.
  • Considered jobification — I asked whether this work can be moved to a Unity Job (Burst-compiled where possible). If it can, it is. If it cannot, the reason is in Notes.
  • No needless { get; set; } properties or access lockdowns — Public fields are fine; Basis is meant to be read and modified freely, so don't wall things off private/internal without a real reason. Don't wrap a field in { get; set; } when the accessors do nothing — property accessors have a real performance cost vs direct field access, and the lead maintainer prefers plain fields (or a method / setter-only property when only the setter needs logic) over a noop-getter pair. For .Instance singletons, callers reassigning Type.Instance is allowed; if that would break your code, log a warning or throw — don't block the assignment. Locking down access is not your call.
  • Camera access goes through BasisLocalCameraDriver — Code that needs the local camera (transform, projection, rig data, etc.) pulls it from BasisLocalCameraDriver rather than looking one up itself. Don't roll a separate camera discovery path.
  • Logging uses BasisDebug — All new logging calls go through BasisDebug.Log / BasisDebug.LogWarning / BasisDebug.LogError (with an appropriate LogTag) instead of UnityEngine.Debug.Log / Debug.LogWarning / Debug.LogError. BasisDebug routes through Basis's tagged, color-coded logger and respects the project-wide LoggingDisabled toggle so logging can be killed at runtime; bare Debug.Log calls bypass that and will be denied.
  • No scene-wide discovery for dependencies — New code is architected so it does not need FindObjectOfType / FindObjectsOfType / GameObject.Find / FindGameObjectsWithTag to locate what it depends on. References are wired in — registered through an existing manager/driver, injected at init, or passed in by the caller — rather than discovered by scanning the scene at runtime. If a scene scan is genuinely unavoidable, justify it under Notes.
  • No allocations in hot paths — Per-frame code (Update / LateUpdate / FixedUpdate, simulation loops, jobs, anything called once per frame or more) does not allocate. No new on reference types, no LINQ, no string concatenation/interpolation, no boxing, no foreach over interface-typed collections. Allocate once at init and reuse the buffer.
  • No debugging in hot paths — No log calls of any kind on per-frame paths, including BasisDebug. Hot-path logging floods the console and incurs cost on every frame regardless of whether the message is filtered out downstream. If a hot-path log is needed while iterating, gate it behind #if UNITY_EDITOR and remove (or leave gated) before merge.
  • Hot-path collection access is optimized — Cache .Count (lists) / .Length (arrays) into a local int before the loop instead of re-reading the property each iteration. Prefer T[] (with a separate length int when the array is over-sized) over List<T> where the data is hot — Unity's mono BCL doesn't expose CollectionsMarshal.AsSpan(List<T>), so a list can't be fed into Span<T> / unsafe paths cleanly. Where the perf justifies it, drop into Span<T> / ref locals / Unsafe.As / unsafe pointer code to skip bounds checks and copies, and call out the invariants you're relying on under Notes so reviewers can sanity-check them.

Testing details

Tick the platforms you actually tested on. Leave the rest unticked — these are informational and do not block merge.

  • Windows
  • Linux
  • Android
  • iOS
  • macOS

Input / control mode coverage:

  • Tested in VR (note headset under Notes)
  • Tested in desktop / non-VR mode
  • Tested with phone controls (mobile touch input)
  • N/A — change does not touch player/XR/input code

Where applicable, confirm these flows still work after your changes:

  • Hot-switching (desktop ↔ VR mode swap at runtime)
  • Avatar swapping
  • Server swapping (joining / leaving / changing servers)
  • N/A — change does not touch any of the above

Notes

  • Verified. In-editor playback across multiple 1080p streams: the top strip is gone, no regression. Backed by a standalone Media Foundation probe on a real 1080p decode, which confirmed the coded surface is 1920×1088 with MF_MT_MINIMUM_DISPLAY_APERTURE = offset (0,0), 1920×1080 (the 8-row bottom pad), that the aperture read returns exactly the crop rect the fix applies, and that the aperture is absent before the first frame and only appears after the stream change — which is why the fix reads it in both places. 720p and other 16-aligned heights don't exercise the pad.
  • Most required boxes are N/A: this is a native Media Foundation crop with no managed hot-path, transform, camera, Addressables, GetComponent, or allocation surface. The video-processor source rect is set once at processor creation, not per frame; the native side logs only via basis_engine_set_error.
  • No Android change: its Surface-render decode path already applies the crop, so the strip doesn't reproduce there.

@towneh towneh force-pushed the fix/mediaplayer-video-geometry branch 2 times, most recently from 3d15288 to 54c2dac Compare July 13, 2026 02:16
@towneh towneh changed the title fix(mediaplayer): correct video geometry — FitInside default + Windows coded-pad crop fix(mediaplayer): crop the coded macroblock pad on Windows so 1080p has no top strip Jul 13, 2026
@towneh towneh added the bug Something isn't working label Jul 13, 2026
@towneh towneh requested a review from dooly123 July 13, 2026 02:35
…as no top strip

The Windows decoder sized its output from the coded NV12 surface. H.264/H.265
round coded height up to a multiple of 16 (1080 -> 1088), and the VideoProcessorBlt
copied the whole coded picture with no source rect, so the ~8 pad rows carried
through; the decode-time vertical mirror then moved them from the bottom of the
coded picture to the top of the displayed frame: a thin black strip on every
protocol at any non-16-aligned height (1080p being the common case; 720p is clean).

Read MF_MT_MINIMUM_DISPLAY_APERTURE from the MFT output type (at configure and on
the first-frame stream change, where most decoders first populate it) and crop the
video processor to it: size the shared/output texture to the visible region and set
the stream source rect to the aperture. The pad never reaches Unity, and Unity
receives the true visible dimensions.

Android is unaffected: it decodes through an AImageReader Surface with render=true,
and MediaCodec applies the crop when rendering to a Surface, so the delivered frame
is already the display size.

Rebuilds the RIST-enabled Windows x64 plugin DLL.
@towneh towneh force-pushed the fix/mediaplayer-video-geometry branch from 54c2dac to 767f7df Compare July 13, 2026 02:42
…r-video-geometry

# Conflicts:
#	Basis/Packages/com.basis.mediaplayer/Plugins/Windows/x86_64/basis_media_native.dll
@dooly123 dooly123 merged commit 65c1ee0 into BasisVR:developer Jul 13, 2026
12 checks passed
@towneh towneh deleted the fix/mediaplayer-video-geometry branch July 13, 2026 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants