Geometric Reciprocity: Unlocking Self-Supervision for Stereoscopic Video Generation (ICML 2026)

Jingyi Lu, Kai Han
Visual AI Lab, The University of Hong Kong

TLDR: Geometric Reciprocity constructs scalable, high-quality stereo inpainting data from real-world monocular videos, using inference-time consistent masks without relying on paired stereo videos.

Quick Start

Step 1: Installation

pip install -r requirements.txt
# or: uv pip install -r requirements.txt

Step 2: Inference

# The first run downloads checkpoints automatically; outputs are saved to output/example/.
python scripts/inference.py --input-video assets/example.mp4 --output-dir output/example --output-mode all

Manual checkpoint download

mkdir -p checkpoints
python -c "from huggingface_hub import hf_hub_download; hf_hub_download('depth-anything/Video-Depth-Anything-Large', 'video_depth_anything_vitl.pth', local_dir='checkpoints', local_dir_use_symlinks=False)"
python -c "from huggingface_hub import hf_hub_download; hf_hub_download('LuJingyi/grt-video-inpaint', 'video_inpaint_large.pth', local_dir='checkpoints', local_dir_use_symlinks=False)"

Data Construction

GRT masks are computed from monocular depth and geometric reciprocity, then used to train the video inpainting model. We use Kinetics-400 for training and DAVIS 2017 480p for evaluation. If you already have matching Kinetics-400 videos, you can use our precomputed Kinetics masks.

from utils.grt import compute_grt_mask

# video: float tensor shaped [T, H, W, 3], normalized RGB frames
masks = []
for frame in video:
    # depth: float array shaped [H, W], estimated from one RGB frame
    depth = depth_model(frame)

    # disparity: torch.float32 tensor shaped [1, H, W], in pixel units
    disparity = normalize(depth).unsqueeze(0) * max_disparity

    # mask: torch.bool tensor shaped [1, H, W], True for disoccluded pixels
    mask = compute_grt_mask(disparity, direction="R2L")
    masks.append(mask)

# masks: torch.bool tensor shaped [T, H, W]

Use the mask generation script on your own video directory:

python scripts/compute_masks.py --input-dir data/videos --output-dir data/masks

The script preserves relative stems:

data/videos/train/example.mp4
data/masks/train/example.npz

Masks align by relative filename stem, so train/example.mp4 uses train/example.npz. The released DAVIS mask follows the same rule for its matching DAVIS video.

Training

After constructing videos and masks, train the video inpainting model with the released setup or your own data.

python scripts/train.py --video-dir data/videos --mask-dir data/masks --save-dir output/train/video_inpaint

Evaluation

DAVIS evaluation:

python scripts/eval.py \
  --checkpoint checkpoints/video_inpaint_large.pth \
  --output-dir output/eval/davis

Acknowledgements

We sincerely thank the authors of the following projects for their excellent work:

• Video Depth Anything and Depth Anything V2.
• Kinetics for providing a large-scale video recognition benchmark.

BibTeX

If you find our work helpful, please cite our paper:

@inproceedings{lu2026grt,
  author    = {Jingyi Lu and Kai Han},
  title     = {Geometric Reciprocity: Unlocking Self-Supervision for Stereoscopic Video Generation},
  booktitle = {International Conference on Machine Learning (ICML)},
  year      = {2026},
}