Refracting Reality: Generating Images
with Realistic Transparent Objects

Generative image models can produce convincingly real images, with plausible shapes, textures, layouts and lighting. However, one domain in which they perform notably poorly is in the synthesis of transparent objects, which exhibit refraction, reflection, absorption and scattering. Refraction is a particular challenge, because refracted pixel rays often intersect with surfaces observed in other parts of the image, providing a constraint on the color. It is clear from inspection that generative models have not distilled the laws of optics sufficiently well to accurately render refractive objects. In this work, we consider the problem of generating images with accurate refraction, given a text prompt. We synchronize the pixels within the object's boundary with those outside by warping and merging the pixels using Snell's Law of Refraction, at each step of the generation trajectory. For those surfaces that are not directly observed in the image, but are visible via refraction or reflection, we recover their appearance by synchronizing the image with a second generated image — a panorama centered at the object — using the same warping and merging procedure. We demonstrate that our approach generates much more optically-plausible images that respect the physical constraints.

Our training-free image generation method, which we name Snellcaster since it casts rays that are curved according to Snell's Law, is visualized in the figure below. First, an image I⁻₀ is generated given a text prompt p⁻, which has any reference to the transparent object removed, and its depth map D⁻ is estimated using MoGe-2. We then generate two images concurrently: a perspective image guided by prompt p, which includes the transparent object, and a panoramic image guided by augmented prompt p₃₆₀, centered at the transparent object's location. At each denoising timestep t, we compute Euler estimates I_0|t and I³⁶⁰_0|t for both views, warp them using precomputed geometric correspondences from ray tracing, and blend them together to enforce physical consistency. Finally, we combine refracted and reflected contributions using the Fresnel equations to produce the final image, which is encoded back into latent space for the next denoising step.