Unity plugin for rendering on eye-tracked 3D light field displays via the DisplayXR OpenXR runtime. Works with any OpenXR-compatible 3D display.
- Overview
- Requirements
- Installing the Plugin
- Enabling the Feature
- Scene Setup
- Stereo Tunables Reference
- 2D UI Overlay
- Preview (Play Mode)
- Building Your App
- Deploying to End Users
- Testing Without Hardware
- Troubleshooting
- Architecture
New to the plugin? Start with the Quick Start Guide — a step-by-step walkthrough that covers installation, demo scenes for both stereo modes, building standalone apps, and end-to-end testing on Windows and macOS.
Want a ready-to-open Unity project? Clone displayxr-unity-samples and open
samples/birp-multipass/— a minimal Unity 6 project that depends on this plugin via UPM. Open it in Unity, hit Play, and you're rendering on a spatial display. (The repo also holds URP, HDRP, and Desktop Avatar samples.)
DisplayXR ships as a custom Unity display provider (IUnityXRDisplay, the same integration route as Oculus/Varjo/Cardboard) that drives the DisplayXR OpenXR runtime directly while Unity keeps rendering the scene. It provides:
- Eye-tracked stereo rendering — runtime-owned Kooima asymmetric frustum projection from real-time eye positions (
XR_DXR_view_rig) - Two stereo rig modes — Camera-centric (add to existing camera) or display-centric (place a virtual display in the scene)
- 2D/3D display zones + 2D UI overlay — frame 3D content to window-pixel zones and route any Canvas to a window-space composition layer with stereo disparity
- BiRP, URP, and HDRP — off-axis projection on all three pipelines (the provider hands Unity a full per-eye projection matrix, so no per-pipeline fix is needed), plus Single-Pass-Instanced (SPI) on URP+Windows+D3D12 and Multi-Pass elsewhere
- In-editor Play Mode preview — pressing Play runs the provider itself, giving full parity with built apps (#171). Play Mode is the preview — there is no separate edit-mode preview window.
How it works: the provider registers a DisplayXR Display display subsystem, binds an OpenXR session to Unity's graphics device, chains the runtime's XR_DXR_view_rig descriptor onto xrLocateViews, and submits Unity's rendered eye textures back to the runtime compositor via xrEndFrame. Enable it under XR Plug-in Management > Standalone > DisplayXR Display (see Enabling the Feature). See docs~/architecture/xr-display-provider.md for the full provider design.
| Requirement | Details |
|---|---|
| Unity | 2022.3 LTS or later (including Unity 6) |
| OpenXR Plugin | com.unity.xr.openxr 1.9.1+ (installed via Package Manager) |
| XR Plugin Management | com.unity.xr.management 4.4.0+ (auto-installed with OpenXR) |
| DisplayXR Runtime | DisplayXRSetup-*.exe (v1.4.0+) — required on every target machine. See Deploying to End Users. |
| Vendor display plug-in (Leia hardware only) | DisplayXRLeiaSRSetup-*.exe — required for Leia SR hardware. Not needed for sim_display testing or non-Leia displays. Install after the runtime. |
| Editor Platform | Build Target | Native Plugin | Status |
|---|---|---|---|
| Windows | Windows x64 | displayxr_unity.dll |
Supported |
| macOS | macOS | libdisplayxr_unity.dylib |
Supported |
| macOS | Windows x64 | displayxr_unity.dll |
Supported (cross-compile) |
Clone and build the native plugin, then add the package from disk:
git clone https://github.com/DisplayXR/displayxr-unity.git
cd displayxr-unity/native~
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release # on Windows, add: -A x64
cmake --build . --config ReleaseIn Unity: Window > Package Manager > + > Add package from disk... → select displayxr-unity/package.json.
Git must be in PATH. Unity's Package Manager requires Git installed and accessible to GUI apps.
- Windows: Install Git for Windows and ensure it's in your system PATH.
- macOS: If Git is installed via Homebrew, Unity launched from Hub/Finder may not find it (GUI apps don't inherit your shell PATH). Fix: run
xcode-select --installto ensure/usr/bin/gitworks, or launch Unity from a terminal. See Troubleshooting for details.
-
In Unity: Window > Package Manager
-
Click + > Add package from git URL...
-
Enter one of:
Pinned to a specific version (recommended for production):
https://github.com/DisplayXR/displayxr-unity.git#upm/v1.0.0Always latest release:
https://github.com/DisplayXR/displayxr-unity.git#upm
Both URLs install from the upm branch, which includes pre-built native binaries (Windows x64 + macOS Universal) — no local build required. See the releases page for available versions.
To update to a newer version later: change the URL fragment (e.g., #upm/v1.0.1) in Packages/manifest.json and use Package Manager → Refresh, or re-add the package with the new URL.
- Download the
.tgzfile from the latest release - In Unity: Window > Package Manager
- Click + > Add package from tarball...
- Select the downloaded
.tgzfile
After installation, the package appears as DisplayXR in the Package Manager.
DisplayXR is enabled as a display provider (the shipping path):
- Go to Edit > Project Settings > XR Plug-in Management
- Under the Standalone tab (Windows or macOS), check DisplayXR Display
- Make sure OpenXR is not also checked on the same tab (the provider drives the runtime itself; running both loaders conflicts)
- Check Initialize XR on Startup so the provider comes up in Play Mode and in built apps
You can verify the runtime status in the DisplayXR rig inspectors and the DisplayXR settings panel (display resolution, eye-tracking state).
This is the simplest setup. Your existing Main Camera stays in place and becomes the nominal viewing position. The plugin creates stereo eye offsets around it.
Step 1: Select your Main Camera in the hierarchy.
Step 2: Add Component > DisplayXRCamera (found under DisplayXR category).
That's it. The component:
- Reads eye tracking data from the runtime each frame
- Computes Kooima asymmetric frustums around the camera position
- Pushes modified FOVs into the OpenXR pipeline before Unity renders
Hierarchy:
Main Camera <-- has Camera + DisplayXRCamera
├── Scene objects...
└── (everything else in your scene)
Inspector tunables:
| Parameter | Default | Description |
|---|---|---|
| IPD Factor | 1.0 | Scales inter-eye distance. <1 = reduced stereo, >1 = exaggerated |
| Parallax Factor | 1.0 | Scales eye X/Y offset from viewing axis |
| Inv. Convergence Distance | 0 | 1/meters. 0 = infinity (parallel projection). Higher values = screen plane closer to camera. The inspector shows the equivalent distance in meters. |
Note: The component inherits the camera's vertical FOV automatically. Change it on the Camera component, not on DisplayXRCamera.
When to use camera-centric mode:
- First-person games and apps
- Retrofitting stereo into an existing project (just add the component)
- When the camera moves freely through the scene
The display is a fixed object in the scene. Eye positions are transformed relative to this virtual display.
Step 1: Create an empty GameObject where the virtual display should be:
- GameObject > Create Empty, name it "VirtualDisplay"
- Position and orient it in the scene (e.g., at
(0, 1.5, 2)facing the player)
Step 2: Add Component > DisplayXRDisplay
Step 3: Make your Main Camera a child of this object, or position it separately — the display's world transform is sent to the native plugin as the "scene transform" applied to raw eye positions.
Hierarchy:
VirtualDisplay <-- has DisplayXRDisplay
└── Main Camera <-- standard Camera component
Inspector tunables:
| Parameter | Default | Description |
|---|---|---|
| IPD Factor | 1.0 | Scales inter-eye distance |
| Parallax Factor | 1.0 | Scales eye X/Y offset from display center |
| Perspective Factor | 1.0 | Scales eye Z only (depth intensity without changing baseline) |
| Virtual Display Height | 0 | Virtual display height in meters. 0 = use physical display height. The inspector shows the computed virtual size. The parent transform's scale acts as zoom. |
When to use display-centric mode:
- Digital signage, museum exhibits, kiosks
- The virtual display is a physical object in the scene (e.g., a TV on a wall)
- You want the display's position and orientation to be an explicit scene element
Try it: Import the Display Scene sample from Package Manager for a ready-made tabletop turntable demo.
Both modes share a common tunable interface that maps to the native plugin's Kooima computation. Here's what each parameter does physically:
Scales the horizontal distance between left and right eye positions.
1.0= natural inter-pupillary distance from eye tracker0.5= half the real IPD (gentler stereo, less eye strain)2.0= double the real IPD (exaggerated depth, "macro" effect)0.0= mono rendering (both eyes at center)
Scales the eye's X and Y offset from the viewing axis (or display center).
1.0= natural head parallax0.0= no parallax (stereo still works via IPD, but no motion parallax)>1.0= exaggerated motion parallax
Scales the eye's Z position (distance from display) without changing the baseline.
1.0= natural depth<1.0= eyes appear closer to display (stronger perspective)>1.0= eyes appear farther (flatter perspective)
Sets the virtual display height in meters for the Kooima projection.
0= use the physical display's actual height (default)0.3= 30 cm virtual display (scene objects appear larger — magnifier effect)0.6= 60 cm virtual display (scene objects appear smaller)
The parent transform's scale acts as zoom: scaling up the DisplayXRDisplay object zooms into the scene.
Inverse convergence distance in 1/meters. Controls where the virtual screen plane sits relative to the camera. Objects at the convergence distance appear at the display surface; closer objects pop out, farther objects recede.
0= infinity (parallel projection — no convergence)2.0= screen plane at 0.5 m from camera1.0= screen plane at 1.0 m from camera- The inspector shows the equivalent distance in meters or "∞"
Note: Camera-centric FOV is inherited automatically from the Camera component. There is no separate FOV tunable.
Route a Canvas to a window-space composition layer that the DisplayXR compositor overlays on both eyes with per-eye disparity shift (renders pre-interlace).
- Create a Canvas (any render mode)
- Add Component > DisplayXRWindowSpaceUI to the Canvas
- Configure position and size in fractional window coordinates [0..1]:
| Parameter | Default | Description |
|---|---|---|
| Position X | 0.0 | Left edge of overlay in window (0 = left, 1 = right) |
| Position Y | 0.0 | Bottom edge (0 = bottom, 1 = top) |
| Width | 1.0 | Fractional width |
| Height | 1.0 | Fractional height |
| Disparity | 0.0 | Stereo disparity in pixels. 0 = at screen plane, positive = in front |
| Resolution | 512 | Render texture resolution (square) |
The overlay is submitted as XrCompositionLayerWindowSpaceDXR and composited by DisplayXR before display processing. This means 2D UI text stays sharp and is not interlaced — ideal for HUDs, menus, and status displays.
Press Play — Play Mode is the preview. With the DisplayXR Display provider active (the shipping path), Play Mode runs the provider itself: the same backend as a built app, weaving to the display in a dedicated editor window (#171). This gives true parity with built apps — provider behavior and bugs surface directly in the editor — with no separate preview session or Play/Stop-free workflow to manage.
Rendering modes are enumerated dynamically from the runtime and depend on the connected display hardware; mode switching is app policy (see DisplayXRModeSwitch). The runtime status (resolution, physical dimensions, eye-tracking state, current mode) is visible in the DisplayXR rig inspectors and settings panel.
- File > Build Settings
- Select Windows, Mac, Linux platform
- Set Target Platform to Windows and Architecture to x86_64
- Verify in Player Settings > XR Plug-in Management > Standalone that DisplayXR Display is enabled
- Click Build or Build And Run
The build output includes displayxr_unity.dll in the Plugins/ folder alongside your executable.
- File > Build Settings
- Select macOS platform
- Verify DisplayXR Display is enabled in Standalone XR settings
- Click Build
The .app bundle includes libdisplayxr_unity.dylib in the plugins folder.
Yes, you can build a Windows app from Unity for Mac. Unity's cross-compilation support handles this:
- Install the Windows Build Support module in Unity Hub:
- Unity Hub > Installs > your Unity version > Add Modules > Windows Build Support (Mono)
- In Unity: File > Build Settings > Windows, Mac, Linux
- Set Target Platform to Windows
- Build as normal
The plugin includes both platform binaries (displayxr_unity.dll for Windows, libdisplayxr_unity.dylib for macOS). Unity automatically selects the correct one based on the build target.
Important: The built Windows .exe still requires the DisplayXR runtime installed on the target Windows machine — see Deploying to End Users. You cannot run the Windows build on macOS.
Your built app is a standard OpenXR application. It needs an OpenXR runtime on the target machine. The plugin is hardware-agnostic — it communicates only through the OpenXR API and does not depend on any specific display vendor SDK.
Two installers, in order:
1. DisplayXR Runtime (always required):
Install DisplayXRSetup-*.exe from the displayxr-runtime releases. Registers the OpenXR runtime JSON, drops the runtime DLLs at C:\Program Files\DisplayXR\Runtime\, sets HKLM\Software\Khronos\OpenXR\1\ActiveRuntime to the DisplayXR manifest, and auto-starts displayxr-service.exe at user logon.
The runtime ships with the sim_display plug-in (vendor-neutral SBS/anaglyph/blend fallback for any GPU). For real 3D hardware you also need the matching vendor plug-in below.
2. Leia SR plug-in (required for Leia SR hardware only):
Install DisplayXRLeiaSRSetup-*.exe from the displayxr-leia-plugin releases. Drops DisplayXR-LeiaSR.dll at C:\Program Files\DisplayXR\Plugins\LeiaSR\ and registers it at HKLM\Software\DisplayXR\DisplayProcessors\leia-sr (ProbeOrder=50). The runtime's registry-driven discovery picks it up at xrCreateInstance time; on Leia hardware it wins the probe and renders through the SR weaver, on non-Leia hardware it declines and the runtime falls back to sim_display.
Install order matters — the Leia plug-in installer hard-prereqs HKLM\Software\DisplayXR\Runtime\InstallPath and refuses install if the runtime isn't already present.
The plug-in cascade-uninstalls with the runtime: uninstalling the runtime via Add/Remove Programs automatically uninstalls all registered vendor plug-ins (#286 fixed cleanup in runtime v1.4.1+).
Why two installers? Pre-v1.4.0 the runtime installer bundled the Leia plug-in. As of v1.4.0 (
ADR-019/ #263), vendor display drivers ship as separate plug-in installers from their own repos so the runtime stays vendor-source-clean and additional vendors don't bloat the runtime install set.
Or, for development/testing without installation, set the environment variable to your dev runtime JSON:
set XR_RUNTIME_JSON=C:\path\to\openxr_displayxr-dev.jsonSet before launching the app:
export XR_RUNTIME_JSON=/path/to/DisplayXR-macOS/share/openxr/1/openxr_displayxr.jsonIf you have a runtime symlink at ~/Library/Application Support/openxr/1/active_runtime.json (the OpenXR loader's standard discovery path), it works for the Unity editor but not for unsigned built .app bundles. macOS' file-access protections block unsigned apps from reading other apps' Application Support directories, so the loader can't find the runtime manifest. Symptom:
Loading OpenXR loader library at path: openxr_loader
[XR] [FAILURE] xrCreateInstance: XR_ERROR_RUNTIME_UNAVAILABLE
[DisplayXR] provider not active.
Two ways around it:
1. Set XR_RUNTIME_JSON explicitly when launching (works with any build, signed or not):
XR_RUNTIME_JSON=/path/to/openxr_displayxr.json /path/to/MyApp.app/Contents/MacOS/MyApp2. Code-sign the .app so macOS grants it normal user-file access. Ad-hoc signing is enough for local development — no Apple Developer account needed:
codesign --deep --force --sign - MyApp.appAfter signing, the runtime symlink is reachable and you can launch the app without XR_RUNTIME_JSON.
For distributing to other users, sign with a Developer ID and notarize:
codesign --deep --force --sign "Developer ID Application: Your Name (TEAMID)" \
--options runtime --timestamp MyApp.app
xcrun notarytool submit MyApp.app --apple-id you@example.com --team-id TEAMID --wait
xcrun stapler staple MyApp.appIf the DisplayXR runtime is not installed, the provider fails to bind an OpenXR session and logs:
[OpenXR] No OpenXR runtime found
The plugin logs a warning but doesn't crash — your app runs in mono (non-stereo) mode. Query the DisplayXR runtime status (via the rig components / DisplayXRProvider) to detect this and show a user-facing message.
The sim_display driver provides a software 3D display for development:
# Windows
set SIM_DISPLAY_ENABLE=1
set SIM_DISPLAY_OUTPUT=sbs # side-by-side stereo
# or: anaglyph (red-cyan), blend (50/50 alpha)
# macOS
export SIM_DISPLAY_ENABLE=1
export SIM_DISPLAY_OUTPUT=sbsWith sim_display:
- Eye tracking is simulated via keyboard/mouse (qwerty driver)
- Display dimensions are synthetic (configurable)
- Output renders as SBS, anaglyph, or alpha-blend in a regular window
This lets you develop and test the full stereo pipeline on any machine.
| Symptom | Cause | Fix |
|---|---|---|
| "No OpenXR runtime found" | XR_RUNTIME_JSON not set or points to missing file |
Set the env var to the DisplayXR runtime JSON path |
| Black screen | DisplayXR not enabled | Check Project Settings > XR Plug-in Management > Standalone > DisplayXR Display, and Initialize XR on Startup. |
| Scene renders 2D side-by-side on Leia hardware (no depth) | Leia plug-in not installed/registered | Install DisplayXRLeiaSRSetup-*.exe from displayxr-leia-plugin. Verify HKLM\Software\DisplayXR\DisplayProcessors\leia-sr exists. Without the plug-in, the runtime falls back to sim_display (SBS output) on any hardware. |
| No stereo (flat image) | Eye tracking not running | Verify the DisplayXR runtime is configured with a display that supports eye tracking, or use sim_display for testing |
| Stereo looks wrong | Tunables misconfigured | Reset to defaults (IPD=1, Parallax=1, Scale=1) |
| UPM "cannot add package from git URL" (Windows) | Git not installed or not in PATH | Install Git for Windows, restart Unity. Verify with git --version in a terminal. |
| UPM "cannot add package from git URL" (macOS) | GUI apps can't find Homebrew Git | Run xcode-select --install to set up /usr/bin/git, or launch Unity from terminal (open -a Unity). Check ~/Library/Logs/Unity/upm.log for details. |
DllNotFoundException: displayxr_unity |
Native plugin not found by Unity | Ensure the plugin binaries are in Runtime/Plugins/Windows/x64/ or Runtime/Plugins/macOS/ |
macOS: XR_ERROR_RUNTIME_UNAVAILABLE in built .app despite the symlink working in editor |
Unsigned .app bundles can't read ~/Library/Application Support/openxr/ due to macOS sandbox protections |
Either set XR_RUNTIME_JSON when launching, or ad-hoc sign with codesign --deep --force --sign - MyApp.app. See macOS Deployment. |
| HDRP stereo artifacts | Single-pass instanced issue | Verify both eye views have correct FOVs in Frame Debugger |
| Play Mode shows "Not Connected" | XR_RUNTIME_JSON not set or runtime not running |
Set the env var before launching Unity; verify the runtime process is active |
| Play Mode shows black | Runtime connected but no output | Check runtime logs; verify sim_display or hardware is configured |
VK_ERROR_EXTENSION_NOT_PRESENT on macOS |
MoltenVK limitation | Known issue — use sim_display for testing |
Enable Log Eye Tracking on the DisplayXRCamera or DisplayXRDisplay component to see per-frame eye positions in the Console:
[DisplayXR] Eyes: L=(0.032, 0.001, 0.504), R=(-0.031, 0.001, 0.504), tracked=True
With the provider active, the DisplayXRCamera / DisplayXRDisplay inspectors and the DisplayXR settings panel show runtime status:
- Connected display properties (resolution, physical size, nominal viewer distance)
- Eye tracking status
DisplayXR ships as a custom IUnityXRDisplay display provider (the shipping path). The provider
registers a DisplayXR Display subsystem, owns an OpenXR session on Unity's graphics device, and hands
Unity's rendered eye textures to the runtime compositor — Unity keeps rendering the scene (BiRP/URP/HDRP,
SPI/Multi-Pass), the runtime owns the Kooima math (XR_DXR_view_rig).
Unity Editor / Player
┌──────────────────────────────────────────────────────────┐
│ C# Layer │
│ │
│ DisplayXRDisplayLoader : XRLoaderHelper │
│ starts the "DisplayXR Display" display subsystem │
│ DisplayXRProvider / DisplayXRProviderDriver │
│ push rig tunables, modes/zones, events (dxr_prov_*) │
│ │
│ DisplayXRCamera DisplayXRDisplay │
│ (camera-centric) (display-centric) │
│ │
│ DisplayXRWindowSpaceUI / DisplayXRLocal2D (2D layers) │
└──────────────────────────────────────────────────────────┘
│ P/Invoke (displayxr_unity.dll / .dylib)
▼
┌──────────────────────────────────────────────────────────┐
│ Native Plugin (C/C++) │
│ Display provider (IUnityXRDisplay): │
│ session on Unity's device → render params (SPI/MP) │
│ xrLocateViews → chain XR_DXR_view_rig descriptor │
│ xrEndFrame → submit projection + zone + 2D layers │
└──────────────────────────────────────────────────────────┘
│ Standard OpenXR API
▼
┌──────────────────────────────────────────────────────────┐
│ DisplayXR Runtime │
│ Compositor → Display Processor → Output │
└──────────────────────────────────────────────────────────┘
See docs~/architecture/xr-display-provider.md for the full
provider design, and docs~/adr/ADR-007-render-path-by-view-count.md
for the roadmap to many-view (light-field / quilt) displays.
Eye Tracker → raw positions in DISPLAY space
↓
Scene Transform (parent camera pose, zoom)
↓
Tunables (IPD factor, parallax, perspective, scale)
↓
Kooima Asymmetric Frustum → XrFovf angles
↓
Unity builds projection matrices → renders stereo
| Path | Purpose |
|---|---|
Runtime/Provider/ |
Display provider (shipping path): DisplayXRDisplayLoader (XR Plug-in Management loader), DisplayXRProvider/DisplayXRProviderDriver (facade + per-frame driver), DisplayXRProviderNative (P/Invoke) |
Editor/Provider/DisplayXRDisplayPackage.cs |
Registers the "DisplayXR Display" provider toggle in XR Plug-in Management |
Runtime/DisplayXRCamera.cs |
Camera-centric stereo rig MonoBehaviour |
Runtime/DisplayXRDisplay.cs |
Display-centric stereo rig MonoBehaviour |
Runtime/DisplayXRDisplayInfo.cs |
Display properties data struct |
Runtime/DisplayXRTunables.cs |
Tunable parameters struct |
Runtime/DisplayXRWindowSpaceUI.cs |
2D UI overlay routing |
Runtime/DisplayXRNative.cs |
P/Invoke bindings to native plugin |
Runtime/Plugins/Windows/x64/ |
Windows native plugin (DLL) |
Runtime/Plugins/macOS/ |
macOS native plugin (dylib) |
Editor/DisplayXRDisplayEditor.cs |
Custom inspector for display-centric mode |
Editor/DisplayXRCameraEditor.cs |
Custom inspector for camera-centric mode |
Editor/DisplayXRSettingsProvider.cs |
Project Settings page |
native~/ |
Native C/C++ plugin source + CMakeLists.txt |