Skip to main content
Version: Reality 5.4

Tracked Billboard

In this section, you'll discover how to use the Tracked Billboard pipeline with Reality 5.4. For our demonstration, we will employ TRAXIS Talent Tracking as a provider.

What Is Tracked Billboard?​

Tracked Billboard enables you to seamlessly integrate your keyed talent into the Reality 5.4 scene, achieving a photorealistic appearance. Furthermore, it can be utilized in a variety of creative scenarios, including Teleportation and Fly Cam.

Process​

Reality Hub Login Screen

  • Login to Reality Hub.
  • Activate the Launcher module and launch your project.

Dragging & Dropping Engine into Nodegraph Canvas

  • Go to the Engines section, drag and drop the UE5 process into Nodegraph canvas.

Node Details Panel

As soon as you click on abovementioned function buttons, following changes occurs as illustrated above:

  1. Reality Actors folder has been created.
  2. New Properties has been added.

AJAIn and AJAOut Nodes

  • Create AJAIn and AJAOut nodes, then designate your reference by right-clicking on each node. In our example, we have selected SingleLink 1 for AJAIn and SingleLink 4 for AJAOut.
  • Establish your corresponding camera tracking node. In our example, we utilized Xync.
  • Connect the SingleLink 1 output pin of the AJAIn node to the Sync input pin to the Xync node.

Breaking Track​

Breaking Track is a procedure for extracting specific output from a Track node. For instance, you might prefer to use only the Location data from your talent tracking software/hardware or on a focus property of Lens Distortion data for your pipeline. This approach also allows you to conduct advanced compositing operations.

Now:

  • Create a Break Track node.
  • Connect the Track output of the Xync node to the Input input pin of the Break Track node.

Cyclorama Connections​

Now that we have AJA nodes for video I/O, talent tracking, camera tracking, and Reality 5.2 nodes, as illustrated in the previous image, it's time to incorporate Cyclorama.

  • Add a Cyclorama node to the canvas.

Distorting and Undistorting​

Most camera lenses exhibit some level of optical distortion, and as a result, you often need to perform undistortion and then distortion operations on your video source to address this issue.

For instance, when dealing with the image from your studio and the 3D Cyclorama mesh you've created inside Reality 5.2, they may not align perfectly in terms of lens distortion. To ensure proper alignment between the Cyclorama and camera image, it's essential to employ the undistortion process.

  • Create two Distort nodes and one Undistort node.

Show as Input

  • Go to the Properties section of each node you created.
  • Right-click the Distortion property for each node and select "Show as Input".

Cyclorama Connections

  • Connect the Lens Distortion output of the Break Track node to the Distortion input pin of the Distort and Undistort nodes.
  • Connect the SingleLink 1 output pin of the AJAIn node to the In input pin to the Undistort node.
  • Connect the AJAIn node's SingleLink 1 output to Track input of the Cyclorama node.
  • Connect the Undistort node's Out output pin to Cyclorama node's Video input.
  • Connect the Cyclorama node's Render output to first Distort node's In input pin.
  • Connect the Cyclorama node's Mask output pin to the second Distort node's In input pin.

Adding RealityKeyer​

Tracked Billboard pipeline requires keyed image of your talent; therefore, you have to use Reality Keyer.

RealityKeyer Connections

  • Create a RealityKeyer node.
  • Connect the Out output pin of the first Distort node to the Clean Plate input of the RealityKeyer node.
  • Connect the Out output pin of the second Distort node to the Clean Plate Mask the input of the RealityKeyer node. Connect the SingleLink 1 output pin to the Input input pin of the RealityKeyer node.
  • Create a second Undistort node, right-click on its Distortion property and select "Show as Input".
  • Connect the Output output pin of the RealityKeyer node to the In input of the second Undistort node.

Talent Location​

In order to place your talent inside the Reality 5.4 scene, you have to provide talent location data via a tracking system. In our example we utilized TRAXIS Talent Tracker.

Talent Data Connections

  • Create a corresponding talent tracking data node. In our example we use FreeD node.
  • Add the second Break Track node.
  • Create a Cast vec3 to vec3d node.
  • Connect the Location output pin of the second Break Track node to vec3 input pin of the Cast vec3 to vec3d node.

Engine Connections​

Engine Connections

  • Connect the vec3d output pin of the Cast vec3 to vec3d node to UE5 node's Reality Tracked Billboard Talent Location input pin.
  • Connect the second Undistort node's Out output pin to Reality Track Billboard Video input of the UE5 node.
  • Connect the Xync node's Track output pin to the Reality Camera Track input pin of the UE5 node.
  • Create a Composite Passes node.
  • Connect UE5 node's Reality Camera Scene output pin to Composite Passes node's Render input.
  • Connect the Xync node's Track output to Track input of the Composite Passes node.
  • Connect the Output output pin of the RealityKeyer to the Video input of the Composite Passes node.
  • Connect the Output pin of the Composite Passes node to SingleLink 4 input pin of the AJAOut node.