GL Texture Output

GL Texture Output

Video objects in Jitter may support GL texture output in addition to the typical matrix output. Textures are image data (or other types of data) stored and processed on your machine's Graphics Processing Unit (GPU). If a, jit.grab or jit.playlist object has texture output enabled (@output_texture 1), video frames are decoded and uploaded directly to an internal object. The outlet type is also changed, outputting the message jit_gl_texture followed by the internal name. Video objects will only output a texture at the native dimensions of their loaded file or device, therefore modifying the dim or adapt attributes will have no effect.

In most cases texture output will be more efficient than matrix output, especially when delivering high definition (HD) content. If an alpha channel is not required, efficiency may be improved further by setting the colormode attribute to uyvy, which requires half the data of the default argb mode without loss in quality. Colormode uyvy with output_texture 1 only affects how the image data is uploaded to the GPU and does not affect the colormode of the internal texture. Therefore, unlike matrix output where uyvy mode outputs a matrix at half the width of argb, uyvy texture output is the same size as argb.

Displaying and Processing Textures

To maintain efficiency when displaying and processing textures, it's important to utilize objects to keep the computations on the graphics card. This precludes the use of matrix processing Jitter objects.

The easiest way to display texture output from video objects is to send directly to a or object. For more control over placement, size, and color use the object. Image skewing and mapping can be managed with the object. To display the image using its native aspect ratio (irrespective of the output window dimensions) set the object's preserve_aspect attribute to 1.

For texture processing and are available. Both objects can take jit.matrix and inputs and will output a loads and runs Jitter shader files and uses Gen patchers. They are otherwise identical. Many Jitter matrix processing objects have corresponding Jitter shaders or Gen patchers. Some examples are jit.brcosa ( @gen brcosa), jit.xfade ( @gen xfade), and jit.alphablend ( @gen alphablend). Most jit.op operations have corresponding shaders, each starting with the prefix op (op.add.jxs, op.max.jxs), or are easily reproduced in a Gen patcher.

See this online article for a more comprehensive list of shader or gen patcher equivalents for common matrix processing objects.

Matrix processing on the left, texture processing on the right

Matrix Readback

Matrix readback, whereby a GL texture is read back into main memory for processing on the CPU, is a relatively costly operation. However certain operations or techniques can only be performed by the CPU using Jitter matrices, such as color tracking with jit.findbounds, or color analysis with jit.3m. In these cases a matrix readback may be the only solution.

Readbacks can be performed asynchronously or synchronously. Async readbacks are typically more performant and therefore the preferred route when possible, and are easily facilitated by the object. Synchronous readbacks are done by simply connecting the texture output to a jit.matrix object. The jit.matrix object will output a matrix matching the dimensions and planecount of the incoming texture. When processing HD images, often times it is less costly to downsample the texture on the GPU prior to matrix readback. This is done by simply inserting a prior to the jit.matrix, with @adapt 0 and dim set to the desired dimensions.

Matrix readback with downsampling

Further Reading

See Also

Name Description
Working with OpenGL Working with OpenGL