GL Texture Output
Video objects in Jitter may support GL texture output in addition to the typical matrix output. Textures are image data (or other types of data) stored and processed on your machine's Graphics Processing Unit (GPU). If a jit.movie, jit.grab or jit.playlist object has texture output enabled (@output_texture 1), video frames are decoded and uploaded directly to an internal jit.gl.texture object. The outlet type is also changed, outputting the message jit_gl_texture followed by the internal jit.gl.texture name. Video objects will only output a texture at the native dimensions of their loaded file or device, therefore modifying the dim or adapt attributes will have no effect.
In most cases texture output will be more efficient than matrix output, especially when delivering high definition (HD) content. If an alpha channel is not required, efficiency may be improved further by setting the colormode attribute to uyvy, which requires half the data of the default argb mode without loss in quality. Colormode uyvy with output_texture 1 only affects how the image data is uploaded to the GPU and does not affect the colormode of the internal texture. Therefore, unlike matrix output where uyvy mode outputs a matrix at half the width of argb, uyvy texture output is the same size as argb.
Displaying and Processing Textures
To maintain efficiency when displaying and processing textures, it's important to utilize jit.gl objects to keep the computations on the graphics card. This precludes the use of matrix processing Jitter objects.
The easiest way to display texture output from video objects is to send directly to a jit.world or jit.gl.layer object. For more control over placement, size, and color use the jit.gl.videoplane object. Image skewing and mapping can be managed with the jit.gl.cornerpin object. To display the image using its native aspect ratio (irrespective of the output window dimensions) set the jit.gl.cornerpin object's preserve_aspect attribute to 1.
For texture processing jit.gl.slab and jit.gl.pix are available. Both objects can take jit.matrix and jit.gl.texture inputs and will output a jit.gl.texture. jit.gl.slab loads and runs Jitter shader files and jit.gl.pix uses Gen patchers. They are otherwise identical. Many Jitter matrix processing objects have corresponding Jitter shaders or Gen patchers. Some examples are jit.brcosa (jit.gl.pix @gen brcosa), jit.xfade (jit.gl.pix @gen xfade), and jit.alphablend (jit.gl.pix @gen alphablend). Most jit.op operations have corresponding shaders, each starting with the prefix op (op.add.jxs, op.max.jxs), or are easily reproduced in a jit.gl.pix Gen patcher.
See this online article for a more comprehensive list of shader or gen patcher equivalents for common matrix processing objects.
Matrix Readback
Matrix readback, whereby a GL texture is read back into main memory for processing on the CPU, is a relatively costly operation. However certain operations or techniques can only be performed by the CPU using Jitter matrices, such as color tracking with jit.findbounds, or color analysis with jit.3m. In these cases a matrix readback may be the only solution.
Readbacks can be performed asynchronously or synchronously. Async readbacks are typically more performant and therefore the preferred route when possible, and are easily facilitated by the jit.gl.asyncread object. Synchronous readbacks are done by simply connecting the texture output to a jit.matrix object. The jit.matrix object will output a matrix matching the dimensions and planecount of the incoming texture. When processing HD images, often times it is less costly to downsample the texture on the GPU prior to matrix readback. This is done by simply inserting a jit.gl.texture prior to the jit.matrix, with @adapt 0 and dim set to the desired dimensions.
Further Reading
See Also
| Name | Description |
|---|---|
| Working with OpenGL | Working with OpenGL |