Jitter Matrix

A Matrix holds multidimensional, numeric data. Each matrix has one or more dimensions, in addition to one or more planes. You can think of a matrix as a grid, where each cell of the grid holds a list of numbers. The length of the list in each cell is the number of planes.

When a matrix holds an image or a frame of video data, it's equivalent to a two-dimensional array, where each cell represents one pixel. The dimension of the matrix would correspond to the resolution of the image or frame, and there would be one plane for each channel of color data (typically red, green and blue), possibly with one additional channel for alpha. Every cell of a matrix must have the same data type, with the possible types being char, long, float32, and float64.

Quick Reference

Matrix processing

Matrix processing will always happen on the CPU, which has advantages and disadvantages. In general, graphics processing can happen on the CPU (Central Processing Unit) or the GPU (Graphics Processing Unit). CPU processing is sequential and generally slower, but can support certain operations that GPU processing cannot. GPU processing is highly parallel and often much faster, but can be more restricted. For video and graphics programming, Max has objects and data structures supporting both kinds of processing. For GPU-based processing, use the jit.gl.* and jit.fx.* families of objects, which make use of textures instead of matrices.

Creating a Matrix

Create an empty matrix by creating a jit.matrix object. The object arguments determine its plane count, data type, and dimension size.

Named matrices

Like other named storage types, each matrix has a unique name. Objects that send matrices to each other send messages like jit_matrix u12345678, where u12345678 is the name of the matrix. Since you can refer to matrices by name, It's possible to define a matrix in one part of your patcher and use it in another.

Loading from a video or image

You can fill a matrix with the contents of a video or image file with the importmovie message. If you're loading a video file, you can supply a time offset to load a particular frame. The jit.matrix object can only load a single image frame this way. To work with videos, use jit.movie, jit.playlist, or jit.matrixset.

Patcher cords

Objects that send out a Jitter matrix will style their outgoing patch cord with black and green stripes. This is only cosmetic, as these just carry regular Max messages of the form jit_matrix u12345678. However, the special styling helps to distinguish matrix patch cords from other patch cords.

Coordinates

Jitter matrixes index from the top left, meaning the origin <0, 0> is in the top left corner. If the matrix is 101 cells wide, then <50, 50> is in the center, <100, 0> is the top right, and <0, 100> is the bottom left.

Frames, Colors, ARGB

Matrices can hold arbitrary data, but Jitter objects that operate on image frames will interpret each plane as corresponding to a particular color. Most Jitter objects work in ARGB format, expecting the first plane of a matrix to represent alpha, the second the red color value, the third green, and the final plane blue. If the type of the matrix is char or long, then the maximum value of a cell, when interpreted as a color, is 255. For float32 or float64 types, the maximum value for a particular color chanel is 1.0.

Data Types

Jitter matrices support four data types:

Getting and Setting Values

Fetch values from a matrix with the getcell message. For example, the message getcell 9 19 would retrieve the value in the 10th column (since we're indexing from zero) and the 20th row. The jit.matrix object will send the contents of the cell as a list out of its right outlet. The length of the list will be the plane count of the matrix.

For setting the values of a matrix, the most common ways are with the setcell, setall, and exprfill messages. The setcell message simply sets the value of one cell to a given value. The message setcell 1 2 val 255 255 0 255 would set the cell at coordinates <1, 2> to the value 255 255 0 255. The setall message fills the whole matrix with a particular value, so the message setall 0 255 0 255 would set every cell of the matrix to the value 0 255 0 255. Finally, the exprfill message lets you fill a matrix parametrically, according to a function that will be evaluated for each cell of the matrix. For more on the expression language used here, see Jitter expr.

Helpful objects for getting and setting values

A handful of jitter objects support setting and retrieving values from a matrix using lists. The jit.fill object lets you "fill" a matrix using a list. Note that jit.fill references a matrix by name, and doesn't output a matrix itself.

Going in the other direction, jit.spill can output a matrix as a list. The attributes @plane and @offset determine the plane and offset into the matrix. The @listlength attribute lets you set the length of the output list.

One more object that serves a similar function is the object jit.iter. This object will output each each cell of the input matrix one at a time, along with the coordinates of each cell.

Adapting and Interpolating

If a jit.matrix object receives a matrix as input that does not match the dimensions of the internal matrix, the object can adapt to or interpolate the incoming data. Interpolating means creating in-between values to fill in gaps where needed. If a 100 by 100 cell jit.matrix object receives a 10 by 10 cell input, it can interpolate to fill in the missing values. Interpolation is off by default. Use the @interp attribute to control interpolation.

By default, the arguments to a jit.matrix object determine the plane count, type, and dimensions of the internal matrix. If you enable the @adapt attribute, then jit.matrix will change its properties to match the dimensions of the incoming matrix, including plane count, dimensions, and type. An empty jit.matrix object with no arguments will have @adapt enabled by default.

Inspecting Matrices

Viewing as images

The jit.window and jit.pwindow objects will display whatever matrix they receive as an image. So, jit.pwindow is a great way to monitor a video stream in a running patcher. The jit.window object is often used as a final render destination, where a series of Jitter processes modify a matrix, and then a jit.window presents the final result.

Matrix descriptors

Several objects can get information about the dimension, plane count, type, or other descriptors of a particular matrix. The jit.fpsgui object can be configured to display any of these descriptors, in addition to the frame rate of a stream of matrices.

If you want to retrieve these same matrix descriptors as Max messages, for further downstream processing, use either the jit.matrixinfo object, or use the getattr object to fetch these values as attributes from a matrix. Both methods are equally efficient and idiomatic, the main difference is that jit.matrixinfo processes an incoming matrix, while getattr must attach to a jit.matrix object.

Matrix contents

For viewing the contents of a jit.matrix object as numerical data, use the jit.cellblock object. It can display a one or two dimensional matrix in a "spreadsheet" format. You can choose which plane of the matrix to view by sending the message plane $1, where $1 is the plane of the matrix that you want to display. You can also send the message plane -1 to view all planes of the matrix simultaneously, with each cell containing a list.

Finally, the jit.print object can print the entire contents of any matrix of any number of dimensions and planes.

Splitting and Recombining Matrices

It's possible to split and recombine matrices across planes or across dimensions. Use jit.unpack and jit.pack to recombine matrices by plane, and use jit.submatrix and jit.concat to recombine by dimension.

Splitting and remapping planes

By default, the jit.unpack object will split a 4-plane matrix into four single-plane matrices, but you can pass an argument to split into a different number of single planes.

If you want to split a multiple-plane matrix into other multiple-plane matrices, use the @offset and @jump attributes to configure jit.unpack. Arguments following @offset indicate, for each outlet, which plane in the input matrix to start reading from. Argments following @jump indicate, for each outlet, how many planes to read from the input.

The jit.pack object combines input matrices into a single multi-plane matrix. Like jit.unpack, it by default expects four input matrices, and the first argument determines the number of inlets.

Using @offset and @jump, you can combine several multi-plane matrices together. The @offset attribute controls the offset into each input matrix (at each inlet), and the @jump attribute determines how many planes to pull from each input matrix. The final plane count will the the sum of all the arguments to @jump.

The jit.matrix object can also remap planes directly, without using any other objects. The @planemap attribute controls the mapping between input and output planes.

Splitting and remapping dimensions

One way to split an input matrix is with the @srcdimstart, @srcdimend, @dstdimstart, and @dstdimend attributes. When a jit.matrix object receives an input matrix, it will look at these attributes to determine how the input is copied into the internal matrix. It's important to note that these attributes will have no effect unless the @usesrcdim and @usedstdim attributes are enabled.

The jit.submatrix object will let you reference just a portion of the input matrix. The @dim attribute determines the size of the portion in each dimension, corresponding to width and height for a two-dimensional matrix. The @offset attribute adjusts the offset of the portion, which you could also think of as the x- and y-coordinate of the submatrix.

The output of jit.submatrix will have smaller dimensions than the input, but it will never reduce the number of dimensions, even a redundant dimension of size 1. The jit.dimmap object can remove whole dimensions from an input matrix, or insert a redundant dimension. It can also remap and invert dimensions, making it useful for transposing and reversing a matrix.

To combine or recombine two matrices, use the jit.concat object. The @concatdim attribute will determine the dimension along which the two matrices will be concatenated. They do not need to have the same size, although there may be empty space in the resulting concatenated matrix, or some data might be lost if the edges of the two matrices don't line up.

More ways to split

There are still other matrix manipulation objects that may be helpful when splitting and recombining matrices:

Coercing

The jit.coerce object can be used to change the interpretation of the contents of a matrix, without copying or changing those contents. For example, you could treat a long matrix as if it were a float32 matrix, or you could treat a 4-plane char matrix as a 1-plane float32 matrix. This is similar to a cast operation in many programming languages.