Tutorial 39: Spatial Mapping
In this tutorial, we will look at ways to remap the spatial layout of cells in a Jitter matrix using a second matrix as a spatial map. Along the way, we’ll investigate ways to generate complete maps through the interpolation of small datasets as well as ways to use mathematical expressions to fill a Jitter matrix with values.
Jitter matrices can be remapped spatially using lookup tables of arbitrary complexity. In Tutorial 12, we learned that the jit.charmap object maps the cell values (or color information) of one Jitter matrix based on a lookup table provided by a second Jitter matrix. In a similar fashion, the jit.repos object takes a Jitter matrix containing spatial values that it then uses as a lookup table to rearrange the spatial positions of cells in another matrix.
The tutorial patch shows the matrix output of a jit.movie object being processed by a new object, called jit.repos. The right inlet of jit.repos receives matrices from the output of a jit.xfade object fed by two named matrices, each of which has been given a name ( and ) so that they share data with jit.matrix objects elsewhere in the patch. To the right of the main processing chain are two separate areas where we construct those two matrices, which will serve as our spatial maps for this tutorial.
Notice that at this point, we only see a solid white frame appear in the jit.pwindow object at the bottom-left of the patch. This is because we haven’t loaded a spatial map into our jit.repos object.
We now see a normal looking image appear in the jit.pwindow at the bottom of our patch.
The Wide World of Repos
The pictslider objects in our patch set individual cells in a 2-plane, 3x3 Jitter matrix of type . This matrix is then upsampled with interpolation to a 320x240 matrix named . This matrix contains the instructions by which jit.repos maps the cells in our incoming matrix. The interpolation here is necessary so that we can infer an entire 320x240 map from just nine specified coordinate points.
The spatial map provided to the jit.repos object in its right inlet contains a cell-by-cell description of which input cells are mapped to which output cells. The first plane (plane 0) in this matrix defines the x coordinates for the mapping; the second plane (plane 1) represents the y coordinates. For example, if the cell in the upper-left hand corner contains the values , then the cell at coordinates from the matrix coming into the left inlet of the jit.repos object will be placed in the upper left-hand corner of the output matrix.
The following example will give us a better sense of what jit.repos is doing with its instructions:
In this example, we have a simple single-plane 5x5 matrix filled with ascending numbers being remapped by a spatial map. Notice the correlation between the values in the spatial map and where the corresponding cells in the output matrix get their original values from in the input matrix. For example, the cell in the lower-left of the spatial map matrix reads
. The cell at coordinate {4, 0} in the incoming matrix is set to value . As you can see, the cell in the lower-left of the output matrix contains that value as well.Similarly, you can see that the cells comprising the middle row of the output matrix are all set to the same value (
). This is because the corresponding cells in the spatial map are all set to the same value ( ). Cell {3, 3} in the input matrix contains the value ; that value is then placed into that entire row in the output matrix.In our tutorial patch, our spatial map needs to be able to contain values in a range that represents the full size (or pictslider ranges are set to 320 x 240. To handle numbers of this size, we need to use a matrix of type for our spatial map. If it was of type , we would be constrained to values in the range of 0 to 255, limiting what cells we can choose from in our spatial map. Note that in order to correctly view these matrices in jit.pwindow objects we use the jit.clip object to constrain the values in the matrix before viewing to the same range as matrices.
) of the matrix to be processed. To this end, theSpatial Expressions
Now that we’ve experimented with designing spatial maps by hand, as it were, we can look at some interesting ways to generate them algorithmically.
The jit.gencoord object by the jit.expr object. The results of this expression, performed on Jitter matrices, are then copied into a matrix that serves as our spatial map. This spatial map causes jit.repos to spatially remap our incoming matrix from Cartesian to polar space, so that the leftmost edge of our incoming matrix becomes the center of the outgoing matrix, wrapping in a spiral pattern out to the final ring, which represents the rightmost edge of the input matrix. The attribute of jit.repos tells the object to wrap coordinates beyond that range back inwards, so that the final edge of our output image begins to fold in again towards the left side of the input image.
spatial map has been set by a mathematical expression operated on the output of theThe jit.gencoord object outputs a simple Cartesian map of floating-point values; when you multiply the values in the matrix by the of the matrix, you can easily get coordinates which literally correspond to the locations of the cells, as shown in this illustration:
The jit.gencoord allow us to manipulate the range of values put out by the object, saving us the step of using multiple jit.op objects to get our Cartesian map into the range we want. For our equation to work, we want our starting map to be in the range of {-1, 1} across both dimensions (hence we multiply the normal Cartesian space by 2 and offset it by -1).
and attributes ofThe jit.expr object takes the incoming matrix provided by jit.gencoord and performs a mathematical expression on it, much as the Max vexpr object works on lists of numbers. The attribute of the jit.expr object defines the expression to use. The notation tells jit.expr that we want to operate on the values contained in plane j of the matrix arriving at inlet i. The keyword returns the size of the dimension i in the matrix. In our case, the notation would resolve to 320; would resolve to 240.
Note that there are actually two mathematical expressions defined in the message box setting the attribute. The first defines the expression to use for plane 0 (the x coordinates of our spatial map), the second sets the expression for plane 1 (the y coordinates).
Placed in a more legible notation, the jit.expr expression for plane 0:
works out to something like this, if x is the value in plane 0 of the input, y is the value in plane 1 of the input, and a and b are the values in the number box objects attached to the second and third inlets of the jit.expr object:
sqrt(ax2+by2)*320
Similarly, the expression for plane 1:
works out to something like this:
(((atan(by/ax)/)/2)+0.5)*240
Therefore, we can see that we are filling the x (plane 0) values of our spatial map with numbers corresponding to the length of the hypotenuse of our coordinates (i.e. their distance from the center of the map), while we set the y (plane 1) values to represent those values’ angle (Ø) around the origin.
The jit.expr object can take Max numerical values (
or ) in its inlets in lieu of matrices. Thus the and notation in the attribute refers to the numbers arriving at the middle and right inlets of the object.An important thing to notice with this subpatch is that the message box objects are directly connected to the jit.matrix object in the main patch. The message to jit.matrix allows us to fill a Jitter matrix with values from a mathematical expression without having to resort to using jit.expr as a separate object. The syntax for the method is slightly different from the attribute to jit.expr: the first argument to is the to fill with values, so two messages (separated by commas) are necessary to generate the spatial map) using this method.
Still Just Matrices
The number box controls the jit.xfade object directly attached to the jit.repos object at the heart of our processing chain. As a result, we can smoothly transition from our manually defined spatial map (the matrix) to our algorithmically defined one (the matrix). The ability to use Jitter processing objects to composite different spatial maps adds another layer of potential flexibility. Because the two matrices containing our maps have the same typology ( , , and ) they can be easily combined to make an infinite variety of composites.
attribute of theSummary
The jit.repos object processes an input matrix based on a spatial map encoded as a matrix sent to its second inlet. The default behavior is for jit.repos to process matrices spatially using absolute coordinates, such that the cell values in the second matrix correspond to coordinates in the input matrix whose cell values are copied into the corresponding position in the output matrix.
Spatial maps can be created by setting cells explicitly in a matrix and, if desired, upsampling with interpolation from a smaller initial matrix. Maps can also be created by generating matrices through algorithmic means, such as processing a “normal” Cartesian map generated by a jit.gencoord object through a mathematical expression evaluated by a jit.expr object. The jit.expr object takes strings of mathematical expressions, one per plane, and parses them using keywords to represent incoming cell values as well as other useful information, such as the size of the incoming matrix. If you want to fill a Jitter matrix with the results of a mathematical expression directly, you can use the message to a jit.matrix object to accomplish the same results without having to use a separate jit.expr object.
The jit.repos object can be used with a wide variety of spatial maps for different uses in image and data processing. The ‘jitter-examples/video/spatial’ folder of the Max ‘examples’ folder contains a number of patches that illustrate the possibilities of jit.repos.
See Also
Name | Description |
---|---|
Working with Video in Jitter | Working with Video in Jitter |
Working with OpenGL | Working with OpenGL |
jit.cellblock | Edit rows and columns of data |
jit.charmap | Map 256-point input to output |
jit.clip | Limit data to a range |
jit.expr | Evaluate an expression to fill a matrix |
jit.gencoord | Evaluate a procedural basis function graph |
jit.matrix | The Jitter Matrix! |
jit.pwindow | Display Jitter data and images |
jit.movie | Play a movie |
jit.repos | Reposition spatially |
jit.xfade | Crossfade between 2 matrices |