Tutorial 6: Adjust Color Levels

This tutorial references the patcher 06jAdjustColorLevels.maxpat

jit.scalebias

This tutorial elaborates on the color discussion of the previous chapter, and introduces an object that is specially designed for modifying the ARGB color planes of a matrix: jit.scalebias.

The term scale in this instance refers to the process of scaling values by a certain factor; that is, multiplying them by a given amount. The term bias refers to offsetting values by adding an amount to them. By combining the processes of multiplication and addition, you can achieve a linear mapping of input values to output values.

Because jit.scalebias is concerned with modifying ARGB color information in an image, it handles only 4-plane matrices of the char data type. (See Tutorial 5 for a discussion of ARGB color and char data.)

Math with char data

As mentioned in the previous chapter, 8-bit char data can be represented as whole number values from 0 to 255 or as fractional values from 0 to 1. In Tutorial 2, for example, we saw that the jit.print object reports char data as integer values from 0 to 255. However, in many cases where we modify char values within a matrix (by changing one of its attributes), the Jitter object will expect to receive the attribute value as a float. Because this can be a little confusing, we've provided a demonstration in this tutorial patch.

Open the tutorial patch. Click on the patcherexplain_scalebias object in the middle of the patch to see the contents of the subpatch [explain_scalebias].

A demonstration of float math on char data.

In the above example, we have created a very small matrix. It has 4 planes of char data, but it has only a single cell. This will to allow us focus on what happens to a single numerical value in the matrix. You can see that we have set the value in plane 2 (the green plane) to 100. At the left side of the example, you can see how this whole number value would be represented as a fractional value between 0 and 1: in this case it's 0.392, which is 100/255 of the way from 0 to 1.

The jit.scalebias object will multiply the values by a certain amount—specified by the scale attribute—and then will add a certain amount to the values—specified by the bias attribute. When that matrix is altered by the jit.scalebias object, all of the values will be treated as float s for purposes of calculation, then will be re-stored in the matrix as char data.

The scale and bias attributes are set here with the scale 2.0 and bias 0.2 messages. The scaling factor (the multiplier)is 2.0 in this case, and the bias (the offset that gets added afterward) is 0.2. So, to understand what is going on inside jit.scalebias, we have to think of the green value as 0.392 times 2.0 plus 0.2, which equals 0.984. The jit.iter object reports the values in each plane of each cell (there's only one cell in this matrix), and we can see that the value (when stored in the matrix as a char) is 251 (or 0.984 on a scale from 0 to 1).

(Just as a mental exercise, can you calculate the values jit.scalebias will produce in the red and blue planes in the above example? Since the values in those planes in the original matrix are 0, the values in the resultant matrix will be 0 times 2.0 plus 0.2, which equals 0.2, which is equal to 51 on a scale from 0 to 255. So the RGB values being displayed by the jit.pwindow object at the bottom are 51 251 51.)

More examples

If the preceding explanation was crystal clear to you, you might want to skip over these additional examples. But in case you're still not totally clear on how math operations with char data (and jit.scalebias in particular) work, here are a few more examples.

One by one, click on each of the presets in the preset object, proceeding from left to right. In the paragraphs below we explain each of the preset examples.

The value in the green plane of the original matrix is 255. (This is equal to 1.0 on a scale from 0 to 1.) The jit.scalebias object multiplies this by 0.5, which results in an internal value of 127.5; however, when storing the value as a char, jit.scalebias truncates (chops off) the fractional part, and stores the value as 127.
The original value is 100, and we double it (scaling factor 2.0) and get the expected result of 200. There is no loss of precision in this case.
The original value is 100 (0.392). We scale it by a factor of 1.0, which leaves it unchanged, then we add -0.2 to it—that is, we subtract 0.2 from it—to get a result of 49 (that is, 0.192).
0.392 times 2.0 plus 0.2 = 0.984. On a scale from 0 to 255, that's 251.
This example and the next one show what happens when the result of the multiplication and addition operations exceeds the capacity of an 8-bit char. jit.scalebias will simply clip (limit) the result to the maximum or minimum limit of a char. Here, 0.392 times 4.0 equals 1.568 (that is, 100 times 4 equals 400), so the result is set to the maximum allowable, 255.
In the other direction, 0.392 minus 0.5 equals -0.108, so the result is set to 0.

It's noteworthy that these imprecisions and limits only occur at the point when the result is re-stored as a char. Up until then, the values are calculated internally as float s, so the precision is retained. Even though the multiplication takes the internal value beyond the 0-1 range, no limiting occurs internally, and the addition operation can bring it back into range. Here, 0.392 times 3.0 (= 1.176) minus 0.5 equals 0.676. When this is stored as a char, however, a small imprecision occurs. 0.676 on a scale from 0 to 255 equals 172.38, but the fractional part is truncated and the stored value is 172 (i.e., 0.675). For no change, the scaling factor should be 1 and the bias offset should be 0.

Try some more values yourself, until you're satisfied that you understand jit.scalebias and the results that occur with 8-bit char data. When you have finished, close the [explain_scalebias] window.

Adjust Color Levels of Images

Now let's apply jit.scalebias to color images. In the top left corner of the tutorial patch, you can see a familiar configuration: a jit.movie object with a readmessage box to load in a movie and a metro object to trigger jit_matrix messages from jit.movie. In this patch we'll modify the matrices with multiplications and additions in jit.scalebias.

Click on the message box read chilis.jpg to read in a JPEG picture. Note that we're reading a still image—not a video—into the jit.movie object. QuickTime can handle a wide variety of media formats, including still images in PICT or JPEG format. jit.movie treats still images just as if they were 1-frame-long videos.

The output of jit.movie will go to jit.scalebias for processing, and will then be displayed in the jit.pwindow. (You can ignore the jit.matrix object for now. We'll discuss its use later in this chapter.) The scale and bias values can be changed by modifying the scale and bias attributes of jit.scalebias.

Click on the toggle to start the metro. Try dragging on the number box above the scale $1message box, to increase the value of the scale attribute to 1.25.

Boost the brightness of the image; with scale , higher values get boosted more.

This will increase all non-zero values in all four planes of the image by a factor of 1.25 (a 25% increase). Note that multiplication has the effect of increasing larger values by a greater amount than smaller values. For example, if the red value of a particular cell in the original matrix is 200, it will be increased to 250 (a net increase of 50), while the blue value of the same cell might be 30 and would be increased to 37 (a net increase of 7).

Try increasing the scale attribute to a very large value, such as 20. Values that were 13 or greater in the original matrix will be pushed to the maximum of 255 (and even the very small values will be increased to a visible level), creating an artificially "overexposed" look.
Try decreasing the scale attribute to a value between 0 and 1. As you would expect, this darkens the image. A scale value of 0 or less will set all values to 0.
Return the scale attribute to 1. Now try adjusting the bias attribute. This adds a constant amount to all values in the matrix. Positive values lighten the image, and negative values darken it.

Boost (or reduce) the level of all values by a constant amount.

Here are a couple of more extreme scale and bias settings you might want to try. Set the scale value to 40 and the bias value to -20. This has the effect of pushing almost all values to either 255 or 0, leaving only a few colors other than white or black. Now try setting the scale value to -1 and the bias value to 1. This has the effect of inverting the color scheme by making all high values low and all low values high. Reducing the scale value even further (to, say, -4 or -8) creates a similar inversion, but only values that were low in the original will be boosted back into the 0-1 range by the positive bias value.

Adjust planes individually

You can adjust the levels in each plane individually in jit.scalebias, using the attributes ascale, abias, rscale, rbias, etc.

Set the scale value back to 1 and the bias value back to 0. Then experiment with adjusting each color plane independently by providing new values for the appropriate attributes.

We've made the process a bit more interactive by giving you a controller that lets you adjust the scaling of all three color planes at once. When you click or drag in the swatch object, it sends out a three-item list representing the RGB color values at the point where your mouse is located. Those values were once expressed on a scale from 0 to 255, but this has been changed to produce values from 0.0 to 1.0. (There is a checkbox in the inspector to provide the old style output if desired.) We use unpack to break that list into three separate floats, and we use those values to alter the rscale, gscale, and bscale attributes of jit.scalebias.

Values from swatch used as attribute values for jit.scalebias

Drag on the swatch object to scale the RGB planes all at the same time. Since this produces scaling values in the range 0 to 1, you're effectively reducing all the levels, so this will generally darken the resulting image somewhat.
You can try all of these operations on different images. Read in some other colorful images such as colorswatch.pict or wheel.mov and experiment with adjusting the color levels.

Reassign planes of a matrix

In the previous tutorial we used jit.unpack and jit.pack to reassign the planes of a matrix. There is another way to do that, using the planemap attribute of the jit.matrix object. In this example, we pass the matrix output of jit.movie through a jit.matrix object just to demonstrate the planemap attribute.

We can reassign the planes of a matrix as it goes through jit.matrix

The planemap attibute of jit.matrix allows us to map (assign) any plane of the incoming matrix to any plane of the outgoing matrix. The word planemap is followed by as many numbers as there are planes in the matrix (four in this case). Each place in the list stands for an output plane (the first place stands for output plane 0, the second place for output plane 1, etc.), and the value of that number states the input plane that will be assigned to it. By default, the planemap values are 0 1 2 3 (etc.), so that each plane in the input matrix is assigned to the same plane in the output matrix. But we can change those assignments as we like. For example, if we send jit.matrix the message planemap 0 3 2 1, we are assigning input plane 3 to output plane 1 (since the value 3 is in the list location for output plane 1), and input plane 1 to output plane 3. Effectively, we're swapping the red and blue color planes of the image.

Click on the message box read wheel.mov and start the metro to display the movie. (Set the scale attribute of jit.scalebias to 1 and the bias attribute to 0, so that you're seeing an unaltered image in the jit.pwindow.) Now, in the bottom right portion of the patch, click on the message box planemap 0 3 2 1 to swap the red and blue planes in the matrix. Click on the message box planemap 0 1 2 3 to go back to the normal plane mapping. If we set all three of the RGB output planes to the same input plane, we get equal values in all three RGB planes, resulting in a greyscale image.
Click on the message box planemap 0 1 1 1 to see this effect. The value 1 is in the list location for each of the three RGB planes, so the red plane of the original is used for all three RGB planes of the output matrix. To rotate through all of the different color plane rotations, we've filled a coll object with various plane mappings (similarly to the way we did in the previous chapter), and we will send those assignments to the jit.matrix to change the settings of its planemap attribute.
Double-click on the patcher r otatecolorplanes object to see the contents of the subpatch. It simply counts from 1 to 6, to step through the different mappings stored in the coll object in the main patch. (And when it's turned off it sends out the number 1 to reset to the default plane mapping.) Close the [rotatecolorplanes] window.
Click on the toggle above the patcherrotatecolorplanes object to step through the different plane mappings at the rate of one setting per second. Change the rate number box above the right inlet to a smaller value (say, 80) to see a flickering effect from rapid plane reassignment.

In the next tutorial chapter, you'll see how to rotate image hue in a subtler way, using jit.hue, and you'll see other ways to adjust color levels with the jit.brcosa object.

Reading and Importing Images

In this tutorial patch, we loaded three different kinds of images into the jit.movie object: PICT and JPEG still images, and a movie. It may seem a little strange to read still images into an object made for playing movies, but in fact QuickTime can read many types of media files, and jit.movie knows how to read them all. (You could even read an AIFF audio file into jit.movie, listen to it with start and stop messages, jump to different locations with the time attribute, etc.! Of course, you won't see any visual matrix info in that case.)

For still images, it's just as easy to load them directly into a jit.matrix object with the importmovie message, as demonstrated in Tutorial 3. If you import a movie that way, only one frame of the movie will be stored in the jit.matrix.

In this patch, we used jit.movie to load in all the images. The first reason is because one of the things we wanted to load was a movie (not just one frame of the movie). The second reason is because we wanted to demonstrate the planemap attribute of jit.matrix. The planemap attribute is only appropriate if there is an actual input matrix (a jit_matrix message coming in the left inlet). If we imported the images directly into jit.matrix with importmovie, the planemap attribute would have no effect.

Summary

The jit.scalebias object uses multiplication and addition to modify all the values in a particular plane of a 4-plane char matrix—or all planes at the same time. The scale attribute is a factor by which each value in the matrix will be multiplied; the bias attribute is an amount to be added to each value after the multiplication takes place. The scale and bias attributes affect all four planes of the matrix. To affect only one plane at a time, use the attributes for that particular plane, such as ascale, abias, rscale, rbias, etc.

You must supply the values for these attributes as float s (numbers containing a decimal point). To perform the multiplication and addition operations, jit.scalebias treats the char values as fractional values in the range 0 to 1, performs the math with float s, then converts the results back to char s (whole numbers from 0 to 255) when re-storing them. Results that exceed the range 0 to 1 will be limited to 0 or 1 before being converted back to char.

You can reassign planes of a matrix using the planemap attribute of jit.matrix. The arguments to planemap are the output planes listed in order, and the values in the list are the input planes to be assigned to each output plane. So, for example to assign the plane 1 of an input matrix to all four planes of the output matrix, the attribute setting should be planemap 1 1 1 1.

jit.scalebias provides a powerful tool for adjusting color levels in a 4-plane char (ARGB color) matrix. More such tools are presented in the next tutorial chapter.