Tutorial 10: Chromakeying
This tutorial explains how to perform chromakeying with two source movies using the jit.chromakey object. We will also learn how to find out the color of any pixel on the screen.
When you open the tutorial patch, Max will automatically read two movies (oh.mov and traffic.mov) into two jit.qt.movie objects by sending appropriate messages to those objects with a loadbang:
Additional parameters we need for this patch are also initialized by the loadbang, which is connected to the message box on the right of the patch. The message box initializes the rest of the patch by sending messages to named receive objects elsewhere in the patch (see Tutorial 25: Managing Messages, in the Max4TutorialsAndTopics.pdf).
• Click the toggle box to start the metro. You should see images appear in the three jit.pwindow objects in the patch. Note that the toggle box not only starts and stops the metro, but also starts and stops the movie transport of the two jit.qt.movie objects.
The lower half of the tutorial patch (with two of the three jit.pwindow objects) looks something like this:
• Click with the mouse on the blue region of the left-hand jit.pwindow object (i.e. the area behind the man's head in the movie).
The third jit.pwindow object (in the lower-right hand of the patch) will look like this:
The jit.chromakey object
Chromakeying—the process of superimposing one image on top of another by selective replacement of color—is accomplished in Jitter by the jit.chromakey object. By specifying a color and a few other parameters, jit.chromakey detects cells containing that color in the first (left-hand) matrix and replaces them with the equivalent cells in the second (right-hand) matrix when it constructs the output matrix. The result is that cells from the first matrix are superimposed onto the second.
• Since any color is fair game for the chromakey, try clicking elsewhere in the lefthand jit.pwindow. Different colors will be knocked out of the man's face to reveal the traffic.
Historical note: Bluescreen compositing, or the process of shooting live footage against a blue matte background only to replace the blue with a separate image later, has been around since the late 1930s. Originally a very expensive film process involving expensive lithographic color separation, bluescreen (and its slightly less common sibling, greenscreen) has evolved into the most commonplace (and effective) special effect in film, television, and video. The ability to perform chromakeying (the technical term for the process) using analog (and later digital) video superimposition has only made it more ubiquitous. Video chromakeying is often referred to in the television industry as CSO (Colour Separation Overlay), the name given to the process by the BBC team that developed it in the 1960s. Petro Vlahos, a bluescreen innovator in the 1960s, was awarded a Lifetime Achievement Award by the Academy of Motion Picture Arts and Sciences in 1994, an acknowledgment of how indispensible the technology had become.
The jit.chromakey object uses the attribute to define the center color to in the chromakey (called the reference color). This attribute is set as a list of values for as many planes as exist in the matrices that are being keyed. The attribute specifies a range of values around the key color. Colors within this range will be keyed as well. When using jit.chromakey with char matrices (e.g. video), the attributes are specified in a floating point range , which is then mapped to the 0-255 range necessary for char data. To set the attribute for a solid green chromakey, therefore, you would set the attribute as , not . A range of will key all values within half of the chromatic distance from the reference color (computed as the sum of the magnitudes of difference in each plane between the reference color and the actual cell value). A range of will treat only the exact reference color as part of the chromakey.
• Try clicking on the blue region in the lefthand movie again, and play with the attribute to see how the chromakey output changes. At low tolerance, some of the bluescreen in the left image will remain in the keyed output. At a very high tolerance, parts of the man's face may disappear.
In the tutorial patch, the attribute to jit.chromakey is set by clicking on an invisible object. If you unlock the patch, you will see a region of concentric red squares that sit on top of the left-hand jit.pwindow object:
The region is a Max user interface object called suckah, which appears on the object palette like this:
The suckah object reports the RGB values of any pixel on the screen that the suckah object overlays. It reports these values as a list of integers out its outlet when you click in the object in a locked patch. For example, clicking on a region of solid blue that has a suckah on top of it will cause the suckah to send out the list .
To set the attribute for our jit.chromakey object, we take the RGB list that comes out of the suckah object and send it through a prepend , which adds an alpha value of to the front of the list. The resulting ARGB list is then divided by using the vexpr object to scale it to the range of . The message is then completed by the prepend . The final message is then sent to jit.chromakey.
The Blue Screen of Death
The jit.chromakey object has additional attributes: , , and . When a matrix arrives in the left inlet, jit.chromakey creates a greyscale (1-plane) mask internally, based on that matrix. Cells in the incoming matrix that have color values within the tolerance () range are set to the attribute's value (the default is ) in the mask. Regions outside the tolerance range are multiplied by the attribute (default is ). If the and are set to and , the resulting image should look white where the keying should take place, and black where the original image is to be retained.
The resulting mask and its inverse are then multiplied by the right and left matrices, respectively. The results of the multiplication are then added to form the composite image. The following diagram shows you a pictorial overview of the process:
As you can see, the attribute sets the strength of the righthand matrix in the output, while the attribute sets the strength of the lefthand matrix. If we were to reverse the and attributes, the chromakey would be reversed, and the following would happen:
The attribute allows for an amount of interpolation between the area being keyed and the area not being keyed. This lets you create a soft edge to the chromakey effect. Colors in the left matrix that are slightly out of bounds of the key tolerance range, yet that are within the range of + from the reference color, are interpolated between their original (unkeyed) color and the color in the same cell of the right matrix. The amount of interpolation is based on how great the value is, and how far the color in question lies outside the tolerance range.
• Try experimenting with different , , , and values for different colors. Watch how the five attributes interact for different keying effects, and how the and values complement one another.
Accurate chromakeying can be a challenging process. Correct values for the and attributes are essential to make sure that the correct regions in the first image are keyed to the second image. In general, very detailed key images will show slight aliasing in spots where the colors rapidly move between keyed and non-keyed regions. In addition, a single key color (e.g. blue) almost never suffices for a complete key, so a range of values must always be used. You will often find, however, that the color you want keyed out of part of the image is somewhat present in the region you want to retain! Balancing all of these factors to get the most convincing effect is the hardest part of using the jit.chromakey object.
Summary
The jit.chromakey object lets you do two-source chromakeying in Jitter. You can set a color range for the key using the and attributes, and use the , , and values to define how the two matrices work in a composite. The suckah user interface object allows you to easily select colors as they appear on the screen by setting the object over a jit.pwindow. Clicking the suckah object will give you the color of the pixel just clicked on the screen.
See Also
| Name | Description |
|---|---|
| jit.chromakey | Keying based on chromatic distance |
| jit.pwindow | In-Patcher Window |
| jit.qt.movie | Play or edit a QuickTime movie |
| loadbang | Send a bang automatically when a patcher is loaded |
| metro | Output a bang message at regular intervals |
| prepend | Put one message at the beginning of another |
| suckah | Get pixel color at display coordinates |
| vexpr | Evaluate a math expression for a list of different inputs |