Tutorial 27: Using MSP Audio in a Jitter Matrix
This tutorial shows how to copy an MSP audio signal into a Jitter matrix using an object called jit.poke~. Along the way, we'll investigate how to use the soundtrack from a movie in the MSP signal network using the sound output component attribute of the jit.movie object and a new MSP object called spigot~.
This tutorial assumes familiarity with routing MSP signals using send~ and receive~. It also uses a simple delay network using tapin~/tapout~ objects. Tutorial 4 and Tutorial 27 in the MSP manual cover these topics.
The jit.movie object at the top left of the tutorial patch reads a movie called rca.mov upon opening.
Our jit.movie object has two attributes set in its object box in addition to its attribute ( by cells). The attribute with a value of tells the jit.movie object to loop the movie as a palindrome. Once the playback of the movie reaches the end, it will play backwards to the beginning of the file, rather than looping around to the beginning and playing forward (the default behavior, when the attribute is set to ). If you watch the movie, you'll see that the arm manipulating the oscillator control moves up and then down again in an endless loop. The movie actually only contains footage of the arm moving upward, but the attribute we've used reverses the playback in the second half of the loop.
The Sound Output Component
The second attribute we've set in our jit.movie object sets the Sound Output Component ( ) for that instance of the jit.movie object. The name specified as an argument to the attribute (in this case ) specifies a new sound output component that MSP can use to acquire the soundtrack of the movie loaded into the jit.movie object. By default, the attribute is set to , which routes the movie's audio output directly to the Sound Manager. A named attribute routes the audio to a spigot~ object with the same name as the component, allowing you to access the audio signal in MSP:
The spigot~ object in the upper-right hand corner of the tutorial patch has an argument ( ) which matches the attribute of our jit.movie object. If a movie file loaded into that jit.movie object has a soundtrack (which the rca.mov file conveniently does), the audio from the movie is sent out as MSP signals from the spigot~. Note that the spigot~ object has two outlets, which correspond to the left and right audio channels of the movie soundtrack. Our rca.mov file has a monaural soundtrack, so we only need to use one of the outlets in our patch.
The soundtrack from the rca.mov file is sent as an MSP signal from the spigot~ object into a two-tap delay line (generated by the tapin~ and tapout~ objects in the patch). The dry audio signal is sent to a send~ object with the name attached to it; the two delay taps are sent to send~ objects named and , respectively. The three audio signals are output by named receive~ objects and summed into the gain~ object at the bottom of the patch, allowing you to hear all of them at once.
The jit.poke~ objects at the right of the tutorial patch, which write MSP audio signals into cells in the matrix. These cells, when displayed in the jit.pwindow object, portray an oscilloscope view of the movie soundtrack, with the dry and two delayed signals appearing as the colors , , and , respectively.Jitter matrix is generated by three
The three similar regions at the right of the screen use the jit.poke~ object to write MSP signal data into our matrix. The jit.poke~ object takes three arguments: the of the Jitter matrix to write into, the number of inlets to use, and the of the destination matrix to write numbers to. All three jit.poke~ objects in our patch write into the matrix . Since is a 2-dimensional matrix, we need inlets to specify where to write the data (one inlet for the column and one inlet for the row). The three objects differ in that they each write to a different of the matrix.
The first inlet of the jit.poke~ object provides the value to write into the matrix cell specified by the other two inlets, which take signals to specify the cell location. We use a sig~ object with a value of to write a constant value into our current position in the scope matrix. The value of gets interpreted as when writing into a matrix containing char data (which is what we're doing in this case).
The other two inlets in our jit.poke~ objects determine where in the output matrix they should write data (this set of coordinates defines the write pointer for the object—you could think of this as the location of the record head, only with two dimensions instead of one). The rightmost inlet receives the audio signal from our named receive~ objects and sets the vertical ( ) coordinate of the write pointer to correspond to the amplitude of the signal. The *~ and +~ objects in the patch scale the output of the audio signal from between and (the typical range for an audio signal) to between and (the range of the vertical dimension of our output matrix).
Sync or Swim
The middle inlet to our jit.poke~ object receives a sync signal that specifies where along the horizontal axis of the matrix we write the current amplitude from the audio signal. This signal is unrelated to the audio data coming from the movie—you could think of it as the horizontal refresh rate of the virtual oscilloscope we've made in this patch. The sync signal is generated by a phasor~ object in the subpatch:
Our phasor~ object generates a repeating ramp signal from to (nearly) . The *~ below it rescales this signal to generate values appropriate to the width of our matrix ( to ). This signal is then passed to a send~ object with the name , which forwards the signal to receive~ objects connected to the middle inlets of our jit.poke~ objects. The frequency of the phasor~ (specified by the number box connected to its first inlet) determines the rate at which our jit.poke~ objects scan from the left to right through the matrix.
The dry audio signal and the two delayed outputs are visualized as the three visible planes of our jit.poke~ objects overlap, different color combinations will appear in the output matrix.matrix ( , , and , or , , and ). When the cells written by the
Now that we understand how the matrix is being written, we need to look into how the matrix clears itself every time a horizontal scan is completed. The relevant parts of the patch are shown below:
The change~ object outputs a value of when the ramp generated by the phasor~ object is on the increase. When the phasor~ snaps back to at the end of the ramp, change~ will briefly output a value of . The ==~ operator, which outputs a when the change~ object does, will output a at that point. When the phasor~ begins to ramp again, the ==~ object will output a , triggering a from the edge~ object (which detects a zero to non-zero transition in the last signal vector). The is then sent to a receive object named , which triggers a message to the jit.matrix object. As a result, our matrix is cleared every time the phasor~ restarts its ramp.
Putting it all Together
The jit.op object is . As a result, our composite is made from the multiplication of the two matrices. Since most of the cells in our matrix are (black), you only see the movie image appear in those cells and planes where the jit.poke~ objects have traced the waveform.attribute we've specified initially for our
The jit.movie object lets you define a named Sound Output Component. The spigot~ object lets you access the soundtrack of a movie as an MSP signal by giving it an argument that matches the attribute of the jit.movie object playing the movie.attribute of the
You can use the jit.poke~ object to write data from MSP signals into a named Jitter matrix. The jit.poke~ object takes arguments in the form of the of the matrix to write to, the number of inlets with which to specify cell coordinates, and the to write to in the matrix. The first inlet of jit.poke~ takes the value to be written into the matrix. Subsequent inlets take MSP signals that specify the cell location in the matrix in which data should be written.
|Working with Video in Jitter||Working with Video in Jitter|
|Video and Graphics Tutorial 8: Audio into a matrix||Video and Graphics 8: Audio into a matrix|
|change~||Report signal direction|
|dac~||Audio output and on/off|
|edge~||Detect logical signal transitions|
|jit.matrix||The Jitter Matrix!|
|jit.op||Apply binary or unary operators|
|jit.poke~||Write an audio signal into a matrix|
|jit.pwindow||Display Jitter data and images|
|jit.movie||Play a movie|
|phasor~||Generate sawtooth signals|
|tapin~||Input to a delay line|
|tapout~||Output from a delay line|