Data Tutorial 5: List Processing

Introduction

This tutorial is a large one, because it covers a big topic: list processing. We cover several of the available modes of the zl object, which provides a central clearinghouse of list processing functions. We also see how we can perform mathematical expressions on lists with the vexpr object, and how to use the prob object to create a table of probabilities for music creation.

Within Max, the list is one of the most powerful data structures available. You can define any combination of values in a list, then have them sent as messages, get processed by objects or get treated as small tables. Much of this is made possible by the zl object, which gives you the ability to query, reorder and access any of the elements in a list. Adding vexpr into the list processing mix allows you to process list elements mathematically without having to break them into their individual components.

The use of the prob object is integral to many generative programs. The prob object allows you to easily create a weighted probability table, with only a bang required to generate the expected data. In our tutorial, we will see how the prob object is used to seed a basic sequencing application.

To open the tutorial patch, click on the green Open Tutorial button in the upper right-hand corner of the documentation window.

Running the sequencing patch

Take a look at our tutorial. This is the quintessential Max patch – the step sequencer. When you open the patch, the first two multislider objects (labeled A and B) are seeded with values, while the third (labeled C) is empty. These objects contain sequences for a MIDI playback system fed by the two noteout objects to the right of them. Double-click the noteout objects to select an available synthesizer. Turn on the metro at the top-left by clicking the toggle, and the sequence will begin playing, as driven by the A and B multislider contents. Click on the C multislider and drag your mouse to set values for the melody line – the result will play back using your chosen synthesizer.

If you are using a General MIDI-compatible synthesizer, you should hear the top two multislider sequences as drums, and the bottom (C) sequence as a melody played on a piano. This is because, according to General MIDI rules, MIDI channel 10 (the argument to the upper noteout) is the drum channel, while the other fifteen channels play melody instruments and generally default to piano. Many synthesizers require you to configure this behavior, but the built-in synthesizers on both Macintosh and Windows machines behave in this way by default.

At the right of the patch are a number of editing routines that use the zl object, which we'll look at in depth in a minute. You can rotate the sequence stored in multislider A, reverse the contents of multislider B, and perform several different functions against the melody contained in multislider C. Click on the button objects that activate the editing routines, and see that the changes are immediately applied against the multislider contents.

The basic function of the step sequencer should be familiar: the metro drives a 32 position counter, whose output is turned into a fetch message for the three multislider objects. The output of the multisliders are sent to two different makenote/noteout combinations; the two drum multisliders are sent to a noteout object assigned to MIDI channel 10, while the melody is sent to a noteout object assigned to MIDI channel 1.

The two drum channels feature a simplified way of selecting drums – their multislider objects are limited to only three values each. The select objects used with these objects are a way of remapping the multislider contents (0, 1 or 2) to an instrument (none, kick/closed hi-hat and snare/open hi-hat, respectively). In this way, the full range of the multislider has significance, and the output value is easy to work with.

Set the entire sequence in the A and B multislider objects to 0 (drag the mouse across them along the bottom). The drums in the sequence will stop. Try adding different patterns to get a sense of how the numbers in the object map to the different sounds you get.

Working with prob (probabilities)

The initial setup of the drum sequences is done using the generate new patterns section of the patch, found below the multislider objects. If you want to see it in action, click on the button connected to the hi-hats’ uzi object. You will see that the A multislider will get scrambled, but that the contents tend to emphasize the 1 value (which corresponds to the closed hi-hat). This occurs because of the probability table set up and enforced by the prob object.

The setup of the probability table is done through the large message triggered when the patcher opened by the loadbang. The way that our probability table works is based on transitions - these are defined by three element lists containing current value, another value and a weighting value used to determine how likely a transition is from current value to another value. So, for example, 0 1 2 means that the weighting applied to a transition from 0 to 1 is 2. What does 2 mean? It doesn’t really mean anything specific – it only has a defined value in relation to all of the other weights for that transition.

The message box used for the hi-hat probability tables sets up all of the possible transitions within a single three value (0,1,2) set. Since you can only be at one location at a time, the probabilities are dependent on the current value. Let’s look at probability calculations if the current step is 0:
Next Step: 0; Weighting: 3; Probability: 3/8 (37.5%)
Next Step: 1; Weighting: 3; Probability: 3/8 (37.5%)
Next Step: 2; Weighting: 2; Probability: 2/8 (25.0%)

In order to understand the calculation of the weighting value, you need to add up all of the weights for the current step, then use that to divide the individual values for a weighting factor. In this example, the total weighting values for all cases where the current step is 0 add up to 8. This gives us the denominator for the calculations we made. Therefore, 25% of the time, we will go from no hi-hat sound to an open hi-hat; 37.5% of the time, we will go to a closed hi-hat and the rest of the time there will be no sound at all.

If you look at the cases when the current step is 2 (an open hi-hat), you will see that the transition to 0 and 1 both are weighted at 4, while the transition to 2 (no change) is given a weight of 0. This means that the prob object’s table will prevent an open hi-hat from ever being repeated, since there is a 0% chance that a current value of 2 will be followed by a 2.

The prob object is driven by an uzi object, set to output 32 bang messages. However, the outputs of prob are individual values, and the multislider is expecting a 32-value list for setup. How do we combine all these messages efficiently?

Working with zl

The zl object is a single object that works in over 20 different modes as set by the object's first argument. The modes perform list processing functions such as sorting, removing unwanted items and joining lists. In the case of the sequence generators for the drums, the zl object is used in group mode, with an argument of 32. This means that the object will collect 32 values, group them into a single list, and then output the list from its left outlet. This is a quick and very efficient way to pack a stream of data into a list of predefined, and turns an otherwise onerous task into a job for a single object.

The tutorial patch shows many uses of the zl object – particularly in the editing functions on the right side. All of these editing routines start with the zl object in reg mode. In this mode, the zl object will accept a list into its right inlet, and store it until it receives a bang in the left inlet. The zl reg object is, in essence, the equivalent of the int, float or value objects, but is designed for list storage.

When you click on the button for each editing routine, it outputs the list from the zl reg into another list processing object. Several of the editing routines use zl in another mode for the list manipulation; for example, the “rotate A” routine uses zl in rot mode with an argument of 1 to rotate the values by one entry – everything moves one position to the right, and the last entry becomes the first. The “reverse B” routine uses zl rev to reverse the order of an incoming list, while the “sort parts of C” combines three zl processing objects to split the list into chunks (iter), sort each and reassemble (group) them into a list. Most zl modes take a second argument or allow a number in the right inlet to change a parameter - for example, the zl iter object breaks a list into sub-lists of a size set by the argument (in our case, 4) that can also be changed by sending a number into the right inlet. If we change the number box connected to the zl iter object to 32 (the entire length of the sequence) and then click the button, the melody stored in multislider C will become sorted from low values to high values.

Perhaps the best way to see all of the zl operating modes is to look at the zl help file. Unlock the patcher, select a zl object and choose “Open zl help” from the contextual menu or from the application help menu. The help file is broken into six tabs, since there are so many modes – but you will be able to see all of the ways that zl can be used to bend, fold and mutilate Max lists.

Working with vexpr

When working with lists, there are times when you want to perform mathematical expressions against all of the elements in a list. Using standard Max math objects (+, etc.) would require splitting, processing and reassembling a list – not the most efficient way of dealing with the problem. There is a solution: the vexpr object.

As you can tell by the name, the vexpr object is very similar to the expr object we learned about earlier, with the exception that it is made specifically to process lists. If we look at the use of vexpr in the “transpose C” routine, we see that the familiar $i1 syntax is used to reference the individual integer values of the list, and that we will add (or subtract) the number 1 from each entry. Therefore, when the zl reg object outputs a 32-value list (sent originally from multislider C), each of the entries will be incremented or decremented, and will be output as a 32-value list.

The “randomize C” routine shows the use of vexpr with two lists used for input. The right-hand side of the routine will generate a 32-value list of random numbers varying from -1 to 1 (the result of the uzi, random 3 object and the – 1 objects sent into a zl group). This is input into the right inlet of vexpr. The left inlet receives the 32-value list from multislider C being held by a zl reg. The vexpr object iterates through the two lists, adding the elements from the right inlet (identified as $i2) to the elements from the left inlet (identified as $i1). This means that each of the list elements will be altered by a different random element, giving a small randomization to the multislider contents. Note that this randomization can be done multiple times to gradually morph the melody into a new pattern altogether.

Summary

We started this tutorial by learning about probability tables, and their use in seeding a multislider object's content. From there we covered several ways that a multislider object's content (provided as a list) could be processed using the zl and vexpr objects. Since lists are an integral data structure in the Max environment, these objects will become an often-used part of your programming toolkit.