Tutorial 8: Keyboard and Mouse Input
This tutorial references the patcher 08mKeyboardAndMouseInput.maxpat
Type, click, and drag
In this tutorial, we will look at controlling Max actions using two common controllers – your computer's mouse and keyboard. We will also see how to convert numbers into letters, and identify particular keystrokes.
It is not always convenient – or even useful – to use external hardware for controlling Max patches. Sometimes the computer itself contains all of the control mechanisms that you need to work with your patches. Working with the mouse and keyboard can give us extensive control of our patches without having to worry about any other devices.
To open the tutorial patch please open 08mKeyboardAndMouseInput.maxpat from the the zip archive, which is available for download at the top of this page.
Watching the mouse
Take a look at the tutorial patcher. At the left side of the window is a small patch that connects a metro object to two new Max objects labeled mousestate and modifiers. Click on the checkbox to turn on the metro, then move your mouse around the screen. You will notice that the number boxes connected to the mousestate object report information as your mouse moves, and that the connected checkbox is turned on whenever you click the mouse button.
The output of the mousestate object provides button-status and cursor-position information about the mouse/cursor when the cursor is positioned within a Max patcher window. It provides a value stating whether the mouse button is "clicked" as well as two sets of coordinates: absolute mouse position on the x and y axis, and a delta position showing the movement since that last time the mousestate was polled. The absolute position is based on screen coordinates, not coordinates within the patcher window, so you can track movement anywhere on your computer screen. It also means that the results of mousestate will vary depending on the size of your computer screen real estate. Screen coordinates on the computer are reverse-cartesian, which is to say that the upper-left hand coordinate of your screen will be 0, 0, and the lower-right coordinate will be the size of your screen minus 1 (e.g. a 1024x768 display will give you the coordinates 1023, 767 when the mouse is at the bottom-right edge).
The delta value reports changes in mouse position – but when are changes actually reported? The change (as well as the absolute position) are only reported when a bang message is received in the object’s inlet – hence, the use of the metro object to generate messages. This is called polling, and allows you to control the density of messages that are created by an object. This is especially important in the case of an object like mousestate, where massive streams of unwanted data could be generated if it acted automatically. The only output that does not require active polling is the mouse button notification (the left outlet of the object).
Watching the keyboard
Adjacent to the mousestate object is the modifiers object. This will track (from left-to-right) the shift and caps-lock keys on your keyboard, as well as the Cmd, Control and Option keys on a Macintosh system or the Alt, right-hand mouse keys or Ctrl keys on a Windows system. All of these outputs require polling as well, but are a great way to capture the state of the modifier keys as special cases within your patch.
The next three small patches demonstrate capturing key presses on your computer keyboard. Unlike mouse messages, keystrokes produce output only when pressed and released, so there is no need to use polling to reduce message density. In the leftmost patch, the key object is used to get the ASCII value of a key when it is pressed. That output is sent to a number box (to display the value), then passed to an itoa object – which translates that integer value into an alphanumeric symbol. This is sent to the Max Console (via a print object), where you can see the letter of the pressed key. Try typing your name and watching the Max Console; notice that the integers coming out of the key object are correctly converted to their corresponding symbol. Keys that generate non-printing characters, e.g. the return key (ASCII 13) and the tab key (ASCII 9) generate the correct numerical values from the key object, but print a single blank line in the Max Console.
Just to the right is another small patch that uses the keyup object. This is almost identical to the key object, except that it doesn’t report any key information until the key is released. This is useful for situations where you want an action to only occur while a key is being pressed; you could turn the action on when you receive a message from key, then turn it off when you get a message from keyup.
The next small patch feeds the output of a key object to an object called numkey. This is a very convenient object that allows you to work with numeric keyboard input in the integer domain. Start by typing in a few numbers – you will see them being accumulated into a number from the right-hand outlet. If you hit the return key, you will see that the number is reported as an integer from the left outlet. This means that the numkey object is performing all of the accumulation and translation of keyboard values into a numeric outlet. Note that this patch is limited to integer output; if you want a floating point output, you would need to add a floating-point argument (like 0.0) to the numkey object to tell it to respect the decimal point and report floating-point output.
Controlling output with mousefilter
Finally, the right-most patch shows an object called mousefilter in action. When we click and drag the upper slider, we see that no output is generated until the mouse button is released. This is another convenience object that can be used to limit the generation of unwanted intermediate data. In this case, you may not want to perform any action until the movement of the slider is complete. By funneling the slider object's output through mousefilter, all output is suppressed until the mouse button is released.
Creating an "intelligent" patch
Let's use some of these objects and techniques to create a patch that works "intelligently" based on keyboard and mouse activity. Create two metro objects with arguments of 250, and connect button objects to their outlets – these will be our test subjects. Using the left-most patch as a starting point, connect the left-most outlet of mousestate (which reports the mouse button state) to the left inlet of the two metro objects. Lock the patch and verify that pressing the mouse button turns on the metro objects, and releasing them causes the metro objects to stop.
Next, let’s track the horizontal mouse position using a slider. Add a slider to the patch, and connect it to the second outlet of mousestate. In order to have this make sense, we will need to change the range of the slider. Use the size message (or use the Object Inspector) to change the maximum range of the slider to the width of your screen in pixels – this will be something like 800, 1024, or 1280 (depending on the resolution of your screen). Now, if you turn on the metro object polling the mousestate, you should see the slider move in tandem with horizontal movements of your mouse.
Let's use this value to control the output speed of the two metro objects. Connect the outlet of the slider to the right inlet of the first metro you created. Now, as you move the mouse (with the button held down), you will see the speed of the first metro change based on the mouse's current position. Let's also use the slider to control the second metro object's speed, but scaled using a math operator. Add a * object to the patch with an argument of 2, and connect it between the slider and the second metro object’s right inlet. Now, as you move the mouse (and press the mouse button), you will see that the second metro takes twice as long to fire as the first, but both are being controlled by the mouse position.
Finally, let's use the keyboard to change the scale of that * object. Using the output of the numkey object's left outlet (the one that only reports values after the return key is hit), we can connect it to the right inlet of the * object. Now, we can change the scaling of the second metro by entering in a number using the number keys – and do so without having to click on the number box to make the change. You can test this by entering a number (try 3 for example), hitting return, then testing the mouse click function. You should see the metro objects respond to the current mouse position.
Summary
This tutorial shows how the computer itself can be used as a controller for your Max patch. Rather than depending solely on user interface elements, you can query the mouse and keyboard directly, and use the resultant values to alter the actions of your work.