For our final project, Mike and I will be working together on an interactive game-like installation using Arduino and Processing. Our idea is to create a game experience where the controller itself is the game, like “Simon” or “Bop-It” toys that we grew up with and are still popular today. Most often in digital games, the “controller” – whether a handheld unit or a mouse and keyboard setup – is really just an interface between the player and the game, and how the player communicates and interacts with the game world. We would like to create an interactive experience where the controller – the interface – IS the game.The line between the interface and game is blurred in dance and rhythm games, of which there are many popular and diverse versions, and we aim to make something along those lines. It would be interesting to play with this idea as part of our metaphor, that we can blur the line between controller and game by engaging different senses between the player and the game machine.

Our core design ideas are that our project will have real-time interactivity, will encompass both the physical components of Arduino and sensors that the players can touch and feel, as well as digital components that work together and receive input from different methods of interaction. We would also like to include audio and visual components, engaging multiple senses of the user in addition to the tactile sense of handling the game/controller itself. The components we have to work with are an Arduino board, a tilt sensor, a buzzer, colored LEDs and push buttons.

Both of us have worked with Arduino with previous projects, but we should research Arduino in relation to exactly what we would like to use it for. We also need to research how Arduino and Processing can work with each other, and if there are any limitations to what we can do when using them together. We should also find out the physical and digital limitations of our components, if there are any, so we are working within the scope of what our parts can accomplish and design with these things in mind.

To bring all of this together, we will create a handheld device that will the the “controller” with the Arduino board, push buttons, LEDs and tilt sensors incorporated. This will be connected via USB to a computer running the Processing part of the game, which will provide the audio and visual interaction. We will have to research how to do sound with Processing, and see if it is possible to do stereo sound in processing, which would be another level of interactivity and feedback we could play with. The Processing programming component will keep track of all of the sensory input from the Arduino hardware handheld controller, and we will have to program the designated outputs for the interactivity of the controller. Once we have the hardware functionally working and have started on getting the Processing part functionally working, we can work getting the two to work together and polishing it until it is the game experience we are aiming for.