Independent Study
I have taken my last semester as a time to delve into topics that I am interested in learning more about. I have chosen to focus on coding in Arduino as well as how it can be utilized as a controller for other technologies, such as Touch Designer.
This semester, I opted to take an Independent Study course in order to explore some technologies that I have been wanted to learn and get more skilled in. Namely, Arduino. I have very minimal past experience with coding, but it is something that I find really interesting and powerful.
I have met with my IS Advisor and formed a Syllabus for myself to follow throughout the semester. My goal for the semesters is to develop skills in Arduino and see if I am able to connect Arduino and TouchDesigner in order to create a light show of sorts.
The technologies that I plan to use throughout the semester include Arduino, Java, TouchDesigner, and LEDs.
Learning Arduino #1
It took a bit to get the materials I needed, but I have been reading Physical Computing: Sensing and Controlling the Physical World with Computers to learn some of the basics of computing, Arduinos and other electrical components.
Once I got my hands on an Arduino Uno, I set out to start learning how to program it to do simple tasks. I started with getting it to light a single LED.
I followed Jeremy Blum's Element 14 Tutorials on Youtube, which you can find a link to HERE.
Following this tutorial, I learned the basic command that are required to turn a simple LED on and off. VERY simple, but for someone who has absolutely no background with Arduino and this kind of technology, I have to admit that I was stoked I got it to work. Below I have inserted a simple video of the code working.
Arduino Tutorial #2
I followed the Arduino Tutorial series I had already started (link HERE) which is titled Tutorial 02 for Arduino: Buttons, PWM, and Functions. Earlier this semester, I was able to grab some Arduino and Physical Computing materials from UT's "Hatchery" and these are what I was able to use when following these tutorials.
This tutorial, took some trial and error and I'm still not sure why. I think there were some connection issues between the button and the breadboard, so when I came back to this tutorial a day or two later, it worked as it should.
This tutorial actually had 3 different parts to it. The first two examples were to get the button hooked up and working with the Arduino. The button turns the LED on when it is held down, and off when it is let go.
The second and third parts consisted of slight variations.
The second consists of a "debounce" code that eliminates random signals that run back and forth which case the button to not always run properly.
In the third variation, the button turns the LED on with the first press, and increases the brightness each time it is pressed until the value is greater than 255. Then, the LED is turned back to 0% brightness, or off. You can see the code for this function below.
Arduino Tutorial #3
When following these tutorials, I learned how to get serial readings from a button as well as a potentiometer, so that I am able to control the output from the Arduino. The potentiometer allows me to control the amount of output more specifically and I can read all of these values in the Arduino's "Serial Monitor."
Arduino Tutorial #4
In the next tutorial, I've been able to learn more about analog inputs. Jeremy Blum's tutorial walks through two different analog input methods, one using a photosensor and the other using a motion sensor, both controlling the brightness of a single LED. Unfortunately, I don't have a motion sensor at my disposal, so instead, I followed this tutorial from Afrotechmods on YouTube. In this tutorial, I was able to use a potentiometer to control the brightness of the LED.
The first version uses the photosensor to turn the LED on and also, if desired, we are able to see the values it is outputting in the "Serial Monitor" window. We just need to add it into our setup code properly.
I think this was the most fun tutorial I have done to date. I'm really interested in human and digital interaction, and using this photosensor is really fun for me.
The video below are the results of the first iteration. As you can see, the photosensor has to be almost completely covered for it to trigger the LED to turn on.
This video is the second iteration. While the change in brightness is choppy, now it adjusts its brightness based on how much light the photosensor is receiving.
The second version of using analog inputs in this tutorial uses the potentiometer to control the LEDs' brightness.
Here, I am able to use the analog inputs connected to the potentiometer in order to control the brightness. The potentiometer is also feeding its values to the Serial Monitor so I can see the exact value that is being output. In order to do this, is requires the potentiometer to be "mapped" to the LED Brightness. The Arduino does the math for us, so it is able to convert the potentiometer's values (0 to1023) to the LED's brightness values (0 to 255).
Connecting Arduino to TouchDesigner #1
After reading the Physical Computing textbook, completing a handful of tutorials, and watching many videos, I wanted to see if I am able to connect Arduino to TouchDesigner. I am more familiar with Touch Designer than I am Arduino and want to see how well they pair with each other and if there is the possibility to develop a system between them.
I poured over online forums and videos and found multiple ways that people have connected the two programs in the past. It was evident that this wasn't going to be as step-by-step as I thought it might be.
I found, on the derivative website, a tutorial to use the "firmata" node to create a connection between the two programs. Arduino has a catalog of examples that are downloaded when you download Arduino. The "Standard Firmata" code is the basis for the connection between the programs. This basically allows Touch Designer to talk to the Arduino.
Below is a screenshot of my initial Touch Designer file.
The firmata node has a list of pin values, which, when matched with the Standard Firmata Arduino code, establishes a connection.
The video to the right shows the connection between the two programs. The LFO node goes to a Math to adjust the range from -1 to 1 into 0 to 1 since the Arduino is unable to convert negative values into the brightness of the LED. The Math is connected to a Null (ledPin) which is connected to the code for Pin 10 under the Pin Values tab in the firmata node.
The LED is displaying the values being caused by the LFO as it's brightness, so it continuously turning on and off.
I want to see if I can take this further. The initial connection is there. Touch Designer is sending values to the Arduino and is controlling this single LED I have hooked up. How can I make it do more?
I want to see if this works with sound. Will the qualities and volume of sound be able to control the LED? If the sound is louder, will the LED glow brighter?
Below is the TouchDesigner node setup for the connection to work.
For the connection to work, we have to keep the Standard Firmata code uploaded to the Arduino and connected to the firmata node in Touch Designer. The main change here will be switching the input for Pin 10 from the LFO to a sound input.
Here, for this system, we have an Audiodevicein node leading to a Math. The Math does the same thing as in our last example, changing the values from -1 to 1 into 0 to -1. We then use a Select node to select one of the two audio channels being displayed which leads to a Null and finally, to the Firmata. I've added an Audiofilter node to exaggerate the levels that are received, but ultimately, it's not necessary.
To the right is a video of the result!
Song used is "Intro Sweet Glory" by Jimkata
Ok, so I'm really enjoying this, so... can I make the LED react to video?
Below is a screenshot of my Touch Designer file in order to make this happen.
This was more difficult than using sound to activate the LED. Instead of just one value, volume, I have a colored video feed which I then need to extract values from and simplify so that the Firmata node is only reading for one dynamic value.
One again, the Standard Firmata code uploaded to the Arduino needs to stay connected to the Firmata node in Touch Designer, so we don't need to edit that. The main change for THIS version, will be finding a way to simplify the incoming video to one number.
I started with Videodevicein which uses my computer camera as the video feed. It is fed it into a Monochrome top to simplify the rgb values to grayscale, and then to a Threshold where I am able to adjust the range of values from the video feed that Touch Designer will read. I set it to a spot that reads me, but not my lighter colored background. The white that is displayed is the elements of the video feed that is being read.
I was able to use some of my knowledge using Touch Designer with an Xbox Kinect to get this particular system to work properly.
The Theshhold feeds into an analyze in order to output numerical values. I decided to change the node type from a Top to a chop since that is the node type that was used for the past two versions, but it may be possible to feed a Top value into the Firmata node. This might be something to come back to later in the semester.
After converting the Analyze top to a chop, we get 4 values, r, g, b, and a which all show the same value. Once again, as before I feed the node into a Math, Select, and Null, and then into the final Firmata node.
Editing the Math to values that worked properly was difficult and took a lot of trial and error to get it to a spot where the values were changing enough to be seen by the LED. I then selected the "a" or "alpha" channel to be the numerical value to feed into the null and Firmata node.
In the end, I was really happy with my results. I was able to connect the two programs like I set out to do, AND edit it to react to different values. IT gives me a lot of hope for its capabilities for the rest of the semester.
Connecting Arduino to LED Tape
Since I was able to connect Arduino and Touch Designer, my next task is to connect Arduino and LED Tape. This will require the power, ground, clock pin, and digital pin to be connected to the Arduino. I watched a coupe of tutorials online and felt confident that I'd be able to establish the connection rather quickly.
Not quite.
I was hoping to find some sample code and tutorials that would walk through the basic connection so I could gain a basic understanding of how it worked and then later, adapt it to work with different systems for specific needs. The tutorials and examples I found weren't working and often were specific to a type of LED tape that I did not have.
To the right is the failed attempt.
To the left is the first sample code I tested.
This code is simple. It designates the type of LED (APA102), begins the serial connection, and calls a command that should turn the LED on and off at a steady pace designated by the "fadeSpeed" integer. Well, after multiple edits to make sure I was calling the right type of LED and calling all of the LEDs instead of just the blue or the green, I still have no luck.
I'm not exactly sure what it is I did wrong in this setup. It may be my coding, my wiring, or simply the LED strip that I have been using. I will probably come back to it later, but for now, it has given me so much trouble that I'm going to try one of the other many methods found on the amazing internet.
Connecting Arduino to LED Tape #2
I've found a simplified version of the previous example code to test the LED tape connection. Instead of passing a function to light the entire strand, the code calsl each specific LED and assigns it a value.
By calling each of the LEDs and assigning them their own individual value, I am able to designate the exact value of each LED.
While this works, and is definitely the first step to more complicated use of Arduino and LED, to program an entire strip of LED like this is cumbersome and only good for about 10 or so before its gets unecessarily repetitive.
By calling each of the LEDs and assigning them their own individual value, I am able to designate the exact value of each LED.
This makes me wonder how I might be able to combine these two lines of code to edit the tape so it slowly increases and decreases in brightness.
Connecting Arduino to LED Tape #3
After getting the LED to light up, I know that the tape is working as it should. This made me want to revisit the first code and see if I can get it to slowly increase and decrease in brightness.
I adapted the TurnOn function from the code that wasn't working and adapted it using the array method in the file that was working. I created integers r, g, and b to correspond with the red, green, and blue LEDs. Under the TurnOn function, I designated LEDs 10, 11, and 12 to be the ones affected and increased the r (red) value by 1, then inserted it at the bottom of my loop function. This way, every time the code goes through the loop, it will add one to the red LEDs brightness.
Below is the result.
I then added a TurnOff function which would bring the red LEDs back to 0 brightness once the value value came to be above 10.
Arduino Tutorial #6
After having some success connecting Arduino to Touch Designer and LED tape, I want to go back to some tutorials and keep learning the different basic functions of Arduino. I'm also realizing that its a lot like learning a whole new language. I came into this semester already kind of knowing this, but it's another thing to experience it. I know its cliche to say, but programming really is a whole language on its own, with syntax, grammar, and an order to sentences. I'm realizing that it is a lot easier for me to follow along and understand someone else's code than it is for me to write my own. Hopefully, with time and practice, I'll become more and more fluent.
In an earlier tutorial, I learned how to read the serial numbers being output by a button and potentiometer, and now I'll be learning how to use the serial connection to trigger an action. In this case, turning a single LED on and off. I'm skipping Tutorial #5 for now, since I don't have all of the materials at hand.
By using If statements, I was able to control a single LED by pressing either "1" or "2" on my keyboard. If any other key was pressed, called by the else function, the Arduino will output "invalid" in the Serial Monitor.
Connecting Arduino to Touch Designer #2
My goal for the end of the semester is to connect Touch Designer to Arduino to LED Tape in order to make cause lights to be triggered by happenings in TouchDesigner. However, I'm unsure of how to do this with the Arduino to TouchDesigner connection I made earlier in the semester.
With the connection using Standard Firmata on Arduino and firmata in Touch Designer, it established a reliable connection, however I don't think it will adapt when instead of controlling a single value, it is controlling a data line connected to the Arduino. So I'm going to see if there are other ways to trigger a connection between Arduino and TouchDesigner.
This was the hardest thing for me to accomplish yet, and I still don't feel confident in creating a connection between the three technologies. I had to go to multiple different resources to get this connection to work properly, and I'm really not confident I could do it again.
The coding is hard to follow, and to be honest, I'm not sure what half of it means. I'm not quite sure How to get it to trigger the LED tape, much less a single LED either.
I am definitely going to need to revisit this later in the semester and see if I can establish connection. As the semester is nearing the end, I'm not sure I'll be able to accomplish this.
Pixel Mapping in TouchDesigner
In order to take a break from the Arduino and connection aspect of this project and semester, I wanted to work on pixel mapping inside the Touch Designer program. I figure, if I am going to get it to work at all, I will need to understand how pixel mapping in Touch Designer works and then maybe... just maybe I'll be able to use that knowledge to trigger it to feed through Arduino to LED Tape. This particular video explained how to pixel for a DMX output.
I followed the course HERE.
In order to render an LED strip in Touch Designer, a line SOP was used with the same number of points as the LED strip that I have.
A ramp TOP was used to declare the color of the LEDs and its phase is being cycled through due to the connection to a speed CHOP. This was all fairly straightforward for myself, since I have used Line SOP, Ramps, and Speeds in previous projects. The difficulty for myself cam in when these elements, led into another Container that contained the nodes required to create the LED Preview.
In the Container, the Sops are converted into Chops using a Sopto CHOP. It is then instanced within the Geography container in order to combine the Ramp effect with the Sop geometry. This results in a visual like the one to the right. We get a mockup of the individual LEDs and a preview as to what they look like when lit up in a certain way. In this case, the Ramp TOP is controlling the color.
Because this is specifically a tutorial for DMX output, I'm not entirely sure how I will be able to adapt it to my needs in the coming weeks. I fear that my original concept may have been a bit above my skill level, and I don't think I have the time to learn all the things I need to make it happen. I really enjoyed following this tutorial and can see myself using similar systems to these in the future, but I don't think that I will use it for my final project this semester.
Controlling the LED through Arduino
I've included the FastLED Arduino Library, so that the program is able to register the LED tap as a multi-parameter object, included the number of LEDS on the step of tape I am using and declared (#define) the Data Pin and Clock Pin to the port they are input to.
Before, when I connected the Arduino to the LED tape, I wasn't using ints to give values. I only did that when I created the TurnOn function in the blog titled "Connecting Arduino to LED Tape #3." I decided to adopt that idea to make it easier for myself to code. Integers r, g, and b, correspond to the red, green, and blue values of the LED Tape.
Using the analogRead function, the Arduino reads the value being input, and I mapped it to fit the values for the tape. In this case, 255. I still needed to declare each individual LED, as I did previously, but since I am using r, g, and b, itss
I've included the FastLED Arduino Library, so that the program is able to register the LED tap as a multi-parameter object, included the number of LEDS on the step of tape I am using and declared (#define) the Data Pin and Clock Pin to the port they are input to.
Before, when I connected the Arduino to the LED tape, I wasn't using ints to give values. I only did that when I created the TurnOn function in the blog titled "Connecting Arduino to LED Tape #3." I decided to adopt that idea to make it easier for myself to code. Integers r, g, and b, correspond to the red, green, and blue values of the LED Tape.
Using the analogRead function, the Arduino reads the value being input, and I mapped it to fit the values for the tape. In this case, 255. I still needed to declare each individual LED, as I did previously, but since I am using r, g, and b, it's a lot easier to control.
Below is the red value on the LED being controlled by a potentiometer.
By using the serial connection, and Serial.print command, I was able to see the exact values that were being output as I turned the potentiometer.
From here, it was fairly easy to add more potentiometers to affect the LEDs' color. I quickly added two more so I had full range in red, green, and blue values.
The main issue I ran into with this code and setup was that the potentiometers, even when I was not touching them, would produce varying values that, sometimes, caused drastic changes in the color produced by the LED Tape. I looked up ways to try and remedy this, however it did not seem to have much effect since the values varied widely.
Below is the code I used in an effort to decrease theses changes
Another, smaller issue I ran into with this code was that the LED Tape would not reach a true "0" in brightness. I tried to remedy this by changing the values that were mapped from the sensor to the LED Tape to coincide more with what I wanted to be output. However, since the potentiometers output varying values, there would be instances when the tape would reach a true "0" in brightness, and then jump to full brightness, due to a negative output value. I switched back my map values to something more conventional, and will have to deal with no true "0" for the time being.
Creating Values from Serial Input
As the end of the semester is nearing, I want to have one final deliverable. Unfortunately, I have been unsuccessful in connecting LED tape to Arduino to Touch Designer like I had originally hoped to accomplish. Looking back now, it was a bit of a stretch in what I could do in just 15 weeks. However, I have been successful in other areas and I beleive have some of the framework down for accomplishing that task, but still some ways to go.
However, where I feel I have really succeeded is connecting the Arduino and Touch Designer platforms. I have been able to establish connection using two different methods (Standard Firmata and Serial) that is fairly stable and effective.
My goal is to take the values input by serial from Arduino and, in TouchDesigner, make them actually useable. This is one of the things I have not come across online. I know it is doable, because people in the forums I have looked at all discuss the possibilities that serial into TouchDesigner open up, but there are not tutorials on it. The closest thing I could find was this video uploaded by Isabelle Rousset on Vimeo. The video talks through the system that is taking inputs from Arduino and using them to manipulate a noise in Touch Designer, but doesn't actually show or name the nodes that are used. Still, this video was extremely encouraging and gave me hope the I could make this work.
Another website that I used to help me through this was this blog by dontjustexist. Since it had been a couple of weeks since I established a serial connection between Arduino and Touch Designer, it was a really good refresher on the process and showed me how it could be edited. This gave me the confidence to edit my code in ways that would benefit me without worrying that I was going to break the code.
Using these resources and what I have already learned from earlier in the semester AND the help of the all knowing google and derivative forum, I was able to convert the Arduino serial inputs into usable chop values.
First Step: Re-establish connection between Arduino and TouchDesigner.
This was a lot easier this time. Since I had already done it before, after a brief refresher using my own code and documentation and dontjustexist's blog post, connection was easy.
NOTE: sometimes Touch Designer doesn't always want to connect. However, if you turn the serial node off while you upload the code to the Arduino, and turn it back on, it should connect just fine. But I ran into an issue where it didn't always want to read the board when I did this. To gain the connection, I went back to Arduino and under the tools tab I re-selected the port the Arduino was plugged in to. It connected almost immediately.
Second Step: Edit the Arduino code so it reflects the values I want to see and need.
After setting ups he Arduino system so that it consists of two potentiometers and a photosensor, the code needs to reflect what is plugged into the board. This means coding the analogWrite codes that tell Arduino to grab these values.
Here, I have constrain and map code which affects the photosensor. The constrain goes the photosensor parameters to stick within, and the map, an inverse relationship, controls what level of brightness should be output due to the input from the sensor. It is mapping its values to each other.
Lastly, the serial information that is being printed must change to reflect the new values being received by the Arduino. In this case, the Serial.print command is printing a label for each of the inputs, and its value. This is what shows up in Touch Designer and what will be the basis for collecting values from serial. It is also shown in the Arduino's serial monitor as below.
Whenever the Arduino connects with Touch Designer, the serial reads become jumbled up. However, after a few minutes, it seems to catch up with itself and output values as it should.
Step Three: Convert the Serial input in TouchDesigner into Usable Values.
This was the most difficult part to figure out, because all I had to help me figure this part out was the video walkthrough posted by Isabelle Rousset on Vimeo. It gives a good description of the system, however, doesn't;t mention the exact nodes being used. I ended up going to google first, and then to the derivative website to try and figure out what nodes it was that they were using to convert these serial inputs into usable values.
Touch Designer's Operator Snippets ended up being the mot helpful when trying to gather this information. After going to the derivative website and discovering some new DATs that I though they might be using, the Operator Snippets showed me examples of how these nodes worked.
From here, I gathered that it wasn't an Evaluate DAT like I though might've been used, but a Convert DAT that took the Serial messages and converted them into Table information. By declaring delimiters, the node split up the values into separate cells. It took me a while to play with both Touch Designer and my Arduino code to find a method that worked best for what I wanted.
I started by sending out all of my outputs in a single line of serial code as seen to the left. However, I found that by editing my code by splitting into separate lines, it made the system in TouchDesigner be much simpler, because it didn't have to convert it in more ways then one.
What worked best ended up being...
- the values From Arduino being sent each in a new line of code
- using the Select DAT to select the last 3 rows being read
- using the Convert DAT to separate the name of the output and output value by using the equal sign ("=") as a delimiter.
This created a table that had the names of the input in the first column (in this case, A1, A2, and A3), and the outputs of each analog input in the second column.
Step Four: Take the data from the newly formed table and transform it into usable values.
By using a Datto CHOP, I am able to convert the DAT table into usable CHOP values. In order for it to translate properly, in the CHOP properties, we must specify which column it is we want to recall. In this case, it is column "1", since the very first column is named "0".
Then, to make it easier for myself, I plugged that CHOP into a Rename CHOP, and then into a Null CHOP which I will use to recall values in the future.
Proof of Concept
Using the input values from the Arduino, I have been able to create and editable visual as a proof of concept for the system I have created.
I created a visual TOP system in Touch Designer that uses the 3 inputs, two variable potentiometers and one button, to adjust different aspects of the visual pictured below.
Using the input values from the Arduino, I have been able to create and editable visual as a proof of concept for the system I have created. I switched the Photosensor to a button for the final iteration because it is much more reliable in its output values and worked with my TouchDesigner system more effectively.
Below is my final code that is being sent to Touch Designer.
I created a visual TOP system in Touch Designer that uses the 3 inputs, two variable potentiometers and one button, to adjust different aspects of the visual pictured below.
The button is controlling the LFO and determines when it is being played. When the button is pressed, it causes the color ramp to shift through its phases. This is demonstrated in the video to the left.
One of the two potentiometers is controlling the scale of the circle inside the Transform TOP. The Circle TOP is a part of a Feedback loop which causes the circle to leave behind traces of its past positions. By using the potentiometer the user has control over the size of the circle, thus controlling how big and how many circles appear.
The second Potentiometer is controlling the Exponent parameter of the Noise TOP. In this portion of the system, and LFO is controlling the seed and the Noise is part of another Feedback system. It is also joined with a Ramp Top in a Composite TOP to give it color, so it isn't monochromatic.
At the end of this system, all portions of the Touch Designer Container are joined into a single Composite TOP to create one single image that is editable by all 3 inputs.
One crucial and important aspect of this system, however, is taking the inputs from the Arduino board and making them work for each individual parameter. Since the button only switched between the values "0" and "1" wedon't have to convert it's values, since those values correspond to "on" and "off" for the LFO. But, since the potentiometers ranges from 0 to 1023, we have to change their values so that they react in an expected way.
After dividing the 3 inputs using Select CHOPs, I fed the two Potentiometers each into a Math CHOP. The Math CHOP takes the input value range and outputs a new range of the programmers choosing. It works very much like the Map code function does for the Arduino. It changes the inputs' values into something that coincided with what it is being used for or with. In this case, the Math CHOP connected to the Potentiometer affecting the Circle changes from "0 to 1023" to "0 to 1" to reflect the limitations of the Scale parameter in this instance. The Potentiometer affecting the Noise Exponent changes from "0 to 1023" to "0 to 4" to reflect the boundaries of the Exponent parameter.
Below is a video showing the System's full capabilities.
In Conclusion
Looking back on this semester, I'm really glad that I chose to do Independent Study instead of Senior Design. And not just because of the whole Covid situation, although I am grateful that I didn't have to complete do a 180 with my projects like I know many people have had to do. This semester gave me the opportunity to dive into subjects I've been wanting to learn for a while, but just hadn't had the time for previously. Namely Arduino, physical computing, and learning more about coding.
One thing I've learned throughout this semester is that coding is a complete language that is really difficult to understand if you dont know the basics. But knowing the basics helps me to pick up context clues throughout a code, an much like foreign languages, I think I'm better at understanding what a code is trying to do rather than writing It myself. I still struggle with certain types of code and the uses of libraries, booleans, and strings. I hope that I do continue to become better at coding and that with time and practice, I will be able to code as easily as the people who's tutorials I watched countless times.
Even though I was unable to accomplish one of the tasks I had hoped, I still think this semester was successful. I was hoping to connect Touch Designer, Arduino and LED Tape and this goal created a direction for me in the content and programs I was learning. A little more than halfway through the semester is when I realized connecting all three technologies was going to be much harder than originally anticipated and when I started to focus my efforts on just Arduino/Touch Designer and Arduino/LED Tape. I still think it's possible that I could accomplish that original task, but it's going to take a bit more time to gather all of the code and systems required.
However, I am still really happy with what I ended up creating and what I was able to accomplish. I know that had I not had this semester, I would probably still have very little knowledge about coding and no knowledge about Arduino. I was able to control a single LED with input from Touch Designer (sound AND visuals), and I was able to control visuals in Touch Designer with input from Arduino, and I think that's pretty cool. Now that I have this knowledge, I'm excited to play with it some more and see what else I can do with Arduino and Touch Designer as a single system.