This is Part I of a tutorial covering various methods for pulling external data into Processing. We’ll start with the basics of the java String class and end with an example that grabs from a mySQL database using PHP code into Processing. At times, I will simply refer to existing content from the Learning Processing book as well as this site’s PHP tutorial. As of now, this page is designed to cover the material for weeks 9 and 10 in ITP’s Introduction to Computational Media course, but ultimately it’s my hope that this material will make its way into a new edition of Learning Processing.

Here is Part II

The String class: who, what, where, when why

Strings are probably not unfamiliar to you. For example, if you’ve printed some text to the message window or loaded an image from a file, you’ve written code like so:

println("printing some text to the message window!");  // Printing a String
PImage img = loadImage("filename.jpg");                // Using a String for a file name

Where do we find documentation for the String class?

Although technically a Java class, because Strings are so commonly used, Processing includes documentation in its reference: http://www.processing.org/reference/String.html.

This page only covers some of the available methods of the String class. The full documentation can be found on java’s String page.

Displaying Text

We’re focused on learning about Strings for the purpose of gathering and parsing data. However, knowledge of Strings is also necessary for drawing text to the screen. We’re going to gloss over this here, but you can explore more by reading Chapter 17 in Learning Processing, some of which is also published on the Processing String tutorial page.

What is a String?

A String, at its core, is really just a fancy way of storing an array of characters. If we didn’t have the String class, we’d probably have to write some code like this:

char[] sometext = {'H', 'e', 'l', 'l', 'o', ' ', 'W', 'o', 'r', 'l', 'd'};

Clearly, this would be a royal pain in the Processing behind. It’s much simpler to do the following and make a String object:

String sometext = "How do I make String? Type some characters between quotation marks!";

Looking at the above, it might appear that a String is nothing more than a list of characters in between quotes. However, this is only the data of a String. We must remember that a String is an object with methods (which you can find on the reference page.) This is just like how in the Pixels tutorial a PImage stores both the data associated with an image as well as the functionality: copy(), loadPixels(), etc.

For example, the method charAt() returns the individual character in the String at a given index. Note that Strings are just like arrays in that the first character is index #0!

String message = "some text here.";
char c = message.charAt(3);
println(c);                // Results in 'e'

Another useful method is length(). This is easy to confuse with the length property of an array. However, when we ask for the length of a String object, we must use the parentheses since we are calling a function called length() rather than accessing a property called length.

String message = "This String is 34 characters long.";
println(message.length());

We can also change a String to all uppercase using the toUpperCase() method (toLowerCase() is also available).

String uppercase = message.toUpperCase();
println(uppercase); 

You might notice something a bit odd here. Why didn’t we simply say “message.toUpperCase()” and then print “message” variable? Instead, we assigned the result of “message.toUpperCase()” to a new variable with a different name — “uppercase”.

This is because a String is a special kind of object. It is immutable. An immutable object is one whose data can never be changed. Once we create a String, it stays the same for life. Anytime we want to change the String, we have to create a new one. So in the case of converting to uppercase, the method toUpperCase() returns a copy of the String object with all caps.

Finally, let’s look at equals(). It is technically possible to compare Strings with the “==” operator as follows:

String one = "hello";
String two = "hello";
println(one == two);

However, when “==” is used with objects, it compares the memory addresses for each object. Even though each String contains the same data — “hello”– if they are different object instances “==” could result in a false comparison. The equals() function ensures that we are checking to see if two String objects contain the exact same sequence of characters, regardless of where that data is stored in the computer’s memory.

String one = "hello";
String two = "hello";
println(one.equals(two));

Although both of the above methods return the correct result, it’s safer to use equals(). Depending on how String objects are created in a sketch, “==” will not always work.

One other feature of String objects is concatenation, joining two Strings together. Strings are joined with the “+” operator. With numbers, plus means add. When used with Strings, however, it means join.

String helloworld = "Hello" + "World";

Variables can also be brought into a String using concatenation.

int x = 10;
String message = "The value of x is: " + x;

Searching in Strings

Let’s take a closer look at the following two String functions: indexOf() and substring().

indexOf() locates a sequence of characters within a String. It takes one argument, a search String, and returns a numeric value that corresponds to first occurrence of the search String inside of the String being searched.

String search = "def";
String toBeSearched = "abcdefghi";
int index = toBeSearched.indexOf(search);
println(index);

The above code will print ’3′ to the message console. Note that Strings are just like arrays, in that the first character is index number zero and the last character is the length of the String minus one. If the search String cannot be found, indexOf() returns a negative one. This is a good choice because “-1” is not a legitimate index value in the String itself and therefore can indicate “not found.” There are no negative indices in a String of characters or in an array.

After finding a search phrase within a String, we might want to separate out part of the String, saving it in a different variable. A part of a String is known as Substring. Substrings are made with the substring() function which takes two arguments, a start index and an end index. substring() returns the substring in between the two indices.

String alphabet = “abcdefghi”;
String sub = alphabet.substring(3,6);

Note that the substring begins at the specified start index (the first argument) and extends to the character at end index (the second argument) minus one. I know. I know. Wouldn’t it have been easier to just take the substring from the start index all the way to the end index? While this might initially seem true, it’s actually quite convenient to stop at end index minus one. For example, if you ever want to make a substring that extends to the end of a String, you can simply go all the way to thestring.length(). In addition, with end index minus one marking the end, the length of the substring is easily calculated as end index minus begin index.

Splitting and Joining Text

Processing has two additional functions that make joining Strings (or the reverse, splitting them up) easy. In sketches that involve parsing data from a file or the web, we will often be presented that data in the form of an array of Strings or as one long String. Depending on what we want to accomplish, it’s useful to know how to switch between these two modes of storage. This is where these two new functions, split() and join(), will come in handy.

Let’s take a look at the split() function. split() separates a longer String into an array of Strings, based on a split character known as the delimiter. It takes two arguments, the String to be split and the delimiter. (The delimiter can be a single character or a String).

// Splitting a String based on spaces
String spaceswords = "The quick brown fox jumps over the lazy dog.";
String[] list = split(spaceswords, " ");
for (int i = 0; i < list.length; i++) {
  println(list[i] + " " + i);
}

Here is an example using a comma as the delimiter.

// Splitting a String based on commas
String commaswords = "The,quick,brown,fox,jumps,over,the,lazy,dog.";
String[] list = split(commaswords, ',');
for (int i = 0; i < list.length; i++) {
  println(list[i] + " " + i);
}

If you want to use more than one delimiter to split up a text, you must use the Processing function splitTokens(). splitTokens() works identically to split() with one exception: any character that appears in the String qualifies as a delimiter.

// Splitting a String based on a multiple delimiters
String stuff = "hats & apples, cars + phones % elephants dog.";
String[] list = splitTokens(stuff, " &,+.");
for (int i = 0; i < list.length; i++) {
  println(list[i] + " " + i);
}

If we are splitting numbers in a String the resulting array can be converted into an integer array with Processing’s int() function.

// Calculate sum of a list of numbers in a String
String numbers = "8,67,5,309";
// Converting the String array to an int array
int[] list = int(split(numbers, ',')); 
int sum = 0;
for (int i = 0; i < list.length; i++) {
  sum = sum + list[i];
}
println(sum);	

The reverse of split() is join(). join() takes an array of Strings and joins them together into one long string. The join() function also takes two arguments, the array to be joined and a separator. The separator can either be a single character or a String of characters.

Consider the following array:

String[] lines = {“It”, “was”, “a”, “dark”, “and”, “stormy”, “night.”};

Using the + operator along with a for loop, we can join a String together as follows:

// Manual Concatenation
String onelongstring = "";
for (int i = 0; i < lines.length; i++) {
  onelongstring = onelongstring + lines[i] + “ “;
}

The join() function, however, allows us to bypass this process achieving the same result in only one line of code.

// Using Processing’s join()
String onelongstring = join(lines,” “);

Data into Processing with Strings!!

A first start: text from user

We saw how Strings can be joined together (referred to as “concatenation”) using the “+” operator. Let’s take a look at an example that uses concatenation to get user input from a keyboard: Example 18-1: User Input.

Reading and Writing Text Files: loadStrings()

Data can come from many different places: web sites, news feeds, databases, etc. And we’re going to see examples of these different sources throughout this tutorial. Nevertheless, when developing an application that involves a data source, such as a data visualization, it’s important to separate out the logic for what the program will ultimately do with the data from the retrieval of the data itself.

In fact, while working out the visuals, it’s especially useful to develop with “dummy” or “fake” data. In keeping with our one-step-at-a-time mantra, once the meat of the program is completed with dummy data, you can then focus solely on how to retrieve the actual data from the real source.

We’re going to follow this model in this section, working with the simplest means of data retrieval: reading from a text file. Text files can be used as a very simple database (we could store settings for a program, a list of high scores, numbers for a graph, etc.) or to simulate a more complex data source.

In order to create a text file, you can use any simple text editor. Windows Notepad or Mac OSX TextEdit will do, just make sure you format the file as “plain text.” It is also advisable to name the text files with the “.txt” extension, just to avoid any confusion. And just as with image files, these text files should be placed in the sketch’s “data” directory in order for them to be recognized by the Processing sketch.

Once the text file is in place, Processing’s loadStrings() function is used to read the content of the file into an String array. The individual lines of text in the file each become an individual element in the array.

String[] lines = loadStrings("file.txt");
println("there are " + lines.length + " lines");
for (int i=0; i < lines.length; i++) {
  println(lines[i]);
}

To run the code, create an text file called “file.txt”, type a bunch of lines in that file, and place it in your sketch’s data directory.

Working with numbers

Text from a data file can be used to generate a simple visualization. Let’s consider a text file that looks like so:

131,85,87,16,169,140,153,72,115,141

The goal here is to turn the above into an array of integers.

int[] data;

You know you can load the text file using loadStrings().

String[] stuff = loadStrings("data.txt");

But there’s only one line here, so all you care about is stuff[0], which you want to split up with the comma as a delimiter.

String[] numbers = split(stuff[0],',');

Finally, you can then convert it to integers:

data = int(numbers);

And use a loop to draw something according to all the numbers.

background(255);
stroke(0);
for (int i = 0; i < data.length; i++) {
  fill(data[i]);
  rect(i*20,0,20,data[i]);
}

Full example: http://www.learningprocessing.com/examples/chapter-18/example-18-2/

Using a text file to load data into an object: http://www.learningprocessing.com/examples/chapter-18/example-18-3/

Saving Data to a text file

saveStrings() is essentially the opposite of loadStrings(), receiving a filename and a String array and saving that array to the file.

String[] stuff = {"Each String", "will be saved", "on a", "separate line"};
saveStrings("data.txt", stuff);

This can be used to write a Processing sketch that will pick up where it last left off. For example, take the following sketch which draws a square every time you click the mouse.

void setup() {
  size(200,200);
  background(255);
  smooth();
}

void draw() {
}

void mousePressed() {
  fill(0);
  rectMode(CENTER);
  rect(mouseX,mouseY,16,16);
}

Every time you run the sketch, it starts with a blank window. But what if, after you add several squares, you could quit the sketch, restart it and pick up where you last left off? This can be done using saveStrings(), however, saveStrings() replaces the entire text file each time you call it. So in this case, it would be more convenient to append to a text file. This can be done with PrintWriter.

PrintWriter output = createWriter("data/data.txt");
output.println("Writing a line to the file.");
output.flush();  // Actually writes the bytes to the file
output.close();  // Closes the file, can't write any more

So if we want to write mouse coordinates to a file, we can say:

output.println(mouseX + "," + mouseY); // Write the coordinate to the file

We must always remember to flush() the data and close() the file. We can do this in Processing’s stop() function. stop() is the opposite of setup() in the sense that it is called the moment the sketch quits.

// This will get called when the sketch quits
void stop() {
  output.flush();  // Flush all the data to the file
  output.close();  // Close the file
}

The data file will look like so:

46,93
160,66
171,151
103,156
103,50
50,75
122,127
170,118
155,94
131,81

If we want our sketch to then read in the data, we can simply loop through all the lines, split each up with a comma and draw rectangles!

// Read all the lines from the file and convert to two integers (xy coordinate)
for (int i = 0; i < positions.length; i++) {
  int[] xy = int(split(positions[i],","));
  fill(0);
  rectMode(CENTER);
  rect(xy[0],xy[1],16,16);
}

Here is the full code:

PrintWriter output;

void setup() {
  size(200,200);
  background(255);
  smooth();
  // Load everything from the file
  String[] positions = loadStrings("data.txt");
  // Create a print writer (will clear the file)
  output = createWriter("data/data.txt"); 

  // It might be null the first time we run the program if there is no data.txt file!
  if (positions != null) {
    // Read all the lines from the file and convert to two integers (xy coordinate)
    for (int i = 0; i < positions.length; i++) {
      int[] xy = int(split(positions[i],","));
      fill(0);
      rectMode(CENTER);
      rect(xy[0],xy[1],16,16);
      // We have to write the data out to the file again
      output.println(xy[0] + "," + xy[1]);
    }
  }

}

void draw() {
}

void mousePressed() {
  // Draw a new rectangle and write it out
  fill(0);
  rectMode(CENTER);
  rect(mouseX,mouseY,16,16);
  output.println(mouseX + "," + mouseY); // Write the coordinate to the file
}

void keyPressed() {
  // Create a print writer (will clear the file)
  output = createWriter("data/data.txt"); 
  background(255);

}

// This will get called when the sketch quits
void stop() {
  output.flush();  // Flush all the data to the file
  output.close();  // Close the file
}

Download example: SavePoints.zip

  • Petar Doychev

    Daniel, the code creates file “/data/data.txt” , but nothing writes inside. Just doesn’t work.
    Tested in Ubuntu 12.04 and Windows XP

  • Petar Doychev

    When I moved only output.flush() from body of stop() to the body of

    mousePressed() worked , bud just for 1 cycle.

  • shiffman

    it’s working for me now if i move just “output.flush()” to mousePressed(), but not output.close(). I think stop() is reliable and so the file is not getting closed properly. But on Mac OS it doesn’t seem to care. Does this work for you?

  • Petar Doychev

    Yes, exactly.
    I noticed, on my sketches, the best plays for PrintWriter.flush() is right after PrintWriter.println(). Haven’t check on Mac OS.
    Thanks for your answer and for your book, I have learned a lot.

  • muhammad usman

    peter please show how to “input integer from user in processing “