This is Part II of a draft tutorial that covers how to get external data into Processing. Part I started with the very basics of how to work with the java String class and covered text parsing and loading from and saving to text files. In Part II, we’ll move onto the basics of PHP and see how we grab data from and save data to a server, ending with an example of how to pull from a mySQL database using PHP code into Processing. At times, I will simply refer to existing content from the Learning Processing book as well as this site’s PHP tutorial. As of now, this page is designed to cover the material for weeks 9 and 10 in ITP’s Introduction to Computational Media course, but ultimately it’s my hope that this material will make its way into a new edition of Learning Processing.

Why PHP? Why a server?

We saw in the Save Points example that you can save data while a Processing sketch is running and load it later. If you are using Processing on its own, however, you can only save data locally to the machine you are working on. What if you want multiple instances of the same sketch running to share a data file? You would need to save data to and load data from a machine that anyone can access — for example, a web server.

Even when a sketch is run as a web applet, it always runs locally on your machine (and therefore is subject to many restrictions, see: Sandbox.) If you write a server-side program, one that executes directly on the server, it can read/write data on that server, as well as connect to other network-based data sources.

This is where PHP comes in. PHP (which originally stood for “Personal Homepage”, but now stands for “Hypertext Preprocessor”) is a scripting language designed for creating dynamic web pages. If you write a PHP script, you are writing a little program that runs on your server. Traditionally, the output of that program would be HTML which you would view from your web browser. For us, however, Processing is going to be our front-end view. And so our PHP programs will just spit out plain text that we’ll read into Processing.

To write a PHP script, simply:

  • Make sure your web server supports PHP. (If you use the ITP server, you’re all set!)
  • Create a file with the PHP extension, i.e. “index.php” or “mypage.php”
  • Write your PHP code in between: <? PHP HERE ?>

A php script is just a plain text file. You can use any text editor to write the PHP code (Notepad on the PC and TextEdit on the mac will do just fine!). You might consider using TextWrangler which will do syntax highlighting for you. I am also a fan of Coda, but it’s not free.

Once you’ve written a script, you’ll want to upload it to a server. If you are using, say, itp.nyu.edu, then look for your “public_html” directory and place it in there (or some subdirectory you create). Let’s say you put it here:

netid/public_html/phpstuff/myscript.php

You would then be able to test your PHP script with the URL: http://itp.nyu.edu/~netid/phpstuff/myscript.php.

Ok, time to get started writing some PHP. Here’s an example displays a random number between 0 and 10 each time you execute it: rand.php.

A random number: <? echo rand(0,10); ?>

Notice how we can execute this code by typing in the URL path to the PHP script in our browser and view the results.

http://www.learningprocessing.com/php/rand.php

This is useful for testing (and the primary way you’d use PHP if you were developing a web site). For us, however, we want to view the results in Processing using loadStrings().

size(200,200);
background(255);
String[] php = loadStrings("http://www.learningprocessing.com/php/rand.php");
fill(0);
text(php[0],10,100);

“echo”, by the way, in PHP-speak means print. rand(0,10) is a call to a function that returns a random number between 0 and 10. This is similar to random() in Processing. The documentation for this function can be found on php.net: http://us2.php.net/manual/en/function.rand.php.

The Fundamentals in PHP

Now that we have a basic sense of why PHP exists in the first place and how we integrate PHP code with Processing, we can arrive at the happy moment where we see that all that fundamental programming stuff we learned with Processing applies to PHP (and pretty much all programming languages) too! (Also, in the example below, notice how “//” also indicates a comment in PHP.)

Variables

Differences with Processing:

  • All variables in PHP must start with a dollar sign: $
  • PHP does not require that you declare the data type for your variable!

Example:

<?
// Declaring and initializing variables (no need to specify a data type!)
$x = 5;
$xspeed = 4.23;
$x = $x + $xspeed;
echo $x;
// Two strings
$name = "Zoog";
$title = "Cosmonaut";

echo $title . " " . $name;
?>

Also, notice how in the above code a period — ‘.’ — is used to concatenate two Strings. In Java/Processing, the plus — ‘+’ — operator is used.

Conditionals

Conditionals (if, else if, else) are exactly the same as in Processing!

<?
$x = rand(0,10);
// Basic conditional in PHP
if ($x > 5) {
  echo $x . " is greater than 5";
} else {
  echo $x . " is less than or equal to 5";
}
?>

Loops

Loops (for, while) are exactly the same as in Processing!

<?
// Loop in PHP
for ($i = 0; $i < 10; $i++) {
  echo $i;
}
?>

Arrays

There are several differences with how arrays work in PHP:

  • Again, just like with variables, you are not required to specify the the data type for an array.
  • You do not have to explicitly declare the size of the array in advance! This makes it especially easy to just add elements to the end of an array.
  • PHP allows for a special “for each” syntax to loop through an array.
  • PHP allows for something called an associative array. An associative array is a list of values where each value is referenced by a “key” (String) rather than an index (integer). In other words, instead of numbering the elements of your array from 0 to 9 (10 elements total), you can name the elements of your array. Array element “fred”, array element “sue”, array element “bob”, etc. There are certain instances where this is especially convenient. Note that you can also do this in Processing, with the use of a HashMap.
  • You can get the size of an array with the sizeof() function

Here is some example array code in PHP that covers the above items:

<?
// Creating an array
$nameList = array("Joe","Jane","Bob","Sue");
echo $nameList[2];

// Adding an element to the end of that array
$nameList[] = "Zoog";
echo $nameList[4];

echo "<br><br>";

// Create an array with a loop
for ($i = 0; $i < 10; $i++) {
  $nums[$i] = rand(0,10);
}

// Using foreach syntax
foreach ($nums as $value) {
  echo $value . " " ;
}

// Getting the size of an array
echo "<br><br>The size of the array is " . sizeof($nums);

// An associative array
$studentIDs["Joe"] = 23;
$studentIDs["Jane"] = 41;
?>

Functions

You can write your own function in PHP:

// Defining a function
function printDate() {
 echo "The Date and Time is: ";
 $today = date(DATE_RFC822);
 echo $today;
}

Objects

You can also write classes and make objects in PHP, with some differences:

  • Variables must be declared as public or private
  • The constructor is always named: “__construct”
  • Instance variables need to be referred to as: “$this->varname”.
<?
// Defining a class
class Human {
  // Declaring instance variables
  public $firstname = "default";
  public $height = 0;
  
  // Writing a constructor
  function __construct($n) {
    // Notice the use of $this->varname
    $this->firstname = $n;
    $this->height = rand(5,6);
  }
   
  // A function
  function show() {
    echo "<br><br>" . $this->firstname . " is " . $this->height . " feet tall.";
  }
}
?>

Creating an object and calling its methods is easy, the only difference is the use of the “->” syntax — i.e. object->function() — instead of what we are used to in java: object.function().

<?
// Creating an object
$frank = new Human("Frank");
// Calling a function on the object
$frank->show();
?>

Here is sample code for all of the above fundamental programming concepts: fundamentals.php.txt.

Execute the script: fundamentals.php

PHP and the Query String

A very common use of PHP is to process input via a URL’s Query String. (In the land of web development the query string will often be tied to an HTML form, but we’ll be writing them ourselves via Processing.)

Let’s take a look at a query string:

http://www.someserver.com/somephpscript.php?mood=happy&time=night&count=5

The above URL path has three “name/value” pairs (also called field/value pairs). We’re telling the PHP script that we want it to run with a mood of happy, a time of night, and a count of 5. It’s like setting some intial variables for the script to run with. PHP can then ask for these “variables” using the built-in array: $_GET. $_GET is technically an associative array, and the values can be retrieved by referencing the name, i.e.:

$mood = $_GET["mood"];
$time = $_GET["night"];
$count = $_GET["count"];

Here is a simple example where the resulting output depends on the query string: query.php, source: query.php.txt.

<?
  // Get the stuff from the query string
  $name = $_GET["name"];
  $mood = $_GET["mood"];
    
  echo $name . ", ";
    
  // Based on those values, pick an output
  if ($mood == "happy") {
    echo " I am jealous of your happiness.";
  } else {
    echo " why so glum?";
  }
?>  

Try manipulating the query string yourself and see what results you get.

Saving and Loading Data with PHP

One of the great things we can use PHP for is persistence. Since a PHP script is executed on the server (as opposed to a Processing applet which runs client side), it can read and write data directly to and from the server rather easily. The first thing we need to do is find built-in PHP functions that allow us to read from and write to a file. All of the built-in functions available to you in PHP (we’ve seen a few so far: date(), rand()) can be found by searching php.net. The functions we need for file I/O are:

  • fopen() — opens a file (or a url). When you open the file, you should declare your intentions, i.e. do you plan on reading, writing a new file, appending to an old file, etc. To write to a file, you pass in the argument: ‘w’, to append: ‘a’, read-only: ‘r’, etc.
  • fread() — returns the content of a file
  • fwrite() — writes content to a file
  • filesize() — returns the size of a file
  • fclose() — closes a file (should be called when you are finished reading or writing)

Using the above built-in functions, we can write a generic function to write to a file:

// Function to write to file
function writeData($filename,$content){
  $dest = fopen("$filename", 'w') ;
  fwrite ($dest,$content);
  fclose($dest);
}

And one to read from a file:

// Function to read from file
function getData($filename){
  $size = filesize ($filename);
  $src = fopen ($filename, 'r');
  $content = fread ($src,$size);
  fclose ($src);
  return $content;
}

A word of caution: anyone can access and modify the data on the server via GET. Anything passed to php via a url query string leaves open the potential for mischief and you should take appropriate cautions, such as limited file reads / writes to a certain directory, etc.

Using PHP for Persistence with Processing

Let’s consider a PHP script that can do three things:

  • Read from a text file and write out the content
  • Append an (x,y) coordinate to a text file
  • Clear the text file of all content

The first thing we want to do is structure our PHP script to receive a command in the form of a URL query, i.e. these three queries will result in a different action:

  • loadstrings.php?type=load
  • loadstrings.php?type=save&x=100&y=100
  • loadstrings.php?type=clear

We can structure the PHP script to have a simple conditional based on the value of “type.” In the code below, we first check to see if type has been passed in via the query string at all. We ask if $_GET['type'] “is set,” using isset().

if (isset($_GET['type'])) {
  $type = $_GET['type'];
} else {
  $type = "null";
}

Next, based on the value of $type, we can execute different operations:

if ($type == "save") {
  // SAVE CODE GOES HERE
} else if ($type == "clear") {
  // CLEAR CODE GOES HERE
} else if ($type == "load") {
  // LOAD CODE GOES HERE
} else {
  echo "This PHP script requires a query string!";
}

Processing can then use loadStrings() to ask the PHP script to perform various tasks. For example, saving a coordinate when the mouse is clicked:

void mousePressed() {
  String[] lines = loadStrings("http://www.learningprocessing.com/php/loadstrings.php?type=save&x=" + mouseX + "&y="+mouseY);
}

Or clearing the file when a key is pressed:

void keyPressed() {
  String[] lines = loadStrings("http://www.learningprocessing.com/php/loadstrings.php?type=clear");
}

And, of course, loading the current text in the file and drawing something based on the data in that file:

String[] lines = loadStrings("http://www.learningprocessing.com/php/loadstrings.php?type=load");
for (int i = 0; i < lines.length; i++) {
  int[] vals = int(split(lines[i],","));
  rectMode(CENTER);
  stroke(0);
  fill(175);
  rect(vals[0],vals[1],10,10);
}

Check out the full example with PHP source here:

http://www.learningprocessing.com/tutorials/php/php-persistence/

PHP and Web Services

You can also use PHP as an intermediary between you and some other data available on the web. There are too many possibilities here to count from flickr to vimeo to twitter to facebook to weather to stock quotes to news articles to itp projects, etc. Let’s just take one example: Twitter. Now, I should point out that you don’t need PHP to access Twitter. There is a Java API for twitter (Twitter4j) and you can also access Twitter’s XML feed directly in Processing. Still, there are a number of advantages for using PHP as an intermediary. It is a highly modular way of working, you can develop in Processing with a dummy text file and only later connect the data (with your PHP outputting an identical format as your TXT file.) In addition, a lot of web services support web technologies more readily in their APIs (like PHP) than Java.

For Twitter, we’re going to use a nice PHP example by Ryan Faerman. First you need to grab his source here: http://greenservr.com/projects/twittersearch/TwitterSearch.phps and create a file called TwitterSearch.php. Ryan has written a class that will perform twitter searches for you. To use it, you just need to make sure you add an include statement (much like a Processing “import” statement with a library.)

include 'TwitterSearch.php';

Then you can create a TwitterSearch object:

$search = new TwitterSearch();
$search->user_agent = 'phptwittersearch:daniel.shiffman@gmail.com';
$search->contains('#Processing');

It’s also good practice to set your user_agent so that Twitter knows who is accessing their API (and contact you if there’s a problem.) Then you can add a search query using the contains() function. In this case, we’re searching for the hashtag #Processing.

Finally, we can request the search results (which returns an array) and loop through them.

$results = $search->results();
// Use a loop to see all the Tweets
// You can also look at time, user profile pic, id, geo info, etc.
foreach ($results as $tweet) {
  echo $tweet->text . "\n";
}

This is great, but it is missing a key component. All we can ever do with this PHP script is get tweets about Processsing. What if we want to get tweets about ITP or Justin Bieber? Add a query string!

$query = $_GET['query'];
$search->contains($query);

Now, you can look at some tweets!

http://www.learningprocessing.com/php/twitter/searchtweets.php?query=Processing.

http://www.learningprocessing.com/php/twitter/searchtweets.php?query=ITP.

Download the source here: twitter_php.zip.

Now we’re ready for Processing to read in the tweets using loadStrings()!


// An array of strings
String[] tweets;

void setup() {
  size(600,400);
  // Filling the array from our PHP script
  tweets =  loadStrings("http://www.learningprocessing.com/php/twitter/searchtweets.php?query=ITP");
}

void draw() {
  background(255);
  // Drawing all the strings
  for (int i = 0; i < tweets.length; i++) {
    fill(0);
    text(tweets[i],10,20+i*20);
  }
}

One thing that is important to note, there are certain characters you can’t use in a query string, such as SPACE or ‘#’. To solve this you must encode them properly, according to URL Encoding rules. For example, for space you would write %20 instead. So here are your tweets about Justin Bieber!

http://www.learningprocessing.com/php/twitter/searchtweets.php?query=Justin%20Bieber.

XML Parsing and PHP and Processing

For grabbing data from the web, an XML (Extensible Markup Language) feed is an excellent option (if available). Unlike HTML (which is designed to make content vieweable by a human’s eyes) XML is designed to make content viewable by a computer and facilitate the sharing of data across different systems.

XML organizes information in a tree structure. Let’s imagine a list of students. Each student has an id number, name, address, e-mail, and telephone number. Each student’s address has a city, state, and zip code. An XML tree for this dataset might look like the following:


<?xml version="1.0" encoding="UTF-8"?>
<students>
  <student>
    <id>001</id>
    <name>Daniel Shiffman</name>
    <phone>555-555-5555</phone>
    <email>daniel@shiffman.net</email>
    <address>
      <street>123 Processing Way</street>
      <city>Loops</city>
      <state>New York</state>
      <zip>01234</zip>
    </address>
  </student>
  <student>
    <id>002</id>
    <name>Zoog</name>
    <phone>555-555-5555</phone>
    <email>zoog@planetzoron.uni</email>
    <address>
      <street>45.3 Nebula 5</street>
      <city>Boolean City</city>
      <state>Booles</state>
      <zip>12358</zip>
    </address>
  </student>
</students>

Note the similarities to object-oriented programming. We could think of the XML tree in the following terms. The XML document represents an array of student objects. Each student object has multiple pieces of information, an id, a name, a phone number, an e-mail address, and a mailing address. The mailing address is also an object that also has multiple pieces of data, such as street, city, state, and zip.

PHP has a nice set of classes and functions for parsing XML: SimpleXML. (Processing can do it too directly: XMLElement). Take a look at the NY Times XML feed for “Most E-mailed Articles”:

http://www.nytimes.com/services/xml/rss/nyt/pop_top.xml.

Here is some PHP code that prints out (on separate lines) each headline:

<?
// Get the RAW XML
$getXML = file_get_contents("http://www.nytimes.com/services/xml/rss/nyt/pop_top.xml");
// Pass the RAW XML to the SimpleXMLElement parser
$xml = new SimpleXMLElement($getXML);
// Print out every item element's headline, i.e. "title" 
foreach ($xml->channel->item as $item) {
   echo $item->title . "\n";
}
?>

Take a look here: http://www.learningprocessing.com/php/xml/nytimes_parser.php

And just like before, we can load and display in Processing!

// An array of strings
String[] headlines;

void setup() {
  size(600,300);
  // Filling the array from our PHP script
  headlines =  loadStrings("http://www.learningprocessing.com/php/xml/nytimes_parser.php");
}

void draw() {
  background(255);
  // Drawing all the strings
  for (int i = 0; i < headlines.length; i++) {
    fill(0);
    text(headlines[i],10,20+i*20);
  }
}

Database and PHP and Processing

The last example we’ll look at here is accessing information from a database. A database is simply a repository for data, organized as a collection of tables, each table having a list of records, each record having several fields. For example, the “ITP” database might have a table called “Students”. The table would have a record for each ITP student (~200 total) and each record would have fields like name, ID#, e-mail address, etc. Database can be “flat” (like a spreadsheet) or “relational” (tables have relationships to each other).



A nice example database for us to start with is the ITP people database. It uses MYSQL, a very popular open source database to store information. While you can access a mysql database directly in Processing, yet again, we’re going to look at a basic example that uses PHP as a layer between Processing and the MYSQL database.



We’re not going to get into the details of how databases work here and the syntax for making MYSQL queries on a database. If you are interested in that stuff, I suggest you start with: Wikipedia’s page on SQL.



The pieces you need to do this are all available on this wiki page:



http://itp.nyu.edu/varwiki/ClassWork/ICM-People-Database.



It’s password protected so that only ITP students have access.

The Sandbox!

Don’t forget. If you are publishing your Processing sketch to the web, there are security restrictions. You’ll be fine if you upload your applet to the same server that houses your PHP script (i.e. if you put both on itp.nyu.edu). But if they live in different places, be careful, you could run into a problem! For more, visit: http://www.learningprocessing.com/tutorials/sandbox/.

Errors

PHP may not be set up to show all errors. You can add the following two lines to enable error output while you are developing:

ini_set('error_reporting', E_ALL);
ini_set('display_errors', true); 
  • Guest

    “It’s password protected so that only ITP students have access.”
    why god, why.

  • shiffman

    Sorry, the example contains private information for ITP only.   To output a sql query from PHP, something simple like:

    $results = sqlQuery($query);
    for ($i = 0; $i < sizeof($results); $i++){etc.will get you started