// Learning Processing
// Daniel Shiffman
// http://www.learningprocessing.com

// Example 16-12: Simple background removal

// Click the mouse to memorize a current background image
import processing.video.*;

// Variable for capture device
Capture video;

// Saved background
PImage backgroundImage;

// How different must a pixel be to be a foreground pixel
float threshold = 20;

void setup() {
  video = new Capture(this, width, height, 30);
  // Create an empty image the same size as the video
  backgroundImage = createImage(video.width,video.height,RGB);

void draw() {
  // Capture video
  if (video.available()) {
  // We are looking at the video's pixels, the memorized backgroundImage's pixels, as well as accessing the display pixels. 
  // So we must loadPixels() for all!
  // Begin loop to walk through every pixel
  for (int x = 0; x < video.width; x ++ ) {
    for (int y = 0; y < video.height; y ++ ) {
      int loc = x + y*video.width; // Step 1, what is the 1D pixel location
      color fgColor = video.pixels[loc]; // Step 2, what is the foreground color
      // Step 3, what is the background color
      color bgColor = backgroundImage.pixels[loc];
      // Step 4, compare the foreground and background color
      float r1 = red(fgColor);
      float g1 = green(fgColor);
      float b1 = blue(fgColor);
      float r2 = red(bgColor);
      float g2 = green(bgColor);
      float b2 = blue(bgColor);
      float diff = dist(r1,g1,b1,r2,g2,b2);
      // Step 5, Is the foreground color different from the background color
      if (diff > threshold) {
        // If so, display the foreground color
        pixels[loc] = fgColor;
      } else {
        // If not, display green
        pixels[loc] = color(0,255,0); // We could choose to replace the background pixels with something other than the color green!

void mousePressed() {
  // Copying the current frame of video into the backgroundImage object
  // Note copy takes 5 arguments:
  // The source image
  // x,y,width, and height of region to be copied from the source
  // x,y,width, and height of copy destination
  • Pandean


    great example, look awesome

    I wonder how to make the green background completely disappear. I tried to play with alpha attribute a couple of times, but no luck yet.

    looking forward to hear with you


  • http://www.learningprocessing.com Daniel Shiffman

    If you want to create the “illusion” of the background disappearing, you can simply replace the green pixels with whatever you want to see “behind” the image. You could also consider writing the pixel data to another PImage object (rather than the display window itself), in which case you could have an ARGB image and set alpha values.

  • Pandean

    Hi Daniel

    I tried to use ARGB before.. It didn’t work, so I tried to use noFill(); magically, it worked.

    Thank you

  • John T.

    What if you wanted to replace the background, not with a static image but rather a constant changing set of objects?

  • http://www.learningprocessing.com Daniel Shiffman

    If you are drawing stuff, you can simply draw everything first and then skip the background pixels entirely or write to a separate PImage object with alpha and set background pixels to alpha of 0.

  • David

    Hi Daniel,

    Thankyou for the great book. I am attempting a project that requires background removal from the webcam, but I am finding it difficult finding a webcam that doesn’t auto adjust the brightness/exposure. I am using a macbook with an isight at the moment, but as you probably know, the brightness auto adjusts. Can you recommend a basic webcam that is suitable?

  • http://twitter.com/shiffman Daniel Shiffman

    I wish I had a good suggestion, I like using the PSEye camera, but not sure if you can turn off automatic settings with macam (you need the macam driver). You may want to just use a DVcam that has lots of settings available on the camera itself.

  • Davidleefoster

    I have been doing some research and found a macam hack for the PSeye over on the openframeworks board!. The guys there have generously updated the driver so the auto adjustments are set to manual. It works for processing. Here’s the link: http://forum.openframeworks.cc/index.php/topic,1182.75.html

    Look for posts by Theo, he has posted numerous iterations of the hack on this thread.

  • Nate

    The camera is not initializing for some reason. I’ve tried looking at other code to see if it will fix it but no luck. Any ideas?

  • Luis Jose Salazar

    Hi Nate,

    Add this line…
    video = new Capture(this, width, height, 30);
    >> video.start();

    Hope it helps!!



  • kitty_viewviie

    I want to pull foreground out of the background
    Can I do it and how?


  • https://www.facebook.com/viewviie kitty_viewviie

    Hi Daniel,
    I want to pull foreground out of the background.
    Can I do it and how?

    Thank you

  • Surya Wöhrle

    Hello Daniel,
    this is really a great example! I was searching for background removal because I need it in a project and was really happy to find your website.
    I have still one question. For this specific project I need the picture with removed background as a png with transparent background. So no green background but no background (I need to put something behind the picture that is moving and should be seen).
    Is it possible to to this? I asked one teacher and he told me to use the alpha channel but this doesn’t work. I also asked a friend of mine who is quite good in processing (I’m just a beginner) and he also didn’t know what to do.
    I hope you can understand my problem (my Englisch is not the best…) and perhaps give me some advice :)
    Thank you!

  • shiffman

    I would try asking this question at forum.processing.org. I think you may be able to do this with createGraphics().

  • Casey Scalf

    If you have a Mac you can use this program, Webcam Settings. It will work with nearly every camera and allow you to adjust the settings and also turn on/off auto brightness for a USB camera using Processing.

    So, I would be there IS a way to interface with a camera. Since this program can do it with many many different cameras there must be some sort of universal protocol.

    So, I think that could possibly work in Processing. Especially if you can send raw bytes. I think this would be best so that *another* app had to be opened; such as Webcam Settings.

    It would be nice to dial that in!