Section 14.2. Working with Video Frames

[Page 494 (continued)]

14.2. Working with Video Frames

As we said earlier, dealing with real video, in real-time, is very hard. We're going to cheat by saving the video as a sequence of JPEG images, manipulate the JPEG images, then convert back into a movie.

To manipulate movies that already exist, we have to break them into frames. The MediaTools application can do that for you (Figure 14.7) as can tools like Apple's QuickTime Pro. The Menu button in the MediaTools application lets you save any MPEG movie as a series of JPEG frame pictures.

Figure 14.7. Movie tools in MediaTools. (This item is displayed on page 495 in the print version)

14.2.1. Video-Manipulating Examples

On your CD is a brief movie of our daughter Katie dancing around. Let's create a movie of Mommy (Barb) watching her daughterwe'll simply composite Barb's head onto the frames of Katie dancing (Figure 14.8).

[Page 495]

Figure 14.8. Frames from the Mommy watching Katie movie.

Program 137. Make Movie of Mommy Watching Katie (This item is displayed on pages 495 - 496 in the print version)

/**  * Method to make a movie of Barb's head moving  * and Katie dancing  * @param dir the directory to read from and write to  */ public void makeMommyWatchingMovie(String dir) {   String barbF = FileChooser.getMediaPath("barbaraS.jpg");   String katieDir =   FileChooser.getMediaPath("kid-in-bg-seq/");   Picture barbP = new Picture(barbF);   FrameSequencer frameSequencer = new FrameSequencer(dir);   Picture currP = null;   // get the array of files in the directory   File dirObj = new File(katieDir);   String[] fileArray = dirObj.list();   // loop through the array of files   for (int i = 0; i < fileArray.length; i++)   {     if (fileArray[i].indexOf(".jpg") >= 0)     {       currP = new Picture(katieDir + fileArray[i]);       currP.copy(barbP,22,9,93,97,i * 3, i * 3); 
[Page 496]      frameSequencer.addFrame(currP);     }   }   // replay the movie   frameSequencer.show();   frameSequencer.replay(30); }

You can test this method with the following main method:

public static void main(String[] args)   {     MovieMaker movieMaker = new MovieMaker();     String dir = "c:/intro-prog-java/movies/mommy/";     movieMaker.makeMommyWatchingMovie(dir);   }

We can certainly do more sophisticated image processing than simple composing or sunsets. For example, we can do chromakey on movies frames. In fact, that's how many computer-generated effects in real movies are made. To try this out, Mark took a simple video of our three children (Matthew, Katie, and Jenny) crawling in front of a blue screen (Figure 14.9). Mark didn't do the lighting right, so the background turned out to be closer to black instead of blue. That turned out to be a critical error. The result was that the chromakey also modified Matthew's and Katie's pants and Jenny's eyes so that you can see the moon right through them (Figure 14.10). Black is another color that one should not use for the background when doing chromakey.

Figure 14.9. Frames from the original movie of the kids crawling in front of a blue screen.

Figure 14.10. Frames from the kids on the moon movie.

[Page 497]

So far we have methods in the Picture class that do chromakey with blue and red backgrounds. Let's create another chromakey method that takes the Color to compare to and a distance to that color to use.

Computer Science Idea: Use Parameters to Make Methods Reusable

Notice that we are adding new methods that are like our old methods in that they have the same functionality, but they take more parameters. Adding parameters to a method will often make it easier to reuse. Remember that you can have many methods with the same name as long as the parameter list is different. You can even have one method call another with the same name as long as the parameter list is correct.

Program 138. New Chromakey Method

/**  * Method to do chromakey using the passed background  * color and the distance to the color  * @param newBg the new background image to use to replace  * @param color the background color to compare to  * @param dist the distance that limits the chromakey  * it will happen if the distance is less than or equal  * to this value  */ public void chromakey(Picture newBg, Color color, double dist) {   Pixel currPixel = null;   Pixel newPixel = null;   // loop through the columns   for (int x=0; x<getWidth(); x++)   {     // loop through the rows     for (int y=0; y<getHeight(); y++)     {       // get the current pixel       currPixel = this.getPixel(x,y);       /* if the color at the current pixel is mostly blue        * (blue value is greater than red and green combined),        * then use the new background color        */       double currDist = currPixel.colorDistance(color);       if (currDist <= dist)       {         newPixel = newBg.getPixel(x,y);         currPixel.setColor(newPixel.getColor());       }     }   } }

[Page 498]

We can use the new chromakey method in the Picture class to create a movie that looks like the kids are crawling on the surface of the moon.

Program 139. Using Chromakey to Put Kids on the Moon

/**  * Method to make a movie of the kids crawling on the moon  * @param dir the directory to write the frames to  */ public void makeKidsOnMoonMovie(String dir) {   String kidsDir = FileChooser.getMediaPath("kids-blue/");   String moonF = FileChooser.getMediaPath("moon-surface.jpg");   Picture moonP = new Picture(moonF);   FrameSequencer frameSequencer = new FrameSequencer(dir);   Picture currP = null;   // get the array of files in the directory   File dirObj = new File(kidsDir);   String[] fileArray = dirObj.list();   // loop through the array of files   for (int i = 0; i < fileArray.length; i++)   {     if (fileArray[i].indexOf(".jpg") >= 0)     {       currP = new Picture(kidsDir + fileArray[i]);       currP.chromakey(moonP,Color.black,100.0);       frameSequencer.addFrame(currP);     }   }   // replay the movie   frameSequencer.show();   frameSequencer.replay(30); }

You can test this method with the following main method:

public static void main(String[] args)   {     MovieMaker movieMaker = new MovieMaker();     String dir = "c:/intro-prog-java/movies/moon/";     movieMaker.makeKidsOnMoonMovie(dir);   }

Mark took a video of fish underwater. Water filters out red and yellow light, so the video looks too blue (Figure 14.11). Let's increase the red and green in the video (yellow is a mixture of red and green light). We had a method that multiplied the red color at each pixel by 1.3. Let's create a new method in the Picture class that will multiply the red and green values by passed multipliers. This way the same method can be used to increase or decrease the red and green values. Let's call this new method changeRedAndGreen.

[Page 499]

Figure 14.11. Some frames from the original too blue movie.

Program 140. General Change Red and Green Values

/**  * Method to change the red and green values in the  * current picture  * @param redMult the amount to multiply the red by  * @param greenMult the amount to multiply the green by  */ public void changeRedAndGreen(double redMult,                               double greenMult) {   Pixel[] pixelArray = this.getPixels();   Pixel pixel = null;   int value = 0;   int index = 0;   // loop through all the pixels   while (index < pixelArray.length)   {     // get the current pixel     pixel = pixelArray[index];     // change the red value     value = pixel.getRed();     pixel.setRed((int) (value * redMult));     // change the green value     value = pixel.getGreen();     pixel.setGreen((int) (value * greenMult));     // increment the index     index++;   } }

[Page 500]

Now let's create a new method in the class MovieMaker that will increase the red and green in each frame of the movie.

Program 141. Change the Color in the Movie of Fish

/**  * Method to change the red and green values in the frames  * @param dir the directory to write the frames to  */ public void makeFishMovie(String dir) {   String movieDir = FileChooser.getMediaPath("fish/");   FrameSequencer frameSequencer = new FrameSequencer(dir);   Picture currP = null;   // get the array of files in the directory   File dirObj = new File(movieDir);   String[] fileArray = dirObj.list();   // loop through the array of files   for (int i = 0; i < fileArray.length; i++)   {     if (fileArray[i].indexOf(".jpg") >= 0)     {       currP = new Picture(movieDir + fileArray[i]);       currP.changeRedAndGreen(2.0,1.5);       frameSequencer.addFrame(currP);     }   }   // play the movie   frameSequencer.play(16); }

The result of executing the following main method is shown in Figure 14.12.

public static void main(String[] args)   {     MovieMaker movieMaker = new MovieMaker(); 
[Page 501]    String dir = "c:/intro-prog-java/movies/fish/";     movieMaker.makeFishMovie(dir);   }