Chapter 20. Working with Images and Other Media

CONTENTS

•  20.1 ImageObserver
•  20.2 MediaTracker
•  20.3 Producing Image Data
•  20.4 Filtering Image Data
•  20.5 Simple Audio
•  20.6 Java Media Framework

Up to this point, we've confined ourselves to working with the high-level drawing commands of the Graphics2D class, using images in a hands-off mode. In this section, we'll clear up some of the mystery surrounding images and see how they are created and used. The classes in the java.awt.image package handle images and their internals; Figure 20-1 shows the important classes in this package.

Figure 20-1. The java.awt.image package

First, we'll return to our discussion of image loading and see how we can get more control over image data using an ImageObserver to watch as it's processed asynchronously by GUI components. Then we'll open the hood and have a look at the inside of a BufferedImage. If you're interested in creating sophisticated graphics, such as rendered images or video streams, this will teach you about the foundations of image construction in Java.

One note before we move on: In early versions of Java (prior to 1.2), creating and modifying images was handled through the use of ImageProducer and ImageConsumer interfaces, which operated on low-level, stream-oriented views of the image data. We won't be covering these topics in this chapter; instead, we'll stick to the new APIs, which are more capable and easier to use in most cases.

20.1 ImageObserver

One of the challenges in building software for networked applications is that data is not always instantly available. Since some of Java's roots are in Internet applications such as web browsers, its image-handling APIs were designed specifically to accommodate the fact that images might take some time to load over a slow network, providing for detailed information about image-loading progress. While many typical client applications do not require handling of image data in this way, it's still useful to understand this mechanism if for no other reason than it appears in the most basic image-related APIs. In practice, you'll normally use one of the techniques presented in the next section to handle image loading for you.

In the previous chapter we mentioned that all operations on image data (e.g., loading, drawing, scaling) allow you to specify an "image observer" object as a participant. An image observer implements the ImageObserver interface, allowing it to receive notification as information about the image becomes available. The image observer is essentially a callback that is notified progressively as the image is loaded. For a static image, such as a GIF or JPEG data file, the observer is notified as chunks of image data arrive and also when the entire image is complete. For a video source or animation (e.g., GIF89), the image observer is notified at the end of each frame as the continuous stream of pixel data is generated.

The image observer can do whatever it wants with this information. For example, in the last chapter we used the image observer built into the base Component class. Although you probably didn't see it happen in our examples, the Component image observer invoked repaint() for us each time a new section of the image became available so that the picture, if it had taken a long time to load, would have displayed progressively. A different kind of image observer might have waited for the entire image before telling the application to display it; yet another use for an observer might be to update a loading meter showing how far the image loading had progressed.

To be an image observer, you have to implement the single method, imageUpdate() , defined by the java.awt.image.ImageObserver interface:

public boolean imageUpdate(Image image, int flags, int x, int y,                            int width, int height)

imageUpdate() is called by the graphics system, as needed, to pass the observer information about the construction of its view of the image. The image parameter holds a reference to the Image object in question. flags is an integer whose bits specify what information about the image is now available. The flag values are defined as static variables in the ImageObserver interface, as illustrated in this example:

//file: ObserveImageLoad.java import java.awt.*; import java.awt.image.*;    public class ObserveImageLoad {      public static void main( String [] args)    {    ImageObserver myObserver = new ImageObserver(  ) {       public boolean imageUpdate(           Image image, int flags, int x, int y, int width, int height)        {          if ( (flags & HEIGHT) !=0 )            System.out.println("Image height = " + height );          if ( (flags & WIDTH ) !=0 )            System.out.println("Image width = " + width );          if ( (flags & FRAMEBITS) != 0 )            System.out.println("Another frame finished.");          if ( (flags & SOMEBITS) != 0 )             System.out.println("Image section :"                 + new Rectangle( x, y, width, height ) );          if ( (flags & ALLBITS) != 0 )            System.out.println("Image finished!");          if ( (flags & ABORT) != 0 )             System.out.println("Image load aborted...");          return true;      }    };        Toolkit toolkit = Toolkit.getDefaultToolkit(  );     Image img = toolkit.getImage( args[0] );     toolkit.prepareImage( img, -1, -1, myObserver );   } }

Supply an image as the command-line argument and observe the output. You'll see a number of incremental messages about loading the image.

The flags integer determines which of the other parameters, x, y, width, and height, hold valid data and what that data means. To test whether a particular flag in the flags integer is set, we have to resort to some binary shenanigans (using the & (AND) operator). The width and height parameters play a dual role. If SOMEBITS is set, they represent the size of the chunk of the image that has just been delivered. If HEIGHT or WIDTH is set, however, they represent the overall image dimensions. Finally, imageUpdate() returns a boolean value indicating whether or not it's interested in future updates.

In this example, after requesting the Image object with getImage(), we kick-start the loading process with the Toolkit's prepareImage() method, which takes our image observer as an argument. Using an Image API method such as drawImage(), scaleImage(), or asking for image dimensions with getWidth() or getHeight() will suffice to start the operation. Remember that although the getImage() method created the image object, it doesn't begin loading the data until one of the image operations requires it.

The example shows the lowest-level general mechanism for starting and monitoring the process of loading image data. You should be able to see how we could implement all sorts of sophisticated image loading and tracking schemes with this. The two most important strategies (to draw an image progressively, as it's constructed, or to wait until it's complete and draw it in its entirety) are handled for us. We have already seen that the Component class implements the first scheme. Another class, java.awt.MediaTracker, is a general utility that tracks the loading of a number of images or other media types for us. We'll look at it next.

20.2 MediaTracker

java.awt.MediaTracker is a utility class simplifies life if we have to wait for one or more images to be loaded completely before they're displayed. A MediaTracker monitors the loading of an image or a group of images and lets us check on them periodically or wait until they are finished. MediaTracker implements the ImageObserver interface that we just discussed, allowing it to receive image updates.

The following code snippet illustrates using a MediaTracker to wait while an image is prepared:

//file: StatusImage.java import java.awt.*; import javax.swing.*;    public class StatusImage extends JComponent {   boolean loaded = false;   String message = "Loading...";   Image image;      public StatusImage( Image image ) { this.image = image; }      public void paint(Graphics g) {     if (loaded)         g.drawImage(image, 0, 0, this);     else {       g.drawRect(0, 0, getSize(  ).width - 1, getSize(  ).height - 1);       g.drawString(message, 20, 20);     }   }   public void loaded(  ) {     loaded = true;     repaint(  );   }   public void setMessage( String msg ) {     message = msg;     repaint(  );   }      public static void main( String [] args ) {     JFrame frame = new JFrame("TrackImage");     Image image = Toolkit.getDefaultToolkit(  ).getImage( args[0] );     StatusImage statusImage = new StatusImage( image );     frame.getContentPane(  ).add( statusImage );     frame.setSize(300,300);     frame.setVisible(true);        MediaTracker tracker = new MediaTracker( statusImage );     int MAIN_IMAGE = 0;     tracker.addImage(image, MAIN_IMAGE);     try {         tracker.waitForID(MAIN_IMAGE); }     catch (InterruptedException e) {}     if ( tracker.isErrorID(MAIN_IMAGE) )         statusImage.setMessage("Error");     else         statusImage.loaded(  );   } }

In this example we have created a trivial component called StatusImage that accepts an image and draws a text status message until it is told that the image is loaded. It then displays the image. The only interesting part here is that we use a MediaTracker to load the image data for us, simplifying our logic.

First, we create a MediaTracker to manage the image. The MediaTracker constructor takes a Component as an argument; this is supposed to be the component onto which the image is later drawn. This argument is somewhat of a hold-over from earlier Java days with AWT. If you don't have the component reference handy, you can simply substitute a generic component reference like so:

Component comp = new Component(  );

After creating the MediaTracker, we assign it images to manage. Each image is associated with an integer identifier we can use later for checking on its status or to wait for its completion. Multiple images can be associated with the same identifier, letting us manage them as a group. The value of the identifier is also used to prioritize loading when waiting on multiple sets of images; lower IDs have higher priority. In this case, we want to manage only a single image, so we created one identifier called MAIN_IMAGE and passed it as the ID for our image in the call to addImage().

Next, we call the MediaTracker waitforID() routine, which blocks on the image, waiting for it to finish loading. If successful, we tell our example component to use the image and repaint. Another MediaTracker method, waitForAll(), waits for all images to complete, not just a single ID. It is possible to be interrupted here by an InterruptedException. We should also test for errors during image preparation with isErrorID(). In our example, we change the status message if we find one.

The MediaTracker checkID() and checkAll() methods may be used to poll the status of images loading periodically, returning true or false indicating whether loading is finished. The checkAll() method does this for the union of all images being loaded. Additionally, the statusID() and statusAll() methods return a constant indicating the status or final condition of an image load. The value is one of the MediaTracker constant values: LOADING, ABORTED, ERROR, or COMPLETE. For statusAll(), the value is the bitwise OR value of all of the various statuses.

This may seem like a lot of work to go through just to put up a status message while loading a single image. MediaTracker is more valuable when you are working with many raw images that have to be available before you can begin parts of an application. It saves implementing a custom ImageObserver for every application. For general Swing application work, you can use yet another simplification by employing the ImageIcon component to use a MediaTracker; this is covered next.

20.2.1 ImageIcon

In Chapter 16, we discussed Swing components that can work with images using the Icon interface. In particular, the ImageIcon class accepts an image filename or URL and can render it into a component. Internally ImageIcon uses a MediaTracker to fully load the image in the call to its constructor. It can also provide the Image reference back. So, a shortcut to what we did in the last few sections getting an image loaded fully before using it would be:

ImageIcon icon = new ImgeIcon("myimage.jpg"); Image image = icon.getImage(  );

This quirky but useful approach saves a few lines of typing but uses a component in an odd way and is not very clear. ImageIcon also gives you direct access to the MediaTracker it is using through the getMediaTracker() method or tells you the MediaTracker load status through the getImageLoadStatus() method. This returns one of the MediaTracker constants: ABORTED, ERROR, or COMPLETE.

20.3 Producing Image Data

There are two approaches to generating image data. The easiest is to treat the image as a drawing surface and use the methods of Graphics2D to render things into the image. The second way is to twiddle the bits that represent the pixels of the image data yourself. This is harder, but it can be useful in specific cases such as loading and saving images in specific formats or mathematically analyzing or creating image data.

20.3.1 Drawing Animations

Let's begin with the simpler approach, rendering on an image through drawing. We'll throw in a twist to make things interesting: we'll build an animation. Each frame will be rendered as we go along. This is very similar to the double buffering we examined in the last chapter, but this time we'll use a timer, instead of mouse events, as the signal to generate new frames.

Swing performs double buffering automatically, so we don't even have to worry about the animation flickering. Although it looks like we're drawing directly to the screen, we're really drawing into an image that Swing uses for double buffering. All we need to do is draw the right thing at the right time.

Let's look at an example, Hypnosis, that illustrates the technique. This example shows a constantly shifting shape that bounces around the inside of a component. When screen savers first came of age, this kind of thing was pretty hot stuff. Hypnosis is shown in Figure 20-2.

Figure 20-2. A simple animation

Here is its source code:

//file: Hypnosis.java import java.awt.*; import java.awt.event.*; import java.awt.geom.GeneralPath; import javax.swing.*;    public class Hypnosis extends JComponent implements Runnable {   private int[] coordinates;   private int[] deltas;   private Paint paint;      public Hypnosis(int numberOfSegments) {     int numberOfCoordinates = numberOfSegments * 4 + 2;     coordinates = new int[numberOfCoordinates];     deltas = new int[numberOfCoordinates];     for (int i = 0 ; i < numberOfCoordinates; i++) {       coordinates[i] = (int)(Math.random(  ) * 300);       deltas[i] = (int)(Math.random(  ) * 4 + 3);       if (deltas[i] > 4) deltas[i] = -(deltas[i] - 3);     }     paint = new GradientPaint(0, 0, Color.blue,         20, 10, Color.red, true);        Thread t = new Thread(this);     t.start(  );   }      public void run(  ) {     try {       while (true) {         timeStep(  );         repaint(  );         Thread.sleep(1000 / 24);       }     }     catch (InterruptedException ie) {}   }      public void paint(Graphics g) {     Graphics2D g2 = (Graphics2D)g;     g2.setRenderingHint(RenderingHints.KEY_ANTIALIASING,         RenderingHints.VALUE_ANTIALIAS_ON);     Shape s = createShape(  );     g2.setPaint(paint);     g2.fill(s);     g2.setPaint(Color.white);     g2.draw(s);   }      private void timeStep(  ) {     Dimension d = getSize(  );     if (d.width == 0 || d.height == 0) return;     for (int i = 0; i < coordinates.length; i++) {       coordinates[i] += deltas[i];       int limit = (i % 2 == 0) ? d.width : d.height;       if (coordinates[i] < 0) {         coordinates[i] = 0;         deltas[i] = -deltas[i];       }       else if (coordinates[i] > limit) {         coordinates[i] = limit - 1;         deltas[i] = -deltas[i];       }     }   }      private Shape createShape(  ) {     GeneralPath path = new GeneralPath(  );     path.moveTo(coordinates[0], coordinates[1]);     for (int i = 2; i < coordinates.length; i += 4)       path.quadTo(coordinates[i], coordinates[i + 1],           coordinates[i + 2], coordinates[i + 3]);     path.closePath(  );     return path;   }      public static void main(String[] args) {     JFrame frame = new JFrame("Hypnosis");     frame.getContentPane(  ).add( new Hypnosis(4) );     frame.setSize(300, 300);     frame.setDefaultCloseOperation( JFrame.EXIT_ON_CLOSE );     frame.setVisible(true);   } }

The main() method does the usual grunt work of setting up the JFrame that holds our animation component.

The Hypnosis component has a very basic strategy for animation. It holds some number of coordinate pairs in its coordinates member variable. A corresponding array, deltas, holds "delta" amounts that are added to the coordinates each time the figure is supposed to change. To render the complex shape you see in Figure 20-2, Hypnosis creates a special Shape object from the coordinate array each time the component is drawn.

Hypnosis's constructor has two important tasks. First, it fills up the coordinates and deltas arrays with random values. The number of array elements is determined by an argument to the constructor. The constructor's second task is to start up a new thread that drives the animation.

The animation is done in the run() method. This method calls timeStep(), which repaints the component and waits for a short time (details to follow). Each time timeStep() is called, the coordinates array is updated. Then repaint() is called. This results in a call to paint(), which creates a shape from the coordinate array and draws it.

The paint() method is relatively simple. It uses a helper method, called createShape() , to create a shape from the coordinate array. The shape is then filled, using a Paint stored as a member variable. The shape's outline is also drawn in white.

The timeStep() method updates all the elements of the coordinate array by adding the corresponding element of deltas. If any coordinates are now out of the component's bounds, they are adjusted, and the corresponding delta is negated. This produces the effect of bouncing off the sides of the component.

createShape() creates a shape from the coordinate array. It uses the GeneralPathclass, a useful Shape implementation that allows you to build shapes using straight and curved line segments. In this case, we create a shape from a series of quadratic curves, close it to create an area, and fill it.

20.3.2 BufferedImage Anatomy

So far, we've talked about java.awt.Images and how they can be loaded and drawn. What if you really want to get inside the image to examine and update its data? Image doesn't give you access to its data. You'll need to use a more sophisticated kind of image, java.awt.image.BufferedImage. These classes are closely related BufferedImage, in fact, is a subclass of Image. But BufferedImage gives you all sorts of control over the actual data that makes up the image. BufferedImage provides many capabilities beyond the basic Image class, but because it's a subclass of Image, you can pass still a BufferedImage to any of Graphics2D's methods that accept an Image.

To create an image from raw data arrays, you need to understand exactly how a BufferedImage is put together. The full details can get quite complex the BufferedImage class was designed to support images in nearly any storage format you could imagine. But for common operations it's not that difficult to use. Figure 20-3 shows the elements of a BufferedImage.

Figure 20-3. Inside a BufferedImage

An image is simply a rectangle of colored pixels, which is a simple enough concept. There's a lot of complexity underneath the BufferedImage class, because there are a lot of different ways to represent the colors of pixels. You might have, for instance, an image with RGB data in which each pixel's red, green, and blue values were stored as the elements of byte arrays. Or you might have an RGB image where each pixel was represented by an integer that contained red, green, and blue component values. Or you could have a 16-level grayscale image with 8 pixels stored in each element of an integer array. You get the idea; there are many different ways to store image data, and BufferedImage is designed to support all of them.

A BufferedImage consists of two pieces, a Raster and a ColorModel. The Raster contains the actual image data. You can think of it as an array of pixel values. It can answer the question, "What are the color data values for the pixel at 51, 17?" The Raster for an RGB image would return three values, while a Raster for a grayscale image would return a single value. WritableRaster, a subclass of Raster, also supports modifying pixel data values.

The ColorModel's job is to interpret the image data as colors. The ColorModel can translate the data values that come from the Raster into Color objects. An RGB color model, for example, would know how to interpret three data values as red, green, and blue. A grayscale color model could interpret a single data value as a gray level. Conceptually, at least, this is how an image is displayed on the screen. The graphics system retrieves the data for each pixel of the image from the Raster. Then the ColorModel tells what color each pixel should be, and the graphics system is able to set the color of each pixel.

The Raster itself is made up of two pieces: a DataBuffer and a SampleModel. A DataBuffer is a wrapper for the raw data arrays, which are byte, short, or int arrays. DataBuffer has handy subclasses, DataBufferByte, DataBufferShort, and DataBufferInt, that allow you to create a DataBuffer from raw data arrays. You'll see an example of this technique later in the StaticGenerator example.

The SampleModel knows how to extract the data values for a particular pixel from the DataBuffer. It knows the layout of the arrays in the DataBuffer and is ultimately responsible for answering the question "What are the data values for pixel x, y?" SampleModels are a little tricky to work with, but fortunately you'll probably never need to create or use one directly. As we'll see, the Raster class has many static ("factory") methods that create preconfigured Rasters for you, including their DataBuffers and SampleModels.

As Figure 20-1 shows, the 2D API comes with various flavors of ColorModels, SampleModels, and DataBuffers. These serve as handy building blocks that cover most common image storage formats. You'll rarely need to subclass any of these classes to create a BufferedImage.

20.3.3 Color Models

As we've said, there are many different ways to encode color information: red, green, blue (RGB) values; hue, saturation, value (HSV); hue, lightness, saturation (HLS); and more. In addition, you can provide full-color information for each pixel, or you can just specify an index into a color table (palette) for each pixel. The way you represent a color is called a color model. The 2D API provides tools to support any color model you could imagine. Here, we'll just cover two broad groups of color models: direct and indexed.

As you might expect, you must specify a color model in order to generate pixel data; the abstract class java.awt.image.ColorModel represents a color model. By default, Java 2D uses a direct color model called ARGB. The A stands for "alpha," which is the historical name for transparency. RGB refers to the red, green, and blue color components that are combined to produce a single, composite color. In the default ARGB model, each pixel is represented by a 32-bit integer that is interpreted as four 8-bit fields; in order, the fields represent the alpha (transparency), red, green, and blue components of the color, as shown in Figure 20-4.

Figure 20-4. ARGB color encoding

To create an instance of the default ARGB model, call the static getRGBdefault() method in ColorModel. This method returns a DirectColorModel object; DirectColorModel is a subclass of ColorModel. You can also create other direct color models by calling a DirectColorModel constructor, but you shouldn't need to unless you have a fairly exotic application.

In an indexed color model, each pixel is represented by a smaller piece of information: an index into a table of real color values. For some applications, generating data with an indexed model may be more convenient. If you have an 8-bit display or smaller, using an indexed model may be more efficient, because your hardware is internally using an indexed color model of some form.

20.3.4 Creating an Image

Let's take a look at producing some image data. A picture is worth a thousand words, and, fortunately, we can generate a picture in significantly fewer than a thousand words of Java. If we just want to render image frames byte by byte, you can put together a BufferedImage pretty easily.

The following application, ColorPan, creates an image from an array of integers holding RGB pixel values:

//file: ColorPan.java import java.awt.*; import java.awt.image.*; import javax.swing.*;    public class ColorPan extends JComponent {   BufferedImage image;      public void initialize(  ) {     int width = getSize(  ).width;     int height = getSize(  ).height;     int[] data = new int [width * height];     int i = 0;     for (int y = 0; y < height; y++) {       int red = (y * 255) / (height - 1);       for (int x = 0; x < width; x++) {         int green = (x * 255) / (width - 1);         int blue = 128;         data[i++] = (red << 16) | (green << 8 ) | blue;       }     }     image = new BufferedImage(width, height,         BufferedImage.TYPE_INT_RGB);     image.setRGB(0, 0, width, height, data, 0, width);   }      public void paint(Graphics g) {     if (image == null)          initialize(  );     g.drawImage(image, 0, 0, this);   }      public void setBounds(int x, int y, int width, int height) {     super.setBounds(x,y,width,height);     initialize(  );   }      public static void main(String[] args) {     JFrame frame = new JFrame("ColorPan");     frame.getContentPane(  ).add(new ColorPan(  ));     frame.setSize(300, 300);     frame.setDefaultCloseOperation( JFrame.EXIT_ON_CLOSE );     frame.setVisible(true);   } }

Give it a try. The size of the image is determined by the size of the application window. You should get a very colorful box that pans from deep blue at the upper-left corner to bright yellow at the bottom right, with green and red at the other extremes.

We create a BufferedImage in the initialize() method and then display the image in paint(). The variable data is a 1D array of integers that holds 32-bit RGB pixel values. In initialize(), we loop over every pixel in the image and assign it an RGB value. The blue component is always 128, half its maximum intensity. The red component varies from 0 to 255 along the y-axis; likewise, the green component varies from 0 to 255 along the x-axis. This statement combines these components into an RGB value:

data[i++] = (red << 16) | (green << 8 ) | blue;

The bitwise left-shift operator (<<) should be familiar to C programmers. It simply shoves the bits over by the specified number of positions in our 32-bit value.

When we create the BufferedImage, all its data is zeroed out. All we specify in the constructor is the width and height of the image and its type. BufferedImage includes quite a few constants representing image storage types. We've chosen TYPE_INT_RGB here, which indicates we want to store the image as RGB data packed into integers. The constructor takes care of creating an appropriate ColorModel, Raster, SampleModel, and DataBuffer for us. Then we simply use a convenient method, setRGB(), to assign our data to the image. In this way, we've side-stepped the messy innards of BufferedImage. In the next example, we'll take a closer look at the details.

Once we have the image, we can draw it on the display with the familiar drawImage() method. We also override the Component setBounds() method in order to determine when the frame is resized and reinitialize the drawing image to the new size.

20.3.5 Updating a BufferedImage

BufferedImage can also be used to update an image dynamically. Because the image's data arrays are directly accessible, you can simply change the data and redraw the picture whenever you want. This is probably the easiest way to build your own low-level animation software. The following example simulates the static on an old black-and-white television screen. It generates successive frames of random black and white pixels and displays each frame when it is complete. Figure 20-5 shows one frame of random static.

Figure 20-5. A frame of random static

Here's the code:

//file: StaticGenerator.java import java.awt.*; import java.awt.event.*; import java.awt.image.*; import java.util.Random; import javax.swing.*;    public class StaticGenerator extends JComponent implements Runnable {   byte[] data;   BufferedImage image;   Random random;      public void initialize(  ) {     int w = getSize().width, h = getSize(  ).height;     int length = ((w + 7) * h) / 8;     data = new byte[length];     DataBuffer db = new DataBufferByte(data, length);     WritableRaster wr = Raster.createPackedRaster(db, w, h, 1, null);     ColorModel cm = new IndexColorModel(1, 2,         new byte[] { (byte)0, (byte)255 },         new byte[] { (byte)0, (byte)255 },         new byte[] { (byte)0, (byte)255 });     image = new BufferedImage(cm, wr, false, null);     random = new Random(  );   }      public void run(  ) {     if ( random == null )         initialize(  );     while (true) {       random.nextBytes(data);       repaint(  );       try { Thread.sleep(1000 / 24); }       catch( InterruptedException e ) { /* die */ }     }   }      public void paint(Graphics g) {     if (image == null) initialize(  );     g.drawImage(image, 0, 0, this);   }      public void setBounds(int x, int y, int width, int height) {     super.setBounds(x,y,width,height);     initialize(  );   }      public static void main(String[] args) {     //RepaintManager.currentManager(null).setDoubleBufferingEnabled(false);     JFrame frame = new JFrame("StaticGenerator");     StaticGenerator staticGen = new StaticGenerator(  );     frame.getContentPane(  ).add( staticGen );     frame.setSize(300, 300);     frame.setDefaultCloseOperation( JFrame.EXIT_ON_CLOSE );     frame.setVisible(true);     new Thread( staticGen ).start(  );   } }

The initialize() method sets up the BufferedImage that produces the sequence of images. We build this image from the bottom up, starting with the raw data array. Since we're only displaying two colors here, black and white, we need only one bit per pixel. We want a 0 bit to represent black and a 1 bit to represent white. This calls for an indexed color model, which we'll create a little later.

We'll store our image data as a byte array, where each array element holds eight pixels from our black-and-white image. The array length, then, is calculated by multiplying the width and height of the image and dividing by eight. To keep things simple, we'll arrange for each image row to start on a byte boundary. For example, an image 13 pixels wide actually uses 2 bytes (16 bits) for each row:

int length = ((w + 7) * h) / 8;

Next, the actual byte array is created. The member variable data holds a reference to this array. Later, we'll use data to change the image data dynamically. Once we have the image data array, it's easy to create a DataBufferfrom it:

data = new byte[length]; DataBuffer db = new DataBufferByte(data, length);

DataBuffer has several subclasses, such as DataBufferByte, that make it easy to create a data buffer from raw arrays.

The next step, logically, is to create a SampleModel. We could then create a Raster from the SampleModel and the DataBuffer. Lucky for us, though, the Raster class contains a bevy of useful static methods that create common types of Rasters. One of these methods creates a Raster from data that contains multiple pixels packed into array elements. We simply use this method, supplying the data buffer, the width and height, and indicating that each pixel uses one bit:

WritableRaster wr = Raster.createPackedRaster(db, w, h, 1, null);

The last argument to this method is a java.awt.Point that indicates where the upper-left corner of the Raster should be. By passing null, we use the default of 0, 0.

The last piece of the puzzle is the ColorModel. Each pixel is either 0 or 1, but how should that be interpreted as color? In this case, we use an IndexColorModel with a very small palette. The palette has only two entries, one each for black and white:

ColorModel cm = new IndexColorModel(1, 2,         new byte[] { (byte)0, (byte)255 },         new byte[] { (byte)0, (byte)255 },         new byte[] { (byte)0, (byte)255 });

The IndexColorModel constructor that we've used here accepts the number of bits per pixel (one), the number of entries in the palette (two), and three byte arrays that are the red, green, and blue components of the palette colors. Our palette consists of two colors: black (0, 0, 0) and white (255, 255, 255).

Now that we've got all the pieces, we just need to create a BufferedImage. This image is also stored in a member variable so we can draw it later. To create the BufferedImage, we pass the color model and writable raster we just created:

image = new BufferedImage(cm, wr, false, null);

All the hard work is done now. Our paint() method just draws the image, using drawImage().

The init() method starts a thread that generates the pixel data. The run() method takes care of generating the pixel data. It uses a java.util.Random object to fill the data image byte array with random values. Since the data array is the actual image data for our image, changing the data values changes the appearance of the image. Once we fill the array with random data, a call to repaint() shows the new image on the screen.

To run, try turning off double buffering by uncommenting the line involving the RepaintManager. Now it will look even more like an old TV with flickering and all!

That's about all there is. It's worth noting how simple it is to create this animation. Once we have the BufferedImage, we treat it like any other image. The code that generates the image sequence can be arbitrarily complex. But that complexity never infects the simple task of getting the image on the screen and updating it.

20.4 Filtering Image Data

An image filter is an object that performs transformations on image data. The Java 2D API supports image filtering through the BufferedImageOp interface. An image filter takes a BufferedImage as input (the source image) and performs some processing on the image data, producing another BufferedImage (the destination image).

The 2D API comes with a handy toolbox of BufferedImageOp implementations, as summarized in Table 20-1.

Table 20-1. Image operators in the 2D API

Name	Description
AffineTransformOp	Transforms an image geometrically
ColorConvertOp	Converts from one color space to another
ConvolveOp	Performs a convolution, a mathematical operation that can be used to blur, sharpen, or otherwise process an image
LookupOp	Uses one or more lookup tables to process image values
RescaleOp	Uses multiplication to process image values

Let's take a look at two of the simpler image operators. First, try the following application. It loads an image (the first command-line argument is the filename) and processes it in different ways as you select items from the combo box. The application is shown in Figure 20-6.

Figure 20-6. The ImageProcessor application

Here's the source code:

//file: ImageProcessor.java import java.awt.*; import java.awt.event.*; import java.awt.geom.*; import java.awt.image.*; import javax.swing.*;    public class ImageProcessor extends JComponent {   private BufferedImage source, destination;   private JComboBox options;      public ImageProcessor( BufferedImage image ) {     source = destination = image;     setBackground(Color.white);     setLayout(new BorderLayout(  ));     // create a panel to hold the combo box     JPanel controls = new JPanel(  );     // create the combo box with the names of the area operators     options = new JComboBox(       new String[] { "[source]", "brighten", "darken", "rotate", "scale" }     );     // perform some processing when the selection changes     options.addItemListener(new ItemListener(  ) {       public void itemStateChanged(ItemEvent ie) {         // retrieve the selection option from the combo box         String option = (String)options.getSelectedItem(  );         // process the image according to the selected option         BufferedImageOp op = null;         if (option.equals("[source]"))           destination = source;         else if (option.equals("brighten"))           op = new RescaleOp(1.5f, 0, null);         else if (option.equals("darken"))           op = new RescaleOp(.5f, 0, null);         else if (option.equals("rotate"))           op = new AffineTransformOp(               AffineTransform.getRotateInstance(Math.PI / 6), null);         else if (option.equals("scale"))           op = new AffineTransformOp(               AffineTransform.getScaleInstance(.5, .5), null);         if (op != null) destination = op.filter(source, null);         repaint(  );       }     });     controls.add(options);     add(controls, BorderLayout.SOUTH);   }      public void paintComponent(Graphics g) {     int imageWidth = destination.getWidth(  );     int imageHeight = destination.getHeight(  );     int width = getSize(  ).width;     int height = getSize(  ).height;     g.drawImage(destination,         (width - imageWidth) / 2, (height - imageHeight) / 2, null);   }      public static void main(String[] args) {     String filename = args[0];        ImageIcon icon = new ImageIcon(filename);     Image i = icon.getImage(  );        // draw the Image into a BufferedImage     int w = i.getWidth(null), h = i.getHeight(null);     BufferedImage buffImage = new BufferedImage(w, h,         BufferedImage.TYPE_INT_RGB);     Graphics2D imageGraphics = buffImage.createGraphics(  );     imageGraphics.drawImage(i, 0, 0, null);        JFrame frame = new JFrame("ImageProcessor");     frame.getContentPane(  ).add(new ImageProcessor(buffImage));     frame.setSize(buffImage.getWidth(), buffImage.getHeight(  ));     frame.setDefaultCloseOperation( JFrame.EXIT_ON_CLOSE );     frame.setVisible(true);   } }

There's quite a bit packed into the ImageProcessor application. After you've played around with it, come back and read about the details.

20.4.1 How ImageProcessor Works

The basic operation of ImageProcessor is very straightforward. It loads a source image, specified with a command-line argument, in its main() method. The image is displayed along with a combo box. When you select different items from the combo box, ImageProcessor performs some image-processing operation on the source image and displays the result (the destination image). Most of this work occurs in the ItemListener event handler that is created in ImageProcessor's constructor. Depending on what option is selected, a BufferedImageOp (called op) is instantiated and used to process the source image, like this:

destination = op.filter(source, null);

The destination image is returned from the filter() method. If we already had a destination image of the right size, we could have passed it as the second argument to filter(), which would improve the performance of the application a bit. If you just pass null, as we have here, an appropriate destination image is created and returned to you. Once the destination image is created, paint()'s job is very simple; it just draws the destination image, centered on the component.

20.4.2 Converting an Image to a BufferedImage

Image processing is performed on BufferedImages, not Images. This example demonstrates an important technique: how to convert an Image to a BufferedImage. The main() method loads an Image from a file using Toolkit's getImage() method:

Image i = Toolkit.getDefaultToolkit( ).getImage(filename);

Next, main() uses a MediaTracker to make sure the image data is fully loaded.

The trick of converting an Image to a BufferedImage is to draw the Image into the drawing surface of the BufferedImage. Because we know the Image is fully loaded, we just need to create a BufferedImage, get its graphics context, and draw the Image into it:

BufferedImage bi = new BufferedImage(w, h,         BufferedImage.TYPE_INT_RGB); Graphics2D imageGraphics = bi.createGraphics( ); imageGraphics.drawImage(i, 0, 0, null);

20.4.3 Using the RescaleOp Class

Rescaling is an image operation that multiplies all the pixel values in the image by some constant. It doesn't affect the size of the image in any way (in case you thought rescaling meant scaling), but it does affect the colors of its pixels. In an RGB image, for example, each of the red, green, and blue values for each pixel would be multiplied by the rescaling multiplier. If you want, you can also adjust the results by adding an offset. In the 2D API, rescaling is performed by the java.awt.image.RescaleOp class. To create such an operator, specify the multiplier, offset, and a set of hints that control the quality of the conversion. In this case, we'll use a zero offset and not bother with the hints (by passing null):

op = new RescaleOp(1.5f, 0, null);

Here we've specified a multiplier of 1.5 and an offset of 0. All values in the destination image will be 1.5 times the values in the source image, which has the net result of making the image brighter. To perform the operation, we call the filter() method from the BufferedImageOp interface.

20.4.4 Using the AffineTransformOp Class

An Affine Transformation is a kind of 2D transformation that preserves parallel lines; this includes operations like scaling, rotating, and shearing. The java.awt.image.AffineTransformOp image operator geometrically transforms a source image to produce the destination image. To create an AffineTransformOp, specify the transformation you want, in the form of an java.awt.geom.AffineTransform. The ImageProcessor application includes two examples of this operator, one for rotation and one for scaling. As before, the AffineTransformOp constructor accepts a set of hints; we'll just pass null to keep things simple:

else if (option.equals("rotate"))   op = new AffineTransformOp(       AffineTransform.getRotateInstance(Math.PI / 6), null); else if (option.equals("scale"))   op = new AffineTransformOp(       AffineTransform.getScaleInstance(.5, .5), null);

In both cases, we obtain an AffineTransform by calling one of its static methods. In the first case, we get a rotational transformation by supplying an angle. This transformation is wrapped in an AffineTransformOp. This operator has the effect of rotating the source image around its origin to create the destination image. In the second case, a scaling transformation is wrapped in an AffineTransformOp. The two scaling values, .5 and .5, specify that the image should be reduced to half its original size in both the x and y axes.

One interesting aspect of AffineTransformOp is that you may "lose" part of your image when it's transformed. For example, when using the rotate image operator in the ImageProcessor application, the destination image will have clipped some of the original image out. Both the source and destination images have the same origin, so if any part of the image gets transformed into negative x or y space, it is lost. To work around this problem, you can structure your transformations such that the entire destination image would be in positive coordinate space.

20.5 Simple Audio

Now we'll turn from images and open our ears to audio. The Java Sound API became a core API in Java 1.3. It provides fine-grained support for the creation and manipulation of both sampled audio and MIDI music. There's space here only to scratch the surface by examining how to play simple sampled sound and MIDI music files. With the standard JavaSound support bundled with Java you can play a wide range of file formats including AIFF, AU, Windows WAV, standard MIDI files, and Rich Music Format (RMF) files. We'll discuss other formats (such as MP3) along with video media in the next section.

java.applet.AudioClip defines the simplest interface for objects that can play sound. An object that implements AudioClip can be told to play() its sound data, stop() playing the sound, or loop() continuously.

The Applet class provides a handy static method, newAudioClip(), that retrieves sounds from files or over the network. (And there is no reason we can't use it in a nonapplet application.) The method takes an absolute or relative URL to specify where the audio file is located and returns an AudioClip. The following application, NoisyButton, gives a simple example:

//file: NoisyButton.java import java.applet.*; import java.awt.*; import java.awt.event.*; import javax.swing.*;    public class NoisyButton {      public static void main(String[] args) throws Exception {     JFrame frame = new JFrame("NoisyButton");     java.io.File file = new java.io.File( args[0] );     final AudioClip sound = Applet.newAudioClip(file.toURL(  ));        JButton button = new JButton("Woof!");     button.addActionListener(new ActionListener(  ) {       public void actionPerformed(ActionEvent e) { sound.play(  ); }     });        Container content = frame.getContentPane(  );     content.setBackground(Color.pink);     content.setLayout(new GridBagLayout(  ));     content.add(button);     frame.setVisible(true);     frame.setSize(200, 200);     frame.setDefaultCloseOperation( JFrame.EXIT_ON_CLOSE );     frame.setVisible(true);   } }

Run NoisyButton, passing the name of the audio file you wish to use as the argument. (We've supplied one called bark.aiff.)

NoisyButton retrieves the AudioClip using a File and the toURL()method to reference it as a URL. When the button is pushed, we call the play() method of the AudioClip to start things. After that, it plays to completion unless we call the stop() method to interrupt it.

This interface is simple, but there is a lot of machinery behind the scenes. Next we'll look at the Java Media Framework, which supports wider ranging types of media.

20.6 Java Media Framework

Get some popcorn Java can play movies! To do this though we'll need one of Java's standard extension APIs, the Java Media Framework (JMF). The JMF defines a set of interfaces and classes in the javax.media and javax.media.protocol packages. You can download the latest JMF from http://java.sun.com/products/java-media/jmf/. To use the JMF, add jmf.jar to your classpath. Or, depending on what version of the JMF you download, a friendly installation program may do this for you.

We'll only scratch the surface of JMF here, by working with an important interface called Player. Specific implementations of Player deal with different media types, like Apple QuickTime (.mov) and Windows Video (.avi). There are also players for audio types including MP3. Players are handed out by a high-level class in the JMF called Manager. One way to obtain a Player is to specify the URL of a movie:

Player player = Manager.createPlayer(url);

Because video files are so large and playing them requires significant system resources, Players have a multistep life cycle from the time they're created to the time they actually play something. We'll just look at one step, realizing. In this step, the Player finds out (by looking at the media file) what system resources it needs to play the media file.

player.realize( );

The realize() method returns right away; it kicks off the realizing process in a separate thread. When the player is finished realizing, it sends out an event. Once you receive this event, you can obtain one of two Components from the Player. The first is a visual component that, for visual media types, shows the media. The second is a control component that provides a prefab user interface for controlling the media presentation. The control normally includes start, stop, and pause buttons, along with volume controls and attendant goodies.

The Player has to be realized before you ask for these components so that it has important information, like how big the component should be. After that, getting the component is easy. Here's an example:

Component c = player.getVisualComponent( );

Now we just need to add the component to the screen somewhere. We can play the media right away (although this actually moves the Player through several other internal states):

player.start( );

The following example, MediaPlayer, uses the JMF to load and display a movie or audio file from a specified URL:

//file: MediaPlayer.java import java.awt.*; import java.net.URL; import javax.swing.*; import javax.media.*;    public class MediaPlayer  {    public static void main( String[] args ) throws Exception {       final JFrame frame = new JFrame("MediaPlayer");       frame.setDefaultCloseOperation( JFrame.EXIT_ON_CLOSE );       URL url = new URL( args[0] );       final Player player = Manager.createPlayer( url );          player.addControllerListener( new ControllerListener(  ) {          public void controllerUpdate( ControllerEvent ce ) {             if ( ce instanceof RealizeCompleteEvent )              {                Component visual = player.getVisualComponent(  );                Component control = player.getControlPanelComponent(  );                if ( visual != null )                    frame.getContentPane(  ).add( visual, "Center" );                frame.getContentPane(  ).add( control, "South" );                frame.pack(  );                frame.setVisible( true );                player.start(  );             }          }       });          player.realize(  );    } }

This class creates a JFrame that holds the media. Then it creates a Player from the URL specified on the command line and tells the Player to realize(). There's nothing else we can do until the Player is realized, so the rest of the code operates inside a ControllerListener after the RealizeCompleteEvent is received.

In the event handler, we get the Player's visual and controller components and add them to the JFrame. We then display the JFrame and, finally, we play the movie. It's very simple!

To use the MediaPlayer, pass it the URL of a movie or audio file on the command line. Here are a couple of examples:

% java MediaPlayer file:dancing_baby.avi % java MediaPlayer http://myserver/mp3s/TheCure/KissMe/catch.mp3

Figure 20-7 shows the "dancing baby" AVI running in the MediaPlayer. Feel free to dance along, if you want.

Figure 20-7. Image of the dancing baby AVI

CONTENTS