How Core Image Manipulates Images

For all its power, Core Image is remarkably easy to use. There are a few basic concepts you have to understand, and some classes your application must manage, but once you have a handle on those you can harness a lot of power quickly.

The focus of Core Image is, of course, pixel-based images. The graphics in Core Image are managed through the aptly named CIImage class. Although the name of this class is very similar to CGImage, the two represent very different concepts. Unlike a CGImage, a CIImage does not represent a collection of pixel data! It represents a series of instructions about how to draw an image. The class may have more appropriately been called an image program.

When the computer draws a CIImage, it actually executes the drawing steps that are stored in the CIImage's program. The program generates the pixels that the computer eventually draws onto a graphics context of some kind. But the pixels themselves are neither calculated nor stored until the CIImage is drawn and its image program runs. This lazy evaluation is an important feature of Core Image.

This concept of CIImages, as instructions for drawing an image instead of as a fixed collection of pixels, allows CIImages to have characteristics that are quite different from other images you may encounter. For example, a CIImage can consist of a drawing instruction like "fill the entire drawing plane with a blue and white checkerboard pattern." In this example, the resulting "image" is unbounded. It fills the entire drawing plane and is infinite in extent.

The connection between a CIImage and a CGImage comes about when the CIImage uses the CGImage as a drawing parameter. The simplest CIImage instruction is probably "Draw these pixels." To provide the particular pixels the image program should draw, an application will provide the CGImage containing the pixels. Note that the CIImage itself doesn't necessarily own a frame buffer with color image data in it. The color image data is owned by the CGImage that is a parameter of a CIImage drawing command. The difference is a subtle but important one.

The CIImage can contain more interesting instructions than just "Draw this image." Consider a simple two-step CIImage where the first step is "collect a bundle of pixels," and the second step is "blur those pixels with a Gaussian blur with a radius of 3." When the computer draws this CIImage, it executes the program steps and the result is a blurred version of the image specified in step 1 of the program. The actual pixels that the computer sets down when it draws the CIImage are calculated when the CIImage is drawn and not before.

To add the Gaussian blur or any other effect to the image program inside of a CIImage, you create a filter object. Filter objects are instances of the CIFilter class. A CIFilter object knows how to generate an image program from a set of input parameters. You might think of them as CIImage factories. The factories take in raw materials, bundles of pixels, input parameters and the like, then use those input parameters to write a program that combines them in some way. The resulting program is a CIImage.

To use a CIFilter, your application creates a particular instance of a CIFilter subclass. Each subclass represents a specific image program step you want the computer to take. You then specify the input parameters for that filter. The input parameters might include other images, numbers, strings, and similar elements that the filter needs for processing. Continuing the preceding example, one subclass of CIFilter accepts an input image and a floating point radius as parameters and performs a Gaussian blur on that image with that radius.

After filling in the input parameters to a CIFilter, you can ask it for its output parameters. The most common output parameter your program will take from a CIFilter is the outputImage parameter. The value of this parameter is a CIImage object, an image program, that encapsulates the input parameters and the drawing commands necessary to carry out the behavior of the filter.

The CIImage that is the output of one CIFilter can become an input to another CIFilter. Your application can use this fact to chain many of filters together. The result of this chain is a CIImage that contains all the processing commands of all the filters. Along the way, the computer collects the various drawing steps, with their input parameters, and records them into another CIImage that becomes the output image of the final filter. Keep in mind that the CIImage is not actually generating pixels as you chain the filters together. It is simply recording the drawing commands and parameters for each filter in the chain.

To actually get pixels out of a CIImage, you have to run the image program that it represents. To do that, you draw the CIImage. Since the filters don't do any processing until you draw the final image, the computer is free to optimize your image program as it collapses the collection of filters into a single rendering step. For example, if your image program specifies that the computer should give all the pixels in an image a purplish tint and then scale the result down to half its original size, the computer might rearrange these steps so that the scaling happens first and the tinting second. The advantage to this is that the computer has to modify fewer pixels to colorize the scaled-down image than it does to colorize the full-scale image.

Just as Core Graphics drawing occurs in a CGContext, Core Image has a CIContext. The context references another drawing context, either a CGCon-text or an OpenGL context, the same way the CIImage references a CGImage to supply pixel data. In addition to drawing into a CGContext, the initial version of Core Image allows you to draw into an OpenGL context.