Now's the time to let slip the dogs of war on your mask in progress. Or at least to bring out all your masking tricks. Consider these strategies for particularly tricky situations.
All-Purpose Strategies to Edit Your Mask
Be realistic. It's easy to lose hours trying to make the perfect mask. To paraphrase Gen. George Patton, "A good mask today is better than a perfect mask tomorrow." And remember the old adage of experienced Photoshop maskers everywhere: "No one sees what you leave out." Sometimes you can't include every strand of hair, so don't drive yourself crazy trying. Ironically, including every last detail may actually make the result look fake. We're after the greater truth here, and if that involves leaving out a strand or two of hair, then so be it. Your subject probably needed a haircut anyway.
Channel masking (when it works) is such an elegant and pleasing technique that you will want to use it all the time. But don't throw away your Selection tools. Stay flexible and open to all possibilities. For example, a channel mask may be just the ticket for the head, but when it comes to the shoulders and torso, you may be better off using your conventional Selection tools.
To get the best results when editing your mask, combine the following techniques and strategies as needed.
Put Your Subject on a Temporary Color Layer
Highlight any problems with your mask by creating a new layer and dragging it beneath your image layer. Fill this layer with a bright color like neon orange or lime green. Warning: This will probably look ghastly. When you have made any necessary adjustments, you can delete the color layer.
Figure 6.13. Use a temporary color layer to bring out problems in the mask.
Choking the Mask
We've detailed this technique in "Choking the Mask" in Chapter 4, "Layer Masks." You can try to apply it here, although presumably most of its potential will have been exhausted in the earlier section in this chapter, "Forcing Contrast."
Using the Blur and Sharpen Tools to Fix Edges
To selectively blur or (less likely) selectively sharpen parts of your mask edge, use the Blur (r) or Sharpen (r) tools on your layer mask. Choose a soft-edged brush and paint around the edges, adjusting the brush Strength as necessary. Alternatively, you can try making a selection on your mask of the edges you want blurred and apply a Gaussian Blur filter. To vary the softness, make several small selections, rather than one big one, and apply varying amounts of blurring. Both this and the previous technique of choking the mask are best done with the final background in place because the image context will determine the amount required.
Defringing with Blend Modes
Depending on the lighting conditions of your image you may be able to use blend modes to fix color fringing around fine details. The most likely blend modes to work will be Multiply or Hard Light. Indeed, if your image is a silhouette you may be able to get away with just using these without a channel mask. With the image of the London Eye on the next page, we see all kinds of nasty fringing around the fine details when we place the masked image on an alternative sky.
Figure 6.14. The original image (example A). The masked image on a white background looks OK (example B), but when viewed against another image layer the fringing is evident (examples C and D).
Open the London Eye image.
Make a channel mask based on a duplicate of the Blue channel using the steps outlined earlier in "Creating a Channel Mask: The Basics." I have included a pre-made alpha channel with this image if you want to skip this step.
Load the alpha channelyours or the one that is saved with the imageand convert the resulting selection into a layer mask revealing the sky layer beneath.
To fix the fringing, duplicate the image layer, delete the layer mask from the duplicate layer, and turn its blend mode to Hard Light. This fixes the fringing but makes the image too dark.
Figure 6.15. Adding a Hard Light layer fixes one problem but creates anotherthe image is now too dark.
To address the too-dark image, add a Curves adjustment layer above the original image layer to brighten the detail. To avoid also brightening the sky, I clipped this adjustment layer to the transparency of the layer beneathhold down Option/Alt and click the horizontal line separating the layers.
Figure 6.16. The finished image (example A), and its Layers palette showing how it is composed (example B).
Here's another example using that holy grail of masking: a curly haired subject. Such images typically require a good deal of experimentation and may take you down some blind alleys. Here's what worked for me in this particular case.
I began with these two images. If you like, you can follow along with your own suitable images.
Figure 6.17. The image layers: a curly-haired subject and the Golden Gate Bridge.
Start with your frizzy-haired subject, using a channel mask to isolate the hair. In my example, the initial channel mask, though a good start, shows color fringing problems when the image is seen in the context of its new background.
Figure 6.18. The channel mask (example A) used to mask the subject (example B). A close-up showing the color fringing problems (example C), and the Layers palette showing how the image is constructed (example D).
To fix the color fringing, duplicate the image layer and its layer mask and set the blend mode of the duplicate layer to Multiply. This goes some way towards fixing the problem, but makes the image too dark.
To correct the dark image, switch the stacking order of the layers, placing the original image layer above the Multiply layer. Then, on the layer mask of the original layer, paint in black around the edges of the hair, hiding the hair on that layer and revealing the hair from the Multiply layer below. This is an improvement, but now the hair has a translucent quality that looks fake.
Figure 6.19. Painting on the layer mask of the image layer reveals the fine strands of hair on the Multiply layer beneath, but now the hair looks oddly transparent.
To fix the transparency, duplicate the Multiply layer and move the copy above the image layer. The original image layer is now sandwiched top and bottom by a duplicated Multiply layer. On this top layer, I painted out most of the detail of the figure on the layer mask, leaving just the fine edges of the hair, and building density and opacity with the other Multiply layer below.
Figure 6.20. A Multiply sandwich: a second Multiply layer on top of the layer stack (example A) reveals just the fine edges of the hair (examples B and C).
Another approach to correct color fringing uses the Color blend mode and paints around the edges of your masked subject on a separate layer. This tends to work best when the subject is fairly uniform in color. The image of a palm tree on page 234 is a good example.
Open the Palm Tree image.
Figure 6.21. The original photo of the palm tree (example A) and the masked version (example B).
Create a channel mask using the steps outlined in the previous section, "Creating a Channel Mask."
Turn the channel mask into a layer mask by first loading it as a selection (drag its thumbnail onto the Load channel as selection icon at the bottom of the Channels palette) and then click the Add Layer Mask at the bottom of the Layers palette.
Create a temporary color layer to highlight the color fringing problems.
Add a new layer above the image layer and make its blend mode Color. Call this layer Fix Fringing. This layer needs to be clipped by the layer below, so hold down Option/Alt and click the line between the layers to make a clipping mask.
With your Eyedropper tool sample the color from just inside the edges of the palm tree where there is no fringing.
Choose a brush size appropriate for the size of your edge and paint over the edge. The blue fringe is painted green to match the tree. Because the defringing layer is clipped by the layer below, the paint will be masked by the image layer's transparency. To put it another way, paint will only go where there's already paint.
Experiment with the opacity of the Fix Fringing layer. The Color blend mode works well in this example, but won't with every type of image.
If the Color blend mode doesn't give the desired results, experiment with Multiply or Darken. Which mode will work will depend on what colors you're working with.
Figure 6.22. The masked palm tree on a color background shows blue color fringing (example A). Painting on a clipped layer above with color sampled from the palm tree fixes the problem (examples B, C, and D).
A variation of this technique is to use the Clone Stamp tool instead of the Brush, making sure that the Sample All Layers option and Aligned options are checked. Clone from just inside the subject edge (where there is no color fringing) onto the edge itself.
Masking transparent objects requires a slightly different approach. Forcing the contrast on the duplicate channel to opaque white will cause the glass to become opaque. Assuming that you want to mask the background completely, force the contrast of the shadow areas to black, but leave the highlight areas light gray. That way, when you convert the channel to a layer mask you'll be able to see through your subject to the layer below. Take a look at this example.
Open the Martini Glass and Wall images.
Figure 6.23. Start with a martini glass and a wall background (examples A and B) to create a finished composite (example C).
Drag the martini glass onto the background image. Turn off the visibility of the Background layer.
Choose Image > Calculations and combine the Red channel (the one with the most contrast) with itself using the Overlay blend mode to create a new channel, Alpha 1. In this case, you're using Calculations to blend two copies of the same channel.
Still on your duplicate (the new Alpha) channel, draw a pen path around the glass. Choose Make Selection from the Paths palette menu to turn the path into a selection. I used a 1-pixel feather radius.
Inverse the selection (Select > Inverse or Command/Ctrl-I) and fill the alpha channel with black to block out the background. Deselect your selection.
Figure 6.24. The channel mask with the background filled with black.
Still on the Alpha 1 channel, lighten the midtone grays of the channel to make the glass more "see-through" when composited with another layer. I used Curves and pulled the Curve down from the midpoint, but using Levels and moving the Midpoint slider to the left would also work.
Return to your composite RGB channel and load Alpha 1 as a selection by dragging its thumbnail onto the Load channel as selection icon at the bottom of the Channels palette.
Click the Add Layer Mask icon at the bottom of the Layers palette to convert the selection to a layer mask.
To soften the mask edge, choke the layer mask by applying a 1.5 pixel Gaussian Blur and then, using Levels, move the Black and Gray point slider to the right.
Figure 6.25. The glass masked but lacking definition.
Now we're ready to return some definition to the glass, which was lost in the flat gray values in the layer mask.
Make sure that the visibility of the Background layer is turned off. Choose Select > Color Range and choose Midtones from the Sampled Colors drop down menu. Click OK to return to your image.
Figure 6.26. Selecting the midtones using Color Range.
Copy this active selection to a new layer (Command/Ctrl-J) and name this layer Midtones.
So that the Midtones layer is clipped by the Glass layer, hold down Option/Alt and click the horizontal line separating the layers.
Change the Blend mode of the Midtones layer to Linear Burn and experiment with the Opacity of the layer. Why Linear Burn? Well, it just worked best in this situation.
To prevent blending mode changes to the Midtones layer from affecting the Background layer, select both the Glass and the Midtones layer, and from the Layers palette menu choose New Group from Layers.
Add a Curve adjustment layer as necessary to brighten the glass.
Figure 6.27. The Layers palette of the image showing how the image is constructed.