Quick Mask and Soft Edges

THIS TEMPORARY MASK LETS YOU PAINT YOUR SELECTIONS IN USING THE BRUSH TOOL

So far, we've made our selections two ways: (1) We used a selection tool (like the Pen tool, or Lasso tool, etc.), and (2) We made our mask by using one of the existing color channels as a starting place. But there's another way. It's quite a popular method, and one that you will either fall in love with and want to use all the time, or you'll just use it on special occasions, when you feel you need it, but either way we have to learn it.

It's called Quick Mask, and it's essentially a temporary Alpha channel (so, although it works similarly to an Alpha channel, once you load your selection, its job is done, and nothing is saved to your Channels palette). So, besides being "quick," the other advantage is that you make your selection by painting (using Photoshop's brush tools). That's right, you enter Quick Mask mode, you paint over the area you want to become a selection, and when you return to Standard mode it becomes one. Quick Mask is ideal when you have an object with soft edges all around, because you can make your selection with a soft-edged brush. It's also ideal when you have an object that has some soft edges and some well-defined or hard edges (like the project you're about to tackle).

Step 1.	OPEN A PHOTO THAT HAS BOTH HARD AND SOFT EDGE AREAS Open a photo that has both well-defined and soft-edged areas that need to be selected (like this hummingbird. Her body can be selected with a standard mask or selection tool, but her wings, with their soft edges and, even worse, the motion blurring the wing farthest away from the camera, will need some serious help). I'll bet you're wondering how long it took to find a hummingbird hovering in front of a gray background. It took a while. Honestly, that's not what I was looking forI was just trying to find a hummingbird, but while searching on iStockphoto.com, I ran across this one and liked the motion in the wings (I cropped the flower in the photo out). ©ISTOCKPHOTO/MARIO ASEHAN
Step 2.	SELECT THE WELL-DEFINED AREAS WITH THE SELECTION TOOL OF YOUR CHOICE As mentioned earlier, we're going to be using Quick Mask to help us select the soft-edged areas in this photo (the wings). But we can make our job easier by doing a little selecting work first on the hard edges or well-defined areas with the standard selection tools. Here I used the Pen tool (P) to put a path around the body of the hummingbird, then I turned that path into a selection by pressing Command-Return (PC: Control-Enter). Although I used the Pen tool, you can choose any selection tool you're comfortable with (the Lasso would work fine, as would the Magnetic Lasso). So, go ahead and make a selection of the bird's body (as shown here).
Step 3.	PRESS Q TO ENTER QUICK MASK MODE Now that our selection is in place, switch to Quick Mask mode by either clicking on the Edit in Quick Mask Mode icon at the bottom of the Toolbox (circled in red here) or just press the letter Q on your keyboard (the shortcut for entering Quick Mask mode). When you enter Quick Mask mode, the masked areas (the areas that are protected) appear in a red tint. The areas that are selected appear clear. That's why the body of the hummingbird appears clear here. See how this relates to a regular Alpha channel, where the black area would be the masked area, and the white area would be where your selected area would be?
Step 4.	DOUBLE-CLICK THE QUICK MASK ICON TO ACCESS THE QUICK MASK OPTIONS So, in Quick Mask, when you paint in white over a red (masked) area, it turns clear, and those clear areas will become a selection when you return to Standard mode. This freaks a lot of people out who aren't comfortable with the idea of painting away red to make a selection. They'd rather that the background was clear, and the area they want selected would then appear in red instead (it actually makes more sense, unless you have a background in traditional prepress). If you're one of those people (and you know who you are), double-click on the Quick Mask icon at the bottom of the Toolbar, and when the Options appear, where it says Color Indicates choose Selected Areas, and click OK.
Step 5.	PICK A SOFT-EDGED, MEDIUM-SIZED BRUSH Go ahead and try painting. As you paint (in black), it paints in a red tint. When you switch back to Standard mode, those red tinted areas will become your selection. Now, at least you know how they both work, and you can choose which method works best for you. Once you make your choice, it is time to start making our soft-edged selection. First, set your Foreground color to white. Next, get the Brush tool (B), and choose a medium-sized, soft-edged brush from the Brush Picker (as shown here). By the way, we're going to go with the default Color Indicates for this tutorial, so red areas are protected, and the clear areas will become your selection later on.
Step 6.	PAINT OVER THE SOFT EDGES OF THE FRONT WING Now take your soft-edged brush and start painting over the wings of your hummingbird. You can see that as you paint, the full-color image is revealed. As you get closer to the edges of your wings, you'll have to shrink the size of your brush a bit (use the Left and Right Bracket keys on your keyboard to change brush sizes as you paint). What you're aiming to do here is paint away the dark red areas on the wings.
Step 7.	INCREASE THE SIZE OF THE SOFT BRUSH AND PAINT OVER THE BACK WING You may have to zoom in to really get a good look at the area you're painting. When you work on the back wing (the one with all the movement), use a larger brush, and let the outside edge of the round cursor touch up against the outside edge of the wing. As you can see, the center of the circle is well within the wing, but the outer edge of your circular brush cursor paints away those soft areas of red where all the motion is. Remember, try to paint away all the dark red areas.
Step 8.	CONTINUE PAINTING UNTIL THE BIRD IS FULLY VISIBLE, CUT OUT OF THE RED TINT OVERLAY Keep painting until all the dark red areas are painted away on the wings (as shown here). If you accidentally paint too far past the wings, you can instantly repair the problem by pressing X to switch your Foreground color from white to black and painting over those mistake areas in black. As you paint, they'll be masked (protected) again. So, toggle back and forth between white (to add to your selection), and black (to paint in red and fix your mistakes) as you need.
Step 9.	HERE'S WHERE YOU CAN CHANGE THE TINT OVERLAY COLOR By the way, I keep talking about the red tint and that's because the default tint color of Quick Mask's overlay is red (it's red because it simulates the red color of traditional rubylith masks that were cut with a knife to make masks back in the days before Photoshop). Okay, so what do you do when you're trying to mask something that's on a red background? Doesn't that red tint make it hard to see what's going on? Yup. That's why Adobe lets you change the color of the tint overlay. Just double-click on the Quick Mask icon to bring up the Quick Mask Options, then click on the red color swatch to choose a new colorone that contrasts better with your background.
Step 10.	SWITCH TO STANDARD MODE AND YOUR SELECTION IS IN PLACE. COPY THE HUMMINGBIRD Now that the hummingbird has been masked using Quick Mask, you can return to Standard mode by either clicking on the Edit in Standard Mode icon (circled here in red), or just by pressing the letter Q again. Once you return to Standard mode, the clear areas of your Quick Mask become a selection (as shown here). Now, if you're looking at this selection, you're probably thinking, "Hey, those wings don't look like they have a soft-edged selection. They just look rounded." Well, that's because the selection border (the marching ants) can't display soft-edged selections, but you'll see that soft edge soon enough. Press Command-C (PC: Control-C) to Copy your selected area into memory.
Step 11.	OPEN A NEW BACKGROUND PHOTO Open the background that you want your selected hummingbird to appear on (like the one shown here, which you can download from the book's companion website, along with the hummingbird, so you can practice along). ©ISTOCKPHOTO
Step 12.	PASTE THE HUMMINGBIRD INTO THE DOCUMENT AND POSITION IT OVER BY THE FLOWERS Press Command-V (PC: Control-V) to Paste your hummingbird into your document as a layer. Since it's on its own layer, you can get the Move tool (V) and position it so it's right next to the flower, as shown here. Notice the soft edges of the wings are perfectly intact. Sweeeeeeet! Of course, there's another problem, and that is the lighting on the hummingbird looks too blue (or "too cold") for the background photo we added her to. This could be a dead giveaway that the two photos were collaged, so we'll go ahead and fix that in the next step.
Step 13.	LOAD THE LAYER AS A SELECTION, THEN ADD A YELLOW PHOTO FILTER TO HELP MATCH BACKGROUND We need to warm her up a bit, but we just want to warm her, not the background. Start by getting your selection of her back. Press-and-hold the Command key (PC: Control key) and click once on her layer thumbnail to reload the selection around her. Now, choose Photo Filter from the Create New Adjustment Layer pop-up menu at the bottom of the Layers palette. Choose Yellow from the Filter pop-up menu, and increase the Density a bit until she looks more yellowish than blue (as shown here). When it looks right to you, click OK, and this adds an adjustment layer to your Layers palette, with a built-in mask over the background, so only the hummingbird is affected.
Step 14.	TO TIGHTEN, OR CHOKE, YOUR SELECTION, USE THE MINIMUM FILTER WHILE IN QUICK MASK MODE Now, we were fairly lucky in that when we pasted the hummingbird onto the background, it was a really good selection. But if you paste yours and find out that some of the gray background she used to be on came along with her, you might need to go back and shrink the selected area of your Quick Mask, then try again. Luckily, there are two filters that help you shrink your entire selected area (called "choking the mask") or increase the size of your selected area, in case you didn't get all of the bird selected (called "spreading the mask"). To choke the mask (shrink it), go under the Filter menu, under Other, and choose Minimum (as shown here).
Step 15.	AS YOU INCREASE THE RADIUS SLIDER, THE SELECTION SHRINKS Here we are back in the original document. When the Minimum dialog appears, there's only one controla Radius slider. As you increase the number (sliding to the right) it shrinks your selected area. In the example shown here, with a radius of 3, you can see that the selected areas have shrunk, because dark red areas are appearing all the way around your hummingbird. Those dark areas are choking your selection (making it smaller). So, if you switched back to Standard mode now, your selection would be 3 pixels tighter in all the way around your hummingbird. We use this trick anytime we see background fringe when we paste a photo into another photo.
Step 16.	WHILE IN QUICK MASK, USE THE MAXIMUM FILTER TO EXPAND, OR SPREAD, YOUR SELECTION The other filter is the Maximum filter, and it "spreads" your selection out, adding more area to your selection. In the example shown here, we've opened the Maximum filter (found under the Filter menu, under Other) and increased the Radius to 3. See how there's now a gray border around the hummingbird? That's the gray background from the original photo (which in this case, we don't want). However, if you pasted the photo onto the background, and the soft areas of the wings were clipped off and looked like hard edges, you might try using the Maximum filter. This will expand your selected area, so those soft-edged wings might be included.
Step 17.	TO TURN A QUICK MASK INTO AN ALPHA CHANNEL, LOAD THE SELECTION, THEN CLICK SAVE CHANNEL Remember, Quick Masks are temporaryonce you switch to Standard mode, they're gone for good, right? So what if you want to save your temporary Quick Mask as an Alpha channel, so you can reload it at any time? Here's what to do: While you're still in Quick Mask mode, go to the Channels palette and you'll see your temporary Quick Mask channel (you'll see "Quick Mask" appear in italics). Command-click (PC: Control-click) on the Quick Mask channel's thumbnail to load your Quick Mask as a selection, then click once on the Save Selection as Channel icon (as shown here). That's ityour new channel appears as Alpha 1, and you can drag your Quick Mask channel into the Trash.
Step 18.	HERE'S THE QUICK MASK SELECTION SAVED AS AN ALPHA CHANNEL Click on your Alpha 1 channel, and you'll see that your old Quick Mask selection is a regular channel. Now, see those small gray areas around the wings (aren't channels normally just black and white)? Those gray areas indicate soft edges. Gray is important in Alpha channels, because it determines visibility. For example, black areas are protected, and not seen. White areas will be seen, right? But what if you only wanted 50% of the image to show through? Then you'd fill that area of the Alpha channel with 50% gray. Want it to be more transparent? Fill it with 25% gray instead (I know, technically it's 25% black, not 25% gray, but you know what I mean).
Step 19.	ONCE SAVED AS AN ALPHA CHANNEL, YOU CAN APPLY MINIMUM AND MAXIMUM Hey, while we've got an Alpha channel, this would be a good time to let you know that you can also use the Minimum and Maximum filters on regular Alpha channels, as well (you probably figured that, but I thought I'd better mention it, just in case). Let's test itclick on your Alpha channel, then bring up the Minimum filter. Since Alpha channels don't have that red tint, the best way to see the effects of the Minimum filter is to click directly in the preview window, inside the Minimum filter itself. This gives you a before and afterwhen you click-and-hold on it, you'll see the before, then release to see the choked version.
Step 20.	GO BACK TO LAYERS IN THE COMPOSITE PHOTO, LASSO JUST THE WINGS Now, let's make one last move to make our two-photo composite more believable. In the Layers palette, click on the hummingbird layer, then get the Lasso tool and draw a very loose selection that completely encompasses the wings (as shown here).
Step 21.	CUT THEM FROM THAT LAYER TO THEIR OWN LAYER, CHANGE MODE TO DARKEN FOR REALISM Press Command-Shift-J (PC: Control-Shift-J) to cut those wings off the hummingbird layer and put them up on their own separate layer. Now, to finish off, let's make the edge areas of the wings appear a little transparent (so it picks up some of the color from the background, which helps the bird look like it was really on this background). Change the layer blend mode from Normal to Darken. Now, take a look at the bottom edges of the wingsyou can see the background peeking through. If we just lowered the opacity, the solid areas of the wings would also appear transparent. By changing the mode to Darken, just the areas darker than her wings show through.
Step 22.	HERE'S THE FINAL IMAGE, WITH THE BACKGROUND SHOWING THROUGH THE WINGS IN DARKEN MODE Here's the final image, without the distracting Layers palette in the way. If you want to dig much deeper into this whole masking and compositing thing, there's a book that I highly recommend and the entire book is dedicated to just those two topics. It's called Photoshop Masking & Compositing by Katrin Eismann (New Riders, ISBN: 0735712794). It is, hands-down, the best book on the topic ever, and it explores all the different techniques and tools used in making intricate selections and realistic composites. Trust me, you'll dig it.

CHANNELS SHORTCUTS	WINDOWS	MACINTOSH
In Quick Mask mode, to toggle back and forth between the red tint showing the selected area, and the red tint showing the unselected area:	Press-and-hold the Alt key and click on the Quick Mask icon on the bottom right of the Toolbox.	Press-and-hold the Option key and click on the Quick Mask icon on the bottom right of the Toolbox.
For new channels, to change from the white mask showing the selected area to showing the unselected area in the Channels palette:	Press-and-hold the Alt key and click on the Save Selection as Channel icon at the bottom of the Channels palette.	Press-and-hold the Option key and click on the Save Selection as Channel icon at the bottom of the Channels palette.
To view an Alpha channel as a red tinted overlay while you're on any channel:	Click on the Eye icon next to the Alpha channel you want to view as a red tinted overlay.	Click on the Eye icon next to the Alpha channel you want to view as a red tinted overlay.