Using alpha channels generated in Photoshop is a great way to add transparency to a layer in AE, but unfortunately it only works well if you are working with still images. If you need moving transparency, a static alpha channel just won't cut it. However, the only easy way to create a moving alpha channel is to work with a 3D animation program. Most 3D apps can automatically generate an alpha channel for each frame of your animation. What if you need to add moving transparency to live video though? In AE, you can do this with masks.
Masks can be applied to each individual layer. You can apply as many masks as you like to a layer, and they can be combined in a variety of ways. Once you have applied a mask to a layer you can turn on keyframing for that mask and change it's shape over time. You can also soften the edges of a mask, and vary it's opacity over time. Masks are not quite as flexible as alpha channels for creating complex transparency, but when it comes time to start animating your transparency masks are faster to work with.
Masks are drawn using 'Bezier splines', and work just like paths in photoshop or the freeform drawing tools in Illustrator or Freehand. The shape of the mask is defined by points, which are then connected by a line which describes the outline of the mask. Each point on the mask can have a curve applied to it by adjusting handles which are tangent to the point, and these handles can be toggled on or off to change a point from a smooth curve to an angular corner.
Masks can be applied to a layer in one of two basic ways:
- Using the square or eliptical mask tools (the button with the dotted-line box on it in the tool palette) you can apply a square, circular or oval mask to a layer. Select a layer, select the tool you want to use, and drag out the mask shape on the layer. These tools work well for quickly cropping a layer or for creating a 'vignette' shape on a layer. Once you have added a mask using one of these tools you can edit it's shape any way you want, but you will always start with a square or round shape with just a few points.
- Using the pen tool you can draw a freeform mask on a layer by adding points one at a time (the points will automatically be connected by a line). When you draw a mask with the pen tool you must close the mask shape by clicking on the first point after you have added the last. Until you close the mask, you will not see any transparency. Once you close the mask, by default anything within the shape will be visible and everything outside the mask will become transparent.
You can add as many points as you need to describe your mask shape, but in general it is best to use as few as you can get away with. The less points you've got, the easier it will be to animate them later.
If you just click to add a point it will default to linear interpolation, meaning the lines will enter and exit the point in straight lines. If you click and drag when you add a point you can pull out handles which will give the line a curve as it passes through the point. You can always remove or add a curve later by using the pen tool on a point - when it is over the point it will become an open arrow, and each time you click the point it will toggle between linear and smooth interpolation.
- The mask feather affects how soft the edges of your mask are. By default there is no feather, which can give a hard, cookie-cutter look to your masks. By adding just a few pixels of feathering to a mask you can make it look much smoother, and by adding a large feather you can make the edges blend very gradually to transparent. You can also add different amounts of feathering in the horizontal and vertical directions.
- Mask opacity lets you set how opaque the visible portion of a layer (the portion within the mask) is. This works just like layer opacity but is only applied to the mask itself, this means that you could have morthan one mask on a layer and each one could have it's own transparency level.
Each time you draw a new mask on a layer, it is by default added to the selection area of the other masks. However, you can also choose different ways to apply masks to each other. Next to the mask name in the timeline there is a drop down menu which should say "add". This is the default setting, but other commonly used settings are:
- Subtract, which subtracts the area within the mask from the other masks. This is useful for punching holes into an area you have already masked off.
- Intersect, which means only the area where that mask and another mask overlap will be visible.
- Difference, the opposite of intersect. Both masks will be visible except where they overlap.
Animating a mask is similar to animating anything else in AE...you just need to turn on keyframing for the mask shape (using the stopwatch icon next to the mask shape property). Once you have done this you can go to a different point on your timeline and change the shape of the mask by moving it's points. AE will blend smoothly from one shape to the next.