With tight production and delivery schedules, the ability to repair or re-purpose a shot can make a big difference. First introduced and evolved in Photoshop, Content-Aware Fill for Video is now available to editors and visual effects artists allowing you to remove unwanted elements from video clips, such as production equipment, boom microphones, vehicles, people, signs, logos, or special effects wires that have inadvertently been included in a shot.
Powered by Adobe Sensei, the new Content-Aware Fill capability in After Effects replaces masked objects, shadows, or blemishes with backgrounds generated from adjacent pixels and other frames in the clip. Incorporating technologies such as optical flow and 3D tracking, Content-Aware Fill estimates motion and depth of the masked object(s) to intelligently synthesize and assemble a replacement background. In other words, crazy magic.
For 360 VR projects, Content-Aware Fill is an indispensable tool since there is nowhere “off-camera” to hide crew, tripods or lights. Content-Aware Fill can also be used to salvage footage by eliminating visual distractions or fixing continuity issues, like a car driving through the background of a scene, dust on a camera lens, or a misplaced prop.
You may have seen this as a sneak peek at Adobe MAX the past couple years with some of the research and development under the codename Project Cloak, and happily this powerful advancement is now available today in After Effects as a fully-functional feature. It’s very exciting and if you’re like us, it will blow your mind! Check out this quick demo:
Install now: Get all new Creative Cloud 2023 direct download links and free trials.
How to Use Content-Aware Fill for Video
To use Content-Aware Fill, you first mask out the object you want removed (leaving a hole in your video), then render a “Fill” layer to re-fill the hole. Content-Aware Fill is a Tool Panel listed under the “Window” menu in After Effects (not an effect that you apply to a clip like color correction or stabilization). It appears in the list of Tool Panels at the right of the screen in the Default Workspace.
Content-Aware Fill has 3 “Fill Methods” or modes of calculating fill pixels: Object, Surface and Edge Blend. You can also use Reference Frames to help the tool learn what pixels to fill your mask with.
- OBJECT Mode is the most popular mode for removing objects that move over time within your shot. This mode looks at frames before and after to find the pixels it can use to fill your mask.
- SURFACE Mode is best used to replace an object that doesn’t move within the scene – i.e., never reveals what is behind itself (poster on a wall, person standing still, etc).
- EDGE BLEND Mode reads the pixels at the edge of your mask and blurs them into the mask area. This is best used to replace flat areas of color such as the back of a laptop or painted wall.
- The REF FRAMES method still uses Object Mode, but adds the use of a Reference Frame (a user-painted single-still clean plate) to teach the Content-Aware Fill algorithm what pixels to fill in your masked area.
For complex shots, you can mask out more items using different Fill Methods in combination, and/or refine your results with reference frames edited in Photoshop.
See more examples of the amazing Content-Aware Fill for Video in action below, plus the step-by-step on how to use it:
Remember, you can download this today and explore its functionality – and even if you aren’t a Creative Cloud subscriber, you can still download and install the software instantly as a free trial:
- The 10 most common myths about Creative Cloud
- What are the differences between Adobe CC vs. CS6?
- Download free Adobe books (choose over 30 titles)
- Free! Watch 200+ new courses from Adobe MAX
- Comparison guide: CC for teams vs. individuals
- Free Adobe Stock! Download 1,000,000+ top images
Do you have any questions about Content-Aware Fill for Video? Just ask them below and we’ll get you answers fast!
To follow this story going forward, subscribe to our RSS feed or like us on Facebook or Twitter… You can also enter your email and have new articles sent directly to your Inbox. We are reader-supported; when you buy through links on our site, we may earn an affiliate commission.