Randy Coppinger

ProTools and H.264 Video

There seems to be plenty of curiosity about video in ProTools, especially the H.264 codec. I’ve previously identified problems using H.264 in ProTools, but let’s take a wider look at how these two interact.

TASTES GREAT / LESS FILLING
If you’ve ever compared video codecs you know H.264 is compact for the quality level it affords. That explains why it’s so popular. The trick is Variable Bit Rate, meaning that each successive frame of picture isn’t fully represented in the data. Most of the time the percentage of stuff that changes frame to frame isn’t high, so the codec just saves the difference. On a static video shot the amount of data needed over time is relatively low. Lots of cuts and motion will require more data. The point is: the amount of memory varies from frame to frame. This can be very efficient in terms of storage, but not in terms of processing overhead. By comparison, Motion JPEG-A is basically the same amount of data for every frame. The files are much larger but it’s easier for your computer to render the image because the data stream is consistent. H.264 requires your computer to work harder, taking more horsepower.

SOUND WITH PICTURE
If that Variable Bit Rate trick sounds familiar it should. AAC (also known as MP4 audio) can vary the bit rate to help keep things smaller. Now take those two codecs and put them together. If the picture is H.264 and the audio is AAC your computer has the increased overhead of decoding two complex streams of data AND it has to keep them in sync! If you have experienced problems with sluggish H.264 video in ProTools, definitely keep away from AAC encoded audio.

WHAT’S A FRAME BETWEEN FRIENDS?
Because the data for a given frame is spread across several other frames, random access of H.264 gets tricky. If you drop somewhere in the ProTools timeline that’s in the middle of an H.264 stream the computer might not have enough data for that frame. So your computer looks back (or maybe forward) to pick up the missing data. That makes it tough for the computer to know exactly which frame to render at any given location. For that reason you may see the wrong frame when you drop the cursor interframe. Let’s say you’re spotting a face slap sound effect. You probably don’t want to be a frame off sync, but that’s exactly what can happen. People who are serious about sync — notably film post and game audio professionals — find H.264 a deal breaker for this reason.

PROTOOLS AND VIDEO
Why is it a video will play fine on it’s own, then die horribly in ProTools? Tough question, but the heart of the issue is how ProTools deals with video.

The ability to play video is built into your OS, not ProTools. So when you play video, ProTools hands off the task. If that task is heavy because you’ve got a big, high resolution video with a variable bit rate, your machine will get busy. When you’re not running ProTools, the video might play fine. But ProTools is a resource hog. If you ask your machine to run ProTools (which tends to tax it) and then ProTools hands off the video playback task (further taxing it), your ProTools may get sluggish. Faster processors, efficient code and tons of memory help video that used to choke machines a few years ago run pretty smoothly today. Moore’s Law means this kind of problem will be less of an issue in the future, but understanding why ProTools bogs down can help you problem solve when you have video trouble.

BEST PRACTICES
By all means, use H.264 to encode and throw those tiny files around to everyone one who needs video. It’s great quality at a low memory price, for quicker up/downloads.
If you need to be frame accurate, convert H.264 to something else before importing into ProTools. I like Motion JPEG-A.
If you use H.264 video and it makes your ProTools rig too slow, first check to see if the audio is AAC. If it is, convert your video or ask for new video with constant bit rate audio.
Slow ProTools and H.264 may simply indicate the limits of your setup. Convert or ask for new video with a smaller frame size, lower quality and/or constrained data rate. In other words, lower the complexity of the stream.
If you’re still having problems with H.264 video after all of that, I say give up. Go ahead and convert to another codec, such as Motion JPEG-A. MPEG Streamclip is free for Mac and Windows, reliable, and can batch convert. (More video conversion options here.)

UPDATE Aug 16, 2012
A video codec based on H.264, called High Efficiency Video Coding (HEVC), is expected from the Motion Pictures Expert Group in 2013.

See also- ProTools + H.264 video = Problem, ProTools: Bounce to Quicktime Movie with Full Options, Conflict: ProTools and Spotlight Indexing, ProTools Sync: The Short Video Problem

Do you have a Pinterest board to collect articles like this one? Cool.