Streaming Media Bible - Streaming Magazine Column

Streaming Magazine Column - January 2003


	Table of Contents

	Chapter Excerpt

	Updates

	Order the Book

	Resources

	Tips & Tricks

	Streaming Magazine Column

	Feedback

	Author

	Home

Codecs Don't Have Eyes or Ears.

A lot of ink has been spilled in this column describing tools & techniques you can employ to produce the best quality streaming media files. In doing so, however, I've broken one of my cardinal rules - don't just tell people what to do, tell them why they're doing it. Give people a fundamental understanding of a technology and they're better able to utilize it.

Perhaps that's not altogether true. I've tried to explain why you should employ certain techniques, but I have left out one key concept - the codecs. A codec (a contraction of compressor-decompressor or coder-decoder) is a piece of software that converts raw media into a format that can be streamed. To do this, a codec must make decisions about what information in a file can be abbreviated or thrown away in an effort to reduce the size of a file.

The Ugly Truth

The amount of data reduction required for streaming media can be mind-boggling. Take for instance a CD-quality file. Raw CD audio contains two channels of data sampled at 44,100 times per second, each sample being 16 bits. The data rate of a CD is therefore

2 * 16 bits/sample * 44,100 samples/second = 1,411,200 bits/second

That's 1.35 Megabits per second. To stream this to a dialup user, who has a whopping 32 kilobits per second available, the data must be reduced by more than a factor of 40. There are certain things that can be done to make the job easier, such as converting the signal to mono instead of stereo, and reducing the frequency range, but neither of these is sufficient on its own for the amount of data reduction required. To achieve the necessary result, codecs have to rely on perceptual models.

Perceptual Models

Our ears can hear a dripping faucet in the next room, or pick out conversations across a crowded restaurant. We can see on bright sunny days, as well as find our way around in the middle of the night. We can do these things because our eyes and ears are incredibly sensitive and because our brain continuously filters the deluge of information our eyes and ears present to us.

For example, when it comes to hearing, our brains treat whatever is loudest as most important. If someone next to you sneezes, you won't hear someone else nearby crumpling a piece of paper. This is partially because the sneeze is louder and masks other sound, but also because our brain pays more attention to the sneeze. Knowing this, an audio codec can discard portions of the audio spectrum that are masked, and things that we don't perceive.

When it comes to vision, motion and color are most important. It is easier for us to percieve things that are moving, and things that are bright and colorful. Using this as a model, a video codec dedicates more bits to things that move rather than static backgrounds. Our eyes are also less sensitive to high-frequencies, so codecs can economize the number of bits allotted to the upper part of the visible frequency spectrum.

By taking into account the ways our eyes and ears function, and how our brains perceive the input that is delivered, codecs can make better decisions about how to reduce the data contained within an audio or video file.

Tying It All Together

If you've read any of the previous columns, some of the tips and techniques described should now make more sense. All the audio techniques are centered on making sure your audio signal is clean and your programming is loud. This virtually guarantees that the audio codec dedicates its resources to encoding your content, and that unwanted background noise is ignored.

The video techniques described have centered on minimizing unnecessary motion, and getting your desired subject correctly lit. Again, this helps a video codec make the right decisions. If your background is stationary, the video codec won't waste unnecessary resources on encoding it.

Make sense? It's really pretty simple once you get the hang of it.

Next Month: Authoring - Keep it Simple