mpeg_lib.tex

mpeng Lib

\documentclass[11pt]{article}
\usepackage{fullpage}
\usepackage{palatino}
\usepackage{psfig}

\title{The MPEG Library\\
       Version 1.2\footnote{Also covers version 1.3}}
\author{Greg Ward\\({\tt [e-mail]})}
\date{February, 1996}

%\renewcommand{\familydefault}{cmss}
%\renewcommand{\bfdefault}{b}   % make ``bold'' come out medium-width

% \code - for typesetting a little bit of computer code (in typewriter font)
\newcommand{\code}[1]{\texttt{#1}}


% ttdescription - a description-like environment where item descriptions are
% typeset using "\tt", followed by a colon

\newcommand{\ttlabel}[1]{\texttt{#1:}\quad\hfil}
\newenvironment{ttdescription}[1] {\newbox\holder
  \setbox\holder=\hbox{\ttlabel#1} \dimen0=\wd\holder
 \begin{list}{}
   {\labelsep=-0.25in \rightmargin=0.25in \leftmargin=\dimen0
     \addtolength{\leftmargin}{0.25in}
  \labelwidth=\leftmargin
  \let\makelabel\ttlabel}}%     this comment is needed to hide the newline
{\end{list}}

% \prototype - for typesetting a function prototype
\newcommand{\prototype}[1]{%
  \textbf{Function prototype:}\par \smallskip \code{#1}\par\medskip }

\newenvironment{Arguments}[1]{%
\noindent\textbf{Arguments:}%
\begin{ttdescription}{#1}}
{\end{ttdescription}\medskip}

\newenvironment{Notes}{%
\noindent\textbf{Notes:}\par\smallskip}
{\medskip}

\begin{document}

\maketitle

\tableofcontents

\section{Introduction and Background}

The MPEG Library is based on an effort from the University of
California at Berkeley to create a portable, software-based MPEG
decoder  \cite{Patel93}.  This resulted in the widely distributed (and
widely modified) \code{mpeg\_play}, a highly-optimized MPEG decoder
that was specifically geared towards displaying under X Windows.  The
value of having a portable, software MPEG decoder is amply
demonstrated by the number of programs that have been adapted from
this original Berkeley source (including ports to the Linux SVGA
library, Silicon Graphics hardware, and a non-display MPEG information
utility).  However, the utility of the decoder was limited by the
difficulty of extracting the useful, MPEG-related source code from the
X11-specific, display-related source.  Essentially what was needed was
a simple interface that would allow a programmer to extract frames
from an MPEG stream (either before or after converting to RGB colour
space), and then to do with the image data as he or she saw fit.

The MPEG Library is intended to fill this need.  It was developed at
the Montreal Neurological Institute in the summer of 1994 in order to
facilitate the development of a high-performance, feature-heavy
MPEG player for Silicon Graphics workstations.  Since then, the
Library has found a use in numerous applications, notably as one of
several optional libraries used for extending the well-known
ImageMagick suite of graphics applications.

\section{Programming with the MPEG Library}

Using the Library is quite straightforward, and is analogous to the
way in which files have been traditionally handled: you open an MPEG
stream to initialize internal data structures, and then read frames
until the stream is exhausted.  At any point, you can rewind the
stream to start over; however, random access is not allowed.  (This is
not due to a fundamental weakness with MPEG; however, due to the
nature of the decoding engine at the heart of the MPEG Library, don't
expect to see it implemented here any time soon.)  When you are
finished with the stream, you close it to clean up.

Here is a simple example program to open an MPEG stream (named by the
first command-line argument) and read all frames from it.  Since
displaying images is as non-portable as it is desirable, I have
included calls to dummy routines \code{InitializeDisplay()} and
\code{ShowFrame()}; actually defining these is up to you.  
\begin{verbatim}
#include <stdio.h>
#include "mpeg.h"

int main (int argc, char *argv[])
{
   FILE       *mpeg;
   ImageDesc   img;
   Boolean     moreframes = TRUE;
   char       *pixels;

   mpeg = fopen (argv[1], "r");
   SetMPEGOption (MPEG_DITHER, FULL_COLOR_DITHER);
   OpenMPEG (mpeg, &img);

   InitializeDisplay (img.Width, img.Height);
   pixels = (char *) malloc (img.Size);
   while (moreframes)
   {
      moreframes = GetMPEGFrame (pixels);
      DisplayFrame (img.Width, img.Height, pixels)
   }
   CloseMPEG ();
   fclose (mpeg);
}
\end{verbatim}

For a concrete example, you might wish to consult \code{easympeg.c}, a
very simple SGI-specific MPEG player included with the Library.  Also,
I have omitted any error-checking or handling here; again, consult
\code{easympeg.c} for a more realistic example.\footnote{For an even
more realistic (but of course considerably larger) example, take a
look at \code{glmpeg\_play}.  This is the full-featured MPEG player
that was the impetus for creating the MPEG Library; it is available by
from the same location as the library itself: 
\code{ftp://ftp.bic.mni.mcgill.ca/pub/mpeg}.}

Note in particular the following points about the above code:
\begin{itemize}
\item The caller must take care of opening and closing the file
containing the MPEG stream; the Library assumes that it is passed a
file ready for reading.
\item The \code{ImageDesc} structure contains all the information that
should be needed to display frames from the MPEG stream (although not
necessarily all the information you could possibly want to know about
an MPEG stream).  
%In particular, the fields \code{Height} and
%\code{Width} are the height and width of each frame in pixels;
%\code{Depth} is the depth (in bits) of each pixel; \code{PixelSize} is
%the actual number of bits stored per pixel; \code{Size} is the size in
%bytes of each entire decoded and uncompressed frame; and
%\code{BitmapPad} gives the ``quantum'' of a scan line.  (Each scan
%line starts on an even multiple of this many bits.)
\item \code{SetMPEGOption()} can be used to control somewhat the
decoding of frames.  In addition to selecting a dithering mode, you
can also select the luminance and chrominance ranges used for
dithering.  Also, note that \code{SetMPEGOption()} should be called
{\em before} \code{OpenMPEG()} when setting the dithering method.
\item The MPEG data can be decoded using a variety of dithering
methods.  (Note that in this context, {\em dithering\/} refers to converting
from the luminance-chromaticity, or YCrCb, colour space in which MPEG
data is encoded, to the more conventional RGB scheme.)
\item You don't need to pass any parameters to \code{GetMPEGFrame()}
  or \code{CloseMPEG()} to tell it which MPEG stream you mean; this is
  because the Berkeley decoding engine (and hence the MPEG Library
  itself) depends heavily on global variables, and unfortunately
  cannot decode more than one MPEG at a time.
\end{itemize}
%Most of these
%involve both pixel values (which are filled in by calls to
%\code{GetMPEGFrame()} and a colour map which maps pixel values to RGB
%values.  The scheme used here, which is the default, uses
%\code{FULL\_COLOR\_DITHER} to return pixels as RGB triples; no colour
%map is involved.  The other useful dithering modes are
%\code{ORDERED\_DITHER} and \code{GRAY\_DITHER}, both of which offer
%better performance and consume less memory at the expense of image
%quality.

More detailed information is provided in sections below.

\section{Concepts and Data Formats}

This section deals with the main concepts needed to control the MPEG
Library and to display the data it returns.  It does {\em not\/} deal
with the details of how MPEG streams are encoded, stored, or decoded.

\subsection{Dithering modes}
\label{sec:dithering}

A large number of dithering modes (in fact, all the modes provided by
the original \code{mpeg\_play}) are available.  A few produce
nonsensical results, but all have been fully tested in the context of
the MPEG Library and found to agree with the results given by
\code{mpeg\_play}.

``Dithering'' in this context is the conversion from
luminance-chrominance colour space (aka YCrCb, YIQ, or YUV, which is how
MPEG streams are encoded and is the same space used by NTSC television
signals) to some form of RGB space.  The implementors of
\code{mpeg\_play} found that outright conversion to red/green/blue
values takes both more time and memory than any other method they
experimented with, so most modes are colour mapped.  This means that
\code{OpenMPEG()} will create a colour map which can be accessed by the
user via the \code{ColormapEntry} pointer in \code{ImageDesc}, and that
the pixel values returned by \code{GetMPEGFrame()} are indeces into this
colour map.  The dithering mode affects the quality of the decoded
images, the number of bits used per pixel, and the colour depth of the
image.

The dithering mode is selected with \code{SetMPEGOption()}, using the
\code{MPEG\_DITHER} option and one of the following values:
\begin{ttdescription}{FULL\_COLOR\_DITHER}

\item[ORDERED\_DITHER] 8-bit colour-mapped; reasonable quality;
  decoding is almost as fast as \code{GRAY\_DITHER}
\item[ORDERED2\_DITHER] 8-bit colour-mapped; reasonable quality
\item[MBORDERED\_DITHER] 8-bit colour-mapped; reasonable quality
\item[FS4\_DITHER] 8-bit colour-mapped; colours are all wrong
\item[FS2\_DITHER] 8-bit colour-mapped; colours are all wrong
\item[FS2FAST\_DITHER] 8-bit colour mapped using Floyd-Steinberg error
  diffusion; reasonable quality
\item[HYBRID\_DITHER] 8-bit colour-mapped; passable colour
\item[HYBRID2\_DITHER] 8-bit colour-mapped; slightly worse than
  \code{HYBRID\_DITHER}
\item[Twox2\_DITHER] 8-bit colour-mapped with pixels doubled; poor
  quality
\item[GRAY\_DITHER] a 256-shade grayscale rendering; nice
  quality and fastest decoding
\item[FULL\_COLOR\_DITHER] a high-quality 24-bit colour rendering;
  results in slowest decoding
\item[MONO\_DITHER] 1-bit monochrome dithering; use as last resort for
  1-bit displays
\item[THRESHOLD\_DITHER] ??
\end{ttdescription}
The descriptions here are my entirely subjective judgments of the image
quality with each dithering mode.  ``Reasonable'' quality is better than
``passable.''  Your mileage may vary.  ``8-bit'' or ``24-bit'' here
refers to the colour depth in the final images, i.e. the minimum number
of bits allocated to each pixel.  Authoritative information about the
actual pixel size can be found in the \code{ImageDesc} structure filled
in by \code{OpenMPEG()}; for instance, if you select
\code{FULL\_COLOR\_DITHER}, the colour depth is 24 bits, but 32 bits are
allocated per pixel.  Thus, the \code{PixelSize} field in
\code{ImageDesc} will be 32, and the \code{Depth} field will be 24.

Note that the dithering mode must be set {\em before\/} \code{OpenMPEG()}
is called.  For example, to select gray-scale dithering and then open
the file \code{example.mpg} as an MPEG stream:
\begin{verbatim}
     char       filename[] = "example.mpg";
     FILE       *mpeg;
     ImageDesc  image;

     mpeg = fopen (filename, "r");
     SetMPEGOption (MPEG_DITHER, (int) GRAY_DITHER);
     OpenMPEG (&image);
\end{verbatim}

\subsection{Colour maps}
\label{sec:colour_maps}

Most dithering modes result in images whose pixel values are indeces to an
8-bit colour map.  This colour map is accessed via the \code{ImageDesc}
structure, and it is created by \code{OpenMPEG()} based on the dithering type
selected by \code{SetMPEGOption()} (this is why the dithering type must be
set before calling \code{OpenMPEG()}).  Note that ``8-bit'' refers to
the size of the colour map: each pixel in the colour-mapped images is 8
bits long, so the colour map itself has 256 entries.