qccvidrwmhencode.3

QccPack-0.54-1 released (2007-04-30) is being developed and tested on Fedora Core Linux. QccPack pro

.TH QCCVIDRWMHENCODE 1 "QCCPACK" ""
.SH NAME
QccVIDrwmhEncode, QccVIDrwmhDecode \-
encode/decode an image sequence using the RWMH algorithm
.SH SYNOPSIS
.B #include "libQccPack.h"
.sp
.BI "int QccVIDrwmhEncode(QccIMGImageSequence *" image_sequence ", const QccFilter *" filter1 ", const QccFilter *" filter2 ", const QccFilter *" filter3 ", int " subpixel_accuracy ", int " blocksize ", QccBitBuffer *" output_buffer ", int " num_levels ", int " target_bit_cnt ", const QccWAVWavelet *" wavelet " const QccString " mv_filename ", int " read_motion_vectors ", int " quiet );
.sp
.BI "int QccVIDrwmhDecodeHeader(QccBitBuffer *" input_buffer ", int *" num_rows ", int *" num_cols ", int *" start_frame_num ", int *" end_frame_num ", int *" num_levels ", int *" blocksize ", int *" target_bit_cnt );
.sp
.BI "int QccVIDrwmhDecode(QccIMGImageSequence *" image_sequence ", const QccFilter *" filter1 ", const QccFilter *" filter2 ", const QccFilter *" filter3 ", int " subpixel_accuracy ", int " blocksize ", QccBitBuffer *" input_buffer ", int " target_bit_cnt ", int " num_levels ", const QccWAVWavelet *" wavelet ", const QccString " mv_filename ", int " quiet );
.SH DESCRIPTION
.SS Encoding
.LP
.B QccVIDrwmhEncode()
encodes an
.I image_sequence
using the redundant-wavelet-multihypothesis (RWMH)
video-coding algorithm by Cui
.IR "et al" .
Essentially, the RWMH algorithm involves traditional
block-based motion estimation and motion compensation wherein
the redundant phases of the RDWT of the reference frame are used
to provide a multihypothesis estimate of motion based on
the diversity of the transform phases; see "ALGORITHM"
below for greater detail.
.LP
.I image_sequence
is the image sequence to be coded and
should indicate a collection of grayscale images of the same size stored
as separate, numbered files; the
filename indicated by
.I image_sequence
must contain one 
.BR printf (3)-style
numerical descriptor which will then be filled in the current
frame number (e.g., football.%03d.pgm will become
football.000.pgm, football.001.pgm, etc.; see
.BR QccPackIMG (3)).
Each frame of
.I image_sequence
must have a size which is an integer multiple of
.I blocksize
both horizontally and vertically.
Both the
.I start_frame_num
and
.I end_frame_num
fields of
.I image_sequence
should indicate the desired starting and stopping frames, respectively,
for the encoding;
these should either be set manually or via a call to
.BR QccIMGImageSequenceFindFrameNums (3)
prior to calling
.BR QccVIDrwmhEncode() .
.LP
.IR filter1 ,
.IR filter2 ,
and
.IR filter3 
are interpolation filters for supporting the subpixel accuracy specified by
.IR subpixel_accuracy
which can be one of
.BR QCCVID_ME_FULLPIXEL ,
.BR QCCVID_ME_HALFPIXEL ,
.BR QCCVID_ME_QUARTERPIXEL ,
or 
.BR QCCVID_ME_EIGHTHPIXEL ,
indicating full-, half-, quarter-, or eighth-pixel accuracy, respectively.
See "SUBPIXEL ACCURACY" below.
.LP
.I block_size
is the size of square blocks to use in the RWMH algorithm.
.LP
.I output_buffer
is the output bitstream
which must be of
.B QCCBITBUFFER_OUTPUT
type and opened via a prior call to
.BR QccBitBufferStart (3).
.LP
.I num_levels
gives the number of levels of dyadic wavelet decomposition to perform,
and
.I wavelet
is the wavelet to use for decomposition, in all the RDWTs in the
RWMH algorithm.
.LP
The RWMH encoder uses a cross-scale distortion measure to determine
a motion vector for a "set" of spatially co-located blocks
in the RDWT of the current frame.
The number of transform scales used in this cross-scale is
.IR num_levels .
The result of this motion-estimation search is an "all-phase"
motion vector for the set of blocks.
.LP
When encoding the current frame,
.BR QccVIDrwmhEncode()
will first output to
.IR output_buffer
all the motion vectors for the frame, and then
an embedded intraframe encoding of the motion-compensated residual.
.I target_bit_cnt
is the desired number of bits to output for each frame,
including motion vectors and motion-compensated residual.
After
.I target_num_bits
have been produced for the current frame,
.BR QccVIDrwmhEncode()
will call
.BR QccBitBufferFlush (3)
to flush the bit-buffer contents to the output bitstream.
Encoding of the next frame starts on the next byte boundary
of the output bitstream.
.LP
.I mv_filename
gives the filename for files of motion vectors.
If
.I read_motion_vectors
is
.BR FALSE ,
then the motion-vectors are written to
.I mv_filename
via
.BR QccVIDMotionVectorsWriteFile (3).
.I mv_filename
should have a
.BR printf (3)-style
numerical descriptor which will then be filled in with
the current frame number before writing, so that motion vectors
are separated into multiple files, one file per frame.
On the other hand, if
.I read_motion_vectors
is
.BR TRUE ,
then motion vectors are read from
.IR mv_filename
via
.BR QccVIDMotionVectorsReadFile (3),
in which case
.BR QccVIDrwmhEncode()
performs no motion estimation itself, using simply the
motion vectors read from the files for coding.
.LP
If
.IR quiet 
= 0,
.BR QccVIDrwmhEncode()
will print to stdout a number of statistics concerning each frame as it
is encoding.
If
.I quiet
= 1,
this verbose output is suppressed.
.SS Decoding
.BR QccVIDrwmhDecodeHeader()
decodes the header information in a bitstream produced by
.BR QccVIDrwmhEncode() .
The input bitstream is
.I input_buffer
which must be of
.I QCCBITBUFFER_INPUT
type and open via a prior call to
.BR QccBitBufferStart (3).
The header information is returned in
.I num_rows
(vertical size of image-sequence frames),
.I num_cols
(horizontal size of image-sequence frames),
.I start_frame_num
(number of the first frame of the sequence),
.I end_frame_num
(number of the last frame of the sequence),
.I num_levels
(number of wavelet-transform levels),
and
.I target_bit_cnt
(number of bits encoded for each frame).
.LP
.B QccVIDrwmhDecode()
decodes the bitstream
.IR input_buffer ,
reconstructing each image of the output image sequence and writing it
to a separate, numbered grayscale-image file.
The filename denoted by
.IR image_sequence
must contain one
.BR printf (3)-style
numerical descriptor which is filled in with the number of the current
frame being decoded.
The bitstream must already have had its header read by a prior call to
.BR QccVIDrwmhDecodeHeader()
(i.e., you call
.BR QccVIDrwmhDecodeHeader()
first and then
.BR QccVIDrwmhDecode() ).
If
.IR quiet
= 0, then
.BR QccVIDrwmhDecode()
prints a brief message to stdout after decoding each frame; if
.IR quiet
= 1, then this message is suppressed.
.LP
.IR filter1 ,
.IR filter2 ,
and
.IR filter3 
are interpolation filters for supporting the subpixel accuracy specified by
.IR subpixel_accuracy
which can be one of
.BR QCCVID_ME_FULLPIXEL ,
.BR QCCVID_ME_HALFPIXEL ,
.BR QCCVID_ME_QUARTERPIXEL ,
or 
.BR QCCVID_ME_EIGHTHPIXEL ,
indicating full-, half-, quarter-, or eighth-pixel accuracy, respectively.
See "SUBPIXEL ACCURACY" below.
.LP
.I mv_filename
gives the name of files of motion vectors.
If
.I mv_filename
is
.BR NULL ,
then
.BR QccVIDrwmhDecode()
simply decodes the motion vectors to use in decoding from the bitstream
(the usual state of affairs).
On the other hand, if
.IR mv_filename
is not
.BR NULL ,
then the motion vectors stored in the bitstream are ignored and, rather,
the motion vectors are read from
.I mv_filename
instead.
.I mv_filename
should have a
.BR printf (3)-style
numerical descriptor which will then be filled in with
the current frame number before reading via
.BR QccVIDMotionVectorsReadFile (3).
.SH "ALGORITHM"
Multihypothesis motion compensation (MHMC) forms a prediction
in the current frame as a combination of multiple predictions in 
an effort to combat the uncertainty inherent in the motion-estimation (ME) 
process. A number of multihypothesis 
techniques for motion compensation (MC) have been proposed.
One approach to MHMC is to implement multihypothesis prediction in 
the spatial dimensions; i.e., the predictions are culled from 
spatially distinct locations in the reference frame. Included in this class 
of MHMC would be subpixel-accurate MC and overlapped block motion 
compensation (OBMC). Another approach is to deploy MHMC in the 
temporal dimension by choosing predictions from multiple 
reference frames. Examples of this class of MHMC are bidirectional prediction 
(B-frames) and long-term-memory motion 
compensation (LTMMC). Cui
.IR "et al" .
introduced a new class of MHMC by extending the 
multihypothesis-prediction concept into the transform domain.
Specifically, Cui
.IR "et al" .
performed ME/MC in the domain of a redundant, or overcomplete, wavelet 
transform, and used multiple predictions that were diverse in transform 
phase. The term redundant-wavelet multihypothesis (RWMH) was coined to 
describe this approach to phase-diversity multihypothesis.
.LP
In the RWMH approach,
both the current and reference frames are transformed
into RDWT coefficients, and both ME and MC take place in this
RDWT domain. However, before calculating the residual frame,
the motion-compensated frame is mapped from the RDWT domain
back to the spatial domain via a multiple-phase inverse RDWT, and
the residual is calculated and coded in the original spatial domain.
.LP
Intuitively, we observe that each of the critically sampled DWTs within an 
RDWT will "view" motion from a different perspective. Consequently, if motion 
is predicted in the RDWT domain, the multiple-phase inverse RDWT converts
these multiple predictions into a single 
multihypothesis prediction in the spatial domain.
Cui
.IR "et al" .
present an analytic derivation that substantiates this intuition by
quantifying the performance gain of RWMH over single-phase 
prediction. Key to this analysis is that noise in the RDWT domain undergoes a 
substantial reduction in variance when the multiple-phase inverse RDWT is 
applied due to the well-known fact that this pseudo-inverse contains a 
projection onto the range space of the forward transform. Consequently, noise 
not captured by the motion model is greatly reduced in an RWMH system, 
leading to substantial reduction in the prediction-residual variance and 
higher coding efficiency.
.LP
We note that Cui
.IR "et al" .
have improved the performance of RWMH by combining it with other,
more traditional forms of multihypothesis, e.g., OBMC.
In the present implementation of RWMH,
these enhancements have not (yet) been implemented.
.SH "SUBPIXEL ACCURACY"
Due to the linearity of the RDWT,
it is possible to implement subpixel interpolation in the RDWT domain in
a manner similar to as done in the spatial domain to support traditional
subpixel accuracy. 
Specifically, one simply interpolates each subband of the RDWT to subpixel
accuracy independently.
.BR QccVIDrwmhEncode()
and
.BR QccVIDrwmhDecode()
both call
.BR QccVIDMotionEstimationCreateReferenceFrame (3)
for each subband of the RDWT of the reference frame to interpolate the subband
to the accuracy specified by
.IR subpixel_accuracy .
The filters
.IR filter1 ,
.IR filter2 ,
and
.IR filter3
are passed to
.BR QccVIDMotionEstimationCreateReferenceFrame (3)
to control whether filtered interpolation or bilinear interpolation
is performed at each step of the subpixel interpolation.
See
.BR QccVIDMotionEstimationCreateReferenceFrame (3)
for more detail.
.SH "SEE ALSO"
.BR rwmhencode (1),
.BR rwmhdecode (1),
.BR QccWAVWaveletRedundantDWT2D (3),
.BR QccWAVWaveletRedundantDWT2D (3),
.BR QccVIDMotionVectorsReadFile (3),
.BR QccVIDMotionVectorsWriteFile (3),
.BR QccVIDMotionEstimationCreateReferenceFrame (3),
.BR QccSPIHTEncode (3),
.BR QccPackVID (3),
.BR QccPackSPIHT (3),
.BR QccPackWAV (3),
.BR QccPackIMG (3),
.BR QccPack (3)

S. Cui, Y. Wang, and J. E. Fowler,
"Motion Compensation Via Redundant-Wavelet Multihypothesis,"
.IR "IEEE Transactions on Image Processing" ,
submitted March 2004. Revised February 2005.

S. Cui, Y. Wang, and J. E. Fowler,
"Multihypothesis Motion Compensation in the Redundant Wavelet Domain,"
in
.IR "Proceedings of the International Conference on Image Processing" ,
Barcelona, Spain, September 2003, vol. 2, pp. 53-56.

H.-W. Park and H.-S. Kim,
"Motion Estimation Using Lowband-Shift Method for
Wavelet-Based Moving-Picture Coding,"
.IR "IEEE Transactions on Image Processing" ,
vol. 9, no. 4, pp. 577-587, April 2000.

.SH AUTHOR
Written by Joe Boettcher <[e-mail]> based on
the originally developed algorithm and code by Suxia Cui.

Copyright (C) 1997-2007  James E. Fowler
.\"  The programs herein are free software; you can redistribute them and/or
.\"  modify them under the terms of the GNU General Public License
.\"  as published by the Free Software Foundation; either version 2
.\"  of the License, or (at your option) any later version.
.\"  
.\"  These programs are distributed in the hope that they will be useful,
.\"  but WITHOUT ANY WARRANTY; without even the implied warranty of
.\"  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
.\"  GNU General Public License for more details.
.\"  
.\"  You should have received a copy of the GNU General Public License
.\"  along with these programs; if not, write to the Free Software
.\"  Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.