vp3-format.txt

mediastreamer2是开源的网络传输媒体流的库

(0, -1). The next 3 bits are 6 which indicate 8 + next 3 bits (7) with
another bit indicating sign (1 in this case, which is negative). Thus,
the X MV component is -15. The next 3 bits are 4 which indicate a Y MV
component of 3 with one more bit for the sign (0 is positive). So the
second motion vector encoded in this stream is (-15, 3).

As an example of the fixed-length entropy method, consider the following
contrived bitstream:

  1010101101010...

The stream is read as:

  1  01010 1  10101 0

The first bit indicates the fixed length entropy method. The first 5 bits
are 10 followed by a negative sign bit. The next 5 bits are 21 followed by
a positive sign bit. The first motion vector in this stream is (-10, 21).

During this phase of the decoding process, it is traditional to assign all
motion vectors for all coded macroblocks that require them, whether they
are unpacked from the motion vector bitstream or copied from previous
coded macroblocks. It is necessary to track the motion vectors for both
the previous macroblock as well as the next-to-last (prior) macroblock.
The general algorithm for this phase is as follows:

  foreach coded macroblock
    last MV = 0
    prior last MV = 0
    if coding mode = MODE_INTER_PLUS_MV or MODE_GOLDEN_MV
      read current MV pair from the bitstream and set all fragment motion
        vectors to that pair
      prior last MV = last MV
      last MV = current MV
    
    if coding mode = MODE_INTER_FOURMV
      read MV for first Y fragment in macroblock
      read MV for second Y fragment in macroblock
      read MV for third Y fragment in macroblock
      read MV for fourth Y fragment in macroblock
      set U & V fragment motion vectors to average of 4 Y vectors,
        calculated as follows:
        if sum of all 4 X motion components is positive, the X
        motion component for the U & V fragments is (sum + 2) / 4,
        otherwise, it is (sum - 2) / 4; repeat the same process for the
        Y components
      prior last MV = last MV
      last MV = MV for fourth Y fragment from this macroblock
      
    if coding mode = MODE_INTER_LAST_MV
      motion vectors for this macroblock are the same as last MV; note
        that in this case, the last MV remains the last MV and the prior
        last MV remains the prior last MV
      
    if coding mode = MODE_INTER_PRIOR_LAST
      motion vectors for this macroblock are the same as prior last MV
      prior last MV = last MV
      last MV = current MV (effectively, swap last and prior last vectors)


Unpacking The DCT Coefficients
------------------------------
After unpacking the macroblock motion vectors, the decoder unpacks the
fragment DCT coefficient data. Each coded fragment has 64 DCT 
coefficients. Some of the coefficients will be non-zero. Many of the 
coefficients will, or should be 0 as this is where the coding method
derives much of its compression.

During this phase, the decoder will be unpacking DCT coefficients, zero
runs, and end-of-block (EOB) codes. The decoder unpacks the the DC 
coefficients for all fragments, then all of the first AC coefficients, 
and so on until all of the 64 DCT coefficients are unpacked from the
bitstream.

To obtain the DCT coefficients, the decoder unpacks a series of VLCs
from the bitstream which turn into a series of tokens ranging from
0..31. Each of these tokens specifies which action to take next. VP3
defines 80 different 32-element histograms for VLC decoding:

  16 histograms for DC token decoding
  16 histograms for group 1 AC token decoding
  16 histograms for group 2 AC token decoding
  16 histograms for group 3 AC token decoding
  16 histograms for group 4 AC token decoding

The decoder fetches 4 bits from the bitstream that will be used to
select a DC histogram and 4 bits that will be used to select 4 AC
histograms, one for each AC group.

The meaning of each of the 32 possible tokens follows. 'EB' stands for
extra bits read from bitstream directly after the VLC token:

0, DCT_EOB_TOKEN
set the current block to EOB, meaning that the block is marked as being
fully unpacked

1, DCT_EOB_PAIR_TOKEN
set the next 2 blocks to EOB

2. DCT_EOB_TRIPLE_TOKEN
set the next 3 blocks to EOB

3, DCT_REPEAT_RUN_TOKEN
set the next (2 EBs + 4) blocks to EOB

4, DCT_REPEAT_RUN2_TOKEN
set the next (3 EBs + 8) blocks to EOB

5, DCT_REPEAT_RUN3_TOKEN
set the next (4 EBs + 16) blocks to EOB

6, DCT_REPEAT_RUN4_TOKEN
set the next (12 EBs) blocks to EOB

7, DCT_SHORT_ZRL_TOKEN
skip (3 EBs + 1) positions in the output matrix

8, DCT_ZRL_TOKEN
skip (6 EBs + 1) positions in the output matrix

9, ONE_TOKEN
output 1 as coefficient

10, MINUS_ONE_TOKEN
output -1 as coefficient

11, TWO_TOKEN
output 2 as coefficient

12, MINUS_TWO_TOKEN
output -2 as coefficient

13, 14, 15, 16, LOW_VAL_TOKENS
next EB determines coefficient sign; coeff = DCT_VAL_CAT2_MIN (3) +
(token - 13) (this gives a range of +/- 3..6)

17, DCT_VAL_CATEGORY3
next EB determines coefficient sign; coeff = DCT_VAL_CAT3_MIN (7) + next
EB (this gives a range of +/- 7..8)

18, DCT_VAL_CATEGORY4
next EB determines coefficient sign; coeff = DCT_VAL_CAT4_MIN (9) + next
2 EBs (this gives a range of +/- 9..12)

19, DCT_VAL_CATEGORY5
next EB determines coefficient sign; coeff = DCT_VAL_CAT5_MIN (13) +
next 3 EBs (this gives a range of +/- 13..20)

20, DCT_VAL_CATEGORY6
next EB determines coefficient sign; coeff = DCT_VAL_CAT6_MIN (21) +
next 4 EBs (this gives a range of +/- 21..36)

21, DCT_VAL_CATEGORY7
next EB determines coefficient sign; coeff = DCT_VAL_CAT7_MIN (37) +
next 5 EBs (this gives a range of +/- 37..68)

22, DCT_VAL_CATEGORY8
next EB determines coefficient sign; coeff = DCT_VAL_CAT8_MIN (69) +
next 9 EBs (this gives a range of +/- 69..580)

23, 24, 25, 26, 27, DCT_RUN_CATEGORY1
coefficient of +/- 1 preceded by a number of 0s; next EB determines sign
of coefficient; skip (token - 22) 0s in the output matrix before
placing the final coefficient (this gives a range of 1..5 0s)

28, DCT_RUN_CATEGORY1B
coefficient of +/- 1 preceded by a number of 0s; next EB determines sign
of coefficient; skip (next 2 EBs + 6) 0s in the output matrix before
placing the final coefficient (this gives a range of 6..9 0s)

29, DCT_RUN_CATEGORY1C
coefficient of +/- 1 preceded by a number of 0s; next EB determines sign
of coefficient; skip (next 3 EBs + 10) 0s in the output matrix before
placing the final coefficient (this gives a range of 10..17 0s)

30, DCT_RUN_CATEGORY2
coefficient of +/- 2..3 preceded by a single zero; next EB determines
sign of coefficient; coefficient = (next EB + 2)

31, DCT_RUN_CATEGORY2B (not specifically named in VP3 source)
coefficient of +/- 2..3 preceded by 2 or 3 0s; next EB determines
sign of coefficient; coefficient = (next EB + 2); skip (next EB + 2) 0s
before placing coefficient in output matrix

Note: EOB runs can, and often do, cross threshold stages and plane
boundaries. For example, a decoder may have decoded all of the AC #2
coefficients for all fragments and still have an EOB run of 2. That
means that during the AC #3 decode process, the first 2 coded fragments
that are not already EOB will be set to EOB.

Let's work through a highly contrived example to illustrate the
coefficient decoding process.



[not finished]




When the decoder is finished unpacking the DCT coefficients, the entire
encoded VP3 frame bitstream should be consumed.


Reversing The DC Prediction
---------------------------
Now that all of the DCT coefficient data has been unpacked, the DC
coefficients need to be fully reconstructed before the IDCT can be
performed.

VP3 uses a somewhat involved process for DC prediction which uses up to
four DC coefficients from surrounding fragments. For each fragment to be
transformed with the IDCT, the DC coefficient is predicted from weighted
sum of the DC coefficients in the left (l), up-left (ul), up (u), and
up-right (ur) fragments, if they are coded (not unchanged from the
previous frame) in a compatible frame (current, previous, or golden).

In a golden frame, the prediction is quite straightforward since all
fragments will be coded. A fragment's DC prediction will fall into 1 of
5 groups:

     abbbbbbbbb
     cdddddddde
     cdddddddde
     cdddddddde
     cdddddddde

* Group a is the top left corner fragment. There is nothing to predict
from. This DC coefficient has a lot of energy and requires many bits to
code.

* Group b is the remainder of the top row of fragments. These fragments
can only predict from the left fragment.

* Group c is the left column of fragments, not including the top left
fragment. These fragments have the top and top-right fragments from
which to predict.

* Group d is the main body of fragments. These fragments have access to
all 4 predictors.

* Group e is the right column of fragments, not including the top right
fragment. These fragments can predict from the left, up-left and up
fragments.

The process of reversing prediction for interframes grows more complex.
First, the decoder must evaluate which candidate fragments (l, ul, u, or
ur) are available for as predictors. Then, it can only use fragments
that are coded within the same frame (current, previous, or golden).
Further, there are auxiliary predictors for each frame type that are
initialized to 0 at the start of each video frame decode operation. The
decoder falls back on these auxiliary predictors when it can not find
any valid candidate predictors for the current fragment.

To work through some examples, consider the following notation, e.g.:

  ul-C = up-left fragment, coded in the current frame
   u-P = up fragment, coded as a motion residual from the previous frame
  ur-C = up-right fragment, coded in the current frame
   l-G = left fragment, coded as a motion residual from the golden frame   
   x-P = current fragment where DC prediction is being performed, coded
         as a motion residual from the previous frame

This is a simple case:

   ul-C   u-C  ur-C
    l-C   x-C

The current fragment predicts from all four of the candidate fragments
since they are coded in the same frame. 

   ul-P   u-C  ur-C
    l-P   x-P

The current fragment predicts from the left and up-left fragments.

   ul-C   u-P  ur-G
    l-P   x-G

The current fragment predicts from the up-right fragment.

   ul-C   u-C  ur-C
    l-C   x-G

The current fragment does not predict from any of the candidate
fragments since the current fragment is a motion residual from the
golden frame. Rather, add the auxiliary golden frame predictor to the
current fragment's DC coefficient. Save the new DC coefficient as the
new golden frame auxiliary DC predictor.

If the decoder only finds one valid candidate predictor, then it is used
by itself. When the decoder finds multiple valid candidate fragments
from which to predict DC, it applies a weighting function to the
surrounding fragments' DC coefficients. The following table presents all 
16 possible combinations of available/not available predictors and what 
to do in each case:

    ul   u  ur   l
    --  --  --  --
     0   0   0   0    no predictors available:
                        use the last predictor saved for the frame type
                        (either intra, inter, or golden)

     0   0   0   1    left predictor available:
                        pred = l.dc

     0   0   1   0    up-right predictor available:
                        pred = ur.dc

     0   0   1   1    up-right, left predictors available:
                        pred = (53 * ur.dc) + (75 * l.dc)
                               --------------------------
                                         128

     0   1   0   0    up predictor available:
                        pred = u.dc

     0   1   0   1    up, left predictors available:
                        pred = (u.dc + l.dc)
                               -------------
                                     2

     0   1   1   0    up, up-right predictors available:
                        discard up-right predictor
                        pred = u.dc

     0   1   1   1    up, up-right, left predictors available: