tutorial03.java

ffmpeg开发指南

        }
      }
      SDL_UnlockMutex(q->mutex);
      return ret;
    }
    

As you can see, we've wrapped the function in a forever loop so we will be
sure to get some data if we want to block. We avoid looping forever by making
use of SDL's SDL_CondWait() function. Basically, all CondWait does is wait for
a signal from SDL_CondSignal() (or SDL_CondBroadcast()) and then continue.
However, it looks as though we've trapped it within our mutex -- if we hold
the lock, our put function can't put anything in the queue! However, what
SDL_CondWait() also does for us is to unlock the mutex we give it and then
attempt to lock it again once we get the signal. 

### In Case of Fire

You'll also notice that we have a global quit variable that we check to make
sure that we haven't set the program a quit signal (SDL automatically handles
TERM signals and the like). Otherwise, the thread will continue forever and
we'll have to kill -9 the program. ffmpeg also has a function for providing a
callback to check and see if we need to quit some blocking function:
url_set_interrupt_cb. 
    
    
    int decode_interrupt_cb(void) {
      return quit;
    }
    ...
    main() {
    ...
      url_set_interrupt_cb(decode_interrupt_cb);  
    ...    
      SDL_PollEvent(&event);
      switch(event.type) {
      case SDL_QUIT:
        quit = 1;
    ...
    

This only applies for ffmpeg functions that block, of course, not SDL ones. We
make sure to set the quit flag to 1. 

### Feeding Packets

The only thing left is to set up our queue: 
    
    
    PacketQueue audioq;
    main() {
    ...
      avcodec_open(aCodecCtx, aCodec);
    
      packet_queue_init(&audioq);
      SDL_PauseAudio(0);
    

SDL_PauseAudio() finally starts the audio device. It plays silence if it
doesn't get data; which it won't right away. 

So, we've got our queue set up, now we're ready to start feeding it packets.
We go to our packet-reading loop: 
    
    
    while(av_read_frame(pFormatCtx, &packet)>=0) {
      // Is this a packet from the video stream?
      if(packet.stream_index==videoStream) {
        // Decode video frame
        ....
        }
      } else if(packet.stream_index==audioStream) {
        packet_queue_put(&audioq, &packet);
      } else {
        av_free_packet(&packet);
      }
    

Note that we don't free the packet after we put it in the queue. We'll free it
later when we decode it. 

### Fetching Packets

Now let's finally make our audio_callback function to fetch the packets on the
queue. The callback has to be of the form void callback(void *userdata, Uint8
*stream, int len), where userdata of course is the pointer we gave to SDL,
stream is the buffer we will be writing audio data to, and len is the size of
that buffer. Here's the code: 
    
    
    void audio_callback(void *userdata, Uint8 *stream, int len) {
    
      AVCodecContext *aCodecCtx = (AVCodecContext *)userdata;
      int len1, audio_size;
    
      static uint8_t audio_buf[(AVCODEC_MAX_AUDIO_FRAME_SIZE * 3) / 2];
      static unsigned int audio_buf_size = 0;
      static unsigned int audio_buf_index = 0;
    
      while(len > 0) {
        if(audio_buf_index >= audio_buf_size) {
          /* We have already sent all our data; get more */
          audio_size = audio_decode_frame(aCodecCtx, audio_buf,
                                          sizeof(audio_buf));
          if(audio_size < 0) {
    	/* If error, output silence */
    	audio_buf_size = 1024;
    	memset(audio_buf, 0, audio_buf_size);
          } else {
    	audio_buf_size = audio_size;
          }
          audio_buf_index = 0;
        }
        len1 = audio_buf_size - audio_buf_index;
        if(len1 > len)
          len1 = len;
        memcpy(stream, (uint8_t *)audio_buf + audio_buf_index, len1);
        len -= len1;
        stream += len1;
        audio_buf_index += len1;
      }
    }
    

This is basically a simple loop that will pull in data from another function
we will write, audio_decode_frame(), store the result in an intermediary
buffer, attempt to write len bytes to stream, and get more data if we don't
have enough yet, or save it for later if we have some left over. The size of
audio_buf is 1.5 times the size of the largest audio frame that ffmpeg will
give us, which gives us a nice cushion. 

### Finally Decoding the Audio

Let's get to the real meat of the decoder, audio_decode_frame: 
    
    
    int audio_decode_frame(AVCodecContext *aCodecCtx, uint8_t *audio_buf,
                           int buf_size) {
    
      static AVPacket pkt;
      static uint8_t *audio_pkt_data = NULL;
      static int audio_pkt_size = 0;
    
      int len1, data_size;
    
      for(;;) {
        while(audio_pkt_size > 0) {
          data_size = buf_size;
          len1 = avcodec_decode_audio2(aCodecCtx, (int16_t *)audio_buf,
&data_size, 
    				  audio_pkt_data, audio_pkt_size);
          if(len1 < 0) {
    	/* if error, skip frame */
    	audio_pkt_size = 0;
    	break;
          }
          audio_pkt_data += len1;
          audio_pkt_size -= len1;
          if(data_size <= 0) {
    	/* No data yet, get more frames */
    	continue;
          }
          /* We have data, return it and come back for more later */
          return data_size;
        }
        if(pkt.data)
          av_free_packet(&pkt);
    
        if(quit) {
          return -1;
        }
    
        if(packet_queue_get(&audioq, &pkt, 1) < 0) {
          return -1;
        }
        audio_pkt_data = pkt.data;
        audio_pkt_size = pkt.size;
      }
    }
    

This whole process actually starts towards the end of the function, where we
call packet_queue_get(). We pick the packet up off the queue, and save its
information. Then, once we have a packet to work with, we call
avcodec_decode_audio2(), which acts a lot like its sister function,
avcodec_decode_video(), except in this case, a packet might have more than one
frame, so you may have to call it several times to get all the data out of the
packet. *** A note: **Why avcodec_decode_audio**2**? Because there used to be
an avcodec_decode_audio, but it's now deprecated. The new function reads from
the data_size variable to figure out how big audio_buf is. Also, remember the
cast to audio_buf, because SDL gives an 8 bit int buffer, and ffmpeg gives us
data in a 16 bit int buffer. You should also notice the difference between
len1 and data_size. len1 is how much of the packet we've used, and data_size
is the amount of raw data returned. 

When we've got some data, we immediately return to see if we still need to get
more data from the queue, or if we are done. If we still had more of the
packet to process, we save it for later. If we finish up a packet, we finally
get to free that packet. 

So that's it! We've got audio being carried from the main read loop to the
queue, which is then read by the audio_callback function, which hands that
data to SDL, which SDL beams to your sound card. Go ahead and compile: 
    
    
    gcc -o tutorial03 tutorial03.c -lavutil -lavformat -lavcodec -lz -lm \
    `sdl-config --cflags --libs`
    

Hooray! The video is still going as fast as possible, but the audio is playing
in time. Why is this? That's because the audio information has a sample rate
-- we're pumping out audio information as fast as we can, but the audio simply
plays from that stream at its leisure according to the sample rate. 

We're almost ready to start syncing video and audio ourselves, but first we
need to do a little program reorganization. The method of queueing up audio
and playing it using a separate thread worked very well: it made the code more
managable and more modular. Before we start syncing the video to the audio, we
need to make our code easier to deal with. Next time: Spawning Threads! 

_**>>** Spawning Threads_

* * *

Function Reference  
Data Reference

email:
dranger at gmail dot com
Back to dranger.com

This work is licensed under the Creative Commons Attribution-Share Alike 2.5
License. To view a copy of this license, visit
http://creativecommons.org/licenses/by-sa/2.5/ or send a letter to Creative
Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA. 
 
  
Code examples are based off of FFplay, Copyright (c) 2003 Fabrice Bellard, and
a tutorial by Martin Bohme.