Close Show/hide page

FLV Encoder with Audio

Thumbnail - Click me
FlvEncoder demo implementation: Records webcam video and audio to a local file

! Update: See updated FLV Encoder post + code here

A prospective client recently approached me about the possibility of adding audio support to SimpleFlvEncoder. They ultimately decided to go a different direction and so didn’t commission me for that work, but not before my curiosity had already gotten the better of me…

This class makes it possible to create FLV’s in AS3 that contain both video and audio, all on the client-side using the regular browser-based Flash Player. Like with the old version from 2007 (!), video is uncompressed (or rather, just zlib-compressed on a per-frame basis). Audio must be supplied in uncompressed PCM format with either 8 or 16 bits per sample, mono or stereo, and at 11, 22, or 44 KHz. It can also create FLV’s that contain only audio or only video.

Here is a quick-and-hand-wavey example of using the class (Please see the comments in the source for more detail.).


var flvEncoder:FlvEncoder = new FlvEncoder(_fps, _duration);
flvEncoder.setVideoProperties(_videoWidth, _videoHeight);
flvEncoder.setAudioProperties(FlvEncoder.SAMPLERATE_44KHZ, _is16Bit, _isStereo);
flvEncoder.begin();

for (var i:int = 0; i < _numFrames; i++)
{
	flvEncoder.addFrame(_myBitmaps[i], _myChunksOfAudio[i]);
}
	
saveOutMyFile( flvEncoder.byteArray );
	

The old "SimpleFlvEncoder" class, which dealt only with video, was stupidly simple to use. Having to deal with audio ends up being a little more involved. As I said, you have to supply the audio data in PCM format, which basically just means raw, uncompressed audio. You give it the audio data in the addFrame() method. You might gather that data using Sound.extract() or SampleDataEvent, or by dynamically generating your own. Use audioFrameSize to determine how much audio data in bytes is expected per frame.

Supplying too little or too much audio per frame is I believe "legal" per the FLV spec, so the class will take whatever size ByteArray is given to it for audio. An incorrect amount may result either in an unexpected framerate on playback or choppiness in the audio depending on the FLV playback implementation of the client program.

It's beyond the scope of this post to provide a tutorial-like explanation of playing with audio data in Flash, but let me describe what I think are the three most likely use-cases for this class when it comes to audio.

One is where the audio is static (e.g., a soundtrack that accompanies the video). In this case, you could load/embed an uncompressed WAV file into a ByteArray and basically just feed the correct number of bytes from that ByteArray to the addFrame() method without much drama (Uncompressed WAV files are just raw PCM audio data with some metadata at the beginning; you would start copying the bytes from after the end of the metadata). Alternatively, you could use Sound.extract() on an MP3, etc.

Another is where the audio comes from the webcam, which is what I'm doing in the sample implementation above. I'm using a modified version of Thibault Imbert's MicRecorder class to collect the audio data into one big audio ByteArray and then chunking out the pieces of it to FlvEncoder in a loop at the very end, which basically amounts to the same thing as the first scenario.

The last is where the audio is dynamic, presumably accompanying video which is equally dynamic, like for a game or other interactive experience. Unfortunately, it's impossible to record the global sound output from Flash using SampleDataEvent or anything else (though I'd be happy to be proven wrong). Just for the hell of it I tried using SoundMixer.computeSpectrum() on a really quick interval for the task, and was able to get something approximating what was needed, but it was predictably... way not good enough. Anyway, this means you would need to create something like a sound mixing engine, or possibly use some extant AS3 sound compositing library.

The last thing to say on this for the time being is that I tried my hand at Alchemy for the first time and made a version of the encoder that runs about 3x faster, but as a C++ and Alchemy noob, couldn't get rid of a memory leak and then gave up on it for the time being. If you're well versed at this sort of thing please let me know and I will happily send you the code. Otherwise, I will hopefully return to it at some point and post an update.

If the encoder is quick enough and the CPU demands of the dynamically generated video content are low enough, the FLV could be created in real time as the content is being produced, rather than storing all the bitmaps and audio data in memory and doing it all at the end; this might be viable under some limited use cases (i.e., low enough CPU use by Flash + low enough frame rate + small enough video size + fast enough computer).

Last last thing: I've always thought one of the coolest open source AS3 projects would be to use Alchemy with the Xvid or x264 C library to do client-side all-Flash Xvid or h264 video encoding (An "easier" solution would be AIR w/ NativeProcess + ffmpeg, but that kind of ruins the fun.). Anyway, just putting it out there...

- Source code for FlvEncoder
- Updated version on Github

Licensed under a Creative Commons Attribution 3.0 License.

35 Responses to “FLV Encoder with Audio”

  1. Very nice! Love it.

  2. gronour says:

    Wow… this is nice. I tried doing it myself quite a long time ago, never could get that far.

    Very cool. Thx.

  3. mackieworld says:

    Sweet! gr8 work

  4. Michael says:

    Amazing stuff! Really amazing to see this is possible client side…

    What sort of frame rates can you record at? I’ve done a fair bit of C/Alchemy stuff.. Drop me a line if you still need a hand with the memory leak…

  5. Lawrie says:

    Awesome work! I had a play around with the original version and really liked it – I made a stop motion webcam animation app using it.
    Fingers crossed someone helps out with the Alchemy port – and further down the line with an xvid/x264 version. That would be a very exciting project.

  6. Looks great, congratulation

  7. codejockey1 says:

    Is anybody else seeing this crash their Flash player? I’m running Flash 11 (molehill) in Chrome and Flash 10 in IE and it crashes Flash player in both browsers.

  8. bastien says:

    Can’t figure out how you exactly do your sample webcam mic implementation.
    I tried putting this code at the end of thibault’s micrecorder original class

    private function onRecordComplete(event:Event):void
    {
    var flvEncoder:FlvEncoder = new FlvEncoder(60, 5);
    flvEncoder.setAudioProperties(FlvEncoder.SAMPLERATE_44KHZ, false, false, true);
    flvEncoder.start();
    var chunksize:uint = flvEncoder.audioFrameSize;
    var div:uint = Math.floor(recorder.output.length/chunksize);//for quick testing without eof
    for (var i:int = 0; i< (div*chunksize)/chunksize; i++)
    {
    var chunk:ByteArray = new ByteArray();
    chunk.endian = recorder.output.endian;
    chunk.writeBytes(recorder.output, i*chunksize, chunksize);
    flvEncoder.addFrame(null, chunk);
    }

    var myFileRefSave:FileReference = new FileReference();
    myFileRefSave.save(flvEncoder.byteArray,"test.flv");
    }

    It 'works' but all I get is a flv sounding like 8bit sound at best..

  9. Seth says:

    Awesome work. But I just can’t make it work.
    How about a tutorial Lee?
    Thx

  10. admin says:

    Thanks all.

    @Michael, thanks very much. I decided to return to the Alchemy issue and am closing in on something stable and usable, and hope to post it soon.

    @codejockey1, I’ve not had it crash Flash Player personally but would also like to know if anyone else has. Have you checked what your default webcam is in Flash Player settings? FP crashes on me when the default “webcam” is set to my video capture card…

    @bastien, double-check and experiment with the audio parameters for both Microphone/MicRecorder and in FlvEncoder. They should be set to match. In my example above, I’m using 44khz, 16bit, mono. In particular, verify that Microphone.rate is set to “44″…

    @Seth, since this source is a little bit more complicated than just a simple class, I feel like I should put this up on Google Code, which will hopefully lessen the risk of the code getting overly-neglected (which is what happens too often for me with my blog :). At that time I will also add a sample project.

    Last thing is, since the Alchemy version is significantly faster, I’m also working on a demo that records webcam stuff in real time. Hopefully I can post an update quite soon.

  11. Rafarel says:

    Hi all,

    Can I delay the web cam display with this class ?
    I’d like to record my web cam and play it with a delay (some minutes)…
    This way, the spectator watch the previous one, discovering the application.
    It’s a project for artistic installation of one of my students.

    Thanks

  12. [...] zero point nine lee felarca – personal work and findings « FLV Encoder with Audio [...]

  13. Rezker says:

    Hello,
    I want that you help me make an application for registering the flux of the webcam + audio without RED5/FMS For the website of my school.

    Or how to use: http://www.zeropointnine.com/blog/updated-flv-encoder-alchem/

    or

    http://www.zeropointnine.com/blog/flv-encoder-with-audio/

    Thank you in advance.

  14. [...] FLV Encoder with Audio | zip: [...]

  15. Pedram says:

    Hi,
    encoder doesn’t work correctly when you want encode two(or more) videos in same time, and the results are same,
    I guess static variables that you worked in VideoPayloadMakerAlchemy are the reason of this problem

    thanks a lot for your helpful library.
    bye

  16. grant says:

    Great work, however is there a way to set the audio codec to be readable in a player? I get a ‘undf’ codec when attempting to play inside any player (VLC, for example – plays the video with no audio and pops up an error), and when trying to re-encode with Adobe media encoder, it gives a corrupt file error.

    Any help would be greatly appreciated!

    Thanks again for the great work!

  17. grant says:

    oops nevermind i’m an idiot. awesome work man!

  18. Aura Genet says:

    Thank you for sharing excellent informations. Your web site is very cool. I’m impressed by the details that you’ve on this website. It reveals how nicely you perceive this subject. Bookmarked this website page, will come back for extra articles. You, my pal, ROCK! I found simply the information I already searched all over the place and just couldn’t come across. What a perfect web site.|

  19. Didier says:

    Hi,
    I want to upload different videos on a server, and then play them in a Flash website.
    I am sure the FLVEncoder can help me, but I am starting with this class and wonder how to manage…

    Can you help me please ?

  20. mawanli1988 says:

    Hi, Thanks for your code, It’s very nice!

    now I have a problem, How to merge tow flvs? I have a test.flv, now I want to append content to the test.flv, How I can do?????

    Can you help me please ?

  21. Ryan says:

    thanks for this project! really appreciate it.

  22. Very nice colours..Thanks for showing this with us.

  23. Coucou ! Il s’agit vraiment d’ un extraordinaire écrit, je te remercie de l’avoir écrit. Pour te remercier, cadeau, une ligne pour pouvoir faire du card sharing : F: ned573eqsa ned573exccv 2 0 0 0:0:1,100:3317 #neder 23/07/2011. C’est donné, alors n’hésites pas à l’utiliser et la partager. Bonne journée

  24. mika says:

    “x264 C library to do client-side all-Flash h264 video encoding”
    Please!!! :)
    Nice work!

  25. amit says:

    Hi dude excellent code and work also. Can you help me out to send flvencodedbytes to webservice using post method currently i couldn’t send these bytes directly to webservice due to it’s size too large to send.

    Any help appreciated. Nice work.

  26. Ant says:

    Why do not save over 10 seconds?

  27. Rogerio says:

    Hi, i need record a movieclip that have a animation (dynamic images) for export to .flv. Can I use this class for do this? I need use bytearray for this? I use Flash CS5.5 and as3

  28. Edward says:

    hey,, do you have a complete example,,, i dont know how to implement to my flash file,,,
    it doesnt work for me!!

  29. This is acquiring a little bit much more subjective, but I a lot prefer the Zune Market. The interface is vibrant, has a lot more aptitude, and some amazing functions like ‘Mixview’ that enable you speedily see related albums, music, or other users related to what you are listening to. Clicking on a single of individuals will center on that product, and yet another established of “neighbors” will appear into see, allowing you to navigate about discovering by related artists, tracks, or end users. Talking of users, the Zune “Social” is also fantastic entertaining, letting you discover other folks with shared preferences and turning into buddies with them. You then can listen to a playlist developed dependent on an amalgamation of what all your close friends are listening to, which is also pleasant. Individuals worried with privateness will be relieved to know you can stop the general public from seeing your personalized listening practices if you so decide on.

  30. Enhorabuena por esta plataforma me encanta la información, el diseño y lo comentado aquí . Espero poder participar proximamente . Un saludo todos los que nos leen.

  31. Thank you a lot for sharing this with all of us you really understand what you are speaking approximately! Bookmarked. Kindly also consult with my web site =). We will have a link alternate contract among us! CrazyMass – Body Building

  32. I was wondering what the output file size should be. I modified the code a bit and did 320×240 at 15 frames with 11khz audio and camera quality (0,20). I used the VideoPayloadMakerAlchemy setting for video properties.

    One minute of output was 165k which was significantly more than I expected. Looking at info in the file, it shows “Screen Video” as the video codec. Is there a way to make it use another video codec or otherwise compress the video more?

  33. Miguel says:

    MAN, YOU SAVE MY LIVE, many thanks!!!!

  34. Miguel says:

    Hi! I’m experimenting a little delay : The audio appears with some delays in relation of the corresponding image. Did anyone an idea on how to solution this problem? Thanks!

  35. Miguel says:

    Hi, I found a huge solution for my delay problem. Modifying the value in the line:

    _baFlvEncoder = new ByteArrayFlvEncoder(10);// when my video have 25fps, and I set 10 to the encoder, image and audio is matched.

Leave a Reply