Close Show/hide page

FLV Encoder with Audio

Thumbnail - Click me
FlvEncoder demo implementation: Records webcam video and audio to a local file

! Update: See updated FLV Encoder post + code here

A prospective client recently approached me about the possibility of adding audio support to SimpleFlvEncoder. They ultimately decided to go a different direction and so didn’t commission me for that work, but not before my curiosity had already gotten the better of me…

This class makes it possible to create FLV’s in AS3 that contain both video and audio, all on the client-side using the regular browser-based Flash Player. Like with the old version from 2007 (!), video is uncompressed (or rather, just zlib-compressed on a per-frame basis). Audio must be supplied in uncompressed PCM format with either 8 or 16 bits per sample, mono or stereo, and at 11, 22, or 44 KHz. It can also create FLV’s that contain only audio or only video.

Here is a quick-and-hand-wavey example of using the class (Please see the comments in the source for more detail.).

var flvEncoder:FlvEncoder = new FlvEncoder(_fps, _duration);
flvEncoder.setVideoProperties(_videoWidth, _videoHeight);
flvEncoder.setAudioProperties(FlvEncoder.SAMPLERATE_44KHZ, _is16Bit, _isStereo);

for (var i:int = 0; i < _numFrames; i++)
	flvEncoder.addFrame(_myBitmaps[i], _myChunksOfAudio[i]);
saveOutMyFile( flvEncoder.byteArray );

The old "SimpleFlvEncoder" class, which dealt only with video, was stupidly simple to use. Having to deal with audio ends up being a little more involved. As I said, you have to supply the audio data in PCM format, which basically just means raw, uncompressed audio. You give it the audio data in the addFrame() method. You might gather that data using Sound.extract() or SampleDataEvent, or by dynamically generating your own. Use audioFrameSize to determine how much audio data in bytes is expected per frame.

Supplying too little or too much audio per frame is I believe "legal" per the FLV spec, so the class will take whatever size ByteArray is given to it for audio. An incorrect amount may result either in an unexpected framerate on playback or choppiness in the audio depending on the FLV playback implementation of the client program.

It's beyond the scope of this post to provide a tutorial-like explanation of playing with audio data in Flash, but let me describe what I think are the three most likely use-cases for this class when it comes to audio.

One is where the audio is static (e.g., a soundtrack that accompanies the video). In this case, you could load/embed an uncompressed WAV file into a ByteArray and basically just feed the correct number of bytes from that ByteArray to the addFrame() method without much drama (Uncompressed WAV files are just raw PCM audio data with some metadata at the beginning; you would start copying the bytes from after the end of the metadata). Alternatively, you could use Sound.extract() on an MP3, etc.

Another is where the audio comes from the webcam, which is what I'm doing in the sample implementation above. I'm using a modified version of Thibault Imbert's MicRecorder class to collect the audio data into one big audio ByteArray and then chunking out the pieces of it to FlvEncoder in a loop at the very end, which basically amounts to the same thing as the first scenario.

The last is where the audio is dynamic, presumably accompanying video which is equally dynamic, like for a game or other interactive experience. Unfortunately, it's impossible to record the global sound output from Flash using SampleDataEvent or anything else (though I'd be happy to be proven wrong). Just for the hell of it I tried using SoundMixer.computeSpectrum() on a really quick interval for the task, and was able to get something approximating what was needed, but it was predictably... way not good enough. Anyway, this means you would need to create something like a sound mixing engine, or possibly use some extant AS3 sound compositing library.

The last thing to say on this for the time being is that I tried my hand at Alchemy for the first time and made a version of the encoder that runs about 3x faster, but as a C++ and Alchemy noob, couldn't get rid of a memory leak and then gave up on it for the time being. If you're well versed at this sort of thing please let me know and I will happily send you the code. Otherwise, I will hopefully return to it at some point and post an update.

If the encoder is quick enough and the CPU demands of the dynamically generated video content are low enough, the FLV could be created in real time as the content is being produced, rather than storing all the bitmaps and audio data in memory and doing it all at the end; this might be viable under some limited use cases (i.e., low enough CPU use by Flash + low enough frame rate + small enough video size + fast enough computer).

Last last thing: I've always thought one of the coolest open source AS3 projects would be to use Alchemy with the Xvid or x264 C library to do client-side all-Flash Xvid or h264 video encoding (An "easier" solution would be AIR w/ NativeProcess + ffmpeg, but that kind of ruins the fun.). Anyway, just putting it out there...

- Source code for FlvEncoder
- Updated version on Github

Licensed under a Creative Commons Attribution 3.0 License.

81 Responses to “FLV Encoder with Audio”

  1. Good info. Lucky me I came across your website by accident (stumbleupon). I’ve book marked it for later!

  2. I couldn’t resist commenting. Very well written!

  3. Can I simply just say what a relief to discover an individual who actually knows what they are discussing on the net. You certainly realize how to bring a problem to light and make it important. A lot more people really need to read this and understand this side of the story. I was surprised that you are not more popular since you certainly possess the gift.

  4. Highly energetic article, I loved that bit. Will there be a part 2?|

  5. This web site certainly has all the information and facts I needed about this subject and didn’t know who to ask. |

  6. Hello There. I found your weblog the usage of msn. This is an extremely smartly written article. I’ll make sure to bookmark it and return to read extra of your helpful info. Thank you for the post. I’ll definitely comeback.|

  7. Transmitir cultura es uno de las apuestas que Signo Editores se puso como meta cuando se fundó la empresa. A lo largo de los años, y con varias décadas vendiendo libros de máxima calidad, la compañía se ha convertido en un referente nacional, con delegaciones repartidas por toda España

  8. divani says:

    But a smiling visitor here to share the love (:, btw great design .

  9. Excellent, what a blog it is! This webpage provides useful data to us, keep it up.|

  10. Pretty nice post. I just stumbled upon your blog and wanted to say that I’ve really enjoyed browsing your blog posts. In any case I will be subscribing to your rss feed and I hope you write again soon!|

  11. Good day! Do you use Twitter? I’d like to follow you if that would be okay. I’m definitely enjoying your blog and look forward to new posts.|

  12. Thanks for the marvelous posting! I quite enjoyed reading it, you are a great author.I will make certain to bookmark your blog and will often come back later on. I want to encourage one to continue your great writing, have a nice afternoon!|

  13. You’re so awesome! I don’t think I’ve truly read through anything like that before. So nice to find another person with a few unique thoughts on this issue. Seriously.. thanks for starting this up. This web site is one thing that’s needed on the web, someone with a bit of originality!|

  14. dumpsters says:

    Thankfulness to my father who told me on the topic of this weblog, this webpage is really awesome.|

  15. This blog was… how do you say it? Relevant!! Finally I have found something that helped me. Thanks!|

  16. I really like what you guys are usually up too. This kind of clever work and coverage! Keep up the fantastic works guys I’ve incorporated you guys to our blogroll.|

  17. exede says:

    I’m not sure why but this blog is loading extremely slow for me. Is anyone else having this issue or is it a problem on my end? I’ll check back later and see if the problem still exists.|

  18. mac lips says:

    When I originally commented I appear to have clicked on the -Notify me when new comments are added- checkbox and from now on every time a comment is added I get four emails with the same comment. Is there a means you are able to remove me from that service? Cheers!

  19. Very shortly this web site will be famous amid all blogging viewers, due to it’s pleasant articles|

  20. I just like the helpful information you provide for your articles. I will bookmark your blog and take a look at once more right here regularly. I’m rather sure I will learn a lot of new stuff proper right here! Good luck for the next!|

  21. Excellent items from you, man. I have remember your stuff prior to and you are simply extremely great. I actually like what you’ve obtained right here, certainly like what you are stating and the way during which you assert it. You make it enjoyable and you continue to take care of to stay it wise. I can’t wait to learn much more from you. That is actually a wonderful website.|

  22. You have made some good points there. I looked on the internet to find out more about the issue and found most individuals will go along with your views on this website.|

  23. I every time used to read article in news papers but now as I am a user of internet therefore from now I am using net for content, thanks to web.|

  24. Good day I am so happy I found your weblog, I really found you by mistake, while I was looking on Askjeeve for something else, Regardless I am here now and would just like to say thanks a lot for a tremendous post and a all round enjoyable blog (I also love the theme/design), I don’t have time to look over it all at the moment but I have book-marked it and also included your RSS feeds, so when I have time I will be back to read a great deal more, Please do keep up the awesome job.|

  25. Ridiculous quest there. What happened after? Good luck!|

  26. Have you ever thought about publishing an e-book or guest authoring on other blogs? I have a blog based upon on the same information you discuss and would really like to have you share some stories/information. I know my readers would value your work. If you are even remotely interested, feel free to send me an e mail.|

  27. I read this post fully about the difference of most recent and preceding technologies, it’s amazing article.|

  28. locksmith says:

    Its like you read my thoughts! You appear to grasp a lot approximately this, such as you wrote the guide in it or something. I feel that you can do with some percent to pressure the message house a little bit, however instead of that, this is great blog. A great read. I will definitely be back.|

  29. What’s up to every one, it’s really a nice for me to pay a visit this web site, it contains priceless Information.|

  30. dishnet says:

    I always used to study piece of writing in news papers but now as I am a user of internet thus from now I am using net for content, thanks to web.|

Leave a Reply