Is it not the norm for the agency developer of whatever flavor to be asked make more progress than what’s reasonable based on the current status of a given project at any given point in time?

Thus, we continuously look for ways to make more progress with less information, even while increasing the risk of wasted work and unseemly displays of developer angst. At the very least, such experiences make a person receptive to finding a few tricks that might shave off a few extra minutes in one’s pursuit to meet the latest unreasonable deadline.

That having been said, the potential utility of this little tool should require no further explanation.

It has its origins in this capricious tweet from some time last winter.

- Download (uses Adobe AIR)

- Source (Flex project) ]]> http://www.zeropointnine.com/blog/fpo-image-generator/feed/ 4 http://www.zeropointnine.com/blog/updated-flv-encoder-alchem/ http://www.zeropointnine.com/blog/updated-flv-encoder-alchem/#comments Wed, 30 Mar 2011 14:32:45 +0000 admin http://www.zeropointnine.com/blog/?p=898

This is an update to FLV Encoder which adds an optional Alchemy routine that’s about 3.5x faster, as well as FileStream support for writing directly to a local file using AIR. The library has been architected in such a way that you can use the package while targeting either the browser (no FileStream support) or AIR, and either Flash 9 (no Alchemy support) or Flash 10 – without getting class dependency compiler errors.

I’ve put the package on Github, along with a couple examples (including the one from the last post). The API has changed a little so be sure to also see the example code below. A method updateDurationMetadata() has been added so the video duration does not have to be declared before the video has been created. Also, a bug where the top-most line of pixels was not being written has been fixed.

Realtime encoding demo:

Because of the increased speed of the Alchemy version, it is now viable to encode FLV’s in realtime as the audio and video is being captured, at least within certain limits. Click on the thumbnail above for an online demo that encodes webcam video and audio to a file at 320×240 in real time. If your system is fast enough, you can keep the framerate set to 15FPS with minimal hiccups. The browser-based version must store the entire FLV in memory before saving to disk, but the equivalent AIR version can save its contents directly to a file so that the only limiting factor is disk space. I’m using a dynamic timing and queuing system to keep video and audio in sync which could be the topic of another post.

Updated usage examples:

[A] To encode an FLV to a ByteArray, start by instantiating ByteArrayFlvEncoder. The rest is similar to the last version, but you can now call updateDurationMetadata() when you’re done to update the duration property in the metadata. Finally, call kill() to prepare the object for garbage collection:

[as3]
var baFlvEncoder:ByteArrayFlvEncoder = new ByteArrayFlvEncoder(myFrameRate);

baFlvEncoder.setVideoProperties(myWidth, myHeight, VideoPayloadMakerAlchemy);
// (Omit the 3rd argument to NOT use Alchemy if you’re targeting Flash 9)
baFlvEncoder.setAudioProperties(BaseFlvEncoder.SAMPLERATE_44KHZ, true, false, true);

baFlvEncoder.start();

baFlvEncoder.addFrame(myBitmapData, myAudioByteArray);
baFlvEncoder.addFrame(myBitmapData, myAudioByteArray); // etc.

baFlvEncoder.updateDurationMetadata();

saveOutMyFileUsingFileReference( baFlvEncoder.byteArray );

baFlvEncoder.kill(); // for garbage collection
[/as3]

[B] To encode an FLV directly to a local file (in AIR), instantiate FileStreamFlvEncoder with a File reference, and open up the exposed FileStream, and then close it when you’re all done:

[as3]
var myFile:File = File.documentsDirectory.resolvePath(“video.flv”);
var fsFlvEncoder:FileStreamFlvEncoder = new FileStreamFlvEncoder(myFile, myFrameRate);
fsFlvEncoder.fileStream.openAsync(myFile, FileMode.UPDATE);

fsFlvEncoder.setVideoProperties(myWidth, myHeight, VideoPayloadMakerAlchemy);
fsFlvEncoder.setAudioProperties(BaseFlvEncoder.SAMPLERATE_44KHZ, true, false, true);

fsFlvEncoder.start();

fsFlvEncoder.addFrame(myBitmapData, myAudioByteArray);
fsFlvEncoder.addFrame(myBitmapData, myAudioByteArray); // etc.

fsFlvEncoder.updateDurationMetadata();

fsFlvEncoder.fileStream.close();

fsFlvEncoder.kill();
[/as3]

A new instance of FlvEncoder should be created for every new FLV to be generated.

If you’re using the Alchemy version, the SWC lives in leelib/util/flvEncoder/alchemy, so be sure add that folder to your project library path. The C++ code is there as well.

Note that the current Alchemy video encoding routine is being done synchronously, which means that although it’s much faster than the regular AS3 version, the main Flash thread must still wait for the Alchemy encoding to be done before continuing. An asynchronous version would allow for a real-time encoding scenario that is more flexible and probably with less ‘dropouts’, and would probably prompt the need an internal queuing system. Anyway, this warrants further development.

- Source code (Github)

Licensed under a Creative Commons Attribution 3.0 License.

I started with a description of the algorithm found on Wikipedia, worked it out in Processing (with 2D, 3D, and isometric views added as a bonus), and then ported it to C++ using openFrameworks for the basic display action.

Presumably it could be used as the start of a game-like concept, if so inclined.

- Processing (source + Win/Mac executables)
- C++/oF (source + Windows executable)

This post announces a 3D framework/library-in-progress I’ve been writing for Android, written in Java with OpenGL ES. Between the names min3d, modest3d, and llama3d, I’ve gone with “min3d” for being the least self-deprecating while still conveying an appropriate level of expectation that I’m comfortable with. :)

I’ve gone legit by putting it up on Google Code (first time for everything, etc.). While just an early build, it should at the very least be useful for educational purposes if like me you’re just starting down the Android path.

My aim while building this was to make a library that required no extra massaging of model data on its way to being used by OpenGL. To that end, data can be created and manipulated directly on the ByteBuffers that are used by OpenGL, without the need for any intermediating (ie, redundant) data structures. And hopefully wrapped in such a way that’s still relatively easy to use.

But this approach also imposes some important restrictions. The maximum number of vertices and faces of an Object3d is fixed after instantiation, and the dynamic addition/removal of vertex or face elements is currently not supported. If/when implemented, these operations will have to be much more costly than, say, with a linked list. But unavoidable, as far as I can tell.

Here’s what’s currently implemented through the API (all conventional stuff…)

• Vertex index lists
• Per-vertex colors
• Vertex normals
• Texture mapping
• Light source (just the one, for now)
• Camera class, with view frustrum settings
• Object scale, euler rotation, and translation properties as expected
• Object children that inherit parent’s transform properties
• Ability to render a subset of an object’s list of faces
• Triangle or point rendermodes (just the two, for now)
• A few canned object primitives

Code examples showing how to actually use “min3d” in its current form can be found through the project home page on Google Code by checking out the source with SVN, and going through the sampleProjects/ directory.

In the immediate future, I’d like to be able to at least add a light manager and a 3d-file format importer, and definitely test and debug more thoroughly. If anyone is interested in collaborating on or even picking up this project, please let me know.

Download installable AIR file
Requires AIR 2 Beta 2 Runtime +
Windows .NET v3.5

My aim here was to get familiar with AIR 2′s new capability of interacting with external processes’ standard streams.

One useful way of learning how to use a new feature is to proceed directly to trying to abuse it.

To that end, this AIR application takes in a fairly continuous stream of uncompressed binary image data from a native process’ standard output. My hope was to get real-time screen capture updates into AIR at a decent frame rate.

Details and source code after the break–

The native process

For my “native process,” I created a Windows .NET application written in C#. When asked, it takes a screen capture of the desktop and sends the image data through the process’ standard output. For various reasons, I chose to set it up as a Windows form rather than as a plain vanilla console or as a real system service. The relevant C# code is only a few dozen lines long. One interesting function there is bitmapToBytes(), which converts a bitmap into a byte array in the correct format for Flash to be able to apply it directly to a BitmapData without any further (costly) conversion.

The AIR application

On the “client-side,” the Flash AIR application has little to do. It simply sends a command to the native process, telling it to take a screen grab, along with image dimension and offset parameters. Because the client takes in the resulting stream of data in packets whose length should not be assumed to be of any given size, it needs to know where in the data a new image starts and where it ends. So it checks the stream for an arbitrary 4-byte start code as well as a termination code. When the transfer is complete, it just applies the resulting data, collected over time into a ByteArray, using BitmapData.setPixels(). It then requests a new image, and so on.

Performance considerations

On a mid-range quad-core PC using 1280×1024 images (5MB per uncompressed frame), I’m getting about 8 frames per second, or 125ms per frame. The time it takes for the 5MB image to transfer through the standard stream is about 65ms, or 50% of the total time for one iteration; without setting up an honest benchmark test, it’d be unfair and/or impolitic to declare that AIR represents a bottleneck here, so … yeah.

The AIR app eats up about 2/3 of one CPU core using the runtime’s main application thread. The remaining 1/3 represents the amount of time the AIR app has to wait while the native process constructs the screengrab. If the app were re-architected in such a way that the native process always has a new screencap queued up and ready to send – leading to little or no idle time between request and response – the AIR app would easily soak a full CPU core. Since the AIR app is doing little of significance besides reading the process’s standard stream and collecting that info into a byte array, it appears that merely handling large amounts of stdout data is quite costly.

Of course, AIR 2 isn’t even final yet. Nevertheless, I thought these preliminary findings were interesting. (This build targets Beta 2, which is the most recent public beta of AIR 2; compiling and running it against the most recent non-public beta of the SDK and runtime dated April 9, 2010 gave the same results.)

Source files

Setting up a native process for use with AIR 2 is well documented so I didn’t bother commenting on that here. However, the included source is I believe a good starting point for adapting to whatever native process craziness you may be cooking up…

For another implementation of AIR 2 screen capture, see this post by Mihai Corlan, also with source, using Java. There are also a few other related examples out there as well.

Licensed under a Creative Commons Attribution 3.0 License.

This one’s pretty niche: A small Windows utility that emulates the left and right mousebuttons using the keyboard. Specifically, using the right Windows key and the right Alt key.

Why? Because I use a MacBook Pro running exclusively in PC-mode, and I can’t stand how the mouseclick is implemented with its button-less trackpad. So yeah, pretty niche…

Sorry, no UI to change the hotkey assignments. If you’re comfortable with C#.NET, the source code is included in the zip file. It may be useful to examine if you’re interested in learning (1) how to capture keystrokes globally in Windows or (2) how to generate mouse operations programmatically, using some dreadful-looking Windows API interop code that I tried to learn as little as possible about in order to get to work.

Download (source included)

Simulates the projection of shadows coming from a spotlight light source onto irregular surfaces. In the supplied example, an animated MD2 model (from Quake 2) is casting a shadow onto the inside of a sphere. Supported parameters include shadow color, alpha, blur, falloff and texture map size, as well as position, rotation and field of view. This evolves out of the projection work posted previously.

Source:

SpotlightShadowCasterManager.as

Usage:

After instantiating the manager –

	_shadowCaster = new SpotlightShadowCasterManager();

– assign one or more DisplayObject3D’s that will block the light from the spotlight:

	_shadowCaster.registerShadowCasterObject(myMesh);

Then register one or more meshes that can have shadows cast upon it with registerShadowCasterObject. As the second argument, you must supply a BitmapMaterial utilized by the mesh on which the manager will draw the shadows. Usually, you’ll want to create a CompositeMaterial, with the BitmapMaterial most likely at the top of the stack. It might look something like this:

	var compositeMaterial = new CompositeMaterial();
	compositeMaterial.addMaterial( new ColorMaterial(0x888888) );
	var mat:BitmapMaterial = new BitmapMaterial( new BitmapData(1,1) );
	// .. the bitmapdata will get overwritten,
	//    but must contain _something_
	var sphere:TriangleMesh3D = new Sphere(compositeMaterial, 1000);

	_shadowCaster.registerShadowReceiverObject(sphere, mat);

To update the shadow textures, call

	_shadowCaster.update();

– probably in your onEnterFrame or onRenderTick function.

That’s enough to get going. From there, it’s just a matter of adjusting the projector’s position and orientation in relation to the objects in your scene with the properties projectorPosition, projectorPitch, and projectorRoll. Check the source for the various configurable properties like blur, alpha, color, etc.

Limitations, bugs:

- As described above, objects must be manually assigned to be either a ‘caster’ or a ‘receiver’. These roles aren’t determined dynamically by the class itself.

- Only one level of shadows is supported. Adding an arbitrary number wouldn’t be difficult, just cumbersome to manage under the current system, and increasingly expensive.

- Unfortunately, a ‘shadow receiver’ object can’t have its own texture in addition to the one used to cast shadows onto, at least not in a way that looks decent. The reason is that a 3d object in Papervision3D has only one set of UV texture data and the shadow texture overwrites this with every update.

- There’s undoubtedly a less expensive way to do the raycasting/projection in the main math routine in the private function updateMesh. If you’re adept at this kind of thing, please send your improvements!

- Ugly and obvious artifacts show up on the ‘projectable’ textures when the spotlight is rotated 90 degrees away them. (You may have already noticed this in the previous two projection experiment posts). Again, I’d love any solutions anyone has for this.

Licensed under a Creative Commons Attribution 3.0 License.

I. Client Side

On the Flash end, two requirements:

Original page view, in Flash

SEO-friendly HTML page, drawing on the same data

(A) Your site must implement deeplinking, presumably using SWFAddress, such that every page view has its own URL path. And your shell class must be intelligent enough to go directly to any given deeplinkable page view on startup via the URL deeplink.

(Sidenote: While not strictly necessary from an SEO standpoint, having gone that far, you’ll probably want all changes to the page view to be driven by the SWFAddress URL-change event, rather than directly by user events like mouse clicks, etc. Which will force you to throw your familiar, tried-and-true coding patterns out the window. And potentially, your laptop thereafter. But is beyond the scope of this entry. ;)

(B) Your site should use an external data source that describes the site structure and contains the site’s text content, as well as, typically, image and/or video URL’s and the like. Usually, this would be a well structured, static XML file whose node structure corresponds to the site’s actual page structure. The deeplink ‘crumbs’ should be included therein as well – as an attribute of each node, for example. (Extra points for using a database or a CMS instead). And it goes without saying that your Flash should be building its site structure, nav, and page content dynamically using all of the above.

II. Server Side

On the server side, nothing too fancy. You want to create a script that parses the XML tree, constructs and writes out an HTML file for every deeplinked page in your site. You’ll run the script every time you make changes to the data, or create a chron job. So in the end, there’s nothing very ‘dynamic’ happening on the back end, just using a script to create static HTML files from a static XML file. (* Extra points for using XSL to magically transform your XML into ready-to-use HTML pages, but since XSL makes my head hurt, I just procedurally concatenated lots and lots of HTML tags together to form one big string…)

The directory path for each generated “index.html” page should correspond to the deeplink used in your Flash (everything after the hash mark). So if you have a page whose URL is:

- your script should create a corresponding HTML page whose filename is

Each HTML page should include all the text content for a given page view, since SEO is our goal, after all. Hopefully, you have a page title attribute on your XML nodes that you can leverage as well. Maybe you want to add some keyword metadata in there, too.

The last thing you want to do is have each of your generated html pages auto-redirect to its Flash equivalent using Javascript, or at least display a prominent link for the user to do it for themselves (Ie, strip “index.html” off the end of the URL and insert “/#” after the domain name). Personally, I chose to do the latter, since I took the trouble to create HTML that could stand ‘on its own’, without the Flash, if necessary. Your pages should link to each other as well; otherwise, the web crawlers may not have a way to get to them all.

So, here’s the PHP script I wrote for my site which may help concretize the procedure described above. And the XML file for your reference, used by both the the Flash and the PHP based on their respective agendas. And for the hell of it, my magic keyword, inserted into the HTML to easily verify if/when they were becoming visible to Google. xD

Licensed under a Creative Commons Attribution 3.0 License.

Or at least, that’s the best easy analogy to describe it.

Gist: The color of each triangle in the mesh is obtained by averaging the pixel colors of the underlying image at the screen positions under each vertex as well as at the triangles’ center point. Some FlatShader-like lighting is used to add a little more three-dimensionality. Finally, a BlurFilter is applied to the source image to keep the transitions between colors from feeling too abrupt.

Source code: PvColorFacets.as

Licensed under a Creative Commons Attribution 3.0 License.

A green-screen/chroma key effect utilizing Pixel Bender. With this version, the live video is compared against a reference image rather than a static color. Additionally, the alpha of the output is graduated based on the color difference between the base image and the incoming video (rather than just being either ‘on’ or ‘off’). Both of these differences may or may not be good for practical use, but can create interesting imagery.

Use the sliders to adjust the cutoff/falloff curve for the alpha. Click “Reset base image” to set the base image with which to compare any further changes in the video against. You can also choose your own background image (click “Browse…” at the bottom). Works best against a plain background and presumably best of all with proper lighting against a green background…

When I wrote the image processing routine in pure Actionscript, it ran like a slideshow. The Pixel Bender version on a multi-core system runs about 30x faster.

Pixel Bender kernel: GreenScreenEffect.pbk

Licensed under a Creative Commons Attribution 3.0 License.