Close Show/hide page

Archive for the ‘flash’ Category

Using NativeProcess in AIR 2 for screencaps

Sunday, April 18th, 2010
Thumbnail

Download installable AIR file
Requires AIR 2 Beta 2 Runtime +
Windows .NET v3.5

My aim here was to get familiar with AIR 2’s new capability of interacting with external processes’ standard streams.

One useful way of learning how to use a new feature is to proceed directly to trying to abuse it.

To that end, this AIR application takes in a fairly continuous stream of uncompressed binary image data from a native process’ standard output. My hope was to get real-time screen capture updates into AIR at a decent frame rate.

Details and source code after the break–

(more…)

More Fun With Texture Projection

Monday, March 22nd, 2010
Thumbnail - Click me

Here I’m just expanding upon the ‘texture extraction’ idea posted before. This version maps video onto 3D objects in a much more direct manner by using ray-casting, like the examples in the last several posts.

Controls:

  • Mouse rotates object
  • Mouse drag moves object
  • [SPACE] remaps the texturing of the 3d object
  • [ENTER] cycles through different 3D primitives
  • [H] shows full list of kludgey hotkey controls

Update with video (8/25/2010):

Although this piece dispenses with the encumbrances of an AR marker or any application-like functionality, I think the method being used here has some potentially interesting uses for an augmented reality-based design application. (Or for more whimsical uses like “upload-your-face” done in 3D). I’ve disabled backface detection, so the imagery gets mapped on all sides of the object rather than just the camera-facing polys, which makes it more visually interesting. Live demo above requires webcam.

Augmented Reality Texture Extraction Experiment

Saturday, September 19th, 2009

This is an AR-based experiment that enables the user to lift textures from real-world objects in live video and apply them onto 3D objects that are overlayed on top of them. Only box primitives are supported here, but the general idea could be extended to other types of 3D primitives or potentially even more complex objects with some clever image compositing and UV mapping.

Thumbnail - Click me
Click for live demo

Click the image on the right to run a live version of the experiment, in case you have the patience to suffer thru the quick-and-dirty hotkey-based UI.

Print the AR marker (PDF | PNG) and point your webcam at it.

It’s running Saqoosha’s (feature-incomplete) Alchemy branch of the FLAR Toolkit, along with Papervision3D.

In the future, it would be nice to figure out is how to apply bilinear filtering to the ‘deperspectivized’ textures. I could also add a feature to export the textured 3d objects into a 3D file format (probably OBJ) if there’s any interest.

Conceptually, this piece builds on these two video projection tests, and is a modest implementation of one of the themes from this post of ideas for augmented reality.

Complex Spotlight Shadow Caster for Papervision3D

Monday, June 8th, 2009
Thumbnail - Click me

Simulates the projection of shadows coming from a spotlight light source onto irregular surfaces. In the supplied example, an animated MD2 model (from Quake 2) is casting a shadow onto the inside of a sphere. Supported parameters include shadow color, alpha, blur, falloff and texture map size, as well as position, rotation and field of view. This evolves out of the projection work posted previously.

Source:

SpotlightShadowCasterManager.as

Usage:

After instantiating the manager –

	_shadowCaster = new SpotlightShadowCasterManager();

– assign one or more DisplayObject3D’s that will block the light from the spotlight:

	_shadowCaster.registerShadowCasterObject(myMesh);

Then register one or more meshes that can have shadows cast upon it with registerShadowCasterObject. As the second argument, you must supply a BitmapMaterial utilized by the mesh on which the manager will draw the shadows. Usually, you’ll want to create a CompositeMaterial, with the BitmapMaterial most likely at the top of the stack. It might look something like this:

	var compositeMaterial = new CompositeMaterial();
	compositeMaterial.addMaterial( new ColorMaterial(0x888888) );
	var mat:BitmapMaterial = new BitmapMaterial( new BitmapData(1,1) );
	// .. the bitmapdata will get overwritten,
	//    but must contain _something_
	var sphere:TriangleMesh3D = new Sphere(compositeMaterial, 1000);

	_shadowCaster.registerShadowReceiverObject(sphere, mat);

To update the shadow textures, call

	_shadowCaster.update();

– probably in your onEnterFrame or onRenderTick function.

That’s enough to get going. From there, it’s just a matter of adjusting the projector’s position and orientation in relation to the objects in your scene with the properties projectorPosition, projectorPitch, and projectorRoll. Check the source for the various configurable properties like blur, alpha, color, etc.

Limitations, bugs:

(more…)

Video Projector Effect, Advanced

Sunday, May 31st, 2009
Thumbnail - Click me

This version handles arbitrary rotations, field of view / focal length, and uses correct math; the previous example faked it somewhat, doing everything basically with 2D math (but why admit to that? :). I found it twice as difficult getting to this point compared to the version posted a few days ago. It turns out that when you want to rotate more than 90 degrees around the x-axis, it’s good to make friends with quaternions, if only superficially.

And it’s interactive this time. By the way, is it not a royal pain to design an interface to control more than two axes in Flash for the web?

There’s still a lot that could be done right / better with this: backface culling (already tried and failed); frustrum culling; per-triangle alpha based on distance and angle to simulate light intensity; per-triangle focus/blur based on distance, maybe (but probably not); a more straightforward way to implement video; shadows… Basically, anything short of building a renderer-inside-a-renderer.

Bonus points for identifying the band to whom the set list pictured in the third image belongs.

Sorry, still no source.

 

Video Projector Effect

Tuesday, May 26th, 2009
Thumbnail - Click me

My general goal for this was to approximate the look of a video projection hitting a ‘non-planar surface’ or otherwise intersecting objects in a 3d scene.

The underlying logic wasn’t too painful. The UV coordinates of each vertex are all that need to be solved for. There’s a linear relationship between the position and orientation of the ‘projector’, and the world space position and UV coordinates of each vertex in the scene — requiring just algebra, which is nice, because that’s about all the math I know. JiglibFlash is being used for the physics action.

In its current implementation, the projector is limited to looking straight down the z-axis. I have a feeling that the most elegant approach for all this would be to extend Camera3D or to piggyback on top of the plumbing of the Papervision framework in some other way (apologies for the mixed metaphor). Should I develop the ‘Projection Manager’ to handle arbitrary projector orientation, I’ll post the source along with an updated example.

An SEO implementation for Flash

Thursday, April 23rd, 2009

I recently added SEO to my current (Flash-based) portfolio site, which I did as an experiment more than for any actual use value. You can navigate the HTML ‘back-end’ pages starting from here, if interested. I figured I should jot down my findings on how it can be done, for the record.

I. Client Side

On the Flash end, two requirements:

(A) Your site must implement deeplinking, presumably using SWFAddress, such that every page view has its own URL path. And your shell class must be intelligent enough to go directly to any given deeplinkable page view on startup via the URL deeplink.

(Sidenote: While not strictly necessary from an SEO standpoint, having gone that far, you’ll probably want all changes to the page view to be driven by the SWFAddress URL-change event, rather than directly by user events like mouse clicks, etc. Which will force you to throw your familiar, tried-and-true coding patterns out the window. And potentially, your laptop thereafter. But is beyond the scope of this entry. ;)

(B) Your site should use an external data source that describes the site structure and contains the site’s text content, as well as, typically, image and/or video URL’s and the like. Usually, this would be a well structured, static XML file whose node structure corresponds to the site’s actual page structure. The deeplink ‘crumbs’ should be included therein as well – as an attribute of each node, for example. (Extra points for using a database or a CMS instead). And it goes without saying that your Flash should be building its site structure, nav, and page content dynamically using all of the above.

(more…)

Faceted Crystal Ball

Tuesday, March 3rd, 2009
Thumbnail - Click me

Or at least, that’s the best easy analogy to describe it.

Gist: The color of each triangle in the mesh is obtained by averaging the pixel colors of the underlying image at the screen positions under each vertex as well as at the triangles’ center point. Some FlatShader-like lighting is used to add a little more three-dimensionality. Finally, a BlurFilter is applied to the source image to keep the transitions between colors from feeling too abrupt.

Source code: PvColorFacets.as

Licensed under a Creative Commons Attribution 3.0 License.

Webcam Green Screen Effect w/ Pixel Bender

Tuesday, February 17th, 2009
Thumbnail - Click me

A green-screen/chroma key effect utilizing Pixel Bender. With this version, the live video is compared against a reference image rather than a static color. Additionally, the alpha of the output is graduated based on the color difference between the base image and the incoming video (rather than just being either ‘on’ or ‘off’). Both of these differences may or may not be good for practical use, but can create interesting imagery.

Use the sliders to adjust the cutoff/falloff curve for the alpha. Click “Reset base image” to set the base image with which to compare any further changes in the video against. You can also choose your own background image (click “Browse…” at the bottom). Works best against a plain background and presumably best of all with proper lighting against a green background…

When I wrote the image processing routine in pure Actionscript, it ran like a slideshow. The Pixel Bender version on a 4-core system runs about 30x faster.

Pixel Bender kernel: GreenScreenEffect.pbk

Licensed under a Creative Commons Attribution 3.0 License.

Webcam image compositor, creating PNG48’s

Monday, February 9th, 2009

How it would look if you took a webcam or camera, and captured several frames of the same scene in quick succession, and then essentially averaged them out and composited them into one image? My guess was that the resulting image would retain most of its sharpness but lose most of the ‘noise’. I made this quick mini-app with that thought in mind.

Here is an example of my results (before and after). The images were captured with a Logitech Quickcam Pro 9000 at a resolution of 1600×1200 (averaged from 128 (!) still images), and were not scaled or retouched (just cropped).

Example - single frame Example - composited version
Example - movement...

As a side benefit, if the camera — or something in the foreground of the scene — moves during the image capturing process, you can get photography-like effects that look like a long exposure…

A PNG48 encoder class…

(more…)