I’m looking forward to posting new and hopefully interesting things here as I progress through the program, the time during which I aim to broaden the scope of my personal sandbox and make experimentation a full-time vocation. But hopefully without losing any of the snark.

It was, to be frank, a gigantic pain in the ass coding the motion for this, since there seems to be no Tweener-like class written in Java (Java, not processing…). Anyone know of one?

Built using min3D.

free version

almost-free version :T

This post announces a 3D framework/library-in-progress I’ve been writing for Android, written in Java with OpenGL ES. Between the names min3d, modest3d, and llama3d, I’ve gone with “min3d” for being the least self-deprecating while still conveying an appropriate level of expectation that I’m comfortable with. :)

I’ve gone legit by putting it up on Google Code (first time for everything, etc.). While just an early build, it should at the very least be useful for educational purposes if like me you’re just starting down the Android path.

My aim while building this was to make a library that required no extra massaging of model data on its way to being used by OpenGL. To that end, data can be created and manipulated directly on the ByteBuffers that are used by OpenGL, without the need for any intermediating (ie, redundant) data structures. And hopefully wrapped in such a way that’s still relatively easy to use.

But this approach also imposes some important restrictions. The maximum number of vertices and faces of an Object3d is fixed after instantiation, and the dynamic addition/removal of vertex or face elements is currently not supported. If/when implemented, these operations will have to be much more costly than, say, with a linked list. But unavoidable, as far as I can tell.

Here’s what’s currently implemented through the API (all conventional stuff…)

• Vertex index lists
• Per-vertex colors
• Vertex normals
• Texture mapping
• Light source (just the one, for now)
• Camera class, with view frustrum settings
• Object scale, euler rotation, and translation properties as expected
• Object children that inherit parent’s transform properties
• Ability to render a subset of an object’s list of faces
• Triangle or point rendermodes (just the two, for now)
• A few canned object primitives

Code examples showing how to actually use “min3d” in its current form can be found through the project home page on Google Code by checking out the source with SVN, and going through the sampleProjects/ directory.

In the immediate future, I’d like to be able to at least add a light manager and a 3d-file format importer, and definitely test and debug more thoroughly. If anyone is interested in collaborating on or even picking up this project, please let me know.

As the geek sits down to learn a new programming language or new programming platform, the first thing he or she must do is learn to make a program that successfully prints the words “Hello, world!” to the screen.

The main significance of this first minor feat as it relates to the programmer is not epistemological, technical, or temporal, but psychological. Crossing the — shall we call it the “Hello world” threshold? — brings with it the realization of the existence of a universe of infinite possibilities. But also the burden and intolerable vertigo that comes with that sense of unlimited freedom.

For, alas, there is no standard “second program” for the geek aspirant to sit down to write. The programmer is on her own. Of the potentially infinite paths that lie in front of her, which will she choose?

Actually though, the most common second program chosen with pedagogical motives in mind — I would submit — would be that venerated bouncing ball, colored 0xFF0000. If you come from a Flash background, you probably know of it well.

Here’s mine, on the Android platform- (Sure wish I could’ve used Flash 10.1… ;)

Download installable AIR file
Requires AIR 2 Beta 2 Runtime +
Windows .NET v3.5

My aim here was to get familiar with AIR 2’s new capability of interacting with external processes’ standard streams.

One useful way of learning how to use a new feature is to proceed directly to trying to abuse it.

To that end, this AIR application takes in a fairly continuous stream of uncompressed binary image data from a native process’ standard output. My hope was to get real-time screen capture updates into AIR at a decent frame rate.

Details and source code after the break–

The native process

For my “native process,” I created a Windows .NET application written in C#. When asked, it takes a screen capture of the desktop and sends the image data through the process’ standard output. For various reasons, I chose to set it up as a Windows form rather than as a plain vanilla console or as a real system service. The relevant C# code is only a few dozen lines long. One interesting function there is bitmapToBytes(), which converts a bitmap into a byte array in the correct format for Flash to be able to apply it directly to a BitmapData without any further (costly) conversion.

The AIR application

On the “client-side,” the Flash AIR application has little to do. It simply sends a command to the native process, telling it to take a screen grab, along with image dimension and offset parameters. Because the client takes in the resulting stream of data in packets whose length should not be assumed to be of any given size, it needs to know where in the data a new image starts and where it ends. So it checks the stream for an arbitrary 4-byte start code as well as a termination code. When the transfer is complete, it just applies the resulting data, collected over time into a ByteArray, using BitmapData.setPixels(). It then requests a new image, and so on.

Performance considerations

On a mid-range quad-core PC using 1280×1024 images (5MB per uncompressed frame), I’m getting about 8 frames per second, or 125ms per frame. The time it takes for the 5MB image to transfer through the standard stream is about 65ms, or 50% of the total time for one iteration; without setting up an honest benchmark test, it’d be unfair and/or impolitic to declare that AIR represents a bottleneck here, so … yeah.

The AIR app eats up about 2/3 of one CPU core using the runtime’s main application thread. The remaining 1/3 represents the amount of time the AIR app has to wait while the native process constructs the screengrab. If the app were re-architected in such a way that the native process always has a new screencap queued up and ready to send – leading to little or no idle time between request and response – the AIR app would easily soak a full CPU core. Since the AIR app is doing little of significance besides reading the process’s standard stream and collecting that info into a byte array, it appears that merely handling large amounts of stdout data is quite costly.

Of course, AIR 2 isn’t even final yet. Nevertheless, I thought these preliminary findings were interesting. (This build targets Beta 2, which is the most recent public beta of AIR 2; compiling and running it against the most recent non-public beta of the SDK and runtime dated April 9, 2010 gave the same results.)

Source files

Setting up a native process for use with AIR 2 is well documented so I didn’t bother commenting on that here. However, the included source is I believe a good starting point for adapting to whatever native process craziness you may be cooking up…

For another implementation of AIR 2 screen capture, see this post by Mihai Corlan, also with source, using Java. There are also a few other related examples out there as well.

Licensed under a Creative Commons Attribution 3.0 License.

Here I’m just expanding upon the ‘texture extraction’ idea posted before. This version maps video onto 3D objects in a much more direct manner by using ray-casting, like the examples in the last several posts.

Controls:

• Mouse rotates object
• Mouse drag moves object
• [SPACE] remaps the texturing of the 3d object
• [ENTER] cycles through different 3D primitives
• [H] shows full list of kludgey hotkey controls

Update with video (8/25/2010):

Although this piece dispenses with the encumbrances of an AR marker or any application-like functionality, I think the method being used here has some potentially interesting uses for an augmented reality-based design application. (Or for more whimsical uses like “upload-your-face” done in 3D). I’ve disabled backface detection, so the imagery gets mapped on all sides of the object rather than just the camera-facing polys, which makes it more visually interesting. Live demo above requires webcam.

This one’s pretty niche: A small Windows utility that emulates the left and right mousebuttons using the keyboard. Specifically, using the right Windows key and the right Alt key.

Why? Because I use a MacBook Pro running exclusively in PC-mode, and I can’t stand how the mouseclick is implemented with its button-less trackpad. So yeah, pretty niche…

Sorry, no UI to change the hotkey assignments. If you’re comfortable with C#.NET, the source code is included in the zip file. It may be useful to examine if you’re interested in learning (1) how to capture keystrokes globally in Windows or (2) how to generate mouse operations programmatically, using some dreadful-looking Windows API interop code that I tried to learn as little as possible about in order to get to work.

Download (source included)

Simulates the projection of shadows coming from a spotlight light source onto irregular surfaces. In the supplied example, an animated MD2 model (from Quake 2) is casting a shadow onto the inside of a sphere. Supported parameters include shadow color, alpha, blur, falloff and texture map size, as well as position, rotation and field of view. This evolves out of the projection work posted previously.

Source:

SpotlightShadowCasterManager.as

Usage:

After instantiating the manager –

	_shadowCaster = new SpotlightShadowCasterManager();

– assign one or more DisplayObject3D’s that will block the light from the spotlight:

	_shadowCaster.registerShadowCasterObject(myMesh);

Then register one or more meshes that can have shadows cast upon it with registerShadowCasterObject. As the second argument, you must supply a BitmapMaterial utilized by the mesh on which the manager will draw the shadows. Usually, you’ll want to create a CompositeMaterial, with the BitmapMaterial most likely at the top of the stack. It might look something like this:

	var compositeMaterial = new CompositeMaterial();
	compositeMaterial.addMaterial( new ColorMaterial(0x888888) );
	var mat:BitmapMaterial = new BitmapMaterial( new BitmapData(1,1) );
	// .. the bitmapdata will get overwritten,
	//    but must contain _something_
	var sphere:TriangleMesh3D = new Sphere(compositeMaterial, 1000);

	_shadowCaster.registerShadowReceiverObject(sphere, mat);

To update the shadow textures, call

	_shadowCaster.update();

– probably in your onEnterFrame or onRenderTick function.

That’s enough to get going. From there, it’s just a matter of adjusting the projector’s position and orientation in relation to the objects in your scene with the properties projectorPosition, projectorPitch, and projectorRoll. Check the source for the various configurable properties like blur, alpha, color, etc.

Limitations, bugs:

- As described above, objects must be manually assigned to be either a ‘caster’ or a ‘receiver’. These roles aren’t determined dynamically by the class itself.

- Only one level of shadows is supported. Adding an arbitrary number wouldn’t be difficult, just cumbersome to manage under the current system, and increasingly expensive.

- Unfortunately, a ’shadow receiver’ object can’t have its own texture in addition to the one used to cast shadows onto, at least not in a way that looks decent. The reason is that a 3d object in Papervision3D has only one set of UV texture data and the shadow texture overwrites this with every update.

- There’s undoubtedly a less expensive way to do the raycasting/projection in the main math routine in the private function updateMesh. If you’re adept at this kind of thing, please send your improvements!

- Ugly and obvious artifacts show up on the ‘projectable’ textures when the spotlight is rotated 90 degrees away them. (You may have already noticed this in the previous two projection experiment posts). Again, I’d love any solutions anyone has for this.

Licensed under a Creative Commons Attribution 3.0 License.

This version handles arbitrary rotations, field of view / focal length, and uses correct math; the previous example faked it somewhat, doing everything basically with 2D math (but why admit to that? :). I found it twice as difficult getting to this point compared to the version posted a few days ago. It turns out that when you want to rotate more than 90 degrees around the x-axis, it’s good to make friends with quaternions, if only superficially.

And it’s interactive this time. By the way, is it not a royal pain to design an interface to control more than two axes in Flash for the web?

There’s still a lot that could be done right / better with this: backface culling (already tried and failed); frustrum culling; per-triangle alpha based on distance and angle to simulate light intensity; per-triangle focus/blur based on distance, maybe (but probably not); a more straightforward way to implement video; shadows… Basically, anything short of building a renderer-inside-a-renderer.

Bonus points for identifying the band to whom the set list pictured in the third image belongs.

Sorry, still no source.

  ]]> http://www.zeropointnine.com/blog/video-projector-effect-advanced/feed/ 5