Close Show/hide page

Augmented Reality Texture Extraction Experiment

This is an AR-based experiment that enables the user to lift textures from real-world objects in live video and apply them onto 3D objects that are overlayed on top of them. Only box primitives are supported here, but the general idea could be extended to other types of 3D primitives or potentially even more complex objects with some clever image compositing and UV mapping.

Thumbnail - Click me
Click for live demo

Click the image on the right to run a live version of the experiment, in case you have the patience to suffer thru the quick-and-dirty hotkey-based UI.

Print the AR marker (PDF | PNG) and point your webcam at it.

It’s running Saqoosha’s (feature-incomplete) Alchemy branch of the FLAR Toolkit, along with Papervision3D.

In the future, it would be nice to figure out is how to apply bilinear filtering to the ‘deperspectivized’ textures. I could also add a feature to export the textured 3d objects into a 3D file format (probably OBJ) if there’s any interest.

Conceptually, this piece builds on these two video projection tests, and is a modest implementation of one of the themes from this post of ideas for augmented reality.

29 Responses to “Augmented Reality Texture Extraction Experiment”

  1. Wez Crozier says:

    Awesome execution and looks like it’s running really fast. Well done Lee!

  2. admin says:

    Thanks Wez. In the spirit of full disclosure I should say that when I recorded the screencast, I set the application to run all tweens 2x slower, and then baked the video to play back 2x faster than the speed at which it was recorded. :)

  3. Wez Crozier says:

    Sneaky. ;) How much performance did you gain by using the Alchemy branc of FLAR?

  4. nicoptere says:

    well done indeed :)
    it reminds me of the early version of carrara.

  5. admin says:

    @Wez Once I started getting into this project, I didn’t get a chance to compare its performance against the ‘trunk’ version of FLARToolkit. But it seemed a lot more responsive.

    Unfortunately, the current Alchemy branch only seems to support sampling bitmaps at 320×240. In the future, it may be nice to take advantage of the added performance to sample bitmaps for marker detection at a higher resolution.

  6. Complete Works of Borges. A stellar choice.

  7. [...] zero point nine » Blog Archive » Augmented Reality Texture Extraction Experiment [...]

  8. Marvin says:

    wow, awesome. keep it up!

  9. Nice work man.

    do you make it on flash? very fast!

  10. [...] zero point nine » Blog Archive » Augmented Reality Texture Extraction Experiment (tags: augmentedreality ar as3 inspiration flartoolkit textures cool fun) Share and Enjoy: [...]

  11. Rémi.T says:

    Amazing ! I love it

  12. KJ says:

    Very nice, incredibly smooth. Can definitely see the potential in this.

  13. Justin says:

    Amazing! Great idea and execution as always Lee. I like the little Borges cameo too – respect for you literary taste ;)

  14. admin says:

    Hehe, thanks Justin.

  15. [...] Very nice Flash Augmented Reality Experiment Idea: Augmented Reality Texture Extraction [...]

  16. Pipo says:

    Could you please explain how to take a snapshot of a particular area and to use it as BitmapMaterial for example on a Cube?

    How do you got such a good performance? :-)

  17. admin says:

    Pipo, I’ve greatly simplified the algorithm for the snapshot part which allows it to be applied to any kind of mesh, not just to a plane or cube face, and hope to post a demo and the code for it soon. (I’ve really wanted to do this sooner but have lacked the necessary alignment of time + energy.)

    Because in this demo the camera is assumed to remain stationary for the most part, I’m doing a simple check of the webcam’s activityLevel, and if it’s below a certain value I skip telling FLAR to do a detectMarker, which is where most of the performance hit comes from.

  18. Pipo says:

    Thanks a lot for your comment! I’m looking forward to your code. Don’t rush with it :-)

  19. Pablo Cabana says:

    Man, congratulations!
    The idea is just amazing. Keep the good work!!

  20. stoli says:

    photoshopped

  21. admin says:

    LOL ;)

  22. Chaco says:

    great idea please in the meanwhile can you explain more about this. do you mean that if there is no activity you put some kind of delay in the detecMarker function?

    [Because in this demo the camera is assumed to remain stationary for the most part, I’m doing a simple check of the webcam’s activityLevel, and if it’s below a certain value I skip telling FLAR to do a detectMarker, which is where most of the performance hit comes from.]

  23. admin says:

    Hi Chaco, here’s some pseudocode:

    function onEnterFrame(e:*)
    {
    if (_webcam.activityLevel < 25) return;

    var b:Boolean = _flarDetector.detectMarkerLite(_flarRaster, 80);
    // etc.
    }

  24. erick says:

    oh please please post the code :) this stuff is amazing!!

  25. Evan says:

    Hi there from the MFADT class!
    Awesome ideas & executions in these projects. I really want to learn more about pv3d and other AS related stuff from you.

  26. Simply desire to say your post is as amazing. The clearness in your information is just great and i can assume you’re an expert on this subjects, you must include more variety of articles such as breast cancer issues, to increase the popularity of your site. Well with your permission let me to grab your RSS feed to keep up to date with forthcoming post. Thanks a million and please keep up the enjoyable work.

  27. Jennie Puig says:

    I am aware this is actually uninteresting and you are basically skipping to the next remark Project Management Professional

  28. new-quotes says:

    You made some good points there. I looked on the internet for the issue and found most people will consent with your site.

  29. My wife and i ended up being absolutely ecstatic when Michael managed to deal with his investigations from the ideas he got out of the weblog. It is now and again perplexing to just happen to be handing out steps which often the rest may have been making money from. Therefore we do know we have the blog owner to appreciate because of that. The most important illustrations you have made, the straightforward blog menu, the relationships you can assist to engender it is mostly fantastic, and it’s letting our son and our family reckon that that subject matter is cool, and that’s unbelievably mandatory. Thanks for all the pieces!

Leave a Reply