360Flex - Doug McCune on Cool Stuff in ActionScript and Flex
First of all, Doug's session was not called "Cool Stuff" but I sanitized the title for those of you reading this at work (the session description was equally colorful). There was a lot of anticipation about Doug's presentation and in fact people were waiting outside the room early just to get a seat. His presentation is about finding inspiration in what you do and finding the time to do things that remind you why you love what you do (in this case, Flex development).
Doug discusses the story of building SpatialKey and how it started as just doing "cool" things with mapping and Flex. However, turning the cool stuff into an actual sellable product was difficult and "not fun." He was, in his words, burnt out and uninspired by Flex. So he decided to submit a presentation to give him a chance to do some things that inspire him. However, none of the items he is showing were built with Flex because he felt constrained by the framework. All the examples were focused on AS3, even if they were still "flex projects." He says he will be posting the code for all of them on his site when he gets the code a little cleaned up.
The first topic he covered was Steganography which is essentially "security through obscurity," or securing messages by hiding them from anyone but the sender and the intended recipient. He used the example of hiding messages within images though he says you could potentially hide data in other file types like audio or possibly in a SWF. You can accomplish the task of hiding the messages by changing one bit within each pixel in an image. This apparently can be accomplished without changing the underlying image. He managed to accomplish this on both BMP and PNG images, though he says it is much easier on BMP since it mostly raw image byte data.
The Flex framework contains a PNEncoder which is useful in accomplishing this task as well as a PNGDecoder class which is available at heriet.info. His AIR app is called STEGAsaurus and it allows you to hide text and/or files within images and then retrieve them. Using this AIR app he showed an example of dragging files onto an image within the application, saving it out and reopening it to retrieve those files. His ideas to take this to the next level would be to try to encrypt a SWF within a PNG and load it at runtine as an RSL or Flex module. Another idea he had was to encrypt messages inside an MP# and playback the message using text to speech software. Another idea was an AIR based web browser which would allow you to see the secret messages on images within the browsed pages.
The next project he did focused on head tracking. He was inspired by a YouTube video of head tracking with a WiiMote which was very impressive (you can see it here http://www.youtube.com/watch?v=Jd3-eiid-Uw). There are some Flash-based demos that exist but he thought there were some issues with how they performed. Both use a number of projects including Marilena (this post has the links http://www.francois-tarlier.com/blog/?p=235) and Deface (http://code.google.com/p/deface/) which are both based on another project called OpenCV. He used an algorithm called Haar Cascades but he optimized it for video since by default it rescans the entire image every time. His library to determine the face location by keeping track of where it last saw a face and starting there. This cuts down on the number of processing runs. His demo looked like the YouTube example and was actually very responsive. He discussed some issues he had to resolve with head tilting.
Using this facial recognition code, he built a "safe sexting" application to, in his description, help all the teenagers getting arrested for sexting. It managed to obscure your face using multiple filters and the blur tracked your face in real time. Then you could take your picture with the obscured face. He wanted to determine a way to send the images via MMS but didn't actually get that completely solved due to the complexities of working with different providers.
The next thing he showed was creating a game with head tracking. He was inspired by another video from YouTube by a guy trying to show how to refuel in the old Top Gun NES game. He recreated a Top Gun interface in Flash and he can use his head movements to dodge incoming missiles. It was more a proof of concept than a completed game which he called "Crappy Top Gun Game," but it worked well. He talks about taking this to the next level by, for example, making a head-tracking multiplayer game using CoCoMo or making an Eigenfaces AS3 implementation. Eigenfaces does more than just identify that a face exists but recognizes the actual individual.
His last example was about augmented reality (AR), which is taking the "real world" (like a webcam) and overlaying something in 3D or 2D. He was inspired by some videos using filters by Grant Skinner as well as the FLARToolkit and JackLinks Sasquatch demos built with Flash. He shows a Flash application for the GE Smart Grid that uses the "marker" in the webcam and adds a 3D image of wind turbines to the marker video in real-time. I mentioned that the Lego store is doing something like this with the boxes showing what the product would look like built. He showed a number of other videos of inspirations he had including some "weird dolls from Japan" that gets a huge laugh from the audience.
So he started exploring FLARToolkit which is a port of a C project called ARToolkit which has a GPL license (http://www.libspark.org/wiki/soquoosha/FLARToolKit/en). Since it is GPL licensed, you would need to purchase a commercial license if you decide to use it for a commercial application. He created an application similar to his "safe sexting" application which he called "play dress up." He takes off his shirt and is immediately pelted with a ton of small rubber balls from the crowd. He had a tattoo on his chest which the application tracked and added 3D fake muscles to, or fake breasts. He thinks that taking this to the next level you could add AR to greeting cards or wearable AR clothing. However, he thinks the real applications for this technology will be on something like the iPhone since that is carried around with you.
At the end he opens the room up for discussion about cool ideas that members of the audience were inspired by.
Legal nit picking, but that's not true - your own code would have to be GPL'ed, iirc, but that doesn't mean you can't sell the application.
PS Cheers for writing this up for those who couldn't go !
PPS Could you tone the CAPTCHA down ? I keep failing it :-)
I was waiting with bated breath to hear about Doug's session. The man creates some incredible applications, and it seems he's found his niche going back to 'fun' development and ideas that inspire him.
Stripping off in front of the crowd as well. He's got balls (apart from the ones thrown at him ;) )
Thanks for the detailed post. Just what I was after