decker labs

Just a quick note that I will be  regularly contributing to Creativity Online’s CATScan blog. It’s a collection of post from various people about digital media and advertising. I’ll be posting one article per month along with Dave Snyder, who is one of the Creative Directors at Firstborn. My first article is titled ‘It Doesn’t Hurt to Twitter Up Close and Personal’ and is about how businesses can use twitter to intimately reach their audience. My first article is linked below:

http://creativity-online.com/news/it-doesnt-hurt-to-twitter-up-close-and-personal/145442

Just a quick update here, we recently relaunched the new Doritos.com, my latest project. I was lead developer on this project with much help from Lucas Motta and Michael Roushey on the flash forefront, and as always Francis Turmel as our technical Sensei. The project was done with the world renowned ad agency Goodby, Silverstein & Partners. We put a lot of hours into this site, but I think it was worth it. My job as lead developer was doing the overall structure and framework of the site. I also developed some helpful debugging utils along the way, which I hope to post on here rather soon.

On a side note, I also brewed a special beer for this project. It was a Doritos style ale, and you can read all about it on my other blog, beer.eric-decker.com. I’ll be brewing a special beer per project from now on.

And on another side note, today was my girlfriend’s birthday. She’s a painter and has a blog which she updates regularly with her work and progress, and you should defiantly check it out at eringregory.com.

I probably should have mentioned this way earlier, but I’ll be speaking along with Mathieu Badimon at OFFF Paris on behalf of Firstborn. Our presentation will be on Friday, June 25th at 12:30. I have to say, this is quite an honor to speak at this conference. If you’re interested, there should be more information on the OFFF Paris 2010 website.

So I realize this is pretty last minute, but I’ll be speaking at FITC Toronto this Monday with Mathieu Badimon. We’ll be speaking on Firstborn’s behalf. Our presentation is titled “Tools & Prototypes: Crafting a Unique Digital Experience.” It’s basically focusing on how for each project we make, we usually need to make demos to test techniques or make tools to aid in development. It’s more of a creative presentation than a technical one, as we really won’t be talking about code directly. So if you’re up at FITC this weekend, we’ll be presenting Monday at 12:10.

http://www.fitc.ca/events/presentations/presentation.cfm?event=102&presentation_id=1227

Read more

Damn, it’s been a long time since I’ve had a site a launch. But I’m happy to announce my latest project at Firstborn has gone live: SoBe.com.  I was the lead flash developer for this project. The current SoBe campaign also featured a facebook app with contest and a mobile version of the site as well (which were not developed by me, but Phil and Miller respectively).

There are two sides to the site. The product ‘flavors’ section features videos of real people stopped and interviewed on street about what they think of SoBe lifewater (which we, firstborn, filmed). We used streaming video from FMS provided by Akamai in order to provide responsive seekable video. This was actually a challenge, as streamed video has some slight programmatic behavior differences than progressive download. For instance, streamed video will not dispatch any Net Events for when it has reached the end of the stream. Another issue I ran into was that due to security restrictions, the  BitmapData.draw function is prohibited unless specifically enabled on the FMS side. The other side of the site, ‘world’, was pretty straight forward, except that since we will be doing updates for the rest of the year the site needed to be extremely dynamic. This section of the site also features actress Ashley Greene in the nude, painted in two skinsuits to promote SoBe lifewater’s two new flavors.

Quick update on my current experiment building a 3D engine in Flash – I’ve added light and shaders. Shaders are special materials that are effected by the direction each face is pointing (called the normal) and another vector, in this case, a light direction. During the render process, each face calculates its brightness thus giving the illusion that light is reflection off of the mesh.

Upon researching, I found on a few occasions that the way to calculate lighting is to simply add the normalized normal and normalized light vector. This, however, seems to be slightly incorrect applicable to this situation. First, the result is another Point3D, where instead I am looking for a brightness value. Upon consulting coworker Mathieu Badimon, he suggested using the dot product of the normal and light vector. This returns a single number value, and when used to calculate brightness seems to work perfectly.

Next step? Finally tackling bitmap distortion in 3D, allowing for bitmap materials. (Which opens the doors to different types of shades as well, such as bump maps and dot3 normal mapping.) Once I’ve tackled that, I plan to completely restructure the engine, as I have a better idea now of how it can work while still sticking to my original plan of excluding a view3D. In the process of restructuring, I plan to write up the core math equations used in the most simplified way that I can, as finding some of these equations in a non-language specific explanation or simplified manor can be difficult.

Check out the new demo here. Click and drag the white dot to move the light source.

A few weeks ago I had to create a simple X-axis based parallax ‘engine’ for a project. Just basic horizontal movement, which is scaled based on an abstract z value. Well, somehow that turned into a full 3D engine. So I am continuing to build on what I had started in order to create my own Flash 3D engine, not because I think I can build a better Papervision, Away3D, or Five3D, (because I’m sure I can’t) but rather in order to learn. Since I end up using 3D engines fairly often now, I figure it’s good experience for me to build my own and see how it can work from the ground up.

So the first thing I learned, which really surprised me, is that building a 3D engine is easy. I’ll be the first to admit that math isn’t necessarily my strong suite, so believe me when I say it’s not too difficult. There are only a few tricky equations you need, and there’s plenty of information on the web since almost everyone it seems has created a 3D engine at one point or another. The tricky part, however, is getting the engine optimized and to run smoothly. That’s really what my challenge is.

I was able to get basic translation (which runs horrendously slow right now) as well as rotation implemented. I expanded on that to create a mesh, which can accept data imported by an .obj file (very easy to write a parser for) and create geometry. The mesh, like most in other 3D engines, consists of an array of vertex points and faces, which themselves are a collection of references to those vertex points. The mesh overall or individual faces can accept a material, as well as an optional back material for the reverse of the normal for when single sided is false. I calculate the z-depth for sorting based on the centroid of the polygon, which is probably not the fastest or most accurate way, but so far it has seems to work pretty well. Currently I have not yet tackled bitmaps or lighting, but that’s an interesting challenge for down the road.

On the note of optimization, I’ve noticed that Papervision, Away3D, and Five3D all use a scene3D and/or view3D. That said, all of your 3D ends up in more or less abstract container that handles the rendering. I suspect the reasoning for this is so that a render function only has to be called once, and all the appropriate transforms can happen here. In my experiment, I have decided against this, and probably at the stake of performance. The reason I don’t want to include a viewport or scene is so that the user can have more direct access to how the ‘Sprite3D’ is placed on the stage and it’s relation to other elements. I would also like to see how much can be achieved without having to use a container for the rendering process. Regardless, it’s been a fun learning experience so far.

Some demos below. The first example shows some of the basics. The second shows an experimental gradientFill material (creates a voronoi effect) and the third show an experimental shader that colors the faces in relation to their space in 3D.

So for the last few weeks I’ve been experimenting with Away3D. Away3D is a free 3D engine for Flash, very similar to Papervision3D. As a previous user of Papervision, I noticed a lot of similarities,  but there are a few differences, both good and bad, to take note of.  There are different features between the two, though for the most part they’re pretty similar.

The different is mainly in the syntax of how things are set up. As with Papervision, you need to set up a Scene3D, a View3D (Viewport3D), and Camera3D. Papervision requires that a Renderer is set up, where as with Away3D the rendering is done just through the view. The syntax is shorter for this in Away3D, as Papervision requires you to pass multiple objects:  renderer.renderScene(scene, camera, viewport), where Away3D you just call view.render().

Another difference is that Away3D uses what they call initObjects, which are objects that contain name-value pairs. So when creating a plane in Papervision, you do so by:  myPlane = new Plane(myMaterial,350,350,3,3),  where there are explicit arguments for the material, width, height, subdivisions x, and subdivisions y. Away3D, on the other hand, doesn’t require you to pass any arguments when you create your plane:  myPlane = new Plane({material:defaultMaterial, width:350, height:350, segmentsW:3, segmentsH:3}). This style has it’s pros and cons, the nice part is that when creating the object, you can set specific properties without having a long list of arguments in the constructor. The downfall is that if you are not carefull, it can fail silently due to typos or by accidently calling a property by a different name, such as something like:  myPlane = new Plane({segmentsW:3}) vs. myPlane = new Plane({segmentsX:3}).

I’m not going to go though all the differences I encountered, but there are more, and I think it boils down to personal preference. For example, when creating shaders in Papervision, you need to pass a specific light object to the material in order to make it work. In Away3D, as long as the light is in the scene, the shader will use it. Another nice feature Away3D has, which I’m not sure if Papervision has, is material caching. When you have multiple objects in a scene, if the object doesn’t move and the camera doesn’t move, that object is not updated on render. If there are objects that are behind the still object that are moving, those will be updated, but the still one will still not be. There’s an example of that here.

One of the features that Away3D does have that Papervision3D does not (yet?) support is Normal Maps.  Normal mapping, basically, is where you take a high polygon mesh and save out the details as a special material that can then be used on a low poly mesh to make it appear to have the higher count. The normal map is basically like an advanced bump map, but instead it is in color because it takes into account angles for where the light reflects and cast shadows. I’m not going to get into details, so check out wikipedia if yo want some more info. Here is an example I made that shows the difference between regular bitmap material and the normal mapping shader, as well as an example that shows you what the normal map looks like wrapped on the head as a texture. The model was made by my talented 3D coworker Andrew Jerez.

Check out the demo.

One issue with the normal map shaders in Away3D is that once you have multiple objects together, if those objects intersect, the render engine breaks and causes artifacts. There’s a few solutions to this. In the example above, the eyes are a different object/mesh. By assigning them to their own render session ( eyes.ownSession = new SpriteRenderSession() ) and then offsetting their z-depth ( eyes.screenZOffset = -10 ). This more or less will break the z depth sorting, so it can’t be used for complex geomety that would wrap around the object that uses the normal mapping. The other solution, which is best especially if you want more than one object to have  normal mapping shader on it, is that all the objects in the scene need to have the same shader material. So for instance, (not seen in demo) we have the head mesh and a hair mesh. We want both to have normal mapping, so we have to assgin the exact same material to each:

shader = new Dot3BitmapMapterial(textureBitmapData, normalsBitmapData);
headMesh.material = shader;
hairMesh.material = shader;

This technique means that you need to make sure that you set up yout UVs in the 3D editor (in this case Maya) so that each object can share the same texture file. Below are some examples showing the texture image, the normal map image, and the UB layout for the face and hair.

Normal maps also take some time to parse and since this happens in a for loop, if you have a large normal map, Flash can freeze for a few seconds while it runs though and creates the normal map. I broke this part out into an enter frame, which on completion dispatches a MaterialEvent. If you are interested, the source is here. Create the Dot3BitmapMaterial as normal, but pass manual:true in the initObject, add a listener for MaterialEvent.DOT3_MATERIAL_BUILT, and then call myMaterial.buildMaterial().

One last fun little thing. Away3D also supportes enviromental shaders. An enviromental shader is more or less like cheating to do reflection. You have a material on the object of the surrounding enviroment, which acts as the reflection result. An example of that is seen here on the Away3D site. The area being reflected is what is in front of the object, which is out of view of the scene and more or less “behind you” as you look at your screen. So a true reflection would be reflecting what’s really in front of that object, which in this case is you looking at your monitor. So hook up a web cam, capture that feed as a material, and there you have a ‘mirror’ material.

If you have a webcam, take a look.

If your camera doesn’t work, you might have to right click>settings, and on the camera tab change it to use the web cam. I’ve noticed this problem on macbooks. It’s slower, but you can also combine materials, such as the environment shader and Dot3Material.

For help with Away3D, visit the dev group they have set up on Google. The Away3D team regularly checks and provides the most help they can.

I’m a little behind here, but here is my latest project from Firstborn: AirTran EveryFlight.com, where I was the front-end developer for the project. We worked with Cramer-Krasselt in order to bring this site to life.  AirTran is a small domestic airliner, and they wanted to run a promotion to see what people wanted on every flight.  So we built a site to collect ideas, then let people vote on them and see how each one ranked on a state and national  level. The site only ran live for about 4 weeks until the winner was announced which happened to be Wi-Fi. You can see the old phase 1 site persevered here on Firstborn’s portfolio. It was a really cool project to work on, especially once it launched to see the suggestions come piling in. Our back end developer for the project, Wes Grubbs, built the database and admin tools, and we used AMF php, which provided to be really simple and easy to use.  And as always, I also got to do some nifty features on the site.

So the first challenge was that we wanted the user to be able to write in their suggestion on one of the cards and actually have a pencil do the writing. There were two important parts to this process. The first is getting each letter traced out. The basic methodology behind this is that we will create for each letter a set of points on a path (anchors and control points for curves) and then use that path to mask out the actual letter. By drawing that path over time, we can create the illusion of the letter being drawn in as it is being reveled. The second part is having a pencil follow the letter, which is simple since everything is done by numbers and the drawing API, so from the current path being drawn we can grab the ‘end’ of it and just have an object track that point, thus having it follow the path being drawn in and the illusion that the pencil is drawing the letter. To create the pencil, I used FIVe3D.  Mathieu had created a pencil for the M&M’s Join the Hunt site, so to avoid having it look too similar I didn’t bother using his model/code for reference, I just built out my own (plus it’s more fun that way).

In order to get the paths for each letter, I created a tracing tool. The tool allowed me to add points and make a path, as well as play with the curves and thickness of the line. Additionally, I added the feature that would allow the pencil to be “picked up” over certain points, meaning it wouldn’t be drawing for that segment (shift+click an anchor point). That is important for drawing letters that have different individual strokes, such as F, where when the pencil was moving I wouldn’t want to unmask a section of that letter as the pencil crossed it to get to another location. Once each letter is traced, it can be saved out as XML which is imported to the site and parsed into usable information.

See the letter tracing demo
See the letter writting demo

Another thing that was kind of interesting, although at this time I don’t have a demo for, was creating the rollover for the stack information. One of our newly hired 3D/Motion Designers, Brett Swanson, first modeled the stack to resemble the initial comps our designer Verena did. In order to make the cross-section, I first made a black and white image of the stack. The black being one side of the paper, and white being the other. Where the two colors met is the corner. So in Flash when the user rolls over the sack, I look across the horizontal strip of pixels from that mouse point and look for the area where white meets black, and I know that’s the corner. I can then do some masking, then calculate the rest of the points around it to create the cross-section plane, which is not 3D in anyway. As for the animation of the card being pulled out, Brett created a static animation that we imported as a PNG sequence, then position it to where the mouse click is, and scale it accordingly to be the correct size. Due to perspective, it doesn’t always match up. But since it is such a quick animation, we can fake the perspective by just slightly rotating the animation a little so that looks like the perspective matches up. As the animation plays, it rotates back to zero and scales back to the default size, and then is swapped out with the flipping animation which is dynamically done in five3D. As I always say, it’s all smoke and mirrors.

(images after the jump)

Read more

Sometimes you’ll need to load an AS2 swf into your AS3 project, and this seems like it can be a nightmare. And it can be, depending on what it is. You loose some functionality, such as loadMovieNum and getURL. On top of that, you can’t control the movie, as in you can’t access any public functions. There can be a workaround, though. The trick is to write an AS2 shell that acts like a bridge. You’ll load the AS2 swf into that shell, and communicate with the swf from that file, then communicate to the AS2 swf  bridge from your AS3 project via localConnection. This comes from a problem I recently was required to tackle, so I fiugred I’d share my solutution.

In this example, we need to load an existing game written in AS2 into an AS3 website. We do not have any access to the source files, just the compiled swf, which needs to be properly disposed in order to stop its audio channel. The solution is to create an AS2 shell that can load and unload the game, and then communicate with the shell via localConnection.

Step 1. Setting up the AS2 shell

Make a new AS2 targeted flash file and on the stage create a new empty MovieClip and give it an instance name, in this case container_mc.

Now as much as this hurts, click on the first frame and open the ActionScript panel. Yes, we’re going to add some very simple code to the timeline. First we want to load the game using loadMovie, targeting the container_mc that we created on the stage. This will load our file into this empty MovieClip so we can easily remove it later. Since we’ll be bringing this into and AS3 swf, we cannot use loadMoiveNum. The one line of code should look like this: loadMovie(“game_as2.swf”,this.container_mc) .

Step 2. Setting up the LocalConnection listener

This file is basically going to wait for a command from a localConnection at some point, so we’ll set that up immediately after loading the movie. In this case, we named the connection ‘gameBridge’ and will need to use that string in out AS3 swf in order to sync the two together. The code looks like this:
var lc:LocalConnection = new LocalConnection();
lc.connect(“gameBridge”);

Now we need to set up a method that will be called from AS3. In this case, it’s a dispose method. In order for the method to be called, it needs to be a method of the localConnection, so the full function name is lc.dispose = function():Void. Remember that lc is our localConnection object, and that in AS2 the datatype for void is capitalized. All we want to do in this function is kill the game, so we’re going to unload it and kill the connection. Just two lines of code, which looks like this:
lc.dispose = function():Void {
unloadMovie(container_mc);
lc.close();
}

That’s it for the AS2 side.

Step 3. Setting up the load in the AS3 project

We load in the AS2 swf just like we do an AS3 one, the only difference is that instead of it being of type MovieClip, it will be a special class called AVM1Movie, which stands for ActionScript Virtual Machine 1 MovieClip. Remember, we’re loading our shell we just made, not the game. So my load method looks like this:

private var game:AVM1Movie;
private var loader:Loader;

private function loadGame():void {
loader = new Loader();
loader.contentLoaderInfo.addEventListener(Event.COMPLETE, hndlGameLoad);
loader.load(new URLRequest(“gameshell.swf”));
}

private function hndlGameLoad(e:Event):void {
game = AVM1Movie(loader.contentLoaderInfo.content);
addChild(game);
}

In this case, the game was set to a frame rate of 30fps, and my app is running at 71fps. That means that the loaded game will now run over twice as fast. There’s not much we can do about this, but we can set the overall stage frame rate to match the loaded game, and when the game is unloaded, just remember to set it back. It’s just one line of code: stage.frameRate = 30;

Step 4. Setting up the LocalConnection call method

The last thing to do is call the AS2 swf and tell it to dispose when we need it to. We just set up a new LocalConnection object and pass it the name of the connection we set up in the AS2 swf and the method that we want to call. Looks like this:

lc = new LocalConnection();
lc.send(“gameBridge”, “dispose”);

And that’s about it. Pretty simple and quick way to load and still be able to dispose of an AS2 file.