decker labs

Damn, it’s been a long time since I’ve had a site a launch. But I’m happy to announce my latest project at Firstborn has gone live: SoBe.com.  I was the lead flash developer for this project. The current SoBe campaign also featured a facebook app with contest and a mobile version of the site as well (which were not developed by me, but Phil and Miller respectively).

There are two sides to the site. The product ‘flavors’ section features videos of real people stopped and interviewed on street about what they think of SoBe lifewater (which we, firstborn, filmed). We used streaming video from FMS provided by Akamai in order to provide responsive seekable video. This was actually a challenge, as streamed video has some slight programmatic behavior differences than progressive download. For instance, streamed video will not dispatch any Net Events for when it has reached the end of the stream. Another issue I ran into was that due to security restrictions, the  BitmapData.draw function is prohibited unless specifically enabled on the FMS side. The other side of the site, ‘world’, was pretty straight forward, except that since we will be doing updates for the rest of the year the site needed to be extremely dynamic. This section of the site also features actress Ashley Greene in the nude, painted in two skinsuits to promote SoBe lifewater’s two new flavors.

Quick update on my current experiment building a 3D engine in Flash – I’ve added light and shaders. Shaders are special materials that are effected by the direction each face is pointing (called the normal) and another vector, in this case, a light direction. During the render process, each face calculates its brightness thus giving the illusion that light is reflection off of the mesh.

Upon researching, I found on a few occasions that the way to calculate lighting is to simply add the normalized normal and normalized light vector. This, however, seems to be slightly incorrect applicable to this situation. First, the result is another Point3D, where instead I am looking for a brightness value. Upon consulting coworker Mathieu Badimon, he suggested using the dot product of the normal and light vector. This returns a single number value, and when used to calculate brightness seems to work perfectly.

Next step? Finally tackling bitmap distortion in 3D, allowing for bitmap materials. (Which opens the doors to different types of shades as well, such as bump maps and dot3 normal mapping.) Once I’ve tackled that, I plan to completely restructure the engine, as I have a better idea now of how it can work while still sticking to my original plan of excluding a view3D. In the process of restructuring, I plan to write up the core math equations used in the most simplified way that I can, as finding some of these equations in a non-language specific explanation or simplified manor can be difficult.

Check out the new demo here. Click and drag the white dot to move the light source.

A few weeks ago I had to create a simple X-axis based parallax ‘engine’ for a project. Just basic horizontal movement, which is scaled based on an abstract z value. Well, somehow that turned into a full 3D engine. So I am continuing to build on what I had started in order to create my own Flash 3D engine, not because I think I can build a better Papervision, Away3D, or Five3D, (because I’m sure I can’t) but rather in order to learn. Since I end up using 3D engines fairly often now, I figure it’s good experience for me to build my own and see how it can work from the ground up.

So the first thing I learned, which really surprised me, is that building a 3D engine is easy. I’ll be the first to admit that math isn’t necessarily my strong suite, so believe me when I say it’s not too difficult. There are only a few tricky equations you need, and there’s plenty of information on the web since almost everyone it seems has created a 3D engine at one point or another. The tricky part, however, is getting the engine optimized and to run smoothly. That’s really what my challenge is.

I was able to get basic translation (which runs horrendously slow right now) as well as rotation implemented. I expanded on that to create a mesh, which can accept data imported by an .obj file (very easy to write a parser for) and create geometry. The mesh, like most in other 3D engines, consists of an array of vertex points and faces, which themselves are a collection of references to those vertex points. The mesh overall or individual faces can accept a material, as well as an optional back material for the reverse of the normal for when single sided is false. I calculate the z-depth for sorting based on the centroid of the polygon, which is probably not the fastest or most accurate way, but so far it has seems to work pretty well. Currently I have not yet tackled bitmaps or lighting, but that’s an interesting challenge for down the road.

On the note of optimization, I’ve noticed that Papervision, Away3D, and Five3D all use a scene3D and/or view3D. That said, all of your 3D ends up in more or less abstract container that handles the rendering. I suspect the reasoning for this is so that a render function only has to be called once, and all the appropriate transforms can happen here. In my experiment, I have decided against this, and probably at the stake of performance. The reason I don’t want to include a viewport or scene is so that the user can have more direct access to how the ‘Sprite3D’ is placed on the stage and it’s relation to other elements. I would also like to see how much can be achieved without having to use a container for the rendering process. Regardless, it’s been a fun learning experience so far.

Some demos below. The first example shows some of the basics. The second shows an experimental gradientFill material (creates a voronoi effect) and the third show an experimental shader that colors the faces in relation to their space in 3D.

So for the last few weeks I’ve been experimenting with Away3D. Away3D is a free 3D engine for Flash, very similar to Papervision3D. As a previous user of Papervision, I noticed a lot of similarities,  but there are a few differences, both good and bad, to take note of.  There are different features between the two, though for the most part they’re pretty similar.

The different is mainly in the syntax of how things are set up. As with Papervision, you need to set up a Scene3D, a View3D (Viewport3D), and Camera3D. Papervision requires that a Renderer is set up, where as with Away3D the rendering is done just through the view. The syntax is shorter for this in Away3D, as Papervision requires you to pass multiple objects:  renderer.renderScene(scene, camera, viewport), where Away3D you just call view.render().

Another difference is that Away3D uses what they call initObjects, which are objects that contain name-value pairs. So when creating a plane in Papervision, you do so by:  myPlane = new Plane(myMaterial,350,350,3,3),  where there are explicit arguments for the material, width, height, subdivisions x, and subdivisions y. Away3D, on the other hand, doesn’t require you to pass any arguments when you create your plane:  myPlane = new Plane({material:defaultMaterial, width:350, height:350, segmentsW:3, segmentsH:3}). This style has it’s pros and cons, the nice part is that when creating the object, you can set specific properties without having a long list of arguments in the constructor. The downfall is that if you are not carefull, it can fail silently due to typos or by accidently calling a property by a different name, such as something like:  myPlane = new Plane({segmentsW:3}) vs. myPlane = new Plane({segmentsX:3}).

I’m not going to go though all the differences I encountered, but there are more, and I think it boils down to personal preference. For example, when creating shaders in Papervision, you need to pass a specific light object to the material in order to make it work. In Away3D, as long as the light is in the scene, the shader will use it. Another nice feature Away3D has, which I’m not sure if Papervision has, is material caching. When you have multiple objects in a scene, if the object doesn’t move and the camera doesn’t move, that object is not updated on render. If there are objects that are behind the still object that are moving, those will be updated, but the still one will still not be. There’s an example of that here.

One of the features that Away3D does have that Papervision3D does not (yet?) support is Normal Maps.  Normal mapping, basically, is where you take a high polygon mesh and save out the details as a special material that can then be used on a low poly mesh to make it appear to have the higher count. The normal map is basically like an advanced bump map, but instead it is in color because it takes into account angles for where the light reflects and cast shadows. I’m not going to get into details, so check out wikipedia if yo want some more info. Here is an example I made that shows the difference between regular bitmap material and the normal mapping shader, as well as an example that shows you what the normal map looks like wrapped on the head as a texture. The model was made by my talented 3D coworker Andrew Jerez.

Check out the demo.

One issue with the normal map shaders in Away3D is that once you have multiple objects together, if those objects intersect, the render engine breaks and causes artifacts. There’s a few solutions to this. In the example above, the eyes are a different object/mesh. By assigning them to their own render session ( eyes.ownSession = new SpriteRenderSession() ) and then offsetting their z-depth ( eyes.screenZOffset = -10 ). This more or less will break the z depth sorting, so it can’t be used for complex geomety that would wrap around the object that uses the normal mapping. The other solution, which is best especially if you want more than one object to have  normal mapping shader on it, is that all the objects in the scene need to have the same shader material. So for instance, (not seen in demo) we have the head mesh and a hair mesh. We want both to have normal mapping, so we have to assgin the exact same material to each:

shader = new Dot3BitmapMapterial(textureBitmapData, normalsBitmapData);
headMesh.material = shader;
hairMesh.material = shader;

This technique means that you need to make sure that you set up yout UVs in the 3D editor (in this case Maya) so that each object can share the same texture file. Below are some examples showing the texture image, the normal map image, and the UB layout for the face and hair.

Normal maps also take some time to parse and since this happens in a for loop, if you have a large normal map, Flash can freeze for a few seconds while it runs though and creates the normal map. I broke this part out into an enter frame, which on completion dispatches a MaterialEvent. If you are interested, the source is here. Create the Dot3BitmapMaterial as normal, but pass manual:true in the initObject, add a listener for MaterialEvent.DOT3_MATERIAL_BUILT, and then call myMaterial.buildMaterial().

One last fun little thing. Away3D also supportes enviromental shaders. An enviromental shader is more or less like cheating to do reflection. You have a material on the object of the surrounding enviroment, which acts as the reflection result. An example of that is seen here on the Away3D site. The area being reflected is what is in front of the object, which is out of view of the scene and more or less “behind you” as you look at your screen. So a true reflection would be reflecting what’s really in front of that object, which in this case is you looking at your monitor. So hook up a web cam, capture that feed as a material, and there you have a ‘mirror’ material.

If you have a webcam, take a look.

If your camera doesn’t work, you might have to right click>settings, and on the camera tab change it to use the web cam. I’ve noticed this problem on macbooks. It’s slower, but you can also combine materials, such as the environment shader and Dot3Material.

For help with Away3D, visit the dev group they have set up on Google. The Away3D team regularly checks and provides the most help they can.

I’m a little behind here, but here is my latest project from Firstborn: AirTran EveryFlight.com, where I was the front-end developer for the project. We worked with Cramer-Krasselt in order to bring this site to life.  AirTran is a small domestic airliner, and they wanted to run a promotion to see what people wanted on every flight.  So we built a site to collect ideas, then let people vote on them and see how each one ranked on a state and national  level. The site only ran live for about 4 weeks until the winner was announced which happened to be Wi-Fi. You can see the old phase 1 site persevered here on Firstborn’s portfolio. It was a really cool project to work on, especially once it launched to see the suggestions come piling in. Our back end developer for the project, Wes Grubbs, built the database and admin tools, and we used AMF php, which provided to be really simple and easy to use.  And as always, I also got to do some nifty features on the site.

So the first challenge was that we wanted the user to be able to write in their suggestion on one of the cards and actually have a pencil do the writing. There were two important parts to this process. The first is getting each letter traced out. The basic methodology behind this is that we will create for each letter a set of points on a path (anchors and control points for curves) and then use that path to mask out the actual letter. By drawing that path over time, we can create the illusion of the letter being drawn in as it is being reveled. The second part is having a pencil follow the letter, which is simple since everything is done by numbers and the drawing API, so from the current path being drawn we can grab the ‘end’ of it and just have an object track that point, thus having it follow the path being drawn in and the illusion that the pencil is drawing the letter. To create the pencil, I used FIVe3D.  Mathieu had created a pencil for the M&M’s Join the Hunt site, so to avoid having it look too similar I didn’t bother using his model/code for reference, I just built out my own (plus it’s more fun that way).

In order to get the paths for each letter, I created a tracing tool. The tool allowed me to add points and make a path, as well as play with the curves and thickness of the line. Additionally, I added the feature that would allow the pencil to be “picked up” over certain points, meaning it wouldn’t be drawing for that segment (shift+click an anchor point). That is important for drawing letters that have different individual strokes, such as F, where when the pencil was moving I wouldn’t want to unmask a section of that letter as the pencil crossed it to get to another location. Once each letter is traced, it can be saved out as XML which is imported to the site and parsed into usable information.

See the letter tracing demo
See the letter writting demo

Another thing that was kind of interesting, although at this time I don’t have a demo for, was creating the rollover for the stack information. One of our newly hired 3D/Motion Designers, Brett Swanson, first modeled the stack to resemble the initial comps our designer Verena did. In order to make the cross-section, I first made a black and white image of the stack. The black being one side of the paper, and white being the other. Where the two colors met is the corner. So in Flash when the user rolls over the sack, I look across the horizontal strip of pixels from that mouse point and look for the area where white meets black, and I know that’s the corner. I can then do some masking, then calculate the rest of the points around it to create the cross-section plane, which is not 3D in anyway. As for the animation of the card being pulled out, Brett created a static animation that we imported as a PNG sequence, then position it to where the mouse click is, and scale it accordingly to be the correct size. Due to perspective, it doesn’t always match up. But since it is such a quick animation, we can fake the perspective by just slightly rotating the animation a little so that looks like the perspective matches up. As the animation plays, it rotates back to zero and scales back to the default size, and then is swapped out with the flipping animation which is dynamically done in five3D. As I always say, it’s all smoke and mirrors.

(images after the jump)

Read more

Sometimes you’ll need to load an AS2 swf into your AS3 project, and this seems like it can be a nightmare. And it can be, depending on what it is. You loose some functionality, such as loadMovieNum and getURL. On top of that, you can’t control the movie, as in you can’t access any public functions. There can be a workaround, though. The trick is to write an AS2 shell that acts like a bridge. You’ll load the AS2 swf into that shell, and communicate with the swf from that file, then communicate to the AS2 swf  bridge from your AS3 project via localConnection. This comes from a problem I recently was required to tackle, so I fiugred I’d share my solutution.

In this example, we need to load an existing game written in AS2 into an AS3 website. We do not have any access to the source files, just the compiled swf, which needs to be properly disposed in order to stop its audio channel. The solution is to create an AS2 shell that can load and unload the game, and then communicate with the shell via localConnection.

Step 1. Setting up the AS2 shell

Make a new AS2 targeted flash file and on the stage create a new empty MovieClip and give it an instance name, in this case container_mc.

Now as much as this hurts, click on the first frame and open the ActionScript panel. Yes, we’re going to add some very simple code to the timeline. First we want to load the game using loadMovie, targeting the container_mc that we created on the stage. This will load our file into this empty MovieClip so we can easily remove it later. Since we’ll be bringing this into and AS3 swf, we cannot use loadMoiveNum. The one line of code should look like this: loadMovie(“game_as2.swf”,this.container_mc) .

Step 2. Setting up the LocalConnection listener

This file is basically going to wait for a command from a localConnection at some point, so we’ll set that up immediately after loading the movie. In this case, we named the connection ‘gameBridge’ and will need to use that string in out AS3 swf in order to sync the two together. The code looks like this:
var lc:LocalConnection = new LocalConnection();
lc.connect(“gameBridge”);

Now we need to set up a method that will be called from AS3. In this case, it’s a dispose method. In order for the method to be called, it needs to be a method of the localConnection, so the full function name is lc.dispose = function():Void. Remember that lc is our localConnection object, and that in AS2 the datatype for void is capitalized. All we want to do in this function is kill the game, so we’re going to unload it and kill the connection. Just two lines of code, which looks like this:
lc.dispose = function():Void {
unloadMovie(container_mc);
lc.close();
}

That’s it for the AS2 side.

Step 3. Setting up the load in the AS3 project

We load in the AS2 swf just like we do an AS3 one, the only difference is that instead of it being of type MovieClip, it will be a special class called AVM1Movie, which stands for ActionScript Virtual Machine 1 MovieClip. Remember, we’re loading our shell we just made, not the game. So my load method looks like this:

private var game:AVM1Movie;
private var loader:Loader;

private function loadGame():void {
loader = new Loader();
loader.contentLoaderInfo.addEventListener(Event.COMPLETE, hndlGameLoad);
loader.load(new URLRequest(“gameshell.swf”));
}

private function hndlGameLoad(e:Event):void {
game = AVM1Movie(loader.contentLoaderInfo.content);
addChild(game);
}

In this case, the game was set to a frame rate of 30fps, and my app is running at 71fps. That means that the loaded game will now run over twice as fast. There’s not much we can do about this, but we can set the overall stage frame rate to match the loaded game, and when the game is unloaded, just remember to set it back. It’s just one line of code: stage.frameRate = 30;

Step 4. Setting up the LocalConnection call method

The last thing to do is call the AS2 swf and tell it to dispose when we need it to. We just set up a new LocalConnection object and pass it the name of the connection we set up in the AS2 swf and the method that we want to call. Looks like this:

lc = new LocalConnection();
lc.send(“gameBridge”, “dispose”);

And that’s about it. Pretty simple and quick way to load and still be able to dispose of an AS2 file.

It gives me great honor to announce that this year I will be giving a presentation at FITC Toronto alongside Firstborn coworker Jens Fischer. I’ve attended FITC for the last three years, and each time I’ve always enjoyed it. Shawn Pucknell has always puts together a great program with the best of the best of the Flash and web community. I have always found great ammount of inspiration from FITC, I always leave feeling creativtily renewed, so I really hope that this year I can be amoung one of the presentors that inspires others.

Our presentation is about the blending of 3D with web to create an engaging experience. We’ll talk about the techniques we have used in recent projects at Firstborn (such as Puma Lift, which launched on this past Friday), as well as technologies and techniques that we’ve just played around with. Here’s the writ-up as it is found on the FITC website:

Dimension Wars. Bridging the Gap Between 2D and 3D in Flash

The focus of this presentation will be the techniques used in creating 3D experience in Flash. We’ll look at different means of accomplishing this; whether it is creating a fully 3D environment right in Flash, combining pre-rendered sequences within a 3D environment, or even faking the 3D completely. We’ll be showing techniques that we have used in projects such as Sport Chalet: The Mountain, M&M’s Join the Hunt, Puma and XM Wild Ride. In addition to these, we’ll also be showing behind the scenes prototypes and demos, as well as internal experiments from our Firstborn Experiment Lab.

We will cover techniques from projects that use alternative means of creating 3D objects (how we created the mountain for the Sport Chalet site, first as based off of noise, then later from a VRML file), compositing rendered stills into a 3D-like environment (M&M’s), displaying 3D assets in a non-3D environment and adding 3D as a detail or interactive component in the Flash experience.

It’s not too late to buy tickets and attend. For more information about tickets, travel, etc, check out the FITC website.

Sport Chalet: The Mountain was featured in April’s issue of .NET Magazine, which is called Practical Web Design here in the ‘States. I was just short and simple write up in their showcase gallery section. It was very cool to see a project I was lead developer for in print in an international magazine. Here’s the short writeup:

TM Advertising approached Firstborn to design
a microsite for its new retail client, Sport Chalet.
Firstborn’s solution used an entire mountain motif
as the gateway for the content. The mountain
itself is modelled in Cinema 4D and exported into
Papervision3D. The result is an exciting and vibrant
interface that provides users with access to winter
apparel and equipment. The mountain serves as a
showcase for brands and their sponsored athletes.
You can check out the rider page of athletes and
see them model their sponsored gear through
video clips as well. Papervision is best used in
conjunction with some Flash, good photography
and web video. The Mountain has an excellent
dose of each.

I ran across two little bitmap related issues in Flash during my last project (should be live by end or March) that I’d thought I share.

The first issue is regarding layers of transparency. Sometimes in a sprite I’ll have multiple shapes layered on top of each other, or various layers of shapes drawn. Now if you change the alpha of that sprite, it will affect all the objects it contains. So, for example, if in my sprite I have one sprite with contains a red rectangle, and on top of it there is another sprite with a blue circle, if I change the alpha of the container sprite, you’ll be able to see though the blue circle since it’s alpha is also being changed. If I want the whole thing to fade together, so I don’t see though the individual parts, I’m out of luck.

So I figured I could use cacheAsBitmap to essentially flatten the sprite, so when the alpha is applied it would be applying it to an image. However, this doesn’t work. Obviously, I could just do it manually and draw the contents to bitmapdata, and swap it out whenever I need to do a fade. I’d like to avoid that as much as possible, so I figured I would try a last ditch attempt – using a empty filter. Basically, if I put a blur filter (or any other) on the container sprite that I am applying the alpha change it, Flash will automatically cache it as a bitmap. Logically, this should be no different than just setting the cacheAsBitmap property to true, as I had done before and failed. However, this works. I don’t know why, but somehow setting a filter to the sprite forces it to cache differently then just using the property. Below is an example (The bars at the top are to show a side to side comparison of each color for each object. It shows that the alpha affects the colors the same way for all of them, since it appears as an optical illusion that the one on the right is different)

(Either JavaScript is not active or you are using an old version of Adobe Flash Player. Please install the newest Flash Player.)

The other thing I’ve run into before, and I’m sure countless others have to, is applying a filter to an object which you have drawn something round in. What happens is that the right-most one or two pixels of the image get cut off.  It looks like Flash is converting the object to a bitmap (which we know it does), but is not taking anti-aliasing into account when setting up the bitmapdata width and height. The solution I used to get arround this issue is when drawing my shape I add a 2px transparent line around it. You won’t see the line, and since it’s a stroke it shouldn’t affect the active mouse area if the target is a button. If you’re drawing a shape  that already has a stroke around it, you’ll either need to draw it twice or do something to give the are just that little extra bit of padding on the edges. Here’s an image to help illustrate what I’m rambeling on about:

I started to play around with pixel bender a few weeks ago, but didn’t have much  time to really accomplish anything interesting. However, I came across the opportunity (or rather excuse) to use it to whip up a quick filter for my current project. Basically, we have a bunch of transparent png images of people or objects. In the images, the figures or objects cast shadows, but the shadow’s transparency is not taken into account (think of it more like a .gif where transparency is either true or false rather than smooth). So I wanted a way to take a selection of the image, and basically have it translate the brigtness value to alpha. The result would be similiar to using a multiply blend mode, if blend modes could be used with only one layer. Anyways, I couldn’t find a way in Photoshop to accomplish the effect to my liking, so I wrote a super simple pixel bender filter, imported it, and voilà, I had exactly what I needed. Pixel Bender is going to be a great too for player 10, but I think there could be a lot of interesting and practical uses for it even outside of Flash.

Here’s the filter (right/option click and save as, I guess Firefox and the like recognize it as a text file) if you’d like, as well as an image that hopefully clarifies what I was trying to describe above.