Texture Streaming feature in Flash Player 11.3
One of the new features in Flash Player 11.3 and AIR 3.3 is called texture streaming. This feature is a new parameter called streamingLevels in the createTexture and createCubeTexture methods of the Context3D class. That means this feature relates to Stage3D. Here is the text from the ActionScript reference documentation:
streamingLevels:int (default = 0) — The MIP map level that must be loaded before the image is rendered. By default, the value is 0 meaning that the highest quality image in the MIP map must be loaded before the image is rendered. This parameter was added for Flash Player 11.3 and AIR 3.3. Use its default value of 0 to recreate the behavior of the previous versions of Flash Player and AIR.
Set streamingLevels to a value between 1 and the number of images in the MIP map to enable texture streaming. For example, you have a MIP map that includes at the highest quality a main image at 64×64 pixels. Lower quality images in the MIP map are 32×32, 16×16, 8×8, 4×4, 2×2, and 1×1 pixels, for 7 images in total, or 7 levels. Level 0 is the highest quality image. The maximum value of this property is log2(min(width,height)). Therefore, for a main image that is 64×64 pixels, the maximum value of streamingLevels is 7. Set this property to 3 to render the image after the 8×8 pixel image loads.
OK, that makes a little sense but what does this mean in terms of implementation? First let’s talk about mipmapping. Mipmapping is used on textures if the fragment shader requests it. Here is what an AGAL fragment shader looks like when requesting mipmapping:
tex ft1, v0, fs0 <2d, nearest, miplinear>
mov oc, ft1
This fragment shader then will use different sized textures (all powers of 2) for different depths on the vertices. There is a great ADC article on this subject by Marco Scabia located here. I am going to use his mipmapping sample code for my example of how to use the texture streaming feature.
In Marco’s mipmapping-sample.zip source code you see that he is creating textures for all the mipmap levels from an embeded checkers.png file.
[Embed( source = "checkers.png" )]
protected const TextureBitmap:Class;
// Roughly line 125:
var bitmap:Bitmap = new TextureBitmap();
texture = context3D.createTexture(bitmap.bitmapData.width, bitmap.bitmapData.height, Context3DTextureFormat.BGRA, false);
uploadTextureWithMipMaps(texture, bitmap.bitmapData);
// Roughly line 198:
public function uploadTextureWithMipMaps( tex:Texture, originalImage:BitmapData ):void
{
var mipWidth:int = originalImage.width;
var mipHeight:int = originalImage.height;
var mipLevel:int = 0;
var mipImage:BitmapData = new BitmapData( originalImage.width, originalImage.height );
var scaleTransform:Matrix = new Matrix();
while ( mipWidth > 0 && mipHeight > 0 )
{
mipImage.draw( originalImage, scaleTransform, null, null, null, true );
tex.uploadFromBitmapData( mipImage, mipLevel );
scaleTransform.scale( 0.5, 0.5 );
mipLevel++;
mipWidth >>= 1;
mipHeight >>= 1;
}
mipImage.dispose();
}
I have posted my modified code at this url https://github.com/renaun/ActionScriptExamples/tree/master/NewGraphicFeatures11_3. I am using the same checkers.png for the texture but added the new texture streaming feature into the sample app.
How this feature works is you can set the mipmap level that has to be uploaded for rendering to start. So for example if I have a 256×256 image it will have 9 mipmap levels (256×256, 128×128, 64×64, 32×32, 16×16, 8×8, 4×4, 2×2, and 1×1). Without texture streaming you have to upload all 9 mipmap level data before you can start rendering. By setting the streamingLevel property to say 7, only the 2×2 and 1×1 texture data has to be uploaded to allow rendering to start.
There are some ramifications to this. On the player side it still has to allocate all the memory for the texture sizes regardless of which ones are uploaded. This can result in more memory usage at runtime vs uploading all data at once. It might save on time to start rendering but doesn’t necessarily help upload texture time.
What it does give you is the ability to start rendering while waiting for high resolution textures to download, for example 1024×1024 textures might be large. It also gives you the ability include default texture data up to a certain size, say mobile small size, and if it’s rendering on a lower end device just use the included default textures (with modifying texture size appropriately). Then for the large device size start rendering with the mipmap level that is included. And then start downloading the higher resolution textures to use for that larger device. This approach takes some conditional logic in your code to use createTexture differently for the small vs large device size but can effectively save you memory on the smaller device size.
Here is what the code looks like to use texture streaming:
// This effectively removes the 512x512 and 256x256 texture memory on the smaller device, for some savings
if (isSmaller)
bData = createMipMapLevelFromBitmap(textureBitmap, 2); // width/4
texture = context3D.createTexture(bData.width, bData.height, Context3DTextureFormat.BGRA, false);
bData.dispose();
for (var i:int = 2; i <= 9; i++)
{
bData = createMipMapLevelFromBitmap(textureBitmap, i);
texture.uploadFromBitmapData( bData, i-2 ); // Shift mipmap levels to map to 0 index
bData.dispose();
}
}
else below code
*/
// Allowing you to include the 128x128 and down textures and only pull in the higher resolution textures for larger screens
bData = createMipMapLevelFromBitmap(textureBitmap, 0);
texture = context3D.createTexture(bData.width, bData.height, Context3DTextureFormat.BGRA, false,2);
for (var i:int = 2; i <= 9; i++)
{
bData = createMipMapLevelFromBitmap(textureBitmap, i);
texture.uploadFromBitmapData( bData, i );
bData.dispose();
}
Then what I do is listen for keyboard events of the numeric values to load the rest of the textures. This was just for testing and you’ll probably have a different mechanism to fire off the rest of the texture uploads. The main take away here is I needed to test how it looked when textures where uploaded, since the order of uploading matters, and this was an easy way to test the different levels.
Here is the method that is called when you press a number (1 maps to 0 etc…):
{
bData = createMipMapLevelFromBitmap(textureBitmap, mipMapLevel);
texture.uploadFromBitmapData( bData, mipMapLevel );
bData.dispose();
}
In my testing and with confirmation from the player team, I found out that rendering will not use the high resolution textures unless they are all there. Meaning if I started with 2-9 levels I should load 1 and then 0 for it to pick up the rendering quality progressively. You can load 0 then 1 but it wont upgrade the rendering until both are uploaded instead of the progressive quality upgrade of loading 1 first then 0.
Here is what the progression of the rendering quality as you upload (stream) in the textures from the example.
mipmap streaming example
Recent Comments