2.0.8 memory concerns

[tobias]

Megazine3 seem to use a LOT of memory even to the point that IE crashes.

During tests, I've used the following setup: no plugins, 22 pages in memory, around 70 pages total, with each page loading a jpg at around 500-1000KB. The exact book element:
<book pagepreview="false" bggradient="false" centercovers="false" cornerhint="false" dragrange="30" dragspeed="1" maxloaded="22" maxscale="3" minscale="2" pagewidth="782" pageheight="1088" plugins="" reflectionskip="false" startpage="1" zoomminscale="0.36764705882352944" zoomwheel="false" jsenabled="true">

I'm using Megazine3 version 2.0.8. I’m profiling with the Flash Builder 4 and IE8. My system has 6GB RAM.

When running the Flex profiler with IE8 the memory consumption seems logical: after megazine has initialized and loaded the first pages (22/2), the memory starts at 280 000K. Then for the first 6 page turns, the memory goes up in 50 000K increments. When it reaches around 550 000K, it "normalizes" and remains for a long time. Sometimes, memory consumption dips down more than 200 000K, to immediately bounce back to around 550 000K. After flipping through the whole book and halfway back again, there's a point when lot of memory is reclaimed, down to around 340 000K. This is quite close to where it started. The memory consumption in IE8 matches that of the profiler, except that it has another 100 000K on top. I guess this accounts for IE itself and maybe some content.

But when running megazine in IE8, WITHOUT the profiler, it's a very different story: memory consumption starts at the same level, around 350 000K, increments in 50 000K steps with each page turn. Now, however, the memory level quickly reaches to 1 000 000K. Next, it drops to around twice the starting amount. For the next 20 or so page turns, one of two things typically happen: if I turn the pages slow, the memory may again drop to twice the starting amount. But if I turn them at a more normal pace, the memory reaches to 1 500 000K, at which IE crashes.

Are these figures normal? What can we do to alleviate the memory consumption?

When turning a page, the memory goes up 50 000K, while the actual content being loaded is only at most 2 000K. That means some 48 000K memory is being allocated, apart from the content. Can you shed some light on this?

Do you have any idea why running the profiler makes such a huge difference?

Best Regards,
Tobias

[tobias]

Well, a lot of testing later ...

We've discovered that the content's file size doesn't really matter: loading 15-17K blank jpgs seem to impact memory just as much as loading 500-1000K jpgs.

What is the relationship between loaded content and the memory consumption? How can loading a couple of 15K jpgs add 50 000K to the browser's private working set? Does anyone have an explanation for this? Does it have to do with large pagewidth and pageheight? Does it have to do with BitmapData drawing? Anyone? I'm a little lost ..

It seems that the profiler anomalies may have been a temporary issue.

[Florian Nücke]

Try reducing the size of the thumbnails, and see if that makes things better (book@thumbscale). Memory for thumbnails is only generated when the thumbnail is first needed, so that's a definite reason for memory consumption growing while flipping through the book.

Although the numbers are quite... extreme. 50MB for just one page? I can't reproduce this at all. In IE8 with debug player I get ~60MB initial memory consumption, and get up to ~130 after flipping around quite a lot. But that's it. In Chrome with the non-debug Player I get almost identical results.
I used your definition as it is (copy paste).
(Note that these... "measurements" were taken using the Windows task manager, so no guarantee on accuracy )

[tobias]

Well, as it turns out, a 50 MB increase may not be so strange after all.

At first, the thought that an 500 KB jpg could add 50 MB to Megazine seemed absurd, but when that jpg gets loaded and added to the display list, it's not so absurd any more. By casting the loader's content to a bitmap, and running flash.sampler.getSize() on its bitmapData propery, the size is reported as over 20 MB. Now, given that there are two jpgs for each page, and adding some generated thumbs and other stuff, the size does add up to 50 MB. So, logically, using maxloaded="22" should reach 600 MB at most. I guess it all comes down to what jpgs are used.

However, 600 MB is far from 1 500 MB, which we do reach in IE 8 on Windows XP. There seems to a problem with IE not garbage collecting aggressively enough. When running the debug player and calling System.gc(), or using the old LocalConnection hack in the release player, the memory stays healthy. So, simply forcing GC solves the problem.

I'm not sure why the jpgs are exploded into 20 MB monsters, and what can be done about it. Anyone?

Also, when building Megazine, I've not been able to target flash player 10 successfully. It does build, but when running, a lot of exceptions are thrown.

Best regards,
Tobias

[Florian Nücke]

Well, as it turns out, a 50 MB increase may not be so strange after all.

At first, the thought that an 500 KB jpg could add 50 MB to Megazine seemed absurd, but when that jpg gets loaded and added to the display list, it's not so absurd any more. By casting the loader's content to a bitmap, and running flash.sampler.getSize() on its bitmapData propery, the size is reported as over 20 MB. Now, given that there are two jpgs for each page, and adding some generated thumbs and other stuff, the size does add up to 50 MB. So, logically, using maxloaded="22" should reach 600 MB at most. I guess it all comes down to what jpgs are used.

Hmm. More interesting than the file size would be the actual dimensions of the images. When generating a BitmapData by hand, I get ~32MB for 2880 * 2880. As expected. So I'd have to assume that your JPGs are even larger than that? In which case the obvious next question would be: do you really need such enormous sizes?
(Quick side remark: interestingly enough, the System.totalMemory variable doesn't show those 32MB... and I don't know why. Second remark: I also tried creating a ByteArray with 2880 * 2880 * 3 bytes [size in pixels * channels], but interestingly enough, the actual size isn't shown when calling getSize, that gives a constant value of 80. But this time it shows in System.totalMemory... and has the expected 32MB gah)

However, 600 MB is far from 1 500 MB, which we do reach in IE 8 on Windows XP. There seems to a problem with IE not garbage collecting aggressively enough. When running the debug player and calling System.gc(), or using the old LocalConnection hack in the release player, the memory stays healthy. So, simply forcing GC solves the problem.

Well, as long as it works... beware that this might stop working some time, though, as System.gc() is only officially supported in the debug player versions (see API). As for the hack, well, it's a hack (plus it can indeed have a very bad influence on performance to enforce garbage collection, especially via the hack. I had that in there whenever a reload occurred for testing once, it caused abysmal lags).

I'm not sure why the jpgs are exploded into 20 MB monsters, and what can be done about it. Anyone?

Well, JPEG is a compressed format, whereas the imagedata is kept uncompressed when in memory, so if your images are large, that's not really all that surprising. Simple formula: imageWidth * imageHeight * 3, one byte for each channel (RGB) for each pixel of the image. So if you have 15 megapixel photos that'd come pretty close.

Also, when building Megazine, I've not been able to target flash player 10 successfully. It does build, but when running, a lot of exceptions are thrown.

Didn't test that for a while, will look into that.
UPDATE: the errors were probably caused by a bug in the FlashPlayer, possibly related to this bug. Anyway, added a workaround, should work fine now.

[tobias]

Our images are 1956 x 2721. With the formula width x height x 3 channels the result is 15966828, or ~15 MB. As I said, when running getSize on the loaded bitmapData, it reports ~20 MB. Not sure what the difference is about.

Yeah, the hack is probably not a VERY good idea  , but what can you do when IE refuses to release the memory? No problem in Firefox.

[Florian Nücke]

Hmm, maybe it's additionally storing the alpha channel? In which case it'd be times 4, thus being exactly your 20MB.
No idea why it'd do that for a jpeg which has no alpha channel, though o.O
But well, as long as it works...

[tobias]

Ok, I found a solution.

I've implemented a kind of BitmapData pooling which works great - now memory remains at an stable level and I don't have to force a gc in IE.

When an image loads, I copy its pixels into a "pooled" BitmapData instance. I dispose the original BitmapData, and instead use the pooled one. With this approach, when all "maxloaded" BitmapData instances are in memory, no more instances need to be created.

Taking the approach further, it's possible to pool the "rasterize" and the "buffer" BitmapData instances, when those settings are used.

Have you considered implementing Object Pooling in Megazine? I think it might boost performance considerably.

[Florian Nücke]

Nice solution.

I have, I've played around with it on a small scale (the Vector2D objects I use for calculation of page position etc are pooled, e.g., because it kinda irked me to see the >50k allocations in the profiler ), but not for the BitmapData, yet. Might be indeed a nice to have. I'll think about it

[tobias]

I went for the BitmapData since it held more than 99 percent of the memory when running the profiler. Until I tried it, I didn't think pooling the BitmapData instances the way I did would work. After all, when I run copyPixels, the loader has already loaded the bitmap, and so it already resides in memory. Perhaps it works because the loaded bitmap is not yet on the display list.

In Megazine there are also quite a few Timeouts, Dictionaries, Shapes and Sprites being allocated. These too, may benefit from pooling.

Also, when turning my head inside out over our memory problems I've come to think about all the pages loaded into Megazine: with a hundred pages and a maxloaded setting of 10, only 10 pages are ever really needed. The other 90 could be pooled, and brought into life only when the user views them. Not sure if it's a good idea, maybe it's just me dreaming 

[Florian Nücke]

I'm pooling the bitmap data for the page buffers now, which is the only manually allocated bitmap data that's ever really freed during an engine's lifetime. The Vector2Ds were basically only a "testcase", where I experimented with it. I don't think I'll do it for the Timeouts and Shapes, they're too small to make a real difference, I think. Sprites on the other hand might be worth it. Will have to see where they are recreated, and how often.

As for the pages: in principle you're right. From the memory perspective that'd make sense. But then the getPage() and getPageSide() functions wouldn't have anything to report back for most pages. Breaking potentially a LOT of stuff Also, the costly process of creating the pages is mostly the creation of their element proxies, where the heavy duty XML processing takes place (for multilanguage / scale based values). So I'm pretty sure it'd actually lead to worse performance if that had to be reprocessed each and every time. Plus I can't really use pooling for the XML objects (as they allocate stuff internally I have no control over), possibly leading to even worse memory behavior.