Flex 4: A multi threading solution
If you develop applications in Silverlight a lot, it becomes more and more annoying that Flex doesn’t support multi threading. This is particularly annoying when you are processing a long running task and need to display some progress. The problem is that you can’t do this task on a background thread, this means that the ui thread is busy processing your task and it can’t update the ui until your task is done, at which the progress of course is 100%. In this article I’m going to show how you can somewhat work around Flex’ and more precisely Flash’ lack of multithreading while still being able to let the ui be responsive and show progress.
The sample Flex application
The sample application in this blog post is a Flex application which turns an image into greyscale. A better option for this would be to use a custom filter built with PixelBender, but this example demonstrates the problem quite well and with PixelBender you lose the ability to report progress. Take a look at the screenshot below:
It has a nice photograph of a parrot, two buttons and a progressbar showing progress when the image is being converted to greyscale. Next up, the code behind:
1: <?xml version="1.0" encoding="utf-8"?>
2: <s:Application >"http://ns.adobe.com/mxml/2009"
3: >"library://ns.adobe.com/flex/spark"
4: >"library://ns.adobe.com/flex/halo" creationComplete="saveOriginalImageData(event)"
5: >
6:
7:
8: <fx:Script>
9: <![CDATA[
10:
11: private var _originalImageData : BitmapData;
12:
13: private function greyWithPauseHandler(event:MouseEvent):void
14: {
15: var clone : BitmapData = _originalImageData.clone();
16: _parrot.source = new Bitmap(clone);
17: setProgress(0,0);
18: convertToGreyScaleWithPauses(clone);
19: }
20:
21:
22:
23: private function convertToGreyScale(toGreyScale: BitmapData) : void {
24: var totalPixels: int = toGreyScale.width * toGreyScale.height;
25: var amountForProgressUpdate : int = totalPixels / 16;
26: setProgress(0, totalPixels);
27:
28: for(var i : int = 0; i < totalPixels; i++) {
29: var y: int = i / toGreyScale.width;
30: var x :int = i - y * toGreyScale.width;
31: var pixel:uint = toGreyScale.getPixel(x,y);
32: var red:uint = (pixel >> 16) & 255;
33: var green:uint = (pixel >> 8) & 255;
34: var blue:uint = pixel & 255;
35: var grey:uint = (red * 0.3) + (green * 0.59) + (blue * 0.11);
36: toGreyScale.setPixel(x,y,(grey<<16) | (grey<<8) | grey);
37:
38: if(i % amountForProgressUpdate == 0 && i != 0) {
39: setProgress(i + 1, totalPixels);
40: }
41: }
42: setProgress(totalPixels, totalPixels);
43: }
44: private function convertToGreyScaleWithPauses(toGreyScale: BitmapData) : void {
45: var totalPixels: int = toGreyScale.width * toGreyScale.height;
46: setProgress(0, totalPixels);
47:
48: UIUtilities.pausingFor(0,totalPixels, function(i:int) : void {
49: var y: int = i / toGreyScale.width;
50: var x :int = i - y * toGreyScale.width;
51: var pixel:uint = toGreyScale.getPixel(x,y);
52: var red:uint = (pixel >> 16) & 255;
53: var green:uint = (pixel >> 8) & 255;
54: var blue:uint = pixel & 255;
55: var grey:uint = (red * 0.3) + (green * 0.59) + (blue * 0.11);
56: toGreyScale.setPixel(x,y,(grey<<16) | (grey<<8) | grey);
57: },setProgress ,this);
58:
59: _parrot.source = new Bitmap(toGreyScale);
60: }
61:
62: private function setProgress(processed : int, amountThatNeedsToBeProcessed : int) : void {
63: _progress.setProgress(processed, amountThatNeedsToBeProcessed);
64: }
65:
66:
67: private function greyHandler(event:MouseEvent):void
68: {
69: var clone : BitmapData = _originalImageData.clone();
70: _parrot.source = new Bitmap(clone);
71: setProgress(0,0);
72: // Need an artificial delay (Timer) to let the Image update itself and show the original image before
73: // the greying begins...
74: convertToGreyScale(clone);
75: }
76:
77: private function saveOriginalImageData(event:Event):void
78: {
79: _originalImageData = Bitmap(_parrot.content).bitmapData;
80: }
81:
82: ]]>
83: </fx:Script>
84:
85: <mx:ProgressBar mode="manual" minimum="0" maximum="1000000000000" id="_progress" horizontalCenter="0" bottom="6" label="Greying: %3%%"/>
86: <mx:Image id="_parrot" source="@Embed('./assets/parrot_heada.jpg')" left="10" right="10" top="10" bottom="77" scaleContent="true"/>
87: <s:Button label="Grey it with pauses" horizontalCenter="-107" bottom="48" click="greyWithPauseHandler(event)"
88: />
89: <s:Button label="Grey it" bottom="48" horizontalCenter="65" click="greyHandler(event)"/>
90:
91: </s:Application>
At this point I encourage you to download the sample project here and run it. If you don’t have Flash Builder installed, I’ve exported a release build here which you can run by double clicking the .html file. First click on the “Grey it” button, you’ll see the ui freeze for a while, when the application has finished converting the image to greyscale, the ui unfreezes and the progressbar jumps to 100%. Next, click on the “Grey it with pauses” button. You’ll see the image being converted to greyscale gradually and you’ll see the progressbar showing progress, you’ll also notice that the “Grey it with pauses” takes more time to complete than the “Grey it” button. Let’s explain what happens:
- Take a look at line 67, this method get’s called when the “Grey it” button is clicked. It clones the original BitmapData (which gets saved on the creationComplete event), resets the progress method using the “setProgress()” method on line 62 and calls the “convertToGreyScale()” method which does all the work. You can see a problem indicated in the comment. The problem is that when supplying the “_parrot” Image component with the original colored image data, the user interface doesn’t update until the “convertToGreyScale()” method has completed. The only way to work around this is to build in an artificial delay using a Timer for example and experiment to find the correct delay in which the Image get’s updated. That’s why I previously told you to first click on the “Grey it” button and then on the “Grey it with pauses” button. The other way around you don’t see the image going from colored to greyscale.
- In the “convertToGreyScale()” method on line 23, the BitmapData data is processed and converted to grayscale using a standard algorithm, maybe it’s not the most optimized algorithm, but it suffices. Most people would use nested for loops, I’ve converted it to a single loop in order to explain the next part more easily. This method updates the progressbar once in a while, in total 16 times, this is determined on line 25. If I updated the progressbar every iteration, I would get a script timeout, even if this timeout was set to 60 seconds, which is the maximum. As you’ve seen, the user only sees the progressbar at 100% when this method is finished, because only then can Flash update the ui. In fact, all the calls to “setProgress()” could be removed from this function, because they don’t have any visual effect.
- Next take a look at the “convertToGreyScaleUsingPauses()” method on line 44. This method get’s called when the user clicks the “Grey it with pauses” button. The heart of this method is on line 48. The “UIUtilities.pausingFor()” method is called, in almost the same way as I configured the for loop in the “convertToGreyScale()” method. The loop body has become an anonymous function that must accept an int (the current iteration) and is called multiple times depending on the first two arguments. Next to the first anonymous function I supply another function which is responsible for setting the progress. The “pausingFor” function makes sure that pauses occur, in which the ui can update itself, hence the name “pausingFor”. Take a look at the code below:
1: package
2: {
3: import mx.core.UIComponent;
4:
5: public final class UIUtilities
6: {
7:
8: /**
9: * Executes a for loop that pauses once in a while to let the UI update itself. The parameters of this method
10: * are derived from a normal
11: * for(var i :int =0; i <10;i++) {
12: * }
13: * loop.
14: * @param fromInclusive The 0 in the loop above.
15: * @param toExclusive The 10 in the loop above.
16: * @param loopBodyFunction The loop body, everything between {} in the loop above. This function must accept an int,
17: * which represents the current iteration.
18: * @param updateProgressFunction The method that needs to be called to update the UI, for example a progressbar.
19: * this method must accept two ints, The first is the number of iterations processed, the other is the total number of
20: * of iterations that need to be processed.
21: * @param componentInDisplayList Any component that is connected to the displaylist. This method makes use
22: * of the callLater() method which is available on any UIComponent. The root Application is an easy choice.
23: * @param numberOfPauses The number of times this method pauses to let the UI update itself.
24: * The correct amount is hardware dependent, 8 pauses doesn't mean you'll see 8 UI updates. Experiment
25: * to find the number that suits you best. A higher number means less performance, but more ui updates and
26: * visual feedback.
27: **/
28: public static function pausingFor(fromInclusive:int, toExclusive :int,loopBodyFunction : Function,updateProgressFunction : Function,componentInDisplayList:UIComponent,
29: numberOfPauses : int = 8) : void {
30: executeLoop(fromInclusive,toExclusive, toExclusive / numberOfPauses, loopBodyFunction,updateProgressFunction, componentInDisplayList)
31: }
32:
33:
34: private static function executeLoop(fromInclusive:int, toExclusive :int,numberOfIterationsBeforePause : int, loopBodyFunction : Function,
35: updateProgressFunction : Function,componentInDisplayList : UIComponent) : void {
36: var i : int = fromInclusive;
37: for(i; i < toExclusive;i++) {
38: //determine the rest of the number of iterations processed and the numberOfIterationsBeforePause
39: //This is needed to determine whether a pause should occur.
40: var rest : Number = i % numberOfIterationsBeforePause;
41:
42: //If the rest is 0 and i not is 0, a pause must occur to let the ui update itself
43: if(rest == 0 && i != 0) {
44:
45: //use callLater to pause and let the UI update.....
46: componentInDisplayList.callLater(
47: //Supply anonymous function to the callLater method, which can be called after the pause...
48: function(index:int) : void {
49: //after pausing, resume work...
50: loopBodyFunction(index);
51: //We need to continue with the executeLoop() method. The current index has already
52: //been processed so continue this method with the next index
53: executeLoop(index + 1,toExclusive,numberOfIterationsBeforePause,loopBodyFunction,updateProgressFunction,componentInDisplayList);
54: },[i]);
55: //When using callLater to let the UI update, my own code must be finished. So break out of the loop
56: break;
57: } else {
58: //No time for a pause
59: loopBodyFunction(i);
60: //Just before a pause occurs, report progress so that a user can set progress values
61: if(rest == numberOfIterationsBeforePause - 1) {
62: updateProgressFunction(i + 1, toExclusive);
63: }
64: }
65: }
66: //Final progress update
67: updateProgressFunction(i + 1, toExclusive);
68: }
69:
70: }
71: }
This is the source code of the UIUtilities class. The only public static function is the “pausingFor” function. Take your time reading the ASDoc, it explains what this function does and what the different parameters are for. This method calls another private static function “executeLoop()” which has almost the same parameters as the “pausingFor” method, except for the “numberOfIterationsBeforePause” parameter. This parameter is the result of the “toExlusive” parameter divided by the “numberOfPauses” parameter of the “pausingFor” method.
Let’s take a look the “executeLoop()” method on line 34. I’ve written inline comments that explain most of what’s going on. The most important part of this function is on line 46. This is the “callLater()” method that any UIComponent has and this is one of the most overlooked methods in Flex. It accepts a function as the first parameter and an array with arguments to supply to that function as the second parameter. When you call this method, Flex schedules the supplied function to be called in the next frame. This way, the ui can update itself for the rest of the duration of the current frame. This means that there must be enough time left in the current frame and there mustn’t be any of your own code that needs to be executed after the call to “callLater()” , or else you still won’t see any ui updates. In the example above I supply an anonymous function that needs to be called in the next frame. In this anonymous function I do two things:
- Let the “loopBodyFunction” parameter execute.
- Afterwards call the “executeLoop()” function again. This way I’m sure that after the ui has had time to update itself, the “executeLoop()” function resumes with the correct iteration after being suspended by the “callLater()” method.
Another interesting part is on line 61. In the iteration just before the iteration in which the “callLater()” function is called, I give the user the chance to set any progress values that the ui needs for updating itself, by calling the “updateProgressFunction” parameter and supplying the number of iterations processed and the number of iterations that need to be processed. This way, I minimize the calls to the ui to when they matter the most: just before a call to “callLater()”.
Conclusion
With my “UIUtilities.pausingFor()” function, I’ve shown how you can let the ui still be responsive and let it update itself, in a way that’s quite similar to using a normal “for” loop in a background thread. My “UIUtilities” class can also be easily expanded with a “pausingForEach()” function, building on the “pausingFor” function. While the api of the “UIUtilities” class is quite simple, be aware that using this api has a performance cost, letting the task finish without giving the ui the time to update itself, is always faster, but not necessarily better for the user of the application. Also, this class is nowhere near a replacement for having the ability to do true multi threading, which would be better for the performance, certainly on multi core processors. But until we have that possibility in Flash, we have to work around the lack of multi threading using solutions similar to the solution provided above. While my post was titled with “Flex 4”, the solution provided above should also work on Flex 3. You can find the working solution with the source code, here.