This post shows you through a couple of infographics what happens when an AIR Native Extension is loaded and unloaded by the AIR Runtime.
First, a couple of useful resources
- What goes into a native extension for iOS? This is our previous infographic-rich post on how a native extension fits together. If you haven’t read it, I recommend you have a look at it first.
- What do the arrows on the diagrams mean? The infographics in this post are in the form of UML Sequence diagrams. Have a look at our post on what these are, if you aren’t familiar with them: UML 2.0 Sequence Diagrams – introduction.
Functions that your native code needs to implement
The Adobe AIR SDK defines a C interface, which your native code needs to implement, so that it can be called by the AIR Runtime. The function signatures you’ll need to implement are defined in FlashRuntimeExtensions.h, which is usually found in the include folder of the Adobe AIR SDK.
These are the functions that you will need to implement:
- The first pair is called by the AIR Runtime when your native extension is loaded or unloaded:
- These two are called when your ActionScript code instantiates flash.external.ExtensionContext:
- And this is the signature that your custom functions will need to have – these are the functions that you will call from ActionScript:
Loading a native extension – initialization phase
Here is what happens at runtime when you use a native extension:
1. Your app makes a call into your ANE’s AIR library code – typically by creating an instance of the class that you have provided as an interface between ActionScript and native code. Let us call that YourNativeExtension.as
2. YourNativeExtension creates an instance of flash.external.ExtensionContext. For what is ExtensionContext have a look at the bottom of the post.
3. This causes AIR Runtime to call your Extension Initializer function, let us call it YourExtensionInitializer(), which implements FREInitializer(). FREInitializer() takes three arguments:
- extDataToSet – a pointer to a pointer to data you may want to set for your extension.
- ctxInitializerToSet – a function pointer; set it to point to your implementation of
FREContextInitializer(). - ctxFinalizerToSet – a function pointer; set it to point to your implementation of
FREContextFinalizer() function
Note that all of these are output arguments, which YourExtensionInitializer() sets.
4. Now that the AIR Runtime has a pointer to your implementation of FREContextInitializer (named YourContextInitializer() on the diagram below), it calls it to finish the initialization of flash.external.ExtensionContext.
YourContextInitializer, in turn, sets the pointers to the functions you have implemented as your native code API. I’ve named them NativeFunctionA and NativeFunctionB on the diagram. Your AIR library now has an extension context that knows all about your native functions and you can use it to make calls to them.
Unloading a native extension – finalization phase
In your native extension you can create one or more instances of ExtensionContext, each with its own type. You can respectively have data that you allocate for each instance – ‘context-specific data’. YourContextFinalizer is the function where you can clean this data up. This function gets called when one of the following happens:
1a. you explicitly call dispose in your ActionScript code and dispose of your extension context:
myContext.dispose();
1b. the AIR Runtime garbage collector detects no references to your extension context and disposes of it;
1c. the AIR Runtime shuts down, normally when your application quits.
Hi Radoslava,
thank you for these insights.
I’m looking for better handling of images in CameraRoll in an AIR application. This means listing albums and thumbnails of the photos in it in a mobile app and having full access to the photos information (id, name, date of shot, …). Also showing the image in full size (scaled down of course for not having to scroll) should be possible.
What do you think: Is that possible with an ANE and how difficult would it be ?
Thank you for your answer.
Regards
Valentin
Hi Valentin,
My hunch is that the information you mention you need about the photos should be accessible though flash.media.CameraRoll and flash.media.MediaPromise without needing an ANE. But I take it you probably need to access stuff that’s not available in AIR (hence the need for better handling of the CameraRoll photos).
If that’s the case, an ANE is certainly possible:
Hope this helps,
Radoslava
Hello Radoslava,
thank you for your detailed answer.
Actually i also thought that i would get some image information in MediaPromise when selecting one photo from CameraRoll, but all properties stay null when getting back from the selection.
Since the possibilities aren’t that great what AIR 3.6 offers regarding having access to the device’s assets i will try now writing an ANE with the goals:
– getting all photo thumbnails and image information from the CameraRoll as an overview in my app
– getting specific thumbnails from the CameraRoll by passing an array of image names (or ids, just a unique identifier that can be used when iterating over the photos in CameraRoll)
– loading one full size photo from CameraRoll that has been selected in the overview in my app
I hope that this is not too difficult and finally fast enough to work with…
Regards
Valentin
Hi Radoslava,
i’ve advanced with my native extension: i can count now the number of photos in CameraRoll and load all the thumbnail images of these photos as UIImage’s into an array. What didn’t work so far is to populate the final result array with image objects that can be returned to the as3 part after async informing it that the result is ready to be fetched. I tried something like this (sorry for the poor quality, i’m not a C programmer):
NSInteger numberOfPhotos = [collector count];
// Create a new AS3 Array, pass 0 arguments to the constructor (and no arguments values = NULL)
FRENewObject((const uint8_t*)”Array”, 0, NULL, &thumbsArray, nil);
FRESetArrayLength(thumbsArray, numberOfPhotos);
UIImage *image;
NSData *data;
Byte *byteData;
NSUInteger len;
int j;
for (j = 0; j < numberOfPhotos; j++) {
image = [collector objectAtIndex:j];
data = UIImageJPEGRepresentation(image, 1.0);
len = [data length];
byteData = (Byte*)malloc(len);
memcpy(byteData, [data bytes], len);
FRESetArrayElementAt(thumbsArray, j, byteData);
}
FREDispatchStatusEventAsync(g_ctx, (const uint8_t*)"LOAD_PHOTO_THUMBNAILS_COMPLETED", (uint8_t*)"");
and
// Transfers the thumbnails loaded in LoadPhotoThumbnails to as3
FREObject GetPhotoThumbnails(FREContext ctx, void* funcData, uint32_t argc, FREObject argv[]) {
NSLog(@"Entering GetPhotoThumbnails()");
NSLog(@"Exiting GetPhotoThumbnails() with");
return thumbsArray; // has been defined outside every function as FREObject thumbsArray;
}
The code doesn't crash but i do not get the final array back to my AIR app. I also don't know what is the best/most performant way to do that. Using image data as ByteArray or BitmapData ?
Do you have an idea how i can do that ?
I guess when i know how to achieve this part the rest of the ANE shouldn't be a problem anymore.
Thank you for your help.
Best regards
Valentin
Hi Valentin,
It looks to me like you are on the right track.
In our camera extension (http://diadraw.com/air-video-camera-native-extension-for-ios/) the image data is passed as a ByteArray between Objective-C and ActionScript like you do.
Are you able to debug the Objective-C side of your extension? It might help to see what thumbsArray looks like at various stages of the calls.
Does the array come back with no elements or is it the correct size, but you can’t access the ByteArray elements you put in it (they are null, etc.)?
What might be easier and possibly more memory-efficient is to transfer the thumbnails one at a time, instead of allocating an array for all of them. This way as a first step you will just need to worry about transferring one ByteArray across.
You can then also only load thumbnails on demand and not have to keep them all in memory (if there are 100 of them, but the user can only see 20 at a time on the screen, for example).
I’m sure you’ll crack it, looks like you are going in the right direction.
Good luck!
Hello Radoslava,
thank you for your answer.
Meanwhile i got it working, for BitmapData AND ByteArray versions
Also i call now the context methods per photo, therefore not passing one array to C anymore.
With 83 photos it takes 9 seconds to get back the thumbnails and draw them into a Sprite with the ByteArray version. Since that’s too slow i guess i have to think about a paging mechanism (what you proposed as well), where only (say) 20 thumbs are loaded at maximum per request.
Also i’m able now to retrieve and return metadata (e.g. creation date, unique url) from the photos.
You see it works quite well, but still not that performant. I keep going and trying.
Have a nice week-end and thank you again!
Regards
Valentin
This is great news, Valentin.
Glad you’ve got it working, keep up the good work!
Radoslava
Hi Radoslava,
in case you’re interested, here the link to the ANE project i’ve created meanwhile:
https://github.com/rivella50/ANE-CameraRoll
Cheers
Valentin
Hi Valentin,
Thank you for sharing your code!
Looks like you’ve put in quite a lot of work in and there is even more to come with the Android version, which is wonderful.
Will there be an example app to demo how the ANE is used?
Thanks a lot and keep up the great work,
Radoslava
Hi Radoslava,
you’re right, an example is still missing. I will add this as soon as possible.
Cheers
Valentin