The challenge
Detect when a target is visible, or not, in the device camera and show the camera feed preview in the application background. Preferably use something that would work with an hybrid mobile application, like Ionic, or if nothing is found go native.
TL;DR
Implemented a plugin for Cordova (iOS and Android) and used OpenCV as the library of choice.
If you are in the search for an image detection/tracking library I recommend you use OpenCV, for most image processing challenges OpenCV does the trick.
Requirements
For requirements we had to find a library that allowed image detection/tracking of a given target and trigger an action on detection. The library shouldn't use an external API and should be free as in free beer and open source.
## Enter the search
In order to find a library that would achieve the application requirements a search was conducted and the result are presented bellow.
| Library/Framework | Paid | Hybrid Plugin |
|---|---|---|
| Wikitude | Yes | Yes |
| Vuforia | Yes | No |
| CrafAR | Yes (has free version) | Yes |
| ezAR | No | Yes |
| Layar | Yes | Yes |
| ARToolKit | No | No |
| OpenCV | No | No |
- Wikitude
- Would work perfectly but the price is not really what most are aiming to pay.
- Vuforia
- Paying and not hybrid, meaning we had to do a native implementation... Hmm... Not really an ideal solution.
- CrafAR
- Hybrid and not really that expensive but having to use their cloud didn't fit the requirements.
- ezAR
- Hybrid and not paid but didn't have the important part, image detection/tracking.
- Layar
- Hybrid and not really that expensive but, again, having to use their cloud didn't fit the requirements.
- ARToolKit
- Fits the requirements so we gave this one a try.
- OpenCV
- Also fits the requirements so it was a backup if ARToolKit didn't work.
## Testing and reaching a conclusion
After choosing ARToolKit as the library to test more in depth, the testing began and we started doing some implementations to see how this would work out.
We did a simple Android native application that detected and launched an alert message saying it was detecting. The reason for developing in native was because it was faster for testing.
The results obtained with the default trained targets were great but problems came up when training new targets and when making the same implementation for iOS. And the fact that training the target was quite time consuming (it only take a few minutes to run the training algorithm but results aren't always the best resulting lot's do overs) contributed in the decision to drop ARToolKit and tryout OpenCV. It also didn't help that we had no experience working in native (either iOS or Android).
Quickly after checking/reading Cordova plugins we made an iOS implementation and soon the Android implementation.
Results where good for most of the targets tested. Changing targets and adding new targets was a lot easier compared to ARToolKit since it didn't involve the amount of training that ARToolKit required. So OpenCV was the final and obvious choice for us.
## Conclusion
So if you are in the search for a library that does image detection/tracking in mobile apps (iOS and Android) give OpenCV a try. You can use template matching or feature matching to do the detection/tracking.
When it come to image processing OpenCV, being a computer vision library, get the job done perfectly. Of course this may not be the best choice but it's free and commonly used meaning you can quickly find quite a lot of information on the web about it in case you run into problems.
In case you want to check the plugin follow this link ImageDetectionCordovaPlugin.
This is it for now, see you in the next post.
