Over the Easter weekend I finally got around to implementing a first prototype of an idea I’ve had for a while, which aims to bring some state of the art computer vision techniques to mobile devices.
Deep Vision uses the implementation of convolutional neural networks provided by libccv to classify images. So it’ll try to figure out whatever is the principal object in an image your provide it with.
At the moment it just has a sample classification database from the ImageNet project, containing 1000 assorted items, however in the future I’d like to see specific classifiers for different tasks (e.g. a classifier trained purely on different plants, so when you’re out for a hike and you want to know what something is you can just point your phone at it and find out.)
Unlike something like Google Goggles it’s doing all the classification on the phone itself without needing to upload the image to any external services.
In preparation for this I’ve put together a simple template for getting started with QML and Box2D development for both desktop Ubuntu and Ubuntu Touch. It’s available in two flavours:
Precompiled version – Includes QML Box2D already compiled for amd64 and armhf (when I have more time I’ll add i386 to this as well).
Source version – Makes it easy to compile everything yourself on whatever architecture you’re interested in.
This means that if your game is going to be purely QML based you can just grab the precompiled version, and run “make click-packages” and have packages built for both desktop and mobile use at the same time.
The template comes populated with an example application (one of the standard QML Box2D demos), which is found in the “src/” directory; so you’ll be able to see something running straight out of the box, then when you’re ready you can just replace this with your own game.
In the future I also plan to extend these templates to provide example packaging for multiple different QML + Box2D compatible mobile platforms (Sailfish, MeeGo, Blackberry, Android, etc.)
If you’re interested in seeing an example of the sort of thing you can achieve fairly easily with QML and Box2D I’ve also uploaded a video of one of my current work in progress projects:
CuteSpotify makes it possible to listen to your Spotify songs on Ubuntu Touch. It’s based on MeeSpot (a MeeGo Spotify client), which I’ve updated to make use of Qt5 and Ubuntu’s QML components. It’s still a little rough around the edges but most of the core functionality is in place now.
One particular problem to look out for is that because of the way Ubuntu Touch currently handles applications CuteSpotify has to be kept in the foreground and the phone has to be kept switched on for music to play (otherwise the application gets suspended). Approaches for handling applications that need to keep running are currently being debated, so hopefully that won’t be the case in future version of Ubuntu Touch.
As a temporary workaround, if your phone is in developer mode (achieved by connecting it up to the Ubuntu SDK) you can run “sudo service powerd stop” (default password is phablet) to stop the phone from going to sleep while you’re listening to music, then “sudo service powerd start” to resume normal power management when you’re done. (See popey’s comment for an alternative method).
Eyrie is a music identification program that I originally created for the Nokia N9 but which I’ve now also ported to Ubuntu Touch. It works in a similar way to commercial applications like Shazam and SoundHound, listening to music through the phone’s microphone and then generating an acoustic fingerprint using the open source EchoPrint algorithm. Currently the EchoPrint database isn’t anywhere near as large as the various commercial offerings, so it won’t recognise everything, but it has reasonable coverage.
Rockwatch allows your N9 to communicate with a Pebble smart watch. It makes it possible to install and manage Pebble apps, upgrade your Pebble’s firmware, receive notifications of e-mails and SMS messages and control your music from your Pebble.
New features in version 1.1
Preliminary support for the HTTPebble protocol which allows watch faces and apps to communicate with HTTP servers via the phone’s network connection and also provides them with location information. For example this provides support for watch faces that display local weather information, like Fair Forecast.
The ability to set the watch’s time from the N9’s clock.
Upcoming features
In the next version I hope to add support for accessing the two-way SDK via DBUS, this will allow other N9 applications to communicate with watch apps and get feedback from the watch.
An article I wrote a little while back for DZone about native Tizen development has just been published, in it I work through the creation of an application to identify music (similar in principle to Eyrie, although greatly simplified). It involves a brief look at the native user interface builder, use of the native sound capture and HTTP APIs, a small amount of audio processing and instructions on how to incorporate a general 3rd party GNU/Linux library into a Tizen project.
Erudite makes it possible to use Amazon’s Cloud Reader on various mobile platforms which aren’t officially supported by Amazon. You can either read books online, or download them for reading offline. Your progress in a book is then also kept synchronised between your phone and other Kindle devices.
Symbian Belle support
Until recently Erudite only supported MeeGo Harmattan and Mer based phones (such as the Nokia N9), but now it’s also available for Symbian Belle phones as well. I’ll try and put together a build for Symbian Anna phones in the near future.
There’s a fairly comprehensive review of Erudite for Symbian over on All About Symbian: Erudite review.
Upcoming features
In the next release for both Symbian and MeeGo I’ll be focusing on orientation switching support, so users can optionally view their books in landscape mode, as well as investigating some apparent issues with very large books.
Download
Symbian Belle version — Nokia 701, Nokia C6-01, Nokia C7-00, Nokia N8-00, Nokia X7-00, Nokia E6-00, Nokia 808 Pureview, Nokia 603, Nokia Oro, Nokia E7-00, and Nokia 700
This is running within the emulator using the Mesa LLVM OpenGL ES 2.0 implementation, so naturally I won’t be able to say for certain if a port is possible for actual phones running Sailfish until I get my hands on some hardware, but it seems hopeful.
Erudite makes it possible to use Amazon’s Cloud Reader service under MeeGo. You can either read books online, or download them for reading offline. Your progress in a book is kept synchronised between your N9 and other Kindle devices.
Name change
Unfortunately since Amazon own a wide range of “Fire” themed trademarks, and the application’s operation is heavily related to the same areas that these trademarks are used in, Amazon couldn’t really let me carry on using the name “Firestarter”. It was entirely my own foolishness that caused this hassle by going for a “clever” name without even considering the potential trademark implications, thankfully the lawyer who got in touch with me was very understanding and patiently helped me work through the various issues. I’m just glad that Amazon handled everything so amicably, rather than wading in with heavy-handed legal threats as some companies might.
So from now on the application is called “Erudite”, a name that’s clever in a different way.
New features
Added in 1.2:
The application name has now changed to Erudite.
A number of buttons and menu entries have been made larger and easier to press.
You can now switch in to and out of fullscreen mode by pressing a toggle button in the bottom right hand corner of the screen.
Added in 1.1:
The application can now start up completely offline. You still need to be online for the very first usage and to download books, but once you’ve downloaded some books you can now always access them offline.
A progress indicator shows when the application is loading.
Keyboard bug is fixed. Previously you needed to tap on text fields twice to bring up the virtual keyboard, now it pops up on the first tap.
Thanks to Daniel Juyung Seo this post is also available in Korean.
Overview
Tizen has the capacity for native applications written using the Enlightenment Foundation Libraries (EFL), however in the 1.0 SDK release this functionality isn’t particularly well exposed. Instead the main focus is currently on HTML5/JavaScript based applications, which is fine, but it’s not appropriate for all applications. As a way to get to grips with Tizen in more detail I decided to try porting one of my existing applications, Eyrie, from MeeGo Harmattan (as used by the Nokia N9) to Tizen.
Eyrie is one such application which clearly isn’t especially suited to being written as a web app. It requires the ability to get raw audio samples from the microphone and perform some signal processing on them to extract acoustic fingerprints. As far as I can see the current Tizen Web API doesn’t really have the capacity to handle audio input at such a low level, and even if it did making use of it for Eyrie would involve completely rewriting the EchoPrint codegen library in JavaScript to handle the fingerprinting.
However by making use of EFL I was able to get the bulk of Eyrie ported to Tizen in less than 24 hours. Thanks to both MeeGo and Tizen using GStreamer as their multimedia framework and the EchoPrint codegen library being easy enough to compile for both platforms all that was really needed was a new UI using EFL instead of QML. Below you can see a video of the initial port running on the Tizen emulator (I don’t have a real device to play with at the moment), the UI is still a little rough around the edges but all the main functionality is working.
In this tutorial we’ll work through the creation, building, packaging and deployment of a very simple EFL application, which I hope will give people a handle on a potential workflow for native application development using the 1.0 SDK release.
Scratchbox
Although it’s not made obvious, the SDK actually contains a full scratchbox environment which the IDE drops into when build packages. While it might potentially be possible to write EFL apps purely using the IDE I found it easier to ignore this and jump into scratchbox directly, this then frees you up to make use of whatever editor/IDE you’re most comfortable with. If you are looking to work just within the IDE you might find the Tizen Platform Developer’s Guide a good starting point.
I’ll be assuming that you installed the Tizen SDK to ~/tizen_sdk/
Note: After installing the SDK it’s necessary to open the IDE and create (and possibly compile) a platform project. Simply select a new platform project and then hit compile. This sets up the scratchbox targets the first time around. You only need to do this once after installing the SDK.
Before we enter the scratchbox environment we should find out what targets are available to us. We can do this by running:
Since I’m working with the emulator in x86 mode I’ll be using the tizen-emulator-1.0.sb2_gcc45sbox2.i386.platform target, to compile packages for real devices you’ll need to replace this with one of the armel targets in the following commands. I’m unclear on the intended separation between the two armel targets, but I expect either should be usable for our purposes.
To access the scratchbox as your normal user you can simply run:
This is perfectly usable for building and editing packages, and you should find that your home directory is available within the scratchbox. However the rest of the environment will be mounted read-only, preventing you from installing new packages.
To start a session in which it’s possible to install packages we can run:
This mode is only really suitable for installing packages and isn’t usable for general development as most compilation tools aren’t available.
We want to try writing a simple EFL application using the Elementary toolkit, so to start with we’re going to need to install the Elementary development package:
apt-get install libelm-dev
As you can probably tell from the above scratchbox is a Debian based environment, so most of the normal Debian tools are available to us. So for example if we didn’t know the exact name of the Elementary package we need we could simply run:
apt-cache search elementary
And we’d get a list of likely looking candidates to choose from. As a side note the Eclipse based IDE does have a simple package management interface which can be found on the default toolbar, however this appears to lacks any way of searching for packages, but if you are trying to do everything through the IDE without dropping into scratchbox you might find it helpful.
Elementary
So now that we have a compilation environment capable of targeting both the emulator and real devices we can start writing our applications. For this tutorial we’ll just create a very simple Elementary application that displays a label:
#include <Elementary.h>
static void on_quit(void *data, Evas_Object *obj, void *event_info) {
elm_exit();
}
EAPI int elm_main(int argc, char *argv[]) {
Evas_Object *win, *bg, *box, *lbl;
/* Create our window */
win = elm_win_add(NULL, "eflexample", ELM_WIN_BASIC);
elm_win_title_set(win, "Tizen EFL Example");
evas_object_smart_callback_add(win, "delete,request", on_quit, NULL);
/* Give it a background */
bg = elm_bg_add(win);
evas_object_size_hint_weight_set(bg, 0.0, 0.0);
elm_win_resize_object_add(win, bg);
evas_object_show(bg);
/* Create a box to place our label inside */
box = elm_box_add(win);
elm_box_horizontal_set(box, EINA_FALSE);
evas_object_size_hint_weight_set(box, EVAS_HINT_EXPAND, 0.0);
evas_object_size_hint_align_set(box, EVAS_HINT_FILL, 0.0);
elm_win_resize_object_add(win, box);
evas_object_show(box);
/* Make a label */
lbl = elm_label_add(win);
evas_object_size_hint_weight_set(lbl, EVAS_HINT_EXPAND, EVAS_HINT_EXPAND);
elm_object_part_text_set(lbl, NULL, "Hello Tizen!");
elm_box_pack_end(box, lbl);
evas_object_show(lbl);
/* Display the window */
evas_object_show(win);
/* Start the Elementary main loop */
elm_run();
return 0;
}
ELM_MAIN();
I’ll assume we’re working in the directory ~/src/tizen/eflexample/, so we can save this as ~/src/tizen/eflexample/eflexample.c
More comprehensive documentation on Elementary can be found on the Enlightenment Wiki.
We’ll keep the build process very simple for this example and just have a small Makefile handle compilation. For more complex projects both autotools and cmake are available.
This gets saved as ~/src/tizen/eflexample/Makefile
By including the $(DESTDIR) variable in our install paths we allow the Debian packaging we’re about to write to override the install destination and place it inside a temporary directory prior to being packed into a .deb.
If we run make in the ~/src/tizen/eflexample directory whilst inside our scratchbox session it should then compile successfully.
Launcher
You might notice in the earlier Makefile a couple of files that we haven’t yet discussed, com.mikeasoft.eflexample.desktop and icon.png. These are going to be used for adding our application to Tizen’s launcher. The .desktop file gets installed to /opt/share/applications/ alongside all other Tizen .desktop launchers, and should look something like:
Name=EFL Example
Type=Application
Exec=/opt/com.mikeasoft.eflexample/eflexample
Icon=/opt/com.mikeasoft.eflexample/icon.png
Comment=An example of how to write native EFL applications for Tizen
Version=0.1
X-TIZEN-TaskManage=True
X-TIZEN-Multiple=False
X-TIZEN-Removable=True
This is then saved to ~/src/tizen/eflexample/com.mikeasoft.eflexample.desktop
You’ll probably notice that there’s a selection of Tizen specific entries. The first, X-TIZEN-TaskManage, controls whether or not our application will appear in the Tizen task manager. The second, X-TIZEN-Multiple, tells the task manager if it is allowed to launch more than one copy of our application. If this is set to false the task manager will instead try to restore a currently running version of our application to the foreground. Because our simple example doesn’t perform the extra necessary work to allow Tizen to restore it to the foreground the task manager will instead simply kill our application when clicked a second time. Finally the last setting, X-TIZEN-Removable, specifies whether or not our application can be uninstalled through the application manager.
For the icon file we’ll simply use the Enlightenment logo:
Which needs to be saved to ~/src/tizen/eflexample/icon.png
Packaging
At this stage we could potentially just copy our eflexample binary to the device and start it via the command line, but a much more complete and redistributable solution is to construct a Debian package for it.
First we create our control file, which describes our package and its various dependencies:
Source: com.mikeasoft.eflexample
Section: user/other
Priority: extra
Maintainer: Michael Sheldon <elleo@gnu.org>
Build-Depends: debhelper (>= 7), libelm-dev
Standards-Version: 3.8.4
Homepage: http://blog.mikeasoft.com
Package: com.mikeasoft.eflexample
Depends: ${shlibs:Depends}, ${misc:Depends}
Description: An example of creating EFL applications for Tizen
Architecture: any
This gets placed in ~/src/tizen/eflexample/debian/control
Because our build system is fairly standard and handles the DESTDIR parameter being passed to it we can use debhelper to automate all of our rules file for us:
#!/usr/bin/make -f
%:
dh $@
This is then saved to ~/src/tizen/eflexample/debian/rules
Then we have our changelog:
com.mikeasoft.eflexample (0.1.0-1) unstable; urgency=low
* Initial release
-- Michael Sheldon <elleo@gnu.org> Sat, 23 Jun 2012 04:32:27 +0100
Which is saved to ~/src/tizen/eflexample/debian/changelog
Finally we can specify which version of debhelper we’re compatible with by running:
echo 7 > ~/src/tizen/eflexample/debian/compat
Now we’re ready to actually build our package. To do this we need to be inside our scratchbox session as a normal user, which we achieve by running:
cd ~/src/tizen/eflexample
dpkg-buildpackage -rfakeroot
After our application has been successfully compiled and packaged it’ll be saved in the parent of our current directory (i.e. ~/src/tizen/).
Deploying
Now that we’ve built a package for our application we can deploy it using the sdb tool provided with the SDK. This should work with either the emulator or a connected device. First we transfer our Debian package to the device and place it in the /tmp/ directory, then we run dpkg on the device to install our new package. If we had used any additional dependencies that aren’t already installed by default we’d need to use apt-get to install them prior to installing our own package. However since libelm is part of the default Tizen system all we need to do is:
