Michael Sheldon's Stuff

I’ve had another stab at getting Jokosher running on the Nokia n900 and I’m getting much closer to something actually usable now, as this screenshot attests:

There’s still a number of issues that need resolving before it’s really ready for use (most notably some playback/recording problems and some dialog boxes that are too large for the screen), but it’s getting there. When it’s working fully it could make the n900 a very useful device for portable podcasting, allowing users to record, edit, mix, encode and upload their roaming shows with nothing more than their phone.

A while back I wrote a few GStreamer plugins that expose OpenCV functionality as GStreamer elements (source code), I haven’t had much time to work on these recently myself, but thankfully a number of other folks have started contributing now. Yesterday Daniil Ivanov kindly packaged gst-opencv for the maemo extras-devel repository, and the n900 performs surprisingly well considering how CPU intensive many of the vision operations performed are.

This first video shows edge detection being performed from the n900’s main camera (whilst simultaneously being encoded):

Example gst-launch line: gst-launch v4l2camsrc device=/dev/video0 ! video/x-raw-yuv,width=480,height=272 ! videorate ! video/x-raw-yuv,framerate=12/1 ! ffmpegcolorspace ! edgedetect ! ffmpegcolorspace ! xvimagesink

This second video shows the faceblur element in action, it detects any faces in the current scene and blurs them out, the frame rate and resolution on this one had to be reduced somewhat due to the complexity of the operation, it looks clearer when performed directly to an xvimagesink rather than attempting to encode at the same time.

Example gst-launch line: gst-launch v4l2camsrc device=/dev/video0 ! video/x-raw-yuv,width=240,height=136 ! videorate ! video/x-raw-yuv,width=240,height=136,framerate=6/1 ! videoscale ! video/x-raw-yuv,width=120,height=68 ! ffmpegcolorspace ! faceblur profile=/home/user/haarcascade_frontalface_default.xml ! ffmpegcolorspace ! xvimagesink

For some more examples of the gst-opencv plugins in action on a normal desktop machine take a look at thiagoss’ blog post and a couple of videos by Alexandre Poltorak (edge detection and face blurring).

This year we’re lucky enough to have three students working on Jokosher as part of the Google Summer of Code, two under GNOME and one under GStreamer.

Andi Miller

Andi is looking into making it possible to do collaborative editing tasks between multiple Jokosher instances, between Jokosher and Pitivi and potentially between Jokosher and a small remote control (so you can use your phone/MID to start Jokosher recording/playing without needing to be sat at your PC). The project is progressing well with some Jokosher information and events already being exposed via a dbus interface.

Pēteris Krišjānis

PÄ“teris is working on finishing up some old work to provide telepathy support within Jokosher and then further extending this to add support for telepathy tubes. This will make it very easy for users to record VoIP sessions within Jokosher and the tubes support will also tie-in with the collaborative editing project, allowing Andi to send dbus messages via telepathy connections to remote users. The most interesting use case from my perspective is for podcasters working over VoIP, in the scenario where both participants are using Jokosher it should be possible for them to carry out a standard VoIP call and then afterwards have the two Jokosher instances automatically synchronise a high quality recording of each participant’s side of the conversation; so while the VoIP call quality might not be perfect the final audio will sound as if they’re both in a studio together.

David Williams

David is attempting to add musical score editing support to Jokosher, allowing people to sketch out musical ideas that can be played back as MIDI instruments alongside normal recorded audio. He’s already made some good progress in creating a python GStreamer element that can output some simple MusicXML based on an internal model (which can then be rendered to MIDI via the musicxml2midi element), this will then later be connected to a score editing UI, with the potential for multiple interface types (traditional scores, guitar tablature, drum events, etc.).

So hopefully by the end of the summer we’ll not only have a number of very exciting new features but also three more core Jokosher developers :).