Oh noes – I’m writing about a Google product, again. The omnipresence of the big G in my daily environment is becoming a bit excessive, so I’m stepping up my vigilance about not getting dependent on their services – though I don’t mind them knowing everything about me. In that light, acquiring another Android communicator may not seem logical, but I’m afraid that it is currently the choice of reason : I would have paid pretty much any price for a halfway decent Meego device, but Nokia’s open rejection of its own offspring is just too disgusting to collude with. The Openmoko GTA04 is tempting, but it is not yet available and I need a device right now.

Android does not quite mean I have to remain attached to the Google tit : thanks to CyanogenMod there is now an Android distribution free of Google applications  – and it also offers a variety features and enhancements… Free software is so sweet !

As a bonus, CyanogenMod is also free of the hardware manufacturer’s pseudo-improvements or the carrier’s dubious customizations – those people just can’t keep themselves from mucking with software… Please keep to manufacturing hardware and providing connectivity – it is hard enough to do right that you don’t have to meddle and push software that no one wants !

So when I went shopping for a new Android device after my one year old daughter disappeared my three year-old HTC Magic, I made sure that the one I bought was compatible with CyanogenMod. I chose the Motorola Defy because it is water-resistant, somewhat rugged and quite cheap too. By the way, I bought it free from access provider SIM lock – more expensive upfront, but the era of subsidized devices is drawing to an end and I’m going to enjoy the cheaper subscriptions.

On powering-on the Defy, the first hurdle is to get past the mandatory Motoblur account creation – not only does Motorola insist on foisting its fat supplements on you, but it won’t let you access your device until you give it an email address… In case I was not already convinced that I wanted to get rid of this piece of trash, that was a nice reminder.

This Defy was saddled with some Android 2.2.2 firmware – I don’t remember the exact version. I first attempted to root it using Z4root, but found no success with that method. Then I tried with SuperOneClick and it worked, after some fooling around to find that USB debugging must not be enabled until after the Android device is connected to the PC – RTFM !  There are many Android rooting methods – try them until you find the one that works for you : there is much variety in the Android ecosystem, so your mileage may vary.

Now that I have gained control over a piece of hardware that I bought and whose usage  should therefore never have been restricted by its manufacturer in the first place, the next step is to put CyanogenMod on it. Long story short : I fumbled with transfers and Android boot loader functionalities that I don’t yet fully understand, so I failed and bricked my device. In the next installment of this adventure, I’m sure I’ll have a nice tale of success to tell you about – meanwhile this one will be a tale of recovery.

This brick situation is a Motorola Defy with blank screen and a lit white diode on its front. The normal combination of the power and volume keys won’t bring up the boot loader’s menu on start. But thanks to Motorola’s hardware restrictions designed to keep the user from modifying the software, the user is also kept from shooting himself in the foot and the Defy is only semi-bricked and therefore recoverable. Saved by Motorola’s hardware restrictions… Every cloud has a silver lining. But had the device been completely open and friendly to alien software, I would not have had to hack at it in the first place, I would not had bricked it and there would have been no need for saving the day – so down with user-hostile hardware anyway !

With the Motorola Defy USB drivers installed since the SuperOneClick rooting, I launched RSD lite 4.9 which is the Motorola utility for flashing Motorola Android devices. Here is the method for using RSD lite correctly. RSD lite immediately recognized the device connected across the USB cord. The trick was finding a suitable firmware in .sbf format. After a few unsuccessful attempts with French Android versions, I found that JRDNEM_U3_3.4.2_117-002_BLUR_SIGN_SIGNED
_USAJRDNEMARAB1B8RTGB035.0R_USAJRDNFRYORTGB_P003_A002_HWp3_Service1FF worked fine and promptly booted me back to some factory default – seeing the dreaded Motoblur signup screen was actually a relief, who would have thought ?

After re-flashing with RSD Lite, I found that there is a Linux utility for flashing Motorola Android devices :  sbf_flash – that would have saved me from borrowing my girlfriend’s Windows laptop… But I would have needed it for SuperOneClick though – isn’t it strange that support tools for Android are Windows-dependent ?

With CyanogenMod in place, my goal will be to make my personal information management system as autonomous as possible – for example I’ll replace Google Contacts synchronization with Funambol. CyanogenMod is just the starting point of trying to make the Android system somewhat bearable – it is still the strange and very un-Unixy world of Android, but is a pragmatic candidate for mobile software freedom with opportunities wide open.

But first I have to successfully transfer it to my Android device’s flash memory… And that will be for another day.

If you need further information about hacking Android devices, great places are Droid Forums and the XDA-Developpers forum – if you don’t go directly, the results of your searches will send you there anyway.

With UMTS now potentially available on all the frequency bands traditionally allocated to GSM, why are we still operating GSM there while UMTS offers nothing but improvements over it and all contemporary handsets support it. The question is particularly pressing since data traffic has for quite a while accounted for more than 90% of network usage in volume and grows faster than backhaul can be deployed and cells made smaller while spectral efficiency has become awfully close to theoretical optima. GSM data modes such as GPRS and its incremental improvements have their purpose well, but they are hacks shoehorning data into a TDM voice world – nothing like the native capabilities of UMTS. Of course, modern marketing knows the value of nostalgia as an advertising vector, but I suspect that the market of users who insist on GSM for nostalgia’s sake may not be sufficient to justify its cost.

Some manufacturers nowadays offer unified RAN infrastructure that supports both UMTS and GSM on a single piece of equipment – and many antennas are now multiband, but there is still an awful amount of specific equipment with the associated duplicated costs… And then there is the effort of maintaining the software for two entirely independent systems, each with its own bugs, quirks and yearly upgrades attempting to squeeze more throughput out of a slice of spectrum that is not going to expand – a single large operator typically has dozens of people whose workload could be cut in half overnight. I for one would love to spend more time on GIS software for the fiber optics infrastructure and less dealing with the Jurassic park.

So what are we waiting for ? Don’t we understand that frequencies are too precious to be wasted on obsolete protocols ? Let’s recycle ! Let GSM retire ! Taiwan’s ministry of transportation and communications is already working on it…

I have owned an an HTC “G2″ Magic for almost two years and one of my biggest disappointments with the Android operating system has been my inability to find a decent Jabber client. On the desktop, my love of Psi has been going on for half a decade but my encounters with mobile Jabber clients have been nothing but disappointments.

On Android in the past two years I have tried them all, including notables such as Jabbdroid, Beem, Jabiru, Yaxim, Emess and many others not even worth citing. Some of them are hampered by a slow graphical user interface, some deplete batteries in a hurry, some lack features I consider essential, some even crash on receiving a message and not a single one is capable of remaining connected while the radio segment hops from GPRS to UMTS to Wi-Fi and back again… They won’t even try to reconnect – leaving me slack-jawed at the lack of such a basic feature when there is even a standard Android class that notifies applications when network connectivity changes.

Enter Xabber – it does everything I expect from an Android Jabber client. Yes, it really does – you can drop that unbelieving face. I’ll spare you the whole features list… Let’s just focus on what I was looking for :

• Permanent tray icon as link to contacts lists
• vCard based avatars
• XMPP priorities
• Groups
• Contacts list management
• TLS/SSL support
• Full Unicode support
• Chat history
• Parameters for just enough customization
• Multi User Chat – you can even join multiple rooms
• Does not deplete the batteries too quickly
• Reconnects promptly after each disconnection while the radio segment hops from GPRS to UMTS to Wi-Fi and back again

As a bonus it publishes geographical location, but I have no idea where it gets it from, nor if it is supposed to implement XEP-0080.

Don’t you love the feeling of discovering a new application and finding that it behaves the way you expect, as if the developers had been reading your mind and making helpful suggestions about the fuzzy parts of what they had read ? On Android K-9 Mail is the only other example I can think about… Yes, Xabber is that good.

The only downside of Xabber is that the code is not free… The site does not even mention a license. So you don’t know what lies hidden inside, you can’t modify it and you are at the mercy of the developer changing his mind and starting to ask for money for further versions. But even as a Free software fanboy I’m willing to live with that for now – I’m so relieved to at last have something that works.

From now on, expect to find me online while I’m on the move !

Edit 20130130 – Xabber is now Free Software !

In France, at least two mobile networks operators out of three (I won’t tell you which ones) have relied on Cell ID alone to identify cells… A mistake because contrary to what the “Cell ID” moniker suggests, it can’t identify a cell on its own.

A cell is only fully identified by combining with the Location Area Identity (LAI). The LAI is an aggregation of Mobile Country Code (MCC), Mobile Network Code (MNC – which identifies the PLMN in that country) and the Location Area Code (LAC – which identifies Location Area within the PLMN). The whole aggregate is called Cell Global Identification (CGI) – a rarely encountered term, but this GNU Radio GSM architecture document mentions it with details.

Since operators run their networks in their own context, they can consider that MCC and MNC are superfluous. And since the GSM and 3G specifications defines the Cell ID as a 16 bit identifier, the operators have believed that they had plenty for all the cells they could imagine, even taking multiple sectors into account – but that was many years ago. Even nowadays there are not that many cells in a French GSM network, but the growth in the number of bearer channels was not foreseen and each of them requires a different CellID – which multiplies the number of cells by their number.

So all  those who in the beginnings of GSM and in the prehistory of 3GPP decided that 65536 identifiers ought to be enough for everyone are now fixing their information systems in a hurry as they run out of available identifiers – not something anyone likes to do on a large critical production infrastructure.

Manufacturers and operators are together responsible for that, but alas this is just one occurrence of common shortsightedness in information systems design. Choosing unique identifiers is a basic modeling task that happens early in the life of a design – but it is a critical one. Here is what Wikipedia says about unique identifiers :

“With reference to a given (possibly implicit) set of objects, a unique identifier (UID) is any identifier which is guaranteed to be unique among all identifiers used for those objects and for a specific purpose.”

The “specific purpose” clause could be interpreted as exonerating the culprits from responsibility : given their knowledge at the time, the use of Cell ID alone was reasonable for their specific purpose. But they sinned by not making the unique identifier as unique as it possibly could. And even worst, they sinned by not following the full extent of the specification.

But I won’t be the one casting the first stone – hindsight is 20/20 and I doubt that any of us would have done better.

But still… Remember kids : make unique identifiers as unique as possible and follow the specifications !

Stumbling upon a months old article by my friend George’s blog expressing his idea of local social networking, I started thinking about Bluetooth again – I’m glad that he made that resurface.

Social networking has been in the air for about as long as Bluetooth exists. The fact that it can be used for reaching out to local people has not escaped obnoxious marketers nor have the frustrated Saudi youth taken long to innovate their way to sex in the midst of the hypocritical Mutaween.

Barely slower than the horny Saudi, SmallPlanet CrowdSurfer attempted to use Bluetooth to discover the proximity of friends, but it apparently did not survive: nowadays none of the likes of Brightkite, Gowalla, Foursquare or Loopt takes advantage of this technology – they all rely on the user checking-in manually. I automated the process for Brightkite – but still it is less efficient than local discovery and Bluetooth is not hampered by an indoor location.

People like George and me think about that from time to time, and researchers put some thought into it too – so it is all the more surprising that there are no mass-scale deployments taking advantage of it. I found OlderSibling but I doubt that it has a large user base and its assumed spying-oriented use-cases are quite off-putting. Georges mentioned Bliptrack, a system for the passive measurement of traffic, but it is not a social networking application. I registered with Aki-Aki but then found that it is only available on Apple Iphone – which I don’t use. I attempted registration with MobyLuck but I’m still waiting for their confirmation SMS… Both MobyLuck and Aki-Aki do not seem very insistent on increasing their user population.

Nevertheless I quite like the idea of MobyLuck and Aki-Aki and I wonder why they have not managed to produce any significant buzz – don’t people want local social networking ?

With indoor navigation looking like the next big thing already rising well above the horizon, I’m pretty sure that there will be a renewed interest in using Blueetooth for social networking – but why did it take so long ?

This morning a French banker provided me with an explanation for the efforts of the banks at selling MVNO contracts to their customers. This explanation does not dismiss the influence of the contemporary urge to use any customer-facing operation as an excuse for selling services entirely unrelated to the core product – but it makes a little more sense.

Banks see the mobile payment wave rising high on their horizon, and they want to be part of the surfing – in a bank-centric model of course. As near field communications are coming soon to a handset near you, getting ready is a rather good idea.

To carve their rôle into the mobile payments ecosystem, banks believe that building a customer base is a good way to make sure that they will have a critical mass of users to deploy their products to when the time comes for that. In the context of a maturing market with decreasing churn rates, this makes sense – especially as the banking and insurance industry enjoys much lower churn rates than mobiles operators, and the banker’s image could have a halo effect on the mobile products they distribute.

But on the other hand, considering the leonine conditions that French mobile license holders grant to the MVNO, this is a fragile position on which to take leverage.

By the way, if you feel like working for a hot mobile payments company, take a look at the openings at Zong and say hello to Stéphane from me !

Twenty two days ago, my periodically running script ceased to produce any check-ins on Brightkite. A quick look at the output showed that the format of the returned place object had changed. Had I used proper XML parsing, that would not have been a problem – but I’m using homely grep, sed and awk… Not robust code in any way, especially when dealing with XML. At least you get a nice illustration of why defensive programming with proper tools is good for you.

So here is a new update of latitude2brightkite.sh – a script that checks-in your Google Latitude position to Brightkite using the Brightkite API and the Google Public Location Badge. Description of the whole contraption may be found in the initial announcement.

The changes are :

% diff latitude2brightkite_old.sh latitude2brightkite.sh
69,70c69,70
< id=`wget -qO- "http://brightkite.com/places/search.xml?q=$latitude%2C$longitude" | grep "<id>" | sed s/\ \ \<id\>// | sed s/\<\\\/id\>//`
< place=`wget -qO- "http://brightkite.com/places/search.xml?q=$latitude%2C$longitude" | grep "<name>" | sed s/\ \ \<name\>// | sed s/\<\\\/name\>//`
---
> id=`wget -qO- "http://brightkite.com/places/search.xml?q=$latitude%2C$longitude" | grep "<id>" | sed s/\ \ \<id\>// | sed s/\<\\\/id\>// | tail -n 1`
> place=`wget -qO- "http://brightkite.com/places/search.xml?q=$latitude%2C$longitude" | grep "<name>" | sed s/\ \ \<name\>// | sed s/\<\\\/name\>// | md5sum | awk '{print $1}'`

I know I should use a revision control system… Posting this diff that does not even fit this blog is yet another reminder that a revision control system is not just for “significant” projects – anything should use one and considering how lightweight Git is in comparison to Subversion, there really is no excuse anymore.

Back to the point… To get the place identifier, I now only take the last line of the field – which is all we need. I mdsum the place name – I only need to compare it to the place name at the time of the former invocation, so a mdsum does the job and keeps me from having to deal with accented characters and newlines… Did I mention how hackish this is ?

Anyway… It works for me™ – get the code !

After days of being pestered by Corseman, here is the list of the Android applications I use after a few months of optimizing my selection. All of them are free, but many are free as in “free beer” rather than free as in “freedom”.

Communications :

• K-9 – Excellent IMAP client. A huge improvement over the barely usable stock Android IMAP client. It makes mobile mail a very tolerable experience. With such a name suggesting Mutt legacy, it could only be a good piece of software !
• May 2011 update… Xabber is the only XMPP Jabber client worth using.
• DaraIRC – a decent IRC client, except that I can’t get it to remain connected reliably while in the background.
• PingDroid – Quick and simple client for posting to the Ping.fm multiple posting application which I have now ceased to use in favor of Pixelpipe.
• Pixelpipe – Like Ping.fm, but better.
• Sipdroid – SIP client. For now I only use it for testing, but it seems to be the only serious game in town.
• ConnectBot – a very good SSH client that even does tunneling.
• AziLink – an application that allows USB tethering for Android-based phones, without requiring root access. Here is a nice Azilink Android tethering Debian/Ubuntu startup script – don’t forget to circumvent lame HTTP user agent blocks.

Utilities :

• Linda file manager – does the job of shuffling files around. The Android’s user-facing design is far from being file-centric, but having file management capability comes handy sometimes. You may also like Astro instead.
• Phonalyzr – analyzes the call log and SMS log to display graphs and usage summaries that provide useful insight into your consumption. I like it a lot.
• Barcode Scanner – In 1D or 2D, it does the job. Works for reading QR coded business cards too.
• Taskiller – Kill those rogue background tasks that make the whole system sluggish. Why doesn’t the Android base system feature a task manager is beyond me. Having an ad-supported and unfree program for such a basic utility irks me.
• StopWatch – Very usable chronometer and countdown.

Geography :

• Maverick – GPS off-road navigation for Android devices. Uses OpenStreetMap, shows heading and provides offline map tiles storage.
• Places Directory – Finds nearby point of interests from an offline database. Useful when in unknown neighborhoods. The database could be better, but it does the job for finding the nearest ATM or a fuel station.
• Google Sky Map – Romantically show the stars to your girlfriend while discreetly checking this awesome augmented reality planetarium to compensate for your utter lack of astronomical knowledge.
• MapDroyd – On top of displaying the sometimes excellent OpenStreetMap data, this application does it offline – which Google Maps will never do. My choice for travel where the Internet does not yet reach.
• Vespucci OSM Editor – OpenStreetMap editor capable of download and upload. But on such a cramped device, I wouldn’t use it for anything but the simplest edits.
• Here I am – Simple application that sends SMS or mail with your coordinates and a link to Google Map.
• Velibike – Paris Velib automated bike rental stations on Google Maps, with bikes and slots availability.

Writing :

After experimenting with loads of notepad applications, I gave up and went with GDocs which synchronizes with Google Docs. Writing on Android is a pain

I love writing and the clumsiness of working with text on this platform is a significant part of what I dislike about it. The other part has something to do with Android’s insularity in the free software world.

The Worpdress client is nice though.

Commerce :

• Pocket Auctions for eBay – Good for sniping on the go.

Personal information management :

• Astrid tasks/todo lists manager – Well balanced user interface, tags and synchronizes with Remember The Milk. Too bad that as with anything on Android compared to Palm OS the graphical user interface is horribly slow.
• Facebook Sync – The lazy way to have a picture for most of your contacts. I like to see the caller’s face. Now superseded by SyncMyPix.
• Gravatar Importer – Same as Facebook Sync, but uses your contact’s email addresses to search for a Gravatar and set it as the contact’s picture.

Sound :

• Frequency Generator – Combine signals of various shapes and frequencies. Nice for the sound curious.
• Ringdroid – Easily cut and use any MP3 file as a ringtone. If you want to use a song as a ringtone, you absolutely need this.
• Robotic Guitarist – Great accompaniment for your lyrical improvisations.
• Sonorox and Loops – Easy tune composition.
• gStrings – A chromatic tuner.

I have yet to find a decent RSS reader that doesn’t choke on what I want to feed it.

For social networking, I use mobile optimized sites – or even the full one, and I have found that no dedicated applications are needed.

You may have noticed that for some applications in that list, I did not provide a link. That is because it is often very difficult to find the developer’s site, drowned in a sea of spammy application review sites.

Next step for me : find Maemo equivalents for all this, as I’ve got my mind set on leaving the disappointing Android environment and migrate to the Maemo platform that makes much more sense to me.

Valued Lessons wrote :

A lot has been written lately about Google Maps Navigation. Google is basically giving away an incredible mapping application with good mapping data for free. Why would they do such a thing? Most of the guesses I’ve seen basically say “they like to give stuff away for free to push more advertisements”. That’s close, but everyone seems to have missed a huge detail, perhaps the most important detail of all.

Google is an advertisement company, particularly skilled at targeted advertisements. Almost all of their revenue comes from being able to show you ads that you want to see when you want to see them. What does this have to do with maps and navigation? Well, this is going to seem really obvious once you read it, but no one seems to have mentioned it yet, so here it goes:

Google will know everywhere you drive, and when.

Valued Lessons goes on to detail ways Google could use that data to refine the targeted advertisement that represents the lion’s share of Google’s revenue. But there is another reason for pushing Google Navigation…

Now that they have found the way to gather start harvesting the data at a really massive scale they are able to go head to head with all the navigation software editors that have provide traffic information. Here is a nice business model :

• Get the free version of Google Navigation deployed to as many terminals as possible.
• Harvest traffic data.
• Sell traffic data as a premium service. Or just give it away and kill everyone else…

Mobile network operators I know are going to hate this. They make partnerships with the likes of TomTom, only to be entirely bypassed by Google ! I love it.

Let’s take a look at what TomTom wanted to do :

TomTom will use two main sources of information, occasionally complemented by others.

First, travel times deduced from the movement patterns of mobile phones. TomTom has made an agreement with Vodafone NL, allowing us to use (anonymously) the country’s 4 million Vodafone customers as a potential source of information and developed the technology to transform this monitoring information from the mobile network into reliable travel time information.

Secondly, historical FCD (Floating Car Data) from our own customers. Every TomTom navigation system is equipped with a GPS sensor, from which one can determine the exact location of a car.

Yes, Google can do all that too.

The process of obtaining data TomTom has developed results in highly detailed traffic information. In the Netherlands, for example, it means up-to-date travel times per road segment for approximately 20,000 km of road (see figure) and historical information per road segment for all major roads in the country, approximately 120,000 km.

TomTom has developed the technology in-house to calculate travel times across the entire road network, by processing the monitoring data from the mobile telephone network through TomTom’s Mobility Framework software.

And that’s information from before 2007… Imagine what can be done today !

Letting Google know where you go and letting Google mine that data is the reason for Google Latitude too… Latitude does not have the same mainstream appeal as a turn by turn navigation application, but with so many Google Maps customers now using it inside their car we are now talking Google scale !

Last week-end I ventured outside of the big city, in the land where cells are measured in kilometres and where signal is not taken for granted. So what surprised me was not to have to deal with only the faint echo of the network’s signal. Cell of origin location on the other hand was quite a surprising feature in that environment : sometimes it works with an error consistent with the cellular environment, but often the next moment it estimated my position to be way further from where I actually was – 34 kilometres west in this case.

The explanation is obvious : my terminal chose to attach to the cell it received best. Being physically located on a coastal escarpment, it had line of sight to a cell on the opposite side of the bay – 34 kilometres away.

But being on the edge of a very well covered area, it was regularly handed over to a nearby cell. In spite of the handover damping algorithms, this resulted in a continuous flip-flop nicely illustrated by this extract of my Brightkite account’s checkin log :

Isn’t that ugly ? Of course I won’t comment on this network’s radio planning and cell neighbourhood settings – I promised my employer I will not mention him anymore. But there has to be a better way and my device can definitely do something about it : it is already equipped with the necessary hardware.

Instant matter displacement being highly unlikely for the time being, we can posit that sudden movement of kilometre-scale distances will result in the corresponding acceleration. And the HTC Magic sports a three axis accelerometer. At that point, inertial navigation immediately springs to mind. Others have thought about it before, and it could be very useful right now for indoor navigation. But limitations seem to put that goal slightly out of reach for now.

But for our purposes the hardware at hand would be entirely sufficient : we just need rough dead reckoning to check that the cell ID change is congruent with recent acceleration. With low quality of the acceleration measurement, using it as a positioning source is out of question, but it would be suitable for dampening the flip flopping as the terminal suffers the vagaries of handover to distant cells.

So who will be the first to refine cell of origin positioning using inertial navigation as a sanity check ?

Five months have elapsed since that first week-end when my encounter with Android was a severe case of culture shock. With significant daily experience of the device, I can now form a more mature judgement of its capabilities and its potential – of course from my own highly subjective point of view.

I still hate having to use Google Calendar and Google Contacts for synchronization.  I hope that SyncML synchronization will appear in the future, make Android a better desktop citizen and provide more choice of end points. Meanwhile I use Google. With that out of the way, let’s move on to my impressions of Android itself.

I am grateful for features such as a decent web browser on a mobile device, for a working albeit half baked packaging and distribution system, and for Google Maps which I consider both a superlative application in its own right and the current killer albeit proprietary infrastructure for location enabled applications. But the rigidly simple interface that forces behaviours upon its user feels like a straitjacket : the overbearing feeling when using Android is that its designers have decided that simplicity is to be preserved at all costs regardless of what the user prefers.

Why can’t I select a smaller font for my list items ? Would a parameter somewhere in a customization menu add too much complication ? Why won’t you show me the raw configuration data ? Is it absolutely necessary to arbitrarily limit the number of virtual desktops to three ? From the point of a user who is just getting acquainted with such a powerful platform, those are puzzling questions.

I still don’t like the Android ‘s logic, and moreover I still don’t quite understand it. Of course I manage to use that system, but after five month of daily use it still does not feel natural. Maybe it is just a skin-deep issue or maybe I am just not the target audience – but some features are definitely backwards – package management for example. For starters, the “My Downloads” list is not ordered alphabetically nor in any apparently meaningful order. Then for each upgradeable package, one must first browse to the package, then manually trigger the upgrade package, then acknowledge system privileges the upgraded package and finally clear the download notification and the update notification. Is this a joke ? This almost matches the tediousness of upgrading Windows software – an impressive feat considering that the foundations of Android package management seem serious enough. Where is my APT ?

Like any new user on a prosperous enough system, I am lost in choices – but that is an embarrassment of riches. Nevertheless, I wonder why basics such as a task manager are not installed by default. In classic Unix spirit, even the most basic system utilities are independent applications. But what is bearable and even satisfying on a system with a decent shell and package management with dependencies becomes torture when installing a package is so clumsy and upgrading it so tedious.

Tediousness in package management in particular and user interaction in general makes taming the beast an experience in frustration. When installing a bunch of competing applications and testing them takes time and effort. Experimenting is not the pleasure it normally is on a Linux system. The lack of decent text entry compounds the feeling. Clumsy text selection makes cut and paste a significant effort – something Palm did make quick, easy and painless more than ten years ago. Not implementing pointer-driven selection – what were the developers thinking ?

PIM integration has not progressed much. For a given contact, there is no way to look at a communications log that spans mail, SMS and telephony: each of them is its own separate universe. There is no way to have a list of meetings with a given contact or at given location.

But there basic functionality has been omitted too. For example when adding a phone number to an existing contact, search is disabled – you have to scroll all the way to the contact. There is no way to search the SMS archive and SMS to multiple recipients is an exercise left to applications.

Palm OS may have been unstable, incapable of contemporary operating system features, offering only basic functionality and generally way past its shelf date. But in the mind of users, it remains the benchmark against which all PIM systems are judged. And to this day I still don’t see anything beating Palm OS on its home turf of  PIM core features and basic usability.

Palm OS was a poster child for responsiveness, but on the Android everything takes time – even after I have identified and killed the various errant applications that make it even slower. Actually, the system is very fast and capable of feats such as full-motion video that were far beyond the reach of the Palm OS. But the interaction is spoilt by gratuitous use of animations for everything. Animations are useful for graphically hinting the novice user about what is going on – but then hey are only a drag. But please let me disable animations as I do on every desktop I use !

The choice of a virtual keyboard is my own mistake and I am now aware that I need a physical keyboard. After five months, I can now use the virtual keyboard with enough speed and precision for comfortable entry of a couple of sentences. But beyond that it is tiring and feels too clumsy for any meaningful work. This is a major problem for me – text entry is my daily bread and butter. I long for the Treo‘s keyboard or even the one on the Nokia E71 – they offered a great compromise of typing speed and compacity. And no multitouch on the soft keyboard means no keyboard shortcuts which renders many console applications unusable – sorry Emacs users.

The applications offering is still young and I cannot blame it for needing time to expand and mature. I also still need to familiarize myself with Android culture an develop the right habits to find my way instinctively and be more productive. After five months, we are getting there – one handed navigation has been done right. But I still believe that a large part of the user interface conventions used on the Android does not match the expectations for general computing.

It seems like everything has been meticulously designed to bury under a thick layer of Dalvik and Google plaster anything that could remind anyone of Unix. It is very frustrating to know that there is a Linux kernel under all that, and yet to suffer wading knee-deep in the marshes of toyland. The more I use Android an study it, the more I feel that Linux is a mere hardware abstraction layer and the POSIX world a distant memory. This is not the droid I’m looking for.

As the Geohack template used by Wikipedia for geographical locations attests (see Paris for example) there are many map publishing services on the Web. But almost all of them rely on an oligopoly of geographical data suppliers among whom AND, Navteq and Teleatlas dominate and absorb a large proportion of the profit in the geographical information value chain :

“If you purchase a TomTom, approximately 20-30% of that cost goes to Tele Atlas who licenses the maps that TomTom and many other hardware manufacturers use. Part of that charge is because Tele Atlas itself, and the company’s main rival Navteq, have to buy the data from national mapping agencies in the first place, like the Ordance Survey, and then stitch all the information together. Hence the consumer having to pay on a number of levels”.

And yet, geographical data is a fundamental pillar of our information infrastructure. A few years ago the realm of specialized geographic information systems, geography is nowadays a pervasive dimension of about every sort of service. When something becomes an essential feature of our lives, nothing short of freedom is acceptable. What happens when that freedom requires collecting humongous amounts of data and when oligopolistic actors strive to keep control and profits to themselves ? Free software collaboration and  distributed data collection of course !

Andrew Ross gives a nice summary of why free geographical data is the way of the future :

“The tremendous cost of producing the maps necessitates that these firms have very restrictive licenses to protect their business models selling the data. As a result, there are many things you can’t do with the data.

[..] The reason why OpenStreetMap will win in the end and likely obviate the need for commercial map data is that the costs and risks associated with mapping are shared. Conversely, for Navteq and TeleAtlas, the costs born by these companies are passed on to their customers. Once their customers discover OpenStreetMap data is in some cases superior, or more importantly – they can contribute to it and the license allows them to use the data for nearly any purpose – map data then becomes commodity”.

The proprietary players are aware of that trend, and they try to profit from the users who wish to correct the many errors contained in the data they publish. But why would anyone contribute something, only to see it monopolized by the editor who won’t let you do what you want with it ? If I make the effort of contributing carefully collected data, I want it to benefit as many people as possible – not just someone who will keep it for his own profit.

Access to satellite imagery will remain an insurmountable barrier in the long term, but soon the map layers will be ours to play with – and that is enough to open the whole world of mapping. Like a downhill snowball, the OpenStreetMap data set growth is accelerating fast and attracting a thriving community that now includes professional and institutional users and contributors. Over its first five years, the Wikipedia-like online map project has delivered great results – and developed even greater ambitions.

I have started to contribute to OpenStreetmap – I feel great satisfaction at mapping the world for fun and for our common good. Owning the map feels good ! You can do it too – it is easy, especially if you are the sort of person who often logs tracks with a GPS receiver. OpenStreetMap’s infrastructure is quite impressive – everything you need is already out there waiting for your contribution, including very nice editors – and there is one for Android too.

If you just want to add your grain of sand to the heap, reporting bugs and naming the places in your favourite neighbourhood are great ways to help build maps that benefit all of us.  Contributing to the map is like giving directions to strangers lost in your neighbourhood – except that you are giving directions to many strangers at once.

If you are not yet convinced, take a look a the map – isn’t it beautiful ? And it is only one of the many ways to render OpenStreetMap data. Wanna make a cycling map with it ? Yes we can ! That is the whole point of the project – we can do whatever we want with the data, free in every way.

And anyone can decide he wants his neighbourhood part of the worldmap, even if no self-respecting for-profit enterprise will ever consider loosing money on such endeavour :

“OpenStreetMap has better coverage in some niche spaces than other mapping tools, making it very attractive resource for international development organizations. Want proof ? [..] we looked at capital cities in several countries that have been in the news lately for ongoing humanitarian situations – Zimbabwe, Somalia, and the Democratic Republic of the Congo. For two our of the three, Mogadishu and Kinshasa, there is simply no contest – OpenStreetMap is way ahead of the others in both coverage and in the level of detail. OpenStreetMap and Google Maps are comparable in Harare. The data available through Microsoft’s Virtual Earth lagged way behind in all three”.

Among other places, I was amazed at the level of detail provided to the map of Ouagadougou. Aren’t these exciting times for cartography ?

I acquired an HTC “G2” Magic less than two days ago. It runs the Android operating system. The feeling of being confronted with something very alien pushed me to record my first impressions with it to give an account of how a foreigner perceives the Android world with his naïve eyes, in his own words. For other systems where I’m a power user, I find the experience of newcomers interesting when they candidly point out problems we ignore because we have simply grown used to them.

This entry relates my feelings from friday night to sunday night. It may seem ridiculous in the future, but it is an instantaneous snapshot – for what it is worth.

First contact with the Android is a severe case of culture shock. More than ten years of Palm OS have shaped my expectations, and the disappointing past year with S60 on the Nokia E71 has not changed them much. But plunging into Android is unlike anything I have used so far – many of the UI conventions feel utterly strange. The home screen has a familiar status bar – but beyond that, Android is a class of its own. For example, instead of grabbing the scroll bar and sliding it, one has to slide the list itself – not illogical, but the contrary of any familiar widget kit I have come accross anywhere else. And many other things are just as alien.

After poking around a little, my first reaction is disorientation from the lack of keyboard. I could write tolerably fast with Palm’s Graffiti, but I was in love with the Treo-style keyboards – on the the Treos as well as on the E71 they let me write considerable volumes fluidly and without excessive strain. But my first attempts at text entry on the G2 are stumbling hit and miss torture with each word containing at least two typing errors. And why does the automatic correction insist on changing my “jm@liotier.org” address to “kn@liotier.org” every single time I enter it in a web form ? Text entry on the G2 is tedious enough without this sort of annoyance…

To be fair, I knew I had to expect text entry woes – I had anticipated that risk when choosing the G2 over the G1. Learning a new tool takes times, especially when low level reflexes are involved, so I have budgeted a few weeks for climbing up the learning curve. Then I’ll decide if I like onscreen keyboards or not. But whatever the learning, it seems that text entry on a virtual keyboard requires to keep one’s eyes on the keyboard – whereas with a physical keyboard, after a while muscle memory sets in and you can forget the keyboard to concentrate on what you are writing. So I’m not optimistic so far – but I’ll keep my mind open. Meanwhile, tactile feedback screens are on the way – I’ll keep an eye on them.

I miss the four-ways arrows button, but the obscene pointing device works rather well although like the rest it will take time to get used to. A “page down” button would be even better than having to swipe the whole screen every time I want to scroll down one step – one page at the time would be more precise than scrolling a random number of lines according to how much inertia the widget takes from the swipe. Screen swipe and inertia are sexy gimmicks, but I don’t understand how heavy users tolerate them for more than five minutes. I’m the sort of user who disables smooth scrolling and any on-screen animation that introduces the slightest lag in my interaction with the system – and I know I’m not the only one who wants responsiveness above everything else.

A combination of importing in Evolution a CSV file generated with Outlook, synchronization from the Nokia E71 to Google and copying native Evolution contacts to Google did not manage to capture at once all of the information I wanted transferred – so I had to munge some of the data and re-enter quite a few of the notes and adresses manually… Hopefully that is the last time I do that. Those contacts had often been through various synchronizations between Palm OS devices, Outlook and Evolution – but getting them to Google seemed lossier than usual. I have read about many other contorted data migration paths, and this one looked straightforward enough – but if I have to do it again I’ll spend time setting up better automation. I’ll concede that it does not have much to do with Android – anywhere you look the synchronization ecosystem seems quite wet behind the ears.

I won’t complain too much about how tightly tied the system is with Google’s applications – after all that is a major feature of Google’s Android business model. With the defaults applications, Android is a seamless extension of the Google universe. Synchronization of calendar and contacts is excellent – although it only happens eventually and you have no way to know when or to trigger it (this is the first time I ever see this implemented with no control by the user). In addition, I am very uncomfortable with the idea of using a third party as my synchronization hub and I’ll look for another way.

But every functionnality seems available through an API – so with the user in control and free to act there is no reason to complain. I’ll ignore the Gmail and Google Talk clients, and replace them with a decent XMPP client and an IMAP client better than the default one – and maybe I’ll even find a decent contacts manager. Meanwhile the native Google Maps client is such a pleasure to use that I could forget everything else (though I wonder why the relief layer has been omitted – I find it very useful for planning human powered movement).

The scarcity of exposed configuration options, output logs and exposed information in general leaves me wanting. For example the Jabiru Jabber client tells me “connexion error” but won’t explain anything, resulting in frustration. Of course, Jabiru is not part of the basic system, but this rarefied atmosphere seems to be the norm in the Android world. And why is there no option to sort the contacts by “family name, first name” instead of the default “first name, family name” ? Would that clutter the interface too much ? Even the simplicity-worshipping Palm OS gave that choice…

I guess that a compromise has been struck in favor of simplicity by default over configurability, and that developper tools are available to provide advanced access to deeply buried parameters – but for example not being able to set the language to english with the french keyboard upsets me a lot. I’m used the english as a device language, and I’m used to the french “AZERTY” keyboard – reading french or typing on a QWERTY keyboard with no accents feels awkward. On any other system I know, keyboard and language are two separate options – but not on Android. I hope I find an application that provides finer grained options. I was also frutrated not to find any configuration option that would solve my above-mentioned problem with the scrolling style.

The Android Market feels sluggish. I have been spoilt by APT caching all packages descriptions locally – and now I have to suffer Android Market loading package descriptions and icons slower than I scroll accross the list that only shows six items per page with no way of getting the device to display smaller characters in order to cram more lines per page. I can understand that the icons must be stored online for storage space’s sake, and maybe the user comments in order to keep them current – but why not load the whole package list at once ? And why does every list on this device use a standard widget that seems sized large enough for legibility by half blind users ? Where is the configuration option ?

Many of my gripes are probably related with the default applications, and after exploring the Android Market for a while I’m sure I’ll feel better. I’m commenting an operating system in its default form, and this is obviously not how I’m going to use it – In a few weeks, after the normal process of appropriation, my Android will hopefully not look and feel like its current state at all.

So see you in a couple of months for a look back at these first impressions – we’ll see which were real problems and which were merely artefacts of the clash of cultures ! For now I have the eery feeling of having stumbled in a sort of Apple-esque Disneyland with my hands tied…

I just released a new update of latitude2brightkite.sh – a script that checks-in your Google Latitude position to Brightkite using the Brightkite REST API and the Google Public Location Badge.

The changes are :

20090607 – 0.3 – The working directory is now a parameter
20090612 – 0.4 – Only post updates if the _name_ of the location changes, not if only the _internal BK id_ of the place does (contribution by Yves Le Jan <inliner@grabeuh.com>).
20090615 – 0.5 – Perl 5.8.8 compatibility of the JSON coordinate parsing (contribution by Jay Rishel <jay@rishel.org>).

Yves’ idea smooths location sampling noise and makes check-ins much more meaningful.

Thanks to Yves and Jay for their contributions ! Maybe it is time for revision control…

Tired of waiting for Google to release a proper Latitude API, I went ahead and scribbled latitude2brightkite.sh – a script that checks-in your Google Latitude position to Brightkite using the Brightkite REST API and the Google Public Location Badge. See my seminal post from yesterday for more information about how I cobbled it together.

Since yesterday I cleaned it up a little, but most of all, as promised, I made it more intelligent by having it compare the current position with the last one, in order to check-in with Brightkite only if the Google Latitude position has changed. Not checking-in at each invocation will certainly reduce the number of check-ins by 99% – and I’m sure that Brightkite will be thankful for the lesser load on their HTTP servers…

So grab the code for latitude2brightkite.sh, put it in your crontab and have more fun with Brightkite and Google Latitude !

There is quite a bit of interest for this script – it seems that I have filled a widely felt need.

Tired of waiting for Google to release a proper Latitude API, I went ahead and scribbled latitude2brightkite.sh – a script that checks-in your Google Latitude position to Brightkite using the Brightkite REST API and the Google Public Location Badge.

This script is an ugly mongrel hack, but that is what you get when an aged script kiddie writes something in a hurry. The right way to do it would be to parse Latitude’s JSON output cleanly using the Perl library. But that dirty prototype took me all of ten minutes to set up while unwinding between meetings, and it now works fine in my crontab.

Apart from Bash, the requirements to run this script are the Perl JSON library (available in Debian as libjson-perl) and Curl.

The main limitation of this script is that your Google Public Location Badge has to be enabled and it has to show the best available location. This means that for this script to work, your location has to be public. The privacy conscious among my readers will surely love it !

This script proves that automatic Google Latitude position check-in in Brightkite can be done, it works for me, and the official Google Latitude API will hopefully soon make it obsolete !

Meanwhile, grab the code for latitude2brightkite.sh, put it in your crontab and have more fun with Brightkite and Google Latitude… To me, this is what both services were missing to become truly usable.

Of course, doing it with “XEP-0080 – User Location” via publish-subscribe (“XEP-0080 – PubSub” would make much more sense than polling an HTTP server all the time, but we are not there yet. Meanwhile this script could be made more intelligent by only checking in with Brightkite if the Google Latitude position has changed. I’ll think about it for the next version…