In the digital world, the folder metaphor has perpetuated the single dimensional limitations of the physical world : each message is present in one and only one folder. The problem of adding more dimensions to the classification has been solved ages ago – whether you want to go hardcore with a full thesaurus or just use your little folksonomy the required technical foundation is the same : tags, labels – or whatever you want to call your multiple index keys.

Noticing how Google has been successful with email tagging, I started exploring free implementation four years ago, aiming to complement my folder hierarchy with tags. But inertia affects all, and I actually never went beyond happy experimentation. As Zoli Erdos puts it in his eloquent description of Google’s 2009 breakthrough in ending the tags vs. folders war :

Those who “just can’t live without folders”, mostly legacy users of Yahoo Mail, Hotmail and mostly Outlook. They are used to folders and won’t learn new concepts, don’t want to change, but are happy spending their life “organizing stuff” and even feel productive doing so.

Ouch – that hits a little too close to home. And even if I had gone forward with tags, that would have been pussyfooting : as Google illustrates, the distinction between tags and folders is a technical one – from a user point of view it should be abstracted. Of course the abstraction is somewhat leaky if you consider what folders mean for local storage, but in the clouds you can get away with that.

For cranky folder-addicted users like myself, born to the Internet with early versions of Eudora, later sucking at the  Microsoft tit and nowadays a major fan Mozilla Thunderbird, there is definitely a user interface habit involved – and one that is hard to break after all those years. It is not about the graphics – I use Mutt regularly; it is about the warm fuzzy feeling of believing that the folder is a direct and unabstracted representation of something tangible on a file system.

Software objects are all abstractions anyway but, with time, the familiar abstraction becomes something that feels tangible. This is why, while I acknowledge the need to get the tagging goodness, I still have desire toward the good old folder look and feel with features such as drag-n-drop and hierarchies. Google seems to know that audience : all those features are now part of Gmail. Now tell the difference between folders and labels !

To make a smooth transition, I want Mozilla Thunderbird to display tags as folders. It looks like it is possible using Gmail with Claws through IMAP. I have yet to learn if I can do that on my own systems using Courier or Dovecot.

Have you tried one more time to convince you parents to switch to web feeds to get updates from the family ? Do you cringe when you see your colleague clumsily wade through a collections of sites main pages instead of having them aggregated in a single feed ? Or did your technophobe girlfriend miss the latest photo album you posted ? With a wide variety of source acknowledging that web feeds ans web feeds readers being perceived as too technical, many of us have scaled back this particular evangelization effort to focus it on users ripe for transitionning from basic to advanced  tools.

Breaking through that resistance outright is beyond our power, but we can get around it. Electronic mail is a mature tool with well understood use cases with which even the least competent users feels comfortable thanks to how easily it maps with the deeply assimilated physical mail model. This is why Louis Gray has started mailing Google Reader items to promote the use of that web feed reader. But we can do better than that by building a fully automated bridge from web feed to email.

Our hope for plugging the late adopters into the information feeds is named rss2email. As its name suggests, Aaron Swartz’s GPL-licensed rss2email utility converts RSS subscriptions into email messages and sends them to whatever address you specify. Despite the name, it handles Atom feeds as well, so you should be able to use it with just about any feed you like. And of course rss2email is available from Debian.

The nice introduction to rss2email by Joe ‘Zonker’ Brockmeier is all the documentation you need – and rss2email is so simple that you probably don’t even need that. I now have some of my favorite late adopters each plugged into his custom subset of my regular information distribution feeds. The relevant news stories get mailed to them without me having to even think about it. And the best part is that they now read them !

Some people notice I am quite dogmatic about open networks. And they are right : to me, open is everything and the rest is details. But even my zeal has its limits : I don’t gratuitously shove tools in the face of people who can’t use them in practical conditions. I have been advocating jabber among my technically minded friends since 2001 and running my own server since 2003, but it took Google joining the XMPP network in 2006 to actually make it a viable option for pushing open instant messaging to the masses of people I don’t want to support myself. Before that I could understand the necessity for joining proprietary networks and run multiprotocol clients to reach people I could not decently drag to Jabber or IRC. But now I can tell them that getting presence information and instant messenging from me is just a Google account away – and since it is a mainstream service offered by an established and well known service provider they can hardly anymore label me a techno-excentric for using it. So – no I will not join your proprietary instant messaging network.

Today we have the same situation with social networking. And while the technological prerequisites for open microblogging have been almost there for a while, they have not yet cristalized into something that can be fed to the masses. That day will come – and we are all pushing toward it. Until then, I have a Facebook profile. But soon I know that I’ll be able to tell the world that my social networking tool is my blog, or whatever other tools I fancy moving to and from thanks to data portability efforts. And it’ll be easy for others to do the same because interoperable services will blossom at the hands of mass-market providers – maybe even Facebook if they ever reach enlightenment. And when that is about to be ready for massive adoption, you know where I’ll be – and you know where I’ll not be anymore !

To me there is an element of religion in those choices. But the techno-apathetic average user can make the same choices out of pure self-interest. If there are a number of comparable offerings on the market, one of which lets the buyer choose between different suppliers and move between them at will, you can bet that the one-time cost of moving away from the proprietary offering will be more than offset by the future value of the open solution. If we look at the history of technologies, examples of such migrations are plenty. Let’s just take e-mail for example : what is the current weight of closed mail systems ? They still exist, but they are insignificant niches and many of them use e-mail for notification…

I work as a project manager for a very large ISP that Dilbert readers would sometimes find strangely familiar. But nothing prepared me for the shock and disbelief I experienced when some of my co-workers in the information systems division asked me why I kept sending them mail written in 10-point Courier New font whereas I was sending them plain text.

Following on their remark I soon found that many people found likewise that my messages are difficult to read because of that poor choice of font. Apparently, no one realized that plain text is rendered as whatever you want it to render as, including Fette Fraktur or even Zapf Dingbats if you fancy hieroglyphic form.

Sometimes I wonder if I am really working for an ISP. If you think that such company is an oasis of Internet culture, then if you joined one nowadays you would be sadly disappointed.

Anyway here is a tip for them, straight from the horse’s mouth :

1. From the main Microsoft Outlook window, on the “Tools” menu, click “Options” and then click the “Mail Format” tab.
2. Click “Fonts“.
3. Next to the “When composing and reading plain text” box, click “Choose Font“.
4. Select the fonts you want, and then click “OK“.
5. Enjoy my plain text messages in your favorite font and size.

Plain text grants the recipient freedom to render as he sees fit, including in Braille or as audio speech – plain text is that flexible.

I do not have a fetish for the spartan aesthetics of plain text in green monospace font on a black text console. What I appreciate is universal portability : I can read plain text on any device in any situation and process it with any tool including old or underpowered ones – and I actually do. That is the power of plain text.

Of course, as an interpreted markup language, HTML can also be rendered in a variety of ways and I could probably use it, but plain provides even more freedom. HTML being a standard I nevertheless welcome it in my mailbox even if I seldom send any HTML mail. A good maxim to live by when you are writing anything which has to interoperate with other systems is : “Be liberal in what you receive, and conservative in what you send”. It is as old as the Internet but it’s a great way to make things highly compatible and interoperable.

Writing this rant probably dates me as a unixian dinosaur… But it could be worse : just wait until someones gets me started about proper mail quoting !

Spam from Gmail accounts has appeared at least three years ago. Two years ago it caused Gmail servers to hit Spamcop’s database. But only now am I starting to notice it on my systems. My existing antispam arsenal may have protected me so far, but now I receive three or four illegitimate messages a day from that source, which is more than I tolerate.

Those messages have valid domain key signatures which lower their Spamassassin score enough to pass them as ham. If Google does not police its mail service better, then domain key signatures will become more than useless. A signature is only as good as its authority.

Meanwhile I will probably have to weight down the negative scores added through Mail::SpamAssassin::Plugin::DomainKeys.

Yes, we do hate stupid email disclaimers that much : they are a completely meaningless waste of electrons, they waste display real estate and they make proper quoting more awkward. And most of all they are unilateral clauses whose acceptation is most dubious.

Lerouge dug out an article by Russel Coker about striking back at worthless email disclaimers by configuring Postfix’s smtpd_banner in /etc/postfix/main.cf with the appropriate legalese of its own. Lerouge suggested that we do likewise and I jumped on the case and implemented it immediately…

Looking for something really outrageous and fit to point out how ridiculous those things are, I found inspiration at ReasonnablAgreement.com and modified it for my purposes…

So there you go :

13:52 jim@kivu ~% telnet localhost 25
Trying 127.0.0.1…
Connected to localhost.
Escape character is ‘^]’.
220 kivu.grabeuh.com ESMTP (M Sexchange) – READ CAREFULLY. By transmitting email to this server you agree personally and on behalf of your employer or organization, to release the recipients from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (hereafter BOGUS AGREEMENTS) that the recipients have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to the recipients ongoing rights and privileges. You further represent that you have the authority to release the recipients from any BOGUS AGREEMENTS on behalf of your employer or organization.

So in case you wondered, here is proof : the humongous disclaimer from hell does fit into smtpd_banner’s limits !

Most social tools come and go. The ones that stay share a common feature : openness. For example, email is open : chose any technology, chose any provider or be your own provider, chose any client, any platform – any way you do it you are still connected to the whole world.

If you have the slightest understanding of your own interest, then there is no way you should even consider using a closed platform as your primary mean of communication. Why would you willingly chose to put your most critical asset outside of your control ?

Many users will object that they gladly surrender control to closed social networking platforms because plain email does not meet their sophisticated communications needs and they are not willing to invest in developing the skills currently required to participated efficiently in the blogging sphere. That is a tragedy because the social graph is quickly becoming the glue of the connected services.

And even if the functionality was sufficient, we would still have a huge mindshare gap to bridge. XMPP provides nice basic instant messenging and presence management in an open environment, but most users still prefer proprietary centralized networks and happily trade freedom for webcam compatibility.

But similar battles have been fought and won in the past : Compuserve, AOL, The Source, Prodigy and their ilk have all dissolved in the Internet. The forces of openness now have a new crusade to embark upon : we must take the best use cases of the closed social networking world and port them in the open !

Open is everything – the rest is details. That is what drew me to the Internet fifteen years ago.

I upgraded the Sympa mailing list manager to 5.2.3-2 using the Debian package from the “Testing” repository. The database part of the upgrade procedure was a bit fussy so instead of solving its problems I simply backed up the tables, dropped them, ran the upgrade procedure and restored them. That workaround worked fine for making the Debian packaging system happy.

But Sympa itself was definitely not happy. On starting Sympa I got the following logs in /var/log/sympa.log :

Apr 25 17:02:39 kivu sympa[657]: Could not create table admin_table in
database sympa : Table ‘admin_table’ already exists
Apr 25 17:02:39 kivu sympa[657]: Could not create table user_table in
database sympa : Table ‘user_table’ already exists
Apr 25 17:02:39 kivu sympa[657]: Could not create table subscriber_table
in database sympa : Table ‘subscriber_table’ already exists
Apr 25 17:02:39 kivu sympa[657]: Could not create table netidmap_table
in database sympa : Table ‘netidmap_table’ already exists
Apr 25 17:02:39 kivu sympa[657]: Unable to execute SQL query : You have
an error in your SQL syntax; check the manual that corresponds to your
MySQL server version for the right syntax to use near ‘.`admin_table’ at
line 1
Apr 25 17:02:39 kivu sympa[657]: Database sympa defined in sympa.conf
has not the right structure or is unreachable. If you don’t use any
database, comment db_xxx parameters in sympa.conf
Apr 25 17:02:39 kivu sympa[657]: Exiting.
Apr 25 17:02:39 kivu sympa[657]: Sympa not setup to use DBI

With no database access, Sympa was not operational. Double plus ungood !

The very strange thing is that the database is fine : the right tables with the right fields and the right records are all present. It even worked with the preceding version of Sympa. It looked like Sympa itself was unable to recognize that my database setup was correct, subsequently reported those errors and thereafter refused to run with it at all.

With a little rummaging inside the Sympa-users mailing list I quickly found a report of something looking suspiciously like my problem. It is probably a bug and Olivier Berger proposed a patch that looked to me like a workable solution : according to Olivier, a faulty regex was the cause of Sympa‘s failure to recognize it’s own.

After making a backup copy of /usr/lib/sympa/bin/List.pm I promptly applied his patch :

17:04 root@kivu /usr/lib/sympa/bin# diff List.pm.dist List.pm
10750a10751
> $t =~ s/^([^.]+\.)?(.+)$/\2/;

I restarted Sympa and it worked fine ever after. Thank you Olivier !

The only problem is that while Sympa was down, people wondered why the messages did not go through and resent some of their messages. None of those messages were lost – they were just piling up in a queue. So when Sympa restarted many duplicates were sent.

But at least now it’s working. So for now I’m going to use dselect to freeze the Sympa Debian package at its current version so that it is kept back next time I upgrade my system.

Last year when I reviewed the IMAP keyword tagging scene, I said :

Thunderbird supports IMAP keywords but only allows five of them. This is a known bug but it has been open for more than four year so I’m not holding my breath for it. The amount of comments made to that bug show that user expectations about IMAP keywords are slowly growing.

Eager taggers rejoice : your wishes have been heard… Among the feature list of the just released Thunderbird 2 I read :

Thunderbird 2 allows you to tag messages with descriptors such as “To Do” or “Done” or even create your own tags that are specific to your needs. Tags can be combined with saved searches and mail views to make it easier to organize email.

This is exciting because last year I was pleasantly surprised to see both IMAP servers and MDA providing apparently mature support for IMAP keywords, but I concluded that we were just some client support away from being able to use them.

So the the last missing piece of the IMAP mail tagging puzzle may have been delivered and I shall soon play with it to see if it lives up to its potential.

This is certainly a classic bit of regex wizardry but since it took me a few minutes of searching and can be valuable in a variety of contexts, it might be valuable to you too…

grep -o ‘[[:alnum:]+\.\_\-]*@[[:alnum:]+\.\_\-]*’

I needed it for extracting the adresses returning a 550 from my Postfix logs. But then I found that Sympa, my mailing list management system, handles bounces automatically very well using a scoring algorithm that the list administrator can optionally override.

We shall call this process “serendipitous ignorance“…

While we are trying to make sense of regular expressions, those curious about them and wishing for an introduction geared toward audiences other than the beard and sandals systems administration crew may appreciate the examples provided in “Egrep for Linguists“.

And yes, I do indulge in sandals and facial pilosity in the hope of mastering regexes one day…

Blue Frog automated the complaint process for each user as they receive spam. It worked so well that spammers considered it a very serious threat to their livelyhood. Blue Security CEO Eran Reshef quoted a spammer as writing “Blue found the right solution to stop spam, and I can’t let this continue“. Under heavy attacks from the spammers, Blue Security called quits in May 2006.

Following the demise of Blue Frog, the Okopipi project aimed to become a distributed replacement of Blue Security’s anti-spam software, based on a P2P network. For now there is only an Okopipi FAQ and a seminal functional overview of the Okopipi system. The official Okopipi forums are quite dead and it is not the only bad sign for the Okopipi project. But Journeyman recently loudly stated that the Okopipi project is still moving forward. So maybe you can still either keep hoping or offer your help…

Whereas Okopipi has a slight rank of Second System Effect, Knujon looks like a bold attempt to take spam control from the technical to the social dimension. Filtering works well but it is only treating the symptom of the spam problem. Knujon vows to bring businesses, governments, law enforcement, security professionals and other users together in collaboration. Filtering is a selfish associal device whereas systemic salvation lies in a multidimensional cooperative approach. As Knujon puts it :

“Organizations and Personal Email users are blocking/filtering millions of junk emails every day. This is to the advantage of spammers as it allows them to target the most vulnerable users who do not have filtering software or technical savvy. Besides helping the junk mailers and identity thieves find their target audience, we are restricting our own use of email.
[..]
Blocking and filtering are not proper solutions for law enforcement or computer security professionals since it they only serve to hide the problem and force the activity to an underground network. Ordinary users must sift through hundreds of quarantined junk emails everyday to search for legitimate messages”.

So help save those clueless “ordinary users” who do not enjoy all the spam filtering goodness ! You can do your bit by simply forwarding your spam mail to yourjunk@knujon.com. There is also a Knujon Thunderbird plugin, or you can automate that process using my script that feeds the content of any maildir to various spam reporting services. Recycle spam, save the planet !

I thought I had spam pretty much under control, with only about one getting though every few days. And then came image spam. No suspicious words, just a load of bayes poison and an image to carry the actual message. Half of my antispam arsenal was suddenly rendered useless. I was back to suffering one or two spam messages every day.

“The level of image spam has increased dramatically this year,” says Carole Theriault, a senior consultant at Sophos cited by New Scientist. According to New Scientist, Sophos estimates that, at the beginning of the year, image spam accounted for only 18% of unsolicited mail but that this has since risen to 40%.

Less impressive but much more useful than statistical FUD from a biased source, were the few articles about using optical character recognition to fight image spam, from Debian Administration and Linux Weekly News among others.

But with a production server on my hands and precious little time to maintain it I wished to stick to packages distributed by Debian so I waited a little longer for packaging while my users suffered. To my great relief, Christmas came a few days earlier this year – FuzzyOcr hit Debian unstable yesterday ! Mail server administrators rejoice ! Somebody must have been even more pissed off than me about image spam and decided to make the Debian packaging work…

So here is the FuzzyOcr Debian package blurb, straight from the horse’s mouth :

This Spamassassin plugin checks for specific keywords in image/gif, image/jpeg or image/png attachments, using gocr (an optical character recognition program). This plugin can be used to detect spam that puts all the real spam content in an attached image, while the mail itself is only random text and random html, without any URL’s or identifiable information. Additionally to the normal OcrPlugin, it can do approximate matches on words, so errors in recognition or attempts to obfuscate the text inside the image will not cause the detection to fail.

But a debug log is worth a thousand words, so here is a choice output :

[2006-12-19 12:03:39] Debug mode: Starting FuzzyOcr...
[2006-12-19 12:03:39] Debug mode: Attempting to load personal wordlist...
[2006-12-19 12:03:39] Debug mode: No personal wordlist found, skipping...
[2006-12-19 12:03:39] Debug mode: Analyzing file with content-type "image/gif"
[2006-12-19 12:03:39] Debug mode: Image is single non-interlaced...
[2006-12-19 12:03:39] Debug mode: Recognized file type: 1
[2006-12-19 12:03:39] Debug mode: Image hashing disabled in configuration, skipping...
[2006-12-19 12:03:40] Debug mode: Found word "price" in line
"lmatpriceguaranteeftdenrey"
with fuzz of 0 scanned with scanset /usr/bin/gocr -i -
[2006-12-19 12:03:40] Debug mode: Found word "price" in line
"lmatpriceguaranteeftdenrey"
with fuzz of 0 scanned with scanset /usr/bin/gocr -l 180 -d 2 -i -
[2006-12-19 12:03:40] Debug mode: Found word "viagra" in line
"viaraloomgaooomaoo"
with fuzz of 0.166666666666667 scanned with scanset /usr/bin/gocr -i -
[2006-12-19 12:03:40] Debug mode: Found word "viagra" in line
"viaqrastloomaaomaa"
with fuzz of 0.166666666666667 scanned with scanset /usr/bin/gocr -i -
[2006-12-19 12:03:40] Debug mode: Found word "viagra" in line
"viaqrastloomaarialisnomaa"
with fuzz of 0.166666666666667 scanned with scanset /usr/bin/gocr -l 180 -d 2 -i -
[2006-12-19 12:03:40] Debug mode: Found word "cialis" in line
"viaqrastloomaarialisnomaa"
with fuzz of 0.166666666666667 scanned with scanset /usr/bin/gocr -l 180 -d 2 -i -
[2006-12-19 12:03:40] Debug mode: Found word "valium" in line
"valiumlomgaooantivanmgalo"
with fuzz of 0 scanned with scanset /usr/bin/gocr -l 180 -d 2 -i -
[2006-12-19 12:03:40] Debug mode: Found word "legal" in line
"vlaraloomgaoorlalisomaoo"
with fuzz of 0.2 scanned with scanset /usr/bin/gocr -l 180 -d 2 -i -
[2006-12-19 12:03:40] Debug mode: Starting FuzzyOcr...
[2006-12-19 12:03:40] Debug mode: Attempting to load personal wordlist...
[2006-12-19 12:03:40] Debug mode: No personal wordlist found, skipping...
[2006-12-19 12:03:40] Debug mode: FuzzyOcr ending successfully...
[2006-12-19 12:03:40] Debug mode: Message is spam (score 10)...
[2006-12-19 12:03:40] Debug mode: Words found:
"price" in 1 lines
"viagra" in 2 lines
"cialis" in 1 lines
"valium" in 1 lines
"legal" in 1 lines
(6 word occurrences found)
[2006-12-19 12:03:40] Debug mode: FuzzyOcr ending successfully...

Sweet isn’t it ? And that antispam OCR goodness is just an ‘apt-get install fuzzyocr’ away !

The only parameters I changed in /etc/FuzzyOcr.cf are the following :

focr_verbose 2.0
focr_logfile /var/log/fuzzyocr.log
focr_timeout 16

The two first are self-explanatory : I just want to know what is going on. The original timeout was 12 seconds and I found that it was often too short for my puny server – apparently 16 seconds are more than enough. I restarted Amavisd-new who handles calling SpamAssassin and I was done !

I was afraid that FuzzyOcr would load my host too much but I found my fears unfounded : FuzzyOcr only scan messages which where not recognized yet as ham or spam by other SpamAssassin rules or plugins. So the additional load was not noticeable among all the heavy antispam and antiviral machinery that already operated. FuzzyOcr is full of nice surprises !

As a conclusion I must say that, on our mail server, FuzzyOcr is a complete success. I recommend that you install it as soon as possible !

Versamail, the default mail application on my Treo 650 is good for casually checking a handful of new messages in the inbox but it comes short for about anything else. For searching accross my 5 GB of mail in a hundred folders and accessing them accross a wet string, Mutt is by far the best remote solution.

Although twenty-four years younger than Vi, Mutt was apparently also designed to thrive in an environment made of appaling latency, tiny bitrates and screens with an even lower definition than the Treo‘s screen. GPRS really feels like my first 2400 baud modem, so to me a command line with Lbdb, Mairix, Mutt and Vim definitely seems to be the perfect match for GPRS on the Treo.

A while ago I published lbdb-fetchalladdresses-firsttime.sh and lbdb-fetchalladdresses-daily.sh, two scripts that scour a maildir hierarchy to records all addresses from the headers in the lbdb m_inmail database in order to feed whole maildirs to LBDB for Mutt’s address completion.

A few days ago, for some unknown reasons this script ceased to work correctly. I traced the problem to an incorrect argument to ‘uniq‘. While I was at it I decided to let ‘sort‘ handle the job of ‘uniq‘ and in one fell swoop enhanced the sort operation with some better criterias (-u -d -f -i -k 1,1). So here is version 0.3 for you :

• lbdb-fetchalladdresses-firsttime.sh
• lbdb-fetchalladdresses-daily.sh

As I mentioned earlier, if you do not have an existing lbdb database, you should run lbdb-fetchalladdresses-firsttime.sh first. Once you have done that you can from time to time update your database with lbdb-fetchalladdresses-daily.sh who does the same thing as lbdb-fetchalladdresses-firsttime.sh but only for messages not older than a certain user-set age.

A month ago I published lbdb-fetchalladdresses-firsttime.sh and lbdb-fetchalladdresses-daily.sh, two scripts that scour a maildir hierarchy to records all addresses from the headers in the lbdb m_inmail database in order to feed whole maildirs to LBDB for Mutt’s address completion.

A month of running these scripts has revealed a few problems wich are now corrected with a speed optimization thrown in for good measure :

• lbdb-fetchalladdresses-firsttime.sh
• lbdb-fetchalladdresses-daily.sh

As I mentioned earlier, if you do not have an existing lbdb database, you should run lbdb-fetchalladdresses-firsttime.sh first. Once you have done that you can from time to time update your database with lbdb-fetchalladdresses-daily.sh who does the same thing as lbdb-fetchalladdresses-firsttime.sh but only for messages not older than a certain user-set age.

As many have said before, Jabber is to instant messaging what SMTP is to email:

Today most people understand instant messaging almost entirely in terms provided by AOL, Microsoft and Yahoo. As with on-line services and operating systems, closed IM systems became familiar to the general public long before the open ones. In terms of architecture and control, AOL’s, Microsoft’s and Yahoo’s instant messaging systems are as closed and non-interoperable as Prodigy, Compuserve and AOL were back in the [early] nineties.

In other words, AOL, Microsoft and Yahoo all want to keep private something that should be no less public than mail and web services. All of those companies’ IM systems use the Internet and offer free clients, but they offer no internet service anybody can build on.

This quote dates back from 2001… After all these years I at last consider the Jabber ecosystem to reach both reasonnable usability and critical mass. I believe we can thank Google for tipping the scale. So what’s next ?

I have said it privately since 2002 : the buddy list will become the center of user interaction with the mobile “phone”. Presence management holds every synchronous media together. Through presence management, messaging has moved from the periphery of the desktop toward its core. It changes the environment in which other services interoperate. For example it makes the answering machine obsolete : no one is going to knowingly initiate a call when there is on one on the other end. And the buddy list is where convergence happens : any communication can be initiated from there, not just what the operator deems reasonnable to provide.

As usual the operators are busy fighting rear-guard action, but they cannot keep the customer from craving the ideal of the Stupid Network. GPRS and UMTS enable innovation at the edge and that is the end of the walled gardens. Of course capital expenditure and spectrum scarcity still provide them with a fairly entrenched position, but we are now firmly in Internet land and there is no going back.

So as we all have been thinking all along, the open protocol takes over the closed ones. It takes time, but it is inexorable. The mobile networks are next and they know it… But that does not mean they can avoid it.

Mobile network operators will not like it but instant messaging is going to be priced at bulk data prices, not anywhere near the EUR 1000 a MB that they can get away with for SMS. Earlier attempts to sell services instead of selling raw data have met mixed responses – it will only get worse and I do not believe this to be only my wishful thinking.