Serendipitous altruism

I use Maildrop to dispatch incoming mail to the appropriate maildir. Such server side filtering is nice because my mail is always tidily waiting for me in the right folder whichever tool I use to read it. But it has a major drawback : each message is present in one and only one folder. For those messages about playing paintball with my grandmother I must choose between the “paintball” folder and the “family” folder – they cannot be in both unless I make a copy which would be a gross hack.

This problem is common to all hierarchical storage of physical items. But it has already been solved many times before with the use of tags. Whether you want to go hardcore with a full thesaurus or just use your little folksonomy the required technical foundation is the same : tags.

Of course, using tags is nothing new to Gmail users : Gmail allows users to categorize their e-mails with “labels.” Labels give users a flexible method of categorizing e-mails, since an e-mail may have any number of labels (in contrast to a system in which an e-mail may belong to only one folder). Users can display all e-mails having a particular label and can use labels as a search criterion. Gmail also allows users to set up filters which label incoming e-mail automatically. Users can simulate the functionality of folder-based filtering by applying labels and archiving mail as it arrives.

But most of the rest of the world has not caught up yet. So let’s do it : it seems to me that we have everything we need to build powerful functionnality on top of our fine toolset. As usual we will not settle for anything that makes us dependant on non-free products or on client side contraptions. So let’s review the available tools :

Server side filtering requires a competent mail delivery agent. A draft RFC describes the “IMAP flag extension” to the Sieve filtering language, but I have not found anything else (a reader later supplemented my lacking research by pointing out that “works on the Sieve language specification is still very much alive and well” – see the comments). Procmail can set keywords and Maildrop supports keywords too. The Maildrop package features a command-line utility (maildirkw) that allows other applications to manually set or clear custom keywords on messages and IMAP keywords may also be set with maildrop itself.

So we potentially have a bunch of maildirs containing appropriately tagged messages. And to serve them we luckily have a whole bunch of IMAP servers supporting IMAP keywords. It came to me as a surprise to see both IMAP servers and MDA providing apparently mature support for IMAP keywords. That is a good surprise. So while we are at it here are a few details about Courier-IMAP’s IMAP keywords implementation.

The less good surprise is the state of client support :

• A couple of weeks ago, Nancy McGough herself posted to evolution-list@gnome.org to enquire about IMAP keywords support in Ximian Evolution – I also failed to find any information about it and I do not take that as a good omen.
• A mutt user asked about IMAP keywords but no useful answer was produced – research yields no further information.
• Squirrelmail has a “Message Flags & Icons” plugin. It could be worse, but development does not seem very lively.
• Thunderbird supports IMAP keywords but only allows five of them. This is a known bug but it has been open for more than four year so I’m not holding my breath for it. The amount of comments made to that bug show that user expectations about IMAP keywords are slowly growing.

So IMAP keywords are as far as I am concerned not ready for production use yet but we are now just some client support away from that stage. I am eagerly looking forward to it ! I will certainly still use physical folders for the two top levels of my classification. But below that I am going to replace a lot of them with tags.

If you install the new 2.3.3 version of amavisd-new via the Debian package, you are in for a few (possibly unpleasant) surprises. Among other changes, /etc/amavis/amavisd.conf no longer works.

The new configuration files in /etc/amavis/conf.d there are:

01-debian
05-node_id
15-av_scanners
15-content_filter_mode
20-debian_defaults
30-template_localization
50-user

To my dismay I also found that virus and spam filtering were also rendered inoperative although mail kept passing through Amavis just fine…

The header of 50-user says : “Place your configuration directives here. They will override those in earlier files“. It should have said “Place most of your configuration directives here” because there is another place where a couple of essential configuration parameters must be set, and it is not made obvious nor even pointed to by the documentation or anything encountered on forums and mailing list. The header of /etc/amavis/conf.d/15-content_filter_mode mentions : “You can modify this file to re-enable SPAM checking through spamassassin and to re-enable antivirus checking“. And indeed, commenting out the couple of lines that obviously needed being commented out re-enabled my favorite content filters.

And the lesson of the day : just porting the old configuration does not cut it, you really need to read every new configuration file – RTFC (Read The Fine Configuration)…

“War upon rebellion is messy and slow, like eating soup with a knife” – TE Lawrence.

In his book “Counterinsurgency Lessons from Malaya and Vietnam: Learning to Eat Soup with a Knife” John Nagl examines how armies learn during the course of conflicts for which they are initially unprepared in organization, training, and mindset. Like “The Seven Pillars of Wisdom” from which TE Lawrence’s quote is excerpted, it is a very useful read for understanding the current situation in Iraq. Written well before Operation Iraqi Freedom it nethertheless shows lessons very relevant to it.

Voicing publicly the private views of many of his peers, Brigadier Nigel Aylwin-Foster, a senior officer in the British Army has published a critical analysis of American military attitudes in Iraq. I do not believe his paper to be the scathing account that the press presents. On the contrary, it confirms the traditional strengths of the US Army and the ability for learning that individuals have demonstrated. But is does point out how heavy institutional inertia prevents widespread adaptation to the nature of counterinsurgency. Compounded doctrinal and cultural factors are shown to be the root cause of the American difficulties in Iraq.

Alwyn-Foster cites Nagl noting that “The American Army’s role from its very origins was the eradication of threats to national survival’, in contrast to the British Army’s history as ‘an instrument of limited war, designed to achieve limited goals at limited cost”. The USA possesses nearly irresistible powers in conventional wars against nation-states. But when confronted with counterinsurgency operations, this strength induces such a bias that it becomes a root of weakness : when the only tool you have is a hammer, everything looks like a nail. For example, Alwyn-Foster underlines that the US Army’s ‘Soldier’s creed’ “enjoins the soldier to have just the one type of interaction with his enemy – ‘to engage and destroy him:’ not defeat, which could permit a number of other politically attuned options, but destroy”. Being a warrior is part, but not all, of being a soldier.

Alwyn-Foster’s paper leads the reader to “the realisation that all military activity is subordinate to political intent, and must be attuned accordingly: mere destruction of the enemy is not the answer”. It is no suprise to me that he reminds me of what my aikido master used to teach us : the goal of martial arts is not the destruction of the opponent, it is the destruction of the conflict. Adversary and partner are one and there can be no winning of the hearts and minds unless this has been realized.

Nowadays, apt has built-in support for GPG verification of package signature. It was silently doing its business of verifying the integrity of Debian packages until a couple of days ago when an ‘apt-get update’ yielded the following error :

W: GPG error: http://ftp2.fr.debian.org testing Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 010908312D230C5F
W: GPG error: http://ftp2.fr.debian.org unstable Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 010908312D230C5F
W: You may want to run apt-get update to correct these problems


Apt is complaining about a package signed with a key it does not know about. We need to import it into the GPG keyring and then tell apt about it. So let’s issue the two following commands as root :

# gpg --keyserver wwwkeys.eu.pgp.net --recv-keys 010908312D230C5F
# gpg --armor --export 2D230C5F | apt-key add -


And that’s it… We’re now back to our regularly scheduled programming…

Can you believe that as of today the string “collaborative adblock” yields no hits on Google ? For now there is the centraly managed list of blockable advertisement at yoyo.org. Conversion scripts make the list available in a large number of popular ad blocking formats and there is even an Adblock filterset updater available for it as a Mozilla extension. You can submit your own items, that is a start. But wouldn’t a truly distributed collaborative peer to peer sharing of Adblock items be staggeringly more powerful ? Something made robust with a reputation system… Let’s seed the meme !