Breaking out of single dimension filtering : IMAP keywords
I use Maildrop to dispatch incoming mail to the appropriate maildir. Such server side filtering is nice because my mail is always tidily waiting for me in the right folder whichever tool I use to read it. But it has a major drawback : each message is present in one and only one folder. For those messages about playing paintball with my grandmother I must choose between the “paintball” folder and the “family” folder – they cannot be in both unless I make a copy which would be a gross hack.
This problem is common to all hierarchical storage of physical items. But it has already been solved many times before with the use of tags. Whether you want to go hardcore with a full thesaurus or just use your little folksonomy the required technical foundation is the same : tags.
Of course, using tags is nothing new to Gmail users : Gmail allows users to categorize their e-mails with “labels.” Labels give users a flexible method of categorizing e-mails, since an e-mail may have any number of labels (in contrast to a system in which an e-mail may belong to only one folder). Users can display all e-mails having a particular label and can use labels as a search criterion. Gmail also allows users to set up filters which label incoming e-mail automatically. Users can simulate the functionality of folder-based filtering by applying labels and archiving mail as it arrives.
But most of the rest of the world has not caught up yet. So let’s do it : it seems to me that we have everything we need to build powerful functionnality on top of our fine toolset. As usual we will not settle for anything that makes us dependant on non-free products or on client side contraptions. So let’s review the available tools :
Server side filtering requires a competent mail delivery agent. A draft RFC describes the “IMAP flag extension” to the Sieve filtering language, but I have not found anything else (a reader later supplemented my lacking research by pointing out that “works on the Sieve language specification is still very much alive and well” – see the comments). Procmail can set keywords and Maildrop supports keywords too. The Maildrop package features a command-line utility (maildirkw) that allows other applications to manually set or clear custom keywords on messages and IMAP keywords may also be set with maildrop itself.
So we potentially have a bunch of maildirs containing appropriately tagged messages. And to serve them we luckily have a whole bunch of IMAP servers supporting IMAP keywords. It came to me as a surprise to see both IMAP servers and MDA providing apparently mature support for IMAP keywords. That is a good surprise. So while we are at it here are a few details about Courier-IMAP’s IMAP keywords implementation.
The less good surprise is the state of client support :
- A couple of weeks ago, Nancy McGough herself posted to evolution-list@gnome.org to enquire about IMAP keywords support in Ximian Evolution – I also failed to find any information about it and I do not take that as a good omen.
- A mutt user asked about IMAP keywords but no useful answer was produced – research yields no further information.
- Squirrelmail has a “Message Flags & Icons” plugin. It could be worse, but development does not seem very lively.
- Thunderbird supports IMAP keywords but only allows five of them. This is a known bug but it has been open for more than four year so I’m not holding my breath for it. The amount of comments made to that bug show that user expectations about IMAP keywords are slowly growing.
So IMAP keywords are as far as I am concerned not ready for production use yet but we are now just some client support away from that stage. I am eagerly looking forward to it ! I will certainly still use physical folders for the two top levels of my classification. But below that I am going to replace a lot of them with tags.
5 responses to “Breaking out of single dimension filtering : IMAP keywords”
Leave a Reply
You must be logged in to post a comment.
I received the following comment from a reader. It sounds like my perception of the liveliness of Cyrus was mistaken and I am thankful to him for providing us with this information that contributes to a more accurate picture of the IMAP keywords support by IMAP servers. I was certainly confused about the Cyrus brand – some of the confusion may stem from lack of research, but maybe it shows that there is a potential for improvement in the communication of the Cyrus IMAP server community toward the audience. Anyway thanks for the feedback – it’s good to see that someone is paying attention !
—–Original Message—–
From: Amos [mailto:a.goo0h@gmail.com]
Sent: Wednesday, February 01, 2006 3:53 PM
To: jim@liotier.org
Subject: About Cyrus
In your post:
http://serendipity.ruwenzori.net/index.php/systems/2006/01/31/breaking-out-of-single-dimension-filtering-imap-keywords
You mention:
“but with Cyrusoft gone bankrupt and the whole Cyrus suite gathering
increasingly thick dust I don’t think it matters that much anymore.”
This is nonsense and displays a considerable degree of ignorance, but
I’ll fill you in on the details.
Cyrusoft primarily developed the Mulberry mail client. The name of the
company originated because the primary, initial developer of the
Mulberry client was Cyrus Daboo. You can still find Daboo very much
involved with on-going IMAP standards development.
I’m not sure exactly what you mean by the Cyrus suite, but the Cyrus
IMAP server that was originally developed by Carnegie Mellon
University (and is the basis of several commercial products) is still
very much alive and well:
http://asg.web.cmu.edu/cyrus/
http://blog.gmane.org/gmane.mail.imap.cyrus
In fact 2.3 BETA, which incorporates server replication, was recently released.
And yes, Cyrus supports user-specified flags and has for quite some
time. I know that for a fact because when I used GNUS with a Cyrus
server several years ago, I noticed back then GNUS was making use of
user-specified flags.
Also, the IETF group MTA-Filters that works on the Sieve language
specification is still very much alive and well too:
http://blog.gmane.org/gmane.ietf.mta-filters
Anyway, hopefully that clears things up.
Amos
I’ve been experimenting with IMAP tags/labels today. Courier IMAP supports them, and if you use Courier’s maildrop program for filtering then you can have them applied automatically at the server-side.
For example, create a file, ~/.mailfilter and assign a comma-separated list of keywords
if (/^From:.*jenny\.p/)
{
KEYWORDS=”family”
to $DEFAULT
}
I already use a .mailfilter for delivering to specific folders, so it is nice to know that I can extend my current practice toward tags !
Since it does not show in the related entries, here is a port from 2007 where I mention the tagging capabilities of Thunderbird 2 : http://serendipity.ruwenzori.net/index.php/2007/04/20/is-thunderbird-2-the-last-missing-piece-of-the-imap-mail-tagging-puzzle
The client side has certainly evolved further since then, but that is about as far as I explored – I have dropped that ball for a while, although I might come back to this problem…
[…] Noticing how Google has been successful with email tagging, I started exploring free implementation …, aiming to complement my folder hierarchy with tags. But inertia affects all, and I actually never went beyond happy experimentation. As Zoli Erdos puts it in his eloquent description of Google’s 2009 breakthrough in ending the tags vs. folders war : […]