PHP :

• Sparkline is a PHP library that produces Edward Tufte inspired “intense, simple, wordlike graphics”. I like the way sparklines spruce up text without interrupting its flow.
• Libchart is a simple PHP charting library that reminds me of the core functionality of the Jpgraph. Simple to deploy and does the basics well.
• Jpgraph can be used to create numerous types of graphs either on-line or written to a file. The range of functionality is very impressive and new features get added all the time. But basic use remains simple. Jpgraph is used by many Free software projects such as Mantis.
• PEAR::Image_Graph was formerly known as GraPHPite. It supports a good choice of graph types, five types of data sources and many output formats.
• Artichow is yet another small PHP charting library. Functionality is limited but it does look clean. The downside is that everything about it is in French… But that may be an upside if you are a French speaker !

Command-line and CGI :

• Ploticus provides a C and Python API, and a Perl command line that can be called from CGI. It is a mature solution that is no longer on the cutting edge but still satisfy many users.

DHTML and Javascript :

• Timeplot is a DHTML-based AJAXy widget for plotting time series and overlay time-based events over them (with the same data formats that Timeline supports). It has limited functionnality, but what it does looks very good and easy to integrate.
• Plotkit is aimed at web applications that require plotting series of data in modern web browsers. It requires MochiKit and supports HTML Canvas and SVG, which makes it a cutting edge way to render graphics. It supports graphing from a dynamic table.
• Plotr is a fork of Plotkit with no need for MochiKit. The result is an incredibly lightweight charting framework : only 12 KB !

Multiplatform :

• Chronoscope aims to bring something like the experience of Google Maps or Google Earth to time series data. It is quite flexible as it supports three authoring models : GWT Widget using Java, Javascript widget, pure HTML+CSS using microformats. In addition it is capable of smart handling of large datasets, even on the client side.

Since I began playing with Net_SmartIRC, I found a new way to put that library to work : a Munin plugin script to monitor the number of users in an IRC channel.

Here is an example of the graphical output provided by Munin :

As you can see, the Debian IRC channel is a very crowded place ! You may also notice small gaps in the data : the script sometimes fails on a refused connection, and I have not elucidated the cause. But as the graph shows, I have coded the script so that those failure cases only result in a null output, which Munin handles well by showing a blank record.

Because my lacking skills and crass lazyness prevented me from writing it all in a single language, I hacked that plugin by simply patching together the parts I could produce rapidly :

• irc_channel_users is a Munin plugin script that calls irc_channel_users.php

• irc_channel_users.php performs the actual login and count on the IRC server.

The PHP script is uses Net_SmartIRC which is available in Debian as php-net-smartirc. It must be configured by modifying the hardcoded server and channel – that may not be what is best in production use, but for the moment it works for me. Here is the full extent of the PHP code :

< ?php
include_once('/usr/share/php/Net/SmartIRC.php');
$irc = &new Net_SmartIRC();
//$irc->setDebug(SMARTIRC_DEBUG_ALL);
$irc->setUseSockets(TRUE);
$irc->setBenchmark(TRUE);
$irc->connect('irc.eu.freenode.net', 6667);
$irc->login('usercount', 'Users counting service for Munin monitoring',
'0', 'usercount');
$irc->getList('#test_channel');
$resultar = $irc->objListenFor(SMARTIRC_TYPE_LIST);
$irc->disconnect();
if (is_array($resultar)) {
    echo $resultar[0]->rawmessageex[4];
} else {
}
?>

The irc_channel_users Bash script is also quite simple. Apart from the barely modified boilerplate adapted from other simple Munin bash scripts, the specific meat of the script is as follow :

work_directory=/home/jim/applications/munin/irc_channel_users
php_interpreter=`which php`
user_population=`$php_interpreter $work_directory/irc_channel_users.php
 | awk -F"#" '{print($1)}' | grep -e '^[0-9]+$'`
echo -n "population.value "
echo $user_population

As you can see, the munin bash script is mostly about setting a few Munin variables, calling the php script and formatting the output.

Here are sample outputs :

15:32 munin@kivu /etc/munin/plugins% ./irc_channel_users autoconf
yes

15:32 munin@kivu /etc/munin/plugins% ./irc_channel_users config
graph_title #b^2 IRC channel users
graph_args --base 1000 -l 0
graph_vlabel population
graph_scale no
population.label users

15:32 munin@kivu /etc/munin/plugins% ./irc_channel_users
population.value 6

No demonstration is available on a public site, but the above graph is about all there is to know about the output of this plugin.

The code resides on its own page and updates if they ever appear shall be stored there.

This experience taught me that coding basic Munin plugins is fun and easy. I will certainly come back to it for future automated graphing needs.

And for those who wonder about the new syntax highlighting, it is produced using GeSHi by Ryan McGeary‘s very nice WP-Syntax WordPress plugin.

Since my migration to PHP 5 revealed a problem with the ancient Lilina 0.7 my interest in Lilina has been rekindled. As I said, I am quite hopeful because I would like to keep using Lilina for small aggregations and avoid deploying the more complex Gregarius where its better scalability is not needed. What first attracted me toward Lilina still holds true.

I started by checking out the development version from Google Code and I deployed it on a handful of small aggregations such as my personal feed. I then immediately started gently pestering the project lead about a few bugs, first by mail and then using the nice Google Code Lilina issue tracker. Ryan Mc Cue has been very responsive and very nice. I look forward making my small contributions in helping him bring Lilina toward 1.0

A year has passed since Ryan announced his intentions. He has already done a lot but his roadmap for Lilina 1.0 is very ambitious so there is still a lot of work remaining. At the current pace it may take a few more months, but this is a journey I am eager to follow : I like Lilina and since I am a novice PHP coder I am quite happy whenever I find a project I like and to which I can give a little help. I am actually quite proud of being mentioned in the credits for having brought RSS output to Lilina : it was the first time I had some of my code included in a significant project.

A few old friends and I often hang together on our usual IRC channel, and while discussing we like to share links. I thought it might be a good idea to use a web tool for that so that we can also share our favorite links with the rest of the world. I thought a private Digg clone could play that role, and I needed an excuse for more PHP development. After some research I found that Pligg was the best candidate. I downloaded Beta 9.7 and I have been quite amazed how fast I got a smooth Pligg site up and running – it seems like much work went into this code.

The first bug that arose was mangled accents in some URLs. I solved it by adding a few cases to a character substitution matrix. This matrix is used to sanitize strings before using them for URLs. The problem is that this method requires each accented character to be treated individually. Maybe a more general solution exists. Anyway I am surprised that no other user noticed that problem with French accents before.

Since I wanted to use Pligg in a way closely associated with our IRC habit, I thought it might be a good idea to make Pligg capable of notifying an IRC channel of an upcoming story. Rumaging through PHP libraries capable of speaking IRC I ended up using Net_SmartIRC which is available in Debian as php-net-smartirc. The examples packaged with the class are quite good at teaching its use so I soon wrote a small proof-of-concept of notifying an IRC channel from PHP.

Some more hacking later I figured how to splice that IRC notification functionality into Pligg. This is a prototype : it works fine for me but you need to overwrite a file from the standard Pligg and edit it with your parameters, and it probably ignores any convention used by those who are familiar with working on Pligg. To reach release quality this functionality should probably be architectured as a proper configurable Pligg plugin. To use it, you need to install Net_SmartIRC which is available in Debian as php-net-smartirc. Make sure that the path of that library as declared in my submit.php is correct.Then you need to edit my submit.php to set connect parameters, login parameters, and the channel. Make a backup of your original submit.php and replace it with mine. That is all !

With IRC notification in place I found that I did not want just anyone spamming our favorite IRC channel. Pligg has three hardcoded user levels : “user”, “admin” and “god”. By default the users can submit posts. I decided to restrict posting to the “admin” and “god” levels. That way we keep control of posting, our friends can vote and comment as “user” and the rest of the anonymous world can read the site.

With the beginning of a familiarity with the Pligg code it did not take me too much grepping to understand the conditional access system and wedge a typical implementation where it should work. And work it did, just as advertised ! I am not sure that the core Pligg developers intended this previously clean switch statement to be hacked like that, but it works for me. A further step would be to make this restriction code conditional and to make the condition a user-configurable parameter in /admin_config.php so that god can choose at which user level submissions are allowed. I am sure that many site operators would appreciate this feature.

So as a summary and conclusion, here is the patch that modifies Pligg’s 9.7 submit.php to provide post submission restriction by user level and IRC notification of posts :

• The original submit.php from Pligg 9.7
• submit.php from Pligg 9.7 patched to provide post restriction by user level and IRC notification of post
• diff between the above files.

Have fun with Pligg ! I’m done with it for now but who knows what other functions I’ll feel like hacking into it in the future…

After migrating an host to PHP5 I found that Lilina 0.7 no longer works and instead produces the following error :

PHP Fatal error: Cannot redeclare class soapclient in /your-lilina-directory/inc/nusoap.php on line 4096

Happily, Robert Mao at “Inmates are Running the Asylum” had already stumbled on this and found a solution.

Robert found a report of the Nusoap library conflicting with PHP5’s built in SOAP functions. The only use of Nusoap in Lilina is the Google API. So Robert found that by disabling the peripheral functionality dependant on the Google API Lilina no longer produced the error.

It works but it is a quick and dirty fix. Enters Ryan Mc Cue who took over Lilina’s development last year. Ryan soon mentioned that the aforementioned functionality is completely disabled in the current development version of Lilina which therefore works fine with PHP5.

There has not been a release of Lilina in quite a while but indeed Ryan and his friends have not been idle and on top of a Brand new web site there have been many commits to the Lilina Subversion repository on Google Code.

So Lilina 1.0 is in development and I’m going to take look at it. I am quite hopeful because I would like to keep using Lilina for small aggregations and avoid deploying the more complex Gregarius where its better scalability is not needed.

I am currently supporting a French centrist political party, mostly helping local militants to improve their web presence by giving them tailored tools and pushing them toward a sensible communication strategy and the organization that goes with it. WordPress and Dokuwiki were of course among the first tools out of the box. I then soon considered the constellation of militant small-time blogs and decided there was a prime target for aggregation.

I first deployed the Lilina PHP news aggregator – I love it and it worked very well for tiny “me and my friends” feed aggregations. With Lilina under new management there were even prospects for improvement. But when the number of inbound feeds began to soar toward 150 I realized that the whole user experience was sinking into a pit of cold molasses. Unexpectedly the host was not even significantly loaded, it was just that updating from that many feeds sequentially was taking much time.

Enter Gregarius. Of course I knew about Gregarius before. But I had no reason to go through the slight hassle of using the Mysql database that Gregarius needs : deploying Lilina only requires the unpacking of my custom distribution of Lilina patched for provinding RSS output. Lilina crawling gave me the reason. On top of that, the Lilina theme for Gregarius really made migration as painless as possible for Lilina refugees who can feel at home right out of the box.

Installation was dead easy and importing all those incoming feeds was done in the single step of entering the URL for Lilina‘s OPML output and waiting a few seconds for all the feeds to be fetched and parsed. And there you are : 130 blogs (and growing) aggregated effortlessly with reasonnable response time and barely any load on the host. Gregarius is even easier than Lilina to administer, and it has categories and tags that Lilina does not, and also does searching.

So from now on I’ll use Lilina for aggregating up to about a couple dozen feeds. Beyond that the territory belongs to Gregarius !

Coldforged mentioned that Lilina “silently failed on several malformed feeds“. Coldforged adds “that wouldn’t be so bad if it survived such indignities but it didn’t. Instead my entire aggregation page would simply not load and the only notice I ever got that something was awry was the PHP error_log in the aggregator directory“. I’ll look out for such behavior, but so far I have had no problems with the few feeds I am using.

Were I looking for something with more features than Lilina I would probably turn to Gregarius with the Lilina theme because I like Lilina’s way so much. And the “RSS View” plugin apparently provides the outgoing something like the aggregated RSS feed that I implemented in Lilina by splicing Feedcreator in.

But for now I’ll stick with Lilina because it is extremely easy to use. The lack of need for a database makes deployment as simple as copying files, setting permissions (the archive of the patched Lilina I provide now features a script to do it for you) and editing a couple of lines in a configuration file. That is hard to beat.

Aggregated RSS feeds presented as HTML and Javascript by Lilina are very sweet. The more we used them, the more we missed having them served as RSS. After much research it seemed to us that there is no nice and easy PHP code capable of mixing RSS as RSS. There are plenty of feed mixers offered as a service but very few offered as a product.

On the fetching and parsing side, Lilina had everything I wanted. All I needed was to make it generate RSS instead of HTML.

I went foraging for RSS creation libraries. The first one I found was XML-RSS-Aggregate . I liked it because the example provided with XML-RSS-Aggregate is an RSS agregator that ouptuts RSS – exactly what I was looking for. But Shlomi Fish mentioned that “this module is unmaintained and no longer works very well. The author (and I) recommend that you use XML::Feed now“. So I took a look at XML-Feed and found it too complex for my meagre skills. And I’m not that hot with Perl anyway. So I went looking somewhere else.

I found my salvation in Feedcreator. Feedcreator creates valid feeds in various formats, features configurable caching, reasonnable documentation and readable code. I found it quite easy to use. All it needs is an array of RSS elements, and that is exactly what Lilina provides.

I took Lilina’s index.php, cleaned up the HTML generation, spliced in the example code from Feedcreator, mapped input to output and lo and behold I had a reasonably valid RSS output by Lilina. Very sweet !

Source code of the modified Lilina with Feedreactor hybridation is available here.

I even added a cute RSS icon to Lilina’s default layout…

Looking for a way to fetch multiple RSS news feeds and present them as a single HTML page I found the wonderful Lilina.

“Lilina is a simple but powerful news aggregator written in PHP. No database is needed, RSS/ATOM parsing is done by the excellent MagpieRSS library”.

That piece of advertisement is all true : Lilina is dead simple to set up, requires no special dependancies and produce very nice aggregated news feeds. This was love at the first sight !

I immediately set up a couple of aggregated news feeds :

• Grabeuh planet shows articles and news items published by my fellow geek friends and myself.
• Jean-Marc’s little corner of the world shows all photos, articles and news items by me.

Next will be a personal feed gathering all my favorite places that publish irregularily. That page will save me quite a lot of clicking around checking for updates.

A while ago I built a Jabber presence indicator on my personal web page by customizing one of the examples provided with class.jabber.php. I have received enthusiastic feedback from a handful of users – that makes me feel warm inside ! Do not neglect the ego of people who provide free services, be they Free software developpers or volunteers for anything : a kind comment or a nice stroke in the right direction go a long way toward feeding their motivation. So next time you enjoy the produce of the gift economy be sure to let people know how much you liked it !

With the kind words came a question which made me aware that a newcomer may not find the required configuration as obvious as I do. One of the least obvious steps is telling the script whose status it is supposed to check. Don’t look for a JID parameter inside the presence indicator script : the presence indicator script itself knows nothing about the user whose status it checks. It is the watcher bot user that has the watched user on his roster. The presence indicator script will return the status of anyone on the watcher bot user’s roster. That makes it quite flexible because it will work for any number of users.

If anything remains unclear to you, please be sure to report it to me.

Nathan Fritz reports Lars Mikkelsen patching class.jabber.php to support Google Talk. Nathan is reviewing the patch and considers including it in a next release of class.jabber.php.

So we learn simultaneously that class.jabber.php does not support Google Talk and that it soon will…

You may have noticed that I have added a to the Jabber presence indicator on my personal web page. It is provided by Edgar, a sophisticated Web Jabber presence indicator based on class.jabber.php. To use the Edgar service, all you need is an image tag such as :

http://edgar.netflint.net/status.php?jid=jim@jabber.grabeuh.com&type=image

Edgar displays status fast because it splits processing in two independant parts sharing a database :

• edgar.php is a command line script that collects presence data into the database.
• status.php reads the database and displays presence on an HTML page.

Edgar is licensed under the GPL so you can download it and set it up on your own host.

I like Edgar and I will certainly study it as an exemple of a non-trivial implementation of class.jabber.php. But I shall continue to use my pathetic Web Jabber presence indicator based on one of the examples provided with class.jabber.php because Edgar does not yet display ressources whereas the script I use does.

One of the problems with my Jabber presence indicator on a web page was that the page’s presence was being sent to the polled account instead of being hidden. I fixed that and the updated code is available. The fix was trivial :

29c29
< $JABBER->SendPresence();
—
> $JABBER->SendPresence(“invisible”);

Although widely implemented (even Edgar does it that way), the “invisible” presence type is not XMPP compliant. JEP 126 explains how to provide selective visibility in a XMPP compliant manner. Of course this is less trivial. But since JEP 126 provides the appropriate XML snippets, I could probably do it the right way using SendPacket($xml) to send the raw XML. Maybe next time… Meanwhile I reported the standard compliance issue on the Edgar bug tracker.

I now rely on Jabber to publish my presence. Not everyone has a Jabber client at hand but Web access is about ubiquitous so it makes sense to me to make my presence available on a Web page. To me this is a personnal itch and an excuse to practice some PHP and learn a little about the exciting world of XMPP.

The source code for my presence indicator is now available for whoever wants it. It is made possible by class.jabber.php, a Jabber library for PHP. I customized one of the examples provided with class.jabber.php. It has many rough edges such as the user’s name sprinkled across the markup code instead of being a proper parameter, or the page’s presence being sent to the polled account instead of being hidden. And it is quite slow because it queries at load time. But I like it better than the example packaged with class.jabber.php.

It basically does the job and I am therefore quite sure that there will be interested users. So this is the “early” part of “Release Early, Release Often”. Take that hack, customize it, tweak it and generally improve it… And most of all please report back somewhere so that everyone can benefit. Here would be a good start.