Code and Debian and Free software and Maps and Openstreetmap25 Aug 2015 at 10:52 by Jean-Marc Liotier

There you are, in some Openstreemap editor, correcting the same typo for the 16th time, cursing contributors who neglect correct capitalization and thinking about how tedious this necessary data gardening is. While JOSM is endowed with unfathomable depths of cartographic potentiality, you long for a way to simply whip out your favourite text editor and apply its familiar power to the pedestrian problem of repeatedly editing text. Or the problem requires editing multiple mutually dependent tags and some XML-aware logic is therefore required – all the same: you just want to perform Openstreetmap editing as text processing.

Of course, as an experienced Openstreetmap gardener, you are well aware of the dangers of casually wielding a rather large chainsaw around our burgeoning yet fragile data nursery. So you understand why automated processing is generally not conducive to improvement in data quality – rare is the automation whose grasp of context equals human judgment. But human judgment could use some power tools… So there.

My overall workflow shall be as follow:

0 – Read the Automated Edits code of conduct
1 – Get data
2 – Edit data
3 – Review data
4 – Commit data

The meticulous reader might object that making the reviewing an explicit step separate from the editing is superfluous since no self-respecting cartographer would commit edited data without having performed a review as a mandatory step integral to edition. But the reader who closely observes Openstreetmap activity might counter that this level of self-disciplined care might not be universal, so the step is worth mentioning. Moreover, I’ll add that as soon as any level of automation is introduced, I consider the reviewing as a necessary checklist item.

So, first let’s get the data ! There are many ways… The normal JOSM way of course – but your mass edition requirement probably means that you wish to edit a body of data much larger than what the Openstreetmap servers will let JOSM download at once – and, if you ever had to repeatedly download rectangles until you have covered you whole working area, you don’t want to do it again.

To illustrate this article, I chose to edit places of worship in Senegal (I am a rather active Openstreetmap contributor for Senegal and places of worship are socially and cartographically important landmarks). This dataset is rather small – in such cases you might want to peruse Overpass Turbo. The relevant Overpass Turbo query is as follow:

 out meta;
 out meta qt;

Another option, viable even for targeting the whole planet, is to use Osmosis (package available from good distributions) to filter a planet extract:

 osmosis \
 --read-pbf file=africa-latest.osm.pbf \
 --bounding-box top=16.7977 bottom=12.0832 \
 left=-17.6317 right=-11.162 \
 --tag-filter accept-nodes amenity=place_of_worship \
 --tag-filter reject-relations \
 --tag-filter reject-ways outPipe.0=nodesPipe \
 --read-pbf file=africa-latest.osm.pbf \
 --bounding-box top=16.7977 bottom=12.0832 \
 left=-17.6317 right=-11.162 \
 --tag-filter accept-ways amenity=place_of_worship \
 --tag-filter reject-relations \
 --used-node outPipe.0=waysPipe \
 --merge inPipe.0=nodesPipe inPipe.1=waysPipe \
 --write-xml senegal-place_of_worship.osm

Yes, I didn’t take relations into account – there are only a couple of amenity=place_of_worship relations in Senegal’s Openstreetmap data… So adding relations to this query is left as an exercise for the reader.

A gigabyte download and a couple of minutes of osmosis execution later, your data is ready and you have found new appreciation of how fast Overpass Turbo is. Our Osmosis computation might have been a little faster if there was a Senegal planet extract available, but we had to contend with taking the whole of Africa as an input and filtering it through a bounding box.

By the way, the dedicated reader who assiduously tries to reproduce my work might notice that the two methods don’t return the same data. This is because the Overpass Turbo query filters properly by intersection with Senegal’s national borders whereas my Osmosis command uses a rectangular bounding box that includes bits of Mauritania, Mali, Guinea and Guinea Bissau. One can feed Osmosis a polygon produced out of the national borders relations, but I have not bothered with that.

Examples OSM XML elements extracted by the osmosis query:

<way id="251666247" version="2" timestamp="2014-03-27T22:16:56Z"
      uid="160042" user="Jean-Marc Liotier" changeset="21354510">
   <nd ref="2578488987"/>
   <nd ref="2578488988"/>
   <nd ref="2578488989"/>
   <nd ref="2578488990"/>
   <nd ref="2578488991"/>
   <nd ref="2748583071"/>
   <nd ref="2578488987"/>
   <tag k="name" v="Grande mosquée de Ndandia"/>
   <tag k="source" v="Microsoft Bing orbital imagery"/>
   <tag k="amenity" v="place_of_worship"/>
   <tag k="religion" v="muslim"/>
   <tag k="denomination" v="sunni"/>

<node id="2833508543" version="2" timestamp="2014-09-01T09:57:07Z"
      uid="160042" user="Jean-Marc Liotier" changeset="25155955"
      lat="14.7069108" lon="-17.      4580774">
   <tag k="name" v="Mosquée Mèye Kane"/>
   <tag k="amenity" v="place_of_worship"/>
   <tag k="religion" v="muslim"/>

<node id="2578488987" version="1" timestamp="2013-12-13T17:49:40Z"
   uid="1219752" user="fayecheikh75" changeset="19436962"
   lat="14.2258174" lon="-16.8134644"/>

The first two example OSM XML elements will not come as a surprise: they both contain <tag k=”amenity” v=”place_of_worship”/> – but what about the third, which does not ? Take a look at its node id – you’ll find it referred by one of the first example’s <nd /> elements, which means that this node is one of the six that compose this way. Including nodes used by the ways selected by the query is the role of the –used-node option in the osmosis command.

But anyway, why are we including nodes used by the ways selected by the query ? In the present use-case, I only care about correcting trivial naming errors – so why should I care about the way’s geometry ? Well… Remember the step “3 – Review data” ? Thanks to being able to represent the way geometrically, I can visually find that an English-language name may not be an error because the node that bears it is located in Guinea Bissau, not in Senegal where it would definitely be an error outside of the name:en tag. Lacking this information I would have erroneously translated the name into French. Actually, I first did and only corrected my error after having reviewed my data in JOSM – lesson learned !

Talking about reviewing, is your selection of data correct ? Again, one way to find out is to load it in JOSM to check tags and geographic positions.

And while in JOSM, you might also want to refresh your data – it might have become stale while you were mucking around with osmosis (do you really think I got the query right the first time ?) and Geofabrik’s Planet extracts are only daily anyway… So hit Ctrl-U to update your data – and then save the file.

This concludes step “1 – Get data” – let’s move on to step ‘2 – Edit data’ ! First, do not edit the file you just saved: we will need it later to determine what we have modified. So produce a copy, which is what we’ll edit – execute ‘cp senegal-place_of_worship.osm senegal-place_of_worship.mod.osm’ for example.

Now take your favourite text processing device and go at the data ! I used Vim – here is how it looks like:

A few edits later:

$ diff -U 0 senegal-place_of_worship.osm senegal-place_of_worship.mod.osm \
  | grep ^@ | wc -l

As an example of modification, let’s have a look at this node :

<node id="2165307529" version="1" timestamp="2013-02-21T13:40:05Z"
      uid="1234702" user="malamine19" changeset="15112373"
      lat="16.0326014" lon="-16.5084412">
   <tag k="name" v="Mosquée Sidy TALL"/>
   <tag k="amenity" v="place_of_worship"/>
   <tag k="religion" v="muslim"/>

A few keystrokes later, its name’s capitalization is fixed :

<node id="2165307529" version="1" timestamp="2013-02-21T13:40:05Z"
      uid="1234702" user="malamine19" changeset="15112373"
      lat="16.0326014" lon="-16.5084412">
   <tag k="name" v="Mosquée Sidy Tall"/>
   <tag k="amenity" v="place_of_worship"/>
   <tag k="religion" v="muslim"/>

Let’s open the file in JOSM and upload this awesome edit to Openstreeetmap – here we go !

“Whaaat ? No changes to upload ? But where are my edits ? You said we could just edit and upload !” – No, and anyway I said that you had to review your data beforehand !

Fear not, your edits are safe (if you saved them before closing your editor…) – it is only JOSM who does not know which objects you edited. Looking at the above data, it has now way to determine if any part has been edited. We’ll have to tell it !

When JOSM modifies any element of an Openstreetmap object, it marks the Openstreetmap object with an action=”modify” attribute. So, we’ll emulate this behaviour.

“Whaaat ? Do I really have to write or copy/paste action=”modify” on the parent Openstreetmap object of every single modification ? You said this article was about automation !” – fear not, I have you covered with this article’s crowning achievement: the OSMXML_mark_modified_JOSM-style script.

Remember when I’ll said earlier “First, do not edit the file you just saved: we will need it later to determine what we have modified. So produce a copy, which is what we’ll edit – ‘cp senegal-place_of_worship.osm senegal-place_of_worship.mod.osm’ ” ? We are now later and the OSMXML_mark_modified_JOSM-style script will not only determine what we have modified but also mark the parent Openstreetmap object of each modification with an action=”modify” attribute.

This blog needs a wider stylesheet, so no inline code in the article – read OSMXML_mark_modified_JOSM-style on Github instead and save me the paraphrasing of my own code ! This script owes everything to XML::SemanticDiff and XML::LibXML – it is a mere ten-line conduit for their blinding awesomeness so all credits go to Shlomi Fish and Kip Hampton.

So, just make sure that you have XML::SemanticDiff and XML::LibXML installed from the CPAN or preferably from your distribution’s packages and execute the command line:

OSMXML_mark_modified_JOSM-style \
    originalOSMfile.xml \

or in our current example

OSMXML_mark_modified_JOSM-style \
    senegal-place_of_worship.osm \

As a result, the parent Openstreetmap object of each modification will have been marked with an action=”modify” attribute – as our example object:

<node id="2165307529" version="1" timestamp="2013-02-21T13:40:05Z"
      uid="1234702" user="malamine19" changeset="15112373"
      lat="16.0326014" lon="-16.5084412" action="modify">
   <tag k="name" v="Mosquée Sidy Tall"/>
   <tag k="amenity" v="place_of_worship"/>
   <tag k="religion" v="muslim"/>

Now open the modified file in JOSM and review the result. As I mention in passing in the script’s comments: BLOODY SERIOUSLY REVIEW YOUR CONTRIBUTION IN JOSM BEFORE UPLOADING OR THE OPENSTREETMAP USERS WILL COME TO EAT YOU ALIVE IN YOUR SLEEP ! Seriously though, take care : mindless automatons that trample the daisies are a grievous Openstreetmap faux pas. The Automated Edits code of conduct is mandatory reading.

Ok, I guess you got the message – you can now upload to Openstreetmap:

If you spent too long editing, you might encounter conflicts. Carefully resolve them without stepping on anyone’s toes… And enjoy the map !

Incidentally, this is my first time using XML::LibXML and actually understanding what I’m doing – I love it and there will be more of that !

Debian and Networking & telecommunications and Systems administration01 Jun 2015 at 15:02 by Jean-Marc Liotier

You have a nice amplifier in a corner of the living-room, tethered to nice loudspeakers. Alas the spot where you want to control the music from is far away – maybe in another corner of the livingroom, maybe in your office room or maybe on another continent… No problem – for about 40€ we’ll let you switch your music to this remote destination just as easily as switching between the headphones and speakers directly connected to your workstation. The average narcissistic audiophile pays more than that for a RCA cable.

So, I grabbed a Raspberry Pi model B (first generation – get them cheap !), a micro-USB power supply from the spares box and a stray SD card (at least 1 GB).

First step is to set it up with an operating system. Since I love Debian, I chose Raspbian.A handy way to install Raspbian quick & easy is raspbian-ua-netinst, a minimal Raspbian unattended installer that gets its packages from the online repositories – it produces a very clean minimal setup out of the box.

So, go to raspbian-ua-netinst’s latest release page and download the .img.xz file – then put it on the SD card using ‘xzcat /path/to/raspbian-ua-netinst-<latest-version-number>.img.xz > /dev/sdX’ for which you may have to ‘apt-get install xz-utils’. Stick that card into the Raspberry Pi, connect the Raspberry Pi to Ethernet on a segment where it can reach the Internet after having been allocated its parameters through DHCP or NDP+RDNSS – and power it up.

Let raspbian-ua-netinst do its thing for about 15 minutes – time enough to find what IP address your Raspberry Pi uses (look at your DHCP server’s leases or just ‘nmap -sP’ the whole IP subnet to find a Raspberry Pi). Then log in over ssh – default root password is raspbian… Use ‘passwd’ to change it right now.

The default install is quite nice, but strangely doesn’t include a couple of important features… So ‘apt-get install raspi-copies-and-fills rng-tools’ – raspi-copies-and-fills improves memory management performance by using  a memcpy/memset implementation optimised for the ARM11 used in Raspberry Pi, rng-tools lets your system use the hardware random number generator for better cryptographic performance. To finish setting up the hardware RNG, add bcm2708-rng to /etc/modules.

Also, the default install at the time of this writing uses Debian Wheezy, which contains a Pulseaudio version too old for our purposes – we need Debian Jessie which offers Pulseaudio 5 instead of Pulseaudio 2. And anyway, Jessie is just plain better – so let your /etc/apt/sources.list look like this:

deb jessie main firmware contrib non-free rpi
deb jessie main contrib non-free rpi

Then ‘apt-get update && apt-get -y dist-upgrade && apt-get -y autoremove’… This should take a while.

Now install Pulseaudio the piece of software that will offer a receiving end to your network audio stream:  ‘apt-get install pulseaudio’. I assume you have Pulseaudio already set up on the emitter station – your favourite distribution’s default should do fine, as long as it provides Pulseaudio version 5 (use ‘pulseaudio –version’ to check that).

Pulseaudio is primarily designed to cater to desktop usage by integrating with the interactive session of a logged in user – typically under control of the session manager of whatever graphical desktop environment. But we don’t need such complex thing here – a dumb receptor devoid of any extra baggage is what we want. For this we’ll use Pulseaudio’s system mode. Pulseaudio’s documentation repeatedly hammers that running in system mode is a bad idea – “nobody should run it that way, with the exception of very few cases”… Well – here is one of those very few cases.

In their zeal to discourage anyone from running Pulseaudio in system mode, the Pulseaudio maintainers do not ship any startup script in the distribution packages – this ensures that users who don’t know what they are doing don’t stray off the beaten path of orthodox desktop usage and end up on a forum complaining that Pulseaudio doesn’t work. But it also annoys the other users, who actually need Pulseaudio to run at system startup – but that is easily fixable thanks to another creation of Lennart’s gang: all we need is a single file called a systemd unit… I copied one from this guy who also plays with Pulseaudio network streaming (but in a different way – more on that later). This systemd unit was written for Fedora, but it works just as well for Raspbian… Copy this and paste it in /etc/systemd/system/pulseaudio.service :

Description=PulseAudio Daemon


ExecStart=/usr/bin/pulseaudio –system –realtime –disallow-exit –no-cpu-limit

Then ‘systemctl enable pulseaudio’ and ‘systemctl start pulseaudio’ – you now have a properly set up Pulseaudio daemon. Now is a good time to take a moment to consider how much more fastidious the writing of a SysVinit script would have been compared to just dropping this systemd unit in place.

Now let’s see the meat of this article: the actual setup of the audio stream. If you stumbled upon this article, you might have read other methods to the same goal, such as this one or this one. They rely on the server advertising its Pulseaudio network endpoint through Avahi‘s multicast DNS using the module-zeroconf-publish pulseaudio module, which lets the client discover its presence and so that the user can select it as an audio destination after having told paprefs that remote Pulseaudio devices should be available locally. In theory it works well and it probably works well in practice for many people but Avahi’s behaviour may be moody – or, in technical terms, subject to various network interferences that you may or may not be able to debug easily… Struggling with it led me to finding an alternative. By the way, Avahi is another one of Lennart’s babies – so that might be a factor towards Pulseaudio’s strong inclination towards integrating with it.

Discoverability is nice in a dynamic environment but, in spite of my five daughters, my apartment is not that dynamic – my office and the livingroom amplifier won’t be moving anytime soon. So why complicate the system with Avahi ? Can’t we just have a static configuration by declaring a hardcoded link once and for all ? Yes we can, with module-tunnel-sink-new & module-tunnel-source-new !

Module-tunnel-sink-new and module-tunnel-source-new are the reason why we require Pulseaudio 5 – they appeared in this version. They are a reimplementation of module-tunnel-sink, using libpulse instead of reinventing the wheel by using their own implementation of the Pulseaudio protocol. At some point in the future, they will lose their -new suffix and officially replace module-tunnel-{sink,source} – at that moment your setup may break until you rename them in your /etc/pulse configuration to module-tunnel-sink and module-tunnel-source… But that is far in the future – for today it is all about module-tunnel-sink-new and module-tunnel-source-new !

Now let’s configure this ! Remember that we configured the Raspberry Pi’s Pulseaudio daemon in system mode ? That means the relevant configuration is in /etc/pulse/ (not in /etc/pulse/ – leave it alone, it is for the desktop users). So add those two load-module lines to the bottom of /etc/pulse/ – the first one to declare the IP addresses authorized to use the service, the second one to declare the IP address of the client that will use it… Yes – it is a bit redundant, but that is the way (two single load-module lines – don’t mind the spurious carriage return caused by this blog’s insufficient width):

load-module module-native-protocol-tcp auth-ip-acl=;2001:470:ca99:4:21b:21ff:feaa:99c9

load-module module-tunnel-source-new server=[2001:470:ca99:4:21b:21ff:feaa:99c9]

It is possible to authenticate the client more strictly using a cookie file, but for my domestic purposes I decided that identification by IP address is enough – and lets leave some leeway for my daughters to have fun discovering that, spoof it and stream crap to the livingroom.

Also, as some of you may have noticed, this works with IPv6, but it works well with legacy IPv4 too – in which case the address must not be enclosed in brackets.

Anyway, don’t forget to ‘systemctl restart pulseaudio’ after configuring.

Then on the client side, add this single load-module line to the bottom of /etc/pulse/ (not in /etc/pulse/ – leave it alone, it is for headless endpoints and your client side is most probably an interactive X session). This is one single load-module line – don’t mind the spurious carriage return caused by this blog’s insufficient width:

load-module module-tunnel-sink-new server=[2001:470:ca99:4:ba27:ebff:fee2:ada9] sink_name=MyRemoteRaspberry

Actually I didn’t use sink_name, but I understand you might want to designate your network sink with a friendly nickname rather than an IPv6 address – though why would anyone not find those lovely IPv6 addresses friendly ?

Anyway, log out of your X session, log back in and you’re in business… You have a new output device waiting for you in the Pulseaudio volume control:

Pulseaudio remote device selection

So now, while some of your sound applications (such as the sweet Clementine music player pictured here) plays, you can switch it to the remote device:

Using Pulseaudio volume control to choose a remote device to stream a local playback to.

That’s all folks – it just works !

While you are joyously listening to remote music, let’s have a word about sound quality. As any sound circuit integrated on a motherboard where it cohabits with a wild bunch of RF emitters, the Raspberry Pi’s sound is bad. The Model B+ claims “better audio – the audio circuit incorporates a dedicated low-noise power supply” but actual testing shows that it is just as bad and sometimes even worse. So I did what I nowadays always do to get decent sound: use a cheap sound adapted on a USB dongle, in the present case a ‘Creative Sound Blaster X-FI Go Pro’ which at 30€ gets you a great bang for the buck.

Luck has it that the Raspberry Pi’s Pulseaudio offers it as the default sink – so I did not have to specify anything in my client configuration. But that may or may not be the case on yours – in which case you must use module-tunnel-sink-new’s sink parameter to tell it which sink to use. Since the Raspberry Pi’s Pulseaudio runs in system mode, you won’t be able to ‘pactl list sinks’ so you’ll have to detour through a run in debug mode to know the name of the sinks available.

Sound quality is also a reason why this method is better than a really long stereo line extension cord whose attenuation would degrade sound noticeably.

Well, that was verbose and long winded – but I hope to have clearly covered everything… If I there is anything you feel I should explain better, please ask questions in the comments !

Networking & telecommunications and Systems administration28 May 2015 at 14:01 by Jean-Marc Liotier
1  15.895697 2600:480e:4000:c00::3 -> 2001:470:1f12:425::2 DNS 94
  Standard query 0x896c  A
2  15.901855 2600:480e:4000:c00::7 -> 2001:470:1f12:425::2 DNS 94
  Standard query 0xe3e6  A Www.RuWEnzoRi.neT
3  16.557423 2600:480e:4000:c00::7 -> 2001:470:1f12:425::2 DNS 93
  Standard query 0x5040  A KiVu.grabEuH.COm
4  16.566121 2600:480e:4000:c00::3 -> 2001:470:1f12:425::2 DNS 93
  Standard query 0x9c91  A KIVU.grabeUH.cOM
5  17.211708 2600:480e:4000:c00::9 -> 2001:470:1f12:425::2 DNS 94
  Standard query 0x7b36  AAAA
6  17.888244 2600:480e:4000:c00::9 -> 2001:470:1f12:425::2 DNS 93
  Standard query 0xc582  AAAA
7  18.041786 2600:480e:4000:c00::7 -> 2001:470:1f12:425::2 DNS 93
  Standard query 0xcb72  AAAA Kivu.GRABEUh.coM

Well… WTF ? Who let the script kiddies out ? No one… Surprisingly: those are actually perfectly well-formed queries, using “0x20 Bit encoding“.

This technique was introduced in a 2008 paper, “Increased DNS Forgery Resistance Through 0x20-Bit Encoding – SecURItY viA LeET QueRieS” :

“We describe a novel, practical and simple technique to make DNS queries more resistant to poisoning attacks: mix the upper and lower case spelling of the domain name in the query. Fortuitously, almost all DNS authority servers preserve the mixed case encoding of the query in answer messages. Attackers hoping to poison a DNS cache must therefore guess the mixed-case encoding of the query, in addition to all other fields required in a DNS poisoning attack. This increases the difficulty of the attack”.

For example, Tor uses it by default for name lookups that a Tor server does on behalf of its clients.

Of course, this clever exploitation of a fortuitous behaviour did not go without inducing bugs… What a surprise !

Well… One less mystery.

Debian and Networking & telecommunications and Systems administration13 May 2015 at 13:34 by Jean-Marc Liotier

Upon reboot after upgrading yet another Debian host to sweet Jessie, I  was dismayed to lose connectivity – a slight annoyance when administering through the Internet. Later, with screen & keyboard attached to the server, I found that the Intel Ethernet interface using the e1000e module had not come up on boot… A simple ‘ip link set eth0 up’ fixed that… Until the next reboot.

/etc/network/interfaces was still the same as before upgrade, complete with the necessary ‘auto eth0′ line present before the ‘iface eth0 inet static’ line. And everything was fine once the interface had been set up manually.

Looking at dmesg yielded an unusual “[    1.818847] e1000e 0000:00:19.0 eth0: Unsupported MTU setting” – strange, considering I had been using a 9000 bits MTU without issue before… That error message let me to the cause of my problem: the driver maintainer chose that from kernel 3.15 onwards, calculation of the Ethernet frame’s length always takes into account the VLAN header, even when none is present… And I was running Linux 3.16:

diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index d50c91e..165f7bc 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -5687,7 +5687,7 @@ struct rtnl_link_stats64 *e1000e_get_stats64(struct net_device *netdev,
 static int e1000_change_mtu(struct net_device *netdev, int new_mtu)
     struct e1000_adapter *adapter = netdev_priv(netdev);
-    int max_frame = new_mtu + ETH_HLEN + ETH_FCS_LEN;
+    int max_frame = new_mtu + VLAN_HLEN + ETH_HLEN + ETH_FCS_LEN;
     /* Jumbo frame support */
     if ((max_frame > ETH_FRAME_LEN + ETH_FCS_LEN) &&

As the author remarked: “The hardware has a set limit on supported maximum frame size (9018), and with the addition of the VLAN_HLEN (4) in calculating the header size (now it is 22) , the max configurable MTU is now 8996″.

So there…

diff --git a/network/interfaces b/network/interfaces
index ee4e27d..a094569 100644
--- a/network/interfaces
+++ b/network/interfaces
@@ -7,7 +7,7 @@ iface lo inet loopback

 auto eth0
 iface eth0 inet static
-       mtu 9000
+       mtu 8996

And a reboot later the host is still connected – problem solved. Now to avoid fragmentation I’ll have to set a few other hosts’ MTU to 8996 too… Damn.

Networking & telecommunications and Radio and Systems and Technology04 Feb 2015 at 23:24 by Jean-Marc Liotier

I was so happy with my pair of Baofeng UV-B6 that I decided to buy four more to entirely replace my fleet of even cheaper Lidl Silvercrest TwinTalker PMR transceivers whose horrendous attrition hints about excessive cheapness.

Alas, as I was using the beloved CHIRP to load them with the family’s standard configuration, I encountered the dreaded ‘Radio did not Ack Programming Mode‘ error.

I was using the USB serial adapter with ID 067b:2303 “Prolific Technology, Inc. PL2303 Serial Port” with of course the Baofeng/Kenwood/etc. specific twin 2.5mm/3.5mm plug.

Some of those Baofeng UV-B6 worked fine with this cable – those are UV-B6 with 29 menu entries (with serial numbers 10B6014828 and 10B6014839)

But others were entirely recalcitrant, with a consistent error pattern – those are UV-B6 with 27 menu entries (with serial numbers 10B6025976, 10B6025999, 10B6026018 and 10B6026047).

As suggested by Miklor I slightly trimmed the plug with a cutter – no change.

I also used a couple of male/female extension cords (2.5mm and 3.5mm) in case the lack of twin plastic molding would provide unimpeded contact – no change either.

I bought two different other cables – they both turned out to also be PL2303 serial adapters with same USB ID (but with different plastic moldings – and of course different commercial names). Still no change – same frustrating results.

My last hope was to get this cable, which turned out having USB ID 0403:6001 “Future Technology Devices International, Ltd FT232 USB-Serial (UART) IC”. The ‘Genuine‘ qualifier in its name and the photocopied sheet that attempted to pass for documentation by merely pointing to Miklor were par for the course and did not inspire me to expect anything different… But actually – it worked ! This is the legendary ‘FTDI’ cable I was reading about, the real thing, the one that works with all Baofeng UV-B6 sub-models. Was I not a militant atheist, I would certainly consider this as a proof of God’s greatness – الله أكبر and all those sorts of things !

Cables with a FTDI chip work with both 27 and 29 menu entries UV-B6 submodels
– Cables with a PL2303 chip only work with 29 menu entries UV-B6

There is still a non-zero probability that all the PL2303 chips I went through were counterfeit and that only the FTDI model was genuine – thus voiding my analysis. But with a sample of three PL2303-based cables from three different vendors, that probability is low enough for me to publish this article. A driver issue is not entirely impossible either – I have only tested with Linux, where both PL2303 and FTDI drivers are part of the standard kernel.

By the way, how does one manage a mixed fleet of 27 and 29 menu entries UV-B6 submodels with CHIRP ? Well – easy:

– If you upload an image originaly downloaded from a 29 menu entries submodel to a 27 menu entries submodel, CHIRP will give you the following error message: “An error has occurred – Radio NAK’d block at address 0x0f10” but you can disregard this message as it only concerns the non-existent menu items – the rest of the configuration has been perfectly transmitted.

– If you you upload an image originaly downloaded from a 27 menu entries submodel to a 29 menu entries submodel, no error occurs – but companding will be disabled. No problem.

Now I would be grateful if someone could explain the interoperability of the companding feature – is it still useful if it is active in only one of the two transceivers involved in a given transmission ?

Uhhh… Anyone wants three PL2303-based cables ? I’ll sell them real cheap !

Edit: also used the FTDI cable successfully with the Pofung UV-B5.

Politics21 Jan 2015 at 11:46 by Jean-Marc Liotier

This drawings say ‘The Coran/Charlie Hebdo is crap: it doesn’t stop bullets !’ – a 16 year old in Nantes has been arrested for publishing the one on the left with the comment ‘ironic’. The drawing on the right is the actual Charlie Hebdo cover it satirizes. I also find that the caricature is indeed ironic – I even laughed when it was published hours after the massacre. So here it is, published on my account – now arrest me: those twin pictures are a wonderful opportunity to explain that freedom of speech must apply equally to all. Streisand me !

Politics20 Jan 2015 at 0:06 by Jean-Marc Liotier

Of course, the righteous nations of the west would never assassinate journalists, except of course when they do – but, you know, it is different because they support an evil regime. In a statement of 8 April 1999, NATO indicated that the Radio Television of Serbia studios in Belgrad would be targeted unless they broadcast 6 hours per day of Western media reports: “If President Milosevic would provide equal time for Western news broadcasts in its programmes without censorship 3 hours a day between noon and 1800 and 3 hours a day between 1800 and midnight, then his TV could be an acceptable instrument of public information. The RTS station was bombed on 23/4/99 – sixteen employees died. Nobody was held accountable for the attack.

Yes, Milosevic was busy with war crimes. Yes, RTS was instrumental to propaganda supporting them.

So, to recapitulate: journalists who support a government that is criminal according to your laws are fair game – especially if your laws are universal laws that obviously apply to all humanity (such as the Universal Declaration of Human rights or the rulings of your favourite religious leaders). Oh noes you say – human rights are natural rights and supporting them is entirely different to any religious creed. Are you sure ? At the very least it makes for an interesting philosophical debate.

The Web13 Jan 2015 at 21:43 by Jean-Marc Liotier

Wading among this blog’s moderation queue, I found that someone’s spam comment generator malfunctioned – instead of the intended randomized comment it posted the template… So here is the template for the curious – rather primitive… I expected proper spammers to use some Marko chains generator and I got this. I’m offended to receive comment spam from such unsophisticated people.

Full text of the template below…


France and Politics11 Jan 2015 at 12:51 by Jean-Marc Liotier

Yes, Charlie would probably have loathed today’s gathering, but he isn’t why I’m going – I don’t really care about him, I never bought its paper and probably never will. However I do care about freedom of speech – I have been annoying people about it for the last twenty years and will keep doing it for at least twice that. I care even more that this is it about the practical exercise of freedom of speech in the country where I chose to live. Hypocritical politicians be damned, see you in Paris this afternoon so that we can count ourselves ! I’ll be silent, especially if someone attempts singing the Marseillaise – now is not the time for warlikeness and our interior minister is an idiot for calling ’war on terror’ : even the British government announced publicly eight years ago that it was abandoning the use of that phrase as they found it to be less than helpful… Anyway, let’s celebrate freedom of speech with Benjamin Netanyahu, Sergueï Lavrov, Ahmet Davutoglu, Ali Bongo and Viktor Orban to remind ourselves that it was all really about satire…

Politics12 Nov 2014 at 12:33 by Jean-Marc Liotier

On HN, Fermigier mentioned that 18 months ago, Genevieve Fioraso, French Minister of Higher Education and Scientific Research said : “Given the high stakes – scientific, economic and societal – the French government reaffirms its support for the principle of open access to scientific information“. Nice – our government seems to get it.

Today, France signs a five-year national deal with Elsevier. How much more hypocritical can politics get ?

It is especially sad as the Dutch are meanwhile taking a stand by embracing open access.

Brain dump and France and Politics17 Oct 2014 at 0:07 by Jean-Marc Liotier

A nation of destitute street boys and peasants, gone bare-feet with old rifles to war against every one of their neighbors, led by visionaries with statements such as “terror is nothing else than justice, prompt, severe, inflexible; it is thus an emanation of virtue“, stained with the blood of all the innocents they beheaded but strong with their belief in ideas that scare all the world’s governments. The French (circa 1793).

Nothing to do with the IS of course, though I  would love members of the IS to read my posts – they would be utterly outraged at being confused with people who fought in the name of the Declaration of the Rights of Man and of the Citizen !

Brain dump and Politics15 Oct 2014 at 11:42 by Jean-Marc Liotier

Queen of HeartsCitizen of a country whose current regime was founded on the corpses of 16594 beheaded people, during a period known as The Terror, I feel uniquely qualified to comment on terrorist beheadings.

Not that I usually need an excuse to open my big mouth, but lets not pass on this excellent one to indulge in some punditry !

Guess why Eugen Weidmann’s guillotine execution on the 17th June 1939 was the last one the French performed in public ? Unbeknownst to Parisian prison officials, a film camera had been set up in one of the apartments overlooking the scene

The public was scandalized by their own violence; the government embarrassed. In response France banned public executions. Weidmann went down in history as the last man in France to be guillotined for the entertainment of the awaiting crowd (a dubious distinction).

The government did not find fault in the grisly execution itself—of course it couldn’t have, that would have been an admission of justice’s guilt—rather it blamed the so-called unruly behavior of the savage crowd. The spectacle of bloodlust was, apparently, too powerful for film. Public guillotining was hidden behind the confines of the prison wall—privatized to conceal the spectacle.

Today, we still sentence to death, but we make sure the gore stays out of sight. As one HN commenter put it:

In some ways, the U.S. has done to executions and automated foreign assassinations what the supermarket has done to eating meat. We are distanced from the act so that we aren’t overly burdened thinking about about what is done in our names, both as citizens and voters. Hence, we do not oppose something that we normally would, were we only more aware of it.

Not having to wipe bloody bone shards and bits of blasted flesh from their tablet’s screen certainly is among the reasons why people outraged at the beheading of innocent on video still tolerate remote airborne executions of no less innocent people.

Michael Leuning sums it best:
Michael Leunig's Beheadings
So ? What do the French, the Saudi and the Queen of Hearts have in common ? They knows better than beheading people in public – it is just a basic matter of marketing communications management.

Military and Politics29 Jul 2014 at 21:36 by Jean-Marc Liotier

“And even if the Jews were to win the war, its end would find the unique possibilities and the unique achievements of Zionism in Palestine destroyed. The land that would come into being would be something quite other than the dream of world Jewry, Zionist and non-Zionist. The ‘victorious’ Jews would live surrounded by an entirely hostile Arab population, secluded into ever-threatened borders, absorbed with physical self-defense to a degree that would submerge all other interests and activities. The growth of a Jewish culture would cease to be the concern of the whole people; social experiments would have to be discarded as impractical luxuries; political thought would center around military strategy…. And all this would be the fate of a nation that — no matter how many immigrants it could still absorb and how far it extended its boundaries (the whole of Palestine and Transjordan is the insane Revisionist demand)–would still remain a very small people greatly outnumbered by hostile neighbors.

Under such circumstances… the Palestinian Jews would degenerate into one of those small warrior tribes about whose possibilities and importance history has amply informed us since the days of Sparta. Their relations with world Jewry would become problematical, since their defense interests might clash at any moment with those of other countries where large number of Jews lived. Palestine Jewry would eventually separate itself from the larger body of world Jewry and in its isolation develop into an entirely new people”

The Jewish Writings‘ by Hannah Arendt is a book of collected works published in 2007 – in this extract she referred to the war of independence in 1948 but the moral degenerateness of living by the sword excluding any other mean is still fresh in the current context.

Free software and Politics and Technology29 May 2014 at 9:35 by Jean-Marc Liotier

I stumbled upon a cute potted guide to open source history and found this paragraph interesting:

Software writers in the 1980s liked to talk about how object technology would be the silver bullet that allowed re-use and composition of software systems, moving programming from a cottage industry where everyone makes everything from scratch to a production-line enterprise where standard parts fit together to provide a base for valuable products. It wasn’t; the sharing-required software license was.

I feel that the author is using object oriented software modeling as a strawman, but his point still stands: the critical enabler of modern software is not technical, it is political.

I would go even further and argue that the critical enabler of modern technology is not technical, it is political – intellectual property law is but one egregious example of how political trumps technical in terms of impact… Technical is essential, but though it may subvert a system, it does not overcome oppression on its own.

So political apathy as shown by staggering voter abstention in the latest European elections has immediate technological impact. Political involvement is not futile – it is actually required for technological progress… Get political  !

Technology28 May 2014 at 10:55 by Jean-Marc Liotier

This passage from “Beacons, marketing and the neoliberal logic of space, or: The Engelbart overshoot” eloquently captures the displacement of pioneer ideals from media attention, replaced by the cult of the gold rushers:

There was a powerful dream that sustained (and not incidentally, justified) half a century’s inquiry into the possibilities of information technology, from Vannevar Bush to Doug Engelbart straight through to Mark Weiser. This was the dream of augmenting the individual human being with instantaneous access to all knowledge, from wherever in the world he or she happened to be standing at any given moment. As toweringly, preposterously ambitious as that goal seems when stated so baldly, it’s hard to conclude anything but that we actually did achieve that dream some time ago, at least as a robust technical proof of concept.

We achieved that dream, and immediately set about betraying it. We betrayed it by shrouding the knowledge it was founded on in bullshit IP law, and by insisting that every interaction with it be pushed through some set of mostly invidious business logic. We betrayed it by building our otherwise astoundingly liberatory propositions around walled gardens and proprietary standards, by putting the prerogatives of rent-seeking ahead of any move to fertilize and renew the commons, and by tolerating the infestation of our informational ecology with vile, value-destroying parasites. These days technical innovators seem more likely to be lauded for devising new ways to harness and exploit people’s life energy for private gain than for the inverse.

In fact, you and I now draw breath in a post-utopian world — a world where the tide of technical idealism has long receded from its high-water mark.

Design and Knowledge management and Politics and Security and Technology26 May 2014 at 14:07 by Jean-Marc Liotier

Skimming an entirely unrelated article, I stumbled upon this gem:

Recently, a number of schools have started using a program called CourseSmart, which uses e-book analytics to alert teachers if their students are studying the night before tests, rather than taking a long-haul approach to learning. In addition to test scores, the CourseSmart algorithm assigns each student an “engagement index” which can determine not just if a student is studying, but also if they’re studying properly. In theory, a person could receive a “satisfactory” C grade in a particular class, only to fail on “engagement

This immediately reminded me of Neal Stephenson’s 1992 novel, Snow Crash where a government employee’s reading behavior has been thoroughly warped into simulacrum by a lifetime of overbearing surveillance:

Y.T.’s mom pulls up the new memo, checks the time, and starts reading it. The estimated reading time is 15.62 minutes. Later, when Marietta does her end-of-day statistical roundup, sitting in her private office at 9:00 P.M., she will see the name of each employee and next to it, the amount of time spent reading this memo, and her reaction, based on the time spent, will go something like this:
– Less than 10 min.: Time for an employee conference and possible attitude counseling.
– 10-14 min.: Keep an eye on this employee; may be developing slipshod attitude.
– 14-15.61 min.: Employee is an efficient worker, may sometimes miss important details.
– Exactly 15.62 min.: Smartass. Needs attitude counseling.
– 15.63-16 min.: Asswipe. Not to be trusted.
– 16-18 min.: Employee is a methodical worker, may sometimes get hung up on minor details.
– More than 18 min.: Check the security videotape, see just what this employee was up to (e.g., possible unauthorized restroom break).

Y.T.’s mom decides to spend between fourteen and fifteen minutes reading the memo. It’s better for younger workers to spend too long, to show that they’re careful, not cocky. It’s better for older workers to go a little fast, to show good management potential. She’s pushing forty. She scans through the memo, hitting the Page Down button at reasonably regular intervals, occasionally paging back up to pretend to reread some earlier section. The computer is going to notice all this. It approves of rereading. It’s a small thing, but over a decade or so this stuff really shows up on your work-habits summary.

Dystopian panoptical horrors were supposed to be cautionary tales – not specifications for new projects…

As one Hacker News commenter put it : in the future, you don’t read books; books read you !

Post-scriptum… Isn’t it funny that users don’t mind being spied upon by apps and pages but get outraged when e-books do ? It may be because in their minds, e-books are still books… But shouldn’t all documents and all communicated information be as respectful of their reader as books are ?

Next Page »