Systems administration archived articles

Subscribe to the RSS feed for this category only

Systems administration and Unix09 Mar 2010 at 17:18 by Jean-Marc Liotier

Solid state drives provide incredible IOPS compared to hard disks. But the consideration of cost rules them out as primary mass storage. But for most applications you would not consider storing everything in RAM either – yet RAM cache is part of any storage system. Why wouldn’t we take advantage of Solid state drives as an intermediary tier between RAM and hard disks ? This reasoning is what hierarchical storage management is about, but Sun took it one step further by integrating it into the file system as ZFS‘s Hybrid Storage Pools.

You can read a quick overview of Hybrid Storage Pools in marketing terms, but you will surely find Sun’s Adam Leventhal’s presentation more substantial as a technical introduction. And most impressive are Sun’s Brendan Gregg’s benchmarks showing 5x to 40x IOPS improvement !

Adding SSD to a ZFS storage pool is done at to locations : the ZFS intent-log (ZIL) device and the Second Level Adaptive Replacement Cache (L2ARC). Usually they are set on two separate devices, but Arnaud from Sun showed that they can share a single device just fine.

The ZIL, also known as Logzilla accelerate small, synchronous writes. It does not require a large capacity. The L2ARC, also known as Readzilla accelerates reads. For the gory details of how Logzilla and Readzilla work, Sun’s Claudia Hildebrandt’s presentation is a great source.

Creating a ZFS storage pool with one or more separate ZIL devices is dead easy, but you then may want to tune your system for performance. It costs some DRAM to reference the L2ARC, at a rate proportional to record size – between 1/40th and 1/80th of the L2ARC depending on the tuning (I have seen several different estimates) – so don’t set a L2ARC larger than your DRAM affords you.

I hope that this sort of goodness will some day come to Linux through Btrfs, but ZFS provides it right now – and it is Free software too… So I guess that in spite of my religious fervor toward the GPL, my storage server’s next operating system will be a BSD licensed one… Who would have thought ?

Email and Knowledge management and RSS and Social networking and Systems administration and The Web and Unix27 Nov 2008 at 13:17 by Jean-Marc Liotier

Have you tried one more time to convince you parents to switch to web feeds to get updates from the family ? Do you cringe when you see your colleague clumsily wade through a collections of sites main pages instead of having them aggregated in a single feed ? Or did your technophobe girlfriend miss the latest photo album you posted ? With a wide variety of source acknowledging that web feeds ans web feeds readers being perceived as too technical, many of us have scaled back this particular evangelization effort to focus it on users ripe for transitionning from basic to advanced  tools.

Breaking through that resistance outright is beyond our power, but we can get around it. Electronic mail is a mature tool with well understood use cases with which even the least competent users feels comfortable thanks to how easily it maps with the deeply assimilated physical mail model. This is why Louis Gray has started mailing Google Reader items to promote the use of that web feed reader. But we can do better than that by building a fully automated bridge from web feed to email.

Our hope for plugging the late adopters into the information feeds is named rss2email. As its name suggests, Aaron Swartz’s GPL-licensed rss2email utility converts RSS subscriptions into email messages and sends them to whatever address you specify. Despite the name, it handles Atom feeds as well, so you should be able to use it with just about any feed you like. And of course rss2email is available from Debian.

The nice introduction to rss2email by Joe ‘Zonker’ Brockmeier is all the documentation you need – and rss2email is so simple that you probably don’t even need that. I now have some of my favorite late adopters each plugged into his custom subset of my regular information distribution feeds. The relevant news stories get mailed to them without me having to even think about it. And the best part is that they now read them !

Social networking and Systems administration and The Web15 Oct 2008 at 13:03 by Jean-Marc Liotier

While FriendFeed is the efficient geek hangout, it is Facebook that provides the social middle ground that bridges the gap between early adopters and the mainstream. What makes Facebook useful is that it attracts mainstream users and entices to publish content – something most of them would never do anywhere else. As Regular Geek puts it : “These are people who thought AOL was the internet“. They only visit a few mainstream sites and to them the Web is almost as read-only as dead tree media. That Facebook manages to turn them painlessly into content producers makes it useful for keeping in touch with everyone who does not know what web feeds are. Be aware that even though Facebook is not a dedicated photo sharing site, it beats them at their own game : with ten billion pictures posted, it is now the site with the most pictures shared.

But with mainstream users comes much cluelessness. Those are the people who mindlessly click the default option on every pop-up dialog in sight and then bring you malware ridden computers for healing while wondering why it is so slow and whether they should buy a new one – all they get from me nowadays is a kind word, an Ubuntu CD and offer to install it. Those users have shown time and time again how the path of least social resistance leads to a torrent of application spam.

Some applications such as are a useful addition to the social framework – among them I particularly like Friend Wheel. Many other are just games or even purely decorative trinkets but they should not be dismissed offhand: they have an important role in evangelizing social tools and in promoting use, and they have the social role of fostering playful interaction that reinforces social links. But if you look below, you’ll probably conclude that the quantity of shiny fluff is a tad overwhelming – this is the list of applications that my Facebook account blocks… And it was gathered in less than two years of activity !

  • Absolut Vodka
  • Addicted to NCIS
  • Addicted to Two and a Half Men
  • A la Antillean
  • Alice Blind Test
  • Amazing Wishlist
  • Animated GIFTS
  • A quel(le) star ressembles-tu le plus physiquement?
  • A quel X-Men ressembles tu ?
  • AREPAS and Venezuelan Food
  • Are you a great lover ?
  • Are You Lucky?
  • Are you Moroccan?
  • Are you romantic?
  • Art
  • ATTACK!
  • aWizard
  • Bathroom Wall
  • Be a Billionaire!
  • Because You’re Special
  • Become Rambo
  • Best Friend Contest
  • Best Match!
  • Best Wishes
  • Birthday Alert
  • Birthday Calendar
  • Birthday Cards
  • Blackjack
  • Bless You
  • Blow A Kiss
  • Books iRead
  • Booze Mail
  • Borat / Ali G Photos, Quotes and Trivia
  • Bowling Buddies
  • BrainFall.com Quiz Results
  • BrewSocial
  • Bubble Town
  • Bumper Sticker
  • Bumper Stickers [Photo Gifts]
  • Call Me on Skype
  • Card for Africa
  • Car IQ
  • Cat Breed Collection
  • Causes
  • Chanel Gifts ?
  • Characteristics
  • Check Your Dudeness
  • Chinese Horoscope
  • Circle of Friends
  • COMINGSOON
  • Comment finirez vous ?
  • Comment s’appellera l’homme de ta vie ?
  • Comparaison
  • Compare People
  • Coolest Friends
  • Coolest Person Contest
  • Crushes
  • Cute vs Sexy
  • Define Me
  • Delux Christmas Tree
  • Denzel Washington
  • De quel arrondissement parisien ètes-vous ?
  • Do people secretly hate you?
  • Drunk Survey
  • ePresident
  • Es-tu fort en histoire ?
  • Etes vous un minimum cultivé ?
  • Etre Marseillais
  • Eurosport – Liste des 23 pour l’Euro 2008
  • Family Guy – Blue Harvest
  • Family Tree
  • Famous Christian Quotes
  • FB Addict – are you hooked on FB?
  • FFR Supporters
  • Fine Wines
  • Flirtable
  • Fortune Cookie
  • Free Gifts
  • Fresh Prince
  • Friend Hug
  • Friends For Sale!
  • Fun Cards!
  • Funnest Person Contest
  • Funny Cards
  • Fun Toys
  • Genius Test
  • Gifts Gallery
  • Good Morning
  • (Green) What fruit are you?
  • Growing Gifts
  • Guerre des gangs
  • Hatching Eggs
  • hello kitty
  • High School Trivia Test
  • Holiday Shoppe (Christmas Tree)
  • Hotness
  • Hot Potato
  • Hottest Person Contest
  • How gangsta are you?
  • How Indian Are You?
  • how smart are u?
  • How stupid are you?
  • How will you die?
  • Hug Me
  • Hug Me
  • Hugs
  • Hug Time
  • Iframer
  • iLike
  • Instant IQ Test
  • IQ Test
  • IQ Test (Advanced Level)
  • is cool
  • iSmile
  • Japanese Foods
  • Japanese Sweets
  • Jedi vs Sith
  • Jetman
  • Jeu de Séduction
  • JungleBook
  • Kisses
  • Kisses!
  • Kiss Me
  • Knighthood
  • Language Exchange
  • Likeness
  • Likeness UNRATED
  • (Lil) Green Patch
  • Local Picks
  • Love Friend
  • LX Champions League
  • LX College Football
  • LX World Cup Football
  • Mario Kart RPG
  • Meet New People
  • Mesmo TV
  • MeteoSun
  • MindJolt Games
  • Mood Ring
  • Most Creative People
  • Most Eligible Singles
  • Most Gorgeous Person Contest
  • Most Wanted Valentine!
  • Mountain Climber
  • Movies
  • My Angels
  • My Aquarium
  • My Boxofun
  • My Drunk Friends
  • MyFlirt
  • My Hebrew Name
  • My Heroes Ability
  • My Music
  • My Personality
  • My Questions
  • MY SEXY FRIENDS
  • MySpace
  • MySpace Link
  • NAB Smart Cookies
  • Name Analyzer
  • NBA Challenge
  • Nicknames
  • Nova Music
  • OneTrack
  • Optical Illusions Challenge
  • OUIFM
  • Owned!
  • Passe Ton BAC !
  • Personality
  • Photo Quizzes
  • Pieces of Flair
  • Pillow Fight
  • Pillow Fight!
  • Pink Ribbon
  • Portrait Chinois
  • Pour quelle boîte Corse es-tu fait(e) ?
  • Pour quelle époque êtiez-vous fait(e)?
  • Pour quelle ville êtes-vous fait(e) ?
  • Pour quelle voiture es-tu fait(e)?
  • PrayLive
  • Premier Football
  • Pro League Rugby
  • PuzzleBee Jigsaw Puzzles
  • Q??l Pa?f?? E? T? ?
  • Que fuyez-vous le plus ?
  • Quel alcoolique êtes vous?
  • Quel chroniqueur du grand journal êtes-vous?
  • Quel écrivain êtes-vous?
  • Quel est ce défaut chez toi qui fait craquer les hommes?
  • Quel est ton degré de connerie??
  • QuEl EsT tOn MeC IdEaL???
  • Quel est ton niveau d’anglais?
  • Quel est ton niveau sexuel ?
  • Quel Festival es-tu?
  • Quel genre de pute es-tu ?
  • Quel joueur de rugby etes vous?
  • Quel joueur du PSG 2007-2008 es-tu ?
  • Quelle citation êtes-vous?
  • Quelle couleur es-tu?
  • Quelle desperate housewife êtes-vous ?
  • Quelle icone glamour es-tu?
  • Quelle ligne de métro êtes vous?
  • Quelle mec te correspond ?
  • Quelle paire de chaussures de créateur êtes-vous ? (pour filles)
  • Quelle princesse de Walt Disney êtes vous?
  • Quelles Vacances VIP es tu?
  • Quel Maman est tu ?
  • Quel mannequin es-tu?
  • Quel méchant de Disney es-tu ?
  • Quel nageur connus es tu ?
  • Quel personnage de desperate housewives es-tu?
  • Quel personnage de FRIENDS es-tu?
  • Quel personnage de KAAMELOTT es-tu ?
  • Quel personnage de la Révolution êtes-vous ?
  • Quel personnage des BRONZES êtes-vous ?
  • quel qualité êtes-vous?
  • Quel séducteur (trice) etes-vous ?
  • Quel sorte d’enfant étais-tu?
  • Quel sous-vêtement êtes-vous?
  • quel star es tu ?
  • Que pensent les autres de toi en secret?
  • Que vas tu faire de ta vie ?
  • Quizzes
  • RAYMOND DEMISSION !!!
  • Rock Paper & Scissors
  • R U CUTE!
  • Save An Alien
  • Say Happy New Year!
  • Say-it-with-Flowers
  • SceneCaster
  • ScoreMe
  • Secret Admirer – CRUSH on ME (PERFECT MATCH)
  • Send Beer
  • Send Chocolate
  • Send Diamonds
  • Send Good Karma
  • Send HOTNESS
  • Send Love Hearts
  • Send Luck
  • Send Sunshine
  • Send Teddy Bears
  • Send Tiaras
  • Send Tux
  • Send Veggie Tales
  • Sexiest Person Contest
  • Sexy Friends
  • Sexy Pillow Fight
  • Sexy Poke!
  • Shots!
  • similaire
  • Six Degrees
  • Sketch Me
  • Skiers vs. Snowboarders
  • Slayers
  • Slide FunSpace
  • Smiles
  • Snowball Fight
  • Snowball Fight!
  • Social Profile
  • Sparkey
  • SpeedDate
  • SpeedDate
  • SpeedDate
  • SpeedDate
  • SpeedDate
  • SpeedDate
  • Sports Fan
  • Status Competition
  • Stickerz
  • Sticky!
  • StyleFeeder
  • Sudoku
  • Suomi-ilmiö
  • Superlatives
  • SuperPoke!
  • Super Slot Machines
  • Super Wall
  • Sweetest Person Contest
  • Tarot
  • Test ton niveau de culture
  • Texas HoldEm Poker
  • The Brain Game
  • The Legend of Zelda Fan
  • The Official 100 Question Geek Test
  • The Sex Compatibility Test
  • The Unofficial Desperate Housewife Quiz
  • The World’s Smallest Political Quiz
  • Top Friends
  • T.O.T. Effect
  • Tower Bloxx
  • Travel Brain
  • Traveler IQ Challenge
  • Trend Setter
  • True Match
  • Truth Box
  • TuneSocial
  • Twirl
  • Twist me!
  • U.S. Citizen Test
  • Vampires
  • Wanna Dance?
  • Water Globe Gifts
  • WC 2010 Euro Predictor
  • We’re Related
  • WereWolves
  • What Beer Are You?
  • What Color Is Your Heart?
  • What Does My Birthday Mean?
  • What Drink Are You?
  • What flower are you?
  • What football player are you?
  • What Is Your Ideal Job?
  • What is Your Secret Sexual Fantasy?
  • What is Your Supermodel Personality?
  • What kind of candy are you?
  • What Kind of Cat Would You Be?
  • What kind of hair best suits you? (for girls)
  • What Kind of Mom Will You Be?
  • What Lost Character Are You?
  • What Mythological Creature Are You?
  • What serial killer are you?
  • What song are you?
  • What’s Your Stripper Name?
  • Whats you true name? (Girls only)
  • What type of person do you attract?
  • What type of warrior are you?
  • When will you get married?
  • Which Disney Princess Are You?
  • Which Disney Song Describes Your Life Right Now?
  • Which Fashion Designer Would You Be?
  • Which Festival best suits you??
  • which F.R.I.E.N.D.S character are you???
  • Which Friends Character Are You?
  • Which Grey’s Anatomy Character Are You?
  • Which Hot Celeb Are You?
  • Which Rockstar Are You?
  • Which Sex and the City Character Are You?
  • Which Simpsons Character Are You?
  • Which WaterAid Country Are You?
  • Who Has The Biggest Brain?
  • Who is Watching You ?
  • Who’s Online
  • Who’s the Coolest Cat?
  • Will you KISS me?
  • Winnie the Pooh
  • Word Challenge
  • Word Twist
  • YES or NO?
  • Your Birthday
  • You’re a Hottie
  • You’re Cute
  • Your Sexyness
  • YouTube Video Box
  • Zombies

You too can escape this minor pantheon of horror – and you can do it without the tedium of blocking each application as it spams your Facebook newsfeed. As you may know, Greasmonkey enables the customization through JavaScript of the way a webpage displays. Auto-Block Facebook Apps is a Greasmonkey script that will block application invitations sent to you by Facebook contacts. After the facebook profile page is loaded, it finds all the applications that your friends have invited you to and blocks them. Whenever you need you can go to the Facebook applications privacy controller and unblock the ones that you find somewhat valuable. With applications spam out of the way, Facebook will remain the neat social watering hole that makes it valuable for interaction with non-geeks.

Jabber and Systems administration and VOIP05 Jun 2008 at 23:59 by Jean-Marc Liotier

Since version 1.4, a Jabber module is available in Asterisk. If you know me, then you probably wonder why it took me that long to discover it. I began playing with it tonight and the short story is that it works, it is simple to configure and it makes telephony aware of presence.

Of course this is still far from the holy grail of a presence-centered converged synchronous communications platform, but it is a start and anything is better than today’s stupid mass-market telephony.

Here is my /etc/asterisk/jabber.conf :

[general]
;Auto register users from buddy list
autoregister=yes
;Jabber service label
[asterisk]
type=client
serverhost=jabber.grabeuh.com
username=asterisk@jabber.grabeuh.com/kisangani
secret=my_password
port=5222
usetls=yes
statusmessage="Watching the telephone"

Yes, that’s it : Asterisk is now registered as a Jabber client. There are other ways to do it using external modules, but this one is the simplest – and since it is now part of the main trunk it is probably the most stable. In particular you should be careful with class.jabber.php which is not maintained anymore and in the process of being replaced with the much more modern XMPPHP.

Now let’s declare a macro for taking full advantage of that. This one takes a look at the user’s presence and routes the call accordingly : if he is online or chatty the call goes to his desk phone – otherwise it goes his mobile phone.

[macro-reach_user]
; ${ARG3} is a destination such as SIP/whatever
exten => s,1,jabberstatus(asterisk,${ARG2},STATUS)
; presence in will be 1-6.
; In order : Online, Chatty, Away, XAway, DND, Offline
; If not in roster variable will = 7
exten => s,2,gotoif($[$[${STATUS}]<3]?available:unavailable)
; GotoIf(condition?label_if_true:label_if_false)
exten => s,3(available),jabbersend(asterisk,${ARG2},"Call from
 ${CALLERID(name)} at number${CALLERID(num)} on
 ${STRFTIME(,GMT-1,%A %B %d %G at %l:%M:%S %p)}")
exten => s,4,Dial(${ARG1})
exten => s,5(unavailable),Dial(${ARG3})

Since we have declared a macro, we have to call it in the context of our choice and assign the relevant values to the macro’s variables :

[whatever_context]
; ${ARG1} is the destination when at desk
; ${ARG2} is a jabber address used at desk
; ${ARG3} is the destination when not at desk
exten => 05600047590,1,Macro(reach_user,SIP/jml-desk,
 jim@jabber.grabeuh.com, SIP/freephonie-out/0666758747);
; repeat last line for each user

That’s all folks ! That is all it takes to have your calls routed to the right phone according to your presence status. It is really that easy.

Why no service provider is offering that is beyond me. The big ones are all waiting for the IMS systems they are going to deploy with a five years roadmap. But if you want the future right now there is no need to wait : all the technology is here today waiting for you to play with it ! And of course it is 100% Free software

Code and Systems administration and Unix and VOIP01 Jun 2008 at 18:55 by Jean-Marc Liotier

Ever since the Linux Advanced Routing & Shaping HOWTO introduced it, I have been a big fan of the Wondershaper, a traffic shaping script that drives Linux‘s class based queuing with stochastic fairness queuing (SFQ) in a pretty effective attempt at maintaining low latency for interactive traffic while at the same time maintaining high throughput. There is even a ‘wondershaper’ Debian package that includes some additional polish. This script is key to the joy of perfectly responsive SSH sessions while peer to peer file sharing traffic saturates the uplink.

Some people have even concluded the resulting quality of service is good enough for voice traffic. But even with the Debian Wondershaper ruling my ADSL link I noticed that SIP and IAX still suffer too much packet loss with the saturating traffic occupying the background. I needed better traffic control.

As usual, being a late adopter I am not the only one to have hit that obstacle, and solutions have already been put forth. After rummaging through various mutations, I found Robert Koch’s version of the Wondershaper for the Asus WL-xxx documented on the Wondershaper package page of the WL-500G wiki to be quite promising. Compared to the standard version it prioritizes VOIP traffic by source port for idiot proof configuration, but also by type of service which is much more flexible and can be used thanks to Asterisk being capable of correctly setting TOS fields. As a bonus, using TOS also makes this version of the script capable of distinction between console interactive SSH traffic and bulk SCP traffic using the same protocol and port. And to top it all, it is based on the better hierarchical token bucket (HTB) discipline which is standard since Linux 2.4.20 while the Debian Wondershaper version uses the more based queuing which used to be the more widespread one.

The first shortcoming I found is that it prioritizes SIP and RTP but not IAX and others which I’ll have to add using the SIP stanzas as templates. The other is that taking lists of low priority ports as arguments could make the command line messy and configuration puzzling for the inexperienced user, so I prefer to have this configuration item as a documented variable allocation inside the script. But those are trifles compared to the new VOIP support, enhanced SSH discrimination and overall upgrade.

Hacking on the script I couldn’t resist reorganizing a few things. I originally intended to provide a diff, but that would be pointless since I ended up touching most of the lines. Also be warned that I do not understand why putting ‘prio 1’ everywhere makes the script work whereas other ‘prio’ values at various places made traffic end up in the wrong class and did not make sense at all. In effect, I think that by putting ‘prio 1’ everywhere I just eschewed the use of priority bands inside the classes, which is just fine with me for the intended use. But this show that my tc fluency is still limited and that there are therefore surely ways to enhance this script. I’ll also welcome feedback – whether it works for you or not.

Anyway – it works ! I had a few VOIP conversations across an IAX trunk with lots of background traffic on the uplink and no perceptible effects on voice quality. Life is good. Now that I have removed the last obstacle to taking full advantage of VOIP at home. Soon all my traffic will be routed through Asterisk and there shall be no more RJ11 nor their French T-sockets alter ego in my home.

Here is my modified wondershaper script in all its glory – contrary to Robert Koch’s version it is a drop-in replacement for Debian’s package. Inheriting from the original Wondershaper it is licensed under the GPL so enjoy, modify and share !

Debian and Systems administration and Unix08 May 2008 at 20:08 by Jean-Marc Liotier

After reading the USP and elevator pitch for Ack, someone took the time to review the hype and make his own opinion about ack. It appears to him that although Ack is not a drop-in replacement for Grep, the DWIM factor makes it a very interesting alternative for many situations where interactive use by a human is involved.

So I decided to give it a spin. ‘apt-cache search ack’ matched 15971 packages related to crack, backup, pack, slack and whatever else you can imagine, but an ‘apt-cache search ack | grep -E ^ack’ solved that problem at once.

In Debian, there was already a kanji code converter named “ack” – but since we don’t intend as far as I know to write Kanji anytime soon I tought it was reasonable to alias ack=’ack-grep’ so that our shell takes advantage of the short name.

After toying with it a little, it appears that Ack is indeed a handy tool which I’ll be definitely using in the future. There is a performance tradeoff when operating with the C locale, but since we strive to be an all UTF-8 shop I don’t care much about that. And anyway, in most interactive situations, brain usage and fingers motricity are far more precious ressources than run time or CPU time…

« Previous Page