Monday, November 2, 2009

Formatting Tool Output for Reports

If you're like me, you use cron to mail you a variety of stats generated during off-peak ours.

You may, at some point, want to start extrapolating patterns from the data you've collected. In my experience, nothing really gets the job done in this scenario quite like a spreadsheet.

That being said, the data generated by most tools is reader-friendly, not import-friendly. Therefore, they must be munged in order to add them to your data set.

Most of them use tabs to do their spacing/formatting. I have written a quick sed hack that removes one or more instances of a tab from a line and converts it to a comma. The end result is essentially a CSV.

So, to format the output of a

du -sm /home/*

you could pipe it through sed and into a mail command.

Ie:

du -sm /home/* | sed -e '/txt/d;s/\t\t*/,/' | mail -s "Usage" nathan@mccourtney.com

the '-e' lets it know that there is more than one command being passed - separated by a ';'.

In this case, I'm using the first part of the command to delete every line with 'txt' in it, since I'm really not worried about the text files. The second part matches at least one tab and all the tabs following it, then replaces them with a single ','. The output it then piped to a mail command and sent to my account.

When it shows up, the output goes from

nsmc@bangkok:~$ du -sm /home/*
1786 /home/nathan

to

1786,/home/nsmc

It's not terribly pretty, but you can easily import it into a spreadsheet. And really, I use a slightly more complex du script that generates a more readable listing before it even gets to sed. This was intended just to show the sed side of it.