Tools for everyday use - www.py - HTML-enabled echo replacement

If you often have to analyze log files or issue commands that produce lots of plain-text output, consider taking a look at my small utility http://www.py (which lives in htmecho GitHub project).

Its purpose is to capture the input, transform it to an HTML with some additional JavaScript payload and open the result in the default browser. This enriched output provides filtering ability and allows assigning colors to lines that match a particular regular expression.

I use it so often, so that if it didn’t exist I would write one today :) For convenience, I usually place it into ~/bin/www (followed by chmod +x ~/bin/www). On Windows I just ensure that ‘py’ file extension is mapped onto Python 3.2 executable.

For example the following console command:
grep -s -r -e "Error:" -e "Warning:" *.log | www
would open a browser (or add a new tab into an already opened browser window) with the output of the grep.
Here is a screen of
dmesg | www
command with BIOS pattern marked as yellow:

Here is a bit of history that might be of interest to developers.

Few years ago, as I was studying I/O area of the C++ standard library, I wrote a small tool that transformed its STDIN input into a temporary HTML file. Though by that time I had only the educational goal, I found this tool to be surprisingly useful in my working environment – as I often have to analyze logs (typically after some grep operations on them) and due to my dual-monitor setup I find it very convenient to issue some command on one screen and see the results on the other.

The only major change before I stopped working on it was adding a ShellExecute call in the end – so that the newly generated HTML file was opened automatically in the default browser (which must be chrome which is chrome in my case). I just put it into a local SCM and kept it for few years without changes.

Recently, as I often rely on LoadRunner logs analysis while handling support cases, few more ideas came to mind as we discussed this tool with my colleague – what if we enrich the generated HTML file with a bit of JavaScript to allow some grep-like functionality right in the browser window? What if we colorize particular filters, so that it will be easy to visually find the interesting spots in the logs? That gave me enough motivation to invest more time in the project and the colleague helped me with the JavaScript part. I also added an imaginary TODO item to ensure that the tool compiles and works on Linux, as I use it on my laptop. Now, a simple one-file educational application became cross-platform with OS-transparent and OS-agnostic layers. I also made a bit of interesting research on how to embed binary resources into Linux executable modules (as I wanted to simulate Windows Portable Executable Resources behavior which allows trivial embedding of whatever you want into the executable).

Next battle was for Unicode support. Or, to put it correctly, total lack of such support in the language. I tried to avoid WinAPIs for that and to be able to handle windows-1251, UTF-16 BE/LE and UTF-8 encodings – I had to spent few nights reading quite a number of materials on iwfstream and codecvt (not the best reading, imo), which is neither simple to use nor beautiful conceptually. And still, removing BOM-checks in C++ classic iwfstream-based code was a great pain. And, OMG!, after I ensured it works on Windows – it just didn’t work on Linux… though it was a personal free-time development, instead of bringing satisfaction and joy – it was just nasty time waste. Finally, though I’ve got to a stable point with support of cp1251 and UTF8, I just didn’t feel I want to proceed any further.

And here comes Python!

Just for sport interest, I tried to reach the same functionality by using Python 3.2 and total time spent until I got it working on two platforms I’m interested in was around 2 hours of coding and debugging.

Benefits brought by Python:
- single-file executable with no dependencies. Even though Python version had platform-agnostic parts too, they were typically one-liners and there was no reason to move them into separate modules. In C++ I had to implement platform-agnostic (and even linker-agnostic for Linux) resource embedding, cross-platform code to open browser and it naturally resulted in multiple headers/source files.
- ability to modify HTML template with JavaScript payload – again, in Python it is native to have multi-line string literals – so, go on and just edit the script. In C++, it was simple too, but only because HTML template had been intentionally kept as an external resource linked into an executable. On windows, it is possible to modify an embedded resource externally with a tool like Resource Hacker. I don’t have a clue on whether anything as simple is available on Linux.
- Unicode. Python 3.x rulez here: standard distribution of the language has plenty of encodings supported out of the box, BOMs are removed automagically granted you’ve specified the correct encoding. It was pity to conclude that pure C++ with its standard library totally suck at this part.
- Performance – for a tiny tool like this I won’t even bother with runtime performance. Productivity is what matters here and Python is an obvious winner here. I admit it wasn’t clever for me to write such a program in C++ in the first place, but as I said it was an educational project and … I like to code in C++ anyway, at least when it’s not a unicode related code.

About these ads