Category Geek

GDMF Exchange!

Yet more reason to hate MS Exchange. Here are the relevant headers and MIME lines from a meeting notification I got recently:

Header:

Subject: XXX Staff Meeting
Content-Type: multipart/alternative;
	boundary="_000_A32122FBAE23C145ABBEFDEA852187CE01E77A1505B7BLADEBLA03V_"
MIME-Version: 1.0

Body:

--_000_A32122FBAE23C145ABBEFDEA852187CE01E77A1505B7BLADEBLA03V_
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

…
--_000_A32122FBAE23C145ABBEFDEA852187CE01E77A1505B7BLADEBLA03V_
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: base64

…
--_000_A32122FBAE23C145ABBEFDEA852187CE01E77A1505B7BLADEBLA03V_
Content-Type: text/calendar; charset="utf-8"; method=REQUEST
Content-Transfer-Encoding: base64

…

--_000_A32122FBAE23C145ABBEFDEA852187CE01E77A1505B7BLADEBLA03V_--

At first glance, all might look normal: there’s a calendar entry with a note attached.

So first we have the plain text version of the note, followed by the HTML version of the note, followed by the vCalendar file describing the meeting itself.

But a closer look at the header shows that the message as a whole has Content-Type: multipart/alternative.

RFC 1521 says:

The multipart/alternative type is syntactically identical to multipart/mixed, but the semantics are different. In particular, each of the parts is an “alternative” version of the same information.

Systems should recognize that the content of the various parts are interchangeable. Systems should choose the “best” type based on the local environment and preferences, in some cases even through user interaction.

In other words, any standards-compliant mail reader should see those three MIME chunks as three different versions of the same information. So if it decides to (or you tell it to) display the HTML version of the note, it shouldn’t display the calendar file. And if it displays the calendar entry, it shouldn’t show the attached note.

Clearly somebody at Microsoft needs to be slapped. Hard.

(And in case you’re wondering, the proper way to do what they’re trying to do would be for the message as a whole to be multipart/mixed with a multipart/alternative chunk for the note, and a text/calendar chunk for the calendar entry. The note chunk would be further subdivided into a text/plain chunk and a text/html chunk.)

I think that when people are first told that Exchange is both a mail server and a calendar server, they think it’s kind of like a goose — something that can competently walk, swim, and fly, even though it may not excel at any of those — but in reality, it’s more like a crocoduck: massive fail at every level, no matter how you look at it.

Using snoop/tcpdump as a Filter

Okay, this is kinda cool. Yesterday, I ran snoop (Sun’s version of tcpdump) to help the network folks diagnose a problem we’ve been seeing. Unfortunately, I let it run a bit too long, and wound up with a 1.5Gb file. And the guy who’s going to be looking at this is at a conference, and would rather not download files that big.

Now, I’d known that snoop can dump packets to a file with -o filename and that that file can be read with -i filename; and of course that you can give an expression to say what kinds of packets you want to scan for. But until now, it never occurred to me to put the three of them together. And it turns out that not only does snoop support that, it Does The Right Thing to boot.

Now, one of the reasons I wound up with 1.5Gb worth of packets is that we didn’t know which port the process we were trying to debug would run on, until it ran. (That, and the fact that I started scanning early because I wasn’t sure when it would run. And ending late because the Internet dangled shiny things in front of me.)

At any rate, I was able to run

# snoop -i old-snoop-log -o new-snoop-log host thehost.dom.ain port 50175

and wind up with a packet capture file of manageable size.

And a bit of experimentation showed that tcpdump does the same thing (adjust arguments as appropriate). I’ll have to remember this.

Countdown to Backpedaling Widget

Over on the right, in the sidebar, you should see a countdown clock entitled “Countdown to Backpedaling”. (If not, then something went wrong.)

If you’ve been listening to Ask an Atheist, then you should recognize this as a widget version of the Countdown to Backpedaling clock. And if not, then you should definitely be listening to them. Because they’re cool.

At any rate, it’s a clock that counts down to May 22, the day after Jesus’ return and the Day of Judgment, when the backpedaling and excuses begin.

So anyway, now you want to know a) where to download this, b) how to install it, and c) how to complain to me about all the problems you’ve had with (a) and (b).

Download

The main download page is .

If you’re using WordPress, you can download , put it in your wp-content/plugins directory, and with any luck, a “Countdown to Backpedaling” widget should magically show up in your “Widgets” control panel. You can then drag it into position, and it should work.

If you’re using some other software, you’ll want . Installation depends on what you’re using, of course, but you should be able to insert it anywhere that takes HTML.

Configuration

The main configuration option is the “show_secs” variable at the top. If you want to see seconds in the countdown, set it to true. If you find the seconds’ flashing annoying, set it to false.

You can also look through the CSS part, and edit as you please. You might need to change the width.

I might improve on this, if time permits and I don’t get raptured before getting around to it.

If you have any comments or complaints, leave a comment. Bug reports accompanied by a rum and Coke will get higher priority. Bug reports accompanied by a patch will get even higher priority.

Bourne Shell Introspection

So I was thinking about how to refactor our custom Linux and Solaris init scripts at work. The way FreeBSD does it is to have the scripts in /etc/rc.d define variables with the commands to execute, e.g.,

start_cmd='/usr/sbin/foobard'
stop_cmd='kill `cat /var/run/foobar.pid`'

run_rc_command "$1"

where $1 is “start”, “stop”, or whatever, and run_rc_command is a function loaded from an external file. It can check whether $stop_cmd is defined, and if not, take some default action.

This is great and all, but I was wondering whether it would be possible to check whether a given shell function exists. That way, a common file could implement a generic structure for starting and stopping daemons, and the daemon-specific file could just set the specifics by defining do_start and do_stop functions.

The way to do this in Perl is to iterate over the symbol table of the package you’re looking for, and seeing whether each entry is a function. The symbol table for Foo::Bar is %Foo::Bar::; for the main package, it’s %::. Thus:

while (my ($k, $v) = each %::)
{
	if (defined())
	{
		print "$k is a functionn";
	}
}

sub test_x() {}
sub test_y() {}
sub test_z() {}

But I didn’t know how to do it in the Bourne shell.

Enter type, which tells you exactly that:

#!/bin/sh

# List of all known commands
STD_CMDS="start stop restart status verify"
MORE_CMDS="graceful something_incredibly_daemon_specific"

do_start="This is a string, not a function"

do_restart() {
	echo "I ought to restart something"
}

do_graceful() {
	echo "I am so fucking graceful"
}

for cmd in ${STD_CMDS} ${MORE_CMDS}; do
	if type "do_$cmd" >/dev/null 2>&1; then
		echo "* do_$cmd is defined"
	else
		echo "- do_$cmd is not defined"
	fi
done

And yes, this works not just in bash, but in the traditional, bourne-just-once shell, on every platform that I care about.

So yay, it turns out that the Bourne shell has more introspection than
I thought.

Music Nerd Joke

I’m going to tag all my Cocteau Twins MP3s and, in the TLAN frame, set the language code to zxx.

Pre-Compressing Web Content

This was definitely a “D’oh!” type of problem.

One thing I’d been meaning to figure out for a while was how to send gzip-compressed files to a browser. That is, if I have a large HTML file, it’d be nice if the server could compress it to save bandwith and transmission time. Yes, Apache has mod_deflate which takes foo.html and gzips it on the fly, setting all the appropriate HTTP headers. But for static content, I should just be able to compress the file in advance. If the browser asked for foo.html, I wanted Apache to see that there’s a foo.html.gz and send that instead, with headers saying that it’s a text/html file that happens to be compressed.

mod_mime seemed like just the thing: just add

AddEncoding x-gzip .gz

to .htaccess. But every time I did that, Apache sent back “Content-Type: application/x-gzip“, so my browser treated it as a random file of unknown type that happened to be compressed.

Then I noticed that my vanilla-ish site-wide Apache config had

AddType application/x-gzip .gz .tgz

so that when Apache saw foo.html.gz, it ignored the .html extension, and saw only the .gz one.

The fix was to add RemoveType to my .htaccess:

RemoveType .gz
AddEncoding x-gzip .gz

And voilà! .gz stops being a file type and becomes an encoding, allowing .html to shine through.

I’ll add that this plays nice with AddLanguage as well. In my test setup, I have foo.html.en.gz, for which Apache returns the headers

Content-Type: text/html
Content-Encoding: x-gzip
Content-Language: en

I.e., it’s an HTML file, it’s gzip-encoded, and it’s in English.

Just as importantly, this works with other file types (e.g., CSS files and JavaScript scripts), and XMLHttpRequest does the Right Thing with them on all of the browsers I care about.

Evil Hack of the Day

MacOS plist XML files are evil; even more so than regular XML. For instance, my iTunes library file consists mostly of entries like:

<key>5436</key>
<dict>
	<key>Track ID</key><integer>5436</integer>
	<key>Name</key><string>Getting Better</string>
	<key>Artist</key><string>The Beatles</string>
	<key>Composer</key><string>Paul McCartney/John Lennon</string>
	<key>Album</key><string>Sgt. Pepper's Lonely Hearts Club Band</string>
	…
</dict>

You’ll notice that there’s no connection between a key and its value, other than proximity. There’s no real indication that these are fields in a data record, and unlike most XML files, you have to consider the position of each element compared to its neighbors. It’s almost as if someone took a file of the form

Track_ID = 5436
Name = "Getting Better"
Artist = "The Beatles"
Coposer = "Paul McCartney/John Lennon"

and, when told to convert it to XML in the name of buzzword-compliance, did a simple and quarter-assed search and replace.

But of course, what was fucked up by (lossless) text substitution can be unfucked by text substitution. And what’s the buzzword-compliant tool for doing text substitution on XML, even crappy XML? XSLT, of course. The template language that combines the power of sed with the terseness of COBOL.

So I hacked up an XSLT template to convert my iTunes library into a file that can be required in a Perl script. Feel free to use it in good or ill health. If you spring it on unsuspecting developers, please send me a photo of their reaction.

dirvish + MySQL

I’ve been using dirvish to do backups at home. One problem I’ve always had is how to back up MySQL databases, since the backup takes a snapshot of the binary files, so if it happens at the wrong time (e.g., in the middle of a transaction), the backup might wind up being in an unusable, inconsistent state. Much better to use mysqldump to export the database to a file (one which, by the way, can also be manipulated with standard tools like perl and emacs, in case I need to repair anything).

The obvious way to do this was to use the pre-client dirvish directive to export to a file, then let that file be backed up. But I could never get it to work.

Cut to the chase: It turns out that for this to work, the pre-client directive (and post-client, if you want to clean up afterward) needs an extra semicolon:

pre-client: ; /usr/local/bin/mysqldump -a -A -e > $DIRVISH_SRC/mysqldump.%Y-%m-%d.sql
post-client: ; /bin/rm $DIRVISH_SRC/mysqldump.%Y-%m-%d.sql

Without those semicolons, things don’t run correctly. My guess is that the semicolon tricks dirvish into thinking that the command consists of multiple commands, which must therefore be run inside a shell, rather than a single command to be executed with fork()/exec().

Another Useful Keybinding

I just found out that the mutt mailreader accepts ~Lexpr in searches. This matches messages where expr appears either as a sender or as a recipient.

Every so often, I need to look for a mail exchange I’ve had with a particular person, or where two people have corresponded and Cc-ed me. So I’d limit to

~farensb ~C(pat|chris) || ~f(pat|chris) ~Carensb

(messages (from arensb and (to or cc-ed) to (pat or chris)) or from (pat or chris) and (to or cc-ed) to arensb, for those who don’t speak mutt-regex).

Now that I know about ~L, the above can be simplified down to

~Larensb ~L(pat|chris)

(messages (from or to or cc-ed to) arensb and (from or to or cc-ed to) (pat or chris)).

My tunnelled carpals thank you, Mutt developers!

Firefox: Reopen Last Tab

Since M. didn’t know about this the other day, I thought I’d mention it, in case it helps someone else: Firefox has a massively-useful feature I’ve been using all the time: reopen the last tab that was closed.

You can find it under History → Recently closed tabs. There’s a Recently closed windows list to go with it.

Reopen last tab is bound to Alt-Shift-T for me, though your mileage may vary, since I’ve adjusted Firefox’s bindings to suit my modifier keys (what’s a Meta for?).

I should also add that Alt-1 is bound to “Go to 1st tab”, Alt-2 to “Go to 2nd tab”, and so forth. I sometimes find this to be quicker than moving my hand off of the keyboard to click on a tab.