Category Hacking

Location-Based Themes in Emacs

There are lots of Emacs themes out there. People mostly use them to change colors. That’s great, but you can do more. As the documentation says:

Custom themes are collections of settings that can be enabled or disabled as a unit.

That can be any setting, not just colors.

I use Emacs in multiple situations. I use it at home, I use it in the office, I use it when I’m working from home, and so on. And there are settings that change depending on how and where I’m using Emacs, the most obvious of which being user-mail-address, which I want to set to user@home.org when I’m sending personal mail from home, and user@work.com when I’m sending professional mail from work. I have other settings that change between locations, like the directory where org files go, because reasons. Your location-dependent settings will be different.

So it seems the natural way to deal with this is to put all of the location-dependent settings in a theme, and load whichever theme is appropriate for what you’re doing. Since themes are groups of settings that can be turned on or off as a group, you can even switch themes in the middle of an Emacs session. For instance, if you’re a consultant working with two clients, you can enable theme client1 in the morning; and then, in the afternoon, disable theme client1 and enable theme client2 to update your settings.

To start with, let’s set up a couple of themes called personal and work, for personal and professional stuff, respectively. Check the value of custom-theme-directory. By default, Emacs looks for themes in the same directory as init.el, but I like to put them in a separate subdirectory, ~/.emacs.d/themes. Customize this to your taste. At any rate, that’s the directory where you’ll be creating theme files. For the rest of this post, I’ll assume that you’re using ~/.emacs.d/themes.

Theme files should go in a file named <theme-name>-theme.el. So create the file ~/.emacs.d/themes/personal-theme.el to hold the personal theme: 

(deftheme personal
  "Settings when working on personal projects."
  :family "location")

(custom-theme-set-variables 'personal
 '(user-mail-address "bob@home.org")
 ;; Add additional variables here.
 )

(provide-theme 'loc-personal)

You can now run M-x load-theme personal to load the theme. Theme files contain Emacs Lisp code, which can be a security risk, so the first time you load it, or any time you make any changes, Emacs will ask you to confirm whether to load the theme, and whether to mark it as safe in the future. If you say yes, the next time it’ll just load the theme asking for confirmation.

Follow the same steps to create a work theme.

Once you have something that works reasonably well, you’ll probably want something that loads the right theme when Emacs starts. For this example, we’ll assume that if the hostname is my-laptop, then you’re working on personal things and want to load the personal theme, while if the hostname is office-workstation, then you’re at work and want to load the work theme.

Add something like the following to your ~/.emacs.d/init.el: 

(add-hook 'emacs-startup-hook
  (lambda nil
    ;; Figure out which theme to load, depending on which machine this
    ;; is running on, and which options were specified.
    (let ((loc-theme 'unknon)
          )
      (cond
       ;; Running on personal laptop.
       ((string= system-name "my-laptop")
        (setq loc-theme 'personal)
        )

       ;; Running on work machine.
       ((string= system-name "office-workstation")
        (setq loc-theme 'work))

       ;; Add any other locations/work modes here.
       ))

      ;; Check whether the theme exists. If it does, load it.
      (if (member loc-theme (custom-available-themes))
          (progn
            (load-theme loc-theme)
            )
        (warn "Can't find location theme \"%s\"" loc-theme)
        ))))

Here, we’re just looking at the hostname, but obviously the condition can be arbitrarily complex. You might pick different themes depending on the day of the week, or whether you’re ssh-ed in from another host, or whatever makes sense in your situation. Add or remove themes as necessary.

One advantage of doing things this way is separation of information. Maybe you want to make your Emacs setup publicly visible on github, but your work setup includes hostnames or other information that shouldn’t leave company premises. In this case, you can store your work-theme.el file on your company machine, perhaps in a separate directory. You can add an entry to custom-theme-load-path, a directory outside of ~/.emacs.d, so that your work theme doesn’t accidentally get added to the git repo that has your init.el and your publicly-visible themes.

There’s one problem that I haven’t found a good solution to: child themes. Let’s say you have your work theme, with a hundred work-related settings. But of those 100 settings, there are three that change depending on whether you’re logged in to host1 or host2.It would be nice to have a host1 theme that automatically includes everything from the work theme, plus the three settings that are specific to host1. I don’t see a good way of doing this. It might make sense to use literate programming to generate multiple *-theme.el files from one source .org file.

On Emacs Theme Precedence

I’ve written before about themes in Emacs, and in particular about using them for something other than just colors. And naturally, you can have more than one theme active at the same time. So what happens when you have a variable set in two active themes?

The list of enabled themes is the custom-enabled-themes variable. And rather intuitively, earlier ones override later ones. Normally, (load-theme 'mytheme) prepends mytheme to custom-enabled-themes, so the settings you just loaded take effect, and last until you load some other settings on top of them, which makes sense.

There’s one twist, though: the settings you change through the “Customize Emacs” system also count as a theme, at least in some ways. If you look at the property list for a symbol defined in two themes, you might see something like 

(symbol-plist 'my-var)

(theme-value
 ((user 'customized-value)
  (second-theme 'second-value)
  (first-theme 'first-value))
 saved-value 'customized-value
 ...)

This tells us that two themes, first-theme and second-theme, have both defined my-var, and set it to first-value and second-value respectively. The symbol’s value is resolved in the order in which it appears here, so second-value overrides first-value because it was loaded later.

But there’s that third user entry, which comes first, and thus overrides the other values: that comes from (customize-variable 'my-var). I find this counterintuitive, since I think of the customization variables as the defaults, to be overridden by themes.

The solution I came up with this was to define my own theme, arensb, to hold all of the settings that might be overridden by other themes, and to load it first. Actually, I already had this theme lying around to store my favorite colors and faces, but themes can hold other values, so it made sense to reuse it.

Return values from Ansible roles

Ansible roles aren’t functions, so they don’t actually return values. But what if they could?

I recently wrote a role to update a software package, which involved asking some questions:

  1. What’s the latest version?
  2. Which version, if any, is currently installed?

I wanted my role to be modular(ish), so it made sense to outsource those questions to other roles. But how would they return their results? I tried adding register: latest_version to the role invocation, but that didn’t work.

But you can set facts inside a role: 

# ... Figure out installed version of Foo package...
- name: Return a value
  set_fact:
    installed_version: "1.2.3"

and if you know that this role sets installed_version, you can treat that as a return value, just as if it were a task with register: installed_version. Also, the fact that it sets, installed_version, is local to the client, so each host will have a copy in hostvars[ansible_host]. This makes sense: after all, you might want to know which of your machines are out of date.

The problem, though, is that installed_version is “global” to each client. So if you’re checking the installed version of multiple packages 

- name: Upgrade if necessary
  roles:
    - role: check_version
      package: Foo
    - role: check_version
      package: Bar
    - role: check_version
      package: Baz
  tasks:
    - debug: var=installed_version

how do you know which package’s installed_version you’re dealing with?

The way to fix this is to realize that you can use string interpolation on the left side of the colon. When we invoke the role, let’s also give it the name of a fact to put the return value in: 

- name: Upgrade if necessary
  roles:
    - role: check_version
      package: Foo
      return: foo_version
    - role: check_version
      package: Bar
      return: bar_version
    - role: check_version
      package: Baz
      return: baz_version

The check_version role then sees return as a role variable, and can use that in a play: 

# ... Look up installed version of the given package...
- name: Return version number
  set_fact:
    "{{ return }}": "1.2.3"

And then just check the appropriate fact for each role invocation. 

- debug: var=foo_version
- debug: var=bar_version
- debug: var=baz_version

Bear in mind that these variables have the same scope as other facts, so if two roles set installed_version, you’re back to the earlier problem of them stepping on each other’s feet. And yes, the whole thing feels like a hack. But it does mean you can return values from roles.

Literate Lists

I’ve written before about literate programming, and how one of its most attractive features is that you can write code with the primary goal of conveying information to a person, and only secondarily of telling a computer what to do. So there’s a bit in my .bashrc that adds directories to $PATH that isn’t as reader-friendly as I’d like:

for dir in \
    /usr/sbin \
    /opt/sbin \
    /usr/local/sbin \
    /some/very/specific/directory \
    ; do
    PATH="$dir:$PATH"
done

I’d like to be add a comment to each directory entry, explaining why I want it in $PATH, but sh syntax won’t let me: there’s just no way to interleave strings and comments this way. So far, I’ve documented these directories in a comment above the for loop, but that’s not exactly what I’d like to do. In fact, I’d like to do something like:

$PATH components

  • /usr/sbin
  • /usr/local/bin
for dir in \
    {{path-components}} \
    ; do
    PATH="$dir:$PATH"
done

Or even:

$PATH components

DirectoryComments
/usr/sbinsbin directories contain sysadminny stuff, and should go before bin directories.
/usr/local/binLocally-installed utilities take precedence over vendor-installed ones.
for dir in \
    {{path-components}} \
    ; do
    PATH="$dir:$PATH"
done

Spoiler alert: both are possible with org-mode.

Lists

The key is to use Library of Babel code blocks: these allow you to execute org-mode code blocks and use the results elsewhere. Let’s start by writing the code that we want to be able to write:

#+name: path-list
- /usr/bin
- /opt/bin
- /usr/local/bin
- /sbin
- /opt/sbin
- /usr/local/sbin

#+begin_src bash :noweb no-export :tangle list.sh
  for l in \
      <<org-list-to-sh(l=path-list)>> \
      ; do
      PATH="$l:$PATH"
  done
#+end_src

Note the :noweb argument to the bash code block, and the <<org-list-to-sh()>> call in noweb brackets. This is a function we need to write. It’ll (somehow) take an org list as input and convert it into a string that can be inserted in this fragment of bash code.

This function is a Babel code block that we will evaluate, and which will return a string. We can write it in any supported language we like, such as R or Python, but for the sake of simplicity and portability, let’s stick with Emacs lisp.

Next, we’ll want a test rig to actually write the org-list-to-sh function. Let’s start with:

#+name: org-list-to-sh
#+begin_src emacs-lisp :var l='nil
  l
#+end_src

#+name: test-list
- First
- Second
- Third

#+CALL: org-list-to-sh(l=test-list) :results value raw

The begin_src block at the top defines our function. For now, it simply takes one parameter, l, which defaults to nil, and returns l. Then there’s a list, to provide test data, and finally a #+CALL: line, which contains a call to org-list-to-sh and some header arguments, which we’ll get to in a moment.

If you press C-c C-c on the #+CALL line, Emacs will evaluate the call and write the result to a #+RESULTS block underneath. Go ahead and experiment with the Lisp code and any parameters you might be curious about.

The possible values for the :results header are listed under “Results of Evaluation” in the Org-Mode manual. There are a lot of them, but the one we care the most about is value: we’re going to execute code and take its return value, not its printed output. But this is the default, so it can be omitted.

If you tangle this file with C-c C-v C-t, you’ll see the following in list.sh:

for l in \
    ((/usr/bin) (/opt/bin) (/usr/local/bin) (/sbin) (/opt/sbin) (/usr/local/sbin)) \
    ; do
    PATH="$l:$PATH"
done

    It looks as though our org-mode list got turned into a Lisp list. As it turns out, yes, but not really. Let’s change the source of the org-list-to-sh() function to illustrate what’s going on:

    #+name: org-list-to-sh
#+begin_src emacs-lisp :var l='nil :results raw
  (format "aaa %s zzz" l)
#+end_src

    Now, when we tangle list.sh, it contains

        aaa ((/usr/bin) (/opt/bin) (/usr/local/bin) (/sbin) (/opt/sbin) (/usr/local/sbin)) zzz \

    So the return value from org-list-to-sh was turned into a string, and that string was inserted into the tangled file. This is because we chose :results raw in the definition of org-list-to-sh. If you play around with other values, you’ll see why they don’t work: vector wraps the result in extraneous parentheses, scalar adds extraneous quotation marks, and so on.

    Really, what we want is a plain string, generated from Lisp code and inserted in our sh code as-is. So we’ll need to change the org-list-to-sh code to return a string, and use :results raw to insert that string unchanged in the tangled file.

    We saw above that org-list-to-sh sees its parameter as a list of lists of strings, so let’s concatenate those strings, with space between them:

    #+name: org-list-to-sh
#+begin_src emacs-lisp :var l='nil :results raw
  (mapconcat 'identity
	     (mapcar
	      (lambda (elt)
		(car elt)
		)
	      l)
	     " ")
#+end_src

    This yields, in list.sh:

    for l in \
    /usr/bin /opt/bin /usr/local/bin /sbin /opt/sbin /usr/local/sbin \
    ; do
    PATH="$l:$PATH"
done

    which looks pretty nice. It would be nice to break that list of strings across multiple lines, and also quote them (in case there are directories with spaces in them), but I’ll leave that as an exercise for the reader.

    Tables

    That takes care of converting an org-mode list to a sh string. But earlier I said it would be even better to define the $PATH components in an org-mode table, with directories in the first column and comments in the second. This is easy, with what we’ve already done with strings. Let’s add a test table to our org-mode code, and some code to just return its input:

    #+name: echo-input
#+begin_src emacs-lisp :var l='nil :results raw
  l
#+end_src

#+name: test-table
| *Name*   | *Comment*        |
|----------+------------------|
| /bin     | First directory  |
| /sbin    | Second directory |
| /opt/bin | Third directory  |

#+CALL: echo-input(l=test-table) :results value code

#+RESULTS:

    Press C-c C-c on the #+CALL line to evaluate it, and you’ll see the results:

    #+RESULTS:
#+begin_src emacs-lisp
(("/bin" "First directory")
 ("/sbin" "Second directory")
 ("/opt/bin" "Third directory"))
#+end_src

    First of all, note that, just as with lists, the table is converted to a list of lists of strings, where the first string in each list is the name of the directory. So we can just reuse our existing org-list-to-sh code. Secondly, org has helpfully stripped the header line and the horizontal rule underneath it, giving us a clean set of data to work with (this seems a bit fragile, however, so in your own code, be sure to sanitize your inputs). Just convert the list of directories to a table of directories, and you’re done.

    Conclusion

    We’ve seen how to convert org-mode lists and tables to code that can be inserted into a sh (or other language) source file when it’s tangled. This means that when our code includes data best represented by a list or table, we can, in the spirit of literate programming, use org-mode formatting to present that data to the user as a good-looking list or table, rather than just list it as code.

    One final homework assignment: in the list or table that describes the path elements, it would be nice to use org-mode formatting for the directory name itself: =/bin= rather than /bin. Update org-list-to-sh to strip the formatting before converting to sh code.

    A Few More Thoughts on Literate Programming

    A while back, I became intrigued by Donald Knuth’s idea of Literate Programming, and decided to give it a shot. That first attempt was basically just me writing down what I knew as quickly as I learned it, and trying to pass it off as a knowledgeable tutorial. More recently, I tried a second project, a web-app that solves Wordle, and thought I’d write it in the Literate style as well.

    The first time around, I learned the mechanics. The second time, I was able to learn one or two things about the coding itself.

    (For those who don’t remember, in literate programming, you write code intertwined with prose that explains the code, and a post-processor turns the result into a pretty document for humans to read, and ugly code for computers to process.

    1) The thing I liked the most, the part where literate programming really shines, is having the code be grouped not by function or by class, but by topic. I could introduce a <div class="message-box"></div> in the main HTML file, and in the next paragraph introduce the CSS that styles it, and the JavaScript code that manipulates it.

    2) In the same vein, several times I rearranged the source to make the explanations flow better, not discuss variables or functions until I had explained why they’re there and what they do, without it altering the underlying HTML or JavaScript source. In fact, this led to a stylistic quandary:

    3) I defined a few customization variables. You know, the kind that normally go at the top for easy customization:

    var MIN_FOO = 30;
    var MAX_FOO = 1500;
    var LOG_FILE = "/var/log/mylogfile.log";

    Of course, the natural tendency was to put them next to the code that they affect, somewhere in the middle of the source file. Should I have put them at the top of my source instead?

    4) Even smaller: how do you pass command-line option definitions to getopt()? If you have options -a, -b, and -c, each will normally be defined in its own section. So in principle, the literate thing to do would be to write

    getopt("{{option-a}}{{option-b}}{{option-c}}");

    and have a section that defines option-a as “a“. As you can see, though, defining single-letter strings isn’t terribly readable, and literate programming is all about readability.

    5) Speaking of readability, one thing that can come in really handy is the ability to generate a pretty document for human consumption. Knuth’s original tools generated TeX, of course, and it doesn’t get prettier than that.

    I used org-mode, which accepts TeX style math notation, but also allows you to embed images and graphviz graphs. In my case, I needed to calculate the entropy of a variable, so being able to use proper equations, with nicely-formatted sigmas and italicized variables, was very nice. I’ve worked in the past on a number of projects where it would have been useful to embed a diagram with circles and arrows, rather than using words or ASCII art.

    6) I was surprised to find that I had practically no comments in the base code (in the JavaScript, HTML, and CSS that were generated from my org-mode source file). I normally comment a lot. It’s not that I was less verbose. In fact, I was more verbose than usual. It’s just that I was putting all of the explanations about what I was trying to do, and why things were the way they are, in the human-docs part of the source, not the parts destined for computer consumption. Which, I guess, was the point.

    7) Related to this, I think I had fewer bugs than I would normally have gotten in a project of this size. I don’t know why, but I suspect that it was due to some combination of thinking “out loud” (or at least in prose) before pounding out a chunk of code, and of having related bits of code next to each other, and not scattered across multiple files.

    8) I don’t know whether I could tackle a large project in this way. You might say, “Why not? Donald Knuth wrote both TeX and Metafont as literate code, and even published the source in two fat books!” Well, yeah, but he’s Donald Knuth. Also, he was writing before IDEs, or even color-coded code, were available.

    I found org-mode to be the most comfortable tool for me to use for this project. But of course that effectively prevents people who don’t use Emacs (even though they obviously should) from contributing.

    One drawback of org-mode as a literate programming development environment is that you’re pretty much limited to one source file, which obviously doesn’t scale. There are other tools out there, like noweb, but I found those harder to set up, or they forced me to use (La)TeX when I didn’t want to, or the like.

    9) One serious drawback of org-mode is that it makes it nearly impossible to add cross-reference links. If you have a section like

    function myFunc() {
        var thing;
        {{calculate thing}}
        return thing;
    }

    it would be very useful to have {{calculate thing}} be a link that you can click on to go to the definition of that chunk. But this is much harder to do in org-mode than it should be. So is labeling chunks, so that people can chase cross-references even without convenient links. It has a lot of work to be done in that regard.

    Readable Code: Variable Overload

    It’s well known that reading code is a lot harder than writing it. But I recently got some insight as to why that is.

    I was debugging someone else’s sh script. This one seemed harder to read than most. There was a section that involved figuring out a bunch of dates associated with a particular object. The author was careful to show their work, and not just have a bunch of opaque calculations in the code. I won’t quote their code, but imagine something like:

    NOW=$(date +%s)
THING_CREATION_EPOCH=$(<get $THING creation time, in Unix time format>)
THING_AGE_EPOCH=$(( $NOW - $THING_CREATION_EPOCH ))
THING_AGE_DAYS=$(( $THING_AGE_EPOCH / 86400 ))

    Now imagine this for three or four aspects of $THING, like last modification time, which other elements use $THING, things like that. The precise details don’t matter.

    Each variable makes sense. But there are four of them, just for the thing’s age since creation. And if you’re not as intimately familiar with it as someone who just wrote this code, that means you have to keep track of four variables in your head, and that gets very difficult very quickly.

    Part of the problem is that it’s unclear which variables will be needed further down (or even above, in functions that use global variables), so you have to hold on to these variables; you can’t mentally let them fall. Compare this to something like

    <Initialize some stuff>
for i in $LIST_OF_THINGS; do
    ProcessStuff $i
done
<Finalize some stuff>

    Here, you can be reasonably sure that $i won’t be used outside the loop. Once you get past done, you can let it go. Yes, it still exists, and it’s not illegal to use $i in the finalization code, but a well-meaning author won’t do that.

    Which leads to a lesson to be learned from this: limit the number of variables that are used in any given chunk of code. You don’t want to have to remember some variable five pages away. To do this, try to break your code down into independent modules. These can be functions, classes, even just paragraph-sized chunks in some function. Ideally, you want your reader to be able to close the chapter, forget most of the details, and move on to the next bit.

    In the same vein, this illustrates one reason global variables are frowned upon: they can potentially be accessed from anywhere in the code, which means that they never really disappear and can’t be completely ignored.

    Campaign Manager: Serializing Recursive Objects

    There’s an idea for a game that’s been rattling around my brain for a while, so I finally decided to learn enough Unity to birth it.

    To get an idea of what I have in mind, think Sid Meier’s Civilization, but for elections: you play a campaign manager, and your job is to get your candidate elected. This is complicated by the fact that the candidate may be prone to gaffes (think Joe Biden) or unmanageable (think Donald Trump); union leaders, state governors, CEOs, etc. can all promise their support, but they’ll all want something in return.

    The thing I learned this week is the Unity JSON Serializer. It’s used to store prefabs and other data structures in the editor, so I figured it would be the natural tool for storing canned scenarios, as well as for saving games. It has a few quirks, though: for one thing, it can only serialize basic types (int, string, and the like), and a few constructs like struct, and arrays and List<T> of basic types. If you’re used to the introspection of languages like Python and Perl, you’re likely to be disappointed in C#. For another, in order to avoid runaway recursion, it only serializes to a depth of 7.

    To start with, I defined class Area to represent an area such as a country, like the US. An Area is divided in various ways: politically, it’s divided into states. Culturally, it’s divided into regions like the Pacific Northwest or the Bible Belt. Other divisions are possible, depending on what you’re interested in. So I added a Division class:

    namespace Map {
    [Serializable]
    public class Area
    {
	public string id;
	public string name;
	public List<Division> divisions = new List<Division>();
    }

    public class Division
    {
	public string id;
	public string name;
	public List<Area> children;
    }
}

    As you can see, this is recursive: an Area can have multiple Divisions, each of which can contain other Areas with their own Divisions. This allows us to divide the United States into states, which are in turn divided into Congressional Districts, which in turn are divided into precincts.

    Since neither of our classes are elementary types, they can’t be serialized directly. So let’s add struct SerializableArea and struct SerializableDivision to hold the structures that will actually be stored on disk, as opposed to Area and Division which will be used in memory at run time, and use the ISerializationCallbackReceiver interface that will give our classes a hook called when the object is serialized or deserialized.

    Without wishing to repeat what’s in the docs: in order to get around the various limitations of the Serializer, the way to serialize a tree of objects is to serialize an array of objects, and use identifiers to refer to particular objects. Let’s say our in-memory Area tree looks something like:

    • Area United States
      • Division Political
        • Area Alabama
        • Area Alaska
        • …
      • Division Regions
        • Area Bible Belt
        • Area Pacific Northwest
        • …

    (That’s just an outline, of course. Each node has more members than are shown here.) We can serialize this as two arrays: one with all of the Areas, and one with all of the Divisions:

    • Area United States
      • List<SerializableArea> childAreas:
        • SerializableArea Alabama
        • SerializableArea Alaska
        • SerializableArea Bible Belt
        • SerializableArea Pacific Northwest
      • List<SerializableDivision> serialDivisions:
        • SerializableDivision Political
        • SerializableDivision Regions

    We don’t want to recurse, but we do want to be able to rebuild the tree structure when we deserialize the above. So SerializableArea contains, not a list of Divisions, but a list of identifiers that we can find in serialDivisions. Likewise, SerialDivision contains not a list of Areas, but a list of identifiers that we can look up in childAreas.

    Naturally, Area and Division each contain a Serialize() method that recursively serializes it and its children.

    The next question is: so you’ve been asked to serialize an Area. How do you know whether you’re supposed to add it to an existing childAreas list or start your own?

    Answer: if the serialization was invoked through OnBeforeSerialize(), then you’re serializing the top level object and should allocate a list to put the children in. Otherwise, append to an existing list, which should be passed in as a parameter to Serialize().

    If anyone’s interested in what it looks like when all is said and done, here it is:

    namespace Map {

    [Serializable]
    public class Area : ISerializationCallbackReceiver
    {
	public string id;
	public string name;

	public List<Division> divisions = new List<Division>();

	[Serializable]
	public struct SerializableArea
	{
	    public string id;
	    public string name;
	    public List<string> divisions;
	}

	public List<Division.SerializableDivision> serialDivisions;
	public List<SerializableArea> childAreas = new List<SerializableArea>();

	public void OnBeforeSerialize()
	{
	    serialDivisions =
		new List<Division.SerializableDivision>(divisions.Count);
	    childAreas = new List<SerializableArea>();

	    for (int i = 0; i < divisions.Count; i++)
		divisions[i].Serialize(ref serialDivisions, ref childAreas);
	}

	// Look up a Division by name, so we can avoid adding it twice
	// to serialDivisions.
	private int FindDivisionById(string id,
				     ref List<Division.SerializableDivision>dlist)
	{
	    for (int i = 0; i < dlist.Count; i++)
		if (dlist[i].id == id)
		    return i;
	    return -1;
	}

	public void Serialize(ref List<SerializableArea> alist,
			      ref List<Division.SerializableDivision> dlist)
	{
	    SerializableArea sa = new SerializableArea();
	    sa.id = id;
	    sa.name = name;
	    sa.divisions = new List<string>(divisions.Count);

	    alist.Add(sa);

	    for (int i = 0; i < divisions.Count; i++)
	    {
		sa.divisions.Add(divisions[i].name);

		int d = FindDivisionById(divisions[i].id, ref dlist);
		if (d < 0)
		    // Don't add a Division to dlist twice.
		    divisions[i].Serialize(ref dlist, ref alist);
	    }
	}
    }

    public class Division : ISerializationCallbackReceiver
    {
	public string id;
	public string name;
	public List<Area> children;

	[Serializable]
	public struct SerializableDivision
	{
	    public string id;
	    public string name;
	    public List<string> areas;
	}

	public void Serialize(ref List<SerializableDivision> dlist,
			      ref List<Area.SerializableArea> alist)
	{
	    SerializableDivision sd = new SerializableDivision();
	    sd.id = id;
	    sd.name = name;
	    sd.areas = new List<string>(children.Count);

	    dlist.Add(sd);

	    for (int i = 0; i < children.Count; i++)
	    {
		sd.areas.Add(children[i].name);

		int a = FindAreaById(children[i].id, ref alist);
		if (a < 0)
		    // Don't add an Area to alist twice.
		    children[i].Serialize(ref alist, ref dlist);
	    }
	}

	private int FindAreaById(string id,
				 ref List<Area.SerializableArea> alist)
	{
	    for (int i = 0; i < alist.Count; i++)
		if (alist[i].id == id)
		    return i;
	    return -1;
	}
    }
}
    Ansible As Scripting Language

    Ansible is billed as a configuration manager similar to Puppet or cfengine. But it occurred to me recently that it’s really (at least) two things:

    1. A configuration manager.
    2. A scripting language for the machine room.

    Mode 1 is the normal, expected one: here’s a description; now make the machine(s) look like the description. Same as Puppet.

    Mode 2 is, I think, far more difficult to achieve in Puppet than it is in Ansible. This is where you make things happen in a particular order, not just on one machine (you’d use /bin/sh for that), but on multiple hosts.

    For instance, adding a new user might involve:

    1. Generate a random password on localhost.
    2. Add a user on the Active Directory server.
    3. Create and populate a home directory on the home directory server.
    4. Add a stub web page on the web server.

    This is something I’d rather write as an ansible play, than as a Puppet manifest or module.

    Which brings me to my next point: it seems that for Mode 1, you want to think mainly in terms of roles, while for Mode 2, you’ll want to focus on playbooks. A role is designed to encapsulate the notion of “here’s a description of what a machine should look like, and the steps to take, if any, to make it match that description”, while a playbook is naturally organized as “step 1; step 2; step 3; …”.

    These are, of course, just guidelines. And Mode 2 suffers from the fact that YAML is not a good way to express programming concepts.  But I find this to be a useful way of thinking about what I’m doing in Ansible.

    Pseudo-Numeric Identifiers

    Let’s say you’re a programmer, and your application uses Library of Congress Control Numbers for books, e.g., 2001012345, or ZIP codes, like 90210. What data types would you use to represent them? Or maybe something like the Dewey Decimal System, which uses 320 to classify a book as Political Science, 320.5 for Political Theory, and 320.973 for “Political institutions and public administration (United States)”?

    If you said “integer”, “floating point”, or any kind of numeric type, then clearly you weren’t paying attention during the title.

    The correct answer was “string” (or some kind of array of tokens), because although these entities consist of digits, they’re not numbers: they’re identifiers, same as “root” or “Jane Smith”. You can assign them, sort them, group them by common features, but you can’t meaningfully add or multiply them together. If you’re old enough, you may remember the TV series The Prisoner or Get Smart, where characters, most of them secret agents, refer to each other by their code numbers all the time; when agents 86 and 99 team up, they don’t become agent 185 all of a sudden.

    If you keep in mind this distinction between numbers, which represent quantities, and strings that merely look like numbers because they happen to consist entirely of integers, you can save yourself a lot of grief. For instance, when your manager decides to store the phone number 18003569377 as “1-800-FLOWERS”, dashes and all. Or when you need to store a foreign phone number and have to put a plus sign in front of the country code.

    Programming Tip: Open and Close at the Same Time

    One useful programming habit I picked up at some point is: if you open or start something, immediately close it or end it. If you open a bracket, immediately write its closing bracket. If you open a file, immediately write the code to close it.

    These days, development environments take care of the niggling little details like matching parentheses and brackets for you. That’s great, but that’s just syntax. The same principle extends further, and automatic tools can’t guess what it is you want to do.

    There’s a problem in a lot of code called a resource leak. The classic example is memory leaks in C: the code asks for, and gets, a chunk of memory. But if you don’t free the memory when you’re done with it, then your program will get larger and larger — like a coffee table where a new magazine is added every month but none are ever taken away — until eventually the machine runs out of memory.

    These days, languages keep track of memory for you, so it’s easier to avoid memory leaks than it used to be. But the best way I’ve found to manage them is: when you allocate memory (or some other resource), plan to release it when you’re done.

    The same principle applies to any resource: if you read or write a file, you’ll need a file handle. If you never close them, they’ll keep lying around, and you’ll eventually run out. So plan ahead, and free the resource as soon as you’ve alocated it:

    Once you’ve written

    open INFILE, "<", "/path/to/myfile";

    go ahead and immediately write the code to close that file:

    open INFILE, "<", "/path/to/myfile";
close INFILE;

    and only then write the code to do stuff with the file:

    open INFILE, "<", "/path/to/myfile";
while ()
{
	print "hello\n" if /foo/;
}
close INFILE;

    The corollary of this is, if you’ve written the open but aren’t sure where to put the close, then you may want to take a look at the structure of your code, and refactor it.

    This same principle applies in many situations: when you open a connection to a remote web server, database server, etc., immediately write the code to close the connection. If you’re writing HTML, and you’ve written <foo>, immediately write the corresponding </foo>. If you’ve sent off an asynchronous AJAX request, figure out where you’re going to receive the reply. When you throw an exception, decide where you’re going to catch it.

    And only then write the meat of the code, the stuff that goes between the opening and closing code.

    As I said, I originally came across this as a tip for avoiding memory leaks. But I’ve found that doing things this way forces me to be mindful of the structure of my code, and avoid costly surprises down the line.