History History

tl;dr Use launchd with Keybase for quick ’n’ dirty backup.

Some time ago, I started losing my bash history on my employer-provided laptop. As someone who uses CTRL-R with fzf to use my bash history as another brain, this felt like being lobotomized.

There’s a good chance this is something that I’ve done. Likely I’ve misconfigured bash’s history settings in some way (the HISTSIZE and HISTFILESIZE variables are good candidates). However, I like to think it’s the company IT department doing me wrong. 😉

In any event, losing this kind of information is something to avoid, whoever the responsible party is. I wanted a way to back it up automatically.

Since this is an Apple laptop, launchd is one of the simpler built-in ways to run something periodically. Keybase provides a simple-to-use encrypted network file system, and I already had it installed. So I fired up the ol’ text editor and wrote me some XML:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>Label</key>
        <string>local.backup_bash_history</string>

        <key>ProgramArguments</key>
        <array>
            <string>/bin/bash</string>
            <string>-c</string>
            <string>/usr/local/bin/keybase fs cp /Users/local username/.bash_history /keybase/private/Keybase username/bash_history_archive/bash_history.$(date +%Y%m%d%H%M%S)</string>
        </array>

        <key>StandardErrorPath</key>
        <string>/tmp/backuperr</string>

        <key>StartCalendarInterval</key>
        <array>
            <dict>
                <key>Weekday</key>
                <integer>1</integer>
                <key>Hour</key>
                <integer>9</integer>
                <key>Minute</key>
                <integer>30</integer>
            </dict>
            <dict>
                <key>Weekday</key>
                <integer>2</integer>
                <key>Hour</key>
                <integer>9</integer>
                <key>Minute</key>
                <integer>30</integer>
            </dict>
            <dict>
                <key>Weekday</key>
                <integer>3</integer>
                <key>Hour</key>
                <integer>9</integer>
                <key>Minute</key>
                <integer>30</integer>
            </dict>
            <dict>
                <key>Weekday</key>
                <integer>4</integer>
                <key>Hour</key>
                <integer>9</integer>
                <key>Minute</key>
                <integer>30</integer>
            </dict>
            <dict>
                <key>Weekday</key>
                <integer>5</integer>
                <key>Hour</key>
                <integer>9</integer>
                <key>Minute</key>
                <integer>30</integer>
            </dict>
        </array>
    </dict>
</plist>

That’s the file in its entirety. I’ll break down the important pieces.

<key>Label</key>
<string>local.backup_bash_history</string>

This just gives the agent a unique identifier. It seems like best practice to give one’s own scripts the local. prefix. Note that the file is named the same thing, with an addition .plist extension: local.backup_bash_history.plist.

<key>ProgramArguments</key>
<array>
    <string>/bin/bash</string>
    <string>-c</string>
    <string>/usr/local/bin/keybase fs cp /Users/local username/.bash_history /keybase/private/Keybase username/bash_history_archive/bash_history.$(date +%Y%m%d%H%M%S)</string>
</array>

This is the meat of the thing, the command that gets run. Fun fact: pre-Catalina, this was a simple cp command, to the volume Keybase had mounted. However, Catalina gave me grief about trying to write to a network volume, and I couldn’t figure out how to whitelist my launch agent. (Likely there is a way, I just couldn’t find it with some cursory web searching.) Fortunately, Keybase was already granted the right permissions, and it ships with a command-line utility with access to all of its features, including the file-system stuff.

Why nest the call to keybase fs cp in a bash invocation? All to get the process substitution to generate a timestamp with date. As far as I was able to determine, there’s no way to set per-execution variables dynamically in a launch agent plist. Another option would have been to bundle it all up in a separate script, but I wanted to keep this as self-contained as possible.

<key>StandardErrorPath</key>
<string>/tmp/backuperr</string>

The location to write errors to. This came in handy when my plain cp version of this started to fail. Granted, it took me way too long to notice things weren’t right; but that’s another rabbit hole for another day!

<key>StartCalendarInterval</key>
<array>
    <dict>
        <key>Weekday</key>
        <integer>1</integer>
        <key>Hour</key>
        <integer>9</integer>
        <key>Minute</key>
        <integer>30</integer>
    </dict>
    <!-- other weekdays here -->
</array>

And this is how you schedule the thing. I had to repeat this four times (so it would run every weekday morning at 9:30 AM, when my computer was likely to be on). It would be nice if there were a cron-like DSL for specifying the schedule, but on the other hand—would it? It’s readable, and I have to admit I haven’t touched it since.

I copied the file to its home in $HOME/Library/LaunchAgents. Then I used launchctl to manage the agent. Here are a few commands worth knowing.

• launchctl load local.backup_bash_history.plist tells launchd to, well, load and schedule the code.
• launchctl start local.backup_bash_history invokes the command immediately, which is extremely helpful when debugging. (Check that StandardErrorPath!)
• launchctl remove local.backup_bash_history pulls it out of launchd.

(There’s a really great website that describes launchd in much greater detail, which I referenced extensively while putting this humble agent together.)

Now it generally hums along without issue. When I CTRL-R and search my history, and I don’t see a command I’m sure I ran, now I can pop over to my archive folder and search (generally with ripgrep). That's not quite as convenient as searching history directly, so the next yak I shave will likely involve merging all these backups back into a single unified history.