It Rains In Triplicate

I spent so much time in this meeting: PHD Comics: A list, plan or outline of things to be done

Course is over. Now I can get some billable work done.

I’m going to have to increase my in-house storage at some point in the next 6 months, I think. Right now, I have a 3×750GB RAID 5 (effective storage of 1.5TB) as my major storage drive. The RAID is in a FW800 enclosure hooked up to a linux box, exposed over a 1000mbit ethernet link.

A few options for the next set of storage:

1. Add another FW800 enclosure and a bunch of disks to the existing linux box. About 150$ for the enclosure + the cost of disks
2. A drobo + drobo share. ~600$ + the cost of disks
3. A netgear NV+ or some other professional RAID solution. ~1000$ + the cost of disks

The first option is cheapest, but probably has the largest maintenance cost. The 3rd is most expensive, and it isn’t clear what it adds over the drobo.

The big issue is that none of these solve the “how do you back up terabytes of data cost effectively”.

Another trip, this time to Colorado Springs. Successful trip, overall. COS is just like Omaha, except there are mountains on the horizon.

By the way, you know how you can tell which mountain is Cheyenne Mountain? It is the one with all the antennas on top.

So during the latest trip, I watched the first several episodes of torchwood.

So if you are a team of people going into constantly dangerous situations, wouldn’t you put the guy who CANNOT DIE as your point person for all ingresses into these situations? Invariably, the people who can die keep going in first.

It is really just poor resource planning.

I’ve always been a version control geek. Back in grad school, I spent a whole lot of time following the alternatives, partially because I had a professional interest in contributing something professionally to the theory (never really did, too many other interests), and partially because I spent a year in RCS hell and wanted to find something much better.

So right now, I have an interest in looking at better accountability and patterns for enterprise software development (I have a paper just submitted on applying patterns to agile data consumers, I should post once I get official public release sign-off).

So, two interesting approaches to how version control systems look at this problem:

1. AEGIS, which tools a very careful approach to developing good software. It had 3 rules:

1. Every change had a unit test, U. Source tree T must always fail U. Source tree T’, T with the change applied, would pass U.
2. Given a tree Tn, every previous unit test, U0 to Un-1 would pass on Tn.
3. Every change would need to approved by a reviewer and integrator before it is officially checked in / made available

Keeping an AEGIS tree going require a lot of rigor, but the product is probably pretty solid. Sounds a lot like good agile programming techniques, doesn’t it?

2. GIT, which is the SCM system being used for linux kernel development, is pretty straightforward. It seems a whole lot like the typical SCM system, i.e., SVN, CVS, bitkeeper, except for a piece called bitsect.

This system assumes that you will always have regressions, and you won’t catch them until they are deep in the code. Further, it (too optimistically) assumes that every regression can be reproduced and traced back to a single checkin.

So, given a tree, with two revisions, X and Y, with there being several checkins between X and Y, such that Y = X+c0, c1, c2, ., cn, and Y has a bug that X does not, we can do the following:

git bitsect good X

git bitsect bad Y

git will give a new tree. You execute the tree and check for the bug.

The outcome determines whether or not you mark ‘git bitsect’ good or bad

The result is another new tree,and the process repeats.

Each new tree is a binary partition of the change set c0, c1, c2, ., cn.

Eventually, bitsect gives you a single change that is the issue.

So, two approaches. One is optimistic, you’ll never have a bug because you’ll put in the upfront effort. The other is pessimistic, you’ll always introduce a bug, but it’ll be a discrete bug.

Unfortunately, neither considers the fact that bugs cascade and may be part of a complex system that can not be traced to a single, identifiable cause.

So why does hibernate use a DTD instead of an XML schema?

The DTD ends up implying an order of elements, making something like:

<map name=”config” lazy=”false”> <key column=”ID”/> <map-key column=”CONFIG_KEY” type=”string”/> <element column=”CONFIG_VALUE” type=”string”/> </map>
valid, and
<map name=”config” lazy=”false”> <key column=”ID”/> <element column=”CONFIG_VALUE” type=”string”/>
<map-key column=”CONFIG_KEY” type=”string”/> </map>
invalid.
bleh. That one cost me a day, but it kept complaining about missing elements.

Fever broke multiple times, but finally went away for good on Friday. Still not feeling completely human, but pretty close today.

Of course my wife is now sick with the same creeping crud.

So I’m writing an application in my copious free time that access a bunch of services and data from authoritative sources and does some searching against them.

The issue is that it is far, far too expensive to query against these data sources every time, so I need to cache. The cache, however, is never going to be authoritative and the data is user driven, so there are no good predictive algorithms for how often the cache will need to be refreshed.

This scenario is increasingly common in social applications, where you are doing analytics of data between different sources. i.e., querying a contacts list from facebook, getting the groups you are a part of in linkedin, etc.

My current approach is to make the cache refresh user-driven, the user makes a conscious decision to reload the cache, and has to wait for it to complete. An alternative is to refresh every night.

It would be nice if there were an easy way to check a list updated field, but none of the services expose one, so far as I know.

Wil Shipley puts it elegantly in Call Me Fishmeal. MacBook Air Haters: Suck My Dick

A sealed, non-upgradable system has rarely, if ever, decided success or failure of a consumer electronics product, and definitely not an apple product. The iphone and ipod are not user serviceable any more than the mba, i.e., you can probably crack it open and replace the drive or battery, but it isn’t easy and may cause damage. The imac or mbp is only a little bit more so, requiring a good amount of expertise to do so.

So, my conclusions on the MBA

1. The utility of a device is not about specs. You can buy a budget dell laptop with better specs on paper. You can buy a cheaper desktop with much better specs then a laptop.
2. It is pretty clear that the MBA is designed to be a secondary or travel device, not your only computer. The remote drive feature really points to that.
3. Some dumbass is going to try to sue apple because it is not user serviceable, just like someone did over the iphone

Given my travel schedule, I would have considered the MBA if it did 4G of RAM. Given the amount of time I spend in VMWare, 2G isn’t enough. I do wish they’ve bring the multitouch trackpad to the MBP line soon.