The Goldfish Chronicles

Earlier in the week, I made an attempt to get an NSLU2 (which I’ve now learned to call a Slug) running on a 64M flash stick. It’s not the intended way, and I got surprisingly far before learning the reasons why it’s not as simple as all that. An unslung slug in fact, but without the ability aka space to install new packages.

I knew it’d be tricky getting it to work on flash as you’re not intended to plug flash sticks into the USB1 slot, just the USB2 slot; but turns out that 64M is too small for the default partitioning that the new firmware OS does. It uses about 150M by default for swap, config etc.

So fresh with the knowledge of what I’d done wrong, I attempted to rollback to the Linksys firmware and start again. Apparantly realising that you’ve got a USB stick still in there when you start the update and pulling it out is a bad idea. To be fair, everything you read warns you not to do this, I was just being an idiot. Voila, one brick (the less affectionate name for a dead slug).

However, provided the bootstrap/bios (known as RedBoot) is still there, you can get it working via one of 3 ways. I tried all 3 without luck before getting encouragement to try the first again and finding it worked from a different machine.

“There and back again, a slug story”

I immediately installed the slug again with the unslung software, against a hard drive. Then I pondered how to get Nagios on there. Not easy it seems, there’s not a package for gcc, so I might have to learn about cross-compiling. Openslug, the alpha-new-OS, does have a gcc available, so possibly I could move to that and do the compiling on the slug. Nagios also needs Perl, but it looks like that is available. The plan is to figure out how much space Nagios needs, then get something of suitable size.

Short followup to last nights ramble about the NSLU2. It’s now handling the backups, the Nagios configuration is on a thumb-drive waiting for a new NSLU2 and the jet-plane 4U is finally quiet.

The silence is quite oppressive, I wonder if I should get a USB fan to help me get used to life without a jet-engine cooled 4U.

I recently obtained a second Linksys NSLU2. I had a couple of hard-drives that I wanted to put into USB Enclosures and put online and I also wanted to be able to play with the NSLU2 Linux project. Last night I did the boring thing and put the new USB drives together and tonight I set the NSLU2 up and ‘unslung’ it; that is I installed the stable nslu2-linux firmware.

There is a bit of magic to figure out as you set it up (cached passwd files, when to set your passwd etc) but it’s all very clearly explained along with many reminders to not attach hard drives while flashing the device. The unslung distro is intended to be a clone of the Linksys firmware, with a few vital improvements. You don’t have to turn it off to (u)mount drives and you can install extra packages on. There’s also something called openslug which is a completely new Linux distro and will have such advantages as being able to hook a USB hub up and have more than the 2-drive limit of the Linksys and Unslung firmware. It’s still in alpha however. Letting the drives power-down is a promised feature too.

Once installed, I telneted in and installed openssh and rsync. There was an alternative to openssh (dropbear I think), but I decided that I’d accept the heavier openssh because I’m comfortable with its configuration and I’ll be moving an ssh key over from a loud noisy 4U server and expecting it to just work. I’m doing this because I’m aiming to have this nslu2 handle backups for my 4 1U servers.

I’m tempted to buy another, they’re addictive at 80 dollars a pop. The only other job the 4U box is doing is being my nagios server, and it would be very cool to have an NSLU2 with a USB thumbdrive acting as a nagios server. This would let me turn the 4U off, which will a) save noise, b) save desk-space and c) save power. The NSLU2 draws 1.5W while a desktop apparantly takes 60W (and the 4U probably takes a fair bit more as it used to have 3 hdd’s and 6 fans or so.

Another feature, which I’ll either install on the original one (which is still on linksys firmware at the moment) or buy yet another nslu2 for, is mt-daapd. An iTunes server so that when we open iTunes our mp3 server will just appear.

Yet another feature is that you can compile a usb 2.0 ethernet driver in and use one of the two usb ports for an usb->ethernet link. This would allow you to use the box as a router, though I keep noticing references to a linksys device that can also be customised a lot and I think that might have been a router.

Java is possible, somebody has instructions for installing JAM but Kaffe apparantly has issues. The biggest limitation here is the 32M (I think) memory that the device has. It seems to fill up pretty quickly with SMB, SSHD, UPNP, thttpd and the various other services that run.

These things are a lot of fun. It amuses me that they are better than the P100, 32M memory, 1Gig hard drive that was my first web/email/bash server. I recommend them to anybody who likes playing with Linux.

Perhaps the best bit about the Payload release is that it happened at all.

Over the last few years, I’ve definitely noticed that each Spring sees a sudden blooming of apathy for all things open-community on my part. I don’t know if anyone else feels the same, but it’s the one time of the year when the time I dedicate to involvement in things open vanishes. This year is especially bad due to the baby boy, a busy time at work and a stomach virus as Spring arrived.

Partly I think it’s because each Spring sees me getting more interested in mundane things like the house, cleaning, building (usually the home network) and rediscovering the basement where most of my computer equipment sits. Summer is too hot, Winter too cold and Autumn sees me too tired after Summer to want to do any of that stuff as I spend most of Summer having to do house stuff when I don’t want to.

There must be something more though. I’ve been dropping off the home-coding each Spring, regardless of whether I have a house, money to fund building or a baby to fill my time. Probably the only reason that work is addictive at the moment is because we’re in effect doing an aggressive spring-clean. Maybe it’s just because the sun is out and it’s beautiful outside, which would imply that it’s better to run a development company in London or Seattle than California and Kentucky.

Also suggests that spring-cleaning projects are a great idea to do each spring to catch the enthusiasm of the team, but this might just be my own malaise.

The Apache lists do seem quiet though. Do other people hit this issue? Anyone know a correct term for it? For the moment I’ll call it SAD-recovery-syndrome Winter is over, so time to shed the old, bring in the new and prepare for a new year. Somebody really screwed up when January 1st was not (generally) the first day of Spring.

Brief mention of the 0.3 release of OSJava Payload.

Payload is a simple library for creating self-extracting jars. Said jars can perform search+replace as they extract and the new version can search+replace inside zip files (in addition to a bugfix for Windows usage).

It’s a critical part of the Genesis build/package/deploy system (mostly written in Perl) which we use at work to manage our Mavenized projects. A group of CVS modules are built, packaged inside a slightly customised JBoss 3.2.x container (unused parts deleted and more in the way of management scripts) and entombed into a self-extracting jar.

The administration team then execute said jar, along with a .properties file specifying the environment, on the desired machine and start things up.

The major advantage of this is that things become very predictable, a Java admin is not largely required and the developers get to manage the entire application without having access to production (so container adjustments work their way through Dev->Test->Staging->Live). Another nice touch is that the J2EE install doesn’t fill up with crap as time goes by, it’s regularly being refreshed with a clean new version.

Probably the biggest downside is that it’s too easy. A handful of lines can build and deploy 2-dozen CVS modules into 6 separate JBoss installs, and there’s less impetus for Developers or Admins to learn about the J2EE container. Still, this can also be good as our Graphics Designer has been doing regular deploys.

My secrets to developing:

* Be tidy and clean, aka care about your code.
* Learn to extract history from your source-code repository.
* Never settle for a result unless you understand how you got there, and its corollary:
* Code from scratch, don’t adapt examples.
* Experiment with tools, but once you’ve experimented, focus on particular ones for particular jobs.
* When you have a language query, experiment yourself. Don’t let search engines replace your brain.
* Become a zone junkie. There are times when coding is bliss, hunt for those times.
* Watch the leading/bleeding edge, but don’t go there unless you feel it is right.
* Always try to keep things to a minimum.
* Be brave. Don’t be afraid to delete, remove, stop.

I’m wearing a disguise at work these days and spending 75% of my time up to my neck in database modelling as I seek to clean up datamodels a bit. Most of the Java I do is directly related to bits I’ve broken with my aggressive DROPs and the rest of my time is spent on trying to schedule it (and various code merging) in JIRA.

All of this reminds me of the one lesson I would hope that every new developer is prepared to learn as soon as possible. It doesn’t matter how full of genius your code is, or how methodic your approach to a problem. It doesn’t matter if you use the newest techs, or build everything in AWT Applets and Model 0 JSP. Be tidy and clean in your code.

Part of this is a dogma of Agile; don’t do unnecessary work, but it’s more than just that. Whatever work you do, be tidy in how you do it. Frankly, I don’t care if your code is commented to the hilts, in fact over-commenting is a sure-sign of a clutterer. Being tidy about your development shows that (and encourages) you to care about your work, and that’s the underlying message. Care about what you do. That’s not to say that you have to think hard and slow; you can hack quickly and still remain clean, still care about the hack.

A nice self-test we can all do is to look at our desktops. Is it organised (as my Windows boxes all tend to be, despite it being my least favourite platform), or is it a mess (as my Powerbook tends to be because I’ve lazily let Firefox download to the Desktop). My Linux habits are different in that I don’t use the Desktop. For that respect I would point to the home directory, which is such a difficult thing to keep tidy.

I suspect I’m now hitting parallels with thought-police who demand a tidy desk at work. Clutter and mess is fine for creation and experimentation. My desk is always covered in a layer of mess. However I periodically tidy it and under the top layer is an organized set of piles, magazines, books etc.

Probably the best ‘blog’ on the web nowadays is the BBC’s world affairs editor’s column. John Simpson has been reporting the news on the BBC for as long as I can remember, and his opinions on Iraq and the Pope have been a breath of fresh air. He does a superb job of hitting the impartial reporting spot that the Beeb is meant to exist to do, and the entries are informative and compelling.

It’s a huge coup for the news.bbc.co.uk site, a major member of the TV side of things getting involved with the new way of doing things. Now if only it had its own RSS feed, I would be able to truly call it a blog.

Well covered elsewhere, but as I recently discovered just how good VLC is, I thought I’d mention (and link) that it believes it is going to be an early casualty to the new European law on patents - http://www.videolan.org/patents.html.

Both Quicktime on our Apples and Windows Media Player on the Windows box proved to be utterly useless when it came to watching video. So I first tried bsplayer on Windows (very good) and then on the Apples (and subsequently Windows box as it is cross-platform) the VLC player. VLC is a superb example of the open program being light years better than the one that gets bundled with the OS.

So it’s a shame to see that the cash-for-laws program that is so widely practiced in Washington DC, and increasingly sounds like it’s being practiced in Brussels is going to grow into a way for stupid companies to protect their dated ideas.

(TDD is Test-Driven Development. Top-Down Development is merely shortened to Top-down here-in).

I’ve long struggled to believe in TDD. After much reflection, my decision was that TDD, while a good idea for bottom-up developers, just didn’t have as much to give top-down developers.

In case I’m using those terms incorrectly; I’m defining a bottom-up developer as someone whose first class in writing an application is the lowest level code, a data structure or a utility; while a top-down developer begins by writing the shell to an application with many stubbed out parts.

For me, top-down development can evolve while bottom-up development really needs to be working to a pre-defined design. Said design was probably top-down, so really the difference is one of good coding being top-down, while one way of creating bad code would be to do bottom-up coding without any initial top-down forethought.

TDD helps people who want to evolve from a bottom-up beginning. It forces them to view their low-level component from the outside before coding, and so the out-of-control problem that bottom-up can lead to is controlled. Top-down already solves this problem, so the part of TDD where it creates better basic units does not apply to the top-down developer.

Before this sounds like too much is being stripped away, I still see two distinct pluses from TDD. While evolving top-down encourages the developer to think about the desired functionality, TDD encourages you to worry about boundary cases, error cases etc before even implementing the code. A bad side of this plus is that the coder is distracted from the development of the major, into the development of a tiny component where they worry about every possible outcome. This seems like a restriction on agility and akin to premature optimisation; though it’s easy to see the argument that catching rare outcomes now is better than catching them at production.

The other plus is that you end up with unit tests as a result, and by virtue of that, code which has been designed to be unit-testable, which while not better code (just better units) at least means you won’t have to redesign at some later stage when you decide to implement unit tests.

I’ve used TDD very happily on two distinct types of occasion. The first is for bug-fixing. Create a test, watch it break, fix the code, watch it pass is a great way to fix bugs. The second is when given a very tight set of inputs and outputs (such as the robots.txt RFC). Writing a test harness and then making the tests pass is the perfect way to develop such a clearly defined problem.

It’s hard to define the negatives to TDD. My main one is that it disrupts the flow of coding; which seems like a very bad thing. Today I will write a Foo application. I begin by writing a unit test for the FooRunner command line class. Then I implement the FooRunner command line class. Its first step will be to call a database to get a list of options; so I begin by writing a test for the database call.

STOP CODING. How will I implement a test on an outside resource? Do I stub out the test call, do I embed a memory-database, spending time figuring out how to create the right state for the call and happy that I can only use the 90% of SQL-99 that is commonly implemented now, or do I run against a dedicate external DB so I can use database-dependent SQL, setting up state there and hoping that nobody else is running a unit test at the same time.

Even if I have decided on exactly which external-resource solution I’ll apply, they all take a fair amount of time.

There’s a 4th solution, which is a variant of the first. Instead of simply stubbing out the test call, I’ll stub out the real call too; which is probably what I’d have done in the original top-down case; and worry about how to implement the code and how to test the code later.