So the Arduino programmer was not that great. AVR programmers are easy to build though, just a handful of zener diodes and resistors and your serial port. Instead of building one, I decided to re-purpose my MiniPOV3.
Doing this is about the same as building one of these. The pinout is easy to remember as an 8-pin AVR's ICSP pins are in the same place. The only thing to do is hook up Vcc, ground and reset.
To program the ATtiny13 on the breadboard just ground the jumper on the bottom left. This holds reset down which is required for programming. There's some way to do that through the serial port, but I was too lazy to figure that out. Now all you have to do is power the circuit and fire up an avrdude command like this.
avrdude -p attiny13 -P com2 -c dasa -t
This gave me an avrdude command prompt where I could examine or write flash or eeprom memories. Similarly you could use a -U command like -U flash:w:someprogram.hex. And this will work for every ATmega and ATtiny (or at least I'm pretty sure). I went ahead and ordered one of these anyway though.