Jay McGavren’s Journal

Most people who’ve used a command line know about the existence of the “zip” utility. It’s basically a scriptable version of the archive tools we all use in our file browsers. Hopefully most people also know about RSync, which brings a directory structure into sync on two machines by creating, deleting, and updating files as necessary.

What people may not know, however, is that RSync will transmit only the updated portions of a file if it already exists on the remote side. That means you can easily maintain downloadable .zip archives, so your audience can retrieve whole collections of files at once. (Say, of your favorite Creative Commons-licensed music, which is the example we’ll use here.)

We start with a command line to add MP3s to a .zip file:

$ zip ~/Documents/media/cc_music.zip \
/Users/jay/Music/iTunes/iTunes\ Music/Ghost/ccMixter/Lullaby.mp3

The long music path is for convenience - that way I can just drag-and-drop files onto a terminal. It’s a nice touch that the full folder structure is preserved (though I will likely want to strike the “Users/jay” directory at some point):

$ unzip -l ~/Documents/media/cc_music.zip
Archive:  /Users/jay/Documents/media/cc_music.zip
 Length     Date   Time    Name
 --------    ----   ----    ----
 3664924  06-26-11 00:58   Users/jay/Music/iTunes/iTunes
Music/Ghost/ccMixter/Lullaby.mp3
...

If we use “&&” to add on an rsync command, the .zip file will automatically be sent to the remote server if compression is successful:

$ zip ~/Documents/media/cc_music.zip
/Users/jay/Music/iTunes/iTunes\ Music/Ghost/ccMixter/Lullaby.mp3 && \
rsync --verbose --compress ~/Documents/media/cc_music.zip \
jay.mcgavren.com:files/cc_music.zip

In the output, we can see the file being zipped, followed by the rsync transmission info:

 adding: Users/jay/Music/iTunes/iTunes
Music/Ghost/ccMixter/Lullaby.mp3 (deflated 3%)
cc_music.zip

sent 3556109 bytes  received 31 bytes  78156.92 bytes/sec
total size is 3554835  speedup is 1.00

Keeping all this as one command allows automatic updates of the remote file as new MP3s are added:

$ zip ~/Documents/media/cc_music.zip \
/Users/jay/Music/iTunes/iTunes\ Music/DoKashiteru/ccMixter/Our\
Slanted\ Voices\ \(ft.\ Colin\ Mutchler\).mp3 && \
rsync --verbose --compress ~/Documents/media/cc_music.zip \
jay.mcgavren.com:files/cc_music.zip
 adding: Users/jay/Music/iTunes/iTunes Music/DoKashiteru/ccMixter/Our
Slanted Voices (ft. Colin Mutchler).mp3 (deflated 1%)
cc_music.zip

sent 5043684 bytes  received 11377 bytes  306367.33 bytes/sec
total size is 8595106  speedup is 1.70

Those transmission sizes represent only the new MP3 that’s been added, not the whole archive. For example, if I add a small README.txt to the archive…

$ zip ~/Documents/media/cc_music.zip README.txt && \
rsync --verbose --compress ~/Documents/media/cc_music.zip \
jay.mcgavren.com:files/cc_music.zip
 adding: README.txt (deflated 36%)
cc_music.zip

sent 1288 bytes  received 20521 bytes  3965.27 bytes/sec
total size is 11636110  speedup is 533.55

…note that only 1KB (of the now-11MB file) is actually transmitted. Updates to files in the middle of the archive likewise cause the retransmission of only that file’s zipped content.

This can of course be converted to a shell script, but personally I think doing it all at the prompt lends itself to ad-hoc, on-the-fly alterations. (I wouldn’t want a script sitting around for every collection I could think to post.)

And I can certainly think of a lot of things to share this way - Project Gutenberg books, Creative Commons wallpapers, your own photos, basically anything that you can freely redistribute. Some care needs to be taken with variance in licenses (I had to add attribution ID3 tags to some of the MP3s, for example). But generally, it should be possible to include the needed documentation in the archive alongside the content. (I am not a lawyer, this is not legal advice, etc. etc.)

I view this as vastly preferable to clicking though dozens of archive.org pages to manually download individual files. If you agree, I hope you’ll use this technique to start sharing your own collections!

Creative Commons (creativecommons.org) is a license you can apply to stuff you create that basically says “anyone can redistribute this as much as they want, they just have to give me credit”. I figured that would enable people to conveniently and legally share huge zip-files of their favorite Creative Commons music, but that hasn’t been the case so far. So, here’s my attempt to fix that:

cc_licensed_music.zip

Just download and unzip; it’s already sorted into folders for you. Drop the whole shebang on top of iTunes to import the tracks. Share with whoever you like; it’s totally legal to do so.

I discovered a couple clever tricks with Linux utilities that will let me easily update this file as I discover new CC music, so feel free to bookmark this URL. And don’t worry, I’m being selective; I know there’s a lot of crap out there. These are real gems, though. If you agree, most of the artists have other music for sale; visit the URLs in each MP3’s comment field to find it.

Received: Tue, 16 Aug 2011 17:35:00 -0700 (PDT)
Subject: New voicemail from (602) 999-9999 at 5:35 PM
From: Google Voice
To: me@example.com

Transcript: The [doctor’s office], calling to remind Jeremy of the an appointment on Tuesday, August 23rd at 11. Hey and in the pediatric office. Please listen to the following options to confirm your appointment, please press 1 to confirm your appointment. Please press 2, to repeat this message. Please press 3, to cancel or reschedule your appointment. Please press forward to hear. Directions to our office. Hello. This is [doctor’s office], calling to remind Jeremy of the an appointment on Tuesday, August 23rd at 11. Hey and in the pediatric office. Please listen to the following options to confirm your appointment, please press 1 to confirm your appointment. Please press 2, to repeat this message, please press 3, to cancel or reschedule your appointment. Please press forward to hear. Directions to our office. Hello. This is [doctor’s office], calling to remind Jeremy of the an appointment on Tuesday, August 23rd at 11. Hey and in the pediatric office. Please listen to the following options to confirm your appointment, please press 1 to confirm your appointment. Please press 2, to repeat this message, please press 3, to cancel or reschedule your appointment. Please press forward to hear. Directions to our office.

Play message:
https://www.google.com/voice/fm/9999999999999999/loremipsumdolorsitametconsecteturadipisicingelitseddoeiusmod

A friend asked me by e-mail what I was passionate about. When I had completed my long-winded answer, I realized that while I think endlessly about my life goal, I had never really written it down before. So, I’ve included it here.

My contribution to this whole mess is going to be tools that enable creative expression. So far it’s been the direct creation of art - generative art and game libraries. But the world is most lacking in tools that allow people to share what they’ve made (and make a living doing it). Sure, there are publication channels like Amazon and the iTunes Store, but you usually have to design what you’re making to fit the channel.

I want people to be able to write a book or a song, mark it as available for sale, and that’s all they have to do. Software should take care of publishing it to a website AND the Android Marketplace AND future channels that haven’t been invented yet. It should take care of promoting your work directly to people who are interested, and it should gather payments for you from all these places and plop them into your bank account. It should also give you feedback on who likes your work and why. If you want to publish your work for free, or allow others to remix it, the same tools should allow you to do that.

Huge thanks to Austin and Zach at Heatsync Labs for their massive efforts to rescue my old Applesoft Basic programs. Don’t have the source yet (it’s on a IIgs 3.5″ floppy that I also can’t read, but it’s an improvement), but I do have some runtime footage!

The Geek ‘n’ Eat group at work had a Pecha Kucha day, and I prepped a quick presentation on learning Ruby/Clojure/Scala/Javascript through koans. (It’s also secretly about flow and how to achieve it in your coding.) I had recently learned how to record audio and export video from Keynote, and applied my newfound knowledge to do a Youtube-ready version:

The mountain.rb (henceforth to be known as Rocky Mountain Ruby) Chronicles are finally out. Pretty darn honored to occupy a page opposite Jim Weirich. (Many other illustrious names in there too.) Great wrapup of a great event.

Hmmm… How about a puzzle game where pieces are “if” statements, and the branch evaluates as soon as they’re “true”? That makes other statements true, causing them to evaluate, which makes other statements true… forming a combo - er - cascade.

Here’s the latest version of my Ruboto presentation… An earlier version of these slides was used by Charlie at JRubyConf 2010, and I updated them for my talk at Philly Emerging Tech.

Pre-rendered HTML

Source (for Showoff presentation software)

These are Creative Commons-licensed, so you’re encouraged to steal these slides for use at your favorite technology user groups!

A couple nights ago, I was playing Score Rush, a twin-stick shooter on XBox, and thinking what an exemplary game it was. Simple, but with highly polished gameplay. Something that other designers and developers could stand to learn from.

That started me wishing there was an open-source game in each genre that was playable and fun, but was first and foremost a reference for developers to learn techniques from. Web framework standards have reference implementations, why can’t games? All the needed infrastructure is out there - freely available game libraries in a variety of languages, GitHub and other source code sites for forking and improving on projects, and a great many motivated (but largely disorganized) open-source game developers. What if we joined forces to make something bigger than any one game?

And so resulted this initial draft of The Reference Games manifesto:

• We seek to meet the needs of both casual and experienced gamers where possible.
  • We believe that affordances for casual gamers should never be obstructive to experienced gamers.
  • In the rare event that needs are mutually exclusive, we strive to provide for both markets through multiple configurations of the same game.
• We prove our assertions through appropriate testing.
  • Example: A/B testing to prove that a particular control scheme reduces player error.
• Although we support innovation, we are a platform for sharing proven best practices first.
  • The reference game codebases can be forked and customized, but only proven techniques usable by a wide audience should be incorporated into core reference games.
• We value gameplay over graphics.
  • The majority of maintenance and development time should be spent on game mechanics.
  • We support users with low-power or low-cost hardware.
  • We allow graphical improvements through abstracted rendering interfaces.
• We are open.
  • We do not rely on proprietary frameworks or formats that are not freely available or could become unavailable in the future.
• We work with the tools most commonly used by the development community.
  • While respecting the openness principle, we choose the tools and languages most widely used by game developers.

I don’t know if I’ll have an opportunity to go anywhere with this, but it’s something I’d love to see. Any enterprising developers out there that want to take this and run with it?