footle

While poking around the database in my rails console, I often found myself jumping to the mysql console just so I could get an easier-to-digest view of the data. I finally had enough of this silliness and wrote up a quick function to dump of set of ActiveRecord objects in the mysql report format.

>> report(records, :id, :amount, :created_at)
+------+-----------+--------------------------------+
| id   | amount    | created_at                     |
+------+-----------+--------------------------------+
| 8301 | $12.40    | Sat Feb 28 09:20:47 -0800 2009 |
| 6060 | $39.62    | Sun Feb 15 14:45:38 -0800 2009 |
| 6061 | $167.52   | Sun Feb 15 14:45:38 -0800 2009 |
| 6067 | $12.00    | Sun Feb 15 14:45:40 -0800 2009 |
| 6059 | $1,000.00 | Sun Feb 15 14:45:38 -0800 2009 |
+------+-----------+--------------------------------+
5 rows in set

Grab it from GitHub and stick it in your .irbrc.

I hadn’t posted this before because it seemed kind of trivial and silly, but I used it again the other day and decided maybe I should share the joy after all.

Far too often I find myself jumping through hoops building ActiveRecord find conditions arrays dynamically depending on what arguments were passed in to the method. So I cooked up a simple little class to clean that up a bit:

# class to construct conditions for AR queries
# cc = ConditionsConstructor.new('foobar > ?', 7)
# cc.add('blah is not null')
# cc.add('baz in (?)', [1,2,3])
# cc.conditions
# # => ["foobar > ? and blah is not null and baz in (?)", 7, [1, 2, 3]]
class ConditionsConstructor
  def initialize(str = nil, *args)
    @conditions_strs = str ? [str] : []
    @conditions_args = args
  end

  def add(str, *args)
    @conditions_strs << str
    @conditions_args += args
  end

  def conditions
    [@conditions_strs.join(' and ')] + @conditions_args
  end
end

We recently released a very slick Firefox extension at Wesabe. It was written by my colleague Tim Mason, but I helped figure out one small piece of it—namely, how to do binary multipart POSTs in Javascript—and since it involved many hours of hair-pulling for both of us, I thought I’d share the knowledge. (Tim says I should note that “this probably only works in Firefox version 2.0 and greater _and_ that it uses Mozilla specific calls that are only allowed for privileged javascript—basically only for extensions.”)

One of the cool features of the plugin is the ability to take a snapshot of a full browser page and either save the snapshot to disk or upload it to Wesabe (so you can, for example, save the receipt for a web purchase along with that transaction in your account). The snapshot is uploaded to Wesabe via a standard multipart POST, the same way that a file is uploaded via a web form.

Tim was having trouble getting the POST to work with binary data at first, and he had other things to finish, so he wanted to just base-64-encode it and be done with it. I was reluctant to do that, as the size of the upload would be significantly larger (about 137% of the original). Also, Rails didn’t automatically decode base-64-encoded file attachments. But Tim had other bugs to fix, so I submitted a patch to Rails to do the base-64 decoding. I was pretty proud of this patch until it was pointed out to me that RFC 2616 specifically disallows the use of Content-Transfer-Encoding in HTTP. Doh. They also realized that it is a colossal waste of bandwidth.

Since Tim was cramming to meet a hard(-ish) deadline set for the release of the plugin, I offered to lend my eyeballs to the binary post problem. This could be a very long story, but I’ll just get to the point: you can read binary data in to a Javascript string and dump it right out to a file just fine, but if you try to do any concatenation with that string, Javascript ends up munging it mercilessly. I’m not sure whether it is trying to interpret it as UTF8 or if it terminates it as soon as it hits a null byte (which is what seemed to be happening), but regardless, doing "some string" + binaryData + "another string", as is necessary when putting together a mutipart post, just does not work.

The answer required employing Rube Goldbergian system of input and output streams. The seed of the solution was found on this post, although that didn’t explain how to mix in all of the strings needed for the post and MIME envelope. So here it is, in all it’s goriness:

var dataURL = this.canvas.toDataURL(this.getImageType()); // grab the snapshot as base64
var imgData = atob(dataURL.substring(13 + this.getImageType().length)); // convert to binary

var filenameTimestamp = (new Date().getTime());
var separator = "----------12345-multipart-boundary-" + filenameTimestamp;

// Javascript munges binary data when it undergoes string operations (such as concatenation), so we need
// to jump through a bunch of hoops with streams to make sure that doesn't happen

// create a string input stream with the form preamble
var prefixStringInputStream = Components.classes["@mozilla.org/io/string-input-stream;1"].createInstance(Components.interfaces.nsIStringInputStream);
var formData =
	"--" + separator + "\\r\\n" +
	"Content-Disposition: form-data; name=\"data\"; filename=\"snapshot_" + filenameTimestamp +
	(this.getImageType() === "image/jpeg" ? ".jpg" : ".png") + "\"\\r\\n" +
	"Content-Type: " + this.getImageType() + "\\r\\n\\r\\n";
prefixStringInputStream.setData(formData, formData.length);

// write the image data via a binary output stream, to a storage stream
var binaryOutputStream = Components.classes["@mozilla.org/binaryoutputstream;1"].createInstance(Components.interfaces.nsIBinaryOutputStream);
var storageStream = Components.classes["@mozilla.org/storagestream;1"].createInstance(Components.interfaces.nsIStorageStream);
storageStream.init(4096, imgData.length, null);
binaryOutputStream.setOutputStream(storageStream.getOutputStream(0));
binaryOutputStream.writeBytes(imgData, imgData.length);
binaryOutputStream.close();

// write out the rest of the form to another string input stream
var suffixStringInputStream = Components.classes["@mozilla.org/io/string-input-stream;1"].createInstance(Components.interfaces.nsIStringInputStream);
formData =
	"\\r\\n--" + separator + "\\r\\n" +
	"Content-Disposition: form-data; name=\"description\"\\r\\n\\r\\n" + description + "\\r\\n" +
	"--" + separator + "--\\r\\n";
suffixStringInputStream.setData(formData, formData.length);

// multiplex the streams together
var multiStream = Components.classes["@mozilla.org/io/multiplex-input-stream;1"].createInstance(Components.interfaces.nsIMultiplexInputStream);
multiStream.appendStream(prefixStringInputStream);
multiStream.appendStream(storageStream.newInputStream(0));
multiStream.appendStream(suffixStringInputStream);

// post it
req.open("POST", "http://yoursite.com/upload_endpoint", true);
req.setRequestHeader("Accept", "*/*, application/xml");
req.setRequestHeader("Content-type", "multipart/form-data; boundary=" + separator);
req.setRequestHeader("Content-length", multiStream.available());
req.setRequestHeader("Authorization", "Basic " + btoa(username + ":" + password));
req.setRequestHeader("User-Agent", "YourUserAgent/1.0.0");
req.send(multiStream);

Update: Spaces removed from multipart boundary per Gijsbert’s suggestion in the comments (thanks!).

I ran across a bug in Date::Format today, and after spending a few hours hacking away at a fix (the date/format.rb code is *uuuuugly* and *sloooow*…someone should really rewrite that. Better yet, rewrite it in C), I thought I’d submit a patch. So I grabbed the ruby_1_8 branch and lo and behold, my issue had already been fixed!

So the question now was how to monkeypatch the entire Date::Format module? Simply require-ing it as a plugin doesn’t work, since Date::Format is already loaded at that point. The trick then is to use load instead of require.

First, I tried this:

load 'date/format.rb'

(note: load needs the actual filename; it doesn’t have the magic that require does) but that gave me an error:

in `load': wrong number of arguments (1 for 0) (ArgumentError)

Turns out Rails::Plugin::Loader defines its own load, so:

Kernel::load 'date/format.rb'

et voila!

I am far more productive when writing Rails code than when writing Java. I just realized that one of the reasons for my lower productivity in Java is the need to recompile every time a make a change to a page on the site. In the 15 seconds or so it takes to recompile and redeploy to Tomcat, I get bored and am apt to go check my new favorite news site, popurls, or my RSS feeds, or (less likely) post to my blog. Suddenly those 15 seconds have become 5 minutes. And this happens many times throughout the day.

With Rails, I make a change, refresh my browser, and there it is. On to the next step.