Time to 'Fess Up

I played a little joke a while back and now it's time to both 'fess up and to explore some of the sadly all-too-explicable reactions to it.

A couple of weeks ago I posted a little ditty on my rants feed about how I had proof that Ruby sucked as a scripting language.  Very little of what I said in it was to be taken seriously at all and I signalled as much in a variety of ways.  To my intense utter lack of surprise, there was a segment of the software community that didn't take the post in the spirit in which it was so obviously intended.

The intent

For those (Hello, HN!) who could not discern it, the intent was simple: I was mocking a certain style of blog post in which people "prove" things using such vapid and/or underhanded techniques that nothing is proven at all.  There were several clues available that this was not a serious attempt at benchmarking.  To wit:

  1. The headline was very obviously an exaggerated mockery of sensationalist headlines from tabloid newspapers and click-baiting tech blogs.
  2. The name of the benchmark was bogobench.
  3. The driver for the loop was called bogoloop.

I think that anybody who doesn't have their head stuck firmly where they can most directly snort their own intestinal methane should recognize the intent from these clues alone, not to mention some of the more subtle clues that were in there as well (among which were my choice of languages to compare).

Some people got the joke

  • The people in the Awk community got the joke.
  • The people in the Lua community got the joke.
  • The people in the Erlang community got the joke.
  • The people in the Prolog community got the joke.
  • The people in the YFL community got the joke.

Even some people who weren't in any of these communities got the joke.  The joke was recognized, appreciated for what it was and, indeed, expanded upon.  Some of the languages I added to the script—Rexx, for example—came from suggestions of people who got the joke, you see.  Indeed I got flooded with so many suggestions for languages I could add to the benchmark that I had to start fending them off with "the source is there; feel free to blog your own version".

Others…didn't

I don't think my disdain for reddit and HN is exactly unknown.  Of course the reaction to this (obvious, note!) joke on HN explains much of why I hold HN in such contempt (and similar experiences with an added dollop of sociopathy are why I hold reddit in such contempt).  Where people in the communities mentioned above (and a few others who weren't in any particular community) understood and expanded upon the joke, the HN response, while the link was still permitted to live, consisted in even parts of pointing out the blindingly obvious, dishonest prestidigitation and just plain not getting things.  I guess being a hipster startup-wannabe strangles your brain cells or something.

Pointing out the blindingly obvious

I’ve no idea how fast Rubinius is compared to MRI 1.9.3 or 2.0 but it’s not exactly the standard interpreter. I wonder why it was chosen?

Oh!  So close to getting it and then fucking it up!

This benchmark is totally stupid.

Wait, you're telling me that a benchmark called bogobench using a script called bogoloop might be totally stupid?  Please, do share your pearls of wisdom.  <frasier>I'm listening.</frasier>

So that means this post is either satire…

Oh!  Someone on HN actually got it!

…or intentionally provocative idiocy.

Or not.<sigh />

Dishonest prestidigitation

You're going to think I'm making this one up to make HN's self-proclaimed "hackers" look bad, but I'm not.

On my macbook:

$ ruby -v

  rubinius 2.0.0.rc1 (1.8.7 release yyyy-mm-dd JI) [x86_64-apple-darwin12.2.0]

$ time ./runner.sh ruby bogobench.rb

  Running 'ruby bogobench.rb' 100 times…
  …done.

  real 0m0.732s
  user 0m0.289s
  sys 0m0.273s

Linode with Ubuntu:

$ ruby -v

   ruby 1.9.3p392 (2013-02-22 revision 39386) [i686-linux]

$ time ./runner.sh ruby bogobench.rb

  Running 'ruby bogobench.rb' 100 times…
  …done.

  real 0m3.701s
  user 0m2.486s
  sys 0m1.202s

I'm going to do the guy the honour of assuming he's being more cleverly sarcastic than I was being.  Because if he intended this to be a serious rebuttal there's quite a few problems here, to wit: he took a comparative benchmark, ran it on different hardware (in this case likely a MacBook with an SSD drive which is of-fucking-course going to have a major impact on load times!), stripped all of the comparisons—and with it the entire point of the even this bogus benchmark—and then supplemented this by running, again without comparisons, the same test on different hardware yet again using a completely different version of Ruby.

 

As I said, I'm assuming that this was more subtle satire in and of itself because it would be really insulting to call him a disingenuous, fatuous shithead.

Just plain not getting things

Heaven knows that saving milliseconds on printing "Hello, world" is a major priority with me.

Milliseconds?  I guess half a second or so is "milliseconds".  About five hundred of the fuckers!

 

I've rarely found scriping languages most important aspect is speed.(sic)

I guess that's why this benchmark, as bogus as it was, was explicitly not measuring speed but, rather, load time.

Runtime speed is the defining metric of a scripting language, then?

I guess that's why this benchmark, as bogus as it was, was explicitly not measuring sp… is there an echo in here?

Write it in C or assembler and it will even be faster. That doesn't make them better choices for a script necessarily. This is one small part of picking the right path for the task at hand.

The only possible reaction to this ninnery.

So why were you so mean to the HN idiots?

Well, I didn't actually target this for HN.  I don't do HN except to occasionally look at links other people toss my way so they can see me shake my virtual head in disbelief.  One of the people who did get the joke posted the link to HN, presumably to share the joke (and not realizing that HN regulars lack the capacity to recognize jokes that aim at their holy cows).  I will instead answer the question you should have asked: "Why did you make this joke?"

 

Well, the origin of this joke was a mistake.  My own mistake.  I was doing some stuff in Ruby and was getting annoyed by seriously long startup times.  It turns out that I had used rvm to switch to rubinius and then plumb forgot that I'd done so.  I decided to benchmark the load time for my own edification and compare it to the proper version of Ruby I wanted to use (1.9.3).  The difference was … surprisingly massive.  That's when I got the idea in my head to compare the load times of a bunch of languages (because Rubinius' load time came as a huge surprise to me).  Seeing Rubinius weigh in at almost 50 seconds while other scripting languages (and, to be fair, MRI Rubies) all come in at under a second then sparked the idea of including other non-scripting languages culminating in Java.

 

That's when I realized I had the makings of a good joke and decided to format the results as one of the typical stupidly-written, stupidly-run, stupidly-explained (and often stupidly-dishonest) benchmark blog entries that so litter the web.  I made the title obviously bogus, I gave the files names that hinted at the underlying bogosity, I inserted the key information needed to explain the bizarre results and then sit back and laughed along with others in various communities who read it (and suggested expansions).

 

Then I laughed at, instead of with, the HN community when the link was posted to that cesspool of wannabes.

TL;DR

Don't trust benchmarks you see on the web.  Most benchmarks published are incompetent and/or dishonest.  Very few of these wil flatly tell you they're being dishonest (bogobench, remember?) and most incompetent people don't realize they're incompetent.  If you see benchmarks, get the source, run them yourself, then look very carefully at both the results and the method being used.  Make sure that you're seeing measured what's claimed; make sure that what's being measured is meaningful.

 

Or better yet, benchmark based on your own needs, not on someone else's incompetently/dishonestly-written microbenchmarks.