Log in

No account? Create an account
entries friends calendar profile my webpage Previous Previous Next Next
The First Law of Development Thermodynamics. - Tina Marie's Ramblings
Red hair and black leather, my favorite colour scheme...
The First Law of Development Thermodynamics.
In one of my RSS feeds today, there was an interview with one of the Twitter developers. He said, amongst other things:
None of these scaling approaches are as fun and easy as developing for Rails. All the convenience methods and syntactical sugar that makes Rails such a pleasure for coders ends up being absolutely punishing, performance-wise. Once you hit a certain threshold of traffic, either you need to strip out all the costly neat stuff that Rails does for you (RJS, ActiveRecord, ActiveSupport, etc.) or move the slow parts of your application out of Rails, or both.
I've not done anything with Ruby or Rails besides install it, but my first thought was, "Duh! You expected Rails to scale?". Someone tried to talk me into a Rails project a year or so ago, and I did a lot of reading about it, but decided not to take the project for other reasons.

Every level of indirection that you put on top of the CPU costs you something. If you want blazing speed, write in assembly. One level up from that is something like C, which used to be considered a high-level language, but now people write drivers in it. By the time you go from C to Rails, you've added dozens of levels of indirection. You've got all the overhead of Rails, the overhead of Ruby, the overhead of an interpreter, the overhead of your database, not to mention all the levels of error handling that you need to keep this all working.

What's the payoff? Ease of development. Apple has it right - "Easy is hard". It's the Law of Conservation Of Developer Energy. If you want fast, scalable, and stable, you have to work hard at it. If you want really fast, really scalable, and really stable, you have to work really hard at it.

Do you always need that? No. If I'm writing a photo manager to share with my relatives that's going to run on a server class machine, why do I care how fast it is (within reason)? It will be fast enough. That's where things like Rails shine - I want to put something together quickly, and I can throw some reasonable hardware at it. I don't want to pay the cost of coding it in cgi-bin Perl - I just want it to work.

And this is where people get into problems with things like Rails. They want to create something. And here's this framework that promises them that it'll make it just work, with no need to worry about all those little details. They put it together, it works for a while, and then they hit a limit. You can code around the minor ones, or throw more hardware at it, but eventually you'll hit a major limitation, and then you either have to live with that limit or go back to the drawing board.

Scalability, speed, and security are all in the details. Any language or framework that speeds development by hiding those details from you is not going to be feasible for a large-scale application. There is no such thing as a free lunch. Except maybe at Google.

Current Mood: sleepy sleepy

1 comment or Leave a comment
alioth1 From: alioth1 Date: April 16th, 2007 07:06 pm (UTC) (Link)
...and it's sometimes so refreshing to write stuff in Z80 assembler. (Although I have also retargeted the z88dk, a C compiler and set of libraries like stdio and friends, to my Z80 computer - because well - trying to do floating point in asm will make you want to claw your own brain out).

I think the epitome of awfulness in layers of abstraction has to be when you add everything up to equal AJAX. Particularly the JavaScript bit. And this has been inflicted upon us because Microsoft just couldn't bring themselves to do Java, so we end up with entire user interfaces written in JavaScript. I'm sorry, but making UI elements by writing HTML fragments to a document object model is a kludge of unbelievable kludginess. This is what Java applets were for. Anyway, enough of that - but when you wind up with the client side of the AJAX stuff for anything moderately complex, the performance is absolutely terrible. A browser based GUI runs like Windows 3.1 did on a 25MHz 80386. On a 3.6 GHz Xeon. And you have to be very careful in the order at which you lay out controls or the performance is utterly unbearable. Most people don't see this because their toolkit is hiding it - then they get ambitious, and well - you know what the inevitable result is.

The inevitable result is my 4MHz Z80 single board computer could have a faster client. It won't be pretty, and it'll be character based, but a 4MHz Z80 will be beating a 3.6GHz 64 bit monster with gigabytes of RAM :-)

I think that all programming courses at university level should involve the students touching bare metal - assembly language. Doesn't matter which one - preferably something simple like an 8 bit arch (the Z80 is ideal, since it's still manufactured, and you can breadboard a simple machine with a straightforward keyboard, a few devices that can raise interrupts, and a simple frame buffer) so they all gain an appreciation for what's really happening behind those layers and layers of abstraction. They will also learn on a gut level why buffer overflows are bad the first time they overwrite the call stack or tromp over a bunch of other variables, and about sanitizing user input when their program ends up doing something weird when the professor tries to break it and they have to spend hours debugging it.
1 comment or Leave a comment