Heck with SQL: is a persistent hash all you need?

Jamie Flournoy compares ActiveRecord to VB: "The easy stuff is easy, but the hard stuff is impossible," he quote James Gosling as saying about VB.

One of the things Jamie says in this very long, well-thought-out article, by someone who obviously has experience with both Rails and databases, gave me pause.

Jamie referred to some of the leaders of the Rails community saying that SQL is a design mistake and that most web apps only need a persistent hash.

This statement follows shortly after me reading an article by Diego Parilla about the impact of ORM on the database, and finding out that Amazon uses basically a highly available, persistent hash for much of its database storage on the web tier.

Michael Stonebraker is even saying that relational databases can not meet everybody's needs any more, and talks about the need for specialized data stores for specific use cases.

I actually like a lot of this. We need to simplify, simplify, when it comes to building web applications - I've been appalled at what a huge mess it is for quite some time, and have found myself avoiding building web apps if at all possible because of this. Also, SQL ties you to a database (yeah, I know, it's a "standard"), and also ties your application to a particular incarnation of the data model in the database. So for all these reasons, eliminating the need for SQL is goodness.

My concern is that if you're not careful, you start with a hash table, but then start implementing your own database on top of it. Jamie mentions this, and I have seen this too:

Satisfying queries is the database’s job, period. It’s just hideously slow to try and do an inner join in the application across a network link to a database. If you find yourself doing this, that’s a pretty good sign that your architecture is broken.

It's easy to think that all you need is a hash table, and when you're building simple, basic web apps, that's probably all you need. Heck, that's all Amazon needs, for the most part. But there are times when you want more. You really do need to do a complex query, or a stored procedure, or text search. You want to be able to run useful reports without having to do joins and sorts within your application.

I think Rails is getting a lot of things right by hiding a lot of complexity from you (as do the Java ORM technologies).

But I agree with Jamie that for those times when you need SQL, it's nice to have the option -- at least until ORM gets so friggin' smart that writing in SQL is like writing in assembly - sure you could do it, but who wants to, and why?