My daughter, Santa Lucia, and the SF Chronicle...

Ariel, my daughter, goes to the East Bay Waldorf School, and a few weeks ago we were asked at the last minute to create a crown for her that would hold eleven candles. So my wife Linda spend a good two hours on Thursday night cutting up coat hangers and putting something together with needle-nose pliers and a glue gun.

Ariel was going to be Santa Lucia, the saint of light in the darkness of winter. Waldorf celebrates holidays from many traditions, including those of Europe. In European villages a girl is chosen to represent Santa Lucia, and she is dressed in robes and a crown of candles.

Ariel came home saying there was a photographer there flashing picture after picture in her face, and she said she would be in the paper. Sure enough, the following Saturday, there she was. And it wasn't just a little picture below the fold, but it was a huge picture filling the whole top fold of the Home section, as well as a thumbnail on the front page. The phone was ringing off the hook that morning with friends asking us if we'd seen the paper. You can read the whole story here at SFGate.

Doesn't she look beautiful - beatific is more like it. She is an amazing little girl (well, not so little any more). And yes, that is a towel on her head. Even a goddess of light needs to protect her hair from the wax...

A RESTful version of Amazon's SimpleDB - subbu.org

Hm, someone dug in to SimpleDB's REST API and found it to be not very RESTful. Doing everything over GET? Sounds like someone didn't do their homework...

http://www.subbu.org/weblogs/main/2007/12/a_restful_versi.html

Fear and greed: the engine of global warming

Corporations and governments seem to act with no moral compass, like refusing to do anything about greenhouse gases even as the world roasts.

Why is this?

Public companies are whores to the Wall Street pimps - if you don't come in with cash tonight, you get beat up. So corporate leaders who are afraid of losing their jobs and the bags of money that come with it will act against what might be their own moral gauge of what is right in order to bring in the top dollar to the company's stockholders.

In the same way, elected officials are highly dependent on funding to win an election, and so they also become whores to those who have money - namely large corporations. An elected official is driven by greed for power or the fear of losing their position to go against their own moral gauge of what is right.

Then there is the engine behind the pressure to act unethically: Wall Street (or your equivalent in your country). These big greedy investment institutions are leading us pell mell to destruction.

But who is Wall Street serving? Where do they get all their money?

From you and I. We want immediate results or we dump our mutual funds. We are afraid of losing money, and we want to make more money. Fear and greed.

So at all levels, it is fear and greed which is driving this, starting with you and me. And what drives fear and greed? Well, I don't know about anyone else, but for me it is tied to deep beliefs in my own smallness, in feeling limited, contracted, vulnerable.

So how do I fight global warming? I try to fight these feelings I have about myself, and remind myself of my own greatness and goodness. In this way I can start making choices that are not based on fear and greed.

An ancient scripture from India says: dhanam atma jnanam: "Knowledge of the Self is charity". As I follow this thread of corrupt and unethical behavior down to its source, I can see the deep wisdom of this statement.

Eventual consistency - following the Middle Path

One of the features of Amazon's SimpleDB is that a write to the database may not be immediately reflected to all readers, and that the user of the interface needs to be aware of this and work with it accordingly.

For those of us used to working with relational databases, which generally provide support for full read consistency, this is a somewhat revolutionary thought. But Werner Vogels, CTO of Amazon.com, explains in his blog that in scalable systems where data must be shared across machines, consistency inherently limits availability, particularly in larger systems where network partitioning is common:

Eric [Brewer] presented the CAP theorem, which states that of three properties of shared-data systems; data consistency, system availability and tolerance to network partition one can only achieve two at any given time. A more formal confirmation can be found in a paper by Gilbert and Lynch.

A system that is not tolerant to network partitions can achieve data consistency and availability, and often does so by using transaction protocols. To make this work, client and storage systems are part of the same environment and they fail as a whole under certain scenarios and as such clients cannot observe partitions. An important observation is that in larger distributed scale systems, network partitions are a given and as such consistency and availability cannot be achieved at the same time. This means that one has two choices on what to drop; relaxing consistency will allow the system to remain highly available under the partitionable conditions and prioritizing consistency means that under certain conditions the system will not be available.

Both require the client developer to be aware of what the system is offering. If the system emphasizes consistency, the developer has to deal with the fact that system may not be available to take for example a write. If this write fails because of system unavailability the developer will have to deal with what to do with the data to be written. If the system emphasizes availability, it may always accept the write but under certain conditions a read will not reflect the result of a recently completed write. The developer then has to make a decision about whether the client requires access to the absolute latest update all the time. There is a range of applications that can handle slightly stale data and they are served well under this model.

He then goes on to describe one form of consistency, called eventual consistency, where there is a time lag between an update and the ability of all clients to read that update.

Eventual consistency. The storage system guarantees that if no new updates are made to the object eventually (after the inconsistency window closes) all accesses will return the last updated value. The most popular system that implements eventual consistency is DNS, the domain name system. Updates to a name are distributed according to a configured pattern and in combination with time controlled caches, eventually of client will see the update.

I love this -- eventual consistency. I think this is the way things actually work on the large scale in the real world. And when you design a system this way, it allows you to, in a sense, breathe, and you end up with something that is much more tolerant and scalable.

It reminds me of the story of the Buddha. He was trying to find God, and he was so strict and severe in his austerities that he was practically starving. He was feeling lost, wondering why all this effort wasn't leading him to God. Then he overheard a man instructing a student how to tune a stringed instrument: "don't tune the string too loose, or it will make no sound. Don't tune it too tight, or the string will break." And Buddha saw this as a message to him to follow the Middle Path, to relax and breathe while still following a healthy discipline.

In a way, requiring strict consistency is like tuning the string too tight. The constraints are so severe that the system is brittle and easily breaks.

So, Grasshopper, when building scalable systems, may we follow the Middle Path of Eventual Consistency.

Ahhh, vacation

Amazing, I am taking three weeks of vacation. How often does this happen? So, I know you all are hanging on my every word, and will feel deep sorrow at my lack of posts, but, alas, you must carry on without me. Have a wonderful holiday and a happy New Year!

See you in 2008!

Throw out your relational db for all but the tough queries

Amazon Simple DB continues the trend that for many many use cases the relational model and SQL queries, even pure data consistency, are overkill. Don MacAskill on his SmugMug blog has this to say about Simple DB:

Like many of you, I’m sure, we’re using much of our RDBMS as a fairly simple data store and aren’t using many advanced RDBMS capabilities. All of those queries could just use SimpleDB and then we could devote our DB iron to just the rare complex queries. We’re not alone - tons of web apps are gonna love this.

I think I have to agree with him here. The relational model is powerful, but it is hard to learn, hard to do right, hard to scale, tends to sacrifice performance for consistency, and adds a lot of complexity to your application. And more and more people are discovering that in many cases, you just don't need it.

Vinod Khosla: We Can Replace Oil in 20 Years or Less

Earth2Tech blog (a great blog on green tech) gives us a video of Vinod Khosla on removing dependency on oil and coal in 20 years. "Don't listen to what most environmentalists say, because they come up with ideas that can't scale"

http://earth2tech.com/2007/12/06/vinod-khosla-video-clip-we-can-replace-oil-in-20-years-or-less/

SmugBlog: Don MacAskill » Companies That Listen: Sun

When you're working hard and you see all the warts, sometimes it's great to read this kind of fan mail. Made my day.

http://blogs.smugmug.com/don/2007/12/13/companies-that-listen-sun/

Build a company with no office and no hardware - no kidding

With Amazon SimpleDB, I think it is now entirely possible to build an massively scalable web-based product without investing in a single piece of server hardware.

Think about that. The up-front costs for starting up a company have never been smaller. You don't even need an office (I'm working at Central Perk this morning).

Sometimes I wish I had a good idea, just so I could have fun building a company like this.

Solid state disk array delivers 800 MB/s with .1 ms access time

I wrote a while back about a dream laptop that included a solid state drive. These things are compelling because they are damn fast, and use a lot less power than a disk drive. As a developer working with databases and NetBeans (which is very I/O intensive with all the class indexing and module loading it does), a fast disk means a fast life.

So I dream about getting a solid state drive - it's just they're so expensive - $650 for 32GB - ouch. But when the prices start going down -- I'm there!

The other area where this could kick the pants off of things is in the server room. Dominic Strippoli does the numbers on a RAID array of solid state drives. This setup ain't cheap, but with the right controller installed (the standard one just couldn't keep up), he was able to get 800 MB/second sustained read with .1ms access times. Using these drives, he got Windows Vista to boot up in five seconds!

Imagine running your database on an array of these things. Yowza! I hope Sun storage is keeping an eye on this trend...

Open source databases: the looming tsunami?

A couple of blogs came across my reader today. The first was by Allan Packer, starting what looks like a series of blogs where he asks the question bound to get him a lot of hits: are proprietary databases doomed?

With the combination of essential features, improved performance, robust support, and compelling price, OSDBs today bear a striking resemblence to Linux a few years ago. Many believe that the wave looming on the horizon is a tsunami.

On the same day in my Dzone database search RSS feed I encountered a link to Brian Hurt's blog, Postgres for the win! (I just noticed it's a 2006 entry, so I guess it's been re-discovered :)).

Brian is doing some serious work with PostgreSQL, and PostgreSQL is succeeding for him

At this point, the only complaint I have is that Postgres is doing so much with so little that I’ll never get cool hardware to play with. I’ll be stuck with cheap low-end boxes forever. Sigh.

The next person who claims “Postgres is slow” is going to get an earful from me. Maybe Postgres 7.0 was slow, I don’t know- but I know for damned sure that Postgres 8.1 isn’t.

If you read Brian's blog, you'll see PostgreSQL is succeeding for him not just because it is fast and can scale, but because he has access, in terms of documentation, community, and the source code itself. It's open-ness gave him the ability to figure out how to solve his problem.

What's my take on this? Well, I think it may happen sooner, or it may happen later, but at some point, the open source databases are going to be just fine for a majority of users. And if you can get enterprise level support, well, all the better!

Illegitimi non carborundum

I read a great editorial by Sean Olender this weekend in the Chronicle on the massive fraud taking place within financial institutions around sub-prime mortgages, and how the government is apparently working with the institutions, in private, to basically prevent the instigators of this fraud from being held accountable.

This seems to happen over and over again. Whenever the economy is doing well, we don't hear about these things. But at some point the apple cart falls over, and everyone sees the ugly crawling bugs underneath. And usually the crooks get away, as they are in cahoots with our government. I find myself surprised, yet not surprised, outraged and resigned.

I hate it that government is in the pockets of people with money. I hate it that criminals of the poor classes are punished with a vengeance, filling up our jails, while white collar criminals are regularly let go with a slap of a hand while they walk aways with millions or billions of our hard-earned money. And our government is one of the better ones in terms of corruption and criminal activity.

I feel myself powerless to do anything about all this. I ask myself: what's the point of trying to live a good life if we're constantly being screwed over by the greedy and immoral and power-hungry?

But then I recognize that all this impotent anger really isn't helping me or anyone else. Not that I won't take action where I can, but to reduce my own stress I gently remind myself to step back, and take on the vision of Shiva, the vision of eternity. I remember how ancient and eternal this universe is. I remember that Death visits us all, and does not recognize or pay homage to wealth, power, race or position. I remember that in the game of life, Nature always wins.

From this perspective, it becomes clear to me that the best I can do is live a life in harmony with the eternal throb of Nature. You can feel it in the emptiness of nature - in the desert, or on the ocean, or in the mountains. That deep throb of stillness and eternity.

From this perspective, I begin to see that all of this is happening as a great dance. It's like watching the waves on the ocean, a constant play of shadow and light. And I remind myself: illegitimi non carborundum - don't let the bastards grind you down.

Tweaks to make Solaris Express work in VMWare

A quick blog to tell you two things you need to do to get Solaris Express to work with VMWare Fusion:

First, if you're using build 79, there is a bug where, during installation, you need to give Solaris just a little bit less than the available disk space. If you don't, the installation fails. This is logged as a P1 bug and hopefully will get fixed by the next build.

Second, after installation, VMWare will complain that you're trying to run a 32 bit operating system in a 64 bit host (or vice versa, I can't remember). To fix this, in add this line to the .vmx file in your image directory:

monitor_control.disable_longmode = 1


JRuby performance: impressive improvements

Tim Bray pointed me to The Great Ruby Shootout, where Antonio Cangiano does an incredibly detailed analysis of the relative performance of the various popular Ruby runtimes out there.

There is a lot to look at, and I recommend you take a gander, but I wanted to highlight this paragraph:

JRuby is truly impressive, being the only implementation that’s able to pass all the tests without raising errors or timing out. Not only that, but in my previous benchmark, JRuby was several times slower than 1.8.5, whereas now it’s faster than the main Ruby interpreter. There have been huge improvements going on over the last few months. Really, just compare the times in the old table with the ones in the new table: unbelievable progress. JRuby is the only one, aside from 1.8.6, which is able to run Rails in production. I wouldn’t be surprised if JRuby turned out to be, on average, just as fast as Ruby 1.9, in a year or so.

That's some pretty impressive news. Kudos to Charlie, Thomas and the entire JRuby community!

Burningbird » Biofuel and Global Warming

Three cheers to Shelley. The more I hear, the more I don't like. Sounds like a boondoggle for get-rich-quickers. We need truly sustainable energy, not a game of whack-a-mole with our natural resources.

http://burningbird.net/environment/biofuel-and-global-warming/

Now THAT is the way to learn math

Our daughter is going to a Waldorf School in the East Bay. We heard about Waldorf from a friend of ours who is a teacher, and the more we heard, the more we liked. Ariel is in second grade, and she and we are liking this school and the way it teaches children more and more.

It's based on a very rich philosophy called anthroposophy, which I have barely scratched the surface of understanding. An important principle is that you involve the entire being of the child when teaching them - physical, emotional, mental and spiritual. You don't just sit them at a desk and give them homework and tests. As some examples, they have a movement class every week, and their first exposure to times tables is by doing step dances in a circle, emphasizing the two-beat, the three-beat, the four-beat, etc.

The latest revelation for me was how she is learning her times tables visually, and is getting her first introduction into geometry. Take a look at the beautiful 10-pointed star below that Ariel drew



Note how the circle has the numbers 0 to 9 around it. The way she draws the star is she places her pen at '0', and then says "seven times one is seven" and draws a line to the 7. Then she says "seven times two is fourteen" and draws a line to the 4, and so on. In this way she draws a ten-pointed star, and then she spends time coloring it in to make it beautiful.

In the last week she has done this about six times at home, and this is without having any homework assigned. She just loves to do it.

I find myself really happy for her, and a little bit jealous - I am terrible at math, in part because I struggled with it at school (in particular my times and addition tables) and now I find myself freezing up when trying to do basic math operations. If I had learned math this way, I think I would have loved it...

tecosystems » “Hybrid” Source, MySQL, and the Economics of Open Source

Stephen O'Grady does an interesting analysis on MySQL going "hybrid source" with their Workbench product. I really like the principle of "pay at the point of value." I would think in most cases that's services and support, but let's see if this works...

http://redmonk.com/sogrady/2007/12/03/mysql_workbench/

A new look

Those of you who actually go to my blog page rather than read my posts through an aggregator will notice I just did a face lift. I just thought my old layout was a bit bland and nondescript.

I'm not a UI expert and I don't have hours to spend on this, but I pulled a template called "K2" from a site called Gecko and Fly (thanks, guys!) and tweaked the CSS a bit to my liking.

At the same time I changed the name of the blog, I kind of like the new one. Makes me smirk when I read it anyway. Often when people ask me how to spell my last name, I say "it's like you were going to Vancouver." Sometimes people get it, sometimes they don't...

NetBeans 6.0 is out!

NetBeans 6.0 is now available from the NetBeans web site. Congratulations to a huge effort by a lot of people.

I've been working on 6.0 for a while now, and I would have a hard time going back. Some of my favorite features include:

• Really fast detection of errors while I code
  
• Showing all the errors in red and all changes in green on the side bar, and if I click on a red or green mark it takes me to that line.
• Greying out of unused variables, and color-coding local variables
• No more two-phase refactoring, unless I request it.
  
• Solid Ruby support, including cool things like auto-completion of ActiveRecord methods by using migration data.
• PostgreSQL and MySQL JDBC drivers included and registered out of the box
  

In the Database Explorer fixed two things that were real annoyances to me (and others):

• No more quoting in SQL DDL! (this caused no end of confusion for users)
  
• Ability to remember password for DB connections (I was so tired of having to re-type passwords

There were also a lot of things that happened under the covers which perhaps weren't so noticeable, including integration of a number of packs/plugins, including VisualWeb Pack, Enterprise Pack, Mobility. and UML. This took a lot of work, and the amazing thing was it was done in a way that maintained the over consistency and solid out-of-the-box experience that is one of NetBean's real strengths.

So, anyway, kudos to all, and now onto the next release! :)