Characterization Tests

I love this very realistic and somewhat funny article on how to test stuff where you have no clear idea exactly how a piece of code should behave, but regardless you do want to assure that if you make changes it keeps behaving the way it used to.

http://www.artima.com/weblogs/viewpost.jsp?thread=198296

Do Messages Want to Be Asynchronous? - Beautiful Code

A nice article which comes to the conclusion that asynchronous messaging is the logical end result of applying "Tell, Don't Ask" methodology to your systems.

http://beautifulcode.oreillynet.com/2007/10/do_messages_want_to_be_asynchr.php

Tiny useful iPhone headset feature

I got a new iPhone recently, and I had been trying to find a good headset that could double as headphones, handle switching between calls and music, had good sound, was easy to put in my pocket, etc.

I had bought one version, returned it dissatisfied, and was trying another one before I left the Apple Store. The sound was horrible, and I complained to a "Concierge". He said "well, why don't you just use the microphone on your headset that came with the phone?" Huh? What button?

Well, if you look closely at the picture, you'll see it on the right. A tiny little microphone. And if you squeeze it between your fingers, it will answer/hang up the phone or pause your music.




I just tried it out and it works perfectly. I am duly impressed.

Microsoft Damage Control

I am downloading the MS SQL Server JDBC driver for some testing I need to do, and saw this in their license agreement:

LIMITATION ON AND EXCLUSION OF REMEDIES AND DAMAGES. You can recover from Microsoft and its suppliers only direct damages up to U.S. $5.00. You cannot recover any other damages, including consequential, lost profits, special, indirect or incidental damages.

That is so generous of them - they are willing to pay your $5.00 if their software causes some sort of damage. For instance, the JDBC driver has a bug that causes data corruption in my central IT database, and I can get $5.00 from Microsoft to cover the damages.

Generous to a fault, those guys are. What will the stockholders have to say about that?

Broadway Photo - Stay Away!

I was this close to ordering our new video camera from Broadway Photo after seeing their amazing prices at half the going rate. Just before I clicked the purchase button, I thought "hm, at those prices, I bet there are no returns" and decided to research their return policy.

I ended up Googling for it, and to my horror got a list of folks saying this shop was a complete scam. I was astounded. They looked totally legit, a very nicely put together web site, but these folks are reputed for calling you to buy very expensive accessories, never shipping the produce, or shipping you used versions of it. I couldn't believe how close I was to losing a very big chunk of cash to these guys.

I'm amazed they're still in business, but I thought I'd do my part to expose them...

E Pluribus Unum

I read Barack Obama's talk yesterday, and it really struck me. I know everybody is talking about how it has to do with race, but his real theme, if you look more closely, is about forming "a more perfect union" - the vision of the Founding Fathers that this country is on a path, a path of union.

I recently read a biography of Abraham Lincoln, and he also studied very carefully the Constitution and tried to interpret the intentions of the founders of our nation. He saw that these founders recognized that the nation, the union, was not perfect, and that although they believed slavery was anathema to their vision of the country, it had to be allowed to exist until a future time.

It reminds me of the story I heard of a cathedral in Europe where they designed it to hold a massive dome on the top, when nobody knew how to create such a dome. But they knew it would take 200 years to build the cathedral, and they had this astounding faith that in that time people would figure out how to build the dome. And that is actually what happened.

I was thinking about this theme of "a more perfect union," and it struck me what the motto is on all of our money - E Pluribus Unum - Out of the Many, One. The Founding Father's vision was not just about uniting a number of separate states into one country, but it was about uniting people - different people from different countries, religions and philosophies into one people.

I think this is what is inspiring people about the vision Mr. Obama is holding for us - he continues to emphasize what is common among us all, about what is good in this country. And he says, unified, we can meet and address our problems. Divided, we fall. As Abraham Lincoln said, "A house divided cannot stand."

Unity is such a common theme in religions, and so it should be. The Indian scriptures say that the root cause of fear is the feeling of being separate, being different. It is the sense of difference, of separateness, that pits man against man, tribe against tribe, husband against wife. But in finding what is common, seeing that we are all the same, promotes love and brings strength to all.

Mr. Obama said something else that struck me: that focusing on these differences of race, of color, and so on, are distractions. And I realized that distraction is key to those who would not have us look at our real problems and solve them together. There are those who benefit from the status quo. And the best thing to do is to keep us distracted, focusing on things like gun control, death penalty, issues of race and sex, immigration, and all these things that keep us separate from and afraid of each other.

I think the Internet has its problems socially, but it has done one thing. It has enabled the individual to participate in democracy like never before. Obama's campaign is it is not financed by the traditional money interests, by lobbyists and large donors. It is financed by you and me, by one small donation after another. So Obama is not beholden to these entrenched interests. He would not have made it, with the platform he is putting forth, if he were getting his money from traditional sources.

It also strikes me that because of the Internet and the read/write web, people can watch the full video of his speech on YouTube, and are not spoon-fed it by the main media. So people can listen to it all, take it in, think, and let it affect their perception of what this nation is about.

And that gives me hope - perhaps there is a way we can bring our democracy - of the people, by the people, and for the people - back to the people, to a place where our dialog is about unity and solving our problems together, not tribal bickering. Audacious hope, yes, but there it is :)

I offered this prayer from the ancient Rg Veda for Kenya and now I offer it for our United States, and for the world:

Let us be united;
Let us speak in harmony;
Let our minds apprehend alike.
Common be our prayer,
Common be the end of our assembly;
Common be our resolution;
Common be our deliberations.
Alike be our feelings;
Unified be our hearts;
Common be our intentions;
Perfect be our unity.

Jeremiah Wright's Speech "Audacity to Hope"

Another perspective of the polarizing figure of Reverend Wright...

http://andrewsullivan.theatlantic.com/the_daily_dish/2008/03/for-the-record.html

MySQL support in NetBeans 6.1

I was going to blog about this, but James beat me to it.

Thanks, James! I think he covers it really well.

I just wanted to comment on why we did this. It's nothing that revolutionary, but if you have MySQL installed on your machine, then these features can help you to get started very quickly. In particular, you can get a connection to MySQL very quickly and start exploring, and you can easily create the sample databases used by tutorials and blogs throughout NetBeans.

Other things of note that didn't make it into the beta but will be there by the time NetBeans 6.1 releases:

• I fixed a bug so that you can again create stored procedures and triggers in the SQL editor. This broke a when we started executing each statement separately. No, if you say 'delimiter X' where X is any string besides ';', then the semi-colons inside the procedure/trigger definition won't confuse our parser. This is the same approach used by the mysql command-line tool

• I'll be putting up a module in NetBeans update center that will make it very easy to create the Sakila sample database in NetBeans

What about the next release of NetBeans? Well, that depends on a lot of things, including what results from my discussions with the MySQL team, what I hear from you, and how many people are available to contribute. Two things I'm thinking about are improving our SQL editing support (including some support embedded within the PHP and Java code editors) and, thinking big here, some form of database schema refactoring.

Addiction and Wall Street

As I read about how the money-and-power addicts of Wall Street (fueled of course by investor's fear and greed), and read about how the government, and thus our tax money, has to come in and save the day, particular for a slimeball company like Bear Stearns, it is very frustrating. And it reminds me of what I know about addiction.

The pattern of addiction is complex, but one aspect of it goes like this. The addict splurges, indulging in whatever it is he or she indulges in. It feels great. It feels like nothing can go wrong. And then at some point the bottom falls out. The addict gets caught in a scandal, or loses his job, and/or the spouse or partner says they've had enough and threaten to leave.

The addict begs, pleads, says it will never happen again. They promise to reform themselves. And the enabling partner decides to give it another shot, and stands firm with the addict.

One would hope that the inflictors of this disaster upon our economy pay the price, or at least don't walk away with bills in suitcases, and that financial regulation is put in place to prevent this sort of thing in the future.

And we'll hope the addict has reformed themselves and won't do it again. I mean they promised, right?

Say Doh! and get a Pez

IMG_0024.jpg, originally uploaded by David Mezzo Van Couvering.

Central Perk, the coffee shop I sometimes frequent for wireless work, has a vast sea of collectible toys, including this shelf of Simpson's Pez Dispensers...

Of note: Jumpchart -- HTML design using Wiki syntax

I was pointed to a very interesting site called Jumpchart, that lets you design web pages using Wiki-like syntax, and then export the result as a set of HTML pages and CSS. A very nice way to quickly pull together a wireframe of your app and then use that to get started. Take a look...

TechCrunch: Yahoo Embraces Hadoop

This is very encouraging, given how important search is and what a proprietary stranglehold Google has on the underlying technology.

http://www.techcrunch.com/2008/02/20/yahoo-search-wants-to-be-more-like-google-embraces-hadoop/

Sittin’ on the CouchDB « ZefHemel.com

A very nice blog describing one person's experience of working with CouchDB. I like it because he compares this with using relational databases and helps a RDB user "migrate" his concepts to CouchDB

http://www.zefhemel.com/archives/2008/03/11/sittin-on-the-couchdb

Threads in a process and servers in a cluster: same issues, different context

I was just re-reading my last two posts, and it triggered an epiphany: language architectures like Scala and Erlang and server architectures like the H-Store, Gigaspaces, memcached (and others) share a very common model: each unit runs independently and collaboration between units is accomplished through asynchronous messaging. This messaging can be thought of as sharing data (e.g. asynchronous replication), or as sending messages that contain data, but at a certain level the result is the same.

Developers struggling with threading, consistency and locking issues in a single process are actually dealing with the same issues that system architects are struggling with when trying to figure out how to scale out a system where data needs to be shared across processing units. Concurrency, consistency and performance - same issues, different context.

Just thought I'd share that little lightbulb moment.

Stonebraker's H-Store: There's something happenin' here

It was fascinating for me to read this overview by Curt Monash of Michael Stonebraker's new database architecture for the web called the H-Store. It is a complete overturning of the traditional database architecture. And I quote:

On the system side, some of their most radical suggestions include:

• No disks or other persistent storage at all.
• No multi-threading.
• No locks
• No redo logs (and perhaps not a lot of undo logs either).

Wow. And here's something else: the engine has a single API: execute transaction(parameters). In other words, all it does is stored procedures. And the language they are proposing for these stored procedures? Ruby.

We're not talking small changes here. What is motivating these kinds of changes? The web. In particular, traditional databases and database APIs like JDBC and ODBC are tuned for a very old style of application, where the user is working over a set of data on a single connection which is holding some level of lock on the database. In Sybase we used to call this "browse mode," and then this got standardized in SQL with the concept of cursors. I remember we had to do a lot of work in the engine and the client API to support browse mode/cursors, because our customers were demanding it.

But now in many cases, this style of interaction with a database is completely irrelevant. Cursors just do not make sense for a web-based application, where the user comes and goes and the last thing you want to do is hold on to a connection across requests. Instead, you have to use loose transactional models like optimistic concurrency to handle conflicts. And when you let go of locks and long-running transactions, you can start letting go of a lot of other aspects of the traditional database.

And then there is web scale. Standard client/server apps needed to scale, but not at the level of consumer-based web applications. A traditional database's transactional, single machine, disk-based model just doesn't cut it. I was at a PHP conference last year, and was amazed to see that the "standard" way of scaling a web application that talks to the database is to do your own partitioning and run multiple instances of a database. When an application has to handle all of the details of managing a self-partitioned database, we know things have gone astray.

The list above is a great summary, but I recommend you spend the time to read the 10-page paper (PDF). It is quite an interesting read. It's not every day you read quotes like There is no redo log, no concurrency control, and no distributed commit processing and H-Store ran 70,416 TPC-C transactions per second. In contrast, we could only coax 850 transactions per
second from the commercial system, in spite of several days of tuning by a professional DBA.

All of this leads me to some deeper (or more strategic) thoughts... The relational database is not going anywhere. But if those of us in the database space are not careful, we could very well be caught off guard by a new wave of database technologies that do the job for a lot of people but have nothing to do with traditional relational databases.

How does that Buffalo Springfield song go - stop, hey, what's that sound, everybody look what's goin' down...

On threading

I've been quiet for a while because I've been heads down working on some new features for NetBeans 6.1 (and now, debugging those features).

One class of bugs has been knocking me down a number of times: there is a chunk of code that is not thread-safe in a very "interesting" way. I am not historically a GUI programmer, and I have been exposed to the amazing intricacies of threading in a GUI application like NetBeans.

I won't go into details (AWT thread cough cough), but suffice it to say that I have had to put on my multi-threaded hat in deeper ways than I have had to for a while.

In the background I've also been reading about Scala, Erlang, threading and scalability. I will ask my readers to please help me, because I can't find the original references, but some comments I have been percolating in the back of my brain as I wade through my bugfixes.

First of all, someone posted that any language that required users to manage their own concurrency was a broken language. That got me thinking - hey, it is pretty crazy that programmers have to deal with this. With Java we got rid of most memory management issues (not all, think weak references), and we have a wonderful exception model, and very clear typing semantics (a reference can't suddenly become an int, for example, like it can in C). All in all it made a safer sandbox for programmers. But then here is this threading and locking complexity that has engendered books upon books, incredibly popular JavaOne talks and so on.

Then I keep hearing about Erlang and Scala, and in particular how they don't do threading, they do message passing, and there is absolutely no shared data between process units. Ah, a light came on in my head. If you follow this model, you avoid ending up in the ditch (or stuck in gridlock).

I remember working on an incredible product at Sybase called Open Server, ahead of its time in many ways, the invention of Tom Haggin, number three employee at Sybase, and someone I was honored to work with.

Open Server was a server that understood the Sybase client/server protocol called TDS, with an open API that let you plug in event handlers to process client requests and send results.

Since it was a server, it had to be multi-threaded. Its recommended mechanism for inter-thread communication was to avoid shared data and use in-memory message queues. You want to talk to another thread, you post a message on its queue. It was so simple and so effective, and so different from how things are done in most desktop applications like NetBeans. At the time I didn't think much about it, I just thought it was an interesting invention.

But my tribulations with Java and threading, my memories of the elegance of the message-passing model, and the ongoing challenge of scaling in multi-core systems, have reminded me of the power of this model, and increased my interest in languages like Scala and Erlang. Now if I can just pull myself out from under these bugs long enough to take a look at them...

Praise be, and deep thanks to Koffi Annan

I just read this opinion piece by Roger Cohen in the New York Times about what Secretary General Koffi Annan was able to accomplish in Kenya. All I can feel is great gratitude, and hope that it will stick.

Here's to the all too rare success of diplomatic intervention. The prayers of many of us appear to have been answered.