Medical records = transparency = trouble for medical industry

Via @timoreilly, an article in Technology Review gives its take on why the medical industry has been so resistant to going electronic. Surprise - it's all about the money...

Dangling $19 billion in front of a $2.4 trillion industry is not nearly enough to get it to reveal the financial secrets that electronic health records are likely to uncover--and upon which its huge profits depend. In those medical records lie the ugly truth about the business of medicine: sickness is profitable. The greater the number of treatments, procedures, and hospital stays, the larger the profit. There is little incentive for doctors and hospitals to identify or reduce wasteful spending in medicine.

Health care: you get what you pay for

Great article by Atul Gawande in the New Yorker where he does some serious snooping around as to why health care costs are skyrocketing.  His conclusion: it's how we pay the doctors.

Somewhere in the United States at this moment, a patient with chest pain, or a tumor, or a cough is seeing a doctor. And the damning question we have to ask is whether the doctor is set up to meet the needs of the patient, first and foremost, or to maximize revenue.

Iran: it wasn't supposed to happen this way

A great guest editorial in the New York Times  from a student in Iran. It starts slow, but has a wonderful finish, really gives you the sense that everybody is surprised by the sweep of events, even those participating directly in them...

Everyone watched everyone else and we wondered how all of this could be happening. Who were all of these people? Where did they come from? These were the same people we pass by unknowingly every day. We saw one another, it feels, for the first time. Now in the second week, we continue to look at one another as we walk together, in marches and in silent gatherings, toward our common goal of having our vote respected.

...

It wasn’t supposed to happen this way. Until last week, Mr. Moussavi was a nondescript, if competent, politician — as one of his campaign advisers put it to me, he was meant only to be an instrument for making Iran a tiny bit better, nothing more. Iranians knew that’s what they were getting when they cast their votes for him. Now, like us, Mr. Moussavi finds himself caught up in events that were unimaginable, each day’s march and protest more unthinkable than the one that came before.

The Guardian seems to have it right

While other news media seem to be flailing in their attempts to keep up with the speed and detail of information coming from social media, the Guardian in the UK has a live page (updated every minute) that seems to get the balance right: keep an eye on the stream of information, and use journalistic discipline to avoid rumors, innuendo and falsehoods.  As an example, Twitter is posting this picture, and The Guardian says

A picture claimed to be of today's opposition rally has been posted to Twitpic. It shows thousands of people marching on a flyover labelled the "Krimkhan bridge". The date of the picture is impossible to confirm, please let us know if you have more information.

 I think this is great, as it gives me a source of information I feel I can trust that is also up-to-date and using new media sources and not just the traditional reporting resources.

New York Times on social media as the only way to get the word out

A good article that seems to capture the power of Twitter and other social media tools to break down the walls put up by repressive regimes.

I've been reading the Twitter #iranelection stream, and a lot of it is chaotic, doubtful, over-enthusiastic, or silly.  But there are also gems in there, with links to articles and lots of pictures showing the emotion of the demonstrators and the violent repression that is going on.

It definitely gives a different more real and emotionally engaging view of what is unfolding from traditional media, with an article here or there, mostly talking about what they've heard is going on through Twitter, and only showing "sanctioned" pictures of pro-government rallies.

But to me what is most important is not what I can learn about what is going on, but what Iranians can communicate with each other even as the government tries to shut down communications.  I read in a NYT article that demonstrations are being organized through Twitter.  That was great news to hear.  And I pray for them, because the risk of serious injury or death is very real, and they are still moving forward.  Incredibly bravery and determination.

Thinking about getting a gTLD - talk to Antony

Here's a nice interview with my brother, Antony Van Couvering, CEO of Minds+Machines, talking about gTLDs (generic Top-Level Domains), why you might want one, why they're useful, and giving you a good overview of what is involved.

For a brand owner, having your brand name as the destination for your customers on the web is ideal. How much money do major corporations spend on branding, and how effective is Internet visibility? To me, it’s a no-brainer to brand yourself instead of a domain extension like .COM which has nothing to do with your brand. For a city, it’s a tremendous aid to tourism and local business, with great possibilities for civic initiatives. Another key advantage, if you keep your TLD clean (in other words, take steps to discourage warehousing, non-working sites, and template sites), is that search engines will pay attention. For instance, .NYC will be about New York City in a meaningful way, and if Google wants to deliver relevant searches (which it does), then it will favor the .NYC TLD when someone types in “nyc” or “new york city” or “manhattan” or any other New York-related term.

Do we *really* want SQL in HTML5?

Jan Lehnardt's tweet pointed me to this ongoing conversation about the new SQL support in HTML5. Vladimir Vukicevic I think summarizes the issue quite well:

There's been a lot of interest around the Web Storage spec (formerly part of whatwg HTML5), which exposes a SQL database to web applications to use for data storage, both for online and offline use. It presents a simple API designed for executing SQL statements and reading result rows. But there's an interesting problem with this; unlike the rest of the HtML5, this section defines a core piece of functionality in terms of an undefined chunk referenced as "SQL".

The initial implementations of Web Storage are both based on SQLite, and expose the dialect of SQL understood by SQLite to web content. I'm actually a big fan of SQLite, and was one of the advocates for pulling it into the Gecko platform. However, SQLite implements a variant of SQL, with a number of deviations from other SQL engines, especially in terms of the types of data that can be placed in columns.

Web content that is created to use database storage with SQLite as the backing is unlikely to work with any other backend database. Similarly, if another database was chosen as a browser's backing implementation, web content that works with it is unlikely to work with anything else. This is a serious interop problem, the root of which is that there really isn't a useful core SQL standard. SQL92 is generally taken as a base, but is often extended or altered by implementations. Even beyond the parser issues (which could be resolved by defining a strict syntax to be used by Web Storage), the underlying implementation details will affect results.

So, the only option is for the Web Storage portion of the spec to state "do what SQLite does". This isn't specified in sufficient detail anywhere to be able to reimplement it from the documents, so it would be even worse — "do what this exact version of SQLite does", because there are no guarantees that SQLite won't make any incompatible changes. For example, a future SQLite 4 may introduce some changes or some new syntax which wouldn't be supported by earlier versions. Thus, it requires every single browser developer to accept SQLite as part of their
platform.

There then follows a lot of discussion about how to handle this, from creating an ORM layer to fully specifying the standard subset of SQL that will be supported in HTML5 (as if there were a valid standard). One person argues that we can simply refer to the ISO SQL standard and then describe extensions to that standard.

OK, well, I have worked with relational databases for over 20 years, and I have seen how completely stuck customers are with a given database vendor. I don't care how "standard" the standard is, it's merely an illusion. There is a reason Oracle can charge such huge sums of money and smile as customers fume. There is a reason Sybase is still in business. It is almost impossible to move a large database application from one vendor to another.

We all know how difficult it is to write code today that is compatible across all browsers. Imagine if each browser had its own SQL database that "implemented the standard." That's where we're heading, and it sure seems hellish to me. I guess if we forced every browser to use SQLite, we'd be OK, but that seems to placing an awful lot of reliance on a single technology. And as Vladimir points out, that doesn't solve issues of incompatibility across versions of SQLite.

It seems to me the requirements are to have rich storage, particularly for Javascript objects, and a way to query over this storage. I know saying is easier than doing, but shouldn't we be defining an API that defines JSON storage and a JSON query language? The Persevere folks have already defined a JSON query language, and they seem to have put a lot of thought into it. Perhaps that could be used as a start.

I can't say I have a solution, but I'm just raising a flag that creating a standard as important as HTML5 with a placeholder for "SQL" is an invitation for major incompatibilities which we'll be digging ourselves out of for years to come.

Michael sings the train blues

Michael got a little geetar for Christmas, and I was sitting here working yesterday when I stopped and listened to him singing along. I couldn't believe it - serious train blues. I got up and went into the living room and there he was sitting in the middle of his train tracks singing along, "Red Train Going Down the Tracks!".

Listen to him singing about the piston at the end. What to say, infinite cuteness.

Middle age: learning acceptance and contentment

To me a lot of what middle age is about - or perhaps life in general - is accepting your life as it is, and finding a place of contentment, an angle of repose.

We often have ideas about how our life should be going, but that's not the way it's going, and we fight and fight and fight. But at some point we are taken down, usually by some bodily failure (which is where middle age comes in) and we are forced to let go and accept. I suspect that this is the seed of wisdom.

This is also the message I see regularly in Indian scriptures. Krishna says "it is better to live your own dharma poorly than someone else's well". The great statements of Vedanta say "All this is God" and "Nothing but God exists." To me, one aspect of this teaching is that if I am seeing my life as "not good enough" or not where I want to be, then this implies I am not seeing God in my life.

Sometimes this is very hard to accept.

Over the last three weeks I went through a debilitating flu and had to go through a root canal (partially caused by grinding my teeth). I took these as signs of a life out of balance and did some self-inquiry.

First of all, I was exhausted, and secondly I was stressed. This much was obvious. Next question was why.

I have just started a new job after being laid off by Sun. I didn't really expect to make this change, and I don't think I had fully accepted the change. At Sun I was working with some serious rock stars of the industry, and doing some "cool stuff" in open source. In my new job, it's a great team and fun work, but I have been unhappy because (to me) it just wasn't that "cool."

I have also been watching all this interesting stuff going on in the web tier - distributed data stores like CouchDB, rich client development using Javascript/HTML5, Flex and JavaFX, potentials for peer-to-peer collaboration, etc. But none of this has anything to do with my day job.

So I started trying to do this stuff on the side. Keep in mind I'm a father of 2 young kids, and I also just took away two hours from my day for commute time. So I was squeeezing this into late nights and early mornings. Thus the exhaustion. And the grinding teeth: I was pushing and pushing myself, I didn't want to let it go and just do my day job - that would be admitting surrender.

But after these illnesses and further thought and I saw that (a) this wasn't a maintainable lifestyle and (b) I was being proud and egotistical. My day job was "below me" and I was meant for "greater things." Hints of Raskolnikov. I wasn't accepting my life as it is.

So with a sigh I saw it was time to let go. No more late nights trying to hack Javascript and CouchDB. Although part of me feels like I'm letting go the chance to ride this wave, I also have to admit I've been much happier and less stressed since this decision.

It's such a fascinating process. This wasn't the first time I've gone through it, and I doubt it's the last.

Great article on CSS development

I have done very very little CSS development, but I'm starting to dabble a bit.  @petr3 posted a link on Twitter to an excellent article by Jonathan Christopher that both lays out some nice principles on CSS design and provides some valuable links to earlier articles he wrote how to write better CSS and improve your CSS processes.

The principles he mentions reminds me a lot of software design in general:

• find the patterns and reuse wherever possible
• work from generic to specific
• work to keep clarity as you maintain it

Definitely worth a read...

What a concept: a steady state economy

From the first time I heard of this idea that growth was essential to prosperity, it's had me shaking my head. I first heard it in terms of an individual company: a public company is always under pressure to grow, grow, grow. You can't just have one coffee shop, you have to have two. You can't just be a single company with a single focus, you have to conglomerate. Grow grow grow! It just seemed wrong for a company to be forced to grow when it could be doing perfectly well where it is. See the story of the Southern fisherman and the Northern industrialist.

Then I heard about this concept in terms of the economy in general.  I read how a steady state was considered the same as stagnation - like some horrible third circle of hell.  But you just do the math in your head, and you wonder where all this growth is going to take you.  Sure enough, it's not sustainable, and bubbles burst.  And I just sigh when I hear all these economists (who got us into this mess) trying to get us back to a trajectory of growth.  Can't we try something else for a change?

So, I thought I'd share with you this very insightful and entertaining article by Herman Daly, posted on the Oil Drum.  Herman Daly has fun skewering the growth economists (who I'm sure are ostracizing him as we speak) and then goes on to describe a vision of a healthy steady state economy.  You know, the old general store instead of Wal-Mart.

Here's one juicy skewer quote:

Some economists in fact think of nature as the set of extractive subsectors of the economy (forests, fisheries, mines, wells, pastures, and even agriculture….). The economy, not the ecosystem or biosphere, is seen as the whole; nature is a collection of parts. If the economy is the whole then it is not a part of any larger thing or system that might restrain its expansion. If some extractive natural subsector gets scarce we will just substitute other sectors for it and growth of the whole economy will continue, not into any restraining biospheric envelope, but into sidereal space presumably full of resource-bearing asteroids and friendly highly-evolved aliens eager to teach us how to grow forever into their territory. Sources and sinks are considered infinite.

Then Mr. Daly goes on to offer 10 proposals to help reach a healthy steady state economy.  I'm not sure I grok or even agree with all of them, but it's a starting point for a healthy dialog about this.  I recommend having a read...

JSR308 (pluggable type checking) is looking promising

Watching the Twitter stream for JavaOne, I saw a number of folks say they were quite impressed with the talk called "Preventing Bugs with Pluggable Type Checking."  I checked out the link for JSR308, and this looks very promising.

Basically, you can add annotations that check for things such as NonNull or Nullable, and then incorporate a checker that supports these annotations into the compiler.

For example, you can say

import checkers.nullness.quals.*;

public class GetStarted {
    void sample() {
        @NonNull Object ref = new Object();
    }
}

and then say

javac -processor checkers.nullness.NullnessChecker GetStarted.java

and the checker will ensure that the @NonNull object indeed is not being set to null anywhere.
Take a look at the page for JSR308, this looks like a good new addition.

Learn CouchDB through online experimentation - highly recommended

Wow, this is very nice.  It's an all JavaScript emulation of CouchDB.  What makes this super-cool is that this allows you to interactively play around with both CouchDB documents and views.  This is particularly useful when you're trying to understand how views work.  Definitely recommended if you're interested in learning CouchDB.

Excellent article describing how to work with CouchDB and jQuery in Firebug

A really nice step-by-step outline by Ben Atkin of how to use the Firebug interpreter to interactively work with CouchDB using jQuery's ajax() method.

Thanks to @nitin for this link, and thanks to Ben for taking the time to write it up! These simple tip blogs are one of the most valuable aspects of the social web for me.

Twitter for communication in a disaster

I just sent this letter to the City of Berkeley disaster preparedness team.  You might want to consider doing the same with your local municipality...

Hello.  I am a homeowner here in Berkeley, and have been using Twitter for the past month or so, and I was impressed at how easily it is for me to stay up-to-date with things happening real-time.  For example, there is a conference going on in SFO, and I just subscribe to the search term for that conference and I see real-time updates from everyone who is at the conference and using that code.

Another example of course is that during the Mumbai terrorist attacks, everyone used the term "#mumbai" in their messages, and anyone subscribing to that term could see upates in real time.

One of the worst problems in a disaster is lack of information and nobody knowing what's going on or what they should be doing.  If a disaster occurs in Berkeley, it would be very nice to get real-time updates through Twitter.

In particular, I can see an "official" channel coming from your office and the City of Berkeley in general, with important information that you want to disseminate.

I can also see an "unofficial" channel where citizens can post messages, things like "fire on 4th and Hearst" or "looting going on near downtown BART, stay away" or "if anyone needs extra firewood I have some".

When I say a channel what I really mean is that when you post a message you use a certain tag like "#berk" or "#coberk" or something like that.   Then those of us trying to keep in touch can subscribe to this term and get real-time updates, and post our own updates as well.

If you set this up, and then let people know ahead of time to make use of these channels, I think this could be a very good thing.

You might want to also set up an agreement with local cellular providers that in the case of emergency everybody should have access to their data services even if they're not paying subscribers, so they can engage in communication through the Internet using their cell phones.

Thank you,

David

Google needs HTML5 and JavaScript

I've been following the Twitter stream for Google I/O, it's been fun to watch (cool that everyone got a phone).

But why are they giving developers phones? Well, being at Sun, I know the strategy: you capture developers, and many good things flow from this.

The other effort to capture developers is all the stuff around HTML5 and making JavaScript faster and better. Take a look at Tim O'Reilly's great post that summarizes the many cool HTML5 features that are already available in most browsers, things like a native JavaScript canvas and easy embedding of videos.

Why is all of this so important to Google? Because if JavaScript and HTML doesn't get much better quickly, then developers will move to the RIA model (ala Flex or Silverlight or Java) to get the beauty and dynamics they need for web-based applications.

And this would be a Big Problem for Google, it seems to me. Why? Because you can't crawl RIAs. You can't add AdSense RIAs. And ads are the cash cow for Google.

It looks like they're doing a great job though - the developers at the Google IO conference are very exicited. My suspicion: the web browser is going to win over RIAs like Flex and Silverlight, because it (a) developers are already used to JavaScript and the browser and (b) it works today on mobile web platforms like the Android and the iPhone and Palm's WebOS.

And by the way, if you're building a rich UI in JavaScript (or in any rich internet app environment), then the role of the server becomes very different - it stops being responsible for handling UI events and generating UI, and instead becomes a simple provider of web services. Just like the good ol' days. I think there are a lot of interesting consequences from that architectural change...

SchemaBank - web-based visual data modeling and change management

Well, this looks pretty darn interesting. I don't have a lot of time to play with this, but SchemaBank, if it delivers on its promise, looks like a very useful ERD tool at a pretty reasonable price point.

You can export a SQL dump from MySQL or PostgreSQL, import this into SchemaBank, and start going. Or you can start with a fresh schema. You can do visual data modeling, versioning, branching, diffing - in other words, both modeling and change control for your schema.

The UI looks sweet, although I'd be curious how it handles super-large schemas, always the bane of ER tools. It does have "Bird's View" which is a good sign they're thinking in the large.

Using CouchDB map/reduce to generate a bloom filter

This is very interesting. Devlin Daley is in the last phases of building a bloom filter for CouchDB. Taking advantage of various reusable components available out there in the Vast Internet, he uses map to set up a bloom filter for a given element of a given document, and the reduce to combine all of these into a single bloom filter. Et voila!

Once this is done, definitely something to keep in the toolbox. Thanks, Devlin!

Gone over to the Twitter side

Remember when I gave up on Twitter? Ah well, I found a tool that actually works for me, TweetDeck, and now I'm actually finding a way for it to work. I particularly like that I can send and view Facebook status updates from the same tool, that really helps keep things together.

The problem is now when I have little comments to make or links to post, I post them on Twitter instead of here. I wish I had an easy way to post the same link to my blog, but I haven't figure that out yet.

I've added the "Follow me on Twitter" button to my blog page, or you can just go here and click on the "Follow" button.

If you're not on Twitter and are wondering what all the fuss is about, the best thing to do is to try it, it's very hard to grok otherwise. And I really do recommend TweetDeck, it helps makes it more manageable and easy to use, and thus you use it more and stay engaged.

I'm sure there are other apps that are similarly good, but without one of these the whole thing just doesn't work.

David

Java is like classical, Javascript is like jazz

I've been looking at Javascript in more detail as part of a side project I'm working on.  I've been reading Javascript code and feeling my head hurt, and it took a while for me to figure out what was causing the dissonance.

I've been a Java programmer for years.  I have always enjoyed its clarity and structure.  The static typing lets you say what you mean and mean what you say. 

Reading Javascript code isn't like that.  Nobody tells you what somethings going to be used for, and it can change over time.  Objects can be of any type and can have any number of elements.  Functions are created and destroyed like wafts of smoke.

I finally realized the Javascript is like impressionistic and avant-garde art, like jazz.  It flows, it molds, it changes, like a dream.  Java is like renaissance art, like classical music.  Its beauty is in its form and structure, but it constrains your ability to express and invent - you have to follow the forms given to you.

Of course this isn't a perfect match, but it helped me remove my Classical hat and put on my Jazz hat when dealing with Javascript.  And it also helped me understand why Java is so popular in the enterprise, while Javascript (and other dynamic languages) are so popular for the creative types building consumer-facing applications.

Sometimes a little metaphor like that can go a long way.  Now all I need to do is start wearing black and pierce my lip...

Larry Ellison wants to keep hardware, SPARC, and storage

From @timbray, a very enlightening interview with Larry Ellison about Sun's hardware business.  Sez he wants to keep SPARC, storage, and the hardware business in general.  I wish him luck - Sun sure wasn't getting anywhere with the "integrated systems" pitch.

Very nice code review tool

I just wanted to quickly mention Review Board - a free code review tool that we're using here at Symantec. It is very easy to create a review based on a diff, pick reviewers, publish it, and manage comments - you can add comments to a specific line and have per-comment discussions.

All review comments are published via email so it can be tied into an open source community where all discussions/reviews are supposed to happen over the email list.

So much better than email reviews, where you have to somehow manage all the comments in a single email, which can be quite daunting.

Inhumane pig conditions are incubator for swine flu

I think each of us knows when something is "not right", which I guess is what you might call "immoral" but that seems so judgemental. But regardless, what I've seen is that often our intuition is telling us something that our rational mind can, well, rationalize.

What I've noticed is that often, when we instinctively know something is not right, often we find out it's true when the unforeseen consequences arise. The latest example of this theme comes from an excellent article in Wired magazine (thanks @larrybrilliant), where they describe in detail how the swine flu now spreading through the world had its origins in insanely packed pig farms:

At an environmental level, the conditions which shaped H3N2 and H1N2 evolution, and increased the variants’ chances of taking a human-contagious form, are well understood. High-density animal production facilities came to dominate the U.S. pork industry during the late 20th century, and have been adopted around the world. Inside them, pigs are packed so tightly that they cannot turn, and literally stand in their own waste.

Diseases travel rapidly through such immunologically stressed populations, and travel with the animals as they are shuttled throughout the United States between birth and slaughter. That provides ample opportunity for strains to mingle and recombine. An ever-escalating array of industry-developed vaccines confer short-term protection, but at the expense of provoking flu to evolve in unpredictable ways.

The horrible packing of pigs for slaughter, standing in their own excrement, combined with an "ever-escalating array of ... vaccines." Sounds horrible, it seems wrong, and as we see, there are consequences.

The world lives in such a beautiful balance, each thing complementing the other in this intricate array of interdependencies. Then we take our logical, rational mind and deconstruct some piece of it and think we can get away with changing it for some increase in efficiency or productivity, without any "unforeseen" consequences.

I really am not interested in placing judgmental or moral overtones on this. This is not about God punishing sinners or some such rot. To me it's more about God's beautiful intricate creation, and how the depth and richness and complexity of it can never be fathomed by the human mind, and if anything I find myself shaking my head at our hubris in thinking we can push things to their limits like this and not expect consequences.

This is why whenever I hear some industry talking head saying how something is "perfectly safe" and "all studies have shown there are no harmful side effects" for some strange mutilation of the natural order (such as fake sugar and fake fat, or massive vaccinations of children, or the heavy use of pesticides and chemical fertilizers) I just go "yeah, right, uh-huh" and wait for the next shoe to drop, which it invariably does ten, twenty, or fifty years later.

Ruminations on what at Sun will stay and what will go

This week I ran into an old friend from Sun who now works at Oracle. I saw him in the lobby of the office building where I work at in San Francisco -- it turns out he's in the same office building! BEA had offices there, and now they're part of Oracle.

We went across the street to Peet's and remembered the good ol' days, and then started talking about the software products we thought were dead meat.

This is all conjecture and hearsay, and I recognize that every project killed means people unemployed, which is no fun as I can personally attest. But that said, one does wonder what's going to get the axe. Here are some possibilities

Glassfish
This is a hard one to swallow, but Oracle already has an app server (Weblogic). So there's a distinct possibility that Glassfish will have to go. Of course it's open source so it could still survive, but I can imagine Oracle pulling the app server folks off of Glassfish and onto Weblogic.

NetBeans
I love NetBeans. I had fun working on it. But let's get real. Oracle is already an Eclipse shop, as well as having their own IDE (JDeveloper). I just can't see why they'd want to keep NetBeans around

JavaFX
It's my opinion that Oracle has no need for this, and will be happy to pull resources off of it and back onto standard Java. However, mobile is taking off, and Java has a place there, so who knows. I am on the fence on this one.

JavaDB
I could see how Oracle would see this as quite valuable and how it could fill a good niche. But they could just as easily take the Most Excellent Java DB engineering team and redeploy them elsewhere, and leave Apache Derby to IBM.

MySQL
Well, everyone's conjecturing about this one. The nice thing for Oracle is now they have the future of MySQL in their hands and can direct it in a way that makes sense for their business. The only wild card I see is the forks that are out there, but I must admit I don't fully understand all the business and political aspects of this to know how viable these forks are if MySQL "proper" is put into mothballs.

There are lots of other ones to talk about - SPARC, cloud computing, and lots of little projects you probably haven't even heard of. But I don't have enough visibility into these to know what the right choice is or could be. So it would really be sticking a finger into the wind.

However, I know we'll be finding out soon enough. I don't envy those folks who get the ax because of the reorganizations Oracle will be doing. May each and every one of you who "get the package" quickly and easefully move on to your new phase in life, whether it's another job or time off or starting your own company. Good luck and God bless!

Into the West - sad to see the Sun set

A number of people have been asking me what I think, as an ex-Sun-employee, of the acquisition of Sun by Oracle.

I am mostly sad.  I really liked Sun as a company.  In the beginning of my career, I was a heavy user of Sun technology, from Sun workstations and servers to, when it came out, Java.  And don't forget NFS.  And Open Office.  And and...  Then I was honored to become an employee at Sun, and regardless of how we fared, I always felt that the company's heart was in the right place, and they did things with passion and intelligence.

Oracle, at least from the outside, just isn't like that.  The culture, as I experienced as a competitor at Sybase and from talking to those who have been there, is aggressive and competetive.  I'm sure that's served them very well, but that's just not my style.  When I was in junior high school in Colorado, soccer was played for the most part by Vietnamese and Koreans.  I played soccer both because I loved it and because it was (at that time) just a bunch of friends getting together to have fun. Baseball and football were ruthless, competetive, and not much fun.

I quit playing soccer in high school when someone from the opposing team came at my head with his feet (I was the goalie) and with a malicious grin on his face.  Thanks, but no thanks.  It just wasn't worth it to me.

I see Oracle as a place where baseball and football are played.  Yes, you are successful, but it's not my idea of fun. 

Perhaps companies like Sun can't really succeed in the business world.  Perhaps you have to be ruthless and aggressive and competetive, both inside and outside your organization.  Perhaps a culture like Sun's only does well in the small - small businesses, small goals, maybe even staying private instead of going public.

But regardless, I'll be sad to see Sun and it's Most Excellent culture and products be subsumed into Oracle.  It's the end of an age - the elves are all leaving Middle Earth... :)  So long, Sun...

Reputation-based security

Today I read a transcript of Enrique Salem's keynote at the RSA conference, and in it he talked about something I found to be very fascinating - reputation-based security. The idea is that you can set security policies for applications you allow to run not based on some impossible-to-maintain whitelist or blacklist, but based on an application's reputation.

This new method isn’t all-or-nothing blocking like we’ve had in the past. This is a policy-based approach, where the administrator can configure the protection based on their own tolerance for risk. For instance, a government agency could forbid installation of all software that is less than 30 days old, hasn’t been installed by at least one million users, and doesn’t have a good reputation. This policy would guarantee that all software installed would first have to be vetted by literally millions of other users.

On the opposite end of the spectrum, an administrator at a university—where students constantly download all sorts of applications—could have a more lenient policy. For example one that requires new software to have a good reputation and have been previously downloaded by at least 100 users.

Think about it a little like a Zagat restaurant guide. Some people with a high risk tolerance may go eat at the new sushi place based on the recommendation of an acquaintance. Someone that is more risk averse might want to first check out the Zagat guide and wait until the place receives a high food rating before they go try it. But what’s most important is that you choose how daring you want to be when it comes to picking a restaurant. You should be able to make a similar choice when it comes to security.

I think this is really cool and makes a lot of sense. I like the way it balances flexibility with security. I like the way it takes advantage of the crowd to help categorize the safety of an application. It also helps me see how there are very interesting problems and potentially fascinating solutions in the world of security.

Security as a service

Now that I'm part of a security company, I've been contemplating this security thing.  As a child of the seventies I notice I have a knee-jerk reaction to see security as a force of totalitarianism and control.  I find myself wanting to 'stick it to the Man.'

But here I am at Symantec, a company that is all about security.  I can either be grimly resistant to the whole focus of the company, or I can examine my attitude and try to find an approach that works for me.

Yes, absolutlely, the name "security" can be lipstick onto the pig of extreme, sometimes brutal control and inflexibility.  In the name of security you can make life miserable for people.  But I have to admit, you can't do without security.  It's a part of life.  In Berkeley-speak we call it "healthy boundaries."  If you have healthy boundaries, then you are actually more able to relax and be yourself.  If I think of it that way, my liberal genomes can relax a bit - "oh, healthy boundaries, that's OK."

I've also been remembering my volunteer offering of security at my place of worship in Oakland many years ago.  I remember the stance we tried to have in that role.  We were vigilant, we kept our eyes open, and were prepared to lovingly but firmly escort someone out of the building if they were dangerous or inappropriate.   But most of the time we were Just There.  We didn't have big badges or scowls on our faces.  We just stood there on the corner or at the door welcoming people and watching the world go by.

To me that's Good security - there when you need it, but quietly present when you don't.  It creates a sense of safety and comfort, not one of being hard-armed and constrained.

So, it's a delicate balance you have to play - you don't want to be too loose, and you don't want to be overly firm and harsh.  You need to catch viruses, but you don't want systems to run 10 times slower or exercise absolute control over exactly what applications people can run.  You need to balance risk against cost.  It's really quite a dancing act. 

But most of all, I believe you need to have the right attitude.  If you approach security with a feeling of fear and panic, you tend make decisions that are likely to be overly dictatorial (see the Bush years as an example).  But if you have an attitude of service, respect, and flexibility, then you are much more likely find solutions that work for everyone.

So armed with that attitude - an attitude of service and respect - I'm ready to go have some fun at Symantec.  Security as a Service.

Dare Obasanjo - AtomPub fades away

Wow, great discussion of the reported demise of AtomPub. I was playing around with AtomPub a while back, and it's true, I've slowly moved over to just using JSON - see CouchDB for example.

The double whammy comes from the fact that although new forms of microcontent have shown up which do encourage the existence of desktop tools ... the services which provide these content types have shunned AtomPub and embraced JSON as the way to expose APIs for rich clients to interact with their content. The primary reason for this is that JSON works well as a protocol for both browser based client apps and desktop apps since it is more compatible with object oriented programming models and the browser security model versus an XML-based document-centric data format.

In my opinion, the growth in popularity of object-centric JSON over document-centric XML as the way to expose APIs on the Web has been the real stake in the heart for the Atom Publishing Protocol.

http://www.25hoursaday.com/weblog/2009/04/18/JoeGregorioOnWhyTheAtomPublishingProtocolAtomPubIsAFailure.aspx

More info about in-memory support in Java DB

Knut Anders gives lots of useful details about the new in-memory support in Java DB 10.5. Very simple to use: just specify "memory" in the URL. I love the tip about backing up your in-memory database prior to exiting the application - it's like doing one big checkpoint at the end. So you get both durability (OK, very large-grained durability :)) and speed.

Javascript implementation of CouchDB API in the browser

Some very interesting stuff is happening in the browser space. Damien Katz gives a link to this post by Atul Varma about creating an initial implementation of the CouchDB API in JavaScript.

Damien takes this and runs with it, saying that CouchDB is specifically intended to run in the client, and to provide a form of location transparency where your entire application can be running locally or remotely and you really don't know the difference except that perhaps some of your data is stale until you reconnect to the server.

This is the problem that CouchDB is designed to solve. Not just putting your data and apps into the cloud, but onto your laptop, your phone and your local office server. Fully query-able and editable in your browser, your data is available wherever you are, despite network outages, air travel or blocked access.

I like how Damien implies that the cloud doesn't include just servers, but also your own machine - that your client machine is part of the cloud. It's a very different way of thinking about web architectures. I highly recommend you keep an eye on this space.

Cool elevators

Here I am on the twentieth floor overlooking the San Francisco bay and watching the ferry come into the Ferry Building. Pretty posh - I'm trying to enjoy it while I can until we move next week.

But what I wanted to write about was the super cool elevators they have in this building. In most office buildings, during "rush hour" in morning, lunch and evening you wait and wait for an elevator, and then on the way up/down you hit every single floor.

But these are "smart" elevators. Instead of pressing "Up" or "Down" buttons, what you do instead is press the floor you want to go to. The EPU (Elevator Processing Unit) then does optimal calculations based on who wants to go where, assigns elevators to routes, and then a number flashes to tell you which elevator to take. So for instance this morning we had stops on the 18th, 19th and 20th floors, and that was it. At lunch time I stopped at one other floor before heading to the lobby.

That is just so cool, and now that I think of it, so obvious. I do wish all elevators worked this way...

Derby 10.5 - in-memory support!

I had lunch today with a bunch of old buddies from Sun, including Rick Hillegas and Francois Orsini from the Java DB/Derby team. They confirmed rumors I had been hearing that 10.5 (currently in release candidate mode) has support for running purely in-memory, and that some operations are running 1000 times faster than on disk.

Why is this interesting? Well, think of the scenario where you need to cache data with your web server instances so that your client requests can access the data they need locally, in-memory. This is a very common need to meet the scalability and performance requirements of web applications.

The problem is, most of the caches out there today are simple key/value hash maps.

What JavaDB in memory gives you is an in-memory cache that gives you all the expressive power of SQL. You can run SQL queries over your cache, or use Hibernate or JPA, rather than being limited to key-based lookup and building your own secondary indexes.

And then there are interesting features such as Java stored procedures and table functions, which I can envision being very powerful and useful in an implementation of an in-memory SQL-based cache.

So take a look at the 10.5 release candidate (or wait until 10.5 releases). It may be just what you need.

Back at work

My first day here at work at Symantec.  It's a very Windows-oriented environment, which is something I have to get used to.  I've already downloaded cygwin to help me have some UNIX comfort.  Next step is to install Perforce, get a build, and try to build this puppy.

The view here is spectacular - a view of the bay, Treasure Island, and if I just squint, I can see my house in Berkeley.  However, alas, this is not to last -- in one month (to the day) we'll be heading to the sixth floor of an office building south of Market, and no more view :(

My dev machine - well, it's hot.  8GB of memory, over 2TB of disk space, and a 4-core CPU.  Plus I get a company-issued laptop to take home with me - but not a Mac (wah!).   It's a Dell running Win XP.  So nineties :)

Hopefully I'll be up and running soon, but I may not be able to talk much about what I'm doing.  This is not open source in any shape or form.  We'll see what we shall see...

I got a new coffee mug and my boss brought cookies and invited everyone over to meet me - it's a very friendly, good-natured group.  I'm glad to be here.

Stratocaster · iPhone Upheaval

Rich Sands has a very nice analysis of the mobile market and the impact of the iPhone - particularly on the carriers.

Consumers love it. Developers, and content creators love it. Apple surely loves it. But carriers are rightly spooked, because this new model cuts them out of the content business and accelerates their inevitable slide into the abyss of commoditized, dumb data pipes, where price and low cost are the only things that matter, and margins get razor thin. But the carriers should have seen this coming. No matter how much control carriers exert, if what they deliver is more about their profits than satisfying end-users, they’ll eventually be attacked by someone who understands that consumers have the ultimate power, and that developers and other content creators are the source of most of the value to those consumers. The iPhone was inevitable.

http://rich-sands.com/wordpress/?p=51

Getting ready to go back to work...

I have been "not working" for 2 1/2 months now. It was very hard getting used to when it started. And now I'm finding myself preparing emotionally for going back to work, and relishing the time I have left.

I have been working on an under-the-radar project with a friend of mine, and that's been a lot of fun. If it ever turns into something, I'll let you know. Meanwhile, it's time to get back to paying the bills. I'm looking forward to working with the Symantec team - I really enjoyed interviewing with them and they seem like a very good group of people. But what can you say - no rushing in the morning, hanging out at wireless cafes, no commuting - these things just don't come around every day. Sigh...

Maybe I should write about the emotional stages of going back to work, to match my blog on the emotional stages of being laid off...

Old Jews telling jokes

My friend turned me on to this last night - OMG this is so hilarious.  You just have to watch these.  See http://www.oldjewstellingjokes.com/.  Be warned - these jokes are not PG rated, but that doesn't meant they're not funny.

Synchronized iPhone todo list on the cheap - Address Book

I've been trying to find a good way to have a simple todo list that is editable on the web, on my Mac, and on my iPhone.

I've been looking at the apps coming in to the App Store, but none of them seemed really worth the effort, especially since they don't have anything that works on the web or on my Mac.

I tried Evernote, but the dumb thing doesn't let you edit entries created on the Mac on the iPhone when you're offline (which I am a lot, because I don't have a data plan, just wifi and T-Mobile pay-as-you-go).  It doesn't matter if you "favorite" them, it doesn't work because the Mac-created entries are created with rich text, which can't be overriden (e.g. you can't use simple ASCII text).  There was even a todo list on the market that costs 50 friggin' dollars plus $10 for their iPhone app.  TODO: fuggetaboutit.

But wait: Address Book contacts have a notes field.  And I just found out you can synchronize Address Book with Google Contacts, and that Google Contacts has its own URL so you don't have to go through Gmail.

Problem solved.  I create a contact called Todo, and in the notes field I put all my action items.  I can even categorize if I want multiple todos (Todo-Family, etc.) but I've noticed that becomes too complex and is a good reason for me to slowly stop using it, and that defeats the purpose.

Now I can edit that note in Address Book on my Mac, on my iPhone, and even on the web if need be.  That works for me.

Obama’s Ersatz Capitalism - NYTimes

A great piece that lays it on the line. Read this, it explains the new bank "partnership" in clear and gory detail.

Paying fair market values for the assets will not work. Only by overpaying for the assets will the banks be adequately recapitalized. But overpaying for the assets simply shifts the losses to the government. In other words, the Geithner plan works only if and when the taxpayer loses big time.

What the Obama administration is doing is far worse than nationalization: it is ersatz capitalism, the privatizing of gains and the socializing of losses. It is a “partnership” in which one partner robs the other. And such partnerships — with the private sector in control — have perverse incentives, worse even than the ones that got us into the mess.

http://www.nytimes.com/2009/04/01/opinion/01stiglitz.html?_r=1

Three wishes

I've been reading a series of great books about djinns with my daughter, and one thing I really like is that the djinn are quite uncomfortable granting three wishes to humans because most of the time they don't work out very well.

So I started thinking, if I had three wishes, what would I wish for? 

But when I started going through possible wishes, I realized that all these wishes seemed off - I found myself feeling kind of "grabby" and small.  Why was this? 

After further thought, what I realized was that the very act of having a wish for something implies that I am dissatisfied and incomplete - it perpetuates a feeling of lack.  So no matter what you wish for, even if you get it, you'll still be dissatisfied and unhappy.  This is one way of understanding Lord Buddha's statement that the root of all suffering is desire.

So, was there anything I could wish for that wouldn't enforce this feeling of lack, that wouldn't create another cycle of dissatisfaction?

There was only one wish, or prayer, that worked for me: the prayer to follow God's will, or if you like to "go with the flow." That wish doesn't carry a feeling of lack. It feels full and complete.

Then I remembered this statement someone told me years ago, by Rabindranath Tagore, a great poet of India:

I slept and dreamt that life was joy. I awoke and saw that life was service. I acted and behold, service was joy.

Job found

Good news - I have found a place to land.  It has been an interesting experience talking to all these different companies and seeing so many different styles and cultures.  There are themes and commonalities, but there are also big differences.

I realized towards the end of this that I would find a job that was probably a near perfect fit, because everybody right now can be so picky.

So, where did I end up?  Well, not necessarily where I expected.  I don't have a big security background and it hasn't been one of my focuses, but I ended up at Symantec , working on their Data Loss Prevention (or DLP) tools.  Think employees losing laptops with medical records, like that.

I'll be one of their senior guys working with what they call "Data At Rest" - data in various storage repositories, be it databases, file shares, wikis, Exchange servers, you name it.  Their tool scans all of these repositories and run them through a detection engine and then raise alerts or run other workflow if something is found.

The group that builds these DLP tools is actually a recent acquisition for Symantec called Vontu .  They appear to be top in this market by quite a large margin.  And this market is continuing to grow, even in these times.  Security has always been an issue, but it's becoming more and more of one as a larger portion of our life starts to go online.

Aside from the technology, what sold me on this team was the way they interviewed.  Yes, they grilled me.  I had to design a data model for a card game and talk about how to implement flow control with it, answer questions file scanning performance, answer architecture questions, etc.  I even had to give a technical presentation followed by Q&A to the entire team as part of my second round of interviews.

But they were respectful, and friendly.  And they had just as many pointed questions about my work style, how I dealt with conflict, how I liked leading people, my opinions and ideas about development process, and so on.  They obviously cared, a lot, about the quality of a person beyond their technical skills.  This is something that has been important to me too, and has been sorely lacking in many of the "hot" Silicon Valley companies I've been talking to.  Some of these guys are so focused on technical prowess that they completely miss the boat in terms of having a respectful, friendly culture.  It's all grim, O log(N) kind of stuff.

The DLP team is based in San Francisco, and I'll be going in most days.  This is huge.  I was convinced I'd have to commute down to the South Bay (think two hours each way on public transit) every day.  That's just where most of the folks are.

This is also huge because this will be the first time in 9 years that I'll be going into an office every day, and meeting my coworkers face to face every day.  Ever since my daughter was born in 2000 I have been working from home in some capacity or another.  And for the past seven years the teams I have worked with have been in Europe or India.  I was looking at my wardrobe and realizing my clothes are all a bit ratty - I just haven't paid much attention to the presentability of my clothes!  Time to help the economy and do a little shopping...

But I am actually looking forward to going in the office every day.  I have loved working from home, and being there when my kids were at home, having some flexibility to help make Linda's life easier, and just being there to see and understand what Linda's life was like, if only a little.  But Michael's three now, getting ready for preschool, and I'm getting ready to start seeing the people I work with again, hanging out with them, and cracking dumb jokes with them.

This will also be the first time in my entire career where I commute daily beyond the East Bay.  That's a whole routine and way of life I will have to get used to - spending most of my time in a town far from home, and being one of the huddled masses on the BART train every day.  But I like trains, just like my son.  I think I'll be fine.  At least I don't have to drive every day (ugh).

I'll be taking two weeks off to breathe after an intense job search.  I'm spending time with the kids, giving Linda some time off, and then for one week I'm going to do a serious hackathon with a friend on a software idea we've been talking about for a while.  Yes, can you believe it, I'm going to code for my vacation.  But this is cool stuff, and what can I say, I'm a geek.  I've been having to shove this into what little extra time I have, so it's a real treat to dig down and code straight, if only for a week.  If anything ultimately comes of it, I'll be sure to let you know.  But even if it doesn't, I'm going to be having fun and learning a lot.

So, I am very grateful.  Let me tell you, this is no time to be looking for a job.  For all of you out there still looking, I'm thinking of you - we're all thinking of you.

Balsamiq Mockups

A friend of mine was raving about this tool called Balsamiq for quickly creating application UI mockups.

http://www.balsamiq.com/products/mockups/tour

Gregor Hohpe describes asynchronous design using Starbucks

Gregor Hohpe has a wonderful article in IEEE Software Design Magazine called Your Coffee Shop Doesn't Use Two-Phase Commit (PDF).

In the article Gregor describes asynchronous messaging design and error handling by comparing it to how drink orders are handled at Starbucks.

What does Starbucks do if they’ve already placed your drink order into the queue and it turns out you can't pay? They either pull your cup from the queue or toss the drink if it has already been made. Likewise, if they deliver a drink that's incorrect or unsatisfactory, they remake it. If the machine breaks down and they can't make your drink, they refund your money. Each of these scenarios describes a different but common error-handling strategy for loosely coupled systems

So cool. This often happens to me - I am thinking about a "mundane" process such as how families work or why freeways get jammed up and I see how it applies to systems design.

Gregor does a fantastic job of this, helping you understand what may fairly complex concepts very easily by imagining yourself at Starbucks. Definitely a good read.

Where was Joseph when we needed him?

My eight-year old daughter's class just put on a play about Joseph (the one with the many-colored coat). I had never heard the story before (I was raised by devout atheists). It was fascinating. Lots to think about and discuss.

But one thing seemed particularly relevant. Joseph could interpret dreams, and he saw that the Pharaoh's dreams signified seven years of plenty followed by seven years of drought, and he convinced the Pharaoh to put aside one fifth of the harvest every year during the years of plenty, and in this way the Pharaoh was able to feed not only his people but the Hebrews as well.

Arnold Scharzenneger is currently proposing a spending cap, taking it to the ballot this summer. The idea is that, even in years of plenty, the state only spends so much money. Any surplus is kept in a "rainy day fund" and can be used in years of drought. Sounds familiar...

I wish we had had more prudence during our last phase of plenty, perhaps we wouldn't be in such a mess as we are now...

Sun gets behind clouds, may be Eclipsed - thoughts from an ex-employee

I've been ruminating about Sun today, given a series of seemingly unrelated blogs and news items.

First of all, Closer To The Ideal commented how Sun hasn't seemed to be make any headway with Java FX in the last two years.  Tell me about it.

I remember when Rich Green did the Big Splash announcement for Java FX at Java One two years ago - I just groaned.  It was basically an "if you announce it they will come" strategy as far as I could tell - there was no there there.  The technology for the most part had not been built.  It was an announcement of an unrealized vision, not an actual working product.  And meanwhile there were not one but two competing solutions - Flex and Silverlight - from two of the best consumer software companies in the business.  Things did not look good - and still don't.

What made this whole Java FX thing really grating to me was that we as a company had to pull out all stops to get this thing to actually happen, since our VP of all software had put his reputation on the line by announcing it.  They pulled the best and the brightest from the NetBeans and Java Swing teams to work on Java FX.  They pulled our UI design resources.  They pulled our QA resources. And the now NetBeans was working on life support, and JavaFX struggled along, and continues to struggle along.  

The other announcement - Sun is going to provide a cloud offering.  OK, so let's remember Way Back to about five years ago, Jonathan was announcing this grand vision of selling CPU for $1 a CPU-minute.  This was the beginning of Sun's Grid effort.  This project ground along for year after year, having (IMHO) completely the wrong focus - focusing on HPC and batch-oriented processing instead of making it easy to host my application on their infrastructure.  Meanwhile Amazon takes the world by storm.  Sigh...

So now Sun finally announces a move into the real cloud, one that people care about.  But at this point they are way behind their competitors.  What makes them different from Amazon, or IBM, or HP?  Oh, I see, it's "REST-based" and it is with an open license.  Ho-hum...

I really like Sun, and liked working for them.  They have some great technology (Solaris, Java, ZFS, DTrace, Thumper, Glassfish, NetBeans), they have contributed enormously to the community, and are one of the most ethical and respectful companies I have worked for. 

But Sun has had this amazing ability to thrash for a very long time on large and ultimately doomed projects (remember N1?), and they just haven't been able to turn the corner and really reinvent themselves.  I think many of us have been rooting for them for a long time. But I'm losing faith.   No it's not that I'm bitter.  I just have been around the block a few too many times with new strategies and not seeing any of them really get any traction.  The recession has not helped - but hey, IBM is doing just fine, thank you, so somebody out there knows how to run a business.

Ah, IBM.   When I heard that IBM may be buying Sun - now that was interesting.  And it reminded me I had better start learning Eclipse... 

Joe Gregorio shows how to write a bloom filter

A very nice, succinct, clear article both on what a bloom filter is and why you would want to use one. I feel very educated, thanks, Joe!

http://bitworking.org/news/380/bloom-filter-resources

The Pragmatic Bookshelf | Erlang in Practice

I am looking into Erlang as part of my involvement with CouchDB, and I bumped into this Most Excellent production from the Pragmatic Programmers shelf. You actually sit with Kevin Smith, an Erlang expert, through a series of screencasts (downloadable onto your iPhone if you like) as he builds a system from soup to nuts.

I don't think there's a better way of learning a language than this. There are these little details about how to work with a language that just can't be covered in a book. Seeing someone work also helps you focus on approach, what's important to focus on, and in general getting a sense for the "gestalt" of the language and its runtime.

So much better than a book - books are good for reference, but just don't convey this "quality" of a technology like hanging out with someone who knows it.

http://www.pragprog.com/screencasts/v-kserl/erlang-in-practice

New TLDs - boon or boondoggle?

When I was reading up about Minds+Machines and what they do, I encountered some articles and blogs complaining that new TLDs are primarily a way to milk money out of trademark owners because they have to defensively register their name in all these new TLDs.  There do seem to be a lot of concerns, and not a clear understanding of its value.

Advertising Age had an article about this that makes it all sounds pretty ugly, and you could find yourself asking "why is ICANN doing this?  It sounds like a mess!"

Antony Van Couvering (yes, we're related), CEO of Minds+Machines, responded to this article in a comment. You can also find his comment in full on the Minds+Machines blog.

His main point is that there are significant benefits to TLDs, that these benefits significantly outweigh the potential costs, and that the marketing/ad industry should look towards how to take advantage of the benefits rather than naysaying and focusing on the costs.

I particularly liked Antony's point that a TLD can be used as a guarantee of quality - it's a brand you control and you can use it as a brand in the real sense of the word:

If BMW had standards for their pre-owned cars (which they do), then they could provide .BMW domain names for dealerships who met their qualifications. BMW = quality; .BMW = dealer you can trust.

It also appears that there has been a pent-up demand for new TLDs that really can't be denied

These new top-level domains are going to happen, because the demand has been simmering for over 10 years for non-Roman-character TLDs for most of the non-English-speaking world, so that people can use the Internet in their own language. Add to this vocal demands for non-brand-related TLDs such as .AFRICA, .EUS for the Basque Country, .NYC for New York City and so on. These are the real demands that are driving this process, and concerns about cost to brand owners, as real as they may be, are not going to stop it. It's true that the new TLDs will cause some headaches for brand owners, although there are workable proposals out there that will allow brand protection across all new TLDs for as little as $125 per brand.

I'm curious what those proposals are, that sounds encouraging.  Antony worked with companies for years as CEO of NameEngine (then acquired by Verisign), helping them establish/protect their brand across domains, so I suspect he knows of what he speaks.

A great conclusion too, that drives the point home 

But to concentrate on the risk and cost when the opportunities are so great strikes me as a great mistake. There is no reason for a company to spend money branding VeriSign (=.com) when they could be branding themselves, even as they claim a permanent part of the Internet at the very top level. Compared to the opportunity, the cost is truly trivial.

 Maybe I should work with Antony to get the .VANCOUVERING TLD :)

Arnon Rotem-Gal-Oz - 10 Papers Every Software Architect Should Read (At Least Twice)

Excellent, excellent resource, thanks Arnon! Another architect friend was just telling me about a paper that shows you can never reach consensus in a distributed system, and then here is the paper in Arnon's list. I love it when that happens.

http://www.rgoarchitects.com/nblog/2009/02/27/10PapersEverySoftwareArchitectShouldReadAtLeastTwice.aspx

Villagers in India Open Their Homes - NYTimes.com

A very nice article on the growing trend of tiny villages in India opening up their homes for tourist stays. Sounds tempting - when the kids grow up... Make sure you look at the slide show.

http://travel.nytimes.com/2009/03/08/travel/08journey.html

Minds + Machines

Hey, look at that, my brother, Antony, along with two other heavy lifters in the domain name industry, Jothan Frakes, Elaine Pruis, just took the domain world by storm.

He pulled the curtains off of his new company, Minds and Machines, at the ICANN meeting in Mexico City last week. They had a booth there and expected some interest, but the news is there was a lot of interest and they were very very busy.

It appears they are giving the bigger companies in the domain industry a run for the money, having moved from zero to 4th in the space in a matter of a week (according to my brother anyway :)).

At Mexico City they announced that the new .eco top-level domain (TLD) is using them, and also got the icing on the cake that Al Gore is supporting and sponsoring the .eco TLD.

Their web site is clean, cool, approachable and understandable even to those not in the domain business. They are also using and enhancing the popular open source domain platform, CoCCA, to make it very easy to create and manage your TLD.

Why "Minds and Machines?" If I understand it right, the idea is that they are combining the strength of their experience in the domain industry (Minds) with the usefulness and productivity of software (Machines) to provide an excellent experience for managing a TLD.

It also allows them to give away m&ms at conferences :)

Many congratulations to my brother and his company, and may this be a source of abundance to him and a service to the world.

Interview questions and poop

As part of the job hunt, I am reviewing potential interview questions and practicing (as much as one can practice this sort of thing).

There are four major groups of interview questions, as far as I can tell:

• General knowledge - discussions about what you know, questions like "what does the synchronized keyword mean?"

• Coding questions - "reverse a linked list"

• Puzzlers with one or two right answers - "you have two candles that burn for 60 minutes. How do you use them to measure 45 minutes?" - no, you can't cut the candle, or make marks with a ruler...

• Puzzlers with no right answer - "you have been reduced to the size of a nickel and placed in a blender. The blender will turn on in 60 seconds. What do you do to save your life?"
  

I haven't personally been asked the last kind of question, but I've read about them. They're supposed to test your creativity and, I don't know, spunk.

However, a few days ago I had an experience with my son that got me thinking, and I realized that as a father and a homeowner I have to face these kinds of questions all the time. Note: none of these situations are made up...

You're in a bookstore with your three year old child who has just recently been potty trained. You are enjoying reading a book while he plays with the train set in the kid's section when all of a sudden he says matter-of-factly "I pooped." You have no change of clothes and no wipes. How do you clean up your son without making a huge mess, embarrassing yourself, or both?

You are on the airplane to Hawaii with your five-year-old daughter, and there is massive turbulence. The flight attendants refuse to provide water, and there are no airsickness bags. Your daughter throws up on herself. Twice. What do you do?

You have a son who likes how things "work". One day you discover most of your daughter's plastic necklaces are missing. The next day brown disgusting water starts bubbling into your bathtub. What do you suspect has happened?

You hear a roaring sound at 2 am and stumble downstairs to find two inches of water in the kitchen. The roaring sound is coming from under the sink. What is happening, and how wet do you think your pajamas actually get?

You have an invasion of rats that have destroyed your futon mattresses in your garage and have eaten their way through your plastic earthquake bin to tear apart your earthquake supplies. They wake you up every night chewing on the drywall just behind the bed. They ignore all traps. How do you get rid of the rats?

So, perhaps I am more prepared for these kinds of questions than I may think...

Turning Japanese - Paul Krugman Blog - NYTimes.com

Scary stuff

Pretending that distressed assets are worth more than they actually are today for regulatory purposes persuades no one besides the regulators, and just gives the banks more taxpayer money to spend down, and more time to impose a credit crunch. These kind of half-measures to keep banks open rather than disciplined are precisely what the Japanese Ministry of Finance engaged in from their bubble’s burst in 1992 through to 1998 …

http://krugman.blogs.nytimes.com/2009/02/25/turning-japanese/

Facebook's Cassandra - what's going on?

I read with interest the SIGMOD paper on Facebook's Cassandra.  It looked very interesting, an open source combination of BigTable and Dynamo (PDF) .  It appears to be running in production in Facebook serving the very demanding inbox notification system.

So I wanted to try it out.  I went to the Google Code project, and it just seemed, well, dead.  The instructions for checking out the source are wrong both on the Sources tab as well as in the Wiki.  The mailing list has almost no activity.  It looks like they are trying to get into Apache incubator, but meanwhile people are asking what's going on, what the status is, and there appears to be no response.

A consideration for using any technology is understanding the strength of the community behind it.  You don't want to bet your infrastructure on something where there is no support and you are basically on your own to maintain it.

So, if you know what's going on with Cassandra, please let me know.  Meanwhile, I'm going to put it on the back burner and take a look at some alternative clustered key-value stores, like Project Voldemort, SimpleDB and HBase.

Hitting the scalability wall - Amdahl's Law

Author's note: I was thinking about this blog and how I didn't quite capture some things correctly, particularly around what it means for processing to become serialized.  Then I saw some comments that repeated my own concerns.  So I have done some rewriting in an attempt to explain more clearly what I think the issues are... 

When examining the challenges of throughput and scalability in the brave new world of Web Scale, one solution I've been looking at is Gigaspaces.

I recently read their article about scalability (follow the link to "The Scalability Revolution" - you may need to register to be able to see it), and they made a claim that gave me pause, to say the least.

Basically they said that if you have a multi-tier architecture, you are doomed - sooner or later you are going to hit a scalability wall, either because of cost, complexity or latency.  I think in particular they meant that if your end-to-end request/response path must travel over multiple tiers to complete, you are doomed.

They have examples to back up this claim, but what was compelling for me was their reference to Amdahl's Law.

Amdahl's Law states if any portion of your system requires processing to be serial instead of parallel  then the system is going to hit a scalability wall where it can go no further, no matter how much hardware you throw at it.   From the Wikipedia article:

The speedup of a program using multiple processors in parallel computing is limited by the time needed for the sequential fraction of the program. For example, if a program needs 20 hours using a single processor core, and a particular portion of 1 hour cannot be parallelized, while the remaining promising portion of 19 hours (95%) can be parallelized, then regardless of how many processors we devote to a parallelized execution of this program, the minimal execution time can not be less than that critical 1 hour. Hence the speed up is limited up to 20x.

Note that the actual numbers don't matter too much.  You can avoid hitting the wall by reducing the amount of serialization, but sooner or later, you are going to hit that wall.  And even before you hit it, the cost of getting a fraction more speedup is going to increase exponentially.  I have heard many stories to vouch for this, where getting 10% more throughput requires 100 times the hardware investment and/or complete rewrites of applications.

If your system architecture requires you to potentially wait for a shared resource within the request/response path, then you are shifting from parallel to serial execution, and you are going to hit Amdahl's Wall.   Here are some examples:

• Reading/writing data in a shared database
• Sending a request over a network which will block if the network is congested (the network is shared by multiple simultaneous requests)
  
• Writing to a disk which may block if it is busy, where the disk is shared between multiple simultaneous requests.

Well, that does seem to throw a wrench in things.  How can a service run independently, with no database access, disk I/O or network I/O, and still be useful?

Well, the key thing is that these things can not happen within the request/response path.  They can still be done in the background, asynchronously. For example, if the user updates their profile, then this update is stored in-memory and then, in the background, this status update is stored in the database.  If you want to send a message to another service, the message is queued up in memory and then delivered to the external service asynchronously.  

Note the consistency implications of this.  You end up with a system where not everybody has a consistent view of the current state of the system.  But we already know through the CAP theorem that this is something you probably need to accept.

There is also a window of possibility where the user thinks the update succeeded but the actual persistence of this update fails or there is some kind of conflict.  This means your company and your application has to know how to apologize.

But if the benefit of this is linear scalability, these tradeoffs may be well worth it.  Better to apologize to a small subset of customers about a lost order than apologize to all of them for terrible latency or to apologize to your management about the cost and time overruns for your system.

Gigaspaces claims to have accomplished this kind of architecture with a runtime environment they call a space which provides in-memory access to all the resources you need to process a request.  The underlying Gigaspaces runtime ensures that each request is routed to the right cluster node and that the node contains all the resources needed, in memory, for that object to get its job done.   To be honest, I am still mystified how they can accomplish this, and this is something I would like to investigate further.

At any rate, I appreciate the Gigaspaces article helping me understand the implications of Amdahl's Law.  As I evaluate a scalability architecture, if something, somewhere, requires the request/response path to potentially wait on a shared resource, then I know that sooner or later we've got trouble.  It's the law.

InfoQ: CouchDB and Me

Wonderful talk by Damien Katz, author of CouchDB. How to get a job doing cool stuff? Close your eyes and jump...

http://www.infoq.com/presentations/katz-couchdb-and-me

Why not just put the whole friggin' thing in memory?

When I was at Sun, I worked on the design for a replicated in-memory data store. One of the principles was, scale and throughput demands are increasing, and memory is getting cheaper, so why not put it all in memory, and use disk/database only as a backing store. We provided durability not through writing to disk but through replication, and then writing back to disk (e.g. to a traditional relational database) in the background. Even if there were a node failure, we would recover from the replica, not from the database.

We couldn't get this project funded for various reasons. It's been frustrating because I knew we were on to something but couldn't make it happen - and now, as many of us predicted, the industry is moving in that direction - a growing belief in denormalization, caching, and eventual consistency. But still, many applications write to the database as part of the cycle of a transaction.

But this article at HighScalability hit the nail on the head - you evolve your application from database-centric to cache-centric to memory-centric. Money quote:

As scaling and performance requirements for complicated operations increase, leaving the entire system in memory starts to make a great deal of sense. Why use cache at all? Why shouldn't your system be all in memory from the start?

Well, that sounds awfully familiar. And it's true. If you are replicating anyway, your risk of data loss is pretty minimal, and as Pat Helland, Amazon ex-architect says, computer suck, and you should just plan to apologize sometimes. If you batch your changes every N minutes, then you have an N minute window where some changes may be lost. But for many applications, that's OK. And the wins you get in terms of throughput are significant.

Latency was slashed because logic was separated out of the HTTP request/response loop into a separate process and database persistence is done offline.

It looks like the way they implemented an all-memory solution was using Gigaspaces, which, by the way, has a solution that deploys and scales automatically on the Amazon EC2 fabric. And the result: near linear scalability and 1 billion events a day[1]. Yeah, that's the ticket.

[1] I think actually this statement is misleading. Reading the article more carefully, they are claiming they can get to 1 billion events a day. This hasn't actually been tested, so take it with a grain of salt.

Posted via email from David Van Couvering's Posterous

RESTing on the Couch - adjusting my CouchDB experiment

As I was playing with my son - a great way to let the right brain chew over a problem - I came up with a different architecture for my memcache/CouchDB experiment, one that I think makes more sense.

Rather than have a traditional PHP front-end, I realized that I need to follow the model I've been espousing for a while now. The server interface, rather than being HTML/browser-centric, is instead a REST web service. The client could be HTML, but I think my first representation of a REST resource will be JSON, and the client user interface will initially be written in JavaScript. Subsequent clients could be written in JavaFX or Flex or Silverlight or an Apache/PHP server, mine or somebody else's... Exercises left to the reader, or to myself time permitting.

Tempting as it may be ("hey, CouchDB is REST with JSON!"), I'm not going to expose CouchDB REST directly to the clients. In general I think it's dangerous (architecturally) to expose your database directly to clients.

Instead, I'm going to put a middleware layer on top so I can do clustering and caching, and also so I can swap in a different caching and storage mechanisms.



Another solution I'd like to try is Gigaspaces, but I'm getting ahead of myself...

.NYC Gets Big Boost from New York City Gov’t » Names@Work

My brother makes big news today - his effort to create a top-level domain for New York City - .nyc - is starting to bear fruit.

New York City will soon have its own place on the web – with dot NYC. Mark Twain famously advised “Buy land, they’re not making it anymore.” Well now we can make more New York addresses – just on the internet! A local business won’t have to outbid a guy in Kansas to get Tony’s Pizza dot com. They’ll be able to get Tony’s Pizza dot NYC, a name associated with the greatest city – and home of the greatest pizza – in the world.

http://www.namesatwork.com/blog/2009/02/12/nyc-gets-big-boost-from-new-york-city-govt#comment-106721

Setup a Memcached-Enabled MAMP Sandbox Environment | Lullabot

Here are some great instructions for getting yourself bootstrapped with memcached on Mac so you can work in a local sandbox.

http://www.lullabot.com/articles/setup-memcached-mamp-sandbox-environment

Setting up a CouchDB cluster fronted by memcached

While I'm looking for work, I thought I'd get to play around with some technologies that I have been very interested in but I haven't been able to get really hands-on with because of my day job.

My first project - set up a CouchDB cluster fronted by memcached and run a simple PHP application on top of that.

The architecture I'm thinking of looks something like this. I'm sure I'll refine it over time after playing around with it, but starting fairly simple first:

• A web tier running Apache and PHP
• A memcached tier
• A CouchDB cluster tier

I will distribute records across the CouchDB cluster by using a distributed hashing algorithm against the keys. The ones that look most promising are Kademlia and Chord. I'm leaning towards Kademlia.

I think it's cool that the same qualities you need for peer-to-peer file sharing are valuable for server-side clustering - even distribution, good performance when nodes come and go, and robustness under heavy changes to node configuration.

Once I have something working, I'll let you know my experiences. Then I'm interested in trying this with SimpleDB and Project Voldemort... I also want to take a look at MemcacheDB. So much interesting stuff, so little time :)

The Canonical Cloud Architecture | High Scalability

This is a very nice and useful architecture with some "patterns" for cloud architectures - although I believe a lot of these patterns can be applied to any system of large enough size. I am particularly a fan of queuing as a great option for tying together loosely coupled system components in a way that can scale and perform. If you can handle delays in your system and don't need a lot of synchronized workflows, it is one architecture I personally would look at very seriously.

http://highscalability.com/canonical-cloud-architecture

Hm, I'm not sure about this new president

You know, I was always bashing Bush, but at least I had a job under his administration. Two days after this Obama guy takes office, and I'm out of a job.  What's that about?

:)

Funny statements from Michael (age 2)

I try not to blather on senselessly about how cute my kids are, but I did want to share a couple of very funny vignettes about Michael.

Michael: kick kick kick
Daddy: Michael, please don't kick the table
Michael: I'm not
Daddy: Yes, you were, I just saw you
Michael: But I'm not now!

He does this a lot!

---

Michael: I went poo!
Mommy: You mean like Winnie the Pooh? (snicker)
Michael: No! This poo is icky and it doesn't talk!

See, see? Isn't he the cutest kid ever?

My new laptop is here!

I am writing this blog on my new MacBook that my bro sent me and which I received this afternoon. 

Very nice machine - 150GB disk, dual cpu, 2G of memory.  Nothing to sniff at, especially for the cost of FedEx shipping.  Thank you, thank you, thank you, Antony (aka my big brother)!

The reason I was able to get going so quickly with this is SuperDuper!  I love this tool.  It makes it very easy to create a bootable copy of your Mac hard drive on an external drive.   I did this before turning in my old laptop.

Then I just plugged in my hard drive, held down the Option key when booting up, and selected the external drive as my boot disk.  And voila, I'm back in my old environment!  It is so nice.

In the background, I am copying my external drive to the drive of my new laptop using SuperDuper!  When it's done, I reboot using the internal drive, and I'm good to go.

One could only wish things on XP were this easy.  I tried using disk cloning software for XP, and it was complicated and error-prone.  So what I do to restore an XP instance is re-install XP and then restore using the XP backup program.  Sigh...  Not nearly as nice.

So, thanks to Antony and SuperDuper!, I am a happy camper.