An expression of joy

A friend just posted this poem by the great Sufi saint Hafiz:

I am happy even before I have a reason.

I am full of light even before the sky can greet the sun or the moon.

Dear companions, we have been in love with God for so very, very, long. What can Hafiz now do but forever dance!

Wow, taking the train rocks

This Christmas we decided to take the train to my Mom's place in New Mexico rather than fly.  What a wonderful experience.

For the same cost of a flight, we got first class service on the Southwest Chief from Los Angeles to Albuquerque.

Someone gave us and our luggage a ride to checkin and then another ride straight to our car on the train. We had a porter who turned down our beds at night and made them back into chairs in the morning.  Free meals, coffee, paper, juice.  Hot steaming washcloths to refresh your face, and hot showers with towels and soap provided.  Serious leg room, quiet quiet cars, an observation car to hang out in, and incredible, gorgeous views.  I mean, incredible, gorgeous views.

Quiet, relaxing, the kids were happy, we were happy, and arrived in Albuquerque refreshed and relaxed.

Then I think about airplanes.  Long lines, zero leg room, security checks, kids are not allowed to move around and have nothing to do.

I'm telling you, trains are the hidden secret of travel.  I highly recommend it.

Foxmarks closing its doors

Update: Foxmarks was bought by LastPass, which is also a great company, so it's great to hear they'll stick around! We'll see how LastPass monetizes this; it's a great service!

----------

I love the Foxmarks service. But it appears they couldn't come up with a business model.

For four years we have offered the synchronization service for no charge, predicated on the hypothesis that a business model would emerge to support the free service.

I am amazed people are still finding investors for a business plan like this. Oy vei..

Remind me, when and if I ever join a startup, to make sure there is money coming in from day one.

I guess I'll need to move to sync for Firefox and Chrome. Sorry to see a cross-browser solution go...

BigCouch: HA, clustered, scalable CouchDB

Not sure how I missed this one, this is awesome news. 

http://github.com/cloudant/bigcouch

“BigCouch is a highly available, fault-tolerant, clustered, mostly api-compliant version of Apache CouchDB. While it appears to the end-user as one CouchDB instance, it is in fact one or more nodes in an elastic cluster, acting in concert to store and retrieve documents, index and serve views, and serve CouchApps. BigCouch has been developed and is continually maintained by Cloudant who offer hosted CouchDB as a service.
Clusters behave according to concepts outlined in Amazon's Dynamo paper, namely that each node can accept requests, data is placed on partitions based on a consistent hashing algorithm, and quorum protocols are for read/write operations”

I worked with the Cloudant team, the folks who created BigCouch, when I was doing some CouchDB work on the side, and they seemed like a smart, fun, dedicated team.

This puts CouchDB on my short list of technologies to consider when looking for a scalable data storage solution.

Republicans promise to abolish all government

I love this line from Krugman's editorial about the Republican budget proposals (where they propose to continue the Bush tax cuts, while the only cut they propose is TARP, and that they will keep their hands off Social Security, Medicare, and the defense budget).

Howard Gleckman of the nonpartisan Tax Policy Center has done the math. As he points out, the only way to balance the budget by 2020, while simultaneously (a) making the Bush tax cuts permanent and (b) protecting all the programs Republicans say they won’t cut, is to completely abolish the rest of the federal government: “No more national parks, no more Small Business Administration loans, no more export subsidies, no more N.I.H. No more Medicaid (one-third of its budget pays for long-term care for our parents and others with disabilities). No more child health or child nutrition programs. No more highway construction. No more homeland security. Oh, and no more Congress.”

Of course what is obviously really going on is that the Republicans are using the populist fiscal responsibility tune as a Trojan horse for handing money to the people who are funding their elections.  It's basically a smoke-and-mirrors approach to looting the treasury and handing out gifties to their buddies.  They don't really care about balancing the budget, they're just using that as a cover.

So what should we do about it?  I don't know.  It seems that the only time people are really ready for change is when the truth can no longer be denied.  It's true for addiction, and I suspect it's true for governance and economic policy as well...

Personal favorites for upcoming Java 7 features

Through the power of a Twitter search column in Tweetdeck, I found a great InfoQ article which did a quick summary with lots of links to what's new for Java 7.

After reading through this, here are some of my favorites:

Automatic Resource Management
Tired of closing input streams, SQL connections, sockets, and so on?  You will no longer need to in Java 7.  Sweet!

Generic Type Inference
Allows you to leave off specifying generic types if they can be easily inferred.

I'd show you an example but this stupid Blogger editor gets completely confused with all the less-than and greater-than symbols, and I simply don't have the time to futz with it.

Closures through Lambda Expressions
This looks like a very promising implementation of closures in Java.  Of course the devil is in the details, but I look forward to making use of this - personally I'm quite tired of anonymous inner classes.

Modules (Project Jigsaw)
I have worked with NetBeans modules and played around with OSGi, and I have seen the power of a well-defined module framework.  I look forward to seeing this as part of the language (again, hoping it is implemented well.  We'll see of course).

Calvin demonstrates socialism for the rich

I've always loved this particular Calvin and Hobbes strip. It's still as appropriate today as when it was written, if not more so...

My visit to Ground Zero

I had the honor and fortune to pay my respects at Ground Zero today, coming back from a business meeting. It really is quite awesome to place yourself there where it all happened. I also feel I got a chance to pay my respects to all those who lost their lives here, whose lives were affected, and who gave (and continue to give) their hearts and bodies to saving lives, digging through the rubble, cleaning it up, and now building new buildings and memorials that honor the power and sacredness of this place.

I watched the construction workers working on the new memorials, and I thought, whereas most construction is "just a job", those working here are truly blessed and are doing a labor of love.

I also went to the small little museum in honor of 9/11. What an incredible, moving exhibit. It was all so personal, with simple artifacts and quotes and stories from those who lived through it. Each artifact had meaning and impact - the melted, destroyed window from a plane, a half-burnt menu from Windows on the World, a fireman's helmet and torn jacket, and a wall covered with copies of all the "missing person" leaflets that were placed out by desperate families looking for their loved ones. I am not one to cry easily, but I was in tears many times as my heart leaped to my throat.

Then, downstairs, was a place you could leave a note for everyone, and many of these were placed lovingly on a wall for all to read. People from around the world expressed their love, their sadness, and their hope. This place is impactful, important, and very moving. I highly recommend everyone try to make it here at least once.

I also saw the plans for the new memorial. Truly beautiful - two square waterfalls going into the ground in the place of the original towers, with the names of all who perished around the walls of the waterfall, inset in a beautiful contemplative garden of oak trees.



Today was a crisp, clear blue September day here in New York, much like the day 9 years ago when this happened. What happened here was so horrific, but now it feels like there is a power here that holds it with great care, with gentle, loving hands. You feel yourself bow inwardly in prayer and respect, and offer your love and blessings.

So, in honor of the memories of all who lost their lives and their loved ones here, I offer my blessing:

May we all experience and offer love and protection, and may we always remember that each and every life in this world is supremely precious and sacred.

Om Shanti Shanti Shantihi (OM peace, peace, peace).

Mass extinctions, basalt flows, and us

Verbatim from my Pa (Dr. John Van Couvering, geologist and all-around Scientist), who wrote this up for a his local neighborhood newsletter in upstate New York

Most of us have heard of the great extinction that wiped the dinosaurs off the face of the earth.  Of course this didn’t really happen, because birds – aha! -- are dinosaurs. But few may be aware that the human presence on the planet follows, not one, but four narrow escapes from extinction, going all the way back before dinosaurs even evolved. We are rather like Fearless Fosdick (Li’l Abner’s comic hero) who pops up unscathed, with the lid on his head, from his hiding place in a trash can that has been riddled to shreds by the mob’s tommy guns. The crooks are gobsmacked. “Fosdick! We shot ya fulla holes!” “Took a bit of dodging,” he jauntily replies.

What did we dodge? In fact, those “extinction events” are deadly beyond belief: moments in planetary history when nearly everything on land dies, as well a great deal of marine life. In the last of these, 65 million years ago, four entire orders of dinosaurs were exterminated, plus all but the airborne varieties of a fifth order, as well as two out of three orders of coral and entire groups of other marine life. To put the extinction of an order in perspective, extinguishing the modern Carnivora would require that every single living animal in that order -- every last cat, lion, tiger, leopard, cheetah, puma, jaguar, hyaena, aardvark, civet, dog, wolf, fox, bear, panda, seal, sea lion, walrus, weasel, mink, skunk, badger, wolverine, otter, raccoon, kinkajou, coati mundi and meerkat – would have to perish. If even one lonely pair of any of these species survived to reproduce, so would the Order Carnivora. The magnitude of disaster that can totally erase not one but a dozen orders is difficult to comprehend.

Huge meteorite impacts have been found to coincide with some major extinction events, but they may have been just the last straw. For one thing, giant impacts are turning out to be surprisingly common, with little or no extinction effect. While the asteroid that punched into Yucatan may have finished off Tyrannosaurus and the tetracorals, they were apparently sick and dying already. A few million years later, the equally large flying rock that blew a 57-mile wide hole into the bedrock under what is now Delaware and Maryland had no effect on earthly life whatever. Not only that, no impact has been found to coincide with the worst extinction of all, the Permo-Triassic event 250 million years ago, that killed off all but one species of non-amphibian land animals – which species, of course, is our ancestor. The real cause of extinction events appears to be flood basalts – extraordinary volcanic episides that go on for a million years as flow after flow of lava wells from cracks in the continent to build up country-sized mounds called “traps”. It is probable, but not yet proven, that the gases emitted from the thousand of cubic kilometers of flood basalts create toxic pollution of land and sea. The Deccan Traps of central India mark the end of the dinosaurs, while the traps that underlie the steppes of Siberia are the same age as the great Permo-Triassic extinction.

Given the odds, it is to be expected that mass extinctions have chancy, even unfair consequences. After the Permo-Triassic massacre our ancestral line succesively developed into anapsids (turtles), synapsids (mammals) and diapsids (everything else). Each of these is a higher stage of evolution, with a basic physiology better adapted to life on land. By some freak chance, a single lowly synapsid got through the 190-my extinction at the end of theTriassic, along with a variety of diapsids.on their way to crocodiles, lizards, and all kinds of dinosaurs. This primitive synapsid line, which seems to have favored bugs and fruit, quietly developed warm-blooded circulation, milk glands and eventually a uterus. And when the grand house of the dinosaurs came crashing down, guess who was hiding under the ruins?

Our final lucky break came around 48 million years ago, when a solitary species of primitive primate got stranded on the (then) island of Africa, never to see its many relatives in North America and Asia again. Bingo, Columbia River basalts. This was not one of the great extinction events, but it was enough to kill every primate in the world – except for that forgotten waif in Africa. Now, many lemurs and lorises and monkeys and apes later, here we are, self-importantly going on about our human-caused extinction event. If we knew how far we have yet to go, perhaps we just might give it up.

Apache Cassandra: materialized queries (one table per query please)

Max Grinev has done an excellent job giving a quick overview of Cassandra here, and then has a followup blog post that discusses how to accomplish various SQL-style operations such as select, join, and group by.  Very helpful, thanks, Max!

Note that in every example, you don't write a query against the existing data.  Instead, you create a new column family to represent the query.  It's as if you write your query as a data structure.   It's basically a materialized view.   Max argues that this is OK for most applications: "However, typically in Web applications and enterprise OLTP applications queries are well known in advance, few in number, and do not change often."

OK, fair enough, but it is definitely something to keep in mind: if you want to support ad-hoc queries in your application, then Cassandra is probably not the right choice.

It also looks like it is your job as a user of Cassandra to update all your various views when data changes.   I agree with Maxim that denormalization and "push on change" rather than "pull on demand" is probably the right approach for highly intertwined systems like Twitter and Facebook.  But it would be nice if the system helped you maintain consistency across all these denormalized copies.

For example, CouchDB has a very similar model of materialized views, but their views are defined as map/reduce operations on the primary document(s), and CouchDB takes care of keeping the views in sync as you update your data.  That saves me a lot of time and worry.

Perhaps something like that exists in Cassandra and I don't know about it?

Anyway, I've been very interested in Cassandra, and Max's blogs were quite helpful in  helping me get my head around it.  I highly recommend reading them if you'd like to learn more about Cassandra, or what a distributed column-oriented key/value store looks like (Cassandra is modeled after Google's BigTable).

Facebook's deal with the Devil

I wanted to share what I thought was a very enlightening blog post by Danah Boyd on the recent ruckus around Facebook and her issues with how Facebook is handling all this.

I think she particularly nailed what's bothering me here:

I’d be a whole lot less pissed off if people had to opt-in in December. Or if they could’ve retained the right to keep their friends lists, affiliations, interests, likes, and other content as private as they had when they first opted into Facebook. Slowly disintegrating the social context without choice isn’t consent; it’s trickery.

It's clear that Facebook really really really wants to make all the content people are posting as public as possible, because they are not making money on ads.

I can just see all these marketers from Big Companies in Big Conversations with Facebook execs in closed rooms waving wads of cash at them saying "Boy do we want this content! People are telling us exactly what they like and what they care about! Just make this content available, and we will give you so much money you'll be swimming in it."

In the other room are the board members saying "so, when are you going to monetize this big thing you've got going?"

So, you take the Devil's deal, what's a little privacy, I mean, people want to be transparent anyway. Rationalize and justify, I know that path. So, hoping nobody's watching, and putting on a show of "choice" and "options", you open up the content, and the guys with the money cheer.

If it were me, and I were at Facebook, I'd have a bad taste in my mouth right now...

Time to tie my camel to the tree

I like to not worry too much about security, and try to feel relaxed about living my life on the planet. But I also know that you need to take necessary care. As the old saying goes "trust in God, but tie your camel to the tree."

I read today in the New York Times that hackers have made 1.5 million Facebook user credentials for sale.

I also got a phone call today from someone saying they were extending my Popular Science subscription, and confirmed my address and name. Then they said if I could just give them my credit card number they could go ahead and extend my subscription.

When I refused, they were OK, but asked for my email so they could send me an email confirmation. When I refused again, they wanted to have me talk to their "manager" and I just hung up. This is the second time I've gotten a call like that. This isn't just random identity theft - this is a business. And I suspect a very profitable one.

I also was reading about online payment services, and how PayPal is targetted for attacks on a regular basis.

And I have a feeling it's only going to get worse - the hackers are on the attack, and from what I know about online sites, they try to be secure (maybe) but with dynamic client-side code and phishing and Trojan horses and worms and insider attacks and just dumb human error, the weaknesses will be found and taken advantage of, more and more and more.

I can't protect myself from all of this, but I do need to take necessary precautions. We already have a shredder and shred anything that has our personal information on it. But I've been lazy about my passwords. I've kept the same one for years; it's short, and it's everywhere. Well, that changed tonight. The key passwords have all been changed, and I'll do more as I bump into them.

Symantec has me change my password every three months. I'll be using that as a trigger for me to change all my online passwords.

I've had enough, it's time to tie my camel to the tree.

Diagrammr - very cool diagramming tool

I just want to do a shout out for a very nice diagramming tool I bumped into via a referral from my Twitter stream.

http://www.diagrammr.com lets you write simple English phrases that describe either relationships between entities/objects/concepts or interactions between them.  For each sentence, the tool creates two nodes (subject and object) and a path between them (either a relationship or an interaction).  You can easily share this, collaborate, or embed it, as below (just right-click on diagram and copy the image URL, and paste it into an img tag)



I can imagine even doing this live during a design walkthrough or presentation, to help make the learning more dynamic.

The web site doesn't say who the author is, but kudos to whoever they are, very creative and very simple.  Thanks!

Bay Area casual carpool - an experiment in market dynamics

For the last year I have been using the "casual carpool" that has evolved on its own here in the Bay Area. It costs $4.00 to cross the bridge, and $3.50 to take BART into San Francisco. The bridge is also always jammed.

However, if you have a carpool of three people, you get a special lane through the bridge tolls, and there is no charge. As a natural law of the free market, the casual carpool developed. There are agreed upon locations, and there is a line of people and a line of cars. Two people get into a car, and together you go across the bridge for no charge and faster than both BART or driving alone. Perfect!

Now CalTrans has thrown a wrench into a beautifully running machine: as of June they will be charging $2.50 for carpools to go across the bridge. Ugh. They could at least have made it $3 so it would be easy to split amongst three people.

So what happens? Do people stop doing casual carpool? If not, do riders pay, or does the driver eat the cost? Do we have competition in the lines, with people jockeying for the front of the line if they're willing to pay more to get into the car? What happens when there is a line of cars and no people? Do cars willing to take riders for no charge get precedence? Do they stick a sign out of their window saying "will take riders no charge?" Does each car have a sign on the window saying how much you have to pay to ride? What happens if one rider pays and the next rider to get in refuses to pay?

As a driver, I think I would just take whatever people are willing to pay me.

As a rider, I would pay what I feel is my fair share and also an amount to maintain good will - I value good will a lot when riding in someone's car :). So I'll probably be prepared to pay $1, regardless of what the other rider pays. I am not interested in haggling with either the driver or the other rider - this just creates unnecessary tension, which is not worth it for me in my daily commute.

How this will ultimately evolve, I'm not sure, but it will be a very interesting experiment in market forces.

In code, a rose by any other name smells less sweet

I regularly get jibed by coworkers for having naming discussions. When my daughter was hanging out at the office the other day and sitting in the back of one of my meetings, she snorted trying not to laugh as we kept going back and forth on how to name something. Just yesterday someone forwarded me Juliet's quote "What's in a name? A rose by any other name would smell as sweet." Very true, very true.

But in code, there is a lot in a name. A different name does not smell as sweet.

The name you give something has a lot of power. In many spiritual traditions, great importance is given to words. Kashmir Shaivism holds that the world is created through the power inherent in syllables and words - matrika shakti. Syllables actually concentrate the universal energy and create form, like vibrations of sound forming shapes in water or grains of sand.



I find this to be particularly true in code. The name you give something gives it a certain power. If you name it well, the role and intent of the class becomes crystal clear. It helps you refine what the class is doing, and strip away what it shouldn't be doing. If it is poorly named, often the code in the class is unclear in its design and role as well.

A great example of fuzzy naming is the famous "Utils" class. You might as well call this class "DumpingGround" because that's what it becomes. Same regularly goes for FooManager, FooHelper, and so on. It doesn't give any real idea what it's for, so it becomes a "whatever" class. Hm, I have a method I want to write, and I don't really know how it fits into the overall design, but hey, here is this Utils class, I'll just use that!

Names also become short-hand for well understood design patterns. If you name something FooBuilder, FooProxy, FooObserver, and so on, then readers expect it to generally follow the pattern. Misusing these well-understood suffixes is asking for trouble.

I often find myself renaming classes over and over again as I work on a piece of code. Each time it helps me refine my understanding. It's like the class and its role in life is slowly taking form out of the primordial ooze, until it stands there clear and strong, with a well-defined shape and purpose.

So laugh and snort all you want, ye plebians. Shakespeare wrote beautiful, beautiful prose, and I am sure he took great care to find the right words. He even invented many words we continue to use to this day. To me writing good, clear code is like the craft of prose - there is a lot to a name, and finding the right name is both an effort and a joy.

Deploying a Google Web Toolkit app to CouchDB using couchapp

I've been working on a side project using Google Web Toolkit (GWT), and I'm using CouchDB as the backend store.

There is a project called couchapp which makes it very easy to write an HTML/Javascript-only application (you know, the new cewl way to build apps), and store it directly in CouchDB.

Huh? What are the benefits of doing this?

Well, offline support is the big one for me. If I install a CouchDB instance on my local machine, and then set up replication, my app and its data can automatically be replicated to my local CouchDB. I lose my connection, that's fine, everything I need is there. I get back online, my changes are automatically replicated back to my CouchDB in the cloud.

It also makes it easy to share apps - just publish to a common 'app hub' and anyone who wants to can subscribe and get the app. If you're feeling kumbaya, your app can be read/write, so that app users not only modify the content, they can also modify your app itself and replicate your changes back up.

One more thing: with my app deployed to CouchDB, I don't need to write server-side code. None. No PHP, servlets, JAXB, XML, MySQL, blah blah blah. I can directly access CouchDB (and other web services like Facebook, Twitter, etc.) from my client-side JavaScript. To me that's a much simpler, cleaner environment.

But the thing is, couchapp by itself just supports HTML, JavaScript, and attachments. After much soul searching and feeling un-hip, I have finally come to terms with the fact that I am comfortable and happy in Java. I could learn JavaScript, but I think and breathe Java, it's what comes naturally to my fingers. But Java isn't native to the browser, and that's not the way the wind is blowing. So recently I took another look at Google Web Toolkit, and I liked what I found:

• Dynamic client-side browser-native apps without having to write a lot of HTML/Javascript. Yeah, I like that.
• Work in Eclipse, a comfortable place for me to be
• Excellent integration with CSS, allowing easy styling of widgets by applying style settings your GWT Java widgets
• Static typing. I actually like that, I have much more confidence that when I run it works.
• Very easy code/run/debug cycle, in Eclipse
• No need to deploy, just refresh your browser and your new changes are in
• Excellent support for AJAX-style HTTP requests
• Lots of widgets, and some very cool libraries, including a visualization library
• Open source under a flexible license
• Strong community

Basically, in the GWT world, JavaScript is like your bytecode - you really don't need to look at it and in general you don't care. As with 'asm' in C, you can 'drop into JavaScript' if you need to, but really that can be kept to a minimum. I highly recommend doing the tutorial, it's one of the best I've worked with in a while.

So, what do couchapp and GWT have to do with each other?

Well, it turns out it's pretty darn easy to drop a GWT app into the couchapp framework, press 'couchapp push', and voila, your GWT app is deployed to a CouchDB instance.

Take a look at the GWT StockWatcher tutorial app running out of CouchDB on cloudant. Add some pretend symbols and it simulates random fluctuations in stock price. This was all written in Java, compiled down to JavaScript, and then deployed to a CouchDB instance on cloudant with a simple one line command...

So, here's all you need to do:

• Get couchapp (on Linux or Mac you should be able to do sudo easy_install -U couchapp)
• Run couchapp generate myapp to generate a directory structure for your app.
• Remove everything under myapp/_attachments. You can keep the views if you want, or remove them.
• Get GWT (preferably the plugin for Eclipse)
• Create a GWT project
• When you're done testing/debugging locally, compile your GWT project for deployment
• Copy everything under the 'war' directory except WEB-INF to the _attachments directory of your couchapp application directory
• Run couchapp push (set up your .couchapprc first with the right settings)
• Go to the URL provided by couchapp push and you should see your app running beautifully

A girl and a fish

Tears in eyes laughing. 

http://hyperboleandahalf.blogspot.com/2010/03/how-fish-almost-destroyed-my-childhood.html

I had a very similar going fishing with my Dad, where I caught a fish and couldn't stand to see it suffer.  Dad told me to whack it with a stick - whack/flop/whack/flop - OMG I am *never* going fishing again.  Horrible.

WTF? - SQLite files as a network messaging format

A colleague told me about an idea his team is considering that at first had me going ewww!

They are thinking of replacing the use of Google protocol buffers with a SQLite file for their network communication format.

Huh?  It didn't make sense at first.  But then I chewed on it some more.

Their protocol is primarily about sending one or more queries, and getting back potentially quite large tabular data sets as a result.  What better format than a compressed relational data format?

The SQLite database format is in the public domain.  It is self-describing.  It is compressed, saving network overhead.  The format is portable across platforms.  It is tabular in nature.  You can use the SQLite APIs to to inspect, slice and dice the results, both for processing as well as for troubleshooting/debugging.  You can also use the SQLite API (e.g. SQL INSERT) to incrementally build up a request or response before sending it over the network.

SQLite is embedded, so you don't have to deal with connecting to the server and yada yada, and it's quite cheap to create new databases (each request and response is a new SQLite database).

The more I think about it, the more it seems like a very compelling approach...  It goes against everything I've learned about network protocols, but for their use case, it actually makes a lot of sense.

Error message 101 - help me fix the problem - Facebook FAIL

Facebook has a new feature that lets you reply to a comment by email.  Very nice!

But if for some reason this doesn't work, here's the message you get by email:

Sorry, your comment could not be added at this time. Please try again later.

Please try submitting this comment through the Facebook website instead.

Thanks,
The Facebook Team

OK, what comment?  For what status update? You could provide me a link to the status update and a copy of the comment I tried to send.

So instead I have to do all this work of tracking down the original email I sent, if I still have it, and then go to the status page and copy and paste it.

Design principle: question your assumptions

A lot about good software engineering and design is about listening - listening to that little voice in your head that's telling you you're pushing it, forcing a design or an implementation onto a problem that's not a good fit.

It's very hard for me to listen to this voice when I've got a particular solution in mind, and I'm plugging away implementing this solution, and if I can just get past this next hurdle, it will work, dammit!

Yesterday I finally noticed I was banging my head against a problem.  I thought about what the struggle was, and I realized that most of my pain was in trying to coordinate two separate threads of control through the system - one doing updates and one posting those updates to listeners.

So I decided to apply a common principle for me: question your assumptions.  Did they really have to be coordinated?  I contacted some other members of the team and our product manager and asked what they thought.  It turns out there was much more flexibility than I thought was possible.  If I tried to stop having absolute control, as it were, relaxed a bit, and let things be a little loosy-goosy, then my overall design became much less coupled and thus much simpler.  Ahhh....

It reminds me of the beauty of eventual consistency - if you let go some of your tight constraints, it's amazing how it opens things up and brings new possibilities into the solution.

Armageddon - to a doctor!

According to House Minority Leader John Boehner, Armageddon is here because the healthcare bill passed.

I suspect for the millions of uninsured or underinsured, that's true. They're all saying "Armageddon to a doctor, finally!"

Pepper in your water? I'm sorry...

Michael loves mechanical things.  So it's no surprise he loves our big fancy pepper shaker that we bring out to the dinner table sometimes.

Tonight when he saw it on the table he asked Linda "can I put pepper in your water?"  She said no.

He asked me next.  I said "I don't have any water." (I don't usually drink water with my meals).

"Why not?"

"Because I don't want anyone to put pepper in it."

---

Fade to fifteen minutes later.  Linda looks at her glass of water, and there are little black granules floating in it. She says "Michael, did you put pepper in my water?"

He says, with great sincerity, "I'm sorry".

We all laugh.

He says, "It was an accident."

The little rascal.

The amazing thing was there were four others sitting at the table, and none of us saw him do it.

Zone of Big Things

Via @ntoll, I really liked this graphic comparing the Project Manager's view vs. the Developer's View of various technologies.

CouchDB - you're up there in the No-Fly Zone! Cassandra and Redis are probably off the map...

Search engines, tech culture, and the public good: my sister's thesis

My sister, Elizabeth Van Couvering (@ejvanc on Twitter), is inches away from getting her Ph. D. at the London School of Economics. Her thesis draft can be found here.

In the case of search engines, the evidence indicates that the culture of the technological entrepreneur is very effective in creating a new mega-business, but less successful in encouraging a debate on issues of the public good or public responsibility as they relate to the search engine industry.

Great stuff from the conclusion:

search engine producers appeared to construct the search index and search results so that the most important markets they serve will have the most relevant results and they will thereby achieve greater customer satisfaction and higher revenues.
...
These processes were often carried out with high levels of secrecy. They create search engine results that are highly opaque to users and a climate in which questions of the public interest in search engine results, as opposed to questions of technical efficiency or market success, are far removed from their everyday search operations.
...
Finally, the prevailing wisdom and common sense within the search engine production community presents challenges for those who wish to discuss public interest or information ethics in connection with the search engine industry as questions in this area infringe on the values of “relevance” or of “customer satisfaction”. It will be the role of actors outside the search engine community, no doubt with the support from certain actors within it, to raise awareness of issues around bias and its implications for the public interest and to provide potential alternatives
...
The “internet culture” is predominantly individualistic, market oriented and opposed to most forms of regulation. Putting the case for a public interest in online media is fraught with difficulty. During this research I was met with bemusement and puzzlement, as well as encouragement, from many sources. Having said that, search engines operate in many parts of the world, notably in Europe, where the tradition of public service media is strong as compared the to the US. These parts of the world may be the best places to encourage discussion about how innovative approaches can be used to create a search engine industry that can help to deliver less biased search results and, arguably in consequence, a fairer information society for all.

Interesting stuff. 

I remember when she was doing the research, she tried very hard to get into the Google campus and "observe" the search engineers, trying to get a sense of bias in their work, but no go.

I can imagine the "bemusement and puzzlement" of techies as she asked them about those namby pamby things like "public service" and "public good."  What is it about our culture, particularly Silicon Valley, that places the social contract ("what's that?") so low on the totem pole.  I suspect it's the same individualistic spirit that drives a lot of innovation - if I can invent this cool new thing, I can make lots of money.  All other considerations are a distraction and an obstacle.

Something to think about and be aware of.  And I agree with her conclusion - the search engine industry and other parts of the tech sector are not going to address issues of public good and information ethics on their own - they need to be forced into action by the users delivering them all this search traffic, and by the governments that represent them.

Tim O'Reilly, John Battelle - you listening?

Oh, and by the way - CONGRATULATIONS, ELIZABETH!

Bubbles for Java

A number of folks have been tweeting about a fascinating IDE currently in closed beta out of Brown  University, called Code Bubbles.  The visual paradigm is very compelling - it lets you organized related snippets of code into "bubbles" - building them up easily as you navigate through a particular thread in your source tree.  So rather than having everything organized by package name, it's organized by a path that groups a set of source files based on a given scenario (for example a particular method invocation and its effects). From the site:

A quantiative user study indicates that Code Bubbles increased performance significantly for two controlled code understanding tasks. A qualitative user study with 23 professional developers indicates substantial interest and enthusiasm for the approach, despite the radical departure from what developers are used to.

Hard to really explain, the best thing to do is watch the video, embedded below...

Code Bubbles is built on the Eclipse platform, so can be run on top of your existing Eclipse environment (although the integration sounds pretty light). It's in closed beta, so you can sign up and they'll get back to you. At some point it will open up for a public beta...

Cool stuff!

Analogue is like...

Via a tweet from @mikeal, I heard about this thing called Analogue:

is like comments, meets im, meets irc, meets your favorite paste app, meets instant coffee.
actually, instant coffee sucks.

Took me a bit to figure it out, but basically you click on a link, it starts a new discussion window with its own unique URL. Then you can send that URL to others who you want to invite to join a discussion.

Very simple, to the point. I'll keep it in mind next time I very quickly want to start a conversation with a bunch of folks.

If you want to contribute, go to the Analogue open source site.

The exciting life of married with children

This article on New York Times talks about a new Twitter/iPhone app called Miso, which lets people with not much of a social life try and have the same fun as people using Foursquare.

I just had to smile. I don't even have time to watch a movie. I don't know if I'd call it a boring life, but it sure is different from those jet setters and swinging singles becoming mayors of all the best restaurants and night clubs. I smiled as I thought of some of the possible Miso checkins:

"Michael clogged up the toilet again, thank goodness for our plunger"

"Fell asleep reading to the kids, no movie tonight."

"Back at the grocery store again, I forgot the butter."

I'd definitely be mayor of our local grocery store and the nearby playground, at least on weekends.

It's a good life - strike that, it's a great life. Just not one that has that zap, zing and zam of the young and restless.

Actually, I'll take the uneventful joy and contentment of my family life over loud restaurants and night clubs and cramming myself into airplanes to head off to Paris, London and Bangalore, thank you very much.

Time to heat up the tamales for dinner...

MySQL Drizzle team joins Rackspace

Well, more defections from Oracle, it's clear where the wind is blowing. It's as if all the cool and interesting stuff is quickly shedding itself from Oracle.

Jay Pipes has a good blog post about the announcement and the history behind them ending up at Rackspace.

Interesting quote:

"Rackspace is also heavily invested in Cassandra, and sees integration of Drizzle and Cassandra as being a key way to add value to its platforms and therefore for its customers".

I look forward to seeing what that's about.

I also liked this from Jay:

"I don't know whether Larry understands that cloud computing and infrastructure-as-a-service, platform-as-a-service, and database-as-a-service will eventually put his beloved Oracle cash cow in its place or not. I don't know whether Oracle is planning on embracing the cloud environments which will continue to eat up the market share of more traditional in-house environments upon which their revenue streams depend. I really don't."

Simon Phipps leaves Oracle

Simon Phipps, Sun's Chief Open Source Officer, announced his departure today, after 10 years at Sun.  I had a great time working with Simon on Java DB, a smart, fine guy to work with.

I am really not at all surprised, Oracle has never thought much of open source.  As with Tim Bray and Ted Leung, their loss.  It's also a sign that a certain cultural style reflected in these folks is not a good match at Oracle.

Excellent online interactive CouchDB view tester

I had seen this before, but it took me a while to track it down again.  This page lets you interactively write map/reduce views against a CouchDB dataset and execute them to see the results.  It's not actually CouchDB, but it mimics it sufficiently to help you get an understanding of views and test various views to help you figure out how to get the results you want.  Very useful!

Twitter as a tool for getting (and providing) customer service

I have to say, one aspect of Twitter that has surprised me with its power is its ability to get attention from service providers.  I am always surprised and appreciative when someone replies to my tweet where I'm complaining about something.

When I am unhappy about something I do tend to gripe about it on Twitter.  This blog itself may arrive as a tweet because the folks at dlvr.it responded when I complained about Twitterfeed not working.

The folks over at Perforce have also been paying attention, although when I come back with a followup reply I tend to get re-routed to support, who re-routes me to my local contact, so oh well.

I also notice when people aren't listening.  For example, when I complained about Sharepoint user interface being, well, painful to say the least, I have to say I'm not expecting to hear from anyone with a followup.

I've even met some good folks this way.  I was whining about Sonic.net (I can't even remember what about), and ended up being followed by (and following) @dane, a dedicated employee at Sonic.net, and now I'm learning all sorts of cool stuff about what sonic.net is up to.  By the way, highly recommend those guys - excellent quality, great customer service, and they have some innovative services on the way (if I can trust Dane :)).

Zack Urlocker, predictive analytics, and R

I just noticed that MySQL's Zack Urlocker Joined the board for Revolution Board

REvolution? Who the heck are they?

Zack says "Predictive analytics is a multi-billion dollar market and it's ripe for disruption".

What the heck is predictive analytics?

Well, that's what Google is for. Turns out it has a lot of uses, and has been in use for a while.

http://en.wikipedia.org/wiki/Predictive_analytics

http://en.wikipedia.org/wiki/R_(programming_language)

http://www.revolution-computing.com/

http://blog.revolution-computing.com/

Well, very interesting. I am in no way a statistics or mathematics guy. But this does sound very compelling and powerful. Combine this with a big data set and you can do some interesting stuff -- for good or evil. Looking to see how Zack and REvolution disrupt this market - I would think by making it more approachable to a larger audience and finding new vertical applications for it.

Silliest 10 billionth (+1) tweet ever

Boy, talk about underscoring the complete vacuousness of a lot of Tweets, here is the 10 billionth (+1) tweet (the 10 billionth was on a private account).

http://twitter.com/SheSz_FUcINgBAD/status/10000000001

Thank goodness I can control who I listen to on Twitter, my goodness.

Nice article about DLP and Data Insight

A nice blog by Jerome Wendt about how tying in access history of a file with an ability to classify a file as sensitive is a big step to tracking down how data is leaking out of an organization.

I've seen demos of this technology (our team worked on it), and it's quite cool. A big problem our customers have is "OK, I've scanned all my data, I see there is sensitive data out there, now what?". Having a good sense of who is accessing the data most frequently, and therefore who is the likely "owner" of the data, is a huge step to helping answer this question.

Apache Cassandra

Meanwhile, I might share that I'm starting to see a serious pickup in Apache Cassandra, with Twitter being one of the latest to commit to using it big time.  It does look very promising - if it has backers like this, and the community is strong, we can expect great things of this project.

Trying to integrate my blog with Twitter and Facebook

I'm trying to use TwitterFeed, which automatically redirects blog postings to Facebook and Twitter.  We'll see how it goes.  What I'm hoping is this will increase my activity on my blog - it doesn't feel right to be so radio silent over here these days.

Now *that's* a meal

Via my cuz, supposedly taken by a road crew in Arizona. As my son would say - eeewwww (grin) !

Wherefore the blog posts?

I am surprised myself to find out how less often I'm blogging now that I'm a more active user of Twitter. I mean, why should that be - two very different mediums, and two very different purposes.

I think it must be a matter of focus and energy. When I have something I want to share, I go to the medium I am most active on/familiar with. Before Twitter (and long after Twitter was around) that was my blog.

But now I find I can share a lot of my thoughts quickly on Twitter. But I also miss the longer musings that I would do on my blogs.

I keep *wanting* to post more stuff on my blog, but it keeps not happening.

It could also be that I'm much busier. Life on NetBeans was a little low key - working from home, the team in Prague, and my feature low on the priority list. Not so at my new job - I've got tight deadlines, I'm the office each day, busy busy busy. So sitting down to write a blog post is often just not an option. I can barely squeeze in a tweet.

I know some of my Dear Readers are not Tweet-aholics and maybe even are avid Twee-totalers. So I did want to share one option - you can view my Twitter stream as an RSS feed. I only post a few things a day at most, so it's not like it will be overwhelming. In particular, this is where I post many of my links of things I have found interesting/compelling on the 'net.

Here's my Twitter RSS feed: http://twitter.com/statuses/user_timeline/15160781.rss

Cheers,

David

The future of UI will be boring

I'm not officially a UI guy, but I liked this blog post:

http://www.scottberkun.com/blog/2010/the-future-of-ui-will-be-boring/

The wiring that powers your home, the plumbing that brings you water, the roads you go to and from work on, work in mostly the same way they always have. This is ok. Lack of upgrade is not a sign of failure. Your genetic code is hundreds of thousands of years old, and seems to be working quite well if you’re reading this. It might just be more important to consider how the tech in question will make your dreams, or someone else’s, come true. If you worry more about the ends, rather than the means, revolution in UI is less important than you suspect it is. I mean, I’m a writer. If a quill pen was good enough for Shakespeare, what do I really have to complain about?

Continuous Deployment - releasing two-three times a day...

Very interesting case study on applying Lean principles to deployment. Serious elimination of waste.

http://www.startuplessonslearned.com/2010/01/case-study-continuous-deployment-makes.html

Continuous Deployment is Continuous Flow applied to software. The goal of both is to eliminate waste. The biggest waste in manufacturing is created from having to transport products from one place to another. The biggest waste in software is created from waiting for software as it moves from one state to another: Waiting to code, waiting to test, waiting to deploy. Reducing or eliminating these waits leads to faster iterations which is the key to success.

Neither Enterpriser or Hacker Be

I know of what Tim speaks around large, unwieldy Enterprise-level projects.  They are money pits and resource pits.  I suspect most of the engineers working on them aren't really having a good time.

But Tim seems to glom Web developers and agile methodologies into the same boat, and I think it's important to make a distinction.  There are strong agile-style developers and there are Hackers.  Hackers do follow iterative development and are focused on user experience, but often testing and reusable design go out the window, and monolithic spaghetti garbage heaps are often the result.

When I was interviewing during the Web 1.0 boom, a common job description was something like "Emergency Senior Engineer."  I would go into a shop that had hacked their way through the first few years and now they were seriously stuck, and were looking for someone with some grey hairs to help clean things up.

It's definitely very hard to get engineers with an Enterprise mentality to adopt an agile approach.  Enterprisers are very uncomfortable when they aren't writing out all the specs and plans up front - they are wanting to cover all the bases and get signoff before they start working - as Tim said "Enterprise IT has spent decades growing a defensive culture based on the premise that you only get noticed when you screw up, so that must be avoided at all costs."

But Hackers are very uncomfortable with having to write tests first, thinking about reusable design, spend time cleaning up code they touch, and so on.  Many times I've seen a roll of the eyes when I bring up these kinds of things.  I think this is because these guys love just whacking out code, are purely focused on technology, and don't have the patience or interest to think about maintainability.  Normally a few hellish releases where you can't get anything done because all you're doing is fixing bugs straightens out this attitude.  But by then it's often too late for the project at hand, without major rewrite.

That said, if I were to be forced to choose between an Enterprise culture or a Hacker culture, I'd probably choose the latter.  Enterprisers take years (and years) to get something to market, and it's usually not what anybody actually wants.  Hackers get hideously coded stuff out quickly, get feedback, and probably have money coming in when it comes time to rewrite.

Also, let's talk about culture.  Enterprise projects have this horrible stuffy dead feeling, like you're working on the 7 and 1/2th floor.  Everything's about specs and standards and ISO and most of the time you're in meetings and doing reviews.  Hackers are often a bit wacko and over-stimulated, but at least there's energy there.

Luckily, right now I'm on a team committed to agile practices, and it's definitely a lot of fun.  Our customers are pretty happy too.  And this, in an enterprise!  Maybe there's hope...