Wednesday, December 16, 2009

Steampunk LCD - classy and cool

At lunch yesterday a colleague from Sun introduced me to the term "steampunk."  Steam what?  Wikipedia showed me I was way behind the times.

I was thinking back to movies and books I've read, realizing they were "steampunk" and I didn't even know it, like The Golden Compass.




This same friend today pointed me to this awesome page where someone shows you step by step how they souped up their LCD monitor to be a steampunk monitor.  My goodness, look at this thing, that is just cool and classy.  That keyboard is pretty cool too, although I'm not sure it's practical (but probably good for preventing RSI).


Tuesday, December 08, 2009

What wakes men up, what wakes women up

From MindLab, by way of the New York Times. Note the complete lack of "baby crying" from the men's top ten list :)

Top 10 sounds most likely to wake women:

* Baby crying
* Dripping tap
* Rowdiness outside
* Snoring
* Buzzing fly
* Drilling/workmen
* Sirens
* Car alarm
* Howling wind
* Noise from drains

Top 10 sounds most likely to wake men:

* Car alarm
* Howling wind
* Buzzing fly
* Snoring
* Noise from drains
* Crickets chirping
* Sirens
* Clock ticking
* Drilling/workmen
* Dripping tap

Some things I would like to add to my list:

* The sound of a mouse scratching in the drywall right behind my bed
* My wife throwing a pillow on my head (long story)
* Too much heat and still air (sadly, what wakes my wife up is too much cold and a draft)
* My iPhone buzzing under my pillow (well, that's on purpose, that's my alarm)

Friday, December 04, 2009

Going Postel

In our code base here, we have a utility class called Observable that makes it easy to apply the observer pattern to a class.

I was using it, and in the process (having been bitten by this in NetBeans), I wrote a unit test to see what would happen if a user of this class added Observers and then dereferenced them without removing them. I used a form of memory leak testing described here.

Sure enough, there was a memory leak, because the underlying list of observers in the Observable class was still hanging on to the observers.

So I modified the class to use WeakReference and ReferenceQueue to detect when an observer was no longer strongly referenced, and then removed "stale" references from the observer list.

When I submitted the code for review, our technical director here, who had written the original Observable code, pushed back, saying it was adding unnecessary complexity to the class.

I argued that my experience with listeners/observers was that people often did not do the right thing and forgot to remove their observers, and Evil Things ensued.

He pointed me to this wonderful, hilarious, incredibly well-written article by Joel Spolsky about (among other things) Martian headsets, the problem of standards, and the disaster caused by Jon Postel's robustness principle ("“Be conservative in what you do, be liberal in what you accept from others.”) I highly recommend you read it, but in case you don't, the point was, this principle encouraged sloppy and incorrect web pages to proliferate, which has put us into a terrible compatibility mess today.

I argued that although this made a lot of sense, we weren't building a standard or pseudo-standard with lots of implementations, and look what Java did to ease programmer's lives by introducing garbage collection to the masses.

He made an interesting point that services should probably follow the robustness principle, but lower-level building blocks, where implementation is more transparent, should expect users to use them correctly and not adapt to incorrect use.

I finally relented because (a) he's the boss, and at some point just to make progress you let the boss make the call :) and (b) I had to admit that we had never actually encountered this problem after being in the field for many years, so perhaps I was inventing a dragon where none existed.

It still makes me nervous. I saw nasty problems with zombie observers in NetBeans. But I'm willing to give it a shot. In general I am happier with code that is simpler and cleaner and only complicate it if you have to. I think we'll just have to see how it goes.

I'm curious, what do you think?

Thursday, November 19, 2009

Applying OO principles to interpersonal relationships

I formally admit my geekiness by seeing the interesting similarities between good OO design and healthy interpersonal relationships.

Healthy boundaries
Objects need to collaborate and interact and depend on each other, but it's important to maintain healthy boundaries.  Other objects are not allowed to touch your private parts, and you get to say what is visible and what is not.

Multitasking
Objects must know how to handle multiple requests at once while maintaining their state.  


Grace Under Pressure
Well-designed objects need to be able to handle stress and overload.


Honoring your commitments
Objects provide public interfaces, which provide an agreement as to what they are willing to do.  You can not change your agreements without talking about it first.

Interdependence
No object is an island unto itself; all objects have dependencies on other objects.  But an object should not be overly dependent on others, as this means it becomes fragile to changes made in the other object.



However, it's important to recognize there are some OO principles that should not be applied to relationships.

Single Responsibility Principle - each object has one and only one responsibility in a system.  Shyeah, right.

Pluggability - you should be able to replace one object with another that plays the same role.  No thanks.



Friday, November 13, 2009

TDD taken to logical conclusion - write acceptance tests first

I'm reading an interesting book called Growing Object-Oriented Software Systems by Steve Freeman and Nat Price (based on @mfeathers recommendation).

For the most part the first two chapters are nothing new for me (I'm expecting that will change in subsequent chapters), but there was one thing that made me stop and step back.  To me TDD has been about unit tests, but they strongly recommend something that I had never though of before but which made a lot of sense.  Not only do you write tests first at the unit level, you also write your end-to-end acceptance test first.

This gave me pause.  I am so used to QA reactively writing acceptance tests either after or in parallel with feature development.  But the more I thought of it, the more it made sense.  It defines a contract in black-and-white, and it also helps you focus on building just as much as you need and no more.

Of course the acceptance tests will break, and then you work towards making it pass by writing unit tests that break and making those unit tests work.  This really makes sense, and I'm going to try this with my next project.  I talked with our QA engineer, and he loves the idea and wants to give it a go.

If it works, I'll talk to the rest of the team about it.  This is the kind of thing that turns standard development process on its head and has to be introduced very gently... :)

Wednesday, October 28, 2009

LinkedIn looking for top-notch database engineers

I recently talked with Jean-Luc Valliant, the CTO at LinkedIn, about a new project they're starting. Of course I was interested, but I'm here at Symantec now and jumping ship after just six months is just not how I do things.

But I did say I'd put the word out. They're looking for top-level database theorists - people who have a solid understanding of and have contributed to database research. They're also looking for people with real-world experience building and deploying high performance, highly scalable database solutions, dealing with things like storage, caching, sharding/clustering, messaging and so on.

If you're interested, contact Jean-Luc Valliant at LinkedIn. If you have trouble reaching him, let me know at david at vancouvering dot com and I'll connect you up.

Tuesday, October 27, 2009

H1N1 Un-scare

I really appreciated this newsletter from our pediatrician, a different perspective on it all, so I thought I'd pass it on...

By the way, Michael and I already got it, and it looks like Linda is getting it too.  Not fun, it hangs on for a while, but no worse than any normal flu.  It was weird having cold-like symptoms and also feeling a bit flu-ish.



Dear Families,

Don't panic. Breathe. Wait, don't breathe, you might catch a germ!  Stay home! Wear masks!  Wash your hands constantly!  Remove your children from all public activities!  Stay home! Build a flu bunker!

Well, you get the idea.  There has been a lot of media coverage of the flu this week.  Unfortunately, the media's job is to obtain viewers, not to give you balanced, non-sensational medical information.  We have been inundated with H1N1 flu questions this past week. Everyone seems much more nervous, due in large part to the escalating news coverage.  Everyone remembers the one exception covered on 60 minutes, but let's set some basic facts straight about this flu:

1.  It is milder than typical winter influenza.

2.  This is a very contagious strain, so it is likely that many members of our community will get it.  The incubation period is 3 days.  If you have no symptoms 5 days after exposure, you likely will not get the virus.

3. If you get the virus, 30% of you won't even get a fever and will think you have just a cold.  If you do get fever, it will typically be for 2-3 days.  A few children have had fever for as long as 5-7 days, but this is very unusual in our patients who are treating their flu with natural means.

4.  Most of the people who have gotten very ill have significant pre-existing medical conditions like chronic lung disease.  In fact, proportionally, more "healthy" people get very ill with regular winter flu than have gotten very ill with this flu.

5.  So far, our families that have had the flu have recovered quickly and easily, with little need for bloodwork, antibiotics, chest x-rays, etc.  Even children with a history of asthma or wheezing seem to be less likely to wheeze with this illness.  We are very happy overall with how our patients are weathering this storm. 

6.  If you have had a flu like illness (fever, muscle aches, headache the first day, with runny nose and cough symptoms) then you may have already had H1N1!  The CDC says unless you know for sure it was H1N1 you should still get the vaccine, but this is likely overkill and you probably don't need it.

7.  Initially we were quite concerned about the H1N1 vaccine because it was going to contain a controversial new ingredient that had not been used in regular vaccines in the U.S.  Luckily, the CDC changed their mind and opted not to use this new adjuvant, squalene.  As far as we know, the vaccine that will be supplied to us from the government will be manufactured in a manner similar to our usual preservative-free flu vaccine.  So, we feel more comfortable with the idea of our patients receiving this vaccine.

We will have a limited supply of preservative-free H1N1 vaccine.  We will prioritize vaccine for high-risk families such as those with pregnant women, or children of any age with a history of pneumonia or asthma.   Once these families are covered, we can open up our supplies to any families that desire the vaccine.  Adults should get their vaccine from their primary care provider.  Pregnant women should be sure to ask for the preservative free version as it does not contain mercury.

8.  Let us be clear, again, that we feel most people don't need this vaccine.  This is a mild flu and it may be better to get it now and have immunity for the next time it goes around, or next year's strain.

Tuesday, October 20, 2009

Online equation solver

I'm working with Retouched Bloom Filters (PDF), and am trying to figure out how to calculate the size of a filter given a desired false positive and false negative rate. I tried to do some of the math, but I'm just too rusty.

Then I remembered my old friend The Web, and sure enough a search for an equation solver uncovered this site. Very useful - enter in a linear equation, and the variable you want to solve for, et voila!

For example, from the paper above, the false negative rate is given by

fn = 1 - (1 - s/(p1 * m))^k

where fn is the false negative rate, s = the number of bits reset, p1 is the probability a given bit is set in the filter after all n elements have been added to it (about 1/2 for an optimized filter), m is the size of the filter, and k is the number of hash functions.

I wanted to know what s is, the number of bits to reset, if I know the other values. So I entered in the equation above to the equation solver, and like magic, I have

s = (1-(1-fn)^(1/k))*m*p1

Nice!

BTW, once I have the numbers worked out to my satisfaction, I'll post them here.

Saturday, October 17, 2009

More on blatant counterfeiting in Wall Street

A well written, if expletive, article by Matt Taibbi in the Rolling Stone.

This is again about naked short selling - shorting stock that you don't actually have, and someone else then selling this same stock, thus driving the stock down.  It turns out that buying and selling stuff you don't acdtually have is also done in commodities, mortgages, and bonds.
A paper presented at the American Bankruptcy Institute earlier this year reports that up to a third of all notes for mortgage-backed securities may have been "misplaced or lost" — meaning they're backed by IOUs instead of actual mortgages.
He can't get his hands on the smoking gun, but all arrows do seem to point towards Wall Street robbing us all blind, with wolves guarding the hen house in Washington.

This week I watched a homeless man standing with handcuffs while two policemen talked to him and searched his stuff.  Meanwhile there are these companies in the center of our financial world who are basically greedy schmucks laughing while they rob us for billions and billions of dollars.  Greed gone wild.
The nation's largest financial players are able to write the rules for own their businesses and brazenly steal billions under the noses of regulators, and nothing is done about it. A thing so fundamental to civilized society as the integrity of a stock, or a mortgage note, or even a U.S. Treasury bond, can no longer be protected, not even in a crisis, and a crime as vulgar and conspicuous as counterfeiting can take place on a systematic level for years without being stopped, even after it begins to affect the modern-day equivalents of the Rockefellers and the Carnegies. What 10 years ago was a cheap stock-fraud scheme for second-rate grifters in Brooklyn has become a major profit center for Wall Street. Our burglar class now rules the national economy. And no one is trying to stop them.
If history is any lesson, this will end, and end ugly.  It's like watching a tidal wave build up behind you, seeing the water all pull away and you wonder if anyone else is noticing.

Friday, October 16, 2009

Pitch for ReviewBoard for code reviews

Back in the nineties, code reviews were painful, horrible experiences. You would be handed a thick parchment of source output and line numbers, and you had to go into a meeting for two hours to review all the code. The waterfall method at work.

Things got a little better when people would quickly send out diffs or attach them to a bug, but not by much. In the open source world, the reviews would go on for days with twenty levels of indents on long long email threads. At the end I'd scratch my head and wonder where things stood.

In my new job here at Symantec, I was introduced to ReviewBoard, an open source tool for code reviews. Wow. What a difference a tool makes. You make a change, and then use a tool to submit it for review. It gets the diff and submits it to Review Board. You then go in and specify who the reviewers are, submit, and you're done.

As a reviewer, you can look at the diffs in context with the source, and then click on any line and you can add a comment. When you're done, submit, and an email is sent out. The change owner can then go to specific comments and reply - each comment can have its own dialog which is logged in the tool. All this is done with AJAX style interaction, which is sweet.

The change owner can submit a new patch for review, and if they use the same changeset id, the tool lets you as a reviewer see the diff between the original and the updated version, so you can focus on what's changed since the last review. When you're satisfied, you click on "ship it" and you're done.

Atlassian also has a code review tool but (a) it's not free and (b) it doesn't handle iterative reviews - where you address comments by a few small changes. Their tool doesn't let you see the differences between iterations in the review, and that's pretty crucial when you have a large patch and the new revision only has a small set of updates. Also, it's difficult to submit a change before it is committed. So I think overall ReviewBoard is the better tool.

Because of this tool, we can now have a policy that every change be submitted for review before it's committed. Of course we are flexible with this, but this is the overall policy. I don't think such a policy would be bearable or enforceable without a tool like ReviewBoard. Thanks, guys!

Calling out to hungry spirits

I was listening to Door of Faith by Krishna Das, and I just had to share this beautiful poem he sang in English. It really touched my heart. To me, this is the heart of God, which we all carry within us.
Calling out to hungry hearts
Everywhere through endless time
You who wander, you who thirst
I offer you this heart of mine

Calling out to hungry spirits
Everywhere through endless time
Calling out to hungry hearts
All lost and left behind

Gather round and share this meal
Your joy and your sorrow I make it mine.

Thursday, October 15, 2009

Short a stock, then bring the company down: sounds like a plan

This blog tells quite an incredible story about the CEO of Overstock.com, Patrick Byrne, bringing to light the strategy of some hedge funds to short a stock and then use various mechanisms to bring the company down, and how the media appears to be involved in a coverup.  I don't know how well sourced this material is, but it's a good read anyway, and to be honest, it doesn't surprise me at all.
A small group of powerful hedge fund managers stop at nothing to annihilate the companies they sell short. Their tactics include: blackmail, smear campaigns, espionage, fraud, harassment, extortion, bribery, rumor-mongering, sabotage, off-shore money laundering, political cronyism, frivolous lawsuits, witness tampering, biased financial research, false identities, bogus credit ratings, bribery, libelous blogs, bad science, forgery, wiretapping, counterfeiting, collusion, lying, cheating, threats and theft.
Sounds like the Wall Street I know and ... well, definitely not love.

Greed gone wild.

"When I despair, I remember that all through history the way of truth and love has always won. There have been tyrants and murderers and for a time they seem invincible, but in the end, they always fall.. think of it, always." - Mahatma Gandhi

Thursday, October 08, 2009

Answers to life's persistent questions

I was recently working on some algebraic problems when I was studying a Retouched Bloom Filter (PDF). I was trying to see if I could calculate how much smaller the Bloom Filter could be depending on how many bits I reset to 0.

I found myself stumped - it has just been too long since I did this stuff in college. My numbers came out all wrong.

So I found it quite appropriate when I found these real-life answers, copied from an email sent to me.






















Thursday, October 01, 2009

Concurrency.next and the suckiness of explicit concurrency management

Tim Bray has a nice blog (with lots of fun comments) about multicore chips, parallelization and concurrency, and how this may bring about a new set of popular languages. Nice analysis, most of it makes sense.

But what he said last struck me the most, because it's been bothering me for a while too and I thought I was a loner.
Assumption · I’m taking the following as an axiom: Exposing real pre-emptive threading with shared mutable data structures to application programmers is wrong. Once you get past Doug Lea, Brian Goetz, and a few people who write operating systems and database kernels for a living, it gets very hard to find humans who can actually reason about threads well enough to be usefully productive.

When I give talks about this stuff, I assert that threads are a recipe for deadlocks, race conditions, horrible non-reproducible bugs that take endless pain to find, and hard-to-diagnose performance problems. Nobody ever pushes back.

When I started working NetBeans, where you the coder are responsible for concurrency, rather than the app server container, I was stunned, literally stunned, at how ugly things could get so easily. I almost immediately introduced deadlocks, data corruptions, and other icky stuff, and found myself desperately reading the bibles on concurrency and thread safety.

But I also found myself shaking my head. Why are we all doing this as if it makes sense for programmers to have to worry about this. This level of complexity and shooting-yourself-in-the-footedness indicates to me something inherently wrong with the programming model. It's like a hotel asking arriving guests to coordinate with each other to assign themselves rooms.

I had heard about the asynchronous, immutable state, message passing model of Erlang and Scala, and these seemed to me what we were looking for. You write an actor, it does what it does, and doesn't share its state with anyone except through copies. Simple, nice, elegant, parallelizable.

It's very hard to move from one language to another, so I'm with Java for now, but I'm looking forward to an opportunity to try out Scala or Erlang for Real Work. I'm sure the opportunity will come soon enough.

It's kind of nice that the new chip architectures are forcing programmers to think about parallelism, and thus good parallel concurrency models, and thus looking at the asynchronous/message-passing architectures of Scala and Erlang. Maybe we'll get out of this synchronized-volatile-deadlock-semaphore-thread-pool-latch madness that exists today in Java.

Excellent article on doing UI right

This is a great article on Software CEO about a new McAfee product (yes, I know, Symantec's sworn enemy :)), how successful it was in terms of usability, and what went behind that.

Lots of great tips about how to build your engineering process around your users, from soup to nuts.  Hiring an outside design team to design your UI - what a concept! :)

Here is just one tip to whet your appetite:
UI tip #6: Let user demand defend against code creep.
" Our team strove to understand what is necessary versus what's desirable," Ries says. "Everything you add in to the product affects documentation, support, and code.

"With every feature, you have to make a design decision. We tried to make reasonable and good assumptions about setting limits; we actively eliminated options, and then validated those options with the users.

"We ran into lots of feature that are good ideas, but they were clipped until we can get enough evidence that there's enough demand. There are always next versions that we can use to include them."

Monday, September 28, 2009

Integrated vs. open: what's more "free"?

OK, first of all, don't get me wrong.  I am totally an open-source advocate.  I worked in open source a lot at Sun, and I make use of a lot of open source in my day job.

But I also have a Mac and an iPhone.  As a matter of fact, many developers in the open source community have Macs and iPhones.  But Apple is the King of Proprietary.  So what's going on?

I recently got a comment on my Facebook page when I mentioned I was trying to unlock my wife's iPhone, basically warning me to stay away from proprietary systems that try to lock you in.  Now, I don't believe phones should be locked, or that you should have to revert to a downloaded software tool that hacks into the phone to unlock it.  But this comment did get me thinking: why do I use the iPhone even though it's proprietary and does try to lock me in, as does my Mac?

The problem with Apple's stuff is, it's just so gawdawfully easy to use.  My wife observed the same thing I did three years ago: "hey, this stuff just works!".

Apple can do this because they're proprietary and closed - they can build a completely integrated system and not worry about having to build something that works when it's pieced together by different vendors with different configurations.  The can focus on their configuration and just make it work.

So, when it comes down to it, I'll take proprietary over open when it offers such incredibly obvious advantages as the Mac and iPhone do.  Have I sold myself to the Devil in a Black Turtleneck?

In the midst of those ruminations, Tim Bray tweeted a link to this article in the Economist, that compares Mac vs. PC with our banking and health care systems.
Like Microsoft operating systems, America's health-insurance system is incoherent, hard to understand, often dysfunctional and bloated by obsolete legacy systems. (Though unlike Windows machines, it's not cheap.) Different parts fail to operate properly with each other, and the whole thing is incomprehensible to most users, patients and doctors alike. But try to set up a central authority like MedPAC to make decisions about how to fix Medicare, or to mandate that policies cover a set of basic conditions, or to make end-of-life counseling available to seniors so they don't go through their final weeks in a blizzard of legal confusion—try to fix any of this stuff, and you'll be accused of "taking the control of health care out of the hands of patients and their doctors." This rhetoric is often driven by vested commercial interests. Medical-industry groups don't want a panel of experts making decisions about Medicare because it reduces their ability to buy concessions through congressional lobbying.

The same goes for the banking and credit-card industries, where small-print legal confusion is used as a tool to extract money from customers, and efforts to ban such practices are attacked as restrictions on consumer freedom. In health insurance or credit cards, freedom's just another word for not understanding what's in your contract. A perfect illustration, from Republican congressman Jeb Hensarling: "The ironically named Consumer Financial Protection Agency (CFPA) would have the power to strip from consumers their freedom of choice and restrict their credit opportunities in the midst of a financial recession—all in the name of 'consumer protection'. Positively Orwellian."

What's Orwellian is describing your credit-card company's ability to arbitrarily raise the interest it charges you on past debt to 35% as "freedom". More broadly, we need to move away from the Orwell "1984" paradigm. It was a brilliant description of the most important threats to freedom in the middle of the 20th century, but it no longer describes the most important threats to freedom today*. It was already clear how creaky the paradigm was in 1984, when the Mac ad came out; it's only gotten creakier over the past 25 years. Orwell didn't pay much attention to the problem of an oppressive blizzard of "choices" designed to take advantage of the consumer or citizen by manipulating asymmetries of information. But that is the way the American commercial and political landscape feels much of the time

Definitely something to think about. What is the true freedom here? And, in terms of health care and other social services, who is benefiting from this 'freedom'?

Friday, September 25, 2009

Why I didn't want to do Test-Drive Development - and why I like it now

I've been thinking about why I didn't do Test-Driven Development for so many years (in particular, writing tests first), and why others don't.

You can be shown lots of books. You can be told it's important. But you still don't do it.

I think it's about discomfort. You're being asked to change. For years, the way I coded was: I did some analysis, thought about the design, and then started coding the design as I saw it, either from my head or from some basic UML.

TDD doesn't let you do that. You have to (ick) write a test first. Why on Earth would I ever want to do that? I'm trying to implement my design, not muck around with tests! So as much as I appreciated from afar the values of TDD, I just never get around to doing it, because it's just not how I did things. It's almost like you're coding naked or with a keyboard with a different layout - it's awkward, uncomfortable, and just feels wrong.

So I came up with all sorts of justifications for not doing it - I don't have time; I'm already writing tests, writing them first seems like overkill; I'm already overwhelmed with things I need to deal with; people who that are just anal design-freaks, and so on. But I think ultimately it came down to - that's just not how I do things.

But recently I finally forced myself to start doing it. I slowly started becoming more comfortable with this approach, and started to see the benefits.

To me the key thing is this concept of letting the design emerge rather than trying to impose a design. It's similar to starting with use cases first, and regularly validating your product with the customer, rather than building a technology and hoping they'll come.

Once I got used to letting the design emerge, I started really enjoying TDD. It was an act of discovery as much as an act of creation. And I truly believe the best creations are done that way - as an act of joyous discovery.

And yes, by the way, you get much better code coverage when you do TDD, because you ensure that each "piece" you build is tested, rather than writing a bunch of code and then trying to figure out how to test it.

And yes, by the way, you get designs that are much less coupled and more cohesive.

But to me my favorite part of it is that joyous process of discovery.

Why make methods small and do one thing?

I've been reading the ObjectMentor blog, and it's got some fun stuff in it. The one I particularly liked is how do you know when a function does "just one thing" - the answer - extract methods until you can't extract any more.

Now, I know there is the feeling of - why should I waste all this time writing lots of tiny little functions? But after trying it for a bit, I'm convinced, for two main reasons (although I am sure there are more):

Readability
When you write small functions, each one is eminently readable.  You look at it, and in an instant you understand what it does and its intent.  You don't have to "decode" the method to understand the intent - it just tells you.  I think this is essential - when you understand a code's intent, you can also understand if it is doing what you want it to.

Discovering New Classes
When you extract functions, sometimes you run into this situation where you are passing umpteen (meaning more than two) parameters into the function.  To me, particularly when the list of parameters is long, this indicates the need to extract a class which encapsulates all this data you're passing around.

And I often find once I've extracted the class, it clarifies the design for me.  You have an idea of a design, but you should really let the design reveal itself to you, and this is one of the great ways I have found that happening.

Also, often once a class is extracted, I can see how it can be used elsewhere - I have a new unit of reusability.

Anyway, I recommend keeping an eye on the ObjectMentor blog, there's good stuff there.

Thursday, September 24, 2009

You should like what you do

I was sitting down for dinner last night, after having grilled some chicken kabobs.  I mentioned that I thought I had cooked them long enough but they seemed undercooked.

My 9-year old daughter, Ariel, after a pause, said "You should like what you do."  I looked at her, confused.  "You're always saying something bad about what you do.  I get tired of it.  And it makes me not want to eat what you make."

She really stopped me in my tracks.  She was right!  And I told her so, "you're absolutely right, Ariel."   The words "never apologize!" from Julia Child's book "My Life in France" came back to me.  She tells a story of making a horrific dish, but she and her guests just shouldered through the dish as best as they could, and Julia never apologized and the guests never complained. 

I was so touched by Ariel's sentiment: "you should like what you do."  It was an admonishment, yes, but more than that, an encouragement - I believe in you, and you should too.

I spent some time looking at myself, and saw that indeed, yes, I have an underlying idea of myself as "not good enough."  We could go back into my history to understand how this happened, but I'd rather work with the present moment.  How can I work with this now?

"You should like what you do."  I see this in two ways.  First of all, whatever I do, take pride in it.  Yes, it may not be "perfect" but all the same take pride in it.  It is a creation from the heart.  Secondly, I should like what I do, rather than wish I were doing something else.  This is the key to a happy life.  "It is not doing the thing we like to do, but liking the thing we have to do, that makes life blessed," said Goethe.

So, I'm not rash enough to believe I can change my underlying perception of myself in a flash, but it's something to practice and be aware of.  When I find myself being overly critical or unhappy with something I'm going or have done, I can remember, "you should like what you do" and be happy about it.  Always room for improvement, but it's great the way it is.  My cooking is great.  My code is great.  My pictures are great.  My words are great.  My nose is great.  I am great.

Friday, September 18, 2009

A balance between flux and immutability

A fun blog post by John Rose about the delicate balance between the world of flux (e.g. volatile memory) and our attempts to enforce immutability through functional languages like Erlang.  Immutability is great, is helps rationalize and simplify your logic, but it just doesn't play well with the notion that "you don't step into the same river twice."  His overall conclusion - work with a language that encourages immutability but allows for in-place mutation where it is needed.  You know a craftsman at work when he sees the issues of software from the perspective of ancient philosophy :)
But we know from common sense (that trusty yet abused helper) that we live somewhere between the fluxy world of Heraclitus and the timeless world of Parmenides. In the days of Plato, it was his student Aristotle who balanced the claims of the two accounts, picking a middle way between the two. In short, the formal ideas we perceive and know inhere (in a way real though bound) in the mutable, moving reality around us. Object identities (what he called "substances") are real, though temporal. If Aristotle had marketers like we do, he might have called his account a hybrid paradigm (more Greek, actually, though to him "hybrid" would mean "having hubris") of flux and form. I think it's likely the story will end similarly for us, in our professional difficulties with software, with some sort of middle way.

Just to speculate wildly, maybe those hybrid functional-and-object-oriented languages (like Scala and F#) can provide a sort of dictionary and grammar of useful terms and constructs. These would somehow describe computations as immutable in most places, and manageably mutable in those few aspects needed to model real-world transitions. (Or to implement localized compute engines, like a quick-sorter.

Thursday, September 17, 2009

Racism in America? Shocking!

What?!  Racism still exists in America?!  I am shocked!  And some of the media and politicians are trying to use this to get attention and money?!  Doubly shocking!

I like Obama's approach: we all know racism exists, let's move on and focus on doing our job: fixing health care, fixing the economy, dealing with global warming, and so on.  I suspect he's done that all his life.  Why waste your time putting energy and attention into something that is divisive and ultimately gets you nowhere.  I've never met someone who stopped being racist because someone said "you're a racist!" to them.

I suspect that underlying all of this is not racism per se but a fear of the unknown.  If we work together create a country where each individual feels safer - in terms of their job, their health, and their overall prospects in life - then there will be less hate-filled speech and actions.

Tuesday, September 15, 2009

Time out on timeouts

A very interesting (and somewhat tragic) article about the long-term damage caused by withholding attention and affection as a way to discipline children. 

The most common tool in this toolbox is "time-outs".  My wife and I just didn't feel comfortable with this - it felt humiliating and mean - and the research seems to be confirming it's not the best approach
The studies found that both positive and negative conditional parenting were harmful, but in slightly different ways. The positive kind sometimes succeeded in getting children to work harder on academic tasks, but at the cost of unhealthy feelings of “internal compulsion.” Negative conditional parenting didn’t even work in the short run; it just increased the teenagers’ negative feelings about their parents.

What these and other studies tell us, if we’re able to hear the news, is that praising children for doing something right isn’t a meaningful alternative to pulling back or punishing when they do something wrong. Both are examples of conditional parenting, and both are counterproductive.

...

In practice, according to an impressive collection of data by Dr. Deci and others, unconditional acceptance by parents as well as teachers should be accompanied by “autonomy support”: explaining reasons for requests, maximizing opportunities for the child to participate in making decisions, being encouraging without manipulating, and actively imagining how things look from the child’s point of view.

The last of these features is important with respect to unconditional parenting itself. Most of us would protest that of course we love our children without any strings attached. But what counts is how things look from the perspective of the children — whether they feel just as loved when they mess up or fall short

Monday, September 14, 2009

Monday, September 07, 2009

Color photographs of Russia from 100 years ago

Go look at these amazing photos on Newsweek.
Between 1907 and 1915, Prodkun-Gorskii rode through even restricted parts of revolutionary Russia in his specially fitted darkroom rail-car, shooting color photos ... by a method of his own invention
Really quite stunning. So used to seeing black-and-white photos of these times, and here it is, the turn of the century in living color.

Color photographs of Russia from 100 years ago

Go look at these amazing photos on Newsweek.
Between 1907 and 1915, Prodkun-Gorskii rode through even restricted parts of revolutionary Russia in his specially fitted darkroom rail-car, shooting color photos ... by a method of his own invention
Really quite stunning. So used to seeing black-and-white photos of these times, and here it is, the turn of the century in living color.

Monday, August 31, 2009

Blocking health care key to GOP's survival

A great article via Tim Bray on USA health care politics.
Michael F. Cannon, a pundit at the libertarian Cato Institute, has written a blog post that highlights the importance of what I believe will be one of the most important issues in play once Barack Obama assumes the U.S. presidency. "Blocking Obama's health plan," he writes, is "key to the GOP's survival."
Note this was written in December, just after Obama's election. Very prescient, and it explains a lot.

Britain serves as an important political lesson for strategists. After the Labor Party established the National Health Service after World War II, supposedly conservative workers and low-income people under religious and other influences who tended to support the Conservatives were much more likely to vote for the Labor Party when health care, social welfare, education and pro-working class policies were enacted by labor-supported governments. ...

Sunday, August 30, 2009

Whence Obesity?

I'd be interested in real invesigative research into the causes of obesity, but here are my suspicions: it's a combination of economics and modern life - but mostly economics.

Corporations selling food like money (it's a corporation's job to make money, as much as possible)
  • The more people eat, the more food they buy
  • Agribusinesses like this, it makes them money
  • Agribusinesses encourage you to eat more (and fight tooth and nail any government policies intended to get people to eat less - read Marion Nestle for some shocking data on this).
  • People who eat more get fat
Corporations selling food like high margins (make more money)
  • Corn is subsidized by the government
  • Thus corn is unnaturally cheap
  • Thus you can make a lot of money selling things made with corn - high margins
  • In particular, corn syrup - it's in everything. Particularly the cheap things.
  • Poor people buy stuff with lots of corn syrup because its sweet and cheap
  • Processed sugars, including corn syrup, makes you fat.

Maybe at some point the health care costs will change our policies, but I think agribusiness will be dragged into accepting this kicking and screaming. Health care costs are there, but they really don't affect their bottom line enough for them to want to change their behavior.

If I Were King, I would find a way to include the cost of disease based on overeating into the costs of foods, so there is a motivation by businesses not just to sell more food but also to keep people healthy. Also, we need to stop subsidizing large farmers and certain crops like corn. That's just incredibly counterproductive, and is based on very old rationales having to do with struggling family farmers, not agribusiness.

But I don't believe these kinds of changes can get very far as long as business runs government, because politicians need money. I would love to find a way for politicians to get their word out and get votes without the poisoning need for money. Solve that problem, and a whole slew of other problems solve themselves. I know it's not simple - even if the politicians themselves don't spend the money, "political action committees" and specially-funded "positioning ads" will do the job for them. I don't have good answer, but I do think this is a key problem.

P.S. At the Santa Cruz boardwalk they were selling (and people were in line for) deep-fried cheesecake and deep-fried Twinkies. OMG.

P.P.S. Just after I posted this I read Paul Krugman's editorial on health care:
We tend to think of the way things are now, with a huge army of lobbyists permanently camped in the corridors of power, with corporations prepared to unleash misleading ads and organize fake grass-roots protests against any legislation that threatens their bottom line, as the way it always was. But our corporate-cash-dominated system is a relatively recent creation, dating mainly from the late 1970s.

And now that this system exists, reform of any kind has become extremely difficult. That’s especially true for health care, where growing spending has made the vested interests far more powerful than they were in Nixon’s day. The health insurance industry, in particular, saw its premiums go from 1.5 percent of G.D.P. in 1970 to 5.5 percent in 2007, so that a once minor player has become a political behemoth, one that is currently spending $1.4 million a day lobbying Congress.

That spending fuels debates that otherwise seem incomprehensible. Why are “centrist” Democrats like Senator Kent Conrad of North Dakota so opposed to letting a public plan, in which Americans can buy their insurance directly from the government, compete with private insurers? Never mind their often incoherent arguments; what it comes down to is the money.

Friday, August 28, 2009

Things to See and Do In Berkeley

Chris Anderson from the CouchDB community just landed in Berkeley, astonishingly near my house.  I had a nice cuppa with him yesterday and realized there was a list of fun things to do, and particularly great eating places, in Berkeley.  So I sent him my list of the Best of the Best, and thought I'd post it here too:

Places to See
  • I mentioned Tilden Park and Lake Anza.  There is also a working farm where you can visit and feed the animals, an old-fashioned merry-go-round, and a real working scale-size steam train you can ride on.  
  • Walk in the North Berkeley Hills.  There's a book around that gives you a bunch of walks to take
  • Fourth Street in West Berkeley, a street of nice (albeit pricey) shops and restaurants just N of University
  • Solano Avenue, particularly at the Colusa intersection
  • Rockridge shopping area at the Rockridge BART station - a nice place to stroll, browse stores, have a cuppa.  Lots of nice restaraunts.
  • Ashby and College is also a very nice walking shopping area.
  • The Bone Store on Solano.  Just check it out, it's weird and interesting :)
  • Botanical Gardens up Strawberry Canyon behind the University
  • Berkeley Rose Garden is a pretty stroll, but dog must be on leash I think
  • Briones Park on the other side of the hill is a stunning set of rolling hills with trails, I think it's dog-friendly.
  • Swimming at the El Cerrito Pool.  It's a salt water pool, affordable, they have hours for "fun" and for laps.  There's also King Pool right up Hopkins from Monterey.
  • At some point you might want to stroll around Bay Street mall - an outdoor mall, much less offensive/oppressive than other malls.  They have an Apple Store! :)  And of course a big movieplex.

Restaurants:
  • Lalime's on Gilman - really nice "night out" California Cuisine
  • Rivoli on Solano - even nicer
  • Cesar - a great tapas bar on Shattuck
  • Cha Am on Shattuck - Thai food
  • Cha Ya on Shattuck - veggie sushi
  • Acme Bakery on Cedar and San Pablo - incredible, delicious fresh bread and pastries
  • Cafe Fanny on Cedar and San Pablo - really good, pricey French cafe
  • Vik's Chaat Cafe in West Berkeley (forget exact location) - excellent, cheap Indian "fast food".  Indians from all over the Bay Area know of this place and swear by it
  • Ajanta on Solano for nice Indian
  • A lot of people like Breads of India on Sacramento and Dwight.  I've never been there
  • Picante Mexican - wow, *excellent* Mexican food, very affordable.  We go there all the time.  6th street south of Gilman.
  • La Farine Bakery on Solano - chocolate croissants to die for
  • Barney's Hamburgers on Solano - fun, delicious gourmet burgers, veggie available
  • Jimmy Bean's on Gilman and 6th - excellent cafe, delicious sandwiches, fries, etc.
  • Zachary's Pizza on Solano - you *must* try their stuffed spinach/mushroom pizza.  Bay Area award winner for years
  • A lot of people like The Cheese Board pizza, I don't care for it
  • Gioia Pizzeria on Monterey and Hopkins - New Yawk pizza by the slice with fresh California ingredients. The real deal.  Another regular haunt for our family.
  • Loard's Ice cream in El Cerrito Plaza - not the best ambiance, but definitely the best (affordable) ice cream around
  • If you want good, expensive gourmet ice cream, there's a little place on 4th street (thou must check out 4th street, a fun walkable set of stores, 4th N of University)
  • Homemade Cafe on Dwight and Sacramento - nice breakfasts, although I haven't been there in a while
  • Cafe Mediterranee at Ashby and College - excellent Greek/Mediterranean food

Groceries/Markets
  • Wine store on Cedar and San Pablo - my wife swears by their help and selection
  • Berkeley Natural on Gilman
  • Monterey Market on Gilman and Hopkins - cheap, fresh, delicious, local produce
  • Farmer's Market Saturday mornings in downtown Berkeley between MLK and Milvia.  There are others around town during the week.
  • The Country Cheese Shop on Hopkins/Monterey.  While you're there, check out the fish market and the butcher (and of course Gioia Pizza and Monterey Market).  A real gem of a corner.

Enjoy!

Friday, August 21, 2009

What is "Erlang Style Concurrency"

In my spare time I continue to try to slowly educate myself about the new concurrent programming languages like Erlang and Scala.  I ran into this article by Ulf Wiger on What Is Erlang-Style Concurrency? which I thought was an excellent overview of what's there and why it's important.

I loved the bit about the value of process monitoring - the system itself takes care of watching processes and if one crashes restarts it.  Serious fault tolerance, embedded in the language runtime.  Cool.

Monday, August 10, 2009

SDD - Scenario Driven Design

The engineering leadership here at Symantec has a strong commitment to TDD - Test Driven Development.  I have been enjoying practicing this approach and experiencing its benefits.

Through this process I have also seen a similarity between TDD and an approach to design that I have applied, which I shall now moniker "Scenario-Based Design" or SDD.

One approach to design is to get a sense of the requirements from a functional spec, and then put together a design that looks like a good fit.  You come up with a set of initial abstractions and then go from there, using TDD to refactor as needed.

Although that approach works, you can commit yourself to a lot of rework if your initial design is off the mark.  You can still get to a good design, but it takes time, and who has time these days?

So I think it behooves to apply TDD principles to design. 

First of all, identify the scenarios you want to support.  Your functional spec should describe the scenarios as use cases.  Even if you're designing an API for a service rather than a user-visible feature, your functional spec should still describe use cases.

At the first draft, just write out all the use cases as one or two sentences, following the principle of "actor- initial condition - goal"  For example "A developer is using the IncidentManager and wants to find out how many incidents exist for a given resource" or "A system administrator has a system which is CPU bound and wants to find out which processes have the highest CPU usage."

Identify which higher-priority scenarios you think will have the biggest impact on design.  Normally this is fairly easy to determine.

Start with the first scenario, and come up with some abstractions that seem reasonable for that scenario.  You can do this with CRC cards or just on a whiteboard or a piece of paper.  Exercise the abstractions with the scenario, refine as needed.  You're looking for a high-level design, don't define the signature of each method; save that for your TDD.

Then go to the next scenario, and repeat.  Normally you'll find that you have to rejigger things a bit to address the new issue.

I recommend thinking about some of the more common error conditions, work through them too.

The value of this approach is similar to TDD - you design only what you need, and you do it based on real-world scenarios, not on your own concepts of what makes sense.  Personally it's quite fun to watch the design evolve on its own.  

This approach reminds me of stories of two great artists/designers in other fields.

When I was in Florence, I went to the museum that had the statue of Michelangelo's David, which is wonderful, but in the same building were a series of unfinished sculptures by Michelangelo.  It was immediately clear that the sculpture was "born" out of the stone - he wasn't forcing his own ideas onto the stone, but was letting the image evolve out of the nature of the stone itself .  When I read his biography later, my impressions were confirmed.  He says "Every block of stone has a statue inside it and it is the task of the sculptor to discover it."[1]

The other story (perhaps apocryphal) [2] is of a woman who was the architect for a huge college in England.  She had the entire campus built, but without any pathways.  When asked about this, she said, "wait until the first snow."  Then when the snow came, she took pictures of all the paths carved out in the snow by the students, and placed the pathways there.  Genius.


[1] http://www.brainyquote.com/quotes/quotes/m/michelange386296.html
[2] Steward Brand, How Buildings Learn

Monday, August 03, 2009

Lies, damn lies, and statistics - health care rescission

Great article by Taunter Media about the damn lies behind the quote from insurance exec Don Hamm that "Rescission is rare. It affects less than one-half of one percent of people we cover. Yet, it is one of many protections supporting the affordability and viability of individual health insurance in the United States under our current system."

When you really do the math, it's clear that that small percentage is actually a very very large percentage of people who have expensive illnesses.  Here's Taunter telling us the disturbing facts around this number:
If the top 5% is the absolute largest population for whom rescission would make sense, the probability of having your policy cancelled given that you have filed a claim is fully 10% (0.5% rescission/5.0% of the population). If you take the LA Times estimate that $300mm was saved by abrogating 20,000 policies in California ($15,000/policy), you are somewhere in the 15% zone, depending on the convexity of the top section of population. If, as I suspect, rescission is targeted toward the truly bankrupting cases – the top 1%, the folks with over $35,000 of annual claims who could never be profitable for the carrier – then the probability of having your policy torn up given a massively expensive condition is pushing 50%. One in two. You have three times better odds playing Russian Roulette.
If we suspect that the insurance industry targets rescission against those people who have expensive illnesses, then it's not an accident, it's very intentional, and basically tells you that you get to have insurance only until and when you really need it, then you're on your own. So we Mr. Hamm says health insurance is affordable, I can only assume he means for his company, not for those who are seriously sick.

But hey, let's let the free market decide who should be covered and how.  The government is only going to get in the way - of making money, that is... 

Saturday, July 18, 2009

Sexuality at camp: this is normal?

My nine year old daughter just came back from a week of sleep-away camp.  She came back swearing.  She told stories of camp counselors doing the bump-and-grind during the camp dance.  They watched sexy PG-13 movies on the bus to camp. As she talks to us about the little vignettes of her time there, there is overall a sense of a "party" atmosphere that is more appropriate for college kids than eight- and nine-year-olds.  I am not sure what I expected, but I didn't expect this. 

For years I've noticed that our culture has become more and more anesthetized to sexuality, and in particular I have noticed kids being exposed to stuff that just seems way too mature for them.  When I see little girls in princess outfits doing hip-grinding moves on the dance floor at dance recitals and Club Med for kids shows including scenes from Cabaret and other adult films, it just seems Wrong.  We don't do TV, but I catch it from time to time, and I can only imagine what it's like for a kid growing up watching what's out there. Somehow I am not surprised girls are reaching puberty earlier and earlier - being exposed to all this sexual energy has got to kick hormones into gear.

Is this really what we want for our kids?  Aren't we confusing them?  Are we doing right by them by providing absolutely no boundaries or guidance around this?  Does anyone even care any more?

I hear myself, and I wonder, jeez, am I just being a prude?  But I don't think so.  I think there is a time and place for everything.  Intuitively I sense that there is a sacredness to childhood, that you are in a certain place emotionally and otherwise, and that this is not the time to be exposed to sexuality.

One of the signs of a dynsfunctional family is kids being forced into adult roles - it is abusive and confusing for the child.  This overexposure to sexuality feels like an form of sexual abuse -- boundaries are being crossed that should not be crossed and leave kids feeling more "adult" than they should. 

What's upsetting is that many parents and adults in general don't seem to think much of it.  It's not that they have explicitly chosen to expose kids to sexuality, they just don't seem to think much about it one way or the other.  Whatever.

The sexual revolution of the sixties brought us a lot of great things.  I think it's healthy for adults to talk about and experiment with sexuality.  I think it's wonderful that people are learning not to be ashamed of their sexuality.  But let's keep it with adults.  I think the sexual revolution also has been misinterpreted by many into "anything goes" attitude, and I also think this is being taken advantage of by advertisers, musicians, and anyone else with something to sell to sell products through sex, and if it exposes kids to inappropriate sexuality, well, that's just collateral damage.  And over time it's just becoming more and more the norm.

I don't have an answer, sometimes you're just in a sea of culture that is moving in a certain direction, and you just have to manage as best you can.  But I do feel the need to say something, as in this case it feels like silence is a form of consent.  I do live in this culture, but this aspect of it, I don't agree with. 

And no, Ariel is not going back to that camp next year.

Thursday, July 09, 2009

James Strachan: Scala is the new Java

I've been hearing about Scala for a while now, and have been contemplating digging into it a bit, trying to write some code and get an idea of its strengths and weaknesses.

I think James Strachan's blog post just put me over the edge.  I'm not sure when I'll find the time, but learning Scala has just moved high up on my "things to do to expand my knowledge" list.

What particularly kicked me into high gear was that James is the author of Groovy, from all I've heard an excellent language in its own right.  In his post he says I can honestly say if someone had shown me the Programming in Scala book by by Martin Odersky, Lex Spoon & Bill Venners back in 2003 I'd probably have never created Groovy.

Wow.  That's some statement.

James' salient points:

  • Scala has type inference - and that's a good thing: "it makes code comprehension, navigation & documentation much simpler"
  • Scala has high order functions and closures "so you can write beautifully concise code"
  • Scala has mixins (called "traits") "so you don't have to muck about with AOP wackiness to get nice modular code"
James summarizes "the beauty of Scala soon becomes apparent; its simplified so many of the gremlins in the Java language, allows you to write very concise code describing the intent behind the code rather than the implementation cruft - together with providing a nice migration path to elegant functional programming which is awesome for building concurrent or distributed software"

Time to get Odersky's book and give it a spin. 


Monday, June 29, 2009

Shut up and write those tests

One of the things I love about my new job is that this team is committed to the craft of software development, and is introducing me to Deep Testing.  I was pointed to this excellent book, Working Effectively with Legacy Code by Michael Feathers, and to this amazing little toolkit called Mockito.


Working Effectively with Legacy Code assumes a Test-Driven Development attitude and philosophy, and then tells you how to go into a hairball of existing code and get it to be testable.  Chapter titles include "I Can't Get This Class into a Test Harness" and "I Don't  Understand the Code Well Enough to Change It," and includes techniques for breaking that bugaboo of unit testing, hard-coded dependencies, like "Extract and Override Method Call," "Introduce Instance Delegator" and "Subclass and Override Method."  Excellent stuff.

Mockito is a mocking framework that is just insanely easy to use.  Here is a piece of code I wrote where the class to test had a large number of dependencies, and I just mocked out all but the relevant ones.  This took me about ten minutes to write, and it was an excellent way to test impact of the changes I was making:

Checkpointable mockCheckpointable = mock(Checkpointable.class);
CrawlerContent mockCrawlerContent = mock(CrawlerContent.class);
List crawlerItems = new ArrayList();
crawlerItems.add(mockCrawlerContent);

ContentStream mockContentStream = mock(ContentStream.class);

when(mockCheckpointable.getItems()).thenReturn(crawlerItems);
when(_mockCrawlerContent.getContentStream()).thenReturn(mockContentStream);

_messageManager.sendCheckpoint(mockCheckpointable);

verify(_mockContentBuffer).handleContentFromCrawler(mockContentStream, mockCrawler);
verify(_mockWalkUpdater).handleCheckpoint(mockCheckpointable);

With tools like this in hand, plus an attitude of commitment to excellence, I now firmly believe you can wrestle your code to the ground and get it into a unit test framework and start improving not only the testability and quality but overall architecture of your code.  I don't care if your code is full of massive, crufty code you can't understand, with lots of accesses to the network and the database, you can write unit tests that test modules independently. 

So shut up and write those tests.  You have no excuses.

Friday, June 26, 2009

Tuesday, June 23, 2009

NYT: Saudi royal family lavishes $$$ on terrorist groups

Looks like the lawyers for the families of Sept 11 got tired of trying to get the justice system to hear them, so they took it to the press. They gave the New York Times a stack of damning evidence linking the House of Saud to Al Qaeda, Taliban, and others.  From the article:
A self-described Qaeda operative in Bosnia said in an interview with lawyers in the lawsuit that another charity largely controlled by members of the royal family, the Saudi High Commission for Aid to Bosnia, provided money and supplies to the terrorist group in the 1990s and hired militant operatives like himself.

Another witness in Afghanistan said in a sworn statement that in 1998 he had witnessed an emissary for a leading Saudi prince, Turki al-Faisal, hand a check for one billion Saudi riyals (now worth about $267 million) to a top Taliban leader.

And a confidential German intelligence report gave a line-by-line description of tens of millions of dollars in bank transfers, with dates and dollar amounts, made in the early 1990s by Prince Salman bin Abdul Aziz and other members of the Saudi royal family to another charity that was suspected of financing militants’ activities in Pakistan and Bosnia.
If this is true, this is a Big Deal.  I don't fully understand why the Saudis would want to support these organizations.  Maybe somebody more versed in the ways of the Middle East can explain it to me.

Medical records = transparency = trouble for medical industry

Via @timoreilly, an article in Technology Review gives its take on why the medical industry has been so resistant to going electronic. Surprise - it's all about the money...
Dangling $19 billion in front of a $2.4 trillion industry is not nearly enough to get it to reveal the financial secrets that electronic health records are likely to uncover--and upon which its huge profits depend. In those medical records lie the ugly truth about the business of medicine: sickness is profitable. The greater the number of treatments, procedures, and hospital stays, the larger the profit. There is little incentive for doctors and hospitals to identify or reduce wasteful spending in medicine.

Monday, June 22, 2009

Health care: you get what you pay for

Great article by Atul Gawande in the New Yorker where he does some serious snooping around as to why health care costs are skyrocketing.  His conclusion: it's how we pay the doctors.
Somewhere in the United States at this moment, a patient with chest pain, or a tumor, or a cough is seeing a doctor. And the damning question we have to ask is whether the doctor is set up to meet the needs of the patient, first and foremost, or to maximize revenue.

Friday, June 19, 2009

Iran: it wasn't supposed to happen this way

A great guest editorial in the New York Times  from a student in Iran. It starts slow, but has a wonderful finish, really gives you the sense that everybody is surprised by the sweep of events, even those participating directly in them...
Everyone watched everyone else and we wondered how all of this could be happening. Who were all of these people? Where did they come from? These were the same people we pass by unknowingly every day. We saw one another, it feels, for the first time. Now in the second week, we continue to look at one another as we walk together, in marches and in silent gatherings, toward our common goal of having our vote respected.

...

It wasn’t supposed to happen this way. Until last week, Mr. Moussavi was a nondescript, if competent, politician — as one of his campaign advisers put it to me, he was meant only to be an instrument for making Iran a tiny bit better, nothing more. Iranians knew that’s what they were getting when they cast their votes for him. Now, like us, Mr. Moussavi finds himself caught up in events that were unimaginable, each day’s march and protest more unthinkable than the one that came before.

Wednesday, June 17, 2009

The Guardian seems to have it right

While other news media seem to be flailing in their attempts to keep up with the speed and detail of information coming from social media, the Guardian in the UK has a live page (updated every minute) that seems to get the balance right: keep an eye on the stream of information, and use journalistic discipline to avoid rumors, innuendo and falsehoods.  As an example, Twitter is posting this picture, and The Guardian says
A picture claimed to be of today's opposition rally has been posted to Twitpic. It shows thousands of people marching on a flyover labelled the "Krimkhan bridge". The date of the picture is impossible to confirm, please let us know if you have more information.
 I think this is great, as it gives me a source of information I feel I can trust that is also up-to-date and using new media sources and not just the traditional reporting resources.

New York Times on social media as the only way to get the word out

A good article that seems to capture the power of Twitter and other social media tools to break down the walls put up by repressive regimes.

I've been reading the Twitter #iranelection stream, and a lot of it is chaotic, doubtful, over-enthusiastic, or silly.  But there are also gems in there, with links to articles and lots of pictures showing the emotion of the demonstrators and the violent repression that is going on.

It definitely gives a different more real and emotionally engaging view of what is unfolding from traditional media, with an article here or there, mostly talking about what they've heard is going on through Twitter, and only showing "sanctioned" pictures of pro-government rallies.

But to me what is most important is not what I can learn about what is going on, but what Iranians can communicate with each other even as the government tries to shut down communications.  I read in a NYT article that demonstrations are being organized through Twitter.  That was great news to hear.  And I pray for them, because the risk of serious injury or death is very real, and they are still moving forward.  Incredibly bravery and determination.

Thursday, June 11, 2009

Thinking about getting a gTLD - talk to Antony

Here's a nice interview with my brother, Antony Van Couvering, CEO of Minds+Machines, talking about gTLDs (generic Top-Level Domains), why you might want one, why they're useful, and giving you a good overview of what is involved.
For a brand owner, having your brand name as the destination for your customers on the web is ideal. How much money do major corporations spend on branding, and how effective is Internet visibility? To me, it’s a no-brainer to brand yourself instead of a domain extension like .COM which has nothing to do with your brand. For a city, it’s a tremendous aid to tourism and local business, with great possibilities for civic initiatives. Another key advantage, if you keep your TLD clean (in other words, take steps to discourage warehousing, non-working sites, and template sites), is that search engines will pay attention. For instance, .NYC will be about New York City in a meaningful way, and if Google wants to deliver relevant searches (which it does), then it will favor the .NYC TLD when someone types in “nyc” or “new york city” or “manhattan” or any other New York-related term.

Do we *really* want SQL in HTML5?

Jan Lehnardt's tweet pointed me to this ongoing conversation about the new SQL support in HTML5. Vladimir Vukicevic I think summarizes the issue quite well:
There's been a lot of interest around the Web Storage spec (formerly part of whatwg HTML5), which exposes a SQL database to web applications to use for data storage, both for online and offline use. It presents a simple API designed for executing SQL statements and reading result rows. But there's an interesting problem with this; unlike the rest of the HtML5, this section defines a core piece of functionality in terms of an undefined chunk referenced as "SQL".

The initial implementations of Web Storage are both based on SQLite, and expose the dialect of SQL understood by SQLite to web content. I'm actually a big fan of SQLite, and was one of the advocates for pulling it into the Gecko platform. However, SQLite implements a variant of SQL, with a number of deviations from other SQL engines, especially in terms of the types of data that can be placed in columns.

Web content that is created to use database storage with SQLite as the backing is unlikely to work with any other backend database. Similarly, if another database was chosen as a browser's backing implementation, web content that works with it is unlikely to work with anything else. This is a serious interop problem, the root of which is that there really isn't a useful core SQL standard. SQL92 is generally taken as a base, but is often extended or altered by implementations. Even beyond the parser issues (which could be resolved by defining a strict syntax to be used by Web Storage), the underlying implementation details will affect results.

So, the only option is for the Web Storage portion of the spec to state "do what SQLite does". This isn't specified in sufficient detail anywhere to be able to reimplement it from the documents, so it would be even worse — "do what this exact version of SQLite does", because there are no guarantees that SQLite won't make any incompatible changes. For example, a future SQLite 4 may introduce some changes or some new syntax which wouldn't be supported by earlier versions. Thus, it requires every single browser developer to accept SQLite as part of their
platform.

There then follows a lot of discussion about how to handle this, from creating an ORM layer to fully specifying the standard subset of SQL that will be supported in HTML5 (as if there were a valid standard). One person argues that we can simply refer to the ISO SQL standard and then describe extensions to that standard.

OK, well, I have worked with relational databases for over 20 years, and I have seen how completely stuck customers are with a given database vendor. I don't care how "standard" the standard is, it's merely an illusion. There is a reason Oracle can charge such huge sums of money and smile as customers fume. There is a reason Sybase is still in business. It is almost impossible to move a large database application from one vendor to another.

We all know how difficult it is to write code today that is compatible across all browsers. Imagine if each browser had its own SQL database that "implemented the standard." That's where we're heading, and it sure seems hellish to me. I guess if we forced every browser to use SQLite, we'd be OK, but that seems to placing an awful lot of reliance on a single technology. And as Vladimir points out, that doesn't solve issues of incompatibility across versions of SQLite.

It seems to me the requirements are to have rich storage, particularly for Javascript objects, and a way to query over this storage. I know saying is easier than doing, but shouldn't we be defining an API that defines JSON storage and a JSON query language? The Persevere folks have already defined a JSON query language, and they seem to have put a lot of thought into it. Perhaps that could be used as a start.

I can't say I have a solution, but I'm just raising a flag that creating a standard as important as HTML5 with a placeholder for "SQL" is an invitation for major incompatibilities which we'll be digging ourselves out of for years to come.

Wednesday, June 10, 2009

Michael sings the train blues

Michael got a little geetar for Christmas, and I was sitting here working yesterday when I stopped and listened to him singing along. I couldn't believe it - serious train blues. I got up and went into the living room and there he was sitting in the middle of his train tracks singing along, "Red Train Going Down the Tracks!".

Listen to him singing about the piston at the end. What to say, infinite cuteness.

Middle age: learning acceptance and contentment

To me a lot of what middle age is about - or perhaps life in general - is accepting your life as it is, and finding a place of contentment, an angle of repose.

We often have ideas about how our life should be going, but that's not the way it's going, and we fight and fight and fight. But at some point we are taken down, usually by some bodily failure (which is where middle age comes in) and we are forced to let go and accept. I suspect that this is the seed of wisdom.

This is also the message I see regularly in Indian scriptures. Krishna says "it is better to live your own dharma poorly than someone else's well". The great statements of Vedanta say "All this is God" and "Nothing but God exists." To me, one aspect of this teaching is that if I am seeing my life as "not good enough" or not where I want to be, then this implies I am not seeing God in my life.

Sometimes this is very hard to accept.

Over the last three weeks I went through a debilitating flu and had to go through a root canal (partially caused by grinding my teeth). I took these as signs of a life out of balance and did some self-inquiry.

First of all, I was exhausted, and secondly I was stressed. This much was obvious. Next question was why.

I have just started a new job after being laid off by Sun. I didn't really expect to make this change, and I don't think I had fully accepted the change. At Sun I was working with some serious rock stars of the industry, and doing some "cool stuff" in open source. In my new job, it's a great team and fun work, but I have been unhappy because (to me) it just wasn't that "cool."

I have also been watching all this interesting stuff going on in the web tier - distributed data stores like CouchDB, rich client development using Javascript/HTML5, Flex and JavaFX, potentials for peer-to-peer collaboration, etc. But none of this has anything to do with my day job.

So I started trying to do this stuff on the side. Keep in mind I'm a father of 2 young kids, and I also just took away two hours from my day for commute time. So I was squeeezing this into late nights and early mornings. Thus the exhaustion. And the grinding teeth: I was pushing and pushing myself, I didn't want to let it go and just do my day job - that would be admitting surrender.

But after these illnesses and further thought and I saw that (a) this wasn't a maintainable lifestyle and (b) I was being proud and egotistical. My day job was "below me" and I was meant for "greater things." Hints of Raskolnikov. I wasn't accepting my life as it is.

So with a sigh I saw it was time to let go. No more late nights trying to hack Javascript and CouchDB. Although part of me feels like I'm letting go the chance to ride this wave, I also have to admit I've been much happier and less stressed since this decision.

It's such a fascinating process. This wasn't the first time I've gone through it, and I doubt it's the last.

Great article on CSS development

I have done very very little CSS development, but I'm starting to dabble a bit.  @petr3 posted a link on Twitter to an excellent article by Jonathan Christopher that both lays out some nice principles on CSS design and provides some valuable links to earlier articles he wrote how to write better CSS and improve your CSS processes.

The principles he mentions reminds me a lot of software design in general:
  • find the patterns and reuse wherever possible
  • work from generic to specific
  • work to keep clarity as you maintain it
Definitely worth a read...

Friday, June 05, 2009

What a concept: a steady state economy

From the first time I heard of this idea that growth was essential to prosperity, it's had me shaking my head. I first heard it in terms of an individual company: a public company is always under pressure to grow, grow, grow. You can't just have one coffee shop, you have to have two. You can't just be a single company with a single focus, you have to conglomerate. Grow grow grow! It just seemed wrong for a company to be forced to grow when it could be doing perfectly well where it is. See the story of the Southern fisherman and the Northern industrialist.

Then I heard about this concept in terms of the economy in general.  I read how a steady state was considered the same as stagnation - like some horrible third circle of hell.  But you just do the math in your head, and you wonder where all this growth is going to take you.  Sure enough, it's not sustainable, and bubbles burst.  And I just sigh when I hear all these economists (who got us into this mess) trying to get us back to a trajectory of growth.  Can't we try something else for a change?

So, I thought I'd share with you this very insightful and entertaining article by Herman Daly, posted on the Oil Drum.  Herman Daly has fun skewering the growth economists (who I'm sure are ostracizing him as we speak) and then goes on to describe a vision of a healthy steady state economy.  You know, the old general store instead of Wal-Mart.

Here's one juicy skewer quote:
Some economists in fact think of nature as the set of extractive subsectors of the economy (forests, fisheries, mines, wells, pastures, and even agriculture….). The economy, not the ecosystem or biosphere, is seen as the whole; nature is a collection of parts. If the economy is the whole then it is not a part of any larger thing or system that might restrain its expansion. If some extractive natural subsector gets scarce we will just substitute other sectors for it and growth of the whole economy will continue, not into any restraining biospheric envelope, but into sidereal space presumably full of resource-bearing asteroids and friendly highly-evolved aliens eager to teach us how to grow forever into their territory. Sources and sinks are considered infinite.
Then Mr. Daly goes on to offer 10 proposals to help reach a healthy steady state economy.  I'm not sure I grok or even agree with all of them, but it's a starting point for a healthy dialog about this.  I recommend having a read...

Tuesday, June 02, 2009

JSR308 (pluggable type checking) is looking promising

Watching the Twitter stream for JavaOne, I saw a number of folks say they were quite impressed with the talk called "Preventing Bugs with Pluggable Type Checking."  I checked out the link for JSR308, and this looks very promising.

Basically, you can add annotations that check for things such as NonNull or Nullable, and then incorporate a checker that supports these annotations into the compiler.

For example, you can say
import checkers.nullness.quals.*;

public class GetStarted {
    void sample() {
        @NonNull Object ref = new Object();
    }
}
and then say
javac -processor checkers.nullness.NullnessChecker GetStarted.java
and the checker will ensure that the @NonNull object indeed is not being set to null anywhere.
Take a look at the page for JSR308, this looks like a good new addition.

Learn CouchDB through online experimentation - highly recommended

Wow, this is very nice.  It's an all JavaScript emulation of CouchDB.  What makes this super-cool is that this allows you to interactively play around with both CouchDB documents and views.  This is particularly useful when you're trying to understand how views work.  Definitely recommended if you're interested in learning CouchDB.

Friday, May 29, 2009

Excellent article describing how to work with CouchDB and jQuery in Firebug

A really nice step-by-step outline by Ben Atkin of how to use the Firebug interpreter to interactively work with CouchDB using jQuery's ajax() method.

Thanks to @nitin for this link, and thanks to Ben for taking the time to write it up! These simple tip blogs are one of the most valuable aspects of the social web for me.

Thursday, May 28, 2009

Twitter for communication in a disaster

I just sent this letter to the City of Berkeley disaster preparedness team.  You might want to consider doing the same with your local municipality...


Hello.  I am a homeowner here in Berkeley, and have been using Twitter for the past month or so, and I was impressed at how easily it is for me to stay up-to-date with things happening real-time.  For example, there is a conference going on in SFO, and I just subscribe to the search term for that conference and I see real-time updates from everyone who is at the conference and using that code.

Another example of course is that during the Mumbai terrorist attacks, everyone used the term "#mumbai" in their messages, and anyone subscribing to that term could see upates in real time.

One of the worst problems in a disaster is lack of information and nobody knowing what's going on or what they should be doing.  If a disaster occurs in Berkeley, it would be very nice to get real-time updates through Twitter.

In particular, I can see an "official" channel coming from your office and the City of Berkeley in general, with important information that you want to disseminate.

I can also see an "unofficial" channel where citizens can post messages, things like "fire on 4th and Hearst" or "looting going on near downtown BART, stay away" or "if anyone needs extra firewood I have some".

When I say a channel what I really mean is that when you post a message you use a certain tag like "#berk" or "#coberk" or something like that.   Then those of us trying to keep in touch can subscribe to this term and get real-time updates, and post our own updates as well.

If you set this up, and then let people know ahead of time to make use of these channels, I think this could be a very good thing.

You might want to also set up an agreement with local cellular providers that in the case of emergency everybody should have access to their data services even if they're not paying subscribers, so they can engage in communication through the Internet using their cell phones.

Thank you,

David