I recently talked with Jean-Luc Valliant, the CTO at LinkedIn, about a new project they're starting. Of course I was interested, but I'm here at Symantec now and jumping ship after just six months is just not how I do things.
But I did say I'd put the word out. They're looking for top-level database theorists - people who have a solid understanding of and have contributed to database research. They're also looking for people with real-world experience building and deploying high performance, highly scalable database solutions, dealing with things like storage, caching, sharding/clustering, messaging and so on.
If you're interested, contact Jean-Luc Valliant at LinkedIn. If you have trouble reaching him, let me know at david at vancouvering dot com and I'll connect you up.
Wednesday, October 28, 2009
Tuesday, October 27, 2009
H1N1 Un-scare
I really appreciated this newsletter from our pediatrician, a different perspective on it all, so I thought I'd pass it on...
By the way, Michael and I already got it, and it looks like Linda is getting it too. Not fun, it hangs on for a while, but no worse than any normal flu. It was weird having cold-like symptoms and also feeling a bit flu-ish.
Dear Families,
Don't panic. Breathe. Wait, don't breathe, you might catch a germ! Stay home! Wear masks! Wash your hands constantly! Remove your children from all public activities! Stay home! Build a flu bunker!
Well, you get the idea. There has been a lot of media coverage of the flu this week. Unfortunately, the media's job is to obtain viewers, not to give you balanced, non-sensational medical information. We have been inundated with H1N1 flu questions this past week. Everyone seems much more nervous, due in large part to the escalating news coverage. Everyone remembers the one exception covered on 60 minutes, but let's set some basic facts straight about this flu:
1. It is milder than typical winter influenza.
2. This is a very contagious strain, so it is likely that many members of our community will get it. The incubation period is 3 days. If you have no symptoms 5 days after exposure, you likely will not get the virus.
3. If you get the virus, 30% of you won't even get a fever and will think you have just a cold. If you do get fever, it will typically be for 2-3 days. A few children have had fever for as long as 5-7 days, but this is very unusual in our patients who are treating their flu with natural means.
4. Most of the people who have gotten very ill have significant pre-existing medical conditions like chronic lung disease. In fact, proportionally, more "healthy" people get very ill with regular winter flu than have gotten very ill with this flu.
5. So far, our families that have had the flu have recovered quickly and easily, with little need for bloodwork, antibiotics, chest x-rays, etc. Even children with a history of asthma or wheezing seem to be less likely to wheeze with this illness. We are very happy overall with how our patients are weathering this storm.
6. If you have had a flu like illness (fever, muscle aches, headache the first day, with runny nose and cough symptoms) then you may have already had H1N1! The CDC says unless you know for sure it was H1N1 you should still get the vaccine, but this is likely overkill and you probably don't need it.
7. Initially we were quite concerned about the H1N1 vaccine because it was going to contain a controversial new ingredient that had not been used in regular vaccines in the U.S. Luckily, the CDC changed their mind and opted not to use this new adjuvant, squalene. As far as we know, the vaccine that will be supplied to us from the government will be manufactured in a manner similar to our usual preservative-free flu vaccine. So, we feel more comfortable with the idea of our patients receiving this vaccine.
We will have a limited supply of preservative-free H1N1 vaccine. We will prioritize vaccine for high-risk families such as those with pregnant women, or children of any age with a history of pneumonia or asthma. Once these families are covered, we can open up our supplies to any families that desire the vaccine. Adults should get their vaccine from their primary care provider. Pregnant women should be sure to ask for the preservative free version as it does not contain mercury.
8. Let us be clear, again, that we feel most people don't need this vaccine. This is a mild flu and it may be better to get it now and have immunity for the next time it goes around, or next year's strain.
By the way, Michael and I already got it, and it looks like Linda is getting it too. Not fun, it hangs on for a while, but no worse than any normal flu. It was weird having cold-like symptoms and also feeling a bit flu-ish.
Dear Families,
Don't panic. Breathe. Wait, don't breathe, you might catch a germ! Stay home! Wear masks! Wash your hands constantly! Remove your children from all public activities! Stay home! Build a flu bunker!
Well, you get the idea. There has been a lot of media coverage of the flu this week. Unfortunately, the media's job is to obtain viewers, not to give you balanced, non-sensational medical information. We have been inundated with H1N1 flu questions this past week. Everyone seems much more nervous, due in large part to the escalating news coverage. Everyone remembers the one exception covered on 60 minutes, but let's set some basic facts straight about this flu:
1. It is milder than typical winter influenza.
2. This is a very contagious strain, so it is likely that many members of our community will get it. The incubation period is 3 days. If you have no symptoms 5 days after exposure, you likely will not get the virus.
3. If you get the virus, 30% of you won't even get a fever and will think you have just a cold. If you do get fever, it will typically be for 2-3 days. A few children have had fever for as long as 5-7 days, but this is very unusual in our patients who are treating their flu with natural means.
4. Most of the people who have gotten very ill have significant pre-existing medical conditions like chronic lung disease. In fact, proportionally, more "healthy" people get very ill with regular winter flu than have gotten very ill with this flu.
5. So far, our families that have had the flu have recovered quickly and easily, with little need for bloodwork, antibiotics, chest x-rays, etc. Even children with a history of asthma or wheezing seem to be less likely to wheeze with this illness. We are very happy overall with how our patients are weathering this storm.
6. If you have had a flu like illness (fever, muscle aches, headache the first day, with runny nose and cough symptoms) then you may have already had H1N1! The CDC says unless you know for sure it was H1N1 you should still get the vaccine, but this is likely overkill and you probably don't need it.
7. Initially we were quite concerned about the H1N1 vaccine because it was going to contain a controversial new ingredient that had not been used in regular vaccines in the U.S. Luckily, the CDC changed their mind and opted not to use this new adjuvant, squalene. As far as we know, the vaccine that will be supplied to us from the government will be manufactured in a manner similar to our usual preservative-free flu vaccine. So, we feel more comfortable with the idea of our patients receiving this vaccine.
We will have a limited supply of preservative-free H1N1 vaccine. We will prioritize vaccine for high-risk families such as those with pregnant women, or children of any age with a history of pneumonia or asthma. Once these families are covered, we can open up our supplies to any families that desire the vaccine. Adults should get their vaccine from their primary care provider. Pregnant women should be sure to ask for the preservative free version as it does not contain mercury.
8. Let us be clear, again, that we feel most people don't need this vaccine. This is a mild flu and it may be better to get it now and have immunity for the next time it goes around, or next year's strain.
Tuesday, October 20, 2009
Online equation solver
I'm working with Retouched Bloom Filters (PDF), and am trying to figure out how to calculate the size of a filter given a desired false positive and false negative rate. I tried to do some of the math, but I'm just too rusty.
Then I remembered my old friend The Web, and sure enough a search for an equation solver uncovered this site. Very useful - enter in a linear equation, and the variable you want to solve for, et voila!
For example, from the paper above, the false negative rate is given by
fn = 1 - (1 - s/(p1 * m))^k
where fn is the false negative rate, s = the number of bits reset, p1 is the probability a given bit is set in the filter after all n elements have been added to it (about 1/2 for an optimized filter), m is the size of the filter, and k is the number of hash functions.
Then I remembered my old friend The Web, and sure enough a search for an equation solver uncovered this site. Very useful - enter in a linear equation, and the variable you want to solve for, et voila!
For example, from the paper above, the false negative rate is given by
fn = 1 - (1 - s/(p1 * m))^k
where fn is the false negative rate, s = the number of bits reset, p1 is the probability a given bit is set in the filter after all n elements have been added to it (about 1/2 for an optimized filter), m is the size of the filter, and k is the number of hash functions.
I wanted to know what s is, the number of bits to reset, if I know the other values. So I entered in the equation above to the equation solver, and like magic, I have
s = (1-(1-fn)^(1/k))*m*p1
Nice!
s = (1-(1-fn)^(1/k))*m*p1
Nice!
BTW, once I have the numbers worked out to my satisfaction, I'll post them here.
Saturday, October 17, 2009
More on blatant counterfeiting in Wall Street
A well written, if expletive, article by Matt Taibbi in the Rolling Stone.
This is again about naked short selling - shorting stock that you don't actually have, and someone else then selling this same stock, thus driving the stock down. It turns out that buying and selling stuff you don't acdtually have is also done in commodities, mortgages, and bonds.
This week I watched a homeless man standing with handcuffs while two policemen talked to him and searched his stuff. Meanwhile there are these companies in the center of our financial world who are basically greedy schmucks laughing while they rob us for billions and billions of dollars. Greed gone wild.
This is again about naked short selling - shorting stock that you don't actually have, and someone else then selling this same stock, thus driving the stock down. It turns out that buying and selling stuff you don't acdtually have is also done in commodities, mortgages, and bonds.
A paper presented at the American Bankruptcy Institute earlier this year reports that up to a third of all notes for mortgage-backed securities may have been "misplaced or lost" — meaning they're backed by IOUs instead of actual mortgages.He can't get his hands on the smoking gun, but all arrows do seem to point towards Wall Street robbing us all blind, with wolves guarding the hen house in Washington.
This week I watched a homeless man standing with handcuffs while two policemen talked to him and searched his stuff. Meanwhile there are these companies in the center of our financial world who are basically greedy schmucks laughing while they rob us for billions and billions of dollars. Greed gone wild.
The nation's largest financial players are able to write the rules for own their businesses and brazenly steal billions under the noses of regulators, and nothing is done about it. A thing so fundamental to civilized society as the integrity of a stock, or a mortgage note, or even a U.S. Treasury bond, can no longer be protected, not even in a crisis, and a crime as vulgar and conspicuous as counterfeiting can take place on a systematic level for years without being stopped, even after it begins to affect the modern-day equivalents of the Rockefellers and the Carnegies. What 10 years ago was a cheap stock-fraud scheme for second-rate grifters in Brooklyn has become a major profit center for Wall Street. Our burglar class now rules the national economy. And no one is trying to stop them.If history is any lesson, this will end, and end ugly. It's like watching a tidal wave build up behind you, seeing the water all pull away and you wonder if anyone else is noticing.
Friday, October 16, 2009
Pitch for ReviewBoard for code reviews
Back in the nineties, code reviews were painful, horrible experiences. You would be handed a thick parchment of source output and line numbers, and you had to go into a meeting for two hours to review all the code. The waterfall method at work.
Things got a little better when people would quickly send out diffs or attach them to a bug, but not by much. In the open source world, the reviews would go on for days with twenty levels of indents on long long email threads. At the end I'd scratch my head and wonder where things stood.
In my new job here at Symantec, I was introduced to ReviewBoard, an open source tool for code reviews. Wow. What a difference a tool makes. You make a change, and then use a tool to submit it for review. It gets the diff and submits it to Review Board. You then go in and specify who the reviewers are, submit, and you're done.
As a reviewer, you can look at the diffs in context with the source, and then click on any line and you can add a comment. When you're done, submit, and an email is sent out. The change owner can then go to specific comments and reply - each comment can have its own dialog which is logged in the tool. All this is done with AJAX style interaction, which is sweet.
The change owner can submit a new patch for review, and if they use the same changeset id, the tool lets you as a reviewer see the diff between the original and the updated version, so you can focus on what's changed since the last review. When you're satisfied, you click on "ship it" and you're done.
Atlassian also has a code review tool but (a) it's not free and (b) it doesn't handle iterative reviews - where you address comments by a few small changes. Their tool doesn't let you see the differences between iterations in the review, and that's pretty crucial when you have a large patch and the new revision only has a small set of updates. Also, it's difficult to submit a change before it is committed. So I think overall ReviewBoard is the better tool.
Because of this tool, we can now have a policy that every change be submitted for review before it's committed. Of course we are flexible with this, but this is the overall policy. I don't think such a policy would be bearable or enforceable without a tool like ReviewBoard. Thanks, guys!
Things got a little better when people would quickly send out diffs or attach them to a bug, but not by much. In the open source world, the reviews would go on for days with twenty levels of indents on long long email threads. At the end I'd scratch my head and wonder where things stood.
In my new job here at Symantec, I was introduced to ReviewBoard, an open source tool for code reviews. Wow. What a difference a tool makes. You make a change, and then use a tool to submit it for review. It gets the diff and submits it to Review Board. You then go in and specify who the reviewers are, submit, and you're done.
As a reviewer, you can look at the diffs in context with the source, and then click on any line and you can add a comment. When you're done, submit, and an email is sent out. The change owner can then go to specific comments and reply - each comment can have its own dialog which is logged in the tool. All this is done with AJAX style interaction, which is sweet.
The change owner can submit a new patch for review, and if they use the same changeset id, the tool lets you as a reviewer see the diff between the original and the updated version, so you can focus on what's changed since the last review. When you're satisfied, you click on "ship it" and you're done.
Atlassian also has a code review tool but (a) it's not free and (b) it doesn't handle iterative reviews - where you address comments by a few small changes. Their tool doesn't let you see the differences between iterations in the review, and that's pretty crucial when you have a large patch and the new revision only has a small set of updates. Also, it's difficult to submit a change before it is committed. So I think overall ReviewBoard is the better tool.
Because of this tool, we can now have a policy that every change be submitted for review before it's committed. Of course we are flexible with this, but this is the overall policy. I don't think such a policy would be bearable or enforceable without a tool like ReviewBoard. Thanks, guys!
Calling out to hungry spirits
I was listening to Door of Faith by Krishna Das, and I just had to share this beautiful poem he sang in English. It really touched my heart. To me, this is the heart of God, which we all carry within us.
Calling out to hungry hearts
Everywhere through endless time
You who wander, you who thirst
I offer you this heart of mine
Calling out to hungry spirits
Everywhere through endless time
Calling out to hungry hearts
All lost and left behind
Gather round and share this meal
Your joy and your sorrow I make it mine.
Thursday, October 15, 2009
Short a stock, then bring the company down: sounds like a plan
This blog tells quite an incredible story about the CEO of Overstock.com, Patrick Byrne, bringing to light the strategy of some hedge funds to short a stock and then use various mechanisms to bring the company down, and how the media appears to be involved in a coverup. I don't know how well sourced this material is, but it's a good read anyway, and to be honest, it doesn't surprise me at all.
Greed gone wild.
"When I despair, I remember that all through history the way of truth and love has always won. There have been tyrants and murderers and for a time they seem invincible, but in the end, they always fall.. think of it, always." - Mahatma Gandhi
A small group of powerful hedge fund managers stop at nothing to annihilate the companies they sell short. Their tactics include: blackmail, smear campaigns, espionage, fraud, harassment, extortion, bribery, rumor-mongering, sabotage, off-shore money laundering, political cronyism, frivolous lawsuits, witness tampering, biased financial research, false identities, bogus credit ratings, bribery, libelous blogs, bad science, forgery, wiretapping, counterfeiting, collusion, lying, cheating, threats and theft.Sounds like the Wall Street I know and ... well, definitely not love.
Greed gone wild.
"When I despair, I remember that all through history the way of truth and love has always won. There have been tyrants and murderers and for a time they seem invincible, but in the end, they always fall.. think of it, always." - Mahatma Gandhi
Thursday, October 08, 2009
Answers to life's persistent questions
I was recently working on some algebraic problems when I was studying a Retouched Bloom Filter (PDF). I was trying to see if I could calculate how much smaller the Bloom Filter could be depending on how many bits I reset to 0.
I found myself stumped - it has just been too long since I did this stuff in college. My numbers came out all wrong.
So I found it quite appropriate when I found these real-life answers, copied from an email sent to me.
I found myself stumped - it has just been too long since I did this stuff in college. My numbers came out all wrong.
So I found it quite appropriate when I found these real-life answers, copied from an email sent to me.
Thursday, October 01, 2009
Concurrency.next and the suckiness of explicit concurrency management
Tim Bray has a nice blog (with lots of fun comments) about multicore chips, parallelization and concurrency, and how this may bring about a new set of popular languages. Nice analysis, most of it makes sense.
But what he said last struck me the most, because it's been bothering me for a while too and I thought I was a loner.
When I started working NetBeans, where you the coder are responsible for concurrency, rather than the app server container, I was stunned, literally stunned, at how ugly things could get so easily. I almost immediately introduced deadlocks, data corruptions, and other icky stuff, and found myself desperately reading the bibles on concurrency and thread safety.
But I also found myself shaking my head. Why are we all doing this as if it makes sense for programmers to have to worry about this. This level of complexity and shooting-yourself-in-the-footedness indicates to me something inherently wrong with the programming model. It's like a hotel asking arriving guests to coordinate with each other to assign themselves rooms.
I had heard about the asynchronous, immutable state, message passing model of Erlang and Scala, and these seemed to me what we were looking for. You write an actor, it does what it does, and doesn't share its state with anyone except through copies. Simple, nice, elegant, parallelizable.
It's very hard to move from one language to another, so I'm with Java for now, but I'm looking forward to an opportunity to try out Scala or Erlang for Real Work. I'm sure the opportunity will come soon enough.
It's kind of nice that the new chip architectures are forcing programmers to think about parallelism, and thus good parallel concurrency models, and thus looking at the asynchronous/message-passing architectures of Scala and Erlang. Maybe we'll get out of this synchronized-volatile-deadlock-semaphore-thread-pool-latch madness that exists today in Java.
But what he said last struck me the most, because it's been bothering me for a while too and I thought I was a loner.
Assumption · I’m taking the following as an axiom: Exposing real pre-emptive threading with shared mutable data structures to application programmers is wrong. Once you get past Doug Lea, Brian Goetz, and a few people who write operating systems and database kernels for a living, it gets very hard to find humans who can actually reason about threads well enough to be usefully productive.
When I give talks about this stuff, I assert that threads are a recipe for deadlocks, race conditions, horrible non-reproducible bugs that take endless pain to find, and hard-to-diagnose performance problems. Nobody ever pushes back.
When I started working NetBeans, where you the coder are responsible for concurrency, rather than the app server container, I was stunned, literally stunned, at how ugly things could get so easily. I almost immediately introduced deadlocks, data corruptions, and other icky stuff, and found myself desperately reading the bibles on concurrency and thread safety.
But I also found myself shaking my head. Why are we all doing this as if it makes sense for programmers to have to worry about this. This level of complexity and shooting-yourself-in-the-footedness indicates to me something inherently wrong with the programming model. It's like a hotel asking arriving guests to coordinate with each other to assign themselves rooms.
I had heard about the asynchronous, immutable state, message passing model of Erlang and Scala, and these seemed to me what we were looking for. You write an actor, it does what it does, and doesn't share its state with anyone except through copies. Simple, nice, elegant, parallelizable.
It's very hard to move from one language to another, so I'm with Java for now, but I'm looking forward to an opportunity to try out Scala or Erlang for Real Work. I'm sure the opportunity will come soon enough.
It's kind of nice that the new chip architectures are forcing programmers to think about parallelism, and thus good parallel concurrency models, and thus looking at the asynchronous/message-passing architectures of Scala and Erlang. Maybe we'll get out of this synchronized-volatile-deadlock-semaphore-thread-pool-latch madness that exists today in Java.
Excellent article on doing UI right
This is a great article on Software CEO about a new McAfee product (yes, I know, Symantec's sworn enemy :)), how successful it was in terms of usability, and what went behind that.
Lots of great tips about how to build your engineering process around your users, from soup to nuts. Hiring an outside design team to design your UI - what a concept! :)
Here is just one tip to whet your appetite:
Lots of great tips about how to build your engineering process around your users, from soup to nuts. Hiring an outside design team to design your UI - what a concept! :)
Here is just one tip to whet your appetite:
UI tip #6: Let user demand defend against code creep.
" Our team strove to understand what is necessary versus what's desirable," Ries says. "Everything you add in to the product affects documentation, support, and code.
"With every feature, you have to make a design decision. We tried to make reasonable and good assumptions about setting limits; we actively eliminated options, and then validated those options with the users.
"We ran into lots of feature that are good ideas, but they were clipped until we can get enough evidence that there's enough demand. There are always next versions that we can use to include them."
Monday, September 28, 2009
Integrated vs. open: what's more "free"?
OK, first of all, don't get me wrong. I am totally an open-source advocate. I worked in open source a lot at Sun, and I make use of a lot of open source in my day job.
But I also have a Mac and an iPhone. As a matter of fact, many developers in the open source community have Macs and iPhones. But Apple is the King of Proprietary. So what's going on?
I recently got a comment on my Facebook page when I mentioned I was trying to unlock my wife's iPhone, basically warning me to stay away from proprietary systems that try to lock you in. Now, I don't believe phones should be locked, or that you should have to revert to a downloaded software tool that hacks into the phone to unlock it. But this comment did get me thinking: why do I use the iPhone even though it's proprietary and does try to lock me in, as does my Mac?
The problem with Apple's stuff is, it's just so gawdawfully easy to use. My wife observed the same thing I did three years ago: "hey, this stuff just works!".
Apple can do this because they're proprietary and closed - they can build a completely integrated system and not worry about having to build something that works when it's pieced together by different vendors with different configurations. The can focus on their configuration and just make it work.
So, when it comes down to it, I'll take proprietary over open when it offers such incredibly obvious advantages as the Mac and iPhone do. Have I sold myself to the Devil in a Black Turtleneck?
In the midst of those ruminations, Tim Bray tweeted a link to this article in the Economist, that compares Mac vs. PC with our banking and health care systems.
Definitely something to think about. What is the true freedom here? And, in terms of health care and other social services, who is benefiting from this 'freedom'?
But I also have a Mac and an iPhone. As a matter of fact, many developers in the open source community have Macs and iPhones. But Apple is the King of Proprietary. So what's going on?
I recently got a comment on my Facebook page when I mentioned I was trying to unlock my wife's iPhone, basically warning me to stay away from proprietary systems that try to lock you in. Now, I don't believe phones should be locked, or that you should have to revert to a downloaded software tool that hacks into the phone to unlock it. But this comment did get me thinking: why do I use the iPhone even though it's proprietary and does try to lock me in, as does my Mac?
The problem with Apple's stuff is, it's just so gawdawfully easy to use. My wife observed the same thing I did three years ago: "hey, this stuff just works!".
Apple can do this because they're proprietary and closed - they can build a completely integrated system and not worry about having to build something that works when it's pieced together by different vendors with different configurations. The can focus on their configuration and just make it work.
So, when it comes down to it, I'll take proprietary over open when it offers such incredibly obvious advantages as the Mac and iPhone do. Have I sold myself to the Devil in a Black Turtleneck?
In the midst of those ruminations, Tim Bray tweeted a link to this article in the Economist, that compares Mac vs. PC with our banking and health care systems.
Like Microsoft operating systems, America's health-insurance system is incoherent, hard to understand, often dysfunctional and bloated by obsolete legacy systems. (Though unlike Windows machines, it's not cheap.) Different parts fail to operate properly with each other, and the whole thing is incomprehensible to most users, patients and doctors alike. But try to set up a central authority like MedPAC to make decisions about how to fix Medicare, or to mandate that policies cover a set of basic conditions, or to make end-of-life counseling available to seniors so they don't go through their final weeks in a blizzard of legal confusion—try to fix any of this stuff, and you'll be accused of "taking the control of health care out of the hands of patients and their doctors." This rhetoric is often driven by vested commercial interests. Medical-industry groups don't want a panel of experts making decisions about Medicare because it reduces their ability to buy concessions through congressional lobbying.
The same goes for the banking and credit-card industries, where small-print legal confusion is used as a tool to extract money from customers, and efforts to ban such practices are attacked as restrictions on consumer freedom. In health insurance or credit cards, freedom's just another word for not understanding what's in your contract. A perfect illustration, from Republican congressman Jeb Hensarling: "The ironically named Consumer Financial Protection Agency (CFPA) would have the power to strip from consumers their freedom of choice and restrict their credit opportunities in the midst of a financial recession—all in the name of 'consumer protection'. Positively Orwellian."
What's Orwellian is describing your credit-card company's ability to arbitrarily raise the interest it charges you on past debt to 35% as "freedom". More broadly, we need to move away from the Orwell "1984" paradigm. It was a brilliant description of the most important threats to freedom in the middle of the 20th century, but it no longer describes the most important threats to freedom today*. It was already clear how creaky the paradigm was in 1984, when the Mac ad came out; it's only gotten creakier over the past 25 years. Orwell didn't pay much attention to the problem of an oppressive blizzard of "choices" designed to take advantage of the consumer or citizen by manipulating asymmetries of information. But that is the way the American commercial and political landscape feels much of the time
Definitely something to think about. What is the true freedom here? And, in terms of health care and other social services, who is benefiting from this 'freedom'?
Friday, September 25, 2009
Why I didn't want to do Test-Drive Development - and why I like it now
I've been thinking about why I didn't do Test-Driven Development for so many years (in particular, writing tests first), and why others don't.
You can be shown lots of books. You can be told it's important. But you still don't do it.
I think it's about discomfort. You're being asked to change. For years, the way I coded was: I did some analysis, thought about the design, and then started coding the design as I saw it, either from my head or from some basic UML.
TDD doesn't let you do that. You have to (ick) write a test first. Why on Earth would I ever want to do that? I'm trying to implement my design, not muck around with tests! So as much as I appreciated from afar the values of TDD, I just never get around to doing it, because it's just not how I did things. It's almost like you're coding naked or with a keyboard with a different layout - it's awkward, uncomfortable, and just feels wrong.
So I came up with all sorts of justifications for not doing it - I don't have time; I'm already writing tests, writing them first seems like overkill; I'm already overwhelmed with things I need to deal with; people who that are just anal design-freaks, and so on. But I think ultimately it came down to - that's just not how I do things.
But recently I finally forced myself to start doing it. I slowly started becoming more comfortable with this approach, and started to see the benefits.
To me the key thing is this concept of letting the design emerge rather than trying to impose a design. It's similar to starting with use cases first, and regularly validating your product with the customer, rather than building a technology and hoping they'll come.
Once I got used to letting the design emerge, I started really enjoying TDD. It was an act of discovery as much as an act of creation. And I truly believe the best creations are done that way - as an act of joyous discovery.
And yes, by the way, you get much better code coverage when you do TDD, because you ensure that each "piece" you build is tested, rather than writing a bunch of code and then trying to figure out how to test it.
And yes, by the way, you get designs that are much less coupled and more cohesive.
But to me my favorite part of it is that joyous process of discovery.
You can be shown lots of books. You can be told it's important. But you still don't do it.
I think it's about discomfort. You're being asked to change. For years, the way I coded was: I did some analysis, thought about the design, and then started coding the design as I saw it, either from my head or from some basic UML.
TDD doesn't let you do that. You have to (ick) write a test first. Why on Earth would I ever want to do that? I'm trying to implement my design, not muck around with tests! So as much as I appreciated from afar the values of TDD, I just never get around to doing it, because it's just not how I did things. It's almost like you're coding naked or with a keyboard with a different layout - it's awkward, uncomfortable, and just feels wrong.
So I came up with all sorts of justifications for not doing it - I don't have time; I'm already writing tests, writing them first seems like overkill; I'm already overwhelmed with things I need to deal with; people who that are just anal design-freaks, and so on. But I think ultimately it came down to - that's just not how I do things.
But recently I finally forced myself to start doing it. I slowly started becoming more comfortable with this approach, and started to see the benefits.
To me the key thing is this concept of letting the design emerge rather than trying to impose a design. It's similar to starting with use cases first, and regularly validating your product with the customer, rather than building a technology and hoping they'll come.
Once I got used to letting the design emerge, I started really enjoying TDD. It was an act of discovery as much as an act of creation. And I truly believe the best creations are done that way - as an act of joyous discovery.
And yes, by the way, you get much better code coverage when you do TDD, because you ensure that each "piece" you build is tested, rather than writing a bunch of code and then trying to figure out how to test it.
And yes, by the way, you get designs that are much less coupled and more cohesive.
But to me my favorite part of it is that joyous process of discovery.
Why make methods small and do one thing?
I've been reading the ObjectMentor blog, and it's got some fun stuff in it. The one I particularly liked is how do you know when a function does "just one thing" - the answer - extract methods until you can't extract any more.
Now, I know there is the feeling of - why should I waste all this time writing lots of tiny little functions? But after trying it for a bit, I'm convinced, for two main reasons (although I am sure there are more):
Readability
When you write small functions, each one is eminently readable. You look at it, and in an instant you understand what it does and its intent. You don't have to "decode" the method to understand the intent - it just tells you. I think this is essential - when you understand a code's intent, you can also understand if it is doing what you want it to.
Discovering New Classes
When you extract functions, sometimes you run into this situation where you are passing umpteen (meaning more than two) parameters into the function. To me, particularly when the list of parameters is long, this indicates the need to extract a class which encapsulates all this data you're passing around.
And I often find once I've extracted the class, it clarifies the design for me. You have an idea of a design, but you should really let the design reveal itself to you, and this is one of the great ways I have found that happening.
Also, often once a class is extracted, I can see how it can be used elsewhere - I have a new unit of reusability.
Anyway, I recommend keeping an eye on the ObjectMentor blog, there's good stuff there.
Now, I know there is the feeling of - why should I waste all this time writing lots of tiny little functions? But after trying it for a bit, I'm convinced, for two main reasons (although I am sure there are more):
Readability
When you write small functions, each one is eminently readable. You look at it, and in an instant you understand what it does and its intent. You don't have to "decode" the method to understand the intent - it just tells you. I think this is essential - when you understand a code's intent, you can also understand if it is doing what you want it to.
Discovering New Classes
When you extract functions, sometimes you run into this situation where you are passing umpteen (meaning more than two) parameters into the function. To me, particularly when the list of parameters is long, this indicates the need to extract a class which encapsulates all this data you're passing around.
And I often find once I've extracted the class, it clarifies the design for me. You have an idea of a design, but you should really let the design reveal itself to you, and this is one of the great ways I have found that happening.
Also, often once a class is extracted, I can see how it can be used elsewhere - I have a new unit of reusability.
Anyway, I recommend keeping an eye on the ObjectMentor blog, there's good stuff there.
Thursday, September 24, 2009
You should like what you do
I was sitting down for dinner last night, after having grilled some chicken kabobs. I mentioned that I thought I had cooked them long enough but they seemed undercooked.
My 9-year old daughter, Ariel, after a pause, said "You should like what you do." I looked at her, confused. "You're always saying something bad about what you do. I get tired of it. And it makes me not want to eat what you make."
She really stopped me in my tracks. She was right! And I told her so, "you're absolutely right, Ariel." The words "never apologize!" from Julia Child's book "My Life in France" came back to me. She tells a story of making a horrific dish, but she and her guests just shouldered through the dish as best as they could, and Julia never apologized and the guests never complained.
I was so touched by Ariel's sentiment: "you should like what you do." It was an admonishment, yes, but more than that, an encouragement - I believe in you, and you should too.
I spent some time looking at myself, and saw that indeed, yes, I have an underlying idea of myself as "not good enough." We could go back into my history to understand how this happened, but I'd rather work with the present moment. How can I work with this now?
"You should like what you do." I see this in two ways. First of all, whatever I do, take pride in it. Yes, it may not be "perfect" but all the same take pride in it. It is a creation from the heart. Secondly, I should like what I do, rather than wish I were doing something else. This is the key to a happy life. "It is not doing the thing we like to do, but liking the thing we have to do, that makes life blessed," said Goethe.
So, I'm not rash enough to believe I can change my underlying perception of myself in a flash, but it's something to practice and be aware of. When I find myself being overly critical or unhappy with something I'm going or have done, I can remember, "you should like what you do" and be happy about it. Always room for improvement, but it's great the way it is. My cooking is great. My code is great. My pictures are great. My words are great. My nose is great. I am great.
My 9-year old daughter, Ariel, after a pause, said "You should like what you do." I looked at her, confused. "You're always saying something bad about what you do. I get tired of it. And it makes me not want to eat what you make."
She really stopped me in my tracks. She was right! And I told her so, "you're absolutely right, Ariel." The words "never apologize!" from Julia Child's book "My Life in France" came back to me. She tells a story of making a horrific dish, but she and her guests just shouldered through the dish as best as they could, and Julia never apologized and the guests never complained.
I was so touched by Ariel's sentiment: "you should like what you do." It was an admonishment, yes, but more than that, an encouragement - I believe in you, and you should too.
I spent some time looking at myself, and saw that indeed, yes, I have an underlying idea of myself as "not good enough." We could go back into my history to understand how this happened, but I'd rather work with the present moment. How can I work with this now?
"You should like what you do." I see this in two ways. First of all, whatever I do, take pride in it. Yes, it may not be "perfect" but all the same take pride in it. It is a creation from the heart. Secondly, I should like what I do, rather than wish I were doing something else. This is the key to a happy life. "It is not doing the thing we like to do, but liking the thing we have to do, that makes life blessed," said Goethe.
So, I'm not rash enough to believe I can change my underlying perception of myself in a flash, but it's something to practice and be aware of. When I find myself being overly critical or unhappy with something I'm going or have done, I can remember, "you should like what you do" and be happy about it. Always room for improvement, but it's great the way it is. My cooking is great. My code is great. My pictures are great. My words are great. My nose is great. I am great.
Friday, September 18, 2009
A balance between flux and immutability
A fun blog post by John Rose about the delicate balance between the world of flux (e.g. volatile memory) and our attempts to enforce immutability through functional languages like Erlang. Immutability is great, is helps rationalize and simplify your logic, but it just doesn't play well with the notion that "you don't step into the same river twice." His overall conclusion - work with a language that encourages immutability but allows for in-place mutation where it is needed. You know a craftsman at work when he sees the issues of software from the perspective of ancient philosophy :)
But we know from common sense (that trusty yet abused helper) that we live somewhere between the fluxy world of Heraclitus and the timeless world of Parmenides. In the days of Plato, it was his student Aristotle who balanced the claims of the two accounts, picking a middle way between the two. In short, the formal ideas we perceive and know inhere (in a way real though bound) in the mutable, moving reality around us. Object identities (what he called "substances") are real, though temporal. If Aristotle had marketers like we do, he might have called his account a hybrid paradigm (more Greek, actually, though to him "hybrid" would mean "having hubris") of flux and form. I think it's likely the story will end similarly for us, in our professional difficulties with software, with some sort of middle way.
Just to speculate wildly, maybe those hybrid functional-and-object-oriented languages (like Scala and F#) can provide a sort of dictionary and grammar of useful terms and constructs. These would somehow describe computations as immutable in most places, and manageably mutable in those few aspects needed to model real-world transitions. (Or to implement localized compute engines, like a quick-sorter.
Thursday, September 17, 2009
Racism in America? Shocking!
What?! Racism still exists in America?! I am shocked! And some of the media and politicians are trying to use this to get attention and money?! Doubly shocking!
I like Obama's approach: we all know racism exists, let's move on and focus on doing our job: fixing health care, fixing the economy, dealing with global warming, and so on. I suspect he's done that all his life. Why waste your time putting energy and attention into something that is divisive and ultimately gets you nowhere. I've never met someone who stopped being racist because someone said "you're a racist!" to them.
I suspect that underlying all of this is not racism per se but a fear of the unknown. If we work together create a country where each individual feels safer - in terms of their job, their health, and their overall prospects in life - then there will be less hate-filled speech and actions.
I like Obama's approach: we all know racism exists, let's move on and focus on doing our job: fixing health care, fixing the economy, dealing with global warming, and so on. I suspect he's done that all his life. Why waste your time putting energy and attention into something that is divisive and ultimately gets you nowhere. I've never met someone who stopped being racist because someone said "you're a racist!" to them.
I suspect that underlying all of this is not racism per se but a fear of the unknown. If we work together create a country where each individual feels safer - in terms of their job, their health, and their overall prospects in life - then there will be less hate-filled speech and actions.
Tuesday, September 15, 2009
Time out on timeouts
A very interesting (and somewhat tragic) article about the long-term damage caused by withholding attention and affection as a way to discipline children.
The most common tool in this toolbox is "time-outs". My wife and I just didn't feel comfortable with this - it felt humiliating and mean - and the research seems to be confirming it's not the best approach
The most common tool in this toolbox is "time-outs". My wife and I just didn't feel comfortable with this - it felt humiliating and mean - and the research seems to be confirming it's not the best approach
The studies found that both positive and negative conditional parenting were harmful, but in slightly different ways. The positive kind sometimes succeeded in getting children to work harder on academic tasks, but at the cost of unhealthy feelings of “internal compulsion.” Negative conditional parenting didn’t even work in the short run; it just increased the teenagers’ negative feelings about their parents.
What these and other studies tell us, if we’re able to hear the news, is that praising children for doing something right isn’t a meaningful alternative to pulling back or punishing when they do something wrong. Both are examples of conditional parenting, and both are counterproductive.
...
In practice, according to an impressive collection of data by Dr. Deci and others, unconditional acceptance by parents as well as teachers should be accompanied by “autonomy support”: explaining reasons for requests, maximizing opportunities for the child to participate in making decisions, being encouraging without manipulating, and actively imagining how things look from the child’s point of view.
The last of these features is important with respect to unconditional parenting itself. Most of us would protest that of course we love our children without any strings attached. But what counts is how things look from the perspective of the children — whether they feel just as loved when they mess up or fall short
Monday, September 14, 2009
Health care is the new sexy
This is very funny. BTW, anyone know why I can't embed this video?
http://www.youtube.com/watch?v=XCw_UoRhTUk
http://www.youtube.com/watch?v=XCw_UoRhTUk
Monday, September 07, 2009
Color photographs of Russia from 100 years ago
Go look at these amazing photos on Newsweek.
Between 1907 and 1915, Prodkun-Gorskii rode through even restricted parts of revolutionary Russia in his specially fitted darkroom rail-car, shooting color photos ... by a method of his own inventionReally quite stunning. So used to seeing black-and-white photos of these times, and here it is, the turn of the century in living color.
Color photographs of Russia from 100 years ago
Go look at these amazing photos on Newsweek.
Between 1907 and 1915, Prodkun-Gorskii rode through even restricted parts of revolutionary Russia in his specially fitted darkroom rail-car, shooting color photos ... by a method of his own inventionReally quite stunning. So used to seeing black-and-white photos of these times, and here it is, the turn of the century in living color.
Subscribe to:
Posts (Atom)