Van Couvering Is Not a Verb

Thursday, September 18, 2008

Barack Obama's Economic Plan

Wow. Direct, to the point, and ( (although I'm not sure about the tax rebate) it actually makes sense. Read more here.

Revamp tax system to work for the average American, not the rich
Regulate Wall Street
Energy independence in 10 years
Crack down on lobbyists
Responsible end to war in Iraq

It won't be easy. But we're Americans. We've met tough challenges before and we can again.

Nicholas Kristof - Need a Job? $17,000 an Hour. No Success Required.

Pointed, painful, outrageous, hilarious.

John McCain seems to think that the problem is that C.E.O.’s are greedy. Well, of course, they are. We’re all greedy. The real failure is one of corporate governance, which provides only the flimsiest oversight to curb the greed of executives like Mr. Fuld....
John Kenneth Galbraith, the great economist, once explained: “The salary of the chief executive of a large corporation is not a market award for achievement. It is frequently in the nature of a warm personal gesture by the individual to himself.”

http://www.nytimes.com/2008/09/18/opinion/18kristof.html?ref=opinion

Friday, September 12, 2008

Paul Krugman - Blizzard of Lies

It is stunning to see advertisements approved by John McCain stating flat, complete, and utter lies.

What it says, I’d argue, is that the Obama campaign is wrong to suggest that a McCain-Palin administration would just be a continuation of Bush-Cheney. If the way John McCain and Sarah Palin are campaigning is any indication, it would be much, much worse.

http://www.nytimes.com/2008/09/12/opinion/12krugman.html?ref=opinion

Wednesday, September 10, 2008

Ludovic Champenois's Blog: GlassFish V3 Prelude Update Tool

Ludo gives a nice demo of the Glassfish V3 update tool in action. An app server that is incrementally upgradable like Ubuntu. Pretty nice!

http://weblogs.java.net/blog/ludo/archive/2008/09/glassfish_v3_pr.html

I am sure a lot of you already know about this, but it took me a while to find this in Google-land, so I thought I'd share it here.

If you want to create a JLabel that does word wrap dynamically, so that if you resize the window the label "does the right thing", then wrap the label text with <HTML> and </HTML>

Memcached User-Defined Functions for Java DB : Knut Anders Hatlen's Weblog

Wow, now this is cool. Some sample code that shows you how to turn Java DB into a memcached engine. Take a look at this snippet:


ij> create trigger insert_trigger after insert on my_table
       referencing new as n
       for each row
       call memcached_set(n.id, n.x);
0 rows inserted/updated/deleted

http://blogs.sun.com/kah/entry/memcached_udf_for_java_db

Drew Westen: What Obama Needs to Do in the Final Sixty Day

Long but great essay with a lot of zing about the problems with standard Democratic strategy and some excellent suggestions (IMHO) or Mr. Obama. I don't know about you, but I am horrified with the decisions McCain is making and am impressed by the ruthlessness of the Republican political machine. I would desperately like to see Obama and his campaign take McCain/Palin by the jugular and stop namby-pambying.
This isn't just about politics. This is about pulling out all the stops to make sure we don't have four more years of horrible Republican policy - I mean, look around at the mess in this house (country)! It's deplorable, and it's been done on the backs of the middle class and poor.

http://www.huffingtonpost.com/drew-westen/what-obama-needs-to-do-in_b_125051.html

Tuesday, September 09, 2008

Bob Herbert - Liberals, Hold Your Heads Up

Great editorial by Bob Herbert reminding all of us the amazing accomplishments of liberals in the US. I highly respect and agree with conservative's championing of traditional values such as honesty, sacrifice, hard work and family.

But let's remember the love, hard work, sacrifice (including jail time, verbal and physical abuse, even lives lost) given by liberals fighting for the repressed and ignored - women, blacks, the elderly, gays, our health, the environment.

Without these honorable fights against a highly resistant conservative status quo, where would we (and our brothers and sisters and children and parents) be today? Hear hear, Bob.

http://www.nytimes.com/2008/09/09/opinion/09herbert.html?_r=1&em&oref=slogin

Monday, September 08, 2008

Skype ignores PayPal siphoning hijack scheme | The Register

Ouch. This is nasty. I use both Skype and Paypal, and now I wonder if I should move on to something else. I stopped using EBay years ago after a number of nasty experiences.

http://www.theregister.co.uk/2008/09/02/mysterious_skype_hijackings/

Sun and NetApp gain external storage market share | The Register

You know, we make excellent products, and it's nice to see people taking notice...

http://www.theregister.co.uk/2008/09/05/sun_and_netapp_gain_market_share_gartner_and_idc_say/

Patrick Keegan's Blog: A Personal Data Storage Application With Embedded Java DB

Patrick has done a very useful blog showing how you can use JPA with embedded Java DB. Thanks, Patrick!

http://weblogs.java.net/blog/pkeegan/archive/2008/08/a_personal_data_1.html

Friday, September 05, 2008

Economic View - Is History Siding With Obama’s Economic Plan? - NYTimes.com

This article review a book that reveals statistical facts that reflect my experience at any rate. The Republicans talk about being for the common man and woman, and perhaps many of them are, but the ones in power have shown by their actions that they are very much for the rich and powerful.

The two Great Partisan Divides combine to suggest that, if history is a guide, an Obama victory in November would lead to faster economic growth with less inequality, while a McCain victory would lead to slower economic growth with more inequality. Which part of the Obama menu don’t you like?

http://www.nytimes.com/2008/08/31/business/31view.html?em

U.S. Rescue Seen at Hand for 2 Mortgage Giants - NYTimes.com

Nice to know that wild speculation is being bailed out on the backs of taxpayers.

I know, I know, better than not doing anything, but it sure reminds me of sending drunken Uncle Bob some cash so he doesn't end up in jail.

Promises of reform to follow, I suspect, followed by a short memory, debates, and a watered down bill in Congress.

http://www.nytimes.com/2008/09/06/business/06fannie.html?hp

And here, this is what really gets my goat:

But privately, senior officials have been critical of top executives at the companies, particularly Freddie Mac. They have raised concerns about major risks to taxpayers of a bailout of companies whose executives have received huge compensation packages. Mr. Syron, for instance, collected more than $38 million in compensation since he joined the company in 2003.

Must be nice, to have the taxpayers foot $38 million in personal income. Consequences, what consequences?

What's even worse is that the reason investors didn't help out these two companies was because everyone knew that the US government would bail them out sooner or later. Fait accomplis...

Sigh...

Major Republican Hypocrisy - The Daily Show | Comedy Central

A friend showed me this video. Hilarious and also very seriously revealing. The two quotes from Karl Rove are particularly damning. Warning - a little strong language in some of the commentary.

http://www.thedailyshow.com/video/index.jhtml?videoId=184086&title=sarah-palin-gender-card

TechCrunch: Cappuccino Brings Cocoa-Like Programming To The Web

OK, this can change everything. If you can use a full-fledged programming language like Objective-C to build web apps, and you don't have to know HTML and CSS (and who would want to if you had a choice), well, now that's interesting.

Cool thing is if you learn this, then you can write stuff for Cocoa as well. As a programmer, anything that lets you do two in one blow is very attractive. I have avoided writing web apps for years because of the hideous, painfully low-level programming environment. This could switch me over.

Also, think about it - the full application is running in Javascript on the IDE. This means the server isn't busy building HTML/CSS, but it simply in the job of delivering Javascript and providing back-end services for rich web clients. Cool.

http://www.techcrunch.com/2008/09/04/cappuccino-brings-cocoa-like-programming-to-the-web/

Thursday, September 04, 2008

Brian Aker - Deleting Your Work, Starting over is good for the soul...

It felt good to delete yesterday's tree and start over again. More often than not this is a best practice :)

A nice short blog about something I have done a number of times. It's good to know I'm not the only one, and yes, now that I think of it, it is a good thing.

http://krow.livejournal.com/608110.html

NetBeans 6.1 has a new patch release available

Hi, all. I thought I'd share this message from our sustaining team:

Hi All,

I would like to inform you that a new patch for NetBeans IDE 6.1 is now available on the Update Center. The patch includes fixes in modules for BPEL, C/C++, Composite Application, Database, Editing Files, GUI Builder, GlassFish, IDE Platform, Java, Java EE, Java Persistence, NetBeans Plugin Development, Platform, RESTful Web Services, SOA, TAX Library, WSDL, Web Services, XML Productivity Tools, XML Schema Support, XML Tools Java Ext, XML and Schema and XSL Support.

The patch is available in English, Japanese, Simplified Chinese and Brazilian Portuguese languages.

For more information, see http://wiki.netbeans.org/wiki/view/NetBeans6.1PatchesInfo.

The update is easy to install when the NetBeans IDE is running. Just click on the globe in the status bar or go to Tools->Plugins->Updates tab, then follow the installation instructions.

Regards,
-Rudolf Balada

Wednesday, September 03, 2008

Government arrests protesters - in USA, not China

Pretty ugly. I just find this astounding, in the country where the principle of free speech was established.

Thursday, August 28, 2008

Engineering @ Facebook : How We Scale Out

Very interesting details into how one of the biggest MySQL customers scales out, particularly how they handle consistency issues with an asynchronous replication and caching architecture.

http://www.new.facebook.com/notes.php?id=9445547199

MySQL Workbench 5.1 Alpha for Linux in Less Than 30 Days

Mike Zinner tells us that his team will be providing an early access of MySQL 5.1 for Linux this month. Those guys are on a roll...

http://dev.mysql.com/workbench/?p=138

Monday, August 25, 2008

Licking summer

Just came back from a very nice family reunion up at Grant's Pass, Oregon.

At one point my little four-year-old-first-cousin-once-removed David (he calls me "cousin me" because we both have the same name) led me up the side of Applegate River next to his house exploring, and we found ourselves picking (and eating) delicious juicy blackberries in the toasty summer sun.

It seemed so idyllic, and I said "mmmm.... this is the taste of summer..."

David was silent for a moment and then he said "we're eating summer..." I said "yeah, I guess we are!"

Then he said, "no, we're licking summer, because it keeps coming back every year. So you can't eat summer ... or spring .... or winter. I guess if you ate all the trees then you could eat fall, because all the trees would be gone and there would be no more fall."

So there we were, licking summer, and enjoying every minute of it.

Wednesday, August 20, 2008

Miserable bastards (at Apple)

I was listening to a fun song on iTunes, I right-clicked on it and I saw "Create Ringtone..." Cool! So I went ahead and decided to see what it would be like to create a ringtone from a song.

Miserable. Bastards.

And yes, that is the name of the song. Just coincidence. Really.

My mothers are getting married

My Mom came out over 25 years ago, and she has been with the same partner, Maryann for over 20 years.

They're coming to visit us here in California this week, and because gay marriage was just made legal here, they're finally going to marry.

My sister asks "But will it last?" :)

Congratulations, Mom (and Mom)!

Tuesday, August 19, 2008

NetBeans Wiki: Best Practices With JPA And BeansBinding

A great technical guide on the NetBeans Wiki which quickly and clearly goes through how to use beans binding with JPA, including the question that comes up all the time: how to use the Beans Binding support with Java DB embedded.

People get very confused because they see the tables and data in the Database Explorer but they aren't there when they run their app. And I quote: "Don't be confused if you don't see the data (or even tables) when you run your application in spite of the fact that you can see them in Services window of NetBeans. Recall that Derby is looking for the DB in the working directory and the working directory of NetBeans is different from the working directory of your project."

http://wiki.netbeans.org/BestPracticesWithJPAAndBeansBinding

Sun Rise, Sun Set - Josh Berkus leaves Sun.

Sorry to see Josh go...

http://it.toolbox.com/blogs/database-soup/sun-rise-sun-set-26078

Arnon Rotem-Gal-Oz - Why the Database as a Service is a bad idea

What Arnon says, I've been thinking for a while. But that's not stopping anyone. It just *seems* to fit so perfectly.

http://www.rgoarchitects.com/nblog/2008/08/17/WhyTheDatabaseAsAServiceIsABadIdea.aspx

So, what is it that you do, Dad?

I kept a promise yesterday by taking my eight-year old daughter into the office. I missed the official "Take Your Daughter To Work Day" but I really wanted to do this.

She was very excited and she told me she really loved my work and wanted to go again.

I found this mystifying, as this was our day:

Take BART from North Berkeley to Union City (45 minutes)
Take the Dumbarton Express bus over the Dumbarton Bridge to the Menlo Park campus
Get some breakfast
Meet with the *one* person in my team who (a) works in the Bay Area and (b) was in the office
Color and listen to stories on her MP3 player while I worked
Go to lunch
Color and listen to stories while I worked
Go to coffee
Color and listen to stories while I worked
Take the bus and BART back home

I tried to get her to tell me what it was she loved about going into the office, but she didn't come up with anything specific.

You know what I think. I think it's just great that she and I spent the day together. She knows it, I know it, and it had nothing to do with what we did. Just in itself, it was precious.

Something to seriously keep in mind.

Sakila Sample database plugin now available from NetBeans Daily Build

As I promised in an earlier post, I have created a plugin that makes it very easy for you to load the MySQL Sakila sample database from NetBeans. Sakila is a serious sample database, with tons of data, and a number of views, stored procedures, triggers and functions to allow you to really explore and test all the various features of MySQL.

If you tried in the past to load Sakila from NetBeans, you were probably quickly dissatisfied. The editor currently can't handle such a large file, and it didn't handle the delimiters correctly, and it also completely barfed when the script tried to insert a raw PNG file.

So, I fixed all those, and now you simply have to install the plugin, then choose Create Database from your MySQL node in the Database Explorer, and you're off and running.

Here are the steps you need to follow.

First of all, you need to get a recent 6.5 development build, this does not work with NetBeans 6.5 beta. Go to http://bits.netbeans.org/dev/nightly/latest/ and get the nightly build from there.

Then you go to Tools->Plugins->Available and choose Sakila Sample Database. To filter the list, type in Sakila in the matchbox. Check the install checkbox and choose Install

Now open the Services tab, expand Databases, and connect to your MySQL server through the MySQL node. If you don't have this node, right-click on Databases and choose Register MySQL Server.

Once you are connected, right-click on the node and choose Create Database...

This brings up a little dialog with a drop-down list of sample names. Note that sakila is on the list - that was added by the plugin.

Choose sakila and click OK.

You are then presented with a dialog to create a new connection for the new database.

Type in the appropriate user and password, press OK, and then press OK when asked to choose a schema (I really should fix this, you don't choose schemas in MySQL as a schema and a database are the same thing).

At this point the plugin merrily creates all the tables, views, stored procedures and triggers for Sakila and then loads up all the data. One thing I need to fix is to add a progress bar so you know when it's done, but just keep your eye on the output window(s). (Another thing to fix: don't have multiple output windows. But hey, this is still in beta :)).

Now you should have all of Sakila loaded up, easy as pie.

Please let me know if you have any feedback, particularly by logging issues.

Friday, August 15, 2008

In India, the cell phone is a religious experience

This should probably go on the list of "things you don't know about me" (well, some of you do), but over the millennium I spent a year in India, living in an ashram north of Mumbai.

One thing that I came to understand during that time was how much religion is part of the warp and woof of Indian life. Temples and images are everywhere - even truck drivers and cab drivers have their favorite deities covering their dashboards and painted over the outside of their trucks.

I was having coffee with an Indian friend of mine today, and he told me that another thing that is very popular in India is cell phones, and that these two things have combined in a wonderful and odd sort of way.

He said one of the most popular applications people download onto their cell phones in India is a "daily worship" application. As I understand it, you pick your favorite deity, and each morning when you open your cell phone it pops up an image of your deity, it plays mantras for the deity, and even shows incense waving in front of it.

The thought of this made me smile and laugh. I also found myself filled with love and respect for the way in which spirituality and devotion is anchored into the very core of Indian life.

Thursday, August 14, 2008

NetBeans development process rocks!

I have worked on a lot of large software projects in the 17+ years I have been in the industry, and I don't say it lightly that I think NetBeans has one of the best development processes I've ever seen.

I've been on (and seen) projects that were so overburdened with processes and documents that it was like walking through a lake of molasses.

I've also been on (and seen) projects where there was so little process that life was exciting, interesting, crazy, but basically one chaotic scramble after another.

NetBeans follow a beautiful middle path - not too much process, but not too little. I'm sure I'm missing a lot of important points and aspects that make this all fall together, but these are the ones that stand out to me:

Planning
The planning phase is fast and lightweight. Each release is given a release boss, and the release boss works to define a theme for the release. The release boss also decides on the dates for milestones and the final release. Then each team is asked to decide what features they feel best meet the theme, and for what milestones those features will be delivered.

Normally the feature list is written up quickly on a Wiki page. Each feature is given an issue number in Issuezilla and is assigned a person who owns it and a milestone date.

The release boss uses reporting features in Issuezilla to track all of the features committed for a given milestone, and you have the plan. Pretty simple. We all know what to do, and when we have to do it by, and we get to work.

Specification and Design

You are expected to specify the user experience and interface for your features. But it's in a very lightweight manner, on a Wiki page. Here is an example UI Spec that I wrote. Pretty simple, but it gets the point across.

Certain defining features get a lot of attention and are reviewed carefully by our User Experience team, but for most smaller features you just need to write up a quick spec, have it reviewed by your team, and you are done.

Milestones
I love milestones. I think every large team should have milestones. Basically these are a mini-release, where everyone has to get features in by a certain date, there is a stabilization period with extended QA and bugs are fixed, and then it gets published on the NetBeans site. Milestones normally happen about once every 4-6 weeks.

The main reason why I like these as a developer is because it keeps me focused. A six month development cycle tends to make you be lazy early on in the cycle. With milestones coming at you once a month, you get very focused, and you are working hard all the way through the release, instead of working insanely at the end. This also means that it is much more likely that we deliver on time, or if we're late, we know sooner rather than later. And what I have seen is that NetBeans has consistently delivered what we say we deliver, when we say we're going to deliver it. There is a lot behind this, but milestones are a key part of this.

Simple, Strong Quality Criteria
Every release has quality criteria. These are soft and hard goals in terms of the number of P1, P2 and P3 bugs. There is a total overall goal, and each team is also assigned specific goals. For some reason a lot of projects I have been on have not been that strict about this, and it has affected the quality of their product

Nightly Builds Available to the Community
What this means is everybody is using our stuff all the time, all the way through the release process, and we hear from them. I think this is great. More and more projects are doing this, and I really can't recommend it highly enough. And if you don't know about Hudson, you really should check it out.

The Team
This is a lot harder to reproduce, but as usual is a key part of it.

The theme running through this all of this and a thousand more details and steps, is a dedication to quality and predictability, without overburdening the teams with process and busywork. It's a difficult balance, and it's executed quite well here.

So I think that is probably the underpinning of why NetBeans development process rock: the team. From developers to management to QE to release engineering, I have never seen such a smart, dedicated, hardworking group of folks. I am really honored to work with them.

Wednesday, August 13, 2008

Drop Memcached, Add More MySQL Servers | High Scalability

A very interesting summary of a different way of looking at things: rather than add a new layer with memcache, consider creating more shards to the point where all your data is running in memory. "Hitting disk is the problem. Shard more and get your data in memory."

http://highscalability.com/strategy-drop-memcached-add-more-mysql-servers

Van Couvering IS a verb!

I just found out that I've been verbized. From the NetBeans users mailing list:

I've googled, nabbled, Vancouveringed (yes, it is a verb now) and even read the manual (gasp), and I'm at a loss. I have no confidence that data changes will still be 'there' when I copy the DB folder to another location.

Well, I give up. I guess I will have to change my blog title now.

Anyway I've been reminded by various English types that if "Vancouvering" were anything it would be a gerund, not a verb. But "Vancouvering is Not a Gerund" just doesn't have the same ring to it. And this way my blog title is slightly more confusing ("um, yeah, did I ever say it was?), and that's always good, makes you scratch your head a bit.

W00t! NetBeans 6.5 beta is a go!

Great news, we just got a "go" for NetBeans 6.5 beta, and you can get it here.

There is a lot of great stuff in this release. Solid PHP, JavaScript and AJAX support. Groovy/Grails support.

The Java debugger and editor also have some nice improvements, including the very nice feature that you Eclipse users are used to, compile on save.

And of course a bunch of cool new database features.

One thing that's not listed yet that I'm just about to put out is support for creating the Sakila sample database directly from NetBeans. This will be available as a plugin on the update center. I'll post more about this when it's ready.

Monday, August 11, 2008

Are you engaged or are you vegging?

I just recently got clear on a distinction between being engaged and vegging. Both of them can happen on a computer.

When I am engaged, my mind is active, I feel productive, and I feel full of energy and creativity.

When I am vegging, it is as if my mind is shut off. Even if my jaw is not slack, I feel slack-jawed. I am not on the computer for any reason but to check out.

I like it when I'm engaged. I dont' like it when I'm vegging. It affects me not only when I do it but after, and I also find I miss out. I am on an anti-veg campaign. My first battle-line - no vegging at night. If I have to work in the evenings, but when I'm done, no vegging. No looking at endless blogs. No link-following, no You-Tubing, no Twittering, no Facebooking. Also, no iPhone vegging.

I do need breaks - one can't work non-stop, it's actually non-productive. But I have so many other great things I could be doing with my break time. Standing up and stretching, going into the kitchen to make myself a cuppa. Breathe deep and relax. Hang out with the kids for a few minutes. Read the paper.

How about you? Do you know when you're vegging and when you're engaged?

Tim Boudreau's Blog: The Capability Pattern - Future-Proof Your APIs

Tim is doing an excellent series on API design. I'm learning a lot in each read. This latest one is on how to use API/SPI and final to create APIs that retain compatibility.

http://weblogs.java.net/blog/timboudreau/archive/2008/08/the_capability.html

Sunday, August 10, 2008

Creating a Custom Java Desktop Database Application - NetBeans 6.1 Tutorial

A new tutorial is out showing how to build a Java Desktop Database app. It's using some of the new MySQL support we put into NetBeans 6.1 to make it easier to work with MySQL from NetBEans.

http://www.netbeans.org/kb/61/java/gui-db-custom.html

Thursday, August 07, 2008

Real-world experience with the new NetBeans PHP and JavaScript editors

One of the challenges for someone who builds products but doesn't necessarily use them is that you can get so busy with your day job that you don't know what it's like to use the product.

I do use NetBeans during my day job, writing Java code for the NetBeans product itself. So I am definitely getting my hands dirtier than when I was working on the app server or Java DB. But NetBeans is vast, and there are some areas that I just don't get enough exposure to, but which are directly relevant to my job.

In particular, a large chunk of database applications today are web applications written with PHP and JavaScript. I am quite aware of the new JavaScript and PHP editors, but had not had much experience with them.

Yesterday I spent the whole day trying to help a friend in need who had waded in too deep with a contract he has. He has no database experience, and was trying to customize a Wordpress plugin that uses PHP, JavaScript and MySQL.

So I downloaded the daily build of the PHP bundle of NetBeans for 6.5 and started working. I had done a previous PHP project using vi, and although I was fairly productive it was somewhat painful. So I was so happy to see so many of the things in the NetBeans Java editor available for JavaScript and PHP: completion, inline documentation, easy refactor/rename, goto declaration, find usages, error highlighting and so on.

I felt much more comfortable and productive in this environment. The PHP support even has FTP integration to make it easy to push to a remote site.

But I also spent a lot of time doing SQL in PHP, and had a number of ideas on how to make it easier to work with SQL in a code editor .

One thing is that for this project, as it is for many, MySQL is on a remote server on the Internet. Our entire Database Explorer is useless in such a scenario, as it requires a JDBC driver connecting to a server available on the LAN.

SSH Tunnel Support

There are HTTP JDBC proxy drivers, but what most people usually do is set up a ssh tunnel using a tool like PuTTY for Windows or OpenSSH for UNIX systems. Wade Chandler provided useful instructions for how to do this with PuTTY. Here is another good page on how to do this.

But I can imagine finding a way to make it very easy to configure a Database Explorer connection to use an ssh tunnel. This simple thing could be a big win for a lot of our users. I'm not sure what platform-specific dependencies there are and if this is possible in a Java environment, but I'm going to check it out.

Add Column Wizard

I have been thinking about how to do SQL completion in the PHP editor for a while. But during my work yesterday another itch came up that I would like to scratch. As I worked on the code, I needed to add a column. It would be great to be able to add the column right there in the PHP editor - some kind of right-click or keystroke to add the column that I have highlighted in my text editor, and a quick dialog comes up to get the details of the column.

Insert DDL Wizard

The Wordpress plugin I was working with does something I have heard a lot of people do: right there in the PHP code, detect if the tables exist and then create the tables (or alter the tables if you are doing an upgrade).

This is different from what I'm used to - I had always created the SQL scripts as separate files - but when you think about the web deployment model, it makes sense.

So, a nice feature would be to provide an Insert Code wizard to generate all the CREATE TABLE statements in your code editor. Generating the ALTER TABLE statements is trickier, but hey, let's set our goals high :)

What do you think? What else could we do to make it easier to work with databases in your language editor?

Stephen O'Grady » The Burden of Choice

A very nice blog, by one of my favorite bloggers.

most [open source] projects could use $5,000. What they could use more, however, is some advice on how to spend it.

http://redmonk.com/sogrady/2008/08/07/heres-5000-let-me-help-you-spend-it/

Monday, August 04, 2008

New plugin for PL/SQL editor in NetBeans

Alexandre Sumbatov was a participant in the NetBeans Innovator Grant project, and he just published the result of his work, a NetBeans plugin that provides the ability to edit and compile Oracle PL/SQL objects within NetBeans. Just download this plugin, and you can browse, edit, save, and compile stored procedures, triggers, functions, package specs, package bodies, type specs and typed bodies all within NetBeans. Take a look, try it out, and let Alexandre know what you think. And thanks to Alexandre for his hard work on this!

Friday, August 01, 2008

Harvesting Money in a Hungry World - Op-Ed - NYTimes.com

An important editorial, well expressed, but I suspect likely to fall on deaf ears. It will likely take massive starvation and uprisings to get anyone to pay attention to this situation.

http://www.nytimes.com/2008/08/01/opinion/01hanson.html?ref=opinion

Tim O'Reilly puts out the call to join the Federation and fight the Borg

I've been thinking a lot over the past few years about data on the web, and in particular open data. How can you build a system such that your data doesn't get locked in to a particular vendor. In his recent blog, Tim O'Reilly takes it a step further, and discusses his concerns about a Web 2.0 OPEC where three or four cloud vendors lock everyone in because they become a black hole for data.

There's a lot of very interesting stuff in his article, and is worth reading twice. Here's my favorite quote, a call to action:

If you care about open source for the cloud, build on services that are designed to be federated rather than centralized. Architecture trumps licensing any time.

He even refers to Prophet, a peer-to-peer database system I didn't even know existed.

With Prophet, it’s easy to build custom database applications and synchronize your data with your friends and coworkers – all without a central server.

Prophet supports arbitrary topology synchronization and has an advanced conflict resolution database which helps make sure that the entire database ‘network’ works its way toward consistency even when disconnected users make conflicting choices about how to resolve any given conflict.

This is way cool - those of you who've been reading my blog know I was thinking about this a while ago. And one of the reasons I think this is very important (and Tim does too) is it gets rid of the need for a centralized data repository like Google or Facebook. You can collaborate and broadcast your content without having to rely on Big Brother to hold it for you. That's fine, thank you very much, I'll keep it for myself.

So I'm definitely going to take a closer look at Prophet...

And here's to Tim's call to action: architecture over licensing! Join the Federation and fight the Data Borg!

jwz - Damned Dirty Ape Mask

From Tim Bray's twitter feed. Serious LOL funny, a MUST read :)

"A chimpanzee face stared back at me from the floor, impassive, but mocking."

http://jwz.livejournal.com/914926.html

Thursday, July 31, 2008

Manish Bharani: NetBeans Database Explorer gets a facelift

Manish provides a nice summary of what's new in the Database Explorer for 6.5. Manish and his team did a great job providing us with a lot of this functionality - the power of open source, even within Sun!

http://manishbharani.blogspot.com/2008/07/netbeans-database-explorer-gets.html

MySQL Workbench 5.1: Mac/Linux support, and a new Query Editor

Mike Zinner, team lead for the MySQL Workbench team, just posted some screen shots for their up-and-coming new version of MySQL Workbench for the Mac.

Note the screen shot above for the new SQL editor, which will replace the old MySQL Query Browser.

Now, some of you may notice, that looks awfully like the query editor in NetBeans.

Yes, there is some crossover, and we've been talking with each other. It would be nice if we could just wave a magic wand and all this database tooling from Sun were in a single tool. But it's not that easy, particularly if one is written in C and one is written in Java. And it's not clear that's actually what we want.

Sometimes one size doesn't fit all. We need to think carefully about two different types of users: those who want an IDE that works with multiple databases and is focused on serious application development productivity, and those who don't necessarily want an IDE or don't want NetBeans, or Java, but do want a top-notch tool for working with MySQL.

Anyway, we're still discussing, and I hope to pass on what our final decisions are once we have them. Regardless of where we end up, you as the user are going to win with both an excellent IDE and a most excellent database tool.

JavaFX Preview available today

A little bird told me the JavaFX preview is available for download today.

So I got the NetBeans 6.1 with Java FX download from their main page and gave it a spin. It is a preview, and it still has some rough edges, but with absolutely no knowledge of Java FX I was able to work through the tutorial and get something up and running pretty quickly, all within the NetBeans environment I know so well.

If you're interested in building rich client apps and want to see what Java FX is about, I recommend trying this preview edition, and get your feedback to the team.

Piracy is alive and well

Pirated DVDs and MP3s for sale in Cuernavaca

The picture above is from a shopping area in downtown Cuernavaca. There was a whole building with stall after stall covered from floor to ceiling with pirated DVDs and CDs and MP3s. It was a wild thing to walk through, with music blaring from either side.

As I walked out of the building, there was a policeman standing on the sidewalk just next door minding his own business. Obviously this was no concern of his.

I remember when DVDs came out, one of their big things was copy protection. Shyeah, that worked!

Wednesday, July 30, 2008

Leal’s Blog » Globo.com moves from Oracle to MySQL

"Globo.com is the biggest portal here in Latin America (newspapers, magazine, Television, and radio), and when the speaker said the line: 'We are moving from Oracle to MySQL', everyone was with the same face as you. :)"

http://www.posix.brte.com.br/blog/?p=151

NetBeans has compile on save - Roumen's Weblog

Very cool.

http://blogs.sun.com/roumen/entry/netbeans_has_compile_on_save

I got apps!

Well, I haven't splunked down the money and extra data usage fees for the iPhone 3G, but that doesn't mean I can't get my apps.

I just upgraded to the iPhone OS 2.0, and thus got the app store. An evening of browing and I got some fun stuff. I'd give you links but everything is available only through the iTunes store, so you'll have to go find it yourself. These are all free, I'm not ready to pony up for anything yet...

Labyrinth LE - remember that old marble game where you tilt the board to move the marble along without falling into a hole. Imagine doing that with your phone, and getting multiple boards to boot. Cool!
Phone Saber - different colors and special effects - you are an iJedi knight!
Apple Remote - use your iPhone as a remote for iTunes. Very nice.
Pandora and AOL radio - streaming Internet radio on my phone - say no more. Just give me Rhapsody and I'm a happy camper
Location-sensitive Yelp - cool!
midomi - Can't remember that song but it's stuck in your head? Hum it to midomi and it will identify it.
Location-sensitive pictures - there are a whole bunch of these to choose from
NearPics - show me pictures near where I am.
Google search - searches both the Internet and my phone (e.g. contacts). Nice.

Well, now we're talking. It's like a whole new phone. Although, as we all know, "phone" is not necessarily the right name for this, this, ... thing...

New NetBeans Tutorial: Creating a Simple Web Application Using a MySQL Database

Nice tutorial to get you started building a database app. I love the name of the application: IFPWAFCAD or the International Former Professional Wrestlers' Association for Counesling and Development. Snork!

http://www.netbeans.org/kb/60/web/mysql-webapp.html

Tuesday, July 29, 2008

Beyond Server Farms In The Sky - InformationWeek

Interesting analysis of what your choices are when you want to scale. I like the phrase "could computing" :)

http://www.informationweek.com/blog/main/archives/2008/07/beyond_server_f.html

Sun may or may not be about to obliterate Oracle and Microsoft | The Register

Warning: colorful language in link. But very funny description of transactional memory (TM), to be supported on Sun's new Rock chip. I have heard about TM for a while within Sun, and yes, there is a very good fit here for databases. 'Nuff said.

http://www.theregister.co.uk/2008/07/28/sun_dziuba_tm/

Monday, July 28, 2008

Kudos for NetBeans exception reporting

I've personally been impressed by the way NetBeans handles reporting an exception for you (and I personally have had a number of bug reports come to me this way, which I don't think I would have otherwise been informed of). It really does help improve the quality of the product.

But here is one user who I think describes the quality of this feature quite well:

So I used the Mercurial support in NetBeans 6.1 (as I have been for some
time), and something went wrong. NetBeans 6.1 did not crash, but recovered and saved the java exception for me, I viewed the exception stacktrace and it was obvious it was a crash inside the NetBeans 6.1 Mercurial support code. There was an option of reporting the crash so I hit the "Report" button to report the problem, then continued on with my work.

Then a few minutes later, I noticed that I had a Firefox tab open to a netbeans site... It said:

Upload of NetBeans Error Report Complete
Thank you very much for sending error information to our server.

After cross-checking your report against others recently received,
your report has been classified as a duplicate of report #64798.
Your comment and logging information were added to this one.

If you believe this issue is severe and you are registered with
netbeans.org, you can use the above link to check all the generated
information in our web application, and you can enter this issue
directly into Issuezilla. To see all reported issues, visit the
Exceptions page.

Thanks again for your help.

The NetBeans development team!

It automatically detected a matching java exception in a previously filed bug report, detecting the duplicate bug report.

So NetBeans 6 did everything right as far as I'm concerned:

1. The tool did not crash or stop or even have to restart due to the error
2. It allowed me the option of reporting the problem
3. It made it trivial to report the problem
4. It automatically submitted the report and data seemlessly and easily
5. It detected it was a duplicate and let me know it was.

Good work guys.

Kaj Arnö - Speaking in Tongues

I was having lunch with Kaj in Prague when he was practicing his speech from his notes, and the locals really were quite impressed. I have never given a talk in a foreign country, but I'm inspired to follow Kaj's example should the opportunity arise.

http://blogs.mysql.com/kaj/2008/06/19/the-why-and-how-to-of-localising-presentations-beyond-english/

Gmail and Google Calendar making the leap offline? » VentureBeat

Looks like Google Gears and SQLite are one of the definite winners in the offline RIA technology wars. Pretty cool that Wordpress is providing offline blogging, and I look forward to offline Gmail.

http://venturebeat.com/2008/07/17/gmail-and-google-calendar-making-the-leap-offline/

Back in the saddle, ruminating on the economics of "free"

I'm back from a nice long vacation, and as usual it was a time of reflection.

I would say that the most interesting and valuable ideas I gained were from reading an excellent book I found on the bookshelf of the home we were doing an exchange at, called The Undercover Economist by Tim Harford.

Mr. Harford layed out in plain and fun English some economic principles I hadn't heard of before that I am still chewing on, including the power of the free market to reveal the truth about what people really value - they put their money where their mouth is - and thus allowing the market to respond in the most efficient way.

But I also liked his descriptions of thing that regularly get in the way of a true free market. Corruption and influence peddling is one for sure. Not taking into account external costs, such as the consumption of resources and the dumping of waste, is another important factor. He argues that when you charge, or "tax", for external costs, it brings more truth into the market, and the market responds. An example that he cites and which I saw directly was the fees charged to cars entering into the center of London, and how that impacted pollution, noise, and transit services. It really is quite amazing.

After reading his book, I realized that in technology we often think something is or should be "free" when it's really not. For example, email has the appearance of being "free", but it actually has a big external cost, particularly managing spam. Companies spend millions of dollars trying to manage spam; spam management itself consumes large amounts of power and CPU.

If there were nominal charge for sending an email, say 1 cent per email, we would immediately throttle back the insane amount of spam being distributed throughout the system.

The problem is, not only is there no centralized point of control on the Internet, there is also no real identity. Would it be possible to set up a system where you are required to provide strong identity before you can access a service like email? You would think this would have been done already if it were possible...

I do think ultimately this problem will need to be solved, and I like the ideas of applying charges for external costs as a way to solve it.

Wednesday, July 16, 2008

Now this is a vacation

Well, it's wonderful here. We are thoroughly enjoying our visit. We are in a huge beautiful house, we are with a friend and her son as well as my wife's aunt, we are visiting places, hanging out, cooking delicious meals, going out for food, and just soaking in the sunshine and the fact that we left all of our many-layered obligations behind us for three weeks.

It was a lot of work getting our house ready, but it was worth it.

The picture above, by the way, is from Las Estacas water park. This is a park surrounding a river that bubbles up from the ground, and then you can ride down in an inner tube or a life jacket and look up at the incredible lush greenery or look down into the crystal clear water. Or you can swing from a rope and jump straight into the water, or wander through the lotus garden.

Ariel and the lotus garden at Las Estancas

To add to the romance and drama, every night there is thick moist stillness and crickets, then a rumble of far off thunder, then wind and lightning and BOOM a huge thunderstorm, power sometimes goes out, rain pours down in sheets, and the next morning all is clear again.

Did I mention the mangoes, the bunches of tiny bananas, delicious Mexican food, a massive kitchen, an inner courtyard, and the fact that our friend sharing this home with us is a chef?

What can I say? This really is the life...

Tuesday, July 01, 2008

Someone is having fun with me

We are getting ready for a wonderful three week vacation to Cuernavaca, Mexico. We are doing a home exchange with a family of four who have a home there. Their place is huge, has a swimming pool and a trampoline, and we are going to have loads of fun.

But then there is getting ready. We really want the place to work well for our guests, and in the past things have happened that were no fun, like the sink clogging up. And I guess Someone Out There is having a little fun, because in the weeks before leaving, we've had one thing come up after another, and for each one, I am so glad that it didn't happen to our guests:

Our sprinkler system got cracked and required a complete overhaul (yes, I did this myself, many trips to the hardware store for pipes and valves and glue and mud and getting sprayed in the face and...)

Our car battery ran out of steam and we kept having to get jumped (we're doing a car exchange too), so the car is at the shop today getting its battery replaced

The day before we were going to get its wings clipped,one of our cockatiels escaped and flew up into a neighbor's tree and wouldn't come down. We drove out there with the bird cage and the other bird, treats and a pillowcase. No luck, and this morning we had to try again. This time it was in another tree, and we finally coaxed it down with a millet stick (his favorite treat). We were very lucky. We lost our first cockatiel for good this way (these two are from an animal shelter).

We had mice and rats in our garage and our food was being raided. Took two weeks to get rid of them, I'll spare you the details

At 4:30 am two nights ago I awoke to a roaring sound. I stumbled downstairs to the kitchen and walked into a pool of water that spread from under the kitchen sink and all the way into the living room. The ominous roar came from under the sink, and taking a deep breath I opened it and water was spraying everywhere. Of course, the shutoff valve was at the back of the opening, so I dove in and spluttering and being smacked by water I got it shut off. Nice way to start your day... Linda and I spent the rest of the morning into lunchtime cleaning up and replacing the water filter, which at that moment had decided to give up the ghost in a dramatic way.

Can you imagine any of that happening to our guests while we were away? Or worse yet (with the water thing) happening when we had no guests? So as crazy as it's been, I actually feel blessed that all these things happened now, instead of later.

Monday, June 30, 2008

Theo Schlossnagle - Traffic spikes through the heart of your app

Theo Schlossnagle's blog is a very interesting read indeed.

What isn't entirely obvious in the above graphs? These spikes happen inside 60 seconds. The idea of provisioning more servers (virtual or not) is unrealistic. Even in a cloud computing system, getting new system images up and integrated in 60 seconds is pushing the envelope and that would assume a zero second response time. This means it is about time to adjust what our systems architecture should support. The old rule of 70% utilization accommodating an unexpected 40% increase in traffic is unraveling. At least eight times in the past month, we've experienced from 100% to 1000% sudden increases in traffic across many of our clients.

That's something to think about.

One likes to think that cloud computing is the silver bullet for scalability - use their stuff, build a system that assumes scale, and no worries, mate. But this makes it clear that there are basic operational issues that need to be kept in mind. Doing it manually may not cut it.

Amazon for example provides no tools for handling traffic increases automatically. It's your job to monitor and kick off new instances. But if this is happening in a matter of 60 seconds, you'd better be very sure you know what to do, and quickly.

Personally, I've never had to deal with this, so I have no tips for you. If I were to start up a new site, I'd go find people who do and have a long talk with them.

Friday, June 27, 2008

New DB Features coming in NetBeans 6.5

Some cool stuff coming down the pike for NetBeans 6.5 database tooling.

SQL Editor code completion - lots of you have been asking for this, and it's in, with more on the way. Note how completion brings up the columns if the SELECT statement already specifies the table name.

SQL History - You can now look at the history of all SQL you've executed, and can to text filtering and/or filter by connection URL, and then select and insert a statement.

Editable, sortable results, multiple result tabs

This isn't integrated yet, but I had to show it to you. With this you can sort the results by double-clicking on a row header (we should have done this a long time ago), modify a row, insert and delete rows, there is full pagination support, and you can optionally create a new tab for each statement that returns results, very nifty for comparing results back and forth.

This feature actually came from the SOA team under a completely different part in Sun - open source works inside as well as outside. Thanks to Ahi and Nithya and Nilesh for this!

Tired of typing in a JDBC URL?

Many of our users, and they're growing, aren't even doing Java, but when they want to connect to a database, the dialog is all JDBC-ish. We've fixed that - now you just type in the parameters and we figure out the URL for you.

It turns out this is no walk in the park, as each vendor has their own parameters and URL format. So for now it works for MySQL, PostgreSQL, Java DB, Oracle and Microsoft SQL Server. For the rest, sorry, you still need to type in the URL, until I can get to them.

BTW, if you still want to see and edit the URL, you can, and we'll update the fields automatically. Nifty.

Tuesday, June 24, 2008

Jerry Seinfeld on George Carlin’s Life and Comedy

I loved George Carlin as a kid. I once thought of a great joke - what is the worst ice cream flavor you ever heard of. I went to a Carlin concert, and he had the same joke, with the best answer: "bologna swirl." Carlin already did it.

http://www.nytimes.com/2008/06/24/opinion/24seinfeld.html?em&ex=1214452800&en=33d98b4f11507cbf&ei=5087%0A

Friday, June 20, 2008

Query languages for Hadoop, AND tooling for the non-relational stores

Tom White just blogged about three, count them three, different query languages being built for Hadoop.

Pig, from Yahoo! and now incubating at Apache, has an imperative language called Pig Latin for performing operations on large data files.
Jaql, from IBM and soon to be open sourced, is a declarative query language for JSON data.
Hive, from Facebook and soon to become a Hadoop contrib module, is a data warehouse system with a declarative query language that is a hybrid of SQL and Hadoop streaming.

As someone working on database tooling, I have been thinking about what we at NetBeans might do to make it easier for developers to build apps against these new platforms/products like Hadoop and CouchDB and Amazon services.

These query languages are an example - I can see an interactive query editor that lets you write and test queries directly against your Hadoop or CouchDB engine, much as you can today with SQL tools.

Another interesting question is - if I write my application in terms of domain objects, what things can a tool do to make it easy to map these domain objects to an underlying non-structured store like Hadoop or CouchDB or SimpleDB.

Definitely some opportunities here to make developers' lives easier.

Thursday, June 19, 2008

Kaj Arnö - Happily using NetBeans to build MySQL

After getting a demo from a NetBeans engineer on the C/C++ extensions to NetBeans, Kaj was converted - "now this was enough even for a sceptic like me to become eager."

http://blogs.mysql.com/kaj/2008/06/18/netbeans-as-ide-for-developing-mysql-itself/

Alex Bunardzic » Software Development Detox Part 5: Session

Someone who is in my boat about session. Although I still wonder if you are overburdening the client if you say "on the web each request contains all the information necessary for the server code to make a decision on what to do next."

http://jooto.com/blog/index.php/2008/06/20/software-development-detox-part-5-session/

Wednesday, June 18, 2008

Alex Bunardzic » World Wide Web is About Self-Serve

Interesting blog challenging us to question if there is a need for services, when really the web is basically about resources. I think the real service is in quality of service, making sure the resource is available when you need it.

http://jooto.com/blog/index.php/2008/06/19/world-wide-web-is-about-self-serve/

Monday, June 16, 2008

AOL Radio

I have been a long-time member of the "AOL sucks" club. I signed my name in blood as a member of this club when I tried for three days to get Internet access working for my laptop in someone's home that had AOL as the Internet provider. AOL has historically provided a horrific user interface which constantly thinks it knows better than you and gets in your way whenever you want to do anything different from the standard.

But I've found one piece of AOL that is just, what can I say, nice... It's AOL Radio, which I bumped into when reading a NY Times article about how AOL Radio will soon be available on the iPhone. I tried the Acoustic Rock station, and it's really quite high quality, and the user interface, is, well, simple and - gasp - easy to use!

I have tried a number of interfaces for free Internet radio, and they are either impossible to use, have a horrible selection, or are filled with ads. AOL Radio has none of these limitations. And all you need is Flash to get going. They do seem to have song-skipping disabled, but that's supposed to come back soon.

Pretty nice, and from such an unexpected source! :)

Friday, June 13, 2008

Facebook's Thrift in Apache Incubator

From the Facebook blog:

Thrift is a lightweight software framework for enabling communication between programs written in different programming languages, running on different computers, or both.

It includes support for C, Java, C++, Ruby, Erlang, Perl, Haskell and many others.

It sounds very interesting. I first heard of Thrift because it's used by a very interesting Amazon-like stack called Thrudb. What's nice is because it works with Thrift, you can use it from any language.

However, I looked at the interface for calling the Thrift APIs, and it's a bit ugly, or at least it takes some getting used to.

Anyway, something to keep an eye on, it may have legs.

Thursday, June 12, 2008

Rhapsody on my Mac

I really like the Rhapsody music service. I have been trying various ways to get this to work on my Mac.

There is a plugin for the browser that lets you access Rhapsody natively on the Mac. The only problem is that it regularly crashes Firefox, and Rhapsody support clearly has fixing this as a very low priority. I tried Safari, and that was working OK, until I upgraded to OSX 10.5.3, and now Safari regularly crashes my entire machine! Sorry, no go.

So I finally gave in, and am happily running Rhapsody natively on my Win XP VMWare instance. I reduced the memory on the instance to 512MB otherwise if I have two instances of NetBeans up my system memory starts thrashing. But once I did that I am happy as a clam. Mozart is lilting to me as I code, and all is well with the world.

Tuesday, June 10, 2008

Open Source EC2 - the beginning of a Scale Stack

In my last post I mentioned that what I wanted to see was the industry coalesce around an open source "scale stack" [1].

We may be seeing the beginnings of this. The High Scalability site (great blog, by the way) recently posted a blog talking about a new kid in town coming out of UC Santa Barbara called Eucalyptus . They are providing an open source implementation an elastic compute infrastructure that is interface-compatible with Amazon's EC2 which you can take and deploy on your own hardware.

This is very encouraging, and I think is a smart approach. Rather than try to build some standard that is lost in committee for years, use the de-facto standard, which in this space is Amazon.

Another piece of the puzzle is Hadoop, an open source implementation of map/reduce. Hadoop also has a distributed file system - one thing that might be worth investigating is building an S3 layer on top of Hadoop's file system.

What about the queuing service? Well, one possibility is to but an SQS API on top of ActiveMQ or OpenJMS.

Throw in CouchDB, and you're starting to get a very interesting stack indeed. I'm not sure about putting a SimpleDB interface on top of this - CouchDB is pretty darn interesting in its own right, and I think the jury is still out on SimpleDB.

[1] I am not sure if he wants me to mention his name, so I won't, but I want to acknowledge that the idea for an open source stack based on Amazon's APIs is not my own, but comes from a colleague at Sun. I think it's a great idea, and may it come to fruition.

Thursday, June 05, 2008

The exponential cost of contention

I enjoy Nati Shalom's blog, although it always has that taste of having the agenda of pushing Gigaspace's solutions. But putting that aside his posts are always well thought-out and well written.

I think his latest blog on the Economies of Non-Scale really drive some points home about scalability, or more to the point, the cost of non-linear scalability.

If 90% of our application is free of contention, and only 10% is spent on a shared resources, we will need to grow our compute resources by a factor of 100 to scale by a factor of 10! Another important thing to note is that 10x, in this case, is the limit of our ability to scale, even if more resources are added.

...

1. The cost of non-linearly scalable applications grows exponentially with the demand for more scale.

2. Non-linearly scalable applications have an absolute limit of scalability. According to Amdhal's Law, with 10% contention, the maximum scaling limit is 10. With 40% contention, our maximum scaling limit is 2.5 - no matter how many hardware resources we will throw at the problem

That's something to chew on. These are real costs, both to your business, to your users, and to the environment. Even if you only have a teeny 10% contention in your system, that 10% will nail you faster than you can say ACID semantics. And as has become very clear to me the final breaking point, the final point of contention in any traditional web application architecture, is the database. Get rid of that and you're home free.

How do you do that? Well, there are a lot of people trying to solve this problem with things like space-based architectures, eventual consistency, distributed map/reduce and Stonebraker's H-Store architecture. Anything to let each instance stand on its own and not have to serialize with the rest of the system at any point, in any form.

Some people argue that scalability is so hard that you shouldn't think about it until you need to. But I really believe that if you do enough to educate yourself and make some wise choices, you will be very glad you did.

What I'd like to see is the industry coalescing around some best practices, an open source "scale stack" ala LAMP, tools, community, and hosting environments like Amazon, that allow developers to easily build applications that will scale from the get go. That's where I want to see things go. That way you don't have to throw your hands up and hope for the best. Because as you can see, the costs can be deadly.

Something to think about...

Tuesday, May 27, 2008

Greed and vision

I was listening to NPR this morning to a fascinating analysis by someone who has talked to everyone from homeowners to lenders to Wall Street about the home loan fiasco. He said everybody up and down the chain was taking unreasonable risks, and everyone was making tons of money off of it.

Then he quoted a financial expert, I wish I could remember who, who said

It's amazing how hard it is for someone to see something when they are being paid a small fortune not to see it.

I just had to laugh, that really sums it up.

Monday, May 12, 2008

It isn't because you're dirty, it isn't because you're clean

My daughter is currently on antibiotics, and she's staying home from school because she has a sniffle.

Why is such an innocent sniffle the cause for keeping her at home, and why is she on antibiotics? Pertussis, or whooping cough.

My daughter goes to a private school, so vaccinations are optional. Many chose not to vaccinate for pertussis (this is actually one we did), and these are the consequences. The Department of Health for the county closed the school for the day on Friday, and is requiring that everyone, regardless of whether they've been vaccinated or not, either take a 5-day protocol of antibiotics or stay home for 21 days. They also have implemented a no-symptoms policy, so anybody with any symptoms, including a sniffle, has to stay home. Sorry, Ariel.

So, swirling around this is the whole debate of vaccinations. As a father, I was stunned at the number and frequency of vaccinations we are supposed to give our children, and which you are required to have before you can enter public schools. I understand, this is all in the name of public health, but as a parent you have to use your intuition a lot, and this just intuitively feels like overkill, literally. It reminds me of the latest discoveries I read about in Science News where there are bacteria in dirt that, when inhaled, create a very strong immunity. Children who are kept away from dirt and kept super-clean are more prone to disease.

I understand vaccinating against horrific and fatal diseases. But many of these vaccinations are for diseases that many of us older folks had as a child: chicken pox, measles, mumps. It was just part of growing up. And there are some real issues here. I am no doctor, but as I understand it, if you are vaccinated against chicken pox and never get it as a kid, then you have a chance of getting it as a teenager, and then it can be *really* bad, whereas if you get it as a kid, then you get a much stronger immunity and generally don't get it when you're older.

I understand then challenge of maintaining public health over trying to protect your child from an army of doctors with needles. I mean, I would be horrified if my child or someone else's child died because of my unwillingness to vaccinate. But at the same time, I wonder what is truly the right thing to do here. We are often so focused on eliminating all and any suffering from this world that we fail to see the ultimate consequences. As an example, part of the reason the plague pounced upon Europe so badly in the middle ages is because humans had been so successful at eradicating the wolf, which, you guessed it, was the primary predator for rats.

I couldn't tell you what the unexpected consequences may be for so much vaccination, but I can tell you that it feels like overkill, and I personally recommend that as a parent you do your research and try to make informed choices about which vaccinations you give your kids, rather than blindly doing none or blindly following the protocol laid out by the powers that be.

Friday, May 09, 2008

MySpace to share data

This just in:
MySpace is going to allow users to share information from their site.

This is definitely a step in the right direction, but notice that sharing is done either by copying/synching, or through their specific APIs. It doesn't fully extract the content into a separate layer with application-provider-agnostic service APIs.

One can imagine an OpenSocial API on top of a social network data service that is used by numerous providers but owned by none.

Giant Food & Biotech Corporations Make Billions in Profit from Growing Global Food Crisis

Big Food is serious big business, and thus is inherently evil (profits over social needs, always - the corporate credo).

http://www.organicconsumers.org/articles/article_12088.cfm

Thursday, May 08, 2008

Internet content by reference, not by value

You may have noticed that I am very interested in how data is managed on the Internet as a platform, at a web scale. In that light, I have been having some very illuminating and interesting conversations with an old friend and colleague, Nitin Borwankar. His thoughts on data property rights and DRY data are concepts that if implemented could result in a major shift in how we manage data on the web.

Data property rights is about laying out a "bill of rights" for data that goes far beyond "the right to move". It also includes the right to access, modify, remove and own your data. So often it happens that once you upload your content to a site, you no longer have full rights to that content, as if somehow in the act of uploading it it is no longer yours. It's like living in a serfdom where you do all the work to plow, seed, tend and harvest the land, but the fruit of your labor is not yours, just because you are using the land that someone else owns.

DRY data is about following the principle of Don't Repeat Yourself for web content. Web applications need to start applying this principle, so that rather than you having to load copies of your content across multiple sites (and losing ownership of it in the process), you place it in one location (your "home" on the Web, as it were), and then you refer application providers to that one place. They can focus on providing added value (for instance, referring it to your friends, enabling collaboration, or helping you organize it or present it in useful ways) rather than on the overhead of building and deploying a scalable storage architecture.

Nitin calls this architecture YINAS (YINAS Is Not A Silo).

The value of DRY for the user is obvious - I only have to put my stuff in one place, and I get to really own my stuff, rather than the vendor owning it. DRY is also very valuable for the vendor, as they can save overhead and complexity by delegating the work of scalable storage and indexing to a "data service provider" rather than having to do it themselves. It's even good for the environment, because you need fewer disk farms sucking up power and space. I guess the only folks who would lose out are the storage and power vendors :)

It's funny, it makes so much sense, but nobody is really doing this.

I pulled Tim Bray aside at Java One to talk to him about these ideas after reading his blog about changing his address, and he suggested that concepts are good, but a simple proof of concept is better. Hm... let me think about that ... :)

Monday, May 05, 2008

Meeting Good Friends at CommunityOne

Lots going on these days, hard to keep up. I have been particularly busy, and that's why I've been quiet. I suspect you probably don't mind too much, dear reader, because if you're like me, you probably experience information overload on a regular basis.

I was at CommunityOne today, and, as seems to be happening these days, I spent most of my time saying Hi to old and new friends catching up and sharing our thoughts and feelings about what is going on in the industry.

I ran into Ted Leung and he shared some of the work he's doing to try and get some of the powerful OpenSolaris features like DTrace working in the various dynamic language environments.

Simon Phipps was hanging out (literally, his feet dangling from the RedMonk UnConference stage during a break) and we ruminated on the fact that Sun seems to be held to a higher standard than many companies in the industry - people see us as a Good Company and so they get very upset when it looks like we're being Corporate and Just Like Everyone Else. There is a sense of betrayal and lost hope. So in a way, it's a positive reflection on Sun - we in general have been good to the community, and on the up and up, so any indication that we're Falling to the Dark Side creates a passionate response.

I bumped into Joe Keller, who was one of my first VPs at Sun when the little company I was at was acquired by Sun. Now he's working for Kapow! (now there's a name), and it's fascinating what they do. He described it, and here's how I interpret it: they're doing data integration at Web Scale. Their customers define a master database schema, look at the web pages that they're interested in (say governmental pages for each state in the USA), and define a mapping between the HTML elements on these pages and the elements in their schema. Then the Kapow! software does its magic, scraping these screens at various semantic levels (screen, HTML, through APIs, etc.), and slurping this data into their database schema. Customers then use their data warehousing solution to analyze the resulting data. As an example, one customer of Kapow! is the company Sun hires to do background checks on potential employees. Yipes!

I saw my old boss Dan Leighton, now an independent consultant, but who was instrumental in setting up the database group and defining our database strategy, working for Rich Green, which ultimately led to our involvement in Java DB, PostgreSQL, and, of course MySQL. So he was happily surprised to hear of Sun's acquisition of MySQL.

I bumped into Rob Stephens, someone else who came from the Clustra acquisition and is now in the Solaris org. He told me I should check out the new release of VirtualBox, the open source virtualization solution whose parent company, which was recently acquird by Sun. Rob's got a very strong sysadmin background, so when he says it's good, I trust him. I had tried out VirtualBox about a year ago, and he says it's really improved. Downloaded, installed, and now pulling down the first GA release of OpenSolaris and will put it in its virtual box and put it through its paces.

I ran into Francois Orsini, who is demonstrating some very interesting stuff with Java DB and Grizzly. I ran into a Sun colleague working on a big Rails project, and we talked about web platforms (Rails vs. PHP vs. Java), scalability, and the place of the relational database in a platform that needs web scale. All very interesting stuff. We'll see how the day goes tomorrow!

OpenSolaris on Amazon EC2

Wow, great news. OpenSolaris has some very powerful technology, particularly DTrace and ZFS. Having this available as an Amazon Machine Instance (AMI) on EC2 is sweet!

http://www.sun.com/third-party/global/amazon/index.jsp

Thursday, April 24, 2008

Windows Live Dev : Introducing Live Mesh

Very nice, very interesting stuff. This is in line with a lot of what I've been thinking about. Too bad your stuff has to go onto *their* servers and is tied to a very proprietary company. Imagine doing the same thing in an open source model.

http://dev.live.com/blogs/devlive/archive/2008/04/22/279.aspx

Tuesday, April 22, 2008

Paul Graham's Business Strategy: Be Good

"Being good is a particularly useful strategy for making decisions in complex situations because it's stateless... it's the only algorithm that works on [a large] scale." Great article.

http://www.paulgraham.com/good.html

Wednesday, April 16, 2008

Seeped in MySQL

This is day three of the MySQL conference for me - usually when people hit burn-out, in my experience. And I'm definitely getting there - I left the Sun party early, my tongue and ears tired of talking and listening.

I have met a lot of great folks here - I was on booth duty a lot and people just kept coming by. This includes saying hello to our new colleagues from MySQL, hello to my old buddies in the database group, and also hello to very good old friends from my Sybase days.

Just as an example, I ran into Robert Hodges, who I used to sit near in the B-Treeum (our building next to the Atrium) in Emeryville. He was the lead on the message queue product, and I led the app server team. Now he's the CTO of Continuent.

Hi, Robert!

Robert Hodges, old friend and CTO of Confluence

I also ran into Wen Hu. I worked for him when he was the lead architect for a massively parallel database engine Sybase was working on with Teradata called Navigation Server. Haven't heard of it? Oh well. Then he moved on to implement the partitioning and parallel query support for the Sybase database kernel. I think MySQL is just now implementing something like that. Now Wen is the CTO for ScaleDB. Sorry, no photo of Wen.

Both of these guys are working on scalability solutions for databases. That is the one theme that repeated here over and over: scalability. All these web properties are encountering scalability requirements that actually go way beyond enterprise scale requirements. You basically need to be able to scale, well, infinitely, and linearly. It's amazing to me that this tiny database engine originally built for quick departmental web apps is now having to address requirements that go way beyond anything Oracle or its peers from the 1980s was built for.

And all the folks I talk to agree, standard transactional/relational approaches just ain't gonna cut it. Something has to give. There's even an acronym for this that I just learned at this conference: CAPS - Consistency, Availability, Performance, Scalability[1].

As Robert explained it at his talk, these are four qualities, and you can't have all of them at once. Something has to give. For a lot of folks, what they are most ready to let go of is consistency. For example, if I write something, I want to see it show up right away. But another reader may not see it for a while (we're talking on the order of seconds, maybe a minute). And that's fine. You get this if, for example, you do asynchronous replication to multiple nodes/sites, rather than trying to do consistent, synchronous replication. And as you can imagine, asynchronous replication scales much better than synchronous replication.

This is the model Amazon has taken, to great success, and I believe it's time for this to spread to the mainstream. The important thing (again, thanks to Robert for crystallizing this for me) is that the levels of consistency need to be clearly defined and guaranteed, like the transaction isolation levels.

I talked to some MySQL folks, and they seem to be looking very seriously at this issue of CAPS, and at these alternate storage solutions like Hadoop/BigTable, SimpleDB and CouchDB. So I look forward to seeing if and how the MySQL folks address this growing need.

[1] I have searched for a reference to CAPS on Google, and can find nothing, so it must be something new. If anyone has a link, please let me know.

Phanfare, S3 and Google Apps

I'm looking very seriously at moving to Phanfare as a replacement for Flickr and YouTube for my photos and videos. Their video quality is much better, and I can use it for video archival as well as display. My friends and family can even download the videos at DVD quality and play them offline or even burn them to DVD. Now that's cool.

The cost: $59 a month. Worth it in my book, and since I'm paying up front I'm pretty sure I don't have to worry about advertising polluting my content. Hey, waddya know, a different business model than free-with-ads, which I am so tired of.

I also like it that they're on S3.

They also have a good blog. I particularly liked their analysis of the benefits of the Amazon Web Services over Google Apps. I think they provide a great testimonial for the value of following open standards - something Sun has always believed in.

Friday, April 11, 2008

This just in: Google Docs offline

I was putzing around with a meeting agenda on Google Docs and noticed an "Offline - New!" link at the top right.

Sure enough, it prompts you to download Google Gears and you can work with your documents offline.

Killer feature. Ouch to Microsoft.

Tuesday, April 08, 2008

I can see clearly now...

Well, a computer career and age have finally gotten the better of me.

The odd thing has been that people haven't really noticed. I was at three different meetings yesterday in the office and each time I had to mention my glasses and then people would say "I thought there was something different about you!"

For me, a huge change, these things always sitting in front of my face, I can't believe people don't even notice. But there you go...

Dooth Booty at MySQL Conference next week

Off to the MySQL conference next week. I still have to remind myself that MySQL is now part of Sun - it's so easy to think of "them" as a separate company that we're just partnering with. It really is something that takes a while to get your head around.

I'll be doing a ton of dooth booty, um booth duty, on Tuesday and Wednesday. Come to the NetBeans booth in the morning and I'll be there, and maybe I can get you a T-Shirt. XXXL of course :)

We'll be showing off what you can do with NetBeans and MySQL, and I also would love to talk to folks about what you want for database tooling in an IDE (and what you don't want).

I'm looking forward to hearing from Mårten Mickos and Jonathan Schwårtz. It should be very interesting to see where these guys see MySQL going next. And then there is Werner Vogels, my "hero" of eventual consistency. Should be good.

I also hope to meet a number of my colleagues from MySQL as well as dear old friends from the Java DB, PostgreSQL and HADB teams (which are now part of the New Database Team).

It should be a time of good information, good friends, and good cheer.

Who is the Dick on My Site? - Dick Hardt

I am just noting one of the best talk titles I have heard in a while. This one is by Dick Hardt on web identity. I'm going just to hear this person who has the cojones and sense of humor to publish such a title for a keynote.

http://en.oreilly.com/mysql2008/public/schedule/detail/2460