Friday, September 28, 2007
Update: Google Reader appears to be affected too, I get nothing but the page header...
Wednesday, September 26, 2007
Most professional programmers know how it feels to see an amateur, unfamiliar with Knuth or any programming book containing equations, implement their own sort routine. That’s how people who understand relational theory feel when they see a badly-designed database. Relational theory and RDBMSs are old and well-established now, so it’s hard not to think a lot of programmers are willfully ignorant.
I have seen this too: "I don't need a friggin' relational db, just give me a flat file." Sometimes this works, but all too often I see the same programmers adding locking semantics, querying capabilities, indexes, and so on, and I just shake my head.
Tuesday, September 25, 2007
I looked for some fancy schmancy network card in his PCMCIA slot, but nothing there. Then I thought, what the heck, and what do you know, and I looked for wifi signals. What do you know, there was an ssid called ac_transit_wifi_bus.
Whoee! It's a nice fast signal too. So here I am zooming across the Dumbarton Bridge and able to work online.
View Larger Map
Now if they could only get it on BART, I'd be a very happy camper.
Monday, September 24, 2007
So that means that if you include an image in your blog, you are copying it, and you have to have rights.
Note that putting in a link to an image can be interpreted as copying, and even if it isn't, some people don't like it; it can also be a big impact on the hosting site for the image, see Tim Bray's blog about this.
Many Dear Readers probably already knew about this, but there is an entire site called the Wikimedia Commons which dedicated to media that you can safely copy, normally licensed under GPL or Creative Commons. The other thing I do is use Flickr, and search for images that have the right Creative Commons license.
By the way, if you ever want to include some of my Flickr images, go right ahead, I've licensed them using basic "Attribution Creative Commons" -- all you need to do is include an attribution when you include. Although I don't know why you'd want to include pictures of my baby son or grainy pictures of Prague.
Friday, September 21, 2007
A very common complaint has been the fact that NetBeans quotes everything when it comes to SQL. When you use the Database Explorer to create a table, index, view, etc., it quotes all the identifiers.
This may or may not be an issue. If, for example, you use all upper case for your identifiers, and the database you are using follows the standard and auto-upper-cases all identifiers, then if you write SQL without quoting, things will work. But how many of us bother to consistently upper-case things?
Here's what can happen to you in NetBeans 5.5:
Well, as of Beta 1, we don't do that any more. We do not quote identifiers when creating SQL objects. If you want them quoted, you can do so when you type in the identifier in the wizard. So now you get the right behavior:
When Beta 2 comes out, we have also made it so that when we generate SQL for you (for instance, when you choose "View Data" on a table, or when you us the visual query editor), we only quote when needed, whereas before we quoted every identifier, whether it was needed or not, misleading users to think they also needed to quote everything, and making the SQL difficult to read.
So, for all of you who have been complaining to me and others on our team about this: you're welcome. Chocolates are always appreciated :)
There have been some interesting comments, and together I believe they give a richer view of the topic that may help you make a more informed decision around the whole matter.
Tuesday, September 18, 2007
One of the things that really sucked with the previous version of the app server was that their session state replication mechanism was completely broken. It would corrupt data or get into deadlocks, and couldn't scale beyond three or four nodes.
This was my first exposure to the demands of session state and how it impacts the back-end architecture. HADB is attractive because it, basically, never goes down, and it is transactional, so it would solve the reliability and scalability problems of the previous implementation.
The more I learned about session state, the more I disliked it. The impact to a system trying to be scalable and available was enormous.
First of all, load balancers have to be able to detect the session id and make sure that a request for an existing session was routed back to the same instance. Only certain hardware load balancers know how to do this, and it requires digging into the HTTP message to pull out a special cookie that tells you what instance owns the session.
Secondly, application servers have to be able to handle all this session state without using up all the memory of the system. So EJB invented this marvelously ugly word, "passivation", by which session state is written out to disk and dropped out of memory - in other words, a virtual memory system for session state.
And then there is the impact on the complexity and scalability of clustered systems. Because you never know when a given instance may go down, session state has to be relocatable to any other instance in the cluster. Trying to build something that does this correctly is complex, painful, error-prone, and, to be honest, annoying.
For the end users, you end up with systems that are always limited in some way or another. If you store session state in a shared database, this impacts response time, and this database is another moving part you have to administer. And it had better be highly available and scalable otherwise it can only handle so many instances -- ultimately the database itself impacts scalability.
If you use in-memory replication, then either you have to make the state available to all instances, which means lots and lots of data flying around the network, constraining scalability, or you have to pair up instances. This is dangerous, because if one instance goes down, the other instance has to handle all the workload of the existing sessions of the downed instance -- you end up with hot spots and if you're not careful, a wonderful cascading failure scenario.
There is a giant sucking sound of processing power, time, money, and intellectual resources being used up on this problem, all because someone wants to store a shopping cart in HttpSession.
All of which leads me to conclude: session state is evil.
When I read about the principles of REST, I was really taken by it. One principle that really stood out for me was: stateless. HTTP was never meant to be stateful, and for good reason. Imposing state on this protocol is a bastardization of its original intent.
Rather than share my potentially confused view of this, I'll quote Roy Fielding directly from his slides on REST that he gave at RailsConf:
- A successful response indicates (or contains) a current representation of the state of the identified resource; the resource remains hidden behind the server interface.
- Some representations contain links to potential next application states, including direction on how to transition to those states when a transition is selected.
- Each steady-state (Web page) embodies the current application state -- simple, visible, scalable, reliable, reusable, and cacheable network-based applications
- All application state (not resource state) is kept on client
- All shared state (not session state) is kept on origin server
So stop asking the server to "hold on to things" for you. Please. If you do that one simple thing when you build an application architecture, you have freed the server infrastructure; you have given it wings.
Today it is easier than ever before to do this. With AJAX and RIA toolkits like JavaFX (shameless plug), you can provide dynamic interaction with the user without having to ping the server all the time and have it store conversational state. You can even use Java DB or Google Gears to keep your client-side state persistent.
Because of these new web client technologies and patterns, I don't believe the concerns that saving state on the client necessarily results in more network traffic. You change your interaction with your server from lots of little requests with little bits of data to much less frequent requests with more data. In general, because of the overhead of a network request, I believe this is a Good Thing.
So, I know it's tempting to use session state. I know it seems easy at first. I know that all the server products out there give you utilities to do it and talk it up (they're even talking about letting you keep even more state around). But Just Say No. Don't Do It.
I have seen where it takes you, and it ain't pretty.
 I have a funny story about the first time I met Roy Fielding. I was at the Derby booth at ApacheCon, and Roy had come over to learn more about it. I didn't know him from Adam, and I asked him why he was interested. He said he was starting a new project in Apache to do document management.
I said with a big smile "welcome to Apache!". He looked at me for a second and said, "well, actually, I've been around Apache for many years..."
It was only later that I discovered that he was one of the founders and is currently a VP at Apache :)
Monday, September 17, 2007
Glassfish Version 2 is out. And what a release it is.
The first release was focused on single instance deployments and developers. This release throws in the works. It has the features you need to run this puppy in an enterprise environment: clustering and HA (including a new in-memory replication feature using JXTA), serious administration and console, record-setting performance, all in a single all-in-one 50+MB bundle rather than confusing users with three separate versions.
I think it's worth looking at how far this team has come and what they have accomplished by looking at a little history (from the biased perspective of yours truly :)).
When I first joined Sun, I was part of Clustra, which became the Database Technology Group here at Sun. We came with a modest little 5-9s highly available, highly scalable RDBMS that was named HADB (Highly Available Database).
I was asked to architect a clustered version of the app server that used HADB as the session store, so I was working very closely with the app server team.
At this time the team was in the middle of completely redoing the app server to move away from the basically overwhelmed and broken C/C++-based codebase inherited from, if I recall, three different app server acquisitions: Netscape, Kivasoft, and NetDynamics.
They were redoing everything by taking a copy of the Reference Implemention, built in 100% Java, and making it "production grade" in terms of features and performance.
It was a difficult and ugly time. The two teams (RI from old Javasoft and the Netscape/Kiva/AOL/you-name-it team) were not getting along very well, and emotions were high. We used to call the Java team "Church" because they were such sticklers for compliance and elegance and we called the production team "State" because they just wanted to get something that customers wanted, that performed, and that was delivered on time.
The processes in place at the time were, well, shocking. They regularly could not get a build to work, and the performance was getting regularly worse instead of better. The product was months late and we had nightly pow-wows with executives to track a dashboard where most of the line-items were red.
I remember trying to get a copy of the codeline and build it. But things were so non-open-source that you had to be on a machine within the Sun network that could auto-mount the necessary drives that contained supporting binaries and infrastructure. In other words, given that I worked from home, I couldn't build it on my machine, and I couldn't find a machine that anybody would give me access to that had these mounts. Pulling down the cvs tree took hours, and then my attempts to build got regular failures. I finally gave up.
After a year, I moved on, and started working on Apache Derby. I have to admit it was a relief...
Since then, I have watched with joy and amazement what the app server team has done.
I have watched them go through the painful process of merging the two codelines (RI and Sun Java Application Server based on the RI) and merging the two teams.
I remember them making the bold decision to take this massive code base and open source it, even though there was already a market-leading open source implementation (JBoss).
I saw them use their blood, sweat and tears to deliver an implementation of Java EE 5 immediately after the release of the spec.
And I started noticing incredible improvements. Getting and using Glassfish has become easier and easier. Today I can just download the bits, put it in a directory, run a simple install script, and voila, everything works. I started hearing users and reading blogs saying, basically, "hey, this ain't half bad. As a matter of fact, it's pretty cool!"
I started feeling proud of our app server.
Then I saw a demo last year of their new modular architecture (coming in version 3), with a sub-second startup time and intelligent auto-loading of modules, and I was just astonished. This was the same app server that I had worked with four years ago? That day over lunch, a Glassfish competitor sharing with me when he saw that demo, he knew that they were in serious trouble.
So, many, many kudos to the Glassfish team. You have done an incredible job, and I am proud of you.
Friday, September 14, 2007
It was also my experience that the DBAs ruled the database. They had Absolute Control, and this allowed them to fine-tune the performance of the database. In particular, they didn't let Just Anybody write SQL. That was for the masters. "Tell us what you want, and we'll write a stored procedure for you. And we'll tell you how to interact with the database. (And a box of chocolates would probably help you get it sooner)."
Just around the time I left Sybase, web applications were taking off. I didn't pay attention as much, but I noticed that more and more applications embedding SQL directly. And I wondered what happened to the mantra of stored procedures.
I have also been watching the growth of ORM technologies like JDO and JPA. And I always wondered -- how do all those DBA-zis I knew were out there in the field feel about this? I found it odd that you can't map objects to stored procedures or views, just tables. What happened to centralized control?
Well, Diego Parrilla explains where things are going , including a hilarious imagined dialog between the DBAs, an IT manager and the developer lead.
IT Manager: Why this [ORM] tool writes crappy SQL?
Development Lead: It does not write poor SQL. It creates simple queries, that's all. We can configure it to create more complex SQL, but sometimes the number of objects explodes and the application run out of memory.
DBAs: Why don't you use the already created Views?
Dev Lead: We cannot map Views to objects.
DBAs: No views!? How am I going to optimize your complex queries?
Dev Lead: Well, we are not going to write complex SQL anymore. The ORM will do.
DBAs: And what about triggers and stored procedures?
Dev Lead: Out. Triggers and ORM does not match very well because it's hard to keep under control the changes performed in the DBMS. And Stored Procedures sucks, we have Java.
IT Manager: So, if there is no views, triggers and stored procedures, the DBAs can set your focus on keeping the system healthy and optimized, but not coding processes. Right?
DBAs: Errr... that's not exactly right...
IT Manager: And we can also transfer all these development tasks to the development team. Right?
I do see these kinds of debates going on the forums from time to time. An old-timer says "you have to use stored procedures to get any performance." But their claims are lost in the noise of all the people building ORM tools and working with data caches. The value of ORM and caching is too great, and stored procedures end up taking a back burner.
One fascinating conclusion Diego makes: as the business logic and data processing (through caching) goes more and more to the application server tier, then the demands on the database are less and less. Maybe it's fine to use basic, good-enough databases.
IT Manager: Should we try an open source database?
From my father, who regularly provides great commentary on current topics:
Today the Kurds announced that they have signed a contract with Hunt Oil Co of Texas to produce and market their oil. The idea of a national oil-sharing agreement for a united Iraq is finished. Since oil is all Iraq has got, a united Iraq is finished.
The real significance is that Mr Hunt is not only a very close personal friend of "we are winning" Bush, but he also sits on the Council of Economic Advisors. If any businessman knows which way the wind is blowing, and what's OK with the White House, he does.
No WMD. No Saddam-Osama link. No real reason to invade Iraq. Just oil deals for the Friends of Bush, while Dubya does the necessary fan dance.
Thursday, September 13, 2007
I don't watch TV and rarely watch movies, and I don't use, or want to use iTunes or an iPod. I mostly want to play Rhapsody music and my own collection from my stereo. Simple needs for a simple man.
Internet Audio Player
I had been dreaming for years about getting a Squeezebox.
It sounded simple, great quality, and people rave about it. The fact that their server, SlimServer, is open source, is important to me too. I know that a popular open source product is almost always better than proprietary solutions that rely on the quality of engineers from a single company.
Then when I learned that Squeezebox is now officially supporting Rhapsody (straight from the box, no need to even have a computer on), I was sold. So, I took the bonus money from last quarter (thanks, Sun!) and splurged.
Man, this thing is sweet. It is so small, cool looking, great screen, and so easy to use. The response time when I do searches and start a song are excellent. And the sound quality is very very good.
Not only does Squeezebox support Rhapsody, it also supports Pandora, and a huge selection of Internet radio stations. All from one little box.
But wait, that's not all! The SlimServer is web-based with UPnP support, so I can administer it from any browser and play it from any UPnP enabled player, like WinAmp, or from the SoftSqueeze player.
Receiver and Wireless Router
I was unhappy with my Harmon Kardon even though CNet recommended it -- too complicated and heavy and hard to use. I have decided to not read "official" reviews of hi-fi equipment because the folks doing reviews are enthusiasts and what they like is not actually what us "normal" people like. So instead I went to Amazon and read reviews from "real" people, and got a Pioneer VSX-516-K. Easy to set up, great sound, perfect. This choice had an unexpected bonus - my DVD player is a Pioneer, and the remote that came with the receiver works out of the box with my DVD player.
I also upgraded my wifi router to a Buffalo WHR-HP-G54 Wireless-G router with greater signal strength (also based on the Amazon reviews, I had never heard of this brand before), and am quite happy with that too.
NAS on the Cheap
My final issue was with how to make my music readily available to the Squeezebox. I really didn't like having to walk into the study, turn on my computer, wait for it to boot up, and log in (I have tried many times to get auto-login to work on this machine, but have failed).
So I considered upgrading, and thought seriously about a ReadyNAS system, that has SlimServer pre-installed. A lot of people really like the ReadyNAS.
But these things are pricey (around $1,100 according to Google Products), and I already had this machine and RAID set up with two 500GB disks. Why couldn't I make my machine like a NAS?
Then I read in my manual that the Squeezebox supports Wake-on-LAN. I checked my machine, and sure enough, it supported it. I configured the network card to wake the computer on a network hit, and bingo, I have, for all intents and purposes, a Squeezebox-enabled NAS system without having to fork over $1,100.
Last night I had fun just randomly picking songs from Rhapsody or my collection, basking in the experience of a well-done system and some serious sound coming from my Bose speakers. It's those little joys in life that make it worth living :)
 After using the OmniFi for a few years, it just didn't "have it", and I suspected that OmniFi would die, and sure enough, they've discontinued their products.
Monday, September 10, 2007
I think of Sun's excellent mobile work program as one of a number of "golden handcuffs" Sun has on my wrists. That, and great pay and great benefits and great people to work with. It really is hard to beat.
Just found a new free wifi hotspot near my home, the Central Perk. It's located right next to the newly refurbished Cerrito Speakeasy Theater.
It's spacious, quiet, good signal, good coffee. What sets this place aside is their vast, vast collection of toys. One could spend a good hour just walking around. I think I may have found a new hangout...
Friday, September 07, 2007
Sam Ruby has some things to say about CouchDB, in particular, how it has the same interface regardless of the type of client.
What API’s/drivers to I need to integrate it with Ruby?
HTTP and JSON.
What API’s/drivers would I need to integrate it with Java?
HTTP and JSON.
What API’s/drivers would I need to integrate it with AJAX?
HTTP and JSON.
What API’s/drivers would I need to integrate it with...
oh, you get my point.
Do HTTP and JSON work with existing J2EE servers? You betcha.
What tooling to I need? Um, a browser, perhaps?
What I think is interesting to note is that JDBC/ODBC/ADO.net are all APIs on top of the same protocol, too. It's just the the underlying protocol is non-standard, whereas HTTP and JSON are standard. So developers can write straight to the underlying protocol rather than needing a specific API for their particular programming environment.
Anyway, time for a second look at CouchDB. I don't know how I missed that the first time I saw it, but it has full bidirectional replication for offline support. That's nothing to be sniffed at...
So why don't I bite? Because the more I read about these devices and the terms and conditions around them, the more they leave a bad, metallic taste in my mouth. I don't like being locked in, I don't like signing up with a single vendor. That's why I like open source so much.
If you think it's just me whining, take a look at this article by Mike Elgan. It's looking pretty ugly.
So, you can take your shiny iPod and submit to the new Borg. Not my cup of tea.
Thursday, September 06, 2007
This is actually a very powerful form of focused/vertical search. I search the people and sources I value for information, rather than the Whole Web. Very simple, very nice.
One interesting choice they made was to do database-level synchronization. I believe database-level synchronization has a lot of issues:
- A service-level transaction may span multiple database transactions with multiple backend databases or services.
- You lose the higher-level semantics of the application when you just work with the database
- You are creating a hardcoded dependency on a particular database schema
No such luck on Mac. Very frustrating - it would appear that the only folder you can share is your home folder. Every now and then the usability folks at Apple just Get It Wrong, and this is one case (another case is their despicable Finder app, which I replaced with PathFinder and I am so much happier).
Some Googling on Mac sharing got me to a Mac Forum thread which suggested all sorts of twisty-turny configuration on the Mac, and then one quick email which said "why not just use SharePoints?".
Ah, all is goodness now...
Wednesday, September 05, 2007
It's pretty easy to do this, but you have to know what you do. You can Enable Debug fom the top-level folder of the Database Explorer (the one called "Databases")
Once you have enabled debug mode, you can perform DDL operations with NetBeans, and you will see the resulting SQL in an output window titled "Database Explorer"
Having just been in Prague and hoping I wasn't saying anything offensive, I remembered the great Hungarian Translation Book sketch, and then Jim Bisso pointed me to it.
A good strong laugh in the middle of the day keeps you young...
Tuesday, September 04, 2007
The quote from the Chinese is almost correct, here is it out of the article:
“Time and again, through the translators, they made clear they did not understand this concept,” said Nick Marchica, an engineer and former agency senior aide. “What they told us was, ‘As far as we are concerned, voluntary means we don’t have to.’ ”
We learn today, in the wake of all the poisoned dog food and harmful toys, that the Consumer Safety Bureau in the Dept of Commerce was cut in half as soon as Bush came in, and was put under a party hack with strict orders to go easy on the safety requirements. Their budget for
monitoring the safety of American consumer goods is now less than half of that for the bureau that monitors animal feeds.
Conservatives say that voluntary guidelines developed by industry are better than guidelines imposed by meddling bureaucrats, and this is the management line now at the Consumer Safety Bureau. Sounds good. But it took the Chinese importers of poisonous and adulterated and unsafe stuff to make it clear. Of course their stuff didn't get stopped at the border
by consumer safety inspectors, because there aren't any, most of the time (they now have to work from home, as their offices were abolished, and their numbers are down by 60 percent), and their testing equipment is whatever they had 10 years ago. It's OK, we can rely on the
manufacturers adhering to voluntary standards.
What did the Chinese say about this? One toy maker summed it up. "We don't understand what is the problem. Voluntary means, we don't have to."
Never mind, we don't have to run tests and require standards. We just wait until a bunch of people get sick and die, then we look into it. Cheaper that way.
P.S. I did start out as a Republican, you know. Stuff like this changed my mind.
I looked at wikis.sun.com. You can stell it's just getting started. But the potential is quite big, and the video really helps capture that.