Derby vs. SQLite - one user chimes in

I haven't used SQLite enough to compare it in any cogent way with Derby. But recently there was a thread on the derby-user list asking to compare Derby with SQLite. A lot of interesting answers, but there was one from Ryan Bobko who apparently has used both, and he had some points to make that I thought I'd share.

Personally, I think there's no comparison between Derby and SQLite.
Derby's an actual database with "all" the bells and whistles, but still
a really compact size. SQLite is an extremely fast database-like system
with a much smaller subset of features and SQL compliance. Plus, if
you're writing in Java, go with Derby. If you're writing in C or C++,
give SQLite a run-through.

Off the top of my head, SQLite doesn't support foreign key constraints,
or use column types (everything is a string, unless it's an int, which
is actually a string). In the quirks department, I've noticed join order
can have a dramatic effect on performance. What's really nice is that
the whole database is a single file, which makes using it as a save file
in your application really nice. Also, startup times are zero. I think
Derby takes a second or two to startup. Both systems support
transactions. Derby can be used in a multi-user mode, while SQLite is
strictly mono-user.

I know both databases claim to be zero-administration, but I'd say
SQLite more serious about it. I don't even know how to configure SQLite.
Derby certainly works great without administration, but there are a
whole lot of options you can muck with if you like.

No Java 6 on Mac Leopard - time to plan a move

I had been waiting patiently for the new version of OS X, code-named Leopard, since early this year. Why? Because I need Java 6. Now I find out that it's not there, and no work from Apple when it will be there.

So, I sigh, and consider that Apple is abandoning the Java developer base, and I am seriously considering making a move. I think I'll start with putting Solaris Express Developer Edition under VMWare Fusion, which sounds like it's just getting better and better (you can get a pre-installed image here).

If that works well, I may set up Solaris under BootCamp and start using Solaris as my primary platform. Even if suspend still doesn't work. At least under VMWare if you suspend your Mac, you suspend Solaris too...

NetBeans Database Tooling - Show Us The Way

I just put together a very quick survey where you can provide us some guidance for database tooling features for the next release of NetBeans. The survey doesn't cover all the features (thank goodness, talk about overwhelm), but the key ones where I'm not sure whether they are important or not.

Your input here is key, very valuable. It takes about ten minutes. Click here and tell us what you want.

Quiet Time, and, how to suppress derby.log in Apache Derby

You probably won't hear much from me in the next few weeks - big deliverable. But I'll Be Back...

A quick tip from the Derby user's list, that I thought was very useful, particuarly in embedded situations where you don't want to leave detritus around when you're running Derby.

A user asks:

I'm about to distribute my java app with an embedded derby database,
but it's really not polite for apps to write unsolicited files to
disc. Anyway of suppressing the derby.log file?

Knut Anders from the Java DB team has a great suggestion:


You could take a look at the derby.stream.error.field property:

http://db.apache.org/derby/docs/10.3/tuning/rtunproper33027.html

If you create a stream object which swallows whatever is passed in to
its write() method and put it in a public static field in the class
MyApp, like this

 public static final OutputStream DEV_NULL = new OutputStream() {
         public void write(int b) { }
     };

then you can suppress the derby.log file by setting

   derby.stream.error.field to "MyApp.DEV_NULL".

I, Cringely . The Pulpit . Oh, and We Also Saved the World | PBS

Bob Cringely pulls together a very interesting story about Google and high-flying wind power. I don't know if I can believe his numbers, but they are very compelling: 1/2 cent per kilowatt, and only needing 3,600 sq miles of land to power the entire USA.

http://www.pbs.org/cringely/pulpit/2007/pulpit_20071012_003200.html

A very personal view on the costs American health care

Recently my aunt passed away after a long bout with breast cancer, the youngest of six brothers and sisters in my Dad's family.

My cousin, my aunt's elder daughter, just sent this out to the family list. I think her message speaks for itself.

I want to say Thank You to all of you. You have been so supportive over the past 5 years and there are no words that can express those feelings that I have. This is truly a difficult time for me and my family.

My mother never had a mammogram until she was diagnosed. There were times, mostly, when she did not have medical insurance. She was afraid that if they ever found anything, she would not be able to get insurance and be treated properly. And if she had to be on county care, she wouldn't get good care.

Does this sound right to you? If you ever have an opportunity to make a difference in the policies that this country has, do it. That is the single most frustrating thing that I have dealt with through this ordeal. If she had had a mammogram at age 50, she would have caught this cancer early, and we would still have her around now. She had those tumors, TWO, for 10 years.

Thank you again for all of your kind words and support. They do not go unappreciated.

Heck with SQL: is a persistent hash all you need?

Jamie Flournoy compares ActiveRecord to VB: "The easy stuff is easy, but the hard stuff is impossible," he quote James Gosling as saying about VB.

One of the things Jamie says in this very long, well-thought-out article, by someone who obviously has experience with both Rails and databases, gave me pause.

Jamie referred to some of the leaders of the Rails community saying that SQL is a design mistake and that most web apps only need a persistent hash.

This statement follows shortly after me reading an article by Diego Parilla about the impact of ORM on the database, and finding out that Amazon uses basically a highly available, persistent hash for much of its database storage on the web tier.

Michael Stonebraker is even saying that relational databases can not meet everybody's needs any more, and talks about the need for specialized data stores for specific use cases.

I actually like a lot of this. We need to simplify, simplify, when it comes to building web applications - I've been appalled at what a huge mess it is for quite some time, and have found myself avoiding building web apps if at all possible because of this. Also, SQL ties you to a database (yeah, I know, it's a "standard"), and also ties your application to a particular incarnation of the data model in the database. So for all these reasons, eliminating the need for SQL is goodness.

My concern is that if you're not careful, you start with a hash table, but then start implementing your own database on top of it. Jamie mentions this, and I have seen this too:

Satisfying queries is the database’s job, period. It’s just hideously slow to try and do an inner join in the application across a network link to a database. If you find yourself doing this, that’s a pretty good sign that your architecture is broken.

It's easy to think that all you need is a hash table, and when you're building simple, basic web apps, that's probably all you need. Heck, that's all Amazon needs, for the most part. But there are times when you want more. You really do need to do a complex query, or a stored procedure, or text search. You want to be able to run useful reports without having to do joins and sorts within your application.

I think Rails is getting a lot of things right by hiding a lot of complexity from you (as do the Java ORM technologies).

But I agree with Jamie that for those times when you need SQL, it's nice to have the option -- at least until ORM gets so friggin' smart that writing in SQL is like writing in assembly - sure you could do it, but who wants to, and why?

Bob Cringley, the inspiration for FON

I didn't know this. Bob Cringely wrote an article on WiFi sharing that inspired the founders of FON. You never know where your next idea will come from...

http://www.pbs.org/cringely/pulpit/2007/pulpit_20071005_003153.html

Recording Artist: Don't be a ZFS Hater

Drew Thaler, an ex-MacOS filesystem engineer, has some very positive things to say about ZFS. "ZFS is a fine candidate to replace HFS+ eventually. It's not going to happen overnight, no. ... . But several years from now? Absolutely."

http://drewthaler.blogspot.com/2007/10/don-be-zfs-hater.html

Out-of-the-box database notifications: JMX and Derby in Java 6

A very interesting article that shows how you can use Java DB's Java procedure mechanism, triggers, and JMX to build a very nice change-notification system to detect changes to your database.

http://chaoticjava.com/posts/out-of-the-box-database-notifications-jmx-and-derby-in-java-6/

Where are my database tables in NetBeans?

All right, another common question, and this one (understandably) can cause a lot of frustration: where the heck are my tables?!

So I think it's time to record some common causes of this. I'm actually finding these 'tips and tricks' blogs very helpful because when the question comes up again I can just point to my blog and save my fingers some typing...

A lot of users create tables in their database, and then are flummoxed when they can't find them in NetBeans.

Here are some possible reasons why this can happen:

Bad Metadata

Older JDBC drivers, particularly Oracle drivers, are notorious for not handling metadata correctly. NetBeans reports exceptions in the log when it can't get metadata, but you may not have noticed that. The visible result is you see no tables when you open the connection, even though you know you created thoses tables. Head banging ensues.

So, make sure you upgrade to the latest JDBC drivers and register them with NetBeans, and see if that helps.

Two Different Schemas

Make sure you're in the right schema. If you create a schema and then create the tables under that schema, they may not show up by default in NetBeans. To fix this, in the connection dialog for NetBeans, choose [Advanced] and then select the correct schema.

Two Different Databases with the Same Name

This is possible particularly with Java DB. Java DB will create databases directories in the location where you start the VM which hosts the database. If you connect to Java DB using the embedded driver, that's the VM of your tool.

So if you use one tool to create your tables, and then use NetBeans to look at the tables, you may actually be looking at two different databases, and so in NetBeans the tables won't be there.

Let's follow this scenario more closely so you can see what I mean:

- You use a tool to create tables. You use a URL like "jdbc:derby:mydb;create=true". Note that if the database does not exist, it is created automatically. The database is created in the directory where your tool was started.

- Then you open up NetBeans, and you connect to your database using the same URL. But NetBeans is started in a different directory, and a different database is created

- You browse the connection in NetBeans, and your tables aren't there! Bang bang goes the head.

The best way to solve this is use an absolute path to the database when you specify the database name in the URL. So instead of "jdbc:derby:mydb" use "jdbc:derby:/the/path/to/my/db/mydb" or "jdbc:derby:C:/the/path/to/my/db" (note that you use forward slashes even for Windows)

---

Update: Some great comments from my Dear Readers, who are respectfully correcting my proposed solution. Take a look at them if you know what's good for you :) But to summarize:

• an alternate to absolute path is setting derby.system.home (or, in NetBeans, setting the database directory, see my previous post on how to do this). Absolute path doesn't always work, and particularly is a problem in production deployments where you don't control your environment
  
• don't use "create=true" after you've created your database, to avoid creating databases willy nilly (although this approach eliminates some usability where in embedded deployments you don't have to worry about the database not existing)
• if you don't know where your database is, search for it

Anyway, my readers say it better than me, so read their comments.

Be productive: stop working

I've been reading a great book called The Power of Full Engagement. These guys worked with top performing athletes, analyzing what makes the best the best, and they came up with a very important conclusion: the best took breaks and re-energized themselves, even if it was a quick pause between serves in tennis. And now they are applying this to the rest of us - what makes us the best performers?

They found that there is something powerful in stopping, getting off the treadmill, taking some deep breaths, walking around, and do anything but think about work. This has an almost magical ability to untie knots in our mental processes and show us new ways to look at a problem. We all know this at some level, but this book has an impact because it's based on solid research -- you just can't argue with it.

I read this book when I was in the middle of a big push at work, and it seemed that the harder I worked, the longer it took to get anything done. I was also exhausted and grumpy, not enjoying life. We all know this mode. My old boss used to call it the "death march."

So I adopted the principles: I made myself not work insane hours, and tried to take more breaks during the day. And I became much happier and much more effective.

Recently I've also been in physical therapy for my lower back, after a very nasty episode where I was in bed and couldn't move for three days. And guess what one of their practices for me is, this time to help me keep track of my physical posture during the work day: take breaks.

Oh, and there is my failing eyesight. One of the key tips to help the eyes not get over-strained -- you guessed it: take breaks.

OK, OK, I get the point. My problem is, I forget to take breaks. I get so completely engrossed in my work, especially when I am coding on some cool feature, that I lose track of time.

I knew they must exist, and sure enough, I found a very nice tool for the Mac called RSIGuard. This tool tracks your mouse and keyboard activity, and then reminds you to take a break. During the break, it shows you little videos of stretches you can do to reinvigorate yourself and avoid physical injury like RSI. You can fine-tune how frequently and for how long you want to take breaks. They also have these nice little "microbreaks" that kind of "wake me up" for a few seconds, and I take a deep breath, adjust my posture, and move on.

So, I highly recommend it. If you are a real bulldog, you can even have it enforce your break by locking up your keyboard and screen. I don't like working that way, I just get pissed off. I like a more gentle approach, and I am a good doggie and take my breaks. And you know what, I really like it.

Off for a cuppa decaf and a walk around the cafe...

Amazon's Dynamo - Serious Non-Relational State Storage

This is getting a lot of buzz, but for us database folks, a very interesting and detailed analysis of a different way to store and query state that meets requirements not met by relational databases.

http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html

The Future of Web Startups

Paul Graham has a great essay that captures a lot of the trends we're seeing with startups and where he sees it all going. Easier, cheaper, standardized, more of them, and who needs college?

http://www.paulgraham.com/webstartups.html

The Inevitable March of Recorded Music Towards Free: Tech Crunch

I have tried telling this to friends, and they think I'm kind of crazy. But I do believe it is inevitable. Maybe we'll also go back to the good ol' days where everyone sang each others' songs without having to pay license fees.

http://www.techcrunch.com/2007/10/04/the-inevitable-march-of-recorded-music-towards-free/

How to change the host and port for Java DB server in NetBeans

This issue has come up a number of times on the mailing list, so I thought I'd post it here for posterity.

The question is: how do you configure the Java DB Network Server, like set its host and port number, for the server that NetBeans starts and stops for you?

The sad answer is: you can't from the IDE. There is a bug logged for this.

However, you can do this by editing the derby.properties file.

First, find out where your database directory is. In NetBeans 6, go to Tools->Java DB Database->Settings and see the value for Database Location

For NetBeans 5.5, go to the Preferences/Options panel for NetBeans, choose Advanced Options, and then under IDE Configuration->Server and External Tools Settings->Java DB Database look at Database Location.

Under that directory, edit derby.properties. The file should be there; if it's not, create it.

Then add

# Change host setting so you will accept connections from any host
derby.drda.host=0.0.0.0

# Change port number if you don't want to use the default of 1527
derby.drda.portNumber=8080

Then for both NetBeans 5.5 and NetBeans 6, do Tools->Java DB Database->Stop Server and then Start Server.

The new properties should now be applied.