Real-world experience with the new NetBeans PHP and JavaScript editors

One of the challenges for someone who builds products but doesn't necessarily use them is that you can get so busy with your day job that you don't know what it's like to use the product.

I do use NetBeans during my day job, writing Java code for the NetBeans product itself. So I am definitely getting my hands dirtier than when I was working on the app server or Java DB. But NetBeans is vast, and there are some areas that I just don't get enough exposure to, but which are directly relevant to my job.

In particular, a large chunk of database applications today are web applications written with PHP and JavaScript. I am quite aware of the new JavaScript and PHP editors, but had not had much experience with them.

Yesterday I spent the whole day trying to help a friend in need who had waded in too deep with a contract he has. He has no database experience, and was trying to customize a Wordpress plugin that uses PHP, JavaScript and MySQL.

So I downloaded the daily build of the PHP bundle of NetBeans for 6.5 and started working. I had done a previous PHP project using vi, and although I was fairly productive it was somewhat painful. So I was so happy to see so many of the things in the NetBeans Java editor available for JavaScript and PHP: completion, inline documentation, easy refactor/rename, goto declaration, find usages, error highlighting and so on.

I felt much more comfortable and productive in this environment. The PHP support even has FTP integration to make it easy to push to a remote site.

But I also spent a lot of time doing SQL in PHP, and had a number of ideas on how to make it easier to work with SQL in a code editor .

One thing is that for this project, as it is for many, MySQL is on a remote server on the Internet. Our entire Database Explorer is useless in such a scenario, as it requires a JDBC driver connecting to a server available on the LAN.

SSH Tunnel Support

There are HTTP JDBC proxy drivers, but what most people usually do is set up a ssh tunnel using a tool like PuTTY for Windows or OpenSSH for UNIX systems. Wade Chandler provided useful instructions for how to do this with PuTTY. Here is another good page on how to do this.

But I can imagine finding a way to make it very easy to configure a Database Explorer connection to use an ssh tunnel. This simple thing could be a big win for a lot of our users. I'm not sure what platform-specific dependencies there are and if this is possible in a Java environment, but I'm going to check it out.


Add Column Wizard

I have been thinking about how to do SQL completion in the PHP editor for a while. But during my work yesterday another itch came up that I would like to scratch. As I worked on the code, I needed to add a column. It would be great to be able to add the column right there in the PHP editor - some kind of right-click or keystroke to add the column that I have highlighted in my text editor, and a quick dialog comes up to get the details of the column.


Insert DDL Wizard

The Wordpress plugin I was working with does something I have heard a lot of people do: right there in the PHP code, detect if the tables exist and then create the tables (or alter the tables if you are doing an upgrade).

This is different from what I'm used to - I had always created the SQL scripts as separate files - but when you think about the web deployment model, it makes sense.

So, a nice feature would be to provide an Insert Code wizard to generate all the CREATE TABLE statements in your code editor. Generating the ALTER TABLE statements is trickier, but hey, let's set our goals high :)

What do you think? What else could we do to make it easier to work with databases in your language editor?

Sittin’ on the CouchDB « ZefHemel.com

A very nice blog describing one person's experience of working with CouchDB. I like it because he compares this with using relational databases and helps a RDB user "migrate" his concepts to CouchDB

http://www.zefhemel.com/archives/2008/03/11/sittin-on-the-couchdb

Derby documentation quick search in Firefox - Knut Anders Hatlen's Weblog

Knut Anders has an excellent tip on how to make it easy to search Derby documentation using Firefox. Thanks, Knut!

http://blogs.sun.com/kah/entry/derby_documentation_quick_search_in

A RESTful version of Amazon's SimpleDB - subbu.org

Hm, someone dug in to SimpleDB's REST API and found it to be not very RESTful. Doing everything over GET? Sounds like someone didn't do their homework...

http://www.subbu.org/weblogs/main/2007/12/a_restful_versi.html

Heck with SQL: is a persistent hash all you need?

Jamie Flournoy compares ActiveRecord to VB: "The easy stuff is easy, but the hard stuff is impossible," he quote James Gosling as saying about VB.

One of the things Jamie says in this very long, well-thought-out article, by someone who obviously has experience with both Rails and databases, gave me pause.

Jamie referred to some of the leaders of the Rails community saying that SQL is a design mistake and that most web apps only need a persistent hash.

This statement follows shortly after me reading an article by Diego Parilla about the impact of ORM on the database, and finding out that Amazon uses basically a highly available, persistent hash for much of its database storage on the web tier.

Michael Stonebraker is even saying that relational databases can not meet everybody's needs any more, and talks about the need for specialized data stores for specific use cases.

I actually like a lot of this. We need to simplify, simplify, when it comes to building web applications - I've been appalled at what a huge mess it is for quite some time, and have found myself avoiding building web apps if at all possible because of this. Also, SQL ties you to a database (yeah, I know, it's a "standard"), and also ties your application to a particular incarnation of the data model in the database. So for all these reasons, eliminating the need for SQL is goodness.

My concern is that if you're not careful, you start with a hash table, but then start implementing your own database on top of it. Jamie mentions this, and I have seen this too:

Satisfying queries is the database’s job, period. It’s just hideously slow to try and do an inner join in the application across a network link to a database. If you find yourself doing this, that’s a pretty good sign that your architecture is broken.

It's easy to think that all you need is a hash table, and when you're building simple, basic web apps, that's probably all you need. Heck, that's all Amazon needs, for the most part. But there are times when you want more. You really do need to do a complex query, or a stored procedure, or text search. You want to be able to run useful reports without having to do joins and sorts within your application.

I think Rails is getting a lot of things right by hiding a lot of complexity from you (as do the Java ORM technologies).

But I agree with Jamie that for those times when you need SQL, it's nice to have the option -- at least until ORM gets so friggin' smart that writing in SQL is like writing in assembly - sure you could do it, but who wants to, and why?

Out-of-the-box database notifications: JMX and Derby in Java 6

A very interesting article that shows how you can use Java DB's Java procedure mechanism, triggers, and JMX to build a very nice change-notification system to detect changes to your database.

http://chaoticjava.com/posts/out-of-the-box-database-notifications-jmx-and-derby-in-java-6/

Amazon's Dynamo - Serious Non-Relational State Storage

This is getting a lot of buzz, but for us database folks, a very interesting and detailed analysis of a different way to store and query state that meets requirements not met by relational databases.

http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html

Why programmers hate relational databases: Greg Jorgensen

Greg Jorgensen writes a fun blog about the common complaints programmers have against relational databases and his responses to them.

Most professional programmers know how it feels to see an amateur, unfamiliar with Knuth or any programming book containing equations, implement their own sort routine. That’s how people who understand relational theory feel when they see a badly-designed database. Relational theory and RDBMSs are old and well-established now, so it’s hard not to think a lot of programmers are willfully ignorant.

I have seen this too: "I don't need a friggin' relational db, just give me a flat file." Sometimes this works, but all too often I see the same programmers adding locking semantics, querying capabilities, indexes, and so on, and I just shake my head.

Why PostgreSQL Instead of MySQL: Comparing Reliability and Speed in 2007

An interesting and very complete analysis/comparison between PostgreSQL and MySQL. Written by PostgreSQL advocates, so take it with a grain of salt, but still a very good in-depth comparison.

http://www.postgresql.org/docs/techdocs.83

Mike Stonebraker: One-size-fits-all databases don't cut it anymore

There's a new database blog, and one of the first postings is from Mike Stonebraker about how specialized databases will become more and more important...

http://www.databasecolumn.com/2007/09/one-size-fits-all.html

Sam Ruby plays with CouchDB

Sam Ruby has some things to say about CouchDB, in particular, how it has the same interface regardless of the type of client.

What API’s/drivers to I need to integrate it with Ruby?

HTTP and JSON.
What API’s/drivers would I need to integrate it with Java?

HTTP and JSON.
What API’s/drivers would I need to integrate it with AJAX?

HTTP and JSON.
What API’s/drivers would I need to integrate it with...

oh, you get my point.


Do HTTP and JSON work with existing J2EE servers? You betcha.

What tooling to I need? Um, a browser, perhaps?


What I think is interesting to note is that JDBC/ODBC/ADO.net are all APIs on top of the same protocol, too. It's just the the underlying protocol is non-standard, whereas HTTP and JSON are standard. So developers can write straight to the underlying protocol rather than needing a specific API for their particular programming environment.

Anyway, time for a second look at CouchDB. I don't know how I missed that the first time I saw it, but it has full bidirectional replication for offline support. That's nothing to be sniffed at...

Terrence Barr's Blog: Need a database with your phone?

Terrence Barr says Java DB is the answer for a compliant database for mobile platforms.

http://weblogs.java.net/blog/terrencebarr/archive/2007/09/need_a_database.html

Trimpath Junction - Synchronization between local and remote databases using JavaScript/Google Gears

I just bumped into Trimpath Junction from the Google Gears mailing list.

Trimpath is a JavaScript-based MVC framework inspired by Ruby On Rails that can run on both the client (browser) and the server (e.g. using Rhino). It also appears to allow synchronization between client- and server-side SQLite databases via Google Gears.

Pretty cool.

One interesting choice they made was to do database-level synchronization. I believe database-level synchronization has a lot of issues:

• A service-level transaction may span multiple database transactions with multiple backend databases or services.
• You lose the higher-level semantics of the application when you just work with the database
• You are creating a hardcoded dependency on a particular database schema

I believe web-service-level synchronization makes more sense, but they seem to feel this is a better approach. I guess if you're tying yourself to Google Gears/SQLite on both ends it could work, but I wonder... I guess we'll have to see...

Prakash Sundaresan of Microsoft: From Database to Complete Data Platform

Microsoft's vision of a Data Platform that covers "all data, all tiers, all services". This is from a new blog by the leader of Microsofts database R&D team in China. I think this is a blog to keep an eye on...

http://blogs.msdn.com/prakas/archive/2007/09/03/from-database-to-complete-data-platform.aspx

Java DB: It's both embedded *and* a server

Some emails I have been reading indicate that not everyone knows there is this cool feature in Java DB/Derby. With a simple call in Java or by setting a property, you can have a VM with an embedded Derby database that also allows other processes to connect to and use this database through the network JDBC driver.

The instructions on how to enable the network server are here.

You either set the property derby.drda.startNetworkServer=true in your derby.properties file or you can start it explicitly:

NetworkServerControl server = new NetworkServerControl();
server.start (null);

Pretty darn simple...

Rick Hillegas talks to us about Java DB

A nice quick podcast where we get to hear Cloudscape old-timer Rick Hillegas talk about Java DB. The interview with Rick starts about 6 minutes in. Really direct, informative podcast.

http://today.java.net/pub/a/today/2007/08/27/javamobility-podcast17.html

Password encryption in Derby 10.3

A reader asked if password encryption is enabled by default in 10.3. I checked with the Derby team, and the answer is, no, the password is still sent in the clear by default in 10.3. You have to enable it by setting the securityMechanism property on the URL.

See http://db.apache.org/derby/docs/dev/ref/rrefattribsecmech.html for more details.

PostgreSQL 8.3 is faster with sequential scans

A nice blog from someone showing the performance improvements for sequential scans in the latest PostgreSQL release.

http://www.commandprompt.com/blogs/joshua_drake/2007/08/postgresql_83_its_faster_really_it_is/

David Duffield's Workday ERP System Uses In-Memory DB and Metatags

Very interesting note from Nicholas Carr about what David Duffield is up to these days.

http://www.roughtype.com/archives/2007/08/the_end_of_erp.php

Detecting the version of the Java VM you're in

This source code comes from the Derby source base. Very useful in cases where you want to load a particular class or do other things based on which VM you are running in. In Derby's case this is used to load the appropriate JDBC driver.

http://svn.apache.org/repos/asf/db/derby/code/trunk/java/engine/org/apache/derby/iapi/services/info/JVMInfo.java