Thursday, September 8, 2011

Application developers should not write frameworks and toolkits (much)

On occasion I'm a little disappointed when I see developers create frameworks and toolkits. I understand that it is enjoyable to do so, but we, as application developers, should be majorly focused on the business needs of developing applications.

Quite often we think that xyz widget doesn't do exactly what is "required" and therefore pursue a path of days (weeks) writing frameworks and toolkits.

Let's not do this anymore.

Instead, every time you, as a developer, get the urge to write a framework or toolkit or even just a utility class, think about extending an existing open source project to meet your needs; or even starting a new open source project. For one thing, your requirements will most likely be challenged (which is a good thing). You will no doubt end up with something better than you could have created yourself (yes, that's hard to swallow, I know!). Importantly though, your organisation will benefit; the less "quirks" particular to an organisation the faster it is for them to ramp up resources and "Get Things Done".

Your organisation will also be able to share stuff between the silos. Imagine that.

I often hear, "but my organisation doesn't contribute to open source" (despite heavily using it). In this instance I always ask if the person in question has actually tried selling the business benefit (above) and attempted to make it happen. More often than not, the developer hasn't tried. Organisational behaviour is driven by the views of their employees and contractors. Organisations change and are changing.

I do believe that at the end of the day, it is better to suffer with an xyz widget that does most of what you need it to do than to re-invent the wheel. If you're an application developer then focus on the application, not frameworks and toolkits; unless you're contributing to open source projects.

Thursday, August 11, 2011

Dependency Injection in Java - @Resource, @Inject or @Autowired?

The @Resource documentation states that the "name" attribute refers to "The JNDI name of the resource". Spring has overloaded this to mean that you can refer to bean identifiers… but as far as the Java spec goes, there's no contractual obligation for a resource name to refer to a bean id. @Resource referring to Spring bean ids is therefore quite Spring-specific.

@Autowired is Spring specific.

@Inject is not Spring specific and performs type-safe injection.

The other benefit you'll get from type-safe injection is the ability to safely re-factor within your favourite IDE, but I personally think that the contractual obligation mentioned above is more important.


Wednesday, August 10, 2011

Versioning RESTful services

I've seen some comment out there regarding how to version RESTful services and so thought it about time to add in my own position on this.

I don't subscribe to crafting your own MIME type as these should refer to well-known entities. I also don't subscribe to passing a version number as an Accept param. Despite there being client/server compatibilities in this regard (Ruby), the developer is not forced to specify the version. I think it is useful to specify the version of a service to use.

I'm an advocate of using a /v1 path in the url of a RESTful service, but it should be noted that great care should be taken in exactly what that version number refers to. I've long subscribed to the major.minor[.maintenance[-build]] approach as per Maven and reasonably discussed at Wikipedia. In the case of URL versioning, only the major identification is used i.e. "1.0.2" becomes just "v1".

Major versions only change when there is an API-breaking change that has been introduced. This should be very rare. API-breaking generally means quite a big shift in the approach an API is taking which is why I found it interesting that Google declare their Google Maps API will always be backwardly compatible. To me that's effectively what held Microsoft back with Windows until Vista came along and started to drop backwards compatibility (Windows 7 of course is the evolution of that good decision).

So I say use /v[major] as a convention, but be diligent in the definition of what is "major".

Friday, July 22, 2011

LMAX and the benefits of keeping things in memory

I've now read Martin Fowler's review of the LMAX architecture and I feel good. This is very similar to what I've been doing for a long time with messaging (Spring Integration, Camel, AMQ, RMQ etc.).

LMAX is an event driven architecture. The Input Disruptor can be your favourite messaging broker. The Business Logic Processors are your message queue consumers. AMQP can certainly attain the goals described for the Output Disruptor (targeted consumers etc.).

The only thing "disruptive" about this article is that it introduces new terminology. I accept that there's probably a lot of optimisation going on, but from an architectural standpoint I don't see anything revolutionary.

Perhaps the most interesting points of the LMAX architecture are that everything is done in memory and that the processing of data is kept close to the data. These are things that we will all no doubt agree on being the main influence on performance.

Sunday, June 26, 2011

RESTful JSON fixtures for testing using CouchDB


I'm working in a distributed team at the moment where I don't have access to infrastructure available within the company's firewall. The application I'm working on has a nice RESTful API with JSON payloads for its logic layer. As I'm mostly developing the presentation layer I needed a quick method of generating something that would simulate the logic layer.

One thing that I felt was important is that I shouldn't have to mutate any settings within my presentation application in order to work either against the simulated logic layer, or the real one. It is all too easy for such settings to creep into production and break things.

I had a few thoughts on how to achieve my objectives considering Spring MVC,JAX-RS re-implementation using our existing interfaces (we're using JAX-RS in the real logic layer), writing a Jetty based servlet handler and node.js. Then I remembered...

CouchDB is a document oriented database that uses JSON as its Data Description Language (DDL). In addition CouchDB provides a RESTful JSON based API to access the database. This got me thinking that I could use sample responses from our real logic layer and enter them directly into the CouchDB database. It turns out you can using the batch interface!

I then thought that I needed to create a CouchDB service that could understand the parameters of the web service request I needed to make. These parameters are properties of my JSON objects that are otherwise unavailable to the URL format that CouchDB provides for querying. The answer here is to provide a View. Here's my view:

function(doc) { emit(doc.surname, doc) }

The surname is a property of my JSON object that I'll want to query.

I call the view the same name as my logic layer service. In my case the view's name is "getCustomerBySurname". If I query my service using CouchDB's URL convention I would issue something like:


This yields a result like:


Note that I'm using a convention for versioning a resource so that I can evolve its API nicely in the future - this is quite important for RESTful requests as versioning isn't something supported by anything within HTTP

I now have a web service that behaves the way the real service behaves. However, it doesn't look the same to the consumer for two reasons:

  • the result contains meta information about the query that the consumer does not expect ("total_rows" etc.); and
  • the URL we provide is not of the same form as the real logic layer service.

Transforming the JSON result

To transform the payload returned by CouchDB into something our consumer expects we create a List in CouchDB-speak. My list function looks like this:

function (head, req) {
  provides('json', function() {
    var results = [];
    while (row = getRow()) { results.push(row.value); }

I've declared my list by the name of "rows". The function is focused on pulling out just the row.value property of each row and returning that back to the consumer. To access the view that I created and have it render through the list you provide a URL like this:


This now yields something like:


...which is what my consumer expects.

Transforming the url

The url is transformed via Apache httpd using a RewriteRule. In order to avoid Same Origin Policy we've had to proxy requests for our presentation component and its web services anyhow. Here is the re-write rule I ended up with:

RewriteEngine On
RewriteOptions Inherit
RewriteRule ^/CustomerDB/services-1.0/rest/Customers?surname=(.*) http://localhost:5984/customerdb/_design/customerService-1.0/_view/getCustomerBySurname?key="$1" [QSA,P]

... and that's it. I now have a means of using CouchDB to provide fixtures for the purposes of development.

One last important thought...

The really interesting thing about using CouchDB for the use-case I set out to solve is that I'm really starting to wonder whether the real logic layer should be written using CouchDB... it is so simple and powerful!



Friday, April 8, 2011

DAO coding paradigms can require some skill

"DAO coding paradigms can require some skill" is a sentence that presently appears at Wikipedia in the description of Data Access Objects. I find DAOs very useful and honing the skill is worth it. The article goes on to state a benefit being, "Improved efficiency and performance of the data layer since it is standard reusable software".

The majority of DAO implementations that I see tend to provide similar functionality to the Active Record pattern i.e. CRUD. Unfortunately these DAOs do not achieve the benefit of improved efficiency and performance. Let's say I have a customer, with orders that relate to products. My use-case is that I want the customers along with the orders and their products. With a straight CRUD approach I tend to get the customers (c) and then I get the orders (o) and then I get the products (p). That's potentially c * o * p or more requests on the database over the network.

The method signatures of my DAOs are shaped by the service layer that drives them. Here are a few methods of one of my real-world DAOs to serve as an example:

public void createSMSSubscription(
String msn, String flightNo,
    String actualText, String receivedParsedText,
Date scheduled, Date eventTime,
AbstractSMSEvent subscriptionEvent) {
public Collection<Subscription> getActiveSubscriptions(
    AbstractRenderedFlight renderedFlight) {
public Collection<ReceivedSMSFlightSubscription>
String msn, Date receivedTime) {

It may be that the service layer simply asks for a Customer to be created, but more often than not, the methods tend to be tailored to what the service layer wants to do e.g. "create an order for a customer that exists and only for products that exist", or "create an order for a customer that has not exceeded their credit limit" and so forth. I like to have my DAOs do as much as possible at the database with the objective of minimising the number of calls made to it. In short, my DAOs tend to comprise of fairly coarsely grained functions instead of finely grained CRUD style operations. I'll admit that it is a bit of an art in getting the granularity correct and that re-factoring commonly occurs, but the overriding goal is to reduce the IO between my logic layer and the data layer. Performance and efficiency then typically follow.

My DAOs are developed as the service layer evolves i.e. they are not developed before the service layer. As stated above I let the service layer determine what functionality the DAO should provide. If I've developed a service with the DAO development alongside, and then develop another service that wants more out of the DAO then I tend to done one of two things:

  • parameterise the existing methods of a DAO such that their behaviour can be varied; or
  • create new methods on the DAO.

One more thing while I'm thinking about DAOs... they're a great place to encapsulate persistence functionality of course. It used to be that we may have wanted to hide away the differences of dealing with multiple types of relational database. However we now find ourselves thinking about providing non-relational stores (NoSQL) as well. DAOs give you the freedom to choose the best persistence implementation to suit your application's requirements.

Long live the DAO!

Sunday, April 3, 2011

Zoneinfo TZ v1.0.0 is now available!

My Zoneinfo TZ library is now available up at the Codehaus. V1.0.0 should make it to the central Maven repo shortly. The project has mailing lists, continuous integration, issue tracking and all of those things you'd come to expect.

Interestingly, the site that hosts the zoneinfo files themselves has disappeared tonight. Fingers crossed that it'll come back soon!

I hope that this project proves useful to someone other than me! Enjoy.

Wednesday, March 2, 2011


Recently I've been doing a lot of development in GWT. I have to say that, despite being a big fan of pure JavaScript RIA, I'm liking GWT a lot. It has been a couple of years since I touched GWT and it has come a long way. I particularly like the Activities and Places API, although there's a bit of a learning curve there.

One thing I don't particularly like though is Sencha's EXT-GWT, or GXT as it is known. For those that don't know, GXT provides an abundance of widgets including a fully featured Grid. This is of course highly enticing, but what I've found is that GXT appears to be an all-or-nothing proposition. GXT is more akin to a framework than a toolkit because it effects how you lay out your code. What isn't clear to me is why it has its own layout architecture and its own event architecture. That's just confusing!

I'm starting to think that GXT plugged a few gaps in GWT 1.x and is becoming increasing less relevant. GWT 2.x deprecates a lot of GXT; even GWT 1.6 introduced a new event model.

Don't even get me started on GXT's Grid separation of concerns; its one of those widgets that likes to influence how the data transfer objects look. Plain wrong. The whole loader concept is terribly confusing as well.

I feel that if GWT's TableCell was as fully featured as GXT's Grid then no one would probably use GXT.

Perhaps I'm being a little harsh though. GXT widgets present nicely. It is just that they don't just "slot in"; you've got to buy into all of GXT to use a little of it.

Thursday, January 13, 2011

I have a new job


I now have a job with VMware starting January 24th and I'm excited! My role is as a vFabric consulting software engineer within the professional services division .

Over the past 12 years I've been an independent contracting developer/architecture/designer and launched quite a few of my own products. I suppose one of the initial drivers to go contracting for me back then was that it was lucrative to do so. I now feel that the gap in remuneration between going permanent and contracting has diminished.

Another incredibly important factor was my family. Contracting can be great but the cash flow can be stressful. With 4 kids I needed more stability.

Finally, I wanted to work with more like-minded people. During my contracting years I've met some truly inspirational people along the way; but I want to meet more! This can be difficult as a contractor as you tend not to stay around long enough to work on the strategic projects and with an organisation's best people.

So, back in May last year I decided to look around for some permanent work. I choose some companies with SpringSource being one of them. I thought, if I go for a permanent job then who would I really want to work for? It is important to "tow the corporate line" of course and so I needed to ensure that this would be a no-brainer. I'm not very good at lying!

I've been using the Spring framework for quite some time and became very interested in the work of some of the SpringSource people; namely Rod Johnson and Ben Alex. Towing the line for SpringSource would not be a difficult proposition... I contacted SpringSource and ended up meeting with Guy Baldwin and the ball then started to roll. I instantly liked Guy and his technical knowledge along with his amiable personality makes him a great face of VMware/SpringSource.

'better had just get one thing straight: VMware acquired SpringSource in 2009 and so SpringSource is now a division of VMware. I'll refer to SpringSource as VMware for the rest of this blog ('hope that is politically correct!).

vFabric is the name given to VMware's Cloud Application Platform, meaning the whole stack of technologies that enable applications to live in a cloud environment. Technically speaking the cloud may not be the actual cloud as in the internet, but live behind the corporate firewall; it doesn't really matter though. What the platform is really about is enabling applications to be scaled in order to meet unanticipated demand.

My specific role is a first here in Asia/Pac. VMware have a vFabric consulting presence in the US and I'm their first consultant for them here in our region. It is exciting.

I'll continue to post my own views on software development here but needless to say, they'll be my views and not necessarily VMware's!

Monday, January 3, 2011

The Low Cloud

I was chatting with a CIO mate today about the future of cloud computing and how his company are looking to build their own Cloud Enabled Application Platform (CEAP). I feel that many companies are sensitive to putting their data in the cloud itself and my friend definitely echoed that sentiment. There are some things his enterprise could put in to the cloud for economic reasons, but most things will reside behind their firewall.

Interestingly my friend's main driver for a CEAP was to achieve horizontal scalability. He has some performance issues that lend themselves well to being addressed in a highly parallel manner.

During this conversation I used the term "low cloud" to describe CEAP hosted behind the enterprise firewall. So, you heard the term, "low cloud" here first. :-)