Jan 5 2009

restful application module

Jonathan

I have been working on a single page web application called Moolah. Moolah uses a restful style to get data from the server in json format. When I started working on this, I couldn’t find a good pluggable module that would allow me to do this easily, so I wrote my own.

This module only works for json currently, but wouldn’t be very much work to extend to xml or other formats.

When POSTing a record, the result is the new record (including the new key). In the case of a validation failure, the errors object is returned instead.

I did some (non-extensive) searching for what a restful url scheme should look like, and didn’t really find anything conclusive. The url patterns that it uses are as follows:

get /data/account/id/123 (get item with id 123
get /data/account/search/123 (get items that contain 123
get /data/account/list/0/10?field=val (get a list from item 0 to item 10, where field=val
post /data/account/ (create a record)
delete /data/account/123 (delete a record with id 123)

It is possible that I should move to a url-pattern that looks more like:

get /data/account/123 (get item with id 123
get /data/account/?query=123 (get items that contain 123 (text search)
get /data/account/?start=0&items=10 (get a list from item 0 to item 10
post /data/account/ (create a record)
delete /data/account/123 (delete a record with id 123)

I didn’t find this urlpattern as natural, but am willing to have my mind changed.

It is written to be compatible with Google App Engine using app-engine-patch and uses the guidelines at app-engine-patch for making reusable modules.

It is completely pluggable and requires very little configuration. To use you must do the following:

  1. Within the main project:
    1. extract the zip file as a new application within your project
    2. modify the root-level settings.py file to add the “restful” application
  2. Within the application:
    1. Create a UserRestfulController class that inherits from restful.RestfulController
      class AccountForm(ModelForm):
          class Meta:
              model = Account

      # exclude any fields that shouldn’t be set or overridden from a POST
      exclude=(‘created’,'owner’)

      class AccountRestfulController(RestfulController):
      modelClass = Account
      formClass = AccountForm

    2. modify the application settings.py file and add “REST_CONTROLLERS={“account”:AccountRestfulController}” (where you want to expose a “User” model object.
  3. Then (as an example using jQuery) you can access the data with something like:
      getAccount: function(key, callback) {
         $.get("/data/account/id/" + key, function(data) {
                  callback(new CORE.Account(JSON.parse(data, CORE.util.stringToDate)[0]));
               });
      },
      getAccounts : function(offset, limit, callback) {
         $.get("/data/account/list/" + offset + "/" + limit + "/", function(data) {
                  callback(CORE.convertJsonListToObject(JSON.parse(data, CORE.util.stringToDate),
                        CORE.Account));
               });
      },
      deleteAccount : function(accountPk, callback) {
         $.ajax({
                  type : "DELETE",
                  url : "/data/account/" + accountPk,
                  success : function() {
                     callback();
                  }
               });
      },

Thats it.

There is a fair bit of flexibility in the way that the controller works by overriding methods in the RestController class. eg: We want to return a list of transactions associated with an account:

"/data/futureTransaction/list/" + offset + "/" + limit + "/?accountPk=" + account.pk;

then we have a RestfulController defined:

class TransactionRestfulController(RestfulController):
    modelClass = Transaction
    formClass = TransactionForm
    def list (self, offset, limit):
        accountPk = self.request.GET['accountPk']
        account = Account.get(accountPk)
        return Paginator(account.futuretransaction_set.order("created"), int(limit)).page((int(offset) / int(limit)) + 1).object_list

This gets the requested account, and gets the list of children objects from there and paginates them using standard Django classes.

Files for this post

  • restful gae module (.tar.gz, extract to the root of your project and follow the instructions above)

Oct 22 2008

Flex MVC

adam

When you look around at all the postings with Flex code examples and advice, it seems clear that the Flex community is largely made up of graphic artists and refugees from the last popular drag and drop programming framework (probably .NET). There are definitely some people out there coding high quality Flex applications but if the code snippets on the web are anything to go by then they are in a tiny minority.

I come from a structured programming background and have been writing Java/J2EE applications almost exclusively for the past 5 or 6 years, I love user interface programming and have done a lot of work on web based UIs, mostly with Javascript, Ajax and HTML. When I first saw the Flex 3 beta and heard that Adobe was going to open source Flex I was blown away, I immediately downloaded the beta Flex Builder 3 and jumped in.

Of course I started Googling for solutions to the problems I was having and pretty soon I realized that most of the code examples, although functional, were a pile of pants. It seems like most of the flex community has never heard of separation of concerns. People just seem to open up a script block and go for it right there in the middle of a presentation page, sometimes they don’t even bother to do that, they just inline it inside the event they are interested in. I must admit that I initially thought this was some kind of joke. A club of deviant bloggers trying to see how successful they could be in encouraging newbies to write rubbish code.

Strangely the problem when you are coding Flex is exactly the opposite to when you are coding Java web applications. In Java structure is easy to find and you want an MVC framework which will make things quicker. In Flex writing applications is quite quick but you desperately need some structure since you know that a couple of months down the track you are going to be supporting your application or building new modules for it and you can quickly see that what you are writing is a no holds barred mess.

When you google for Flex best practice you get very little and what there is is mostly very general and stolen from the last Agile, TDD slide set the presenter watched… There are exceptions and there are some people out there doing really nice code. Some guys at Adobe had a misguided attempt to develop an MVC framework, presumably with the intention of trying to correct the horror code that was beginning to be associated with the Flex brand. Cairngorm was born. Cairngorm is basically a Singleton-fest who’s only redeeming feature, IMHO, is the ServiceLocator class.

Nobody was thinking too deeply the day Cairngorm was born. They have few ideas and no structural assistance for separating plumbing code from the view. Most likely coders who go to the trouble of implementing a Cairngorm application will at least try to work out some type of mediator or code behind implementation of their own since core separation of view and controllers has to constantly be wired in manually in Cairngorm. Of course using Cairngorm is considerably better than using no structure at all, but all in all it is pretty lame.

After some time spent suffering and becoming seriously tempted to write my own MVC framework I came across PureMVC. The guys at PureMVC have a very nice ActionScript 3 implementation of their MVC framework which finally provides some proper structure for separating concerns. They provide a Mediator base class for extending to do your plumbing code. PureMVC also uses a more registry based approach rather than just encouraging you to conjure up singletons whenever you get to lazy too sort out your code properly. It is by no means a perfect framework, I do find myself using quite a bit of boiler plate code but with care, this can largely be factored out into factories or utility classes.

There definitely needs to be more work in the Flex community on application structure and separation of concerns. There is still a lot of work to do in developing a best practice for binding form fields. The Flex Binding framework seems to verge on Voodoo and behaves very unpredictably for anyone who is not aware of what is really going on under the covers. i don’t think that a really good MVC framework has been written for Flex but at least there are some options starting to spring up.


Oct 22 2008

Flex MenuBar – allowing top level elements to be selected

adam

Out of the box the MenuBar control kind of insists that you have children for every top level menu item in order to navigate. If you don’t have any children for a MenuBarItem you don’t get an ITEM_CLICK event when it is clicked. Fixing this is pretty simple but can be a little annoying since you need to listen for MouseEvent.CLICK as well as MenuEvent.ITEM_CLICK.

Putting another listener on the menu bar is pretty simple but you need to look out for the user clicking on the menu bar but not on a menu item. My click listener looks simething like this:

public function handleMenuClick(event : MouseEvent) : void {

if(event.target is MenuBarItem) {

var menuItem : MenuItem = event.target.data as MenuItem;

if(menuItem.children == null) {

app.views.selectedIndex = menuItem.navIndex;

}

}

}

In the above example i am using a very basic custom MenuItem for the data structure of my menu. It just contains the Label, Icon and a navigation index which I apply to my ViewSstacks (in the above case the ViewStack is ‘app.views’) as the selectedIndex the class looks like this:

public class MenuItem {

public var label : String;

public var icon : String;

public var navIndex: int;

public var children : Array;

public function MenuItem(label : String, navIndex : int, icon : String) {

this.label = label;

this.icon = icon;

this.navIndex = navIndex;

}

}

If the children Array is null, I know that it is a navigation target.


Oct 14 2008

Flex Form Fields Submit on Enter

adam

It’s just a small thing but it can be quite annoying when you have to use the mouse in order to submit a form, particularly something like a login form where you just want to quickly type in your details and get into the application. HTML web forms have this behaviour by default, however in Flex you have to ask for it. i have not yet found a way to do it for the whole form but you simply add a listener to the FlexEvent.ENTER event for each field on the form.

It will look something like this:

<form field>.addEventListener(FlexEvent.ENTER, doLogin);

Where doLogin, obviously, is your login function and <form field> is a field on your form.


Sep 22 2008

Excluding test Classes from WTP deploy

adam

I was finding that test classes were somehow ending up compiled into the web application deployed onto Tomcat. I spent a fair bit of energy reading through the Eclipse ETP help files but there wasn’t any reference that I could find to reconfiguring an existing dynamic web application. In desperation I started hunting through the configuration files under the .settings folder in my project ad low and behold there i found it.

Just in case i forget or someone else has this problem the file is called org.eclipse.wst.common.component in that file are wb-resource entries for each source folder which is being compiled and/or copied into the classes directory under WEB-INF. I deleted the entries for source/test/java and source/test/resources and that took care of things.


Sep 20 2008

Battle of the IDEs

adam

I have always been a die hard Eclipse fan. Ever since the days of 2.1. A few years ago I started using the MyEclipse plugin to get hot-deploy to Tomcat. It quickly became indispensable to me and even when I started using maven and, what was then a very flaky plugin, M2Eclipse I persisted and wore the pain.

About 18 months ago I bought a Macbook Pro and my love affair with Eclipse and particularly MyEclipse plunged. I just could not seem to get a stable build. It would constantly crash…constantly. After many calls to MyEclipse and hundreds of different configurations I took the plunge an moved to IntelliJ.

Wow, what a lovely IDE. Smooth, everything just works (except the PMD plugin), but I kept feeling just a little bit dirty, just a little like I had sold out. As if forsaking Linux wasn’t enough, now I was forsaking my open source IDE! Also the use of space in IntelliJ kind of sucked. All the wasted space in margins makes it hard to use with multiple panels open unless you have a monster screen. I think most of that is due to the incredibly poor attention to detail in Java Swing UI components. Making a sexy looking Swing app is like walking over hot coals.

A little while ago I got involved in a project with a bunch of friends who were using Eclipse Ganymede without the need for MyEclipse. They were using the native functionality for hot deploy to the server. I was initially reluctant to go back to Eclipse after all the past pain but developing in different physical locations using different IDE’s really started to make some things more difficult than they needed to be so I downloaded Ganymede and gave it a go…

Rock solid on the Mac. I have been using it for the past few months without a single crash. The SWT UI is obviously a lot sexier than the IntelliJ Swing UI. The hot deploy is a bit of a pain in the arse to sort out when combined with M2Eclipse, but after a bit of tinkering around that is all working well too. The recent work on M2Eclipse has improved it out of site and integration with Maven 2 in Eclipse is now closing on what IntelliJ has had all along. The major down side is the SVN integration. Subversive is not bad, although a right pain in the arse to install, yet it is not a pinch on the IntelliJ native Subversion support. This is absolutely superb, the 3 way merge tool is brilliant, the ability to view annotations in your code to see who changed what, the constant monitoring of your repo all go to making it a pleasure to use.

I don’t have a conclusion on which is the best for me. I am back to using Eclipse almost exclusively and enjoying the experience but every now and then I really miss a feature from IntelliJ. I will download 8.0 when it gets out of Beta and have a look but it would need to provide some pretty compelling reason for me to switch back, mostly because most of the developers I work with are using Eclipse and the difference in IDEs can make some things difficult.


Sep 17 2008

Unit Testing with mock objects

adam

The main reason for using mock objects to write unit tests is so that they are unit tests. Without mock implementations, unit tests quickly become integration tests. There is nothing wrong with integration tests however they tend to make coverage data inaccurate, since code which is exercised incidentally rather than being directly tested gets marked as covered, and they can end up slowing down the test suite since these types of tests usually involve having to load the complete application context in a Spring application or they require a container and a JNDI instance for other J2EE apps.

My motto is keep it simple. I don’t like to think too much, particularly about my tests. When I see a red bar I don’t want to have to wonder whether it is the method under test which is broken or if it is one of its dependencies. I like to be certain that it is the method under test. It greatly reduces the time it takes me to get the green bar of goodness back.

I, personally, use EasyMock as my mock framework of choice. Combined with Unitils it takes a great deal of drudgery out of mocking out dependencies. Using the Unitils EasyMock annotations you can mark an interface as a mock using @Mock and mark the class under test which will have the mocks injected using @TestedObject. This doesn’t seem to work on classes using constructor injection, however it works fine for setter injection and means there is no need to manually setup your mocks and inject them into the class under test.

Complete details on how to use these annotations can be found here


Sep 3 2008

Eclipse Ganymede plugins

adam

It has been quite a while since I posted on Eclipse plugins I am using. All the URLs have changed and so have many of the plugins I find essential for development. Here is an update on the plugins I recommended last time and the ones I’m using now:

  1. MyEclipse: I have ditched this as an unstable, annoying, and largely useless waste of money. Ganymede WST provides all the Sever hot deploy stuff that was good in the older MyEclipse editions out of the box, most of the rest can be found in various free  plugins I will detail here. When I was working mostly in Windows or Linux this plugin was okay, but on the Mac MyEclipse crashes almost constantly and is a complete time waster. Virtually nothing interesting, which might save you money even works on the Mac, ie. UML, Javascript debug, image editing. Also the new installer is crap… I’m over these guys, $60/yr is a nice price point for a useful plugin set but really this is worse than useless if you are a Mac user.
  2. Spring IDE – If you are doing Spring development you cannot live without this plugin. Provides Spring configuration validation and help with code completion. URL: http://springide.org/updatesite/
  3. Subclipse: I have trashed this. The conflict resolution was kind of buggy and failure prone. Maybe it was operator error, but I got sick of trying to manage broken merges and migrated to Subversive.
  4. Subversive: This is becoming the Eclipse default SVN plugin. Why Ganymede does not have an SVN plugin out of the box is beyond me, is anyone still using CVS these days??? The only downside of Subversive is that it is slightly tricky to install as the install instruction have you jumping all over the web to get the update site urls for the connectors and the actual plugin. Here is the long and the short of it…
    1. Plugin site URL:http://download.eclipse.org/technology/subversive/0.7/update-site/
    2. Connector site URL: http://www.polarion.org/projects/subversive/download/eclipse/2.0/update-site/

    If you are using OS X or Linux you will need to unselect some of the windows JavaHL connectors

  5. M2Eclipse: This plugin has really come a long way since it was released a year or so ago. Previously the only useful thing about it was the repository search. Now it allows quite reasonable management of your Maven 2 project from within Eclipse. For those afraid of editing XML the new pom editor is really useful. I still don’t think the integration is quite as seamless as IntelliJ but it is now quite acceptable. URL: http://m2eclipse.sonatype.org/update/
  6. Eclemma is still the best free coverage plugin I am aware of. Again it is annoying that the IntelliJ emma integration is more seamless and intuitive but the Eclipse integration is quite workable. I had troubles installing this on Ubuntu, although it installed fine on my Mac. If you have problems it is quite simple to go to the eclemma site (www.eclemma.org) and download it. Unzip it and drop the contents of the plugin/ and features/ folders into their respective folders under your eclipse installation. URL: http://update.eclemma.org/
  7. PMD: This plugin is very handy for getting quick feedback on code style. We have been using PMD for a while and have a company ruleset.xml. I like the way you can prioritize problems and then filter by priority in the IDE, but I am still in 2 minds as to whether this is actually better than Checkstyle. URL:  http://pmd.sourceforge.net/eclipse

Aug 15 2008

Installing Postgres on Ubuntu 8.04

adam

This is  a fairly simple process (the client install is optional as pgadmin is a far better client):

sudo apt-get install postgresql postgresql-client

Install pgadmin:

sudo apt-get install pgadmin3

then you will need to reset the password for the postgres user:

$ sudo su postgres -c psql
postgres=# ALTER USER postgres WITH PASSWORD 'password';
postgres=# \q

You can now connect to postgres using pgadmin using the password you entered above.

If you want to be able to access Postgres from other computers than the one it is installed on then you will need to modify the /etc/postgresql/8.3/main/pg_hba.conf  (this is the location in Ubuntu).
The file has a large commented section describing how to configure access from other machines on your network but it boils down to adding an entry something like this:
host all postgres 192.168.1.0/24 md5
The above line tells postgres that the postgres user can access all databases from any machine in the 192.168.1.* subnet. Credentials will be sent encoded using md5. There are a bunch of more complicated ways that you can setup authentication but this basic example will get you started. Later you may want to use the ident authentication method and maintain dbusers in the pg_ident.conf file (located in the same directory as the ph_hba.conf file).


Jul 20 2008

No Test, No Agile

adam

This should be obvious, but I seem to find myself re-explaining this concept quite a bit. Without testing there can be no Agile development. Tests are not an optional extra, there can be no negotiation. If you are doing agile, you are doing tests. You can test first, my personal preference, or you can test immediately after, but you must write tests.

One of the core values of the Agile manifesto is responding to change over following a plan. Agile development is predicated on writing the simplest solution which meets business requirements and nothing more (YAGNI principle). This can only be done if the developer(s) are able to refactor later as requirements become more complex or change.

Developers will either not refactor at all or take an extremely long time to achieve refactorings if they don’t have tests in place that give them confidence that their changes have not broken existing functionality. No test, no agile. Other features of agile development may or may not be optional, but testing is definitely not.