redredred

There have been a couple of releases of the Simple Object Assembler in quick succession over the last week. These changes are all in aid of simplifying it’s use and can dramatically reduce the number of converters required in some projects due to enhancements to the auto-mapping capabilities.  One project i’m using it on saw the converters drop from 34 to 14!

A summary of the release notes from the last week are below:

0.4.1

This is a small api change release that simplifies a couple of commonly used methods. It will require changes to any existing converters. Please see upgrade notes below.

• Changed the return type of the alwaysIgnoreFields() method on a converter from a HashSet<String> to an internally defined IgnoreSet which has a string varargs constructor and is chainable. This makes for simpler implementation as you can simply do this… return new IgnoreSet("field1", "field2", "fieldN").
• Removed the return type on the overridable convert(...) method of a converter as it was unnecessary. This method is now void.

Upgrade notes:

1. Change all converters that implement the convert(...) method to return void instead of the destination object.
2. Change all converters that implement the alwaysIgnoreProperties() method to return an IgnoreSet.

0.4.0

• You no longer need to implement the methods getSourceObjectClass() and getDestinationObjectClass() as these are inferred via generics.
• You can now set the object assembler to attempt to convert objects automatically if no converter is explicitly registered for the source / destination classes. This is very convenient for basic situations where there are no special rules for the conversion as no converter needs to be written.
• As an extension to the auto-mapping described above, you can also inject an EntityDao? for looking up JPA / Hibernate mapped entities by id when converting to an entity mapped using the javax.persistence.Entity annotation. Basic Hibernate and JPA implementations have been provided but you can plug-in your own as required. While I haven’t tested it with anything other than Hibernate, it’s highly likely that this will work with any JPA implementation but you’ll need to inject your own. One caveat with this however is that in order to look up the entity using the supplied EntityDao, SimpleObjectAssembler needs to know what field in your dto holds the id of the entity to fetch. To achieve this, annotate your dto with the EntityDto annotation supplying the field to get the entity id from. Eg. @EntityDto(id = “id”)
• Fixed problem where source objects must have a property for every method prefixed with ‘get’. This meant that methods that did things like combine other properties or perform calculations would cause the assembler to fail. These ‘computed’ getters can now be mapped directly to destination objects.
• Added default converters for mapping strings and numbers.

Going back a few years, it took a little while for me to realise the benefits of providing a clean separation of the core business domain from any consuming client, be it a web ui or remote service. I guess I always saw it as a lot of work (which it can be) and the simpler apps I was writing didn’t seem to need it – or at least I couldn’t really see the benefit at the time. While exposing dtos to the outside world is not necessarily a better option in all cases (many modern app frameworks offer rapid development because they don’t do this), i’ve found that erring on the side of caution in most of my development has set me up for a better experience as an application grows, allowing me to develop my domain without worry about how it might be used externally. This approach also has a nice side effect of allowing me to develop from the outside in, giving me exactly what I want in a web client and exactly what I want in my domain… often these are a little different.

I’ve used a few different techniques for converting domain objects to dtos and back again and it’s always turned out to be an ugly part of the application. I gave Dozer a go for a while and while it’s very capable, I found that once you have nested object graphs that need selective conversion for different scenarios, it became more and more difficult to follow, with more chance of things happening that you don’t want. Also having the mapping rules defined in xml, there were too many places to check for any different scenario. It’s been a while since I used Dozer and it’s quite possible that any statements I make about the way it works may be wrong so i’ll just say that it had some things a while back that lead me away from it and I haven’t gone back for another look since. So after trying Dozer out and not being entirely happy, I went back to having a pair of domain / dto assemblers that had dedicated methods for converting one object to another.

For example:

public class DomainAssembler {

public User assembleNewUser(NewUserRequest newUserRequest) {
//… manually convert objects here.
}

}

As you can imagine, this class will get large very quickly so you’ll want to break it down. Depending on the application, it might not be clear where or how much and what the deciding factor is. If you’re using course grained services, maybe separating assemblers on a per-service basis is the way to go. The thing with that is that you end up with lots of inter-dependencies between service based assemblers that is also hard to manage. The worst thing is that you also end up with lots of special methods for converting different parts of a graph. Clear naming is something I always find difficult, probably because I place a lot of importance on it.

So for example, this assembler has a three different methods for returning a different view of a user:

public class DtoAssembler {

public UserDto assemblerBasicUser(User user) {
//… manually convert objects here.
}

public UserDto assemblerUserWithProjects(User user) {
//… manually convert objects here.
}

public UserDto assemblerUserWithProjectsAndTasks(User user) {
//… manually convert objects here.
}

}

Depending on your application, this may or may not be a problem but you can see where it’s going. Anyway, this all bothered me a bit so I started writing something that has helped to simplify things for me a little on a couple of apps. There were a few basic requirements that I wanted to be able to satisfy and some assumptions about the environment that it would exist within. These included:

• Runs in a Spring 2.5.x based application and it’s ok to have a dependency on this but try to avoid if possible.
• Likely to be a Hibernate / JPA based application but again don’t tie to the Hibernate API if at all possible.
• Utilise Convention Over Configuration where possible but don’t impose it.
  • Fields of same names are mapped automatically
  • Fields of same name but different types can be mapped by registering converters for the combination. It’s recursive.
  • Fields of different names can be mapped by explicit configuration
  • Fields to be ignored can be configured
  • Any details about how much of an object graph should be populated must be configurable at runtime
• Can be completely overridden if necessary or at different stages in the conversion cycle.
• Configuration information is not defined in xml and is as close to the code that uses it as possible. This being a differentiator from Dozer.
• Should be able to detect problems in configuration such as mistyped property names.
• Destination objects should be able to be constructed in any way required but if not defined should be done reflectively.
• Should be capable of handling conversion of objects proxied by hibernate or JPA.
• Can handle mapping of collections, using existing objects within the destination collection when it makes sense for integrity or performance reasons.
• Can handle circular relationships
• Should use direct field access to negate the need for getters and setters simply for object conversion

Object Assembly / Conversion

The object assembler is the entry point to object conversion. It maintains a registry of ObjectConverters that are written by the application developer to convert one object to another. The registry enables a single converter to exist for one type to another.

You generally call the object assembler like this:

DestinationObject destinationObject objectAssembler.assemble(sourceObject, DestinationClass.class);

This will look up it’s registry for a converter with the same source / destination combination. A converter in it’s simplest form might look something like this:

public class SourceToDestinationTestObjectConverter extends    AbstractObjectConverter<SourceObject, DestinationObject> {

public Class<DestinationObject> getDestinationObjectClass() {
return DestinationObject.class;
}

public Class<SourceObject> getSourceObjectClass() {
return SourceObject.class;
}

}

You’ll notice that this converter extends the AbstractObjectConverter which has most of the conversion logic. The end result of calling this converter via the assembler as shown above is that all properties of the same name will be converted from the source to destination object. If two properties with the same name but different types are encountered, the registry will be consulted and the assembler will recursively call itself until all properties are converted.

Selective Conversion

If you want to exclude any part of the object graph from automatically being converted, you can do so in three ways.

1. By overriding the ‘disableAutoMapping()’ method on your converter implementation and returning ‘true’
2. By overriding the ‘alwaysIgnoreProperties()’ method on your converter and returning a Set of field names to never convert
3. By passing an array or var-args of property names to ignore into the assembler

The first two approaches are useful when you want this behaviour for every invocation of the converter but if you want to be able to specify what to convert upon invocation, the third approach is much more flexible.

Example:

DestinationObject destinationObject objectAssembler.assemble(sourceObject, DestinationClass.class, “ignoreFieldA”, “ignoreFieldB”);

The ignore field paths are aware of nested object graphs so it’s possible to ignore any part of the graph, no matter how deep.

DestinationObject destinationObject objectAssembler.assemble(sourceObject, DestinationClass.class, “ignoreCollection.fieldA”, “ignoreFieldB.fieldValue”);

This approach is very similar to web field binding except that collections are not indexed. This means that while you can refer to a field of an object within a collection, it will always apply to all objects in the collection, not a single one. For example you cannot specify ‘ignoreCollection[0].fieldName’ to ignore only the ‘fieldName’ property of the first element in the ignoreCollection. I can’t think of a situation where this would be that useful. Having said that, adding this capability would be reasonably straight forward.

Wildcard Exclusions

There are times when you want to convert an object that is a property of another but not actually convert any of it’s properties. Say for example, you want to look up a staff member’s manager from the database and populate the manager field in the staff member but you don’t want to actually map any of the ManagerDto fields onto the Manager object from the database. In this case you can use a wildcard in your ignorePath.

StaffMember staffMember objectAssembler.assemble(staffMemberDto, StaffMember.class, “manager.*”);

Mapping fields with different names

Sometimes your source and destination objects don’t have the same names. In this case, it’s possible to define pairs of ’source > destination’ names for mapping. This is done at the converter level by overriding the ‘customConverterFieldMappings()’ method. This method should return a Set of ConverterFieldMappings.

Configuration Validation

It’s easy to mistype a property name so you’ll want some checking to make sure your property names actually exist. Due to the way converters are registered, this can’t be done at startup time yet (hopefully it will soon). It can be done on first execution though so it’s easy to pick this up as part of your test suite. Any fields marked to ignore or fields mapped that don’t exist will result in an exception with a clear indication as to the source of the problem.

Constructing your own Destination Object

The default behaviour is for the converter to attempt to construct a destination object based on it’s destination type. This requires your destination object to have a default no-arg constructor. If you want to be able to construct your own object you can do this by overriding the createDestinationObject(SourceObject sourceObject) method. Because the sourceObject is passed into this method it’s entirely possible to manually do some conversion in here but it’s not the intended place and it’s possible that your values will be overridden during the actual conversion which happens next so it’s not advised. There is a method for this which i’ll come to. This is however a good place to do things like retrieve an object from the database based on the id of the source object. This is certainly the most common case.

Example of a Converter with the createDestinationObject method being used.

@Component
public class UserDtoToUserConverter extends AbstractObjectConverter<UserDto, User> {

private UserDao userDao;

@Autowired
public UserDtoToUserConverter(UserDao userDao) {
super();
this.userDao = userDao;
}

@Override
public User createDestinationObject(UserDto userDto) {
if (userDto.getId() == null) {
return new User();
} else {
return userDao.findById(userDto.getId());
}
}

public Class<UserDto> getSourceObjectClass() {
return UserDto.class;
}

public Class<User> getDestinationObjectClass() {
return User.class;
}

}

Custom Conversion Logic

There are cases where you need to perform some custom conversion logic because it’s not simple enough to just map one field to another. In this case you can override the convert(SourceObject sourceObject, DestinationObject destinationObject) method. This is called after all auto conversion has been performed and enables any custom conversion to be done. If you wanted to perform a full custom conversion for a converter you simply override disableAutoMapping() to return true and write your custom logic here.

Conclusion

This is not a perfect solution and it certainly won’t make sense in all situations. Some people, justifiably will take issue with the idea of passing strings of property fields to ignore / exclude into the assembler. I find this a bit awkward myself however i’m not sure that it’s really all that different to putting it outside in an xml file. I feel this line is starting to be crossed with some implementations of annotations vs their xml configuration approaches. Sure, it’s compiled code, but it’s also a lot more visible at the point at which it matters so there are some clarity benefits there. It would also be possible to externalise this configuration as an option but the more that’s done, the closer we get to Dozer which undeniably has a much deeper feature set and is almost certainly much ’smarter’ under the hood with the benefit of years in the wild. The intention for this was a simple solution to a simple mapping problem, sitting halfway between dozer and a fully handwritten approach. I came across an interesting post the other day by Gavin King, talking about the new typesafe Criteria API for JPA 2.0. I’ve been pondering whether one of the features they are using from the Java 6 compiler plugin architecture might be useful here although introducing yet another annotation could become a bit of a bore!

Another potential problem comes about when new properties are introduced to objects that are already mapped. It’s possible that an introduced relationship with another large part of the object graph could go unnoticed and result in conversion of much larger sets of objects than originally intended. I guess this is a trade off of taking blacklist vs whitelist approach for property mapping.

Get the Code

I’ve published the code on google code. At this point it’s not packaged up so you’ll have to check it out and build it from the source.  It’s a maven project though so this will be trivial as long as you’re familiar with maven.

I have found a couple of tutorials on this lying around the web, however they all take slightly different approaches, and although the information was useful, I think there is a cleaner way to do the integration. Keeping in mind of course that now that the spring guys have released the spring-flex module there will hopefully be complete integration with Spring Security soon without the need to even wire it up the way I describe here.

What I am going to describe here uses:

• Spring 2.5.5
• Spring-Security 2.0.4 (core & tiger)
• BlazeDS 3.2
• Flex3

My approach is not to protect end-points using BlazeDs’s security. The reason is that I often expose my services as web services as well as AMF endpoints and I don’t want to create my security mappings twice. Instead I use method invocation based security and protect the services directly. All that BlazeDS needs to do is get the users credentials out of the request and delegate to SpingSecurity to create a SecureContext, authenticate and put the authentication object in the SecureContext (effectively thread local). After that normal Spring Security applies and secure methods will be protected in the normal way.

A typical security scenario in one of my applications goes something like this: Either none of the application or a small part of the application is available anonymously as is the login service. Users access them without credentials, usually the User doesn’t even have menu options to navigate to UI areas with secure functions until they have logged in. After a successful login I will cache the user’s credentials on the client and re-log them in on each service request. This keeps the server completely stateless, all state is stored on the client. I configure a lightweight cache on the server since if someone logs in once they will likely make many more login requests (one for every service they are going to use). This reduces database hits. I implement a service registry on the client which stores the credentials and automatically adds them to all RemoteObjects and HttpServices.

I’m going to work this assuming that you already have a Flex/BlazeDS/Spring application going. It can be as simple as hello world, there is an excellent article on how to do this using the new spring-flex solution here . The way that i am applying security means that it is largely irrelevent whether you are using the Adobe SpringFactory or the spring-flex MessageBrokerHandlerAdapter.

Step1. Add spring-security to your web application’s lib my pom entry looks like this:

<dependency>

   <groupId>org.springframework.security</groupId>

   <artifactId>spring-security-core</artifactId>

   <version>2.0.4</version>

   </dependency>

<dependency>

<groupId>org.springframework.security</groupId>

<artifactId>spring-security-core-tiger</artifactId>

<version>2.0.4</version>

</dependency>

   </dependencies>

Step2. Add the spring security filter to your web.xml

<filter>

<filter-name>springSecurityFilterChain</filter-name>

<filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>

</filter>

<filter-mapping>

<filter-name>springSecurityFilterChain</filter-name>

<url-pattern>/data/messagebroker/*</url-pattern>

</filter-mapping>

This will secure everything and is mostly necessary if you are going to use other transports than just AMF via BlazeDS.
Step3. Configure Spring Security. I am just going to put a very basic config here. For details on configuring spring security you can look at the spring security documentation here

<security:http auto-config=“true”>

<security:intercept-url pattern=“/**” filters=“none”/>

</security:http>

<security:global-method-security>

<security:protect-pointcut

expression=“execution(* net.oneadam.app.services.*Manager.*(..))”

access=“ROLE_ADMIN”/>

</security:global-method-security>

<authentication-provider>

   <user-service>

<user name=”adam” password=”adamspassword” authorities=”ROLE_USER, ROLE_ADMIN” />

   <user name=”bob” password=”bobspassword” authorities=”ROLE_USER” />

   </user-service>

  </authentication-provider>

This configuration passes everything through at the URL level and relies on method level security. I have used pointcuts here since it doesn’t require me to cut and paste classes and methods in order to demonstrate how annotations work. The above pointcut requires the user for all methods in any class ending in the word Manager in the services package to have the role ROLE_ADMIN.
Step4. Implement the flex.messaging.security.LoginCommand Interface so that the doAuthentication creates a UsernamePasswordAuthenticationToken and authenticates using Spring Security, if it is successful it should put the authentication object into the SecureContext where it will be accessible to your application and the Spring Security annotations or security pointcuts. here is my implementation:

package net.oneadam.web.security;

import java.security.Principal;

import java.util.List;

import java.util.Map;

import javax.servlet.ServletConfig;

import javax.servlet.http.HttpServletRequest;

import org.apache.commons.logging.Log;

import org.apache.commons.logging.LogFactory;

import org.springframework.context.ApplicationContext;

import org.springframework.security.AbstractAuthenticationManager;

import org.springframework.security.Authentication;

import org.springframework.security.context.SecurityContextHolder;

import org.springframework.security.providers.AbstractAuthenticationToken;

import org.springframework.security.providers.UsernamePasswordAuthenticationToken;

import org.springframework.web.context.support.WebApplicationContextUtils;

import flex.messaging.FlexContext;

import flex.messaging.io.MessageIOConstants;

import flex.messaging.security.LoginCommand;

public class SpringSecurityLoginCommand implements LoginCommand {

   private static final Log LOG = LogFactory.getLog(SpringSecurityLoginCommand.class);

   private AbstractAuthenticationManager authenticationManager;

   public Principal doAuthentication(String username, Object credentials) {

String password = extractPassword(credentials);

if (password == null) {

   return null;

}

if(authenticationManager == null) {

   init();

}

Authentication auth = new UsernamePasswordAuthenticationToken(username, password);

try {

   SecurityContextHolder.getContext().setAuthentication(authenticationManager.authenticate(auth));

   LOG.info(“User [" + username + "] logged in.”);

   return (AbstractAuthenticationToken) SecurityContextHolder.getContext().getAuthentication();

}

catch (RuntimeException e) {

   LOG.info(“User [" + username + "] failed authentication.”);

   throw e;

}

   }

   /**

* We are not going to perform Authorization at this level since we would like SpringSecurity

* to manage Authorization at the method level.

*/

   @SuppressWarnings(“unchecked”)

   public boolean doAuthorization(Principal principal, List roles) {

return true;

   }

   public boolean logout(Principal principal) {

HttpServletRequest request = FlexContext.getHttpRequest();

if (request != null && request.getSession(false) != null) {

   try {

request.getSession().invalidate();

   }

   catch (IllegalStateException e) {

// Expected.

   }

}

request.getSession(true); // Session re-created to avoid Flex error when

// it also attempts to invalidate the session.

LOG.info(“User [" + principal.getName() + "] logged out.”);

return true;

   }

   public void start(ServletConfig config) {

//My tests show this never gets called

   }

   public void stop() {

authenticationManager = null;

   }

   @SuppressWarnings(“unchecked”)

   protected String extractPassword(Object credentials)

   {

   String password = null;

   if (credentials instanceof String)

   {

   password = (String)credentials;

   }

   else if (credentials instanceof Map)

   {

   password = (String)((Map)credentials).get(MessageIOConstants.SECURITY_CREDENTIALS);

   }

   return password;

   }

   private void init() {

HttpServletRequest request = FlexContext.getHttpRequest();

ApplicationContext ctx =

   WebApplicationContextUtils.getWebApplicationContext(request.getSession().getServletContext());

authenticationManager = (AbstractAuthenticationManager) ctx.getBean(“_authenticationManager”);

   }

}

You will notice that the doAuthentication method just returns true. That is because I am not interested in securing BlazeDS destinations. I prefer to secure the service facade methods themselves, that way if I expose them using CXF or some other Web Service transport my security does not have to be redone.
Step5. Wire up security in the services-config.xml file to use your SpringSecurityLoginCommand it will look something like this:

<security>

<login-command class=“com.renewtek.cmf.web.security.SpringSecurityLoginCommand” server=“Tomcat”>

   <per-client-authentication>true</per-client-authentication>

</login-command>

<security-constraint id=“roleuser”>

   <auth-method>Custom</auth-method>

   <roles>

<role>ROLE_USER</role>

<role>ROLE_ADMIN</role>

   </roles>

</security-constraint>

   </security>

The roles are actually unnecessary since I return true from all authorization requests at this level. I have just left them in for completeness. If you wanted to reject a request at this level that is where you would put the roles and if you are using the Adobe style of defining destinations in your remoting-config.xml file then you could say what destinations are allowed for what roles. What is important to the strategy I described above is the per-client-authentication entry. it needs to be set to true since we don’t want to use server based sessions. it is false by default.

Optimistic locking in JPA relies on a version field which can be a number or a date of some type. Using date objects can be problematic if they are not detailed enough. For example a date which does not persist at least milliseconds is likely not to be accurate enough to do versioning. I generally use a Long since it is going to survive a very long time indeed, even in a high transaction system. The version filed is identified using the @Version annotation and when an object containing a version field is updated JPA will read up the version number in the database before attempting to do the update and if the version numbers do not match then it will throw an OptimisticLockException. It is a good practice to make the version field private and not to provide getters and setters. it is critical that the field not be altered except by the JPA implementation.

Ok, so far so good, we implement this strategy on systems where we have a greater proportion of reads to writes and we would like data to be highly available. Basically a user can always get a read, no matter what, however if they want to do a write we will check that the object they are modifying has not been written to since they read it. Very many web based application use this strategy today and as I described above it is supported out of the box with JPA. Of course there are endless arguments about how you manage the OptimisticLockException, but that is a separate issue.

When your web application is using DTOs or some other representation of the system data which is not your actual domain another level of complexity enters the frame. If the DTOs don’t carry the version field with them and re-present it for the update then the whole optimistic lock strategy is dead in the water. If the web tier represents an object without keeping the version field then when it the user requests to update the domain, JPA must retrieve it again, over-write the fields provided and then perform the update. Of course if someone had changed the data between the user’s read and write, that would be over-written.

So we have established that maintaining the version field into the web tier is critical. We already established that the version field should not be user modifiable so it would be good to make the version field in the DTOs private as well. If you are hand mapping between DTOs and your domain then you will need to use reflection to map this field across. I generally user dozer and it allows you to set is-accessible=”true” in a field description which tells it to access this field directly without using the getter and setter.

When you add Flex and BlazeDS to the picture (and of course that means an ActionScript domain) you add a whole new level of complexity and at this point a real problem arises. Of course BlazeDS will auto-magically transform your Java DTOs into your ActionScript domain and perform all transport in AMF3. Thats peachy, but unfortunately it does not transport private fields without getters or setters. That means that although your version field gets mapped onto the DTO when it arrives on the Flex client, it is gone.

Well that just won’t do, we already established that without the version field we have no locking strategy at all. Currently my only work around is to provide getters and setters on the DTOs. This ensures that they make the trip to the client and back but it risks the client accidentally altering them and inadvertently causing an OptimisticLockException. I’m a little unhappy with this and intend to spend a little bit of time with the BlazeDS source code to see if there is something that can be done to allow private field mapping. If you are interested stay tuned or feel free to post your ideas as comments here.

WTP and M2Eclipse do not play very nicely together. We have identified 2 main problems:

1. Converting Maven projects to WTP projects so that they can be auto-deployed to the servlet container.
2. Multi-modules projects where the web project has dependencies on one or more of the other modules and you want to be able to debug freely into any module. In this case if you allow a project per module, the other modules never seem to get deployed to the web project.

I will deal with issue 1 first. Most of this solution came from a posting on the eclipse.webtools newsgroup, however it is quite incomplete with respect to Maven projects:

1. Open .project file for the project to be converted to web project.
2. Insert the following to natures element:
   <nature>org.eclipse.wst.common.project.facet.core.nature</nature>
   <nature>org.eclipse.wst.common.modulecore.ModuleCoreNature</nature>
3. Open the properties for the project and go to the section for Project Facets
4. Add Java and Dynamic Web Module facets to the project. (I forgot the Java facet first time around and had a lot of confusion until I worked it out)

The above will work for a non-maven project the rest you will need if you are using m2eclipse:

Open the .classpath file and add the following:

<classpathentry exported=”true” kind=”con” path=”org.maven.ide.eclipse.MAVEN2_CLASSPATH_CONTAINER”>
<attributes>
<attribute name=”org.eclipse.jst.component.dependency” value=”/WEB-INF/lib”/>
</attributes>
</classpathentry>

Open the org.eclipse.wst.common.component file in the .settings folder of your project and add this line inside the wb-resource block:<wb-resource deploy-path=”/” source-path=”<nested module name if you have one>/src/main/webapp”/>

Issue 2 is really annoying to me since most of our applications use multiple modules to decouple the services from the way they are exposed, ie. via a Java MVC framework, Web Services or via BlazeDs to a Flex RIA. Having this limitation forces us to have a single eclipse project for the services and the web tier or transport tier. This can have the effect of hiding coupling between your services and the web or transport tier.
Nevertheless it is a reality that has to be dealt with right now. You can manage it very easily in IntelliJ, however, even there I can, as yet, find no way to tell the editor that the service tier jar source code is actually in the editor so that I can set break points. Probably this could be dealt with but I was reluctant to spend time on it when I know our management would flip if I suggested paying for all developers to have IntelliJ  licenses
Okay, so the way to solve this problem with m2eclipse is to have the service or core application and the web server application as a single project with multiple maven modules. This sucks for a couple of reasons. The first is that if you have decomposed your application into more that 2 modules it is very hard to manage all the source folders in eclipse. The second reason I mentioned above, this approach will actually have eclipse compiling all classes into the one class folder and deploying that to your web server once you set WTP up. I have seen a number of instances where this effectively makes decomposing your application into multiple modules a waste of time or worse since it gives you the impression that you are enforcing a decoupling when in fact you are not.
If you use this approach then the instructions above still apply in terms of setting up WTP once you have created your multi-module project. In the last step you will need to prepend the web server module name to src/main/webapp (the angle brackets in the description above show exaclty where this goes).
I hope this helps someone, at the very least it will help me the next time I have to deal with this.

I was finding that test classes were somehow ending up compiled into the web application deployed onto Tomcat. I spent a fair bit of energy reading through the Eclipse ETP help files but there wasn’t any reference that I could find to reconfiguring an existing dynamic web application. In desperation I started hunting through the configuration files under the .settings folder in my project ad low and behold there i found it.

Just in case i forget or someone else has this problem the file is called org.eclipse.wst.common.component in that file are wb-resource entries for each source folder which is being compiled and/or copied into the classes directory under WEB-INF. I deleted the entries for source/test/java and source/test/resources and that took care of things.

The main reason for using mock objects to write unit tests is so that they are unit tests. Without mock implementations, unit tests quickly become integration tests. There is nothing wrong with integration tests however they tend to make coverage data inaccurate, since code which is exercised incidentally rather than being directly tested gets marked as covered, and they can end up slowing down the test suite since these types of tests usually involve having to load the complete application context in a Spring application or they require a container and a JNDI instance for other J2EE apps.

My motto is keep it simple. I don’t like to think too much, particularly about my tests. When I see a red bar I don’t want to have to wonder whether it is the method under test which is broken or if it is one of its dependencies. I like to be certain that it is the method under test. It greatly reduces the time it takes me to get the green bar of goodness back.

I, personally, use EasyMock as my mock framework of choice. Combined with Unitils it takes a great deal of drudgery out of mocking out dependencies. Using the Unitils EasyMock annotations you can mark an interface as a mock using @Mock and mark the class under test which will have the mocks injected using @TestedObject. This doesn’t seem to work on classes using constructor injection, however it works fine for setter injection and means there is no need to manually setup your mocks and inject them into the class under test.

Complete details on how to use these annotations can be found here

It has been quite a while since I posted on Eclipse plugins I am using. All the URLs have changed and so have many of the plugins I find essential for development. Here is an update on the plugins I recommended last time and the ones I’m using now:

1. MyEclipse: I have ditched this as an unstable, annoying, and largely useless waste of money. Ganymede WST provides all the Sever hot deploy stuff that was good in the older MyEclipse editions out of the box, most of the rest can be found in various free  plugins I will detail here. When I was working mostly in Windows or Linux this plugin was okay, but on the Mac MyEclipse crashes almost constantly and is a complete time waster. Virtually nothing interesting, which might save you money even works on the Mac, ie. UML, Javascript debug, image editing. Also the new installer is crap… I’m over these guys, $60/yr is a nice price point for a useful plugin set but really this is worse than useless if you are a Mac user.
2. Spring IDE – If you are doing Spring development you cannot live without this plugin. Provides Spring configuration validation and help with code completion. URL: http://springide.org/updatesite/
3. Subclipse: I have trashed this. The conflict resolution was kind of buggy and failure prone. Maybe it was operator error, but I got sick of trying to manage broken merges and migrated to Subversive.
4. Subversive: This is becoming the Eclipse default SVN plugin. Why Ganymede does not have an SVN plugin out of the box is beyond me, is anyone still using CVS these days??? The only downside of Subversive is that it is slightly tricky to install as the install instruction have you jumping all over the web to get the update site urls for the connectors and the actual plugin. Here is the long and the short of it…
   1. Plugin site URL:http://download.eclipse.org/technology/subversive/0.7/update-site/
   2. Connector site URL: http://www.polarion.org/projects/subversive/download/eclipse/2.0/update-site/

   If you are using OS X or Linux you will need to unselect some of the windows JavaHL connectors

5. M2Eclipse: This plugin has really come a long way since it was released a year or so ago. Previously the only useful thing about it was the repository search. Now it allows quite reasonable management of your Maven 2 project from within Eclipse. For those afraid of editing XML the new pom editor is really useful. I still don’t think the integration is quite as seamless as IntelliJ but it is now quite acceptable. URL: http://m2eclipse.sonatype.org/update/
6. Eclemma is still the best free coverage plugin I am aware of. Again it is annoying that the IntelliJ emma integration is more seamless and intuitive but the Eclipse integration is quite workable. I had troubles installing this on Ubuntu, although it installed fine on my Mac. If you have problems it is quite simple to go to the eclemma site (www.eclemma.org) and download it. Unzip it and drop the contents of the plugin/ and features/ folders into their respective folders under your eclipse installation. URL: http://update.eclemma.org/
7. PMD: This plugin is very handy for getting quick feedback on code style. We have been using PMD for a while and have a company ruleset.xml. I like the way you can prioritize problems and then filter by priority in the IDE, but I am still in 2 minds as to whether this is actually better than Checkstyle. URL:  http://pmd.sourceforge.net/eclipse

This should be obvious, but I seem to find myself re-explaining this concept quite a bit. Without testing there can be no Agile development. Tests are not an optional extra, there can be no negotiation. If you are doing agile, you are doing tests. You can test first, my personal preference, or you can test immediately after, but you must write tests.

One of the core values of the Agile manifesto is responding to change over following a plan. Agile development is predicated on writing the simplest solution which meets business requirements and nothing more (YAGNI principle). This can only be done if the developer(s) are able to refactor later as requirements become more complex or change.

Developers will either not refactor at all or take an extremely long time to achieve refactorings if they don’t have tests in place that give them confidence that their changes have not broken existing functionality. No test, no agile. Other features of agile development may or may not be optional, but testing is definitely not.

Powerful ORM frameworks like hibernate and Ibatis and more recently JPA based ORM frameworks like Toplink, Hibernate and openJPA have taken a great deal of drudgery and error prone code out of the average enterprise application codebase. Many IDEs now provide powerful tool support for writing and maintaining code based on these frameworks. One effect of all this has been to move attention away from the database and into code, particularly in instances where developers are using automatic ddl generation from their ORM framework.

In such a situation, developers often don’t think about the effect that changes to their java code are having on the underlying database. Before there has been an initial release this may not necessitate migration of production data, but it can lead to large amounts of time wasted when changes in code necessitate schema changes, particularly if these changes clash with schemas other developers have established in their development environment. After an initial production release it can lead to serious problems if developers alter domain objects which are mapped using ORM frameworks. Simple XML or Annotation changes can wipe out or make redundant whole segments of the production database when released into the wild, or completely alter the semantics of existing data.

There is no doubt that a new discipline is required for developers when writing code using ORM frameworks. Paritcularly where there are multiple developers and potentially multiple production releases (pretty well every agile project I have ever known). This discipline should focus on the database schema that ORM frameworks are generating. Such schemas need to be versioned in the same way as the codebase is versioned. Luckily a tool called Liquibase exist for exactly this problem.

Liquibase provides for the creation and maintanence of database schemas and their data. I have not yet used it to make changes on production databases but we have been using it in development and it has had a very positive effect on productivity as developers no longer have to spend time working out why their tests are no longer working only to eventually discover that it is due to data changes conflicting with their existing schema.

Liquibase manages schemas and data through change sets and the database itself. It stores the change set ids in the database along with a checksum for the change set so that it can determine if the change set has been altered, invalidating it against that database.

With Liquibase in place the database can be generated and/or updated without the use of the ORM frameworks auto generation of DDL, in fact it is essential that this be turned of. For Spring applications a data migrator bean can be configured to automatically generate or update the database on startup of the application context and check that the schema is valid for application data mappings. If this is not desirable Liquibase has Ant targets and Maven goals that can be executed.