Moving on....

I am currently moving my old blog entries to this new blog. Some old posts will not be transferred because time and software also moved on. The old blog address will be abandoned and disappear in the near future.

ActiveMQ and Network Interfaces

When you are using ActiveMQ as a message broker, the need can arise to bind the queue listener to a specific network-interface. Although this is not very common and some argue that the OS should do the routing, you can face certain situations in which your software decides which network-interface SHOULD or MUST be used for the communication.

The reason could be that some communication-channels are expensive and the machine needs to switch network-interfaces on the fly because of you business logic. When the servers are not under your control you cannot simply change the OS routing tables.

There is a simple trick in ActiveMQ to select the network-interface to use, although you cannot find it in the manual -- but it is in the code. Just use the following scheme for the binding:

tcp://remote-queue-ip:61616@network-interface-ip:61616

You can also use other protocols, of course.

A basic outline of a workflow system using J2EE and CDI

For one of my customers, we needed a very simple workflow-framework which coordinates the flow between methods inside a J2EE6 application on Glassfish. The business required to implement the following simple logic:

1. Create a job.
2. When step 1 was successful, then start a job (which affects some thousand objects) and send an email
3. End a job when step 2 was successfull.
4. Mark the job as failed when one of the steps couldn't be completed due to the implemented business-logic.
5. Provide a handler to handle exceptions.

Taking a fully fledged workflow-system was way over the top to solve this problem. The workflow will also remain unchanged; and when it is changed, the code needed to be altered to. We needed to come up with a simple workflow-management which is bound to transitions and methods.

So, first we decided to create an observer the change the transitions of the workflow-steps. This is basically a very simple method which alters the status and stores it in the database. The next step was to create some annotations to add to the methods which needed to be executed during a certain step. We annotate our workflow methods as follows:

    @Asynchronous
    @TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
    @Workflow
    @WorkflowStep(from = Status.IDLE, to = Status.PROGRESS)
    public void startJob(
            @WorkflowActionHandler
            @Observes(notifyObserver = Reception.ALWAYS,
            during = TransactionPhase.AFTER_SUCCESS)
            final JobWorkflowEvent event) {}

This straight-forward approach is very clean and simple. Basically it says that this method will be executed after the transition from a Status.IDLE to Status.PROGRESS. Although we use an enum here, you could take any arbitrary integer. Using the CDI annotations we get some additional power. This method will only be executed after the success of the previous step. Here you can combine the full power of CDI with some basic workflow concepts to create a nice system.

Now remains the problem of the transition handling. The transitions are handled in an interceptor which is marked by the @Workflow annotation.

    @Interceptor @Workflow
    public class WorkflowInterceptor {
    }

This interceptor does not do much more than changing the transition state when the method pinged by CDI has the correct workflow-steps. It uses some reflection to figure that out and allows access to the method. 

To handle exceptions, we introduce a new annotation:

    @Asynchronous  
    @TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
    @Workflow @WorkflowException
    public void excJob(
            @WorkflowActionHandler 
            @Observes(notifyObserver = Reception.ALWAYS, 
            during = TransactionPhase.AFTER_FAILURE) 
            final JobWorkflowEvent event) {}

Whenever one step fails unexpected, control is transferred to this method where certains actions can be executed. We need this annotation to prevent other methods to pick this event up. 

Using CDI together with some custom annotations really did the job and works fine.

JPA, Criteria and custom objects

Suppose you have a SQL-statement where you select custom fields and or the result of some functions. To make it more interesting, we want to combine this with a group-by statement and an ordering. Let's us for example say you want to execute the following query in MySQL:

SELECT uuid, GROUP_CONCAT(field1, ':', field2), 
  MIN(ss_date_started), 
  MAX(ss_date_ended) 
FROM my_table 
WHERE key = 'xxx' 
GROUP BY group_key 
ORDER BY id ASC

The functions GROUP_CONCAT, MIN and MAX are availble in MySQL. The intention of the GROUP_CONCAT is to get some information out of all the group-rows with issuing additional selects. The question is how we can create the criteria for this query and how we can retrieve the results. 

 

We want the results to be stored in the following object:

 

public class MyGroup implements Serializable { 

    private String uuid;
    private Map<String, String> map = new HashMap<>();
    private Date dateStarted;
    private Date dateEnded; 

    public MyGroup() {
    }
}

In the map we want to store the GROUP_CONCAT strings as a map to avoid other SELECTs. First of all, we get the CriteriaBuilder and the root of the query. The MyEntityForTheTable is the entity which is mapped to the table. This object will act as the base to map our fields.

final CriteriaBuilder cb = 
  this.getEntityManager().getCriteriaBuilder();

final CriteriaQuery< ... > cq = ...; // we'll talk about this later

final Root<MyEntityForTheTable> root = 
  cq.from(MyEntityForTheTable.class); 

The WHERE clause is pretty simple:

cq.where(cb.equal(root.get(MyEntityForTheTable_.key), key));

The GROUP-BY and the ORDER-BY is even more simple:

cq.groupBy(root.get(MyEntityForTheTable_.groupKey)).
   orderBy(cb.asc(root.get(MyEntityForTheTable_.id)));

Now we need to construct the function calls. We extract the MIN and MAX date out of the group using by writing the following expressions:

cb.function("MIN", Date.class, 
    root.get(MyEntityForTheTable_.dateStarted))

cb.function("MAX", Date.class, 
   root.get(MyEntityForTheTable_.dateEnded)) 

We start with the GROUP_CONCAT in MySQL which can be written as:

cb.function("GROUP_CONCAT", byte[].class, 
    root.get(MyEntityForTheTable_.field1), 
    cb.literal(":"), 
    root.get(MyEntityForTheTable_.field2))

First of all, we cannot take String.class to read out the results. MySQL returns the string as byte-code, so we need to map it to a byte[]. The following arguments are the parameters for the function. The first and the last is the result of the columns. We also want to have a semicolon between the two values which can be achieved by cb.literal(String) function.
Now we need to execute the queries and map it to an object. We can use the construct() method to instantiate a new object inside our query. Unfortunately, our main domain object does not have the proper constructor. So we wrap the POJO class in a new class and add the specific constructor.

 
public static class WrappedGroup extends MyGroup {

  public WrappedGroup() { }

  public WrappedGroup(
     final String uuid,
     final byte[] serviceStatus,
     final Date dateStarted,
     final Date dateEnded) { }
}

We make sure that this constructor has the byte[] parameter. In this constructor we can convert the byte-array to a string. The you can convert the string to a map of your choice. So, we are almost done. Here is the complete code:

final CriteriaBuilder cb = 
  this.getEntityManager().getCriteriaBuilder();

final CriteriaQuery<WrappedGroup> cq = 
  cb.createQuery(WrappedGroup.class);

final Root<MyEntityForTheTable> root = 
  cq.from(MyEntityForTheTable.class); 

And the select part looks like;

cq.select(cb.construct(WrappedGroup.class,
   root.get(MyEntityForTheTable_.uuid), 
   cb.function("GROUP_CONCAT", byte[].class, 
      root.get(MyEntityForTheTable_.field1), 
      cb.literal(":"), 
      root.get(MyEntityForTheTable_.field2)),
   cb.function("MIN", Date.class, 
      root.get(MyEntityForTheTable_.dateStarted)),
   cb.function("MAX", Date.class, 
      root.get(MyEntityForTheTable_.dateEnded)))); 

We then use a TypedQuery to get the results, and we are done:

final TypedQuery<? extends MyGroup> query =
   this.getEntityManager().createQuery(cq).
   setFirstResult(startResult).
   setMaxResults(maxResults);

The resulting list can be cast to List<MyGroup>.

Et voilá, we have a very dynamic query which we can easily refactor and adapt.

Startup script for Ubuntu

When you need to start a service (like Glassfish or Play) in Ubuntu, you can take this script as a template:

#! /bin/sh
case "$1" in
    start)
 ... execute a shell script here ...
        ;;
    stop)
       ... execute a shell script here ...
        ;;
    restart)
       ... execute a shell script here ...
        ;;
esac
exit 0

Make sure to make your scripts executable. In order to get it running during startup, add a link to the init.d directory and register the service-script to make it run as first service (depends on your system which priority you assign, take care here, I just took 01 as an example) 

cd /etc/init.d
sudo ln -sf  ~/your-script.sh your-script.sh
sudo update-rc.d your-script.sh defaults 01 01

Reboot and you'll see that your services get started.

Setting the UTF8 character set in Ubuntu or Debian in EC2

The EC2 AMI sometimes have the wrong character set. This could be very annoying in for example the German regio where characters like ä und ö are displayed incorrectly. Issue the following commands to switch your Ubuntu/Debian server to the correct character set:

Make sure these variables are set:

export LANGUAGE=en_US.UTF-8

export LANG=en_US.UTF-8

export LC_ALL=en_US.UTF-8

And don't forget to reconfigure the character set.

locale-gen en_US.UTF-8

dpkg-reconfigure locales

ImageMagick, jMagick and Glassfish on a EC2 machine

Recently I had to roll out a REST-Webservice on Glassfish which used jMagick as a library. Now, ImageMagick is a C++ library and is available in Java through JNI. The installation on a standard EC2-Ubuntu AMI is pretty straightforward once you know the tricks. I executed the following steps to get things up and running:

1. Make an EC2-Instance, attach an EBS and install JDK7 and GFv3.2.1 (these are the version which I installed)
    
2. Install ImageMagick using sudo yum install ImageMagick. You can't use apt-get or aptitude since Amazon prefers yum.
    
3. Next thing is to install jMagick. You can download the latest from ftp://ftp.imagemagick.org/pub/ImageMagick/java/. Use wget to get the RPM (32 or 64 bit - I used the 64bit version).
   wget ftp://ftp.imagemagick.org/pub/ImageMagick/java/jmagick-6.4.0-3.x86_64.rpm
   
4. Install the RPM using sudo yum  jmagick-6.4.0-3.x86_64.rpm
    
5. When all goes well, you should be able to see a handful of Magick-files using ls /usr/lib64/libM* (which were installed in step 2) and of course the Java library ls /usr/lib64/libJMagick.so
   
6. There should also be an accompagning JAR file for the library. Check if /usr/lib64/jmagick-6.4.0.jar is there.
    
7. Now we get to the Glassfish part. Suppose your domain is called "domain1". Copy the JAR into the /lib directory of your domain so it will get loaded during the startup of Glassfish.
    
8. Point your browser to the admin-console or open the domain.xml. You need to add a JVM-parameter to get this running. In the console go to the configurations / server-config / JVM-settings and then the JVM-options tab and add the following line:
   
   -Djmagick.systemclassloader=no
   
   or add in the domain.xml the line
   
   <jvm-options>-Djmagick.systemclassloader=no</jvm-options>
   This line prevents jMagick of using the system-class-loader as you might have expected. Not adding this line leads to errors.
    
9. Restart your domain and deploy the application. The System.loadLibrary("JMagick"); in Magick.java runs as planned. (http://jmagick.svn.sourceforge.net/viewvc/jmagick/trunk/src/magick/Magick.java?revision=91&view=markup) 

We can now redeploy our application in Glassfish at will because the class is loaded by Glassfish during the startup. When not, you can expect exception when redeployen when your jMagick.JAR is in your project. The classloader will tell you that the classes are already loaded when you deploy a second time.

PS: when running Maven tests outside the container, do not forget to specifiy the -Djava.library.path=/your-path-to/libJMagick.so. 

PS2: if you install JMagick using a repository, then the JMagick.so files are stored in the /usr/lib64/jni directory. Copy the libJMagick.so file to /usr/lib64 and restart your domain. There is no need to add java.library.path to the JVM parameters in the domain.xml (something which apparently doesn't function well).

Copying symlinks

JDK7 offers a whole new set of methods to work with symbolic links. This was in pre-JDK7 times pretty difficult and often you had to execute barebone copy statements through the runtime. There is one method which I missed in the API. You can open or follow the link, or test whether your file is a link, but you cannot "copy" the link itself. You need to resolve the original link and create a new link as illustrated in the class below.

The class could surely be optimized, but it gives you an idea on how the symbolic links function.

public class LinkFileVisitor extends SimpleFileVisitor<Path> {

    private Path fromPath;

    private Path toPath;

    public LinkFileVisitor(final Path fromPath, final Path toPath) {
        this.fromPath = fromPath;
        this.toPath = toPath;
    }

  public static void copy(final Path from, final Path to)
            throws IOException {

        // make sure it is a directory
        if (!Files.isDirectory(from)) {
            throw new IllegalArgumentException(
                    String.format("%s is not a directory", from.toString()));
        }

        Files.walkFileTree(from, new HashSet(),
                Integer.MAX_VALUE, new LinkFileVisitor(from, to));

    }

    @Override
    public FileVisitResult preVisitDirectory(final Path dir,
            final BasicFileAttributes attrs)
            throws IOException {
        Path targetPath = toPath.resolve(fromPath.relativize(dir));
        if (!Files.exists(targetPath)) {
            Files.createDirectory(targetPath);
        }
        return FileVisitResult.CONTINUE;
    }

    @Override
    public FileVisitResult visitFile(Path file, BasicFileAttributes attrs)
            throws IOException {
         if (Files.isSymbolicLink(file)) {
            Path link = toPath.resolve(fromPath.relativize(file));
            if (Files.exists(link, LinkOption.NOFOLLOW_LINKS)) {
                Files.delete(link);
            }
            Files.createSymbolicLink(link, Files.readSymbolicLink(file));
        } else {
            Files.copy(file, toPath.resolve(fromPath.relativize(file)),
                    StandardCopyOption.REPLACE_EXISTING);
        }
        return FileVisitResult.CONTINUE;
    }
}

Is a @Singleton bean really a singleton?

Well, as you might have expected from the title, the answer is "yeah well, not really". A singleton in the J2EE world is not a singleton as you know it from the normal Java world. In a non-J2EE environment you typically write something like this to generate a singleton:

public class MySingleton {

     final private static MySingleton instance = new MySingleton();

     protected String name = "whatever";

     private MySingleton() {}

     public static MySingleton instance() {
          return instance;
     }
}

You can get exactly one instance of this bean and you could access the fields directly or through getter and setters. There is only one instance of this object and all information is stored there. This pattern is well known to every programmer. The demand was great to port this pattern to the J2EE world. A new type of bean was invented, the Singleton bean. The idea behind all this is to provide a single instance of a particular bean within your application. You can construct a singleton bean with the following code:

     @LocalBean
     @Singleton
     @Startup
     public class MySingleton {

     }

First we declare it as a local bean, and we load it during startup. The @Singleton annotation makes a singleton out of this class. But is this class really instantiated once when using the @LocalBean annotation? No, it is instantiated twice! Let's test this out with by adding a static field in the bean. 

    static int i=1;

    public MySingleton() {
        System.out.println("INSTANCE: " + i++);
    } 

You will see in the logs that you get two different instances. If you print out the reference of the class, you will notice that one of the instances is a proxy object. The proxy object is instantiated too and the constructor is instantiated again. When you write some kind of initialisation logic inside the constructor, you will run into somekind of unwanted behaviour because it executes your code twice where you expect that the constructor is executed only once. So, don't use the constructor to set up your data but use the @PostConstruct annotation to execute a method with your initialisation logic in it.

    @PostConstruct
    public void init() {
     ... do your init here ....
    }         

You will see that the proxy object does not execute this code (which is obvious of course because the code is not in the constructor anymore). Another pitfall might be that during some kind of rapid prototyping action you store some state-data in a non-private field in the singleton or a stateful session bean. When you do not provide a getter/setter for the field, the field-value of the injected bean will always be null because the proxy gets injected and not the "real" instance. The proxy object does not store state. Let's test this:

     @LocalBean
     @Singleton
     @Startup
     public class MySingleton {
         protected String name = "whatever";
         .....
         public String getName() {
              return this.name;
         }
     }

We inject this bean in another session: 

     @LocalBean
     @Stateless
     public class MyTest {

          @EJB private MySingleton mySingleton;
          public void test() {
               // mySingleton.name --> null; (proxy)
               // mySingleton.getName() ---> "whatever" (real object)

         }
     }

The injected reference is the proxy object without state. Getting the value of the field directly must always return null when it is not initialised. The real state is maintained in the singleton. You can get the "real" data when you use the method "getName()" because the proxy object tunnels your method and not the field. This is the reason behind the fact that you may not use fields directly in session-beans. 

Well, as you might have expected, it is really a bad idea to get the field from a singleton or other bean directly (in general, it is an anti-pattern in J2EE). Try to encapsulate your data with nice getters and setters and keep in mind that your objects can get dynamic proxies depending on the annotations you add to the class.