Clojure Atom, Swap!, Reset! and Concurrency

I've encourage myself to keep writing simple Clojure codes. Obviously, this encouragement should extend beyond the traditional "Hello World!" program, shouldn't they?

Recently, I've attempted to write a in-memory database program that keeps track of person record. This in-memory database is nothing but a list (in an abstract sense) and each element in the list contains a map of properties that describes a person.

So, we'll have a variable called "db-ref" bounded to an atom of Clojure vector:

(def db-ref (atom []))

We also need a function to create a person containing id, first name, last name and email :

(defn person [pid lname fname email]

  {:id pid :lname lname :fname fname :email email})

(Does this remind you of a factory pattern?)



To add a person to the database

(defn add-person [p]

  (swap! db-ref conj p))

As you can see, we're adding the parameter "p" (for person) into db-ref using "swap!"



We also need a way to remove a person by their "id". All we have to do is use the same pattern as "add-person", but as oppose to conjoining, we use the remove function (how difficult could this be?):

(defn find-id-p [id element]

  (let [k (:id element)]

    (if (= id k)

      element)))

(defn delete-person [id]

  (swap! db-ref (into [] (remove (partial find-id-p id) @db-ref))))

Surely, our test code below should quickly tell us how awesome we are after the first attempt...

(deftest delete-person-test

  (testing

  (is (= 0 (count-db)))

  (add-person (person "100" "fred" "flintstone" "fred@flintstone"))

  (add-person (person "101" "barney" "rubble" "barney@rubble"))

  (is (= 2 (count-db)))

  (delete-person "100")

  (is (= 1 (count-db)))

  (delete-person "101")

  (is (= 0 (count-db)))))

Running "lein test":

ERROR in (delete-person-test) (APersistentVector.java:292)

Uncaught exception, not in assertion.

expected: nil

  actual: java.lang.IllegalArgumentException: Key must be integer

 at clojure.lang.APersistentVector.invoke (APersistentVector.java:292)

  ...

  ...

  ...

Oopsie! A big fat error telling us something is terribly wrong. Maybe we could sidestep the problem by using reset! function.

(defn delete-person [id]

  (reset! db-ref (into [] (remove (partial find-id-p id) @db-ref))))

Re-running the test should tell us we could totally sidestep the issue.

0 failures, 0 errors.

But before we conclude, Clojure documentation says,

(reset! atom newval)

Sets the value of atom to newval without regard for the
current value. Returns newval.

Our in-memory database may not work correctly in concurrent environments.

Alright, let's try to revisit the problem. Afterall, we're awesome aren't we?

The swap! function documentation says:

(swap! atom f) (swap! atom f x) (swap! atom f x y) 

    (swap! atom f x y & args)

Atomically swaps the value of atom to be:
(apply f current-value-of-atom args). Note that f may be called
multiple times, and thus should be free of side effects.  Returns
the value that was swapped in.

It looks like we shouldn't dereference "db-ref" when calling on swap.

Below is another take to delete a person using the swap! function:

(defn remove-person [id db-ref]

  (remove #(find-id-p id %) db-ref))

(defn delete-person [id]

  (swap! db-ref (partial remove-person id)))

We added an extra helper function to call on the actual remove function. 

Running the test displays:

0 failures, 0 errors.

Destructuring Clojure Vector and Map

Clojure documentation describes destructuring as:

Destructuring is a way to concisely bind names to the values inside a data structure. Destructuring allows us to write more concise and readable code.

So, what does this mean?

Suppose we have a vector that contains the bridge officers found in Star Trek original series:

(def star-trek-bridge-officers [ :kirk :spock :sulu :uhura ])

To retrieve the values, we could destructure the elements this way:

(let [[a b c d ] star-trek-bridge-officers] 

    println (str a " " b " " c " " d))

;; ":kirk :spock :sulu :uhura"

What happened here was the keyword :kirk is bound to the variable a, :spock to b and so on and so forth.


Suppose we have a slightly different vector, where :sulu and :uhura belongs to another subvector:

(def star-trek-bridge-officers-2 [ :kirk :spock [:sulu :uhura]])

To retrieve the values:

(let [[a b [c d]] star-trek-bridge-officers-2] println (str a " " b " " c " " d))

;; ":kirk :spock :sulu :uhura"

If we're not interested with Spock and Uhura, we replace the variables with underscore character "_":

(let [[a _ [c _]] star-trek-bridge-officers-2] 

    println (str a " " c))

;; ":kirk :sulu"

Jim, you can't risk your life on theory!

Now, let's talk about destructuring a map.

Suppose we have a map that contains the rank and names of the crew:

(def star-trek-crew { :captain "kirk" :lt-cmd "spock" :lt-1 "sulu" :lt-2 "uhura" })

To retrieve the values by the keys:

(let [{a :captain b :lt-cmd c :lt-1 d :lt-2 } star-trek-bridge-officers] 

    println (str a " " b " " c " " d))

;; "kirk spock sulu uhura"

The variable a is bound to the value of the key :captain and b is bound to the value of the key :lt-cmd.

To complicate the map structure a bit, I'm going to add a few more members as an embedded map:

(def star-trek-officers { :captain "kirk" :lt-cmd "spock" :lt-1 "sulu" :lt-2 "uhura" 

    :non-bridge-officers {:lt-cmd-1 "scotty" :lt-cmd-2 "mccoy" :nurse "chapel"}})

To retrieve the values including the embedded map:

(let [{a :captain b :lt-cmd c :lt-1 d :lt-2 {x :lt-cmd-1 y :lt-cmd-2 z :nurse} 

    :non-bridge-officers} star-trek-officers] 

        println (str a " " b " " c " " d " " x " " y " " z))

;; "kirk spock sulu uhura scotty mccoy chapel"

As you can see, we need to use :non-bridge-officers as the key to refer to the embedded map.

We all have our darker side. We need it; it's half of what we are.
        It's not really ugly, it's human.

Clojure Plan, Map and Courage

This is one of those moments when the computer says NO!

Let's look at the interesting nature of Clojure's map function.

(defn foo[filenames]
    (map #(touchFile (str "/tmp/" %)) filenames))

Suppose we have a pre-existing function called touchFile that creates an empty file.

This foo function never created any files when called. I must have made a mistake with the directory path or perhaps my filesystem ran out of space or perhaps a bug somewhere in my touchFile function or perhaps...

Hang on, my test code for touchFile passes without any problem!

An attempt to prove something is wrong with the map call....

(defn foo[filenames]
    (map #(println (str "/tmp/" %)) filenames))

And nothing was printed out on my screen!

Something really sinister is going on here...

All you need is the plan, the road map, and the courage to press on to your destination.

After going back to the docs, it mentioned that "map returns a lazy sequence of...". Did you just hear the penny dropped?

So, the fix here is to invoke doall to force the map to evaluate each items immediately.

(defn foo[filenames]
    (doall (map #(touchFile (str "/tmp/" %)) filenames)))

Yay it worked!

Looking at a simple problem at 2 in the morning doesn't help a lot. It's probably time for me to hit the Zs.

In Search For A Better CI/CD

This is going to be an exciting time!

Imagine me, writing me first Clojure app!

I'm halfway through writing my first Clojure program called "relman", short for "Release Manager".

It's a simple CLI application to help me deploy specific WAR files to a dockerised Tomcat. 

The recipe here is a simple pull deployment (unless I've hit unforeseen issues - fingers crossed).

Firstly, I've configured my Jenkins build to rename any successful elwood-parent builds to <feature>-<issue number>-<issue description>-<#build number>.war and copy this to a mounted volume shared with Tomcat.

Secondly, running relman on host to copy the war file as ROOT to the designated Tomcat directory mounted as "/webapp".

Thirdly, umm.. I think that's it. Trying to keep the solution simple.

This POC worked by manually copying the file across directories, so technically this idea should work!

Furthermore, I think this recipe is universal and applicable when I use different Git branches whenever Jenkins kicks in.

As with relman in Clojure, this is a work in progress but I think I'm not far off from my end goal.

In summary, this is my first step to get CI/CD working and it's by no means an end to this exciting journey!

And the quest for CI/CD continues... (Clojure included)

Clojure Exercise #1: Bend My Mind With MapSet

Write a function, mapset, that works like map except the return value is a set:
(mapset inc [1 1 2 2])
;=> #{2 3}


Answer:

(defn mapset [fn elements]
(let [uniq (into #{} elements)]
(set (map fn uniq))))

How can I write a piece of code without a unit test?

(deftest mapset-test

(is (= #{ 2 3 } (mapset inc [1 1 2 2]))))

Invoking mapset:

(mapset inc [1 1 2 2])

;=> #{3 2}

I Keep Coming Back to the Land Of Lisp

Don't ask me why. For some reason, I keep coming back to the Land of Lisp.

A few weeks ago, I was reading a few chapters about Common Lisp before I went to bed.
I remember vividly on one particular night, my mind was mumbling about the cryptic CAR, CDR, CDAR, CADR operations repeatedly, while looking at an unknown rectangular block chopped off from the top.

Well, the good thing is - this isn't one of Freddy Krueger's episode.


Lisp has been around for quite some time and to think about this old language coming back to life in the modern times or as an incarnation is exciting.

Welcome Clojure!


I hear people complained about the excessive (((((parenthesis))))) that comes with it. True, but has anyone complained about excessive semis;colons;and;{braces{around{blocks}}} from other programming languages? (No Visual Basic 6 please)


Though quite new to Clojure, I thought of trying a few simple things to see how it tastes like.

To compute for an extended price of an imaginary item (I was told that I should use a map to represent the properties of an item):

(defn line-extended [{qty :qty price :price discount :discount}]
   (* (- price discount) qty))

Simple enough.

Next is what I find Clojure interesting.

Suppose I need to sum all quantity, price and discount:

(defn line-items-summary [items]
   (apply merge-with + (for [item items]
   select-keys item [:qty :price :discount]))))

The code performs a loop for each item, add the quantity, price and discount and returns the sum of these properties.

So invoking this function as follows:

(line-items-summary [
    {:qty 1 :price 10 :discount 2}
    {:qty 2 :price 1000 :discount 200}])

Returns:

{:qty 3, :price 1010, :discount 202}

The syntax looks simple, elegant and most of all appealing. 

To sum up the three adjectives - it's POWERFUL!

♕ Keep Calm We're Moving To Gitflow GitFlow + Maven = No Joy ☹

I've have my own custom Jenkins that contains some basic configuration to build a tiny library called lyeung-common-test.

I've been trying to get GitFlow to work as a proof of concept.

I've played around with different plugins but to get GitFlow and Maven to work together proves more challenging than I initially thought.

The fundamental problem seems to be the pom version. Maven standard practice requires the version number to be defined upfront. However, when GitFlow comes in and starts merging branches, I get merge conflicts on version numbers.

Little Johnny is not a happy boy...

♔ Don't Panic See You In GitHub

After a few months on Hosted Redmine/Bitbucket, I've decided to pull all Elwood developments/wiki/issue tracker into one spot - GitHub. There's nothing wrong with Hosted Redmine and Bitbucket, they both provided me absolutely great services at $0.

It all boils down to splitting projects into personal and public projects. You see, I own 2 Git accounts: Bitbucket and GitHub. My overall intention was to keep all things private in Bitbucket and the rest publicly in GitHub. Elwood was suppose to be just a thought, a result of N-number of days day dreaming about writing a personal CI server. I was expecting this thought to vaporise as days go on, hence a private repository for late night coding activity.

Happy GitHubbing everyone!

♕ Keep Calm We're Moving To Gitflow

I'm evaluating Elwood development with Gitflow. One thing I like about Gitflow is that it gives the ability to better track changes, releases and bug fixes. It also allows me to stay focus in writing one feature at a time. You see, there is an adrenaline rush in me that I sometimes find myself shoving unrelated changes to my timeline. I could commit one large snowball of unrelated codes that may stretch the completion time to 1.5 times longer if I'm lucky or 3 times longer if I'm not-so-lucky.

This definitely contradicts the spirit of agile with the emphasis of delivering in small increments. (Ahhh, agile that starts with a lower case... How pragmatic do I sound?)

I did ask myself if this could be too much of an overhead for a lone developer like me?

Well, time will tell...

Elwood Release 1.1.0 (Aluminium+AL) Is Now Officially Released!

Please see the official release wiki page for more information:
http://www.hostedredmine.com/projects/elwood-parent/wiki/110_Release

How Do I Laugh At Thee? Let Me Count The Ways...

I hate to admit that I'm resource poor, in the sense that I can't afford to keep running my hardwares for 24x7. Neither can I afford to keep installing a new Linux image (JDK/Maven/Jenkins) every time I find a temporary shelter.

Currently, I have Jenkins/Linux running on an old machine. There were a few attempts to have it "VirtualBoxed" on my old laptop. However, since there is a sad fact about me and my laziness, all attempts ended in failure.

How do I laugh at thee? Let me count the ways

I laugh at thee from morning till noon till night

My soul rants which, when Jenkins s'out of sight

For constant installs take hours to days.

By nature, VirtualBox allows me to move the image between hardwares, but I need something much more than this. I need something that I could bring up on demand, something that I can move the data around, something that is lightweight - and Docker seems to fit the bill!

I was fortunate to have a hands-on experience with Docker at work, coincidentally, after playing around a bit with it in my spare time. At that time, I was looking for possible solutions for my quest on CI/CD. 

After spending roughly 2 months, I finally ended up getting Jenkins 1.6¹ and Redis images running on its own subnet with data directories mounted to host. I had to build my own Redis image as the official Docker image had a few issues when persisting to files. It could be something as easy as disabling persistence which I don't need, but nonetheless, time is ticking and I have to get back to Elwood development!

Below is a diagram of POC:

The proof of concept is very basic, something that's good enough to keep me going and something to iron out when I come back.

Hopefully, when the dust settles, I'll push these images into Docker Hub.

Repository: https://github.com/lyeung/lyeung-jenkins-ci-tools


¹Why am I using Jenkins as oppose to using Elwood for my build? Elwood is light years behind Jenkins and any mature products out there.

AngularJS 1.5 And PhantomJS Compatibility Issue

Last week, I've committed the relevant changes¹ to issue #508680² but Codeship³ reported a broken build. I swear I ran the UI test suite before committing and Grunt said everything was good! I decided to take some time to investigate this issue tonight to get to the very bottom of this. 

Broken builds keep me awake at night.

It appears that I've been building from the recent release of AngularJS 1.5 (previously 1.4) and there seems to be some incompatibility with PhantomJS 1.x. 

Interestingly, Codeship reported:

Error: [$injector:modulerr] Failed to instantiate module ng due to: TypeError: '' is not an object (evaluating 'Function.prototype.bind.apply')

After some lucky searching, this link came up:

https://github.com/angular/angular.js/issues/13794

The suggested fix was to move up to PhantomJS2 and update the karma-phantomjs-launcher module.

As to why the test suite worked on my machine but not Codeship's is a big mystery to me...

Note: Last October, as a true believer of CI, I've got Codeship to automatically build Elwood every time I push my changes to BitBucket. Codeship has been doing a great job for the past couple of months happily doing the builds in the cloud. (Hoping CD to come soon but everything must come in baby steps :)) It's an irony that I run a local copy of Elwood and manually trigger the build on Elwood sources. Does this sound like eating my own dog food?

¹https://bitbucket.org/lyeung/elwood-ui/commits/706cb2ef8025fc4fe46de2c7e18366f45107cfaf?at=master

²http://www.hostedredmine.com/issues/508680

³https://codeship.com - a cloud based Continuous Delivery platform and it comes with free tier.

AngularJS Custom Directive: Display Elwood Build Result Stats

I've written my first successful custom AngularJS directive to display build result stats (see issue #508680).

The directive initially displays the build status indicating success, failure or in-progress. Adjacent to this is a clickable icon¹ that goes back to the server and retrieve the build result statistics and displays them on the screen.

The journey to get here is challenging  (especially for someone who works at the back office) but very fulfilling except for one last thing - unit testing.


Below is the directive code:

angular.module('elwoodUiApp')
  .constant('BuildResultStatsUrl', 'http://localhost:8080/buildResultStats/:key/:count')
  .factory('BuildResultStatsResource', function($resource, BuildResultStatsUrl) {
    return $resource(BuildResultStatsUrl, {}, {
      'get': {method: 'GET'}
    });
  })
  .directive('elwBuildResultStats', function() {
    var controller = function($scope, BuildResultStatsResource, ToKeyCount) {
      $scope.toggle = false;
      $scope.getBuildResultStats = function(keyCount) {
        BuildResultStatsResource.get({'key': keyCount.key, 'count': keyCount.count},
          function (successResult) {
            console.log(successResult);
            if (!$scope.buildResultStats) {
              $scope.buildResultStats = [];
              $scope.buildResultStats[ToKeyCount(keyCount)] = {
                'successCount': successResult.successCount,
                'failedCount': successResult.failedCount,
                'ignoredCount': successResult.ignoredCount
              };
            }
            $scope.toggle = !$scope.toggle;
          }, function (errorResult) {
            console.log(errorResult);
          }
        );
      };

      $scope.isShowBuildResultStats = function(keyCount) {
        return $scope.toggle
          && ($scope.buildResultStats
            && ($scope.status == 'SUCCEEDED' || $scope.status === 'FAILED'));
      }
    };

    return {
      restrict: 'A',
      templateUrl: 'views/buildresultstats.html',
      controller: controller,
      scope: {
        status: '@',
        keyCount: '=',
      }
    };
  });

And below is the test code:

describe('Controller: BuildResultStatsCtrl', function () {

  // load the controller's module
  beforeEach(module('elwoodUiApp'));

  var
    scope,
    httpBackend;

  describe('BuildResultStatsCtrl', function() {
    var BuildResultStatsCtrl,
      buildResultStatsResource,
      toKeyCount;

    // Initialize the controller and a mock scope
    beforeEach(inject(function ($controller, $rootScope, $compile, $httpBackend, BuildResultStatsResource, ToKeyCount) {
      scope = $rootScope.$new();
      httpBackend = $httpBackend;
      buildResultStatsResource = BuildResultStatsResource;
      toKeyCount = ToKeyCount;

      var elem = '<div elw-build-result-stats="" key-count="keyCountTuple" status="{{buildStatus}}"></div>';
      scope.status = 'SUCCEEDED';
      scope.keyCount = {
        key: 'PRJ',
        count: '10'
      };

      httpBackend.whenGET('views/buildresultstats.html').respond(200, '');
      var template = $compile(elem)(scope, buildResultStatsResource, toKeyCount);
      console.log(template);

      scope.$digest();
    }));

    it('should display when status is SUCCEEDED', function() {
      var url = 'http://localhost:8080/buildResultStats/PRJ/10';

      expect(scope).toBeDefined();
      expect(scope.toggle).toBeFalsy();
      expect(scope.isShowBuildResultStats).toBeFalsy();
      expect(scope.buildResultStats).toBeUndefined();

      var mockData = {
        successCount: 10,
        failedCount: 3,
        ignoredCount: 2
      };

      httpBackend.expectGET(url).respond(mockData);

      expect(buildResultStatsResource).toBeDefined();
      scope.getBuildResultStats(scope.keyCount);
      httpBackend.flush();

      expect(scope.toggle).toBeTruthy();
    });
  });
});

After compiling the element at line number 30, I'm expecting the scope variables and functions will be fully defined and by the time I invoke "scope.getBuildResultStats(scope.keyCount)" the mocked GET should return the mock data.

Well, it looks like it didn't materialise for some unknown reason.


It's almost quarter past 2 in the morning.

Changes aren't committed yet.

I feel very tired but I think I'm almost there...

Update: (2016-02-23 12:10 AM) Extracting the directive's function into a full blown controller and referring the controller with a controller name might be an option. This allows me to test the controller as an standalone component outside of this directive.

.controller('BuildResultStatsCtrl', function($scope, BuildResultStatsResource, ToKeyCount) {

  // ...

}

return {
  restrict: 'A',
  templateUrl: 'views/buildresultstats.html',
  controller: 'BuildResultStatsCtrl',
  scope: {
    status: '@',
    keyCount: '=',
  }
};

¹Generated from http://www.amp-what.com and I find this website very cool.

Java 7 and Multi-Catch Exception

Java 7 has been out for a few years and I admit this is one of the features I seem to neglect. Perhaps, I've been living under a rock, yeah?

I've got JDK 8 installed and because of my sheer laziness, I'll stick to this version of the compiler to analyse how multi-catch exception is implemented.

She saw first some bracelets, then a pearl necklace, then a Venetian gold cross set with precious stones, of admirable workmanship. She tried on the ornaments before the mirror, hesitated and could not make up her mind to part with them, to give them back.

She kept asking: "Haven't you any more?"

Let's define a few exceptions before dipping a toe in the water.

PearlNecklaceException.java

package org.lyeung.thenecklace;

public class PearlNecklaceException extends RuntimeException {
    // do-nothing
}

VenetianCrossException.java

package org.lyeung.thenecklace;

public class VenetianCrossException extends RuntimeException {
    // do-nothing
}

DiamondNecklaceException.java

package org.lyeung.thenecklace;

public class DiamondNecklaceException extends RuntimeException {
    // do-nothing
}

Followed by the main class we'll look at:
Mathilde.java

package org.lyeung.thenecklace;

public class Mathilde {

    private void retrieveNecklace(int value) throws PearlNecklaceException,
            VenetianCrossException, DiamondNecklaceException {
        if (value == 0) {
            throw new PearlNecklaceException();
        } else if (value == 1) {
            throw new VenetianCrossException();
        } else {
            throw new DiamondNecklaceException();
        }
    }

    public void findNecklace1(int value) {
        try {
            retrieveNecklace(value);
        } catch (PearlNecklaceException e) {
            e.printStackTrace();
        } catch (VenetianCrossException e) {
            e.printStackTrace();
        } catch (DiamondNecklaceException e) {
            e.printStackTrace();
        }
    }

    public void findNecklace2(int value) {
        try {
            retrieveNecklace(value);
        } catch (PearlNecklaceException e) {
            handleException(e);
        } catch (VenetianCrossException e) {
            handleException(e);
        } catch (DiamondNecklaceException e) {
            handleException(e);
        }
    }

    private void handleException(Exception e) {
        e.printStackTrace();
    }

    public void findNecklace3(int value) {
        try {
            retrieveNecklace(value);
        } catch (VenetianCrossException
                | PearlNecklaceException
                | DiamondNecklaceException  e) {
            e.printStackTrace();
        }
    }
}

All three variants of find necklaces call retrieveNecklace which throws different exceptions base on input value.

The first variant of find necklace is pretty old school. We declare different catch blocks for each exception thrown.

The second variant of find necklace, similar to the first variant, but minimises code duplication by moving the print stack trace into a common method.

The third variant uses multi-catch exception that results to a more readable and more compact code.

Under the hood, these variants produce different bytecodes.


The disassembled code for the first variant:

public void findNecklace1(int);
  Code:
     0: aload_0
     1: iload_1
     2: invokespecial #8                  // Method retrieveNecklace:(I)V
     5: goto          29
     8: astore_2
     9: aload_2
    10: invokevirtual #9                  // Method org/lyeung/thenecklace/PearlNecklaceException.printStackTrace:()V
    13: goto          29
    16: astore_2
    17: aload_2
    18: invokevirtual #10                 // Method org/lyeung/thenecklace/VenetianCrossException.printStackTrace:()V
    21: goto          29
    24: astore_2
    25: aload_2
    26: invokevirtual #11                 // Method org/lyeung/thenecklace/DiamondNecklaceException.printStackTrace:()V
    29: return
  Exception table:
     from    to  target type
         0     5     8   Class org/lyeung/thenecklace/PearlNecklaceException
         0     5    16   Class org/lyeung/thenecklace/VenetianCrossException
         0     5    24   Class org/lyeung/thenecklace/DiamondNecklaceException

The exception table says,  "For program counter offset 0 to 5, inclusive and exclusive respectively, the program counter should jump to target offset 8 if PearlNecklaceException is thrown, 16 if VenetianCrossException is thrown and 24 if DiamondNecklaceException is thrown."

All offsets 8, 16 and 24 invoke the caught exception's printStackTrace() method.

The disassembled code for the second variant:

public void findNecklace2(int);
  Code:
     0: aload_0
     1: iload_1
     2: invokespecial #8                  // Method retrieveNecklace:(I)V
     5: goto          32
     8: astore_2
     9: aload_0
    10: aload_2
    11: invokespecial #12                 // Method handleException:(Ljava/lang/Exception;)V
    14: goto          32
    17: astore_2
    18: aload_0
    19: aload_2
    20: invokespecial #12                 // Method handleException:(Ljava/lang/Exception;)V
    23: goto          32
    26: astore_2
    27: aload_0
    28: aload_2
    29: invokespecial #12                 // Method handleException:(Ljava/lang/Exception;)V
    32: return
  Exception table:
     from    to  target type
         0     5     8   Class org/lyeung/thenecklace/PearlNecklaceException
         0     5    17   Class org/lyeung/thenecklace/VenetianCrossException
         0     5    26   Class org/lyeung/thenecklace/DiamondNecklaceException

The exception table is not much different from the first variant.

The disassembled code for the last variant:

public void findNecklace3(int);
    Code:
       0: aload_0
       1: iload_1
       2: invokespecial #8                  // Method retrieveNecklace:(I)V
       5: goto          13
       8: astore_2
       9: aload_2
      10: invokevirtual #14                 // Method java/lang/RuntimeException.printStackTrace:()V
      13: return
    Exception table:
       from    to  target type
           0     5     8   Class org/lyeung/thenecklace/VenetianCrossException
           0     5     8   Class org/lyeung/thenecklace/PearlNecklaceException
           0     5     8   Class org/lyeung/thenecklace/DiamondNecklaceException
}

Now, the exception table looks different. As you can see, the target values all point at 8, where offset 8 prints the stack trace!

The generated bytecodes are definitely smaller and more compact than the previous variants.


Below is the disassembled code for entire Mathilde class:

Compiled from "Mathilde.java"
public class org.lyeung.thenecklace.Mathilde {
  public org.lyeung.thenecklace.Mathilde();
    Code:
       0: aload_0
       1: invokespecial #1                  // Method java/lang/Object."":()V
       4: return

  private void retrieveNecklace(int) throws org.lyeung.thenecklace.PearlNecklaceException, org.lyeung.thenecklace.VenetianCrossException, org.lyeung.thenecklace.DiamondNecklaceException;
    Code:
       0: iload_1
       1: ifne          12
       4: new           #2                  // class org/lyeung/thenecklace/PearlNecklaceException
       7: dup
       8: invokespecial #3                  // Method org/lyeung/thenecklace/PearlNecklaceException."":()V
      11: athrow
      12: iload_1
      13: iconst_1
      14: if_icmpne     25
      17: new           #4                  // class org/lyeung/thenecklace/VenetianCrossException
      20: dup
      21: invokespecial #5                  // Method org/lyeung/thenecklace/VenetianCrossException."":()V
      24: athrow
      25: new           #6                  // class org/lyeung/thenecklace/DiamondNecklaceException
      28: dup
      29: invokespecial #7                  // Method org/lyeung/thenecklace/DiamondNecklaceException."":()V
      32: athrow

  public void findNecklace1(int);
    Code:
       0: aload_0
       1: iload_1
       2: invokespecial #8                  // Method retrieveNecklace:(I)V
       5: goto          29
       8: astore_2
       9: aload_2
      10: invokevirtual #9                  // Method org/lyeung/thenecklace/PearlNecklaceException.printStackTrace:()V
      13: goto          29
      16: astore_2
      17: aload_2
      18: invokevirtual #10                 // Method org/lyeung/thenecklace/VenetianCrossException.printStackTrace:()V
      21: goto          29
      24: astore_2
      25: aload_2
      26: invokevirtual #11                 // Method org/lyeung/thenecklace/DiamondNecklaceException.printStackTrace:()V
      29: return
    Exception table:
       from    to  target type
           0     5     8   Class org/lyeung/thenecklace/PearlNecklaceException
           0     5    16   Class org/lyeung/thenecklace/VenetianCrossException
           0     5    24   Class org/lyeung/thenecklace/DiamondNecklaceException

  public void findNecklace2(int);
    Code:
       0: aload_0
       1: iload_1
       2: invokespecial #8                  // Method retrieveNecklace:(I)V
       5: goto          32
       8: astore_2
       9: aload_0
      10: aload_2
      11: invokespecial #12                 // Method handleException:(Ljava/lang/Exception;)V
      14: goto          32
      17: astore_2
      18: aload_0
      19: aload_2
      20: invokespecial #12                 // Method handleException:(Ljava/lang/Exception;)V
      23: goto          32
      26: astore_2
      27: aload_0
      28: aload_2
      29: invokespecial #12                 // Method handleException:(Ljava/lang/Exception;)V
      32: return
    Exception table:
       from    to  target type
           0     5     8   Class org/lyeung/thenecklace/PearlNecklaceException
           0     5    17   Class org/lyeung/thenecklace/VenetianCrossException
           0     5    26   Class org/lyeung/thenecklace/DiamondNecklaceException

  private void handleException(java.lang.Exception);
    Code:
       0: aload_1
       1: invokevirtual #13                 // Method java/lang/Exception.printStackTrace:()V
       4: return

  public void findNecklace3(int);
    Code:
       0: aload_0
       1: iload_1
       2: invokespecial #8                  // Method retrieveNecklace:(I)V
       5: goto          13
       8: astore_2
       9: aload_2
      10: invokevirtual #14                 // Method java/lang/RuntimeException.printStackTrace:()V
      13: return
    Exception table:
       from    to  target type
           0     5     8   Class org/lyeung/thenecklace/VenetianCrossException
           0     5     8   Class org/lyeung/thenecklace/PearlNecklaceException
           0     5     8   Class org/lyeung/thenecklace/DiamondNecklaceException
}

JUnit Test Categorisation With Maven

To group by test types, we annotate our test codes with JUnit's org.junit.experimental.categories.Category and customise Maven's test plugin to allow certain groups to run before others.

But just before we start, we need to provide group names to our test codes. As an example, we'll model these group types as Felicite's early and later life.

package org.lyeung.simpleheart;
public interface EarlyLife {}

package org.lyeung.simpleheart;
public interface LaterLife {}

Below is a sample test class:
public class TestFelicity {

    @Category(EarlyLife.class)
    @Test    public void isKnowsTheodore() {
        System.out.println("isKnowsTheodore");
        assertTrue(true);
    }

    @Category(EarlyLife.class)
    @Test    public void isKnowsPaul() {
        System.out.println("isKnowsPaul");
        assertTrue(true);
    }

    @Category(EarlyLife.class)
    @Test    public void isKnowsVirginie() {
        System.out.println("isKnowsVirginie");
        assertTrue(true);
    }

    @Category(LaterLife.class)
    @Test    public void isKnowsFatherColmiche() {
        System.out.println("isKnowsFatherColmiche");
        assertTrue(true);
    }

    @Category(LaterLife.class)
    @Test    public void isParrotLouLou() {
        System.out.println("isParrotLouLou");
        assertTrue(true);
    }
}

As you can see from above, we've categorised our test methods by annotating @Category with the desired group types.

To enable early life tests when Maven runs package goal, we need to update the pom file to customise Surefire plugin:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.18.1</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit47</artifactId>
            <version>2.18.1</version>
        </dependency>
    </dependencies>
    <configuration>
        <groups>org.lyeung.simpleheart.EarlyLife</groups>
    </configuration>

</plugin>

To enable later life tests when Maven runs integration test or after, we need another update to Failsafe plugin:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.18.1</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit47</artifactId>
            <version>2.18.1</version>
        </dependency>
    </dependencies>
    <configuration>
        <groups>org.lyeung.simpleheart.LaterLife</groups>
    </configuration>
    <executions>
        <execution>
            <id>failsafe-integration-tests</id>
            <phase>integration-test</phase>
            <goals>
                <goal>integration-test</goal>
            </goals>
            <configuration>
                <includes>
                    <include>**/*.class</include>
                </includes>
            </configuration>
        </execution>
    </executions>
</plugin>

What this means is that when Maven runs package goal, only tests with early life are invoked. When Maven runs integration test goal or after, both early life and later life tests are invoked¹.

"From the threshold of the room she saw Virginia, stretched on her back, her hands joined, her mouth open, and her head thrown back under a black cross bending towards her, between motionless curtains, less white than her face."

Below is a snippet log for package goal:
mvn clean package

-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.lyeung.simpleheart.TestFelicity
isKnowsVirginie
isKnowsPaul
isKnowsTheodore
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec - in org.lyeung.simpleheart.TestFelicity

Results :

Tests run: 3, Failures: 0, Errors: 0, Skipped: 0

Below is a snippet log for package verify goal:

mvn clean verify
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.lyeung.simpleheart.TestFelicity
isKnowsVirginie
isKnowsPaul
isKnowsTheodore
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec - in org.lyeung.simpleheart.TestFelicity

Results :

Tests run: 3, Failures: 0, Errors: 0, Skipped: 0

-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.lyeung.simpleheart.TestFelicity
isKnowsFatherColmiche
isParrotLouLou
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec - in org.lyeung.simpleheart.TestFelicity

Results :

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

¹http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html

Why Test Groups Matter

I'm having poach eggs today.

Upon checking the fridge, to my horror, the egg trays are empty!

Just as I'm about to rush out to the nearest supermarket, I just remembered my stovetop broke down last night.

My instinct kicks in. Before I run towards the door, I quickly check the stovetop.

If it decides to take a holiday today, I'll surely head back to my bedroom and get more sleep.

Well, this is just an analogy.

---

Test tiers are usually made up of unit tests, integration tests and end-to-end tests (E2E).

And if we are to visualise the Agile Test Pyramid...

Unit test, as the name implies, tests for logical and boundary correctness. Most of these tests are in-memory, therefore, they run trail-blazing fast.

Integration test is an aggregation of unit tests that requires external resources. These resources could be transaction boundaries, filesystem or network calls. Resources are usually expensive to setup and perhaps torn down and they need to be made available to the integration test beforehand.

E2E tests for the whole workflow and may touch multiple integration points. It is usually more complex than an integration test. An example of E2E test is a test on checkout workflow. The test must log into the system, select a list of products, put them into the shopping cart then make a checkout.

As you can see, the test cases become more complex as we move up the tiers.

---

Many years ago, I said to my colleague that he needs to run the whole test suite before committing. He replied back saying, "But it takes too long to run the whole suite."

This made me realise that only a handful of people are willing to sit and wait while they twiddle their thumbs as they stare emptily at the screen. As for myself, I make a quick exit to the lift to get another cup of flat white.

---

From my experience, if test codes are grouped and ordered to run from bottom to the top of the pyramid, the development cycle will find and fix defects quicker and minimise waste.

You wouldn't run to the closest supermarket without ensuring your stovetop works, would you?

Below is an inverted test pyramid in running time.

Elwood Build Process High Level Design

Below is a diagram of how the build process works at a conceptual level.

Elwood High Level Design

"A picture is worth a thousand words..."

Elwood Maven Surefire Integration Part 2

I find this solution slightly obtrusive. After the project is checked out, Elwood alters the original pom and adds a small dependency snippet to pull down the custom run listener (elwood-maven-runlistener) during the build. This is evidently found in the pom.xml sitting under the META-INF directory marked with provided scope to prevent this from becoming an official dependency of the artefact.

I could potentially use a custom Maven that has the run listener JAR as one of it's libraries. But then, this has to be another day.

I had to abandon the original idea of making the run listener POST the build statistics using a 3^rd party REST library because I may inadvertently introduce a class loader issue when the official project dependency has this library defined. Same argument goes with Redis. I also wouldn't want roll up my sleeves and start writing network calls on my own as time is a scarce resource.

My other concern with using network calls is that the back-end REST service could be busily serving requests from the browser, which I suspect may slow down the build process. Note that the front-end code retrieves the build status every few seconds.

I don't want to stress the back-end service too much.

I've also tested the idea of keeping the statistics in memory as a JMX bean. However, as each test classes are started up and torn down, the numbers are lost. In the end, I had to come back to my old friend – the filesystem¹.

Note: All changes² related to capturing the Maven build statistics are found in “maven-integration” branch.

¹https://bitbucket.org/lyeung/elwood-parent/commits/f3b45eb95717fc0b48247c69ba0a15918d37e22d

²https://bitbucket.org/lyeung/elwood-parent/commits/822ee1c8343598210c38f3016dbe9a66ec5ca04d