JVx and Java 8, Events and Lambdas

Post to Twitter

With version 8, Java has received support for lambda expressions and of course also JVx and its users directly profit from this new feature.

In case you do not know what a lambda expression is, it is basically the possibility to inline functions, sometimes also called "anonymous functions" and should not be confused with "anonymous classes", which Java has supported for quite some time now. Let's look at a simple example of anonymous classes:

new Thread(new Runnable()
{
    public void run()
    {
        // your code
    }
}).start();

This launches a thread that does something. The Runnable in our example is the anonymous class. Now with the new lambda support we don't need to write the interface implementation, but can directly tell it to run a function:

new Thread(() -> { /* your code */ }).start();

The empty parentheses and the arrow indicate a new anonymous function. The parentheses contain the list of parameters of the function that is going to be invoked, in our case it is empty because Runnable.run() does not have any parameters. So the anatomy of a lambda looks like this:

new Thread((optional explicit cast to target interface)(parameters) ->
{
    function body
});

Internally this is still compiled into an anonymous class, but the code becomes a lot smoother and easier to read.

Additionally it is now possible to reference functions directly, like this:

// instance
new Thread(this::worker).start();
// static
new Thread(YourWorkClass::worker).start();

Especially the last possibility is very exciting, as our event system has a very similar mechanism but until recently did not have compile-time safety and checks.

The new lambda feature is backwards compatible and can easily be used in JVx. Here is a simple test window that shows the new lambda expressions in combination with JVx events:

import javax.rad.genui.component.UIButton;
import javax.rad.genui.container.UIFrame;
import javax.rad.genui.layout.UIBorderLayout;
import javax.rad.ui.event.UIActionEvent;

public class LambdaShowcaseFrame extends UIFrame
{
    public LambdaShowcaseFrame()  
    {
        initializeUI();
    }

    public void doJVxAction(UIActionEvent pActionEvent)
    {
        System.out.println("JVx");
    }

    public void doLambda2(UIActionEvent pActionEvent)
    {
        System.out.println("Lambda 2");
    }

    private void initializeUI()
    {
        //before Java 8
        UIButton buttonJVx = new UIButton("JVx Style");
        buttonJVx.eventAction().addListener(this, "doJVxAction");

        // with Java 8

        UIButton buttonLambda1 = new UIButton("Lambda 1");
        buttonLambda1.eventAction().
                addListener(pActionEvent -> System.out.println("Lambda 1"));

        UIButton buttonLambda2 = new UIButton("Lambda 2");
        buttonLambda2.eventAction().addListener(this::doLambda2);

        setLayout(new UIBorderLayout());
        add(buttonJVx, UIBorderLayout.NORTH);
        add(buttonLambda1, UIBorderLayout.WEST);
        add(buttonLambda2, UIBorderLayout.EAST);
    }
}

RaspberryPi + 1-wire + i2c + camera and kernel pain

Post to Twitter

Our last research project was real fun: Build a Webcam with temp and light sensor to do some home/office automation.

We took a RaspberryPi model B, temp sensor (DS18B20) and light sensor (TSL45315).

The temp sensor had a 1-wire interface and the Internet had many tutorials to connect the sensor to a RasPi. It should be a trivial job, if you read the tutorials and it should be ready in max. 10 minutes. Should!

The theory wasn't the reality and the tutorials didn't cover our environment, because we had a new Raspbian ISO with kernel 3.16 and 3.18 some hours later.

We tried to connect the sensor as described (with and without pullup resistor, with parasite power and without) but it wasn't detected. It was horrible because the driver was loaded without problems and logged a simple success message.
We didn't find information about the problem because we didn't see an error. It simply did nothing and the filesystem didn't contain the relevant files and directories. Our /sys/bus/w1/devices directory was clean and empty. After some hours error searching, without any ideas we tried an older Raspbian image with kernel 3.6 and our sensor worked out-of-the-box without modifications. We saw additional log information from the 1-wire module which we didn't have with our 3.18 kernel. So we knew that something has changed in the kernel or kernel config...

But we weren't happy because the whole thing should work with an up-to-date setup. After we knew that the kernel caused our problems, we tried to find a solution and found: http://www.raspberrypi.org/forums/viewtopic.php?t=97216&p=676793 (first posting on 2nd page). There was a hint about the device tree. With this information, we found: https://github.com/raspberrypi/documentation/blob/master/configuration/device-tree.md

We disabled the device-tree by adding device_tree= to our config.txt and everything worked after a reboot, with our 3.18 kernel. So far so good.

It also worked with device_tree_overlay=overlays/w1-gpio-overlay.dtb. This option was our preferred one because it didn't disable the device_tree.

After our temp sensor worked without problems we connected the Raspicam and made some tests. Everything worked without problems. The last piece of our project was the light sensor. It was a littly bit tricky to use the sensor with our Pi because the source code example wasn't available for Java/Pi4J, but here it is:

I2CBus bus = I2CFactory.getInstance(I2CBus.BUS_1);

I2CDevice device = bus.getDevice(0x29);

device.write((byte)(0x80 | 0x0A));

byte[] byData = new byte[1];
device.read(byData, 0, 1);

System.out.println(byData[0]);

System.out.println("Power on...");

device.write((byte)(0x80 | 0x00));
device.write((byte)0x03);

System.out.println("Config...");

device.write((byte)(0x80 | 0x01));
device.write((byte)0x00);   //M=1, T=400ms
//device.write((byte)0x01);   //M=2, T=200ms
//device.write((byte)0x02);   //M=4, T=100ms

byData = new byte[2];

while (true)
{
    device.write((byte)(0x80 | 0x04));
   
    device.read(byData, 0, 2);
   
    int l = byData[0] & 0xFF;
    int h = byData[1] & 0xFF;
   
    int lux = (h << 8) | (l << 0);
   
    lux *= 1;   //M1
    //lux *= 2;   //M2
    //lux *= 4;   //M4
   
    System.out.println(lux);
   
    Thread.sleep(1000);
}

We compared the output with an Arduino board and the values were "the same".

It wasn't possible to use the light sensor without required modules. We simply added device_tree_param=i2c1=on to our config.txt and I2C worked. You should install

sudo apt-get install i2c-tools

to verify connected sensors, via:

i2cdetect -y 1

After we thought that everything will work, we put all pieces together: temp sensor, cam, light sensor.
During development, we didn't use all parts together - only the relevant sensor, to rule out errors.

Everything worked fine after starting the Pi, but the cam stopped capturing after some minutes... frustrating.
So again we tried to find an error without detailed information and without logs. We thought it might be a module loading problem and found some hints: https://github.com/raspberrypi/linux/issues/435. But nothing worked for us. The only working solution was: disable the device tree :(

The whole system is up and running for some days and there we no problems.

We currently don't know what's the problem with the device tree or what we should configure to avoid problems, but if you have similar problems, this posting could help. If you have a working solution with the device tree... leave a comment :)

RaspberryPi, Jetty, Vaadin & Push support

Post to Twitter

I wrote in a previous article about my problems with Jetty and Vaadin push support.

I found the problem in the meantime and Jetty works well on my Raspi. The performance is a little bit better than with Tomcat 8.

What was the problem?

Vaadin 7.1 push implementation was incompatible with Jetty 9.2.6. I had to use Jetty 9.0.7.

I didn't use Jetty as embedded server, it runs standalone via bin/jetty.sh. It was a little bit tricky to add an external lib to the classpatch because the commandline argument changed from 9.0 to 9.2.

I had to add following to my jetty.sh:

export JETTY_HOME="/opt/jetty/"
export JETTY_BASE="/opt/jetty/"
export JAVA="/opt/java/bin/java"
export JETTY_ARGS="--module=websocket --module=continuation path=/usr/share/java/RXTXcomm.jar"

With 9.2:

export JETTY_ARGS="--module=websocket --module=continuation --lib=/usr/share/java/RXTXcomm.jar"

You have to know the pitfalls :)

It seems that Vaadin' websocket implementation for Jetty is better than for Tomcat because Vaadin handles multiple rotate/resize events faster with less communication. I know that Vaadin 7.3 has a better websocket implementation and it's also up-to-date and should work with Jetty 9.2.6, but I still use Vaadin 7.1.

I recommend Jetty for your embedded project because it's fast and simple. If you prefer Tomcat, you can't do anything wrong.

Oh... I also solved the problem with missing request from / to /index.html. The redirection won't work if you don't have a file with the name index.html in your public web area because Jetty does nothing in that case. I didn't find an option for that, but creating an empty index.html solved my problem because I had a Servlet which was listening on /index.html.

Vaadin UI with SmartTV

Post to Twitter

It's not rocket science and we didn't write code, but it's nice to see our UIs on SmartTVs :)

Here are some impressions:

SmartTV - Statistic

SmartTV - Statistic

SmartTV - Data

SmartTV - Data

SmartTV - Window Shutter

SmartTV - Window Shutter

Window shutter UI - WiiU

Post to Twitter

Because I got some emails, here's the window shutter UI how it looks on my Wii U. Sure, the space is not perfectly used, but the UI wasn't optimized for Wii U. Also the agent name of the browser wasn't checked to optimize controls.

But it's ok and control just works :)

The pictures (phone cam):

Wii-U controller

Wii-U controller

Wii-U (TV)

Wii-U (TV)

IoT: Window shutter control with JVx, Vaadin and RaspberryPi

Post to Twitter

It's about two years ago that I wrote about controlling window shutters. The system run for 2 years without problems but it was time to change "something" because it was vacation time and I thought it would be useful to showcase our vaadin UI :)

The old system was controled via (good old) SMS - without GUI. It had a lot of commands but only en/disable up/down were heavily used because the integrated public holiday and weekend automatism solved most use cases. The SMS contol was great and worked without problems but it was a problem if you didn't know the commands and syntax by heart!

Another consideration was a simple real-world showcase for our vaadin UI, the responsive feature and push support. Our application frame was already responsive and worked on desktop, tablet and smartphone browsers. I thought about using AngularJS or different "hypeware" but I didn't have weeks to research and implement. My new GUI had to be finished in few hours. And it should be maintainable for the next two (or more) years without problems.

I love researching hypeware... but simply read some articles if you want:

There are too many frameworks, everyone is different - no real standard APIs (didn't count HTML and CSS). You can't be sure that the framework is future-proof because developers love to change software to be incompatible with previous versions just to be state-of-the-art.
And why should a developer use 5 or more frameworks to create a simple application for desktop and mobile? You'll need more libs if your application needs DB and CRUD support.
C'mon we need simple and small frameworks and not what we have right now! The world is talking about microservices and what about "micro frameworks"?

Conclusio: I didn't find something comparable to JVx. Of course, it's my opinion and I'm JVx developer, but I'm a researcher and the technology isn't a problem if it makes sense to use it.

To be honest: Our Vaadin UI in combination with other JVx features is unbeatable for most application types. Of course, it's not recommended for web pages with custom layout requirements. If you need an "application style" solution with menu and screen/work area or form based - there's nothing comparable.

After I knew that my new GUI would be a JVx' vaadin solution, I had to think about the details. The whole application should be responsive to support every modern device and desktop browsers as well. The application server should be Jetty or Tomcat and it should run on the RasPi. The performance is not critical because the hardware is limited (but cheap).

Software stack: clear
Hardware: clear
Deployment type: clear

Next step was development. This was easy because I re-used existing classes and added some get/set methods to access internal members. The most important change was an event listener because every vaadin client should be updated immediate via push. It wasn't possible without notification mechanism. I thought about an event bus and played around with mbassador. It's a nice and small lib, fast and just works (recommendable).
To be honest: I didn't need it because I didn't need more than one listener and an event bus is not much more than a listener abstraction (with some extras).

After some hours developing and refactoring, everything was ready for a test.... The next decission: Jetty or Tomcat?
I worked with Tomcat for many years and didn't know much about Jetty, only that it's the number one for embedding into applications, supports SPDY and has websocket support. I played around with Jetty and got it running. The installation of my GUI worked like a charme and everything worked except push. I found solutions for embedded run mode but not for standalone Jetty. I'm pretty sure that I missed something, but I wasn't very patient and switched to Tomcat 8.
Not only because of my push problem but Jetty didn't handle requests as expected:

http://localhost/house/ didn't redirect to http://localhost/house/index.html automatically. I didn't have an index.html file in the public file area, anyway the request should be executed. Such requests work with Tomcat without additional configuration.

I'm not happy that my Jetty server didn't work but I'll retry it in the next days :)

Here's a simple screencast of my solution because it's easier to understand:

Window shutter web control


It took me less than one day to finish my GUI and the best: It's family safe.
I tried the GUI with PS4 and WiiU integrated browser and it was great to see shutters moving.

Summary

I've used a RaspberryPi, installed Tomcat 8 with Java8 and deployed my application as war file. The GUI is responsive and supports push via websockets and runs on every device in my house. It wasn't planned using gaming consoles :)

Oh, I forgot to mention: During my tests I found out that Android and iOS support custom Bookmark images for Desktop links but this needs additional tags in index.html (works without, but it's better to include the tags). Our VaadinUI now has built-in support for such images :)

The switch component is available in the Vaadin Directory.

Stress testing a Vaadin application with Selenium Grid, a summary

Post to Twitter

For quite some time there has been the idea in our heads to run a stress test against a JVx/Vaadin installation to see if there are any major problems with performance and multithreading. We knew that Tomcat, Vaadin and JVx could serve a lot of users simultaneously, that's what all three were designed to do, that's what all three do well in many different environments, but we never really tried to stress it out. So we set out to change that.

The plan

Some weeks ago we started discussing and outlining how to perform such a stress test. After some research we came up with a plan:

  1. Design a test case using Selenium for the DemoERP application
  2. If necessary, design and create solutions to allow that
  3. Set up a Selenium Grid which can be used for testing
  4. Perform the actual test and fix anything that needs to be fixed

As simple as it sounds, it had quite some gotchas on the way and taught us some valuable lessons.

Name all the things

The first thing you need for automated testing are names, components need names so that they are easily identifiable by whatever you're using for testing. Already back in September we started to name components in a unique and easily recognizable fashion, our efforts are covered by ticket #1103. Of course you can still set your own names, if you want that, as the default names will only be applied if no other name is set.

In short, components are now receiving a name based on their position in the component tree. For example take this simple workscreen:

Class                 Name

WorkScreen:           WS-XX
    SplitPanel:       WS-XX_SP1
        Panel:        WS-XX_SP1_P1
            Label:    WS-XX_SP1_P1_L1
            Editor:   WS-XX_E_COLUMN
        Panel:        WS-XX_SP1_P2
            Label:    WS-XX_SP1_P2_L1
            Button:   WS-XX_B_DOPRINT

The naming starts with the "root" component and every child component does append it's name. So you can easily guess where a component is assigned to, simply from its name. Every name is guaranteed to be unique, either by its position in the tree or by incrementing a counter.

Additionally there are "special snowflakes" which are getting better names, like the name of a button which is only prefixed with the name of the workscreen and a "B" and then followed by the actions which are assigned to it, e.g.

public class DashboardWorkScreen extends ...
{
    ...
    public void initializeUI()
    {
        ...
        UIButton butUp = new UIButton("UP");
        butUp.setSize(80, 80);
        butUp.setStyle(new Style("up"));
        butUp.eventAction().addListener(this, "doUp");
    }
}

The button "butUp" will get the name Das-WV_B_DOUP. The prefix Das is the short name for the DashboardWorkScreen, followed by -WV which is like a prefix checksum/number. The letter B stands for Button and the action name DOUP is the last part.

With such a naming scheme in place, it's easy to create automated UI tests as every component is uniquely identifiable in the component tree. In the case of Vaadin the names are also used as ID's on the elements

<button type="button" class="v-nativebutton v-widget up v-nativebutton-up daswv_b_doup v-nativebutton-daswv_b_doup default-padding v-nativebutton-default-padding v-has-width v-has-height" id="Das-WV_B_DOUP" tabindex="0" style="width: 80px; height: 80px;">
  <span class="v-nativebutton-caption">UP</span>
</button>

The test case

For everyone who does not know our DemoERP application, it is a simple ERP application which features customer, product, offer and order management together with a nice statistic screen. All this can either be run as Swing application or (powered by a Tomcat Server) as Vaadin application directly in the browser.

The test case that we designed is rather simple:

  1. Load the application website and perform the initial login
  2. Open the customer management screen and add a new customer
  3. Open the offer management screen and search for an offer
  4. Switch to the order of said offer and edit it
  5. Open the statistic screen
  6. Logout

This gives a nice workload for testing, as it touches close to all areas of the application, changes values nearly simultaneous and also adds data with every run. Also note that the testcase would perform these steps as fast as possible, without any artificial waiting in between.

Thanks to the Selenium IDE the test case was recorded within a matter of minutes, and quite fast copied into a unit test. The first round of testing happened with the Chromium WebDriver, which allowed us to watch the test case do its work and make tweaks wherever necessary to allow the test to succeed.

Setting up Selenium Grid

Selenium Grid (or Grid2) is a simple, scalable framework for distributed Selenium tests, allowing to control multiple browsers on different platforms. The central server is called a "hub" and the clients are called "nodes". The nodes are registering themselves at the hub, and from there a client (for example a unit test) can send a request to the hub and it will receive a node matching the requested requirements. On the client side this is happening by simply creating a new instance of a RemoteWebDriver provided by Selenium. It will automatically contact the hub, wait for a node to be ready for it and then forward all commands to that node.

This is useful if you want quickly test an application in multiple browsers on multiple platforms without leaving your IDE or to perform a simple stress test.

Setting it up is as easy as starting the hub on one machine, and starting the nodes on any other machines. We decided to go with PhantomJS as browser, as we intended to run many instances of said browser on one node and we also had a wide variety of machines, some without a GUI. PhantomJS does already have it's own built-in node, so we just needed to start PhantomJS on the machines and tell it where it can find the hub.

At the end of day we had 12 physical machines in place and we maxed them out to get a total of 82 nodes which can be used for our stress testing.

Sessions

One thing we noticed very quickly was that lingering sessions can be a real issue and the timeouts really need to be tailored to your environment. While testing we quickly noticed that sessions kept lingering on, this was less an issue for the server and Tomcat, as more for our MariaDB installation because at some point it simply stopped accepting new connections. So we needed to reduce the amount of time a session would linger on to not clog our test server.

There are two important settings controlling the amount of time a session will linger on in such a setup:

  1. The timeout setting of the Tomcat server
  2. The heartbeat interval and closing of idle sessions in Vaadin

As said, these settings should be tailored to whatever environment the server is running in. In our test environment it didn't make any sense to keep sessions longer than a few minutes, because if the test case ended or was interrupted for some reason, the session would not resume. Armed with that knowledge we reduced the amount of time a session could linger on and we could finally start the real test.

Running the test

The server in our case would be a rather unspectacular laptop, with a Core-i7 (1.80GHz) and 16 GB of RAM. Tomcat 6 would deliver the application and the data would be provided by a local MariaDB/MySQL database (with an InnoDB engine).

The test itself, which ran several times for several hours, was rather boring to be honest. 82 clients swarmed the server as fast as it could answer requests without any noticeable effect on it's performance. The laptop, despite not being the typical server machine and for sure not intended as such, could easily handle the load that 82 clients produced. We did expect the application to stay in a usable state, however we did not expect that it wouldn't have any noticeable effect at all from a users perspective.

Here is a short summary of the data we gathered (using jvisualvm and ProcessHacker):

  • CPU peaked at 60%, but stood most of the time between 40% and 50%
  • Tomcat peaked at 4GB RAM, but would have been able to work with as little as 2GB
  • Over 21.000 datasets were created during the test run
  • The GUI was not noticeably slower during the test

Our setup could handle that amount of requests with ease. Unfortunately we could not get a hold of more machines to add further nodes for now.

We broke it, we fixed it

During the tests two nasty looking issues stood out, which kept happening in roughly 1% of all test runs with seemingly no pattern behind them.

The first was a simple NullPointerException which kept happening in UIImage, but that was fixed rather easily as certain methods where not synchronized and so destined to fail.

The second was a set of Exceptions which send us to our toes: NullPointerExceptions, "Object 'offer' was not found" and "Remote storage returned 15 value(s) but 22 were expected". Clearly there was something very wrong, and with the low volume with which these exceptions occurred, it could only be a race condition somewhere. Looking for possible causes send us to all corners of the codebase without any clue in sight what might cause them.

It only dawned upon us when we got a hold of the map which did not contain the object "offer", it only contained the objects "v_statistic_order_offer_year" and "v_statistic_order_offer_month". Looking at the lifecycle objects immediately showed the problem, the session did have a completely different lifecycle object. That did also explain the wrong amount of values, as two lifecycle objects happened to contain identical named objects. But how was that possible? We immediately checked the complete source code associated with creating a session, and in AbstractSession we found the culprit. Even though the complete code for creating a session was properly synchronized where necessary, one simple statement had slipped past:

private static long lSessionCount = 0;
...
private final Long lObjectId = Long.valueOf(++lSessionCount);

For everyone not seeing the problem, here is a simplified explanation. If an object is instantiated, the fields are initialized even before the constructor is run, this is not and can't be directly synchronized. A simple increment operation (++) is compiled to six operations:

  1. Load a reference to a variable onto the stack
  2. Put into that reference the value of the static field (in our case lSessionCount)
  3. Put the value "1" onto the stack
  4. Add both values on the stack together
  5. Duplicate the values
  6. Put the new value into the static field

Six instructions, a lot of time for two threads to overtake each other and cause problems. Instantiating two sessions at once had the potential to give both the same ID. Of course we immediately fixed it and applied some other optimizations.

Another run verified that this had fixed all three issues at once and the tests were running without any further incidents for hours.

Conclusion

We were not even close to pushing the limits of what a JVx application/server can handle, and judging from the performance of the server I can only call our test setup "humble". Still it gave us a good measure of what is for sure possible and showed us that we need a bigger setup to really push the limits.

During the tests we also created various utilities to aid us with testing, unfortunately they are in a rather barebone state and can not be released by now. Also there is now a more important project awaiting, one that we think you've been waiting for for far too long.

Cross platform GUI development in the press

Post to Twitter

My latest article about cross platform GUI development is available in current edition of Java aktuell. It's a German magazine and the article is in German!

It's all about application development for Desktop, Web and Mobile (native or web). It's based on Single sourcing and JVx.

You can read the article here (if German isn't a problem). Feel free to post your Feedback!

Switch button for Vaadin 7.1

Post to Twitter

There's a nice AddOn in the Vaadin Directory. It's a switch button/checkbox. There's an older version for vaadin 6.6+ and the current version for 7.2+.

But the 7.2+ version is not compatible with vaadin 7.1+. We checked the source code and made a version for 7.1.

If you need a 7.1 compatible switch AddOn, simply use this archive. It's a zip archive which contains the pre-compiled class files and changed source files. Simply use the zip (or rename it to jar) to rebuild your widgetset.

Don't use the 7.1 version for 7.2 because the API is different and it won't work. Simply use the original version for 7.2 and newer.

Page Navigation

Post to Twitter

Plain JVx Vaadin applications don't have page navigation as known from e.g. JSF applications. If you press "back" in your browser, the whole application is lost and "forward" will create a new application. Same problem with "reload".

It has nothing to to with JVx because it's a technology restriction. Isn't it?
Not necessarily!

You should know that vaadin has support for Browser Navigation, but this feature has no standard implementation because every application is different and has different requirements. If you create your own Vaadin application, you'll have to implement your own Navigation. It's not tricky but you should know how it works. A good starting point can be found in the book of vaadin. The main thing is that you should create your application with different views. A view is more or less a page. The navigator supports navigation between views. If you have different views, it's easy to use the Navigator.

But our generic application is a little bit different to plain Vaadin applications because JVx has work-screens. A work-screen is like a View or a page, but technology independent. A work-screen can be used for desktop applications and web applications without code changes. A desktop application usually doesn't offer Navigation as known from web browsers. So it's a good idea to have work-screens because it's independent of the used GUI technology and platform features.

If you use a JVx application as web application with Vaadin UI, you'll miss page navigation - for sure. We made some experiments with vaadin' page Navigation and are happy to present an awesome solution for this problem.

We did a generic implementation in our application frame and now it's possible to navigate between work-screens - out-of-the-box. It's implemented once in our application frame. Simply use it and page navigation works!

Here's a short demonstration:

Browser Navigation