Posts tagged: Vaadin

Stress testing a Vaadin application with Selenium Grid, a summary

Post to Twitter

For quite some time there has been the idea in our heads to run a stress test against a JVx/Vaadin installation to see if there are any major problems with performance and multithreading. We knew that Tomcat, Vaadin and JVx could serve a lot of users simultaneously, that's what all three were designed to do, that's what all three do well in many different environments, but we never really tried to stress it out. So we set out to change that.

The plan

Some weeks ago we started discussing and outlining how to perform such a stress test. After some research we came up with a plan:

  1. Design a test case using Selenium for the DemoERP application
  2. If necessary, design and create solutions to allow that
  3. Set up a Selenium Grid which can be used for testing
  4. Perform the actual test and fix anything that needs to be fixed

As simple as it sounds, it had quite some gotchas on the way and taught us some valuable lessons.

Name all the things

The first thing you need for automated testing are names, components need names so that they are easily identifiable by whatever you're using for testing. Already back in September we started to name components in a unique and easily recognizable fashion, our efforts are covered by ticket #1103. Of course you can still set your own names, if you want that, as the default names will only be applied if no other name is set.

In short, components are now receiving a name based on their position in the component tree. For example take this simple workscreen:

Class                 Name

WorkScreen:           WS-XX
    SplitPanel:       WS-XX_SP1
        Panel:        WS-XX_SP1_P1
            Label:    WS-XX_SP1_P1_L1
            Editor:   WS-XX_E_COLUMN
        Panel:        WS-XX_SP1_P2
            Label:    WS-XX_SP1_P2_L1
            Button:   WS-XX_B_DOPRINT

The naming starts with the "root" component and every child component does append it's name. So you can easily guess where a component is assigned to, simply from its name. Every name is guaranteed to be unique, either by its position in the tree or by incrementing a counter.

Additionally there are "special snowflakes" which are getting better names, like the name of a button which is only prefixed with the name of the workscreen and a "B" and then followed by the actions which are assigned to it, e.g.

public class DashboardWorkScreen extends ...
{
    ...
    public void initializeUI()
    {
        ...
        UIButton butUp = new UIButton("UP");
        butUp.setSize(80, 80);
        butUp.setStyle(new Style("up"));
        butUp.eventAction().addListener(this, "doUp");
    }
}

The button "butUp" will get the name Das-WV_B_DOUP. The prefix Das is the short name for the DashboardWorkScreen, followed by -WV which is like a prefix checksum/number. The letter B stands for Button and the action name DOUP is the last part.

With such a naming scheme in place, it's easy to create automated UI tests as every component is uniquely identifiable in the component tree. In the case of Vaadin the names are also used as ID's on the elements

<button type="button" class="v-nativebutton v-widget up v-nativebutton-up daswv_b_doup v-nativebutton-daswv_b_doup default-padding v-nativebutton-default-padding v-has-width v-has-height" id="Das-WV_B_DOUP" tabindex="0" style="width: 80px; height: 80px;">
  <span class="v-nativebutton-caption">UP</span>
</button>

The test case

For everyone who does not know our DemoERP application, it is a simple ERP application which features customer, product, offer and order management together with a nice statistic screen. All this can either be run as Swing application or (powered by a Tomcat Server) as Vaadin application directly in the browser.

The test case that we designed is rather simple:

  1. Load the application website and perform the initial login
  2. Open the customer management screen and add a new customer
  3. Open the offer management screen and search for an offer
  4. Switch to the order of said offer and edit it
  5. Open the statistic screen
  6. Logout

This gives a nice workload for testing, as it touches close to all areas of the application, changes values nearly simultaneous and also adds data with every run. Also note that the testcase would perform these steps as fast as possible, without any artificial waiting in between.

Thanks to the Selenium IDE the test case was recorded within a matter of minutes, and quite fast copied into a unit test. The first round of testing happened with the Chromium WebDriver, which allowed us to watch the test case do its work and make tweaks wherever necessary to allow the test to succeed.

Setting up Selenium Grid

Selenium Grid (or Grid2) is a simple, scalable framework for distributed Selenium tests, allowing to control multiple browsers on different platforms. The central server is called a "hub" and the clients are called "nodes". The nodes are registering themselves at the hub, and from there a client (for example a unit test) can send a request to the hub and it will receive a node matching the requested requirements. On the client side this is happening by simply creating a new instance of a RemoteWebDriver provided by Selenium. It will automatically contact the hub, wait for a node to be ready for it and then forward all commands to that node.

This is useful if you want quickly test an application in multiple browsers on multiple platforms without leaving your IDE or to perform a simple stress test.

Setting it up is as easy as starting the hub on one machine, and starting the nodes on any other machines. We decided to go with PhantomJS as browser, as we intended to run many instances of said browser on one node and we also had a wide variety of machines, some without a GUI. PhantomJS does already have it's own built-in node, so we just needed to start PhantomJS on the machines and tell it where it can find the hub.

At the end of day we had 12 physical machines in place and we maxed them out to get a total of 82 nodes which can be used for our stress testing.

Sessions

One thing we noticed very quickly was that lingering sessions can be a real issue and the timeouts really need to be tailored to your environment. While testing we quickly noticed that sessions kept lingering on, this was less an issue for the server and Tomcat, as more for our MariaDB installation because at some point it simply stopped accepting new connections. So we needed to reduce the amount of time a session would linger on to not clog our test server.

There are two important settings controlling the amount of time a session will linger on in such a setup:

  1. The timeout setting of the Tomcat server
  2. The heartbeat interval and closing of idle sessions in Vaadin

As said, these settings should be tailored to whatever environment the server is running in. In our test environment it didn't make any sense to keep sessions longer than a few minutes, because if the test case ended or was interrupted for some reason, the session would not resume. Armed with that knowledge we reduced the amount of time a session could linger on and we could finally start the real test.

Running the test

The server in our case would be a rather unspectacular laptop, with a Core-i7 (1.80GHz) and 16 GB of RAM. Tomcat 6 would deliver the application and the data would be provided by a local MariaDB/MySQL database (with an InnoDB engine).

The test itself, which ran several times for several hours, was rather boring to be honest. 82 clients swarmed the server as fast as it could answer requests without any noticeable effect on it's performance. The laptop, despite not being the typical server machine and for sure not intended as such, could easily handle the load that 82 clients produced. We did expect the application to stay in a usable state, however we did not expect that it wouldn't have any noticeable effect at all from a users perspective.

Here is a short summary of the data we gathered (using jvisualvm and ProcessHacker):

  • CPU peaked at 60%, but stood most of the time between 40% and 50%
  • Tomcat peaked at 4GB RAM, but would have been able to work with as little as 2GB
  • Over 21.000 datasets were created during the test run
  • The GUI was not noticeably slower during the test

Our setup could handle that amount of requests with ease. Unfortunately we could not get a hold of more machines to add further nodes for now.

We broke it, we fixed it

During the tests two nasty looking issues stood out, which kept happening in roughly 1% of all test runs with seemingly no pattern behind them.

The first was a simple NullPointerException which kept happening in UIImage, but that was fixed rather easily as certain methods where not synchronized and so destined to fail.

The second was a set of Exceptions which send us to our toes: NullPointerExceptions, "Object 'offer' was not found" and "Remote storage returned 15 value(s) but 22 were expected". Clearly there was something very wrong, and with the low volume with which these exceptions occurred, it could only be a race condition somewhere. Looking for possible causes send us to all corners of the codebase without any clue in sight what might cause them.

It only dawned upon us when we got a hold of the map which did not contain the object "offer", it only contained the objects "v_statistic_order_offer_year" and "v_statistic_order_offer_month". Looking at the lifecycle objects immediately showed the problem, the session did have a completely different lifecycle object. That did also explain the wrong amount of values, as two lifecycle objects happened to contain identical named objects. But how was that possible? We immediately checked the complete source code associated with creating a session, and in AbstractSession we found the culprit. Even though the complete code for creating a session was properly synchronized where necessary, one simple statement had slipped past:

private static long lSessionCount = 0;
...
private final Long lObjectId = Long.valueOf(++lSessionCount);

For everyone not seeing the problem, here is a simplified explanation. If an object is instantiated, the fields are initialized even before the constructor is run, this is not and can't be directly synchronized. A simple increment operation (++) is compiled to six operations:

  1. Load a reference to a variable onto the stack
  2. Put into that reference the value of the static field (in our case lSessionCount)
  3. Put the value "1" onto the stack
  4. Add both values on the stack together
  5. Duplicate the values
  6. Put the new value into the static field

Six instructions, a lot of time for two threads to overtake each other and cause problems. Instantiating two sessions at once had the potential to give both the same ID. Of course we immediately fixed it and applied some other optimizations.

Another run verified that this had fixed all three issues at once and the tests were running without any further incidents for hours.

Conclusion

We were not even close to pushing the limits of what a JVx application/server can handle, and judging from the performance of the server I can only call our test setup "humble". Still it gave us a good measure of what is for sure possible and showed us that we need a bigger setup to really push the limits.

During the tests we also created various utilities to aid us with testing, unfortunately they are in a rather barebone state and can not be released by now. Also there is now a more important project awaiting, one that we think you've been waiting for for far too long.

Switch button for Vaadin 7.1

Post to Twitter

There's a nice AddOn in the Vaadin Directory. It's a switch button/checkbox. There's an older version for vaadin 6.6+ and the current version for 7.2+.

But the 7.2+ version is not compatible with vaadin 7.1+. We checked the source code and made a version for 7.1.

If you need a 7.1 compatible switch AddOn, simply use this archive. It's a zip archive which contains the pre-compiled class files and changed source files. Simply use the zip (or rename it to jar) to rebuild your widgetset.

Don't use the 7.1 version for 7.2 because the API is different and it won't work. Simply use the original version for 7.2 and newer.

Page Navigation

Post to Twitter

Plain JVx Vaadin applications don't have page navigation as known from e.g. JSF applications. If you press "back" in your browser, the whole application is lost and "forward" will create a new application. Same problem with "reload".

It has nothing to to with JVx because it's a technology restriction. Isn't it?
Not necessarily!

You should know that vaadin has support for Browser Navigation, but this feature has no standard implementation because every application is different and has different requirements. If you create your own Vaadin application, you'll have to implement your own Navigation. It's not tricky but you should know how it works. A good starting point can be found in the book of vaadin. The main thing is that you should create your application with different views. A view is more or less a page. The navigator supports navigation between views. If you have different views, it's easy to use the Navigator.

But our generic application is a little bit different to plain Vaadin applications because JVx has work-screens. A work-screen is like a View or a page, but technology independent. A work-screen can be used for desktop applications and web applications without code changes. A desktop application usually doesn't offer Navigation as known from web browsers. So it's a good idea to have work-screens because it's independent of the used GUI technology and platform features.

If you use a JVx application as web application with Vaadin UI, you'll miss page navigation - for sure. We made some experiments with vaadin' page Navigation and are happy to present an awesome solution for this problem.

We did a generic implementation in our application frame and now it's possible to navigate between work-screens - out-of-the-box. It's implemented once in our application frame. Simply use it and page navigation works!

Here's a short demonstration:

Browser Navigation


Responsive Web Application

Post to Twitter

We have a great new feature in our generic web application. It's now responsive and supports big and small devices like smartphones, without additional coding. Everything will work out-of-the-box.

The application is generic which means that it's an application frame that can be used for you own requirements. We have a simple demo application for our internal tests.

Have a look:

JVx application with Vaadin UI


The menu changes the position and size. Also padding and margin are different if there's not enough space.

VaadinUI 1.2 released

Post to Twitter

Our VaadinUI 1.2 is available!
It's a feature release with some bugfixes.

The biggest change was the update to vaadin 7.1.15 and we've already 7.3.3 in our development branch (welcome Valo theme).

What's new?

  • Download extension

    Download dynamic content on-demand or whenever you want via WebSockets. More information and an example project for this extension.

  • Configuration of preserveOnUIRefresh

    Simply use preserveOnRefresh as init-param (true | false) in your web.xml or as URL parameter to preserve the application on browser refresh.

  • Use external CSS

    Use an external CSS file to configure your application style. There's no need to change genereated styles.css for simple tasks. Simply configure the init-param externalCss, e.g.

    <init-param>
      <param-name>externalCss</param-name>
      <param-value>../demoapp.css</param-value>
    </init-param>

    Place the demoapp.css in the root directory of your application (not application server ROOT).

    The css will be placed at the end of the stylesheet links in the generated index.html page.

  • Mobile check

    We check if the UI is running on mobile browsers and set parameters for your application: Web.mobile (= VaadinUI.PROP_MOBILE) and Web.mobile.tablet(only for iPad at the moment) (= VaadinUI.PROP_TABLET). Simply check the parameter in your application.

  • Cache for dynamic images

    If you use images from your classpath, vaadin generates internal URLs for you. This URLs are different per image and the browser cache doesn't work. This handling produces unnecessary requests. We solved the problem with an image cache on server side and custom Image resources. With this changes, the browser cache works as expected.

  • IDs for menu items

    We have an experimental solution for setting IDs on menu items. It's experimental because it's possible that vaadin implements support for that. In that case we won't need our solution anymore. But till then, it works. More information...

  • Responsive support

    It's now possible to register a resize event on the application launcher. Use this event to change your application on-demand. We have some impressions for you.

  • Self-contained packages for Portlets

    We have built-in support for self-contained portlets. It's possible to toggle between global and self-contained mode. Simply set the init-param vaadin.shared (true | false) in your web.xml to use vaadin shared from your application server or the self-contained vaadin.

    General information about self-contained packages.

  • invokeLater and WebSockets

    We've completely changed our invokeLater mechanism because it didn't work in previous versions.

Please check the changelog for a complete list.

Responsive Web Design

Post to Twitter

Hurray, we have a new buzzword: Responsive.

It's not brand new, but nowadays it's important because we use applications on different devices and the same application should work on different screen sizes/resolutions.

What does responsive mean?

Wikipedia:

Responsive web design (RWD) is a web design approach aimed at crafting sites to provide an optimal viewing experience—easy reading and navigation with a minimum of resizing, panning, and scrolling—across a wide range of devices (from mobile phones to desktop computer monitors).

My opinion:

Don't waste space for unnecessary information.

If you have big menus and toolbars in your application, you should reduce the used space dependent of the available space.

We did some experiments with our vaadin UI and have some impressions for you:

App full space

App full space

 
App with limited space

App with limited space

The application on the right has a small menu without additional text and with small padding. All gaps are smaller than in the orginal application on the left.

We used custom css and source code to change the UI because not everything was possible with CSS e.g. set tooltips instead of button text. A set of instructions was posted from Matti Tahvonen.

We made some smaller changes in our Vaadin UI to support technology independent handling of resize changes.

It will be possible to do following:

launcher.eventComponentResized().addListener(this, "doLauncherResized");

...

public void doLauncherResized(UIComponentEvent pEvent)
{
    IDimension dim = ((ILauncher)pEvent.getSource()).getSize();
   
    if (dim != null)
    {
        int iWidth = dim.getWidth();
       
        if (iWidth < 800)
        {
            layoutMode = LayoutMode.Small;
        }
        else
        {
            layoutMode = LayoutMode.Full;
        }

        resizeMenu();
    }
}

One application, different styles

Post to Twitter

Impressions

The first image shows the legacy mode of our standard application style (check last image). It turns a SDI application to MDI - without coding or re-deployment. Simply awesome :)

The second image doesn't use a toolbar and the third one is without a menubar. Not all applications need multiple navigation elements, known from desktop applications.

The fourth image shows, what we prefer.

Legacy mode (MDI)

Legacy mode (MDI)

 
Legacy mode (no toolbar)

Legacy mode (no toolbar)

Legacy mode (no menubar)

Legacy mode (no menubar)

Modern mode (SDI with popups)

Modern mode (SDI with popups)

One more ;-)

Modern style (menu right)

Modern style (menu right)

[Update]

The same application with Swing UI

Swing UI

Swing UI

Add IDs to your Vaadin Menu items

Post to Twitter

It's a shame :) but Vaadin doesn't have built-in support for setting an ID on menu items. There are several tickets about this problem: http://dev.vaadin.com/ticket/3873, http://dev.vaadin.com/ticket/11307, http://dev.vaadin.com/ticket/14294

BTW, the whole menu solution is a little bit tricky and wrapped.

There are no excuses - a web technology should fully support web concepts. We tried to touch menu items, by ID, during our UI test automation efforts. It wasn't a problem to find an xpath for accessing menu items, but... it's a principle thing.

It was a nice task to implement ID support for menu items. We found a "general" solution - it's still a workaround, but as long as vaadin doesn't support it, it might help you.

Summarized

Extend server-side MenuBar and add a map for menuitem ids. Send the map to the client and add the id to the matching menu item. The client-side needs an extended VMenuBar and an extended MenuBarConnector.

We've used MenuBar for caching ids because it wasn't possible to extend menu item handling and MenuBar already has a connector.

An example

Menuitem IDs

Menuitem IDs

Details

The implementation is available in our Vaadin UI implementation. Simply use following classes:

MenuBar
VMenuBar
MenuBarConnector

And don't forget the replacement mappings for your widgetset before you start the compiler:

Widgetset replacements

The solution was built with vaadin 7.1 but we checked the implementation in 7.3.0 and it didn't change. So our workaround should work without problems.

We didn't need a lot LoC to solve the problem and it should be easy for you to understand what we did. If you want to know more details - simply ask.

Usage

A short code snippet:

MenuBar bar = new MenuBar();

MenuItem miInsert = bar.addItem("Insert");
MenuItem miUpdate = bar.addItem("Update");

bar.setId(miInsert, "mi_insert");
bar.setId(miUpdate, "mi_update");

As written before, it's not possible to set the ID on the item because of original implementation in vaadin. Don't use getId of menu item because this will return the "internal" id.

Keep coding :)

New application frame feature - No menu

Post to Twitter

All our applications have a menu and a toolbar. This is great for backend applications but not always good for web frontends. Our web UIs have a different menubar, styled for web, but it's always visible. If you have simple web forms, you won't use a menu. We did some changes and have new options for applications without menu and toolbar :)

Some impressions with standard (backend) Desktop application and as (frontend) web application:

Backend application (Swing)

Backend application

 
Frontend application (Vaadin)

Frontend application

It's very simple to hide the menu. Simply set an application property via application.xml or directly via launcher - that's it. We changed our web menu and allow access to internal panels and components. It will be possible to hide buttons, change layouts, etc.

It's still possible to create your own, custom, application frame or extend our pre-defined frame. The new feature will be available in the next VisionX update.

Vaadin UI application frame and windows

Post to Twitter

Our custom Vaadin application frame isn't a MDI. It's "web SDI" . We didn't show popup windows aka internal frames with work-screen content. We did embedd the content directly into the web page. But we changed the default implementation a little bit because it makes sense to show popup windows for modal work-screens. Modal frames are important for database applications. Such frames could be used for showing record details, for creating new records, ...

So it makes sense that our application frame supports such specific work-screens. And the good news is that we've support for this.

It could look like following screenshot:

Modal frame

Modal frame


It's a screenshot from one of our test applications. The feature isn't generally available yet, but it will be part of the next VisionX release.