It's now been a year since I joined SIB Visions, I guess that's a good time to take a short break and look at everything that has happened so far.

Know the framework

The first day at SIB Visions was filled with getting familiar with the JVx application framework, including a crash course through the complete architecture of it. I have to be honest here and say that I have maybe understood half of what I was told, but it was an awesome and extremely useful overview over the framework.

Right after that I made myself roughly familiar with the framework by looking through the code, I got assigned my first tickets, simple feature requests and bugs, easy enough to do. The next weeks I read through quite many areas of the codebase and learned to navigate through it. Nothing too exciting.

FMB Importer - Oracle Forms to JVx

The first really big task I was assigned to was writing an FMB importer from scratch. In case you don't not know what FMB is, it is the file format used by Oracle Forms Builder to save the created forms. It basically comes in two flavors, binary and XML. The goal was to be able to import a form created with Oracle Forms Builder directly into VisionX, preserving as much information as possible and making the migration of the GUI as seamless as possible.

Reading the XML file is easy, processing its content not so much. The concepts of JVx UI and Oracle Forms UI are quite different. In JVx the controls are forming a tree structure/hierarchy, with the inheritance dictating the size and location of a control. Oracle Forms uses a coordinate approach, with each control knowing its exact coordinates and size on the form. This can be clearly seen in the following picture:

On the left is the JVx control tree (Swing), on the right the Summit example in Oracle Forms Builder, the XML format looks quite the same as the view in the designer. After some trial and error, I decided to take a two step approach to converting this structure:

1. Convert the XML nodes into custom objects and place them in a tree structure.
2. Convert the custom objects to JVx classes and perform the layouting in this step.

Step 1 starts rather easy, I encapsulated the XML nodes in a custom class to have a thin interface over them, that allowed me to be able to provide support methods and yet still have all the information of the XML node easily accessible. The tricky part becomes the creation of the tree structure and the conversion to the custom objects, because the controls of the form do not have any sort of structure and are split in two groups, graphical elements (lines, circles, etc.) and controls (text fields, radio button, etc.). The best working solution I found was to create a processor object that does know everything necessary to create the custom objects and also builds the tree structure. That processor starts with the window and finds all canvases that can be on that window, then it finds all controls and graphics that can be on these canvases. From there it recurses through all controls and graphical elements, always finding the children of the current one by checking their coordinates and size. Extremely simplified it looks like this:

"Extremely simplified" might be an understatement, because there are a lot of edge cases that need to be taken into account and you can't treat all elements the same, but basically that's the core principle of the processor. If you also go with naming your functions after what they are doing, don't be surprised if you end up with a function named transformItemNodesOnCanvasOnTabPageInBoundary. In the same step as the creation of the tree is the creation of a custom object hierarchy. I decided against converting them directly into JVx classes, because that would add a lot of logic to the processor and would mix two tasks, creating the tree structure and converting the objects.

Now that we have a nice tree of objects, converting them to JVx classes is easy. Ideally all custom objects that were created in the previous step would inherit from one base, and that base ideally would have a method like toJVxComponent(). With that in place converting the custom objects becomes only a matter of calling root.toJVxComponent() after the tree was generated. Again, the second part of this step is what gets you, the layouting. A lot of trial and error went into this one and while Oracle Forms does only support one layout, coordinates, JVx has multiple:

• Pane
• SplitPane
• BorderPane
• FormPane

So we need some form of heuristic approach to determine what layout we will be using for each container based on:

• The number of children.
• The position and size of these children.

Again, very simplified the approach might look like this:

Again, this misses quite a few edge cases and optimizations. But now the really hard part is missing, how do we convert a coordinate base layout to a FormPane layout?

I tried many multiple methods but ended up using a Sweep line algorithm. The basic idea is to have a line sweep from left to right/top to bottom and creating anchors on the go as necessary.  We'll sweep from left to right, for example, and create a new anchor everytime we encounter the start of a new element and and element has ended before. Assume the following layout:

We'll sweep from left to right, create anchors and stretch all components to their respective anchors:

Even though the layout looks quite nice already, there are two problems here. First, the second/third and fourth/fifth anchors are empty, they do not contain a component ("contain" as in a component that does start and end in that anchor). So we can eliminate all anchors that do not contain a component.

The second problem are the overlapping components in the middle. We can easily fix that by pushing the components around.

All that is missing now is to two a second pass from top to bottom. So we now have everything in place to import the UI. Of course there are still some parts missing, like data binding and wiring up events. But that wraps up the approach to importing FMB files.

EPlug - JVx Eclipse Plugin

I enjoyed working on the plugin very much. The Eclipse environment was a completely new system to work with and I had never before written a plugin for a development environment. After the basic functionality was determined, I set to work and made myself familiar with Eclipse. As it turns out, the Eclipse plugin system is quite huge, but once you know the basics it is very easy to use. Every plugin consists of a simple "core" of files:

• plugin.xml
• Manifest.mf
• Activator

The plugin.xml contains all the needed information for the Eclipse runtime to load and run the plugin. Also this is where you register all the extensions you want either to use or provide. The Manifest.mf contains some additional information and the Activator is the main lifecycle class for the plugin. Registering extensions is as easy as implementing the interface required by the extension point and then adding your implementation to the list of the extension point. Your class will be automatically instantiated and used from the class it is registered at.

The first big obstacle was implementing auto completion for various code parts, for example our listeners. As it turned out JDT (Java Development Tools) does only have limited support for it. If you use their parser, you can only have complete lines, it was not possible to get incomplete statements. So we had to rely on internal classes provided by the "JavaCompletionProposalComputer" and the "JavaContentAssistInvocationContext" which also allowed us to get the information needed for incomplete statements.

The other big part of the plugin is launching an application in a container, that allows to extract information about it (like lifecycle objects, server metadata, etc.). That part is quite big and complicated, so let me shortly outline the approach for it:

• Create a new Server.
• Gather all necessary dependencies for the application.
• Create a ClassLoader that contains all these dependencies and inject it into the ObjectProvider of the server.
• Create yourself a new session.

Obviously this is only a rough outline, for example there's the security manager. We took the approach to change the configuration of the application on the fly and inject our own security manager that way, which obviously allowed us to do everything within the application. Also our session is a custom implementation that can never expire.

From there it is only a small step to receiving metadata and other information needed to provide auto completion and compile time errors. Speaking of compile time errors, implementing those was as easy as eating a cake. The only two things needed are to register a build watcher, which will be notified of every file that is touched by the build process, and to run the file you want through the parser to receive the AST. Walking the AST is thanks to the visitor pattern very easy and you can quickly select only those parts of the AST that you're interested in.

One of the biggest features we can offer is auto completion and compile time support of DataBook columns. From a technical point of view, the support for RemoteDataBooks is kinda boring, we simply query the server for the metadata. However the support for MemDataBooks was a challenge. Because of the very nature of MemDataBooks, there only were two options to receive the needed MetaData:

1. Run the code and extract the needed information.
2. Parse the code/AST.

The first possibility got axed extremely quickly, as it would be too hard, if not impossible to do it in a sane manner. So we ended up with parsing the code/AST directly for any information about the MemDataBook which we tried to provide support for. This is a tedious work, as you basically have to consider all the permutations of creating a MemDataBook and everything related with it, for example the following code:

If we now need to get the metadata for this MemDataBook, it would be roughly the following steps:

• Find the variable that contains the MemDataBook.
• Find the declaration or last assignment of that variable.
• Find all usages of that variables.
• Get all values from the .getRowDefinition().addColumnDefinition calls.

This is quite some work and that does not take into account that the row definition might be put into a variable, or might be provided by a different variable altogether. In the end, it comes down to implementing all the edge cases that you run across, and I think we did that very well.

Also we had to create a few GUI forms with SWT and I have to say that it is very good.

I'm looking forward to return to this project for the 1.2 release and implementing even more awesome features and doing some refactoring on the components I did not quite get right the first time.

GUI Tests - Stress testing Vaadin and automating JVx applications

I was looking forward to this research project. I was always fascinated with the idea to have GUI unit tests, now I had the chance to actually do some research on them and implement everything necessary in JVx to make them possible.

You can read about the results of the Vaadin stress tests on a previous blog post.

After trying different possibilities, including the awesome Sikuli, we settled with AssertJ-Swing for Swing and Selenium for Vaadin. The first rough tests showed that testing was only reasonable possible if we would name all components by default and so we derived a default naming scheme and implemented it. While experimenting with AssertJ-Swing I realized that one important thing was missing from it, the possibility to easily record tests. Selenium already offers a simple recorder for most browsers, which simply records all events and exports them in the correct format, however AssertJ-Swing does not have such a facility unfortunately.

Swing itself is build on top of AWT, which allows to register a global event listener by using the Toolkit which means that one can see all events that are happening, which itself makes it rather easy to record these events and put them in an easy to use format. The format in our case would be Java code that can be copied and pasted into our test cases. To have a reasonable layer of abstraction, the recorder would filter and prepare the events and forward them to a "plugin" that is able to omit Java code. With this facility in place it was easy to record tests for Swing, however replaying them was not so easy at first. AssertJ-Swing has very concrete requirements when it comes to how it is launched and JVx has its own ideas regarding that. The solution to this was to derive a test harness, which would manage both, the AssertJ-Swing setup and also the start of the JVx application. The unit tests would then simply extend said test harness and would be able to launch the application on their own, executing the test case and then shutting it down again. This worked extremely well and we were even able to record and replay test cases for VisionX with ease.

These experiments have shown that it is very easy to perform automated GUI tests on JVx applications, no matter if Swing or Vaadin is used as frontend. The created code during these experiments is not public at the moment, as there is still quite a lot to do and it's in dire need of some refactoring. But I'm very confident that we will be able to return to it and one day provide you with the tools to easily create automated GUI tests for JVx.

JavaFX - Thee shall be my nemesis

JavaFX has been in the works for quite some time now and has been coming along greatly. It was overdue that JVx does support JavaFX as a frontend and with the beginning of this year the work started to make that happen.

Event though JavaFX has been coming along greatly, it is still quite rough around the edges and contains bugs that we needed to work around and some missing features. Currently we are monitoring over 40 requests in the bug tracker of JavaFX that are either directly or indirectly related to our JVx implementation,  a little less than half of these were reported by us. Those span little bugs to quite big feature requests, some more critical than others.

One of the biggest missing features was the missing MDI system, which we simply need for ERP applications. There is RT-22856 which is concerned with creating such a system, but that hasn't moved in quite some time and so we set out to create our own. Our efforts are quite well covered by the blog post I linked to above, we managed to come up with a very nice implementation:

• The FXInternalWindow can either be used with or without a desktop pane. They might as well live in a normal Pane (or any other).
• The content of internal windows is not limited in any way (you could also nest another desktop pane into it).
• The FXDesktopPane does offer various convenience features, like a pluggable window manager system.
• Implementing a new and custom window manager is a breeze, there are already two implementations provided by us: The FXDesktopWindowManager and the FXTabWindowManager.

But the MDI system is not the only system or control that we created to aid in our implementation of JVx on top of JavaFX, there is a long list of controls that were created by us:

• A DesktopPane for JavaFX
• Drag'n'Drop support for JavaFX' TabPane
• JavaFX: Styled stage and MDI system
• JavaFX FlowPane vs. FXFluidFlowPane
• Charts with JVx and JavaFX
• And more to come...

But one thing that really drove me up the wall several times was the usage of the "final" keyword and "package-private" modifiers. Multiple times problems could have been easily solved by overriding a method, which turned out to be final, or invoking a method or accessing a variable, which turned out to the be (package) private. Good API design dictates that both are used to protect the API from misuse, but for our work on the JVx/JavaFX project, most of the time it stood more in the way than helped. The take away for me personally is, that of course you need to design the API in a way that limits the possibility for abuse, but you also have to allow users to shoot themselves in the foot from time to time, that means that they can experiment and use the API for possibilities it might not have been intended for.

The look ahead

And now we're in the present, phew, what a year! There are already two projects waiting for me as soon as we're done with the first version of our JVx/JavaFX implementation.

I'm looking forward for another year with the awesome people at SIB Visions!