Blog @ SIB Visions » Development

Category: Development

Pair Programming experiment

Some months ago we were asked if we want to participate a pair programming experiment. The guy who asked was "Code-Cop". Of course, this is not his real name but it's easier for you to understand what he does. He is fanatic about code quality. He likes his code being in order, e.g. nicely formatted, readable, proper named, designed, tested etc. And trust me, these are not just words for him.

Read the full story about Code-Cop and his craftsmanship tour on his blog.

My first thoughts

I did pair/xtreme programming in 2001 and thought it would be nice to do it again. But I thought it could be a risk because I found my own source style and defined my own quality standards over the years. I thought that my quality standard is very high and I'm a professional developer with a lot of experience. You could say that I have a lot of self-confidence... hm... not really bad. What I also learned over the years was that you can't be successful without changes. Every change is a risk because you learn/see something new and maybe you don't like it. But you don't know what you like until you've tried it. This is the most important experience - from my point of view.

So it was great to have this new opportunity and it's always cool to work with same minded people

The experiment

We planned to work 3 days with Code-Cop. It was not clear if one person or two persons should work with Code-Cop. We wanted to use the experiment as mechanism of getting input from "outside". The more different perspectives, the better. We had many different tasks for the 3 days, but not all were good because I'm researcher and love playing around with new technologies/frameworks and products. If you do pair programming, you should develop.

I had one task that was open for a while and it frustrated me that it was open. It was the optimization of our JVx build process and fixing some failed test cases. Don't misunderstand me... I hate red test cases and my tests are always green, but it's boring if your tests are green on your developer engine but fail on the build server (different encoding, different OS, ...). Another bad thing was that the build ran about 80 minutes!
The problem were our unit tests because we mixed manual (performance, timeouts, ...) and automatic tests. My plan was a build time around 10 minutes.

The optimization and tuning was the task for one day. Another task was the implementation of a new feature for our product VisionX (a very short task for about a half day). The rest of the time was hardcore coding with Martin, because we have one really complex model class in JVx. Its the MemDataBook class and it grew over the years. It's about 5000 lines with about 3000 LoC. It's a big class but it's not a monster. It's the biggest class in JVx and has the most logic. It was the right class for pair programming

How we worked together

We worked together on the same machine with a big external screen, two keyboards and two mice. The first day was a challenge because Code-Cop and I tried to find out who's the top dog Not that bad, but it was a big difference to work with another person on the same machine and the same problem (and it didn't help that we are ambitious and dominant). We had different ideas and sometimes different solutions for a problem. I tried to dominate the keyboard because I was used to. One tip: Don't do this, because you don't focus on the problem.

We had great discussions and smaller brainstormings about code quality. It was awesome.

From the second day, everything was in flow because the first day was to get to know the other person. We did our tasks and I had some extra tasks/ideas that could be useful.

The rest of the second day and the third day were reserved for Martin. It was funny to see them in action, because Martin got a "Mouser" and "Keyboarder". It was a long awaited dream of him - a person who understands what he think and can produce source code of his ideas. Of course, they had a tricky task and it wasn't possible to finish the task in two days, but Martin got some new ideas. And last but not least - they had fun!

All 3 days were awesome because Code-Cop was productive from the first minute. We didn't waste time to explain how our framework works. This only is possible if you work with geeks.

Lessons learned

It's great to try new things
Work with geeks who are professional and have a lot of experience (if you find someone)
Use the keyboard and learn most important shortcuts of your IDE. You save time!
Don't think you are professional enough. There's always space, but be solution and practice-oriented.

The experiement was awesome and Code-Cop is a pro' geek. We had a great time and I can recommend such experiments.

Development | rjahn | 19/10/2013 | Comments (0)

VaadinUI 1.0

Our new vaadin UI is available. It's the first release and it's awesome.

Get the binaries from our project site. It's licensed under Apache 2.0 and free for all.

The installation is very simple:

Download jvxvaadin-1.0.zip
Unzip the archive
Add all files to your existing JVx application
Change Deployment descriptor (web.xml) and replace simpleapp with your application

Have fun

News, Release notes | rjahn | 13/10/2013 | Comments (0)

JVx, vaadinUI

Vaadin OnlineHelp 1.0

Our new Online Help system is available. It's the first release with our new vaadin UI. It's fully compatible with our old, GXT based, help system.

Get the binaries from our project site. It's licensed under Apache 2.0 and free for all.

The installation is very simple:

Download jvxhelp-1.0.zip
Unzip the archive
Copy the new directory to your application server, e.g. /webapps/
Open http://yourserver:port/jvxhelp-1.0/help/

The help content is saved under /structure. The system reads all available files and directories from the structure directory and creates a table of contents.

News, Release notes | rjahn | | Comments (0)

Online help

JVx 1.2 is out

We released JVx 1.2 today - as planned

The binaries are available on SourceForge or via Maven central. We also updated our Archetype to version 1.2.0 and it should be available in Maven central in the next days.

We told you that the release contains about 90 tickets. The real number of changes is 123.
Check the changelog for a complete list.

Next version will be 2.0

We did decide that version 1.2 is the last release before 2.0. It's not because of new killer features or big API changes.
The higher version number should represent the maturity of JVx.

JVx was started in 2008 and the low version numbers were fine for our own goals, but our users asked for bigger steps. If we compare our 1.2 with other frameworks, we could use 5.0 without problems.

The version 2.0 will be a smaller feature release that changes MetaData handling on server-side. We'll introduce a new caching mechanism that allows manual change of storage metadata.

We plan the release for the end of this year - without guarantee.

We also plan maintenance releases starting with 2.0

Currently, we don't fix bugs in old JVx releases. We only fix bugs in our development version. We offer nightly builds and maven snapshots and we thought that's enough, but some users want to keep old releases. No worries, we'll do our best to make you happy!

News, Release notes | mhandsteiner | 04/10/2013 | Comments (0)

JVx

Create CSV files of JVx storages

Our AbstractStorage implementations have a writeCSV feature already implemented and you can call this method via action call mechanism. But sometimes you need different CSV exports or ZIP archives with different CSV files. You simply need control over the export.

There's a small project with the name AES Storage Export that could help you solving CSV export problems. It has one simple class that takes one or more storages and exports data as UTF-8 CSV files. All CSV files will be added to a zip archive (optionally AES encrypted)

Simple use the class in an server action and create your custom CSV exports.

Test case:

StorageExport export =new StorageExport();
ICondition condFilter = new GreaterEquals("ID", BigDecimal.valueOf(10)).and(new LessEquals("ID", BigDecimal.valueOf(20)));
StorageEntry entryColumns = new StorageEntry("columns.csv", createStorage(), condFilter);

entryColumns.setColumnNames("ID", "VALUE");
StorageEntry entryColumnsStorage = new StorageEntry("columns_storage.csv", createStorage(), condFilter);

entryColumnsStorage.setColumnNames(createColumnStorage(), "NAME");
export.add(new StorageEntry("first.csv", createStorage()));

export.add(new StorageEntry("filtered.csv", createStorage(), condFilter));

export.add(entryColumns);

export.add(entryColumnsStorage);

export.setPassword("testcase");

export.setSeparator(",");
File fiTemp = new File(System.getProperty("java.io.tmpdir"), "aesarchive.zip");
RemoteFileHandle rfh = new RemoteFileHandle();

export.export(rfh.getOutputStream());
FileUtil.save(fiTemp, rfh.getInputStream());
System.out.println(fiTemp);

Development | rjahn | 28/08/2013 | Comments (0)

JVx

Project news

Some years ago, we started a project with the name OnlineHelp. It was used to create simple online help pages for applications. The project was implemented with GXT 2.1 and was really useful. It's still useful but the technology is old, and GXT is not our best friend. Of course newer versions are really fancy, but the license is not the best for commercial applications.

Some month ago, we decided to switch completely from GXT to vaadin - all our projects. The first project was our webUI and we make good progress. We're near to a first release, but we have no definite date for it (Q4/13 should be possible). The second project is the online help. We make good progress and have a first preview:

Online Help with vaadin

The style is similar to our new web application style but not a big change compared to our old online help, because our users like it:

Online Help with GXT

The new style is fresh but not too much... We added some new features like full-text search and topic navigation (previous, next). It's a first preview and not more!

The project has now it's own project page and is not anymore a sub project of JVx. The license is still Apache 2.0.

Development | rjahn | 19/07/2013 | Comments (0)

JVx, Online help

Maven deployment via Ant

We are no Maven lovers, because it helps to forget how things work. But we think the dependency management is useful. It's great for developers who need specific library versions without managing them manually.

The user aspect is one thing, but nobody tells you that it's not trivial to release libraries. One problem is the pre-deployment process. Before you are able to deploy your jar files, you have to do a lot of things like GPG key generation, publishing GPG key, find the right repository, prepare your build, special pom cration and so on.

If you already have release builds with version numbers, javadoc and source archives, you'll save a lot of time. If you don't have complete and clean release builds - see you later.

Let's start with a good documentation about requirements:
https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide

Of course, it assumes that you'll use Sonatype as your repository, but all others are not too different.

This posting is not a complete documentation. It simply shows the problems we had.

Our simple pom files:
pom.xml (jvxall)
pom.xml (jvxclient)

The first is with dependencies and the second without, because our client doesn't have dependencies.

The real problems started with the integration in our ant script, because the documentation of Maven plugins were awful. It's easier to read the source code than to find out how plugins work. One example: Read following plugin documentation and tell me the valid values for "types" and "classifier". There are no examples on the page! Google around and copy/paste a little bit - awful!

Here's a working ant call

<typedef resource="org/apache/maven/artifact/ant/antlib.xml" 
             uri="antlib:org.apache.maven.artifact.ant" 
             classpath="${build}/maven/maven-ant-tasks-2.1.3.jar" />
<condition property="gpgexecutable" 

      value="C:\Program Files\GNU\GnuPG\pub\gpg.exe" else="gpg">

    <and>

        <os family="Windows" />

   </and>

</condition>
<artifact:mvn>

    <arg value="org.apache.maven.plugins:maven-gpg-plugin:1.4:sign-and-deploy-file" />

    <arg value="-Durl=${mvn.url}" />

    <arg value="-DrepositoryId=${mvn.id}" />

    <arg value="-DpomFile=${maven.tmp}/jvxall/pom.xml" />

    <arg value="-Dfile=${mvn.jvx.jar}" />

    <arg value="-Dfiles=${mvn.jvx.sources.jar},${mvn.jvx.javadoc.jar}" />

    <arg value="-Dclassifiers=sources,javadoc" />

    <arg value="-Dtypes=jar,jar" />

    <arg value="-Pgpg" />

    <arg value="-Dgpg.executable=${gpgexecutable}" />

</artifact:mvn>

Above call submits e.g. jvx-1.1.jar, jvx-1.1-javadoc.jar and jvx-1.1-sources.jar to the repository.

Why one call instead of 3 separate calls, as described in the documentation?

Short: It's better
Long: It's better to submit all dependent files in one "maven session". Above call creates a new maven project and if you call this command per file, there's no logical connection between them. If you plan to deploy snapshot releases - forget it - it doesn't work with different calls because every upload gets a new buildnumber. But all files need the same buildnumber! Such deployments can't be used. If you read the Sonatype document, you saw that "deploy" task didn't deploy javadoc and sources! The "stage" task did. I'm not sure, but I think they had the same problem with separate uploads! Trust me, above call works with snapshots and final releases.

Our naming conventions

<property name="mvn.jvx.jar" value="${release}/maven/${release.name}-${versionnumber}${maven.version.postfix}.jar" />

<property name="mvn.jvx.sources.jar" value="${release}/maven/${release.name}-${versionnumber}${maven.version.postfix}-sources.jar" />

<property name="mvn.jvx.javadoc.jar" value="${release}/maven/${release.name}-${versionnumber}${maven.version.postfix}-javadoc.jar" />

(${maven.version.postfix} is -SNAPSHOT for snapshot releases and empty for final releases)

Repository Id, URL

${mvn.id} = sonatype-nexus-staging
${mvn.url} = https://oss.sonatype.org/service/local/staging/deploy/maven2

Other problems?

Proxy settings for Maven, Use an external GPG key, Autentication.

If you copy the following settings.xml to <users.home>/.m2 and modify it to fit your needs, it should solve all problems:

<?xml version="1.0" encoding="UTF-8"?>
<settings>

  <servers>

    <server>

      <id>sonatype-nexus-snapshots</id>

      <username>username</username>

      <password>password</password>

    </server>

    <server>

      <id>sonatype-nexus-staging</id>

      <username>username</username>

      <password>password</password>

    </server>

  </servers>

  <profiles>

    <profile>

      <id>gpg</id>

      <properties>

        <gpg.passphrase>gpgkey</gpg.passphrase>

      </properties>

    </profile>

  </profiles>

  <proxies>

    <proxy>

      <id>firewall</id>

      <active>true</active>

      <protocol>http</protocol>

      <host>10.0.0.1</host>

      <port>3128</port>

      <username></username>

      <password></password>

      <nonProxyHosts>localhost,127.0.0.1</nonProxyHosts>

    </proxy>

    <proxy>

      <id>firewall-2</id>

      <active>true</active>

      <protocol>https</protocol>

      <host>10.0.0.1</host>

      <port>3128</port>

      <username></username>

      <password></password>

      <nonProxyHosts>localhost,127.0.0.1</nonProxyHosts>

    </proxy>

  </proxies>

</settings>

We configured the proxy via ant build.xml

<target name="proxy.check">
<condition property="proxy.enabled">
<and>
<socket server="10.0.0.1" port="3128"/>
</and>
</condition>
</target>
<target name="proxy" depends="proxy.check" if="proxy.enabled">

  <property name="proxy.host" value="10.0.0.1"/>

  <property name="proxy.port" value="3128"/>

  <property name="proxy.user" value=""/>

  <property name="proxy.pass" value=""/>

    

  <setproxy proxyhost="${proxy.host}" proxyport="${proxy.port}" 

             proxyuser="${proxy.user}" proxypassword="${proxy.password}"/>

</target>

Simply add "proxy" task as dependency of another task.

Maybe it's easier to release libraries with other build systems or maybe it works out-of-the-box with Maven itself, but the whole process is really bad.

Good luck!

Development | rjahn | 16/07/2013 | Comments (0)

Ant, JVx, Maven

JVx - Maven snapshots

We now provide our JVx nightly via Maven Snapshots. Our nightly build job automatically uploads new JVx snapshots. If you want to use the last JVx snapshot in your project, simply add:

<repositories>

  <repository>

    <id>sonatype-nexus-snapshots</id>

    <name>Sonatype Snapshots</name>

    <url>https://oss.sonatype.org/content/repositories/snapshots/</url>

  </repository>

</repositories>

to your pom.xml and set JVx' version to: 1.2_beta-SNAPSHOT.

If you use our new JVx archetype 1.1.9, simple change the version in the master project:

<properties>

  <jvx.version>1.2_beta-SNAPSHOT</jvx.version>

</properties>

All SNAPSHOT versions contain debugging information. The release versions don't.

Development, News | rjahn | 15/07/2013 | Comments (0)

JVx

New project: toPDF

We tried to find a simple solution to convert MS Office files to pdf, without online services. We tried OpenOffice but the results were awful! There are a lot of free, and commercial, PDF printers available. But they are for desktops and a user has to print manually. We wanted a solution that works without user interaction.

There is a very useful open source project called PDFCreator. It also is a printer but has a useful API. The API is available via COM, which is not the best technology for Java, but it's also not bad.

We didn't find a ready-to-use solution for our idea and it shouldn't cost money. The solution had to be open source. We found some great commercial tools and SDKs but all of them were not cheap.

We spent some hours and used PDFCreator, Jacob and some other open source tools to create an "Online service for PDFCreator". The result of our work is toPDF.

What is toPDF?

It's a small library that allows conversion of files to PDF, via PDFCreator. It's also a web application that offers services for remote conversion via http. The application has a REST service and a simple servlet service.

Simply POST binary data via http request and receive a PDF in the response. The servlet supports multipart form-data and simple application/octet-stream as requests. The REST service also supports multipart form-data but also JSON requests.

A short example:

URL url =newURL(getServletService());
URLConnection ucon = url.openConnection(); 

ucon.setDoOutput(true);

ucon.setDoInput(true);

ucon.setUseCaches(false);

ucon.setRequestProperty("Content-Type", "application/octet-stream");

ucon.setRequestProperty("Content-Disposition", "attachment; filename=\"Forms.docx\";");
FileUtil.copy(ResourceUtil.getResourceAsStream("/com/sibvisions/topdf/Forms.docx"), 

              ucon.getOutputStream());
byte[] byData = FileUtil.getContent(ucon.getInputStream());

or as Multipart:

MultipartUtil multipart =new MultipartUtil("UTF-8");

multipart.addDataPart("data", "Forms.docx", 

                  ResourceUtil.getResourceAsStream("/com/sibvisions/topdf/Forms.docx"));
byte[] byData = multipart.post(getServletService());

The conversion via PDFCreator works great, but not perfect. There are different problems with small page margins in Word documents, problems with OpenOffice documents, ...

The problem is not toPDF, because it works as good as PDFCreator does. If PDFCreator doesn't convert a document, toPDF has no chance to convert it.

We had problems with simple image conversions to PDF because default windows print dialog appeared and we didn't associate image extensions with another tool. We solved the conversion of images with iText instead of PDFCreator. Now it's possible to create PDFs from images very easily without pop-ups.

License?

AGPL 3.0, because PDFCreator is licensed under GPL and iText is licensed under AGPL.

Used tools and libraries

toPDF is a mixture of different open source projects:

PDFCreator
iText
RESTlet
Jackson
JVx
Apache commons FileUpload and IO
Jacob
PDFCreator4J

Installation?

toPDF was written in Java, but the installation only makes sense on Windows (same requirements as PDFCreator)
Install PDFCreator (default desktop installation, with COM)
Deploy topdf.war on Tomcat or JBoss or your preferred Java application server. If your application server runs as windows service, be sure that it runs as OS user.

Development | rjahn | 03/07/2013 | Comments (0)

PDF