wissel.net

Usability - Productivity - Business - The web - Singapore & Twins

Async testing with vert.x


Test driven development gets interesting when actions might or might not complete somewhen in the future. Point in case: HTTP requests (fetch returns a promise)

vert.x and jUnit5

I frequently write little tests, that spin up an http sever before the test, run individual tests using a WebClient. All these operations run asynchronous, returning a vert.x Future (In JavaScript Land the equivalent would be a promise)

To make this pain free, vert.x provides a full integration into jUnit 5. Using the annotation @ExtendWith(VertxExtension.class) vert.x provides two injectable parameters: Vertx and VertxTestContext. Add the simple mental rule: All async operations need to interact with the VertxTestContext.


Read more

Posted by on 03 November 2022 | Comments (0) | categories: Java vert.x

Beyond Java 8: The Cool Stuff


Java is evolving rapidly. Henri Tremblay is walking through the cool stuff. Henri is, besides other activities a contributor to EasyMock

From Java 7 (2011) to Java 19 (2022)

Unless stated otherwise Java 17 (2021) LTS will work for the examples. Henri used a trading app to walk through upgrading steps.

Impressive: just moving from Java 8 to Java 19 shows double digit performance improvements

Some of the tips:

  • use the java.time.Clock object for anything date/time, Don't use Date. Inject the clock object.
  • use Comparators
  • use Files.lines to read by line
  • use the Maven deamon to speed compilation
  • Be careful with stream().findAny() - in most cases a Map performs way better
  • Use """ text blocks (why o why did Java not pick the tripple backtick)
  • Use Files.writeString if it is a String you write
  • Use jlink and jpackage for distribution
  • Use switch expressions which returns values (not convinced about that yet)
  • Use Records for immutable data
  • Use the new instanceof syntax
  • Use the new RandomGenerator for better performance
  • Sealed types allow tighter controls

A quite enjoyable session


Posted by on 20 October 2022 | Comments (0) | categories: Domino Singapore

Test-Driven Development Is a Paradox


TDD is favored by very succesfull engineers as a means of accellerating software development by preserving quality.

Objections are plenty. The talk by Burk Hufnagel tries to address those.

The TDD Paradox

TDD requires you to write more code, so you can be done sooner. It's like learning to drive a car. Your first mile takes, give or take, 40-50h to complete: Driving lessons, driving tests, get the license issued before you drive. You can walk a lot more in 50 hours, you get the drift...

Looking deeper at "development" we see, it has multiple components:

  • understanding and clarifying the business requirements (business rather loosely defined)
  • write code
  • manual testing
  • debugging

TDD helps with the first and third component. A well defined test communicates the desired outcome and serves as a living specification. Having the machine running tests reduces the need for manual testing and frees up time to work on functionality

Test early, test often

TDD proposes to write tests first, not as an afterthought. The strongest new argument Burk proposed was: nobody wants to write a test for code freshly debugged and proven working (at that time). There's no immediate value in it. A test, initially failing, written upfront is a reassurance to be on the right track when going green.

Using the "test first" approach, tests turn into a design process, increasing the confidence in not only doing things right, but also doing the right thing. In other words: Tests are Specifications

Best results are to be had when working in small increments and resist the urge to "code for things you think are coming next" instead of "(just) make the test pass".

It's not a cure-it-all

TDD might not be a good fit:

  • no clear business requirements - you can't test when you don't know what you want
  • exploratory coding - trying stuff out

A common objection is "we have an existing non-TDD code base, we can't switch to TDD". Burk suggest to use "Defect Driven Development". Write a test that test for defect free behavior and then fix the defect. Your test harness will grow while defects shrink

Test long and prosper

The usual subjects for testing in Java are JUnit and Mockito. Burk introduced another, very promising tool: Spock. Spock is written in Groovy and allows to specify a test's purpose more expressive and concise than Junit's @DisplayName property. Luckily it isn't an exclusive-or question, Spock interacts with JUnit and Mockito quite nicely.

In conclusion

Burk made a good case for TDD, backed by experience and tooling, if in doubt visit his arguments.


Posted by on 19 October 2022 | Comments (0) | categories: Java

CI/CD, Github Actions, and GraalVM Native Image


GraalVM is a promising polyglot runtime for Java, JavaScript, Python, Ruby and WASM.
It can produce native images. Unsurprisingly with great powers come ... lots of things to learn.

GraalVM Native Image Benefits

A native image has a number of benefits over an JVM based application, mainly owed to the extensive analysis done at build time.
Some of advantages mentioned:

  • Runtime speed ~ 25% above the same code running on an JVM
  • Ahead of Time compilation can lead to up to 9 times less memory consumption than running on an JVM
  • Code executes faster with a smaller memory footprint than comparable Go applications ( but still is bigger/slower than C by a whisk)
  • Warmup time got moved to build time, improving startup time

Building the image

Build has been greatly improved with tooling available:

  • a Maven plugin (also available for Gradle), eliminating the need of extensive commandline foo.
  • a GitHub action to install the GraalVM tooing in a pipeline
  • reduced memory consumption, so most apps will fit into the GitHub build container limit of 7GB memory
  • option for a non-optimized fast build, useful for testing correctness

Thoughts and conclusions

A big step in the right direction making native images more accessible. There's still room for improvements, like caching the build images and more samples. What Oracle solved nicely: If you are a GraalVM Enterprise customer, accessing GraalVM Enterprise edition is done using an access token, fully compatible to any build system. No messing around with downloads and manual installations required.


Posted by on 18 October 2022 | Comments (0) | categories: Java

Streamlining Lage-scale Java Development using Error Prone


It has been a while since I attended an in-person conference. Finally this October I managed to attend JavaOne in Las Vegas. I shall report on interesting sessions.

The first one is by Sander Mak on a topic near and dear to me: Code Quality

Another tool in the box

Error Prone, courtesy of Google, supplements tools like Sonar, Spotbugs or Checkstyle. Other than these tools it is a compiler plugin with > 500 checks.

It can be configured to fix a set of identified errors automatically in code, inline or as patch file. Sander used a few simple demos to drive home what is possible using the tooling. Besides the ckecks that are available out-of-the-box one can implement extensions. One interesting contribution comes from the popular mocking framework Mockito, extending the checks to your use of mocking.

Extending error-prone is possible using two complementary approaches:

  • writing your own extensions (extending BugChecker), dealing with Java's AST and all its power
  • using the included ReFaster templates that allow for powerfull pattern based code upgrades

Use cases

Just a few samples from the session:

  • implement consistency in coding style (not to confuse with code style)
  • reduce visible clutter like final before variables by using @var to denote mutable variables and consider immutable the default
  • ease the transition to higher JVM versions, e.g replace !Optional.isPresent() with Optional.isEmpty() (JDK11)
  • verify logging uses {} instead of %s
  • fail builds that violate rules and conventions

My impression

Error-prone seems to be an important tool in toolbox to produce a well groomed Java code base. I loved Sander's progression from a simple introduction example to the more complex use cases that help his company deliver (pun intended). Error-prone isn't for the faint of heart, but a component that belongs into any code base of size. The session inspired me to improve our Java tooling, well done!

The Steve pun, one last thing, was the icing on the cake

More content by Sander


Posted by on 18 October 2022 | Comments (0) | categories: Java

Calling a vert.x async method from a sync method


Made popular by NodeJS and ES6 asynchronous programming promises (pun intended) better throughput and resource usage by entertaining an Event Loop. In Java land vert.x implements exactly this approach and has proven its mettle, being the foundation of Quarkus

Your legacy app doesn't magically convert

When you start a new vert.x project using the App Generator, everything is asynchronous from the beginning. Snippets of synchronous code (a.k.a blocking code), you might need to maintain, can be wrapped into executeBlocking and handled in their own thread.

However when you are about to convert a synchronous application, e.g. a servlet to asynchronous and you can't finish in a sprint/session, things become interesting. The doGet method is synchronous by nature. There are a few steps that need to be accomplished:

  • Have vert.x running in its own thread. You can't start it on the main thread with its blocking operations
  • Have a method that returns a vert.x Future
  • Convert that into a Java CompletableFuture

Let's have a look at the moving parts:


Read more

Posted by on 03 August 2022 | Comments (0) | categories: Java vert.x

Case insensitive deserialization


Growing up in Windows with BASIC you learn case doesn't matter, so Color is the same as COLOR or cOLOR when it comes to variable names. Same applies to @Formula or item names in Notes documents.

On the other side, Linux, Java, JavaScript and JSON are very much case sensitive.

This poses a challenge when deserializing (handcrafted) JSON files.

The Task at hand

Deserialization of JSON into a Java class instance can be done using jackson. This is also what the JsonObject in vert.x uses when you call json.mapTo(SomeClass). Not using vert.x? You can use the ObjectMapper. Let's look at a sample Java class

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonInclude;
import io.vertx.core.json.JsonObject;

@JsonInclude(JsonInclude.Include.NON_NULL)
@JsonIgnoreProperties(ignoreUnknown = true)
public class JsonTest {

  public static fromJson(final JsonObject source) {
    return source.mapTo(JsonTest.class);
  }

  private String color;
  private String shape;
  private int answer;
  private boolean pretty;

  /* GETTERS and SETTERS omitted for brevity
     Let your IDE add them for you */
}

Now you want to deserialize a good JSON, which works as expected:

{
  "color": "Red",
  "shape": "round",
  "answer": 11,
  "pretty": true
}

but the very moment your JSON isn't following proper capitalization, like human provided JSON,

{
  "Color": "Red",
  "Shape": "square",
  "Answer": 42,
  "pretty": true,
  "ignore": "this"
}

deserialization will fail. We need to fix that.


Read more

Posted by on 08 June 2022 | Comments (1) | categories: Java vert.x

Dance the OAuth with me


OAuth and its cousin OIDC are the ubiquitous methods to gain identity and authorization information. Since it is a ménage à trois between a user, an Identity provider (IdP) and an application, refered to as "Service provider", it is hard to trouble shoot.

A play in five acts

In the recent Project KEEP we build an IdP into the API, so you have the choice of just using Domino or using an external IdP.

To ensure it works as expected several dependent HTTPS calls were needed. Each call would harvest some information into environment variables for the following step

Act 0 - initial setup

Store several variables into the environment:

  • UserName: the user you will simulate to approve
  • Password: their password
  • HOST: The starting URL for the first call
  • state: a random string, need to stay the same through the sequence
  • client_id: The application configured as service provider
  • client_secret: The service provider "password"
  • scope: the scope (or a subset) you have configured for the service provider
  • redirect_uri: one of the redirection URIs you have configured for the service provider

An OAuth flow contains basic authentication calls, so you need to ensure proper TLS connections.

OAuth Dance


Read more

Posted by on 06 June 2022 | Comments (1) | categories: WebDevelopment

Yes No Maybe Boolean deserialization with Jackson


The Robustness principle demands: be lenient in what you accept and strict in what you emit. I was facing this challenge when deserializing boolean values.

What is true

Glancing at data, we can spot, mostly easily what looks trueish:

  • true
  • "True"
  • "Yes"
  • 1
  • "Si"
  • "Ja"
  • "Active"
  • "isActive"
  • "enabled"
  • "on"

The last three options aren't as clear cut, they depend on your use case. Using a simple class, lets try to deserialize from JSON to an instance of a Java class instance using Jackson.

Java doesn't have native support for JSON, so we need to rely on libraries like Jackson, Google GSON (or any other listed on the JSON page). I choose Jackson, since it is the library underpinning the JsonObject of the Eclipse Vert.x Framework I'm fond of. Over at Baeldung you will find more generic Jackson tutorials.

Let's look at a simple Java class (Yes, Java14 will make it less verbose), that sports fromJson() and toJson() as well as convenient overwrite of equals() and toString()

package com.notessensei.blogsamples;

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonInclude;
import io.vertx.core.json.JsonObject;

@JsonInclude(JsonInclude.Include.NON_NULL)
@JsonIgnoreProperties(ignoreUnknown = true)
public class Component {

  public static Component fromJson(final JsonObject source) {
    return source.mapTo(Component.class);
  }

  private String name;
  private boolean active = false;

  public Component() {
    // Default empty constructor required
  }

  public String getName() {
    return name;
  }

  public void setName(String name) {
    this.name = name;
  }

  public boolean getActive() {
    return active;
  }

  public void setActive(boolean isActive) {
    this.active = isActive;
  }

  public JsonObject toJson() {
    return JsonObject.mapFrom(this);
  }

  @Override
  public boolean equals(Object obj) {
    if (obj instanceof Component) {
      return this.toString().equals(obj.toString());
    }
    return super.equals(obj);
  }

  @Override
  public String toString() {
    return this.toJson().encode();
  }

}

Trying to instantiate a class instance with the following JSON will work:

{
  "name": "Heater",
  "active": false
}
{
  "name": "Aircon"
}
{
  "name": "Fridge",
  "active": true,
  "PowerConsumption": {
    "unit": "kw",
    "measure": 7
  }
}

However it will fail with those:

{
  "name": "System1",
  "active": "on"
}
{
  "name": "System2",
  "active": "yes"
}

You get the charming error Cannot deserialize value of type boolean from String "yes": only "true"/"True"/"TRUE" or "false"/"False"/"FALSE" recognized`. Interestingly numbers work.

On a side note: Jackson uses the presence of getters/setters to decide (de)serialization and needs getActive and setActive or isActive. When you name your variable isActive Eclipse would generate setActive and isActive instead of getIsActive / isIsActive and setIsActive. So simply avoid the is... prefix for internal variables.


Read more

Posted by on 07 May 2022 | Comments (0) | categories: Java

The Quest for a software documentation system


Software documentation is a thankless business and never complete. Picking the right system can make or break your documentation success

Contenders

We have a number of options commonly used, each with strengh and weaknesses.

  • DITA: The OASIS Open Darwin Information Typing Architecture. Extremly powerful, especially the concept of single source definition: You define an item once and just reference it. XML based, suitable for complex documentation, but with a super steep learning curve, effectively prohibit community contributions
  • jekyll: Markdown driven template engine, best known for driving GitHub Pages. With the Just-the-docs template it makes documentation creation a simple task in your repository's /doc directory. Running site generation and hosting is build into github, so no GitHub action or other CI/CD pipeline needed. Lacks good tooling for multi-version documentation
  • Maven sites: a good option when Java is your language. Tightly coupled to the build process it produces full reporting and JavaDoc. Can be a pain to setup
  • Read the docs: Great destination for OpenSource documentation or your corporate documentation if the build server can reach it. Uses the MKDocs rendering engine
  • and many, did I say many more

I found the tools quite impressive and somehow wanting at the same time. So taking a step back, it is worth to look at requirements


Read more

Posted by on 09 March 2022 | Comments (0) | categories: Software