wissel.net

Usability - Productivity - Business - The web - Singapore & Twins

Calling a vert.x async method from a sync method


Made popular by NodeJS and ES6 asynchronous programming promises (pun intended) better throughput and resource usage by entertaining an Event Loop. In Java land vert.x implements exactly this approach and has proven its mettle, being the foundation of Quarkus

Your legacy app doesn't magically convert

When you start a new vert.x project using the App Generator, everything is asynchronous from the beginning. Snippets of synchronous code (a.k.a blocking code), you might need to maintain, can be wrapped into executeBlocking and handled in their own thread.

However when you are about to convert a synchronous application, e.g. a servlet to asynchronous and you can't finish in a sprint/session, things become interesting. The doGet method is synchronous by nature. There are a few steps that need to be accomplished:

  • Have vert.x running in its own thread. You can't start it on the main thread with its blocking operations
  • Have a method that returns a vert.x Future
  • Convert that into a Java CompletableFuture

Let's have a look at the moving parts:


Read more

Posted by on 03 August 2022 | Comments (0) | categories: Java vert.x

Case insensitive deserialization


Growing up in Windows with BASIC you learn case doesn't matter, so Color is the same as COLOR or cOLOR when it comes to variable names. Same applies to @Formula or item names in Notes documents.

On the other side, Linux, Java, JavaScript and JSON are very much case sensitive.

This poses a challenge when deserializing (handcrafted) JSON files.

The Task at hand

Deserialization of JSON into a Java class instance can be done using jackson. This is also what the JsonObject in vert.x uses when you call json.mapTo(SomeClass). Not using vert.x? You can use the ObjectMapper. Let's look at a sample Java class

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonInclude;
import io.vertx.core.json.JsonObject;

@JsonInclude(JsonInclude.Include.NON_NULL)
@JsonIgnoreProperties(ignoreUnknown = true)
public class JsonTest {

  public static fromJson(final JsonObject source) {
    return source.mapTo(JsonTest.class);
  }

  private String color;
  private String shape;
  private int answer;
  private boolean pretty;

  /* GETTERS and SETTERS omitted for brevity
     Let your IDE add them for you */
}

Now you want to deserialize a good JSON, which works as expected:

{
  "color": "Red",
  "shape": "round",
  "answer": 11,
  "pretty": true
}

but the very moment your JSON isn't following proper capitalization, like human provided JSON,

{
  "Color": "Red",
  "Shape": "square",
  "Answer": 42,
  "pretty": true,
  "ignore": "this"
}

deserialization will fail. We need to fix that.


Read more

Posted by on 08 June 2022 | Comments (1) | categories: Java vert.x

Dance the OAuth with me


OAuth and its cousin OIDC are the ubiquitous methods to gain identity and authorization information. Since it is a ménage à trois between a user, an Identity provider (IdP) and an application, refered to as "Service provider", it is hard to trouble shoot.

A play in five acts

In the recent Project KEEP we build an IdP into the API, so you have the choice of just using Domino or using an external IdP.

To ensure it works as expected several dependent HTTPS calls were needed. Each call would harvest some information into environment variables for the following step

Act 0 - initial setup

Store several variables into the environment:

  • UserName: the user you will simulate to approve
  • Password: their password
  • HOST: The starting URL for the first call
  • state: a random string, need to stay the same through the sequence
  • client_id: The application configured as service provider
  • client_secret: The service provider "password"
  • scope: the scope (or a subset) you have configured for the service provider
  • redirect_uri: one of the redirection URIs you have configured for the service provider

An OAuth flow contains basic authentication calls, so you need to ensure proper TLS connections.

OAuth Dance


Read more

Posted by on 06 June 2022 | Comments (1) | categories: WebDevelopment

Yes No Maybe Boolean deserialization with Jackson


The Robustness principle demands: be lenient in what you accept and strict in what you emit. I was facing this challenge when deserializing boolean values.

What is true

Glancing at data, we can spot, mostly easily what looks trueish:

  • true
  • "True"
  • "Yes"
  • 1
  • "Si"
  • "Ja"
  • "Active"
  • "isActive"
  • "enabled"
  • "on"

The last three options aren't as clear cut, they depend on your use case. Using a simple class, lets try to deserialize from JSON to an instance of a Java class instance using Jackson.

Java doesn't have native support for JSON, so we need to rely on libraries like Jackson, Google GSON (or any other listed on the JSON page). I choose Jackson, since it is the library underpinning the JsonObject of the Eclipse Vert.x Framework I'm fond of. Over at Baeldung you will find more generic Jackson tutorials.

Let's look at a simple Java class (Yes, Java14 will make it less verbose), that sports fromJson() and toJson() as well as convenient overwrite of equals() and toString()

package com.notessensei.blogsamples;

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonInclude;
import io.vertx.core.json.JsonObject;

@JsonInclude(JsonInclude.Include.NON_NULL)
@JsonIgnoreProperties(ignoreUnknown = true)
public class Component {

  public static Component fromJson(final JsonObject source) {
    return source.mapTo(Component.class);
  }

  private String name;
  private boolean active = false;

  public Component() {
    // Default empty constructor required
  }

  public String getName() {
    return name;
  }

  public void setName(String name) {
    this.name = name;
  }

  public boolean getActive() {
    return active;
  }

  public void setActive(boolean isActive) {
    this.active = isActive;
  }

  public JsonObject toJson() {
    return JsonObject.mapFrom(this);
  }

  @Override
  public boolean equals(Object obj) {
    if (obj instanceof Component) {
      return this.toString().equals(obj.toString());
    }
    return super.equals(obj);
  }

  @Override
  public String toString() {
    return this.toJson().encode();
  }

}

Trying to instantiate a class instance with the following JSON will work:

{
  "name": "Heater",
  "active": false
}
{
  "name": "Aircon"
}
{
  "name": "Fridge",
  "active": true,
  "PowerConsumption": {
    "unit": "kw",
    "measure": 7
  }
}

However it will fail with those:

{
  "name": "System1",
  "active": "on"
}
{
  "name": "System2",
  "active": "yes"
}

You get the charming error Cannot deserialize value of type boolean from String "yes": only "true"/"True"/"TRUE" or "false"/"False"/"FALSE" recognized`. Interestingly numbers work.

On a side note: Jackson uses the presence of getters/setters to decide (de)serialization and needs getActive and setActive or isActive. When you name your variable isActive Eclipse would generate setActive and isActive instead of getIsActive / isIsActive and setIsActive. So simply avoid the is... prefix for internal variables.


Read more

Posted by on 07 May 2022 | Comments (0) | categories: Java

The Quest for a software documentation system


Software documentation is a thankless business and never complete. Picking the right system can make or break your documentation success

Contenders

We have a number of options commonly used, each with strengh and weaknesses.

  • DITA: The OASIS Open Darwin Information Typing Architecture. Extremly powerful, especially the concept of single source definition: You define an item once and just reference it. XML based, suitable for complex documentation, but with a super steep learning curve, effectively prohibit community contributions
  • jekyll: Markdown driven template engine, best known for driving GitHub Pages. With the Just-the-docs template it makes documentation creation a simple task in your repository's /doc directory. Running site generation and hosting is build into github, so no GitHub action or other CI/CD pipeline needed. Lacks good tooling for multi-version documentation
  • Maven sites: a good option when Java is your language. Tightly coupled to the build process it produces full reporting and JavaDoc. Can be a pain to setup
  • Read the docs: Great destination for OpenSource documentation or your corporate documentation if the build server can reach it. Uses the MKDocs rendering engine
  • and many, did I say many more

I found the tools quite impressive and somehow wanting at the same time. So taking a step back, it is worth to look at requirements


Read more

Posted by on 09 March 2022 | Comments (0) | categories: Software

Maven JNA macOS and LD_LIBRARY_PATH


When running Java applications on a *nix style of operating system that need to load native libraries, you will need to set the LD_LIBRARY_PATH environment variable (or something similar). That's not an issue on Linux.

macOS: I won't let you, it's for your own good

On macOS the System Integrity Protection (SIP) prevents these variables to be set in your shell (bash, zsh). It works inside Eclipse, when you define environment parameters, but not in any shell script. Unfortunately Maven's command line mvn is a shell script.

The Notes challenge

Since the Notes client is a cross-platform product, the library locations aren't where a macOS program would look for:

  • The application directory. That's where the Java runtime is at home, not the notes executable
  • In a library location, here looking for notes instead of libnotes.dylib
  • /Users/[YOURNAME]/Library/Frameworks/notes.framework/
  • /Library/Frameworks/notes.framework/
  • /System/Library/Frameworks/notes.framework/

You could try to symlink the first library: ln -s /Applications/HCL\ Notes.app/Contents/MacOS/libnotes.dylib ~/Library/Frameworks/notes.framework/notes (after creating the rewuired directories) to run into the next challenge.


Read more

Posted by on 12 January 2022 | Comments (0) | categories: Domino Java

Async Java with vert.x


I wrote about more modern Java syntax and streams before.
There is more to it. Non Blocking I/O and Event Loops allow for
better performance. It's not a magic bullet, some readjustment is required

Adjusting methods, exceptions and return values

Initially it might look daunting, but the adjustments are not too big. Let's look at some examples. A classic Java method looks like this:

String someResult throws DidnWorkException {
    // Working code goes here
    if (someCondition) {
        throw new DidnWorkException();
    }
    return "It worked";
}

Its asynchronous counter-part looks like this:

Future<String> someResult() {
    return Future.future(promise -> {
        // Working code goes here
        if (someCondition) {
            promise.fail("It didn't work"); // Could use a Throwable too
        } else {
            promise.complete("It worked");
        }
    });
}

Read more

Posted by on 06 January 2022 | Comments (0) | categories: Domino Singapore

Deploying your frontend as webJar


In an API driven world back-end and front-end are clearly separated and might live on different servers alltogether. However for smaller applications serving static files happens from the same place as your backend lives

So many choices

The web server that proxies your application server could have a rule for static files, your firewall could so that, you use a static directory on your application server or pack, that's the story here, your front-end into a jar. I'm not discussing the merits of the different approaches here, that's a story for another time, but describe the workflow and tools for the JAR approach.

vert.x static routes

In Vertx a static route can be declared with a few lines of code:

Router router = Router.router(vertx);
router.route("/ui/*")
      .handler(StaticHandler.create("uitarget"));

Vertx will then look for the folder uitarget in its current working directory or on the classpath. So you will need to put your jar on the classpath

The swagger-ui example

There are lots of prepackaged UI jars available and easy to integrate into vert.x. For example the Swagger UI. Define a dependency in your pom.xml and a one liner to access the code:

<dependency>
    <groupId>org.webjars</groupId>
    <artifactId>swagger-ui</artifactId>
    <version>4.1.3</version>
</dependency>
Router router = Router.router(vertx);
router.route("/assets/lib/*").handler(StaticHandler.create("META-INF/resources/webjars"));

Packing your own front-end

Most modern build front-ends owe their executable form to an npm build command. If you are not sure check the documentation for React, Angular, Lightning, Vue, Ionic or whatever framework you fancy.

There are two plugins for maven that can process front-end work:

  • The Frontend Maven Plugin: Specialized module that handles download of NodeJS and running your NodeJS based build tools. Great when you don't have NodeJS installed anyway
  • The Exec Maven Plugin: Generic plugin to run stuff. Doesn't download NodeJS for you. More work to setup (that's what I picked)

The steps you will need to perform, actually not you, but your mvn package run:

  • run npm install
  • run npm build
  • move files into the target directory structure
  • build the Jar

All of this can be wrapped into your pom.xml. I usually add the front-end as a module to the whole project, so a build is always complete


Read more

Posted by on 27 December 2021 | Comments (0) | categories: Java JavaScript vert.x

Refresh local git repositories


I keep all my software that is under version control below a few directories only. E.g. OpenSource projects I cloned to learn from them live below ~/OpenSource. Keeping up with updates requires to pull them all.

Pulling the main branch

My little helper does:

  • change into each first level sub directory
  • check if it is under version control
  • capture the current branch
  • switch to main or master branch, depending on which one is there
  • capture the name of the tracked remote
  • fetch all remotes
  • pull the tracked remote
  • switch back to the branch it was in

The script does not check if the current branch is dirty (preventing checkout) or pushing back changes. Enjoy

#!/bin/bash
# Pull all repos below the current working directory

do_the_sync() {
  for f in *; do
      if [ -d $f -a ! -h $f ]; then
         cd -- "$f";
         if [ -d ".git" ]; then
            curBranch=$(git branch --show-current)
            mainBranch=nn
            echo "Working on $f";
            if [ "`git branch --list main`" ]; then
              mainBranch=main
            else
              mainBranch=master
            fi
            remoteBranch=$(git rev-parse --abbrev-ref ${mainBranch}@{upstream})
            IFS='/' read -r remoteSrv string <<< "$remoteBranch"
            echo "working on $mainBranch tracking $remoteSrv"
            git fetch --all
            git pull $remoteSrv
            git checkout $curBranch
         fi
         cd ..
      fi;
  done;
};

do_the_sync
echo "DONE!"

As usual YMMV


Posted by on 23 December 2021 | Comments (0) | categories: GitHub Software

Spotless code with a git hook


When developing software in a team, a source of constant annoyment is code formatting. Each IDE has slightly different ideas about it, not even getting into the tabs vs. spaces debate. Especially annoying in Java land is the import sort order

Automation to the rescue

I switch between editors (if you need to know: Eclipse, Visual Studio Code, OxygenXML, IntelliJ, Sublime, Geany, nano or vi (ESC :!wq)) frequently, so an editor specific solution isn't an option.

Spotless to the rescue. It's a neat project using Maven or Gradle to format pretty (pun inteded) much all code types I use. The documentation states:

Spotless can format <antlr | c | c# | c++ | css | flow | graphql | groovy | html | java | javascript | json | jsx | kotlin | less | license headers | markdown | objective-c | protobuf | python | scala | scss | sql | typeScript | vue | yaml | anything> using <gradle | maven | anything>.

Setup

I opted for the eclipse defined Java formatting, using almost the Google formatting rules with the notable exception not merging line breaks back.

There are 3 steps involved for the Maven setup:

  • Obtaining the formatting files, outlined here. Just make sure you are happy with the format first
  • Add the maven plugin (see below)
  • Add a git hook (see below)

pom.xml

This is what I added to my pom.xml. By default spotless would run check only, so I added apply to enforce the formatting

<properties>
   <spotless.version>2.4.1</spotless.version>
</properties>

<build>
    <plugins>
        <plugin>
            <groupId>com.diffplug.spotless</groupId>
            <artifactId>spotless-maven-plugin</artifactId>
            <version>${spotless.version}</version>
            <executions>
               <execution>
                 <goals>
                   <goal>apply</goal>
                 </goals>
               </execution>
            </executions>
            <configuration>
                <formats>
                    <format>
                        <!-- Markdown, JSON and gitignore -->
                        <includes>
                            <include>*.md</include>
                            <include>*.json</include>
                            <include>.gitignore</include>
                        </includes>
                        <trimTrailingWhitespace />
                        <endWithNewline />
                        <indent>
                            <spaces>true</spaces>
                            <spacesPerTab>2</spacesPerTab>
                        </indent>
                    </format>
                </formats>
                <!-- ECLIPSE Java format -->
                <java>
                    <toggleOffOn />
                    <importOrder>
                        <file>${maven.multiModuleProjectDirectory}/spotless.importorder</file>
                    </importOrder>
                    <removeUnusedImports />
                    <eclipse>
                        <file>${maven.multiModuleProjectDirectory}/eclipse-java-keep-style.xml</file>
                    </eclipse>
                </java>
            </configuration>
        </plugin>
    </plugins>
</build>

A few remarks:

  • I run apply rather than check
  • the directory variable ${maven.multiModuleProjectDirectory} is needed, so sub projects work
  • you want to extend the configuration to include JS/TS eventually

.git/hooks/pre-commit

Create or edit your [projectroot]/.git/hooks/pre-commit file:

#!/bin/bash
# Run formatting on pre-commit
files=`git status --porcelain | cut -c 4-`
fulllist=''
for f in $files; do
    fulllist+=(.*)$(basename $f)$'\n'
done;
list=`echo "${fulllist}" | paste -s -d, /dev/stdin`
echo Working on $list
# Activate Java 11
export JAVA_HOME=`/usr/libexec/java_home -v 11.0`
/usr/local/bin/mvn spotless:apply -Dspotless.check.skip=false -DspotlessFiles=$list
  • You might not need the line with Java
  • swap apply for check when you just want to check

As usual YMMV


Posted by on 10 December 2021 | Comments (0) | categories: GitHub Java Software