In a recent chat a store manager shared how tedious the annual physical stock taking is, since the company doesn't provide handheld barcode scanners. Why not use your phone I replied. So after a short discussion it was agreed to build a MVP.

The geek corner of my brain went into overdrive: databases, authentication, protocols, libraries - the whole shebang. If you don't reign into that, you end up with the famous "swing design"

(shamelessly borrowed from here)

Luckily there's the agile way: add value incrementally. Our increment frequency was one improvement per 15min, tiny step by step ;-).

The result is POPIAH: Poor Operator's Personal Inventory Assessment Helper (not that one). The final (for now - pending making it pretty) result looks like this:

Step by step

Our epics (defined looking back) were:

• make scanning work
• make the screen useful
• allow using the data elsewhere

Time-based one-time passwords (TOTP) are a common security feature in Identity Providers (IdP). There are use cases beyond IdP, mine was "Understanding what it takes").

TOTP interaction

You have two phases: enrollment and use. During enrollment a secret is generated and (typically) presented as QR Code. A user points one of the many Authenticator apps to it and gets a numeric code that changes once a minute.

When you use it, you pick the current number and paste it into the provided field. The backend validates the correctness with some time leeway.

What it is not

Typically when enrolling you also get recovery codes, sometimes called scratch codes. They are NOT part of TOTP and implementation is site specific and not standardized. An implementer might choose to check your recovery codes when your TOTP fails or provide a separate interaction using those.

The initial confirmation, is actually the first instance of "use" and one could have a successful enrollment without it. This is depending on the implementation.

It isn't foolproof. An attacker could trick you into typing your TOTP code into a spoofed form or just hijack your session (cookie). That's why responsible web apps run a tight security with CSP and TLS (and once browser support is better Permission Policy)

Setting up a sample application

We need a landing page and its supporting files (css, js, png) served statically and 3 routes:

• request
• save
• verify

A common approach for web applications is to serve them behind a reverse proxy. My favorite there is nginx. It has a fairly understandable configuration, supports http/2 and is fully supported by Letsencrypt's certbot.

proxy_pass can be tricky

The configuration for reverse proxying is a combination of location and a proxy_pass statement, as well as some headers. In it simplest form the URL path and the proxy_path URL are the same, so you don't need a translation between direct access (e.g. local testing) and access through nginx. A configuration could look like this:

location /someurl {
        proxy_pass http://127.0.0.1:3000/someurl;
        proxy_http_version 1.1;
        proxy_cache_bypass $http_upgrade;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Port $server_port;
    }

The headers ensure your application knows the original request source and allow upgrade from http to websockets.

It gets interesting when location and proxy_pass don't match. Differences can range from different url path to regex and rewrites. Especially when inheriting (or googling) configurations, understanding can be a challenge.

Express to the rescue

A few lines of expressjs will echo back what ends up at the backend. Together with curl it provides you a practical learning and testing environment.

const express = require('express');
const app = express();
const port = 3010;

app.get('*', (req, res) => {
  res.setHeader('Content-Type', 'text/json');
  let result = JSON.stringify(
    {
      headers: req.headers,
      url: req.url,
      query: req.query,
      hostname: req.hostname
    },
    null,
    2
  );
  res.send(result);
  console.log(result);
});

app.listen(port, () => {
  console.log(`Echo app listening for GET on port ${port}`);
});

Enjoy, as usual YMMV!

Over the years I developed a certain style developing in Java (and other languages) that made my code more readable. This isn't about the formatting style guide (Just stick to Google) or making code spotless, but about style.

Objectives

Code needs to be testable, easy to adopt and understandable when someone else, like your future self, tries to read it. Starting with SOLID principles, the guidelines help to get there. They work for any style of development, while I strongly recommend looking at TDD, Dave agrees, so does Allen. In no particular order, let's dig in.

Our brave new world of containers and microservices runs on a combination of YAML and shell scripts.
Getting them to run in the desired sequence can be a challenge.

When ready isn't ready

All container environments have a "depends" clause, so the container runtime can determine the correct startup sequence for the zoo of containers comprising the application to be launched. Each container will usually signal when it is ready.

However ready can mean different things to different applications.

In the container world it should be: the service is available.
However it could be: service was successfully started, but might be busy with house keeping. In the later case the start scripts of the dependent services need to do their own waiting

curl to the rescue

Curl is the Swiss Army Knive of network tools. It probably takes a lifetime to understand all its possibilities. Make sure you have the current version, but at least 7.52.0 from 2016.

Whipping up a little shell script provides what we need:

#!/bin/bash
# Wait for a http service to be available
export TARGET=http://baseservice.local/
curl -v \
     --retry 10 \
     --retry-delay 20 \
     --retry-connrefused \
     $TARGET

if [ "$?" -ne 0 ]; then
  echo "Failed to reach $TARGET"
  exit 1
else
  echo "$TARGET has replied!"
fi

Let's break it down:

• export TARGET defines the URL you want to reach. Works for both http and https. Potentially works for other protocols supported by curl as well, but I haven't tested that (yet). You might swap it out for $1 to capture the first parameter called
• \ denominates line continuation. The whole command needs to be on one line which is hard to read. \ tells Linux to treat the following line as belonging to the current one. On Windows you need to replace \ with ^ and alter the syntax according to Powershell
• curl -v Verbose output, so the logs tell the full story. When you work with privat CA or self signed certificates (Don't do self signed, use your own CA), you need to add -k or --insecure to make this work
• --retry 10 try the connection 10 times
• --retry-delay 20 Wait 20 sec until the next try
• --retry-connrefused retry even on "connection refused" responses. By default curl retries only http responses not in the 2xx range. Connection refused is HTTP's way telling you: there's nothing taking your call, which curl, without --retry-connrefused takes as final, fatal error and terminates retries. When you have curl v7.71.0 (from 2020 or later), you can use --retry-all-errors instead. The documentation warns This option is the "sledgehammer" of retrying. Use it wisely
• if [ "$?" -ne 0 ]; then Commands usually return a numeric result. Anything other than 0 indicates a failure condition of some sort. When curl has exhausted the number of retries a value greater 0 will be returned. To capture the return value we us $?
• return 1 We indicate that our script failed. So it can be called from other scripts

We could swap out $TARGET for $1 which makes the script callable handing over the target URL e.g waitforme.sh http://someURL

Beyond containers

The script is also useful in "classic" situations, where you start a service in the background (which by definition is asynchonous). Your local environment uses anything like:

• ./startdb &
• systemctl start myservice
• domino[_container] start

You can use the curl retry for any automation. Adjust the wait time and the retry time to your needs.

As usual YMMV

When the HCL Labs is hiring engineers, we conduct technical interviews. I had and have the priviledge to interview, hire and work with incredible talented individuals in the process. Lets look behind the curtain of the process.

What we are looking for

The technical interview is conducted by the interviewie's potential future colleagues. We try to gauge skills, conduct and team fit. Fit includes: does it improve diversity. You got a tatoo, tell me how you picked the design? You like tailored suits, who's your tailor? Punk is your thing, what's your favorite band? - You get the drift.

Can the candidate explain technology to peer engineers and understands the fundamental concepts? We found a surprising number of developers who don't know how their platform actually works. Topics like "The Node event loop" or "What does Maven do".

Knowing your environment is essential. Have you kept uptodate with latest changes, even your current role holds you back on applying them.

Does the candidate know about our "Buzzwords" YAGNI, DRY, KISS, SOLID or pattern

What we ask

Questions come from all areas: general project conducts, APIs, Code (mostly JS, TS & Java), HTTP and databases. We are looking not only for "I have x years experience", but also general awareness.

Example: you might not have a chance to work with a graph database, but you know they exist, what they are supposed to solve. etc.

A little taste:

• How much Scrum does one entertain to be agile and what's the essence of agile
• How to interact and resolve ambiguity in tasks and asks
• On object oriented vs functional programming styles
• Types of NoSQL databases and their uses
• How do the joins (left, right, inner, outer) in an RDBMS work
• Use cases for SQL and NoSQL
• HTTP Verbs: POST vs PUT vs PATCH
• REST experience including a tool or another
• HTTP return codes. Everyone knows 200, some 201 and the difference to 200, and a few 418
• CORS and CSP
• Callbacks vs. Promises
• let vs var vs const
• function vs =>
• Loops vs streams
• Testing: TDD, UnitTests and Integration Tests
• SQL Injection and prevention
• Maven & NPM
• Git & GitHub
• What do you use NodeJS for

Coding challenges

Coding challenges in interviews are controversial and they are usefull.
We keep ours simple and explicitely allow questions, dialogue or googling for answers, with the warning label: you need to explain your solution, no matter what.

Being a little sneaky we keep the challenge requirements fuzzy to check if the candidate will ask for clarifications.

The exercises are light and not too far from practical use:

• talk to an API
• refactor code
• fix performance

Work with us

HCL in Manila is looking for engineers of all levels and career stages. Come join us, some current openings from Linkedin (links are temporary, grab them while they are hot):

• Senior Software Developer
• Lead Software Engineer
• UX Designer
• Documentation Lead (Technical Writer)

Test driven development gets interesting when actions might or might not complete somewhen in the future. Point in case: HTTP requests (fetch returns a promise)

vert.x and jUnit5

I frequently write little tests, that spin up an http sever before the test, run individual tests using a WebClient. All these operations run asynchronous, returning a vert.x Future (In JavaScript Land the equivalent would be a promise)

To make this pain free, vert.x provides a full integration into jUnit 5. Using the annotation @ExtendWith(VertxExtension.class) vert.x provides two injectable parameters: Vertx and VertxTestContext. Add the simple mental rule: All async operations need to interact with the VertxTestContext.

Java is evolving rapidly. Henri Tremblay is walking through the cool stuff. Henri is, besides other activities a contributor to EasyMock

From Java 7 (2011) to Java 19 (2022)

Unless stated otherwise Java 17 (2021) LTS will work for the examples. Henri used a trading app to walk through upgrading steps.

Impressive: just moving from Java 8 to Java 19 shows double digit performance improvements

Some of the tips:

• use the java.time.Clock object for anything date/time, Don't use Date. Inject the clock object.
• use Comparators
• use Files.lines to read by line
• use the Maven deamon to speed compilation
• Be careful with stream().findAny() - in most cases a Map performs way better
• Use """ text blocks (why o why did Java not pick the tripple backtick)
• Use Files.writeString if it is a String you write
• Use jlink and jpackage for distribution
• Use switch expressions which returns values (not convinced about that yet)
• Use Records for immutable data
• Use the new instanceof syntax
• Use the new RandomGenerator for better performance
• Sealed types allow tighter controls

A quite enjoyable session

TDD is favored by very succesfull engineers as a means of accellerating software development by preserving quality.

Objections are plenty. The talk by Burk Hufnagel tries to address those.

The TDD Paradox

TDD requires you to write more code, so you can be done sooner. It's like learning to drive a car. Your first mile takes, give or take, 40-50h to complete: Driving lessons, driving tests, get the license issued before you drive. You can walk a lot more in 50 hours, you get the drift...

Looking deeper at "development" we see, it has multiple components:

• understanding and clarifying the business requirements (business rather loosely defined)
• write code
• manual testing
• debugging

TDD helps with the first and third component. A well defined test communicates the desired outcome and serves as a living specification. Having the machine running tests reduces the need for manual testing and frees up time to work on functionality

Test early, test often

TDD proposes to write tests first, not as an afterthought. The strongest new argument Burk proposed was: nobody wants to write a test for code freshly debugged and proven working (at that time). There's no immediate value in it. A test, initially failing, written upfront is a reassurance to be on the right track when going green.

Using the "test first" approach, tests turn into a design process, increasing the confidence in not only doing things right, but also doing the right thing. In other words: Tests are Specifications

Best results are to be had when working in small increments and resist the urge to "code for things you think are coming next" instead of "(just) make the test pass".

It's not a cure-it-all

TDD might not be a good fit:

• no clear business requirements - you can't test when you don't know what you want
• exploratory coding - trying stuff out

A common objection is "we have an existing non-TDD code base, we can't switch to TDD". Burk suggest to use "Defect Driven Development". Write a test that test for defect free behavior and then fix the defect. Your test harness will grow while defects shrink

Test long and prosper

The usual subjects for testing in Java are JUnit and Mockito. Burk introduced another, very promising tool: Spock. Spock is written in Groovy and allows to specify a test's purpose more expressive and concise than Junit's @DisplayName property. Luckily it isn't an exclusive-or question, Spock interacts with JUnit and Mockito quite nicely.

In conclusion

Burk made a good case for TDD, backed by experience and tooling, if in doubt visit his arguments.

GraalVM is a promising polyglot runtime for Java, JavaScript, Python, Ruby and WASM.
It can produce native images. Unsurprisingly with great powers come ... lots of things to learn.

GraalVM Native Image Benefits

A native image has a number of benefits over an JVM based application, mainly owed to the extensive analysis done at build time.
Some of advantages mentioned:

• Runtime speed ~ 25% above the same code running on an JVM
• Ahead of Time compilation can lead to up to 9 times less memory consumption than running on an JVM
• Code executes faster with a smaller memory footprint than comparable Go applications ( but still is bigger/slower than C by a whisk)
• Warmup time got moved to build time, improving startup time

Building the image

Build has been greatly improved with tooling available:

• a Maven plugin (also available for Gradle), eliminating the need of extensive commandline foo.
• a GitHub action to install the GraalVM tooing in a pipeline
• reduced memory consumption, so most apps will fit into the GitHub build container limit of 7GB memory
• option for a non-optimized fast build, useful for testing correctness

Thoughts and conclusions

A big step in the right direction making native images more accessible. There's still room for improvements, like caching the build images and more samples. What Oracle solved nicely: If you are a GraalVM Enterprise customer, accessing GraalVM Enterprise edition is done using an access token, fully compatible to any build system. No messing around with downloads and manual installations required.