wissel.net

Usability - Productivity - Business - The web - Singapore & Twins

Domino Docker and Debugging


Given that Domino once was build to run on 486 capacity of servers, Docker and Domino are posed to be a match made in heaven (eventually). Jesse shared shared his Weekend Domino-Apps-in-Docker Experimentation, Paul shared his learning points and Daniel provided the invaluable Domino on Docker build scripts. So it's time to contribute my share. The topic is slightly more exotic

Debug a Java application running on Domino in a Docker container

Before we can get cooking, we need to know what ingredients we need:

Our objective: Create a Domino image that loads the Java application from its host file system, so we do not need to rebuild the container on Java changed. An instance of this image shall allow to connect a debugger to that Java application

Foundation: the Domino image

First we have to build a Domino Docker image, configure a server using a docker volume. This has been mapped out in the domino-docker project and its slighly hidden documentation. Just a quick recap:

  • Build the image using ./build domino
  • Create a volume using docker volume create keep_data
  • Run the instance once to setup the domino
docker run -it -e "ServerName=Server1" \
    -e "OrganizationName=MyOrg" \
    -e "AdminFirstName=Doctor" \
    -e "AdminLastName=Notes" \
    -e "AdminPassword=passw0rd" \
    -h myserver.domino.local \
    -p 80:80 \
    -p 1352:1352 \
    -v keep_data:/local/notesdata \
    --stop-timeout=60 \
    --name server1 \
    hclcom/domino:11.0.1

We shut down the instance once you have confirmed it works. We don't need it thereafter, we only need the volume and image. Of course there's no harm keeping it around


Read more

Posted by on 30 June 2020 | Comments (1) | categories: Docker Domino HCL Notes

Watching the EventBus


I'm quite fond of Event-driven architecture, so to no surprise, I like vert.x's EventBus and its ability to enable polyglot programming. So it is time to have a closer look

Dem Volk aufs Maul geschaut

(That's a word play on Martin Luther loosly translated as "Watch them how they talk")

I wanted to know, what exactly is happening "on the wire", without disrupting the regular flow. Turns out, there is an easy way to do this. The vert.x EventBus provides the methods addOutboundInterceptor and addInboundInterceptor that provide you with access to a Handler with a DeliveryContext.

From there you can get to the Message or directly the message's body. So I took it for a spin in conjunction with a Websocket. This allows me to watch as the messages flow through:

final HttpServer server = this.vertx.createHttpServer();
server.websocketHandler(this::handlerWebsockets);


Read more

Posted by on 28 April 2020 | Comments (0) | categories: Java vert.x

SimpleXMLDoc revisited


It is 2020, JSON is supposed to have won, with a challenger in sight. XML with its fine distinction between Elements, Attributes and clear ownership demarked by name spaces, was supposed to be gone. But OData made it necessary to look again, as did CalDAV

Out into the OutputStream

The initial version was introduced in the context of XAgents which mandated an OutputStream. I find that adequate and useful, so I kept that. If you just want a String, a ByteArrayOutputStream will do quite nicely

Fluent methods

The big change to the revamped version is the addition of a fluent API. Each method call returns the object instance itself, so you can chain your document creation to look modern (and type less)

Namespace and attributes

Originally I though "simple" would be sufficient to create Elements only. But as time goes by one starts to appreciate name spaces and Attributes, so I added support for these too. To keep things simple: once we specify the namespace at the beginning of the document, we can simply refer to it by its alias name.

A sample:

    final ByteArrayOutputStream out = new ByteArrayOutputStream();
    final SimpleXMLDoc doc = new SimpleXMLDoc(out);
    doc.addNamespace("X", "https://xmen.org")
    .addNamespace("", "https://whyOhWhy.com/xml")
    .setXmlStyleSheet("somestle.xslt")
    .openElement("Endpoints")
    .openElement(doc.element("X:Endpoint")
          .addAttribute("name", "A Name")
          .addAttribute("url", "http://anywhere/")
          .addAttribute("meta", "meta not metta"))
     .closeElement(1)
     .addSimpleElement("description", "Something useful")
     .closeDocument();
    System.out.println(out.toString());

Key methods

  • addNamespace: adds one name space and establishes the alias. To keep it simple, namespaces are defined only at the beginning fo the document
  • setXmlStyleSheet: Same here, needs to be defined at the beginning - after all this class streams the result and stylesheets only start at the beginning
  • OpenElement starts a new XML Element. When provided with a string, it is an attribute free element, that can include the namespace abbreviation. When using a doc.element, we can add attributes
  • addSimpleElement: add an element, its String content and close it
  • closeElement: write out a number of closing tags. It deliberately uses number of tags and not tag names, so you don't need to track the names you have opened. Ensures that XML stays valid
  • closeDocument: closes all remaining Elements in the correct sequence and closes the document. Can be called once only

Check the full source code for details

As usual YMMV


Posted by on 13 April 2020 | Comments (0) | categories: Java XML

vert.x and CORS


One of the security mechanism for AJAX calls is CORS (Cross-Origin Resource sharing), where a server advice a browser if it can request resources from it, coming from a different domain.

It is then up to the browser to heed that advice. To complicate matters: when the browser wants to POST data (or other similar operations), it will go through a preflight request adding to site latency.

I have to admit, I never fully understood the rationale, since only browsers adhere to CORS, any webserver, Postman or CURL ignore CORS happily.

None, One or All, but not Some

There's another trouble with CORS: The specification only allows for no-access, all-access (using * as value for Access-Control-Allow-Origin, with restrictions) or one specific domain, but not a list of domains.

Mozilla writes

Limiting the possible Access-Control-Allow-Origin values to a set of allowed origins requires code on the server side to check the value of the Origin request header, compare that to a list of allowed origins, and then if the Origin value is in the list, to set the Access-Control-Allow-Origin value to the same value as the Origin value.


Read more

Posted by on 07 April 2020 | Comments (0) | categories: Salesforce Singapore

My Maven starter template


Maven is to Java what npm is to JavaScript. It can be a harsh mistress or your best companion. It depends

Beyond dependencies and builds

Maven removes the need to download and manages your dependencies. Unfortunately it doesn't come with mvn install <packagename> like npm (or I haven't learned that yet), so keeping that pom.xml current is a little PITA. However once we make peace with it, the power of plugins makes development in auto-pilot a breeze. Some of the things you can do:

  • Generate a project site
  • Generate various reports: code quality, code coverage
  • Run unit tests

Check out the complete list to get an idea. I'm specifically fond of the site generation capability. It allows us to keep your documentation in the same repository as the project, so we have one place less to worry about.

We simply add /src/site/ to our project and content can be created in multiple formats. My favorite one is Markdown. Besides my handcrafted pages, I generate reports:

  • Issue management
  • Licenses
  • Plugins
  • Source code location
  • Team
  • JavaDoc
  • PMD and CPD
  • Surefire (Test results) and JaCoCo (Test coverage)

All this involved a bit of boilerplate in the pom.xml so I keep a template around,


Read more

Posted by on 06 April 2020 | Comments (0) | categories: Java WebDevelopment XML

eMail etiquette - the 60ties are calling


With WFH being en vogue these days, not only video conferencing and chat, but also eMail.
Dating back to the 1960ties, we had 6 decades to develop etiquette, which seems tobe lost to current users, so here we go again

Addressing

eMail has To, CopyTo (also called CC for Carbon Copy) and BCC (Blind Carbon Copy) as a means of addressing people. They serve distinct purposes:

  • TO: This is the person (or people) we want to act on our message, do something, reply etc. A good email has only few names, ideally one. If we have an ongoing eMail thread that involves multiple actors, we most likely use the wrong channel and are better of using collaborative software like HCL Connections, HCL Sametime, Slack, Teams or Chatter
  • CC: People, we think, who should be keept in the loop. We don't expect any action or reaction of them. A lot of eMail veterans automatically route those messages to a low priority place
    -- BCC: all receipients here get the message and the rest of the addressies won't know. I used to call it the "mobbing copy". BCC is especially fun when someone there hits "reply all" and reveals the readership. There are few legitimate uses for this. One is distribution lists (see below), the other archive/record keeping. Our external readers don't need to know that your compliance archive has the eMail address compliance@acme.com If we really want someone outside the visible thread to take note - forward the message

Subject line

It is like a tweet about our content. The subject needs to justify why it is worth the time and attention to open it. So "Status", "Report" or "Important" don't cut it. Common practise we can see are qualifiers, e.g Opportunity codes or project IDs at the beginning. Something like [T3453] - makes it easier to filter.

The biggest competitor to inbox attention by subject is the sender identity. We probably open a message one or two reporting managers up even with bad subject lines.

Content

Let's keep it crisp and short, best below 5 sentences.

We state:

  • the information we want to provide
  • the exact ask what action we expect, from whom and when
  • name the person "team please look into... " doesn't cut it and is an indicator of a broken process

If there is a lot of information, it might better live in a Wiki, a project place or even a file share. We then provide the news cast overview and a link - Would you like to know more?

There are some interesting cultural differences. In Anglosaxon or Eastern culture we would politely address the person and add a whiff of smalltalk, something along the lines "hope that finds you well". Germans, Dutch and other Nordics consider this a waste of space and time and consider it as the ultimate courtesy to cut through the chase and get to the point.

When we address close co-workers, who value efficiency, it even is OK to skip the greeting. We need to dread carefully here, it needs to be clarified otherwise it is seen ultra rude.

Replies

Do we reply to the sender or all of the addressies together. It seems to be the default for many "replyToAll". This is especially hillarious when a distribution list sneaked on the addressies. The rationale here is: the sender wanted to keep all these in the loop, so I won't break it. For a small group, I hit replyAll, for larger ones only reply.

I would wish the eMail software would warn when you blast a reply. The guardian agrees: don't replyAll.

A special mention: cherry-picking replies. We hit reply all and remove the mailing lists - good. We just remove the project manager we compete with - bad. So we need to be careful of the ramifications. Other receipients might wonder: why are Jane and Joe no longer in this conversation?

Distribution lists

They firmly belong in BCC - avoids ReplyAll armageddon. When we use private distribution list, we need to make sure, they resolve before sending otherwise people can't reply. However - most likely - that group of people would be better served with a shared channel. A good strategy: we put it in BCC, write a two sencent summary and provide a link to the full info. Co-workers who are not into eMail will find in in their [insert the collaborative tool you use]

As usual YMMV


Posted by on 27 March 2020 | Comments (1) | categories: GTD Intercultural

Running Java applications with Notes on macOS


My main work computer is a MacBook running macOS. Thank to Logitech, they keyboard is just fine. As you know macOS neither features Domino Designer or Domino Admin. For recent development I wanted to make sure that my applications can run on the client (I've done that before).

Java, Java on the wall, what's the right path of them all?

(Sorry Schneewitchen)

In Notes/Domino R9.0.1FP8 - (client FP10) the Java runtime was updated to Java8 and in R11 changed to AdoptOpenJDK.

On macOS that led to a particular situation. The jvm packaged with the HCL Notes.app can be found in the path
HCL Notes.app/jre/Contents/Home with bin, lib and lib/ext as we can expect from a JVM. Suspiciously absent are Notes.jar, websvc.jar and njempcl.jar. They can be located at HCL Notes.app/Contents/MacOS.jvm/lib/ext.

While this isn't an issue for the Notes client, it is an obstacle when you try to run an external jar file. java -jar somejar.jar ignores any classpath setting outside the JVM and only loads resources from the default JBM path (lib/ext).

I suspect the separation was neccesary due to the AdoptOpenJDK distribution rules.

To solve this, we can use a start script that creates a symbolic link in the right place. Takeing the usual suspects like DYLD_LIBRARY_PATH and LD_LIBRARY_PATH into account we end with a script like this:

#!/bin/bash
# MacOS Keep Starter file
# Keep locations - update as needed - leave the TLS stuff empty if you don't have it
export KEEPJAR=$HOME/keep/projectkeep.jar
export LOG_DIR=$HOME/keep/logs
export TLSFile=$HOME/keep/private/demoserver.projectkeep.io.pfx
export TLSPassword=supersecret

# Don't change anything below unless you are sure what you are doing
# Java files places unfortunately troublesome, so we link some
cd /Applications/HCL\ Notes.app/jre/Contents/Home/lib/ext
export SRCDIR="../../../../../Contents/MacOS/jvm/lib/ext"
if [ ! -f njempcl.jar ]; then
	ln -s $SRCDIR/njempcl.jar .
    echo "Linked njempcl.jar"
fi
if [ ! -f Notes.jar ]; then
	ln -s $SRCDIR/Notes.jar .
    echo "Linked Notes.jar"
fi
if [ ! -f websvc.jar ]; then
	ln -s $SRCDIR/websvc.jar .
    echo "Linked websvc.jar"
fi

# Local Keep Server
export DEBUG=true
export PATH=/Applications/HCL\ Notes.app/Contents/MacOS:$PATH
export JAVA_HOME=/Applications/HCL\ Notes.app/jre/Contents/Home
export GodMode=true
export DYLD_LIBRARY_PATH=/Applications/HCL\ Notes.app/Contents/MacOS
export LD_LIBRARY_PATH=/Applications/HCL\ Notes.app/Contents/MacOS
echo $LD_LIBRARY_PATH ..
cd $HOME/Library/Application\ Support/HCL\ Notes\ Data
/Applications/HCL\ Notes.app/jre/Contents/Home/bin/java -jar $KEEPJAR
cd ~
echo Done!

This script presumes thay we have admin permissions. We could contemplate to check for existing symbolic links and remove the ones we did set. I decided, that smells too much like YAGNI.

As usual: YMMV


Posted by on 17 March 2020 | Comments (0) | categories: HCL Notes Java macOS

OData - the fine print


For an upcoming event I had to dig a little deeper into OData and learned a few things along the lines.

Same but different

OData's current version is 4.0 and that's the only version I will focus on in this article.

OData is the love child of an effort that joined Microsoft and SAP at the hip to create an universal internet data access standard, that on first sight has a lot of NIH issues to sort out. But I'm sure on a second (or third or forth) look it will all make sense.

Key components of OData

  • a service document in JSON format. It describes what data endpoints are available and where. Depending on the requested scope (more on that later) that JSON contains different levels of details
  • a meta data document. That document is referenced in the service document. The format is XML. It describes the data format found in each record, relations to other record and actions that can be taken. Looks like XML Schema, but not quite. Record is used loosely here, data can be hierarchical and nested
  • a query language squeezed into an URL, so data can be fetched using HTTP GET without using a body submitted. Query allows data selections, filtering, output size and sorting to be specified
  • Query results in JSON format enhanced with OData specific Metadata

What I liked a lot: OData.org offers a rich set of documentation and tutorials, including a Postman collection to learn

Happy soup

So dealing with OData you can enjoy a mix of technologies:

  • JSON for service document and payloads
  • XML for meta data
  • a dot notation (like dotNet or Java) for property values like data types (go learn about it). E.g. Edm.String defines a String value or Org.OData.Core.V1.ResourcePath the path to data
  • another query language
  • individual items by default get addressed in an uncommon format: serviceurl/People('johndoe'). Luckily this is optional and you can use the more common format for a route: serviceurl/People/johndoe. This seems to be born out of squezing the square pin URL into the round hole of database schemas and flexible primary keys

On the upside: Your Salesforce and Excel users will love it. Import and CRUD operations are well supported.

Initially I was looking at Apache Olingo to implement my needs. Unfortunately the library is too close to the Java servlet model and static defined data models (like your one application, not your data platform), so I disected my needs the hard way.

Lessons learned along the way

  • A OData client (think Excel, Tableau, Salesforce) will initially connect to the service document url. It sends a GET request. To specify what level of detail is required the ACCEPT header is used. So when testing your provider implementation you need to take that into account
  • Excel sends ACCEPT application/json;odata.metadata=minimal
  • Salesforce sends ACCEPT application/json;odata.metadata=full
  • OpenAPI can specify the top level application/json or the full qualified content type in the response. If you fully qualify it, you might end up with a 406 and wonder what happens
  • The meta data url gets specified in your service document and is usually the service document url extended by /$metadata
  • I ran into a potential Salesforce bug, where the meta data URL was ignored and the service document URL extended by $metadata. Note the missing /. Caused a bit of grief since Excel does /$metadata
  • Everything is case sensitive, which makes sense, but is fun in Domino that treats item names case insensitive
  • I couldn't get Salesforce to play nice until I added a field ExternalId to my form. Might be my lack of understanding how to change the primary key Salesforce will play nice with or something missing in my Metadata

Same, same, but different

Swap out the GET for a POST and the XML for something not-quite-JSON and you get GraphQL that tries to address a similar set of functionalities. We really love wheels reinvented

More lessons to be learned, stay tuned!


Posted by on 04 March 2020 | Comments (0) | categories: OpenSource WebDevelopment

Domino Administration Back to Basics (Part 2) - Networking


In Part 1 we learned about the marvelous world of Notes Names, X400 and the perils of messing with certificates. One big difference to X509 is the (almost) absence of certificate Command Line tools that can be so much fun.

Domino Networking - protocols as you like it

Domino predates the rise of TCP/IP and the internet. To no surprise it has its own idea about networking. Starting with protocol support:

  • Netbios using NDIS (doesn't route) and
  • IPX/SPX A protocol from days long past, when red boxes weren't Redhat but Novell
  • X.PC DialUp - Yes. A modem or something that takes modem commands and will establish a serial connection, no longer ships with Notes
  • A few more obscure protocols: Vines, SPXII
  • last not least TCP/IP

Having this zoo of protocols, Notes needs its own version of name resolution. That version is called Notes Named Network

One step back: What makes a Notes Domain?

A Notes Domain consists out of one or more server that use a Domino directory (a.k.a Public Name & Addressbook a.k.a names.nsf) with the same replica id (a story for another time) as the other member servers and have the same Domain name in their server document (that's where most of the server's setting are stored).

A popular point of confusion: Notes Names (from Part1) and Notes Domains: It is quite common to name your Domain after your orgID, but not mandatory. SO you could have HeavyRock/Acme@Acme or Sandstone/Acme@ToonsInc or Machine/Blowup@Acme The first and the last would be in the same Domain, while the first and second share the Org certifier. Anything goes, but to keep it simple, keep OrgId and Domain the same - unless you have 5 good reasons not to.

Another one: NEVER name your Notes Domain so it could be mistaken for an internet Domain. So no . in the name. Spaces interestingly are OK!


Read more

Posted by on 05 February 2020 | Comments (2) | categories: Domino Networking

Domino Administration - Back to Basics (Part 1) - Certificates


Domino is different, a lot of its concepts predate the internet and quite often inspired the standards. This is not an step-by-step instruction, but an introduction into concepts. The "step by step" approach is another story for another time.

In the beginning was the Certificate

Notes and Domino run using ID files. This are not merly files that can arbitrarily reconstructed, but cryptographically created Public Private key pairs. To avoid naming collisions the key names are hierarchical (Since R2), so anyone can call their server Server1 without confusion (sort of). This hierarchy is achieved using X400 naming conventions, an early competitor to DNS naming. A X400 name can consist out of multiple parts, these are the ones Domino is using:

X500 Name

So the minimum is a common name and an Org. The starting point of each Domino journey is the creation of the OrgID. All other parts depend on it. Note: there isn't a country ID, even if country mentioned after the Org. When you create your OrgID (while setting up the very first server), you can specify the country and it becomes part of the OrgID.

In practise however I haven't seen many OrgIDs that would carry a country code in them. So you can skip that part.

Signing Ids

Using the OrgID as signing certificat, one can go and create server and user certificates. E.g using the OrgID /O=Acme one can create /CN=Coyotee/O=Acme. For convenience the qualifiers are usually omitted, their meaning results from the position (and the fact that country is 2 letter if present and Orgs have 3 or more). So instead of /CN=Coyotee/O=Acme one can write Coyotee/Acme.

In practise however more enlightened organisation use their OrgCert as cautiously as root certificates in the X509 world and only register/sign Organizational Unit (/OU=) Ids which then can be used to sign server and user certificates.

Trust between certificate is hierarchical, similar to Internet certificates, so IDs having the same root (/O) certifier recognizse each other. The hierachy can be used in Access Control (another story for another time) to grant access to all IDs at a given level e.g. /OU=Management/O=Acme


Read more

Posted by on 04 February 2020 | Comments (3) | categories: Domino