Usability - Productivity - Business - The web - Singapore & Twins

By Date: November 2011

Documenting Notes Development

We all know: developers are to documentation are like cats to dogs. There are cute pictures of co-existence in the internet but in reality one chases the other in a never ending fight. One of the big reasons: nobody reads documentation until a problem occurs or there is a change of watch, none of which appears in either "important" or "interesting" category of a regular developer. A partial solution for this dilemma is to automate documentation creation. I'm fully aware that a lot of generated documentation is not very useful, so be careful what you wish for.
What I describe in the following is not something readily available, but it would make an excellent plug-in for the Domino Designer (if I can find some brothers in crime). Some of the tools are ready, some need to be created and all of them need to be glued together in an automated process. So this is how it can look like:
  1. The documentation process would be governed by an Apache ANT script. Such a script can be run from within Domino designer and you even could automatically run it whenever you run a build of your application
  2. Ideally the reports would run (makes things easier) on individual files and, for cross design element reports, on a summary file. When connecting an NSF to a version control system, the individual files are created on the file system, so that's sufficient (or you can use Guo Yi's Import/Export plug-in (which incidentially has an ANT interface ) or call DXLMagic from the ANT script (it is like calling a command line Java) with the DesignExtractorSINGLE mode. DXLMagic useful for classic stuff, not for XPages, so you stick to the export tool or the version control for these
  3. Run XSLT based reports on the XML: View selection formula, XPages field validation, Buddy list reports, Hide-When formulas, Replication formulas, Field reports, View structure, XRef documents etc.
  4. Use JavaDoc to document all Java classes you might have used
  5. Use LSDoc to document your LotusScript (how to get all of it into a script library is subject to a future post)
  6. Use JSDoc to document all the JavaScript
  7. Use Visustin to generate flowcharts from all the language source code. With a little tweaking the *Doc generators can be tweaked to include the images of visustin (after all it is just HTML which could be post-processed)
  8. Use Crap4J to check for the quality of the Java used (CRAP stands for: "Change Risk Analysis and Predictions")
  9. Use JSHint to check all the JavaScript (based on some opinion it will work better for our purpose than the original JSLint
  10. The missing piece: LSLint. It would be way cool to have something like that. On the other hand: some code better should not be scrutinized
  11. Finally all results are uploaded into one WIKI (or another), so the WIKI would take care of "what has changed"
What would you add? What XML/XSLT driven reports could be created?

Posted by on 29 November 2011 | Comments (3) | categories: Show-N-Tell Thursday

View selection reports

In the last entry I described a method to replace view selection formulas wholesale with boolean expressions. One question I got asked was: " Do we need this?". The clear answer is: " It depends". To make an informed decision, you need to look at your view selection formulas. To make that easier I designed a report you can run against your DXL export of your database design (using DXLMagic) that will highlight stuff that needs fixing. Of course beautiy is in the eye of the beholder. Eliminating @Now, @Today, @Tomorrow and @Yesterday is a no-brainer for anyone. Disputed items are @IsResponseDoc (you probably want @AllDescendants) or @Contains, as well as the total length or the number of *OR* conditions. Anyway, the report with a little automation can be part of your standard system documentation.
Sample for Selection Formula Report
The report is done in XSLT and you are free to add your own priorities in the matching formulas.

Read more

Posted by on 29 November 2011 | Comments (0) | categories: Show-N-Tell Thursday

Brute Force View Tuning

Complex view selection formulas can slowdown your Domino performance substantially. So keeping them simple is a mandate for high performance applications. One of my favorite approaches here is to have a set of hidden fields in my forms that compute to true or false and then simply use SELECT vsf_Demo as selection formula, where " vfs" stands for "View Selection Formula". In large applications that results in a rather nice performance. Downside of the approach: you need to compute these values on each alteration of a document (but that's not too hard). You can use this method even to retrofit existing applications to improve performance. These are the steps:
  1. Create a FORM with one field for each view that contains the view selection formula
  2. Change the view selection formula (short of the SELECT @All ones) to point to that field
  3. Alter/Create the QuerySave event to: switch to that form, do a computeWithForm, switch back to the original form
  4. Create a script library with an easy function that takes a document and tells you if the document's selection formulas have been updated
  5. Add a call to that function to all agents
  6. Add an agent that runs on "documents created or changed" to catch any case where the two calls above didn't work (e.g. @Formula agents)
  7. Test, Test, Test
  8. Enjoy better performance
The code you need to add to the querySave event is rather light: while this is the code in the querySave event of any form:
Option Declare
Use "CoreHandlerLibrary"

Sub Querysave (Source As Notesuidocument, Continue As Variant )
    Dim handler As DocumentHandler
    Dim doc As NotesDocument
    Set doc = Source. Document
    Set handler = New DocumentHandler (doc )
      'We recompute here, since we will save anyway we don't care for the result
    Call handler. docHasChangedAfterUpdate ( )
End Sub
Being lazy and notoriously prone to typos I created some code that does step 1 and 2 for me in one go.
    Class ViewTuner
    Description: Creates a form with all view selection formulas and updates all views

Public Class ViewTuner
    Private ourForm As NotesDOMDocumentNode
    Private formName As String
    Private fieldNamesUsed List As String
    Private resultStream As NotesStream
    Private parser As NotesDOMParser
    Private s As NotesSession
    Public Sub New
        formName = "ViewSelectionFormulas"
        Set me. s = New NotesSession
    End Sub
        Property Set formToUse
        Description: Overwrite the formName if necessary
    %END REM

    Public Property Set formToUse As String
        me. formName = formToUse
    End Property

        Function getEmptyForm
        Description: Get an empty form DOM in DXL
    %END REM

    Private Function getEmptyForm (formName As String )   As NotesDOMDocumentNode
        Dim rawform As NotesStream
        Dim s As New NotesSession
        Set rawForm = s. Createstream ( )
        Set resultStream = s. Createstream ( )

        Call rawform. Writetext ( "<?xml version='1.0' encoding='utf-8'?>" )

Read more

Posted by on 24 November 2011 | Comments (3) | categories: Show-N-Tell Thursday

Doing one thing really well

One of my absolute favorite Android apps is Skitch (recently acquired by Evernote). It does exactly one thing: let you doodle on pictures and share them. While they market it for "fun and happiness" I can see applications for maintenance and evidence collection. Skitch is an excellent example of the principle Do one thing and do it outstanding that utilitarian applications should follow. Skitch is available for Android and for the Mac and according to their website for iOS soon.

Posted by on 23 November 2011 | Comments (3) | categories: After hours Mobile

Workflow and Organisational Directory

An interesting question was raised in a recent development discussion: " What is the best way to keep the organisation data of our workflow application(s)?" My first answer was easy: "Definitely not inside the application itself". But then the choices start (too many of them are not good for you): Keep them in the Domino directory, in a custom Notes application, in a RDBMS, pull from HR, pull from LDAP? Each of these approaches do have their merit and their drawbacks. I'll elaborate.
Of course the first question to ask: Should I build my own workflow engine? There are a number of choices readily available. You can start with Ironworkflows on OpenNTF which even provides a graphical workflow editor running in the browser (and full source code included). You could have a look at XFlow, that will gain with some help based on a a great library a graphical editor in due time. It also can work with different engines. If you are not into "complete source provided" type of applications, you can checkout PAVONE Espresso Workflow which runs both on Domino and Websphere and got a nice graphical editor too. The graphical editors seem to be a must in these days, but might only determine a fraction of the usefulness of a workflow engine (I know that statement is a candidate for flame proved underwear).
But back to the initial question: Where to store the organisation data? If you build your own engine I would strongly suggest, regardless of where you store, to abstract workflow logic from storage. Define the API your application will call (like getWFDefinition(...), getApprovers(...), resolveRole(...) etc. What calls would make sense in an engine warrants a post on its own. Then code against the API (this is called contract first programming). This way it doesn't matter if your storage backend changes and you can keep the considerations for storage separate from the considerations for your applications. In Domino such an API would be best implemented using the Extensibility API (that's what XFlow is doing). Now to the storage options:
  • Domino directory (my preference)
    If there wasn't THE ADMIN the Domino directory is an excellent choice. It has ample fields to capture organisational information like department, managers etc. You even have well defined extension points (yes: places where you legally can mess with the directory).
    Big advantage: easy to sync (we give you TDI), available for other applications via LDAP and when a user goes the roles go with him. That is the general idea of a directory service.
    Biggest challenge: the admins jealously guard the directory and won't let you get anywhere near it to customise or update data. Cited reason: "If you mess with the directory, IBM won't support us" and "We can't grant the access rights without risking other parts of the data being messed up". There *is* a slight difference between altering the directory design (=fun with support) and updating the schema. Dealing with the schema (and LDAP) requires to leave the protected yellow bubble for a moment and learn about the finer details of LDAP (e.g. I didn't know that you can't use # for a group name, but & is allowed).
    Open to discussion: how well does Domino Directory serve to capture all the different roles? I've seen very few engines using the directory alone. What I have seen is the Domino directory in use for lookups, but a custom front application for data entry, so the workflow owners can't mess with other data in the directory. Also popular: sync the Domino directory using TDI with an external backend like the HR system
  • Custom Notes application
    That's the most prevalent form of corporate storage for organisation data. The original Prozessware used that approach as do the tools mentioned above. Clear advantage is the flexibility of a solution, since you can tailor it to whatever you fancy without the risks perceived in adding to the Domino directory. Of course you loose the exposure via LDAP, which makes it a solution confined to Domino (unless you add some web services, which would be a good idea) and you need to keep the Domino directory in sync with the org database. What I've seen very often are org directories that synchronise with all sorts of backends: RDBMS, HR Systems, web services etc. This of course opens the eternal question of sync vs. direct access
    As long as you look for a Domino solution there is little reason to entertain a RDBMS (unless of course your architect was hiding under a rock and thinks NoSQL isn't a database). So typically this approach can be found where e.g. HR systems run on different platforms. I'm not a big fan of direct RDBMS access since it ties you down to a rigid structure and, more important, adds another breaking point: if your RDBMS is down your workflow stops, so you need to queue your apps for processing. You can sync the data or at least use LEI/DECS to avoid language rojak
  • Pull from HR
    A good HR system would have a higher level API (probably a web service) that allows to pull org information by your workflow engine. While more business like than the RDBMS approach these APIs are rarer. If your HR system has one - good for you, use it. Again you need to decide sync vs. direct access
  • LDAP
    If your LDAP server isn't plagued with a fragile engine it is a good place to store org data in it. LDAP can be queried by a lot of platforms and is language neutral. In Domino LDAP goes well with XPages (I've seen a LDAP data source somewhere) and Java, but not so much with LotusScript. Of course if you use Domino Directory as (one of your) LDAP engine(s) you get the best of both. A good idea when working LDAP (Domino or others) is to have a sound LDAP front-end to view and enter data. Apache's Directory Studio is a good choice here
I'll cover more thoughts on what the org directory should provide in a later post. As usual YMMV.

Posted by on 21 November 2011 | Comments (4) | categories: Show-N-Tell Thursday

The Vulcan UI vision

By now everybody in the little yellow bubble should have seen the Project Vulcan Demo:

But how would you explain it to a non-technical user? I use the following little story:
Just imagine one of your favorite artists performs at The Esplanade and you invested into some premium tickets. When you arrive a purser doesn't ask you for your ticket class, but asks: " How did you come here?" When you look astonished, he will continue: " If you came by car, please use entry A. If you came by bus, entry B is for you. Taxi passenger use entry C, while pedestrians please use entry D" - and these are the entries to the concert hall, not the building.
Silly isn't it? But that's exactly what we do with computers today: " If you arrived via SMTP, please show up in eMail. If you arrived via XMPP, show up in the chat client. If you arrived via RSS/ATOM please surface in the feed reader, etc.". Vulcan will put an end to this. First it will unify the application access and then arrange content by your criteria (from whom, what content, what priority, what project etc.) rather than distribution channel (which will be just one option).
Feel free to expand, remix and reuse this little story (and go buy some tickets)

Posted by on 21 November 2011 | Comments (2) | categories: Software

I want one - 2011 edition

From the marketing speak:
The world’s first real smartwatch. Multi-tasking, easy to use and love:  touch, drag, swipe or pinch. Get your intuitive interface in a complex world.

Posted by on 21 November 2011 | Comments (0) | categories: After hours

Information Technology and Education

Today Jotterlab launched their eLearning platform to Singapore's schools. Since CHS is one of them they approached me to deliver a keynote to their audience. I used the "deck" below. Some of the items only make sense with "sound" which I haven't recorded. But you get the idea!

Posted by on 17 November 2011 | Comments (0) | categories: eLearning

The appraisal

Made me cringe with laughter:
John Doe, my assistant developer, can always be found
hard at work in his cubicle. John works independently, without
wasting company time talking to colleagues. John never
thinks twice about assisting fellow employees, and he always
finishes given assignments on time. Often John takes extended
measures to complete his work, sometimes skipping
coffee breaks. John is an individual who has absolutely no
vanity in spite of his accomplishments and profound
knowledge in his field. I firmly believe that John can be
classed as a high-calibre employee, the type which cannot be
dispensed with. Consequently, I duly recommend that John can be
promoted to executive management, and a proposal will be
executed as soon as possible.

Once you read it slightly different:

Read more

Posted by on 15 November 2011 | Comments (0) | categories: After hours

Yong Green Food Melbourne

My favorite nice lives in Melbourne. Having a local guide makes discovery of awsome food places easy. We had dinner at Yong Green Food in Fizroy. The lovingly prepared vegetarian food was a joy to look at, to smell and to eat. Highly recommended! The pictures below were taken in low light with an Huawei Ideos phone (no match to the high end phones)

Young Organic Food

Young Organic Food

Young Organic Food

Young Organic Food

Posted by on 09 November 2011 | Comments (0) | categories: Travel

Protecting sensitive information in Notes documents

Even in the most social businesses there is information that is available only on a "need to know" basis. Sometimes it is only a small subset of the fields in a document. In a Notes client, as long as I can see a document, I can see all the item values in it (using the document properties box), even if the field is hidden in the current form. So what are my options to protect a few sensitive fields?:
  1. Encrypt the fields with an encryption key that only the authorised user have. Problem here: you need to manage these keys and need a different set for each use case - messy. Also you can't show any of these fields in a view
  2. Hide the database design when applying the template. Looks good on the first view, but a semi skilled user can easily bypass that (e.g. copy & paste into an empty database, create a private view, use a @Prompt smart icon, use the free Document viewer or NotesPeek) - security by obscurity never works
  3. Store the sensitive data in a second document that is protected by reader names that are different (more restrictive) than the main document. This approach also keeps (given you have security set) curious admins out, a capability RDBMS is lacking
  4. Change the process and remove the need for the fine grained access control
Let's have a closer look at option 3. Notes allows to pull in values from a different document using @GetDocField. So once we have the secret document, we can pull in these values. If a user can't see that document no value is retrieved. The formula for such a computed for display field would look like this:
@If (RiskAssessmentID= "";@Return ( "n/a" ); "" );
tmp := @GetDocField (RiskAssessmentID; "RiskAssessment" );
@If ( @IsError (tmp ); "nothing to retrieve";tmp )
To create such a "secondary document" a few lines of LotusScript in a button make it a seamless experience:
Sub Click (Source As Button )
    Dim doc As NotesDocument
    Dim riskDoc As NotesDocument
    Dim uiDoc As NotesUIDocument
    Dim db As NotesDatabase
    Dim w As New NotesUIWorkspace
    Set uidoc = w. CurrentDocument
    Set doc = uidoc. Document
    If doc. IsNewNote Then
        Call doc. Save ( True, True )
    End If
    Set db = doc. ParentDatabase
    If doc. RiskAssessmentID ( 0 ) = "" Then
        Set riskDoc = db. CreateDocument
        Call riskDoc. ReplaceItemValue ( "Form", "RAF" )
        Set riskDoc = db. GetDocumentByUNID (doc. RiskAssessmentID ( 0 ) )
        If riskDoc Is Nothing Then
            Set riskDoc = db. CreateDocument
            Call riskDoc. ReplaceItemValue ( "Form", "RAF" )        
        End If
    End If
    Call riskDoc. MakeResponse (doc )
    If  w. DialogBox ( "RAF", True, True, False, False, False, False, "Please provide your Risk Assessment",riskDoc, True ) Then
        Call riskDoc. Save ( True, True )
        Call doc. ReplaceItemValue ( "RiskAssessmentID",riskDoc. UniversalID )
        Call doc. Save ( True, True )
    End If
    Call uidoc. Refresh
End Sub
To keep the 2 documents in sync we add some code into the PostSave event of the main form:
Sub Postsave (Source As Notesuidocument )
    Dim doc As NotesDocument
    Dim riskDoc As NotesDocument
    Dim db As NotesDatabase
    Set doc = source. Document
    If doc. RiskAssessmentID ( 0 ) = "" Then
    End If
    Set db = doc. ParentDatabase
    Set riskDoc = db. GetDocumentByUNID (doc. RiskAssessmentID ( 0 ) )
    Call riskDoc. ReplaceItemValue ( "Subject",doc. subject )
    'Repeat with other fields needed in views, for workflow and access control
    Call riskDoc. Save ( True, True )
End Sub
We are almost done. The response documents need some handling when we try to open them.

Read more

Posted by on 04 November 2011 | Comments (1) | categories: Show-N-Tell Thursday

Balsamiq Mockups in XPages

I write most of my technical blogs to document solutions of problems I have encountered in "real" applications. The popular XAgents post is no exception here. Then I was investigating how to make Confluence plug-ins work in XPages. The specific plug-in I had in mind was Balsamiq Mockups, the tool that has proven invaluable for any screen design, both here on the blog as well as in countless customer discussions. Once I figured the XAgent it was rather easy:

If you want to embed Balsamiq Mockups in your own XPages application, reach out to Peldi or Valerie and buy the Balsamiq Plug-in for Confluence.
Update: To make it very clear: Using the Confluence plug-in in XPages is not covered by the Balsamiq license for Confluence, so you need to individually negotiate with Peldi and clearly indicate that you want to run that in XPages. You also, for the time being, need to promise not go go back for support for any challenge that can't be verified in a Confluence installation.
Update 2: Peldi and I are talking an there will be a solution. Stay tuned

Posted by on 02 November 2011 | Comments (9) | categories: XPages