Usability - Productivity - Business - The web - Singapore & Twins

Large scale workflow application performance using Push Replication

Imagine the following scenario (live from a very large customer of mine):
A workflow application on a central server has at any time 500-800k of active documents. Any normal user would have access to about 50 of them, while an approver typically sees 1000-1500. Using a Notes client Domino will be mainly busy not to display a record (even if you follow my advise). Contrast that with a local replica of the same application. Since documents a user can't see are not replicated these local replicas would be tiny in comparison and offer a beautiful user experience. The only catch: if you work on a local replica you most likely will screw up notifications and an approver will get a request before the document is in her local replica. The sequence that needs to be followed looks like this:
8 steps of local workflow
  1. User creates a new request in a local workflow database and submits it
  2. Local replica replicates back to the server
  3. Approver gets notified that new data is waiting
  4. New data is replicated from the server to the client
  5. The approver makes a decision and submits it
  6. Data is replicated back to the server
  7. Requester is notified on updated data
  8. Data is replicated from server back to the requester's workstation
It is easy to see why workflow databases are hardly exist as local replicas. Replication as background process typically runs on schedule and doesn't tell when it is finished (other than in the replicator page). There is no trigger to tell a local database: now it is time to fetch. But if it was different? What if the requester would only do step 1 and 2-4 happens automagically? If the approver would get the notification after the data has arrived in step 4? If the approver does only step 5 and steps 6-8 also happen automagically, with the notification after local data has arrived?
This is exactly what Dragon Li from our Beijing Lab and I are working on. The prototype runs quite beautifully but currently requires both users to be online. We are using machine to machine notification, so the automatic steps can be completed in the background without disturbing the users before they get notified. The hooks for a notification persistence are ready and just need to be implemented. The beautiy of this implementation: we use the time tested replication, just we trigger it differently. No new protocol or emerging unratified standard is used. The application works through an innovative combination what is there in the Notes client for quite a while already. Pending our internal process this will hit OpenNTF soon.

Posted by on 26 September 2011 | Comments (1) | categories: Show-N-Tell Thursday


  1. posted by Lothar Mueller on Tuesday 27 September 2011 AD:
    I remember having a similar situation in a customer project several years ago. We solved it in a way that all workflow notifications were sent by the spoke server only using Domino agents. That worked quite nice. Only problem: there was no way to control WHEN the users would replicate their data, or whether the replicated data and the notifications would arrive at the same time. Looking forward to learn what you're up to Emoticon wink.gif