Simultaneous Lotus notes server-side agents

Question

In my Lotus Notes workflow application, I have a scheduled server agent (every five minutes). When user's act on a document, a server-side agent is also triggered (this agent modifies the said document, server-side). In the production, we are receiving many complaints that the processing are incomplete or sometimes not being processed at all. I checked the server configuration and found out that only 4 agents can run concurrently. Being a global application with over 50,000 users, the only thing that I can blame with these issues are the volume of agent run, but I'm not sure if I'm correct (I'm a developer and lacks knowledge about these stuffs). Can someone help me find if my reasoning is correct (on simulteneous agents) and help me understand how I can solve this? Can you provide me references please. Thank you in advance!

Answer 1

Important thing to remember.

Scheduled agents on the server will only run one agent from the same database at any given time!

So if you have Database A with agent X (5 mins) and Y (10 mins). It will first run X. Once X completes which ever is scheduled next (X or Y) will run next. It will never let you run X + Y at the same time if they are in the same database.

This is intended behaviour to stop possible deadlocks within the database agents.

Also you have an schedule queue which will have a limit to the number of agents that can be scheduled up. For example if you have Agent X every 5 minutes, but it takes 10 minutes to complete, your schedule queue will slowly fill up and then run out of space.

So how to work around this? There is a couple of ways.

Option 1: Use Program Documents on the server.

Set the agent to scheduled "Never" and have a program document execute the agent with the command.

tell amgr run "dir/database.nsf" 'agentName' 

PRO:

1. You will be able to run agents in <5 minute schedule.
2. You can run multiple agents in the same database.

CON:

1. You have to be aware of what the agent is interacting with, and code for it to handle other agents or itself running at the same time.
2. There can be serious performance implications in doing this. You need to be aware of what is going on in the server and how it would impact it.
3. If you have lots of databases, you have a messy program document list and hard to maintain.
4. Agents via "Tell AMGR" will not be terminated if they exceed the agent execution time allowed on the server. They have to be manually killed.
5. There is easy way to determine what agents are running/ran.

Option 2: Create an agent which calls out to web agents.

PRO:

1. You will be able to run agents in <5 minute schedule.
2. You can run multiple agents in the same database.
3. You have slightly better control of what runs via another agent.

CON:

1. You need HTTP running on the server.
2. There are performance implications in doing this and again you need to be aware of how it will interact with the system if multiple instances run or other agents.
3. Agents will not be terminated if they exceed the agent execution time allowed on the server.
4. You will need to allow concurrent web agents/web services on the server or you can potentially hang the server.

Option 3: Change from scheduled to another trigger.

For example "When new mail arrives". Overall this is the better option of the three.

...

In closing I would say that you should rarely use the "Execute every 5 mins" if you can, unless it is a critical agent that isn't going to be executed by multiple users across different databases.