Disable a JobProcessor that isn't working well anymore

[christian.hombach]

For example, if a job processor machine has trouble obtaining a license, it currently “kills” all tasks in the scheduler for its fellow job processors.

Preferred behavior: after, for example, 3 consecutive failed tasks of the same type, the affected job processor should stop processing further tasks. It would also be helpful to send an email notification to the administrator.

[DominikHall_CADPDM]

Fully agree – we’ve seen the same behavior in multi-processor setups.

A single unstable Job Processor (e.g. license or session issues) can disrupt the whole queue instead of being isolated. A simple fail-threshold per job type with auto-disable or cooldown would already improve stability a lot.

Admin notification would be a strong addition as well, since these issues are easy to miss without active monitoring.

[ihayesjr]

@christian.hombach 

Thank you for posting the idea. 

Question: What if the problem is not the Job Processor? Do you still want it to disable itself?

For example, there is an issue creating PDF jobs for a particular set of drawings in a specific dataset.

If it fails on three drawings, but there are more PDF jobs in the queue that won't fail.

What should happen with the Job Processor?

[Nick_Hall]

It seems to me that all Job Processor errors fall into one of three categories

1. Definitely the Job Processor
   eg Failed to acquire a license
2. Definitely the data
   eg Update Revision Table - A qualifying Inventor installation is required to perform this operation
   
3. Everything that isn't A or B

If the errors were categorised, it would be possible to stop the Job Processor for multiple A errors, but not B or C

Another, more flexible solution would be to use a configuration file containing patterns to search for within errors. Multiple hits on these could cause the Job Processor to stop. The patterns could be simple wildcards or, for more flexibility/granularity, regexes.

Nick

[ihayesjr]

@Nick_Hall 

Thanks for the feedback.

[christian.hombach]

@Nick_Hall 

Exactly! 

The feature should be disabled by default as it doesn't make sense in a single Job Processor environment. But if Job Processor A is blocking the other e.g. 4 caused by "Failed to acquire a license" it's really bad.
An option would of course also be a self healing option   -   e.g. 3 times "Failed to acquire a license" in a row is triggering Windows to reboot.