Job queues and thread related questions
Hello! I have a few questions regarding job queues and threads, please assist:
1. Queue naming / thread pools
I've noticed when using ThreadPools or multiple scheduled jobs, I'm guessing each process needs its own queue to avoid conflicts. Since queue names are free-text, it’s hard to know what already exists. Is there a recommended best practice for naming or managing these queues?
2. Automatic thread vs. sync processing
Can Decisions automatically assign a unique thread/queue for larger jobs so they don’t block the default queue? For example, run files under 10,000 rows normally, but route larger ones to their own queue?
3. Dynamic queue names
Can queue names be created dynamically, and would using Thread Jobs, ThreadJob Async, or running a linked flow async achieve this and prevent future jobs from being blocked by long-running ones? Note: I am processing millions of records in short intervals.
Comments
-
Hello there,
Thank you for reaching out on our forum! Regarding your first question, you can view the jobs with specified queue names you created as well as default job queue names that are set up when you install Decisions by navigating to System > Administration > System Tools > Event Viewer > Thread Jobs. Below are screenshots of how to specify a job queue name as well as how it would look in the Thread Jobs dashboard.
To answer your second question, you can manage this based on the step you choose. For example, Run Flow For List can spin up multiple threads under a unique queue name. Decisions doesn’t let you control threads directly as each queue is single-threaded, but you can run things in parallel by giving each process its own queue name. So you can build out logic to go down different paths (i.e. more than or less than 10,000 rows) with the Get Line Count step. Below is an example build of the process mentioned above (may not be fit for your specific use case, just for visualization).
Finally, depending on what you need, you can either use Run Flow For List to balance the workload across threads, or use batch-processing steps to balance it across servers when dealing with datasets this large. The Run Flow For List step can’t use a dynamic worker-queue name, it will always use whatever queue name you manually set.
About Sync and Async Sub Flows
Regards,
Vinh Tran | Decisions Support