Bulk Upload

Unknown

Bulk Upload

Unknown

Hello Decisions,

I’m currently running a query to get back results that contain more than 8,000 records. I’ve tried to use a ForEach step to apply some additional steps before it uploads into my data structure but it crashed our Dev server. Instead of using a ForEach step that does one record at a time is there an alternative that won’t crash our server?

Unknown

Hi,

There is a ForEach (Chunking) Step or Branch ForEach (Chunking) that you can try. The ForEach(Chunking) will send item lists through of a set size per iteration. The Branching ForEach (chunking) will do the same, but in parallel.

Let me know if this is what you are looking for.

Unknown

So the logic behind the chunking was good however it outputs a list. Is there a step that would output a single record because in the later steps we are substringing a ZIP code and utilizing truth tables for that single record.

Thanks,

Unknown

If you still need a single item, you can nest the ForEach loop inside the chunking loop.

Unknown

For clarification you’re insisting the following:
Execute Query -> Branch ForEach (chunking) -> ForEach step -> Substring step etc…?

Unknown

Yes,

This will divvy up the threads that are running in parallel and work some tasks sequentially, so instead of having 1000 processes happening at once, we can have 100 processes with 10 sequential steps happening at once.

best,