Workflow Waiting Queue - Possible Scenarios

Common scenarios:

Incoming file that has no corresponding workflow on region

File UnexpectedIncomingData.dat arrives on region

NDFA adds information about this file to global error queue:

1. Ndfa gets PARAMETER_BLOB_VALUE by key ORCH_INSTANCE:-418:PARAMETER_NAME:global_error_queue

2. NDFA parses through queue, throwing out records that exceeded lifespan

3. NDFA adds file information to the end of queue

4. NDFA updates PARAMETER_BLOB_VALUE by key ORCH_INSTANCE:-418:PARAMETER_NAME:global_error_queue

Record that NDFA adds for unexpected file to queue:

No excessive information needed – only addtimestamp, signal correlation id (to recognize file in cmw/in folder) and remotefilename – to find out original file name.

Incoming file that has corresponding workflow on region but no workflow task waiting for file

File ExpectedIncomingData.dat arrives on region that is expected by load_files workflow, load_subprocess subprocess and load_task task.

ndfa_exec creates and sends setEvent CM.

execwkfl stream receives CM for SetEvent and updates workflow specific wait queue:

1. Execwkfl retrives workflow specific waiting queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue

2. Parses through workflow specific queue to find tasks waiting for this file (and throws away records that exceed lifespan threshold)

3. Gets to an end of queue without finding task for file

4. Adds record about file waiting for task to process to the end of queue

Added record looks like:

Incoming file that has corresponding workflow on region being canceled before processing

File ExpectedIncomingData.dat arrives on region that is expected by load_files workflow, load_subprocess subprocess and load_task task.

ndfa_exec creates and sends setEvent CM.

execwkfl stream receives CM for SetEvent and updates workflow specific wait queue:

1. Execwkfl retrives workflow specific waiting queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue

2. Parses through workflow specific queue to find tasks waiting for this file (and throws away records that exceed lifespan threshold)

3. Gets to an end of queue without finding task for file

4. Adds record about file waiting for task to process to the end of queue

5. Updates wait queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue

Added record looks like:

User decides to cancel file load and sends RTR for execwkfl to cancel file processing.

execwkfl stream receives CM for SetEvent and updates workflow specific wait queue:

1. Execwkfl retrives workflow specific waiting queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue

2. Parses through workflow specific queue to find file record (by signal correlation id), throwing records that exceed record lifespan.

3. Finds file record and removes it from wait queue

4. Updates wait queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue

5. Retrieves error_queue by ORCH_INSTANCE:-418:PARAMETER_NAME:load_files_wait_queue

6. Parses through it (throwing out records which lifespan exceeds threshold) and adds record about canceled file to end of error queue

Added record looks like:

Incoming file that has corresponding workflow on region being canceled before processing

File ExpectedIncomingData.dat arrives on region that is expected by load_files workflow, load_subprocess subprocess and load_task task.

ndfa_exec creates and sends setEvent CM.

execwkfl stream receives CM for SetEvent and updates workflow specific wait queue:

1. Execwkfl retrives workflow specific waiting queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue

2. Parses through workflow specific queue to find tasks waiting for this file (and throws away records that exceed lifespan threshold)

3. Gets to an end of queue without finding task for file

4. Adds record about file waiting for task to process to the end of queue

5. Updates wait queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue

Added record looks like:

Workflow for this file launches and consumes file.

execwkfl stream gets to step that consumes this file:

1. Execwkfl retrives workflow specific waiting queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue

2. Parses through workflow specific queue to find file record (by signal correlation id), throwing records that exceed record lifespan.

3. Finds file record and removes it from wait queue.

4. Updates wait queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue

5. Retrieves workflow specific wait_queue by ORCH_INSTANCE:{workflow instance}:PARAMETER_NAME:wait_queue

6. Parses through it and updates record about consumed file

7. Updates workflow instance wait queue

Added record looks like: