Common scenarios:
Incoming file that has no corresponding workflow on region
File UnexpectedIncomingData.dat arrives on region
NDFA adds information about this file to global error queue:
- Ndfa gets PARAMETER_BLOB_VALUE by key ORCH_INSTANCE:-418:PARAMETER_NAME:global_error_queue
- NDFA parses through queue, throwing out records that exceeded lifespan
- NDFA adds file information to the end of queue
- NDFA updates PARAMETER_BLOB_VALUE by key ORCH_INSTANCE:-418:PARAMETER_NAME:global_error_queue
Record that NDFA adds for unexpected file to queue:
addtimestamp|20181120 094750|signalcorrelationid|15TVOUI2OIF4JHGPDATA|remotefilename|UnexpectedIncomingData.dat
No excessive information needed – only addtimestamp, signal correlation id (to recognize file in cmw/in folder) and remotefilename – to find out original file name.
Incoming file that has corresponding workflow on region but no workflow task waiting for file
File ExpectedIncomingData.dat arrives on region that is expected by load_files workflow, load_subprocess subprocess and load_task task.
ndfa_exec creates and sends setEvent CM.
execwkfl stream receives CM for SetEvent and updates workflow specific wait queue:
- Execwkfl retrives workflow specific waiting queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue
- Parses through workflow specific queue to find tasks waiting for this file (and throws away records that exceed lifespan threshold)
- Gets to an end of queue without finding task for file
- Adds record about file waiting for task to process to the end of queue
Added record looks like:
updtimestamp|20181120 094750|addtimestamp|20181120 094750|processworkflowname|load_files|workflow|load_subprocess|event|load_task |instance||step||correlationid||processcorrelationid||signalcorrelationid| 45TBCY2OIF4JHGPDATA|eventprocesscorrelationid||remotefilename| ExpectedIncomingData.dat|eventinstance||autostart|0|isset|1
Incoming file that has corresponding workflow on region being canceled before processing
File ExpectedIncomingData.dat arrives on region that is expected by load_files workflow, load_subprocess subprocess and load_task task.
ndfa_exec creates and sends setEvent CM.
execwkfl stream receives CM for SetEvent and updates workflow specific wait queue:
- Execwkfl retrives workflow specific waiting queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue
- Parses through workflow specific queue to find tasks waiting for this file (and throws away records that exceed lifespan threshold)
- Gets to an end of queue without finding task for file
- Adds record about file waiting for task to process to the end of queue
- Updates wait queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue
Added record looks like:
updtimestamp|20181120 094750|addtimestamp|20181120 094750|processworkflowname|load_files|workflow|load_subprocess|event|load_task |instance||step||correlationid||processcorrelationid||signalcorrelationid| 45TBCY2OIF4JHGPDATA|eventprocesscorrelationid||remotefilename|ExpectedIncomingData.dat|eventinstance||autostart|0|isset|1
User decides to cancel file load and sends RTR for execwkfl to cancel file processing.
execwkfl stream receives CM for SetEvent and updates workflow specific wait queue:
- Execwkfl retrives workflow specific waiting queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue
- Parses through workflow specific queue to find file record (by signal correlation id), throwing records that exceed record lifespan.
- Finds file record and removes it from wait queue
- Updates wait queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue
- Retrieves error_queue by ORCH_INSTANCE:-418:PARAMETER_NAME:load_files_wait_queue
- Parses through it (throwing out records which lifespan exceeds threshold) and adds record about canceled file to end of error queue
Added record looks like:
updtimestamp|20181120 104750|addtimestamp|20181120 094750|processworkflowname|load_files|workflow|load_subprocess|event|load_task|signalcorrelationid| 45TBCY2OIF4JHGPDATA|remotefilename|ExpectedIncomingData.dat|eventinstance||autostart|0|isset|canceled
Incoming file that has corresponding workflow on region being canceled before processing
File ExpectedIncomingData.dat arrives on region that is expected by load_files workflow, load_subprocess subprocess and load_task task.
ndfa_exec creates and sends setEvent CM.
execwkfl stream receives CM for SetEvent and updates workflow specific wait queue:
- Execwkfl retrives workflow specific waiting queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue
- Parses through workflow specific queue to find tasks waiting for this file (and throws away records that exceed lifespan threshold)
- Gets to an end of queue without finding task for file
- Adds record about file waiting for task to process to the end of queue
- Updates wait queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue
Added record looks like:
updtimestamp|20181120 094750|addtimestamp|20181120 094750|processworkflowname|load_files|workflow|load_subprocess|event|load_task |instance||step||correlationid||processcorrelationid||signalcorrelationid| 45TBCY2OIF4JHGPDATA|eventprocesscorrelationid||remotefilename|ExpectedIncomingData.dat|eventinstance||autostart|0|isset|1
Workflow for this file launches and consumes file.
execwkfl stream gets to step that consumes this file:
- Execwkfl retrives workflow specific waiting queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue
- Parses through workflow specific queue to find file record (by signal correlation id), throwing records that exceed record lifespan.
- Finds file record and removes it from wait queue.
- Updates wait queue by key ORCH_INSTANCE:-200:PARAMETER_NAME:load_files_wait_queue
- Retrieves workflow specific wait_queue by ORCH_INSTANCE:{workflow instance}:PARAMETER_NAME:wait_queue
- Parses through it and updates record about consumed file
- Updates workflow instance wait queue
Added record looks like:
updtimestamp|20181120 104750|addtimestamp|20181120 094750|processworkflowname|load_files|workflow|load_subprocess|event|load_task|correlationid|C1D40JJ8OSFGB012|processcorrelationid|B81C1D40QM4OEPLI|signalcorrelationid|45TBCY2OIF4JHGPDATA|remotefilename|ExpectedIncomingData.dat|eventinstance||autostart|0|isset|consumed