Pausing/Killing a Workflow or MC Stream

Owned by Dmitry Donosiyan (Unlicensed)
Last updated: Nov 08, 2024 by Kinga Huszno
9 min read

Cancel a Single Stream

For RTR for Cancel Single Stream the following parameters are available:

  • ActionType = ‘CANCEL’ (required)

  • StreamName = the name of stream, where cancel steps for single stream will be implemented (required). The exact stream name will be known when the development is complete

  • CancelStreamName = The name of stream to cancel (required)

  • CancelMsgStatId = Message Stat Id of stream to cancel (optional). If empty, all running streams with name CancelStreamName will be cancelled.

<EagleML eaglemlVersion="2-0" eaglemlType="RunTaskRequest" xsi:schemaLocation="http://www.eagleinvsys.com/2011/EagleML-2-0 eagleml-main-2-0.xsd" xsi:type="RunTaskRequest" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0"> <header> <messageId>EA27DCC0ED860BD0</messageId> <sentBy>http://www.eagleinvsys.com/</sentBy> <sendTo>http://www.eagleinvsys.com/eagle_ml-2-0_default_cm_control_message</sendTo> <creationTimestamp>2016-12-15T09:12:28-05:00</creationTimestamp> </header> <taskIdentifier> <correlationId correlationIdScheme="correlationIdScheme">BT142658_SRV20170724101</correlationId> <businessTaskId correlationIdScheme="businessTaskIdScheme">BT142658_SRV20170624</businessTaskId> </taskIdentifier> <taskParameters> <taskParameter> <name>ActionType</name> <dataType>S</dataType> <value>CANCEL</value> </taskParameter> <taskParameter> <name>StreamName</name> <dataType>S</dataType> <value>eagle_ml-2-0_default_cm_control_process</value> </taskParameter> <taskParameter> <name>CancelStreamName</name> <dataType>S</dataType> <value>bt142658_task_translator</value> </taskParameter> <taskParameter> <name>CancelMsgStatId</name> <dataType>S</dataType> <value>2252596</value> </taskParameter> </taskParameters> </EagleML>

  1. Delete processing file from ‘processing’ folder of stream directory (example, ‘/estar/tpe/data/msgcnetr/out/CSV/bt142658_task_translator/processing/’)

  2. Update status of processing MC task and message in msgcenter_dbo.msg_message_stat and msgcenter_dbo.tasks in order to mark them as completed

    update msgcenter_dbo.tasks set status='S', processing_status ='' where batch_id like '%UNIQUE_MASK_OF_PROCESSING_FILE%' escape '\' and STATUS = 'E' and PROCESSING_STATUS='E'
    update msgcenter_dbo.msg_message_stat st set st.status=3, end_processing_time = sysdate where st.file_name  like '%UNIQUE_MASK_OF_PROCESSING_FILE%' escape '\'  and st.end_processing_time is null

  3. Restart MC processor, which processes current file

If this step is not performed, the processor will continue processing file even if the status in msgcenter_dbo.msg_message_stat and msgcenter_dbo.tasks tables was updated.

Cancel a EJM Process

For RTR for Cancel EJM Process the following parameters are available:

  • ActionType = ‘CANCEL’ (required)

  • StreamName = the name of stream, where cancel steps for EJM process will be implemented (required). The exact stream name will be known when the development is complete

  • CancelCorrelationId = Correlation Id of EJM process to cancel (required)

  • CancelStreams = 1 or skipped (optional). If 1, underlying streams executed under EJM process will be cancelled too.

<EagleML eaglemlVersion="2-0" eaglemlType="RunTaskRequest" xsi:schemaLocation="http://www.eagleinvsys.com/2011/EagleML-2-0 eagleml-main-2-0.xsd" xsi:type="RunTaskRequest" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0"> <header> <messageId>EA27DCC0ED860BD0</messageId> <sentBy>http://www.eagleinvsys.com/</sentBy> <sendTo>http://www.eagleinvsys.com/eagle_ml-2-0_default_cm_control_message</sendTo> <creationTimestamp>2016-12-15T09:12:28-05:00</creationTimestamp> </header> <taskIdentifier> <correlationId correlationIdScheme="correlationIdScheme">BT142658_SRV20170627097</correlationId> <businessTaskId correlationIdScheme="businessTaskIdScheme">BT142658_SRV20170624</businessTaskId> </taskIdentifier> <taskParameters> <taskParameter> <name>ActionType</name> <dataType>S</dataType> <value>CANCEL</value> </taskParameter> <taskParameter> <name>StreamName</name> <dataType>S</dataType> <value>eagle_ml-2-0_default_cm_control_process</value> </taskParameter> <taskParameter> <name>CancelCorrelationId</name> <dataType>S</dataType> <value>BT142658_EJM_TC1_21</value> </taskParameter> </taskParameters> </EagleML>

The cancel steps for EJM process include all steps for cancel single stream and one additional step – sending trigger to task reporter in order to EJM process can finish correctly

  1. Delete processing files from ‘processing’ folders of streams directories (example, ‘/estar/tpe/data/msgcnetr/out/CSV/bt142658_task_translator/processing/’)
    For EJM process sometimes the file should be deleted not only from target stream (for example inbound stream for loading data), but also from the eagle control streams such as ‘eagle_ml-2-0_default_cm_datafile_control’ or ‘eagle_ml-2-0_default_cm_parallel_exec’. These streams are waited for completion of target stream. File from ‘processing’ folder of such streams should be deleted firstly and after that file from ‘processing’ folder of target stream.

  2. Update status of processing MC task and message in msgcenter_dbo.msg_message_stat and msgcenter_dbo.tasks in order to mark them as completed

    update msgcenter_dbo.tasks set status='S', processing_status ='' where batch_id like '%CORRELATION_ID_FROM_RTR%' escape '\' and STATUS = 'E' and PROCESSING_STATUS='E'
    update msgcenter_dbo.msg_message_stat st set st.status=3, end_processing_time = sysdate where st.file_name  like '%CORRELATION_ID_FROM_RTR' escape '\'  and st.end_processing_time is null

  3. Send trigger file to the ‘eagle_ml-2-0_default_cm_task_reporter’ stream to finish the EJM process
    Example of trigger data:

    CorrelationId:CORRELATION_ID_FROM_RTR:BusinessTaskId:CORRELATION_ID_FROM_RTR:ProcessCorrelationId:CORRELATION_ID_FROM_RTR:TaskParams:FileStream;TARGET_STREAM!DataFileName;PROCESSING_DATA_FILE_NAME!:varTaskStatus:FAILED:varErrorDescr:Task cancelled:IsEOFW:1:

  4. Restart MC processor, which processes current file

Cancel an Entire Workflow

For RTR for Cancel Entire Workflow the following parameters are available:

  • ActionType = ‘CANCEL’ (required)

  • StreamName = ‘eagle_ml-2-0_default_cm_execwkfl’ (required)

  • CancelWorkflowName = The name of workflow to cancel (required)

  • CancelCorrelationId = Correlation Id of workflow instance to cancel (optional). If empty, all instances of CancelWorkflowName will be cancelled

  • CancelEvents = 1 or skipped (optional). If 1, all events for CancelWorkflowName will be cancelled

  • CancelStreams = 1 or skipped (optional). If 1, underlying streams executed under steps of CancelWorkflowName will be cancelled too.

<EagleML eaglemlVersion="2-0" eaglemlType="RunTaskRequest" xsi:schemaLocation="http://www.eagleinvsys.com/2011/EagleML-2-0 eagleml-main-2-0.xsd" xsi:type="RunTaskRequest" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0"> <header> <messageId>EA27DCC0ED860BD0</messageId> <sentBy>http://www.eagleinvsys.com/</sentBy> <sendTo>http://www.eagleinvsys.com/eagle_ml-2-0_default_cm_control_message</sendTo> <creationTimestamp>2016-12-15T09:12:28-05:00</creationTimestamp> </header> <taskIdentifier> <correlationId correlationIdScheme="correlationIdScheme">BT142658_WRF_TC1_CANCEL_30</correlationId> <businessTaskId correlationIdScheme="businessTaskIdScheme">BT142658_WRF_TC1_CANCEL</businessTaskId> </taskIdentifier> <taskParameters> <taskParameter> <name>ActionType</name> <dataType>S</dataType> <value>CANCEL</value> </taskParameter> <taskParameter> <name>StreamName</name> <dataType>S</dataType> <value>eagle_ml-2-0_default_cm_control_process</value> </taskParameter> <taskParameter> <name>CancelWorkflowName</name> <dataType>S</dataType> <value>bt142658_test_workflow_for_research</value> </taskParameter> <taskParameter> <name>CancelCorrelationId</name> <dataType>S</dataType> <value>BT142658_WRF_TC1_14</value> </taskParameter> <taskParameter> <name>CancelEvents</name> <dataType>S</dataType> <value>1</value> </taskParameter> <taskParameter> <name>CancelStreams</name> <dataType>S</dataType> <value>1</value> </taskParameter> </taskParameters> </EagleML>

or

<EagleML eaglemlVersion="2-0" eaglemlType="RunTaskRequest" xsi:schemaLocation="http://www.eagleinvsys.com/2011/EagleML-2-0 eagleml-main-2-0.xsd" xsi:type="RunTaskRequest" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0"> <header> <messageId>EA27DCC0ED860BD0</messageId> <sentBy>http://www.eagleinvsys.com/</sentBy> <sendTo>http://www.eagleinvsys.com/eagle_ml-2-0_default_cm_control_message</sendTo> <creationTimestamp>2016-12-15T09:12:28-05:00</creationTimestamp> </header> <taskIdentifier> <correlationId correlationIdScheme="correlationIdScheme">BT142658_WRF_TC1_CANCEL_30</correlationId> <businessTaskId correlationIdScheme="businessTaskIdScheme">BT142658_WRF_TC1_CANCEL</businessTaskId> </taskIdentifier> <taskParameters> <taskParameter> <name>ActionType</name> <dataType>S</dataType> <value>CANCEL</value> </taskParameter> <taskParameter> <name>StreamName</name> <dataType>S</dataType> <value>eagle_ml-2-0_default_cm_control_process</value> </taskParameter> <taskParameter> <name>CancelWorkflowName</name> <dataType>S</dataType> <value>bt142658_test_workflow_for_research</value> </taskParameter> <taskParameter> <name>CancelCorrelationId</name> <dataType>S</dataType> <value>BT142658_WRF_TC1_14</value> </taskParameter> <taskParameter> <name>CancelEvents</name> <dataType>S</dataType> <value>1</value> </taskParameter> <taskParameter> <name>CancelStreams</name> <dataType>S</dataType> <value>1</value> </taskParameter> </taskParameters> </EagleML>

The cancel steps for entire workflow include steps #1-5 for updating workflow specific features such as statuses in ORCH tables, processing and global events, etc. These steps are already implemented in EagleML and can be executed via sending RTR with ActionType – CANCEL, specified CancelWorkflowName and CancelCorrelationId. Also cancel steps for entire workflow include all steps for cancel EJM process (steps #6.a-6.c)

  1. Collect list of correlation ids for all executed subtasks of workflow (store in ‘eagle\estar\tpe\dynamic\msgcenter\eagle_ml-2-0_cm\proctasks_WORKFLOW_CORRELATION_ID.dat’ file)

  2. For each wrf subtask the it is necessary:

    • Update PROC_STATUS and ORCH_STATE in ORCH table for current task

    • Send trigger file to the ‘eagle_ml-2-0_default_cm_task_reporter’ stream for current task
      Example of trigger data:

      CorrelationId:SUB_TASK_CORRELATION_ID:WorkflowStatus:FAILED:TargetMessageType::ErrorDescr:Task cancelled.

    • Remove ‘procwaitstate_’, ‘procstate_’, ‘proctasks_’ files for current task

  3. For each not wrf subtask if necessary:

    • Update PROC_STATUS and ORCH_STATE in ORCH table for current task

  4. Cancel global event waits for workflow

  5. Cancel all log records for workflow

  6. To cancel running streams executed from workflow subtask:

    1. Delete processing files from ‘processing’ folders of streams directories (example, ‘/estar/tpe/data/msgcnetr/out/CSV/bt142658_task_translator/processing/’)
      For some workflows the file should be deleted not only from stream to cancel (for example inbound stream for loading data), but also from the eagle control streams such as ‘eagle_ml-2-0_default_cm_datafile_control’ or ‘eagle_ml-2-0_default_cm_parallel_exec’. These streams are waited for completion of target stream. File from ‘processing’ folder of such streams should be deleted firstly and after that file from ‘processing’ folder of stream to cancel.

    2. Update status of processing MC task and message in msgcenter_dbo.msg_message_stat and msgcenter_dbo.tasks in order to mark them as completed

      update msgcenter_dbo.tasks set status='S', processing_status ='' where batch_id like '%SUB_TASK_CORRELATION_ID%' escape '\' and STATUS = 'E' and PROCESSING_STATUS='E'
      update msgcenter_dbo.msg_message_stat st set st.status=3, end_processing_time = sysdate where st.file_name  like '%SUB_TASK_CORRELATION_ID%' escape '\'  and st.end_processing_time is null

    3. Send trigger file to the eagle_ml-2-0_default_cm_task_reporter stream to finish the EJM process
      Example of trigger data:

      CorrelationId:SUB_TASK_CORRELATION_ID:WorkflowStatus:FAILED:TargetMessageType::ErrorDescr:Task cancelled.

  7. Restart of MC processor, which processes current file

Cancel a Workflow Substep

Before canceling a workflow substep the workflow should be paused and after that should be an opportunity to resume workflow. The steps for pause/resume workflow is described in the next chapter.

The cancel steps for workflow substep include steps #1-5 for updating workflow specific features such as statuses in ORCH tables, processing and global events, etc. These steps are near the same as for cancel entire workflow, but not implemented in EagleML yet. Also cancel steps for workflow substep include all steps for cancel EJM process (steps #6.a-6.c)

  1. Collect a list of correlation ids for all executed subtasks of workflow substep to cancel

  2. For each wrf subtask if it is necessary:

    • Update PROC_STATUS and ORCH_STATE in ORCH table for current task

    • Send trigger file to the ‘eagle_ml-2-0_default_cm_task_reporter’ stream for current task
      Example of trigger data:

      CorrelationId:SUB_TASK_CORRELATION_ID:WorkflowStatus:FAILED:TargetMessageType::ErrorDescr:Task cancelled.

    • Remove ‘procwaitstate_’, ‘procstate_’, ‘proctasks_’ files for current task

  3. For each not wrf subtask the it is necessary:

    • Update PROC_STATUS and ORCH_STATE in ORCH table for current task

  4. Cancel global event waits for workflow

  5. Cancel all log records for workflow

  6. To cancel running streams executed from workflow subtask:

    1. Delete processing files from ‘processing’ folders of streams directories (example, ‘/estar/tpe/data/msgcnetr/out/CSV/bt142658_task_translator/processing/’)
      For some workflow the file should be deleted not only from stream to cancel (for example inbound stream for loading data), but also from the eagle control streams such as ‘eagle_ml-2-0_default_cm_datafile_control’ or ‘eagle_ml-2-0_default_cm_parallel_exec’. These streams are waited for completion of target stream. File from ‘processing’ folder of such streams should be deleted firstly and after that file from ‘processing’ folder of stream to cancel.

    2. Update status of processing MC task and message in msgcenter_dbo.msg_message_stat and msgcenter_dbo.tasks in order to mark them as completed

      update msgcenter_dbo.tasks set status='S', processing_status ='' where batch_id like '%SUB_TASK_CORRELATION_ID%' escape '\' and STATUS = 'E' and PROCESSING_STATUS='E'
      update msgcenter_dbo.msg_message_stat st set st.status=3, end_processing_time = sysdate where st.file_name  like '%SUB_TASK_CORRELATION_ID%' escape '\'  and st.end_processing_time is null

    3. Send trigger file to the ‘eagle_ml-2-0_default_cm_task_reporter’ stream to finish the EJM process
      Example of trigger data:

      CorrelationId:SUB_TASK_CORRELATION_ID:WorkflowStatus:FAILED:TargetMessageType::ErrorDescr:Task cancelled.

  7. Restart of MC processor, which processes current file

Pause a Workflow

For RTR for Pause Workflow the following parameters are available:

  • ActionType = ‘PAUSE’ (required)

  • StreamName = ‘eagle_ml-2-0_default_cm_execwkfl’ (required)

  • PauseWorkflowName = The name of workflow to pause (required)

  • PauseCorrelationId = Correlation Id of workflow instance to pause (optional). If empty, all instances of PauseWorkflowName will be paused.

You can set the following parameters for PAUSE:

  • RETRY_CHECK_WORKFLOW_PAUSED is  w_custom_config parameter to set count of cycle

  • RetryCheckWorkflowPaused is RTR parameter to set count of cycle

  • W_PROCESS_TIMEOUT is w_custom_config parameter to set time of waiting of changing Pause Status

  • ProcessTimeout is RTR parameter to set time of waiting of changing Pause Status

<EagleML eaglemlVersion="2-0" eaglemlType="RunTaskRequest" xsi:schemaLocation="http://www.eagleinvsys.com/2011/EagleML-2-0 eagleml-main-2-0.xsd" xsi:type="RunTaskRequest" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0"> <header> <messageId>EA27DCC0ED860BD0</messageId> <sentBy>http://www.eagleinvsys.com/</sentBy> <sendTo>http://www.eagleinvsys.com/eagle_ml-2-0_default_cm_control_message</sendTo> <creationTimestamp>2016-12-15T09:12:28-05:00</creationTimestamp> </header> <taskIdentifier> <correlationId correlationIdScheme="correlationIdScheme">BT142658_WRF_TC1_PAUSE_WRF_01</correlationId> <businessTaskId correlationIdScheme="businessTaskIdScheme">BT142658_WRF_TC1_PAUSE_WRF</businessTaskId> </taskIdentifier> <taskParameters> <taskParameter> <name>ActionType</name> <dataType>S</dataType> <value>PAUSE</value> </taskParameter> <taskParameter> <name>StreamName</name> <dataType>S</dataType> <value>eagle_ml-2-0_default_cm_control_process</value> </taskParameter> <taskParameter> <name>PauseWorkflowName</name> <dataType>S</dataType> <value>bt142658_test_workflow_for_research</value> </taskParameter> <taskParameter> <name>PauseCorrelationId</name> <dataType>S</dataType> <value>BT142658_WRF_TC1_02</value> </taskParameter> </taskParameters> </EagleML>

Resume a Workflow

For RTR for Resume Workflow the following parameters are available:

  • ActionType = ‘RESUME’ (required)

  • StreamName = ‘eagle_ml-2-0_default_cm_execwkfl’ (required)

  • ResumeWorkflowName = The name of workflow to resume (required)

ResumeCorrelationId = Correlation Id of workflow instance to resume (optional). If empty, all instances of ResumeWorkflowName will be paused. 

<EagleML eaglemlVersion="2-0" eaglemlType="RunTaskRequest" xsi:schemaLocation="http://www.eagleinvsys.com/2011/EagleML-2-0 eagleml-main-2-0.xsd" xsi:type="RunTaskRequest" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0"> <header> <messageId>EA27DCC0ED860BD0</messageId> <sentBy>http://www.eagleinvsys.com/</sentBy> <sendTo>http://www.eagleinvsys.com/eagle_ml-2-0_default_cm_control_message</sendTo> <creationTimestamp>2016-12-15T09:12:28-05:00</creationTimestamp> </header> <taskIdentifier> <correlationId correlationIdScheme="correlationIdScheme">BT142658_WRF_TC1_RESUME_WRF_01</correlationId> <businessTaskId correlationIdScheme="businessTaskIdScheme">BT142658_WRF_TC1_RESUME_WRF</businessTaskId> </taskIdentifier> <taskParameters> <taskParameter> <name>ActionType</name> <dataType>S</dataType> <value>RESUME</value> </taskParameter> <taskParameter> <name>StreamName</name> <dataType>S</dataType> <value>eagle_ml-2-0_default_cm_control_process</value> </taskParameter> <taskParameter> <name>ResumeWorkflowName</name> <dataType>S</dataType> <value>bt142658_test_workflow_for_research</value> </taskParameter> <taskParameter> <name>ResumeCorrelationId</name> <dataType>S</dataType> <value>BT142658_WRF_TC1_02</value> </taskParameter> </taskParameters> </EagleML>