Sequencer CleanUp

In some rare cases (especially related to emergency situations in env, such as MC crash, MC stopped/restarted, files got stuck) you might find that one or more of launched workflows have “In progress” status, however there are no launched sub-steps (streams). For example:

Open

Please note, that this situation differs from a hanging extract stream (in case of hanging extract stream you would be able to expand a workflow line in MT and see that there is a stream or streams with "Processing" status (not a workflow))

This page with help you cleanup the stuck process using built-in EJM instruments and recover Sequencer functionality.

On this page

• 1 Reset Sequencer using eagle_ml-2-0_default_cm_sequencer_cleanup 
  • 1.1 Reset current stuck process (recommended method)
  • 1.2 Reset the entire queue
• 2 Reset Sequencer using eagle_ml-2-0_default_cm_nuggetizer_recover
• 3 Sequencer states validation

Reset Sequencer using eagle_ml-2-0_default_cm_sequencer_cleanup 

Reset current stuck process (recommended method)

1. Create a .txt file with any name with the following line:

1176:CUR_[workflow_name]:1177:[POSTFIX]:

where:

• CUR_  – is a prefix used when you want to cleanup only current process (which is stuck),

• [workflow_name] – is workflow name,

• [POSTFIX] – is defined as:

  • the value of <processingSequencerIdentifier> element of the initial RTR (if this node is not empty and exists). For example, on Sequencer Execution in Parallel Mode page:

    […] <taskIdentifier>     <processingSequenceNumber>1</processingSequenceNumber>     <processingSequencerIdentifier>PROCSEQID</processingSequencerIdentifier>     <correlationId correlationIdScheme="correlationIdScheme">correlationId</correlationId>     <businessTaskId correlationIdScheme="businessTaskIdScheme">businessTaskId</businessTaskId> </taskIdentifier> […]

    The cleanup command should look like this:

    1176:CUR_g_os_eddf_mc_eagle_extract:1177:PROCSEQID: 

  • OR the value of the corresponding tag 1177 from related pace event (you can just copy it in Automation Center):
    
    In this case the cleanup command should look as follows:

    1176:CUR_g_os_eddf_mc_eagle_extract:1177:MSGCENT:

2. Drop that file to eagle_ml-2-0_default_cm_sequencer_cleanup stream.
3. Wait eagle_ml-2-0_default_cm_sequencer_cleanup stream to complete the cleanup.
4. Done! You can resume Sequencer, it will continue processing the queue with the next sequence number

Reset the entire queue

You should form a message as above, but with ALL_ prefix. 

1176:ALL_[workflow_name]:1177:[POSTFIX]:

Example:

Reset Sequencer using eagle_ml-2-0_default_cm_nuggetizer_recover

1. Go to the Message Center Console/Editor.

2. Find eagle_ml-2-0_default_cm_nuggetizer_recover stream and right click on it

3. Choose Run Now With Overrides:
   

4. In the opened Run Now With Overrides window, right click somewhere in the List Options, press Debug
   

5. And tick Show Hidden Cells:
   

6. Add the following parameters: Mode = Recovery, Option = All Queue and Workflow Name = eagle_wrf_generic_load:

   Open

   

7. Press Ok

The sequencer will be unstuck and will continue processing the queue with the next sequence number.

Sequencer states validation

Check database with the next query:

If you see some record with proc_status='FAILED', you need to change it back to 'ACK':