Manage the Eagle WRF Cloud Extract Workflow

The eagle_wrf_cloud_extract workflow was created to periodically execute EBS extracts.

WRF Workflow Default Parameters

The workflow contains one ExecuteCM task only. The task runs eagle_wrf_generic_extract_mc2 workflow with the following default parameters:

Parameter

Default Value

Parameter

Default Value

ebs:default:DeltaBased

Y   to extract delta only

ebs:default:DeltaBasedUID

AZURE_CLOUD  unique uID for delta

ebs:default:OutputFormat

EagleJSON   - to extract in EagleJson format

ebs:default:TargetSinkType

azureblob  - to publish extract data to azure

DisableDistributionStep

Y - to skip distribution step in mc2 workflow

FeedType

ebs:warehouseposition, ebs:warehouseperformance, ebs:warehousenav, ebs:exchangeprice,ebs:smf,ebs:genericentity, ebs:codevalue, ebs:issuefxrate,ebs:warehousetrade, ebs:issueprice,ebs:errordefinition

The values for FeedType and ebs:default:TargetSinkType can be overridden by the following variables in the w_config_custom.inc file:

       w_ebs_cloud_interfaces

       w_ebs_cloud_targetsink

For instance, to extract SMF interface only and to disable distribution to Azure, define the parameters as shown below:

<CODE>    :w_ebs_cloud_interfaces: := 'ebs:smf';    :w_ebs_cloud_targetsink: := ' '; </CODE>

The workflow can be executed periodically using schedule rule/events.

Add a New Interface in IWS

There are two methods for adding a new interface such as an interface called mycustomsmfextract.

  • Method One: Override the Default Feed type Value

  • Method Two: Add a New Interface in IWS

To override the default feedtype value using the w_ebs_cloud_interfaces variable.

  1. Open tpe\dynamic\msgcenter\eagle_ml-2-0_custom_cm\w_config_custom.inc

  2. Add/modify w_ebs_cloud_interfaces value.

    <CODE>    :w_ebs_cloud_interfaces: := 'ebs:smf,ebs:mycustomsmfextract'; </CODE>

     

  3. Save the file.

  4. Re-initialize the classic MC cache.

To add the new interface in IWS:

  1. Open eagle_wrf_cloud_extract in IWS

  2. Expand CustomCMParams for execute_ebs_extracts

  3. Add the interface at the end of value before last single quote.

  4. Save and Publish the solution.

  5. Re-initialize the classic MC cache.

  6. To save changes in Bitbucket, commit the following files:
    tpe\dynamic\msgcenter\eagle_ml-2-0_workflows\eagle_wrf_cloud_extract.wrf
    tpe\dynamic\msgcenter\eis\ejmwf\eagle_wrf_cloud_extract.ejmwf

Add New Task Parameter for EBS Extracts

The common parameter name has ‘ebs:default:’ prefix. For example, to reduce extracts, apply maxrows=100 to all EBS extracts.

To add new task parameter for EBS extracts:

  1. Open eagle_wrf_cloud_extract in IWS.

  2. Expand CustomCMParams for execute_ebs_extracts.

  3. Add a new parameter with the name ebs\5C3Adefault\5C3Amaxrows and value of 100.
    Note: IWS does not replace the colon with \5C3A for parameter names automatically, so you must write \5C3A in place of colon in the parameter name.

  4. Save and publish the solution.

  5. Re-initialize the classic MC cache.

  6. To save changes in Bitbucket, commit the following files:
    tpe\dynamic\msgcenter\eagle_ml-2-0_workflows\eagle_wrf_cloud_extract.wrf
    tpe\dynamic\msgcenter\eis\ejmwf\eagle_wrf_cloud_extract.ejmwf

Add EBS Extract Specific Parameter

Extract specific parameter name has ‘ebs:[interface name]:’ prefix. For instance, you can reduce mycustomsmfextract by applying maxrows=100 to the extract.

To add EBS extract specific parameter:

  1. Open eagle_wrf_cloud_extract in IWS.

  2. Expand CustomCMParams to execute_ebs_extracts.

  3. Add a new parameter with the name ebs\5C3Amycustomsmfextract\5C3Amaxrows and a value of 100.
    NOTE: IWS does not replace the colon with \5C3A for parameter names automatically, so you must write \5C3A in place of the colon in the parameter name.

  4. Save and publish the solution.

  5. Re-initialize the classic MC cache.

  6. To save changes in Bitbucket, commit the following files:
    tpe\dynamic\msgcenter\eagle_ml-2-0_workflows\eagle_wrf_cloud_extract.wrf
    tpe\dynamic\msgcenter\eis\ejmwf\eagle_wrf_cloud_extract.ejmwf

Use Profile to Run the Workflow

To support custom parameters, MC2 uses the workflow profile for eagle_wrf_cloud_extract workflow.

To use profile to run the workflow:

The following parameters can be set in the profile:

Parameter

Value

Parameter

Value

interface

The list of ebs interfaces separated by comma

OutputFormat 

Default value is EagleJSON

TargetSinkType 

Default value is azureblob

ExcludeZero 

Default value is Y

Includenull  

Default value is Y

DisableDistributionStep

Default value is Y

  1. To create a new profile, open Monitoring Tool and select the Activities tab.

  2. Click the Add Profile button.

  3. Complete the appeared form as shown below.

  4. Select Save Profile As to save the profile.
    You see the saved path in the Profile field.

  5. Action – Execute Workflow
    Workflow – eagle_wrf_cloud_workflow
    Task Parameters - the parameters to change

  6. To use the profile in workflow execution by schedule, add in Automation Center to schedule the rule.
    1166 tag  with profile name, for example, eagle_wrf_cloud_extract_mlperf.profile.
    For MC versions 2017 and higher, you can add the parameter ProfileName instead of 1166 tag.