Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 6 Next »

The eagle_wrf_cloud_extract workflow was created to execute ebs extracts periodically.

The workflow contains one ExecuteCM task only. The task runs eagle_wrf_generic_extract_mc2 workflow with following pre-selected parameters:

ebs:default:DeltaBased=Y   - to extract delta only

ebs:default:DeltaBasedUID=AZURE_CLOUD – unique uid for delta

ebs:default:OutputFormat=EagleJSON   - to extract in EagleJson format

ebs:default:TargetSinkType=azureblob  - to publish extract data to azure

FeedType=ebs:warehouseposition,ebs:warehouseperformance,ebs:warehousenav,ebs:exchangeprice,ebs:smf,ebs:genericentity,ebs:codevalue,ebs:issuefxrate,ebs:warehousetrade,ebs:issueprice,ebs:errordefinition

FeedType  and ebs:default:TargetSinkType Values can be overridden by the following variables in w_config_custom.inc:

        w_ebs_cloud_interfaces

       w_ebs_cloud_targetsink


For instance to extract smf interface only and to disable distribution to azure we should define the parameters as shown below:

<CODE>

   :w_ebs_cloud_interfaces: := 'ebs:smf';

   :w_ebs_cloud_targetsink: := ' ';

</CODE>


 The workflow can be executed periodically using schedule rule/events.

How to add new interface.

For example you have to add new interface mycustomsmfextract.

First way.

We can override default feedtype value using w_ebs_cloud_interfaces variable.

  • Open tpe\dynamic\msgcenter\eagle_ml-2-0_custom_cm\w_config_custom.inc
  • Add/modify w_ebs_cloud_interfaces value

<CODE>

   :w_ebs_cloud_interfaces: := 'ebs:smf,ebs:mycustomsmfextract';

</CODE>

  • Save the file
  • Re-init classic MC cache

Second way.

We can add the new interface in IWS

  • Open eagle_wrf_cloud_extract in IWS
  • Expand CustomCMParams for execute_ebs_extracts

  • Add the interface at the end of value before last single quote

  • Save &Publish solution
  • Re-init classic MC cache
  • To save changes in bitbucket , commit the following files:

                   tpe\dynamic\msgcenter\eagle_ml-2-0_workflows\eagle_wrf_cloud_extract.wrf

                   tpe\dynamic\msgcenter\eis\ejmwf\eagle_wrf_cloud_extract.ejmwf

How to add new task parameter for EBS extracts.

Adding common parameter for all ebs extracts.

Common parameter name has ‘ebs:default:’ prefix.

For instance we have to reduce extracts and should apply maxrows=100 to all ebs extracts.

  • Open eagle_wrf_cloud_extract in IWS
  • Expand CustomCMParams for execute_ebs_extracts
  • Add a new parameter with name ebs\5C3Adefault\5C3Amaxrows and value 100. Please note: IWS doesn’t replace colon with \5C3A for param names automatically, so we should write \5C3A instead of colon in parameter name.

 

  • Save &Publish solution
  • Re-init classic MC cache
  • To save changes in bitbucket , commit the following files:

                 tpe\dynamic\msgcenter\eagle_ml-2-0_workflows\eagle_wrf_cloud_extract.wrf

                 tpe\dynamic\msgcenter\eis\ejmwf\eagle_wrf_cloud_extract.ejmwf

Adding ebs extract specific parameter.

Extract specific parameter name has ‘ebs:[interface name]:’ prefix.

For instance we have to reduce mycustomsmfextract by applying maxrows=100 to the extract.


  • Open eagle_wrf_cloud_extract in IWS
  • Expand CustomCMParams for execute_ebs_extracts
  • Add a new parameter with name ebs\5C3Amycustomsmfextract\5C3Amaxrows and value 100. Please note: IWS doesn’t replace colon with \5C3A for param names automatically, so we should write \5C3A instead of colon in parameter name.

  • Save &Publish solution
  • Re-init classic MC cache
  • To save changes in bitbucket , commit the following files:

                    tpe\dynamic\msgcenter\eagle_ml-2-0_workflows\eagle_wrf_cloud_extract.wrf

                    tpe\dynamic\msgcenter\eis\ejmwf\eagle_wrf_cloud_extract.ejmwf

Using Profile to run the workflow

To support custom parameters we decided to use workflow profile for eagle_wrf_cloud_extract workflow.

The following parameters can be set in profile:

FeedType – the list of ebs interfaces separated by comma

OutputFormat  - default value is EagleJSON

TargetSinkType  - default value is azureblob

ExcludeZero  - default value is Y

Includenull   - default value is Y

To create new profile please open Monitoring Tool, go to “Activities” tab and press Add Profile button:

Fill the appeared form as shown below

Action – Execute Workflow

Workflow – eagle_wrf_cloud_workflow

Task Parameters: - the parameters to change

Then press Save Profile As.. button and save profile

The saved path will be shown in Profile field:

To use the profile in workflow execution by schedule you should add in Automation Center to schedule rule

1166 tag  with profile name(E.g. eagle_wrf_cloud_extract_mlperf.profile). For mc versions  2017 and higher you can add parameter ProfileName instead of 1166 tag.


 





  • No labels