EagleML Release Notes - March 2020

Owned by Olga Galis (Deactivated)
Last updated: yesterday at 4:14 pm by Kinga Huszno
6 min read

Enhancements:

SDP-33185: Enhanced Signage Logic for Warehouse Trade Extract

Signage logic was enhanced to support additional fields: bookValue and bookValueLocal.
Signage logic is applied only when the RTR parameter EnableSignageLogic is set to Y or
the global parameter W_ENABLE_SIGNAGE_LOGIC is set to Y.
If the value is N or null, the numeric fields are presented as stored in the database. If the value is Y, the identified and documented numeric fields have their own algorithm for signing the values.

<taskParameter>
<name>EnableSignageLogic</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_wrhs_trade_signage_logic.inc

SDP-33800: Introduced a global parameter to control the signage logic for all extracts where this functionality is supported

A new global parameter, W_ENABLE_SIGNAGE_LOGIC, was introduced to control signage logic across an environment. It may be set in the w_config_custom.inc setting the value to Y. The Global parameter will be overwritten by a local parameter from the RTR / control message.

Example of usage: <COL TAG="W_ENABLE_SIGNAGE_LOGIC" EXPRESSION="'Y'"/>

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_acct_settled_signage_logic.inc
eagle_ml-2-0_cm/out/tagvalue/extract_acct_traded_signage_logic.inc
eagle_ml-2-0_cm/out/tagvalue/extract_wrhs_trade_signage_logic.inc

SDP-33656: Translation CorrelationId was added to the trigger file for profiled workflow

The trigger file that starts the translation process for the profiled workflow included the correlationId of unpacker but not the correlationId for the translation process.

The logic was adjusted to store the correlationId of the unpacker as the OriginatingCorrelationId and write the correlationId of the translator to the trigger file.

Files changed:
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/unpacker/custom_routing.inc

SDP-33645: Enhanced Resiliency of Transform Pairs include file

In the past, if changes made for the transform_pairs.inc and the sequence of a number of transforms was broken, then the files for transformation were not be picked up by a translator. The issue was corrected, going forward to enable files to be transformed and loaded even if the transform_pairs.inc is missing some pairs.

Files changed:
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/unpacker/custom_routing.inc

SDP-33607: Introduced the Effective Date filter for the Issuer ESG outbound interface

Added support for the Effective Date to the Issuer ESG outbound interface. The filter by Effective Date may be used with the historyonlyflag only.

File changed:
eagle_ml-2-0_cm/out/tagvalue/extract_issuer_esg.inc

SDP-33432: New EagleML FeedType SECURITYDEFAULTEXTRACT

A new outbound interface was added to extract data from the SECURITYDBO.SECURITY_DEFAULT table. The feedtype is defined as SECURITYDEFAULTEXTRACT.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_ref_security_default.inc
eagle_ml-2-0_cm/out/tagvalue/extract_ref_security_default.xml
eagle_ml-2-0_cm/out/tagvalue/extracts.rsf
eagle_ml-2-0_cm/out/xml/t_eagleml-2-0_streaming.rsf
eagle_ml-2-0_cm/out/xml/t_ref_security_default_streaming.inc
eagle_ml-2-0_cm/out/xml/t_ref_security_default_streaming.xml
eagle_ml-2-0_cm/w_config.inc

SDP-32796: SecurityDefault Interface was enhanced to support Update and Delete

The SecurityDefault Interface was enhanced by adding Update and Delete support logic.

Files changed:
metadata/bind/eagleml_ref_security_default.xml
eagle_default/in/xml/xml-ref_security_default.xml

SDP-33825: The SWAGGER File was updated with additional feedtypes - SECURITYDEFAULTEXTRACT and GLOBALTAXRATEEXTRACT

The Feedtypes, SECURITYDEFAULTEXTRACT and GLOBALTAXRATEEXTRACT, were added to the SWAGGER file.

Files changed:
eagleml/swagger-extracts.json

SDP-33353: EJM Monitoring functionality was enhanced to support Production Parallel Autorecovery System

Added support of the dkm_autorecovery system and single dkm_monitoring execution.
The eagle_ml-2-0_default_cm_monitoring stream is now able to use the dkm_autorecovery system and may be launched in the dkm_monitoring mode only. Sender Policy Framework (SPF) profiles, which allows clients to set the type of notification, list of the workflows to check, some specific variables like thresholds, periods and emails are excluded from the default task monitoring system. The dkm_monitoring and dkm_autorecovery systems are turned off by default.

This functionality is optional and is controlled by a parameter within the w_config_custom.ini.
<!—Monitoring enable/disable variables. Y for enable, N for disable -->
<COL TAG="W_DKM_MONITORING_ENABLE" EXPRESSION="'Y'"/>
<!—Autorecovery enable/disable variables. Y for enable, N for disable -->
<COL TAG="W_DKM_AUTORECOVERY_ENABLE" EXPRESSION="'Y'"/>

Files added:
eagle_ml-2-0_cm/workflow_states_query.inc

Files changed:
eagle_ml-2-0_cm/out/xml/monitoring.xml
eagle_ml-2-0_cm/out/xml/read_task_info.inc
eagle_ml-2-0_cm/out/xml/read_workflow_profiles.inc
eagle_ml-2-0_custom_cm/w_config_custom_example.inc

SDP-33348: Security Resolution logic for the Use First As Main with Match All was simplified

Simplified the security resolution process. Before the change, all additional options that were specified in the Asset Resolution Option based on the Batting Order (e.g. XREF_EXCHANGE, ASSET_CURRENCY, MATURITY_DATE, NRA_TAX_COUNTRY) were included to resolve the Security Alias on Step 1.
Step 1 - The existing security resolution logic for mode Use First As Main + Match All was updated. After the fix, resolution does not include any additional options: The Security Alias is resolved by the 1st pair of Xref Id + Xref Type.
Step 2 - The security resolution was not changed and is still applicable if the Security Alias is not resolved by pair 1, it tries to resolve the Security Alias by pairs 2 – 10 of Security Xreference Identifiers + all additional options

File changed:
eagle_default/in/xml/security_resolution_generic.inc

SDP-33184: Improved the filtering logic for the SPDDELTATRIGGERLOGEXTRACT when the minAccountingDateFlag is used

The filtering logic was modified to calculate the Minimum Accounting Date based on the records calculated by the DeltaBased filter.
The same filter is used to calculate the Minimum Accounting Date within the whole table or different date range when the minAccountingDateFlag is used.
Support of additional date filters including the updatedate, fromdate, and todate were improved applying the same logic. This sequence change ensures that the update date for the delta records was applied prior to identifying the Minimum Accounting Date and other related date filters.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_spd_delta_trigger_log.inc

SDP-32999: ExcludeSecurityAlias logic was improved for the GenericSMF Outbound Interface

The logic for the ExcludeSecurityAlias parameter was modified to support this parameter for the Underlying Security Alias and correct a mapping issue for the main Security Alias. When applied, it disables the extraction of the security alias in the Asset Identifiers model.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_smf.inc
eagle_ml-2-0_cm/out/xml/t_smf_streaming.inc
eagle_ml-2-0_cm/out/xml/t_smf_streaming_eml.inc

SDP-32979: GenericEntity Inbound Interface was updated to support additional elements

New fields were added to the GenericEntity object. The following fields are now supported by the inbound interface:

RULESDBO.ENTITY_STAR_PROCESSING.BUS_PROCESS_GRP
RULESDBO.ENTITY_STAR_PROCESSING.EXPIRY_DATE_PROC_FLAG
RULESDBO.ENTITY_STAR_PROCESSING.EXP_REIMB_TIER_RULE_2
RULESDBO.ENTITY_STAR_PROCESSING.FOF_ALLOCATION_RULE
RULESDBO.ENTITY_STAR_PROCESSING.ILB_VALUATION_METHOD
RULESDBO.ENTITY_STAR_PROCESSING.INTRADAY_NAV_ELECTION
RULESDBO.ENTITY_STAR_PROCESSING.MINIMUM_BALANCE
RULESDBO.ENTITY_STAR_PROCESSING.NASDAQ_INST_TYPE
RULESDBO.ENTITY_STAR_PROCESSING.NAV_RULE
RULESDBO.ENTITY_STAR_PROCESSING.PREF_SHARES_PROCESSING
RULESDBO.ENTITY_STAR_PROCESSING.PRORATA_CA_CLOSE_FLAG
RULESDBO.ENTITY_STAR_PROCESSING.RELEASE_PRO_PLG
RULESDBO.ENTITY_STAR_PROCESSING.SPECIAL_WAIVER
RULESDBO.ENTITY_STAR_PROCESSING.VOL_RESTRICT_PROC
RULESDBO.ENTITY.MIL_RATE_ROUNDING
RULESDBO.ENTITY.NUMBER_OF_MIL_RATE_DIGITS
RULESDBO.ENTITY.MONEY_MARKET_TYPE
RULESDBO.ENTITY.FINAL_NPORT_FILING_DATE
RULESDBO.ENTITY.EXCHANGE
RULESDBO.ENTITY.PROSPECTUS
RULESDBO.ENTITY.ENTITY_FUND_TYPE
RULESDBO.ENTITY_ACCT_BASIS.REGULATORY_CAT_EFF_DATE
RULESDBO.ENTITY_ACCT_BASIS.DERV_FEE_RULE_ID
RULESDBO.ENTITY_ACCT_BASIS.SPD_DATE_TYPE
RULESDBO.ENTITY_ACCT_BASIS.INC_POST_TREATMENT
RULESDBO.ENTITY_ACCT_BASIS.IO_INC_AMORT_FLAG
RULESDBO.ENTITY_ACCT_BASIS.MTM_ACCOUNTING_FLAG
RULESDBO.ENTITY_ACCT_BASIS.MTM_INT_DIV_INC_ELECTION

Files changed:
metadata/bind/eagleml_ent_genericentity.xml
eagle_default/in/include/entity_acct.inc
eagle_default/in/include/entity_aliases.inc
eagle_default/in/include/entity_panels.inc
eagle_default/in/xml/xml-ent_entitygeneric.xml

SDP-32738: Generic Entity Outbound Interface was updated to support additional elements

The ENTITYEXTRACT was updated to support additional elements:

RULESDBO.ENTITY_STAR_PROCESSING.ILB_VALUATION_METHOD
RULESDBO.ENTITY_STAR_PROCESSING.NASDAQ_INST_TYPE
RULESDBO.ENTITY_ACCOUNT_BASIS.PRORATA_CA_CLOSE_FLAG
RULESDBO.ENTITY_ACCOUNT_BASIS.IO_INC_AMORT_FLAG
RULESDBO.ENTITY_ACCOUNT_BASIS.MTM_INT_DIV_INC_ELECTION
RULESDBO.ENTITY.MIL_RATE_ROUNDING
RULESDBO.ENTITY.NUMBER_OF_MIL_RATE_DIGITS

Files changed:
eagle_ml-2-0_cm/out/xml/t_entity_streaming.inc
eagle_ml-2-0_cm/out/xml/t_entity_streaming_eml.inc

SDP-32489: The eagle_wrf_generic_extract workflow was enhanced with an additional post extract sub-process

The eagle_wrf_generic_extract workflow was modified with the additional sub-process post extract step.
This step may trigger any workflow which enables each client the ability to have their own workflow for data quality validation.

There are two required parameters to control the cache validation post extract sub-process and the parameters may be passed via an RTR or set as a global parameter.

Global Parameters:
W_ENABLE_POST_EXTRACT_VALIDATION – This option enables the step. Required parameter.
W_POST_EXTRACT_VALIDATION_WORKFLOW - The Name of the workflow which will be triggered in this step. Required parameter.

RTR Example:
<taskParameter>
<name>EnablePostExtractValidation</name>
<value>Y</value>
</taskParameter>
<taskParameter>
<name>PostExtractValidationWorkflow </name>
<value>any_workflow_name</value>
</taskParameter>

w_config_custom.ini settings:
<CODE>
CreateVariable( 'W_ENABLE_POST_EXTRACT_VALIDATION ','Y' );
CreateVariable( 'W_POST_EXTRACT_VALIDATION_WORKFLOW ','any_workflow' );
</CODE>

Files changed:
eagle_wrf_generic_extract workflow related files

SDP-31569: Option to overwrite the SFTP_FILES_LOCATION within the Control Message

Incoming files were only able to be setup via configuration file and settings to define the FTP/SFTP location.
A new RTR parameter, CustomInboundLocation, is now supported to provide additional flexibility to the end users.

Example of RTR parameter:
<taskParameter>
<name>CustomInboundLocation</name>
<dataType>S</dataType>
<value>CMW/inbound_custom/</value>
</taskParameter>

File changed:
runtaskrequest.xml
collect_task_parameters.inc

SDP-31304: Order by added to sort the rows in the result set for TRADEDCAEXTRACT

To maintain consistency, the Order By was added to the TRADEDCAEXTRACT to produce the same sort order per execution. The default Order By is based on the ESTAR_TRADED_CASH_ACTIVITY.INSTANCE field.

Files changed:
extract_acct_traded_cash_activity.inc

SDP-31086: Warehouse Performance Inbound Interface was updated to Support PSR_ANALYTICS Table on Load

The core stored procedure, PERFORMDBO.EGL_GEN_PERFORM.PRF_EGL_GEN_INTERFACE, was updated to support the load of additional fields into the database table PERFORMDBO.PSR_ANALYTICS.
The two new fields supported via WarehousePerformance Inbound Interface are netAssetValue and notionalMarketValue.

The changes are available starting from the v17R2.19 Eagle Core release.

Files changed:
wrhs_performance.xsl
xml-dbdirect_wrhs_delete_statement.inc
extract_warehouse_performance.inc
t_warehouse_performance_streaming.inc

SDP-30102: GenericSMF Derivatives Model was extended with additional fields

The GenericSMF XSD, Inbound and Outbound rules were updated to provide support for new fields. The following fields were added to the Inbound Interface:

SECURITYDBO.DERIVATIVES_EXT.NATIONAL_COUPON
SECURITYDBO.DERIVATIVES_EXT.NOTIONAL_MAT_TERM
SECURITYDBO.DERIVATIVES_EXT.CONTRACT_EXPIRATION_DATE
SECURITYDBO.DERIVATIVES_EXT.CTD_COUPON
SECURITYDBO.DERIVATIVES_EXT.CTD_MAT_DATE
SECURITYDBO.DERIVATIVES_EXT.GENERIC_CONTRACT_MONTH
SECURITYDBO.DERIVATIVES_EXT.CTD
SECURITYDBO.DERIVATIVES_EXT.HEDGER_MARGIN_MIN
SECURITYDBO.DERIVATIVES_EXT.SPEC_MARGIN_MIN
SECURITYDBO.DERIVATIVES_EXT.NOTICE_FIRST_DAY
SECURITYDBO.DERIVATIVES_EXT.NOTIONAL_BOND
SECURITYDBO.DERIVATIVES_EXT.NOTIONAL_MAT_DATE
SECURITYDBO.DERIVATIVES_EXT.ROLL_DATE
SECURITYDBO.DERIVATIVES_EXT.SPEC_MARGIN_BALANCE_MIN
SECURITYDBO.DERIVATIVES_EXT.INT_RATE_END_DATE
SECURITYDBO.DERIVATIVES_EXT.INT_RATE_START_DATE
SECURITYDBO.DERIVATIVES_EXT.INT_RATE_DAY_CNT_DESC
SECURITYDBO.DERIVATIVES_EXT.AGREEMENTS_OUTSTANDING
SECURITYDBO.DERIVATIVES_EXT.FUT_VALUATION_PT
SECURITYDBO.DERIVATIVES.RATE.CONVERSION_RULE
SECURITYDBO.DERIVATIVES.DERV_CHAR26

The Outbound rules were updated to support two new fields:

SECURITYDBO. DERIVATIVES.RATE_CONVERSION_RULE
SECURITYDBO. DERIVATIVES. DERV_CHAR26

Files changed:
eagleml-sec-shared-2-0.xsd
smf_egl_gen_interface_template.xsl
smf_egl_gen_interface_template_mssql.xsl
t_smf_streaming_eml.inc
smf_aliases.inc
derivative_ext.inc
generic_smf_base_model.inc

SDP-26851: New Outbound Interface GLOBAL TAX RATE EXTRACT

A new extract, GLOBALTAXRATEEXTRACT, was implemented. It is also supported in the generic extract workflow.

Supported filters:
startdate
enddate
distributiontype
entitycountrycode
issuecountrycode
entitytaxqualifier
entitytaxtype
issuetaxtype
overrideindicator
primaryexchangecode

If the startdate or enddate filters are not part of RTR, then all the records are extracted.

Files added:
eagle_ml-2-0_cm/out/tagvalue/extract_globaltaxrate.inc
eagle_ml-2-0_cm/out/tagvalue/extract_globaltaxrate.xml
eagle_ml-2-0_cm/out/xml/extract_service_globaltaxrate.xml
eagle_ml-2-0_cm/out/xml/t_globaltaxrate_streaming.inc
eagle_ml-2-0_cm/out/xml/t_globaltaxrate_streaming.xml

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extracts.rsf
eagle_ml-2-0_cm/out/xml/extract_service_start.inc
eagle_ml-2-0_cm/out/xml/t_eagleml-2-0_out_extract_service.rsf
eagle_ml-2-0_cm/out/xml/t_eagleml-2-0_streaming.rsf
eagle_ml-2-0_cm/w_config.inc

SDP-20743: Added support for the NO_DATA status into the final TSR for the non-extract process

The status NO_DATA was only supported for extracts. The TSR logic was improved to support the status for the non-extract processes like load. The following description is available via the TSR for the extract: No Data returned as result of extract

The following description is part of the TSR for non-extracts: No Data found. Possibly the file was not processed

File changed:
eagle_ml-2-0_cm/w_config.inc
eagle_ml-2-0_cm/out/xml/task_reporter_gen_step_summary.inc

SDP-9641: Order by added to sort the rows in the result set for UNSETTLEDTRANSACTIONSEXTRACT

The order by was added for the UNSETTLEDTRANSACTIONSEXTRACT. The default order by is based on the record key.

Files changed:
extract_acct_unsettled_transactions.inc

Production Bugs:

SDP-33623: EJM Alerts Issue was resolved

EJM Alerts were not delivered due to the wrong stream name. The stream package was corrected to point to the correct stream which is eagle_ml-2-0_default_cm_eas_distribution.

File changed:
eagle_ml-2-0_default_cm_eas_distribution.pkg

SDP-33318: Corrected the TriggeredWorkflowCorrId in the file statistics for the bundle based profiled workflow

The BundleTrigger file for the profiled workflow overwrote the correlationId for the eagle_wrf_generic_load_profile_executor workflow while generating the control message. The new instance of the profiled workflow was added to store the initial correlationId as the TriggeredWorkflowCorrId parameter before publishing it to external data.

Files changed:
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/ndfa/launch_load_by_profile.inc
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/ndfa/ndfa_exec.xml
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/run_wrf_instance.inc

SDP-33144: Incorrect Status for externalData event was corrected

The externalData event remained In Progress from Automation Center if triggered via the Profiled Generic Load after the file was loaded successfully and the workflow was completed.
The logic for the Profiled Generic Load Workflow was adjusted. The status for the externalData event was corrected in Automation Center when the extended file statistic is used with and without the profiled generic load workflow.

The Event Status is in sync between AutomationCenter and in EJM Monitoring Tool.

Files changed:
eagle_ml-2-0_cm/ndfa/ndfa_exec.xml
eagle_ml-2-0_cm/unpacker/send_to_wrf.xml
eagle_ml-2-0_workflows/eagle_wrf_generic_load_profile_executor.wrf
eis/ejmwf/eagle_wrf_generic_load_profile_executor.ejmwf

SDP-22710: The eagle_wrf_generic_delete workflow was updated

The delete process was improved for the IssuePcPrice and ExchangePrice feed types. The logic of using priceExchangeCode from incoming message was modified. If the priceExchangeCode is part of the incoming message, it is used as the EXCHANGE_CODE for the delete. In case it is not passed, the EXCHANGE from the corresponding SMF is used as the EXCHANGE_CODE.
The ExchangePrice may be deleted via the eagle_ml-2-0_default_in_xml_reference message stream if the action DELETE is set in the referenceTracsaction header section.

File added:
eagle_default/in/xml/include/ref_exchangeprice_delete.inc

Files changed:
eagle_default/in/xml/include/parse_entity_xrefs.inc
eagle_default/in/xml/t_eagle_ml-2-0_default_in_xml_delete_exchange_price.
eagle_default/in/xml/xml-ref_delete.xml
eagle_default/in/xml/xml-ref_objects.rsf

Â