EagleML Release Notes - February 2020

Enhancements:

SDP-33042: Added the ability to disable rollback for the Warehouse Position and OpenLot if keys file sent for processing

The following was implemented:

Users are able to define if rollback should happen on keys file processing or not
Rollback will execute by default for Warehouse Position and Warehouse OpenLot in xmldbapi format if a keys file provided for processing
Rollback may be disabled if the parameter ‘W_WRHS_EXECUTE_ROLLBACK_SWITCH’ set to ‘N’ in w_config_custom.inc

Files changed:
estar/tpe/dynamic/msgcenter/eagle_default/in/xml/xml-dbdirect_invar_rollup.xml

SDP-32750: Corporate Action Rebook Process was improved

During the rebook process for Corporate Actions, a number of elements were maintaining their original value during the rebook process. Mapping for the following elements was corrected:

Priorities for mapping of the <splitRatio> element were updated: an incoming element <splitRatio> is mapped to the Rebooked Corporate Action record if it is not null.
Priorities for mapping of the <cpCashCurrency> element were updated: an incoming element <cpCashCurrency> is mapped to the Rebooked Corporate Action record if it is not null.
The mapping of <cpOffer> element was added to the Rebooked Corporate Action record.
Fixed error resolving the PIK/PIKBOND Transaction Type.

Files changed:
eagle_default/in/xml/xml-ref_ca.xml

SDP-32581: EagleML Binds will automatically regenerate after Eagle Core Update

Additional functionality was added to the eagle_ml-2-0_default_cm_w_setup stream to validate the Eagle Core version after every Message Center Service restart, regenerate the EagleML binds and restart the Message Center Service. It is a required step going forward as if the binds are not regenerated, then it may lead to functional issues. In case the Eagle Core was not upgraded, nothing will be done.

Files changed:
tpe/dynamic/msgcenter/eagle_mc/deploy/out/tagvalue/eagle_default_deploy.xml

SDP-32336: EntityXref object type was enabled

The EntityXref object type was enabled via Eagleml XSD.

SDP-32932: Added possibility to change the XrefId via the entityXref object type

Rule changes were completed to support the update of existing XrefId via n entityXref object type.

Files changed:
eagle_default/in/xml/xml-ent_entityxref.xml

SDP-32795: Added changes for the load PcPrice in the appropriate leg based upon the Security Processing Type - PST

Rule changes were enhanced to load the PcPrice to the appropriate leg of a multileg SMF based upon processing security type.

When the PST has a contract leg of SWCOTR, the pay or receive leg that does not have the PST of SWLEAC will be priced.
When the PST contract leg is SWCOIR, SWCOCR, SWCDCX or SWCDCO, the contract leg will be priced.

Files changed:
eagle_default/in/xml/include/match_all_sec_resolution.inc
eagle_default/in/xml/include/security_resolution_generic.inc
eagle_default/in/xml/include/xml-ref_security_pricing_common.inc

SDP-32487: eagle_wrf_generic_extract workflow was modified with additional Cache Validation sub-process

The eagle_wrf_generic_extract workflow was modified with the additional sub-process CacheValidation, prior to the extract step.
This step may trigger any workflow enabling each client to have their own workflow for cache validation and reload.
The current version of the workflow may reload ALL caches which are used in the EBS extracts.

There are a few required parameters to control the cache validation sub-process:

EnableValidateCache (RTR) and global parameter W_ENABLE_VALIDATE_CACHE to enable the step prior to extract
CacheValidationWorkflow (RTR) and global parameter W_CACHE_VALIDATION_WORKFLOW which specifies the Name of the workflow which will be triggered via the sub-process
CacheReloadPeriod (RTR) and global parameter W_CACHE_RELOAD_PERIOD which controls how often cache validation will run. The established setup time is in seconds.
Example:
<taskParameter>
<name>EnableValidateCache</name>
<value>Y</value>
</taskParameter>
<taskParameter>
<name>CacheValidationWorkflow</name>
<value>any_workflow_name</value>
</taskParameter>
<taskParameter>
<name>CacheReloadPeriod</name>
<value>2000</value>
</taskParameter>

<CODE>
CreateVariable( 'W_ENABLE_VALIDATE_CACHE','Y' );
CreateVariable( 'W_CACHE_VALIDATION_WORKFLOW','any_workflow_name' );
CreateVariable( 'W_CACHE_RELOAD_PERIOD','3000' );
</CODE>

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_calculate_date.inc
eagle_ml-2-0_cm/out/tagvalue/extract_delta_based_get_date.inc
eagle_ml-2-0_cm/out/xml/task_reporter.xml
eagle_wrf_generic_extract workflow related files

SDP-32485: Post Load Data Quality Check

A Post Load Data Quality Check was added to the profiled generic load processing via the eagle_ml-2-0_default_cm_profiled_load_validation stream. The Post Load step was implemented for the profiled generic load to validate data after a load.

Post Load Data Quality Check will:

Collect source file(s) processing statistics – the number of records in the source files that were recognized on translation, the number of records that were filtered.
Collect translation results statistics - number of records generated during translation step, number of filtered records, number of skipped duplicates, number of records processed on load and number of records failed on load.
Report Status via TSR, either in additionalData node in case of successful validation or in the error description in case of a failed validation.
Perform load validation according to provided validation definitions (use extract_interface and comparison_interface to define interfaces to use for load validation. Use prefix comparison_ or extract_ to pass additional parameters to corresponding steps) and adds validation information to task report.

File added:
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/unpacker/profiled_load_validation.rsf
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/unpacker/profiled_load_validation.xml

SDP-32482: Translation statistics were added to file statistics

The following was implemented:

The File statistics were expanded by the translation statistic and stored in the database PACE_MASTERDBO.ORCH_QUEUE.ORCH_STATE_CLOB to support the creation of the data load validation.
The ORCH_STATE_CLOBE table contains:

statistics for the source file: partition count; count of filtered records before the translation; total batches for the main destination; count of input records; and count of records passed to translation.
statistics for translated files: total translated records; number of batches and filtered records for the destination; translated records for partition; outgoing filename; and count of records in key file for roll-up process.
To enable translation statistics functionality a new parameter should be added to the w_config_custom.inc

Files changed:
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/tagvalue/execute_wrkfl_event.xml
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/parse_translator_stat.inc
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/unpacker/send_to_wrf.xml

SDP-32421: New EagleML Inbound interface - SecurityDefault

Added support of the new object type SecurityDefault to allow the load of data into the SECURITYDBO.SECURITY_DEFAULT.
Inbound interface introduced as a part of the reference loading process.
This load only supports the load of the security default by security or security type. The ability to load by entity / security is out of scope.

Files changed:
eagle_default/in/xml/xml-ref_objects.rsf

New files:
eagle_default/in/xml/xml-ref_security_default.xml

SDP-32061: Added support for the Begin Accrual Date and End Accrual Date fields for Derivative Trades

Support for beginAccrualDate - Tag 2857 and endAccrualDate - Tag 4412 was added in the XSD schema and rules for the following panels:

pan-cdxopen.htm
pan-cdxclose.htm
pan-swcdopen.htm
pan-swcdtermination.htm
pan-crswapopen.htm
pan-crswapclose.htm
pan-irswapopen.htm
pan-irswapclose.htm
pan-trswapopen.htm
pan-trswapclose.htm

Files changed:

csv-buy_columns.inc
csv-conversion_columns.inc
csv-sell_columns.inc
multiplelegtrade_aliases.inc
multiplelegtrade_common_columns.inc
trade_aliases.inc
xml-accttrade_multipleleg.xml

SDP-31843: SFTP Delivery Method was Optimized

The data file was not received from the SFTP server if SSL certificate or SSH remote key were invalid.
Support of the option "--insecure" was added.
To configure, set the SFTP_INSECURE parameter equal to Y in the w_config_custom.inc to bypass / ignore the issue. Default value is N.

File changed:
runtaskrequest.xml

SDP-31578: New Fields added to the XSD Schema and Warehouse Position Extract

The XSD Schema and the Warehouse Position Extract were enhanced.
The following fields are supported now:
originalAcquisitionCostBase
originalAcquisitionCostLocal
originalAcquisitionNotionalCostBase
originalAcquisitionNotionalCostLocal

These fields are now available for the Element Reduction Selection Criteria functionality.

File changed:
eagle_ml-2-0_cm/out/xml/t_warehouse_position_streaming.inc
XSD schema files

SDP-31381: Inbound Interface Future Contract SMF was updated with additional field

A new field, promptDate, was added to the XSD for the SMF Future object:

The corresponding rules were updated to include this new field.

Files changed:

smf_alias_inc
smf_panels.inc
futurecontractsmf_model.inc

SDP-28246: The Warehouse Position Inbound Interface was updated with additional elements

The Interface was only partially updated due to a dependency on a Core Stored Procedure update.
Four new fields were added into the XSD schema for Warehouse Position:
originalAcquisitionCostBase
originalAcquisitionCostLocal
originalAcquisitionNotionalCostBase
originalAcquisitionNotionalCostLocal
These fields were added into the XSLT transformation for future use once the dependent stored procedure is updated

Files changed:
wrh_egl_gen_interface3.xsl

SDP-23727: Issuer Relationship and new processing option - ISSUER_BUILD_COMPLIANT_SWITCH

An additional processing option was introduced for the Issuer Relationship to retain data in compliance with the Issuer Build Engine.
A new processing option ISSUER_BUILD_COMPLIANT_SWITCH provides the ability to identify and delete records for the following KEY: ROLE TYPE + SOURCE + FROM ISSUER if the <fromIssuer> and <toIssuer> are the same in the incoming file or the <toIssuer> is Null.

Files changed:

xml-ref_issuerrelationship.xml

SDP-10604: EJM SFTP delivery method was updated to work with PRIVATEKEY / PUBLICKEY options

Provided support for files distributing via SFTP with PRIVATE KEY and PUBLIC KEY options which previously were unavailable, for example:
<COL TAG="SFTP_PRIVATEKEY" EXPRESSION="'id_rsa'" />
<COL TAG="SFTP_PUBLICKEY" EXPRESSION="'id_rsa.pub'" />

File changed:
reporter_dist_methods.inc

Production Bugs:

SDP-31220: Corrected the issue related to SWAP transaction type logic leading to incorrect updates for PIK,NONTAXMERGR and NONTAXSPINP corporate actions.

An overlap of the transtype swap logic in the panel and in the rule for xml-ref_ca.xml was causing incorrect resolution of existing records in the DB. The logic for resolving existing records was updated for the following transaction types:

PIK
NONTAXMERGR
NONTAXSPINP
TAXSPINP

Files changed:
xml-ref_ca.xml

SDP-29940: Corrected Mapping Issue for Corporate Action Exchange Offers

Logic to calculate the Treatment of Interest Indicator - Tag 1709 based on the resolved PST’s per the from security - Tag 2300 and the to security - Tag 2301 was added to accommodate additional condition 9153 != 'MC_MODE' in the Corporate Actions Announcements - Exchange Offers panel.

Files changed:

eagle_default/in/xml/xml-ref_ca.xml

SDP-32968: Fixed OutputFormat parameter overwrite for Warehouse objects in Generic Extract Workflow

Corrected an issue when the W_GEN_EXTRACT_OUTPUT_FORMAT global parameter was overriding the OutputFormat parameter from the profile or control message.

The change now sets the profile or RTR parameter to have a higher priority overwriting the global parameter setting.

Files changed:
eagle_ml-2-0_workflows/eagle_wrf_generic_extract.wrf
eis/ejmsln/eagle_wrf_generic_extract.ejmsln
eis/ejmwf/eagle_wrf_generic_extract.ejmwf