EagleML Release Notes - October 2019

Enhancements:

SDP-28462: The Inbound Interface for Accounting Asset Cash was updated to support Accounting Cash Miscellaneous Expenses

The Inbound Interface for the Accounting Asset Cash was updated.  The AMISCEXP, assetCashType='AMISCEXP', is now supported.
The logic for tag85 and mapping for tag5363 was corrected. Users can now specify the Settlement Currency separately from the asset currently via the RTR.

File changed:
eagle_default\in\include\csv-cash_misc_inc_exp.inc

SDP-28432: Enhanced Security Resolution logic for the Warehouse Broker Trade Quotes inbound interface when a security with the same XREF identifiers and different currencies is processed

Following improvements were made to tradeQuotes:

• Implemented the Generic EagleML Security resolution to the inbound interface
• Implemented the Generic EagleML Entity resolution to the inbound interface
• Added support of the tradeQuotes extract for MSSQL
• Added the assetIdentifiers model to outbound interface

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_ref_trade_quotes.inc
eagle_ml-2-0_cm/out/tagvalue/extract_ref_trade_quotes.xml
eagle_ml-2-0_cm/out/xml/t_eagleml-2-0_streaming.rsf
eagle_ml-2-0_cm/out/xml/t_ref_trade_quotes_streaming.inc
eagle_ml-2-0_cm/w_config.inc
eagle_default/in/xml/xml-ref_tradequotes.xml

File removed:
eagle_default/in/xml/include/tradequote.inc

SDP-28360: EagleML Inbound Interface for Price Classification

Introduced the capability to add price classification values to be loaded via the stream eagle_ml-2-0_in_xml_reference into the SECURITYDBO.SMF_PRICE_CLASSIFY table.  The Security. Generic Load workflow supports the new interface. The Inbound rule is supported for Oracle and MSSQL platforms.

Files changed:
metadata\bind\ eagleml_ref_priceclassification.xml
eagle_default\in\xml\eagleml_reference.xml
xml-ref_price_classification.xml
xml-ref_objects.rsf

SDP-27846: The logic for the calcTransactionId and calcCancelTransactionId was modified in the Warehouse Trade Extract - WRHSTRADEEXTRACT

The fields calcTransactionId and calcCancelTransactionId are concatenated with starTag26 when the parameters EnhanceCalcTransactionId = Y and IncludedExtendedDetails = Y or ALL. The logic of these fields was modified as following: starTag26 is now applied only if the recordType of the trade is 350 or 450 when the parameters EnhanceCalcTransactionId = Y and IncludedExtendedDetails = Y or ALL.

Files changed:
extract_warehouse_trade.inc

SDP-27841: Fields buysell and amortizationParValue were added into the IncludeExtendedDetails mode

Two new fields were added to the IncludeExtendedDetails extract mode:

SECURITYDBO.FIXED_INCOME.AMORTIZATION_PAR_VALUE - amortizationParValue
SECURITYDBO.DERIVATIVES.BUYSELL- buysell

In order to extract these fields, IncludeExtendedDetails should be set as ALL.

<taskParameter>
<name>IncludeExtendedDetails</name>
<dataType>S</dataType>
<value>ALL</value>
</taskParameter>

Files changed:

extract_asset_properties_fields.inc
t_asset_properties_fields.inc
t_asset_properties_fields_start.inc
t_asset_properties_fields_streaming.inc

SDP-27767: Additional fields added to GenericSMF Data Model

Three new fields were added to the XSD for the GenericSMF object:

SECURITYDBO.SECURITY_MASTER_EXT.FINAL_RATE_ROUND_DIR
SECURITYDBO.SECURITY_MASTER_EXT.FINAL_RATE_ROUND_PREC
SECURITYDBO.SECURITY_MASTER_EXT.PROJECT_RATE

The Inbound and outbound rules were updated to support these new fields.

Files changed:
eagleml-sec-shared-2-0.xsd 
smf_egl_gen_interface_template.xsl
smf_egl_gen_interface_template_mssql.xsl
t_smf_streaming_eml.inc

SDP-27290: I2I Interface was improved to retrieve TSR and RTR messages

A new rule, i2i_control_messages.rsf, was created to read both messages (RTR and TSR) instead of having two separate rulesets t_i2i_message_distribution.xml (OUT/XML) and control_messages.rsf(IN/XML).
As a result, a single JMS inbound stream may process incoming TSR’s and RTR’s.
A new rule, i2i_message_distribution.xml (IN/XML), was created and common includes which are used in both rules i2i_message_distribution.xml and t_i2i_message_distribution.xml.

Files changed:

eagle_ml-2-0_cm/in/xml/i2i_control_messages.rsf
eagle_ml-2-0_cm/in/xml/i2i_message_distribution.xml
eagle_ml-2-0_cm/out/xml/t_i2i_message_distribution.xml
eagle_ml-2-0_cm/out/xml/t_i2i_message_distribution_columns.inc
eagle_ml-2-0_cm/out/xml/t_i2i_message_distribution_dist.inc
eagle_ml-2-0_cm/out/xml/t_i2i_message_distribution_resmsgcolumns.inc

SDP-26502: Two new processing options introduced for IssuerRole batch mode

Two new processing options introduced for Issuer Role batch:

ISSUE_ROLE_UPSERT_MODE - this option disables delete steps (both core table and history) during batch load process for the Issuer Role.
USE_LATEST_START_DATE - this option allows detecting the highest start date among incoming issuer roles and sets it for each role during load. Such an approach allows period-by-period update logic.

Files changed:
xml-ref_issuerrole_batch.xml

SDP-26467: Added new EagleML object - Global Tax Rates

Added support via EagleML to load, update and delete Global Tax Rates.  This interface supports only Inbound rules to INSERT,.DELETE and UPDATE.
The new object was added to the default generic load workflow.

Files changed:
eagle_default/in/xml/xml-ref_global_tax_rate.xml
eagle_default/in/xml/xml-ref_objects.rsf
workflow related files

SDP-25042: Added support for load and extract of Variation Margin Override Rule Name in the GenericEntity

Added mapping for the Variation Margin Override Rule Name - tag1081 which is used to resolve the Variation Margin Override Rule Instance - tag16723.
Also added to the extract of the following fields in GenericEntity:

SECURITYDBO.ACCOUNTING_RULES.RULE_NAME – varMarginOverrideAccountingRuleUniqueKey/accountingRuleName
SECURITYDBO.ACCOUNTING_RULES.RULE_DESC - varMarginOverrideAccountingRuleUniqueKey/accountingRuleDesc
In EntityAccountingBasis:
SECURITYDBO.ACCOUNTING_RULES.RULE_DESC - urglAccountingRuleUniqueKey /accountingRuleDesc

Files changed:

eagleml_ent_genericentity.xml
xml-ent_entitygeneric.xml
entity_aliases.inc
entity_acct.inc
entity_panels.inc
extract_entity.inc
extract_entity_accounting_basis.inc
t_entity_streaming.inc
t_entity_streaming_eml.inc
t_entity_accounting_basis_streaming.inc

SDP-22873: Two additional fields added to GenericSMF Data Model

Two new fields were added to the XSD for the GenericSMF object:

SECURITYDBO.MBS.FMAE_ISSUER_CONCENTRATION
SECURITYDBO.MBS.FMAC_ISSUER_CONCENTRATION

The Inbound and Outbound rules were updated to support these new fields.

Files changed:
eagleml-sec-shared-2-0.xsd
smf_egl_gen_interface_template.xsl
smf_egl_gen_interface_template_mssql.xsl
t_smf_streaming_eml.inc

SDP-22186: Corrected issue resolving SOAP Port for Date Rule requests

Data was not loaded because the SOAP port was not calculated correctly.
The SOAP port calculation was improved. The RUNPREPROCESSOR is now used instead of the MSGEXCHANGE to get only a string with the port number. Also, retry logic was added in case port is not found.

The parameters below will allow customization of the retry logic via the w_config_custom.inc if required:

<CODE>
:W_CONFIG_CHECK_SOAPPORT_RE_TRY_CNT: :=5;
:W_CONFIG_CHECK_SOAPPORT_RE_TRY_WAIT: :=1;
</CODE>

Files changed:
get_soap_port.inc
calc_date_from_rule.inc
w_setup_eagle_version.xml

SDP-19651: Added possibility to include details of the Successful messages into the Consolidated TSR

A Consolidated TSR represents an additional report of the workflow, summing up the TSR's for each of the subprocesses of the workflow. The Consolidated TSR was improved to include SUCCESS Messages now.  A new flag IncludeAllDetails (W_INCLUDE_ALL_DETAILS) was added. Possible values: Y/N. The default value is N.
The flag allows to include details for SUCCESS messages in the Consolidated TSR like:
<transactionId>B90DBM55FL6PIXJB</transactionId>
<eagleDetailId>4839327</eagleDetailId>
<identifier><identifierName>fromId</identifierName><identifierValue>1</identifierValue>
</identifier><identifier><identifierName>account</identifierName><identifierValue>261</identifierValue>
</identifier><identifier><identifierName>referenceNumber</identifierName><identifierValue>-3862</identifierValue>
</identifier><identifier><identifierName>identifier2</identifierName><identifierValue>19900101</identifierValue>
</identifier><identifier><identifierName>identifier3</identifierName><identifierValue>99</identifierValue>
</identifier>

This flag works in conjunction with RTR parameters CreateConsolidatedTSR and IncludeRecordErrorDetails.

Example of usage in RTR:
<taskParameter>
<name>CreateConsolidatedTSR</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>
<taskParameter>
<name>IncludeAllDetails</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>
<taskParameter>
<name>IncludeRecordErrorDetails</name>
<dataType>S</dataType>
<value>N</value>
</taskParameter>

The Global flag is set in the "w_config_custom.inc":

<COL TAG="W_INCLUDE_ALL_DETAILS" EXPRESSION="'Y'"/>

Files changed:
mc2_task_reporter_generate_task_unit.inc
task_reporter_gen_step_summary.inc
task_reporter_gen_task_unit_realtime.inc
task_reporter_generate_task_unit.inc

Production Bugs:

SDP-29036: Generic Extract Workflows were updated where the wrong status No_Data was returned under MC2 Engine

Extracts below used to return a NO_DATA status if the eagle_wrf_generic_extract was launched under MC2 engine:

• SOURCEEXTRACT
• XREFCONFIGEXTRACT
• PACESYSEXTRACT
• PRICINGSYSSETTINGEXTRACT
• RPTTIMESTAMPS
• CLIENT

The issue was related to incorrect condition in Generic Extract workflow for these feed types.

Files changed:
Generic extract related files.

SDP-28434: Added securityAlias to Security Notes extract

The security alias element was added to the SECURITYNOTESEXTRACT.

Files changed:

extract_ref_security_notes.inc

SDP-28396: Sink Schedule amount cannot exceed 1.00

A rounding operation was corrected to allow more accurate tolerance in case of the sum of all sink factors.

Files changed:
eagle_default/in/xml/xml-ref_schedule.xml
eagle_default/in/xml/xml-ref_schedule_mi.xml

SDP-28287: Resolved Pricing Center Event Timeouts

The Pricing Center event for Price composite without child event was timing out due to a continuous loop of the failing SQL request. Added changes to check if the Pricing Center event has child events.

If it has child events, the rule checks the subevent statuses. If not, the rule escapes this logic.

Files changed:
exec_pace_event_pricing_check.inc

SDP-28129: Entitlement filter for extracts was changed to avoid duplicate records in Entity Filter V2

The Entity Resolution logic filter was modified to avoid duplicate records.  To use the latest changes for the Entity List Resolution and corresponding Extracts, a new flag was introduced and supported in the w_config_custom.inc.

<COL TAG="W_USE_ENTITY_RESOLUTION_VERSION" EXPRESSION="2"/>

The Entitlement filter (usergroups) is a part of Entity filter for many extracts. The rule extract_filter_entity_entitlement.inc is called inside the extract_filter_entity.inc file.
The Entity and Entitlement filters use two separate temp tables for storing values.
The Entitlement filter was called outside of the extract_filter_entity.inc for the below list of extracts:

• ENTITYBANKRELEXTRACT
• ENTITYCOMPOSITELISTEXTRACT
• ENTITYLISTEXTRACT
• ENTITYMANAGEREXTRACT
• ENTITYMANAGERRELEXTRACT
• ENTITYREGULATORYEXTRACT
• ENTITYSHARECLASSEXTRACT
• ENTITYXREFEXTRACT
• WRHSSPDDELTALOGEXTRACT

In the case of the Entity Filter V2 and the Entitlement filter, data was inserted twice into the temp table: the first time in a separate call extract_filter_entity_entitlement.inc; and the second time in the Entity Filter V2 itself.
As a result, the temp table for the Entitlement filter contained duplicates and multiple result set of records after the join. As a result, the extract files contained duplicates. For the above list of extracts, the separate call of extract_filter_entity_entitlement.inc was deleted and moved inside the extract_filter_entity.inc as it works for most parts of the objects.

Files changed:

extract_filter_entity.inc
extract_filter_entity_entitlement.inc
extract_entity_bank_relationship.inc
extract_entity_composite_list.inc
extract_entity_list.inc
extract_entity_manager.inc
extract_entity_manager_relationship.inc
extract_entity_regulatory.inc
extract_entity_share_class.inc
extract_entity_xref.inc
extract_warehouse_spd_delta_log.inc

SDP-27918: GenericSMF Load replaces some Xref data with Primary Asset or Alt Asset values

When the GenericSMF is loaded via panels (accountingValidationFlag=Y) and the Primary Asset Type and/or Alt Asset Type has any values in 'INTERNAL', 'FWDXREF', some xrefs were replaced by the Primary Asset Type or Alt Asset Type values. Added logic to reorder the xrefs before sending to panel based on panel-replacement logic. Changes were made for the SMF type Forwards (FWXXXX).

File changed:
smf_forwrd.inc

SDP-27850: Mapping issue for OriginalSource Field was corrected in the IssueFxRate Inbound rule

The IssueFxRate inbound rule was missing a mapping for tag4003 (OriginalSource). The issue was corrected.

Files changed:
xml-ref_fxrate.xml

SDP-27770: Allowed use of local currency for security resolution on load to Warehouse Trade Extension Table

The local currency was added for security resolution for the Trade Ext messages in single-load mode, allowing more flexibility with security resolution logic (:USE_TRADE_EXT: = '1')

Files changed:
eagle_default/in/xml/include/xml-dbdirect_sec_parse_xrefs.inc

SDP-27726: Added Mapping for Additional Fields in Exchange Price Inbound/Outbound Interfaces

Exchange Price data object was updated; Added support for additional fields.
The Inbound and Outbound rules were updated.

SECURITYDBO.PRICE_EXCHANGE. CONFIDENCEINTERVAL - confidenceInterval
SECURITYDBO.PRICE_EXCHANGE. PERCENT_CHANGE_1M - percentChange1m
SECURITYDBO.PRICE_EXCHANGE. PERCENT_CHANGE_3M - percentChange3m
SECURITYDBO.PRICE_EXCHANGE. PERCENT_CHANGE_6M - percentChange6m
SECURITYDBO.PRICE_EXCHANGE. PERCENT_CHANGE_1YR - percentChange1yr
SECURITYDBO.PRICE_EXCHANGE. PERCENT_CHANGE_WTD - percentChangeWtd
SECURITYDBO.PRICE_EXCHANGE. PERCENT_CHANGE_MTD - percentChangeMtd
SECURITYDBO.PRICE_EXCHANGE. PERCENT_CHANGE_QTD - percentChangeQtd
SECURITYDBO.PRICE_EXCHANGE. PERCENT_CHANGE_YTD - percentChangeYtd
SECURITYDBO.PRICE_EXCHANGE. PX_DIRTY_CLEAN - pxDirtyClean
SECURITYDBO.PRICE_EXCHANGE. PX_SETTLE - pxSettle
SECURITYDBO.PRICE_EXCHANGE. PX_SETTLE_LAST_DATE - pxSettleLastDate
SECURITYDBO.PRICE_EXCHANGE. QUOTED_SPEC - quotedSpec
SECURITYDBO.PRICE_EXCHANGE. SETTLEMENT_TYPE - settlementType

Files changed:
metadata/bind/eagleml_ref_exchangeprice.xml
eagle_default/in/xml/xml-ref_exchangeprice.xm
eagle_ml-2-0_cm/generate_sp_on_fly.inc
eagle_ml-2-0_cm/out/xml/t_issueprice_exchange_streaming.inc

SDP-27452: Cancel EJM Process was improved

The Cancel EJMProcess continued running, therefore the status was changed from CANCELLED back to SUCCESS.
The issue was corrected by including a restart MC Process command at the end of the Canceled Process for all MC Instances.

Files changed:
control_cancel_process.inc
control_process.xml

SDP-26575: FX Rates load was optimized

The Fx Rate inbound rule was improved to use a direct-SQL call instead of a panel call, which is much slower compared to the direct SQL query. The load rate was improved
from 20 rec/sec up to 200 rec/sec.  Additionally, the Fx Rate load is currently supported via an XMLDBAPI request with the load rate being about 350 rec/sec.
This is the non-default mode and requires a parameter in the incoming EagleML message to not use the panel load and validations:
<accountingValidationFlag>N</accountingValidationFlag>

Files changed:
xml-ref_fxrate.xml