EagleML Release Notes - November 2019

Enhancements:

SDP-29856: List of identifiers was added to EDITCHECK error message for Rating

The Rating object was updated. The error message was improved to return the list of security identifiers in case of a failure.

Files changed:
eagle_default/in/xml/xml-ref_issuerating_all.xml

SDP-29645: RTR execution after MC restart

There were two ways of workflow execution after MC restart:
1) one processing branch continued its work, the second one stopped with a "Duplicate correlation id" error
2) one processing branch stopped with "Duplicate correlation id" error, the second one hung

Added changes to continue the workflow execution in one branch and not to start the second one. This logic is implemented for subproccesses only. If MC will be restarted during the very first initial RTR, it will be processed as it is described in case 1 above.

Files changed:
runtaskrequest.xml

SDP-29456: The DEMANDLIST Extract was enhanced to support Date Rules filters

The DEMANDLIST Extract was enhanced to support Date Rules filters and to support streaming transformation. The Generic Extract workflow was also updated to support the Date Rules filters for the DEMANDLIST extract.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_demandlist.inc
eagle_ml-2-0_cm/out/tagvalue/extract_demandlist.xml
eagle_ml-2-0_cm/out/xml/t_demandlist_streaming.inc
eagle_ml-2-0_cm/out/xml/t_eagleml-2-0_streaming.rsf
eagle_ml-2-0_cm/out/xml/t_eml_header_streaming.inc
eagle_ml-2-0_cm/w_config.inc
various workflow files

Files added:
eagle_ml-2-0_cm/out/xml/t_demandlist_streaming.xml

SDP-29171: Added mapping for the flag calculateValidateLastPaymentDate to the SMF inbound rule in Accounting mode

The inbound SMF rule was enhanced to map the flag calculateValidateLastPaymentDate to validate or calculate the last coupon date within the panel. This date is calculated when the parameter calculateValidateLastPaymentDate is set to 'Y'. When it is set to 'N', the value received for the last coupon date in the message is validated.

Files changed:
eagle_default/in/xml/include/sm_detail_model.inc
eagle_default/in/xml/include/smf_aliases.inc
eagle_default/in/xml/include/smf_forwrd.inc
eagle_default/in/xml/include/smfflags_model.inc

SDP-29052: Enhanced the Inbound EML Receive / Deliver rules to support direct mapping tags transmission to the Receive/Deliver panels

Added missing mappings for the delivery and receive Accounting Trades (buySellIndicator with value 'DEL' and 'REC' respectively) to transmit them directly to the delivery/receive panels.
Added the following mappings:
tag49 - tradedInterest
tag50 - localNetAmount
tag478 - baseNetAmount

Also added the new element yieldIndicatorCode (tag5533) to be able to map it directly.

Files changed:
trade_aliases.inc
csv-deliver.inc
csv-receive.inc

SDP-28843: Warehouse Performance Extract was enhanced

The WRHSPERFORMANCEEXTRACT was enhanced to support two additional tables: PSR_ANALYTICS and PERF_SEC_RETURNS_EXT.
The XSD schema was changed to support the fields from the PSR_ANALYTICS and PERF_SEC_RETURNS_EXT tables.

Files changed:
extract_warehouse_performance.inc
t_warehouse_performance_streaming.inc

SDP-28786: Added Support for Remapping Fields on the PACE Side for Warehouse Trade Extract

The RTR parameter EnableS2PReMappingFlag and configuration parameter W_ENABLE_S2P_REMAPPING_FLAG were added to support S2P remapping for the following fields secFee(Base), otherFee(Base) and exchangeFee(Base).
<taskParameter>
<name>EnableS2PReMappingFlag</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>

If set to Y, the following fields are remapped:
'secFee' is mapped into 'otherFee'
'secFeesBase' is mapped into 'otherFeeBase'
'exchangeFee' is mapped into 'secFee'
'exchangeFeeBase' is mapped into 'secFeeBase'
'otherFee' is mapped into 'exchangeFee'
'otherFeeBase' is mapped into 'exchangeFeeBase'.

If the RTR parameter EnableS2PReMappingFlag or configuration parameter W_ENABLE_S2P_REMAPPING_FLAG is set as N or is null, the default mapping is used for these fields.

Files changed:
extract_warehouse_trade.inc

SDP-28587: Support entity resolution conditions longer than 500 characters on the extract in Entity filter

Corrected length limitation greater than 500 characters for conditions to define DynamicEntityList or DynamicPortfolioList. This eliminates the ORA-06502: PL/SQL: numeric or value error: character string buffer too small error which was being returned.

File changed:
/eagle_ml-2-0_cm/extract_filter_entity_resolution_v2.inc

SDP-28517: Improved the WRHSTRADEEXTRACT to expand the Signage Logic for the Dirty Principal Base and Dirty Principal Local fields

Signage logic was added for the following fields: dirtyPrincipalBase and dirtyPrincipalLocal.
Signage logic is applied when the parameter EnableSignageLogic is set to Y. If the value is N or null, the numeric fields are presented as stored in the database. If the value is Y, a predefined algorithm will be used for changing the sign of the field value.

<taskParameter>
<name>EnableSignageLogic</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_wrhs_trade_signage_logic.inc

SDP-28443: Added support of extensions for the Generic Issue Analytic Inbound Interface

The GenericIssueAnalytic object used to support a limited set of database fields.
The following support was added:

The ability to generate stored procedures to load extended data based on metadata and created extension tables
The ability to add custom mapping to elements from the incoming file

Files changed:
eagle_ml-2-0_cm/generate_sp_on_fly.inc

SDP-24950: Real Time Processing Supported via MC2

Real time processing fully supported via MC2. The set of changes was implemented to make Real time processing fully operational and synced up with MC0.

Files changed:
eagle_default/in/xml/xml-create_tsr.xml
eagle_ml-2-0_cm/collect_task_parameters.inc
eagle_ml-2-0_cm/in/xml/realtime_rtr.xml
eagle_ml-2-0_cm/mc2/ejm/reporter/mc2_ejm_reporter.xml
eagle_ml-2-0_cm/mc2/ejm/reporter/mc2_reporter_file.inc
eagle_ml-2-0_cm/mc2/ejm/reporter/mc2_reporter_file_summary.inc
eagle_ml-2-0_cm/mc2/ejm/reporter/mc2_task_reporter_generate_task_unit.inc
eagle_ml-2-0_cm/out/xml/task_reporter_gen_task_unit_realtime.inc
eagle_ml-2-0_cm/out/xml/task_reporter_gen_task_unit_realtime_req.inc

SDP-24455: SPDDELTATRIGGERLOGEXTRACT was modified to return one row per Entity

A new parameter, minAccountingDateFlag, was introduced for the SPDDELTATRIGGERLOGEXTRACT.

<taskParameter>
<name>minAccountingDateFlag</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>

The row returned for each entity has the earliest accounting date for the message subtype of 'SPD_VALUATION_CALCS' which indicates a revaluation.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_spd_delta_trigger_log.inc

Production Bugs:

SDP-29609: Issuer Extract updated to correct data extracted from the LEI_DETAILS table

The wrong data was extracted as part of the LEI model via the Issuer Extract. The GENISSUEREXTRACT logic was corrected.

Files changed:
t_generic_issuer_streaming.inc

SDP-29434: Added the securityAlias and other missing nodes to the Issuer Role Extract

The following Security elements were added to Issuer Role - ISSUERROLEEXTRACT outbound interface:

securityAlias
xrefIdentifiers
uniqueProductId
uniqueSwapId

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_issuer_role.inc
eagle_ml-2-0_cm/out/tagvalue/extract_issuer_role.xml
eagle_ml-2-0_cm/out/xml/t_issuer_role_streaming.inc

SDP-29381: Additional attribute dateToUse was added for DateRule

Only the EndDate was calculated when the DateRule was used. A new attribute added flexibility to the DateRule. The attribute dateToUse supports two values: StartDate or EndDate. The default value is EndDate.
Example of using the attribute dateToUse in a data file:

Files changed:
eagle_default\in\xml\include\date_rule_calculation.inc
XSD Schema

SDP-29380: XML Grouping Issue in the Trial Balance Extract was Corrected

The grouping key for the Trial Balance Extract - TRIALBALANCEEXTRACT was fixed to have each line of the report be reflected as a separate TrialBalance node.

Files changed:
t_acct_trial_balance_streaming.inc

SDP-29351: Adjusted the Issuer Role batch load and update for UPSERT MODE

The ISSUE_ROLE_UPSERT_MODE is properly supported now for the Issuer Role batch load with the records being properly updated.
The issue was fixed by resetting the varAddFlagProcessCheck for each xml-ref_issuerrole_common.inc call which updates the batch records accordingly.

Files changed:
xml-ref_issuerrole_common.inc

SDP-29348: Changed the Tag number for the finalRateRoundingPrecision in the GenericSMF Interface

The number of the tag for the finalRateRoundingPrecision was changed from 2283 to 12402 to map Tag 12402 directly and not resolve it based upon 2283 in the GenericSMF Interface for both accounting and SP modes.

Files changed:
eagle_default/in/xml/include/sm_extension.inc

SDP-29227: Removed duplicated aliases for Accounting Entity Inbound Interface

The Add Entity Xreferences functionality did not work due to duplicated alias for the objectType which caused the loss of value for this element. The replica alias was removed and the Entity Xreferences are added as expected.

Files changed:
entity_aliases.inc

SDP-28980: Profile Loader was improved - FileName starts from Correlation ID

Data processing via the profile loader was failing in cases where the filename was greater than 200 characters.
The issue was corrected by starting a filename with the correlationId to improve the execution process.

Files changed:
eagle_ml-2-0_cm\unpacker\send_to_wrf.xml

SDP-28699: Corrected an issue with the element clientId for the Generic Entity inbound and outbound interfaces

The element clientId has the following logic in the Generic Entity inbound interface:

First there was an attempt to calculate the INSTANCE field by CLIENT_ID field in the RULESDBO.CLIENT DB table; clientId mapped to CLIENT_ID field in the lookup;
Then if the INSTANCE was calculated, it mapped to tag 1864 (in_client_id parameter of stored procedure to load Generic Entity);
If the INSTANCE was not calculated, clientId from an incoming message was mapped directly to tag 1864;
If the clientInstance field is null and the INSTANCE was not calculated by clientId and clientId field is numeric, the clientId will be mapped to tag 1864.

The element clientId has the following logic in the Generic Entity outbound interface:

2 DB fields are selected from the RULESDBO.CLIENT DB table in the Query rule: RULESDBO.CLIENT.CLIENT_ID (type CHAR(20) ) and RULESDBO.ENTITY.CLIENT_ID (type NUMBER).
If RULESDBO.CLIENT.CLIENT_ID is not null, it was mapped to the element clientId in the transformation rule.
If the RULESDBO.CLIENT.CLIENT_ID is null, the field RULESDBO.ENTITY.CLIENT_ID was mapped to the element clientId.

XSD Schema was updated:

The element clientInstance (type xsd:integer) was added for inbound and outbound interfaces Generic Entity
The type of the element clientId was updated from xsd:integer to xsd:string

Corresponding rule files were changed:

Added the mapping for the element clientInstance: it is now always mapped to the RULESDBO.ENTITY.CLIENT_ID;
Updated the mapping for the element clientId: it is now always mapped to the RULESDBO.CLIENT.CLIENT_ID.

Files changed:

eagle_default/in/xml/xml-ent_entitygeneric.xml
eagle_ml-2-0_cm/out/xml/t_entity_streaming.inc
eagle_ml-2-0_cm/out/xml/t_entity_streaming_eml.inc

SDP-27747: Corrected issues related to the output format when the eagle_wrf_generic_extract workflow is executed via MC2

The Open Lot - WRHSOPENLOTEXTRACT, Position - WRHSPOSITIONEXTRACT and Warehouse SPD Delta Log - WRHSSPDDELTALOGEXTRACT Extracts completed with NO_DATA Status, in case OutputFormat parameter was other than EagleML as part of RTR.
The workflow was corrected and the issue fixed.

Files changed:
workflow related files

SDP-27640: Corrected Mapping for the priceCurrency element in IssuePrice Interface

The IssuePrice Interface was updated. The priceCurrency element is mapped to the SECURITYDBO.PRICE.PRICE_CURRENCY field with higher priority than the assetCurrency.

Files changed:
eagle_default/in/xml-ref_issuerprice. xml

SDP-27275: Fixed an issue with incorrect load of Warehouse Open Lots in Sub-Batch Mode

The Warehouse Open Lots were loading incorrectly in sub-batch mode where the W_WRHS_IGNORE_POS_FLAG is set to Y. Each first sub-batch was deleted by each new batch.
The issue was identified and corrected.

Files changed:

xml-dbdirect_wrhs_delete_statement.inc

SDP-26619: DateTime Format Supported for changeDate element on the SMF Load

Added support for the DateTime format of the changeDate element. The inbound and outbound interfaces were updated for the genericSMF and multilegSMF.
The change is available starting from Eagle Core V17R2.17.
The time value will be ignored for prior versions. The changeDate value will be properly extracted via Extract Service (MC2).

Files changed:
contract_mapping.inc
pay_mapping.inc
receive_mapping.inc
sm_extension.inc
smf_egl_gen_interface_template.xsl
smf_egl_gen_interface_template_mssql.xsl
t_smf_streaming.inc
setup_bind.inc
msgxchg_sec_core.inc

SDP-29933: Workflow Restart functionality was corrected

There was a problem with workflows launched from Automation Center with the timeout option (EnableTimeoutCancel = Y).

RTR's for cancel these workflows were sent after the timeout was over, regardless of the workflow state.
For example, it could be successfully processed, but after the RTR was sent, the status was switched from SUCCESS to CANCELLED.
The condition to cancel workflow was adjusted to have the RTR be sent only when a workflow is in progress (wait or processing files).

Specific parameters in Automation Center:
EnableTimeoutCancel (tag1179) with value Y (The workflow will be cancelled when the timeout is completed in the stream generic_load_workflow_restart)
EJMRunTimeout (tag1180) set to a value [Default value 6000]. The timeout for processing the generic_load_workflow_restart will be overridden with seconds.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/workflow_restart.xml