EagleML Release Notes - January 2022

Owned by Jeffrey Santora (Deactivated)
Jan 14, 2022
6 min read

Enhancements

SDP-56180 SMF FWD resolution by xId/xIdType

Generic SMF resolution logic was updated to enable Forward securities to be resolved by xId and xIdType.

In order to enable this logic, the assetResolutionOption with the required parameters must be included. However, the ASSET_CURRENCY flag is a required value in that case. For example, <assetResolutionOption>BCUSIP,ASSET_CURRENCY</assetResolutionOption>

Files changed:
eagle_default/in/xml/include/security_resolution_generic.inc

SDP-57168 Enhanced EagleML to support single-row IRS and TRS trades

Single-row IRS and TRS trades can now be booked via EagleML. In order to enable this feature, the processingOptions must include parameter SINGLE_LEG_PANEL. For example,
<processingOptions>SINGLE_LEG_PANEL</processingOptions>

File added:
eagle_default/in/xml/xml-accttrade_singleleg.xml

Files changed:
eagle_default/in/xml/include/tradeleg_columns.inc
eagle_default/in/xml/xml-acct_trades.rsf

SDP-55523 WRHSOPENLOTEXTRACT - Warehouse Open Lot extract was enhanced to support the ability to extract the priceDate field

The priceDate field from the HOLDINGDBO.POSITION_DETAIL was added to the WRHSOPENLOTEXTRACT - Warehouse Open Lot.

New DB field: POSITION_DETAIL.PRICE_DATE
XPath: EagleML/warehouseTransaction/warehouseOpenLot/priceDate

Files changed:
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/tagvalue/extract_warehouse_openlot.inc
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/xml/t_warehouse_openlot_streaming.inc
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/xml/t_warehouse_openlot_xmldbapi_streaming.inc

SDP-57810 WRHSPOSITIONEXTRACT and WRHSOPENLOTEXTRACT  outbound interfaces were improved to support custom xreference identifiers for the XMLDBAPI format

The WRHSPOSITIONEXTRACT - Warehouse Position and WRHSOPENLOTEXTRACT - Warehouse OPen Lot outbound interfaces were improved to support custom xreference identifiers in the XMLDBAPI format for I2I purposes.

The purpose for using XMLDBAPI format is to limit the size of the data in the implementation of I2I, especially for positions and performance data. The use of custom xreference identifiers is meant as a capability for I2I to improve the success rate of security matching.

When a defined identifier type matches an existing custom identifier types for a security, it will be added to the first pair of xreference identifiers.

This feature is enabled by default and the default xreference type is ‘SPOKEID’. This can be overridden by passing the new parameter in an RTR or defining the parameter in the w_config_custom.inc include file.

New RTR parameter: PrimaryI2ISecXrefType

Example of usage:

<taskParameter>
                <name>PrimaryI2ISecXrefType</name>
                <value>SPOKEID</value>
</taskParameter>

New config parameter: W_PRIMARY_I2I_SEC_XREF_TYPE

Example of usage:
<CODE>:W_PRIMARY_I2I_SEC_XREF_TYPE: := 'SPOKEID';</CODE>

File changed:
eagle_ml-2-0_cm/out/xml/xref_map_xmldbapi_extract_streaming.inc

SDP-57650 Issuer Entity Relationship inbound interface was enhanced to load data into the ISSUER_ENTITY_MAP table

The Issuer Entity Relationship inbound interface was enhanced to load data into the SECURITY.ISSUER_ENTITY_MAP table as well as the SECURITYDBO.ISSUER_ENTITY_RELATION table.

The ISSUER_ENTITY_MAP table has a unique key issuer_alias + entity_id. All records in the ISSUER_ENTITY_RELATION table having the same issuer_alias and entity_id key will result in one record being created in the ISSUER_ENTITY_MAP table.

The delete functional was also improved. Records from the ISSUER_ENTITY_MAP table will be deleted after the last record with the same issuer_alias and entity_id is also deleted from the ISSUER_ENTITY_RELATION table.

Files changed:
eagle_ml-2-0_cm/generate_sp_on_fly.inc
eagle_default/in/xml/include/ref_issuerentityrelationship_delete.inc
eagle_default/in/xml/xml-ref_issuerentityrelationship.xml

SDP-51411 Rebook logic was implemented for the Schedule interface to check if a record exists prior to the load

To reduce unnecessary updates at the database level, Rebook logic was introduced to address when Schedule data is loaded without changes. The process will validate that the original record exists and it will ignore that message.

The comparison logic includes the list of fields that will be validated in order to skip a record update.

Common fields for all schedule types:
SECURITYDBO.SCHEDULE.NEXT_CALL_DATE - 1241 - EagleML/referenceTransaction/schedule/nextCallDate
SECURITYDBO.SCHEDULE.NEXT_CALL_PRICE - 1242 - EagleML/referenceTransaction/schedule/nextCallPrice
SECURITYDBO.SCHEDULE.NEXT_PUT_DATE - 1243 - EagleML/referenceTransaction/schedule/nextPutDate
SECURITYDBO.SCHEDULE.NEXT_PUT_PRICE - 1244 - EagleML/referenceTransaction/schedule/nextPutPrice
SECURITYDBO.SCHEDULE.STEP_UP_COUPON - 1252 - EagleML/referenceTransaction/schedule/stepUpCoupon
SECURITYDBO.SCHEDULE.STEP_UP_DATE - 1253 - EagleML/referenceTransaction/schedule/stepUpDate
SECURITYDBO.SCHEDULE.SCHEDULE_STATUS - 1670 - EagleML/referenceTransaction/schedule/scheduleStatus
SECURITYDBO.SCHEDULE.SCHEDULE_FLOAT1 - 5534 - EagleML/referenceTransaction/schedule/scheduleFloat1

Schedule type dependent fields:
SECURITYDBO.SCHEDULE.BYPASS_SINK_CHECK - 17479 - EagleML/referenceTransaction/schedule/bypassSinkCheckSwitch (CSTMSINK)
SECURITYDBO.SCHEDULE.SCHEDULE_AMOUNT - 1669 - EagleML/referenceTransaction/schedule/scheduleAmount (EXINTSCH, CSTMSINK, CPNRESET, PIKMTM, SINK)
SECURITYDBO.SCHEDULE.SCHEDULE_PRICE - 1249 - EagleML/referenceTransaction/schedule/price, EagleML/referenceTransaction/schedule/rate (REFUND, CALL, PUT)
SECURITYDBO.SCHEDULE.PRE_REFUNDED_DATE - 1247 - EagleML/referenceTransaction/schedule/preRefundedDate (REFUND)
SECURITYDBO.SCHEDULE.SCHEDULE_DATE1 - 3669 - EagleML/referenceTransaction/schedule/scheduleDate1 (EXINTSCH)
SECURITYDBO.SCHEDULE.RESET_DATE - 16408 - EagleML/referenceTransaction/schedule/resetDate (CPNRESET)
SECURITYDBO.SCHEDULE.FIXING_DATE - 16410 - EagleML/referenceTransaction/schedule/fixingDate (CPNRESET)
SECURITYDBO.SCHEDULE.CASH_PAYMENT_DATE - 16411 - EagleML/referenceTransaction/schedule/cashPaymentDate (CPNRESET)
SECURITYDBO.SCHEDULE.END_DATE - 73 - EagleML/referenceTransaction/schedule/endDate (CPNRESET, PIKMTM)

The new logic is only available when the processing option SCHEDULE_TRUE_REBOOK is set in the incoming file or as a global parameter W_SCHEDULE_TRUE_REBOOK in w_config_custom.inc.

Usage:
<header>
<objectType>Schedule</objectType>
<processingOptions>SCHEDULE_TRUE_REBOOK</processingOptions>
</header>
or
<COL TAG="W_SCHEDULE_TRUE_REBOOK" EXPRESSION= "'Y'"/>

Files changed:
eagle_default/in/xml/xml-ref_schedule.xml

SDP-54421 Enhanced Signage logic for the cashEventType element

Signage logic for the cashEventType element was adjusted for MBS securities where the Payment with the Payment Loss Factor results in a negative calculation. This will set the Settlement Amount to a cash type of DISB. The changes are in effect when using the CashTypeSignageFlag parameter.

<taskParameter>
<name>CashTypeSignageFlag</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>

The CashTypeSignageFlag parameter impacts the TRADE_DETAIL.CASH_TYPE field only and evaluates the Cash Type flipping the value. It is activated when passed when the EnableSignageLogic is set to Y and the IncludeExtendedDetails is set to Y or ALL.

<taskParameter>
<name>EnableSignageLogic</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>
<taskParameter>
<name>IncludeExtendedDetails </name>
<dataType>S</dataType>
<value>ALL</value>
</taskParameter>

File changed:
eagle_ml-2-0_cm/out/tagvalue/extract_wrhs_trade_signage_logic.inc

SDP-54413 The Warehouse Trade Inbound interface was enhanced to support an additional field

The Warehouse Trade interface was updated to support loading the additional field TRADESDBO.TRADE_DETAIL.SETTLE_AMOUNT_BASE. This is required because Settle Amount Base is needed for swaps since the Net Amount Base shows the notional value and settlement amount shows the money movement. Therefore, the base money movement is required.

XPath: EagleML/warehouseTransaction/warehouseTrade/settleAmountBase

Files changed:
eagle_default/in/xml/xslt/wrh_egl_gen_interface3.xsl

SDP-58187 Updated the WRHSTRADEEXTRACT to revise the calcMasterTransactionID for NONTAXMERGP events and to expand support of Technical Trades for the Settlement Amount elements to set the amounts to 0

Revised the calcMasterTransactionID to include the event types NONTAXMERGP and NONTAXMERGPCANCEL for the MTCASH logic. Corrected non-MTCASH events to include TRADE.STAR_TAG_25 for the default logic.

[

Revised the logic for the Settlement Amount Base and Settlement Amount Local fields to support Technical Long and Short Trades setting the values to 0 when it is an MTCLOSE or an MTOPEN. Signage logic for the Settlement Amount Base and Settlement Amount Local is enabled through an opt-in element. The SettlementSignageFlag is used in tandem with the EnableSignageLogic set to Y. When the SettlementSignageFlag is set to Y, signage logic is applied to the SETTLE_AMOUNT_BASE field. The SETTLE_AMOUNT_BASE field is supported in v17.R2.30 forward

Signage is enabled when the following parameters are included:
<taskParameter>
<name>EnableSignageLogic</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>
<taskParameter>
<name>SettlementSignageFlag</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_wrhs_trade_signage_logic.inc
eagle_ml-2-0_cm/out/tagvalue/extract_warehouse_trade.inc

SDP-57919 Improved data pipeline from the In-market environment to the Cloud region in History mode

To correctly upload data to history tables in Snowflake, the historyOnlyFlag set to Y was included for every record in the output file as part of the In-Market EDS extract. This enhancement was applied to resources where history tables are supported. The following resources were updated:

genericsmf
genericentity
entitydetail
entityregulatory
issuerorganization
issuerrole
issueresg
 
The functionality will be applied to the above EDS extracts in history mode and to all supported output formats, like EagleML, CSV, and JSON. The enhanced logic works for In-Market and Cloud processing.

Files changed:

metadata/core/ontology/entity/entitydetail-models.json
metadata/core/ontology/entity/entityregulatory-models.json
metadata/core/ontology/entity/genericentity-models.json
metadata/core/ontology/reference/genericsmf-models.json
metadata/core/ontology/reference/issueresg-models.json
metadata/core/ontology/reference/issuerorganization-models.json
metadata/core/ontology/reference/issuerrole-models.json

SDP-57770 Added support for Delta Extracts in EDS

EDS Delta-based extracts are now supported for the In-Market/Oracle environment.
Supported task parameters are:

DeltaBased
DeltaBasedUID
increaselastrundatetime

The functionality is in sync wtih EagleML [ ]
The RTR for the delta genericsmf EDS extract is below:

<EagleML xmlns="[http://www.eagleinvsys.com/2011/EagleML-2-0%22|http://www.eagleinvsys.com/2011/EagleML-2-0%22|smart-link ] xmlns:xsi="[http://www.w3.org/2001/XMLSchema-instance"|http://www.w3.org/2001/XMLSchema-instance%22] xsi:type="RunTaskRequest" eaglemlVersion="2-0" xsi:schemaLocation="[ ] eagleml-main-2-0.xsd" eaglemlType="RunTaskRequest">
<header>
<messageId>E30X5FTN4N317JFF</messageId>
<sentBy>MC2EJM</sentBy>
<sendTo><[ ]
<creationTimestamp>2018-04-13T09:52:39.73-05:00</creationTimestamp>
<expiryTimestamp/>
<expiryLoadFlag/>
</header>
<taskIdentifier>
<processingSequenceNumber/>
<correlationId>genericsmf_eds_extract_\{GUID}</correlationId>
<businessTaskId>genericsmf_eds_extract</businessTaskId>
<sequenceNumber/>
</taskIdentifier>
<taskTypeEnum>NEW</taskTypeEnum>
<synchronousExecution>yes</synchronousExecution>
<taskParameters>
<taskParameter>
<name>ServiceName</name>
<value>EBS</value>
</taskParameter>
<taskParameter>
<name>ServiceMethodName</name>
<value>extract</value>
</taskParameter>
<taskParameter>
<name>ResourceName</name>
<value>genericsmf</value>
</taskParameter>
<taskParameter>
<name>DeltaBased</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>
<taskParameter>
<name>DeltaBasedUID</name>
<dataType>S</dataType>
<value>mydelta1</value>
</taskParameter>
<taskParameter>
<name>increaselastrundatetime</name>
<dataType>S</dataType>
<value>ALWAYS</value>
</taskParameter>
<taskParameter>
<name>dbprovider</name>
<value>oracle</value>
</taskParameter>
<taskParameter>
<name>OutputFormat</name>
<dataType>S</dataType>
<value>EagleML</value>
</taskParameter>
<taskParameter>
<name>SaveSQLQuery</name>
<value>Y</value>
</taskParameter>
</taskParameters>
</EagleML>

Files changed:
mc2py/cachemanager/delta_extract_info_store.py
mc2py/cachemanager/poll_eagledb_driver.py
mc2py/dbdataframe.py
mc2py/ds_query_info.py
mc2py/idsdataframe.py
mc2py/idsdataframe_delta.py
mc2py/idsdataframe_props.py
mc2py/instructions.py
rdf_metadata/rdf_outflow.py

SDP-52235 EDS fieldatrribute Extract

EDS fieldatrribute resource is similar to EagleML extract. The following filters are supported:

  • fieldattributeid

  • fieldname

  • fieldtype

This resource contains meta data information about Eagle fields like field type, name, data base table, description and etc. This is basic implementation which supports only common information for core fields, custom, derived.

Files changed:

metadata/core/ontology/eagle-vocab-enumerations.json
metadata/core/ontology/reference/fieldattribute-models.json
metadata/core/ontology/reference/fieldattribute.json

Production Bugs

SDP-57424 Issuer Resolution logic was fixed for the Line of Credit Facility Inbound interface and Issuer Delete logic

The Issuer Resolution Options were not supported for the Line of Credit Facility interface due to an incorrect variable name. The same issuer was found when deleting the Issuer Organization, Issuer Role, Issuer Entity Relationship or Issuer Analytics via eagle_ml-2-0_default_in_xml_reference stream with the action DELETE.

The Variable name for the Issuer Resolution Options was also fixed for all the cases described above.

In certain cases, the Issuer Resolution process returned an incorrect IssuerID in the Line of Credit Facility interface. Related variables were also fixed in the Line if Credit Facility interface in order to resolve to the proper IssuerID.

Files changed:
eagle_default/in/xml/xml-ref_line_of_credit.xml
eagle_default/in/xml/include/parse_issuer_xref.inc

SDP-57429 Accounting Reclaim Cash object was corrected to allow data to load via postDate

The Accounting Reclaim Cash inbound interface was corrected to allow data to load to the postDate XML element. The mapping for the post date was not part of the load therefore it was not being applied. The element was added and mapped to the XSD.

XPath:
EagleML/accountingTransaction/accountingReclaimCash/postDate

File changed:
eagle_default/in/include/cash_reclaim_aliases.inc
msgcenter/eagle_default/in/include/csv-reclaim.inc

SDP-57392 maxrows filter was fixed for Accounting Rule Extract

The maxrows filter was being overwritten by the SQL query that is generated by the Accounting Rule extract. The extract query generation was therefore fixed to include a maxrows filter for both MC and MC2.

In addition to the maxrows filter change, support was provided for streaming transformation for MC.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_rule_accounting.inc
eagle_ml-2-0_cm/out/tagvalue/extract_rule_accounting.xml
eagle_ml-2-0_cm/out/xml/t_eagleml-2-0_streaming.rsf
eagle_ml-2-0_cm/out/xml/t_rule_accounting_streaming.inc
eagle_ml-2-0_cm/w_config.inc

SDP-57987 An incorrect error description related to the Warehouse Security resolution was resolved

There was incorrect error description in the status of the warehouse object load that did not have a security resolution in the EagleML rules. This occurred in specific cases when using the ASSET_CURRENCY assetResolutionOption without a localCurrency and a wrong primaryAssetId.

Files changed:
eagle_default/in/xml/include/xml-dbdirect_sec_parse_xrefs.inc
eagle_default/in/xml/include/xml-dbdirect_sec_resolution.inc

SDP-46387 The Warehouse Performance EDS extract was improved to support additional filters

The Warehouse Performance EDS extract was enhanced to support the following filters:

dictionaryid - dictionary id filter
perffreqcode - performance frequency code filter
perfrollupreturnsid - performance rollup return id filter
perfsumarytype - performance summary type filter

File changed:
metadata/core/ontology/warehouse/warehouseperformance.json

SDP-53653 Corrected wrong data mashup for IssuerRating

The issue was related to the mashup not working correctly based on the equal set of keys: SOURCENAME + ISSUERALIAS == ISSUERALIAS + SOURCENAME. The order of unique keys was adjusted to correct the issue.

Files changed:
metadata/core/datasets/issuerrating.json

SDP-54520 The query profile definition logic was corrected

The queryprofiledefinition logic was improved to correctly handle profiles for EDS resources where the same tables are used with a different prefix in the query definition more than once.

Files changed:
src/eagleinvsys/metadata/metainfo_query_description_parser.py