EagleML Release Notes - February 2022

Enhancements

SDP-58451 Xreference load via GenericSMF interface was fixed for MSSQL environments

Two parameters were removed from the INSERT_UPDATE_XREF_INFO_ASM stored procedure in Eagle Core v17 R2.32. This was causing an error during Xreference load via the GenericSMF interface.

The INSERT_UPDATE_XREF_INFO_ASM stored procedure call was fixed on the EagleML rule side to be in order to remain in sync with Eagle Core.

Files changed:
eagle_default/in/xml/include/ins_upd_xrefs.inc

SDP-58272 Expanded logic for the calcTransactionId to differentiate PIK and COUPONs

Column calcTransactionId now appends by the transaction code for PIKBOND, COUPON, PIKBONDCANCEL, COUPONCANCEL values.

File changed:
eagle_ml-2-0_cm/out/tagvalue/extract_warehouse_trade.inc

SDP-58469 Warehouse objects support using assetCurrency instead of localCurrency.

Provided the ability to use assetCurrency instead of localCurrency if local currency is not present. localCurrency still has a higher priority than assetCurrency. This was developed for to load in the following warehouse objects that use localCurrency in the load.

Warehouse Position, Warehouse Trade, Warehouse Cash Activity, Warehouse Cash Flow Projection, Warehouse Close Lot, Warehouse Disclosure Status, Warehouse NAV Multi, Warehouse Open Lot, Warehouse GL Balance.

Files changed:
eagle_default/in/xml/xslt/wrh_egl_gen_interface3.xsl
eagle_default/in/xml/xml-wh_gl_detail_posting.xml
metadata/bind/eagleml_wrhs_warehouseglbalances.xml

SDP-57649 Ability to display Schedule Name instead of businessTaskId for scheduled EJM Processes

Two features have been added to change the businessTaskId for scheduled EJM Process events. The new businessTaskId will be shown as the Schedule Name of the EJM Process in Automation Centers monitoring GUI. These features can be used for events that are executed by the workflow_restart Ruleset File.

Usage:

“On Data” and “Schedule“ events.
- To enable this feature you need to add a new flag in custom configuration:
  - <COL TAG="W_SCHD_BUS_TASK_ID_EXTENDED" EXPRESSION="'Y'"/>
“Scheduled“ events. Go to Run Parameters section in Automation Center where you configure the schedule. Set the “IVAR_CHAR_7249 (7249)” tag in “Tags“ or BUSINESSTASKID parameter in “Parameters“. BusinessTaskId (Schedule Name in AC) will be equal to the value of the tag or parameter. Please refer to the SDP item for further details.

File changed:
eagle_ml-2-0_cm/out/tagvalue/workflow_restart.xml

SDP-58830 Load and extract now supported for the PACE_MASTERDBO.EGL_GENERIC_XLATION_ML table

The load and extract rules for Code Value Feed Type were both updated to support the following new fields.

referenceTransaction/codeValue/codeValueLongDescription
referenceTransaction/codeValue/languageId

These fields map into EGL_GENERIC_XLATION_ML.DESCRIPTION and EGL_GENERIC_XLATION_ML.L_ID columns.

This allows for clients to leverage codeValueLongDescription which provides the capability to load and extract more than 50 characters of data for a max of up to 255 characters.

Files changed:
eagle_default/in/xml/xml-ref_codevalue.xml
eagle_ml-2-0_cm/out/tagvalue/extract_codeval.xml
eagle_ml-2-0_cm/out/tagvalue/extract_codeval.inc
eagle_ml-2-0_cm/out/xml/t_codeval.xml
eagle_ml-2-0_cm/out/xml/t_codeval_streaming.inc

SDP-58846 YieldCurve object was enhanced to support the maturityDate element

The maturityDate element is now supported on both inbound and outbound interfaces for the YieldCurve object. If maturity date is however passed on the incoming message on inbound interfaces it will go to the database as is. Otherwise, it is being calculated based on effectiveDate and duration.

Files changed:
eagle_ml-2-0_cm/generate_sp_on_fly.inc
eagle_ml-2-0_cm/out/xml/t_ref_yield_curve_streaming.inc
eagle_default/in/xml/xml-ref_yieldcurve.xml

SDP-58752 Warehouse Trades inbound interface was updated to revert back the previously removed Security and Trade resolutions

Redundant Security and Trade Resolutions were previously removed from the Warehouse Trade processing logic. As a result, that led to an unexpected behavior in a client's custom auto-includes. The Security and Trade Resolution logic was therefore reverted back.

Files changed:
eagle_default/in/xml/include/xml-warehouse_common.inc
eagle_default/in/xml/xslt/wrh_egl_gen_interface3.xsl

SDP-58549 The Accounting Trade inbound interface was fixed in order to correctly process Original Face value for RECEIVE, DELIVER and CONVERSION messages.

RECEIVE, DELIVER and CONVERSION messages were publishing Original Face (tag41) value for all security types but this behavior was incorrect. Only Sinkable securities require Original Face (tag41).

The Accounting Trade inbound interface logic was adjusted for RECEIVE, DELIVER and CONVERSION messages. The Sink Flag Indicator value is checked and if it is other than Y (N, null), then the Original Face is set to null.

Files changed:
eagle_default/in/include/csv-conversion_columns.inc
eagle_default/in/include/csv-deliver.inc
eagle_default/in/include/csv-receive.inc

SDP-58776 MultilegSMF interface was enhanced for new elements related to ARR processing

Additional elements resetLockoutPeriod, couponRateFixingMethod, finalRateRoundingPrecision and finalRateRoundingDirection are now supported for the MultilegSMF interface for ARR processing.

XPath:
/referenceTransaction/multilegSMF/contractLeg/FixedIncomeModel/resetLockoutPeriod
/referenceTransaction/multilegSMF/contractLeg/FixedIncomeModel/couponRateFixingMethod
/referenceTransaction/multilegSMF/contractLeg/SMExtension/finalRateRoundingPrecision
/referenceTransaction/multilegSMF/contractLeg/SMExtension/finalRateRoundingDirection

Files changed:
eagle_default/in/xml/include/contract_leg_alliases.inc
eagle_default/in/xml/include/contract_mapping.inc
eagle_default/in/xml/include/pay_leg_alliases.inc
eagle_default/in/xml/include/pay_mapping.inc
eagle_default/in/xml/include/receive_leg_alliases.inc
eagle_default/in/xml/include/receive_mapping.inc
eagle_default/in/xml/xml-smf_multileg_swap.xml

SDP-58770 Process Center Inbound Interface

Added the ability to load a Process Center entry via the Process Center Inbound Interface. Support has been provided for the following five items. ADD_PROCESS_CENTER, UPD_PROCESS_CENTER, UPD_PROCESS_CENTER_CCDT, UPD_PROCESS_CENTER and DEL_PROCESS_CENTER. Please note, that updateSource field can not be overridden as the panel is set to default value.

Sample Message:

<objectType>ProcessCenter</objectType>
<objectId>ProcessCenter</objectId>
<action>ADD</action>
</header>
<processCenter>
<objectType>ProcessCenter</objectType>
<objectId>ProcessCenter</objectId>
<objectDescription>OBJECT_DESCRIPTION</objectDescription>
<updateSource>MCADMIN</updateSource>
<processCenterName>V9</processCenterName>
<processCenterDescription>V9 TEST</processCenterDescription>
<currentDate>2018-07-16</currentDate>
<ccCurrentDate>2018-07-11</ccCurrentDate>
<ccMonthEndDate>2018-08-31</ccMonthEndDate>
<rollPriorPeriod>Y</rollPriorPeriod>
</processCenter>
</referenceTransaction>
</EagleML>

Files changed:
eagle/estar/EAGLE_ML_2_0.pkg
eagle/estar/eagle_ml-2-0_default_in_xml_reference.pkg
eagle_default/in/xml/xml-ref_objects.rsf
eagle_default/in/xml/xml-ref_processcenter.xml

SDP-58658 YieldCurve object was enhanced to support extensions

The YieldCurve object was enhanced to support extensions for both inbound and outbound interfaces.
Before adding the extension, an extension table should be created. The table must have the same key field as the main table (SECURITYDBO.YIELD_CURVE_DETAIL): YIELD_CURVE_ID, PAR_DURATION. After the table is created, the YieldCurve extension can then be added through the IWS xPanles tool.

Example of YieldCurve object with an extension:
Node customExtensions should always be present for all extensions. Node yieldCurveExt is created from extension name in xPanels and ‘Ext’ postfix.

<referenceTransaction>
<header>
<objectType>YieldCurve</objectType>
</header>
<yieldCurve>
<objectType>YieldCurve</objectType>
<objectId>YieldCurve</objectId>
<objectDescription>REFERENCE</objectDescription>
<sourceName>EAGLE PACE</sourceName>
<updateSource>EMLTEST</updateSource>
<effectiveDate>2022-01-30</effectiveDate>
<yieldCurveCode>CH</yieldCurveCode>
<duration>1</duration>
<parYield>4.935</parYield>
<maturityDate>2011-12-31</maturityDate>
<customExtensions>
<yieldCurveExt>
<custExt1>Ext1123</custExt1>
<custExt2>2022-01-28</custExt2>
<custExt3>321.456</custExt3>
</yieldCurveExt>
</customExtensions>
</yieldCurve>
</referenceTransaction>

Along with this enhancement, the yieldcurvecode filter was implemented. To use this filter, the yieldcurvecode parameter should be passed in the RTR message.

Usage example:
<taskParameter>
<name>yieldcurvecode</name>
<dataType>S</dataType>
<value>CH</value>
</taskParameter>

Files changed:
eagle_default/in/xml/include/custom_fields.inc
eagle_default/in/xml/xml-ref_yieldcurve.xml
eagle_ml-2-0_cm/dbkeysmap.inc
eagle_ml-2-0_cm/generate_sp_on_fly.inc
eagle_ml-2-0_cm/w_config.inc
eagle_ml-2-0_cm/out/tagvalue/extract_ref_yield_curve.inc
eagle_ml-2-0_cm/out/xml/t_eagleml-2-0_streaming.rsf
eagle_ml-2-0_cm/out/xml/t_ref_yield_curve_streaming.inc
dynamic/eagleml/swaggerswagger-extracts.json

SDP-58659 IssuerESG object was enhanced to support extensions

The IssuerESG object was enhanced to support extenstions for inbound and outbound interfaces.
Before adding the extension, an extension table should be created. The table must have the same key field as main table (SECURITYDBO.ISSUER_ESG): ISSUER_ALIAS. After the table is created, the IssuerESG extension can be added through the IWS xPanles tool.

Example of IssuerESG object with extension:
Node customExtensions - should always be present for all extensions. Node issuer_esg_extExt is created from extension name in xPanels and ‘Ext’ postfix.

<referenceTransaction>
<header>
<objectType>IssuerESG</objectType>
</header>
<issuerESG>
<objectType>IssuerESG</objectType>
<objectId>IssuerESG</objectId>
<objectDescription>REFERENCE</objectDescription>
<sourceName>EAGLE PACE</sourceName>
<effectiveDate>2015-03-08</effectiveDate>
...
<adultIntertainmentIndicator>N</adultIntertainmentIndicator>
<customExtensions>
<issuer_esg_extExt>
<esgExt1>Ext1</esgExt1>
<esgExt2>2022-02-02</esgExt2>
<esgExt3>888</esgExt3>
</issuer_esg_extExt>
</customExtensions>
</issuerESG>
</referenceTransaction>

Along with the enhancement there were some elements with misspelling that were fixed. They are as follows. socialWeight, governanceWeight and adultEntertainmentIndicator.

Files changed:
eagle_default/in/xml/include/custom_fields.inc
eagle_default/in/xml/xml-ref_issuer_esg.xml
eagle_ml-2-0_cm/dbkeysmap.inc
eagle_ml-2-0_cm/out/tagvalue/extract_issuer_esg.inc
eagle_ml-2-0_cm/out/xml/t_issuer_esg_streaming.inc
dynamic/metadata/bind/eagleml_ref_issueresg.xml

SDP-58311 WRHSTRADEEXTRACT - Signage Revisions

Changes were made to the follow fields for updated Signage Keys and Conditions .

SETTLEMENT_AMOUNT
TRADE_AMOUNT
NETSETTLEAMT
ACCRUED_INTEREST
BASEINTERESTADJ
SETTLEMENT_AMOUNT_BASE

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_wrhs_trade_signage_logic.inc

SDP-58874 Improved processing rule for EDS fieldattribute extract

The processing rule for EDS fieldattribute extract was improved to support new field -fieldAttributeOntologyDescription. The value for this field is populated from ontology using mashup file generated based on ontology element definition.

As a result, the EDS fieldattrbute extract and ontology are linked by the key: {{|mainds.TABLE_NAME| + '.' + |mainds.FIELD_NAME|}}

{noformat}"mashups": {
"ontology_metadata": "select first FIELD_DEFINITION from ontology_metadata where (|mainds.TABLE_NAME| + '.' + |mainds.FIELD_NAME|)=(|FIELD_PATH|)"
}{noformat}

This field will be part of the EDS extract.

More details on the extract: https://eagleinvsys.atlassian.net/wiki/spaces/MC2/pages/3333586953

Files changed:

EDS Component files:
src/eagleinvsys/rdf_metadata/rdf_canonical.py
src/eagleinvsys/rdf_metadata/rdf_meta_enum.py
src/eagleinvsys/rdf_metadata/rdf_meta_main.py
src/eagleinvsys/rdf_metadata/rdf_meta_utils.py
src/eagleinvsys/rdf_metadata/rdf_ontology_generator_templates.py
src/eagleinvsys/rdf_metadata/rdf_outflow.py
src/eagleinvsys/rdf_metadata/rdf_vocab.py

Ontology files:
src/resources/ontology/definition/eagle-vocab-definition-common.json
src/resources/ontology/eagle-mashups.json
src/resources/ontology/eagle-vocab-common.json
src/resources/ontology/eagledb-vocab-common.json
src/resources/ontology/reference/fieldattribute-models.json
src/resources/ontology/reference/fieldattribute.json

SDP-58779 EDS warehouseposition resource was updated to support additional fields for load and extract

EDS warehouseposition resource was updated to provide support for additional fields on the load and extract for both Oracle and Snowflake database providers.

List of new fields for the *HOLDINGDBO.POSITION_DETAIL* database table:

DB Name vs XSD element name
CALL_DATE < - > callDate
RATINGS_CHANGED < - > ratingsChangedFlag
HEDGE_LINK < - > hedgeLink

List of fields for the *HOLDINGDBO.POSITION_COST_DETAIL* database table*:*
DB Name vs XSD element name

ORIG_ACCRUED_INCOME_BASE < - > originalAccruedIncomeBase
TAX_PAYABLE_LOCAL < - > taxPayableLocal
TAX_PAYABLE_BASE < - > taxPayableBase
TAX_PAYABLE_SYS < - > taxPayableSys
ORIG_TAX_PAYABLE < - > originalTaxPayable

Files changed:
warehouseposition.json
warehouseposition_in.json
warehouseposition-models.json
eagle-vocab-acc-shared.json
eagle-vocab-definition-acc-shared.json
eagle-vocab-wh.json
eagle-vocab-definition-wh.json
eagle-vocab-shared.json
eagle-vocab-definition-shared.json

SDP-58766 The Ontology Query REST endpoint was improved as part of Eagle Metadata API

The Ontology Query REST endpoint was enhanced to return unique key, index and cache mode information as part of the response. The Endpoint works the same way as before but only the response was extended to include additional details. Those additional details are cache_mode, index and unique_key.

Cloud DEV endpoint:
https://apps.sd.dev.az.eagleinvsys.com:8443/api/vault/metadata/api/doc#/metadata-service-rest-api/getOntologyQuery
In-market endpoint:
http://10.80.104.23:20427/mc2/swagger-ui/index.html?configUrl=%2Fmc2%2Fswagger-config&urls.primaryName=METADATA#/ontology_query/getOntologyQuery

Files changed:
src/eagleinvsys/rdf_metadata/rdf_vocab.py
src/eagleinvsys/rdf_metadata/rdf_meta_main.py

SDP-58414 EDS and Data Cache functionality

The EDS core functionality related to work with Redis resource dataset cache has been improved. Currently, there are two types of dataset cache that exist in Redis cache. One for ontology based resources and the other for core datasets.

Please review corresponding pages for additional details on data cache functionality : https://eagleinvsys.atlassian.net/wiki/spaces/MC2/pages/2951086081

There are additional post-install steps for existing EDS clients: [https://eagleinvsys.atlassian.net/wiki/spaces/MC2/pages/3202908179/Deploying+EDS+core+code+and+Ontology+to+In-Market#4.-Refresh-Resource-Dataset-Cache-and-Data-Cache-Policies ] .

Files changed:

src/eagleinvsys/mc2py/idsdataframe.py

SDP-58133 Cache Policy logic was enhanced to allow refresh cache based on ontology or datasets

Delta Cache Update Policies for the following core resources have been added:

smfidentifiers - check for any change/update every 60 seconds

entityidentifiers - check for any change/update every 60 seconds

codevalue - check for any change/update every 30 seconds

codecategory - check for any change/update every 30 seconds

sources - check for any change/update every 30 seconds

If new records were loaded or existing records were updated for the above resources in the In-Market region, then after 30-60 seconds these changes will be automatically uploaded to the Redis Cache. The end users are no longer required to intervene and manually send a RTR message to refresh dataset cache in Eagle regions.

The code was improved to check the two locations below for resource dataset definitions:

App server folder - estar/tpe/dynamic/metadata/core/datasets

Redis cache

Prior to the changes, the Cache Update Manager was validating one of the above locations and if the dataset definition was not defined, then the error was raised despite the fact of existing definition in another location. Currently, Cache Update Manager checks both locations.

Files changed:

eagle/estar/tpe/dynamic/metadata/core/cachepolicies/codecategory.json
eagle/estar/tpe/dynamic/metadata/core/cachepolicies/codevalue.json
eagle/estar/tpe/dynamic/metadata/core/cachepolicies/entityidentifiers.json
eagle/estar/tpe/dynamic/metadata/core/cachepolicies/smfidentifiers.json
eagle/estar/tpe/dynamic/metadata/core/cachepolicies/sources.json
src/eagleinvsys/ebs/ebs_interfaces_manager.py

SDP-55031 Generic load workflow eagle_wrf_generic_load was enhanced to support EDS load

Generic load workflow eagle_wrf_generic_load was enhanced to support EDS load via MC. A few new parameters have been created to enable and disable EDS processing via the generic load workflow.

A new task parameter should be added UseEDS which is set to N by default:
<taskParameter>
<name>UseEDS</name>
<value>Y</value>
</taskParameter>

New global config parameter is also available W_USE_EDS:
<COL TAG="W_USE_EDS" EXPRESSION="'Y'"/>

Additionally, the parameter w_eds_load_params should be set to use EDS load via workflow eagle_wrf_generic_load for specific resource from config w_config_eds.inc:
<COL TAG="w_eds_load_params" EXPRESSION="'\!resourceName:warehouseposition:fileMask:_WAREHOUSEPOSITION_\!resourceName:genericsmf:fileMask:_GENERICSMF_'"/>
In case parameter w_eds_load_params does not exist or is empty, the generic load workflow will function as before without EDS Service.
Currently, only one file can be loaded for each resource for a single workflow execution.

Files changed:
estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/tagvalue/execute_wrkfl_event.xml
estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/tagvalue/set_eds_params.inc
estar/tpe/dynamic/msgcenter/eagle_ml-2-0_custom_cm/w_config_eds.inc

SDP-52549 The EDS entityaccountingbasis resource was enhanced

Load to oracle is supported via stored procedure PACE_MASTER.ESTAR_ENTITY_RULES.MODIFY_ACCT_BASIS.

The distinct expression was implemented to exclude duplicates on extract:

{{"|portfolioAcct| + |acctBasis| + iif(StringLength(convert(date(|effectiveDate|, 'yyyy-mm-dd'), 'yyyy-mm-dd')) = 0, '', convert(date(|effectiveDate|, 'yyyy-mm-dd'), 'yyyymmdd'))"}}

Files changed:
ontology/entity/entityaccountingbasis -models.json
ontology/entity/entityaccountingbasis_in.json
ontology/templates/eagle_ent_entityaccountingbasis_insertupdate.txt

SDP-58930 The EDS sources resource was enhanced

Load to Oracle is supported through the stored procedure PACE_MASTER.ESTAR_PACE.INSERT_UPDATE_SOURCE_INFO.

Files changed:
ontology/reference/sources -models.json
ontology/reference/sources _in.json
ontology/templates/eagle_ref_sources _insertupdate.txt

SDP-58904 The EDS codecategory resource was enhanced

Load to Oracle is supported by the stored procedure PACE_MASTER.ESTAR_CODES.INSERT_UPDATE_CODES_INFO.

Files changed:
ontology/reference/codecategory-models.json
ontology/reference/codecategory_in.json
ontology/templates/eagle_ref_codecategory_insertupdate.txt

Production Bugs

SDP-58399 WRHSTRADEEXTRACT - Defaulted the Trade Amount and Net Settlement Amount to 0 when it is a TRRESETCOST

Revised instances where the TRRESETCOST is the TRANS_CODE. Set the Trade Amount - EagleML/WarehouseTransaction/warehouseTrade/tradeAmount and Net Settlement Amount - EagleML/WarehouseTransaction/warehouseTrade/netsettleamt to default to 0. This logic mimics the Net Cash and Trade Base Amount for the TRRESETCOST transaction code only.

Files Changed
dynamic/msgcenter/eagle_ml-2-0_cm/out/tagvalue/extract_wrhs_trade_signage_logic.inc

SDP-58091 IssuerEntityRelationship object load fails if an Entity doesn't exist

Processing will be interrupted if attempting to load IssuerEntityRelationship objects for an entity that does not exist in the database. The resulting error message will be displayed as “Entity ID is empty or not found in the database.” Prior to the fix, it was possible to load IssuerEntityRelationship records to the database for an entity that did not exist.

File changed:
eagle_default/in/xml/xml-ref_issuerentityrelationship.xml

SDP-58160 Enhanced Amortize Notional Swaps CA Announcement security resolution

Security resolution for Amortize Notional Swaps CA Announcement was adjusted to select and process on the Contract leg (SWCOCR and SWCOIR).

File changed:
eagle_default/in/xml/xml-ref_ca.xml

SDP-46304 IssuerRole delete process was corrected for MSSQL environments

The delete process for Issuer Role failed on MSSQL environments due to an incorrect delete statement.
The delete logic was therefore corrected to provide the ability to delete single or multiple records for Issuer Role based on the combination of security_alias, issuer_alias and role_type key.

Files changed:
eagle_default/in/xml/include/ref_issuerrole_delete.inc

SDP-59070 buysell case element in Asset Properties model does not correspond to the XSD

The buysell element in the Asset Properties model was inconsistent with the XSD on the extract transform side. buySell element was returned instead of buysell. This case issue was fixed to return the proper case.

Files changed:
eagle_ml-2-0_cm/out/xml/t_asset_properties_fields.inc
eagle_ml-2-0_cm/out/xml/t_asset_properties_fields_start.inc
eagle_ml-2-0_cm/out/xml/t_asset_properties_fields_streaming.inc

SDP-59081 calcMasterTransactionId misspelling for TAXMERGP

calcMasterTransactionId had a misspelling for TRANS_CODE TAXMERGP. It was spelled as TAXMERGRP with an additional R that needed to be removed.

Files Changed

eagle_ml-2-0_cm/out/tagvalue/extract_wrhs_trade_signage_logic.inc
eagle_ml-2-0_cm/out/tagvalue/extract_warehouse_trade.inc