EagleML Release Notes - January 17, 2023
- Eric Fardig (Deactivated)
Enhancements
SDP-69212 Inactive Security Resolved Incorrectly
During an SMF update, the inactiveFlag was mapped from the call to the eagle/star/pan-qxref-s-msgcntr.htm panel. This panel does not have this variable in the outbound cursor and therefore, does not return and the inactiveFlag was nullified. As a result, during the next call of the panel, all the records were returned regardless if the inactiveFlag was supplied in the incoming message and an error indicating a mismatch was presented.
Tag11043 was deleted from all lookups which could not return it and the mismatch error has been eliminated.
Files changed:
eagle_default\in\xml\include\security_resolution_generic.inc
SDP-69028 IssuePrice Inbound and Outbound interfaces were enhanced to support new fields
An enhancement was made to the IssuePrice object to support additional fields for the Price Load Direct Database and the Extract - ISSUEPRICEEXTRACT.
The below mapping was added for the following fields and Xpath::
SECURITYDBO.PRICE.PAY_LEG_PRICE_OR_MV - Tag 7680 - EagleML/referenceTransaction/issuePrice/payPriceLeg/marketValue
SECURITYDBO.PRICE.REC_LEG_PRICE_OR_MV - Tag 7681 - EagleML/referenceTransaction/issuePrice/receivePriceLeg/marketValue
SECURITYDBO.PRICE.PRICE_CONV_FX_SOURCE - Tag 7108 - EagleML/referenceTransaction/issuePrice/priceConversionFxSource
SECURITYDBO.PRICE.PRICE_CONVERSION_FX_RATE - Tag 7682 - EagleML/referenceTransaction/issuePrice/priceConversionFxRate
SECURITYDBO.PRICE.ENTITY_ID - Tag 7107 - EagleML/referenceTransaction/issuePrice/pricingEntityId
SECURITYDBO.PRICE.PAY_LEG_NOTIONAL - Tag 7683 - EagleML/referenceTransaction/issuePrice/payPriceLeg/notionalAmount
SECURITYDBO.PRICE.REC_LEG_NOTIONAL - Tag 7684 - EagleML/referenceTransaction/issuePrice/receivePriceLeg/notionalAmount
SECURITYDBO.PRICE.PX_YEST - Tag 7728 - EagleML/referenceTransaction/issuePrice/pxYest
SECURITYDBO.PRICE.ASK_SCORE - Tag 18348 - EagleML/referenceTransaction/issuePrice/bloombergAskPriceScore
SECURITYDBO.PRICE.BID_SCORE - Tag 18349 - EagleML/referenceTransaction/issuePrice/bloombergBidPriceScore
SECURITYDBO.PRICE.MID_SCORE - Tag 18350 - EagleML/referenceTransaction/issuePrice/bloombergMidPriceScore
SECURITYDBO.PRICE.QUOTE_TYPE - Tag 10381 - EagleML/referenceTransaction/issuePrice/bloombergQuoteCode
SECURITYDBO.PRICE.CONFIDENCE_INTERVAL - Tag 10956 - EagleML/referenceTransaction/issuePrice/confidenceInterval
Files changed:
eagle/estar/tpe/dynamic/metadata/bind/eagleml_ref_issueprice.xml
eagle/estar/tpe/dynamic/msgcenter/eagle_default/in/xml/xml-ref_issueprice.xml
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/xml/t_issueprice_streaming.inc
SDP-68808 IssuePrice, ExchangePrice and IssuePcPrice inbound interfaces were enhanced to support SWAP resolution by the Pay Receive Indicator
SWAP resolution for Prices has specific logic based upon the Processing Security Type and only one leg could be priced. The logic was updated to support SWAP resolution based on the Security Resolution and Pay Receive Indicator.
The logic is an opt-in feature and only available when the processing option PRICE_RESOLVE_SW_BY_PAYRECIND is set in the incoming file or as a global parameter where the parameter, W_PRICE_RESOLVE_SW_BY_PAYRECIND, is applied to the w_config_custom.inc.
Usage:
{noformat}<header>
<objectType>IssuePrice</objectType>
<processingOptions>PRICE_RESOLVE_SW_BY_PAYRECIND</processingOptions>
</header>{noformat}
or
<COL TAG="W_PRICE_RESOLVE_SW_BY_PAYRECIND" EXPRESSION= "'Y'"/>
Files changed:
eagle_default/in/xml/xml-ref_issueprice.xml
eagle_default/in/xml/xml-ref_exchangeprice.xml
eagle_default/in/xml/xml-ref_issuepcprice.xml
eagle_default/in/xml/include/xml-ref_security_pricing_common.inc
SDP-68567 Fixed Index Hint in MSSQL Lock Query
There was identified slowness in the Orchestration Queue which has been addressed by changing the index hint from PK_ORCH_QUEUE to use EX2_ORCH_QUEUE in SQL Server only.
Files changed:
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/lock_req_state.inc
SDP-68508 New Elements added to the core Entity Share Class object
Entity Share Class Extract was enhanced to support the below 2 elements:
RULESDBO.ENTITY_SHARE_CLASS.CLASS_HEDGING - Tag 18045 - EagleML/entityTransaction/entityShareClass/classHedging
RULESDBO.ENTITY_SHARE_CLASS.HEDGING_METHOD - Tag 18451 - EagleML/entityTransaction/entityShareClass/hedgingMethod
Files changed:
eagle_ml-2-0_cm\out\xml\t_entity_share_class.xml
eagle_ml-2-0_cm\out\xml\t_entity_share_class_streaming.inc
SDP-67703 Updated Element names in the EDS warehousetrade resource
Mappings for the gainSecurityLocal and lossSecurityLocal were changed to be populated from the TOTAL_GAIN_LOCAL and TOTAL_LOSS_LOCAL respectively. New elements gainSecurityBase and lossSecurityBase were added and mapped to the SECURITY_GAIN and SECURITY_LOSS. Changes were applied both for the extract and the load.
Files changed:
metadata/core/ontology/definition/eagle-vocab-definition-wh-shared.json
metadata/core/ontology/eagle-vocab-wh-shared.json
metadata/core/ontology/warehouse/warehousetrade-models.json
metadata/core/ontology/warehouse/warehousetrade_in.json
SDP-67503 Separate entity-level price source for Private Equity
The GenericEntity and EntityAccountingBasis Inbound and Outbound (ENTITYACCBASISEXTRACT and ENTITYEXTRACT) interfaces were enhanced to support the new element privateEquityPriceSource.
EagleML XPath:
EagleML/entityTransaction/genericEntity/acctBasis/privateEquityPriceSource
EagleML/entityTransaction/entityAccountingBasis/privateEquityPriceSource
Field in database: RULESDBO.ENTITY_ACCT_BASIS.PRIVATE_EQUITY_PRICE_SOURCE
Files changed:
eagle_default/in/include/entity_acct.inc
eagle_default/in/include/entity_aliases.inc
eagle_default/in/include/entity_panels.inc
eagle_default/in/xml/xml-ent_entityaccountingbasis.xml
eagle_ml-2-0_cm/out/tagvalue/extract_entity.inc
eagle_ml-2-0_cm/extract_language_list.inc
eagle_ml-2-0_cm/out/xml/t_entity.xml
eagle_ml-2-0_cm/out/xml/t_entity_streaming.inc
eagle_ml-2-0_cm/out/xml/t_entity_streaming_eml.inc
eagle_ml-2-0_cm/out/xml/t_entity_accounting_basis.xml
eagle_ml-2-0_cm/out/xml/t_entity_accounting_basis_streaming.inc
SDP-66635 The fieldattribute extract was extended by RDC information
The below five additional fields are now currently supported in our fieldattribute object:
fieldAttributeReference
enrichmentRule
validation
rdcFieldGroup
rdcDataStrategy
Additional logic was introduces to support group values where new a TEMP table is used for join. Grouped values are separated by the semicolon symbol.
Files changed:
Ontology:
estar/tpe/dynamic/metadata/core/ontology/definition/eagle-vocab-definition-ref-shared.json
estar/tpe/dynamic/metadata/core/ontology/eagle-vocab-ref-shared.json
estar/tpe/dynamic/metadata/core/ontology/reference/fieldattribute-models.json
estar/tpe/dynamic/metadata/core/ontology/reference/fieldattribute.json
Common Core:
src/common_core/dataset/ds_filters/fieldattribute_filter/fieldattribute_filter_cursor_handler.py
src/common_core/dataset/ds_filters/fieldattribute_filter/fieldattribute_filter_sql.py
SDP-66525 EDS exchnageprice resource was enhanced to support new fields in PRICE_EXCHANGE table
The EDS exchangeprice Resource was enhanced to support the below fields for both load and extract to SECURITYDBO.PRICE_EXCHANGE tables for both Oracle and Snowflake database providers.
New elements:
referenceTransaction/exchangePrice/pricingEntityId
referenceTransaction/exchangePrice/priceConversionFxRate
referenceTransaction/exchangePrice/priceConversionFxSource
referenceTransaction/exchangePrice/pxYest
referenceTransaction/exchangePrice/payPriceLeg/notionalAmount
referenceTransaction/exchangePrice/payPriceLeg/price
referenceTransaction/exchangePrice/payPriceLeg/marketValue
referenceTransaction/exchangePrice/receivePriceLeg/notionalAmount
referenceTransaction/exchangePrice/receivePriceLeg/price
referenceTransaction/exchangePrice/receivePriceLeg/marketValue
Files changed:
metadata/core/ontology/definition/eagle-vocab-definition-ref-shared.json
metadata/core/ontology/eagle-vocab-ref-shared.json
metadata/core/ontology/reference/exchangeprice-models.json
metadata/core/ontology/reference/exchangeprice.json
metadata/core/ontology/reference/exchangeprice-in.json
SDP-66524 EDS issueprice resource was enhanced to support new fields
EDS issueprice resource was enhanced to support the below fields for both load and extract to SECURITYDBO.PRICE tables for both Oracle and Snowflake database providers.
New elements:
referenceTransaction/issuePrice/confidenceInterval
referenceTransaction/issuePrice/pricingEntityId
referenceTransaction/issuePrice/priceConversionFxRate
referenceTransaction/issuePrice/priceConversionFxSource
referenceTransaction/issuePrice/pxYest
referenceTransaction/issuePrice/payPriceLeg/notionalAmount
referenceTransaction/issuePrice/payPriceLeg/price
referenceTransaction/issuePrice/payPriceLeg/marketValue
referenceTransaction/issuePrice/receivePriceLeg/notionalAmount
referenceTransaction/issuePrice/receivePriceLeg/price
referenceTransaction/issuePrice/receivePriceLeg/marketValue
Files changed:
metadata/core/ontology/reference/issueprice-models.json
metadata/core/ontology/reference/issueprice.json
metadata/core/ontology/reference/issueprice-in.json
SDP-64040 Added support for IN_UPDATE_INSERT_FLAG in Warehouse GL Balances
Logic has been added into the warehouse leger object to support incrementing and deleting the existing ledger balance value in the warehouse ledger table.
Logic for XMLDBAPI format: Implemented logic adding the IN_UPDATE_INSERT_FLAG parameter to the database request with a default value of Y. Possible values I - INCREMENT; Y – OVERRIDE; and D - DELETE. If the parameter is presented without a value, it will be set to Y.
Logic for the EagleML format: the 'action' element was added to the inbound EagleML interface with a default value of Y. Possible values I or INCREMENT; Y or OVERRIDE; and D or DELETE. If the action element is not present or has an incorrect value, ‘Y’ will be passed to the Stored Procedure. The action element is case insensitive.
Files changed:
dynamic/msgcenter/eagle_default/in/xml/include/xml-warehouse_common.inc
dynamic/msgcenter/eagle_default/in/xml/xml-wh_gl_detail_posting.xml
dynamic/bind/eagleml_wrhs_warehouseglbalances.xml
SDP-16640 Added SINK schedules loading logic for tags 54 and 1670
Logic for tags 54 - Corporate Action Status and 1670 - Schedule Status were added to the sink schedule object. If Tag 1250 - Schedule Sub Type is equal to ‘SINK’ then it will now null out the value for tag 1670, else there is no change to this tag. Added in mapping for tag 54 in the rule file to find the current status of the sink schedule.
Files changed:
eagle/estar/tpe/dynamic/msgcenter/eagle_default/in/xml/xml-ref_schedule_mi.xml
SDP-68797 Added Support for the Floating Price Source in Fixed Income SMF Object
GenericSMF inbound and outbound interfaces were enhanced to support the below elements:
SECURITYDBO.FIXED_INCOME.VARIABLE_RATE_FLAG - Tag 16727 - referenceTransaction/genericSMF/FixedIncomeModel/floatingPriceInstrumentIndicator
SECURITYDBO.FIXED_INCOME.VARIABLE_RATE_SOURCE - Tag 16728- referenceTransaction/genericSMF/FixedIncomeModel/floatingPriceSource
In accounting mode, added a condition to process these tags only for the DBSTST and EQEQMF processing Security Types.
Files changed:
eagle_default/in/xml/include/fixedincomesmf_model.inc
eagle_default/in/xml/include/smf_aliases.inc
eagle_default/in/xml/xslt/smf_egl_gen_interface_template.xsl
eagle_ml-2-0_cm/out/xml/t_smf_streaming.inc
eagle_ml-2-0_cm/out/xml/t_smf_streaming_eml.inc
SDP-66635 The fieldattribute extract was extended by RDC information
Five additional fields are currently supported:
fieldAttributeReference
enrichmentRule
rdcValidation
rdcFieldGroup
rdcDataStrategy
Additional logic was introduced to support group values where the new TEMP table is used during a join. Grouped values are separated by the semicolon “;” symbol.
Files changed:
Ontology:
estar/tpe/dynamic/metadata/core/ontology/definition/eagle-vocab-definition-ref-shared.json
estar/tpe/dynamic/metadata/core/ontology/eagle-vocab-ref-shared.json
estar/tpe/dynamic/metadata/core/ontology/reference/fieldattribute-models.json
estar/tpe/dynamic/metadata/core/ontology/reference/fieldattribute.json
Common Core:
src/common_core/dataset/ds_filters/fieldattribute_filter/fieldattribute_filter_cursor_handler.py
src/common_core/dataset/ds_filters/fieldattribute_filter/fieldattribute_filter_sql.py
SDP-68596 All EDS resources are supported by the conversion service and conversion workflow
By default all EDS extracts are supported except resources that do not have either the effectivedate nor updatedate filters.
*The exception list of non-supported resources,* based on EDS extracts:
entitymanager
controlcentereventlog
calendarbusinessdates
calendarholiday
earningstimeperiod
entityentitlementlist
errordefinition
gipscomposite
gipsdisclosures
issueresg
processcenter
securitydefault
securitynotes
smfidentifierconfig
spddeltatriggerlog
user
SDP-69498 Ontology for WarehouseCashActivity resource was updated
The Datatype and length for field COMMENTS1 was increased from VARCHAR(100) to VARCHAR(255)
Files changed:
warehousecashactivity-models.json
SDP-69687 Data rules improvements
There was a set of changes introduced to Data rules Metadata to improve overall data processing:
Error message count to limit max number of error messages
Support RIMES XML attribute format as part of schema drift
Do not raise an error on Empty file processing
Ontology was updated for exchangeprice and issueprice extract, the field PX_VOLUME must be Integer
The RTR for the buildexecutionplan as part of conversion workflow was improved
SDP-67902 Ontology was updated for the GenericEntity resource
The length for a set of string elements was increased as part of the genericentity resource:
ENTITY.PRIMARY_TRADING_SYSTEM changed length from 20 to 100
ENTITY.ENTITY_SUB_TYPE changed length from 20 to 30
ENTITY_EXTENSION.USER_XL_CHAR1changed length from 255 to 1000
ENTITY_EXTENSION.STRATEGY changed length from 10 to 100
ENTITY_EXTENSION.SUB_STRATEGY changed length from 10 to 100
Files changed:
metadata/core/ontology/entity/genericentity-models.json
SDP-68596 All EDS resources are supported by Conversion Workflow
By default all EDS extracts are supported except resources that don have neither effectivedate nor updatedate filters.
*The exception list of non-supported resources,* based on EDS extracts:
entitymanager
controlcentereventlog
calendarbusinessdates
calendarholiday
earningstimeperiod
entityentitlementlist
errordefinition
gipscomposite
gipsdisclosures
issueresg
processcenter
securitydefault
securitynotes
smfidentifierconfig
spddeltatriggerlog
user
Production Bugs
SDP-68862 SPDDELTATRIGGERLOGEXTRACT enhancement to add in date filter when the spd_delta_trigger_log_hist is not purged
The SPDDELTATRIGGERLOGEXTRACT was enhanced by adding dates conditions to join the filter for the PRICE_FX_EXCEPTION query. This was done in the event that the PRICE_FX_EXCEPTION was not purged which may cause additional rows in the table and the extract would have incorrect data.
Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_spd_delta_trigger_log.inc
SDP-68625 Logic for SOURCE_DATE, RELEASE_DATE fields
New canonical elements were added to eagle-vocab:
releaseTimestamp
releaseTimeStamp
*Extract:*
Date from SOURCE_DATE is mapped to sourceDate
Timestamp from SOURCE_DATE is mapped to sourceTimeStamp
Date from RELEASE_DATE is mapped to releaseDate
Timestamp from RELEASE_DATE is mapped to releaseTimeStamp
*Load:*
Load to Snowflake is using logic:
SOURCE_DATE
Load sourceTimeStamp if it is provided, else load sourceDate
RELEASE_DATE
Load releaseTimeStamp if it is provided, else load releaseDate
*Enhanced resources:* genericsmf, factor
SDP-69081 Data rules extensions functionality was improved
The logic for EDS extensions was improved to support extensions off the composite and off the history tables for both database providers - Oracle & Snowflake. Data upload to extension tables is supported in insert and update mode.
SDP-69748 Warehouse Openlot Rollup Procedure was Improved to Work Without PositionCostLot Data
The Warehouse Openlot Rollup procedure HOLDINGDBO.EGL_GEN_HOLDING.LOTFINAL_EGL_GEN skips the summation if the table POSITION_COST_LOT does not have a record. In EagleML, the core procedure LOT_EGL_GEN_INTERFACE creates a record in the POSITION_COST_LOT every time during the Warehouse Openlot load. EDS does not create a record if there is no data. As a result, the Rollup procedure LOTFINAL_EGL_GEN worked incorrectly.
In EDS, the core procedure LOTFINAL_EGL_GEN is called by dynamic SQL block. This dynamic SQL block was improved to execute the Warehouse Openlot Rollup procedure every time.
Files changed:
eagle_wrhs_warehouseopenlot_reinitbatch.oracle.txt
eagle_wrhs_warehouseopenlot_rollup.oracle.txt