Enhancements

SDP-68647 Added rounding for assetToBaseFxRate element

The SETTLEDCAEXTRACT and TRADEDCAEXTRACT include the element assetToBaseFxRate.

This element is dynamically calculated during the extract and does not have a corresponding database field. The element calculates the asset currency to the base currency fx rate.

To include this element in extract set the parameter IncludeAssetToBaseFxRate to Y in extract RTR:

The calculation of the element was updated to round to 12 decimals.

Files changed:

eagle_ml-2-0_cm/out/tagvalue/extract_acct_settled_cash_activity.inc
eagle_ml-2-0_cm/out/tagvalue/extract_acct_traded_cash_activity.inc

SDP-68023 Security Resolution for EDS load for Oracle was improved

EDS Security Resolution was improved for Oracle and Snowflake. The following Security resolution logic is in place on data upload:

• Resolution mashup logic will be applied for all xrefIdentifiers instead of just the first xId1 pair as it was before.

• If the resolution via the mashup fails, then the stored procedure will be called. A new Stored Procedure similar to the Oracle procedure was created for Snowflake.

By default, the USE_FIRST_AS_MAIN will be used. Additional security resolution options and Batting Order are also supported.

SDP-68518 EDS Security Resolution - MATCH_ALL Option

EDS Security resolution logic was enhanced to support MATCH_ALL mode in which all identifiers specified in the battingOrder node are used for resolution, so a security is resolved only when ALL identifiers belong to the same security.

Files changed:

src/eagleinvsys/etl/core_etl/resolution_service.py
ontology/resolution/defaultassetresolutionservice.json
ontology/templates/includes/defaultassetresolutionservice.jso

SDP-68498 Expression support for DbCall T-Item in data rules interfaces

Expression support was added for the dbcall T-Item in the Data Rules interfaces to use dynamic SQL queries.

The new parameter should be added after the condition for execution to process the SQL query as expression.

Usage example:

{noformat}[dbcall, get_event_id,

              [EVENT_ID, :EVT_ID:],
              [EVT_ID, EVT_ID],
              [SQL, 'SELECT EVT_ID FROM '+:ESTAR_PARTITION_NAME:+'.ESTAR_FINANCIAL_EVENT WHERE 1=1 and EVT_ID = :EVENT_ID'],
              [:is_sell_subtype: = 'Y' and :sell_lot_selection_method: = 'IDLOT'],
              [Y]
]{noformat}

The new expression parameter can be set only to Y and does not expect any expressions.

Files changed:
eagleinvsys/etl/core_etl/genetl_dbcall.py
eagleinvsys/etl/core_etl/genetl_items_enum.py
tests/ebstests/test_ebs_with_mock.py

SDP-68351 Extract file name pattern was adjusted as part of Conversion Workflow

The pattern for the conversion service extract files was changed to synchronize with the Cloud integration.

From:
{noformat}@CORRID_CONVERSION_{from_effective_date or START}_{to_effective_date or END}_@FeedType_@DateStamp.xml{noformat}
To:
{noformat}@CORRID_ASISCOPY_CONVERSION_{from_effective_date or START}_{to_effective_date or END}_@FeedType_@DateStamp.xml{noformat}

SDP-68295 New Processing Option APPLY_DIRECT_MAPPING added for the GenericSMF

The GenericSMF inbound interface was updated to support a new processing option APPLY_DIRECT_MAPPING for the Stored Procedure mode.

This option allows to maintain the genericSMF/FixedIncomeModel/delayDays value with the genericSMF/MBSModel/delayDays value when the first one is empty in the incoming message.

<header>
<processingOptions>APPLY_DIRECT_MAPPING</processingOptions>
<objectType>GenericSMF</objectType>
</header>

Files changed:
eagle_default/in/xml/include/generic_smf_fields_logic.inc

SDP-68257 The Signage logic for DL_OPEN_COST_LOCAL was updated for the WRHSTRADEEXTRACT

The WRHSTRADEEXTRACT - Warehouse Trade - TRADESDBO.DISPOSAL_LOTS.OPEN_COST_LOCAL field was modified.

DISPOSAL_LOTS.OPEN_COST_LOCAL - DL_OPEN_COST_LOCAL

EagleML/WarehouseTransaction/warehouseTrade/openCostLocal

[Fields Affected by Signage Logic ]

Signage logic for the Open Cost Local is enabled through an opt-in element with the EnableSignageLogic set to Y.

Signage is enabled when the following parameters are included:
<taskParameter>
<name>EnableSignageLogic</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_wrhs_trade_signage_logic.inc
eagle_ml-2-0_cm/out/tagvalue/extract_warehouse_trade.inc
eagle_ml-2-0_cm/out/xml/t_warehouse_trade_streaming.inc

SDP-67839 Exchange Rate Inbound and Outbound interfaces were enhanced to support new fields

Additional fields were added to the Load - ExchangePrice and Extract - ISSUEEXCHANGEPRICEEXTRACT for the Exchange Rate to PRICE_EXCHANGE table.

Mappings were added for the following fields and Xpath::

SECURITYDBO.PRICE_EXCHANGE.PAY_LEG_PRICE_OR_MV - Tag 7680 - EagleML/referenceTransaction/exchangePrice/payPriceLeg/marketValue
SECURITYDBO.PRICE_EXCHANGE.REC_LEG_PRICE_OR_MV - Tag 7681 - EagleML/referenceTransaction/exchangePrice/receivePriceLeg/marketValue
SECURITYDBO.PRICE_EXCHANGE.PRICE_CONV_FX_SOURCE - Tag 7108 - EagleML/referenceTransaction/exchangePrice/priceConversionFxSource    
SECURITYDBO.PRICE_EXCHANGE.PRICE_CONVERSION_FX_RATE - Tag 7682 - EagleML/referenceTransaction/exchangePrice/priceConversionFxRate    
SECURITYDBO.PRICE_EXCHANGE.PX_YEST - Tag 7728 - EagleML/referenceTransaction/exchangePrice/pxYest      
SECURITYDBO.PRICE_EXCHANGE.ASK_SCORE - Tag 18348 - EagleML/referenceTransaction/exchangePrice/bloombergAskPriceScore   
SECURITYDBO.PRICE_EXCHANGE.BID_SCORE - Tag 18349 -  EagleML/referenceTransaction/exchangePrice/bloombergBidPriceScore  
SECURITYDBO.PRICE_EXCHANGE.MID_SCORE - Tag 18350 - EagleML/referenceTransaction/exchangePrice/bloombergMidPriceScore   
SECURITYDBO.PRICE_EXCHANGE.QUOTE_TYPE - Tag 10381 - EagleML/referenceTransaction/exchangePrice/bloombergQuoteCode
SECURITYDBO.PRICE_EXCHANGE.ENTITY_ID - Tag 7107 - EagleML/referenceTransaction/exchangePrice/pricingEntityId    
SECURITYDBO.PRICE_EXCHANGE.PAY_LEG_NOTIONAL - Tag 7683 -  EagleML/referenceTransaction/exchangePrice/payPriceLeg/notionalAmount   
SECURITYDBO.PRICE_EXCHANGE.REC_LEG_NOTIONAL - Tag 7684 - EagleML/referenceTransaction/exchangePrice/receivePriceLeg/notionalAmount    

Files changed:
eagle/estar/tpe/dynamic/metadata/bind/eagleml_ref_exchangeprice.xml
eagle/estar/tpe/dynamic/msgcenter/eagle_default/in/xml/xml-ref_exchangeprice.xml
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/generate_sp_on_fly.inc
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/xml/t_issueprice_exchange_streaming.inc

SDP-66895 The Split rule for reference objects was updated to support IssuerXref interface

There was an issue with duplicate records being loaded through the IssuerXref interface. If two records with the same Xref Type are loaded in parallel they sometimes will both get inserted which created duplicates.

The load through this interface was changed to be sequential eliminating the potential for duplicates to be loaded in parallel and enforcing the edit check to catch all of the duplicates.

The split key for Issuer Xref inbound interface was added and now it is IssuerId

Files changed:
eagle_default/in/xml/xml-ref_objects.spl

SDP-66769 Set Position Balances To Null When Batch Execution Of Open Lots Or Positions Fails

When running Batch Mode, all records are initially deleted within the Open Lots and Position related tables. A roll up process updates the position table upon completion. When the Batch load failed, the roll up process did not initiate and the position balances were not recalculated from the lots.

Changes were made for when the Batch load fails, positions are updated and the balances are nullified.

The following fields of the POSITION table are involved in the balances calculation and will be set to null upon a failure:       

File changed:
eagle_default/in/xml/include/xml-dbdirect_wrhs_delete_statement.inc

SDP-66594 Impairments inbound interface was enhanced to support Average Cost Entities

The logic of Impairments inbound interface was updated to write down Position Impairments for Average Cost Entities. Conditions for the pan-qentitysummary_portid.htm and pan-qasofpositiononesecurityalias.htm lookups were updated in order to find the correct data for Average Cost Entities

Files changed:
eagle_default/in/xml/xml-impairments.xml

Production Bugs

SDP-67035 Issuer Organization interface was updated for extension processing

The Issuer Id resolution and load process was updated to avoid duplicates in the Database. By default, the IssuerId is converted to uppercase during the resolution and the load process.

For backward compatibility, there is an ISSUER_ID_CASE_SENSITIVE processing option that returns the old resolution and load process.

<header>
<processingOptions>ISSUER_ID_CASE_SENSITIVE</processingOptions>
<objectType>GenericSMF</objectType>
</header>

The Issuer Organization inbound and outbound interfaces were updated to support extensions in history mode.

The history extension table name should have the pattern [master_table_name]_HIST. The history table should contain the ISSUER_ID, EFFECTIVE_DATE and the SRC_INTFC_INST fields, and they be unchecked when creating the extension. The ISSUER_ALIAS cannot be part of the key as it is not business key.

The following behavior is expected:

• historyOnlyFlag = Y and new record

  • If it is a new record in the Issuer Organization table, the record is inserted in the master and the history extension tables

• historyOnlyFlag = Y and record exists

  • If the record exists in the Issuer Organization table, the record is inserted/updated only in the history extension tables

• historyOnlyFlag != Y

  • The record is inserted/updated in the master and the history table

• If there is no history extension table, but historyOnlyFlag = Y then a warning is shown and data is inserted/updated into the master table in any mode. On the extract - ISSUERORGEXTRACT, the data is not extracted from the extension tables if there is historyOnlyFlag = Y but no History extension tables.

Files changed:
eagle_default\in\xml\include\custom_fields.inc
eagle_default\in\xml\include\issuer_xref_resolution.inc
eagle_ml-2-0_cm\dbkeysmap.inc
eagle_ml-2-0_cm\extract_extension_table_fields.inc

SDP-65515 Added Missing Schedule Fields to the Inbound/Outbound EagleML Interfaces

The below schedule elements are now supported for loading and extracting via EagleML interfaces.

SECURITYDBO.SCHEDULE.BASE_CURR_SCH - Tag 3676
SECURITYDBO.SCHEDULE.PUT_NOTIFICATION_MAX_DAYS - Tag 1657
SECURITYDBO.SCHEDULE.PUT_NOTIFICATION_MIN_DAYS Tag 1658
SECURITYDBO.SCHEDULE.EXERCISE_PRICE -Tag 3661

Xpath:

Load within GenericSMF:
EagleML/referenceTransaction/genericSMF/schedule/baseCurrSch
EagleML/referenceTransaction/genericSMF/schedule/putNotificationMaxDays
EagleML/referenceTransaction/genericSMF/schedule/putNotificationMinDays
EagleML/referenceTransaction/genericSMF/schedule/exercisePrice

Load/Extract (SCHEDULEEXTRACT) Schedule object:

EagleML/referenceTransaction/schedule/baseCurrSch
EagleML/referenceTransaction/schedule/putNotificationMaxDays
EagleML/referenceTransaction/schedule/putNotificationMinDays
EagleML/referenceTransaction/schedule/exercisePrice

XSD changes:

eagle/estar/tpe/dynamic/eagleml/schema/eagleml-sec-shared-2-0.xsd
eagle/estar/tpe/dynamic/eagleml/schema/eagleml-shared-2-0.xsd
eagle/estar/tpe/dynamic/eagleml/schema/eagleml-ref-shared-2-0.xsd

Files changed:
eagle/estar/tpe/dynamic/metadata/bind/eagleml_ref_schedule.xml
eagle/estar/tpe/dynamic/msgcenter/eagle_default/in/xml/xml-ref_schedule.xml
eagle/estar/tpe/dynamic/msgcenter/eagle_default/in/xml/xml-ref_schedule_mi.xml
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/xml/t_schedule_streaming.inc

SDP-64401 New Logic for the SOURCE_DATE, RELEASE_DATE fields

New canonical elements were added to the eagle-vocab:
releaseTimestampE
releaseTimeStamp

*Mapping was corrected on Extract side:*

Date from SOURCE_DATE is mapped to sourceDate

Timestamp from SOURCE_DATE is mapped to sourceTimeStamp

Date from RELEASE_DATE is mapped to releaseDate

Timestamp from RELEASE_DATE is mapped to releaseTimeStamp

*Load:*

Load to Snowflake is using logic:
SOURCE_DATE

Load sourceTimeStamp if it is provided, else load sourceDate

RELEASE_DATE

Load releaseTimeStamp if it is provided, else load releaseDate

*Enhanced resources:* genericsmf, factor

SDP-63753 First load of a security to prevent uncomposited/unvalidated data landing on the Gold Copy record.

Multi Leg Swap to load history only on the first load of a security to prevent uncomposited/unvalidated data landing on the Gold Copy record.

Files changed:

eagle_default\in\xml\xml-smf_multileg_swap.xml
eagle_default\in\xml\include\smf_panels.inc
eagle_default\in\xml\include\smfobject.inc
eagle_default\in\xml\include\smfaccobject.inc
eagle_default\in\xml\include\security_resolution_generic.inc

SDP-67267 Custom Extension Tables

load history only on the first load of a security to prevent uncomposited/unvalidated data landing on the Gold Copy record.

Files changed.

eagle/estar/tpe/dynamic/msgcenter/eagle_default/in/xml/include/wrhs_ext_custom_fields.inc

eagle/estar/tpe/dynamic/msgcenter/eagle_default/in/xml/include/xml-warehouse_common.inc

eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/dbkeysmap.inc