Enhancements

SDP-83635 IRS Swap Multileg Trade was enhanced with additional mapping for settlement fields

New tags are added to mapping for the IRS Swap Multileg trade:

Tag 17989 - <commissionSettlementAmount>
Tag 17988 - <otherFeeSettlementAmount>
Tag 18397 - <swapSettlementAmount>
Tag 66 - <settlementExhangeRate>

Corrected typo for EagleML element from comimissionSettlementAmount to _commissionSettlementAmount_

Files changed:
eagle_default/in/xml/include/multiplelegtrade_aliases.inc
eagle_default/in/xml/include/tradeleg_aliases.inc
eagle_default/in/xml/xml-accttrade_multipleleg.xml

SDP-82556 Entity Extract was enhanced to include entity history details

A New RTR parameter ‘includeEntityDetailHistory’ was introduced for ENTITYEXTRACT feedtype. Supported values are Y/N.

<taskParameter>
<name>includeEntityDetailHistory</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>

If enabled, then the ENTITYEXTRACT returns all entity detail historical records for the specific entity.
EffectiveDate filter is also available.

The parameter historyOnly which runs history mode only overrides this option.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_entity.inc
eagle_ml-2-0_cm/out/xml/t_entity_streaming_eml.inc

SDP-82226 Сash Segregation as part of WRHSPOSITIONEXTRACT

Custodian Bank Account elements available now within the Position Extract - WRHSPOSITIONEXTRACT as part of the support for cash segregation. 

The following fields are supported:
BankAcctNum - ENTITY_CUSTODY_RELATIONSHIP.CUSTODY_BANK_ACCT_NUM
BankCode – ENTITY_CUSTODY_RELATIONSHIP.CUSTODY_BANK_CODE
BankDesc - ENTITY_CUSTODY_RELATIONSHIP.CUSTODY_BANK_DESC

EagleML Xpath:
EagleML/warehouseTransaction/warehousePosition/custodyBankAccountNumber
EagleML/warehouseTransaction/warehousePosition/custodyBankCode
EagleML/warehouseTransaction/warehousePosition/custodyBankDesc

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_warehouse_position.inc
eagle_ml-2-0_cm/out/xml/t_warehouse_position_streaming.inc

SDP-82048 The MultilegSMF Interface was enhanced to support Time Sensitive functionality

The EagleML MultilegSMF interface was enhanced to support Time Sensitive functionality. Currently the Time Sensitive functionality is working for the initial SMF load and for the SMF update. This functionality can be enabled by passing the FixedIncomeModel/timeSensitiveIndicator = Y in the MultilegSMF message in the PAY or the RECEIVE leg record.

A new element, asOfEfectiveDate, is introduced for MultilegSMF.

File changed:
eagle_default/in/xml/xml-smf_multileg_swap.xml

SDP-84106 Cloud workflow eagle_wrf_cloud_extract was updated

ebs:default or ebs:[resourcename] prefix is required for all parameters when running the eagle_wrf_generic_extract workflow for EDS Extracts.

The eagle_wrf_cloud_extract workflow did not have correct parsing for the calculatechecksum and usedeltabatch parameters which resulted to the workflow not including them correctly.

Parsing of the calculatechecksum and usedeltabatch parameters was updated for the eagle_wrf_cloud_extract workflow to pass the parameters with the correct prefix for further processing via the eagle_wrf_generic_extract workflow.

Files changed:
eagle_ml-2-0_workflows/eagle_wrf_cloud_extract.wrf

SDP-83664 Profile Templates for Cloud Workflows were updated for many resources

The usedeltabatch parameter was set incorrectly in profiles.

eds:default or eds:[resourcename] prefix is required.

The eds:default:usedeltabatch parameter was updated for the cloud profile core templates for below resources:

warehouseposition, warehouseopenlot, warehousecloselot, warehousetrade, warehousecashactivity, warehousenav, warehousenavmulti, warehousenavmonthend, warehouseperformance, custombenchmark, warehouseglbalances, tradedcashactivity, settledcashactivity, smfidentifier

SDP-83935 The Filters todate and fromdate were corrected for tradedcashactivity and settledcashactivity resources

The todate and fromdate filter data types were set as DateTime instead of Date for the tradedcashactivity and settledcashactivity resources.

Files changed:
estar/tpe/dynamic/metadata/core/ontology/accounting/tradedcashactivity.json
estar/tpe/dynamic/metadata/core/ontology/accounting/settledcashactivity.json

SDP-79888 New Data Rules Resources - securityjournal and genericjournal to support Delete functionality

To enable delete functionality, enabled new resources genericJournal/securityJournal as part of the E2V pipeline which work the same way as any other Data Rules resource. These resources are very similar, the only difference is that securityjournal is designated to Security information and genericjournal to Entity.

genericjournal resource supports PACE_MASTERDBO.JOURNAL_MASTER and PACE_MASTERDBO.GENERIC_JOURNAL_DETAILS
securityjournal resource supports PACE_MASTERDBO.SECURITY_JOURNAL_MASTER, PACE_MASTERDBO.GENERIC_JOURNAL_DETAILS

Load on vault side works via the Reinit stored procedure because the load happens into tables with a one-to-many relation. The difference with the traditional load is that both resources have an extra Stored Procedure to execute after data was moved to Vault.
PACE_MASTERDBO.GENERIC_JOURNAL_EAGLE_SYNC and PACE_MASTERDBO.SECURITY_JOURNAL_EAGLE_SYNC SP execute delete in the Vault based on data received from Eagle.
In other words, when delete happens on the Eagle Side, new record(s) are inserted into journaling tables with information which entity_id or security_alias were deleted and in which tables. This information is extracted from Eagle and loaded into snowflake where the same delete operation has to be performed in order to sync data between Eagle and Vault.

In case of temporal load, the soft delete will be executed, records will not be deleted, but will be marked closed, the zcreated_at_end field will be updated with a corresponding date.

New Extracts support below filters:
MASTERTABLE.EFFECTIVE_DATE @(:effectivedate)
MASTERTABLE.INSTANCE @(:journalid)
MASTERTABLE.JOURNAL_TYPE @(:journaltype)
GENERIC_JOURNAL_DETAILS.COLUMN_NAME @(:columnname)
GENERIC_JOURNAL_DETAILS.DATABASE_NAME @(:databasename)
GENERIC_JOURNAL_DETAILS.TABLE_NAME @(:tablename)
GENERIC_JOURNAL_DETAILS.UPDATE_TYPE @(:updatetype)

Files changed:
metadata/core/ontology/definition/eagle-vocab-definition-ref.json
metadata/core/ontology/definition/eagle-vocab-definition-shared.json
metadata/core/ontology/definition/eagle-vocab-definition-wh-shared.json
metadata/core/ontology/eagle-vocab-enumerations.json
metadata/core/ontology/eagle-vocab-ref.json
metadata/core/ontology/eagle-vocab-ref.json
metadata/core/ontology/eagle-vocab-shared.json
metadata/core/ontology/eagle-vocab-wh-shared.json
metadata/core/ontology/eagle-vocab-wh-shared.json
metadata/core/ontology/reference/genericjournal-models.json
metadata/core/ontology/reference/genericjournal.json
metadata/core/ontology/reference/genericjournal_in.json
metadata/core/ontology/reference/securityjournal-models.json
metadata/core/ontology/reference/securityjournal.json
metadata/core/ontology/reference/securityjournal_in.json
metadata/core/ontology/templates/db_oper/snowflake/genericjournal.reinit_batch_many.sql
metadata/core/ontology/templates/db_oper/snowflake/genericjournal.sync.sql
metadata/core/ontology/templates/db_oper/snowflake/securityjournal.reinit_batch_many.sql
metadata/core/ontology/templates/db_oper/snowflake/securityjournal.sync.sql
metadata/core/ontology/templates/eagle_ref_genericjournal_reinitbatch.snowflake.txt
metadata/core/ontology/templates/eagle_ref_securityjournal_reinitbatch.snowflake.txt

SDP-83329 The SMF and Entity Delete Stored Procedure logic in Vault

The Delete SMF and Delete Entity SP in Vault mimic the corresponding Eagle Delete Stored Procedure. During the SecurityJournal/GenericJournal Eagle to Vault data migration, if the delete SMF/Entity event found, then the stored procedure for delete SMF/Entity will be executed on the Vault side as well in order to keep data in sync between Eagle and Vault.

Stored Procedure for delete SMF: ESTAR.DELETE_SMF executed inside PACE_MASTERDBO.SECURITY_JOURNAL_EAGLE_SYNC
Stored Procedure for delete ENTITY: ESTAR.DELETE_ENTITY_SETUP executed inside PACE_MASTERDBO.GENERIC_JOURNAL_EAGLE_SYNC

Files changed:
metadata/core/ontology/templates/db_oper/snowflake/delete_entity.sql
metadata/core/ontology/templates/db_oper/snowflake/delete_smf.sql
metadata/core/ontology/templates/db_oper/snowflake/genericjournal.sync.sql
metadata/core/ontology/templates/db_oper/snowflake/securityjournal.sync.sql

SDP-82949 Data Rules Entity resources were updated to support missing fields

The genericentity and entityaccountingbasis resources were updated to support the load and extract for additional fields for the Oracle database. The metadata was updated by implementing mapping for missed canonical elements.

genericentity:

Database Field Canonical Element
ENTITY_STAR_PROCESSING.WAM_WAL_DATA_SRC wamWalDataSource
ENTITY_STAR_PROCESSING.AVG_DAILY_YIELD_CALC_7_30 averageDailyYieldCalc7or30

Entityaccountingbasis:

Database Field Canonical Element
ENTITY_ACCT_BASIS.PERF_NAV_LONG_PRICE perfNavLongPrice
ENTITY_ACCT_BASIS.PERF_NAV_SHORT_PRICE perfNavShortPrice
ENTITY_ACCT_BASIS.ME_FX_SOURCE meFxSource
ENTITY_ACCT_BASIS.ME_FWD_PRICE_SOURCE meFwdPriceSource
ENTITY_ACCT_BASIS.PERF_NAV_FORWARD_PRICE_SOURCE perfNavForwardPriceSource
ENTITY_ACCT_BASIS.PERF_NAV_VALUATION_FX_SOURCE perfNavValuationFxSource

Files Changed:
eagle-vocab-entity-shared.json
eagle-vocab-definition-entity-shared.json
genericentity-models.json
entityaccountingbasis-models.json
entityaccountingperiods-models.json

SDP-82264 The DateTime format supported for EDS Oracle Extensions

The DateTime format is supported via the nmTradeDate2 element for EDS Oracle Extensions.

Code change was implemented for the ESTAR.MM_TXN_EXT_DTN.NM_TRADE_DATE2 field to support dateTime 'yyyy-mm-dd\"t\"hh24:mi:ss' format.

Files changed:
rdf_metadata/rdf_meta_enum.py
rdf_metadata/rdf_meta_utils.py
rdf_metadata/rdf_ontology_generator.py
rdf_metadata/rdf_vocab.py
metadata/core/ontology/eagle-vocab-base.json
metadata/core/ontology/eagledb-inventory.json

Production Bugs

SDP-84021 Python translator change for JSON parser

The custom client workflow for translator was working incorrectly. The raw input file and translated file did not match and missing transaction records were reported after upgrade to the EagleML October, 2023 Monthly Release. The issue was identified and code for the Python translator was corrected.

Files changed:
src/common_core/ds_parser/dsjsonparser.py

SDP-80176 Forward Trades routed to the Cancel Trade panel

Forward Cancels are now routed to the Cancel Trade panel only (eagle/star/trade/pan-cxltrade.htm) since the Batch Cancel panel does not support Forward Trades. Changes were done by reviewing the processingSecurityType outmapped by the eagle/star/pan-canceltradequery.htm lookup. If tag3931 = ‘FWXXXX’, then tag701 (batchId) is removed and the data passes to the Cancel Trade. Already canceled trades are removed from the trade lookup leaving only active trades eligible for cancelling.

Both legs of the Forward trade can be canceled by the entityId and tradeTicketNumber.

Files changed:
eagle_default/in/include/csv-cancel_columns.inc

SDP-73792 The EDS genericsmf resource mapping was enhanced

The mapping for the following fields was updated to allow accurate data load to Oracle:
SECURITYDBO.SECURITY_MASTER.CURRENCY_CODE - EagleML/referenceTransaction/genericSMF/assetCurrency
SECURITYDBO.SECURITY_MASTER.EXCHANGE - EagleML/referenceTransaction/genericSMF/primaryExchangeCode
SECURITYDBO.SECURITY_MASTER.CURRENCY_PRECISION - EagleML/referenceTransaction/genericSMF/FixedIncomeModel/currencyPrecision
SECURITYDBO.EQUITY_DETAIL.PRIMARY_EXCHANGE - EagleML/referenceTransaction/genericSMF/primaryExchange

Files changed:
metadata/core/ontology/eagledb-vocab-common.json
metadata/core/ontology/reference/genericsmf-models.json