Enhancements

SDP-82928 An effectivedate filter was added for the clientaddress resource

The effectivedate filter is supported now for the clientaddress resource.

Files changed:
estar/tpe/dynamic/metadata/core/ontology/reference/clientaddress.json

SDP-82883 The SPDDELTATRIGGERLOGEXTRACT will now return data for the non-primary partition

The SPDDELTATRIGGERLOG EXTRACT interface has been updated to return results for non-primary partitions. The schema name for the TEMP_RESOLVE_COMPOSITE table will always be ‘ESTAR’.

Files changed:
eagle_ml-2-0_cm\out\tagvalue\extract_spd_delta_trigger_log.inc

SDP-82777 Updates made for Profile Templates for Conversion and Cloud Workflows

The following changes were applied to the Data Rules Core Profiles as part of the Eagle to Vault Pipeline:

• New profile templates were created for the warehouseglbalances, securityjournal and genericjournal resources for both the Conversion and Cloud workflows

• calculatechecksum parameter is set to Y by default for every cloud profile template

• usedeltabatch parameter is set to Y by default for the cloud profile templates for the below resources:

  • warehouseposition, warehouseopenlot, warehousecloselot, warehousetrade, warehousecashactivity, warehousenav, warehousenavmulti, warehousenavmonthend, warehouseperformance, custombenchmark, warehouseglbalances, tradedcashactivity, settledcashactivity

• composite and history profile templates were combined into a single profile for the Cloud workflow

  • To set a separate DeltaBasedUID for composite and history resources in one profile, the DeltaBasedUID_composite and DeltaBasedUID_history parameters should be used

Files changed:
eagle/estar/EAGLE_EXTRACTSERVICE.pkg
eagle/estar/eagle_default_workflows.pkg

SDP-82726 The Warehouseopenlot resource was updated to load the cashAccountAlias field correctly

The EDS warehouseopenlot was updated to support the cashAccountAlias canonical element. Previously, an error occurred on the load to Oracle in case of an empty field in the file. The error was raised when a NULL value was loaded into the field declared as not null by default. The issue was corrected and a new default value of ZERO will be loaded if the field cashAccountAlias is not present in the data file.
XSD name: cashAccountAlias
DB field: CASH_ACCT_ALIAS NUMBER(38) default 0 not null

Files changed:
warehouseopenlot_in.json
warehouseopenlot-models.json

SDP-82641 The AVRO schema generator was improved to produce valid schema against the Maven AVRO plugin

The AVRO schema generator was improved whereby enum objects could have values with - whitespaces, special characters, etc. and Maven AVRO plugin used to fail on validation. The issue was resolved by changing the type from enum to string.

Files changed:

src/eagleinvsys/rdf_metadata/rdf_schema_generator.py
metadata/core/ontology/eagle-vocab-enumerations.json

SDP-82545 The historyOnlyFlag logic was corrected for the genericentity resource on data load to Oracle

The historyOnlyFlag was not used on the load to the Oracle database in the genericentity resource. The logic was corrected to apply the historyOnlyFlag. The logic for the Oracle Extension for the genericentity resource was updated to be in sync with the EagleML extension for the same resource:

• If it is a new Entity, then the data is loaded into the Composite and History tables regardless of whether the historyOnlyFlag is present or not

• If the historyOnlyFlag is not present, then the data is loaded into the Composite and History tables

• If the historyOnlyFlag is present, then the data is loaded into the History table only

Files changed:
metadata/core/ontology/entity/genericentity-models.json
metadata/core/ontology/entity/genericentity_in.json
metadata/core/ontology/templates/eagle_ent_genericentity_insertupdate.txt

SDP-82370 The originalAcquisitionNotionalCostBase and originalAcquisitionNotionalCostLocalelements elements were added to the WRHSOPENLOTEXTRACT

The WRHSOPENLOTEXTRACT was enhanced to support below elements:

warehouseTransaction/warehouseOpenLot/originalAcquisitionNotionalCostBase - HOLDINGDBO.POSITION_COST_DETAIL.ORIG_ACQ_NOTIONAL_COST_B
warehouseTransaction/warehouseOpenLot/originalAcquisitionNotionalCostLocalelements - HOLDINGDBO.POSITION_COST_DETAIL.ORIG_ACQ_NOTIONAL_COST_L

Files changed:
eagle_ml-2-0_cm/out/xml/t_warehouse_openlot_streaming.inc

SDP-82242 Several new fields supported in the EagleML settled/traded cash activity extract

The below fields are now supported in the SettledCashActivity and TradedCashActivity extracts in EagleML.

Fileds in DB:
RULESDBO.ENTITY_CUSTODY_RELATIONSHIP.CUSTODY_BANK_ACCT_NUM
RULESDBO.ENTITY_CUSTODY_RELATIONSHIP.CUSTODY_BANK_CODE
RULESDBO.ENTITY_CUSTODY_RELATIONSHIP.CUSTODY_BANK_DESC

EagleML Xpath:
EagleML/accountingTransaction/tradedCashActivity/custodyBankAccountNumber
EagleML/accountingTransaction/tradedCashActivity/custodyBankCode
EagleML/accountingTransaction/tradedCashActivity/custodyBankDesc

EagleML/accountingTransaction/settledCashActivity/custodyBankAccountNumber
EagleML/accountingTransaction/settledCashActivity/custodyBankCode
EagleML/accountingTransaction/settledCashActivity/custodyBankDesc

Files changed:
eagle_ml-2-0_cm\out\tagvalue\extract_acct_traded_cash_activity.inc
eagle_ml-2-0_cm\out\tagvalue\extract_acct_settled_cash_activity.inc
eagle_ml-2-0_cm\out\xml\t_acct_settled_cash_activity_streaming.inc
eagle_ml-2-0_cm\out\xml\t_acct_traded_cash_activity_streaming.inc

SDP-82047 The EagleML MultilegSMF interface was enhanced to support TimeSensitive functionality for the Insert / Load only

The EagleML MultilegSMF interface was enhanced to support Time Sensitive functionality. Currently Time Sensitive functionality is working only for the initial SMF load and is not supported for SMF updates which will be enhanced in a future release. This functionality can be enabled by passing FixedIncomeModel/timeSensitiveIndicator = Y in the MultilegSMF message in the PAY or RECEIVE leg record.

File changed:
eagle_default/in/xml/xml-smf_multileg_swap.xml

SDP-81976 The EDS warehouseglbalances Resource was enhanced

The EDS warehouseglbalances resource was enhanced to support the Eagle - Vault pipeline, to extract data from Eagle and upload to Snowflake.

Supported modes: Batch, Temporal
Table: LEDGERDBO.GL_DETAIL_POSTING
Batch Key: ENTITY_ID + GLDP_EFFECTIVE_DATE + SRC_INTFC_INST

Extract Filters:
sourcename
processingsecuritytype, securitytype, investmenttype, releasestatus, primaryassetid, xrefid, ticker, securityalias, securitysubtype, longshortindicator
heldonlyfilter, entityfilter, xrefaccountid
ledgeraccount
fromdate, todate,
effectivedate, fromeffectivedate, toeffectivedate
accountingdate, fromaccountingdate, toaccountingdate
monthendaccountingdate, frommonthendaccountingdate, tomonthendaccountingdate
postdate, frompostdate, topostdate
deltabatchfilter

Files changed:
metadata/core/ontology/warehouse/warehouseglbalances.json
metadata/core/ontology/warehouse/warehouseglbalances-models.json
metadata/core/ontology/definition/eagle-vocab-definition-wh.json
metadata/core/ontology/eagle-vocab-wh.json

Files added:
metadata/core/ontology/warehouse/warehouseglbalances_in.json
metadata/core/ontology/templates/db_oper/snowflake/warehouseglbalances.reinit_batch_many.sql
metadata/core/ontology/templates/eagle_wrhs_warehouseglbalances_reinitbatch.oracle.txt
metadata/core/ontology/templates/eagle_wrhs_warehouseglbalances_reinitbatch.snowflake.txt

SDP-81856 The EDS genericentity resource was enhanced to support several fields

EDS genericentity resource was improved to support additional fields and correct the mapping:

Database field Canonical element
ENTITY.BLENDED_MILL_RATE blendedMillRate
ENTITY.DEFAULT_DERIV_REASON defaultDerivativeReason
ENTITY_EXTENSION.ACTIVE_PASSIVE activePassive
ENTITY.DBA_HEDGE_AT_COST dbaHedgeAtCost
ENTITY_STAR_PROCESSING.RECOUPMENT_YEARS

Files changed:
eagle-vocab-entity-shared.json
eagle-vocab-definition-entity-shared.json
genericentity-models.json

SDP-81649 The EDS warehouseopenlot resource was updated

The metadata for the warehouseopenlot resource has been updated to be in sync with the Eagle EDM. Additional mapping was added to support the load and extract for both database providers for the following fields:

• HOLDINGDBO.LOT_LEVEL_POSITION
  CASH_ACCT_ALIAS cashAccountAlias
  LOCAL_TAX_COST_VALUE lotTaxCostValueLocal
  CALL_DATE callDate
  SLEEVE_ID sleeveId

• HOLDINGDBO.POSITION_COST_LOT
  AMORT_AT_DISP_LTD_B amortizedAtDisposalLtdBase
  AMORT_AT_DISP_LTD_L amortizedAtDisposalLtdLocal
  CASH_ACCT_ALIAS cashAccountAlias
  COMMITMENT commitmentAmount
  DIST_INV_INC_B distributedInvestmentIncomeBase
  DIST_INV_INC_L distributedInvestmentIncomeLocal
  ECL_BASE expectedCreditLossAmountBase
  ECL_FX_URGL expectedCreditLossFxUrgl
  ECL_LOCAL expectedCreditLossAmountLocal
  ECL_PREV_BASE expectedCreditLossPreviousAmountBase
  ECL_PREV_LOCAL expectedCreditLossPreviousAmountLocal
  ECL_PREV_STAGE expectedCreditLossPreviousStageAmount
  ECL_STAGE expectedCreditLossStageAmount
  EMA_COST_FLAG equityMethodAcctCostIndicator
  EMA_TREATMENT equityMethodAcctTreatmentMethod
  EX_UNEARNED_INT_B exUnearnedInterestBase
  EX_UNEARNED_INT_PTD_L exUnearnedInterestPtdLocal
  EX_UNEARNED_RECLAIM_B exUnearnedReclaimBase
  EX_UNEARNED_WITHHOLDING_B exUnearnedWithholdingBase
  FX_IMPAIRMENT fxImpairmentRate
  ORIG_ACQ_NOTIONAL_COST_B originalAcquisitionNotionalCostBase
  ORIG_ACQ_NOTIONAL_COST_L originalAcquisitionNotionalCostLocal
  ORIG_EX_UNEARNED_INT_PTD_B originalExUnearnedInterestPtdBase
  ORIG_UNACCRUED_RECLAIM_B originalUnaccruedReclaimBase
  ORIG_UNACCRUED_WITHHOLDING_B originalUnaccruedWithholdingBase
  PRIOR_QUANTITY priorQuantity
  UNACCRUED_RECLAIM_L unaccruedReclaimLocal
  UNACCRUED_WITHHOLDING_L unaccruedWithholdingLocal
  UNDIST_INV_INC_B undistributedInvestmentIncomeBase
  UNDIST_INV_INC_L undistributedInvestmentIncomeLocal

Additionally mapping for the updateTimestamp element was corrected:
POSITION_INCOME_LOT.UPDATE_DATE
POSITION_COST_LOT_USER.UPDATE_DATE
REAL_ESTATE_POSITION_LOT.UPDATE_DATE

and Mapping for the updateSource element was implemented:
POSITION_INCOME_LOT.UPDATE_SOURCE
POSITION_COST_LOT_USER.UPDATE_SOURCE
REAL_ESTATE_POSITION_LOT.UPDATE_SOURCE

Files changed:
warehouseopenlot-models.json
eagle-vocab-common.json
eagle-vocab-acc-shared.json
eagle-vocab-wh-shared.json
eagle-vocab-definition-common.json
eagle-vocab-definition-acc-shared.json
eagle-vocab-definition-wh-shared.json

SDP-81085 Added the ability to provide a file mask for EDS load

The ability to provide a file mask for an EDS load was added. When the parameter DataFileName is provided with the file mask, the load will be executed by multiple processes for each file which fits to this mask. It is required to use the DataFileName parameter NOT inFileName.

<taskParameter>
<name>DataFileName</name> <value>load_func_test/8C6C4AFC9A01DE5C_ASISCOPY_ADHOC_20161229_20161229_*_WAREHOUSEOPENLOT_20240206.csv</value>
</taskParameter>

SDP-72836 The EDS genericissueanalytic resource was enhanced

The EDS genericissueanalytic resource was enhanced to support the full data load to Oracle.
Mappings were updated and added to load data into all related tables:
SECURITYDBO.SECURITY_ANALYTICS
SECURITYDBO.SECURITY_ANALYTICS_DERV
SECURITYDBO.SECURITY_ANALYTICS_EQUITY
SECURITYDBO.SECURITY_ANALYTICS_EXT
SECURITYDBO.SECURITY_ANALYTICS_FI
SECURITYDBO.SECURITY_ANALYTICS_MTG
SECURITYDBO.SECURITY_ANALYTICS_USER
SECURITYDBO.SECURITY_EPS_EQUITY
SECURITYDBO.SECURITY_PRICE_EARN_EQUITY

Files changed:
metadata/core/ontology/eagledb-vocab-common.json
metadata/core/ontology/reference/genericissueanalytic-models.json

Production Bugs

SDP-83057 The EDS Extract Filters logic was corrected

The EDS Extract Filters logic was adjusted to work with filters without parameters in the bind description, similar to the one below:
"filter": "MASTERTABLE.EFFECTIVE_DATE in (SELECT BC.BUSINESS_DATE FROM PACE_MASTERDBO.BUSINESS_CALENDAR BC WHERE BC.SRC_INTFC_INST = 1 AND BC.MONTH_END_FLAG = 1)"

Files changed:
pyrules/eagleinvsys/common_core/dataset/ds_filters/query_filter_processor_registry.py

SDP-82823 Detect encoding functionality was enhanced as part of translators

The error was reported during data transformation for an ASCII file, where the value could not be decoded. The code was changed and the issue was corrected.

Files changed:
src/common_core/descriptor/dbdescriptor_sf.py
src/common_core/descriptor/idescriptor.py

SDP-82515 Fix for curl command in curl_ftp_process.inc

The Curl library was updated in Eagle Core in the 2017.2.46 release which exposed flaws in the ML Curl code. The ML commands were fixed in the curl_ftp_process.inc include file.

Files changed:
eagle_ml-2-0_cm\curl_ftp_process.inc

SDP-82197 Custom XSLT translation is supported in NG mode via MC2 for warehouse objects

There was a type mismatch in the XSLT transformation call condition. The issue was fixed and the custom XSLT transformation can be executed in the NG procedure mode via MC2 for warehouse objects.

File changed:
eagle_default/in/xml/xml-warehouse_v2.xml

SDP-81429 Traded Interest removed from the accruedInterestType lookup for trade

During previous development, Tag 49 - Traded Interest was added with a lookup for the accrued interest type. This led to an incorrect calculation for trade records. With Tag 49 removed from lookup, trade records loaded correctly.

Files changed:
eagle_default/in/include/csv-buy_columns.inc
eagle_default/in/include/csv-sell_columns.inc