Enhancements
SDP-82928 An effectivedate filter was added for the clientaddress resource
The effectivedate filter is supported now for the clientaddress resource.
Files changed:
estar/tpe/dynamic/metadata/core/ontology/reference/clientaddress.json
SDP-82883 The SPDDELTATRIGGERLOGEXTRACT will now return data for the non-primary partition
The SPDDELTATRIGGERLOG EXTRACT interface has been updated to return results for non-primary partitions. The schema name for the TEMP_RESOLVE_COMPOSITE table will always be ‘ESTAR’.
Files changed:
eagle_ml-2-0_cm\out\tagvalue\extract_spd_delta_trigger_log.inc
SDP-82777 Updates made for Profile Templates for Conversion and Cloud Workflows
The following changes were applied to the Data Rules Core Profiles as part of the Eagle to Vault Pipeline:
New profile templates were created for the warehouseglbalances, securityjournal and genericjournal resources for both the Conversion and Cloud workflows
calculatechecksum parameter is set to Y by default for every cloud profile template
usedeltabatch parameter is set to Y by default for the cloud profile templates for the below resources:
warehouseposition, warehouseopenlot, warehousecloselot, warehousetrade, warehousecashactivity, warehousenav, warehousenavmulti, warehousenavmonthend, warehouseperformance, custombenchmark, warehouseglbalances, tradedcashactivity, settledcashactivity
composite and history profile templates were combined into a single profile for the Cloud workflow
To set a separate DeltaBasedUID for composite and history resources in one profile, the DeltaBasedUID_composite and DeltaBasedUID_history parameters should be used
Files changed:
eagle/estar/EAGLE_EXTRACTSERVICE.pkg
eagle/estar/eagle_default_workflows.pkg
SDP-82726 The Warehouseopenlot resource was updated to load the cashAccountAlias field correctly
The EDS warehouseopenlot was updated to support the cashAccountAlias canonical element. Previously, an error occurred on the load to Oracle in case of an empty field in the file. The error was raised when a NULL value was loaded into the field declared as not null by default. The issue was corrected and a new default value of ZERO will be loaded if the field cashAccountAlias is not present in the data file.
XSD name: cashAccountAlias
DB field: CASH_ACCT_ALIAS NUMBER(38) default 0 not null
Files changed:
warehouseopenlot_in.json
warehouseopenlot-models.json
SDP-82641 The AVRO schema generator was improved to produce valid schema against the Maven AVRO plugin
The AVRO schema generator was improved whereby enum objects could have values with - whitespaces, special characters, etc. and Maven AVRO plugin used to fail on validation. The issue was resolved by changing the type from enum to string.
Files changed:
src/eagleinvsys/rdf_metadata/rdf_schema_generator.py
metadata/core/ontology/eagle-vocab-enumerations.json
SDP-82545 The historyOnlyFlag logic was corrected for the genericentity resource on data load to Oracle
The historyOnlyFlag was not used on the load to the Oracle database in the genericentity resource. The logic was corrected to apply the historyOnlyFlag. The logic for the Oracle Extension for the genericentity resource was updated to be in sync with the EagleML extension for the same resource:
If it is a new Entity, then the data is loaded into the Composite and History tables regardless of whether the historyOnlyFlag is present or not
If the historyOnlyFlag is not present, then the data is loaded into the Composite and History tables
If the historyOnlyFlag is present, then the data is loaded into the History table only
Files changed:
metadata/core/ontology/entity/genericentity-models.json
metadata/core/ontology/entity/genericentity_in.json
metadata/core/ontology/templates/eagle_ent_genericentity_insertupdate.txt
SDP-82370 The originalAcquisitionNotionalCostBase and originalAcquisitionNotionalCostLocalelements elements were added to the WRHSOPENLOTEXTRACT
The WRHSOPENLOTEXTRACT was enhanced to support below elements:
warehouseTransaction/warehouseOpenLot/originalAcquisitionNotionalCostBase - HOLDINGDBO.POSITION_COST_DETAIL.ORIG_ACQ_NOTIONAL_COST_B
warehouseTransaction/warehouseOpenLot/originalAcquisitionNotionalCostLocalelements - HOLDINGDBO.POSITION_COST_DETAIL.ORIG_ACQ_NOTIONAL_COST_L
Files changed:
eagle_ml-2-0_cm/out/xml/t_warehouse_openlot_streaming.inc
SDP-82242 Several new fields supported in the EagleML settled/traded cash activity extract
The below fields are now supported in the SettledCashActivity and TradedCashActivity extracts in EagleML.
Fileds in DB:
RULESDBO.ENTITY_CUSTODY_RELATIONSHIP.CUSTODY_BANK_ACCT_NUM
RULESDBO.ENTITY_CUSTODY_RELATIONSHIP.CUSTODY_BANK_CODE
RULESDBO.ENTITY_CUSTODY_RELATIONSHIP.CUSTODY_BANK_DESC
EagleML Xpath:
EagleML/accountingTransaction/tradedCashActivity/custodyBankAccountNumber
EagleML/accountingTransaction/tradedCashActivity/custodyBankCode
EagleML/accountingTransaction/tradedCashActivity/custodyBankDesc
EagleML/accountingTransaction/settledCashActivity/custodyBankAccountNumber
EagleML/accountingTransaction/settledCashActivity/custodyBankCode
EagleML/accountingTransaction/settledCashActivity/custodyBankDesc
Files changed:
eagle_ml-2-0_cm\out\tagvalue\extract_acct_traded_cash_activity.inc
eagle_ml-2-0_cm\out\tagvalue\extract_acct_settled_cash_activity.inc
eagle_ml-2-0_cm\out\xml\t_acct_settled_cash_activity_streaming.inc
eagle_ml-2-0_cm\out\xml\t_acct_traded_cash_activity_streaming.inc
SDP-82047 The EagleML MultilegSMF interface was enhanced to support TimeSensitive functionality for the Insert / Load only
The EagleML MultilegSMF interface was enhanced to support Time Sensitive functionality. Currently Time Sensitive functionality is working only for the initial SMF load and is not supported for SMF updates which will be enhanced in a future release. This functionality can be enabled by passing FixedIncomeModel/timeSensitiveIndicator = Y in the MultilegSMF message in the PAY or RECEIVE leg record.
File changed:
eagle_default/in/xml/xml-smf_multileg_swap.xml
SDP-81976 The EDS warehouseglbalances Resource was enhanced
The EDS warehouseglbalances resource was enhanced to support the Eagle - Vault pipeline, to extract data from Eagle and upload to Snowflake.
Supported modes: Batch, Temporal
Table: LEDGERDBO.GL_DETAIL_POSTING
Batch Key: ENTITY_ID + GLDP_EFFECTIVE_DATE + SRC_INTFC_INST
Extract Filters:
sourcename
processingsecuritytype, securitytype, investmenttype, releasestatus, primaryassetid, xrefid, ticker, securityalias, securitysubtype, longshortindicator
heldonlyfilter, entityfilter, xrefaccountid
ledgeraccount
fromdate, todate,
effectivedate, fromeffectivedate, toeffectivedate
accountingdate, fromaccountingdate, toaccountingdate
monthendaccountingdate, frommonthendaccountingdate, tomonthendaccountingdate
postdate, frompostdate, topostdate
deltabatchfilter
Files changed:
metadata/core/ontology/warehouse/warehouseglbalances.json
metadata/core/ontology/warehouse/warehouseglbalances-models.json
metadata/core/ontology/definition/eagle-vocab-definition-wh.json
metadata/core/ontology/eagle-vocab-wh.json
Files added:
metadata/core/ontology/warehouse/warehouseglbalances_in.json
metadata/core/ontology/templates/db_oper/snowflake/warehouseglbalances.reinit_batch_many.sql
metadata/core/ontology/templates/eagle_wrhs_warehouseglbalances_reinitbatch.oracle.txt
metadata/core/ontology/templates/eagle_wrhs_warehouseglbalances_reinitbatch.snowflake.txt
SDP-81856 The EDS genericentity resource was enhanced to support several fields
EDS genericentity resource was improved to support additional fields and correct the mapping:
Database field Canonical element
ENTITY.BLENDED_MILL_RATE blendedMillRate
ENTITY.DEFAULT_DERIV_REASON defaultDerivativeReason
ENTITY_EXTENSION.ACTIVE_PASSIVE activePassive
ENTITY.DBA_HEDGE_AT_COST dbaHedgeAtCost
ENTITY_STAR_PROCESSING.RECOUPMENT_YEARS
Files changed:
eagle-vocab-entity-shared.json
eagle-vocab-definition-entity-shared.json
genericentity-models.json
SDP-81649 The EDS warehouseopenlot resource was updated
The metadata for the warehouseopenlot resource has been updated to be in sync with the Eagle EDM. Additional mapping was added to support the load and extract for both database providers for the following fields:
HOLDINGDBO.LOT_LEVEL_POSITION
CASH_ACCT_ALIAS cashAccountAlias
LOCAL_TAX_COST_VALUE lotTaxCostValueLocal
CALL_DATE callDate
SLEEVE_ID sleeveIdHOLDINGDBO.POSITION_COST_LOT
AMORT_AT_DISP_LTD_B amortizedAtDisposalLtdBase
AMORT_AT_DISP_LTD_L amortizedAtDisposalLtdLocal
CASH_ACCT_ALIAS cashAccountAlias
COMMITMENT commitmentAmount
DIST_INV_INC_B distributedInvestmentIncomeBase
DIST_INV_INC_L distributedInvestmentIncomeLocal
ECL_BASE expectedCreditLossAmountBase
ECL_FX_URGL expectedCreditLossFxUrgl
ECL_LOCAL expectedCreditLossAmountLocal
ECL_PREV_BASE expectedCreditLossPreviousAmountBase
ECL_PREV_LOCAL expectedCreditLossPreviousAmountLocal
ECL_PREV_STAGE expectedCreditLossPreviousStageAmount
ECL_STAGE expectedCreditLossStageAmount
EMA_COST_FLAG equityMethodAcctCostIndicator
EMA_TREATMENT equityMethodAcctTreatmentMethod
EX_UNEARNED_INT_B exUnearnedInterestBase
EX_UNEARNED_INT_PTD_L exUnearnedInterestPtdLocal
EX_UNEARNED_RECLAIM_B exUnearnedReclaimBase
EX_UNEARNED_WITHHOLDING_B exUnearnedWithholdingBase
FX_IMPAIRMENT fxImpairmentRate
ORIG_ACQ_NOTIONAL_COST_B originalAcquisitionNotionalCostBase
ORIG_ACQ_NOTIONAL_COST_L originalAcquisitionNotionalCostLocal
ORIG_EX_UNEARNED_INT_PTD_B originalExUnearnedInterestPtdBase
ORIG_UNACCRUED_RECLAIM_B originalUnaccruedReclaimBase
ORIG_UNACCRUED_WITHHOLDING_B originalUnaccruedWithholdingBase
PRIOR_QUANTITY priorQuantity
UNACCRUED_RECLAIM_L unaccruedReclaimLocal
UNACCRUED_WITHHOLDING_L unaccruedWithholdingLocal
UNDIST_INV_INC_B undistributedInvestmentIncomeBase
UNDIST_INV_INC_L undistributedInvestmentIncomeLocal
Additionally mapping for the updateTimestamp element was corrected:
POSITION_INCOME_LOT.UPDATE_DATE
POSITION_COST_LOT_USER.UPDATE_DATE
REAL_ESTATE_POSITION_LOT.UPDATE_DATE
and Mapping for the updateSource element was implemented:
POSITION_INCOME_LOT.UPDATE_SOURCE
POSITION_COST_LOT_USER.UPDATE_SOURCE
REAL_ESTATE_POSITION_LOT.UPDATE_SOURCE
Files changed:
warehouseopenlot-models.json
eagle-vocab-common.json
eagle-vocab-acc-shared.json
eagle-vocab-wh-shared.json
eagle-vocab-definition-common.json
eagle-vocab-definition-acc-shared.json
eagle-vocab-definition-wh-shared.json
SDP-81085 Added the ability to provide a file mask for EDS load
The ability to provide a file mask for an EDS load was added. When the parameter DataFileName is provided with the file mask, the load will be executed by multiple processes for each file which fits to this mask. It is required to use the DataFileName parameter NOT inFileName.
<taskParameter>
<name>DataFileName</name> <value>load_func_test/8C6C4AFC9A01DE5C_ASISCOPY_ADHOC_20161229_20161229_*_WAREHOUSEOPENLOT_20240206.csv</value>
</taskParameter>
SDP-72836 The EDS genericissueanalytic resource was enhanced
The EDS genericissueanalytic resource was enhanced to support the full data load to Oracle.
Mappings were updated and added to load data into all related tables:
SECURITYDBO.SECURITY_ANALYTICS
SECURITYDBO.SECURITY_ANALYTICS_DERV
SECURITYDBO.SECURITY_ANALYTICS_EQUITY
SECURITYDBO.SECURITY_ANALYTICS_EXT
SECURITYDBO.SECURITY_ANALYTICS_FI
SECURITYDBO.SECURITY_ANALYTICS_MTG
SECURITYDBO.SECURITY_ANALYTICS_USER
SECURITYDBO.SECURITY_EPS_EQUITY
SECURITYDBO.SECURITY_PRICE_EARN_EQUITY
Files changed:
metadata/core/ontology/eagledb-vocab-common.json
metadata/core/ontology/reference/genericissueanalytic-models.json
Production Bugs
SDP-83057 The EDS Extract Filters logic was corrected
The EDS Extract Filters logic was adjusted to work with filters without parameters in the bind description, similar to the one below:
"filter": "MASTERTABLE.EFFECTIVE_DATE in (SELECT BC.BUSINESS_DATE FROM PACE_MASTERDBO.BUSINESS_CALENDAR BC WHERE BC.SRC_INTFC_INST = 1 AND BC.MONTH_END_FLAG = 1)"
Files changed:
pyrules/eagleinvsys/common_core/dataset/ds_filters/query_filter_processor_registry.py
SDP-82823 Detect encoding functionality was enhanced as part of translators
The error was reported during data transformation for an ASCII file, where the value could not be decoded. The code was changed and the issue was corrected.
Files changed:
src/common_core/descriptor/dbdescriptor_sf.py
src/common_core/descriptor/idescriptor.py
SDP-82515 Fix for curl command in curl_ftp_process.inc
The Curl library was updated in Eagle Core in the 2017.2.46 release which exposed flaws in the ML Curl code. The ML commands were fixed in the curl_ftp_process.inc include file.
Files changed:
eagle_ml-2-0_cm\curl_ftp_process.inc
SDP-82197 Custom XSLT translation is supported in NG mode via MC2 for warehouse objects
There was a type mismatch in the XSLT transformation call condition. The issue was fixed and the custom XSLT transformation can be executed in the NG procedure mode via MC2 for warehouse objects.
File changed:
eagle_default/in/xml/xml-warehouse_v2.xml
SDP-81429 Traded Interest removed from the accruedInterestType lookup for trade
During previous development, Tag 49 - Traded Interest was added with a lookup for the accrued interest type. This led to an incorrect calculation for trade records. With Tag 49 removed from lookup, trade records loaded correctly.
Files changed:
eagle_default/in/include/csv-buy_columns.inc
eagle_default/in/include/csv-sell_columns.inc