EagleML Release Notes - July 25, 2024

Enhancements

SDP-87252 EagleML Voluntary Election Inbound interface was enhanced with Entity Bank relationship resolution for Update and Delete Actions

The EagleML Voluntary Election Inbound interface was enhanced with Entity Bank relationship resolution for Update and Delete operations.

Files changed:
eagle_default/in/xml/xml-ref_voluntarycaelections.xml

SDP-86971 The Logic for Xreference Identifiers was adjusted to exclude duplicates

The logic for SMF xreference mapping was enhanced with an additional check to validate if the identifier is duplicated in the incoming data to pass the set of unique pairs for xId/xIdType.

For example, if the incoming data has bloombergId as an xref and as separate tag, then they are mapped to the same pair of xId/xIdType:

{noformat}<bloombergId>EQ86971</bloombergId>
<xrefIdentifiers>
  <xId26>
        <xId>EQ86971</xId>
        <xIdType>BBID</xIdType>
  </xId26>
</xrefIdentifiers>{noformat}

This pair {{EQ86971}}/{{BBID}} will be passed into DB only once. This ensures that EagleML will not miss identifiers low in priority because of duplicated ones.

Files changed:
tpe/dynamic/msgcenter/eagle_default/in/xml/include/smf_identifiers_map.inc

SDP-86417 The multipleLegAccountingTrade inbound interface enhanced with new elements

The multipleLegAccountingTrade multiple leg inbound interface has been enhanced with new elements:
accountingTransaction/multipleLegAccountingTrade/beginAccrualDate
accountingTransaction/multipleLegAccountingTrade/endAccrualDate

Files changed:
eagle_default/in/xml/include/multiplelegtrade_aliases.inc

SDP-86194 Signage logic changes for BUYCVR and BUYCVRCANCEL events

The ACCRUED_INTEREST and BASEINTERESTADJ signage logic has been changed for BUYCVR and BUYCVRCANCEL events. In case the signage logic is applied these fields will be presented differently.

Signage is activated when passed when the EnableSignageLogic is set to Y and the IncludeExtendedDetails is set to Y or ALL.

<taskParameter>
<name>EnableSignageLogic</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>
<taskParameter>
<name>IncludeExtendedDetails </name>
<dataType>S</dataType>
<value>ALL</value>
</taskParameter>

Files changed:
tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/tagvalue/extract_wrhs_trade_signage_logic.inc

SDP-85883 Warehouse Position load supports srcSysC element

The Warehouse Position Inbound interface was enhanced to support srcSysC for load in all modes.

Files changed:
eagle_default/in/xml/xslt/wrh_egl_gen_interface3.xsl
eagle_ml-2-0_cm/dbkeysmap.inc
eagle_ml-2-0_cm/out/xml/t_warehouse_position_xmldbapi.xml
eagle_ml-2-0_cm/out/xml/t_warehouse_position_xmldbapi_streaming.inc

SDP-85849 Added new flag ValuationSource for PRICINGRDCREPORTEXTRACT

A new flag, ValuationSource, has been added for the PRICINGRDCREPORTEXTRACT. When it set to MONTHLY, pricing data will be extracted from the ESTAR.MONTHLY_PORTFOLIO_VALUATION. When it set to DAILY or not presented in the RTR, data will be extracted from the ESTAR.MONTHLY_PORTFOLIO_VALUATION table. The default value is DAILY.

The functionality can also be specified in the /eagle_ml-2-0_custom_cm/w_config_custom.inc file via the global W_PRICINGREPORTEXTRACT_VALUATION_SRC variable for all PRICINGRDCREPORTEXTRACT requests.

The RTR has higher priority over the custom include file. When both the ValuationSource and W_PRICINGREPORTEXTRACT_VALUATION_SRC are not null, the pricing table is chosen from the RTR parameter.

<taskParameter>
<name>ValuationSource</name>
<dataType>S</dataType>
<value>MONTHLY</value>
</taskParameter>

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_pricing_report.inc

SDP-85947 Python CSV2XML: name 'getvariable' is not defined - adjust getVariable functionality

A new object called getvariable has been implemented to support dynamic expression calculation for variables.

Dynamic expression calculation means the presence of two execution steps, as opposed to the traditional single execution step seen with a variable, where expression can be only static.

For example:

[
"getvariable",
"extUSER_CHAR",
"iif(stringlength(get_instruction_param('USER_CHAR', :GVAR_RTR: )>0,get_instruction_param('USER_CHAR', :GVAR_RTR: ,'')"
],

In this case there will be two executions (calculations):

  1. The first Execution will calculate expression (iif(stringlength(get_instruction_param('USER_CHAR', :GVAR_RTR: )>0,get_instruction_param('USER_CHAR', :GVAR_RTR: ,'')). In our example the result of this expression is |57|;

  2. The second Execution will use the result of the 1st execution, it works as expression for the second execution. In this scenario, the value will be taken from column 57 of the incoming CSV file.

This variable then can be used in the mapping as typical variable:

[
":extUSER_CHAR:",
"",
false,
null,
"",
"accountingTrade/userChar5"
],

In the resulting file, we should expect to find the value from column 57 in the element accountingTrade/userChar5.

It works the same way if the variable is defined with static expression:

[
"variable",
"extUSER_CHAR",
"|57|"
],

However, in the scenario, it's expected that the definition will be dynamically defined and the column number will be modified.

SDP-82063 Added new flag ValuationSource for PRICINGREPORTEXTRACT

A new flag, ValuationSource, has been added for the PRICINGREPORTEXTRACT. When it set to MONTHLY, pricing data will be extracted from the ESTAR.MONTHLY_PORTFOLIO_VALUATION. When it set to DAILY or not presented in the RTR, data will be extracted from the ESTAR.MONTHLY_PORTFOLIO_VALUATION table. The default value is DAILY.

The functionality can also be specified in the /eagle_ml-2-0_custom_cm/w_config_custom.inc file via the global W_PRICINGREPORTEXTRACT_VALUATION_SRC variable for all PRICINGREPORTEXTRACT requests.

The RTR has higher priority over the custom include file. When both the ValuationSource and W_PRICINGREPORTEXTRACT_VALUATION_SRC are not null, the pricing table is chosen from the RTR parameter.

<taskParameter>
<name>ValuationSource</name>
<dataType>S</dataType>
<value>MONTHLY</value>
</taskParameter>

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_pricing_report.inc

SDP-85845 VoluntaryCAElection interface resolves the custody elements

A new custodian element, entityBankAlias, is supported in the Corporate Actions Voluntary Elections for load.
If none of the new elements, custodianBankCode, custodyBankAccount, custodyBankSubaccount are sent, nothing changes for existing clients. However, if at least one of the new elements is present in the upload file, new logic is applied to check all custodian fields. Once all three elements are resolved, the entityBankAlias is resolved as a result. If even one of the identifiers is not found in the panel lookup or was not sent in the file, an "entityBankAlias" error is produced.

Additionally, a new processing option has been introduced:
<processingOptions>USE_ENTITY_BANK_ALIAS</processingOptions>
When using this option, the entityBankAlias is expected during the load process.

EagleML Xpath:
EagleML/referenceTransaction/voluntaryCAElection/custodianBankCode - Tag 1176
EagleML/referenceTransaction/voluntaryCAElection/custodyBankAccount - Tag 1660
EagleML/referenceTransaction/voluntaryCAElection/custodyBankSubaccount - Tag 1661
EagleML/referenceTransaction/voluntaryCAElection/entityBankAlias - Tag 16

Files changed:
eagle_default/in/xml/xml-ref_voluntarycaelections.xml

SDP-85455 Asset Resolution Options for Warehouse Trade Interface

Two resolution options were added for the warehouseTrade object:

  • RESOLVE_PRIMARY is used to resolve the primaryAssetId; this can be useful to validate the securityAlias

<assetResolutionOption>RESOLVE_PRIMARY</assetResolutionOption>

  • SKIP is used to use the securityAlias passed from the incoming data without resolution

<assetResolutionOption>SKIP</assetResolutionOption>

Both options allow to load trades via the warehouseTrade object using only the securityAlias.

Files changed:
eagle_default/in/xml/include/xml-dbdirect_sec_resolution.inc

SDP-85425 SFTP Distribution with Optional Public Key

Optional SFTP Distribution Key. If the SFTP_PUBLICKEY parameter is not defined, then the process will use the SFTP_PRIVATEKEY instead.

Files changed:
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/reporter_dist_methods.inc

SDP-85213 DBTATA (TBA) Trades updated To Support All Lot Selection Methods

The AccountingTrade interface has been enhanced for TBA trades. All Eagle Lot Selection Methods or Tag 27 are available for the Processing Security Type (PST) DBTATA trades in EML now. Previously, IDLOT was only allowed for DBTATA.

Files changed:
eagle_default/in/include/csv-sell_columns.inc

SDP-84964 VOLCAELECTIONEXTRACT enhanced to support new elements

New custodian elements custodyBankAccount, custodianBankCode, custodianBankName, custodyBankSubaccount, and entityBankAlias are supported in the Corporate Actions Voluntary Elections for extract.

Xpath | DB mapping:
voluntaryCAElection/custodianBankCode | RULESDBO.ENTITY_CUSTODY_RELATIONSHIP.CUSTODY_BANK_CODE
/voluntaryCAElection/custodianBankName | RULESDBO.ENTITY_CUSTODY_RELATIONSHIP.CUSTODY_BANK_DESC
voluntaryCAElection/custodyBankAccount | RULESDBO.ENTITY_CUSTODY_RELATIONSHIP.CUSTODY_BANK_ACCT_NUM
/voluntaryCAElection/custodyBankSubaccount | RULESDBO.ENTITY_CUSTODY_RELATIONSHIP.CUSTODY_BANK_SUB_ACCT
/voluntaryCAElection/entityBankAlias | SECURITYDBO.VOL_CORP_ACTION_ELECTION.ENTITY_BANK_ALIAS

Files changed:
eagle_ml-2-0_cm/out/xml/t_voluntary_ca_election_streaming.inc
eagle_ml-2-0_cm/out/tagvalue/extract_voluntary_ca_election.xml

SDP-84906 The multipleLegAccountingTrade single event inbound interface enhanced with new elements

multipleLegAccountingTrade single event inbound interface enhanced with new elements

<payTradeLeg>
<tradedInterest>30</tradedInterest> <!-- Tags 17937, 18280 -->
<tradedInterestBase>25</tradedInterestBase> <!-- Tags 17942, 18281 -->
<firstPeriodCouponRate>3</firstPeriodCouponRate> <!-- Tag 18282 -->
<lotLevelDatedDate>2024-01-04</lotLevelDatedDate><!-- Tag 18283 -->
</payTradeLeg>
<receiveTradeLeg>
<tradedInterest>40</tradedInterest> <!-- Tags 17938, 18280 if not exist in pay leg -->
<tradedInterestBase>35</tradedInterestBase> <!-- Tags 17943, 18281 if not exist in pay leg -->
<firstPeriodCouponRate>3</firstPeriodCouponRate> <!-- Tag 18282 if not exist in pay leg -->
<lotLevelDatedDate>2024-01-04</lotLevelDatedDate><!-- Tag 18283 if not exist in pay leg -->
</receiveTradeLeg>

Files changed:
eagle_default/in/xml/xml-accttrade_singleleg.xml

SDP-83690 Additional Load Fields for the ExchangePrice Load and ISSUEEXCHANGEPRICEEXTRACT

New fields added for the load and extract for the ExchangePrice Load and ISSUEEXCHANGEPRICEEXTRACT:

EagleML/referenceTransaction/exchangePrice/actualSettlementPrice SECURITYDBO.PRICE_EXCHANGE.PX_SETTLE_ACTUAL - Tag 18582
EagleML/referenceTransaction/exchangePrice/actualSettlementPriceDate SECURITYDBO.PRICE_EXCHANGE.PX_SETTLE_ACTUAL_DATE - Tag 18583
EagleML/referenceTransaction/exchangePrice/marketStatus SECURITYDBO.PRICE_EXCHANGE.MARKET_STATUS - Tag 18584
EagleML/referenceTransaction/exchangePrice/foreignInstitutionalInvestorLimit SECURITYDBO.PRICE_EXCHANGE.FII_LIMIT - Tag 18585
EagleML/referenceTransaction/exchangePrice/cleanDirtyFlag SECURITYDBO.PRICE_EXCHANGE.CLEAN_DIRTY_FLAG - Tag 18586
EagleML/referenceTransaction/exchangePrice/exchangeListedFlag SECURITYDBO.PRICE_EXCHANGE.EXCHANGE_LISTED_FLAG - Tag 18587
EagleML/referenceTransaction/exchangePrice/holidayFlag SECURITYDBO.PRICE_EXCHANGE.HOLIDAY_FLAG - Tag 18588

Also, following pricing flags are added for existing fields: closingPriceCondition, oidYieldPriceCondition, highPriceCondition, lowPriceCondition, referencePriceCondition, nominalDirtyPriceCondition, cleanInclOptionPriceCondition, dirtyInclOptionPriceCondition, cleanExclOptionPriceCondition, dirtyExclOptionPriceCondition, callOptionPriceCondition, putOptionPriceCondition, fairValuePriceCondition, averagePriceCondition, fixingPriceCondition, grossAskPriceCondition, grossBidPriceCondition, grossMidPriceCondition, midClosePriceCondition, meanPriceCondition, nominalPriceCondition, officialClosePriceCondition, vwapClosePriceCondition, askAdrPriceCondition, bidAdrPriceCondition, lastAdrPriceCondition, lastUpdatePriceCondition, naicPriceCondition, navCurrentPriceCondition, userFee1PriceCondition, userFee2PriceCondition, userFee3PriceCondition, userNav1PriceCondition, userNav2PriceCondition, userNav3PriceCondition, userNav4PriceCondition, userNav5PriceCondition, pxSettleCondition, actualSettlementPriceCondition, pxYestPriceCondition.

Files changed:
metadata/bind/eagleml_ref_exchangeprice.xml
eagle_default/in/xml/xml-ref_exchangeprice.xml
eagle_default/in/xml/include/xml-ref_security_pricing_common.inc
eagle_ml-2-0_cm/out/xml/t_issueprice_exchange_streaming.inc

SDP-87039 The Warehouse Trade EagleML Inbound interface was enhanced with additional elements

The warehouseTrade EagleML inbound interface enhanced with new elements:

EagleML/warehouseTransaction/warehouseTrade/eventSequence
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate1
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate2
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate3
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate4
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate5
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate6
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate7
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate8
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate9
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate10

File changed:
eagle_default/in/xml/xslt/wrh_egl_gen_interface3.xsl

SDP-86384 Data Rules Profile Templates for Conversion and Cloud Workflows for New Resources

Added Data Rules profile templates for the Conversion and Cloud Workflows for issuerentityrelationship, entityliquidity and entitypeergroupranking resources.

Files added:
msgcenter/profile/workflows/EagleToVault-cloud-entityliquidity_tmp.profile
msgcenter/profile/workflows/EagleToVault-cloud-entitypeergroupranking_tmp.profile
msgcenter/profile/workflows/EagleToVault-cloud-issuerentityrelationship_tmp.profile
msgcenter/profile/workflows/EagleToVault-conversion-entityliquidity_tmp.profile
msgcenter/profile/workflows/EagleToVault-conversion-entitypeergroupranking_tmp.profile
msgcenter/profile/workflows/EagleToVault-conversion-issuerentityrelationship_tmp.profile

Files changed:
eagle/estar/EAGLE_EXTRACTSERVICE.pkg
eagle/estar/eagle_default_workflows.pkg

SDP-86309 Additional fields supported for Oracle load via entityaccountingbasis resource

The fields PERF_NAV_FORWARD_PRICE_SOURCE (performanceNavForwardPriceSource) and PERF_NAV_VALUATION_FX_SOURCE (performanceNavValuationFxSource) now supported for load.

Files changed:
entityaccountingbasis-models.json

SDP-86235 The warehouseposition resource was updated to support a few missing fields

The warehouseposition resource has been enhanced to support additional fields for both database providers for load and extract. The mapping was implemented for missing canonical elements.

Database Field Canonical Element
POSITION_DETAIL.SHARE_CLASS - shareClass
POSITION_DETAIL.CUST_PRINCIPAL_BALANCE - custodianPrincipalBalanceBase
POSITION_DETAIL.CUST_PRINCIPAL_BALANCE_LOCAL - custodianPrincipalBalanceLocal
POSITION_DETAIL.CUST_INCOME_BALANCE - custodianIncomeBalanceBase
POSITION_DETAIL.CUST_INCOME_BALANCE_LOCAL - custodianIncomeBalanceLocal

Files Changed:
eagle-vocab-wh-shared.json
eagle-vocab-definition-wh-shared.json
warehouseposition-models.json

SDP-86200 Warehouseposition Reinitbatch Stored Procedure was updated for Oracle load to ingest data correctly

All warehouseposition fields loaded data passed without null values. Previously, a number of fields passed where ignored during the load process.

Files Changed:
metadata\core\ontology\templates\eagle_wrhs_warehouseposition_reinitbatch.oracle

SDP-86196 Information in data load error is not sufficient to triage issue

Logic for reporting error messages to the end users was improved to inherit global parameter for length, also this value can be adjusted via RTR parameter: procstatusmsgsize. The Default size is set to 300.

For troubleshooting purposes you can increase value to get more details via error message, but please be aware that too long error messages can cause unexpected behavior.

If the error has a line number, then the error message will have text from this line for troubleshooting purposes.

Files changed:

common-core/descriptor/dbdescriptor.py
common-core/processing-env/proc_log_stru.py
common-core/processing-env/processing_context.py
common-core/processing-env/processing_log.py
common-core/processing-env/proc_log_policy.py

SDP-85755 Cloud workflow eagle_wrf_cloud_extract was updated

The eagle_wrf_cloud_extract workflow can be executed in two different ways:

I - default setup or original implementation where the eagle_wrf_cloud_extract workflow runs as an MC process. In this instance, it triggers the eagle_wrf_generic_extract workflow as a separate MC2 process.
The main disadvantage of this method is that the workflow only creates CMs/triggers for the eagle_wrf_generic_extract and does not track further execution.

II - eagle_wrf_cloud_extract workflow runs as an MC2 process where the eagle_wrf_generic_extract workflow runs as a subprocess.
The advantage of this method is that all extracts work within the eagle_wrf_cloud_extract workflow and therefore the final TSR is generated when all the corresponding processes are completed.

Files changed:
eagle_ml-2-0_workflows/eagle_wrf_cloud_extract.wrf

SDP-85709 Python translator was updated to support missed header elements

Python translator destination logic was updated to support the following header elements:

header/checkActivity
header/processingOptions
header/enableSourceMappingFlag
header/codeTranslations

Files changed:
src/common_core/enums/eml_enum.py
src/eagleinvsys/etl/default_etl_impl.py

SDP-85139 Data Rules Security Resolution Service was Enhanced

The logic for MATCH_ALL, USE_FIRST_AS_MAIN, USE_EXIST_XREF_ONLY was implemented. Additional rejections were implemented for different use cases to improve end user experience.

Files changed:
metadata/core/ontology/resolution/defaultassetresolutionservice.json
metadata/core/ontology/templates/includes/defaultassetresolution.json

SDP-85065 Data Rules support for warehousebrokertradequote Resource

Data Rules warehousebrokertradequote resource was created to support data load into Oracle using the core stored procedure - TRADESDBO.EGL_GEN_TRADE.TRADE_QUOTE_INSUPD.
Resource also supports Eagle - Vault pipeline, extract data from Eagle and upload to Vault.

Supported modes: Batch, Temporal
Table: TRADESDBO.TRADE_QUOTES
Unique Key: TRADE_ID, BROKER
Batch Key: TRADE_ID

Extract Filters:
securityalias, securitytype, securitysubtype, investmenttype, primaryassetid, xrefid, ticker, releasestatus, processingsecuritytype
entityfilter, heldonlyfilter
fromdate, todate,
sourcename,
masterticketnumber,
deltabatchfilter

Files changed:
metadata/core/ontology/eagle-editchecks.json
metadata/core/ontology/warehouse/warehousebrokertradequote.json
metadata/core/ontology/warehouse/warehousebrokertradequote-models.json

Files added:
metadata/core/ontology/warehouse/warehousebrokertradequote_in.json
metadata/core/ontology/templates/db_oper/snowflake/warehousebrokertradequote.reinit_batch_many.sql
metadata/core/ontology/templates/eagle_wrhs_warehousebrokertradequote_insertupdate.txt
metadata/core/ontology/templates/eagle_wrhs_warehousebrokertradequote_reinitbatch.oracle.txt
metadata/core/ontology/templates/eagle_wrhs_warehousebrokertradequote_reinitbatch.snowflake.txt
msgcenter/profile/workflows/EagleToVault-cloud-warehousebrokertradequote_tmp.profile
msgcenter/profile/workflows/EagleToVault-conversion-warehousebrokertradequote_tmp.profile

SDP-85064 Data Rules support for entitypeergroupranking Resource

The Data Rules entitypeergroupranking resource was created to support data load into Oracle using core stored procedure - ESTAR.INS_UPD_ENT_PEER_GRP_RANKINGS.
Resource also supports Eagle - Vault pipeline, extract data from Eagle and upload to Vault.

Supported modes: Delta, Temporal
Table: RULESDBO.ENTITY_PEER_GRP_RANKINGS
Key: ENTITY_ID,SRC_INTFC_INST,EFFECTIVE_DATE,TIME_PERIOD,PEER_GROUP_ID

Extract Filters:
entityfilter, entitytype
fromdate, todate,
effectivedate, fromeffectivedate, toeffectivedate
sourcename

Files changed:
metadata/core/ontology/eagle-resolution-processing.json
metadata/core/ontology/entity/entitypeergroupranking-models.json
metadata/core/ontology/entity/entitypeergroupranking.json

Files added:
metadata/core/ontology/entity/entitypeergroupranking_in.json
metadata/core/ontology/templates/eagle_ent_entitypeergroupranking_insertupdate.txt

SDP-85063 Data Rules support for entityliquidity resource

Data Rules entityliquidity resource is supported now for data load into Oracle via the stored procedure - ESTAR.INS_UPD_ENTITY_LIQUIDITY
Resource also supports Eagle - Vault pipeline, extract data from Eagle and upload to Vault.

Supported update mode: Delta, Temporal
Table: RULESDBO.ENTITY_LIQUIDITY
Key: ENTITY_ID,EFFECTIVE_DATE,SRC_INTFC_INST

Files added:
metadata/core/ontology/entity/entityliquidity_in.json
test-metadata/core/ontology/templates/eagle_ent_entityliquidity_insertupdate.txt
expected-results/entityliquidity_resource_output.json

Files changed:
metadata/core/ontology/entity/entityliquidity-models.json
metadata/core/ontology/entity/entityliquidity.json
metadata/core/ontology/vault/eagle-vault-model.json
expected-results/all_create_snowflake_ddl.sql
expected-results/entitylist_resource_output.json
rdfmeta/test_rdf_meta_convert.py

SDP-85062 The issuerentityrelationship resource is supported via Data Rules

New resource issuerentityrelationship is supported for both database providers - Oracle and Snowflake. It is also supports Eagle - Vault pipeline, extract data from Eagle and upload to Vault.

Supported modes: Delta, Temporal
Table: SECURITYDBO.ISSUER_ENTITY_RELATION
Key: ISSUER_ALIAS,ENTITY_ID,START_DATE

Extract Filters:
entityfilter
issuerid, issuername, issueralias
fromdate, todate,
startdate, fromstartdate, tostartdate
enddate, fromenddate, toenddate

Files changed:
metadata/core/ontology/reference/issuerentityrelationship-models.json
metadata/core/ontology/reference/issuerentityrelationship.json

Files added:
metadata/core/ontology/reference/issuerentityrelationship_in.json
metadata/core/ontology/templates/eagle_ref_issuerentityrelationship_insertupdate.txt

SDP-83065 Data Rules entityidentifiers Resource was enhanced to support DeltaBatch mode

The entityidentifiers resource was enhanced to support DeltaBatch mode on extract and load.
The deltabatchfilter is now supported for the entityidentifiers extract.

Supported updatemode: Delta, Temporal, Batch
Delta - default load mode
Temporal - works at record and at batch level
Batch - triggered by file with _ASISCOPY_DELTABATCH_ file mask. The eagle_wrf_cloud_extract workflow will generate the mask if parameter usedeltabatch is set to Y in the profile for the cloud_extract.

It is recommended to use Batch mode only with ASISCOPY mode.

Batch Key: ENTITY_ID in ASISCOPY mode

Files added:
metadata/core/ontology/templates/db_oper/snowflake/entityidentifiers.reinit_batch_many.sql
metadata/core/ontology/templates/eagle_ent_entityidentifiers_reinitbatch.oracle.txt
metadata/core/ontology/templates/eagle_ent_entityidentifiers_reinitbatch.snowflake.txt

Files changed:
metadata/core/ontology/entity/entityidentifiers.json
metadata/core/ontology/entity/entityidentifiers_in.json

SDP-81857 warehousetrade Resource was updated to support additional fields

Below changes were implemented:

  • Additional elements are currently supported for TRADESDBO.TRADE_DETAIL:
    trdDetUserChar1 - USER_CHAR1
    trdDetUserChar2 - USER_CHAR2
    trdDetUserChar3 - USER_CHAR3
    trdDetUserFloat1 - USER_FLOAT1
    trdDetUserFloat2 - USER_FLOAT2
    trdDetUserFloat3 - USER_FLOAT3

  • Added mapping for element assetLiabilityHedgeFlag for the database field TRADE_DETAIL_EXT.AL_HEDGE_FLAG

  • Removed XSD element trdDetExtUserFloat1 and DB field TRADE_DETAIL_EXT.USER_FLOAT1, the field does not exist in Oracle Database

Files changed:
warehousetrade_in.json
warehousetrade-models.json
eagle-vocab-wh-shared.json
eagle-vocab-definition-wh-shared.json

SDP-55999 Oracle load implemented for issuerrelationship resource via Data Rules

The issuerrelationship resource is supported now for data load for both database providers, including Oracle via the stored procedure - PACE_MASTERDBO.ESTAR_ISSUER.INS_UPD_ISSUER_REL.

Files changed:
metadata/core/ontology/reference/issuerrelationship-models.json
metadata/core/ontology/reference/issuerrelationship_in.json
metadata/core/ontology/templates/eagle_ref_issuerrelationship_insertupdate.txt

Production Bugs

SDP-86442 warehousecloselot resource was updated to correct database field type

The metadata for the warehousecloselot resource was corrected to keep the DISPOSAL_LOTS.ORIGINAL_SOURCE_CODE filed length in sync with the Eagle Core database. The length was changed from 25 to 255 characters.

Files changed:
metadata/core/ontology/warehouse/warehousecloselot-models.json

SDP-86292 Cancel for AccountingTrade used to fail for future dates

The AccountingTrade with the buySellIndicator = 'CANCEL' and empty reportEndDate was failing if the original trade was booked with future dates.

The Cancel failed because the reportEndDate - Tag 221 had a default value of Today which did not cover any future trades in lookup as it looks for trades from the initial Begin Date to the resolved End Date.

Now, if the reportEndDate is not sent, the default value is taken as maximum of the END_DT from the accounting period table for an entity. If an accounting period table does not have any periods for an entity, then Tag 221 will be set to 'Today + 7 days' for a lookup.

Files changed:
eagle_default\in\include\csv-cancel_columns.inc

SDP-86283 DeltaBatch Filter for Warehouse Performance Was Improved to Include UPDATE_DATE and MPC_UPD_DATE

The DeltaBatch filter was improved and now uses multiple fields to identify changed records.
The DeltaBatch Filter for warehouseperformance was corrected to include UPDATE_DATE and MPC_UPD_DATE.

Files changed:
warehouseperformance.json
eagleinvsys\rdf_metadata\rdf_outflow.py
common_core\dataset\ds_filters\delta_batch_filters\delta_batch_filter_cursor_handler.py

SDP-86049 User and ScimUser logic changes for Business and Application groups

User and ScimUser load interfaces were enhanced for Business and Application groups.

If Business group is not sent or does not exist, then the load is failed with error “'Business Group not found“.

If the Application group is not sent or does not exist, then the default EAGLE DEFAULT USER GROUP will be assigned to the user.

The accountingValidationFlag is supported for both User and ScimUser interfaces to switch between the panel or Stored Procedure load mode.

Files changed:
eagle_default\in\xml\xml-user_user.xml
eagle_default\in\xml\xml-user_user_scim.xml
eagle_default\in\xml\include\user_common.inc

SDP-86019 Automation Center Jobs and EJM events Statuses are out of Sync for initial Delta Based run

The EJM event Stop Date was not populated by the process. Delta Based logic was corrected for a case when the new DeltaBasedUID is created and the Start Date and Stop Date are accurately populated. All DeltaBaseUID processes are now completing as expected with correct status in Automation Center.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_delta_wrf.inc

SDP-85841 Missing Errors in the Consolidated TSR

Some extract rules did not have editcheck logic implemented at the end of rule. As a result, the final TSR did not include information about failed steps. The issue was corrected.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_acct_fair_value_comparison.xml
eagle_ml-2-0_cm/out/tagvalue/extract_acct_cash_journal.xml
eagle_ml-2-0_cm/out/tagvalue/extract_acct_realized_gain_loss.xml
eagle_ml-2-0_cm/out/tagvalue/extract_acct_trial_balance.xml
eagle_ml-2-0_cm/out/tagvalue/extract_acct_weighted_average.xml
eagle_ml-2-0_cm/out/tagvalue/extract_acct_working_trial_balance.xml
eagle_ml-2-0_cm/out/tagvalue/extract_globaltaxrate.xml

SDP-85840 Error from distribution stream was not reported in the final TSR

Previously the error from eagle_ml-2-0_default_cm_file_distribution was not caught by editcheck and the final TSR did not include failed distribution step. The issue was corrected by implementing the editcheck at the end of the distribution rule.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/eagle_default_cm_file_distribution.xml

SDP-85676 securitypolicy profile was updated

One missed data nugget was reported on load via eagle_wrf_generic_load. The securitypolicy profile was corrected to not raise a false error for the SQL injection expressions.

Files changed:

metadata/core/securitypolicy.json

SDP-85571 IWS Translator issue with Unpacker

Previously EXEC EDS task failed to resolve the correct path to the file in case of uncompressed task as part of the workflow and an uncompressed result was passed to the EXEC EDS task. The existing logic for eagle_ml-2-0_default_cm_exec_eds stream was improved in case it is called after unpacker in the workflow.

Files changed:
eagle_ml-2-0_cm/in/xml/exec_eds.xml