EagleML Release Notes - July 25, 2024
Enhancements
SDP-87252 EagleML Voluntary Election Inbound interface was enhanced with Entity Bank relationship resolution for Update and Delete Actions
The EagleML Voluntary Election Inbound interface was enhanced with Entity Bank relationship resolution for Update and Delete operations.
Files changed:
eagle_default/in/xml/xml-ref_voluntarycaelections.xml
SDP-86971 The Logic for Xreference Identifiers was adjusted to exclude duplicates
The logic for SMF xreference mapping was enhanced with an additional check to validate if the identifier is duplicated in the incoming data to pass the set of unique pairs for xId/xIdType.
For example, if the incoming data has bloombergId as an xref and as separate tag, then they are mapped to the same pair of xId/xIdType:
{noformat}<bloombergId>EQ86971</bloombergId>
<xrefIdentifiers>
 <xId26>
       <xId>EQ86971</xId>
       <xIdType>BBID</xIdType>
 </xId26>
</xrefIdentifiers>{noformat}
This pair {{EQ86971}}/{{BBID}} will be passed into DB only once. This ensures that EagleML will not miss identifiers low in priority because of duplicated ones.
Files changed:
tpe/dynamic/msgcenter/eagle_default/in/xml/include/smf_identifiers_map.inc
SDP-86417 The multipleLegAccountingTrade inbound interface enhanced with new elements
The multipleLegAccountingTrade multiple leg inbound interface has been enhanced with new elements:
accountingTransaction/multipleLegAccountingTrade/beginAccrualDate
accountingTransaction/multipleLegAccountingTrade/endAccrualDate
Files changed:
eagle_default/in/xml/include/multiplelegtrade_aliases.inc
SDP-86194 Signage logic changes for BUYCVR and BUYCVRCANCEL events
The ACCRUED_INTEREST and BASEINTERESTADJ signage logic has been changed for BUYCVR and BUYCVRCANCEL events. In case the signage logic is applied these fields will be presented differently.
Signage is activated when passed when the EnableSignageLogic is set to Y and the IncludeExtendedDetails is set to Y or ALL.
<taskParameter>
<name>EnableSignageLogic</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>
<taskParameter>
<name>IncludeExtendedDetails </name>
<dataType>S</dataType>
<value>ALL</value>
</taskParameter>
Files changed:
tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/tagvalue/extract_wrhs_trade_signage_logic.inc
SDP-85883 Warehouse Position load supports srcSysC element
The Warehouse Position Inbound interface was enhanced to support srcSysC for load in all modes.
Files changed:
eagle_default/in/xml/xslt/wrh_egl_gen_interface3.xsl
eagle_ml-2-0_cm/dbkeysmap.inc
eagle_ml-2-0_cm/out/xml/t_warehouse_position_xmldbapi.xml
eagle_ml-2-0_cm/out/xml/t_warehouse_position_xmldbapi_streaming.inc
SDP-85849 Added new flag ValuationSource for PRICINGRDCREPORTEXTRACT
A new flag, ValuationSource, has been added for the PRICINGRDCREPORTEXTRACT. When it set to MONTHLY, pricing data will be extracted from the ESTAR.MONTHLY_PORTFOLIO_VALUATION. When it set to DAILY or not presented in the RTR, data will be extracted from the ESTAR.MONTHLY_PORTFOLIO_VALUATION table. The default value is DAILY.
The functionality can also be specified in the /eagle_ml-2-0_custom_cm/w_config_custom.inc file via the global W_PRICINGREPORTEXTRACT_VALUATION_SRC variable for all PRICINGRDCREPORTEXTRACT requests.
The RTR has higher priority over the custom include file. When both the ValuationSource and W_PRICINGREPORTEXTRACT_VALUATION_SRC are not null, the pricing table is chosen from the RTR parameter.
<taskParameter>
<name>ValuationSource</name>
<dataType>S</dataType>
<value>MONTHLY</value>
</taskParameter>
Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_pricing_report.inc
SDP-85947Â Python CSV2XML: name 'getvariable' is not defined - adjust getVariable functionality
A new object called getvariable has been implemented to support dynamic expression calculation for variables.
Dynamic expression calculation means the presence of two execution steps, as opposed to the traditional single execution step seen with a variable, where expression can be only static.
For example:
[
"getvariable",
"extUSER_CHAR",
"iif(stringlength(get_instruction_param('USER_CHAR', :GVAR_RTR: )>0,get_instruction_param('USER_CHAR', :GVAR_RTR: ,'')"
],
In this case there will be two executions (calculations):
The first Execution will calculate expression (iif(stringlength(get_instruction_param('USER_CHAR', :GVAR_RTR: )>0,get_instruction_param('USER_CHAR', :GVAR_RTR: ,'')). In our example the result of this expression is |57|;
The second Execution will use the result of the 1st execution, it works as expression for the second execution. In this scenario, the value will be taken from column 57 of the incoming CSV file.
This variable then can be used in the mapping as typical variable:
[
":extUSER_CHAR:",
"",
false,
null,
"",
"accountingTrade/userChar5"
],
In the resulting file, we should expect to find the value from column 57 in the element accountingTrade/userChar5
.
It works the same way if the variable is defined with static expression:
[
"variable",
"extUSER_CHAR",
"|57|"
],
However, in the scenario, it's expected that the definition will be dynamically defined and the column number will be modified.
SDP-82063 Added new flag ValuationSource for PRICINGREPORTEXTRACT
A new flag, ValuationSource, has been added for the PRICINGREPORTEXTRACT. When it set to MONTHLY, pricing data will be extracted from the ESTAR.MONTHLY_PORTFOLIO_VALUATION. When it set to DAILY or not presented in the RTR, data will be extracted from the ESTAR.MONTHLY_PORTFOLIO_VALUATION table. The default value is DAILY.
The functionality can also be specified in the /eagle_ml-2-0_custom_cm/w_config_custom.inc file via the global W_PRICINGREPORTEXTRACT_VALUATION_SRC variable for all PRICINGREPORTEXTRACT requests.
The RTR has higher priority over the custom include file. When both the ValuationSource and W_PRICINGREPORTEXTRACT_VALUATION_SRC are not null, the pricing table is chosen from the RTR parameter.
<taskParameter>
<name>ValuationSource</name>
<dataType>S</dataType>
<value>MONTHLY</value>
</taskParameter>
Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_pricing_report.inc
SDP-85845 VoluntaryCAElection interface resolves the custody elements
A new custodian element, entityBankAlias, is supported in the Corporate Actions Voluntary Elections for load.
If none of the new elements, custodianBankCode, custodyBankAccount, custodyBankSubaccount are sent, nothing changes for existing clients. However, if at least one of the new elements is present in the upload file, new logic is applied to check all custodian fields. Once all three elements are resolved, the entityBankAlias is resolved as a result. If even one of the identifiers is not found in the panel lookup or was not sent in the file, an "entityBankAlias" error is produced.
Additionally, a new processing option has been introduced:
<processingOptions>USE_ENTITY_BANK_ALIAS</processingOptions>
When using this option, the entityBankAlias is expected during the load process.
EagleML Xpath:
EagleML/referenceTransaction/voluntaryCAElection/custodianBankCode - Tag 1176
EagleML/referenceTransaction/voluntaryCAElection/custodyBankAccount - Tag 1660
EagleML/referenceTransaction/voluntaryCAElection/custodyBankSubaccount - Tag 1661
EagleML/referenceTransaction/voluntaryCAElection/entityBankAlias - Tag 16
Files changed:
eagle_default/in/xml/xml-ref_voluntarycaelections.xml
SDP-85455 Asset Resolution Options for Warehouse Trade Interface
Two resolution options were added for the warehouseTrade object:
RESOLVE_PRIMARY is used to resolve the primaryAssetId; this can be useful to validate the securityAlias
<assetResolutionOption>RESOLVE_PRIMARY</assetResolutionOption>
SKIP is used to use the securityAlias passed from the incoming data without resolution
<assetResolutionOption>SKIP</assetResolutionOption>
Both options allow to load trades via the warehouseTrade object using only the securityAlias.
Files changed:
eagle_default/in/xml/include/xml-dbdirect_sec_resolution.inc
SDP-85425 SFTP Distribution with Optional Public Key
Optional SFTP Distribution Key. If the SFTP_PUBLICKEY parameter is not defined, then the process will use the SFTP_PRIVATEKEY instead.
Files changed:
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/reporter_dist_methods.inc
SDP-85213 DBTATA (TBA) Trades updated To Support All Lot Selection Methods
The AccountingTrade interface has been enhanced for TBA trades. All Eagle Lot Selection Methods or Tag 27 are available for the Processing Security Type (PST) DBTATA trades in EML now. Previously, IDLOT was only allowed for DBTATA.
Files changed:
eagle_default/in/include/csv-sell_columns.inc
SDP-84964 VOLCAELECTIONEXTRACT enhanced to support new elements
New custodian elements custodyBankAccount, custodianBankCode, custodianBankName, custodyBankSubaccount, and entityBankAlias are supported in the Corporate Actions Voluntary Elections for extract.
Xpath | DB mapping:
voluntaryCAElection/custodianBankCode | RULESDBO.ENTITY_CUSTODY_RELATIONSHIP.CUSTODY_BANK_CODE
/voluntaryCAElection/custodianBankName | RULESDBO.ENTITY_CUSTODY_RELATIONSHIP.CUSTODY_BANK_DESC
voluntaryCAElection/custodyBankAccount | RULESDBO.ENTITY_CUSTODY_RELATIONSHIP.CUSTODY_BANK_ACCT_NUM
/voluntaryCAElection/custodyBankSubaccount | RULESDBO.ENTITY_CUSTODY_RELATIONSHIP.CUSTODY_BANK_SUB_ACCT
/voluntaryCAElection/entityBankAlias | SECURITYDBO.VOL_CORP_ACTION_ELECTION.ENTITY_BANK_ALIAS
Files changed:
eagle_ml-2-0_cm/out/xml/t_voluntary_ca_election_streaming.inc
eagle_ml-2-0_cm/out/tagvalue/extract_voluntary_ca_election.xml
SDP-84906 The multipleLegAccountingTrade single event inbound interface enhanced with new elements
multipleLegAccountingTrade single event inbound interface enhanced with new elements
<payTradeLeg>
<tradedInterest>30</tradedInterest> <!-- Tags 17937, 18280 -->
<tradedInterestBase>25</tradedInterestBase> <!-- Tags 17942, 18281 -->
<firstPeriodCouponRate>3</firstPeriodCouponRate> <!-- Tag 18282 -->
<lotLevelDatedDate>2024-01-04</lotLevelDatedDate><!-- Tag 18283 -->
</payTradeLeg>
<receiveTradeLeg>
<tradedInterest>40</tradedInterest> <!-- Tags 17938, 18280 if not exist in pay leg -->
<tradedInterestBase>35</tradedInterestBase> <!-- Tags 17943, 18281 if not exist in pay leg -->
<firstPeriodCouponRate>3</firstPeriodCouponRate> <!-- Tag 18282 if not exist in pay leg -->
<lotLevelDatedDate>2024-01-04</lotLevelDatedDate><!-- Tag 18283 if not exist in pay leg -->
</receiveTradeLeg>
Files changed:
eagle_default/in/xml/xml-accttrade_singleleg.xml
SDP-83690 Additional Load Fields for the ExchangePrice Load and ISSUEEXCHANGEPRICEEXTRACT
New fields added for the load and extract for the ExchangePrice Load and ISSUEEXCHANGEPRICEEXTRACT:
EagleML/referenceTransaction/exchangePrice/actualSettlementPrice SECURITYDBO.PRICE_EXCHANGE.PX_SETTLE_ACTUAL - Tag 18582
EagleML/referenceTransaction/exchangePrice/actualSettlementPriceDate SECURITYDBO.PRICE_EXCHANGE.PX_SETTLE_ACTUAL_DATE - Tag 18583
EagleML/referenceTransaction/exchangePrice/marketStatus SECURITYDBO.PRICE_EXCHANGE.MARKET_STATUS - Tag 18584
EagleML/referenceTransaction/exchangePrice/foreignInstitutionalInvestorLimit SECURITYDBO.PRICE_EXCHANGE.FII_LIMIT - Tag 18585
EagleML/referenceTransaction/exchangePrice/cleanDirtyFlag SECURITYDBO.PRICE_EXCHANGE.CLEAN_DIRTY_FLAG - Tag 18586
EagleML/referenceTransaction/exchangePrice/exchangeListedFlag SECURITYDBO.PRICE_EXCHANGE.EXCHANGE_LISTED_FLAG - Tag 18587
EagleML/referenceTransaction/exchangePrice/holidayFlag SECURITYDBO.PRICE_EXCHANGE.HOLIDAY_FLAG - Tag 18588
Also, following pricing flags are added for existing fields: closingPriceCondition, oidYieldPriceCondition, highPriceCondition, lowPriceCondition, referencePriceCondition, nominalDirtyPriceCondition, cleanInclOptionPriceCondition, dirtyInclOptionPriceCondition, cleanExclOptionPriceCondition, dirtyExclOptionPriceCondition, callOptionPriceCondition, putOptionPriceCondition, fairValuePriceCondition, averagePriceCondition, fixingPriceCondition, grossAskPriceCondition, grossBidPriceCondition, grossMidPriceCondition, midClosePriceCondition, meanPriceCondition, nominalPriceCondition, officialClosePriceCondition, vwapClosePriceCondition, askAdrPriceCondition, bidAdrPriceCondition, lastAdrPriceCondition, lastUpdatePriceCondition, naicPriceCondition, navCurrentPriceCondition, userFee1PriceCondition, userFee2PriceCondition, userFee3PriceCondition, userNav1PriceCondition, userNav2PriceCondition, userNav3PriceCondition, userNav4PriceCondition, userNav5PriceCondition, pxSettleCondition, actualSettlementPriceCondition, pxYestPriceCondition.
Files changed:
metadata/bind/eagleml_ref_exchangeprice.xml
eagle_default/in/xml/xml-ref_exchangeprice.xml
eagle_default/in/xml/include/xml-ref_security_pricing_common.inc
eagle_ml-2-0_cm/out/xml/t_issueprice_exchange_streaming.inc
SDP-87039 The Warehouse Trade EagleML Inbound interface was enhanced with additional elements
The warehouseTrade EagleML inbound interface enhanced with new elements:
EagleML/warehouseTransaction/warehouseTrade/eventSequence
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate1
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate2
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate3
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate4
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate5
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate6
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate7
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate8
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate9
EagleML/warehouseTransaction/warehouseTrade/trdDetExtUserDate10
File changed:
eagle_default/in/xml/xslt/wrh_egl_gen_interface3.xsl
SDP-86384 Data Rules Profile Templates for Conversion and Cloud Workflows for New Resources
Added Data Rules profile templates for the Conversion and Cloud Workflows for issuerentityrelationship, entityliquidity and entitypeergroupranking resources.
Files added:
msgcenter/profile/workflows/EagleToVault-cloud-entityliquidity_tmp.profile
msgcenter/profile/workflows/EagleToVault-cloud-entitypeergroupranking_tmp.profile
msgcenter/profile/workflows/EagleToVault-cloud-issuerentityrelationship_tmp.profile
msgcenter/profile/workflows/EagleToVault-conversion-entityliquidity_tmp.profile
msgcenter/profile/workflows/EagleToVault-conversion-entitypeergroupranking_tmp.profile
msgcenter/profile/workflows/EagleToVault-conversion-issuerentityrelationship_tmp.profile
Files changed:
eagle/estar/EAGLE_EXTRACTSERVICE.pkg
eagle/estar/eagle_default_workflows.pkg
SDP-86309 Additional fields supported for Oracle load via entityaccountingbasis resource
The fields PERF_NAV_FORWARD_PRICE_SOURCE (performanceNavForwardPriceSource) and PERF_NAV_VALUATION_FX_SOURCE (performanceNavValuationFxSource) now supported for load.
Files changed:
entityaccountingbasis-models.json
SDP-86235 The warehouseposition resource was updated to support a few missing fields
The warehouseposition resource has been enhanced to support additional fields for both database providers for load and extract. The mapping was implemented for missing canonical elements.
Database Field Canonical Element
POSITION_DETAIL.SHARE_CLASS - shareClass
POSITION_DETAIL.CUST_PRINCIPAL_BALANCE - custodianPrincipalBalanceBase
POSITION_DETAIL.CUST_PRINCIPAL_BALANCE_LOCAL - custodianPrincipalBalanceLocal
POSITION_DETAIL.CUST_INCOME_BALANCE - custodianIncomeBalanceBase
POSITION_DETAIL.CUST_INCOME_BALANCE_LOCAL - custodianIncomeBalanceLocal
Files Changed:
eagle-vocab-wh-shared.json
eagle-vocab-definition-wh-shared.json
warehouseposition-models.json
SDP-86200 Warehouseposition Reinitbatch Stored Procedure was updated for Oracle load to ingest data correctly
All warehouseposition fields loaded data passed without null values. Previously, a number of fields passed where ignored during the load process.
Files Changed:
metadata\core\ontology\templates\eagle_wrhs_warehouseposition_reinitbatch.oracle
SDP-86196 Information in data load error is not sufficient to triage issue
Logic for reporting error messages to the end users was improved to inherit global parameter for length, also this value can be adjusted via RTR parameter: procstatusmsgsize
. The Default size is set to 300.
For troubleshooting purposes you can increase value to get more details via error message, but please be aware that too long error messages can cause unexpected behavior.
If the error has a line number, then the error message will have text from this line for troubleshooting purposes.
Files changed:
common-core/descriptor/dbdescriptor.py
common-core/processing-env/proc_log_stru.py
common-core/processing-env/processing_context.py
common-core/processing-env/processing_log.py
common-core/processing-env/proc_log_policy.py
SDP-85755 Cloud workflow eagle_wrf_cloud_extract was updated
The eagle_wrf_cloud_extract workflow can be executed in two different ways:
I - default setup or original implementation where the eagle_wrf_cloud_extract workflow runs as an MC process. In this instance, it triggers the eagle_wrf_generic_extract workflow as a separate MC2 process.
The main disadvantage of this method is that the workflow only creates CMs/triggers for the eagle_wrf_generic_extract and does not track further execution.
II - eagle_wrf_cloud_extract workflow runs as an MC2 process where the eagle_wrf_generic_extract workflow runs as a subprocess.
The advantage of this method is that all extracts work within the eagle_wrf_cloud_extract workflow and therefore the final TSR is generated when all the corresponding processes are completed.
Files changed:
eagle_ml-2-0_workflows/eagle_wrf_cloud_extract.wrf
SDP-85709 Python translator was updated to support missed header elements
Python translator destination logic was updated to support the following header elements:
header/checkActivity
header/processingOptions
header/enableSourceMappingFlag
header/codeTranslations
Files changed:
src/common_core/enums/eml_enum.py
src/eagleinvsys/etl/default_etl_impl.py
SDP-85139 Data Rules Security Resolution Service was Enhanced
The logic for MATCH_ALL, USE_FIRST_AS_MAIN, USE_EXIST_XREF_ONLY was implemented. Additional rejections were implemented for different use cases to improve end user experience.
Files changed:
metadata/core/ontology/resolution/defaultassetresolutionservice.json
metadata/core/ontology/templates/includes/defaultassetresolution.json
SDP-85065 Data Rules support for warehousebrokertradequote Resource
Data Rules warehousebrokertradequote resource was created to support data load into Oracle using the core stored procedure - TRADESDBO.EGL_GEN_TRADE.TRADE_QUOTE_INSUPD.
Resource also supports Eagle - Vault pipeline, extract data from Eagle and upload to Vault.
Supported modes: Batch, Temporal
Table: TRADESDBO.TRADE_QUOTES
Unique Key: TRADE_ID, BROKER
Batch Key: TRADE_ID
Extract Filters:
securityalias, securitytype, securitysubtype, investmenttype, primaryassetid, xrefid, ticker, releasestatus, processingsecuritytype
entityfilter, heldonlyfilter
fromdate, todate,
sourcename,
masterticketnumber,
deltabatchfilter
Files changed:
metadata/core/ontology/eagle-editchecks.json
metadata/core/ontology/warehouse/warehousebrokertradequote.json
metadata/core/ontology/warehouse/warehousebrokertradequote-models.json
Files added:
metadata/core/ontology/warehouse/warehousebrokertradequote_in.json
metadata/core/ontology/templates/db_oper/snowflake/warehousebrokertradequote.reinit_batch_many.sql
metadata/core/ontology/templates/eagle_wrhs_warehousebrokertradequote_insertupdate.txt
metadata/core/ontology/templates/eagle_wrhs_warehousebrokertradequote_reinitbatch.oracle.txt
metadata/core/ontology/templates/eagle_wrhs_warehousebrokertradequote_reinitbatch.snowflake.txt
msgcenter/profile/workflows/EagleToVault-cloud-warehousebrokertradequote_tmp.profile
msgcenter/profile/workflows/EagleToVault-conversion-warehousebrokertradequote_tmp.profile
SDP-85064 Data Rules support for entitypeergroupranking Resource
The Data Rules entitypeergroupranking resource was created to support data load into Oracle using core stored procedure - ESTAR.INS_UPD_ENT_PEER_GRP_RANKINGS.
Resource also supports Eagle - Vault pipeline, extract data from Eagle and upload to Vault.
Supported modes: Delta, Temporal
Table: RULESDBO.ENTITY_PEER_GRP_RANKINGS
Key: ENTITY_ID,SRC_INTFC_INST,EFFECTIVE_DATE,TIME_PERIOD,PEER_GROUP_ID
Extract Filters:
entityfilter, entitytype
fromdate, todate,
effectivedate, fromeffectivedate, toeffectivedate
sourcename
Files changed:
metadata/core/ontology/eagle-resolution-processing.json
metadata/core/ontology/entity/entitypeergroupranking-models.json
metadata/core/ontology/entity/entitypeergroupranking.json
Files added:
metadata/core/ontology/entity/entitypeergroupranking_in.json
metadata/core/ontology/templates/eagle_ent_entitypeergroupranking_insertupdate.txt
SDP-85063 Data Rules support for entityliquidity resource
Data Rules entityliquidity resource is supported now for data load into Oracle via the stored procedure - ESTAR.INS_UPD_ENTITY_LIQUIDITY
Resource also supports Eagle - Vault pipeline, extract data from Eagle and upload to Vault.
Supported update mode: Delta, Temporal
Table: RULESDBO.ENTITY_LIQUIDITY
Key: ENTITY_ID,EFFECTIVE_DATE,SRC_INTFC_INST
Files added:
metadata/core/ontology/entity/entityliquidity_in.json
test-metadata/core/ontology/templates/eagle_ent_entityliquidity_insertupdate.txt
expected-results/entityliquidity_resource_output.json
Files changed:
metadata/core/ontology/entity/entityliquidity-models.json
metadata/core/ontology/entity/entityliquidity.json
metadata/core/ontology/vault/eagle-vault-model.json
expected-results/all_create_snowflake_ddl.sql
expected-results/entitylist_resource_output.json
rdfmeta/test_rdf_meta_convert.py
SDP-85062 The issuerentityrelationship resource is supported via Data Rules
New resource issuerentityrelationship is supported for both database providers - Oracle and Snowflake. It is also supports Eagle - Vault pipeline, extract data from Eagle and upload to Vault.
Supported modes: Delta, Temporal
Table: SECURITYDBO.ISSUER_ENTITY_RELATION
Key: ISSUER_ALIAS,ENTITY_ID,START_DATE
Extract Filters:
entityfilter
issuerid, issuername, issueralias
fromdate, todate,
startdate, fromstartdate, tostartdate
enddate, fromenddate, toenddate
Files changed:
metadata/core/ontology/reference/issuerentityrelationship-models.json
metadata/core/ontology/reference/issuerentityrelationship.json
Files added:
metadata/core/ontology/reference/issuerentityrelationship_in.json
metadata/core/ontology/templates/eagle_ref_issuerentityrelationship_insertupdate.txt
SDP-83065 Data Rules entityidentifiers Resource was enhanced to support DeltaBatch mode
The entityidentifiers resource was enhanced to support DeltaBatch mode on extract and load.
The deltabatchfilter is now supported for the entityidentifiers extract.
Supported updatemode: Delta, Temporal, Batch
Delta - default load mode
Temporal - works at record and at batch level
Batch - triggered by file with _ASISCOPY_DELTABATCH_ file mask. The eagle_wrf_cloud_extract workflow will generate the mask if parameter usedeltabatch is set to Y in the profile for the cloud_extract.
It is recommended to use Batch mode only with ASISCOPY mode.
Batch Key: ENTITY_ID in ASISCOPY mode
Files added:
metadata/core/ontology/templates/db_oper/snowflake/entityidentifiers.reinit_batch_many.sql
metadata/core/ontology/templates/eagle_ent_entityidentifiers_reinitbatch.oracle.txt
metadata/core/ontology/templates/eagle_ent_entityidentifiers_reinitbatch.snowflake.txt
Files changed:
metadata/core/ontology/entity/entityidentifiers.json
metadata/core/ontology/entity/entityidentifiers_in.json
SDP-81857 warehousetrade Resource was updated to support additional fields
Below changes were implemented:
Additional elements are currently supported for TRADESDBO.TRADE_DETAIL:
trdDetUserChar1 - USER_CHAR1
trdDetUserChar2 - USER_CHAR2
trdDetUserChar3 - USER_CHAR3
trdDetUserFloat1 - USER_FLOAT1
trdDetUserFloat2 - USER_FLOAT2
trdDetUserFloat3 - USER_FLOAT3Added mapping for element assetLiabilityHedgeFlag for the database field TRADE_DETAIL_EXT.AL_HEDGE_FLAG
Removed XSD element trdDetExtUserFloat1 and DB field TRADE_DETAIL_EXT.USER_FLOAT1, the field does not exist in Oracle Database
Files changed:
warehousetrade_in.json
warehousetrade-models.json
eagle-vocab-wh-shared.json
eagle-vocab-definition-wh-shared.json
SDP-55999 Oracle load implemented for issuerrelationship resource via Data Rules
The issuerrelationship resource is supported now for data load for both database providers, including Oracle via the stored procedure - PACE_MASTERDBO.ESTAR_ISSUER.INS_UPD_ISSUER_REL.
Files changed:
metadata/core/ontology/reference/issuerrelationship-models.json
metadata/core/ontology/reference/issuerrelationship_in.json
metadata/core/ontology/templates/eagle_ref_issuerrelationship_insertupdate.txt
Production Bugs
SDP-86442 warehousecloselot resource was updated to correct database field type
The metadata for the warehousecloselot resource was corrected to keep the DISPOSAL_LOTS.ORIGINAL_SOURCE_CODE filed length in sync with the Eagle Core database. The length was changed from 25 to 255 characters.
Files changed:
metadata/core/ontology/warehouse/warehousecloselot-models.json
SDP-86292 Cancel for AccountingTrade used to fail for future dates
The AccountingTrade with the buySellIndicator = 'CANCEL' and empty reportEndDate was failing if the original trade was booked with future dates.
The Cancel failed because the reportEndDate - Tag 221 had a default value of Today which did not cover any future trades in lookup as it looks for trades from the initial Begin Date to the resolved End Date.
Now, if the reportEndDate is not sent, the default value is taken as maximum of the END_DT from the accounting period table for an entity. If an accounting period table does not have any periods for an entity, then Tag 221 will be set to 'Today + 7 days' for a lookup.
Files changed:
eagle_default\in\include\csv-cancel_columns.inc
SDP-86283 DeltaBatch Filter for Warehouse Performance Was Improved to Include UPDATE_DATE and MPC_UPD_DATE
The DeltaBatch filter was improved and now uses multiple fields to identify changed records.
The DeltaBatch Filter for warehouseperformance was corrected to include UPDATE_DATE and MPC_UPD_DATE.
Files changed:
warehouseperformance.json
eagleinvsys\rdf_metadata\rdf_outflow.py
common_core\dataset\ds_filters\delta_batch_filters\delta_batch_filter_cursor_handler.py
SDP-86049 User and ScimUser logic changes for Business and Application groups
User and ScimUser load interfaces were enhanced for Business and Application groups.
If Business group is not sent or does not exist, then the load is failed with error “'Business Group not found“.
If the Application group is not sent or does not exist, then the default EAGLE DEFAULT USER GROUP will be assigned to the user.
The accountingValidationFlag is supported for both User and ScimUser interfaces to switch between the panel or Stored Procedure load mode.
Files changed:
eagle_default\in\xml\xml-user_user.xml
eagle_default\in\xml\xml-user_user_scim.xml
eagle_default\in\xml\include\user_common.inc
SDP-86019 Automation Center Jobs and EJM events Statuses are out of Sync for initial Delta Based run
The EJM event Stop Date was not populated by the process. Delta Based logic was corrected for a case when the new DeltaBasedUID is created and the Start Date and Stop Date are accurately populated. All DeltaBaseUID processes are now completing as expected with correct status in Automation Center.
Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_delta_wrf.inc
SDP-85841 Missing Errors in the Consolidated TSR
Some extract rules did not have editcheck logic implemented at the end of rule. As a result, the final TSR did not include information about failed steps. The issue was corrected.
Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_acct_fair_value_comparison.xml
eagle_ml-2-0_cm/out/tagvalue/extract_acct_cash_journal.xml
eagle_ml-2-0_cm/out/tagvalue/extract_acct_realized_gain_loss.xml
eagle_ml-2-0_cm/out/tagvalue/extract_acct_trial_balance.xml
eagle_ml-2-0_cm/out/tagvalue/extract_acct_weighted_average.xml
eagle_ml-2-0_cm/out/tagvalue/extract_acct_working_trial_balance.xml
eagle_ml-2-0_cm/out/tagvalue/extract_globaltaxrate.xml
SDP-85840 Error from distribution stream was not reported in the final TSR
Previously the error from eagle_ml-2-0_default_cm_file_distribution was not caught by editcheck and the final TSR did not include failed distribution step. The issue was corrected by implementing the editcheck at the end of the distribution rule.
Files changed:
eagle_ml-2-0_cm/out/tagvalue/eagle_default_cm_file_distribution.xml
SDP-85676 securitypolicy profile was updated
One missed data nugget was reported on load via eagle_wrf_generic_load. The securitypolicy profile was corrected to not raise a false error for the SQL injection expressions.
Files changed:
metadata/core/securitypolicy.json
SDP-85571 IWS Translator issue with Unpacker
Previously EXEC EDS task failed to resolve the correct path to the file in case of uncompressed task as part of the workflow and an uncompressed result was passed to the EXEC EDS task. The existing logic for eagle_ml-2-0_default_cm_exec_eds stream was improved in case it is called after unpacker in the workflow.
Files changed:
eagle_ml-2-0_cm/in/xml/exec_eds.xml