Enhancements

SDP-76625 Enhanced mapping of EDS resources

The mapping of the PERF_SEC_RETURNS.SECURITY_ALIAS field was fixed in the asiscopy mode for the warehouseperformance resource. The mapping of the TRADE_DETAIL.POSITION_ID and TRADE_DETAIL.PRINCIPAL fields in the warehousetrade resource were fixed. The mismatches in common fields like effective_date were fixed for all resources.

Files changed:
metadata/core/ontology/eagledb-vocab-common.json
metadata/core/ontology/entity/custombenchmark-models.json
metadata/core/ontology/entity/entityanalytic-models.json
metadata/core/ontology/entity/genericentity-models.json
metadata/core/ontology/reference/client-models.json
metadata/core/ontology/reference/clientrelationship-models.json
metadata/core/ontology/reference/genericissueanalytic-models.json
metadata/core/ontology/reference/genericsmf-models.json
metadata/core/ontology/reference/issuerrole-models.json
metadata/core/ontology/warehouse/warehousecloselot-models.json
metadata/core/ontology/warehouse/warehouseopenlot-models.json
metadata/core/ontology/warehouse/warehouseperformance-models.json
metadata/core/ontology/warehouse/warehouseposition-models.json
metadata/core/ontology/warehouse/warehousetrade-models.json

SDP-76508 The EDS entitypolicymap and entitypolicydetail resources were updated to support additional filters

The EDS entitypolicymap now supports the additional filters policymapinstance and policydetailinstance.

Files changed:|
ontology/eagle-interfaces.json
ontology/entity/entitypolicydetail.json
ontology/entity/entitypolicymap.json

SDP-76137 A new Concatenated ID calcCloseUniqueId field was added to WRHSCLOSELOTEXTRACT

A new field, calcCloseUniqueId, has been added for CloseLot Extracts which is created if the RTR parameter EnhanceCalcTransactionId = 'Y'. This id is intended to help distinguish close lot records that spawn across multi-leg corporate actions.

Files changed:
eagle_ml-2-0_cm\out\xml\t_calc_link_identifiers_fields_streaming.inc
eagle_ml-2-0_cm\out\tagvalue\extract_warehouse_closelot.inc

SDP-76089 Enhancement to the EDS languagetranslation resource

The languagetranslation resource was implemented to support the extract from Oracle and load to Snowflake as part of the Eagle - Vault pipeline.

Table: PACE_MASTERDBO.EGL_GENERIC_XLATION_ML

Load Modes:
Delta
Temporal

Supported filters:
fromdate
todate
instance
eglcategory
languageid

Files changed:
ontology/eagle-interfaces.json
ontology/eagle-vocab-enumerations.json
ontology/eagle-vocab-shared.json
ontology/reference/codevalue.json

Files added:
ontology/reference/languagetranslation-models.json
ontology/reference/languagetranslation_in.json
ontology/reference/languagetranslation.json
ontology/templates/eagle_ref_languagetranslation_insertupdate.txt

SDP-76049 Implemented new eagle_ml-2-0_default_in_xml_route_to_eds stream for EDS routing

A new eagle_ml-2-0_default_in_xml_route_to_eds stream has been implemented which processes EagleML formatted data and routes them to EDS for load as a single step. The data file should contain objects of the same feed type only.
Files added:
eagle/estar/eagle_ml-2-0_default_in_xml_route_to_eds.pkg
eagle/estar/tpe/dynamic/msgcenter/eagle_default/in/xml/route_eagleml_to_eds.xml
eagle/estar/tpe/dynamic/msgcenter/eagle_default/in/xml/route_to_eds.rsf

SDP-76010 Added support of batch cancel trade for entities with segregated derivatives

EagleML now supports the batch cancel trade for entities with segregated derivatives. To enable this functionality, the entity should have a value of RULESDBO.ENTITY_STAR_PROCESSING.SEGREGATE_DERIVATIVES = 'Y' set. In addition, the incoming message should include the clearingBrokerName element with a corresponding value illustrated below.

<clearingBrokerName>EAGLE</clearingBrokerName>

The value of Tag16253 - segregateDerivatives, will be retrieved by a DB call automatically on the entity.

Files changed:
eagle_default/in/include/trade_aliases.inc
eagle_default/in/include/csv-cancel_columns.inc

SDP-75447 New parameters added for waiting task_reported step within workflow execution

The EJM processing was enhanced to support two new parameters which allow freezing of the execwkfl step and wait until last task_reporter step finishes.

Local RTR parameters:
WaitLastTaskReporterSeconds - to set how long to wait each time.
WaitLastTaskReporterRetryCounts - to set how many times to wait.

Global parameters:
W_WAIT_LAST_TASK_REPORTER_SECONDS
W_WAIT_LAST_TASK_REPORTER_RETRY_COUNTS

The logic will be activated only when setting WaitLastTaskReporterSeconds parameter with value greater than 0 and integer values are supported. The default value of WaitLastTaskReporterRetryCounts is 3.

File added:
eagle_ml-2-0_cm/waiting_task_reporter.inc
Files changed:
eagle_ml-2-0_cm/out/tagvalue/execute_wrkfl_event.xml

SDP-73700 EDS resource custombenchmark enhancements

The EDS resource custombenchmark was enhanced with the below.

1. Populate z-fields for CUSTOM_INDEX_ATTRIBUTES

2. Added missing mappings

3. Added reinit Many procedure

Files changed:
custombenchmark.json
custombenchmark-models.json
custombenchmark_in.json

SDP-70927 Additional filters Source and EntityID supported in the Conversion Workflow

Additional filters sourcename and ENTITYID can be used now to trigger the conversion workflow. It’s recommended to use additional parameters like sourcename and deletedata set to N to move additional data without deleting the existing data from Snowflake within the given date range.

Files changed:
msgcenter/eagle_ml-2-0_workflows/eagle_wrf_cloud_conversion.wrf
msgcenter/eis/ejmwf/eagle_wrf_cloud_conversion.ejmwf
src/eagleinvsys/rdf_metadata/rdf_outflow.py

SDP-65517 The Conversion Workflow was updated to support all partitions to calculate record count

Previously, the execution plan was built based on number of records for each date and taking into account only default partition. This logic was improved to calculate statistics using all existing partitions in the environment.

Files changed:
src/eagleinvsys/etl/conversion_main.py

Production Bugs

SDP-77050 Conversion Workflow fails due to duplicates in the files name

A regression issue was introduced by previous changes which were implemented for EDS stream regarding duplicated correlation ID’s and as a result the following error was raised:
File Name yf7viishcsgwmkld_asiscopy_adhoc_815_custombenchmark_20230814.csv.gzip matches more than 1 file masks.

The code was updated to correct this issue.

Files changed:
src/eagleinvsys/etl/conversion_main.py

SDP-76500 Expanded settlementDate format for loading for AccountingAssetCash

Within the AccountingAssetCash interfaces, the settlement date was expanded to support the conversion of the “YYYY-MM-DD” format.
<settlementDate>2022-04-20</settlementDate>

The BLINC, Coupon, Cash Dividend and Miscellaneous Income and Miscellaneous Expense were modified to support the date format. The modification changes the date to “YYYYMMDD” which is the expected format.

Files changed:
eagle_default/in/include/csv-cash_blinc.inc
eagle_default/in/include/csv-cash_coupon.inc
eagle_default/in/include/csv-cash_div.inc
eagle_default/in/include/csv-cash_misc_inc_exp.inc

SDP-76474 The EDS resource for genericsmf was updated to correct the mapping for the SECURITY_MASTER.TICKER field

Unmatched records were calculated on the genericsmf load to Snowflake. The Checksum did not match between the target and source if the XREF.TICKER and SECURITY_MASTER.TICKER were different. The genericsmf mapping was updated to correct this issue.
EDS genericsmf extract in ASISCOPY mode takes the ticker element from SECURITYDBO.SECURITY_MASTER.TICKER.
EDS genericsmf extract with mashup logic uses the ticker element from SECURITYDBO.XREFERENCE.TICKER.

Files changed:
ontology/reference/genericsmf-models.json
ontolgoy/reference/genericsmf_in.json

SDP-76242 Handling of cancel panels errors in Accounting Trade Auto Rebook logic

The Accounting Trade Auto Rebook logic was enhanced and now terminates the process if the cancel panel generates an error. The error will be trapped in mc console.

Files changed:
eagle_default\in\include\accttrade_true_rebook.inc

SDP-75515 Updated the Entity and EntityAccountingBasis extracts to align with the XSD schema

The order of the elements in the ENTITYACCBASISEXTRACT and ENTITYEXTRACT are now in sync with XSD schema.

Model “AccountingProcessing.model” was added in complexType "EntityAccountingBasis".

Files changed:

eagle_ml-2-0_cm/out/xml/t_entity_accounting_basis_streaming.inc
eagle_ml-2-0_cm/out/xml/t_entity_streaming.inc
eagle_ml-2-0_cm/out/xml/t_entity_streaming_eml.inc
dynamic/eagleml/schema/eagleml-entity-2-0.xsd