Enhancements

SDP-81198 The Conversion Workflow was enhanced to support the scenario where the conversion date is NULL

Previously, if database table contained NULL values in the conversion date field (UPDATE_DATE, EFFECTIVE_DATE, etc.), then these records were skipped during conversion process. New logic now allows to take into account records with NULL values in the conversion date field while building an execution plan. A separate query with the condition WHERE MASTERTABLE was created.

{DATE_FIELD}

IS NULL is executed. A separate RTR for the extract is created with NULL values instead of date filters.

<taskParameter>
<name>fromeffectivedate</name>
<value>NULL</value>
</taskParameter>

In this case, the file mask of the result file contains NULL instead of dates:
K26C9A17D6FE1891_ASISCOPY_CONVERSION_NULL_NULL_10_GENERICSMF_20231225.csv

The Extract Filtering logic was improved to work with NULL values in filters.

The Conversion Workflow Delete logic was updated to support NULL values on delete also.

Files changed:
src/eagleinvsys/etl/conversion_main.py
pyrules/eagleinvsys/common_core/dataset/ds_filters/filter_operation.py
pyrules/eagleinvsys/common_core/dataset/ds_filters/query_filter_processor_registry.py
pyrules/eagleinvsys/common_core/processing_env/instructions.py
pyrules/eagleinvsys/common_core/core_utils/rdf_meta_runtime_utils.py

SDP-80854 Extended the CA EagleML extract to support cpSubPriority

The cpSubPriority element was added to the Extended CA model and is part of the extract for the WRHSTRADEEXTRACT, WRHSCLOSELOTEXTRACT, TRADEDCAEXTRACT and SETTLEDCAEXTRACT.

TRADESDBO.CORP_ACT_HIST.CP_SUB_PRIORITY - Tag 3961 - cpSubPriority

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_ca_properties_fields.inc
eagle_ml-2-0_cm/out/xml/t_ca_properties_fields_streaming.inc

SDP-80803 The InterportfolioTransfer interface was enhanced to support Settlement Date

The InterportfolioTransfer load interface did not support the settlement date field mapping and was setting it equal to the trade date from the incoming data file. The interface now supports the settlementDate element and will set the value based upon the incoming data.

EagleML/accountingTransaction/interportfolioTransfer/settlementDate - Tag 37

Files changed:
eagle_default\in\xml\xml-acct_interporttransfer.xml

SDP-80685 The EagleML Entity Stream was enhanced to support Fund Level WAM/WAL Data Source

The Generic EagleML entity stream now supports Fund Level WAM/WAL Data Source. The data file will need to include the wamWalDataSource with a valid value to be used in processing.

EagleML/entityTransaction/genericEntity/wamWalDataSource - Tag 18564

Files changed:
eagle/estar/tpe/dynamic/msgcenter/eagle_default/in/csv/csv-entity_masterfund.xml
eagle/estar/tpe/dynamic/msgcenter/eagle_default/in/star/star-entity_masterfund.xml

SDP-80092 Added support for tag 1293 in the forwardSMF and multilegSMF load

The journalizingFlag, Tag1293 element, was added to the forwardSMF and multilegSMF interfaces. It is now possible to enable/disable journaling the underlying information into the PACE_MASTERDBO.SECURITY_JOURNAL_MASTER and PACE_MASTERDBO.GENERIC_JOURNAL_DETAILS tables. When Tag1293 = ‘EXCLUDE_JOURNAL’, then no information about inserting, changing or deleting underlying information will populate the journal tables.

Files changed:
/eagle_default/in/xml/include/forwardsmf_und_link.inc
/eagle_default/in/xml/xml-smf_multileg_swap.xml

SDP-79119 The custombenchmark resource was enhanced to support the data load to Oracle

The custombenchmark EDS resource was enhanced to support the load to Oracle via the core stored ESTAR.INS_UPD_CUSTOM_BENCHMARKS stored procedure.

Below mappings were updated:
EagleML/entityTransaction/customBenchmark/dictionaryTarget - contains dictionary name for the resolution on SP side instead of value from RULESDBO.CUSTOM_INDEX_ATTRIBUTES.TARGET_DICT_ID
EagleML/entityTransaction/customBenchmark/dictionarySource - contains dictionary name for the resolution on SP side instead of value from RULESDBO.CUSTOM_INDEX_ATTRIBUTES.SOURCE_DICT_ID

New XML elements:
EagleML/entityTransaction/customBenchmark/dictionaryTargetId - RULESDBO.CUSTOM_INDEX_ATTRIBUTES.TARGET_DICT_ID
EagleML/entityTransaction/customBenchmark/dictionarySourceId - RULESDBO.CUSTOM_INDEX_ATTRIBUTES.SOURCE_DICT_ID
EagleML/entityTransaction/customBenchmark/perfRollupReturnsId - RULESDBO.CUSTOM_INDEX_ATTR_DETAIL.PERF_ROLLUP_RETURNS_ID
EagleML/entityTransaction/customBenchmark/resetDateRule - contains Date Rule Name for the resolution on SP side

Files changed:
metadata/core/ontology/entity/custombenchmark-models.json
metadata/core/ontology/entity/custombenchmark.json
metadata/core/ontology/templates/eagle_ent_custombenchmark_insertupdate.txt
metadata/core/ontology/templates/db_oper/snowflake/custombenchmark.reinit_batch_many.sql
metadata/core/ontology/templates/eagle_ent_custombenchmark_reinitbatch.oracle.txt
metadata/core/ontology/definition/eagle-vocab-definition-shared.json
metadata/core/ontology/definition/eagle-vocab-definition-wh-shared.json
metadata/core/ontology/eagle-vocab-entity-shared.json
metadata/core/ontology/eagle-vocab-shared.json
metadata/core/ontology/eagle-vocab-wh-shared.json

SDP-76813 The GenericSMF load and extracts were enhanced to support the UK Taxable Flag

A new element ukTaxableFlag was added to the GenericSMF load and extract interfaces.

EagleML Xpath:
EagleML/referenceTransaction/genericSMF/SMExtension/ukTaxableFlag - SECURITYDBO.SECURITY_MASTER_EXT.UK_TAXABLE_FLAG - Tag 18499

Files changed:
eagle_default\in\xml\xslt\smf_egl_gen_interface_template.xsl
eagle_default\in\xml\include\smf_aliases.inc
eagle_default\in\xml\include\sm_extension.inc
eagle_ml-2-0_cm\out\xml\t_smf_streaming.inc
eagle_ml-2-0_cm\out\xml\t_smf_streaming_eml.inc

SDP-68903 The Metadata REST API was enhanced to improve the ontology query for extensions output

The Metadata ontology query output was enhanced for metadata-service-rest-api/getOntologyQuery endpoint.

• added main table information

  • table - table name

  • owner - schema name

  • key - key column, if PK is complex columns are divided with commas

Example of ontology query output enhancement for genericsmf:

"main_table": {
"table_name": "SECURITY_MASTER",
"owner": "SECURITYDBO",
"key": "SECURITY_ALIAS"
}

Production Bugs

SDP-80794 The element for the trade_demo2 element was replaced by tradeMemo2 in warehouseCloseLot interface

The warehouseCloseLot interface has been modified and the trade_demo2 element was replaced by the tradeMemo2 because the Trade_memo2 element did not meet EagleML XSD elements naming style. To provide backward compatibility, the warehouseCloseLot inbound interface has been modified to support both the tradeMemo2 and Trade_memo2. If both of these elements are passed in on the incoming message, then the tradeMemo2 element will have a higher priority.

Files changed:
msgcenter/eagle_ml-2-0_cm/out/xml/t_warehouse_close_lot.xml
msgcenter/eagle_ml-2-0_cm/out/xml/t_warehouse_closelot_streaming.inc
msgcenter/eagle_default/in/xml/xslt/wrh_egl_gen_interface3.xsl

SDP-80174 The warehousetrade resource and and corresponding dataset definitions were updated to support transactioncode filter

Added in the missing transactioncode filter to the warehousetrade resource.

MASTERTABLE.TRANS_CODE @(:transactioncode)

File changed:
metadata/core/ontology/warehouse/warehousetrade.json
metadata/core/datasets/warehousetrade.json