EagleML Release Notes - December 14, 2023

Owned by Larry Ackerman
Last updated: Dec 14, 2023
3 min read

Enhancements

SDP-79873 EagleML updated with latest core changes for DELETE_UNDERLYING and INSERT_UPDATE_UNDERLYING_INFO

The EagleML rules were updated to remain in sync with the latest core changes for the DELETE_UNDERLYING and the INSERT_UPDATE_UNDERLYING_INFO stored procedures, where new parameters in_journal_flag and in_update_source were added.

Files changed:
metadata/bind/eagleml_ref_underlyingsmf.xml
eagle_default/in/xml/include/forwardsmf_und_link.inc
eagle_default/in/xml/include/generic_smf_fields_logic.inc
eagle_default/in/xml/include/underlying_link_ins_upd.inc
eagle_default/in/xml/t_eagle_ml-2-0_default_in_xml_delete_smf_mi_underlying.xml
eagle_default/in/xml/t_eagle_ml-2-0_default_in_xml_delete_smf_mi_underlying_floater.xml
eagle_default/in/xml/t_eagle_ml-2-0_default_in_xml_delete_underlying_smf.xml
eagle_default/in/xml/xml-smf_multileg_swap.xml
eagle_default/in/xml/xml-smf_underling.xml
eagle_default/in/xml/xml-smf_underling_batch.xml
eagle_default/in/xml/xslt/eagleml_smf_gen_fields.xsl
eis/ejmtsk/eagle_ml-2-0_default_in_xml_delete_objects/t_eagle_ml-2-0_default_in_xml_delete_smf_mi_underlying.ejmtr
eis/ejmtsk/eagle_ml-2-0_default_in_xml_delete_objects/t_eagle_ml-2-0_default_in_xml_delete_smf_mi_underlying_floater.ejmtr
eis/ejmtsk/eagle_ml-2-0_default_in_xml_delete_objects/t_eagle_ml-2-0_default_in_xml_delete_underlying_smf.ejmtr

SDP-79669 Rejection reprocessing through EDS - reprocessing for Eagle environments (Oracle) from ontology configured

As part of the framework of the EDS Rejection Reprocessing, work was done to generate in load processing rules to incorporate the following additional strings:

{noformat}""eagledb_log_process_id"": ""eagle_default_eds_warehouse"",
""eagledb_needs_record_logging"": true,{noformat}

Change affects Oracle (inMarket) only, no changes for cloud.

Files modified:
src/eagleinvsys/rdf_metadata/rdf_inflow.py

SDP-79203 New PRICINGRDCREPORTEXTRACT Available in MC2

A new outbound interface was added to the new RDCPricingReport EagleML object. This interface is based upon the original PRICINGREPORTEXTRACT built which is utilized for Pricing Center. The new PRICINGRDCREPORTEXTRACT is only available using MC2 and provides data for clients using RDC.

Feed type: {{PRICINGRDCREPORTEXTRACT}}

Sample RTR - Generic Extractor Workflow:

<EagleML xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance"" xsi:type=""RunTaskRequestSync"" eaglemlType=""RunTaskRequest"" eaglemlVersion=""2.0"">
	<header>
		<messageId/>
		<sentBy><http://www.client.com</sentBy>>
		<sendTo><http://www.eagleinvsys.com/eagle_ml-2-0_default_cm_wrkfl_listener%3C/sendTo%3E%3E 
		<replyTo>http://www.client.com/Eagle Control Messages</replyTo>
	</header>
	<taskIdentifier>
		<correlationId>PRICINGRDCREPORTEXTRACT_MC2_WF_{GUID}</correlationId>
		<businessTaskId>PRICINGRDCREPORTEXTRACT_MC2_WF</businessTaskId>
	</taskIdentifier>
	<taskTypeEnum>NEW</taskTypeEnum>
	<taskParameters>
		<taskParameter>
			<name>ActionType</name>
			<dataType>S</dataType>
			<value>EXECUTE</value>
		</taskParameter>
		<taskParameter>
			<name>StreamName</name>
			<dataType>S</dataType>
			<value>eagle_ml-2-0_default_cm_execwkfl</value>
		</taskParameter>
		<taskParameter>
			<name>WorkflowName</name>
			<dataType>S</dataType>
			<value>eagle_wrf_generic_extract</value>
		</taskParameter>
		<taskParameter>
			<name>FeedType</name>
			<dataType>S</dataType>
			<value>PRICINGRDCREPORTEXTRACT</value>
		</taskParameter>
		<taskParameter>
			<name>effectivedate</name>
			<dataType>S</dataType>
			<value>2023-06-28</value>
		</taskParameter>
		<taskParameter>
			<name>entityselectiontype</name>
			<dataType>S</dataType>
			<value>EntityID</value>
		</taskParameter>
		<taskParameter>
			<name>entityselectionvalue</name>
			<dataType>S</dataType>
			<value>L(FNDXYZ01%)</value>
		</taskParameter>
	</taskParameters>
</EagleML>

Sample RTR - Direct Request:

<EagleML xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance"" xsi:type=""RunTaskRequestSync"" eaglemlType=""RunTaskRequest"" eaglemlVersion=""2.0"">
	<header>
		<messageId>CC59ADD40E1E069</messageId>
		<sentBy><http://www.client.com</sentBy>>
		<sendTo><http://www.eagleinvsys.com/eagle_ml-2-0_default_out_extract_service%3C/sendTo%3E%3E 
		<replyTo>http://www.client.com/Eagle Control Messages</replyTo>
	</header>
	<taskIdentifier>
		<correlationId>PRICINGRDCREPORTEXTRACT_{GUID}</correlationId>
		<businessTaskId>PRICINGRDCREPORTEXTRACT</businessTaskId>
	</taskIdentifier>
	<taskTypeEnum>LOAD</taskTypeEnum>
	<taskParameters>
		<taskParameter>
			<name>ActionType</name>
			<dataType>S</dataType>
			<value>EXTRACT</value>
		</taskParameter>
		<taskParameter>
			<name>StreamName</name>
			<dataType>S</dataType>
			<value>eagle_ml-2-0_default_out_extract_service</value>
		</taskParameter>
		<taskParameter>
			<name>FeedType</name>
			<dataType>S</dataType>
			<value>PRICINGRDCREPORTEXTRACT</value>
		</taskParameter>
		<!--taskParameter>
			<name>primaryassetid</name>
			<dataType>S</dataType>
			<value>1234SMON2</value>
		</taskParameter-->
		<taskParameter>
			<name>effectivedate</name>
			<dataType>S</dataType>
			<value>2023-06-28</value>
		</taskParameter>
		<taskParameter>
			<name>entityselectiontype</name>
			<dataType>S</dataType>
			<value>EntityID</value>
		</taskParameter>
		<taskParameter>
			<name>entityselectionvalue</name>
			<dataType>S</dataType>
				<value>ENTIDV001,ENTIDV002</value>
		</taskParameter>
		<!--taskParameter>
			<name>usergroups</name>
			<dataType>S</dataType>
			<value>Sample Group</value>
		</taskParameter-->
		<!--taskParameter>
			<name>sourcename</name>
			<dataType>S</dataType>
			<value>STARDIRECT</value>
		</taskParameter-->
		<taskParameter>
			<name>maxrows</name>
			<dataType>S</dataType>
			<value>10000</value>
		</taskParameter>
		<!--IncludeZeroBalances -->
		<taskParameter>
			<name>IncludeZeroBalances</name>
			<dataType>S</dataType>
			<value>Y</value>
		</taskParameter>
		<!--ZeroBalanceSwitch -->
		<taskParameter>
			<name>ZeroBalanceSwitch</name>
			<dataType>S</dataType>
			<value>ShareParValue</value>
		</taskParameter>
		<!--UsePriceExchangeMarketDateSource -->
		<taskParameter>
			<name>UsePriceExchangeMarketDateSource</name>
			<dataType>S</dataType>
			<value>Y</value>
		</taskParameter>
	</taskParameters>
</EagleML>

Supported filters:
required filters: common entity filter or usergroups filter, effectivedate filter
optional filters: IncludeZeroBalances, ZeroBalanceSwitch, UsePriceExchangeMarketDateSource, sourcename, common SMF filters

Added Files:
eagle_ml-2-0_cm/out/tagvalue/extract_pricing_rdc_report.inc
eagle_ml-2-0_cm/out/xml/extract_service_acct_pricing_rdc_report.xml
eagle_ml-2-0_cm/out/xml/t_pricing_rdc_report_streaming.inc

Changed Files:
eagle/estar/EAGLE_EXTRACTSERVICE.pkg
eagle/estar/eagle_ml-2-0_default_out_extract_service.pkg
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/xml/extract_service.rsf
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/xml/extract_service_start.inc
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/xml/t_eagleml-2-0_out_extract_service.rsf
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_workflows/eagle_wrf_generic_extract.wrf
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_workflows/eagle_wrf_generic_extract_accounting_objects.wrf

SDP-76061 Primary Key and grouping for EDS custom resource

EDS generator for custom resources was enhanced:

  • Any vocabulary element can be marked as PK using {{is_primary_key_column}} property

  • Grouping for custom resources is supported by defining {{group_key_expr}} in interface

Files changed:
src/eagleinvsys/rdf_metadata/rdf_ontology_generator.py
src/eagleinvsys/rdf_metadata/rdf_ontology_generator_templates.py

SDP-79024 The warehousecashactivity resource was enhanced to support data load to Oracle

The warehousecashactivity EDS resource has been enhanced to allow load to Oracle using the core stored procedure CASH.EGL_GEN_CASH.CSA_EGL_GEN_INTERFACE.

Files changed:
metadata/core/ontology/warehouse/warehousecashactivity-models.json
metadata/core/ontology/warehouse/warehousecashactivity_in.json

Files added:
metadata/core/ontology/templates/eagle_wrhs_warehousecashactivity_insertupdate.txt

SDP-78953 Profile Templates for Conversion and Cloud Workflows as part of Product

Added profile templates for the Conversion and Cloud Workflows supported by the core product.

Profile templates EagleToVault-conversion-[resource]-[mode]_tmp.profile were created for every supported resource for the eagle_wrf_cloud_conversion workflow and added to the package.
Profile templates EagleToVault-cloud-[resource]-[mode]_tmp.profile were created for every supported resource for the eagle_wrf_cloud_extract workflow and added to the package.

Overall 186 profiles are part of the Data Rules product.

Files changed:
eagle/estar/EAGLE_EXTRACTSERVICE.pkg
eagle/estar/eagle_default_workflows.pkg

SDP-77841 EDS support for issueranalytic Resource

The EDS issueranalytic resource was created to support the Eagle To Vault pipeline to extract data from Eagle and upload to Snowflake.

Supported modes: Delta, Temporal
Table: SECURITYDBO.ISSUER_ANALYTICS

Supported Extract Filters:
sourcename
issuerid, issuername, issueralias
fromdate, todate,
effectivedate, fromeffectivedate, toeffectivedate
releasedate, fromreleasedate, toreleasedate

Files changed:
metadata/core/ontology/reference/issueranalytic.json
metadata/core/ontology/reference/issueranalytic-models.json
Files added:
metadata/core/ontology/reference/issueranalytic_in.json

SDP-80433 SrcSys is added to AccountingAssetCash load interface

The element srcSysC is added for the load process to the AccountingAssetCash interface for all supported asset cash types.

EagleML/accountingTransaction/accountingAssetCash/srcSysC - Tag 98

Files changed:
eagle_default\in\xml\xml-acct_cash_asset.xml
eagle_default\in\include\csv-cash_misc_inc_exp.inc
eagle_default\in\include\csv-cash_lot.inc
eagle_default\in\include\csv-cash_coupon.inc
eagle_default\in\include\csv-cash_div.inc
eagle_default\in\include\csv-cash_blinc.inc

SDP-68807 Entity Active Status check for Warehouse streams

A new global option, W_WRHS_CHECK_ENTITY_ACTIVE_STATUS, is added for eagle_ml-2-0_default_in_xml_warehouse and eagle_ml-2-0_default_in_xml_warehouse_preproc streams for all supported warehouse objects. It should be set in the w_config_custom.inc and is an optional parameter.

If the option is set to 'Y', then the active status of an entity will be checked during a processing where if the entity is not active (RULESDBO.ENTITY.ACTIVE_STATUS = N), then the process will be failed with the next error:

Error Tag: 1283;
Error Code: 1283;
Sample Error Text: Entity Id - ENT12345 is not able to load records into the Warehouse as the Entity is not in Active Status and is prohibited from being loaded as a result of a Global Configuration. The entity may be returned to Active Status and the data reloaded.
Exception Code in Automation Centers Exception: 1283.

If the entity's status is active (RULESDBO.ENTITY.ACTIVE_STATUS = Y) or the active status is empty, then the record will be processed without errors.

Files changed:
eagle_default\in\xml\include\xml-warehouse_common.inc

Production Bugs

SDP-80324 DEMANDLIST extract fixed to work correctly with panel profiles placed in both disk and database

Issue was in DemandListProfileName parameter which is used with DEMANDLIST extract. It works with pan-demandlist.htm profiles, and after general migration panel profiles in database in eagle core 17.43 version, extract still was looking for profiles on disk and that's why failed. Fix resolves that and now DemandListProfileName can work with both: profiles located in database for version ≥ .43 and located on disc profiledat/entry_public/eagle/star/reference/ for all earlier versions, based on system parameter 287.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_demandlist.inc

SDP-73673 MATCH_ALL security resolution for dually listed securities

Bug for MATCH_ALL security resolution was fixed. Query for resolution was modified to filter results by identifiers from batting order. Now it allows duplicate xreferences on any place in batting order. Query returns error only if more than one security was resolved by set of identifiers. Also, record counter was added in match_all_new_record_check.inc to consider if filter should be applied.

Files changed:
eagle_default/in/xml/include/match_all_new_record_check.inc
eagle_default/in/xml/include/match_all_sec_resolution.inc