EagleML Release Notes - September 20th 2022

Enhancements

SDP-64729 Added support for WROFF event type.

The Accounting trades inbound interface was enhanced to support the WROFF event type.

Files changed:
eagle_default/in/include/accttrade_true_rebook.inc
eagle_default/in/include/csv-deliver.inc
eagle_default/in/xml/xml-acct_trades.rsf
eagle_default/in/xml/xml-accttrade_deliver.xml

SDP-65191 EDS IssuerESG Resource

EDS IssuerESG Resource was updated to support the ability to both load and extract to SECURITYDBO.ISSUER_ESG/ISSUER_ESG_HIST tables for both Oracle and Snowflake database providers.

Files changed:
metadata/core/ontology/reference/issueresg-in.json
metadata/core/ontology/templates/eagle_ref_issueresg_insertupdate.txt
metadata/core/ontology/reference/issueresg.json
metadata/core/ontology/reference/issueresg-models.json
metadata/core/ontology/definition/eagle-vocab-definition-issuer.json
metadata/core/ontology/eagle-vocab-issuer.json

SDP-65768 WRHSTRADEEXTRACT - Updated signage logic for CASH_TYPE field

Signage Keys and Conditions were adjusted for the TRADE_DETAIL.CASH_TYPE field.

CASH_TYPE - EagleML/warehouseTransaction/warehouseTrade/cashEventType

Affected transactionCodes: OPENFUTURE.
Affected recordType = 150, longShortIndicator = L.

File changed:
eagle_ml-2-0_cm/out/tagvalue/extract_wrhs_trade_signage_logic.inc

SDP-65633 NaicDesignation EagleML outbound interface supports streaming transformation

NaicDesignation EagleML outbound interface now supports streaming transformation. This provides the ability to transform raw data to the EagleML format quickly. Elements were also re-ordered accordingly to the proper EagleML XSD schema. Mapping was updated for the sourceName element. It used to be displayed as source instance whereas now it’s source name.

Files added:
eagle_ml-2-0_cm/out/xml/t_ref_naic_designation_streaming.inc
eagle_ml-2-0_cm/out/xml/t_ref_naic_designation_streaming.xml

Files changed:
eagle_ml-2-0_cm/w_config.inc
eagle_ml-2-0_cm/out/xml/t_eagleml-2-0_streaming.rsf
eagle_ml-2-0_cm/out/tagvalue/extract_naic_designation.xml

SDP-64996 EDS Security Resolution - added ability to read resolution options from w_config_custom.inc

Added the ability to read Asset Resolution Options and Batting Order from w_config_custom.inc in EDS.

If Asset Resolution Options and Batting Order are not specified in an incoming message, Option can be used from /estar/tpe/dynamic/msgcenter/eagle_ml-2-0_custom_cm/w_config_custom.inc file.

Files changed:

eagle_default/in/include/accttrade_true_rebook.inc
eagle_default/in/include/csv-deliver.inc
eagle_default/in/xml/xml-acct_trades.rsf
eagle_default/in/xml/xml-accttrade_deliver.xml

Options should be stored in EagleML rule format in the variables CONFIG_SEC_RES_OPTION and CONFIG_BATTING_ORDER.

Example:

<CODE>
CreateVariable( 'CONFIG_SEC_RES_OPTION','XREF_EXCHANGE,MATURITY_DATE,ASSET_CURRENCY');
CreateVariable( 'CONFIG_BATTING_ORDER','cusip,isin,sedol,reuters,bloombergId,sicovmId,valorenId,cedel,internalId,cinsId,ticker');
</CODE>

SDP-64662 EDS Schedule Resource

EDS Schedule Resource is available now for load and extract. Data load to SECURITYDBO.SCHEDULE table implemented for both database providers Oracle and Snowflake.

Files changed:
metadata/core/ontology/reference/schedule_in.json
metadata/core/ontology/templates/eagle_ref_schedule_insertupdate.txt
metadata/core/ontology/reference/schedule.json
metadata/core/ontology/reference/schedule-models.json

SDP-63574 EDS issuerrelationshipsummary resource is supported for Oracle and Snowflake database providers

New resource issuerrelationshipsummary supports extract from Oracle and Snowflake and upload to SECURITYDBO.ISSUER_RELATIONSHIP_SUMMARY Snowflake table. There is no Stored Procedure for Oracle load. Metadata vocabulary was updated to support new fields.

Files changed:
metadata/core/ontology/definition/eagle-vocab-definition-shared.json
metadata/core/ontology/eagle-vocab-common.json
metadata/core/ontology/eagle-vocab-enumerations.json
metadata/core/ontology/eagle-vocab-shared.json
metadata/core/ontology/reference/issuerrelationshipsummary-models.json
metadata/core/ontology/reference/issuerrelationshipsummary.json
metadata/core/ontology/reference/issuerrelationshipsummary_in.json

SDP-64718 DeltaBatch Mode was implemented for EDS WarehousePerformance and WarehousePosition resources

The DeltaBatch mode is not supported for WarehousePerformance and WarehousePosition resources. The DeltaBatch mode will extract all records for a given batch if one or more records in the batch were updated in Eagle.

The extracted data should then be loaded in Batch mode to Azure. To use DeltaBatch Extract, the following parameter should be used.

Usage of RTR parameter:

set usedeltabatch to Y

<taskParameter>
<name>usedeltabatch</name>
<value>Y</value>
</taskParameter>

or "updatemode": "batch"

Files changed:
ontology/eagledb-crud-filters.json
ontology/warehouse/warehouseperformance.json
common_core/dataset/ds_filters/delta_batch_filters/delta_batch_filter_cursor_handler.py

 

Production Bugs

SDP-65118 More than one extension table can be supported by the transformation rule for extension table extracts

The logic for generating *_streaming_init.inc and *_streaming_onend.inc was updated in order to support more than one extension table.

Files changed:
eagle_ml-2-0_cm/generate_extension_fields.inc

SDP-65405 Mashup logic for ontology based extracts was corrected

The Mashup logic for Data Rules Ontology based extracts was enhanced to support mapping from {alias}.SECURITY_ALIAS field into {alias}.securityAlias

SECURITY_ALIAS mapped into toCurrency/securityAlias
FROM_SECALIAS mapped into fromCurrency/securityAlias

Files changed:
eagle-mashups.json

SDP-65409 The CSV Encoding was enhanced for EDS codevalue extract

The encoding for EDS extracts in a CSV format was enhanced to support the EOL symbol generated as part of the extract file on load.

Files changes:
string_utils.py

SDP-65547 The EDS Issuer resolution was improved for load to Snowflake

The error raised for Issuer resolution on load to Snowflake was fixed for all resources below:

  • issuerrating

  • issuerrelationship

  • issuerrelationshipsummary

  • issuerrole

The Issuer resolution was corrected on EDS extract side and load side. Additional logic was implemented to support fromissuer and toissuer as part of Issuer resolution for load to Snowflake.

Files changed:
metadata/core/ontology/eagle-mashups.json
metadata/core/ontology/eagle-resolution-processing.json
metadata/core/ontology/reference/issuerrelationship_in.json
metadata/core/ontology/reference/issuerrelationshipsummary_in.json

SDP-65803 Default Logic for Warehouse Position and Warehouse Openlot was updated for load in updatemode Batch

EDS warehouseposition resource had the openLotDeleteSwitch flag set to Y which allowed to delete data from the HOLDINGDBO.LOT_LEVEL_POSIITON table during data upload in batch mode. This default behavior was corrected.
In order to resolve this issue, the openLotDeleteSwitch flag was set to N by default. Also, the Warehouse Openlot procedureFlag was also set to N by default for data upload in Batch mode to Snowflake. This change will prevent data being deleted from the HOLDINGDBO.POSITION_DETAIL table.

Files changed:
warehouseposition_in.json
warehouseopenlot_in.json

Â