EagleML Release Notes - July 2020

Owned by Olga Galis (Deactivated)
Last updated: Aug 14, 2020
7 min read

Enhancements:

SDP-37655: Pricing Report Extract logic was revised, to refine records returned

The PRICINGREPORTEXTRACT had been displaying records when the PRS_INSTANCE on the estar_portfolio_valuation table had value of 0 or null. There were instances when the PRICINGREPORTEXTRACT was returning duplicate records in the case of multiple price-validation rules on a single Target Source.
The extract logic was revised, and issues were corrected:

  • The filter by the PRS_INSTANCE > 0 was removed, and an additional key TARGET SOURCE INSTANCE + TARGET SOURCE FIELD is applied to join the records.

  • In order to filter out the unnecessary records, an additional flag was introduced to allow bringing back pricing records only when the price validation rule has an Active status. In most configurations, there will be only one active flag set with a value of Y.

  • The new flag is supported via w_config_custom.inc and via RTR parameter, the Default value is Y.

<CODE>
:W_PRICINGREPORTEXTRACT_ACTIVE_FLAG: = 'N';
</CODE>

<taskParameter>
<name>pricingactiveflag</name>
<dataType>S</dataType>
<value>N</value>
</taskParameter>

File changed:
eagle_ml-2-0_cm/out/tagvalue/extract_pricing_report.inc

SDP-38766: Pricing Report Extract logic was revised to exclude records not available in the Portfolio Valuation Table

The Pricing Report Extract, PRICINGREPORTEXTRACT, was improved to exclude records that have a target source instance of null in the Portfolio Valuation Table. This change is intended to eliminate records that do not have a linked Pricing Record from the Pricing Report. Any position not valued is expected to be captured using other system tools. The PRICINGREPORTEXRACT will return only those positions valued for accounting with applied prices.

File changed:
eagle_ml-2-0_cm/out/tagvalue/extract_pricing_report.inc

SDP-38632: New processing option UNIQUE_RECORD_PER_UNDERLYING_TYPE for GenericSMF

A new processing option, UNIQUE_RECORD_PER_UNDERLYING_TYPE, was introduced with the following logic while loading Underlying SMF:
There will be only one record, meaning that only one underlying SMF will be open for a combination of Main SMF + Underlying Type.
If there is an open record for the same combination of Main SMF and Underlying Type as that of the incoming record, the open record will now be closed with the end date = incoming begin date -1.
This option may be turned on via the global flag W_UNDERLYINGSMF_UNIQUE_RECORD_PER_TYPE_SWITCH set to Y or within the incoming file <processingOptions>UNIQUE_RECORD_PER_UNDERLYING_TYPE</processingOptions>.
This new option may be used independently or in combination with the W_UNDERLYINGSMF_DELETE_REINSERT_SWITCH.
If both the W_UNDERLYINGSMF_DELETE_REINSERT_SWITCH and W_UNDERLYINGSMF_UNIQUE_RECORD_PER_TYPE_SWITCH are set to Y, the expected behavior is as follows:

  1. The existing records with the same security alias (main security) + underlying alias (underlying security) as that of the incoming record are deleted before load.

  2. All open records with the same Main security and Underlying Type will be closed with the end date = incoming begin date -1.

  3. An error is thrown when the incoming begin date is less than the existing begin date in the database.

  4. A new record is stored in the database.

Usage:
<processingOptions>UNIQUE_RECORD_PER_UNDERLYING_TYPE</processingOptions>

Global settings via w_config_custom.ini
<COL TAG="W_UNDERLYINGSMF_UNIQUE_RECORD_PER_TYPE_SWITCH" EXPRESSION="'Y'"/>
or
<COL TAG="W_UNDERLYINGSMF_UNIQUE_RECORD_PER_TYPE_SWITCH" EXPRESSION="'Y'"/>
<COL TAG="W_UNDERLYINGSMF_DELETE_REINSERT_SWITCH" EXPRESSION="'Y'"/>

Files changed:

eagle_default/in/xml/include/xml-smf_underling_common.inc
eagle_default/in/xml/xml-smf_multileg_swap.xml

SDP-38591: SPDDELTATRIGGERLOGEXTRACT added support for the minimum accounting date logic from the Post Date

The SPDDELTATRIGGERLOGEXTRACT logic was adjusted to enhance the minAccountingDateFlag filter to select the minimum accounting date from the post_date value if the minAccountingDateFlag is set to P. The valid values are now P for Post Date and Y for Accounting Date.
For Example:
<taskParameter>
<name>minAccountingDateFlag</name>
<dataType>S</dataType>
<value>P</value>
</taskParameter>

File changed:
eagle_ml-2-0_cm/out/tagvalue/extract_spd_delta_trigger_log.inc

SDP-38556: New Inbound interface EntityToEntity

Added support of the new object type EntityToEntity, to allow load of Account Xreferences into the RULESDBO.ENTITY_DETAIL.
The inbound interface was introduced as part of the entity load process.

Files changed:
eagle_default/in/xml/xml-ent_objects.rsf

New files:
eagle_default/in/xml/ xml-ent_entitytoentity.xml

SDP-38549: ENTITY COMPOSITE LIST EXTRACT was updated to support Effective Date under entityDetail node

The XSD schema and ENTITYCOMPOSITELISTEXTRACT were updated to support the Effective Date under the entityDetail node.
The default generic extract workflow was also updated to support the task parameter historyonlyflag for the ENTITYCOMPOSITELISTEXTRACT.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_entity_composite_list.inc
eagle_ml-2-0_cm/out/xml/t_entity_composite_list_streaming.inc
eagle_ml-2-0_workflows/eagle_wrf_generic_extract_entity_objects.wrf

SDP-38329: RTR parameter LocalFileFolder works with the Data parameter, which contains base64 encrypted data embedded in the control message

RTR reader allows to download Data parameter, which contains base64 encrypted data from the RTR to the custom local directory. It is supported only for a single EJM task.

Example of RTR parameters:
<taskParameter>
<name>Data</name>
<value>H4sIAJRt/F4AA+/8gTt90ocgAoAAA=</value>
</taskParameter>
<taskParameter>
<name>WorkflowData</name>
<value>EML_EAGLE_ML_GENERICSMF_SMFCNGDT01.tar.gz.base64</value>
</taskParameter>
<taskParameter>
<name>LocalFileFolder</name>
<value>in/RAW/eagle_ml-2-0_default_cm_ndfa/incoming/
</value>
</taskParameter>

Files changed
eagle_ml-2-0_cm/check_loc_file_folder.inc
eagle_ml-2-0_cm/file_base64_encode.inc
eagle_ml-2-0_cm/get_workflow_data.inc
eagle_ml-2-0_cm/in/xml/runtaskrequest.xml
eagle_ml-2-0_cm/mc2/ejm/workflow/workflow_listener.xml

SDP-38240: Enhanced logic to resolve the Voluntary Corporate Action by using the clientSpecificId

Added logic in the Voluntary CA Election to resolve the original corporate action with the clientSpecificId.

File changed:
eagle_default/in/xml/xml-ref_voluntarycaelections.xml

SDP-38232: Warehouse Trade Outbound Interface was updated to extract additional fields via the IncludeExtendedDetails filter

The Extended CA Properties Model for Corporate Actions was updated to support additional elements for the WRHSTRADEEXTRACT:

EagleML/warehouseTransaction/WarehouseTrade/cpShareTaxability
EagleML/warehouseTransaction/WarehouseTrade/cpCashTaxability
EagleML/warehouseTransaction/WarehouseTrade/cpCash
EagleML/warehouseTransaction/WarehouseTrade/cpCallPx
EagleML/warehouseTransaction/WarehouseTrade/fromFmvPrice
EagleML/warehouseTransaction/WarehouseTrade/toFmvPrice
EagleML/warehouseTransaction/WarehouseTrade/cpNewPx
EagleML/warehouseTransaction/WarehouseTrade/consentRt
EagleML/warehouseTransaction/WarehouseTrade/userGroup1
EagleML/warehouseTransaction/WarehouseTrade/userGroup3
EagleML/warehouseTransaction/WarehouseTrade/userGroup4
EagleML/warehouseTransaction/WarehouseTrade/userGroup5

The additional Corporate Action fields are included in the extract file via the IncludeExtendedDetails filter when the CA value is supplied and the transaction is generated as part of a corporate action:

<taskParameter>
<name>IncludeExtendedDetails</name>
<dataType>S</dataType>
<value>CA,ALL</value>
</taskParameter>

If the extract is executed via the generic_extract_workflow, the prefix with feedtype name is required.
Also, the ALLEXTRACTS_ prefix may be used.

For example:
<taskParameter>
<name>WRHSTRADEEXTRACT_IncludeExtendedDetails</name>
<dataType>S</dataType>
<value>CA,ALL</value>
</taskParameter>

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_ca_properties_fields.xml
eagle_ml-2-0_cm/out/xml/t_ca_properties_fields_streaming.xml

SDP-38214: WarehouseTrade extract revised to correspond XSD schema

Elements that do not belong to the WarehouseTrade object were replaced with elements according to the XSD schema:
closeAmortizedLocal replaced by clsAmortLocal
closeAmortizedBased replaced by clsAmortBase

File changed:
eagle_ml-2-0_cm/out/xml/t_warehouse_trade_streaming.inc

SDP-38076: Client Inbound Interface was enhanced via the Generic Load Workflow

The Generic Load workflow, eagle_wrf_generic_load, was updated to support an additional sub-process with a file mask, CLIENTRELATIONSHIP, and to allow sequential inserts and updates of Client Relationship and Client Communication objects.

Files changed:
eagle_wrf_generic_load.wrf
eagle_wrf_generic_load_client_objects.wrf

SDP-37983: Mapping for UserGroup Fields was Added to Generic CA Inbound Interface

Added additional elements to the GenericCA interface:
EagleML/ReferenceTransaction/genericCA/userGroup1 userGroup1
EagleML/ReferenceTransaction/genericCA/userGroup3 userGroup3
EagleML/ReferenceTransaction/genericCA/userGroup4 userGroup4
EagleML/ReferenceTransaction/genericCA/userGroup5 userGroup5
to allow load into the TRADESDBO.CORP_ACT_HIST database to support Corporate Actions insert, update and rebook.

Files changed:
xml-ref_ca.xml

SDP-37955: GenericSMF Extract was enhanced to support a primary asset id type override

The GenericSMF extract supports a new RTR parameter, idtypeoverride. It may be passed directly to the SMFEXTRACT or to the eagle_wrf_generic_extract workflow with a feedtype prefix, SMFEXTRACT_idtypeoverride. The parameter contains a string which will be used as the alternative value in case the database returns a null value for the primary asset id type. In the case where the parameter was not passed or the database SMF record has an incorrect value, the primary asset id type will not be overritten.

The override occurs in the SMFEXTRACT query step/eagle_ml-2-0_default_out_q stream. When the parameter is passed, the database query will be expanded with the NVL statement. In case of a NULL value, it will be overwritten to the value from the idtypeoverride parameter.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_asset_identifiers_fields.inc

SDP-37904: Adjusted minimum accounting-date logic for the SPD Delta Trigger Log Extract

The SPDDELTATRIGGERLOGEXTRACT logic was revised to allow the minAccountingDateFlag filter to pull back the earliest accounting date record per entity, regardless of message_subtype.

File changed:
eagle_ml-2-0_cm/out/tagvalue/extract_spd_delta_trigger_log.inc

SDP-37862: Traded Cash Activity and Settled Cash Activity Outbound Interfaces were updated to extract additional fields via IncludeExtendedDetails filter

The Extended CA Properties Model for Corporate Actions was updated to support additional elements for the SETTLEDCAEXTRACT and TRADEDCAEXTRACT:

EagleML/accountingTransaction/TradedCashActivity(SettledCashActivity)/cpShareTaxability
EagleML/accountingTransaction/TradedCashActivity/cpCashTaxability
EagleML/accountingTransaction/TradedCashActivity/cpCash
EagleML/accountingTransaction/TradedCashActivity/cpCallPx
EagleML/accountingTransaction/TradedCashActivity/FmvPrice
EagleML/accountingTransaction/TradedCashActivity/toFmvPrice
EagleML/accountingTransaction/TradedCashActivity/cpNewPx
EagleML/accountingTransaction/TradedCashActivity/consentRt
EagleML/accountingTransaction/TradedCashActivity/userGroup1
EagleML/accountingTransaction/TradedCashActivity/userGroup3
EagleML/accountingTransaction/TradedCashActivity/userGroup4
EagleML/accountingTransaction/TradedCashActivity/userGroup5

The additional Corporate Action fields are included in the extract file via the IncludeExtendedDetails filter when the CA value is supplied and the transaction is generated as part of a corporate action:

<taskParameter>
<name>IncludeExtendedDetails</name>
<dataType>S</dataType>
<value>CA,ALL</value>
</taskParameter>

If extract is executed via generic_extract_workflow, a prefix with feedtype name is required.
Also, ALLEXTRACTS_ prefix can be specified.
For example:
<taskParameter>
<name>SETTLEDCAEXTRACT_IncludeExtendedDetails</name> or <name>TRADEDCAEXTRACT_IncludeExtendedDetails</name>
<dataType>S</dataType>
<value>CA,ALL</value>
</taskParameter>

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_ca_properties_fields.xml
eagle_ml-2-0_cm/out/xml/t_ca_properties_fields_streaming.xml

SDP-37700: Warehouse Performance Inbound Interface was Updated to support the Analytics Tables on Load

The inbound interface of the Warehouse Performance was updated to support the data load to analytics tables: PERFORMDBO.PSR_ANALYTICS and PERFORMDBO.PERF_SEC_RETURNS_EXT.
Multiple partitions are supported.

Files changed:
generate_sp_on_fly.inc
xml-warehouse_common.inc
eagleml_wrhs_performance_ext.xml
eagleml_warehouse.xml

SDP-37502: Warehouse Trade Quotes inbound Interface was updated to work with the core stored procedure

The Warehouse Trade Quotes inbound interface was enhanced to utilize the core stored procedure from the EGL_GEN_TRADE package. By default, the non-core TRADE_QUOTE_UPLOAD procedure is still supported for backward compatibility.

File changed:
eagle_default/in/xml/xml-wh_brokertradequote_v2.xml

SDP-37450: MultilegSMF load processing without panel validations

The MultilegSMF Interface was enhanced to support the data-load without accounting validations in the panel. Setting noLoadValidationFlag to Y allows data-insert and update directly via SP INSERT_UPDATE_SECURITY_INFO without any calls to the panel.
Underlying links for the Pay and Receive legs are created with a begin date which corresponds to the effective date for the Contract leg in the incoming file. In the case of a MultilegSMF update, the underlying links are updated only if the new record has an effective date higher than the previously loaded begin dates on the Pay and Receive legs for the corresponding Contract leg.

Files changed:
eagle_default/in/xml/xml-smf_multileg_swap.xml

SDP-36627: Rebook functionality was added for Accounting Trades DEL/REC

The Rebook functionality was added for the Accounting Trades Deliver and Receive types.
To enable the Rebook option, the allowAutoRebookFlag set to Y or global parameter W_ACCTTRADE_ALLOW_AUTOREBOOK set to Y should be used.

Files changed:
eagle_default/in/include/accttrade_true_rebook.inc
eagle_default/in/xml/xml-accttrade_deliver.xml
eagle_default/in/xml/xml-accttrade_receive.xml

SDP-36517: Added the ability to calculate netAmoundBase and localToBaseFXRate values for the Accounting Asset Cash Types AMISCEXP and AMISCINC

A new element, calculateBaseAmountsFlag, was added to the XSD and inbound interface for Accounting Asset Cash Types AMISCEXP and AMISCINC.
The new flag calculateBaseAmountsFlag activates the following logic when set to Y within the incoming message:

  • The localToBaseFXRate is calculated if not available from the incoming message else it is taken from the incoming message

  • The netAmoundBase is always calculated

Calculations logic:

The First Base Currency and Local Currency values are resolved.
Base Currency is resolved from the Entity record, database field RULESDBO.ENTITY.BASE_CURRENCY.
Local Currency is equal to the settlementCurrency from the incoming message or if the settlementCurrency is null, then it is resolved from the Security record, database field SECURITYDBO.SECURITY_MASTER.CURRENCY_CODE.

Case 1: Base Currency = Local Currency
localToBaseFXRate = 1
netAmoundBase = value from the incoming element netAmountLocal

Case 2: Base Currency != Local Currency
localToBaseFXRate = value calculated in lookup of Fx Rates.
netAmoundBase = value from the incoming element netAmountLocal / calculated localToBaseFXRate

Files changed:
eagle_default/in/xml/xml-acct_cash_asset.xml
eagle_default/in/include/csv-cash_misc_inc_exp.inc

SDP-34149: CorrelationIdPrefix functionality was improved for all sub-processes as part of restart workflows

The CorrelationIdPrefix was not correctly passed to all sub-processes as part of the restart workflows.
The issue was corrected for all the sub-processes as part of restart workflows and as part of the restart schedule streams.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/workflow_restart.xml

SDP-27495: Calendar Extract supports Nuggetizer

The REFCALENDAREXTRACT via Classic MC was updated to support nuggetizer functionality.

Example of usage:
<taskParameter>
<name>MicroBatchKeyRecordCount/name>
<dataType>S</dataType>
<value>50</value></taskParameter>
<taskParameter>

File changed:
eagle_ml-2-0_cm/out/xml/t_eagleml-2-0_streaming.rsf
eagle_ml-2-0_cm/w_config.inc

SDP-26361: Profile inheritance functionality was implemented for workflow restart rule

If the profile contains the includeProfiles parameter, then all parameters of the parent profile are taken into account on the workflow restart.
This option allows for sharing common parameters between profiles. It is important for profile maintenance and simplifies the configuration of the similar processes.

Files changed:
eagle_ml-2-0_cm\get_correlation_id_prefix.inc
eagle_ml-2-0_cm\read_profile.inc
eagle_ml-2-0_cm\out\tagvalue\workflow_restart.xml

Production bugs:

SDP-38866: SKIPCANCEL functionality was corrected for scheduled workflows

The Task Parameter SKIPCANCEL was duplicated in the RTR for scheduled workflows if passed from the profile. The parameter is case sensitive and is effective only in the upper case.
The issue was corrected.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/workflow_restart.xml

SDP-38163: Corrected the Last Run Date logic for DeltaBased Extracts

The last run date was being calculated more than once, via the extract_delta_based.inc and the extract_save_and_execute_query.inc. The last value was stored in the database which created the time gap. The issue was fixed, now the last run time is calculated only once when the extract_delta_based.inc is called.

File changed:
/eagle_ml-2-0_cm/out/tagvalue/extract_save_and_execute_query.inc

SDP-37687: The Accounting Trade Interface was updated to correctly load the exchangeCode from the data file

The AccountingTrade interface did not load the Exchange Code due to an incorrect alias. The mapping was corrected to resolve this issue.

Files changed:
eagle/estar/tpe/dynamic/msgcenter/eagle_default/in/include/csv-buy_columns.inc
eagle_default/in/include/csv-buy_panels.inc
eagle_default/in/include/csv-cancel_columns.inc
eagle_default/in/include/csv-cancel_panels.inc
eagle_default/in/include/csv-capfundinvestments_columns.inc
eagle_default/in/include/csv-capfundinvestments_panels.inc
eagle_default/in/include/csv-conversion_columns.inc
eagle_default/in/include/csv-deliver.inc
eagle_default/in/include/csv-fxfwd_columns.inc
eagle_default/in/include/csv-fxspot.inc
eagle_default/in/include/csv-receive.inc
eagle_default/in/include/csv-sell_columns.inc
eagle_default/in/include/csv-sell_panels.inc
eagle_default/in/include/trade_aliases.inc
eagle_default/in/xml/xml-accttrade_conversion.xml
eagle_default/in/xml/xml-accttrade_deliver.xml
eagle_default/in/xml/xml-accttrade_fxfwd.xml
eagle_default/in/xml/xml-accttrade_fxspot.xml
eagle_default/in/xml/xml-accttrade_multipleleg.xml
eagle_default/in/xml/xml-accttrade_receive.xml

SDP-38961: XSD element <reason> was updated to optional in the XSD schema

The element <reason> was updated from required to optional in the Record complex type in the XSD schema. TSR file is valid by the XSD in case of success.

File changed:
eagleml/schema/eagleml-shared-2-0.xsd