EagleML Release Notes - June 2020 (First)

Owned by Olga Galis (Deactivated)
Last updated: Aug 13, 2020
6 min read

Enhancements:

SDP-37014: Warehouse delete process in batch mode was improved

To improve the performance of the Warehouse delete process run in batch mode, the logic of retrieving the list of Position Analytics and Performance Extension tables was improved. The request is now triggered only once during the deployment of AllServices and uses the correct list of database tables.

Files changed:
eagle_default/in/xml/include/xml-dbdirect_olot_pos_delete_statement.inc
eagle_default/in/xml/include/xml-dbdirect_wrhs_delete_statement.inc
eagle_ml-2-0_cm/in/xml/w_setup_eagle_version.xml
eagle_ml-2-0_cm/w_config.inc

SDP-36644: Additional buySellIndicator routing types supported for Accounting Trades

Added additional support for buySellIndicators to support additional Accounting Trades beyond CONVERSION, BUY, SELL, SHORTSELL and BUYCVR. The following values will now direct the trades to the proper processing: WRITE, CDSOPEN, TBAOPEN, TBACLOSE, OPENSWAP, and CLOSESWAP.

File changed:
eagle_default/in/xml/xml-acct_trades.rsf

SDP-36638: Multiple extracts were updated to support Nuggetizer functionality via Extract Service

The main purpose of the Nuggetizer Workflow is to create nuggets and send them for further processing. The process is intended to increase process performance.

The following interfaces were updated to support Nuggetizer functionality via Extract Service:

  • CLIENTCOMMEXTRACT

  • CONTROLCENTEREVENTLOGEXTRACT

  • EAGLEADMINISTRATIONSETTINGSEXTRACT

  • ENTITYANALYTICEXTRACT

  • STATICENTITYLISTEXTRACT

  • ENTITYPEERGROUPRANKINGEXTRACT

  • ENTITYRATINGEXTRACT

  • ENTITYREGULATORYEXTRACT

  • ENTITYSTATISTICEXTRACT

  • RULEMONTHENDDATEEXTRACT

Example of usage:

<taskParameter>
<name>MicroBatchKeyRecordCount</name>
<dataType>S</dataType>
<value>50</value></taskParameter>

or
<taskParameter>
<name>RecordCountPerNugget</name>
<dataType>S</dataType>
<value>50</value>
</taskParameter>

The MicroBatchKeyRecordCount has a higher priority than the old parameter and is used to specify the maximum number of records for each extract file.

Files changed:
eagle_ml-2-0_cm/out/xml/extract_service_clientcommunication.xml
eagle_ml-2-0_cm/out/xml/extract_service_control_center_event_log.xml
eagle_ml-2-0_cm/out/xml/extract_service_eagle_administration_settings.xml
eagle_ml-2-0_cm/out/xml/extract_service_entity_analytic.xml
eagle_ml-2-0_cm/out/xml/extract_service_entity_list_list.xml
eagle_ml-2-0_cm/out/xml/extract_service_entity_peer_group_ranking.xml
eagle_ml-2-0_cm/out/xml/extract_service_entity_rating.xml
eagle_ml-2-0_cm/out/xml/extract_service_entity_regulatory.xml
eagle_ml-2-0_cm/out/xml/extract_service_entity_statistics.xml
eagle_ml-2-0_cm/out/xml/extract_service_rule_month_end_date.xml

SDP-36565: Generic Entity Update logic was improved to allow update of Accounting Basis elements only

The Generic Entity update logic was improved to allow update of elements on the accounting basis table only as the process impacts both the entity and the accounting basis table.

Files changed:
eagle_default/in/include/entity_panels.inc

SDP-36507: Corporate Action Voluntary Elections updated with new fields and inner calculation logic

The following new fields were added for Corporate Action Voluntary Elections:

prorataCreditNoncreditLossIndicator (tag16157)
volRestrictProc (tag16658)
expiryDate (tag12876)
expiryDateFlag (tag12877)
reorgFeeBase (tag 9707)
reorgFee (tag 9706)
babyBondPrice (tag 9613)

The reorgFeeBase may be used in place of the internalFloat3 element.
The reorgFee may be used in place of the internalFloat2 element.
The babyBondPrice may be used in place of the internalFloat1 element.

Logic for resolution of the batchEventId (tag 701) and the transtype (tag 1257) were also improved to support loads when the accountingValidationFlag is set to Y.

Files changed:
eagle_default/in/xml/xml-ref_voluntarycaelections.xml

SDP-36439: Corrected Workflow Status from Pending to Failed on Cancel

The request to cancel/stop all workflow instances was enhanced. Pending files cancelled from the load queue will now have a status of Failed now.

Files changed:
eagle_ml-2-0_cm/cancel_global_event_wait.inc
eagle_ml-2-0_cm/cancel_workflow.inc

SDP-36379: Added support for multiple sets of credentials for SFTP/FTP protocols (GET Only)

A new option was added to support multiple sets of SFTP/FTP credentials specified in the w_config_custom.ini providing the ability to call specific credentials from a workflow.

A new global configuration parameter “W_CREDENTINALS_ID” should be set to 1,2, 3 etc to enable alternative settings or the new RTR parameter “CredentialsId” may be used to invoke specific set of SFTP/FTP credentials from a workflow. The supported values are 1,2,3 etc.

For example:
<taskParameter>
<name>CredentinalsId</name>
<value>1</value>
</taskParameter>

Custom config parameters for FTP delivery method:

  • FTP_URL1 - required

  • FTP_USER1 - required

  • FTP_PWD1 - required

  • FTP_PWD_ISENCRYPTED1

  • FTP_FILES_LOCATION1- required

Custom config parameters for SFTP delivery method:

  • SFTP_URL1 - required

  • SFTP_USER1 - required

  • SFTP_PWD1 - required

  • SFTP_PWD_ISENCRYPTED1

  • SFTP_FILES_LOCATION1- required

  • SFTP_INSECURE1

Files changed:
eagle_ml-2-0_cm/collect_task_parameters.inc
eagle_ml-2-0_cm/get_workflow_data.inc
eagle_ml-2-0_cm/mc2/ejm/workflow/workflow_listener.xml

SDP-36345: ReferenceElementOverride inbound interface was updated to support additionalInfo element

The ReferenceElementOverride was updated to allow data insert into rulesdbo.rdc_overrides.addl_info
A new element additionalInfo (tag 2761) is supported only on the Insert.

Files changed:
xml-ref_elementoverride.xml

SDP-36297: DeltaBased filter was enhanced for Warehouse Trade Extract

By default, the WRHSTRADEEXTRACT returns all records within a Batch (trades with same Batch Key: Entity+Source+Effective Date+Security), even if only one trade from the Batch was updated since the prior run. An additional DeltaBased global parameter was introduced, in order for WRHSTRADEEXTRACT to return transactions loaded only after the prior run.

<CODE>:W_TRADE_DELTA_BASED_FILTER_MODE: := 'WAREHOUSE';</CODE>

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_warehouse_trade.inc
eagle_ml-2-0_custom_cm/w_config_custom_example.inc

SDP-36225: The Generic Issue Analytic Interface was updated to support additional fields

The IssueAnalytic.model was updated to support two additional fields, as part of the Generic Issue Analytic Interface. The following new fields were added to the XSD, inbound and outbound rules:
futEquivModifiedDurationCTD - FUT_EQV_MOD_DUR_CTD
daysFromTodayToNotionalMat - DAYS_TDY_TO_NOTNL_MTY

Files changed:
eagle_default\in\xml\xml-ref_genericissueanalytic.xml
metadata\bind\eagleml_ref_genericissueanalytic.xml
eagle_ml-2-0_cm\out\xml\t_generic_issue_analytic_streaming.xml

SDP-36172: Added a new processing option to exclude Issuer roleType from batch processing via the SMF MI interface

A new processing option was added for the IssuerRole via the SMF MI interface. ISSUER_ROLE_PROCESSING_TYPE_E(ROLETYPE)_PROCESSING_TYPE to exclude a particular roleType from batch processing. The excluded roleType will be loaded outside of batch processing, as an independent record.

For example:
<processingOptions>ISSUER_ROLE_GROUPING,ISSUER_ROLE_PROCESSING_TYPE_E(ISSUER)_PROCESSING_TYPE</processingOptions>

Files changed:
eagle_default/in/xml/xml-ref_issuerrole_batch.xml

SDP-36160: SQL statement joining to the ESTAR.ESTAR_INCOME_ACTIVITY in the Warehouse OpenLot Extract was improved

When the RTR parameter IncludeAccrualRate was set to Y, the Warehouse OpenLot Extract triggered the sub-query to the ESTAR.ESTAR_INCOME_ACTIVITY to find the last instance in the table. The query was executing very slowly. As a result, the SQL statement was corrected to improve performance. Also, the Entity filter is now supported by the corresponding sub-query to limit the extracted data.

Files changed:
extract_warehouse_openlot.inc

SDP-36155: GenericSMF Interface was updated to support a missing field as part of Fixed income Model

The GenericSMF Interface was updated to support a missing field as part of the Fixed income Model: SECURITYDBO.FIXED_INCOME_DETAIL.FID_CHAR36. The EagleML XSD, and corresponding inbound and outbound rules were updated to include the fiDetChar36 within the Fixed Income Model.

EagleML/ReferenceTransaction/genericSMF/FixedIncomeModel/fiDetChar36
<FixedIncomeModel>
<fiDetChar36>fid</fiDetChar36>
</FixedIncomeModel>

Files changed:
eagleml-sec-shared-2-0.xsd
smf_egl_gen_interface_template.xsl
smf_egl_gen_interface_template_mssql.xsl
fixedincomesmf_model.inc
t_smf_streaming_eml.inc

SDP-36079: PreserveRemoteFileName functionality was enhanced to allow to retain original file name as part of businessTaskId for child ACK/TSR

Previously, the PreserveRemoteFileName option was added to keep the original file name in the final TSR. The PreserveRemoteFileName functionality was improved to also keep the original file name as part of businessTaskId for the child ACK and child TSR.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/execute_wrkfl_event.xml
eagle_ml-2-0_cm/out/xml/task_reporter_gen_step_summary.inc

SDP-36078: Issuer ESG resolution logic was updated

The Issuer ESG automatic issuer resolution logic was updated. The security is resolved based on the xref identifiers and then uses the role type from incoming message to find the issuer id from the table securitydbo.issuer_role_details where the security_alias = resolved security_alias, role_type = role type from incoming file, and the end_date is null.
The percent owned, which was earlier used in the issuer resolution logic, was removed from the SQL query. An error is produced when more than one record is returned during the SQL execution.

Files changed:
eagle_default/in/xml/xml-ref_issuer_esg.xml

SDP-36029: Corrected mapping inconsistency for GenericEntity Outbound Interface between Accounting and Generic processing mode

The GenericEntity Interface was working incorrectly in cases where the Accounting Validation Flag was set to N, due to a wrong mapping.
The issue was corrected by correcting the mapping as part of the outbound interface:
DB field STAR_ACCT_FIELD_9 to EagleML element acctBasis/netFuturesPositions
DB field STAR_ACCT_FIELD_11 to EagleML element acctBasis/futuresClearingBrokerCode
DB field STAR_ACCT_FIELD_12 to EagleML element acctBasis/swapFeeAccrual
DB field STAR_ACCT_FIELD_14 to EagleML element acctBasis/couponSpreadingPeriod
DB field STAR_ACCT_FIELD_16 to EagleML element acctBasis/netOptionPositions
DB field STAR_ACCT_FIELD_18 to EagleML element acctBasis/futureCostLedgerMethod
DB field STAR_ACCT_FIELD_25 to EagleML element acctBasis/entityEngineNumber
DB field STAR_ACCT_FIELD_29 to EagleML element acctBasis/cashDividendDistribution

Files changed:
eagle_ml-2-0_cm/out/xml/t_entity_streaming.xml
eagle_ml-2-0_cm/out/xml/t_entity_streaming_eml.xml

SDP-35743: Added support of Real Estate SMF Model via Generic SMF Interface

Multiple SECURITYDBO models were enhanced as part of the XSD, including the SECURITYDBO.REAL_ESTATE.model and corresponding GenericSMF inbound and outbound rules to support new fields:

UNDERLYING_TIC_AND_EXCH
IS_FLOAT_TO_FIX
IS_INDEX_LINKED
IS_FIX_TO_FLOAT
MTG_DEAL_CALL_DT
ID_BB_SEC_NUM_DES
IS_CASH_SETTLED
PROPERTY_SUB_TYPE
METRO
MSA
PROJECT_TYPE
YEAR_BUILT
RISK_PROFILE
LIFECYCLE
RENOVATED
LOAN_TERM_IN_MONTHS_OR_YEARS
RESET_LOCKOUT_PERIOD

Sample Elements:

FixedIncomeModel>
<underlyingTicAndExch></underlyingTicAndExch>
<isFloatToFix></isFloatToFix>
<isIndexLinked></isIndexLinked>
<isFixToFloat></isFixToFloat>
<isCashSettled>AS</isCashSettled>
<resetLockoutPeriod></resetLockoutPeriod>
</FixedIncomeModel>
<MBSModel>
<mtgDealCallDate></mtgDealCallDate>
</MBSModel>
<mtgLoanModel>
<loanTermInMonthsOrYear></loanTermInMonthsOrYear>
</mtgLoanModel>
<realEstateModel>
<propertySubType></propertySubType>
<metro></metro>
<msa></msa>
<projectType></projectType>
<yearBuilt></yearBuilt>
<riskProfile></riskProfile>
<lifecycle></lifecycle>
<renovated></renovated>
</realEstateModel>
<SMExtension>
<idBbSecNumDes></idBbSecNumDes>
</SMExtension>

Files changed:
eagleml-sec-shared-2-0.xsd
eagle_default/in/xml/xslt/smf_egl_gen_interface_template.xsl
eagle_default/in/xml/xslt/smf_egl_gen_interface_template_mssql.xsl
eagle_ml-2-0_cm/out/tagvalue/extract_smf.inc
eagle_ml-2-0_cm/out/xml/t_smf_streaming.inc
eagle_ml-2-0_cm/out/xml/t_smf_streaming_eml.inc

SDP-35566: Resolved performance issues for the SMFEXTRACT in History Mode

A new Temp table was added for the processing of Underlying Security data in the Held filter.
The previous version was creating a sub-query with a date filter to restrict Underlying Security by the EffectiveDate returned by Held filter.
This sub-query was working correctly, but the date filter impacting throughput.
In the current version, the data of Underlying Security table filtered by the EffectiveDate will be saved into the Temp table inside the Held filter.
The result query of the SMF Extract uses a new Temp table to filter Underlying Security records.

Files changed:
eagle_ml-2-0_cm/extract_filter_held.inc
eagle_ml-2-0_cm/out/tagvalue/extract_smf.inc

SDP-34965: Enabled data load into the HOLDINGDBO.REAL_ESTATE_POSITION_DETAIL table via Warehouse Position Inbound Interface

The Warehouse Position Inbound Interface was enhanced to support additional realEstateUDF elements and the possibility of loading data into the HOLDINGDBO.REAL_ESTATE_POSITION_DETAIL table via the core stored procedure POS_EGL_GEN_INTERFACE. The ability to Insert, Update, Delete, and Extract are supported.
The Warehouse Position Outbound Interface supports realEstateUDF elements as well.

File changed:
eagle_default\in\xml\xslt\wrh_egl_gen_interface3.xsl

SDP-31034: RTR reader supports download of the WorkflowData file from the SFTP/FTP server to the custom local directory

RTR reader functionality was improved to enable the download of the WorkflowData file from the SFTP/FTP server to the custom local directory.
It is supported only for the SFTP/FTP delivery method and a single EJM task.
A new RTR parameter was introduced, LocalFileFolder, to support custom local directory, where the value is a relative path to the data/msgcenter folder.

Example of RTR parameters:

<taskParameter><name>WorkflowData</name><value>FILE_NAME.zip</value></taskParameter>
<taskParameter><name>DeliveryMethod</name><value>SFTP</value></taskParameter>
<taskParameter><name>LocalFileFolder</name><value>in/RAW/eagle_ml-2-0_default_cm_ndfa/incoming/</value>

Files changed:
get_workflow_data.inc
runtaskrequest.xml

Production Bugs:

SDP-36632: Added Nuggetizer support for Pricing Report Extract via Extract Service

PRICINGREPORTEXTRACT via MC2 was updated to support nuggetizer functionality.

Example of usage:

<taskParameter>
<name>MicroBatchKeyRecordCount/name>
<dataType>S</dataType>
<value>50</value></taskParameter>
<taskParameter>

or
<name>RecordCountPerNugget</name>
<dataType>S</dataType>
<value>50</value>
</taskParameter>

File changed:
eagle_ml-2-0_cm/out/xml/extract_service_acct_pricing_report.xml

SDP-36161: Corrected Issue updating python translator rule on Publish

The Python translator rule was not properly updated when publishing from IWS.
The issue happened only when a previous copy of the file had a newer modification date.
This scenario occurred when multiple users from different time zones worked on the same file.

Files changed:
eagle_ml-2-0_cm/installer/mc_installer.prf