EagleML Release Notes - August 2015

This release uses XSD Revision 209.

There was an overlay file that is currently packaged as part of the EagleML monthly release. Please refer to About EagleML Overlays for specific details and impact to you.

Enhancements

EagleML

Added Ability to Specify Fields To Be Included in Extract
Clients expressed an interest in extracting files with less elements than the default settings. To allow for this, the new task parameter QueryProfileName was created in the outbound q rules. The basic approach is a column filter where you list what you want included in the extract within a text file. The column names must abide by the OWNER.TABLE.COLUMN format. The name of the file and two task parameters in the control message, FeedType and QueryProfileName, work together to make this option possible. The FeedType parameter still works the same way and specifies the type of object that should be extracted. The new parameter, QueryProfileName, takes the last part of the text file name as a parameter. This file is located in /estar/dynamic/msgcenter/profile/extracts and its name follows the format:

profile_[FEED TYPE]_[NAME].txt

The FEED TYPE is taken from the FeedType parameter and the NAME is read in from the value of the QueryProfileName parameter. An example of a text file name containing the columns desired for extract would be:

profile_GENERICISSUEANALYTICEXTRACT_tst1.xml

This enhancement allows for a more concise extract file which reports the columns containing only the data that you want.
This issue was tracked as Item #131421.


Allow For a Price of Zero on Trade Date
The accounting trade and the multileg accounting trade rules received a new additional tag. The notional reset indicator tag was mapped in both rules with the alias 3599. The corresponding path names were accountingTransaction/accountingTrade/notionalResetIndicator for the accounting trade and accountingTransaction/multipleLegAccountingTrade/notionalResetIndicator for the multileg trade. This addition allows the STAR engine to take a value of zero for the price on the trade date.
This issue was tracked as Item #131419.


EagleML Prepayment Speed Time Series Rule Now Consistent with CSV Version
The prepayment speed time series EagleML rule had a few inconsistencies with its CSV rule equivalent. The CSV rule uses the /eagle/star/reference/pan-addtimeseries.htm panel. The xml-ref_timeseries.xml rule now also takes advantage of this panel. When this feature was first added there was a missing mapping to the prepayment time series release status tag. This issue has been corrected and the eagle_ml-2-0_default_xml_in_reference stream supports the updated rule.
This issue was tracked as Item #130768.


New Impairments Object Added to the EagleML Library
A new impairments object has been added to the EagleML library. The object's corresponding tables is ESTAR.ESTAR_FINANCIAL_EVENTS and data from objects of this type reach the table via the eagle_default/in/xml/xml-impairments.xml rule which is attached to the new eagle_ml-2-0_default_in_xml_impairments stream. There are two basic event types that exist for this new object:

  • WRITEDOWN
  • BATCHCANCEL

The WRITEDOWN event adds data to the ESTAR_FINANCIAL_EVENTS table and the BATCHCANCEL event cancels this action.
This issue was tracked as Item #131569.


Extend Issue Analytic Model to Include New SECURITY_ANALYTICS_USER Table
The Issue Analytic Model has been extended to include the new table SECURITY_ANALYTICS_USER to allow for inbound and outbound functionality for the table. The xml-ref_genericissueanalytic.xml has allows for the upload of data to the new table. The outbound portion is handled by the extract_generic_issue_analytic.xml rule and is executed by sending a control message with the appropriate task parameters to the eagle_ml-2-0_default_cm_control_message stream.
This issue was tracked as Item #131900.


Added New Code Translation Functionality to Ratings
Code translation logic has been added to ratings in EagleML. The ref-issuerating_all.xml rule was restructured and the old ref-issue_rating.xml and ref-rating.xml rules have been removed. The ref-issuerating_all.xml rule now includes all the logic pertaining to ratings in EagleML. With this enhancement, the execution of the load of code categories and code values has been split into two distinct processes. Looking at the accountingValidationFlag, the rule either handles the addition of the above mentioned objects through a panel or through the stored procedure. If the value inside the accountingValidationFlag is "Y", then the eagle/star/reference/pan-addcodetrans.htm panel is launched to handle the addition of the code translations. Otherwise, the stored procedure will be executed. The result of all changes is a more simplified and efficient process of adding code translations.
This issue was tracked as Item #132239.


Updates to Extractor Panel to Support Improved Default EagleML Workflows
To support the new workflow for extract and load, the /eagle/messagecenter/extract/pan-eagle_extractor_demand_v2.htm panel has been updated. Corporate actions, accounting and financial events extracts are now available through this panel. In addition, security alias, ticker, primary asset Id, source, held, and custom FTP folder filters have been added to the panel. The default parameters have also been included.
This issue was tracked as Item #132295.


Support for Special Characters in Outbound SMF and Warehouse EagleML Interfaces
The Outbound SMF and Warehouse EagleML interfaces now support XML special characters, including:

<

>

&

"

All rules connected to these outbound interfaces have been updated to support these special character.
This issue was tracked as Item #128639.


Issue with HeldOnly Option on SMF Extracts
When you used both the Held Security Option and the entity ID filter on the SMF Extract, the entity Id filter was not being applied. The code was updated so that both the entity Id filter and the held security option are applied to the extract when selected.
This issue was tracked as Item #132398.


Created Cancel Option When One Part of a Multi Leg Trade Failed
When loading a multileg trade there could be instances where one of the legs failed but the other successfully loaded. This created issues since only one leg was present in the system. To complete the load of the multileg trade, all the legs would have to be changed and reloaded which was inefficient. The code was enhanced to allow for the ability to cancel the initial load when there was a failure with one of the legs.
This issue was tracked as Item #132467.

EJM

Created a Restart Option for EJM Tasks
An option for restarting an EJM task has been added to the EagleML framework. In the scenario where an EJM task failed due to a technical error, the option to send a new control message with the same exact correlation Id and a <taskTypeEnum> field with a value of "RESTART" has been added. The /eagle_ml-2-0_cm/close_corrid.inc include file has been created to handle the new action. This control message will clear out the remnants of the schedule to allow the job to be executed with the same correlation Id. This new restart functionality eliminates the need of having to change the correlation Id and simplifies the process of launching an equivalent job in EJM.
This issue was tracked as Item #131709.


Added GPG Encryption Support
GPG Encryption support has been added to EJM. It is now possible to include a task parameter that will encrypt the extract file in the GPG2 format when sending a control message to extract data from the system. The task parameter name is ENCRYPTION_MODE and the corresponding value is gpg2. The resulting extract file should have the .gpg extension and should only be accessible from the FTP.
This issue was tracked as Item #132734.


Extensions on Encrypted Files
The extensions for FTP .gpg or .pgp encrypted file names were updated. The original file name plus the .gpg or .pgp extentsion will now be used, instead of just replacing original extension with .gpg or .pgp.
This issue was tracked as Item #133132.

Software Fixes

EagleML

Fixed reason tag formatting for extracts
There was a small issue with the extract of the traded cash activity object related to the reason tag. The value inside the reason tag was not being formatted and delivered correctly. The /eagle_ml-2-0_cm/out/xml/t_acct_traded_cash_activity.xml rule was edited to improve the decoding process for this tag. The value is now correctly decoded and reported in the extract file. This fix helps avoid confusion which could pop up as a result of the improperly shown value.
This issue was tracked as Item #127462.


MSSQL only - Issue with Warehouse Batch Mode Function
The warehouse batch mode function was not working properly in the MSSQL environments only. There are two options for how to send files to the warehouse stream:

  • single mode
  • batch mode

The single mode creates a new record in the database with a unique ID, while the batch mode deletes the old record and adds the new one in its place. The core part of this process is the delete block which finds the old record according to the batch key field in the new file and purges it from the database. The new record is then added. The delete block logic was missing in the MSSQL platform. The code was updated to resolve this issue and now the warehouse batch mode has the same functionality across all environments.
This issue was tracked as Item #129829.


Entity Manager Relationship Issue Due to Apostrophe
The entity manager relationship object did not properly load to the database if the entity Id contained an apostrophe. The eagle_default/in/xml/xml-ent_entitymanagerrelationship.xml rule has been updated and the object now loads without issue.
This issue was tracked as Item #130806.


Resolved EagleML Warehouse Trade Extract Issues
There were a few issues with the warehouse trade extract. The tags prepaySpeed, strategy and timeOfTrade were not included in the extract message. The commissionCurrency and tradeCurrency fields were both being mapped into the same tag. Also, the netAmountLocal tag was being over written by the netCash tag. In addition, several tags were not supported by the rule and therefore were not included in the extracts. The code, specifically the wrh_egl_gen_interface3.xsl XSLT file, was updated to resolve these issues.
This issue was tracked as Item #131253.


Fixed Grouping by Ticket Number in TRDX_EGL_GEN_INTERFACE
The TRDX_EGL_GEN_INTERFACE did not support grouping by master ticket number. The XSLT generated only one message for several items when it should have generated an individual message for each of the trades. The rule has been change to correct this issue and allow the creation of separate messages when the trades are grouped by master ticket number.
This issue was tracked as Item #131349.


Inconsistent Name in Underlying SMF Rule
The underlying SMF rule had an incorrectly named node which was coded as altSecMinorType. The corresponding alias definition in eagleml_smf_underlying.xsl was code as altSecurityMinorType instead. This discrepancy caused errors and improperly loaded data. The xml-smf_underlying.xml rule has been updated and data is now loaded properly in this field.
This issue was tracked as Item #131407.


Fixed Incorrect TSR Formatting
The task status reply control message was formatted improperly due to a small inconsistency in the TSR rule. The XSD states that the reason items should come before the unit task item, but one of the sets of reason nodes was actually placed after the unit task nodes. The part of the rule that coded for the task unit nodes was included with the code related to the reason nodes. This was changed in the eagle_ml-2-0_cm/out/xml/task_reporter.xml rule to reflect the TSR appearance described by the XSD. The task status response messages are now sent properly formatted.
This issue was tracked as Item #131813.


Issues with Data Roll Up for Batch Mode Open Lot Warehouse Objects
When you send a warehouse open lot file to the eagle_ml-2-0_default_in_xml_warehouse_preproc stream, the lot_level_position table is updated correctly. However, if if <batchMode> is set to 'Y', the the data was not properly rolled up in the position_detail table. This error occurred due to an issue with the XSLT wrh_egl_gen_interface3.xsl rule. The data roll up is controlled by setting the procedureFlag to "PD" and the rule did not properly handle the setting of this field to the expected value when batch mode was on. The code was updated and the data is now rolled up correctly.
This issue was tracked as Item #131977.


Fixed Xreference Filter for Multi Leg Securities Extract Through Panel
If you extracted multileg securities by the xref Id through the panel, only one record would be extracted. This problem was the result of a setting in the panel execution that calculated the security alias based on the xref Id. This was an incorrect setting since securities can have the same xref Id but different security aliases. The panel has been adjusted to allow the security alias to be entered as a separate field and when the xref Id is entered the security alias no longer gets auto filled. This update permits proper extract by xref Id.
This issue was tracked as Item #132207.


Fixed FWD SMF Extract Issues
During the extract and load of multi leg securities several fields were omitted due to the FWD SMF leg not being extracted correctly. The inbound and outbound rules were updated to ensure that all fields are properly extracted and loaded.
This issue was tracked as Item #132235.


Added Deletion When Failed Leg of a Multi Leg SMF Load
When one leg of a multileg SMF was loaded with an error, if the other two legs loaded into the database the overall record would incorrectly return a success status. Also, if you then loaded the same SMF with all three legs corrected, the overall status would incorrectly return a failure. This issue was a result of the other two legs being present in the database. A panel has been added to the xml-MF_multileg_swap.xml rule which to delete the legs that passed through to the database if any of the other legs had errors. This resolves the issue of having only a partial multileg SMF in the database until being deleted manually.
This issue was tracked as Item #132390.


Issue Canceling Corporate Actions
There was an issue with cancelling corporate actions via EagleML. In the CSV equivalent stream, only three fields were mandatory, but when these same three fields were passed to the EagleML version of the rule, Message Center returned errors referencing missing identifiers. The rule has been modified to allow the rule to work properly with the three fields being passed to it.
This issue was tracked as Item #132394.


Resolved Corporate Actions Trans Types Compatibility in EagleML
The trans types in the EagleML corporate action inbound rule pointed to the same panel as the CSV equivalent, but the records were not processed. This issue was a result of the trans types not being compatible with any Eagle jobs because they were not EagleML trans types. All the relevant trans type names have been changed in the inbound xml rule to allow compatibility with the Eagle panel.
This issue was tracked as Item #132452.


Added Functionality to Xreference Lookups Within Multi Leg SMF
The multileg SMF did not support xreference identifiers and only allowed for lookups based on the primary asset Id. Support for xref identifiers was added by changing the multi leg SMF inbound rule. The contract leg is the basis for all the xreferences and the pay and receive leg's identifiers are evaluated based on the contract leg. The multileg trade rule has also received support for the xreference identifiers. It is now possible to load the xreferences through a call to the stored procedure within both rules.
This issue was tracked as Item #132590.


Resolved Targeted Mapping Issues for Multi Leg SMF Rule
The following fields were missing mappings in the Multi Leg SMF:

  • derivChar1
  • derivChar2
  • derivChar3
  • derivChar4
  • derivChar5
  • derivChar6
  • derivChar7
  • derivChar8
  • derivChar9
  • derivChar10
  • derivDate1
  • pRollConvention
  • rRollConvention
  • rollConvention
  • SMExtension/prvtPlace

The eagle_default\in\xml\xml-smf_multileg_swap.xml rule was updated to include these mappings.
This issue was tracked as Item #132593.


Transaction Id and Reference Number Now Included in TSR
When an error occurred in an inbound process, the task status response message did not include the transaction Id. The xml-warehouse_v2.xml and xml-wh_brokertradequote_v2.xml rules were updated to include mapping for the transaction Id. The problem occurred because of the missing mapping as well as the reference number being hard coded. Both of these issues were corrected and the transaction Id as well as the reference number are properly related in the task status response.
This issue was tracked as Item #132619.


Fixed GenericIssueAnalytic Extension Panel Issue
When you created an extension for the GenericIssueAnalytics panel, you would get the error

Error reading a message from the channel. Status Code: -3 Channel Status Code: -3 Error Message: No data. /estar/tpe/dynamic/eagleml/extension directory doesn't contain any genericissueanalytic.ext file(s)

This issue occurred due to improper case in the extension config file. The code was updated to resolve this issue.
This issue was tracked as Item #132626.


Issue with LongShortIndicator of Multi Leg Trades
The LongShortIndicator incorrectly loaded with the same value for contract, pay and receive legs of a multileg trade. This issue was due to improper mapping that would over write the calculated values returned by the panel. The following rules and include files have had changes in the mapping to correct this problem:

  • eagle_default\in\xml\xml-accttrade_multipleleg.xml
  • eagle_default\in\xml\include\multiplelegtrade_aliases.inc
  • eagle_default\in\xml\include\multiplelegtrade_common_columns.inc

This issue was tracked as Item #132935.


CM_READER stream Updated
The July month end release included an enhancement to the CM_READER stream to query an Eagle ACCESS database instead of polling the FTP site (tracked under item #131094). In the case where the file was not found in the database, the system would not report the error correctly and the processing within the streams continued. The code was enhanced so that if one of its entry is processed with errors or it is unable to locate files in the database, the process is stopped immediately and an alert is sent out.
This issue was tracked as Item 133312.

EJM

New CustomWhere Task Parameter
The extract_filter_common.inc include file now allows for a task parameter that you can enter a value to specific the location from which data should be extracted during a SQL statement. This task parameter is named CustomWhere. The value of the parameter will be appended to the WHERE (1=1) AND… part of the SQL query. This enhancement allows you to select data for extracts with the help of a SQL query.
This issue was tracked as Item #129523.


False Success Status of TSR Message When Cash Flow Projections Timed Out
When sending an RTR message to the control message stream with the goal of generating cash flow projections, the task status response message would return a false success status. The process would actually time out and not execute, but the TSR reported that the process was successfully completed. To resolve this issue, an IsPRFW parameter equal to one was added in the xml-task_reporter.xml rule. This change in the rule will properly return a failure status in case of a timeout.
This issue was tracked as Item #132173.


Incorrect Success Status in TSR when a Subtask Fails
The task status response message did not reflect a failure within a subtask in the parent workflow. The overall status of the workflow would be noted as success in the TSR even when a sub process executed with errors. A task parameter field, ExecuteToFirstFailure, was added to the control message to resolve this issue. This parameter allows the workflow to run only up to the first failure it encounters. If a failure occurs, it immediately send the TSR with a failure status. This fix helps avoid numerous problems that can occur as a result of false success statuses.
This issue was tracked as Item #132343.


CM_Reader Stream Updated to Handle Errors
The July month end release included an enhancement to the CM_READER stream to query an Eagle ACCESS database instead of polling the FTP site (tracked under item #131094). In the case where the file was not found in the database, the system would not report the error correctly and the processing within the streams continued. The code was updated so that if one of its entry is processed with errors or it is unable to locate files in the database, the process is stopped immediately and an alert is triggered.
This issue was tracked as Item #133312.


UpdateTimestamp Omitted from Client Extracts
The updateTimestamp field was omitted from client extracts due to a misnamed field in the include file. The field name was changed to reflect the correct grouping within the rule and the field now shows up as expected in the extract files.
This issue was tracked as Item #133313.