EagleML Release Notes - September 2015

This release uses XSD Revision 211.

There was an overlay file that is currently packaged as part of the EagleML monthly release. Please refer to About EagleML Overlays for specific details and impact to you.

Enhancements

EagleML

Allow Lookup Based on Master_ticket_number for WarehouseCloseLot
The warehouse close lot rule's lookup found a close lot object by its entity Id and security alias. However, there are occasions when these two fields are blank. The code was enhanced to address this scenario. In this instance, the warehouse preproc stream is now able use the master ticket number and check in the TRADESDBO.TRADE DB table for a matching one, then grab that record's security alias and entity Id and return them. The enhancement is driven by the SCR_INTFC_INST which was added. It will return an error if the stream returns anything other than one matching record.
This issue was tracked as Item #129963.


EagleML Miscellaneous Income Lot Specific Load
The accounting asset cash interface was expanded to allow for the storage of Miscellaneous Income/Expense Lot Specific objects. The xml-acct_cash_asset.xml rule received a new include file with proper mappings for handling the new object type. An incoming message should be sent to the eagle_ml-2-0_default_in_xml_acct_asset_cash stream and should have a <assetCashType> field with a value of 'LMISCINC' or 'LMISCEXP'. The message is received by the stream and sent to the eagle/star/cash/pan-miscincome_lotspecific.htm panel for loading to the database.
This issue was tracked as Item #130742.


Allow Held Filter to be Used with Cash Activity and Corporate Actions
The Held filter was functional only with the positions table. The code was enhanced to allow the Held filter to be used with cash activity and corporate actions. The extract_filter_held.inc and extract_filter_entity.inc include files were both updated to allow the extraction of the two new types.
This issue was tracked as Item #132607.


Added HUB Security_alias as a Security Xref in SPOKE
The SPOKE services was enhanced to support SMF security alias and xreference identifiers fields. The data is initially stored in the HUB and the SPOKE will send a request to HUB for the information. This enhancement added the support of xreference identifiers and security alias filters for these types of extract. Now when a request is made, the xref Ids and the security alias data gets updated to the securitydbo.xreference table in the database and subsequently loaded into the SPOKE infrastructure.
This issue was tracked as Item #132608.


Track the Last Successful Load in SPOKE to Drive Delta Extracts from HUB
The code was enhanced to allow the SPOKE environment to track the last successful load to itself, store the resulting value and then send a run task request to the HUB environment. The RTR will send an extraction request for the last record that was loaded which will be sorted out by the value that the SPOKE environment was supposed to track and store. Once the extraction is made, a task status response is sent from the HUB environment to the SPOKE environment. This process involved the altering of all outbound rules in the HUB environment as well as new streams in both environments that allow them to send and receive the necessary messages.
This issue was tracked as Item #132609.


Add Configuration to Exclude Security Alias in Extracts
New functionality was added to the Instance to Instance interface to allow you to decide whether to exclude the security alias in the extracted files. The new parameter <excludeSecurityAlias> was created and coded for in the security related outbound rule. This option is controlled just like any other task parameter in the control message. A value of "Y" or "YES" will act as a signal not to include the alias in the extract file.
This issue was tracked as Item #132624.


Add Reference Object Support for the Generic Delete Option
The generic delete option which allows the elimination of data from the database before the load of a specified object was enhanced to support reference object types. This feature is currently used by internal regression testing tools.
This issue was tracked as Item #132907.

EJM

Schedule-based IWS Workflows Can't Receive Parameters from Automation Center Event
Schedule-based IWS workflows had issues receiving parameters from the Automation Center due to their limited scope of the restart option. This option only allowed the workflow to be restarted exact as it was previously run. It could not handle passing new task parameters to the workflow between restarts. This code was enhanced with a new addition to the *_restart template. The tag contains a call to an external file created inside the workflow, which contains task parameters to be passed to the engine during restarts. The *_restart.xml rule also received a new feature that lets all these functions be controlled through the cm stream with the use of a run task request file.
This issue was tracked as Item #132106.


Enhancement for HUB and SPOKE to Handle Additional Delivery Types
Enhancements were made to allow the config files of both the HUB and SPOKE environments to handle more than one delivery type. The desired available types include: MQ, FTP, SFTP and JMS protocols. This enhancement allows diversified communications between the two environments making using them simpler while also making it more expansive and open ended.
This issue was tracked as Item #132610.


Added Option to Create an Empty File When No Data
EJM was enhanced with a new feature to create an empty zero byte sized file and return that instead of returning a status of NO DATA. This feature was added to address the fact that there are some systems that do not respond to a status, but can pick up and interpret an empty file. This option is controlled with a parameter in the run task request. The parameter is called ForceExtractFileCreation and will create an empty file if the value stored in it is 1.
This issue was tracked as Item #133263

Software Fixes

EagleML

Issue Extracting Generic Entity Elements
When extracting a generic entity, several fields would either be extracted with an incorrect value or not present at all in the extracted message. The issue was due to lack of inbound mapping in the stored procedure. This caused the fields to never be updated to the database when the entity was initially sent to the system. The source name and month day fields were missing and returned with an improper value. The problems have been fixed by creating a new datamap column for outbound only fields as well as fixing mappings within the outbound rules. All values that are expected now have full functionality in generic entity extracts.
This issue was tracked as Item #131306.


False TSR Error When Compress Mode Used and No Data to Output
When compress mode is applied during an extract and there is actually no data returned to be compressed, the task status response message returns an error. It should be able to recognize that no data was extracted and simply return this as a status. The logic has been adjusted to identify this scenario and the TSR will no longer return an error.
This issue was tracked as Item #131340.


Issue Including SMF Reference Data When Running Financial Event Extract
When running a financial event extract through the extractor demand panel, there is an option to include SMF reference data in this extract. This is done by selecting the SMF + ENTITY option under the drop down menu for the financial event. The panel was unable to correctly keep the settings, however, and if any other filters were added the selections would all be cleared and the panel would run in default mode. This problem has been fixed to allow extracts to include the SMF reference data and support any filters for financial events as well as for reference or SMF data.
This issue was tracked as Item #132068.


Issue with EagleML Cancel Accounting Trade
The feature which allows you to cancel any accounting trade load by sending a control message with the cancel action and the targeted Id present was not working properly. Message Center was unable to identify tag F26. This tag contains the transaction Id and since it was incorrectly mapped, Message Center was unable to pick it up. The estar\tpe\dynamic\msgcenter\eagle_default\in\include\csv-cancel_columns.inc and estar\tpe\dynamic\msgcenter\eagle_default\in\include\csv-cancel_panels.inc includes were updated to include the proper mapping for F26. The cancel feature now functions as expected.
This issue was tracked as Item #132615.


Filter Issues with ISSUE FX RATE and ISSUE FX PC RATE Extracts
Investment type and security alias filters not working in ISSUE FX RATE and ISSUE FX PC RATE extract. The code was updated and these filters now work as expected.
This issue was tracked as Item #132859.


Warehouse Position Load Deleting Lot Level Data in Batch Mode
When loading warehouse positions in batch mode, there was an issue around the delete block of the eagle_default/in/xml/include/xml-dbdirect_wrhs_delete_statement.inc include file. When the batch mode positions were loaded, the delete block triggered and cleared all the lots in the lot level tables. This block should not be reached unless the lots are being loaded. The rule was updated with conditional logic. This conditional logic is presented in the form of <openLotDeleteSwitch> flag which takes values "Y" or "N". Now the lot level tables will only be cleared if specified.
This issue was tracked as Item #132916.


Generic SMF Issue
The generic SMF could result in the error:

Error executing the request: Invalid character value for cast specification

This error resulted from a problem within the INSERT_UPDATE_SECURITY stored procedure. A new parameter called in_der_RATE_CONVERSIO_RULE was added to this stored procedure resolve the the issue.
This issue was tracked as Item #132998.


Issue Receiving Replies from Web Service Synchronous Jobs
There were problems receiving replies from web service synchronous jobs. This issue was because the expected format for file names that WS is preset to was not being met. In the config file there was a parameter that was naming files based on a different mask than what web services expected and therefore causing issues. The solution was to create a copy of that file and give it a name that the WS receiver expected. This was accomplished with the eagle_ml-2-0_cm\reporter_dist.inc include file in which there is code for handling the file names. The files now correctly make it to the cmw/reply/ folder.
This issue was tracked as Item #133189.


Issue with LongShortIndicator in Warehouse Trade
The loaded element <longShortIndicator> in the Warehouse Trade file did not populate the database field TRADESDBO.TRADE_DETAIL.LONG_SHORT_INDICATOR. Instead, the element <starLsInd> populated this field. This issue resulted in incorrect data stored for both <longShortIndicator> and <starLsInd>. The code was updated to resolve this issue.
This issue was tracked as Item #133270.


Issue with Corporate Actions Load Process
There was an issue with the corporate actions load process that was caused by an incorrect mapping for the correlation Id. The mapping in the rules did not match up with the xPath which caused the stream to return an error when the actions were loaded. The mapping in the inbound rule has been corrected to match the xPath and now the corporate actions load works as expected.
This issue was tracked as Item #133287.


Issue with Underlying Model When Swap Extracted
The underlying model was not always included when the swap was extracted. This problem was fixed with the creation of two new include files. The first include file calculates the security alias of the underlying model and then second files handles the linking of the model to the database table. The underlying models are now consistently included during extracts.
This issue was tracked as Item #133306.


Issue Loading Underlyings When Primary Asset Id is Null
There was an issue loading underlyings in the the eagle_ml-2-0_default_in_xml_smf_generic stream. An error would be raised and the process stopped if the underlyings did not include a primary asset Id or the primary asset Id was null. The code was updated to allow the stream to ignore any underlying SMF message sent without a primary asset Id. Now instead of an error stopping the stream, it simply reports on it and moves on.
This issue was tracked as Item #133323.


Issue with Earnings Field in the Eagle_ml-2-0_default_in_xml_reference Stream
There was an issue with tag 97 in the eagle_ml-2-0_default_in_xml_reference stream. Tag 97 corresponds to the EARNINGS field and failed to load in the case of a conditional statement being equal to 1. The /eagle_default/in/xml/xml-ref_earnings.xml rule was updated. The naming of the primary asset Id field was corrected and the code was changed to prevent the lookup results from being erased when the lookup result was 1. The field names are now all functional and the lookup no longer encounters issues when the resulting value is 1.
This issue was tracked as Item #133383.


Issue with SWAP Security Loading Due to Environment Setting
There was an issue loading SWAP securities to the database which resulted in only the contract leg begin uploaded. This issue was actually the result of a setting in the particular environment on which the failures were occurring. The setting did not allow for duplicate entries in fields that had been unknowingly filled in with duplicates. To avoid this error, it is necessary to select the proper settings that match the format of the data being sent to the database.
This issue was tracked as Item #133384.


Reference Extension Objects Were Not Loading Correctly
EagleML reference extension objects were not loading properly. The inbound reference extension rule was updated to allow these objects to load and update the correct fields in the respective database tables. Reference extensions now load as expected.
This issue was tracked as Item #133425.


Completed Mapping for Multi Leg SMF Rule
In the August release, targeted mapping issues were resolved for the multileg SMF rule under BT #132593. The code was updated further to address all other outstanding mapping issue with this rule.
This issue was tracked as Item #133449.


Issue with Underlyings in the Eagle_ml-2-0_default_in_xml_reference stream
The eagle_ml-2-0_default_in_xml_reference stream had issues with underlyings messages being loaded. It showed the incoming messages as empty even when they actually had data. This issue occurred since the previous settings of the stream allowed records to pass as long as at least one element within the underlying model group was not null. The primary asset Id node was actually required in this model, so the inconsistency could sometimes result in records without a primary asset id being passed to the stream. To avoid this issue, the conditional logic has been rewritten to require the primary asset Id. The code will now read as: the model group must have at least the primary asset Id node present.
This issue was tracked as Item #133513.


T Stream Sending Message Prior to Copy
The XMLDBAPI cache link directory was not set up which resulted in problems in the eagle_ml-2-0_default_out_q stream during extraction. The errors showed on the q stream because the out_t stream was acting too quickly with the copy command. The process is meant to copy the message and then send it to the q stream, but the t stream was sending the message out before it had a chance to be copied. This problem was fixed with a catch mechanism to ensure that the file was fully written before being sent along to the next stream.
This issue was tracked as Item #133683.


Updated Stream Categories
EagleML, Eagle Job Manager, Integrated Workflow Studio streams were reviewed and updated to have the correct stream categories and DB update levels.
This issue was tracked as Item #133747.


Issue Selecting the Correct t_stream
There was an issue where EagleML would not always correctly choose when to use the eagle_ml-2-0_default_out_t_streaming stream versus the eagle_ml-2-0_default_out_t stream. This issue was due to the config file not having the proper ranges for the versions which require each rule. These ranges have been corrected and the the appropriate stream is now always selected.
This issue was tracked as Item #133870.


Warehouse Open Lot Extract not publishing Entity ID
Entity ID was not being included in the warehouse open lot extract. The code was updated to resolve this issue.
This issue was tracked as Item #133930.


XSD BuySellIndicatorEnum Changes
Changes were made to the BuySellIndicatorEnum in the XSD. These changes included the addition of the openLotDeleteSwitch to the warehouse position complex type, the addition of the 'REC' value to the BuySellIndicatorEnum simple type and the addition of the 'EXTRACT' value to the TaskTypeEnum simple type.
This issue was tracked as Item #133995.


Unable to Overwrite Some Parameters Using W_config_custom
The w_config_custom include was originally added near the top of the the config file. This location prevented you to be able to manipulate many parameters. The include was moved to the end of the file to allow you to overwrite parameters.
This issue was tracked as Item #134327.


EagleML Changes Around First and Last Coupon Date Calculations
In the July release, functionality was added to calculate the first/last coupon dates if you set the value of the parameter calculateFirstLastCouponDatesFlag to 'Y' in the incoming file. If this parameter was set, the First and Last Coupon Dates were both calculated and if the First and/or Last Coupon Dates were sent in the incoming file, they were overwritten with calculated values. That item was tracked as Item #130173. There were issues were identified with these changes. The identified issues have been partially addressed in this release. In this release the logic was updated so that if you pass in a First Coupon Date and have this parameter set AND also have the accountingValidationFlag parameter set to "N", it will no longer overwrite the First Coupon Date. Note this change is only applicable if you have the accountingValidationFlag parameter set to "N".

Important Note: While the above changes are included in this EagleML monthly release, there are still core Eagle changes that are required in order for the First and Last Coupon Date to be appropriately calculated for some asset types and until those core changes are delivered, the above changes are not necessarily beneficial. Also, Eagle is planning to enhance RDC in an upcoming release to be able to calculate the dates. Once this enhancement is available, you may no longer want to pass the EagleML parameter of calculateFirstLastCouponDatesFlag to 'Y' at all. Please review with your Eagle representative.
This issue was tracked as Item #133943.

EJM

Loading Issue When Warehouse Object Has Extension File
The warehouse preproc stream would occasionally return an error when a warehouse object sent for load had an extension file. The stream would not be able to recognize the extension object and returned an error that said there was no data in the extension file. This issue was a result of the stream not locating the proper xslt file to transform the extension file. The stream is was updated to recognize when extension files are present with the original warehouse object and then select the appropriate xslt.
This issue was tracked as Item #133482.


Corrected File Extension Issue
When sending a run task request which included the parameters CustomLoadStream and CustomLoadFile1 to EJM, the specified loading stream crashed due. This issue was a result of an error with the logic in the inbound rule. The logic for resolving the fields was rewriting the extension of whatever file was passed in the CustomLoadFile1 parameter as .xml. This caused the load stream to fail as it was unable to recognize the file since the extension was incorrect. This logic was corrected and the original extension is now kept at the end of the file name.
This issue was tracked as Item #134454.