EagleML Release Notes - May 2016

This release uses XSD Revision 223.

There were several issues identified in the May release and on May 25 a special amendment was issued that should be installed alongside the May release package.

There are four overlay files that are currently packaged as part of the EagleML monthly release. Please refer to About EagleML Overlays for specific details and impact to you. In the May 2016 release only, there is one additional overlay file used as listed below. The functionality that requires this particular overlay is only available today in Eagle release BW18 only:

eagle/estar/tpe/dynamic/custdat/eagle/messagecenter/extract/pan-eagle_wrf_trr_reset_demand.custom.xml

Enhancements

EagleML

New Custom Chart of Accounts Interface

EagleML was enhanced to include a new interface called Custom Chart of Accounts. A new stream and new set of rules were added which correspond to the RULESDBO.CUST_CHART table.
This issue was tracked as Item #137897.


New Factor Interface

EagleML was enhanced to include a new interface called Factor. This interface corresponds to the SECURITYDBO.FACTOR table. New rules and feed types as well as outbound streams were created to accommodate this interface.
This issue was tracked as Item #137898.


New Ledger Notes Interface

A new ledger notes interface was added to EagleML. This new interface with corresponding rules tied to the ESTAR.ESTAR_LEDGER_NOTES table was added to the EagleML library
This issue was tracked as Item #137899.


Changes in the Warehouse Trade Strategy Mapping

The warehouse trade interface element strategy was incorrectly mapped to the in_strategy_id. The mapping was corrected to in_strategy instead of in_strategy_id.
This issue was tracked as Item #138443.


Ability to Load Cross Reference Configurations and Interface Detail
EagleML was enhanced to allow you to load Cross Reference Configurations and Interface Details data from files across different regions.  The stream eagle_ml-2-0_default_in_xml_reference stream was leveraged and the following files changed to accommodate this enhancement:

  • /eagle_default/in/xml/xml-ref_sourcefeed.xml
  • /eagle_default/in/xml/xml-ref_source.xml
  • /eagle_default/in/xml/xml-ref_smfidentifierconfig.xml
  • /eagle_default/in/xml/xml-ref_objects.rsf

This issue was tracked as Item #138583.


GenericLoad Workflow Dependency Enhancements

The generic load workflow was improved to account for dependencies during load. Certain objects have to be loaded prior to others due to the relationships these objects have with each other. Therefore, the Generic Load was enhanced to allow for you to control dependency not only on data feed level but also on Feed sub type level.
This issue was tracked as Item #138915.


New Accounting OpenRecPayMulti Extract

There was a need to create new EagleML extract calling the panel eagle/star/cash/pan-openrecpaymulti.htm. A new model group, object type and new outbound rules were created to support the above mentioned extract. This extract is supported in Oracle only as all Accounting Extracts are supported in Oracle only.
This issue was tracked as Item #138992.


New Accounting UnapprovedVariationMargin Extract

Extracts from the Unapproved VariationMargin was added to the EagleML library. A new model group, object type and new outbound rules were created to support the above mentioned extract. This extract is supported in Oracle only as all Accounting Extracts are supported in Oracle only.
This issue was tracked as Item #138994.


Generic CA Reference Interface Improvements with To Asset Logic

In the Generic CA interface, the logic for resolving the "to_asset" security has been improved. The "to_asset" is found more efficiently and the value is not changed when the original action is being updated or rebooked.
This issue was tracked as Item #139309.


New User Group Filter Added to Three Extracts

The user group parameter was added to generic entity, warehouse open lot and warehouse position extracts. The parameter can now be used in control messages and it will filter by Entity Entitlement. The outbound q rule was changed to include filtering by this new parameter.
This issue was tracked as Item #139818.


Non-streaming support for Entity Share Class Extract

In the previous release, the new entity share class had been created with extract functionality that only allowed for streaming mode only. A non streaming rule was added for backwards compatibility with older Eagle versions that do not support streaming mode. The outbound entity share rules were edited to allow for either streaming or non-streaming to be used appropriately.
This issue was tracked as Item #140671.

EJM

EJM TSR improvements for Datamart jobs

A few changes were made to the content of the TSR messages sent by EJM for Datamart jobs. Any successful substeps will not have any information included.

With this change, by default all the steps (information for them is shown in <taskUnit> node) with a status = SUCCESS will be skipped and will not be shown in the TSR.

The example of skipped step:

<taskUnit>
  <status>SUCCESS</status>
  <severityCode>0</severityCode>
  <eventDetails>
    <eventName>Model Manager for [Fund Summary] [As of 12/31/2014] [15861]</eventName>
    <eagleInstanceId>1721338</eagleInstanceId>
  </eventDetails>
</taskUnit>

To switch ON the displaying success steps for PACE jobs in the TSR, the setting W_CONFIG_SKIP_SUCCESS_STEPS = N should be set up in the configuration file _w_config_custom.inc.

A second enhancement is that Event Entity Summary Details are displayed for failed DataMart Model Manager steps. The details contain the following:

  • Summary details: Number of failed entities, total number of processed entities
  • Entity Details: The set of failed entity Ids

By the default the Event Entity Details are switched off. To enable adding Event Entity Details to the Final TSR, the task parameter ReportEventEntitiesDetails = Y should be populated in the RTR message or the setting W_CONFIG_INCLUDE_ENTITIES_DETAILS = N should be set up in the configuration file _w_config_custom.inc.

The example of task parameter in the RTR:

<taskParameter>
       <name>ReportEventEntitiesDetails</name>
       <dataType>S</dataType>
       <value>Y</value>
</taskParameter>

Also, while testing this enhancement an issue was uncovered where an invalid TSR could be generated (empty status mode). The invalid TSR was generated due to the database returning job subevent status message with a carriage return character. This caused issues in rule with parsing this data and TSR generation. That issue was corrected. 
This issue was tracked as Item #138468.


Introduced New Total Return Swap Reset Workflow

A new workflow was created for resetting Total return swaps. Previously, the steps to correct a Total Return Reset that was processed incorrectly was manual and time consuming. A new workflow was created to streamline this process. This process can be triggered through panels or automated using IWS via control message. Please note that this functionality is currently only available today in Eagle release BW18 only. It is not available currently for clients using any other version of Eagle.
This issue was tracked as Item #140561.

I2I

New History Only Option for Generic Entity Extracts

A new option was added to the generic entity extract. Previously, there was no capability to extract entities in history only mode. The generic entity outbound rules received new additions which allow for extracting entities from history tables.
This issue was tracked as Item #133684.


Performance Load Option Improvements

There was an issue with a specific scenario in which the back dated periods are set as final so that an accounting event can be rebooked. The performance interface was unable to appropriately handle this case and the accounting event would be deleted and not re entered. The resulting action was the deletion of the original record. The action should replace it, but the updated record failed to load since the incoming file had a period set to final. This issue only occurred in the above scenario when I2I is used to leverage the performance interface. Code was changed so that the periods being locked down as final is acceptable. There are two ways to specify the capability: the custom config file or the incoming file itself.
This issue was tracked as Item #137964.


SMF Source Mapping

Previously there no capability to reassign source names based on more than one criteria. The filters have been improved to convert source names based on predetermined logistics defined in the custom config file. The HUB source name can now be converted based on predetermined rules to find the matching mapping in the SPOKE.
This issue was tracked as Item #138596.

Software Fixes

EagleML

Reference Interfaces Were Loading Fields as Blank Instead of NULL

If the incoming file does not have a set value for a field, you expect to see a NULL value loaded into the database. However, in the case of the character fields in the SECURITY_ANALYTIC_USER table, blanks were loaded instead of NULL. The code was changed to correctly populate the field with a NULL value rather than blank in the case that the incoming file has a blank field.
This issue was tracked as Item #135297.


Performance Improvements to Task Reporter Stream

The throughput of the task_reporter stream was noticeable slower when multiple file masks were processed at the same time. The code was improved to increase throughput. The stream works the same if there is only one file mask to process, but the area of significant improvement involved code changes that altered the processing of data with multiple file masks.
This issue was tracked as Item #136309.


Mapping missing in Warehouse Trade

The in_principal_trd mapping was missing from the wrh_egl_gen_interface3 rule for warehouse trade. As a result of it missing, the element would not show up in databases or on extracts. The mapping was added to the interface and is now available for use in databases as well as outbound rules.
This issue was tracked as Item #136370.


Enable Streaming for Corporate Actions Announcement
There were problems with large volume extracts for Corporate Actions Announcement. The extracts could time out and not complete. A new streaming rule was created to allow for extracting large amounts of Corporate Actions Announcements quickly.
This issue was tracked as Item #137140.


Enable Streaming for Pending Trades Extracts

There were problems with large volume extracts for Pending Trade extracts. The extracts could time out and not complete. A new streaming rule was created to allow for extracting large amounts of Pending Trades quickly.
This issue was tracked as Item #137151.


Enable Streaming for Entity Extracts

There were problems with large volume extracts for entities. The extracts could time out and not complete. A new streaming rule was created to allow for extracting large amounts of entities quickly.
This issue was tracked as Item #137161.


Issues with Concurrent GenericCA Load

There was a problem inserting records via concurrent loading due to one of the instances lagging behind. The second instance would fail since it would be processed a little after the first one and therefore be reported as a duplicate record. Logic in the eagle_default/in/xml/xml-ref_generic_ca.xml rule was altered to allow for duplicate instance numbers to resolve this issue.
This issue was tracked as Item #139096.


Corporate Action Interfaces Fixes

There were several issues with the Corporate Action interface. The change_ca lookup was not filtering on tag 54. The ref_ca rule allowed tag 60 to be passed as a parameter. In addition, the cancel rebook was not working properly. The above issues were corrected through improved message mappings in the relevant rules.
This issue was tracked as Item #139349.


Swap Trade Logic Added

There was a difference between Book Trade Panel and pan-trsswapclose.htm requirements. When an OPENSWAP for a TRS is booked through the UI, if there is an error on the pay leg, the pay leg is prevented from posting, however the other two legs, the contract and the receive leg, are posted. In the stream, when there is an error on one of the three legs for a trade, batch cancel is called so that none of the legs posted. In order to implement logic same to the panel, a new include has been created and added into the xml-accttrade_multipleleg.xml. The include name is add_fields_clone. It has logic that checks incoming files for additional leg fields and either gets them from the incoming file or calculate them according to the panel logic.
This issue was tracked as Item #139407.


Issues with Parallel Execution

Parallel loading was not operating correctly and would send error messages. The code was changed to switch the order of the task parameter, allowing this parameter to be picked up by the rules whereas it was ignored before. Also the custom streams are now supported for parallel loading and the parameter has been equipped with a default value for better execution.
This issue was tracked as Item #140343.


Issue Processing TAR.GZ files for Eagle_wrf_generic_load
There was an issue handling TAR GZIP files from FTP when using a control message to call the workflow eagle_wrf_generic_load. When the control message was sent to the stream eagle_ml-2-0_default_cm_control_message, the stream placed the retrieved file from the FTP into the cmw/in directory with an filename that included the correlation Id appended between the tar and .gz extensions. Because of this, the unpacker stream couldn't recognize the file as tar gz archive and take appropriate action and the task failed. The cause of this issue was incorrect logic for building the file name for the workflow data. To resolve the issue, a change to the runtaskrequest rule was required. The logic to build the new name for the file was updated.
This issue was tracked as Item #140759.


Dates Missing from the Unsettled Transaction Extracts

The trade date and effective date were not showing up in the unsettled transaction extracts. Actually, the effectiveDate and tradeDate come from the same field. The panel contains field Trade Date with mapping from DB field ESTAR.ESTAR_TRADED_CASH_ACTIVITY.EFFECTIVE_DT. The message mapping for the effective date was corrected so that the dates now shows up in extracts.
This issue was tracked as Item #140471.


Issue with IRS Multileg Swaps
In loading an IRS multileg swap via the EagleML generic SMF stream, the load failed with the error "Errors found during loading - Action for Contract \ Pay \ Receive" even though security aliases were returned successfully. The issue appears only for records for which in the incoming message data for SWAPS_HIST/DERIVATIVES_HIST/FIXED_INCOME_HIST/SECURITY_MASTER_DETAIL_HIST/SECURITY_MASTER_EXT_HIST/EQUITY_HIST tables is exactly the same for the existing record matched by condition in the stored procedure. Based on this logic the stored procedure does not insert the record with new effective date if the it is greater then effective date for existing record and the fields are exactly the same or empty for current table for existing and loading record.  The code was updated to resolve this issue.
This issue was tracked as Item #140536.


Option to Handle Duplicates in Entity Extracts

A new parameter was added to allow or disallow duplicate entities being extracted. With this parameter, you have more control over which entities are extracted. The parameter name is W_REMOVE_ENTITY_DUPLICATES. By default it is set to N. You can set this in the w_config_custom.inc file or you can pass in an RTR. For example, below is section of an RTR where this parameter is set to Y:

<taskParameter>
<name>RemoveEntityDuplicates</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>

This issue was tracked as Item #140584.


Error Reading Encrypted Passwords

There was a problem reading files from FTP with encrypted passwords. The code was updated to resolve this issue and encrypted passwords are no longer problematic.
This issue was tracked as Item #140666.


Malformed Output in FX Rate and Warehouse Open Lot Extracts
Both the FX Rate extract and the Warehouse Open Lot Extracts could produce malformed outputs. The root cause of this issue was incorrect grouping in the t streams. The code was updated to resolve this issue. For the reference extract, the t rules for issue fx rate and issue pc rate were updated. For the warehouse open lot extract, the extract rule was updated.
This item was tracked as #141287.

EJM

No Acknowledgement Message for Delayed ACK

After processing a message, there was no acknowledgement response. In the case of a delayed message, the acknowledgement should still show up after processing. The problem was due to the fact that the processing stream could not be found due to a missed mapping. Tag 55 was added which resolved this problem and allowed the processing stream to be located and send the ACK message.
This issue was tracked as Item #139451.


Improved TSR Messages
Previously, when either an empty file was sent or the incorrect file was sent, the TSR would contain the general error of  'unable to find status information'. The logic was improved to specifically list these issues in the TSR instead of displaying 'unable to find status information'.
This issue was tracked as Item #139747.


False success status when TST with failure message failed to be delivered via webservices

In the case when a webservices callback call failed, the overall task status was shown as "SUCCESS" instead of "FAILED". To resolve this issue, the task reporter rules were changed. For task distribution the following variables were added:

  • varDistributionErrors – contains distribution errors description
  • varDistributionStatus – can be SUCCESS or FAILED, status of distribution

The variables are set during distribution step in the reporter_dist.inc file. After distribution step is completed, the task reporter rule checks distribution status, then in case distribution is failed, the task reporter modifies TSR message (sets status to FAILED and adds errors description) and updates overall task status to "FAILED". There is ability to disable task status and TSR changes in task reporter rule. The w_ignore_distribution_errors parameter should be set to 1 to disable this feature (by default the parameter is undefined). You can set the parameter in w_config_custom.inc.
This issue was tracked as Item #140161.

I2I

Update Date Not Updated in Perf Summary

The perf summary update date was not getting updated in EagleML. When the perf calcs gets updates for a previous effective date, the update_date should be updated in both the perf_summary table and the perf_sec_returns table. However, it was only being updated in the perf sec return table. The code was updated to resolve this issue. An update mode was added to the eagle_default/in/xml/include/xml-dbdirect_wrhs_perf_delete_statement.inc include which now allows for the update date to be populated correct during updates.
This issue was tracked as Item #137677.


Issue with I2I Entity Xref Logic

The dbdirect_invar.xml rule had an incorrectly coded statement for the entity xref field. It was listed as:

<COL TAG="ENTITY_XREF" EXPRESSION=":ENTITY_XREF:" />

instead of

<COL TAG="ENTITY_XREF" EXPRESSION="|ENTITY_XREF|" />

The impact was that the entity xref was unable to be used properly. The code was updated to resolve this issue.
This issue was tracked as Item #140796.