EagleML Release Notes - August 2016

This release uses XSD Version 229.

There was an issue identified with the automated installation process for August 2016.  During this process there is a record placed in the eagle_product_version table to reflect the EagleML version installed. The issue is that the next instance of this table was not properly updated afterwards. This issue will be corrected in the next EagleML release. If you installed August 2016 release using the automated installation process though, you will need take one additional step after the install to resync up the next instance. After the install is complete, log into your Eagle environment. Bring up the Unique Id screen (which you can find in Metadata Center or System Mgt Center, depending on your Eagle version) and select the row for EAGLE_PRODUCT_VERSION and then click on the Update button.
Note: The August 2016 EagleML release was repacked to address an issue with the manual install process. Please ensure you have the latest package if you plan to install via the manual method.
There are four overlay files that are currently packaged as part of the EagleML monthly release. Please refer to About EagleML Overlays for specific details and impact to you.

Enhancements

EagleML

Added Ability to Skip Loading to Generic SMF

EagleML was enhanced to allow for the ability for skip the loading process to Generic SMF. This ability is controlled by a parameter in the w_config_custom.inc file. To leverage this feature, add the following code to w_config_custom.inc:

<COL TAG="skipRecordProcess" EXPRESSION="'Y'"/>

This issue was tracked as Item #131313.


Added NG Framework for EagleML Warehouse Interfaces

The NG framework was created for Warehouse feeds in EagleML. Warehouse objects did not support the NG framework and these interfaces, Positions in particular, needed this change to be consistent with other EagleML interfaces as well as client requirements. The following rules were altered to allow for the NG framework to be supported by Warehouse interfaces:

  • estar\tpe\dynamic\msgcenter\eagle_default\in\xml\xml-warehouse_v2.xml
  • estar\tpe\dynamic\msgcenter\eagle_default\in\xml\include\ng_exec_sp.inc
  • estar\tpe\dynamic\msgcenter\eagle_default\in\xml\xslt\wrh_egl_gen_interface3.xsl

This issue was tracked as Item #141696.


Created New EagleML Extract for PACE Dictionary

The EagleML library received a new outbound interface for the PACE dictionary extract. A new outbound interface was added to accommodate the new extract. The outbound t and streaming t rules were created to support this interface.
This issue was tracked as Item #141712.


Updated Timeout Value

If there were many files in the region, the script did not have enough time to execute because 100 seconds was too short a time to execute. The timeout has now been set to 6000 seconds instead of 100 to allow the script to finish executing.
This issue was tracked as Item #142015.


Added SMF Related Data Mashup Extract

A new extract was needed that would allow SMF and related data to be grouped together into one large extract. There are scenarios when SMF data is helpful to view along with Security related data such as issuePrice and other similar objects. In the previous version of EagleML, there was no ability to gather this data in one place efficiently and setting up extracts for other feeds required using many parameters which resulted in complex control messages. A new workflow called eagle_mashup_smf was created to allow the extract of data that is pertinent to SMFs (issuerOrganization, issuePrice, and issueFxRates objects) based on the original request for an SMF extract. The objects mentioned above must be related to the SMFs extracted in the main branch via security identification in order to make the mashup extract.
This issue was tracked as Item #142260.


Added a New Block

A new block was needed in EagleML that combines Holdings, SMF and Entity extracts. A new workflow and flag were created in order to combine Holdings, SMF, and Entity extracts within a new block.
This issue was tracked as Item #142478.


Added Entity Entitlement Filter

The Entity Entitlement filter needed to be added to entity extract rules. This filter will allow all entitylist and entitycomposite extracts to execute correctly. The SQL will also contain the Entitlement filter. The Entity Entitlement filter has now been added to entity extract rules.
This issue was tracked as Item #142776.


Added Entitlement Filter to All Warehouse Trade Extracts

EagleML was enhanced to add the Entitlement filter to all Warehouse Trade extracts.This enhancement allows you to extract using the PACE_MASTERDBO.GROUP_ENTITIES table by the name of groups. It also allows for a list of entity IDs to be written into the TEMP table and be extracted as a limit of entities in the SQL data. The Entitlement filter has been added across all Warehouse Trade extracts.
This issue was tracked as Item #142826. 


Fixed Partition Positions in Warehouse Extracts

On the Oracle platform, partitions such as RULESDBO and RULESDBO100 exist. Entity Entitlement did not work properly for these partitions. When a Warehouse extract was completed for an Entity not from the main partition, the extract returned NO_DATA. To resolve this issue, code has been updated to successfully extract entities that are not in main partitions.
This issue was tracked as Item #144087.

Software Fixes

EagleML 

Fixed GenericLoad Save Functionality

GenericLoad did not save original file names for the HUB2SPOKE interfaces and there was no way to recover them from any variable. To resolve this issue, changes were made to task_reporter.xml to preserve the initial archive file name through all steps of the process.
This issue was tracked as Item #130840.


Fixed Processing Security Types and Pass Incoming Value Logic in Generic SMF

Tag 12 in the Generic SMF inbound rule was missing logic. To resolve this issue, logic has been added for processing security types and pass incoming value.
This issue was tracked as Item #131131. 


XSLT Performance Tuning

The eagleml_issuer_org.xsl and eagleml_issuer.xsl XSLT operations were not executing efficiently. In the files eagleml_issuer_org.xsl and eagleml_issuer.xsl, there was reference to directories in the system which were given as absolute paths. These paths were changed to relative to increase the speed at which the directories can be located. This change improved overall performance.
This issue was tracked as Item #138266.


Issue with Load Issuers Sequence in eagle_wrf_generic_load_issuer_objects

In the eagle_wrf_generic_load, the WH Close Lot was not in a correct sequence with other load issuers. It was incorrectly placed before Trade References. The code was updated to resolve this issue. Now, the WH Close Lot is placed after Trade References and the new sequence is as follows:

  1. ISSUER and ISSUER_ORGANIZATION - concurrent load
  2. ISSUER_RELATIONSHIP and ISSUER_XREF - concurrent load

This issue was tracked as Item #139913.


Removed Vendor Objects from EagleML

The vendor objects did not belong in the EagleML library. These objects were DIS related and therefore should not be included in EagleML. All vendor streams were removed. Additionally, the col_tag_to_var.xslt has been moved from vendor/bbps to tpe\dynamic\msgcenter\eagle_ml-2-0_cm. Also, parse_msg_objects_data.inc and col_tag_to_var.xsl have been moved to the eagle_ml-2-0_default_cm_control_message.pkg.
This issue was tracked as Item #141796.


Corrected Misspelling in RTR

An RTR included errors where the word "Cancel" was misspelled. All instances of misspelling of the word "Cancel" have been corrected.
This issue was tracked as Item #142206.


Updated Logic for Date Filters

The CASHJOURNALEXTRACT did not produce the same output as the panel. The issue was due to the Date filter not working as intended. To resolve this issue, the logic of the out_q rule was updated. Now, the direct SQL is called PANEL. The advantage of this method is that the number of records in the extract result will now be the same as returned by the panel in Web Panel Designer.
This issue was tracked as Item #142315.


Added Support for EntityComposite in Entity Composite Filter

The EntityComposite filter did not support lists of composite accounts. Therefore, if a number was inputted into FailureTolerance with an error of less than 1%, the system reported a failure if the number of errors did not equal 0. The code was updated and the EntityComposite filter now supports lists of composite accounts. The FailureTolerance also now works as expected.
This issue was tracked as Item #142645.


Benchmark Name Not Included in Entity Extract

Benchmark details under an Entity Extract did not contain the Benchmark name. You were not able to view the Entity Detail ID name and the entityDetailIdName field was not in the XSD. The code was updated to resolve this issue. With the new addition of the entity field into the XSD, the Entity Detail ID name can now be viewed.
This issue was tracked as Item #142646.


Issue with Multileg Swaps

When a new multiLeg swap was being inserted, the underlyingCusip values carried from history for updates were not being propagated correctly and were made invalid. This error occurred when inserting a new multileg swap via EagleML where the incoming record had underlyingSecurity set to CPURNA. The underlyingCusip was updated so that when the Multileg SMF object is faced due to incorrect parsing, the result of the request will not return an empty incoming value.
This issue was tracked as Item #142673.


Issue with Special Characters in Generic SMF

When certain special characters were loaded to the generic SMF stream, they were not being translated correctly. This issue was related to Message Center's XSLT processing where Message Center replaced these characters twice for parameters. To resolve this issue, new templates were added to accommodate these special characters. Also, the output method was changed to make sure that these special characters are loaded properly.
This issue was tracked as Item #142858.


Issue with Update Timestamp

The node <updatetimestamp> was missing in the ISSUERFXRATE extract. This meant that the parameter UPDATE_DATE was being returned in the <updatetimestamp> node, not UPDATE_TIMESTAMP. To resolve this issue, the query in out_q stream was changed. UPDATE_DATE was converted to UPDATE_TIMESTAMP in the rule "t_issue_fx_rate.xml".
This issue was tracked as Item #143313.


Issue with Creating Invalid TSR File when Special Character is Present

If the special character "&" is part of the request, an invalid TSR file is created by the task_reporter. To resolve this issue, changes were made to task_reporter_generate_task_unit_old.inc. The special character "&" has been replaced by "&amp".
This issue was tracked as Item #143428.


Issue with Entity Extracts when Special Characters are Used

Entity extracts returned NO_DATA when special characters were used in the extracts. This error typically occurred when a colon character was used in a colon character test. To resolve this issue, a new taskParameterValue transformation was added. An additional transformation was also added to the out_q stream.
This issue was tracked as Item #143432.


Forego Dependency on Entity ID for the Remove Duplicates Filter

The logic for the Remove Duplicates filter was hinged upon the Entity ID being passed in along with the RemoveEntityDuplicates = Y. This was an issue because you may not always want to find Entity IDs while running entity extracts with the intent to exclude duplicate entities. To resolve this issue, the logic was changed in the extract_entity.xml rule which made the Remove Duplicates filter work independently of the Entity ID being passed in.
This issue was tracked as Item #143481.


Issue with Missing Backslash in control_message Stream

An error in the control_message stream occurred due to the missing backslash. To resolve this issue, backslash fix logic was added for the FTP file path creation.
This issue was tracked as Item #143526.


Entity List and Entity Composite Extract Included only PORT Entity Type

The ENTITYCOMPOSITELISTEXTRACT is used to fetch entities, but it did not include the underlying benchmarks. This means that only entities with Entity_Type PORT were returned. The code was updated to resolve this issue. The entity list and entity composite extract now includes all entity types.
This issue was tracked as Item #143539.


Issue with Investment Type and Security Type Filters with Heldonly Filter

When the Security Type and Investment Type filters were used in tandem with the Heldonly filter, the SMF extract returned a NULL value. While it is possible that the extract may return zero files, the fact that it was returning a NULL signified an issue with the performance of the combination of these filters. The code was updated to resolve this issue. The extract_filter_entity.inc file was edited to allow the filters to interact properly and allow the underlying query to properly return results rather than returning a NULL.
This issue was tracked as Item #143540.


Issue with Loading Multileg Contract Xref Ids

There was an issue with the Multileg Contract leg load where the xref identifiers were not populating. This issue occurred when the batting order was not set to default. To resolve this issue, calls were added and names of variables were updated for certain files affecting the Multileg Contract leg xref load. The xref Ids now load as expected.
This issue was tracked as Item #143541.


Issue with Position Extracts Across Partitions

Position extracts failed when entities across multiple partitions were submitted. This caused no data to be returned in Position extracts. To resolve this issue, additional sorting logic was added for when partitions are used in Position extracts.
This issue was tracked as Item #143606.

EJM

Issue with False Positives in task_reporter.xml

The task_reporter.xml has logic that will produce a successful TSR if the Failure Tolerance is set to 0 and the percentage of failures is greater than 0, but less than 1%. This means that the task_reporter.xml will report false positives and cannot account for a fraction of a percent. The logic in the task_reporter.xml has been updated and corrected to account for these scenarios. The task_reporter.xml will no longer report false positives on fractions of percents.
This issue was tracked as Item #141625.


Issue with Lookups Based on Exact Matches of Stream Names

EJM needed to be updated so that the include /eagle_ml-2-0_cm/get_msg_objects.inc can have exact match for the panel eagle/messagecenter/pan-lookup-q_msg_streams.htm. This change allows you to make lookups based on exact matches of stream names. EJM has been updated in /eagle_ml-2-0_cm/get_msg_objects.inc to implement these changes.
This issue was tracked as Item #141704.


Improved Handling of Incomplete TSR

When the EagleML Web client received certain types of SOAP messages, errors were generated. These errors occurred when an incomplete ACK message was received by a valid SOAP message in response to the EagleML Web client. A check was added to resolve this issue. If a user receives an ACK message without certain elements, this check will prevent errors from generating.
This issue was tracked as Item #142018.


Issue with EJM Not Skipping Records Under Certain Conditions

EJM was not properly skipping records based on certain conditions.This issue was due to incomplete logic around the FOREACH condition. The file /eagle_ml-2-0_cm/get_stream_folder.inc was updated to add a condition to properly handle records when the STREAM_STATUS is null. 
This issue was tracked as Item #142726.


Issue with Workflow Retrieval State

When a workflow state was retrieved with a subsequent update, the encoding resulted in a very long string that crashes Message Center. Every time a "\" symbol was sent to the database, it was replaced by "\5C", creating long strings that crashed Message Center. To resolve this issue, the colon logic and backslash replacement logic were changed to eliminate these Message Center crashes.
This issue was tracked as Item #142768.


Issue with varFileName in Extracts

In extracts, if certain parameters were input before being passed to the code, the MsgExchange GetFile failed. This issue was due to the variable varFileName being set to a value that caused the MsgExchange to fail. To resolve this issue, a new FILE_TYPE was added to the code to allow for the variable varFileName to execute properly based on any value.
This issue was tracked as Item #142782.

I2I

Added Ability to Apply Entity Filter <onlyheld> to Extracts

The ability apply the Entity Filter <onlyheld> was added to the newer extracts of:

  • FACTOREXTRACT
  • RATINGSEXTRACT
  • SCHEDULEEXTRACT
  • ISSUERORGEXTACT
  • GENISSUEREXTACT
  • GENCORPACTIONEXTRACT

The ability to filter this way is vital for the Instance to Instance functionality.
This issue was tracked as Item #143170.