EagleML Release Notes - August 2016
This release uses XSD Version 229.
Enhancements
EagleML
Added Ability to Skip Loading to Generic SMF
EagleML was enhanced to allow for the ability for skip the loading process to Generic SMF. This ability is controlled by a parameter in the w_config_custom.inc file. To leverage this feature, add the following code to w_config_custom.inc:
<COL TAG="skipRecordProcess" EXPRESSION="'Y'"/>
This issue was tracked as Item #131313.
Added NG Framework for EagleML Warehouse Interfaces
The NG framework was created for Warehouse feeds in EagleML. Warehouse objects did not support the NG framework and these interfaces, Positions in particular, needed this change to be consistent with other EagleML interfaces as well as client requirements. The following rules were altered to allow for the NG framework to be supported by Warehouse interfaces:
- estar\tpe\dynamic\msgcenter\eagle_default\in\xml\xml-warehouse_v2.xml
- estar\tpe\dynamic\msgcenter\eagle_default\in\xml\include\ng_exec_sp.inc
- estar\tpe\dynamic\msgcenter\eagle_default\in\xml\xslt\wrh_egl_gen_interface3.xsl
This issue was tracked as Item #141696.
Created New EagleML Extract for PACE Dictionary
The EagleML library received a new outbound interface for the PACE dictionary extract. A new outbound interface was added to accommodate the new extract. The outbound t and streaming t rules were created to support this interface.
This issue was tracked as Item #141712.
Updated Timeout Value
If there were many files in the region, the script did not have enough time to execute because 100 seconds was too short a time to execute. The timeout has now been set to 6000 seconds instead of 100 to allow the script to finish executing.
This issue was tracked as Item #142015.
Added SMF Related Data Mashup Extract
A new extract was needed that would allow SMF and related data to be grouped together into one large extract. There are scenarios when SMF data is helpful to view along with Security related data such as issuePrice and other similar objects. In the previous version of EagleML, there was no ability to gather this data in one place efficiently and setting up extracts for other feeds required using many parameters which resulted in complex control messages. A new workflow called eagle_mashup_smf was created to allow the extract of data that is pertinent to SMFs (issuerOrganization, issuePrice, and issueFxRates objects) based on the original request for an SMF extract. The objects mentioned above must be related to the SMFs extracted in the main branch via security identification in order to make the mashup extract.
This issue was tracked as Item #142260.
Added a New Block
A new block was needed in EagleML that combines Holdings, SMF and Entity extracts. A new workflow and flag were created in order to combine Holdings, SMF, and Entity extracts within a new block.
This issue was tracked as Item #142478.
Added Entity Entitlement Filter
The Entity Entitlement filter needed to be added to entity extract rules. This filter will allow all entitylist and entitycomposite extracts to execute correctly. The SQL will also contain the Entitlement filter. The Entity Entitlement filter has now been added to entity extract rules.
This issue was tracked as Item #142776.
Added Entitlement Filter to All Warehouse Trade Extracts
EagleML was enhanced to add the Entitlement filter to all Warehouse Trade extracts.This enhancement allows you to extract using the PACE_MASTERDBO.GROUP_ENTITIES table by the name of groups. It also allows for a list of entity IDs to be written into the TEMP table and be extracted as a limit of entities in the SQL data. The Entitlement filter has been added across all Warehouse Trade extracts.
This issue was tracked as Item #142826.
Fixed Partition Positions in Warehouse Extracts
On the Oracle platform, partitions such as RULESDBO and RULESDBO100 exist. Entity Entitlement did not work properly for these partitions. When a Warehouse extract was completed for an Entity not from the main partition, the extract returned NO_DATA. To resolve this issue, code has been updated to successfully extract entities that are not in main partitions.
This issue was tracked as Item #144087.
Software Fixes
EagleML
Fixed GenericLoad Save Functionality
GenericLoad did not save original file names for the HUB2SPOKE interfaces and there was no way to recover them from any variable. To resolve this issue, changes were made to task_reporter.xml to preserve the initial archive file name through all steps of the process.
This issue was tracked as Item #130840.
Fixed Processing Security Types and Pass Incoming Value Logic in Generic SMF
Tag 12 in the Generic SMF inbound rule was missing logic. To resolve this issue, logic has been added for processing security types and pass incoming value.
This issue was tracked as Item #131131.
XSLT Performance Tuning
The eagleml_issuer_org.xsl and eagleml_issuer.xsl XSLT operations were not executing efficiently. In the files eagleml_issuer_org.xsl and eagleml_issuer.xsl, there was reference to directories in the system which were given as absolute paths. These paths were changed to relative to increase the speed at which the directories can be located. This change improved overall performance.
This issue was tracked as Item #138266.
Issue with Load Issuers Sequence in eagle_wrf_generic_load_issuer_objects
In the eagle_wrf_generic_load, the WH Close Lot was not in a correct sequence with other load issuers. It was incorrectly placed before Trade References. The code was updated to resolve this issue. Now, the WH Close Lot is placed after Trade References and the new sequence is as follows:
- ISSUER and ISSUER_ORGANIZATION - concurrent load
- ISSUER_RELATIONSHIP and ISSUER_XREF - concurrent load
This issue was tracked as Item #139913.
Removed Vendor Objects from EagleML
The vendor objects did not belong in the EagleML library. These objects were DIS related and therefore should not be included in EagleML. All vendor streams were removed. Additionally, the col_tag_to_var.xslt has been moved from vendor/bbps to tpe\dynamic\msgcenter\eagle_ml-2-0_cm. Also, parse_msg_objects_data.inc and col_tag_to_var.xsl have been moved to the eagle_ml-2-0_default_cm_control_message.pkg.
This issue was tracked as Item #141796.
Corrected Misspelling in RTR
An RTR included errors where the word "Cancel" was misspelled. All instances of misspelling of the word "Cancel" have been corrected.
This issue was tracked as Item #142206.
Updated Logic for Date Filters
The CASHJOURNALEXTRACT did not produce the same output as the panel. The issue was due to the Date filter not working as intended. To resolve this issue, the logic of the out_q rule was updated. Now, the direct SQL is called PANEL. The advantage of this method is that the number of records in the extract result will now be the same as returned by the panel in Web Panel Designer.
This issue was tracked as Item #142315.
Added Support for EntityComposite in Entity Composite Filter
The EntityComposite filter did not support lists of composite accounts. Therefore, if a number was inputted into FailureTolerance with an error of less than 1%, the system reported a failure if the number of errors did not equal 0. The code was updated and the EntityComposite filter now supports lists of composite accounts. The FailureTolerance also now works as expected.
This issue was tracked as Item #142645.
Benchmark Name Not Included in Entity Extract
Benchmark details under an Entity Extract did not contain the Benchmark name. You were not able to view the Entity Detail ID name and the entityDetailIdName field was not in the XSD. The code was updated to resolve this issue. With the new addition of the entity field into the XSD, the Entity Detail ID name can now be viewed.
This issue was tracked as Item #142646.
Issue with Multileg Swaps
When a new multiLeg swap was being inserted, the underlyingCusip values carried from history for updates were not being propagated correctly and were made invalid. This error occurred when inserting a new multileg swap via EagleML where the incoming record had underlyingSecurity set to CPURNA. The underlyingCusip was updated so that when the Multileg SMF object is faced due to incorrect parsing, the result of the request will not return an empty incoming value.
This issue was tracked as Item #142673.
Issue with Special Characters in Generic SMF
When certain special characters were loaded to the generic SMF stream, they were not being translated correctly. This issue was related to Message Center's XSLT processing where Message Center replaced these characters twice for parameters. To resolve this issue, new templates were added to accommodate these special characters. Also, the output method was changed to make sure that these special characters are loaded properly.
This issue was tracked as Item #142858.
Issue with Update Timestamp
The node <updatetimestamp> was missing in the ISSUERFXRATE extract. This meant that the parameter UPDATE_DATE was being returned in the <updatetimestamp> node, not UPDATE_TIMESTAMP. To resolve this issue, the query in out_q stream was changed. UPDATE_DATE was converted to UPDATE_TIMESTAMP in the rule "t_issue_fx_rate.xml".
This issue was tracked as Item #143313.
Issue with Creating Invalid TSR File when Special Character is Present
If the special character "&" is part of the request, an invalid TSR file is created by the task_reporter. To resolve this issue, changes were made to task_reporter_generate_task_unit_old.inc. The special character "&" has been replaced by "&".
This issue was tracked as Item #143428.
Issue with Entity Extracts when Special Characters are Used
Entity extracts returned NO_DATA when special characters were used in the extracts. This error typically occurred when a colon character was used in a colon character test. To resolve this issue, a new taskParameterValue transformation was added. An additional transformation was also added to the out_q stream.
This issue was tracked as Item #143432.
Forego Dependency on Entity ID for the Remove Duplicates Filter
The logic for the Remove Duplicates filter was hinged upon the Entity ID being passed in along with the RemoveEntityDuplicates = Y. This was an issue because you may not always want to find Entity IDs while running entity extracts with the intent to exclude duplicate entities. To resolve this issue, the logic was changed in the extract_entity.xml rule which made the Remove Duplicates filter work independently of the Entity ID being passed in.
This issue was tracked as Item #143481.
Issue with Missing Backslash in control_message Stream
An error in the control_message stream occurred due to the missing backslash. To resolve this issue, backslash fix logic was added for the FTP file path creation.
This issue was tracked as Item #143526.
Entity List and Entity Composite Extract Included only PORT Entity Type
The ENTITYCOMPOSITELISTEXTRACT is used to fetch entities, but it did not include the underlying benchmarks. This means that only entities with Entity_Type PORT were returned. The code was updated to resolve this issue. The entity list and entity composite extract now includes all entity types.
This issue was tracked as Item #143539.
Issue with Investment Type and Security Type Filters with Heldonly Filter
When the Security Type and Investment Type filters were used in tandem with the Heldonly filter, the SMF extract returned a NULL value. While it is possible that the extract may return zero files, the fact that it was returning a NULL signified an issue with the performance of the combination of these filters. The code was updated to resolve this issue. The extract_filter_entity.inc file was edited to allow the filters to interact properly and allow the underlying query to properly return results rather than returning a NULL.
This issue was tracked as Item #143540.
Issue with Loading Multileg Contract Xref Ids
There was an issue with the Multileg Contract leg load where the xref identifiers were not populating. This issue occurred when the batting order was not set to default. To resolve this issue, calls were added and names of variables were updated for certain files affecting the Multileg Contract leg xref load. The xref Ids now load as expected.
This issue was tracked as Item #143541.
Issue with Position Extracts Across Partitions
Position extracts failed when entities across multiple partitions were submitted. This caused no data to be returned in Position extracts. To resolve this issue, additional sorting logic was added for when partitions are used in Position extracts.
This issue was tracked as Item #143606.
EJM
Issue with False Positives in task_reporter.xml
The task_reporter.xml has logic that will produce a successful TSR if the Failure Tolerance is set to 0 and the percentage of failures is greater than 0, but less than 1%. This means that the task_reporter.xml will report false positives and cannot account for a fraction of a percent. The logic in the task_reporter.xml has been updated and corrected to account for these scenarios. The task_reporter.xml will no longer report false positives on fractions of percents.
This issue was tracked as Item #141625.
Issue with Lookups Based on Exact Matches of Stream Names
EJM needed to be updated so that the include /eagle_ml-2-0_cm/get_msg_objects.inc can have exact match for the panel eagle/messagecenter/pan-lookup-q_msg_streams.htm. This change allows you to make lookups based on exact matches of stream names. EJM has been updated in /eagle_ml-2-0_cm/get_msg_objects.inc to implement these changes.
This issue was tracked as Item #141704.
Improved Handling of Incomplete TSR
When the EagleML Web client received certain types of SOAP messages, errors were generated. These errors occurred when an incomplete ACK message was received by a valid SOAP message in response to the EagleML Web client. A check was added to resolve this issue. If a user receives an ACK message without certain elements, this check will prevent errors from generating.
This issue was tracked as Item #142018.
Issue with EJM Not Skipping Records Under Certain Conditions
EJM was not properly skipping records based on certain conditions.This issue was due to incomplete logic around the FOREACH condition. The file /eagle_ml-2-0_cm/get_stream_folder.inc was updated to add a condition to properly handle records when the STREAM_STATUS is null.
This issue was tracked as Item #142726.
Issue with Workflow Retrieval State
When a workflow state was retrieved with a subsequent update, the encoding resulted in a very long string that crashes Message Center. Every time a "\" symbol was sent to the database, it was replaced by "\5C", creating long strings that crashed Message Center. To resolve this issue, the colon logic and backslash replacement logic were changed to eliminate these Message Center crashes.
This issue was tracked as Item #142768.
Issue with varFileName in Extracts
In extracts, if certain parameters were input before being passed to the code, the MsgExchange GetFile failed. This issue was due to the variable varFileName being set to a value that caused the MsgExchange to fail. To resolve this issue, a new FILE_TYPE was added to the code to allow for the variable varFileName to execute properly based on any value.
This issue was tracked as Item #142782.
I2I
Added Ability to Apply Entity Filter <onlyheld> to Extracts
The ability apply the Entity Filter <onlyheld> was added to the newer extracts of:
- FACTOREXTRACT
- RATINGSEXTRACT
- SCHEDULEEXTRACT
- ISSUERORGEXTACT
- GENISSUEREXTACT
- GENCORPACTIONEXTRACT
The ability to filter this way is vital for the Instance to Instance functionality.
This issue was tracked as Item #143170.