EagleML Release Notes - July 2015

This release uses XSD Revision 207.

Enhancements

EagleML

New DeltaBased Parameter for Code Categories, Code Values and Code Translation Extracts
A new parameter has been added to the extracts for code categories, code values and code translations. This new parameter named <deltaBased> allows a document to be created in the cmw/extracts/delta folder of the respective environment which contains the time stamp representing when the execution of the extract was successfully completed in the out_q stream. Each type of object (code category, code value and code translations) will have its unique folder. These folders are named extract_code_data, extract_code_value_data, and extract_code_translation_data and are created when the extract with the <deltaBased> parameter set to Y is carried out for the first time. Subsequent runs will simply update the content of the file with a new time stamp and only extract records that have been loaded after the time stamp of the previous extract.
This issue was tracked as Item #129675.


Added Reference to DIS Include File to W_Config_Custom
A link to the w_config_dis.inc file was added to the w_config_custom.inc file to allow the w_config_custom.inc file to have access to the core parameters of the w_config_dis.inc file without the risk of the entire w_config_custom.inc client file being overwritten. This change is a precaution for DIS clients to avoid overwriting customer data.
This issue was tracked as Item #130232.


New EagleML Vendor Cash Flow Interface
The new EagleML interface Vendor Cash Flows has been created. In support of this interface a new rule has been developed and linked to the eagle_ml-2-0_default_in_xml_reference stream to facilitate the load and delete of Vendor Cash Flow objects. There are two ways to execute the load or delete, through a stored procedure or through a panel. The selection of either method is dictated by the accountingValidationFlag field. These new developments are key additions to the new accounting tables and allow a direct way to load and delete data.
This issue was tracked as Item #130741.


Support Added for Security_Analtyic_User Table
Support has been created for the new SECURITY_ANALYTICS_USER table. This table now has an accompanying load and extract function. The inbound portion is handled by the eagle_ml-2-0_default_in_xml_reference stream and the extracts (outbound portion) is supported by the eagle_ml-2-0_default_cm_control_message stream. Data can now be loaded into the table via the above mentioned reference stream and extracted through a control message sent to the control message stream.
This issue was tracked as Item #131074.


Added Support for WarehousePendingTrade
A new EagleML warehouse interface has been created. The eagle_ml-2-0_default_in_xml_warehouse_preproc stream is used to load data files to this new interface. This new interface supports object of the type Warehouse Pending Trade. It is available on all the EagleML environments and has received xslt support.
This issue was tracked as Item #131092.


AccountingValidationFlag Has Been Added to All Entity Objects
The <accountingValidationFlag> field has been added to all entity objects in order to regulate whether or not the panel will be called from the rule which adds and updates entities. A value of "Y" in the The <accountingValidationFlag> field will call the Add/ChangeEntity panel in Eagle  version 2015 R2 or higher and will call either AddEntity or ChangeEntity (depending on the action being performed) in any version earlier than 2015R2. In addition, a number of new fields have been included in the Add/ChangeEntity" panel.
This issue was tracked as Item #131441.


Support for the Load and Extract Warehouse Position Analytics UDFs
The inbound and outbound rule were enhanced with new support for the load and extract of warehouse positions and numerous position analytics UDFs. The rule has been equipped with proper mapping to support load to and extracts from the position analytics user tables.
This issue was tracked as Item #131816.

EJM

New EJM Option to Allow Processing Loads in Parallel
Previously during the load of warehouse objects the eagle_ml-2-0_default_in_xml_warehouse_preproc stream would handle each file sent sequentially due to the way the stream was set up. It would create a new name for each record that it would process and add these to the system one at a time. This enhancement increases the efficiency of the load process by allowing all the files in the same control message to be process simultaneously. This can be accomplished through an optional <parallelExecution> parameter in the control message. When the value of this field is set to "Y" the files in the control message will be processed as one file set, thus opening the possibility of parallel processing by processors. The files within the file set are evenly distributed among all operating processors within the environment.
This issue was tracked as Item #131330.

Software Fixes

EagleML

Empty ScheduleType Field Caused Edit Check
There was a problem with the <scheduleType> field being empty and causing an edit check to be triggered when sending SMFs to the eagle_ml-2-0_default_in_xml_smf_mi stream. The reason for the empty field was a result of a lack of mapping in the xslt. The <scheduleType> field has been mapped in the eagleml_smf_schedule_delete.xsl file to properly fill the parameter and avoid the edit check being triggered.
This issue was tracked as Item #130763.


Issue with Position Extract When Position_Detail Null
During a Positions Extract, individual records should be split up with separate <warehousePosition> tags. The main grouping key

POSITION_DETAIL.POSITION_DETAIL_ID

allows POSITION_DETAIL to be null, yet the extract combines several unique records under one such tag. To avoid this error, the grouping key has been changed to:

POSITION.POSITION_ID + POSITION_DETAIL.POSITION_DETAIL_ID

This change makes it possible for the extract to be generated with separate <warehousePosition> tags for each unique record regardless of the POSITION_DETAIL being null.
This issue was tracked as Item #130801.


Support for Special Characters in Warehouse Inbound Interfaces
The Warehouse inbound interfaces now support XML special characters, including:

<

>

&

All rules connected to the warehouse interface have been updated to support these special character.
Note: If you are not using IWS, you must ensure that these special characters are properly encoded.
This issue was traced as ITEM #128634


Issues with Filters and DeltaBased Mode in Financial Event Extract
The outbound rule for the financial event object type had a few shortcomings in the form of missing filters and inability to support DeltaBased mode. It is now possible to filter the extract by the effective and the update dates as well as by security type, investment type, message or event type and by security identifiers for the objects of type SMF, PST and entity along with other entity base objects such as entity lists. DeltaBased mode is now available as a task parameter as well and will allow the user to extract only files sent after the processing of the previous control message with a DeltaBased mode task parameter in it. These changes adds a substantial amount of versatility to the accounting trades and financial event extraction process.
This issue was tracked as Item #130802.


Update Source Issue in FxRate Rule
When sending Fx rate records for updating, the field <updateSource> would not be properly reflected in the database. Regardless of the value in this field in the incoming file, the database would record the update source as the default value MCADMIN. The ref_fxrate.xml rule has been edited to allow the database to properly display the update source which is derived from the incoming file.
This issue was tracked as Item #130842.


Issues with Underlying Security Alias Column in the Derivatives table
When loading an SMF without an existing underlying SMF, the underlying security alias column would be left blank. This issue was connected to two issues with the smf_generic.rpf rule. First, code responsible for generating the underlying SMF for a new SMF was incorrectly commented out. Additionally the distribution of the underlying SMF was coded after the distribution of the regular SMF. Changes were made to ensure that the underlying SMF is generated and sent to the stream before the regular SMF is. This way the underlying security alias column will be filled since the underlying SMF will already exist in the system by the time the SMF file is processed.
This issue was tracked as Item #131086.


Updated the Order of Include Files in the Generic SMF Rule
The generic SMF rule contained a set of two include files that were in the opposite order than they should be. The original order had the security_resolution_full.inc file after the sml-smf_generic_columns_after.inc file. This was a problem because the resolution logic should have been accessed first and then the details of the columns_after file would handle any special resolution scenarios. The rule file was edited to ensure that the two include files were placed in the proper order.
This issue was tracked as Item #131351.


Issue When Xref and Xref Type Pairing Were Not Listed as the First Pairing
When adding a record where the primary asset Id and primary asset type were taken from xref Id and xref type, there were issues if the information was stored in any xref and xref type pairing other than the first one. The rule would not pick up on the pairings that came after any empty pairing and therefore would try to add the record instead of successfully finding that such a record exists in the database and updating it. The dbdirect_sec_resolution.inc file was modified to catch any xref and xref type pairing and map the proper primary asset Id and primary asset type from that pairing.
This issue was tracked as Item #131555.


Match All Security Resolution Could Display Incorrect Status With Ratings Load
When trying to update ratings in the accounting database with the Match All mode set as the security resolution option, the database showed no update despite the stream returning a success status. The rule has been fixed to ensure that the Match All security resolution will not return a false success status and will return the proper error when the record is not found and updated.
This issue was tracked as Item #131769.


UPDATE_SOURCE Issue on Corporate Action Records
When updating Corporate Action records in the database the UPDATE_SOURCE field would get overwritten to the default value of MCADMIN. The issue was due to an incorrect mapping in the eagle/star/reference/pan-addfactorca.htm panel. This panel has been overlaid with proper mapping which does not allow the update source to be overwritten through the panel lookup. The inbound rule has now been updated to allow the update source tag to not be overwritten when updating records.
This issue was tracked as Item #131800.


Issue With Sequencer Stream if Time is 00:00
The eagle_ml-2-0_default_cm_sequencer stream had an issue with automatic execution at exactly midnight. When it registered the time as 0 (which occurs at midnight), the start and end suspension time periods were incorrectly compared by the stream. This improper comparison results in the vSuspensionPeriodFlag being set to 1 which in turn halted the stream's processing. The eagle_ml-2-0_cm\out\tagvalue\seq_get_params.inc rule has been changed to handle this particular scenario and make sure that the vSuspensionPeriodFlag is not mistakenly set to one allowing the process to continue correctly when the time is 00:00.
This issue was tracked as Item #132080.


MSSQL Only - Issue with Schedule Reference
On the MSSQL platform only, the secm_date_update.inc rule had two errors that caused the dates not to be properly updated. When sending a message to the eagle_ml-2-0_default_in_xml_reference stream, you would get the error:

Invalid column name sysdate

The code was updated so that the improper field sysdates were replaced with the function getdate(). The stream now works correctly and the dates are updated as expected.
This issue was tracked as Item #132169.


Ratings_float Fields Were Not Being Updated
The element rating_float1 could be loaded to the database but was not supported by the inbound reference rule. The field called FORCEADDTAGS in that rule has been edited to allow the loading of rating_float1-3 and rating_char1-20.
This issue was tracked as Item #132178.


Added Web Sevices Delivery Method for Multileg SMFs
There was no support for multileg SMFs in the Web Services component. Changes were made to the inbound SMF rule so that it is now possible to load multileg SMF files through Web Services. This adds a complimentary delivery method for multileg SMFs to work alongside the typical file delivery method.
This issue was tracked as Item #132120.


Issue with SMF Extract History Only Flag Mode
The extract of SMFs through the use of the <historyOnlyFlag> field within the control message caused issues. The extract pulled every record from the database that had a valid xreference identified and extracted it. The rule has been updated to only allow the newest security in the database with a valid xreference identifier to be extracted with the control message unless there are other fields that specify otherwise. This change makes it much easier to control the number of records being extracted with each control message.
This issue was tracked as Item # 132523.

EJM

Improved Load and Extract Workflows
The load and extract rules have been reworked and new functionality has also been added. The new names for these workflows are eagle_wrf_generic_load and eagle_wrf_generic_extract. The new workflows are now more efficient and support all the financial objects within the EagleML structure.

Note: The load workflow does not function with objects that only have an outbound (extract) rule in Message Center.
This issue was tracked as Item #130346.


Created New Monitor Stream For Cm_reader 
A new stream has been created to monitor the cm_reader stream. This monitoring stream acts as a safety net and checks every 10 minutes to make sure that the cm_reader stream is active and is not hung.

Note: In order to leverage this new monitor stream, you must be an Eagle ACCESS client and have at least the EagleML July monthend package in your environment. Also, you will need to work with your Eagle representative regarding configurations.
This issue was tracked as Item #131079.


Cm_reader Now Queries Eagle ACCESS Database Instead of Polling the FTP Site
Previously the FTP site was constantly polled at regular intervals for all control messages that had been processed through FTP. This process experienced some problems. During times of high volume or poor connection, the entire system would be dampened as a result of this polling. The cm_reader stream was enhanced to eliminate the necessity of polling the FTP site every [x] amount of seconds. The Eagle ACCESS SiteSee database which records every control message sent to the system and stores the record in a corresponding table is now checked instead. This enhancement greatly reduces the stress on the system and improves performance.

Note: In order to leverage this new feature, you must be an Eagle ACCESS client and have at least the EagleML July monthend package in your environment. Also, you will need to work with your Eagle representative regarding configurations.
This issue was tracked as Item #131094.


Issue with Generic Load Workflow Not Properly Completing
There was a problem with the generic load workflow getting hung up and not properly completing. This behavior was caused by an inconsistency in the custom_config file. The FTP file directories should have a slash at the end of the file path, but one of the paths (CMW/in) was missing this final slash. This missing slash created problems since the Eagle Job Manager waited for the load file to be sent to the stream when it should have automatically taken it from the cmw/in folder in the FTP. The config file was updated with the proper directory syntax to resolve this issue. In addition, the path name of the load file would be displayed with a duplicated slash in the stream. That issue has also been addressed.
This issue was tracked as Item #131852.


Update Issue with Issuer Organization  Interface
The stored procedure would not be called from within the rule if the issuer organization object was sent with the intention of an update. The rule previously only had the capability to add such object. The rule was updated to now allow for the capability for updating through the stored procedure. This makes it possible to update records directly through the rule.
This issue was tracked as Item #132305.