EagleML Release Notes - February 2018

This release uses XSD Revision 285

There are four overlay files that are currently packaged as part of the EagleML monthly release. Please refer to About EagleML Overlays for specific details and impact to you.

Enhancements

Enterprise

EagleML

Reduced Joined Tables in Entity Extract
Request to reduce joined tables in Entity extract.
Developed functionality to use profiles to reduce joined tables for the Entity extract.

This issue was tracked as Item #144545.

Added New Filters for Warehouse Performance Extract

Request to add filters for Dictionary Name, Performance Frequency Code and Return Level.
The ability to use these filters was introduced in the outbound warehouse performance interface.

Added new filters:

dictionaryname - one or several values, separated by commas

perffrequency - one or several values, separated by commas

levelofreturns - can be “Total”, “Security”, “Sector” or “ALL”. Or any combinations of these values, separated by commas.

<taskParameter>

<name>dictionaryname</name>

<dataType>S</dataType>

<value>eglPrfPc-Total Level Model,1</value>

</taskParameter>

<taskParameter>

<name>perffrequency</name>

<dataType>S</dataType>

<value>D,A,2</value>

</taskParameter>

<taskParameter>

<name>levelofreturns</name>

<dataType>S</dataType>

<value>Security,total,sector</value>

</taskParameter>

This issue was tracked as Item #154846.

Support Business Day Processing Between Date Ranges
A data extract within a date range needs to have the ability to exclude non business days.
A new BusinessCalendarSource parameter was added. This parameter’s value will be assigned to the SourceName and select the calendar used. The date ranges can be specified by using date parameters such as effectivedate, fromeffectivedate, and toeffectivedate. The date rule for EffectiveDate can also be used by passing the date rule parameters. In case there are no business days in the selected date range the user should expect the following error: Business dates are not found for Source = [value passed in BusinessCalendarSource].

<taskParameter>

<name>BusinessCalendarSource</name>

<dataType>S</dataType>

<value>EAGLE PACE</value>

</taskParameter>

This issue was tracked as Item #154847.

Support PreserveNull Flag for Generic Entity
The generic entity inbound interface did not support the preserveNull flag.
The xml-ent_entitygeneric.xml rule was changed to support the preserveNull flag.

This issue was tracked as Item #154891.

Added Stream Name in processing information for each file (Consolidated TSR)
There was a request to add the corresponding stream name for each processed file into the consolidated TSR.
Two new nodes were added for each processed file in order to reflect the processing stream name in the consolidated TSR:
<taskType>Stream</taskType>
<taskName>eagle_ml-2-0_default_in_xml_warehouse_preproc</taskName>

<taskParameter>

<name>CreateConsolidatedTSR</name>

<dataType>S</dataType>

<value>Y</value>

</taskParameter>

This issue was tracked as Item #156362.

Refined the WRHSNAVMULTIEXTRACT to distinguish the use of the enable calculations and IncludeReportDetails
To produce additional elements within the WRHSNAVMULTIEXTRACT, two parameters were required. The logic was modified to only require the IncludeReportDetails parameter to publish out the transferAgentMillRate, rawNavChange and reportingNavChange elements.

Setting the value of enablecalculations to Y will maintain the behavior to round the millrate, HOLDINGDBO.NAV_MULTI.MILL_RATE, to 9 decimals. The default is N.

<taskParameter>

<name>IncludeReportDetails</name>

<dataType>S</dataType>

<value>Y</value>

</taskParameter>

<taskParameter>

<name>enablecalculations</name>

<dataType>S</dataType>

<value>Y</value>

</taskParameter>

This issue was tracked as Item #157092.

Created Inbound and Outbound Interfaces for User Group and Modified Interfaces for User Objects
The new XSD group of User objects have been created and added to the XSD schema.
The User object was moved from the Reference to the User object group.
Created new inbound stream for User objects.
Created new rules:
xml-user_user.xml - inbound rule User,
extract_user_group.xml - Q rule User,
t_user_user_streaming.xml - T Streaming rule User,
xml-user_group.xml - inbound rule User Group
extract_user_group.xml - Q rule User Group,
t_user_group_streaming.xml - T Streaming rule User Group

Inbound rule User:
- allows addition, updating and deletion of User records
- has 2 modes: load with panel or with stored procedure. The record will be loaded with stored procedure if Business Group Name and/or Application Group Name is null.

Inbound rule User Group:
- allows addition and deletion of one or more Users to a Business or Application group or to the Center Role
- has 3 modes: add/delete User(s) to Business group, add/Delete User(s) to Application group, assign/remove User(s) to Center Role.

Outbound rules User:
- extract from the DB table: PACE_MASTERDBO.PACE_USERS
- shows Groups of User in repeating mode <group>
- available filters: User, Filter, Date, Common
- supports streaming mode

- EagleML Object is USEREXTRACT

Outbound rules User Group:
- extract from the DB tables: PACE_MASTERDBO.PACE_USER_GROUPS, PACE_MASTERDBO.STARSEC_USER_GROUP, PACE_MASTERDBO.EGL_CENTER_ROLE
- shows Users in the Group in the repeating node <member>
- available filters: User, Filter, Date, Common
- supported streaming mode
- EagleML Object is USERGROUPEXTRACT

Also created new include extract_filter_user.inc to contain the following User filters:
:userid: - filter for the db field PACE_USERS.USERNAME
:businessgroupname: - filter for the db field PACE_USER_GROUPS.GROUP_NAME
:usergroupname: or :applicationgroupname: - filter for the db field STARSEC_USER_GROUP.STAR_GROUP_NAME
:centerrolename: - filter for the db field EGL_CENTER_ROLE.CENTER_ROLE_NAME
:centercode: - filder for the db field EGL_CENTER_ROLE.CENTER_CODE
:accountstate: - filter for the db field PACE_USERS.ACCOUNT_STATE

All rules are supported on Oracle and MSSQL environments.

This issue was tracked as Item #158230.

EagleML - installer should perform restart correctly after binds installation

The installer should be able to perform a restart correctly after installing binds.

The binds were updated to a cache to allow this behavior. The eagle_wrf_deploy workflow was changed for this functionality in the following ways:

- added a stop mccommandexecutor

- sending commands to restart messagecenter on all instances

- added start mccommandexecutor after successful restart

This issue was tracked as Item #158365.

New Approach Implemented for the Bind and Include in the Inbound Issuer Rules
NG works very slowly, because a dynamic stored procedure is generated for each record.
The NG process was updated and currently NG and CORE use the same logic for loading.
The performance for both methods is similar.
The following inbound rules received these changes:
xml-ref_issuer_xref.xml (158831)
xml-ref_issuerrelationship.xml (158830)
xml-ref_issuerorganization.xml (158829)

All meta data was added in XSD for these objects.
This meta data is used to generate includes with the code which create DBML request automatically on installation EagleML step(Via Installer)

This issue was tracked as Item #158829.

Added accounting option for UNAPPROVEDVARMARGINEXTRACT
There are different tables used to retrieve the data for the UNAPPROVEDVARMARGINEXTRACT data. In regions with version 13.1 or lower installed, the extract was failing to produce the proper data set.
An accounting option was added for the UNAPPROVEDVARMARGINEXTRACT. This option allows the extract to use the stored procedure which avoids the table issue in the environments with version 13.1 or lower.

<taskParameter>
<name>accountingvalidationflag</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>

This issue was tracked as Item #158932.

EJM

Added Support for Datamask in prf File to manage TSR alerts
Added the ability to use the datamask option in the profile for TSR alerts.
Added support for the datamask in the prf file. File format was extended by updating the read_task_info.inc.

This issue was tracked as Item #154072.

Support Execution of OLAP Pace Events via EJM with Overrides
Added support to execute OLAP Pace Events via EJM with the following override parameters:
- Entity/Multiple Entities
- Date Range (From/To Date)
- Date Rule
Parameters overrides can be specified in the RTR. For now this functionality is only supported in ORACLE.
The ability to configure the user name for the user who is executing the PACE Event.

<taskParameter>

<name>ActionType</name>

<dataType>S</dataType>

<value>PACEEvent</value>

</taskParameter>

<taskParameter>

<name>StreamName</name>

<dataType>S</dataType>

<value>eagle_ml-2-0_default_cm_execute_pace_event</value>

</taskParameter>

<taskParameter>

<name>PACEEventName</name>

<dataType>S</dataType>

<value>sampleeventname</value>

</taskParameter>

<taskParameter>

<name>PACEEventType</name>

<dataType>S</dataType>

<value>sampleeventtype</value>

</taskParameter>

This issue was tracked as Item #156453.

Enhanced Capabilities of IWS Stream to Move Files
It had not been possible to use the msgexchange functionality to rename files.
The eis_rename_file.xml rule was changed to allow renaming files by using the msgexchange in this rule. It is now possible to use the following parameters with this functionality:
UseMsgexchange=Y/N
RemoveSource=Y/N

<name>UseMsgexchange</name>

</taskParameter>

<name>RemoveSource</name>

</taskParameter>

This issue was tracked as Item #159182.

Software Fixes

Enterprise

EagleML

Fixed Issue with EagleML GenericSMF underlying Xref Security Resolution
The underlying Security Model allows securities to have xref Ids, but the underlying security is resolved based on underlying primary asset Id only.
The batting order variable for the underlying security resolution changed to allow underlying security resolution by xref Ids.

This issue was tracked as Item #152403.

Field UpdateSource is Updated in Yield Curves Inbound Rule
The field 'UpdateSource' was not being updated when the Yield Curves data was getting loaded. All records contained the value 'UPDATING' in the 'UpdateSource' field.
The 'UpdateSource' field was added into the update/insert SQL statements in the Yield Curves inbound interface.

This issue was tracked as Item #153134.

Enumeration of Entity Xref Fields is Added for All Extracts
Enumeration of the entity xref model across all objects was required.

These existing parameters were used for the entity xref model:
- RTR-parameter: UseNumerationForXrefs
- w_config parameter W_USE_NUMERATION_FOR_XREFS
Default value is : N

<name>UseNumerationForXrefs</name>

</taskParameter>

When value of this parameter is Y, the enumeration is included for each ENTITY XREF:

<entityXrefs>
<entityXref1>
<xrefAccountId>ID1</xrefAccountId>
<xrefAccountIdType>IDTYPE1</xrefAccountIdType>
</entityXref1>
<entityXref2>
<xrefAccountId>ID2</xrefAccountId>
<xrefAccountIdType>IDTYPE2</xrefAccountIdType>
</entityXref2>
</entityXrefs>

This issue was tracked as Item #154441.

EagleML Added Support for Special Characters Like "&" In Extract Filter Parameters
EagleML was unable to handle special characters like "&" in extract filter parameters.
Logic to process special characters (i.e. &,:,!,’) in extract filter parameters was added in the extract_filter_common.inc, extract_filter_entity.inc, extract_filter_held.inc, extract_filter_listprm.inc, extract_filter_sql.inc rules.

This issue was tracked as Item #154965.

Added Switch Between Inner Join and Left Outer Join in Warehouse Position Extract
If a warehouse position has had its position details removed WRHSPOSITIONEXTRACT will return no data. This is caused by the inner join between the POSITION_DETAIL and SECURITY_MASTER tables.
The existing parameter extractLevel was used to add logic that would determine whether the inner join should be switched to an outer join.
When 'extractLevel' = 'A', the tables :
- HOLDINGDBO.POSITION_DETAIL
- SECURITYDBO.SECURITY_MASTER
will be joined to POSITION by left outer join instead of inner join.

<name>extractLevel</name>

</taskParameter>

This issue was tracked as Item #155772.

Added Timeout Error Message to the Compress Mode
Incorrect status was showed for compress mode
Added code to display correct status in compress_extract.inc

This issue was tracked as Item #155916.

Incorrect Record Count for GENISSUEANALYTICEXTRACT
If there is more than one GenericIssueAnalytic record for the same security, extra records will appear in the extract.
The effective date and the source interface instance fields (EFFECIVE_DATE and SRC_INTFC_INST) were added to the left inner join in the "extract_generic_issue_analytic.inc" file.

This issue was tracked as Item #156266.

Adjusted Task Reporter to Show Editcheck Error Description
There were scenarios in which the task_reporter stream was failing with an error and the error description from the edit check was not being displayed.
The \eagle_ml-2-0_cm\out\xml\task_reporter.xml file was changed so that the editcheck error description can be displayed.

This issue was tracked as Item #157109.

UnderlyingModel was added into XSD, XSLT and mapping fixed for underlyingAsset
The UnderlyingModel object was in the core EagleML/EJM mappings, but it was not in the XSD. The XSD actually had the UnderlyingSMF and there were no mappings for underlyingSMF/underlyingAsset/primaryAssetId and underlyingSMF/underlyingAsset/primaryAssetType.
The UnderlyingModel was added into the XSD and the XSLT was changed to process underlyingAsset correctly. The mapping for underlyingAsset was fixed in the underlyingSMF rules.

This issue was tracked as Item #157169.

EagleML - Not able to add service reference to EagleML request
The soap-envelop.xsd file had attributes with single quotes around them. These single quotes were changed to double quotes in the soap-envelop.xsd file to resolve the reported issue.

This issue was tracked as Item #157384.

Added Ability to Change dailyFlag via EagleML
The Pace Calendar interface was unable to update the Daily Flag field via EagleML.
1. Added the ability to read the flag from the PaceCalendar EagleML message
2. Added the ability to update the Pace Calendar record in the database with dailyFlag
3. Added the ability to insert the calendar record with the dailyFlag from the PaceCalendar EagleML message

This issue was tracked as Item #158650.

Changed includeOnlyAcctBaseCurrency filter to use ENTITY_ACCT_BASIS table instead of ENTITY
The includeOnlyAcctBaseCurrency filter was returning the baseCurrency from the ENTITY table for multi-basis entities.
1. The condition to check Base Currency in the ENTITY table was changed to check that element in the ENTITY_ACC_BASIS table instead.
2. A join to the ENTITY_ACC_BASIS table was added for the case that Extended Details are not included in the GL Balances Extract.

<name>IncludeOnlyAcctBaseCurrency</name>

</taskParameter>

This issue was tracked as Item #159058.

Added support of special characters for the User Administration Role field in the User inbound interface
If the <userAdministrationRole> field contained special characters, such as - &, ', <, >, " -, the User record load would fail with the error: "User Role does not exist in a database". The lookups in the User inbound rule could not resolve the User Administration Role.
Special character support for tag2961 - <userAdministrationRole> was added:
Special characters in the incoming message should be encoded in the following format:
& = &
‘ = '
< = <
> = >
“ = "

This issue was tracked as Item #159238.

EJM Stale File Error When Making Web Service Call
If a file is not completely finished being written in the extract process and the WS call attempts to retrieve that file from cmw/reply a stale file error is returned.
The "taskstatus_reply.inc" file was changed to:
1. Save the reply message to a staging folder
2. Move the reply to the TSR and move it to the cmw/reply folder

This issue was tracked as Item #159272.

Added Logic for Updating an existing Issuer Name for Issuer Organization
There was no way to update the issuer name for an existing issuer Id in an issuer organization record.
Added logic in the xml-ref_issuerorganization.xml file to update the issuer name for an issuer Id using the value from an incoming message.

This issue was tracked as Item #159428.

Added Validation Logic for Asset Currency for FWDs in Generic SMF (Non-Accounting)
The generic SMF interface was not validating the asset currency therefore allowing invalid values to be loaded into the database.
Validation logic was added for the asset currency in the generic_smf_validations.inc file for the forward SMF.

This issue was tracked as Item #159602.

Added fix for password in uppercase and added Pace Role to User inbound and outbound interfaces
The password was uppercased on env. 17.2 ORA (Panel mode only). The custom changes were added for the Add and Change User Panel, which is used in the User inbound rule.
The Pace Role can be added to User in the User and Group inbound rules. Added Pace Role tables to extract User and Group.
Custom changes were made to eagle/star/admin/pan-adduser.htm and eagle/star/admin/pan-chguser.htm panels to unset notforsubmit panel flag.
User mode was added to the user inbound rule. If the Pace Role specified in the node groups/group with groupCode = 'PaceRole', the User will be assigned to the Pace Role.
User to Group mode was added to the Group inbound interface. If groupCode = 'PaceRole', all specified Users in the node members/member will be assigned to the Pace Role.
The ability to delete a User from a Group was added in the Group inbound interface. If groupCode = 'PaceRole', all specified Users in the node members/member will be removed from the Pace Role.
Added PACE_USER_ROLE_DETAILS and PACE_USERS databases tables to the User outbound rule. (This adds records with TYPE = 'R') to the User extract. The pacerolename filter was also added.
A section was added to the SQL for the Pace Role in the Group outbound rule. If the filter groupcode = 'ALL' or contains 'PaceRole' (example: 'CenterRole,PaceRole'), the data will be extracted from PACE_USER_ROLE_DETAILS and PACE_USERS database tables. (This adds records with TYPE = 'R') to the User extract. The pacerolename filter was also added

This issue was tracked as Item #159693.

EJM

Added fix for logic of reading a file name by specified file mask in the stream eagle_ml-2-0_default_cm_ndfa_exec

Logic was changed to allow the support of two new special characters in file masks for inbound files.

The '*' character represents any amount of any characters and the '^' symbol signifies that the match has to be foudn at the very beginning of the name. These changes were applied in the read_datamaps.inc file.

This issue was tracked as Item #157891.

I2I

Change Default Logic for CUSIP_CHANGE
EagleML load in non-accounting mode has a Default value = N for the CUSIP_CHANGE, but the accounting load does not have it. As a result the HUB and SPOKE are not in sync when running I2I.
The default logic for CUSIP_CHANGE was changed. This field will be mapped only when it is present in the incoming file. If it is empty in the incoming file, it will be NULL in Data Base.

This issue was tracked as Item #154833.