EagleML Release Notes - December 2017

This release uses XSD Revision 276

Enhancements

Enterprise

EagleML

Added CustomBenchmark Type in Entity Stream
This enhancement added the functionality of CustomBenchmark logic in entity inbound stream. This is only available on Oracle platforms.
New logic for loading CustomBenchmark was added, including Entity, EntityDetail and Security Resolutions.

This issue was tracked as Item #153545.

The <splitRatio> Logic in the xml-ref_ca.xml was Updated
In a true rebook for corporate actions the splitRatio value (if not defined) must be calculated by dividing the value in tag 1710 by the value in tag 1723.
A new variable var1001 was added with a condition that evaluates the splitRatio value in Rebook mode. In addition, the to_ignore flag and status message were updated inside the LOOP for cpStatus=RD.

This issue was tracked as Item #153751.

The Security Resolution With xRefs updated for toAsset

The security resolution for toAsset should be updated to allow the use of either primary asset id + primary asset type or the xreferences to resolve the security.
LOOKUP "to_security_alias", RUNPANEL "delete_ca" and the :ins_upd: variable were modified to successfully resolve a security for toAsset in 3 cases:
1. Only primaryAssetId\primaryAssetIdType specified
2. Only xRefs specified
3. Both primaryAssetId\primaryAssetIdType and xRefs specified

This issue was tracked as Item #154210.

Enhanced EagleML Core to Generate Binds Automatically Based on List of Parameters of Stored Procedure
When core stored procedures are updated with new parameters the call of these SPs from EagleML should also be updated, otherwise the call will fail.
Support was added to generate binds automatically based on the list of parameters of the stored procedure for EagleML Interfaces.
Generic SMF load uses generated bind by default. To switch it off flag W_GENERATE_WITH_BIND=N can be set in ‘w_config_custom.inc’.

This issue was tracked as Item #154560.

IncludeDailyYieldMillRate Added to Include dailyYieldMillRate Model in WHRSNAVMULTIEXTRACT
estar.estar_daily_yield_mill_rate support as a part of the WHRSNAVMULTIEXTRACT extract was requested.
New model added into WHRSNAVMULTIEXTRACT - dailyYieldMillRate. It is turned on by including the IncludeDailyYieldMillRate = 'Y' flag in the RTR as a task parameter.

This issue was tracked as Item #155058.

New XSD object was created - AccountingPositionAsOf
We added new object in XSD – AccountingPositionAsOf and created inbound and outbound rules.
New extract support following filters:

• Entity

  • EntityId

  • EntityList

  • EntityComposite

• SMF

  • primaryassetid:

  • issuename:

  • securitytype:

  • investmenttype:

• reportStartDate:

• reportEndDate: 

• accountingBasis:

• BaseCurrency:

This issue was tracked as Item #155341.

Changed Error Message to Ignore Message if securityAlias Unresolved
Adjusted parameter for xml-ref_ca.xml to return ignore message instead of an error message if securityAlias is not resolved.
The warning and break procedures were added before EDITCHECK with a condition that generates a warning message and stops the rule execution if tag10 is not resolved.

This issue was tracked as Item #155776.

Added Mapping for Tag 1257 to ISSUER_ORGANIZATION
Tag 1257 was added to the Issuer Organization type.
Mapping for tag 1257 was added in the following rule: xml-ref_issuerorganization.

This issue was tracked as Item #156005.

Added Extension Extract to Warehouse Performance
Warehouse Performance did not support extension extracts.

1. In dbkeysmap.inc logical keys were added for Warehouse Performance

2. Exclude fields were added to the extract_extension_table_fields.inc include. These fields are used for PERF_SUM_INST resolution, but not used in extracts.

3. Includes with extension logic were added to the extract_warehouse_performance.inc include file.

4. Customer extension includes were added to the following files: t_warehouse_performance.xml, t_warehouse_performance_streaming.inc and t_warehouse_performance_streaming_eml.inc.

This issue was tracked as Item #156185.

Security Resolution Full was Added in Generic Delete
Only primaryAssetId/primaryAssetIdType were used for security resolution.
Mapping for all security identifiers was added to all rules of the generic delete workflow where security resolution is used, and security_resolution.inc was replaced with security_resolution_full.inc

This issue was tracked as Item #156207.

Updated Issuer Resolution Logic for Issuer Role in eagle_ml-2-0_default_in_xml_smf_mi
New Issuer resolution logic was not applied to Issuer Role in the eagle_ml-2-0_default_in_xml_smf_mi stream.
The issue occurs, when one uploads an SMF with Issuer Role through the eagle_ml-2-0_default_in_xml_smf_mi stream.

This issue was tracked as Item #156374.

Field 'totalNumberShareClasses' is Modified to Filter Out 'TF' Value
Enhanced the Extended Entity Model to provide a field that represents the Total Number of Shares, taking into account the ability to filter out the TF value when the count > 1
Added new RTR-parameter: CountTFShareClass. Values are:
- Y (or missed): default mode, no filter for ShareClasses
- N : excluding of 'TF' value for counting of ShareClasses
Value of field 'totalNumberShareClasses' if CountTFShareClass = N:
if number of Share Classes is equal 1 then totalNumberShareClasses = 1
else totalNumberShareClasses = number of Share Classes excluding of 'TF' value.

This issue was tracked as Item #156475.

Added New Fields to Entity Extract
The new fields added to Entity extract and XSD schema are:
Master Entity Id, Master Entity Name, Sector Entity Id, Sector Entity Name, Entity Detail Xreference.
XSD schema:
1. Added 4 new elements entityMasterName, entityMasterId, entitySectorName, entitySectorId to GenericEntity complex type
2. Added element of complex type entityDetailXrefs to Entity Detail section of Generic Entity (group name EntityDetail.model):
<xsd:element name="entityDetailXrefs" type="EntityXrefIdentifiers" minOccurs="0" maxOccurs="1"/>\
The new element contained the Entity Xreference Identifiers in the root node <entityDetailXrefs>:

Outbound Q rule:
Introduced the ability to add the new fields if the flag IncludeExtendedDetails set to 'ALL':

1. Master Entity Id, Master Entity Name - if the Sector Type (DB field RULESDBO.ENTITY_STAR_PROCESSING.SECTOR_TYPE) = 'SECTOR' then the current Entity is Detail Entity and the Master Entity will be selected

2. Sector Entity Id, Sector Entity Name - if the Sector Type (DB field RULESDBO.ENTITY_STAR_PROCESSING.SECTOR_TYPE) = 'SECTOR' then Sector Entity = current Entity

3. Entity Details Xreference fields - Xref Account Id and Type, Class Code selected from ENTITY_XREFERENCE DB table for Detail Entity of current Master Entity

This issue was tracked as Item #156481.

Added New Fields to Warehouse Position Extract - Position Analytics Section
New fields from DB tables have been added to Position Analytics section of the position extract. The DB tables are:
HOLDINGDBO.POSITION_DETAIL_ANL
HOLDINGDBO.POSITION_DETAIL_ANL_FI
XSD schema:
The following elements have been added to PositionDetailAnl.model group (Position Analytics group for elements HOLDINGDBO.POSITION_DETAIL_ANL DB table) -
liquidityClassification
liquidityScore
rarQuantity
daysToSell
avgDailyVolume
The following elements have been added to PositionDetailAnlFi.model group (Position Analytics group for elements HOLDINGDBO.POSITION_DETAIL_ANL_FI DB table) -
stateScore
priceScore
issueSizeScore
sectorScore
subSectorScore
indexEligibilityScore
runOffScore
highLowPriceScore
fedHoldingPercentScore
topParentIssuerScore
ratingScore
capitalStructureScore
daysToMaturityScore
Mappings for these fields were added to the outbound t rule of the position extract.

This issue was tracked as Item #156656.

Added Several New Fields to Performance Extract
Several fields from the PSR_CURVE_ANALYTICS table were added to the performance extract.
The following fields were added to the XSD:
LIQUIDITY_AMOUNT
LIQUIDITY_PERCENT
LIQUIDITY_CLASSIFICATION
HLIM_THRESHOLD_BREACH
ILLIQUID_THRESHOLD_BREACH

This issue was tracked as Item #156658.

Created Inbound and Outbound Interfaces for User Object
New data object - user - was created.

1. A new reference based model (user) was added to the XSD

2. An inbound rule (extract_ref_user.xml) was created which allows the insert/update/delete of a user object and assignment/deletion of central roles to it/from it.

3. Created extract rules and includes: extract_ref_user.xml and extract_ref_user.inc, t_ref_user_streaming.xml and t_ref_user_streaming.inc. The extract supports the following filters: username, businessgroupname, usergroupname, centerrolename, accountstate

This issue was tracked as Item #156664.

Adjusted XSD with New Columns for EntityPeerGroupRanking and Added them into the xml-ent_entity_peer_group_ranking.xml; Added entityPeerGroupRanking into the eagle_wrf_generic_load
We needed to add several fields to EntityPeerGroupRanking.
New fields in the DB could not be populated through the corresponding EagleML rule.
Added several fields to EntityPeerGroupRanking:
absRankNet
categorySizeNet
percentRank
medianReturn
Changed peerGroupId element - assigned EntityCommonIdModel type to it.

This issue was tracked as Item #156726.

Adjusted ENTITY_RATINGS table into the EagleML XSD, EagleML Inbound Rules and eagle_wrf_generic_load
We needed to add a new XSD object - EntityRating.

Inbound and Outbound rules have been created, and the new type was added to generic load and extract workflows.

This issue was tracked as Item #156727.

Added Enumeration for Nodes <distributionLogLevel[N]> and the Root Node <distributionLogLevels> in the Warehouse NAV Multi Extract with Included NAV Report Details
The data could not be parsed inside the repeated nodes <distributionLogLevel> without enumeration and the root node for all Distribution Log Level records.
The flags IncludeReportDetails (or W_INCLUDE_REPORT_DETAILS) and IncludeCustomReportDetails (or W_INCLUDE_CUSTOM_REPORT_DETAILS) should be set to 'Y' to include Distribution Log Level records in Warehouse NAV Multi extract.

1. Added the new flag to the outbound Q and T streaming rules: CM parameter EnableEnumerationFlag or global variable W_ENABLE_ENUMERATION_FLAG. If the value of EnableEnumerationFlag set to 'Y' then the nodes Distribution Log Levels will be numerated as <distributionLogLevel1>, <distributionLogLevel2>, etc. In other case the repeated nodes <distributionLogLevel> will be added to extract

2. Added the root node <distributionLogLevels> to T rule and XSD schema
   
   Example of Distribution Log Level with the flag EnableEnumerationFlag = 'Y':

This issue was tracked as Item #156981.

Enhanced Generic SMF/ Underlying Model to support Investment Type and Processing Security Type During the Insert of Security for Underlying
Initially Security for Underlying, which is loaded as a part of Generic SMF was inserted with empty Investment Type and Processing Security Type = DEFAULT (escape Forwards and Swaps).
The elements: investmentType and processingSecurityType, were added to GenericSMF in the UnderlyingModel section. Appropriate mappings were added into the inbound rules for the eagle_ml-2-0_default_in_xml_smf_generic and eagle_ml-2-0_default_in_xml_smf_mi streams as well as the Generic SMF extract.

This issue was tracked as Item #157034.

FX Rate Load can Resolve Security Alias based on Security Alias from the hub
Currently, EagleML FX rate Load process uses panels. It uses pan-qfxratesnew.htm to query the FX rate and pan-addfxratesnew.htm to add FX Rates. These panels use the currency value of from and to securities (received as input to validate against TICKER in SECURITYDBO.SECURITY_MASTER. The two nodes used are:
referenceTransaction/issueFxRate/fromCurrency/assetCurrency and
referenceTransaction/issueFxRate/toCurrency/assetCurrency
In the I2I implementation model, where the SMF is fed from IAS and multiple instances of Eagle, there are multiple security records in the SECURITYDBO.SECURITY_MASTER table and the panel returns error when multiple records are returned in response to a query on SECURITYDBO.SECURITY_MASTER.TICKER.
We updated the Fx Rate T rule (Streaming version) and added mapping for XrefIdentifiers fields. This allowed us to use the Security Alias which is in this section for Security Resolution.

This issue was tracked as Item #157071.

Added Ability to Exclude Zero Positions from the Warehouse Position and OpenLot Extracts
Cash Positions records are pushed/created regardless of the Quantity/Value. A new switch is necessary to exclude zero positions from the extracts.
Used an existing parameter: IncludeZeroBalances.
Logic of parameter for Extracts Warehouse Position and Warehouse OpenLot was modified as follows:
If value = Y or parameter is missed in RTR-file, then zero balance records are included into the extract (default behavior).
If value = N, zero balance records are excluded from extract.

This issue was tracked as Item #157091.

XML structure in Raw Service
Need an ability to build structures for XML format files and load those structures into DB.
Added an ability to build XML structures in Raw Service (IWS). Changed the main load Raw Service workflow, namely, the step for loading global values has been moved into the "header_load" step, and the load data step has been modified to use "eagle_ml-2-0_default_in_xml_warehouse" stream.

This issue was tracked as Item #157185.

EJM

Added Several Rules to EagleML Packages and Changed Restart Logic
We have not been checking that the "restart workflow" step is complete.
Added logic that makes it possible to check that the "restart workflow" step is complete.
Was created the following rules
workflow_restart.sch
get_stream_name.inc
workflow_restart.sch
workflow_restart.rsf
workflow_restart.xml
workflow_restart.wrf

This issue was tracked as Item #157264.

I2I

Added Support for the Date Rule Parameter in I2I Workflow
User wasn't able to pass Date Rule as a parameter to the I2I workflow.
We made a common solution which allow passing of any parameters to the Generic Extract workflow through the I2I workflow without any changes in IWS including the Date Rules.
The following code was added in the i2i include:
i2i_common_include.inc

This issue was tracked as Item #155638.

Software Fixes

Enterprise

EagleML

Fixed the Xref Extract in History Mode, Added Support on Environments with MSSQL Database, Added Streaming T Rule

1. The Xref extracts in History mode (when the flag HistoryOnlyFlag set to 'Y') contain duplicated Security Xreference identifiers records in the nodes <xId> or numbered <xId[N]>

2. Missing support on MSSQL environments

3. Missing streaming mode

1. Fix for WHERE section in the History mode of Xref extract: added the following condition:
   AND SECURITYDBO.XREFERENCE_HIST.UPDATE_DATE <= SECURITYDBO.SECMASTER_HISTORY.UPDATE_DATE

2. Added SQL code for MSSQL database

3. Added T streaming rule and grouping in SQL code

This issue was tracked as Item #154948.

New Flag W_ISSUEROLE_UPDATE_CLOSED Added to Support Closed Records Update
It was requested to support updating of records with END_DATE not NULL (END_DATE and the percentage owned should be possible to update). According to the current logic such records are considered as closed and are never updated.
A new flag, W_ISSUEROLE_UPDATE_CLOSED, was added to support updating of closed records (END DATE not NULL).Flag should be set as Y in order to work and allows to update the end date and the percentage owned for records with specified start date. (only works with new SP version)

This issue was tracked as Item #156267.

Added New Field in Warehouse Position for Total Return Value from Position Detail Table
We added a new field in the Warehouse Position interface(XSD) – assetTotalReturns. This field contains the value from the total return field from the Position_detail table. Once the SP has been updated we will add support for loading this value. For now we can only extract it.

This issue was tracked as Item #156457.

Added Mapping for Element ‘contractualCashSettlementRule’ into Entity Accounting Basis Load, Entity and Entity Accounting Basis Extracts
Mapping for the element ‘contractualCashSettlementRule’ was missing from the Entity Accounting Basis Load rule as well as the Entity and Entity Accounting Basis Extract rules.
A new element, contractualCashSettlementRule, was added into Accounting Basis Interface and appropriate mappings were added in the Entity Accounting Basis Load and Extract rules. The missed mapping for the contractualCashSettlementRule element was added to the Entity Extract.

This issue was tracked as Item #156533.

Fixed Incorrect Base Currency for Non-Primary Basis in the RealizedGainLoss Extract
When the RealizedGainLoss extract is executed for a non-primary basis then the Base Currency comes from the primary basis.
Base Currency should be found from corresponding record in the ENTITY_ACCT_BASIS table for the applicable 'accountingBasis'.

This issue was tracked as Item #156560.

Added Flag to Remove EDITCHECK in xml-ref_generic_ca.xml Rule
Tag302 can be optional.
Added flag W_GENERICCA_IGNORE_EDITCHECK302 to disable edit check in the case that Security Alias was not resolved for tag 302.

This issue was tracked as Item #156755.

Ability to Include or Exclude Zero Balances in the Insurance Valuation Extract
Added ability to include or exclude zero balances in the insurance valuation extract.
Used an existing parameter: IncludeZeroBalances.
Logic of parameter for Extract InsuranceValuation was modified as follows:
If value = Y or parameter is missing in RTR-file, then zero balance records are included into the extract (default behavior).
If value = N, zero balance records are excluded from extract.

This issue was tracked as Item #157321.

Changed Sql Request Because Underlying Security Resolution Using the Begin Date / End Date Parameters for BT154541 Had a Negative Affect 
We saw a negative affect not providing securities.
SQL was changed to use the sub-request to UnderlyingTable.

This issue was tracked as Item #158146.

I2I

Fix Problem When Held Filter is Used in Accrued Income Extract
Accrued Income Extract generating duplicate rows when Held filter is used. The reason this happens when the held and entity filters are used together. We have separate includes for these two filters. Entity Filters is called first and then Held Filter. Held Filter uses Entity Filter include as well and as a result it overwrites some variables of initial Entity Include for the main table.
We updated Q rule for Accrued income extract and changed order of the entity and held includes in the extract_accrued_income.inc.

This issue was tracked as Item #156889.

Fixed Issue for I2I and Entity Extract with Effective Date filter
Effective Date filter was passed to the I2I extract. Entity Extract doesn't have Effective Date in composite table and we use UPDATE DATE field instead. Due to this, the client didn't receive the necessary data.
Fixed extract_entity.inc. Deleted Effective Date filter for non-history mode.

This issue was tracked as Item #158122.