EagleML Release Notes - March 2016

This release uses XSD Revision 221.

There are four overlay files that are currently packaged as part of the EagleML monthly release. Please refer to About EagleML Overlays for specific details and impact to you.

Enhancements

EagleML

Added New filter for Security Types

A new filter that extracts based on investment types has been created. Sometimes it is necessary to extract only securities with certain investment types. This filter allows multiple types to be extracted. You can extracting anywhere from 1 investment type to all except 1. All security extracts are now fitted with this filter. A new task parameter called investmenttype was added to the eagle_ml-2-0_cm/extract_filter_smf.inc include.
This issue was tracked as Item #136928.


Added Entity Xref ids to all Warehouse Extracts

No entity xreference identifiers showed up on Warehouse extracts. This scenario occurred in all Warehouse extracts. Message mappings were changed so that entity xreferences were set to appear in all Warehouse extracts by default.
This issue was tracked as Item #136971.


Added SMF Xrefs to all Warehouse Extracts

Security xref ids were not showing up on warehouse extracts. SMF xrefs can be in tracing a security to a warehouse transaction. This scenario occurred in all Warehouse extracts. Message mapping was altered to include the SMF xrefs by default on any warehouse extract.
This issue was tracked as Item #136973.


Improved Support of Xref ids in Issuer_role

Code for copying identifiers was added to issuer role xsl rule. All possible identifiers to the issuer role sml rule and security resolution was switched to security resolution full in the issuer role include file. This enhancement allows for more comprehensive use of all xref identifiers. The message mappings and resolution logic was updated to allow for this change.
This issue was tracked as Item #137313.


Introduced New Elements for Warehouse NAV Object

Upon analysis of the warehouse NAV object, it was determined that adding new fields would be beneficial. These fields include:

  • priorValuationDate
  • priorRawNAV
  • priorNetAssetsNAV
  • mktValueBaseCash
  • mktValueForeignCash

The rules related to WarehouseNAVMulti object received new elements with corresponding new mappings. The following rules and include files were altered:

  • eagle_ml-2-0_cm/out/tagvalue/extract_warehouse_nav_multi.xml
  • eagle_ml-2-0_cm/out/tagvalue/extract_nav_multi_details_fields.inc
  • eagle_ml-2-0_cm/out/tagvalue/extract_nav_multi_details_tables.inc
  • eagle_ml-2-0_cm/extract_filter_nav_multi_exclude_TF.inc
  • eagle_ml-2-0_cm/out/xml/t_warehouse_nav_multi.xml
  • eagle_ml-2-0_cm/out/xml/t_warehouse_nav_multi_streaming.xml

This issue was tracked as Item #135868.


Extract only newly added securities

You can now filter to extract only newly added securities. Previously filters could pull an old security if it was recently updated and think of it as new. Now, a new parameter, onlyNewSecurities, avoids this issue by looking to the create date. Any SMF extract can use this filter. The enhanced logic only extracts securities based on the create date.
This issue was tracked as Item #137351.


Improved Support for xref ids in Smf_schedule

Code for copying identifiers was added to SMF schedule xsl rule. All possible identifiers to SMF schedule xml rule and security resolution was switched to security resolution full in the issuer role include file. This enhancement allows more comprehensive use of all xref identifiers. Message mappings and resolution logic in the smf_schedule were updated to accommodate this change.
This issue was tracked as Item #137362.


Batch Issue for ALADDIN Source

In the case of a batch load, if the source was ALADDIN the securities in the batch could include securities be of different variable rate. This scenario impacts only SMF loads with a source of ALADDIN. The logic for assembling this batch was changed so that securities with different variable rates would not be loaded at the same time.
This issue was tracked as Item #137530.


Where Clause For Warehouse Delta Extracts

Previously the warehouse delta extracts included the instance in the where clause. This limited the ability of what was extracted. A new option has been added to exclude the instance and use the update time only.
This issue was tracked as Item #138641.


Share Class Added to Warehouse NAV Multi

The share class was added to the Warehouse NAV multi extracts. The code was updated to include the share group in the XSD for the WarehouseNAVMulti.model group
This issue was tracked as Item #138643.


Introduced New Extract for ESTAR_SETTLED_CASH_ACTIVITY

A new outbound interface for DB table ESTAR.ESTAR_SETTLED_CASH_ACTIVITY has been created. Supported filters include:

  • Date filter (by Effective Date and Update Date)
  • Entity filter (by Portfolio Acct/Entity Id)
  • SMF filter (by Security Alias and Security fields)
  • Common filter

For this enhancement, XSD type SettledCashActivity (Accounting group) has been created. The XSD group SettledCashActivity.model with elements for all DB fields of ESTAR_SETTLED_CASH_ACTIVITY table has been added. The extract rules extract_acct_settled_cash_activity.xml (q_rule), t_acct_settled_cash_activity.xml (t_rule) and t_acct_settled_cash_activity_streaming.xml (t_rule_streaming) were created.
This issue was tracked as Item #138989.


New Accounting Accrued Income Extract

EagleML was enhanced to include a new accounting accrued income extract. It is now possible to extract accounting accrued income objects.
This issue was tracked as Item #138999.

EJM

Included Entity Xrefs in Accounting Basis Extract

With accounting basis extracts, no information about entity xreferences would get extracted. The code was enhanced so that entity xreferences are now included in these extracts.
This issue was tracked as Item #133264.


Eagle Default Nuggetizer Recovery Stream

When the Eagle Default Nuggetizer workflow hangs, it can require a significant amount of manual work to resolve it. A new recovery stream was created to avoid this manually intensive effort. The new stream name is eagle_ml-2-0_default_cm_nuggetizer_recover. Please refer to the Nuggetizer Recovery Steps for information on how to use this new stream.
This issue was tracked as Item #137673.

I2I

Allow Held Only Filter for SMF Related Data extracts

The held only filter previously worked for SMFs only, but sometimes SMF related data is needed. This same filter could be useful when extracting SMF related data. The code was enhanced so that SMF related data is now available through the held only filter. Message mapping in the outbound SMF rule was altered to include related data for extracts.
This issue was tracked as Item #138650.

Software Fixes

EagleML

Update to EagleML Split Rules

While the CSV generic interfaces support the split rule function, there was no equivalent split rule on the EagleML platform. This rule is responsible for the parallel processing of records with different keys and consecutive processing for records with the same key. The code has been enhanced. The new EagleML split rule, called xml-acct_trades.spl, was created to mirror the CSV version.
This issue was tracked as Item #136655.


Incorrect Processing Sequence for Warehouse Close Lot

The Warehouse Close Lot is being processed in parallel with the trades references such as trade quotes and trade charge. The incorrect order of processing can lead to performance decrease as well as potentially unwanted results. This is a problem in the eagle_wrf_generic_load workflow. The error occurred every time this workflow was launched. The logic for processing sequences was improved to make the Close Lot get processed after all the trades references have been completed.
This issue was tracked as Item #137051.


Enabled Streaming for Warehouse Trade Activity

There were problems with large volume extracts for warehouse trade activity objects. The extracts could time out and not complete. A new streaming rule was created to allow for extracting large amounts of warehouse trade activity data quickly.
This issue was tracked as Item #137144.


Enabled Streaming for Warehouse Cash Activity

There were problems with large volume extracts for warehouse cash activity objects. The extracts could time out and not complete. A new streaming rule was created to allow for extracting large amounts of warehouse cash activity data quickly.
This issue was tracked as Item #137145.


Enabled Streaming for Disposal Lots

There were problems with large volume extracts for disposal lots objects. The extracts could time out and not complete. A new streaming rule was created to allow for extracting large amounts of disposal lots data quickly.
This issue was tracked as Item #137146.


Enabled Streaming for Warehouse Traded Cash Activity

There were problems with large volume extracts for warehouse traded cash activity objects. The extracts could time out and not complete. A new streaming rule was created to allow for extracting large amounts of warehouse traded cash activity data quickly.
This issue was tracked as Item #137147.


Enabled Streaming for Warehouse Position

There were problems with large volume extracts for warehouse position objects. The extracts could time out and not complete. A new streaming rule was created to allow for extracting large amounts of warehouse position data quickly.
This issue was tracked as Item #137153.


Enabled Streaming for Warehouse Performance

There were problems with large volume extracts for warehouse performance objects. The extracts could time out and not complete. A new streaming rule was created to allow for extracting large amounts of warehouse performance data quickly.
This issue was tracked as Item #137160.


Issue with Symbols ":" and "/"

Loading records with the special characters ":" or "/" would result in the database being populated with records that have the symbols encoded. With the symbols encoded, the names are then not correct and it is also difficult to through SQL queries. This issue was a problem specifically with the generic SMF and generic entity inbound streams. The streams did not encode the two symbols after the rule processed their xml equivalent. The code was updated to resolve this issue. The rules encode and decode the special symbols as expected.
This issue was tracked as Item #137223.


Missed Reinvest Based On Field

The Reinvest Based On field was missing the GenericCA section of the XSD. The code was updated to resolve this issue. The field was added to the XSD.
This issue was tracked as Item #138004.


Multiple Issues Associated with accountingValidationFlag = Y

There were several issues identified when the accountingValidationFlag was sent as Y. The issue name was not populated correctly for currency forwards. Also, there were a number of fields that did not get populated in the database including from the DERIVATIVES table the fields:

  • DERV_CHAR1
  • DERV_CHAR2
  • DERV_CHAR3
  • DERV_CHAR4
  • DERV_CHAR5
  • DERV_CHAR6
  • DERV_CHAR7
  • DERV_CHAR8
  • DERV_CHAR9
  • DERV_CHAR10
  • DERV_CHAR11
  • DERV_DATE

From the SECURITY_MASTER table the fields:

  • PRVT_PLACE

From the SECURITY_MASTER_DETAIL table the fields:

  • USER_GROUP_DESC3
  • USER_GROUP_DESC4
  • USER_GROUP10

And from the SECMASTER_DETAIL_EXT the fields:

  • SECURITY_TYPE
  • SECURITY_TYPE2

Also, the SECURITY_MASTER.UPDATE_SOURCE was always mapped as 'MCADMIN' . The code was updated to resolve these issues. These fields are now populated as expected when the accountingValidationFlag = Y.
This issue was tracked as Item #138025.


Issue with PACE User Data Parameter

For the task parameter PACEUserData, if the data you passed in the value contained an exclamation point, the data was truncated at the  exclamation point before being stored in the database. This issue was due to the fact that the exclamation point was treated as a field delimiter. The code was updated to allow for an exclamation point for the PACEUserData parameter.
This issue was tracked as Item #138066.


Issue with Corporate Action Lookup

The Corporate Action look up was not working as expected. It caused records to fail when they were sent to be updated. This issue occurred when corporate actions were updated. The lookup resolved several records, but it did not chose the record with the RD status and therefore the wrong panel was called. The code was updated to resolve this issue. Additional logic introduced to the query and the lookup is functioning properly now.
This issue was tracked as Item #138505.


Security Resolution Issues on SMF Loads

The resolution logic for non multileg SMFs was not working properly. It did not properly handle the use case of several records with the same primary asset ID. In addition, if a record with an existing primary asset ID, but different xref IDs got sent to the database, it would fail because the resolution logic incorrectly assumed it was a new security. This scenario affected SMF loads. The code was updated to resolve these issues. The resolution logic was edited so that a record with an existing primary asset ID is treated as an update as long as the xreference IDs are unique.
This issue was tracked as Item #138559.


Issue with Issuer Extract

The issuer extract could fail with the following error:

-2. ErrorMessage: ORA-01427: single-row subquery returns more than one row

This error prevented you from getting results from an issuer extract run. The issuer outbound rule was updated to resolve this issue. Issuer Extract now functions as expected.
This issue was tracked as Item #138649.


GPG Decryption logic

There was an issue with the GPG decryption logic. This issue was resolved.

Note: This feature has not been tested in Eagle's QA regions as GPG does not exist there.
This issue was tracked as Item #138537.


Update Issue with Index Swap and Locks Swap

Non-multileg SMF swaps (Index Swap and Locks Swap) failed on updates. This issue was due to the security resolution logic not working for non-multileg SMF swap records. The security resolution logic was updated to resolve this issue.
This issue was tracked as Item #138738.


SMF Generic Stream Incorrectly Updated Composite Tables In History Only Mode

When the core in_xml_smf_generic stream updated the shell record in history only mode, it actually updated both the security history tables and the composite tables. It should only update the history table. Because it is also updated the composite table, its affected RDC's ability to composite the security correctly. This issue was introduced in the EagleML February release. The code was updated to resolve the issue. Only the history table is updated now.
This issue was tracked as Item #139073.


Issue With PostLoad Workflow

The PostLoad workflow was not able to pick up the unpacked file from the generic load workflow. The PostLoad workflow is supposed to activate after the generic load is completed. New parameters were added to the PostLoad workflow so that it is able to locate files processed by the generic load workflow.
This issue was tracked as Item #137674.

EJM

TSR Reporting of No Data and Warnings

When no data was returned, the TSR reported the status as a failure. In addition, the TSR reported the overall status as a failure if sub steps contained a warning. The code was enhanced to allow you to have the TSR report these scenarios as success. A new task parameter called TreatNoDataAsWarning. This parameter hands both scenarios. By setting TreatNoDataAsWarning parameter, the Pace Jobs no date steps will be reported with NO DATA status and treated as warnings and the overall status will be success. By the default the TreatNoDataAsWarning is switched OFF, to switch on it the following parameter should be included into RTR message:

<taskParameter>
<name>TreatNoDataAsWarning</name>
<dataType>S</dataType>
<value>Y</value>
</taskParameter>

This issue was tracked as Item #138134.

I2I

Incorrect Results for SMF Extracts Due to Date Filter

In the case of a false date being used in the date filter for SMF extracts, all records held by the Spoke were returned. Returning all records when a false date is used in the extract is incorrect and misleading. This issue only occurred only when the filter is not filled in with a date that exists on any of the records in the database. The code was updated to resolve this issue. All include files related to the SMF extracts received an error check to handle the scenario in which the effective date was not matched to any records in the database. Now a mismatched date returns the following error:

HELD Error message (8) - Format of Held Effective Date should be - yyyy-mm-dd or yyyymmdd. Input value is incorrect - 'date_from_filter'

This issue was tracked as Item #136960.


Improved Logic for SMF Extracts

The heldpositionsource and heldcashactivitysource filters were not being properly processed and returned all records in the database when it encountered an improper filter. This scenario occurred if either of the above mentioned parameters are filled with at least one incorrect source name. The logic was rewritten for these two parameters. Now there is an edit check that verifies that each filter passes to the parameter in the RTR is actually a valid source that can be found in the database. Only if all filters passed in the parameter are correct will the extract be run without an error message.
This issue was tracked as Item #136963.


I2I Issue with Extract security alias

The security alias was not being extracted when web services was used. This affected I2I (Instance to Instance) processes and only occurred in web services SMF extracts. The standalone EagleML interfaces do not have this problem. The code was updated to resolve this issue. The I2I process is now set to extract the security alias form HUB and send it to SPOKE.
This issue was tracked as Item #137352.