EagleML Release Notes - April 2016

This release uses XSD Revision 222.

For the April 2016 EagleML Release only, there is an additional amendment package for clients using Performance Extracts. This extra install package is to address a significant issue with the Performance Extracts that was uncovered after the April 2016 EagleML was released. The issue is that where clauses were not being applied properly and therefore resulted too much data being extracted as well as a large performance hit. You only need this additional package if you specifically use Performance Extracts.
There are four overlay files that are currently packaged as part of the EagleML monthly release. Please refer to About EagleML Overlays for specific details and impact to you.

Enhancements

EagleML

New Forward Points Extract

EagleML was enhanced to include a new forward points extract. This extract pulls data from the SECURITYDBO.FORWARD_POINTS table. With this change a new XSD type was added and a new outbound rule was created that has streaming capabilities for large volume extracts.
This issue was tracked as Item #135717.


Changes to Issuer Role End Date

With the issue role object, the end date (tag 73) was changed to only accept values from incoming files as well as receiving a new mapping in the xml-ref_issuerrole_common.inc file. Previously, the tag could be calculated from the beginning date under some circumstances. This capability has been disabled. Additionally, the mapping in the issuer role rule has been corrected.
This issue was tracked as Item #137789.


Added new Outmapping for Tag 1432 in Security Resolution

The outmapping in security_resolution.inc was altered to allow for correct resolution. Tag 1432 is required for some security resolution processes so it was important to include it in the out mappings. The outmapping received the new tag as well as a slight correction to old mapping that was susceptible to improper matching of the primary asset Id and the primary asset Id type.
This issue was tracked as Item #137841.


Added ESTAR.ESTAR_SUBSCRIBE_REDEEM

The ESTAR.ESTAR_SUBSCRIBE_REDEEM table did not exist in EagleML. This table is primarily needed to extract shareholder capital stock activity. EagleML was enhanced to include this table. To add the table and an extract rule for it, a new model group called SubscribeRedeem.model was created with a corresponding complex type and object of the same name. These were changes to the XSD.
This issue was tracked as Item #137894.


Added ESTAR_DAILY_YIELD_MILL_RATE

EagleML did not have a database table for the ESTAR.ESTAR_DAILY_YIELD_MILL_RATE. This table contains data that clients would like to use via an extract. EagleML was enhanced to include ESTAR_DAILY_YIELD_MILL_RATE. The new model group, complex type and object were all created and added to the EagleML XSD.
This issue was tracked as Item #137895.


Added Client Xreference Lookup

You could have issues using the client Id field to do lookups due to the Id field being too long. This scenario occurred in both inbound and outbound client interfaces. The XSD was updated with an additional model group and complex type to accommodate xreference extensions.
This issue was tracked as Item #132595.


Added Entity Share Class

EagleML was enhanced to include Entity Share Class. A new object type, XSD model group and outbound rules were created for this type to exist and be available for extracts.
This issue was tracked as Item #137896.


Improvement Handling of Blank Xrefs

Blank xreference Ids would create problems on load since they would cancel the validity of previously loaded xrefs of the same type. New security resolution was updated to ignore blank xrefs and load the rest of the file regularly.
This issue was tracked as Item #137946.


Changes to xml-ref_ca.xml Rule Around Trans Type Parameter

The trans type parameter used to only take one value, but this was inaccurate since it could actually take on different values. The trans type depends on the tag 3931, which is process_sec_type in the SECURITYDBO.SECURITY_MASTER table. The trans type should therefore be calculated based on tag 3931. This issue affected corporate actions interface. Code was added to allow the trans type to be calculated based on the process_sec_type value stored in tag 3931.
This issue was tracked as Item #138040.


New Outbound Data Entitlement Interface

EagleML was enhanced to include a new outbound interface that can be supplied with a user Id, business user group and list of entities which will selectively extract a certain group of entities. Either the user Id or the business user group should be supplied but not both. This will return either all entities associated with that user or user group or if a list of entities is specified, only these entities (as long as they are associated with the user or group passed). The new interface was added to the XSD and received a corresponding outbound xml rule. Names as follows:

  • Name of q-rule: extract_ref_entity_entitlement_list.xml
  • t-rule streaming: t_ref_entity_entitlement_list_streaming.xml

This issue was tracked as Item #138120.


Physical Partition Added to Set of Entity Fields

The entity interface received a new field called physical partition. Now when inquiring information about entities, it is possible to get the physical partition of where the particular entity is located. The code was enhanced to include new message mapping, thus allowing data about the table space of the entity to be displayed alongside already available data.
This issue was tracked as Item #138642.


Extended AssetIdentifiers.model

Three new elements were added to the Asset Identifiers. The following elements were added to the XSD:

SECURITYDBO.SECMASTER_DETAIL_EXT. SECURITY_TYPE – <smSecurityType>
SECURITYDBO.SECMASTER_DETAIL_EXT. SECURITY_TYPE2 – <securityType2>
SECURITYDBO.SECURITY_MASTER_DETAIL – <securitySubType>


These new elements will now be present in extracts if the parameter IncludeAssetProperties is set to Y in the RTR. Or, you could specify the variable :W_INCLUDE_ASSET_PROPERTIES: in the the w_config_custom.inc. For example:

<CODE> :W_INCLUDE_ASSET_PROPERTIES: = 'Y' </CODE>

This issue was tracked as Item #138645.


Fixed Missing Multileg Swap Links

When loaded through Message Center the contract, pay and receive legs would all be processed as separate records which would lead to the total settlement to be calculated incorrectly since the swap was essentially calculated twice. This scenario occurred with the Multileg SMF load. To resolve this issue, a new tag was added to keep track of the total traded interest so that the settlement would not be calculated twice.
This issue was tracked as Item #138805.


New Realized Gain Loss Extract

EagleML was enhanced to include a new Realized Gain Loss Extract.
This issue was tracked as Item #138993.


Added Unsettled Transaction

A new object, complex type and model group were added to the EagleML for Unsettled Transaction. The XSD received the following changes:

  • New object unsettledTransactions with complex type UnsettledTransactions
  • group UnsettledTransactions.model

This issue was tracked as Item #138995.


Added ConsolidatedEarnedIncome Extracts

EagleML was enhanced to include extract based on the RPT_POS_CONSO_INC_EARNED stored procedure. A new object type, complex type and model group were added to the XSD. Also new outbound rules including rules that had streaming capacity were created to support extracts.
This issue was tracked as Item #139374.


Added REPORT_CASH_JOURNAL Extracts

EagleML was enhanced to allow extracts from the table REPORT_CASH_JOURNAL. A new feed type and outbound rules were created to allow for extracts form this table.

This issue was tracked as Item #139375.


IWS Default Solution Files

The default IWS Solution files are now included as part of the EagleML monthly installation package. By including these solution files, it allows you to see the graphical interruption of these solutions within the EJM Monitoring Tool. These files all start with the name "eagle_wrf".
This issue was tracked as Item #128068.

EJM

TSR for Jobs that Timeout

Sometimes it can be difficult to determine if a job is hung. If a task status response (TSR) was sent to notify you of a time out error, it would help you identify there is a problem. The code was updated. MSGEXCHANGE receives a catch block that is responsible for recognizing time out errors and reporting them correctly to you through a TSR.
This issue was tracked as Item #136543.

I2I

Ability to Delete Xref Id Hub and Reflect Changes in Spoke

I2I was enhanced to be able to delete xref identifier for a security in the Hub and reflect this change in the Spoke. For this enhancement, the generic extract workflow was updated and logic to extract info related to canceled xref identifiers was added. The generic load workflow was also updated to load information on the canceled identifiers prior to SMF loading. It's important to note that info about canceled SMF identifiers is loaded BEFORE SMF loading in eagle wrf generic load workflow.

Note: The extract to obtain info about canceled xref identifiers will be processed only if there is taskparameter "onlycancelsmfxref" = "Y" in RTR and extract feed type is SMFEXTRACT. You can't extract separately info about canceled xref identifiers, this can only be done together with SMF extraction.
This issue was tracked as Item #138684.


Ability to Use Double Entity List Filter

It is now possible to apply a double EntityList filter when using the entity list extract. Code was changed regarding the entity list filter to allow a second set of entitySelectionType and entitySelectionvalue parameters to be passed for extracts.
This issue was tracked as Item #138687.

Software Fixes

EagleML

Issue Handling Duplicate Entity Xref Ids in Warehouse Stream

In the case of two different entity Ids having the same xreference Id, the rule would always update the first of the two records. This is incorrect behavior. The desired result is an error that reports on the existence of the duplicate xref Ids. This issue could occurring during any xref Id lookup within the warehouse object type. The code was updated to resolve this issue. An edit check was added that will throw an error if there are duplicate securities.
This issue was tracked as Item #129894.


Added I2I Support for Warehouse Trade Extracts

The warehouse trade object did not support I2I processing for trade extracts. The w_config_custom.inc file now has a parameter that allows or disallows I2I processing. This parameter is W_I2I_FORCE_WRHS_TRADE_INS and can be set manually or through a control message. If this parameter is set to Y, I2I processing is allowed.
This issue was tracked as Item #131003.


Issue Extracting Client Communication Data with the Client Extract

When extracting through the outbound q stream, the client communication object was extracted separately from the client object. It is preferable to see the client communication data with the client extract. The code was updated to resolve this issue. The stream eagle_wrf_generic_extract is now capable of combining these two objects into one extract through the use of specific filters in the RTR. The feed type parameter should include both of these objects in order for them to both be extracted.
This issue was tracked as Item #131828.


Enabled Streaming for Prices

There were problems with large volume extracts for price objects. The extracts could time out and not complete. A new streaming rule was created to allow for extracting large amounts of price data quickly.
This issue was tracked as Item #137142.


Enable Streaming for FX Rates

There were problems with large volume extracts for FX Rates. The extracts could time out and not complete. A new streaming rule was created to allow for extracting large amounts of FX rates quickly.
This issue was tracked as Item #137143.


Security Resolution Updates for Multileg Swaps

There were two issues resolved. First, the security resolution via security alias was returning contracts and legs with information from only the contract. As a result, the underlying leg records would be created incorrectly. The second issue had to do with the effective date causing confusion in the system as a result of it viewing contracts and legs with different dates as separate records rather than recognizing the swap security. Both these errors could lead to loss of data due to the system viewing the legs and contracts as separate objects rather than one complex swap object. These issues occurred in multileg securities. The resolution logic was changed to avoid these issues. The resolution by effective date was eliminated and the security alias resolution was edited to pick up on all parts of the multileg swap.
This issue was tracked as Item #137847.


Security resolution updates for multileg swaps

There were two basic issues fixed in this bug:

  1. The security resolution via security alias was returning contracts and legs with information from only the contract. As a result the underlying leg records would be created incorrectly.
  2. The second problem had to do with the effective date causing confusion in the system as a result of it viewing contracts and legs with different dates as separate records rather than recognizing the swap security.

Both these errors could lead to loss of data due to the system viewing the legs and contracts as separate objects rather than one complex swap object. The resolution logic was changed to avoid both problems discovered above. The resolution by effective date was eliminated and the security alias resolution was edited to pick up on all parts of the multileg swap.
This issue was tracked as Item #137914.


Forwards Do not Create Record in the Underlying_securing Table

When inserting or updating currency forwards through the generic SMF stream, the underlying security would not be created. This was initially out of the scope of the stored procedure. The impact is that underlying security were not created for forward records. This issue occurred for multileg securities, specifically forwards. The issue was addressed by adding code to the stored procedure that can handle forwards and properly create a placeholder underlying record with default values. These shell securities can be edited later.
This issue was tracked as Item #138033.


Warehouse Trade Extract Always Used Trade Date

For the Enhance Warehouse Trade Extract, the effective date was always populated off the Trade Date. The code was modified to allow you to specify in a parameter another date to use instead. The possible dates that can be used include the following:

  • Effective Date - TRADE.EFFECTIVE_DATE
  • Accounting Date - TRADE.ORIG_ACCT_DATE / TRADE.LEDGER_EFFECTIVE_DATE
  • Month End Accounting Date - TRADE.MONTH_END_DATE
  • Post Date - TRADE.POST_DATE
  • Trade Date - TRADE.TRADE_DATE
  • Settle Date - TRADE.SETTLEMENT_DATE

This issue was tracked as Item #138206.


Symbol Encoding/Decoding Issues in Reference Exchange Price

The ":" (colon) was being displayed as the xml encoding (\3A). This issue occurred with the reference exchange price feed. The code was changed to encode and decode the special symbols and display them properly.
This issue was tracked as Item #138539.


Multileg Swaps Issue Updating History Tables

The history tables would not be updated completely on loading of multileg swaps. This issue occurred if the swap was being updated. The default is to have the fields populated through a panel. However, sometimes a stored procedure is necessary as well which can be the case for updating. The stored procedure can overwrite fields with blanks if the values from those fields are not re-passed to the stored procedure. The values that are populated by the panel are leveraged to be used by the stored procedure. This way the stored procedure effectively overwrites the values with equivalent ones rather than blanks.
This issue was tracked as Item #138602.


New Tag in Generic SMF Interface

The journalizingFlag element (tag1293) was added to the Generic SMF interface. The new element was added to the smf_aliases.inc include file.

Note: While this change was made to EagleML, there is also a stored procedure change required in Eagle core for clients on SQL platform. This procedure change has not yet been scheduled.
This issue was tracked as Item #138652.


Incorrect Underlying type for Multileg Swap Legs

The underlying types were not being calculated correctly. This issue occurred for Multileg Swaps. The message mapping was not tracking the correct tag. It was using the value from tag 916 when the underlying type is calculated based on tag 7103. The code was updated to resolve this issue.
This issue was tracked as Item #138821.


Sequencer Issues

In the EagleML March release, a regression issue was introduced which caused the sequencer to not work properly. This would then also impact the execution of the nuggetizer. The code was updated to resolve this issue. One line of code was added to the cancel_subtasks.inc and execute_wrkfl_event.xml files so that the sequencer operates properly again.
This issue was tracked as Item #139487.


Issues with TRUNC Function in Extracts

The TRUNC function would cause the SQL query index not to be used if the index was meant to be a short date. Indexes are used to search quickly in the database so losing such a capability is problematic for efficiency. To resolve this issue the short dates POST_DATE, ACCOUNTING_DATE, MONTH_END_ACCT_DATE, EFFECTIVE_DATE, had the TRUNC function made unavailable to them.
This issue was tracked as Item #139565.


Issue with Forward accounting SMF if Buy and Sell Currency Not Populated

The forward accounting SMF would stay in the processing folder indefinitely if the buy and sell currency fields were not filled in. The fields were made mandatory and the record now fails in the case of these fields not being filled in.
This issue was tracked as Item #139728.


Remove RTRIM function From Holdings Extract

The RTRIM function was causing performance issues in holdings extracts. The RTRIM function was eliminated from character fields and replaced with the TRUNC function. This increased processing speed.
This issue was tracked as Item #139858.

EJM

Improvements in eagle_ml-2-0_default_ea_ftp_reader stream

The eagle_ml-2-0_default_ea_ftp_reader stream was tuned to be able to read the FTP files faster, thus increasing throughput.
This issue was tracked as Item #138949.


Issues with SingleIncomingFile Parameter

When the SingleIncomingFile parameter was set to "1" there would be errors in processing. The problem lay within the wrong file path that would be used in the case the parameter had a value of 1. This path was fixed to grab the file from the expected directory and it now works as it should. The first file in the correct directory is processed and then the execution terminates.
This issue was tracked as Item #139316.


Issues with Cm Load Control Flow Stream

The cm load control flow stream was not working properly on some 12.1.5 and 13.1.2 regions. The code was updated to resolve this issue. Code was changed in the eagle_ml-2-0_cm\out\tagvalue\cmw_load_flow.prf rule.
This issue was tracked as Item #139655.


Issue with Custom Transform Stream

EJM allows the submitter to forward the output of 'q' stream to a custom transform stream by using taskparameter T_StreamName and providing the name of the stream or the path to the incoming directory of the stream. In a prior release, enhancements were made to accommodate streaming functionality and the routing from 'q' to transform was split into two paths. One path goes to eagle_ml-2-0_default_out _t and the second path goes to eagle_ml-2-0_default_out _streaming_t. In the path going to eagle_ml-2-0_default_out _streaming_t, the feature to send the output to custom transform stream was not included. The code was updated to resolve this issue.
This issue was tracked as Item #139920.

I2I

Improper extracts through I2I

When running an I2I extract for entities with the entity Id value set to a value that does not exist in the database, the workflow returns data. This should not be the case since that value should not be in the database and therefore the expected result is either failure or no data. The I2I streams, however were actually creating a record with that entity Id before extracting and returning it. The code was updated to resolve this issue. The extract_filter_held.inc include was improved to handle such situations and not allow for the creation of a record.
This issue was tracked as Item #136968.


Issue with Currencies Included in SMF Extract

With the SMF extract, only currencies from specified fund should be extracted. However, there was an issue with the HeldOnly filter that would return the desired currencies as well as all the core currencies in Eagle. A new parameter was introduced to filter the appearance of currencies in SMF extracts. The parameter is called vIncludeCurrencyAssets. It needs to be set to Y for the currency assets to be extracted.
This issue was tracked as Item #138488.


Adjustments to Held Only Filter for SMF Extracts

The held only filter extracts the record as many times as there accounts which hold it. This is unnecessary since the record is one and the same despite the fact that it might be held by several accounts. This issue occurred with SMF Extracts. The outbound rule was fixed to recognize that a record is the same even if it is present in more than one account.
This issue was tracked as Item #138496.