EagleML Release Notes - September 2016

This release uses XSD Version 230.

There are four overlay files that are currently packaged as part of the EagleML monthly release. Please refer to About EagleML Overlays for specific details and impact to you.

Enhancements

EagleML

Introduced New NaicDesignation Object

EagleML was enhanced to include a new object to allow for loading and extracting from the NAIC Ratings table (istar.istar_naic). For this enhancement, the XSD object NaicDesignation was created which includes the following models:

  • AssetIdentifiers.model
  • IssuerIdentifiers.model
  • NaicRates.model

This issue was tracked as Item #140595.


Added Ability to Determine Your EagleML version from the Front End

Previously in order to find out what EagleML version your environment was using, you had to look at the file /eagle_ml-2-0_cm/w_config.inc file. The code was enhanced to make it easier to find out your EagleML version. The new stream eagle_mc_server_support stream can now be run with overrides in order to return the EagleML version in use. The Action field must be set to: "EAGLEML VERSION" and the ActionType field must be set to: "Get Version" before the run is executed. In the messages that this stream returns, the EagleML version can be found.
This issue was tracked as Item #141567.


Added Asset Properties Model in Accounting Extracts

The Asset properties Model was not included in accounting extracts as it is in the warehouse extracts. The capability to allow this model was added in the following feed types:

  • GENCORPACTIONSEXTRACT
  • SETTLEDCAEXTRACT
  • TRADEDCAEXTRACT
  • FACTOREXTRACT
  • GLACTIVITYEXTRACT
  • RATINGSEXTRACT
  • SCHEDULEEXTRACT

The inclusion of this model in these extracts is controlled by adding a parameter called IncludeAssetProperties equal to "Y" in the RTR or by setting the :W_INCLUDE_ASSET_PROPERTIES: variable to "Y" in the w_config.inc file.
This issue was tracked as Item #142283.


Add additional details to TSR for STAR Processes
The TSR logic was enhanced to include additional details for STAR processes. Specifically, entity Id information that shows in the event details in the STAR Schedulure is now included in the TSR. As the entity Id information was determined to be the most helpful, the code was updated to include entity Id data now. In future releases, the TSR will be enhanced further to include additional data captured in the event details as well.
This issue was tracked as Item #142698.


Source Interface Name Added to ISSUEPRICEEXTRACT

Source Interface Name was added to the ISSUEPRICEEXTRACT. The code that handles the extract from the database was added to the existing SQL, allowing the framework to grab the extra field and include it in core extracts.
This issue was tracked as Item #143199.


Added Entitlement and Partition Support to Several EagleML Extracts

EagleML was enhanced to support the entitlement and partition filters that were added to several EagleML interfaces. The following interfaces were updated to allow using the filter during extracts:

  • UNSETTLEDTRANSACTIONSEXTRACT (Entitlement model delivered in August, only Partition needed)
  • WRHSCLOSELOTEXTRACT (Entitlement model delivered in August, only Partition needed)
  • SETTLEDCAEXTRACT
  • TRADEDCAEXTRACT
  • WRHSCASHACTIVITYEXTRACT
  • CASHPROJECTIONEXTRACT
  • ENTITYACCPERIODSEXTRACT
  • ACCOUNTINGTRADEEXTRACT

This issue was tracked as Item #143617.


Added Option to Extract SMF History with Xrefs from Xreference table

In I2I there was a problem when extracting SMF objects. The procedure requires all xrefs to be extracted from the SPOKE, but the xrefs are grabbed from the xref_hist table which may not capture all of the xrefs. As a result of this, there could be missing xreferences in the extract. To resolve this issue, a new parameter was created to allow the xreferences to be extracted from the xreference table rather than the xref_hist table as long as the value in the parameter is "Y".
This issue was tracked as Item #144145.


Entitlement and Partition Support Added to Several EagleML Extracts

Support for the entitlement and partition filters were added to several EagleML interfaces. The following interfaces were updated to allow using the filter during extracts:

  • CASHJOURNALEXTRACT PANEL (Y)
  • REALIZEDGAINLOSSEXTRACT

This issue was tracked as Item #144591.

EJM

Automation of Engine Cache Clearing

Previously, the only way to refresh the engine cache was manually. EJM was enhanced to include functionality that allows the engine cache to be refreshed by a control message in the form of a run task request that includes three pertinent parameters. The streamName will always be eagle_ml-2-0_default_cm_refresh_engine_cache. The commands parameter can take either a single command or multiple commands separated by the "|" (pipe) symbol. Finally, the runType param will either refresh every process if set to "Y" (ignoring all the commands in the command param) or it will defer to the commands parameter for execution.
This issue was tracked as Item #142440.


Limit Size of Output for ODS Consumption

Sync WS API calls sometimes return large data payloads which can be difficult to deal with. EJM was enhanced to handle situations like this. The Java code was edited to allow the payload output to be compressed in a zip file in the case of large output volume.
This issue was tracked as Item #144756.

Software Fixes

EagleML 

UniqueProductId and UniqueSwapId Elements Were Not Mapped in Credit Index Swap SMFs

The uniqueProductId and the uniqueSwapId elements were not showing up in the Credit Index Swap SMFs. These are required elements for this security type and must be loaded into the database so that they are available for extracts and data manipulation. To resolve the issue, the elements were added to the smf_aliases.in file and were also defined in the varDefaultBattingOrder tag within the security_resolution_generic.inc file.
This issue was tracked as Item #142548.


Updated Support of Aladdin DMF

Due to the elimination of the underlyingsmf_model.inc file from the xml-smf_generic.xml rule, a part of the ignore logic was lost for underlying records. This issue only occurred with already existing ALADDIN SMFs. The impact of this problem was that in the case that an underlying SMF that already existed in the database is loaded, the load should be stopped and the record should not be updated. The following code in the underlyingsmf_model.inc was added to the generic_smf_aladdin.inc file so that the ignore logic can be preserved:

<COL TAG="var_sql_result" EXPRESSION="IIF(Find(Upper(|objectType|),'UNDERLYING',0,1)!=-1 AND Atoi(:tag10:) > 0, 'Y', 'N')"/>

This issue was tracked as Item #142888.


ContraSecAlias was not Updated on Pay or Receive Legs

During updates on SWAPs, the ContraSecAlias (tag 915) was not populated for the pay or receive leg. Only the contract leg was populated with this information. The tag was dependent on the resolution of tag CON_10, which was dependent on the security resolution for the swap. Since the pay and receive leg don't have to go into the security resolution loop within the rule, tag CON_10 does not get any value and therefore neither does tag 9151. The logic was changed so that tag 9151 is not dependent on the security resolution of the swap.
This issue was tracked as Item #142917.


Resolved Issue with Fx Rate
Previously, you could load Issue Fx Rate data with only set <assetCurrency> nodes in the <fromCurrency> and <toCurrency> branches. The code was updated to allow you to define assetCurrency by using security parameters, such as <ticker> and/or <primaryAssetId> in the <fromCurrency> and <toCurrency> branches.
This issue was tracked as Item #142393.


Corrected LastCouponDate Typo

In the generic_smf_aladdin.inc file there was a small mistake where lastCouponDate variable was misnamed as firstCouponDate. This error was throwing off the logic, since the lastCouponDate should be populated after the firstCouponDate. Instead, due to the error the lastCouponDate was actually overwriting the firstCouponDate. To resolve this issue, the variable was switched to lastcouponDate which restored proper structure to the include file.
This issue was tracked as Item #142918.


Mapping Issue for Amereuro Field

In the generic_smf_aladdin.inc file there is a lookup that requires tag 1142 which used to be located in the generic_smf_base_model.inc file. The link to this file and therefore to the mapping for tag 1142 were unintentionally excluded as part of the June release. The exclusion of the generic_smf_base_model.inc file made the lookup in the generic_smf_aladdin.inc file inoperable. To resolve the issue, the mapping for the tag was added to the generic_smf_aladdin.inc file as well as the addition of Amereuro to the input xml alias in the eagleml_smf_gen_fields.xsl file.
This issue was tracked as Item #142921.


Restored Functionality to Reason Node in Warehouse Platform

The reason node was not being populated since the exceptions_catch.inc was placed after the xsl transformation that populated the IN_MESSAGE variable. This variable is used by the exceptions_catch.inc to assign values to the reason tag, but since the xslt was overwriting values in the IN_MESSAGE the reason tag would be ignored. To resolve this issue, the exceptions_catch.inc was moved ahead of the xslt so that the IN_MESSAGE variable would not be overwritten.
This issue was tracked as Item #143109.


ExchangeRateDate Returned Null

The exchangeRateDate returned null values because it was mapped incorrectly to the POSITION_COST_DETAIL.EXCHANGE_RATE_DATE table when it should have been mapped to the POSITION_DETAIL.EXCHANGE_RATE_DATE table. The POSITION_COST_DETAIL.EXCHANGE_RATE_DATE table did not always contain the right data and sometimes no data at all. The SQL was updated to include both tables, with preference going to the POSITION_DETAIL.EXCHNAGE_RATE_DATE table. The POSITION_COST_DETAIL.EXCHANGE_RATE_DATE table is accessed only if the first table is found to be null.
This issue was tracked as Item #143124.


Issue with Installer Stream

There were problems with the installer stream grabbing files that were located above the ../estar/tpe level. As a result, STAR Engine files were not installed when they should have been since the "starengine" folder was located in a directory above the prior mentioned pathway. To resolve this issue, the eagle_mc_server_support.inc file was edited to give capacity to the stream for installing all necessary files.
This issue was tracked as Item #143153.


Adjusted Handling of Large Amounts of Errors in Task Reporter Stream

There was an issue with a consolidate TSR message not being generated after the Bloomberg full workflow was run. This issue stemmed from the large amounts of errors in the multiple records processed from a Bloomberg full run. The current logic uses a MSGEXCHANGE that can write run data to a file on the app server in case the data flow is large. This data would then be transferred back to a TSR. The issue was that there were too many errors to put on the TSR and therefore it would not get generated. The code was updated to resolve this issue. In such cases the MSGEXCHANGE is now responsible for simply creating a TSR with a parameter that says "too many errors" and leaves all the actual descriptions within the file on the app server.
This issue was tracked as Item #143258.


Removed Tags from Voluntary CA Lookup

The voluntary CA lookup was overwriting tags that already had proper values from the original CA election record. This was causing a situation in which the ex date was later than the effective date which is not allowed and causes an error. The following three tags were removed from the voluntary CA lookup in order for this look up to not over write the effective date: 1001, 1109, 1257.
This issue was tracked as Item #143261.


Fixed Issue with Credit Default Swap SMF Extract Not Returning Data

The Credit Default Swap SMF extract was returning no data even when there was actually data that could have been extracted based on given parameters. The problem revolved around the logic that was used to replace the "(" and ")" symbols in the SQL responsible for launching the extract. This logic was causing the primary asset Id filter to malfunction. The logic was updated to allow the filter to work properly.
This issue was tracked as Item #143352.


Xref Identifiers Trimmed After Special Characters

The xref identifiers would get trimmed after any special character when being processed by EagleML streams. This would cause the xref identifiers to load incorrectly, making the data invalid. To resolve this issue, the eagle_ml-2-0_default_out_q stream xreference logic was changed to handle the special characters.
This issue was tracked as Item #143408.


Fixed issue with \5C character

The task_reporter.xml rule was not decoding the "\5C" character in the description node of the TSR. Because of this, the TSR contained many \5C snippets. The code in this part of the rule was fixed to decode the symbol correctly into its proper format.
This issue was tracked as Item #143894.


Cash Journal had Value Always Set for DetailIndicator

In the Cash Journal, the value for the detailIndicator was incorrectly hardcoded as "SETTLED ACTIVITY". The hardcoded value was replaced with an update to the cash journal rule. The eagle/star/reports/pan-ec_cashjournal.htm panel is now responsible for populating the detailIndicator based on the value in the reportView node.
This issue was tracked as Item #144009.


Fixed Binding Parameters in Financial Events Extracts

There was an issue with the feature to use bind variables when using the xref filter for extracts. The SQL generated by this feature would fail with the error: "Not all variables found". The binding parameters have been fixed in the extract_financial_event.xml rule.
This issue was tracked as Item #144057.


Resolved XML to Excel Issues with Xref Ids

When an EagleML file with more than ten xreference Ids was exported to Excel there could be problems with keeping all the ids on the same line of the excel file. This was caused by the fact that the xref nodes were set as repeating nodes and were not numbered. The Excel could not interpret them correctly as a result. To resolve this issue, a new flag called UseNumerationForXrefs was created. By default, the value for UseNumerationForXrefs is Y. This indicates that numerated xrefs will be contained to EagleML extract by default.
This issue was tracked as Item #144131.


Increased Parameter Capacity in RTRs

The number of bind parameters that it was possible to include in an RTR message was limited to 50. This limit could cause you issues if you wanted to more bind parameters. The capacity was increased to handle 100 parameters.
This issue was tracked as Item #144178.


Fixed Duplicate Securities not Reported in Batch Mode TSR

There is a feature that allows you to report all errors are once during a batch load for certain loads. This feature had an issue that involved not reporting duplicate securities in the TSR. This issue only occurred if the duplicate records were the only ones with errors in the batch file. If there are any other errors, the duplicate records got flagged and reported as well. This issue stemmed from the one by one rollback of errors. To correct this, the procedure was contained within a loop instead which resolved the issue of the duplicate records not getting caught.
This issue was tracked as Item #144597.


Next Instance Not Properly Updated

The Next Instance field was not being properly updated by EagleML. The top value from the table is grabbed and used as the max instance which is incorrect. This was causing issues with EDM inserts. The logic for getting the next instance was changed to make sure that it is calculated correctly. On a slightly unrelated note the EagleML version number format was also changed slightly by adding periods between each part of the version date.
This issue was tracked as Item #144694.

EJM

Resolved Issue with Duplicate Processing Handling

If the Message Center process is restarted mid load (for any data type on any stream), the process will attempt a restart once the Message Center process boots up. The load is executed in part by a SQL statement that inserts a request into the ORCH_REQUEST_PARAMS table. When Message Center restarts, this is carried out a second time and therefore ultimately leads to the record failing due to duplication. Logic in the eagle_ml-2-0_cm\ orch_param_add.inc rule was changed so that data is not reentered into the ORCH_REQUEST_PARAMS table and therefore will not fail due to duplicate execution.
This issue was tracked as Item #142132.


Fixed Issues with extract_demandlist.xml

The demand list extract had an incorrect file path to profiles that the rule uses as well as some missing tags that are not populated by the panel in this rule. Due to this issue, the process would wait for a profile definition that was not located in the directory that was being searched by the rule. The path was changed to set the rule onto the right directory. Tags were also added prior to the panel and to the panel INMAP section so that the extract can operate correctly based on the selected profile.
This issue was tracked as Item #143537.

I2I

Changes to I2I Delta Mode

During I2I processing there could be cases where trades that had previously been loaded to the database and both the HUB and SPOKE would be updated, resulting in a mismatch on HUB and SPOKE. The logic for processes like this revolved around using a batch key (comprised of Entity+ Effective+Source+SecurityAlias) to delete all records with the matching key and then load incoming records. This was problematic since there were times where not all records with the batch key should have been deleted. These scenarios were what cause the HUB and SPOKE to fall out of synchronization. The logic for delta mode loads for trades was changed to be similar to single row extracts. Instead of deleting all previously loaded records, the process now deletes the specific records that match the incoming file and loads them from the incoming file. Instead of using the batch mode, the load is now executed on a record by record basis.
This issue was tracked as Item #141619.