EagleML Release Notes - August 2019

Owned by Olga Galis (Deactivated)
Last updated: Nov 13, 2024 by Kinga Huszno
6 min read

Enhancements:

SDP-25546: Cancelled Status was Implemented for Cancelled EJM Tasks

A ‘Failed’ status was previously displayed for canceled EJM processes in Automation Center and the Monitoring Tool which did not reflect the actual status. A revised status of ‘Canceled’ was implemented for cancelled EJM Tasks starting with v17R2.16.  Now when a process is cancelled, it will now display a status of 'Canceled.'

Files changed:

cancel_subtasks.inc
control_cancel_process.inc
task_reporter.xml

SDP-25032: DeltaBased Warehouse Position Extract was Updated

A new, additional filter was implemented for DeltaBased Mode Warehouse Positions Extract, extractLevel = 'PDB' (position Detail Batch).
The extract will pull all the records from the POSITION_DETAIL table tied to one position, if at least one of them was updated since previous extract.

RTR parameter example:

<taskParameter>

<name>DeltaBased</name>

<dataType>S</dataType>

<value>Y</value>

</taskParameter>

<taskParameter>

<name>extractLevel</name>

<dataType>S</dataType>

<value>PDB</value>

</taskParameter>

Files changed:

eagle_ml-2-0_cm/out/tagvalue/extract_warehouse_position.inc

SDP-24970: Generic Extractor Optimization

The Generic Extractor workflows were improved and redesigned to allow parallel execution of extracts.

The following workflows were redesigned:

  • eagle_wrf_generic_extract

  • eagle_wrf_generic_extract_system_objects

  • eagle_wrf_generic_extract_warehouse_objects

Redundant sub-processes were removed from the workflows listed below, and moved under the main workflow:

  • eagle_wrf_generic_extract_entity_and_rule_objects.wrf

  • eagle_wrf_generic_extract_financial_event.wrf

  • eagle_wrf_generic_extract_reference_objects.wrf

  • eagle_wrf_generic_extract_reference_objects_ind.wrf

  • eagle_wrf_generic_extract_smf_objects.wrf

Files changed:

eagle_ml-2-0_workflows/eagle_wrf_generic_extract.wrf
eagle_ml-2-0_workflows/eagle_wrf_generic_extract_system_objects.wrf
eagle_ml-2-0_workflows/eagle_wrf_generic_extract_warehouse_objects.wrf

SDP-24646: Resolution Logic was Improved for Issuer ESG Interface

Logic was added to resolve the Issuer id based on security identifiers, when the Issuer Id is not in the incoming message for the IssuerESG Load interface.

The new enhanced logic for the ESG interface resolves the security based on the xref identifiers and then uses the role type from the incoming message to find the Issuer Id

from the table securitydbo.issuer_role_details where the security_alias = resolved security_alias, role_type = role type from the incoming file, the end_date is null and the percent_owned = 1.

Files changed:
xml-ref_issuer_esg.xml

SDP-24308: Batch Mode was Implemented for Issuer Role

Batch Mode was Implemented for Issuer Role. Batch Underlying Load process first deletes all existing Issuer Role records from tables ISSUER_ROLE_DETAILS and ISSUER_ROLE_DETAILS_HIST with the following key: security + role type and after that loads incoming records.

Records are grouped by security + role type and sorted by time periods (from earliest to latest) and contain element referenceTransaction/header/batchKey as SECURITY IDENTIFIER^IDENTIFIER TYPE^ROLE TYPE. After that, common Issuer Role load logic is applied.

Files changed:

xml-ref_issuerrole_common.inc
xml-ref_objects.rsf

New files:

xml-ref_issuerrole_batch.xml
eagleml_smf_issuer_role_tagvalue.xsl

SDP-24298: TSR Message Distributed to JMS Broker Available in EJM Monitoring Screen

Added support to display the TSR message in EJM and Monitoring Tool for In-Memory Processing.

The feature to display the TSR is optional. The TSR will be displayed in case the ENABLE_TSR_LOGGING flag is set in <processingOptions> for the payload message, or if the EnableTSRLogging task parameter is set to 'Y' in RTR, or if the process fails.

<taskParameter>

<name>EnableTSRLogging</name>

<dataType>S</dataType>

<value>Y</value>

</taskParameter>

Files changed:
realtime_ejm_stat.xml
realtime_rtr.xml
task_reporter.xml

SDP-24286: Generic Extract Workflow was enhanced and may be launched under both Engines MC0 or MC2

The MC2 workflow was outdated and did not contain newly added interfaces. In order to avoid double maintenance and to synchronize the code base, the generic extract workflows for classic mc and mc2 were merged into a single Generic Extract Workflow (eagle_wrf_generic_extract), which may be launched under either engine.

Files changed:
eagle_wrf_generic_extract workflow related files

SDP-24151: Corrected filenames for underlying feeds within the eagle_ml-2-0_default_in_xml_issuer_mi stream in case of a ConsolidatedTSR

The logic for filename creation was updated for underlying feeds like Issuer Organization, Issuer Relationship and Issuer Xref to be in sync with other interfaces.

Files changed:
eagle_default/in/xml/xml-ref_issuer.prf

SDP-24150: Mapping for Underlying Security was Corrected

Previously if tag5399 was passed, it was assigned to tag1348. The logic was adjusted by removing the conditon for tag5399.  The change enables the underlyingCusip to be properly mapped and the correct value assigned.

Files changed:
eagle_default/in/xml/include/get_additional_underlying_smf_fields.inc

SDP-23124: Additional Logic was Introduced for Underlying SMF

A new load option was added for the Underlying SMF to control the logic for the start date and end date for underlying security relationships.

A new optional parameter underlyingRetainBeginDateSwitch may be set to Y in the incoming message (referenceTransaction/underlyingSMF/underlyingRetainBeginDateSwitch).

The new setting enables the following logic:

  • Maintains a single underlying record per relationship with a null end date, where the record key= Security alias + Underlying type

  • Updates the existing DB record in case the record key= Security alias + Underlying type + Underlying security alias, but does not update the existing begin date in DB

  • Retains all the default rules for underlying SMF records

Files changed:
xml-smf_underling.xml
xml-smf_underling_common.inc

SDP-23045: New Concatenated ID was added to the WRHSCLOSELOTEXTRACT to supplement the calcMasterTransactionId

A new field, 'calcMasterCloseTransactionId', has been added for the CloseLot Extracts which is created if te RTR-parameter 'EnhanceCalcTransactionId' = 'Y'.
The field contains the concatenated value from the 'acctBasis', 'entityId', 'securityAlias', 'longShortIndicator' and 'eventId'. The new ID links the Disposal lot back to the Closet Lot Sell transaction.

<taskParameter>

<name>EnhanceCalcTransactionId</name>

<dataType>S</dataType>

<value>Y</value>

</taskParameter>

Files changed:
extract_warehouse_closelot.inc
t_warehouse_closelot_streaming.inc
t_calc_link_identifiers_fields_streaming.inc

SDP-22633: The AccountingCash Interface was enhanced to expand support for Miscellaneous Income and Expense Cash

The AccountingCash Interface was enhanced to support the cashEventType element for Miscellaneous Income and Expense Cash message types (Cash TransactionType = MISCINC or MISCEXP).
Possible values for the Cash Event Type: DISB, RECPT.  Examples: <cashEventType>DISB</cashEventType> or <cashEventType>RECPT</cashEventType>

Files changed:
eagle_default\in\include\cash_aliases.inc
eagle_default\in\include\csv-cash.inc

SDP-22206: New EarningsTimePeriod Extract

Created a new interface for the EarningsTimePeriod EagleML object.

The new feed type, EARNINGSTIMEPERIODEXTRACT, was implemented. The Extract is supported via the generic extract workflow.

Supported filters:

  • Date filters (begindate, enddate, updatedate);

  • SMF filters

  • Common filters

  • DeltaBased

Files changed:

  • metadata/bind/eagleml_reference.xml

  • eagle_default/in/xml/xml-ref_objects.rsf

  • eagle_ml-2-0_cm/extract_filter_date.inc

  • eagle_ml-2-0_cm/out/tagvalue/extracts.rsf

  • eagle_ml-2-0_cm/out/xml/t_eagleml-2-0_streaming.rsf

  • eagle_ml-2-0_cm/out/xml/extract_service.rsf

  • eagle_ml-2-0_cm/out/xml/extract_service_start.inc

  • eagle_ml-2-0_cm/out/xml/t_eagleml-2-0_out_extract_service.rsf

New files:

  • metadata/bind/eagleml_ref_earningstimeperiod.xml

  • eagle_default/in/xml/xml-ref_earningstimeperiod.xml

  • eagle_ml-2-0_cm/out/tagvalue/extract_earnings_time_period.inc

  • eagle_ml-2-0_cm/out/tagvalue/extract_earnings_time_period.xml

  • eagle_ml-2-0_cm/out/xml/t_earnings_time_period_streaming.inc

  • eagle_ml-2-0_cm/out/xml/t_earnings_time_period_streaming.xml

  • eagle_ml-2-0_cm/out/xml/extract_service_ref_earnings_time_period.xml

Production Bugs

SDP-25743:  CorrelationId Field was Adjusted to 100 Characters for Real Time and for EJM Processes

A prior alphanumeric limit set to 50 characters for the in_corr_trans_id in the update message statistics SQL statements was creating challenges when the value was over 50. Also, the Correlation ID length was not validated properly by the Real Time Processing. In order to handle the error for the character limit for correlation ID's the following change was introduced:
The configuration parameter g_nCorrIdMaxLength will be used by Real Time and EJM processes.
The default g_nCorrIdMaxLength is set to 100.
The CorrelationID limit check for RealTime processes was extended to 100 and error message will now be produced if the limit is exceeded.

Files changed:
eagle_ml-2-0_cm/check_cm_limits.inc
eagle_ml-2-0_cm/out/tagvalue/realtime_ejm_stat.xml
eagle_ml-2-0_cm/in/xml/realtime_rtr.xml
eagle_ml-2-0_cm/w_config.inc

SDP-25539: Field acementNumber was deleted for Warehouse Trade Interfaces

The field 'acementNumber' was removed from Inbound and Outbound rules for Warehouse Trade.

Files changed:
t_warehouse_trade.inc
t_warehouse_trade_streaming.inc

SDP-25441: Corrected Failures on the User load in case Panel is used for processing

When the passwordSetDate field was null, a failure was being produced.  To prevent this behavior, the inbound rule was enhanced to use the default value for this field as Date for versions less than v17 and Date + Time for versions higher than v17.   The XSD file was also updated to support DateTime format.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_user_user.inc
eagle_ml-2-0_cm/out/xml/t_user_user_streaming.inc
eagle_default/in/xml/user_common.inc

SDP-25372: GPG/PGP File Extensions are part of TSR for encrypted extract files

When the extract file is encrypted, the corresponding extension is added to file name (.gpg if GPG or GPG2 and .pgp if PGP). This was not part of previous eagle_wrf_generic_extract logic and is implemented by the eagle_ml-2-0_default_cm_file_distribution task which is responsible for the extract distribution and handling after packing.  This also updates the main workflow extractName status variable in case encryption is part of distribution.

Files changed:
eagle_ml-2-0_cm/out/tagvalue/eagle_default_cm_file_distribution.xml
eagle_ml-2-0_cm/reporter_dist_methods.inc

SDP-25033: Delete Logic for Batch Mode was Corrected

The logic was adjusted to correct delete behavior for lots and positions. Records were missing from the LOT_LEVEL_POSITION and the POSITION_DETAIL tables if the W_WRHS_IGNORE_POS_FLAG was set to Y.  The SQL query 'Run Batch Del for Open Lot(4 key batch)' was corrected.

Files changed:
\in\xml\include\xml-dbdirect_wrhs_delete_statement.inc

SDP-24873: Fixed an issue with Custom filters not working for WRHSSPDDELTALOGEXTRACT in Delta Mode

When requesting the WRHSSPDDELTALOGEXTRACT the custom filters were not applied to the extract query. The root case was an incorrect sequence within the include files in the corresponding extract rule. The rule file was fixed to include the custom filters.

Files changed:
eagle_ml-2-0_cm\out\tagvalue\extract_warehouse_spd_delta_log.xml

SDP-24688: EntityToComposite Update Logic was Corrected

EntityToComposite update logic was adjusted for existing records in case the data is loaded with the submitOptionsFlag value set to Y (Load Composite & History) or F (Apply Forward).

When the EntityToComposite data is loaded with the submitOptionsFlag value H (Load History Only) or the submitOptionsFlag is omitted and record with the same effective date already exists or incoming effective date is less than max effective date, then the error will be returned based on the SP and the panel logic.

Files changed:
eagle_default/in/xml/xml-ent_entitytocomposite.xml
metadata/bind/eagleml_entity.xml
metadata/bind/eagleml_ent_entitytocomposite.xml

SDP-24245: Inbound Interface for Accounting Asset Cash was updated for Accounting Cash Income Misc

The inbound interface for Accounting Asset Cash for assetCashType=’AMISCINC’ was updated.  The logic for tag85 and mapping for tag5363 was corrected. Now users can specify the Settlement Currency separately from the asset currently via the RTR.

Files changed:
eagle_default/in/xml/xml-acct_cash_asset.xml
eagle_default/in/include/csv-cash_misc_inc_exp.inc

SDP-23759: Retry Logic was added for Warehouse load in Batch Mode

Retry logic was introduced for the Warehouse load. This logic will be triggered only in sub-batch load mode and if the stored procedure returns an integrity constraint error. In this case, the rule will try to load the failed sub-batch one more time. If after max retry counts errors still exist, the message will be marked as failed. 

Retry logic may be adjusted via the w_config_custom.inc:
• W_CONFIG_WRHS_MAX_RE_TRY_CNT is MAX number of re-try attempts. The default value is 3. 
To disable retry logic, set the value to 0.
• W_CONFIG_WRHS_RE_TRY_WAIT – Amount of seconds between the next load attempt. the default value is 1.

SDP-23397: Key for GROUPBY was corrected in the ISSUEFXRATEEXTRACT

The ISSUEFXRATEEXTRACT was working incorrectly if the streaming rule was used for processing. The key for GROUPBY was updated to fix the issue with grouping.

Files changed:
eagle_ml-2-0_cm/out/xml/t_issue_fx_rate_streaming.inc

SDP-23025: Correlation ID added to the Task File Name

When an extract requested returned NO_DATA, the task file was created with a non-unique file name like FeedType + totalCount + currentDate.  It resulted in an incorrect display of the workflow steps that ran previously that did not belong to the current workflow instance.  To resolve, the Correlation ID was added to the Task File Name to eliminate identical file names.

Files changed:
eagle_ml-2-0_cm/extract2t.inc