Enhancements:
SDP-23032
...
: Updated index hint to
...
address throughput in execution of message streams
Updated additional index hint to address slowness in execution of message streams
Hint in A hint in the task reporter query for getting summary was updated which gets the summary information about loaded files was updated. It should improve the throughput time of the query execution.
Changed files:
task_reporter_generate_task_unit.inc
SDP-23005
...
: The FTP/SFTP file transfer was timing out via the Message Center curl command if files are larger than 150 MB
The issue was resolved by For larger files greater than 150MB, a timeout was occurring. The issue was resolved by adding an additional call to curl the curl command with a timeout set to 1800 seconds. New The new, optional parameter is defined as W_FTP_CURL_TIMEOUT_30MIN is available and should be set to ‘Y’ in with a value of Y in the w_config_custom.inc.
File changed:
reporter_dist_methods.inc
SDP-22742
...
:
...
Created new
...
Inbound interface for EntityLiquidity
Created a new inbound interface to load Entity Liquidity object records to the database
.Created functionality to delete the Entity Liquidity object records from the database via Inbound Interface
.The Inbound interface for EntityLiquidity
addedwas added to the eagle_wrf_generic_load workflow
.
Changed files:
eagleml_entity.xml
ent_entityliquidity_delete.inc
generate_sp_on_fly.inc
parse_ent_common_fields.inc
xml-ent_delete.xml
xml-ent_objects.rsf
Added files:
eagleml_ent_entityliquidity.xml
ent_entityregulatory_delete.inc
xml-ent_entityliquidity.xml
SDP-22672
...
: GENISSUEANALYTICEXTRACT was updated to support streaming
Rules The rules for GENISSUEANALYTICEXTRACT the GENISSUEANALYTICEXTRACT were updated to support streaming.
Files changed:
t_generic_issue_analytic_streaming.xml
t_eagleml-2-0_streaming.rsf
extract_generic_issue_analytic.inc
w_config.inc
SDP-22107
...
: TIBCOMode
...
supported for streaming extracts (T-transformation)
The parameter TIBCOMode has an impact on the mapping of attributes for extract data.
The EagleML and transactionMessage nodes do did not contain important attributes for TIBCO mode when using streaming mode.
Support was provided to the last group of extracts below:
WORKINGTRIALBALANCEEXTRACT
...
PREFERREDSHAREEXTRACT
...
UNAPPROVEDVARMARGINEXTRACT
...
WEIGHTEDAVERAGEEXTRACT
...
MDCLENEAGEGRAPHEXTRACT
...
WASHSALEEXTRACT
Extra grouping in the out_t_streaming includes was deleted (includes were optimized) for the listed extracts:
WRHSBROKTRDQUOTEEXTRACT
...
WRHSCASHACTIVITYEXTRACT
...
WRHSCASHFLOWPROJEXTRACT
...
WRHSCLOSELOTEXTRACT
...
WRHSCSTDACTIVITYEXTRACT
...
WRHSCSTDBALANCEEXTRACT
...
WRHSNAVEXTRACT
...
WRHSPENDINGTRADESEXTRACT
...
WRHSPERFORMANCEEXTRACT
...
WRHSPOSITIONEXTRACT
...
WRHSTRADEEXTRACT
...
WRHSNAVMONTHENDEXTRACT
Files changed:
1. t_eml_header_streaming.inc
2. t_rule_start_obj_header_streaming.inc
3. t_rule_start_trx_header_streaming.inc
4. t_acct_preferred_share_streaming.inc
5. t_acct_unapproved_var_margin_streaming.inc
6. t_acct_weighted_average_streaming.inc
7. t_ref_mdc_leneage_graph_streaming.inc
8. t_ref_mdc_leneage_graph_streaming.xml
9. t_warehouse_brokertradequote_streaming.inc
10. t_warehouse_brokertradequote_streaming.xml
11. t_warehouse_brokertradequote_streaming_eml.inc
12. t_warehouse_cash_activity_streaming.inc
13. t_warehouse_cash_activity_streaming.xml
14. t_warehouse_cash_activity_streaming_eml.inc
15. t_warehouse_cash_flow_proj_streaming.inc
16. t_warehouse_cash_flow_proj_streaming.xml
17. t_warehouse_cash_flow_proj_streaming_eml.inc
18. t_warehouse_close_lot_streaming_eml.inc
19. t_warehouse_closelot_streaming.inc
20. t_warehouse_custody_activity_streaming.inc
21. t_warehouse_custody_activity_streaming.xml
22. t_warehouse_custody_activity_streaming_eml.inc
23. t_warehouse_custody_balance_streaming.inc
24. t_warehouse_custody_balance_streaming.xml
25. t_warehouse_custody_balance_streaming_eml.inc
26. t_warehouse_nav_streaming.inc
27. t_warehouse_nav_streaming.xml
28. t_warehouse_nav_streaming_eml.inc
29. t_warehouse_pending_trades_streaming.inc
30. t_warehouse_pending_trades_streaming.xml
31. t_warehouse_pending_trades_streaming_eml.inc
32. t_warehouse_performance_streaming.inc
33. t_warehouse_performance_streaming.xml
34. t_warehouse_performance_streaming_eml.inc
35. tt_warehouse_position_streaming.inc
36.t_warehouse_trade_streaming.inc
37.t_warehouse_trade_streaming.xml
38. t_warehouse_trade_streaming_eml.inc
39. t_wash_sale_streaming_eml.inc
40. t_warehouse_nav_month_streaming.xml
41. t_acct_working_trial_balance_streaming.inc
42. t_acct_working_trial_balance_streaming.xml
SDP-21983
...
: Added support for
...
BatchMode
...
for
...
the Warehouse DeltaBased extracts
Logic was enhanced to allow BatchMode to be disabled for DeltaBased Warehouse Extracts via a RTR parameter:
<taskParameter>
<name>BatchMode</name>
<dataType>S</dataType>
<value>N</value>
</taskParameter>
Files changed:
...
eagle_ml-2-0_cm/extract_start.inc
SDP-21389
...
: Swagger
...
Add missing description for extracts to Swagger
Definition file was updated to support additional feed types
The definition of the following feed types have been added to the swagger-extracts.json file:
ENTITYCOLLATERALSUMMEXTRACT
ENTITYANALYTICEXTRACT
ENTITYPEERGROUPRANKINGEXTRACT
ENTITYRATINGEXTRACT
ENTITYSECLENDINGEXTRACT
ENTITYSTATISTICEXTRACT
GENISSUEANALYTICEXTRACT
PRICINGREPORTEXTRACT
ISSUERRELEXTRACT
ISSUEEXCHANGEPRICEEXTRACT
ISSUERENTRELEXTRACT
ISSUERESGEXTRACT
ISSUERRATINGEXTRACT
ISSUERREGULATORYEXTRACT
CONTROLCENTEREVENTLOGEXTRACT
EAGLEADMINISTRATIONSETTINGSEXTRACT
Support of the following Feeds have been added in Extract Service Services (MC2).
ENTITYCOLLATERALSUMMEXTRACT
ENTITYANALYTICEXTRACT
ENTITYPEERGROUPRANKINGEXTRACT
ENTITYRATINGEXTRACT
ENTITYSECLENDINGEXTRACT
ENTITYSTATISTICEXTRACT
PRICINGREPORTEXTRACT
ISSUERENTRELEXTRACT
ISSUERESGEXTRACT
ISSUERRATINGEXTRACT
ISSUERREGULATORYEXTRACT
CONTROLCENTEREVENTLOGEXTRACT
EAGLEADMINISTRATIONSETTINGSEXTRACT
SDP-20827
...
: Corrected the mapping for the AccountingTrade Python rule
The python CSV-to-EagleML transform for the AccountingTrade did not recognize security tags 1233 and 1234. Added mapping Mapping for this these tags was added in Python the python rule for the AccountingTrade, which is required for in-memory only processing.
Files changed:
t_eml_acct.py
SDP-20549
...
: Allow the Profile Loader to work without any translation required
It is now possible to configure profiled the profile generic load loader to pass an incoming file without translation. In order to do that, To configure, the user should enter "no translation" in the field where the name of translator supposed to beis expected.
The Incoming file will be passed to load by profiled profile generic load loader instance.
Files changed:
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/unpacker/custom_routing.inc
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/unpacker/send_to_wrf.xml
SDP-20513
...
:
...
Updated the Warehouse Performance rules for loading and extracting the new Liquidity data model
The EagleML Warehouse Performance incoming transformation rule and corresponding extract were enhanced to process data for the PERFORMDBO.PSR_LIQUIDITY table.
Note: The PSR_LIQUIDITY is a new table and exists is available only on for certain Eagle versions. Core A corresponding core stored procedure also should be upgraded changes outside of EagleML are required to support data loading and updates to this table.
Files changed:
...
eagle_default/in/xml/xslt/wrhs_performance.xsl
...
eagle_default/in/xml/include/xml-dbdirect_wrhs_delete_statement.inc
...
eagle_ml-2-0_cm/out/tagvalue/extract_warehouse_performance.inc
...
eagle_ml-2-0_cm/out/xml/t_warehouse_liquidity_fields_streaming.inc
...
eagle_ml-2-0_cm/out/xml/t_warehouse_performance_streaming.inc
SDP-20512
...
:
...
Outbound interface for Entity Liquidity was created
Created Outbound interface for Entity Liquidity
The outbound interface for EntityLiquidity object was created.
Feed type: ENTITYLIQUIDITYEXTRACT.
Supported EagleML filters:
Date Filter
Sources Filter
Entity Filter
User Group Filter
Common Filter
Changed files:
extracts.rsf
extract_service.rsf
t_eagleml-2-0_streaming.rsf
w_config.inc
Added files:
extract_entity_liquidity.xml
extract_entity_liquidity.inc
t_entity_liquidity_streaming.xml
t_entity_liquidity_streaming.inc
extract_service_entity_liquidity.xml
SDP-20149: Added XSD Enumeration for the User and the UserGroup in SCIM format
Added XSD Enumeration for the User and
...
Added XSD Enumeration for User and UserGroup the UserGroup objects in SCIM format
The Extract rules were updated to provide support for Object Types – ScimUser and ScimGroup.
File changed:
• /eagle_default/in/xml/xml_user_objects.rsf
• /eagle_ml-2-0_cm/out/xml/t_user_group_scim_streaming.inc
• /eagle_ml-2-0_cm/out/xml/t_user_user_scim_streaming.inc
SDP-19866
...
: The information about incoming
...
files is stored in
...
the database by default
Following The following parameters have been set with a default value in the eagle_ml-2-0_cm/w_config.inc:
:W_WAITEVENTS_DATABASE_MODE: := 'Y';
:W_ENABLE_EXTENDED_FILE_STATISTICS: := 'Y';
These options enable logic for storing extended information about the incoming files in Data Base the database instead of a file on the app server.
SDP-19850
...
:
...
Updated the Warehouse Position rules for loading and extracting the new Liquidity data model
The EagleML Warehouse Position incoming transformation rule and corresponding extract were enhanced to process data for the HOLDINGDBO.POSITION_DETAIL_LIQUIDITY table.
Note: The POSITION_DETAIL_LIQUIDITY is a new table and exists available only on for certain Eagle versions. Core A corresponding core stored procedure also should be upgraded changes outside of EagleML are required to support data loading and updates to this table.
Files changed:
/eagle_default/in/xml/xslt/wrh_egl_gen_interface3.xsl
/eagle_default/in/xml/xslt/wrhs_performance.xsl
/eagle_default/in/xml/include/xml-dbdirect_wrhs_delete_statement.inc
/eagle_ml-2-0_cm/out/tagvalue/extract_warehouse_position.inc
/eagle_ml-2-0_cm/out/xml/t_warehouse_liquidity_fields_streaming.inc
/eagle_ml-2-0_cm/out/xml/t_warehouse_position_streaming.inc
SDP-19625
...
: Added support of JMSExtractDeliveryStream and CollectDistributionErrors RTR parameters
...
in generic extract workflow
Support of Added support for two new RTR parameters JMSExtractDeliveryStream and CollectDistributionErrors was provided in the eagle_wrf_generic_extract workflow if when the ExtractDeliveryMethod='JMS'.
The corresponding global parameter is are also supported with the W_COLLECT_DISTRIBUTION_ERRORS
Files changed:
eagle_default_cm_file_distribution.xml
reporter_dist_methods.inc
eagle_wrf_generic_extract.wrf
eagle_wrf_generic_extract.ejmwf
SDP-18029 Update PRICINGREPORTEXTRACT to include additional elements
global parameter.
RTR Sample Parameters:
<taskParameter>
<name>CollectDistributionErrors</name>
<dataType>S</dataType>
<value>JMS</value>
</taskParameter>
<taskParameter>
<name>JMSExtractDeliveryStream</name>
<dataType>S</dataType>
<value>JMS_stream_sample_extract</value>
</taskParameter>
w_config_custom Sample Parameters:
<COL TAG="W_COLLECT_DISTRIBUTION_ERRORS" EXPRESSION= "'JMS'"/>
<COLTAG="W_JMS_DEF_EXTRACT_STREAM"EXPRESSION="'JMS_EAGLE_extract'"USERDESCRIPTION="Default JMS EXTRACT Delivery Stream"/>
Files changed:
eagle_default_cm_file_distribution.xml
reporter_dist_methods.inc
eagle_wrf_generic_extract.wrf
eagle_wrf_generic_extract.ejmwf
SDP-18029: The PRICINGREPORTEXTRACT was updated to include additional elements
Following The following new elements were added to Pricing Report:
priceLevel
levelComments
reason
comments
marketDate
marketDaysStale
Added MC2 support for PRICINGREPORTEXTRACT
Modified common source filter to support MC2the PRICINGREPORTEXTRACT
Changed files:
extract_filter_sources.inc
extract_pricing_report.inc
t_pricing_report_streaming.inc
SDP-16994
...
: The XSD version was added to the w_config_version.inc file
Extract The extract always gets the XSD version from the main xsd file. Added To simplify debugging, made changes allow to store the XSD version as the global parameter g_EagleML20_XSDRevision in “wthe w_config_version.incâ€inc. The parameter is set by the SetupEagleVersion process via the eagle_ml-2-0_default_cm_w_setup stream. Currently extracts Extracts now read this parameter g_EagleML20_XSDRevision once, instead of from the eagleml-main-2-0.xsd file each timeexecution.
Files changed:
w_setup_eagle_version.xml
get_revision_num.inc
SDP-15946
...
: Optimized execution of sql queries for most used EJM includes
The execution of sql requests was optimized.
The bind Bind variables were added when executing sql queries for most used includesthe most common shared include files.
Files changed:
execute_task.inc
control_workflow_paused.inc
get_stream_info.inc
check_env_name.inc
calc_date_from_rule.inc
check_stream_status.inc
include\get_sp_params_list.inc
SDP-15247
...
: Rating extract supports date and datetime format
The Rating Extract was enhanced to support both date and datetime format for submittedDate field submittedDate.
Files changed:
/eagle_ml-2-0_cm/out/tagvalue/extract_rating.inc
/eagle_ml-2-0_cm/out/xml/t_rating_streaming.inc
SDP-14722
...
: Error message was improved in case of failure of accounting trade
Previously the 'UNKNOW' error was returend returned in case no panel was selected for the accounting trade load. This is generally because the message is missing a valid event type, processing security type, etc.
The Error description was imporvedimproved:
'No panel found to load. Please check values for tag55, tag3931 and tag4533.'
File changed:
accttrade_routing_check.inc
csv-buy_panels.inc
csv-sell_panels.inc
xml-accttrade_buy.xml
xml-accttrade_sell.xml
SDP-10431
...
: Enhanced the Profile Loader to support processing of multiple archived files
Support for the processing of archived
...
User is now able to use packages that contain one or several files as source files for profiled generic load and configure translation and outgoing feed types for each file from package separately.
To specify packaged files user should use prefix of package mask separated from processed file by slash symbol, data has been added to the profile loader workflow. In the past, it could only receive plain uncompressed files. It is now possible to configure the loader for the condition when data comes in from one or multiple archives.
The following changes in the profiles syntax have been added:
If the file is part of the archive, the archive mask should be specified before the file mask as follows: [package mask]/[
...
sourсe file mask]
...
Only one file per profile should be a bundle trigger.
...
To mark a file as a bundle trigger, prefix it with the tilde symbol: ~bundle_archive/source_file
Archived and plain files may appear together in one profile.
Files changed:
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/ndfa/launch_load_by_profile.inc
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/ndfa/ndfa_exec.xml
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/unpacker/custom_routing.inc
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/unpacker/send_to_wrf.xml
eagle/estar/tpe/dynamic/pyrules/eagleinvsys/wrf_generator/_main_.py
Files added:
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/run_wrf_instance.inc
SDP-9466: Added the ability to extract a Static or Dynamic Entity List
Added two new FeedTypes were added:
STATICENTITYLISTEXTRACT
DYNAMICENTITYLISTEXTRACT
Both feeds are based on ENTITYLISTLISTEXTRACT rules and have the same output structure and are supported by Generic Extract workflow.
STATICENTITYLISTEXTRACT – extracts only static Entity with type LIST, dynamic lists will be ignored.
EntityFilter(Optional) – type can be only EntityList or ProcessCenter.DYNAMICENTITYLISTEXTRACT– extracts only dynamic Entity with type LIST, static lists will be ignored.
EntityFilter(required) with type EntityList
Changed files:
eagle_ml-2-0_cm/extract_filter_entity.inc
eagle_ml-2-0_cm/out/ndfatagvalue/launchextract_loadentity_bylist_profilelist.inc
eagle/estar/tpe/dynamic/msgcenter/_ml-2-0_cm/out/tagvalue/extracts.rsf
eagle_ml-2-0_cm/out/ndfaxml/ndfaextract_execservice.xmlrsf
eagle/estar/tpe/dynamic/msgcenter/_ml-2-0_cm/out/xml/extract_service_start.inc
eagle_ml-2-0_cm/unpacker/custom_routing.inc
eagle/estar/tpe/dynamic/msgcenter/eagleout/xml/t_eagleml-2-0_out_extract_service.rsf
eagle_ml-2-0_cm/out/unpackerxml/send_to_wrf.xml
eagle/estar/tpe/dynamic/pyrules/eagleinvsys/wrf_generator/_main_.py
Files added:
eagle/estar/tpe/dynamic/msgcenter/t_eagleml-2-0_streaming.rsf
eagle_ml-2-0_cm/out/xml/t_entity_list_list_streaming.inc
eagle_ml-2-0_cm/runw_wrf_instance.inc
Production Bugs
SDP-23322 task_reporter failes to assemble ConsolidatedTSR for workflow that waits for external data if waitevents DB mode or extended statistics is enabled
...
config.inc
Production Bugs
SDP-23322: Corrected logic for CreateConsolidatedTSR query
The CreateConsolidatedTSR query was working incorrectly in case of when the varchar columns in sub-select the subselect contained non-alpha values, and as alphanumeric values. As a result, the ConsolidatedTSR was not created. This issue was fixedidentified and corrected.
Changed files:
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/xml/task_reporter.xml
SDP-22713
...
: Fixed lastTradeDate mapping for Exchange Price Object type
Value The value for the lastTradeDate is loading correctly now via the RDC Price Evnet Event and panel.
Changed files:
xml-ref_security_pricing_common.inc
SDP-22593
...
: Resolved issue with malformed CASHJOURNALEXTRACT file related to Records grouping
The records grouping was removed from transformation. Every extracted record is now enveloped under a separate <accountingTransaction> node.
Changed file:
t_acct_cash_journal_streaming.inc
SDP-22083
...
: Logic for the Distribution Rate Summary model was improved
...
within the WRHSNAVMULTIEXTRACT
The total summary value in the Distribution Rate model was equal to sum of rate the rates of all processed records for a given Portfolio Account, Valuation Date and Share Class.
Such approach had a problem in case of canceled records. Calculations The subtotals were corrected to properly account for canceled records.
The logic was changed to reflect the correct total sum reversing cancels when they are present in the dataset.
Changed rules:
• extract_nav_multi_me_details_tables.inc
SDP-20418
...
: GenericSMF load replaces some xref data with Primary Asset or Alt Asset values
When the GenericSMF is was loaded via panels (accountingValidationFlag=Y) and the Primary Asset Type, and/or Alt Asset Type has had any values in the Xref ('ISIN', 'SEDOL', 'RIC', 'BBID', 'SICOVM', 'VALOREN', 'CMM', 'INTERNAL', 'CINS') some xrefs are were replaced by the Primary Asset Type or Alt Asset Type values.
Added logic to reorder of the xref before sending to the panel based on the panel replacement logic. Changes were made for the following SMF types: Equity ('EQCSCS', 'EQCSPF', 'EQCSPK'), ; Mutual Fund ('EQEQMF'), ; and Long Term Debt ('DBFLTP', 'DBIBFD', 'DBIBMU', 'DBIBPK', 'DBIVIV', 'DBMFDD', 'DBAMTL', 'DBIBTL').
SDP-20235
...
: Entitlement filter for SP-Based Extracts was improved
Fixed issues related to the Entitlement filter with the attribute "all entities are included" in SP-Based Extracts.
The following changes were implemented:
1) If the Entity filter and the Entitlement filter (RTR parameter is ‘usergroups’‘usergroups’) are used at the same time, the Entitlement filter works as an additional restriction for Entity filter (default logic).
2) If the Entitlement filter is only used (RTR parameter is ‘usergroups’‘usergroups’), the extract performed performs according to the Business group assigned:
When the Business group with a list of specific entities is used,
thenit works as the source for allowed portfolios for next processing in extract (default logic).
When the Business group with the attribute "all entities are included" is used, the extract will be cancelled with the error message: "User Group with access to all entities cannot be used without Entity filter".
This logic works only for following extracts:
CASHJOURNALEXTRACT Accounting Cash Journal
COMPLIANCE2A7PRICINGEXTRACT Accounting Compilance 2a7 Picing Report
CONSEARNEDINCOMEEXTRACT Accounting Cons Earned Income
EXPENSEACTIVITYEXTRACT Accounting Expense Activity
FAIRVALUECOMPARISONEXTRACT Accounting Fair Value Comparison
POSITIONASOFEXTRACT Accounting Position AsOf
REALIZEDGAINLOSSEXTRACT Accounting Realized Gain Loss
TRIALBALANCEEXTRACT Accounting Trial Balance
UNAPPROVEDVARMARGINEXTRACT Accounting Unapproved Var Margin
UNSETTLEDTRANSACTIONSEXTRACT Accounting Unsettled Transactions
WEIGHTEDAVERAGEEXTRACT Accounting Weighted Average
WORKINGTRIALBALANCEEXTRACT Accounting Working Trial Balance
...