Enhancements
SDP-83345 Data Rules Rejection Reprocessing Logic for Batch load to the Oracle database
Implemented batch reprocessing logic. This logic works with the RTR parameter only:
...
The dumped file in PSV pipe delimited format with the header no matter which format was initially loaded.
SDP-85570 Data Rules Rejection Reprocessing Logic for Batch load to the Oracle database
Improved Data Rules Rejection Reprocessing Logic for the Batch load to the Oracle database to handle row level errors. In addition, the eds_reprocess rule was also adjusted. Previously, when only single record reprocessing was implemented, eds_reprocess set the DO_NOT_DELETE variable to Y, in order to avoid deleting the entire batch when a single record for this batch is reprocessed. Now, when the entire batch is reprocessed, DO_NOT_DELETE has to be set to N, in order to delete the data and reload it.
Files changed:
metadata/core/ebs/eds_reprocess.json
SDP-83344 Data Rules Rejection Reprocessing functionality for single record
Below reprocessing improvements were implemented for single record:
Fixed problem with reprocessing when original message in XML format
Eliminated existing size limitation for original message
SDP-81724 Improved data structure for the SQL parameters, to store more information on validation
As part of this story, refactored new structure for SqlParam incorporated, rather than just list structure which had been in place.
...
The Validation is used only on Oracle and the next step is to extend processing rules information about (length, mandatory, dialect for date\datetime). Basic coverage of this data can be already gathered from ontology.
SDP-84271 Data Rules Validation Process as part of Rejection Reprocessing Functionality
Data Rules metadata was enhanced to have more information about the parameters for validation. Additional information was added to the ontology around stored procedure parameters in order to validate these parameters during the transformation and load process in Eagle/Oracle:
...
In addition, the process used for generating processing rules was improved to store more data about the parameters and use it during the transformation and load processes.
SDP-84987 The validation process was enhanced for data upload to the Oracle database via Data Rules
To improve end user experience, the errors returned have more details now.
For example:
for string parameter - parameter name and max length are added:
{{Param(in_investment_type) value too long DDDDDDDDDDDDDDDDDDDDDDDDD max length 2}}
for number or date - the parameter name is added.
SDP-79581 Data Rules smfidentifier Resource
The Data Rules smfidentifier resource was added to support the Eagle To Vault pipeline, to extract data from Eagle and upload to Snowflake.
The Oracle load is also supported via the PACE_MASTER.ESTAR_SEC.INSERT_UPDATE_XREF_INFO Stored Procedure.
...
Files changed:
rdf_metadata/rdf_meta_utils.py
rdf_metadata/rdf_physical_schema_generator.py
metadata/core/ontology/eagledb-inventory.json
metadata/core/ontology/eagle-editchecks.json
metadata/core/ontology/eagledb-vocab-common.json
metadata/core/ontology/reference/smfidentifiers-models.json
metadata/core/ontology/reference/smfidentifiers_in.json
metadata/core/ontology/reference/smfidentifiershist-models.json
SDP-85377 Accounting Periods Extract was enhanced to support accountingBasis filter
The ENTITYACCPERIODSEXTRACT has been enhanced to filter data based upon the ESTAR.ESTAR_ACCOUNTING_PERIODS.ACCT_BASIS using the accountingBasis parameter which was added as a paramter:
<taskParameter>
<name>accountingBasis</name>
<dataType>S</dataType>
<value>GAAP</value>
</taskParameter>
...
Files changed:
eagleml/swagger/swagger-extracts.json
eagle_ml-2-0_cm/out/tagvalue/extract_entity_accounting_periods.inc
eagle_ml-2-0_cm/out/xml/t_entity_accounting_periods.xml
eagle_ml-2-0_cm/out/xml/t_entity_accounting_periods_streaming.inc
SDP-85261 Python translator change to allow creates correct DBML request for close lots
There was a problem with the incorrect submessage type when the python translator was used. The Close Lot XSLT template was modified. The python translator now pulls the correct CLL_EGL_GEN_INTERFACE value for submessage type.
File changed:
eagle_default/in/xml/xslt/wrh_egl_gen_interface3.xsl
SDP-84859 Warehouse Open Lot outbound interface has been enhanced with new elements
The WRHSOPENLOTEXTRACT outbound interface was enhanced to support below elements:
...
Files changed:
eagle_ml-2-0_cm/out/xml/t_warehouse_openlot_streaming.inc
SDP-84771 Enhanced Data Rules extract logic to support override of group_key_expr using groupBy task parameter
Enhanced the Data Rules extract functionality by supporting the override of group_key_expr via groupBy task parameter.
...
editcheck
creatertr
executeeds
match
daterule
datastorage
SDP-84693 Intrafund Cash Transfer Interface added to generic load workflow
The Intrafund Cash Transfers is supported by the eagle_wrf_generic_load workflow with _INTRAFUNDCASHTRANSFER_ file mask.
Files changed:
eagle_ml-2-0_workflows/eagle_wrf_generic_load.wrf
eagle_ml-2-0_workflows/eagle_wrf_generic_load_accounting_objects.wrf
eis/ejmwf/eagle_wrf_generic_load.ejmwf
eis/ejmwf/eagle_wrf_generic_load_accounting_objects.ejmwf
eis/profiles/eagle_wrf_generic_load.ejmspf
SDP-84490 genericsmf resource was updated to support a few new missing fields on load to Oracle database
The genericsmf resource was updated to support additional fields for the Oracle database.
...
Files Changed:
eagle-vocab-sec-shared.json
eagle-vocab-definition-sec-shared.json
genericsmf-models.json
SDP-83786 EJM Processing Logic was enhanced for files with long file names
The EJM process logic was enhanced for file names longer than 200 characters.
...
Files changed:
eagle_ml-2-0_cm\get_w_state.inc
eagle_ml-2-0_cm\upd_w_state.inc
eagle_ml-2-0_cm\out\tagvalue\data_file_task.xml
eagle_ml-2-0_cm\out\xml\task_reporter.xml
SDP-83452 Added TBA Position netting via EagleML interface to the genericEntity
A new EagleML element was added for entityTransaction/genericEntity/netTBAEntityIndicator - Tag18577 to support TBA position netting via the EagleML interface as part of Entity election. The new XML element is supported for the load and the extract using the genericentity interface via the panel and direct SP load.
...
eagleml/schema/eagleml-entity-2.0.xsd
SDP-82995 Warehouse Position Outbound Interface has been enhanced with new elements
The WRHSPOSITIONEXTRACT outbound interface was enhanced to support additional elements for the load and the extract:
...
Files changed:
eagle_ml-2-0_cm/out/xml/t_warehouse_position_streaming.inc
SDP-82969 The WRHSOPENLOTEXTRACT logic was corrected for entities segregating derivatives
The issue was reported where the WRHSOPENLOTEXTRACT is appearing to duplicate lots on cash securities for entities segregating derivatives. The extract logic was updated as requested to correct the issue.
Files changed:
estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/tagvalue/extract_warehouse_openlot.inc
SDP-81860 warehouseperformance resource was updated to support data load to additional fields
Two database fields were missed on the data upload in the Eagle Oracle environment.
...
Files changed:
warehouseperformance-models.json
SDP-81702 New EagleML Interface for Intrafund Cash Transfers Between Custodians
The EagleML Intrafund Cash Transfers Between Custodians interface is available for load. It enables the transfer of cash for a given entity between cash accounts related to different custodian banks. Entity should be established to use cash segregation for such cash transfers.
Object type name: CustodyCashTransfer
Stream for load: eagle_ml-2-0_default_in_xml_acct_cash
Generic load workflow file mask: CUSTODYCASHTRANSFER
...
Files changed:
eagle_default/in/xml/xml-acct_cash.rsf
SDP-81701 New EagleML interface for Intrafund Cash Transfers
IntrafundCashTransfers interface is available for the EagleML load. It allows to transfer cash between cash accounts for an entity. If an entity uses cash segregation, then the client can use this option to move cash within the entity's custody basis. The interface includes entity resolution and validation checks for the entity, accounting basis, accounting periods, bank account and and cash accounts. It also can search for suitable cash accounts related to an entity in case of cash segregation.
...
Files changed:
eagle_default/in/xml/xml-acct_cash.rsf
eagle_default/in/xml/xml-ent_cash_account.xml
SDP-80988 Added feature to resolve the resourceName from the file and pass for processing to EDS
The generic load workflow, eagle_wrf_generic_load, can route files for processing by Data Rules/EDS without configuration files using a global parameter.
New RTR parameters should be used at the same time : *GetResourceName* and *UseEDS*
...
Files changed:
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/tagvalue/execute_wrkfl_event.xml
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/set_eds_params.inc
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/w_config.inc
SDP-80208 Additional Security Master fields TYPE_OF_BOND and ARM_INITIAL_FIXED_PERIOD_MONTHS to support DIS BBPS SMF
New security elements typeOfBond and armInitFixedPeriodMonths are supported in the GenericSMF interface for the load and the extract for composite and history mode.
...
Files changed:
eagleml/schema/eagleml-sec-shared-2-0.xsd
eagle_default\in\xml\xslt\smf_egl_gen_interface_template.xsl
eagle_default\in\xml\include\smf_aliases.inc
eagle_default\in\xml\include\sm_extension.inc
eagle_default/in/xml/include/mbs_extension.inc
eagle_ml-2-0_cm\out\xml\t_smf_streaming.inc
eagle_ml-2-0_cm\out\xml\t_smf_streaming_eml.inc
SDP-78140 Warehouse Position and Open Lot outbound interfaces have been enhanced with srcSysC element
The srcSysC element has been added to the Warehouse Position - WRHSPOSITIONEXTRACT and Open Lot - WRHSOPENLOTEXTRACT outbound interfaces.
...
Files changed:
eagle_ml-2-0_cm/out/xml/t_warehouse_position_streaming.inc
eagle_ml-2-0_cm/out/xml/t_warehouse_openlot_streaming.inc
SDP-52544 The issueranalytic resource was enhanced to support data load to Oracle
The issueranalytic Data Rules resource has been enhanced to support the load to Oracle using the core stored procedure ESTAR.INS_UPD_ISSUER_ANALYTICS.
...
Files added:
metadata/core/ontology/templates/eagle_ref_issueranalytic_insertupdate.txt
Production Bugs
SDP-85244 The mapping for the commitmentAmount element was corrected as part of Warehouse Position interface
The WarehousePosition load interface is fixed and now applies the value for the commitmentAmount element.
Files changed:
eagle_default/in/xml/xslt/wrh_egl_gen_interface3.xsl
SDP-85220 The Cancel Trade functionality was improved for IPT including Batch Cancels
IPT cancels used to fail in case of closed accounting period. The issue was corrected, the cancel transaction is applied to the first available accounting period now.
...
Files changed:
eagle_default/in/include/csv-cancel_columns.inc
SDP-84942 todate and fromdate filters were corrected for the reportdictionarylinkedasset resource
The todate and fromdate filters were referring to the RULESDBO.DICTIONARY.UPDATE_DATE field, but the RULESDBO.SECURITY_DICTIONARY_LINK table has the SECURITY_DATE field that works as update date field. In some cases, the SECURITY_DATE field can be empty and the DICTIONARY.UPDATE_DATE field should be used.
...
Files changed:
estar/tpe/dynamic/metadata/core/ontology/reference/reportdictionarylinkedasset.json
SDP-84915 Signage logic changes for CloseSwap and TRRESET events
The SETTLE_AMOUNT_BASE and SETTLEMENT_AMOUNT signage logic has been changed for the TRRESET / RECPT and TRRESET / DISB events in the WRHSTRADEEXTRACT. When signage logic is applied these fields will apply the revised logic for signage.
Files changed:
tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/tagvalue/extract_wrhs_trade_signage_logic.inc
SDP-84807 Forward Trades always routed to Cancel Trade panel
Forward Cancels are routed to the Cancel Trade panel only (eagle/star/trade/pan-cxltrade.htm) since the Batch Cancel panel does not support Forward Trades. Changes were done by interpreting the processingSecurityType outmapped by the eagle/star/pan-canceltradequery.htm lookup. If tag3931 = ‘FWXXXX’, then the batchId - Tag 701is removed and the data is passed to the Cancel Trade. Already canceled trades are removed from the trade lookup leaving only active trades eligible for cancelling.
...
Files changed:
eagle_default/in/include/csv-cancel_columns.inc
SDP-84481 MultilegSMF inbound interface enhanced with pulling data for existing security from DB for security update
The issue was that on initial security load some fields were calculated. On security updates, it was required to pass these fields in the incoming message.
...
File changed:
eagle_default/in/xml/xml-smf_multileg_swap.xml
SDP-83891 GenericSMF inbound interface was enhanced with skipping delisted securities functionality
Once a security becomes delisted, the Reuters Internal ID for it adds a caret symbol ^ to identify delisting of a security.
For example, the new ID will be ESRX.OQ^L18 as opposed to ESRX.OQ.
The problem was that even though the delisted Reuters ID sent against the same identifiers, a new security was getting created as the existing rule logic treated the combination of characters as a new security.
...
File changed:
eagle_default/in/xml/include/security_resolution_generic.inc
SDP-83856 Warehouse Position and Taxlots extracts did not return data for secondary basis from portfolio valuation table
The logic for the warehouse position - WRHSPOSITIONEXTRACT and warehouse openlot - WRHSOPENLOTEXTRACT extracts was corrected to include the secondary basis in the query. RTR filters have a higher priority over default logic.
...