ESP-367: - EagleML - Optimize requests in task_reporter and updated to use bind parameters
...
event_orch.inc
exec_pace_event.inc
exec_pace_event.xml
monitor_delayedevent.inc
monitoring.xml
reporter_status.inc
ESP-1200: set Set uncompress_transform task status as failed if any translation step was failed
...
Files changed:
\dynamic\msgcenter\eagle_default\in\xml\xml-ref_ca.xml
ESP-1956: Encoding issue in generic extract
The EntityComposite object can now be resolved by xref id as well as the entity id.
The inbound EntityComposite interface was edited to allow for this new resolution logic.
File changed - xml-ent_entitycomposite.xml
ESP-1961: The EJM Sequencer needs to be able to work for multiple distinct instances of an EJM Workflow
Ability to execute tasks in sequencer queue in parallel was added to EJM. Now multiple instances of the same workflow can be executed simultaneously. For example the executing the generic extract workflow for different sources and data types at once is now possible. The new processingSequencerIdentifier element can be added to the header of an RTR to enable this functionality.
RTR example:
<EagleML 1938: EagleML- Include Accounting Trades as a part of in-memory processing (specific to CSV format only)
Added support for Accounting Trades in realtime processing:
- Added the eagle_ml-2-0_default_in_xml_acct_trades_no_split stream, which is based on core rules, but loads records sequentially.
- Updated python script to allow a CSV to EagleML transformation for Accounting Trade objects.
- Updated TSR generation for handling scenarios where the number of errors exceeds 1000.
- Improved query performance.
Files Changed:
realtime_rtr.xml
task_reporter_gen_task_unit_realtime.inc
ESP-1956: Encoding issue in generic extract
The EntityComposite object can now be resolved by xref id as well as the entity id.
The inbound EntityComposite interface was edited to allow for this new resolution logic.
File changed - xml-ent_entitycomposite.xml
ESP-1961: The EJM Sequencer needs to be able to work for multiple distinct instances of an EJM Workflow
Ability to execute tasks in sequencer queue in parallel was added to EJM. Now multiple instances of the same workflow can be executed simultaneously. For example the executing the generic extract workflow for different sources and data types at once is now possible. The new processingSequencerIdentifier element can be added to the header of an RTR to enable this functionality.
RTR example:
<EagleML xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="RunTaskRequest" eaglemlVersion="2-0" xsi:schemaLocation="http://www.eagleinvsys.com/2011/EagleML-2-0 eagleml-main-2-0.xsd" eaglemlType="RunTaskRequest">
<header>
<messageId>EXTRACTOR</messageId>
<sentBy>http://www.client.com</sentBy>
<sendTo>http://www.eagleinvsys.com/eagle_ml-2-0_default_cm_control_message</sendTo>
<replyTo>http://www.client.com/Eagle Control Messages</replyTo>
<creationTimestamp>2012-08-02T22:13:32.272-05:00</creationTimestamp>
</header>
<taskIdentifier>
<processingSequenceNumber>1</processingSequenceNumber>
<processingSequencerIdentifier>PROCSEQID</processingSequencerIdentifier>
<correlationId correlationIdScheme="correlationIdScheme">correlationId</correlationId>
<businessTaskId correlationIdScheme="businessTaskIdScheme">businessTaskId</businessTaskId>
</taskIdentifier>
<taskTypeEnum>NEW</taskTypeEnum>
<taskParameters>
<taskParameter>
<name>StreamName</name>
<dataType>S</dataType>
<value>eagle_ml-2-0_default_cm_execwkfl</value>
</taskParameter>
<taskParameter>
<name>ActionType</name>
<dataType>S</dataType>
<value>EXECUTE</value>
</taskParameter>
<taskParameter>
<name>WorkflowName</name>
<dataType>S</dataType>
<value>eagle_wrf_generic_extract</value>
</taskParameter>
<taskParameter>
<name>FeedType</name>
<dataType>S</dataType>
<value>SMFEXTRACT</value>
</taskParameter>
<taskParameter>
<name>maxrows</name>
<dataType>S</dataType>
<value>10</value>
</taskParameter>
</taskParameters>
</EagleML>
...
Files changed
• eagleml-sec-shared-2-0.xsd
• eagleml-wh-shared-2-0.xsd
• smf_egl_gen_interface_template.xsl
• smf_egl_gen_interface_template_mssql.xsl
• t_smf_streaming.inc
• t_smf_streaming_eml.inc
ESP-2435: unpacker stream to remove BOM if not supported by core.
Streaming rules are unable to handle any UTF-8 BOM characters.
Logic has been added to the task_unpacker stream to eliminate such characters from source files.
Changes are specifically for UNIX environments and for the following list of environments:
• EagleML20_Version < '15.2'
• EagleML20_Version = '15.2' and EagleML20_Revision < 26
• EagleML20_Version >= '17.0' and EagleML20_Version < '17.2'
• EagleML20_Version = '17.2' and EagleML20_Revision < 4
The W_RS_REMOVE_BOM_IF_UNSUPPORTED global variable controls this new functionality.
Files changed:
wrf_unzip.prf
remove_bom_chars.inc
ESP-2463: Added empty destination compatibility for profiled generic load
...
Files changed:
eagle/estar/tpe/dynamic/msgcenter/eagle_ml-2-0_cm/unpacker/custom_routing.inc
ESP-2662: GIPSDisclosures: add filters for disclosureName and disclosureType to Extract
New filters:
<taskParameter>
<name>DisclosureName</name>
<dataType>S</dataType>
<value>NAME</value>
</taskParameter>
<taskParameter>
<name>DisclosureType</name>
<dataType>S</dataType>
<value>TestValue</value>
</taskParameter>
• DisclosureName - RULESDBO.DISCLOSURE.NAME
• DisclosureType - RULESDBO.DISCLOSURE.TYPE
Files changed:
extract_ref_gips_disclosure.inc
ESP-2695: 2017ORA - REF_006_Client REF_007_ClientCommunication test cases problem
...
Files changed:
xml-ent_delete.xml
ESP-2730: EagleML Delta Based mode doesn't work correctly sometimes
The SQL query for retrieving delta information was occasionally executed incorrectly due to the PACE_MASTERDBO.ORCH_QUEUE table being included in the query. This table does not hold any critical information for delta processing, but would sometimes make the query return no records.
The table was removed from the query for more consistent delta based processing.
Changed files:
extract_delta_orch.inc
ESP-2742: Stored procedure for non-existing tables should not be created
...
File changed:
• \msgcenter\eagle_ml-2-0_cm\generate_sp_on_fly.inc
ESP-2755: add support of MS SQL for ClientCommunication object
The inbound ClientCommunication rule was updated to use Meta Information and auto-generated includes instead of the MSGESXCHANGE.
MSSQL queries were introduced for resolving instances and support for MSSQL was added to the outbound rule.
Streaming mode was enabled and new filters were implemented:
Old filters: (Still supported))
<taskParameter>
<name>selectiontype</name>
<dataType>S</dataType>
<value>CommunicationType</value>
</taskParameter>
<taskParameter>
<name>selectionvalue</name>
<dataType>S</dataType>
<value>PHONE</value>
</taskParameter>
New filters:
<taskParameter>
<name>selectiontype<<name>CommunicationType</name>
<dataType>S</dataType>
<value>CommunicationType<<value>PHONE</value>
</taskParameter> taskParameter>
<taskParameter>
<name>selectionvalue<<name>ClientId</name>
<dataType>S</dataType>
<value>PHONE</value>
</taskParameter>
New filters:
<taskParameter>
<name>CommunicationType</name>
<dataType>S</dataType>
<value>PHONE</value>
</taskParameter>
<taskParameter>
<name>ClientId</name>
<dataType>S</dataType>
<value>TestValue </value>
</taskParameter>
ESP-2759: xml-smf_generic.xml - businessCalendarName is being truncated to 10 characters
There was logic that would truncate the businessCalnedarName to ten characters. This was the correct logic for Eagle version 13 and lower, but was incorrect for versions 15 and higher.
The logic was updated to check for the version and only truncate the value if the version was 13 or lower.
These changes affected the loads of the following objects: GenericSMF, GenericSMF with accountingValidationFlag=Y, MultilegSMF.
...
dataType>
<value>TestValue </value>
</taskParameter>
ESP-2759: xml-smf_generic.xml - businessCalendarName is being truncated to 10 characters
There was logic that would truncate the businessCalnedarName to ten characters. This was the correct logic for Eagle version 13 and lower, but was incorrect for versions 15 and higher.
The logic was updated to check for the version and only truncate the value if the version was 13 or lower.
These changes affected the loads of the following objects: GenericSMF, GenericSMF with accountingValidationFlag=Y, MultilegSMF.
changed files:
contract_mapping.inc
fixedincomesmf_model.inc
generic_smf_fields_logic.inc
pay_mapping.inc
receive_mapping.inc
ESP-2774: EJM- Remove unnecessary file copy function
During extracts files were being copied unnecessarily. This was related to there being places in the rules where a file would get copied instead of moved.
Additional logic was added to the RENAME preprocessor which would make the preprocessor not run if the final file name matched the original.
The MSGEXCHANGE that is responsible for sending files to JMS was changed to move the files rather than copy them.
If the delivery method for a file is JMS then the file will no longer be moved to the cmw/extracts folder.
An additional flag was added for deleting the original archive in case of base 64 encoding.
Files changed:
compress_extract.inc
reporter_dist_methods.inc
ESP-2775: Running SMFEXTRACT in history mode showing ORA-00904: "INTERFACES"."INSTANCE": invalid identifier
...
Files changed:
-eagle_ml-2-0_cm/out/tagvalue/extract_smf.inc
ESP-2778: GENCORPACTIONSEXTRACT running through Generic Extract in MC not producing a file to a custom directory when no records
The parameters were being ignored by the outbound rule since along the way the parameter names were all getting converted to lowercase.
This logic for this was fixed and the extract goes to the correct location now.
Files changed:
eagle_ml-2-0_cm/extract2t.inc
ESP-2779: Subscribe Redeem extract - Shares Outstanding Incorrect for Intraday Nav Funds
...
Files changed:
tpe/dynamic/msgcenter/eagle_ml-2-0_cm/out/tagvalue/data_file_task.xml
eagle\estar\tpe\dynamic\msgcenter\eagle_ml-2-0_cm\w_config.inc
ESP-2837: Missing reference to include t_warehouse_istarlot_streaming.inc
The missed reference has been added to the necessary packages:
EAGLE_ML_2_0.pkg
eagle_ml-2-0_default_out_q.pkg
eagle_ml-2-0_default_out_streaming_t.pkg
ESP-2890: Warehouse Trade extract fails on env where ext table is not available
The Warehouse Trade extract with the EnableSignageLogic set to “Y” was failing in some regions due to the new table TRADE_EXT not being present on all regions.
A lookup for the table was added and the results are stored in the vIsTradeExtTableExist variable. Depending on the value in this variable the fields that are in the TRADE_EXT table will either be included or excluded.
Files changed:
\dynamic\msgcenter\eagle_ml-2-0_cm\out\tagvalue\extract_wrhs_trade_signage_logic.inc