Pattern #2 - Loading Accumulated Messages (non-compressed, non-encoded)

Owned by Natalie Gesin (Deactivated)
Last updated: Dec 05, 2017
4 min read

Pattern 2 allows loading of accumulated messages (non-compressed, non-encoded).

Note

Node <synchronousExecution> doesn't matter in this case. 
W_USE_PARALLEL_LOAD does matter.  The default value of W_USE_PARALLEL_LOAD is 'Y'. 

Loading Accumulated Messages in Parallel Mode (Simultaneous Loading)

In this scenario,  non-compressed, non-encoded messages come in a defined location and then  need load them. Each of these messages is valid by XSD EagleML data file.  The order of loading these files doesn't matter. Usually this pattern is used to load EagleML message with the different types (for example, IssueFxRate file and WarehouseOpenLot file).

To load these messages , the ACQUIRE method will be used.

Messages should be located in the cmw/in/ server directory or in the CMW/inbound FTP directory. If you have another delivery method please put the messages into cmw/in/ server directory using default eagle_ml-2-0_default_cm_data_load_cmw_in stream OR custom streams special configured for that (see Delivery Methods).

Tip: Send any data file to default eagle_ml-2-0_default_cm_data_load_cmw_in stream to put the message into cmw/in/ server directory.

To load all of these messages use following RTR .  Also, the W_USE_PARALLEL_LOAD global variable flag must NOT be set to  'N' in the custom configuration file (see more about configuration files here - Configuration Files). 

The RTR should be sent to eagle_ml-2-0_default_cm_control_message stream:

<EagleML xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="RunTaskRequest"eaglemlVersion="2-0" xsi:schemaLocation="http://www.eagleinvsys.com/2011/EagleML-2-0 eagleml-main-2-0.xsd" eaglemlType="RunTaskRequest">
    <header>
        <messageId>TEST</messageId>
        <sentBy>http://www.eagleinvsys.com</sentBy>
        <sendTo>http://www.eagleinvsys.com/eagle_ml-2-0_default_cm_control_message</sendTo>
        <replyTo>http://www.client.com/Eagle Control Messages</replyTo>
        <creationTimestamp>2012-08-02T22:13:32.272-05:00</creationTimestamp>
    </header>
    <taskIdentifier>
        <correlationId correlationIdScheme="correlationIdScheme">accumulated_messages_sample_1</correlationId>
        <businessTaskId correlationIdScheme="businessTaskIdScheme">Test send message</businessTaskId>
    </taskIdentifier>
    <taskTypeEnum>NEW</taskTypeEnum>
    <taskParameters>
        <taskParameter>
            <name>ActionType</name>
            <dataType>S</dataType>
            <value>ACQUIRE</value>
        </taskParameter>
        <taskParameter>
            <name>StreamName</name>
            <dataType>S</dataType>
            <value>eagle_ml-2-0_default_cm_acquire_data</value>
        </taskParameter>
        <taskParameter>
            <name>WAREHOUSE</name>
            <dataType>S</dataType>
            <value>WarehouseOpenLot_sample.xml</value>
        </taskParameter>
        <taskParameter>
          <name>SMFGENERIC</name>
          <dataType>S</dataType>
          <value>SMF_sample.xml</value>
        </taskParameter>
        <taskParameter>
          <name>REFERENCE</name>
          <dataType>S</dataType>
          <value>IssueFxRate_sample.xml</value>
        </taskParameter>
        <taskParameter>
          <name>REFERENCE</name>
          <dataType>S</dataType>
          <value>IssueFxRate_sample_2.xml</value>
        </taskParameter>
    </taskParameters>
</EagleML>

Important Note

Be sure that xsi:type attribute is "RunTaskRequest". Also correlation ID should be unique for each RTR.
EagleML data messages, which are defied in the file task parameters (SMFGENERIC, REFERENCE and so on), are the simple EagleML data messages (without correlation id).

For example IssueFxRate_sample_2.xml:

<EagleML eaglemlVersion="2-0" xsi:schemaLocation="http://www.eagleinvsys.com/2011/EagleML-2-0 eagleml-main-2-0.xsd" xsi:type="ReferenceTransactionMessage" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0">
    <referenceTransaction>
    <header>
        <objectType>IssueFxRate</objectType>
    </header>
    <issueFxRate>
        <objectType>IssueFxRate</objectType>
        <objectId>IssueFxRate</objectId>
        <sourceName>EAGLE PACE</sourceName>
        <updateSource>MCADMIN</updateSource>
        <effectiveDate>2015-01-15</effectiveDate>
        <fromCurrency>
            <assetCurrency>USD</assetCurrency>
        </fromCurrency>
        <toCurrency>
            <assetCurrency>EUR</assetCurrency>
        </toCurrency>
        <exchangeRate>636.978</exchangeRate>
        <offerrate>479.072</offerrate>
        <originalSource>ORIGINAL_SOURCE</originalSource>
        <for1MnthRate>809</for1MnthRate>
        <for2MnthRate>668.875</for2MnthRate>
        <for3MnthRate>594.985</for3MnthRate>
        <for4MnthRate>734.659</for4MnthRate>
        <for5MnthRate>669.5</for5MnthRate>
        <for6MnthRate>633.966</for6MnthRate>
        <for7MnthRate>576.249</for7MnthRate>
        <for8MnthRate>827.639</for8MnthRate>
        <for9MnthRate>963.893</for9MnthRate>
        <for10MnthRate>539.896</for10MnthRate>
        <for11MnthRate>566.699</for11MnthRate>
        <for12MnthRate>527.102</for12MnthRate>
        <for13MnthRate>734.047</for13MnthRate>
        <for14MnthRate>488.872</for14MnthRate>
        <for15MnthRate>639.225</for15MnthRate>
        <for16MnthRate>375.258</for16MnthRate>
        <for17MnthRate>281.898</for17MnthRate>
        <for18MnthRate>291.081</for18MnthRate>
        <for19MnthRate>142.987</for19MnthRate>
        <for20MnthRate>435.818</for20MnthRate>
        <for21MnthRate>627.04</for21MnthRate>
        <for22MnthRate>482.984</for22MnthRate>
        <for23MnthRate>182.35</for23MnthRate>
        <for24MnthRate>541.054</for24MnthRate>
        <bidRate>549.359</bidRate>
        <fxRatesFactors>FX_RATES_FACTORS</fxRatesFactors>
        <for7dayRate>894.144</for7dayRate>
        <UDF1>200.908</UDF1>
        <UDF2>247.516</UDF2>
        <onDepositRate>315.607</onDepositRate>
        <tnDepositRate>737.886</tnDepositRate>
    </issueFxRate>
    </referenceTransaction>
</EagleML>

You can read more about ACQUIRE methods and all options in the related wiki-topic (Define ACQUIRE Options). 

You will see the following in the EJM Monitoring Tool after dropping the RTR (simultaneous loading): 

Note

Please pay attention to the Start Time column (for example, you can see that SMF Generic loading stream and Reference loading stream start in the same time). 

Loading Accumulated Messages in Sequential Mode (Loading Based on Order)

In this scenario,  non-compressed, non-encoded messages come in defined location and then need load them. Each of these messages is valid by XSD EagleML data file. Order of loading these files does matter. Usually this pattern is used to load EagleML message with the same type (for example, two IssueFxRate files).

To load these messages ACQUIRE method will be used.

Messages should be located in the cmw/in/ server directory or in the CMW/inbound FTP directory. If you have another delivery method please put the messages into cmw/in/ server directory using default eagle_ml-2-0_default_cm_data_load_cmw_in stream OR custom streams special configured for that (see Delivery Methods).

Send any data file to default eagle_ml-2-0_default_cm_data_load_cmw_in stream to put  the message into cmw/in/ server directory.

To load all of these messages use following RTR. The  W_USE_PARALLEL_LOAD global variable flag must be 'N' in the custom configuration file (see more about configuration files here - Configuration Files). 

The RTR should be sent to eagle_ml-2-0_default_cm_control_message stream:

<EagleML xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="RunTaskRequest" eaglemlVersion="2-0" xsi:schemaLocation="http://www.eagleinvsys.com/2011/EagleML-2-0 eagleml-main-2-0.xsd" eaglemlType="RunTaskRequest">
    <header>
        <messageId>TEST</messageId>
        <sentBy>http://www.eagleinvsys.com</sentBy>
        <sendTo>http://www.eagleinvsys.com/eagle_ml-2-0_default_cm_control_message</sendTo>
        <replyTo>http://www.client.com/Eagle Control Messages</replyTo>
        <creationTimestamp>2012-08-02T22:13:32.272-05:00</creationTimestamp>
    </header>
    <taskIdentifier>
        <correlationId correlationIdScheme="correlationIdScheme">accumulated_messages_sample_2</correlationId>
        <businessTaskId correlationIdScheme="businessTaskIdScheme">Test send message</businessTaskId>
    </taskIdentifier>
    <taskTypeEnum>NEW</taskTypeEnum>
    <taskParameters>
        <taskParameter>
            <name>ActionType</name>
            <dataType>S</dataType>
            <value>ACQUIRE</value>
        </taskParameter>
        <taskParameter>
            <name>StreamName</name>
            <dataType>S</dataType>
            <value>eagle_ml-2-0_default_cm_acquire_data</value>
        </taskParameter>
        <taskParameter>
            <name>WAREHOUSE</name>
            <dataType>S</dataType>
            <value>WarehouseOpenLot_sample.xml</value>
        </taskParameter>
        <taskParameter>
          <name>SMFGENERIC</name>
          <dataType>S</dataType>
          <value>SMF_sample.xml</value>
        </taskParameter>
        <taskParameter>
          <name>REFERENCE</name>
          <dataType>S</dataType>
          <value>IssueFxRate_sample.xml</value>
        </taskParameter>
        <taskParameter>
          <name>REFERENCE</name>
          <dataType>S</dataType>
          <value>IssueFxRate_sample_2.xml</value>
        </taskParameter>
    </taskParameters>
</EagleML>


Important Note

Be sure that xsi:type attribute is "RunTaskRequest". Also, the  correlation ID should be unique for each RTR.

EagleML data messages, which are defied in the file task parameters (SMFGENERIC, REFERENCE and so on), are the simple EagleML data messages (without correlation id).

For example IssueFxRate_sample_2.xml:

<EagleML eaglemlVersion="2-0" xsi:schemaLocation="http://www.eagleinvsys.com/2011/EagleML-2-0 eagleml-main-2-0.xsd" xsi:type="ReferenceTransactionMessage" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0">
    <referenceTransaction>
    <header>
        <objectType>IssueFxRate</objectType>
    </header>
    <issueFxRate>
        <objectType>IssueFxRate</objectType>
        <objectId>IssueFxRate</objectId>
        <sourceName>EAGLE PACE</sourceName>
        <updateSource>MCADMIN</updateSource>
        <effectiveDate>2015-01-15</effectiveDate>
        <fromCurrency>
            <assetCurrency>USD</assetCurrency>
        </fromCurrency>
        <toCurrency>
            <assetCurrency>EUR</assetCurrency>
        </toCurrency>
        <exchangeRate>636.978</exchangeRate>
        <offerrate>479.072</offerrate>
        <originalSource>ORIGINAL_SOURCE</originalSource>
        <for1MnthRate>809</for1MnthRate>
        <for2MnthRate>668.875</for2MnthRate>
        <for3MnthRate>594.985</for3MnthRate>
        <for4MnthRate>734.659</for4MnthRate>
        <for5MnthRate>669.5</for5MnthRate>
        <for6MnthRate>633.966</for6MnthRate>
        <for7MnthRate>576.249</for7MnthRate>
        <for8MnthRate>827.639</for8MnthRate>
        <for9MnthRate>963.893</for9MnthRate>
        <for10MnthRate>539.896</for10MnthRate>
        <for11MnthRate>566.699</for11MnthRate>
        <for12MnthRate>527.102</for12MnthRate>
        <for13MnthRate>734.047</for13MnthRate>
        <for14MnthRate>488.872</for14MnthRate>
        <for15MnthRate>639.225</for15MnthRate>
        <for16MnthRate>375.258</for16MnthRate>
        <for17MnthRate>281.898</for17MnthRate>
        <for18MnthRate>291.081</for18MnthRate>
        <for19MnthRate>142.987</for19MnthRate>
        <for20MnthRate>435.818</for20MnthRate>
        <for21MnthRate>627.04</for21MnthRate>
        <for22MnthRate>482.984</for22MnthRate>
        <for23MnthRate>182.35</for23MnthRate>
        <for24MnthRate>541.054</for24MnthRate>
        <bidRate>549.359</bidRate>
        <fxRatesFactors>FX_RATES_FACTORS</fxRatesFactors>
        <for7dayRate>894.144</for7dayRate>
        <UDF1>200.908</UDF1>
        <UDF2>247.516</UDF2>
        <onDepositRate>315.607</onDepositRate>
        <tnDepositRate>737.886</tnDepositRate>
    </issueFxRate>
    </referenceTransaction>
</EagleML>

You can read more about ACQUIRE methods and all options in the related wiki-topic (Define ACQUIRE Options). 

You will see the following in the EJM Monitoring Tool after dropping the RTR (loading depends on strong order):

Note

Please pay  attention to Start Time column (for example, you can see that Reference loading stream starts only after SMF Generic loading stream finishes).