Page Comparison

Pattern 1 allows loading of many individual small data messages (non-compressed, non-encoded, order doesn't matter).

Info

title	Note

W_USE_PARALLEL_LOAD global variable flag doesn't matter in this case.

Node <synchronousExecution> does matter. The default value of <synchronousExecution> is 'no'.

Loading thru FILE, FTP, JMS Data Delivery Channels

In this common scenario, there are large volumes of individual non-compressed, non-encoded small data messages which are needed to load. Each of these messages is valid by XSD EagleML data file (for instance: SMF, Reference, Accounting transactions, etc) with correlation ID. Order of loading these files doesn't matter. Message delivery channel is FILE (FileSystem), FTP (SFTP) or JMS (MQ).

Here is example of Reference Transaction Message with correlation ID:

Code Block

language	xml

<EagleML eaglemlVersion="2-0" xsi:schemaLocation="http://www.eagleinvsys.com/2011/EagleML-2-0 eagleml-main-2-0.xsd" xsi:type="ReferenceTransactionMessage"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0">
    <referenceTransaction>
    <header>
        <objectType>IssueFxRate</objectType>
    </header>
    <taskIdentifier>
        <correlationId>IssueFxRate_sync_sample_01</correlationId>
        <businessTaskId>Test send message</businessTaskId>
    </taskIdentifier>
    <synchronousExecution>yes</synchronousExecution>
    <issueFxRate>
        <objectType>IssueFxRate</objectType>
        <objectId>IssueFxRate</objectId>
        <sourceName>EAGLE PACE</sourceName>
        <updateSource>MCADMIN</updateSource>
        <effectiveDate>2015-01-15</effectiveDate>
        <fromCurrency>
            <assetCurrency>USD</assetCurrency>
        </fromCurrency>
        <toCurrency>
            <assetCurrency>EUR</assetCurrency>
        </toCurrency>
        <exchangeRate>636.978</exchangeRate>
        <offerrate>479.072</offerrate>
        <originalSource>ORIGINAL_SOURCE</originalSource>
        <for1MnthRate>809</for1MnthRate>
        <for2MnthRate>668.875</for2MnthRate>
        <for3MnthRate>594.985</for3MnthRate>
        <for4MnthRate>734.659</for4MnthRate>
        <for5MnthRate>669.5</for5MnthRate>
        <for6MnthRate>633.966</for6MnthRate>
        <for7MnthRate>576.249</for7MnthRate>
        <for8MnthRate>827.639</for8MnthRate>
        <for9MnthRate>963.893</for9MnthRate>
        <for10MnthRate>539.896</for10MnthRate>
        <for11MnthRate>566.699</for11MnthRate>
        <for12MnthRate>527.102</for12MnthRate>
        <for13MnthRate>734.047</for13MnthRate>
        <for14MnthRate>488.872</for14MnthRate>
        <for15MnthRate>639.225</for15MnthRate>
        <for16MnthRate>375.258</for16MnthRate>
        <for17MnthRate>281.898</for17MnthRate>
        <for18MnthRate>291.081</for18MnthRate>
        <for19MnthRate>142.987</for19MnthRate>
        <for20MnthRate>435.818</for20MnthRate>
        <for21MnthRate>627.04</for21MnthRate>
        <for22MnthRate>482.984</for22MnthRate>
        <for23MnthRate>182.35</for23MnthRate>
        <for24MnthRate>541.054</for24MnthRate>
        <bidRate>549.359</bidRate>
        <fxRatesFactors>FX_RATES_FACTORS</fxRatesFactors>
        <for7dayRate>894.144</for7dayRate>
        <UDF1>200.908</UDF1>
        <UDF2>247.516</UDF2>
        <onDepositRate>315.607</onDepositRate>
        <tnDepositRate>737.886</tnDepositRate>
    </issueFxRate>
    </referenceTransaction>
</EagleML>

Info

title	Important Note

Be sure that the xsi:type attribute matches the objectType of this message. Also, the correlation ID should be unique for each message and synchronousExecution node value should be "yes".

The messages should be sent to eagle_ml-2-0_default_cm_control_message stream. In this case to increase performance and payload EJM core routes the incoming data message directly to loading stream.

You will see the following in the EJM Monitoring Tool for each incoming data file. For example:
one of IssueFxRate data messages with correlation ID is sent to eagle_ml-2-0_default_in_xml_reference loading stream directly:

If we send 3 different EagleML types messages (for example: GenericSMF, IssueFxRate, AccountingTrade) at the same time to eagle_ml-2-0_default_cm_control_message stream, we can see in the EJM Monitoring Tool the following:

Each of these messages is sent to corresponding loading stream directly.

Expand

title	Examples of these SMF, IssueFxRate, AccountingTrade messages (click to expand)

Code Block

language	xml

Example of SMF:
<EagleML xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ReferenceTransactionMessage" eaglemlVersion="2-0" eaglemlType="ReferenceTransactionMessage" actualBuild="1" xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0">
    <header>
        <messageId messageIdScheme="http://www.eagleinvsys.com/coding-scheme/message-id-2-0">d123b0-7ec2-4eae-8990-42508d167414</messageId>
        <sentBy>http://www.client.com</sentBy>
        <sendTo>http://www.eagleinvsys.com/eagle_ml-2-0_default_cm_control_message</sendTo>
        <replyTo>http://www.client.com/Eagle Control Messages</replyTo>
        <creationTimestamp>2016-10-24T14:44:04.3675929-05:00</creationTimestamp>
    </header>
    <taskIdentifier>
        <correlationId>gaSMF_sync_p1_de2</correlationId>
        <businessTaskId>Test send message</businessTaskId>
    </taskIdentifier>
    <synchronousExecution>yes</synchronousExecution>
    <referenceTransaction>
        <header>
            <objectType>GenericSMF</objectType>
        </header>
        <genericSMF>
            <objectType>GenericSMF</objectType>
            <objectId>GenericSMF</objectId>
            <sourceName>EAGLE PACE</sourceName>
            <updateSource>MCADMIN</updateSource>
            <effectiveDate>2014-09-30</effectiveDate>
            <noLoadValidationFlag>N</noLoadValidationFlag>
            <accountingValidationFlag>N</accountingValidationFlag>
            <primaryAssetId>TSTSMFCUSIP_003</primaryAssetId>
            <primaryAssetType>CUSIP</primaryAssetType>
            <issueName>TSTSMFISSUENAME_003</issueName>
            <unicodeIssueName>TSTSMFUNIISSUENAME_003</unicodeIssueName>
            <issueDescription>TSTSMFISSUEDESC_003</issueDescription>
            <bloombergId>TSTSMFBLOOMBERGID_003</bloombergId>
            <internalId>TSTSMFINTERNAL_003</internalId>
            <xrefIdentifiers>
                <xId1>
                    <xId>TSTSMFXREFID_003</xId>
                    <xIdType>TSTSMFXID1</xIdType>
                </xId1>
            </xrefIdentifiers>
            <primaryExchange>US</primaryExchange>
            <primaryExchangeCode>US</primaryExchangeCode>
            <assetCurrency>USD</assetCurrency>
            <assetResolutionOption>MATCH_ALL</assetResolutionOption>
            <battingOrder>cusip,internalId,bloombergId,xId1</battingOrder>
            <investmentType>CASH</investmentType>
            <processingSecurityType>EQCSCS</processingSecurityType>
            <incomeCurrency>USD</incomeCurrency>
            <issueCountry>CND</issueCountry>
            <settlementCurrency>USD</settlementCurrency>
            <priceMultiplier>1.0</priceMultiplier>
            <quantityScale>1.0</quantityScale>
            <releaseStatus>1</releaseStatus>
            <maturityDate>2015-06-30</maturityDate>
        </genericSMF>
    </referenceTransaction> 
</EagleML>

Example of IssueFxRate:
<EagleML xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ReferenceTransactionMessage" eaglemlVersion="2-0" eaglemlType="ReferenceTransactionMessage" actualBuild="1" xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0">
    <header>
        <messageId messageIdScheme="http://www.eagleinvsys.com/coding-scheme/message-id-2-0">d123b0-7ec2-4eae-8990-42508d167414</messageId>
        <sentBy>http://www.client.com</sentBy>
        <sendTo>http://www.eagleinvsys.com/eagle_ml-2-0_default_cm_control_message</sendTo>
        <replyTo>http://www.client.com/Eagle Control Messages</replyTo>
        <creationTimestamp>2016-10-24T14:44:04.3675929-05:00</creationTimestamp>
    </header>
    <taskIdentifier>
        <correlationId>gaIssueFxRate_sync_p1_de7</correlationId>
        <businessTaskId>Test send message</businessTaskId>
    </taskIdentifier>
    <synchronousExecution>yes</synchronousExecution>
    <referenceTransaction>
        <header>
            <correlationId>gaIssue_test_001</correlationId>
            <objectType>IssueFxRate</objectType>
        </header>
        <issueFxRate>
            <objectDescription>REFERENCE</objectDescription>
            <objectType>IssueFxRate</objectType>
            <objectId>IssueFxRate</objectId>
            <updateTimestamp>2015-08-21T12:00:00-01:00</updateTimestamp>
            <sourceName>EAGLE PACE</sourceName>
            <updateSource>MCADMIN</updateSource>
            <effectiveDate>2015-10-05</effectiveDate>
            <fromCurrency>
                <assetCurrency>USD</assetCurrency>
                <issueName>US Dollar</issueName>
                <primaryAssetType>CASH</primaryAssetType>
                <primaryAssetId>CASHUSD</primaryAssetId>
                <issueDescription>US Dollar</issueDescription>
                <securityAlias/>
            </fromCurrency>
            <toCurrency>
                <assetCurrency>EUR</assetCurrency>
                <issueName>EURO</issueName>
                <primaryAssetType>CASH</primaryAssetType>
                <primaryAssetId>CASHEUR</primaryAssetId>
                <issueDescription>Euro</issueDescription>
                <securityAlias/>
            </toCurrency>
            <exchangeRate>636.978</exchangeRate>
            <offerrate>479.072</offerrate>
            <originalSource>Eagle Pace</originalSource>
            <for1MnthRate>809.1</for1MnthRate>
            <for2MnthRate>668.875</for2MnthRate>
            <for3MnthRate>594.985</for3MnthRate>
            <for4MnthRate>734.659</for4MnthRate>
            <for5MnthRate>669.5</for5MnthRate>
            <for6MnthRate>633.966</for6MnthRate>
            <for7MnthRate>576.249</for7MnthRate>
            <for8MnthRate>827.639</for8MnthRate>
            <for9MnthRate>963.893</for9MnthRate>
            <for10MnthRate>539.896</for10MnthRate>
            <for11MnthRate>566.699</for11MnthRate>
            <for12MnthRate>527.102</for12MnthRate>
            <for13MnthRate>734.047</for13MnthRate>
            <for14MnthRate>488.872</for14MnthRate>
            <for15MnthRate>639.225</for15MnthRate>
            <for16MnthRate>375.258</for16MnthRate>
            <for17MnthRate>281.898</for17MnthRate>
            <for18MnthRate>291.081</for18MnthRate>
            <for19MnthRate>142.987</for19MnthRate>
            <for20MnthRate>435.818</for20MnthRate>
            <for21MnthRate>627.04</for21MnthRate>
            <for22MnthRate>482.984</for22MnthRate>
            <for23MnthRate>182.35</for23MnthRate>
            <for24MnthRate>541.054</for24MnthRate>
            <bidRate>549.359</bidRate>
            <fxRatesFactors>FX_RATES_F</fxRatesFactors>
            <for7dayRate>894.144</for7dayRate>
            <UDF1>200.908</UDF1>
            <UDF2>247.516</UDF2>
            <onDepositRate>315.607</onDepositRate>
            <tnDepositRate>737.886</tnDepositRate>
        </issueFxRate>
    </referenceTransaction>
</EagleML>

Example of AccountingTrade:
<EagleML xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" eaglemlVersion="2-0" eaglemlType="AccountingTransactionMessage" xsi:schemaLocation="http://www.eagleinvsys.com/2011/EagleML-2-0 eagleml-main-2-0.xsd" xsi:type="AccountingTransactionMessage">
    <header>
        <messageId messageIdScheme="http://www.eagleinvsys.com/coding-scheme/message-id-2-0">d123b0-7ec2-4eae-8990-42508d167414</messageId>
        <sentBy>http://www.client.com</sentBy>
        <sendTo>http://www.eagleinvsys.com/eagle_ml-2-0_default_cm_control_message</sendTo>
        <replyTo>http://www.client.com/Eagle Control Messages</replyTo>
        <creationTimestamp>2016-10-24T14:44:04.3675929-05:00</creationTimestamp>
    </header>
    <taskIdentifier>
        <correlationId>gaAcctTrade_sync_p1_de7</correlationId>
        <businessTaskId>Test send message</businessTaskId>
    </taskIdentifier>
    <synchronousExecution>yes</synchronousExecution>
    <accountingTransaction>
        <header>
            <identifier>
                <transactionId>DEQOPENFRKIENG23</transactionId>
            </identifier>
            <objectType>AccountingTrade</objectType>
        </header>
        <accountingTrade>
            <buySellIndicator>BUY</buySellIndicator>
            <xrefAccountId>ITEMLXREFTST</xrefAccountId>
            <xrefAccountIdType>PORTM</xrefAccountIdType>
            <cusip>546475PS8</cusip>
            <isin>US546475PS74</isin>
            <bloombergId>FF000ED6110</bloombergId>
            <assetResolutionOption>ALL</assetResolutionOption>
            <tradeDate>2014-07-04</tradeDate>
            <settlementDate>2014-07-06</settlementDate>
            <sharesParQuantity>4700.00</sharesParQuantity>
            <price>93.12</price>
            <taxAmount>10</taxAmount>
            <impliedCommissionIndicator>Y</impliedCommissionIndicator>
            <commission>663.968</commission>
            <secFee>875.645</secFee>
            <stampDutyTax>9.99</stampDutyTax>
            <localToBaseFXRate>335.617</localToBaseFXRate>
            <otherFee>10.1</otherFee>
            <autoSettleIndicator>YES</autoSettleIndicator>
            <custodianBankCode>1</custodianBankCode>
            <custodianBankName>CUSTODIAN_BANK_NAME</custodianBankName>
            <custodyBankAccount>11</custodyBankAccount>
            <custodyBankSubaccount>CUSTODY_BANK_SUBACCOUNT</custodyBankSubaccount>
            <cashAccountName>CASH</cashAccountName>
            <cashAccountNumber>1</cashAccountNumber>
            <exchangeCode>236</exchangeCode>
            <brokerCode>1</brokerCode>
            <brokerName>1</brokerName>
            <clearingBrokerCode>BONY</clearingBrokerCode>
            <clearingBrokerName>BANK OF NY</clearingBrokerName>
            <settleLocation>SETTLE_LOCATION</settleLocation>
            <settleInstructions1>SETTLEINSTRUCTION</settleInstructions1>
            <settleInstructions2>SETTLE_INSTRUCTIONS2</settleInstructions2>
            <settleInstructions3>SETTLE_INSTRUCTIONS3</settleInstructions3>
            <settleInstructions4>SETTLE_INSTRUCTIONS4</settleInstructions4>
            <transactionNotes>Other</transactionNotes>
            <originalTradeTicketNumber>544828</originalTradeTicketNumber>
            <orderNumber>157.096</orderNumber>
            <userDefinedPriority>441061</userDefinedPriority>
            <fas115Code>Trading</fas115Code>
            <regulatoryIntent>HFT</regulatoryIntent>
            <holdingTerm>CE</holdingTerm>
            <calculateLocalAmountsFlag>Y</calculateLocalAmountsFlag>
            <calculateBaseAmountsFlag>Y</calculateBaseAmountsFlag>
        </accountingTrade>
    </accountingTransaction>
</EagleML>

Loading Through WS Delivery Channel

In this common scenario, there are large volumes of individual non-compressed, non-encoded small data messages which are needed to load. Each of these messages is valid by XSD EagleML data file (for instance: SMF, Reference, Accounting transactions, etc) with correlation ID. The order of loading these files doesn't matter. Message delivery channel is WS.

You need to wrap the EagleML message into the SOAP envelope.

Here is example of Reference Transaction Message with correlation ID wrapped into the SOAP envelope:

Code Block

language	xml

<soap:Envelope xmlns:addr="http://www.w3.org/2005/08/addressing" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
    <soap:Body>
        <runTaskRequest xmlns="http://www.eagleinvsys.com/2011/wsdl/EagleML-2-0">
            <EagleML xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ReferenceTransactionMessage" eaglemlVersion="2-0" eaglemlType="ReferenceTransactionMessage" actualBuild="1" xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0">
                <header>
                    <messageId messageIdScheme="http://www.eagleinvsys.com/coding-scheme/message-id-2-0">d123b0-7ec2-4eae-8990-42508d167414</messageId>
                    <sentBy>http://www.client.com</sentBy>
                    <sendTo>http://www.eagleinvsys.com/eagle_ml-2-0_default_cm_control_message</sendTo>
                    <replyTo>http://www.client.com/Eagle Control Messages</replyTo>
                    <creationTimestamp>2016-10-24T14:44:04.3675929-05:00</creationTimestamp>
                </header>
                <taskIdentifier>
                    <correlationId>gaIssueFxRate_sync_p1_de12</correlationId>
                    <businessTaskId>Test send message</businessTaskId>
                </taskIdentifier>
                <synchronousExecution>yes</synchronousExecution>
                <referenceTransaction>
                    <header>
                        <objectType>IssueFxRate</objectType>
                    </header>
                    <issueFxRate>
                        <objectDescription>REFERENCE</objectDescription>
                        <objectType>IssueFxRate</objectType>
                        <objectId>IssueFxRate</objectId>
                        <updateTimestamp>2015-08-21T12:00:00-01:00</updateTimestamp>
                        <sourceName>EAGLE PACE</sourceName>
                        <updateSource>MCADMIN</updateSource>
                        <effectiveDate>2015-10-05</effectiveDate>
                        <fromCurrency>
                            <assetCurrency>USD</assetCurrency>
                            <issueName>US Dollar</issueName>
                            <primaryAssetType>CASH</primaryAssetType>
                            <primaryAssetId>CASHUSD</primaryAssetId>
                            <issueDescription>US Dollar</issueDescription>
                            <securityAlias/>
                        </fromCurrency>
                        <toCurrency>
                            <assetCurrency>EUR</assetCurrency>
                            <issueName>EURO</issueName>
                            <primaryAssetType>CASH</primaryAssetType>
                            <primaryAssetId>CASHEUR</primaryAssetId>
                            <issueDescription>Euro</issueDescription>
                            <securityAlias/>
                        </toCurrency>
                        <exchangeRate>636.978</exchangeRate>
                        <offerrate>479.072</offerrate>
                        <originalSource>Eagle Pace</originalSource>
                        <for1MnthRate>809.1</for1MnthRate>
                        <for2MnthRate>668.875</for2MnthRate>
                        <for3MnthRate>594.985</for3MnthRate>
                        <for4MnthRate>734.659</for4MnthRate>
                        <for5MnthRate>669.5</for5MnthRate>
                        <for6MnthRate>633.966</for6MnthRate>
                        <for7MnthRate>576.249</for7MnthRate>
                        <for8MnthRate>827.639</for8MnthRate>
                        <for9MnthRate>963.893</for9MnthRate>
                        <for10MnthRate>539.896</for10MnthRate>
                        <for11MnthRate>566.699</for11MnthRate>
                        <for12MnthRate>527.102</for12MnthRate>
                        <for13MnthRate>734.047</for13MnthRate>
                        <for14MnthRate>488.872</for14MnthRate>
                        <for15MnthRate>639.225</for15MnthRate>
                        <for16MnthRate>375.258</for16MnthRate>
                        <for17MnthRate>281.898</for17MnthRate>
                        <for18MnthRate>291.081</for18MnthRate>
                        <for19MnthRate>142.987</for19MnthRate>
                        <for20MnthRate>435.818</for20MnthRate>
                        <for21MnthRate>627.04</for21MnthRate>
                        <for22MnthRate>482.984</for22MnthRate>
                        <for23MnthRate>182.35</for23MnthRate>
                        <for24MnthRate>541.054</for24MnthRate>
                        <bidRate>549.359</bidRate>
                        <fxRatesFactors>FX_RATES_F</fxRatesFactors>
                        <for7dayRate>894.144</for7dayRate>
                        <UDF1>200.908</UDF1>
                        <UDF2>247.516</UDF2>
                        <onDepositRate>315.607</onDepositRate>
                        <tnDepositRate>737.886</tnDepositRate>
                    </issueFxRate>
                </referenceTransaction>
            </EagleML>
        </runTaskRequest>
    </soap:Body>
</soap:Envelope>

Info

title	Important Note

The xsi:type attribute must match the objectType of this message. Also , the correlation ID should be unique for each message and synchronousExecution node value should be "yes". The delivery stream name is defined in the node <sendTo>.

The messages should be sent to eagle_ml-2-0_default_cm_control_message stream (<sendTo>http://www.eagleinvsys.com/eagle_ml-2-0_default_cm_control_message</sendTo>) . In this case to increase performance and payload EJM core routes the incoming data message directly to loading stream.

For example, sending IssueFxRate data message using SoapUI:

In the EJM Monitoring Tool:

In this case SoapUI shows the final TSR and message is sent to loading stream directly.

Versions Compared

Old Version 10

New Version 11

Key

Loading thru FILE, FTP, JMS Data Delivery Channels

Loading Through WS Delivery Channel