Page Comparison

In this page:

...

Info
This functionality requires Python to be installed. Please refer to Pypy Deployment

Info
Currently available only for ORA environments

For a number of business processes a near-real - time processing pattern focused around capturing small changes in the system and packaging them into messages is used. Eagle handles this with different object types which often require a specific order of processing. This order is not a thing some of the clients would like to control, so Eagle has the Generic Load workflow which follows proper dependencies allowing to package various objects into a single message and send to Eagle for processing.

...

1. RTR processor step – System receives RTR message (in most of the cases reads them from queue) with base64 encoded archive data. This rule uses a python utility which is able to receive RTR via stdin and then parse incoming RTR, base64-decode and decompress data in memory and create a single EagleML message which contains all objects sorted in accordance with Generic Loader dependencies. This single EagleML message is then piped into stdout. MC rule is able to consume stdout to get the single EagleML message and to add special new object called CreateTSR with information for TSR creation. Then this message is sent to the next step using ProcessInStream rule element.See an example of RTR on this page.

Expand

title	Example of RTR (click to expand):

Code Block

language	xml

<EagleML
 xmlns="https://protect-us.mimecast.com/s/9QZ3C9rpDgHkAGpRIBevTv?domain=eagleinvsys.com"
 xmlns:xsi="https://protect-us.mimecast.com/s/_zz8C0R20LHGQ1LJsBl0uB?domain=w3.org"
 eaglemlVersion="2-0"
 eaglemlType="RunTaskRequest"
 xsi:schemaLocation="https://protect-us.mimecast.com/s/9QZ3C9rpDgHkAGpRIBevTv?domain=eagleinvsys.com eagleml-main-2-0.xsd"
 xsi:type="RunTaskRequest">
	<header>
		<messageId>SMF_331624_71454390-9726-41d2-95f6-b4c16e125cf0</messageId>
		<creationTimestamp>2018-01-24T18:10:44.483-06:00</creationTimestamp>
	</header>
	<taskIdentifier>
		<correlationId>LOAD_TEST_3</correlationId>
		<businessTaskId>Generic_Loader_Bundle</businessTaskId>
	</taskIdentifier>
	<taskTypeEnum>LOAD</taskTypeEnum>
	<taskParameters>
		<taskParameter>
			<name>StreamName</name>
			<dataType>S</dataType>
			<value>eagle_ml-2-0_default_cm_execwkfl</value>
		</taskParameter>
		<taskParameter>
			<name>ActionType</name>
			<dataType>S</dataType>
			<value>EXECUTE</value>
		</taskParameter>
        <taskParameter> 
			<name>EnableACKSwitch</name>
			<dataType>S</dataType>
			<value>N</value>
		</taskParameter>
        <taskParameter> 
			<name>DeliveryMethod</name>
			<dataType>S</dataType>
			<value>JMS</value>
		</taskParameter>
		<taskParameter> 
			<name>ReplyDeliveryMethod</name>
			<dataType>S</dataType>
			<value>JMS</value>
		</taskParameter>
		<taskParameter> 
			<name>ACKDeliveryMethod</name>
			<dataType>S</dataType>
			<value>JMS</value>
		</taskParameter>
		<taskParameter> 
			<name>JMSReplyDeliveryStream</name>
			<dataType>S</dataType>
			<value>tst_jms_stream_tsr</value>
		</taskParameter>
		<taskParameter>
			<name>JMSAckDeliveryStream</name>
			<dataType>S</dataType>
			<value>tst_jms_stream_tsr</value>
		</taskParameter>
		<taskParameter>
			<name>WorkflowName</name>
			<dataType>S</dataType>
			<value>eagle_wrf_generic_load</value>
		</taskParameter>
		<taskParameter>
			<name>CreateConsolidatedTSR</name>
			<dataType>S</dataType>
			<value>Y</value>
		</taskParameter>
		<taskParameter>
			<name>ConsolidatedTSROnly</name>
			<dataType>S</dataType>
			<value>Y</value>
		</taskParameter>
		<taskParameter>
			<name>WorkflowData</name>
			<dataType>S</dataType>
			<value>TST_EagleML.zip.base64</value>
		</taskParameter>
		<taskParameter>
			<name>Data</name>
			<dataType>S</dataType>
			<value>UEsDBBQAAAAIAFVgiUxelmrmhwQAAIoMAAAQAAAAX0dFTkVSSUNDQV8xLnhtbKVWW4/SQBR+1sT/
sNl3eqG0hU1tUtvuSgTEFo0+bWo7C6O94Exh2X/vmbYz04JGXRNCmO985zLnxjhhss3RcnF1KvKS
vr7e1fX+RlUfHx8VxCS4PNInqqRVoY41XVc7+mg80q5bnZsTxQO9R0OpyBbomq5+Xi7idIeKZIRL
Widliq6vGrtF/gkRiqvy9TWYEuDmaY9eX0foAREE5A1JSpqkNfCWiNJkC+rg7oY2NhdVmjDRP0XN
PY2KBJcMUE40a63Wf3Tuvnr5wiG/IDDBC2eHkgwR9hsO1ddvKK3Zhdw7VCKCU99zVIm2KmpPx9ly
3r+Y4LR51idxbEAJEE0J3jcBR+FtGIUrPwTuubBTOuyzpEYbXCAoXbF3IZXmSJuOxvpGH99oGnxG
mg7fjnpG7QzQ6kBStEoK5Ibe3SK8WnvMn4QHjuIGdpe+FyznK7ApUU5EDw8QKT6iACQs</value>
		</taskParameter>
	</taskParameters>
</EagleML>

Rule File used is eagle_ml-2-0_cm\in\xml\realtime_rtr.xml

Expand

title	Example of single EagleML message , which also contains with a CreateTSR object (click to expand):

Code Block

language	xml

<EagleML eaglemlVersion="2-0"  xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:SchemaLocation="http://www.eagleinvsys.com/2011/EagleML-2-0 eagleml-ext-2-0.xsd" xsi:type="ReferenceTransactionMessage">
  <referenceTransaction>
    <header>
      <objectType>IssuerOrganization</objectType>
    </header>
    <issuerOrganization>
      <objectType>IssuerOrganization</objectType>
      <objectId>IssuerOrganization</objectId>
      <objectDescription>REFERENCE</objectDescription>
      <updateTimestamp>2015-08-21T12:00:00-01:00</updateTimestamp>
      <updateDate>2015-08-21</updateDate>
      <sourceName>EAGLE PACE</sourceName>
      <updateSource>MCADMIN</updateSource>
      <effectiveDate>2015-10-05</effectiveDate>
      <issuerId>TST_TEST_ISSUER_01</issuerId>
      <issuerName>TST_TEST_ISSUER_01</issuerName>
      <address>1 WELLS</address>
      <city>NEWTON</city>
      <state>MA</state>
      <countryOfRiskCode>US</countryOfRiskCode>
      <countryOfIncorporationCode>US</countryOfIncorporationCode>
      <currency>USD</currency>
      <industryCode>TECHNOLOGY</industryCode>
      <countryExposure>US</countryExposure>
    </issuerOrganization>
  </referenceTransaction>
  <referenceTransaction>
    <header>
      <objectType>GenericSMF</objectType>
    </header>
    <genericSMF>
      <objectType>GenericSMF</objectType>
      <objectId>SMF</objectId>
      <objectDescription>SMF</objectDescription> 
      <sourceName>EAGLE PACE</sourceName>
      <updateSource>MCADMIN</updateSource>
      <effectiveDate>2016-06-14</effectiveDate>
      <primaryAssetId>TST_TEST_SMF</primaryAssetId>
      <primaryAssetType>INTERNAL</primaryAssetType>
      <issueName>TST_TEST_SMFISSUENAME</issueName>
      <issueDescription>TST_TEST_SMFSSUEDESC</issueDescription>
      <primaryExchangeCode>ALL</primaryExchangeCode>
      <assetCurrency>USD</assetCurrency>
      <investmentType>FI</investmentType>
      <processingSecurityType>DBIBTL</processingSecurityType>
      <securityType>CS</securityType>
      <securitySubType>COMMON STCK</securitySubType>
      <altSecurityMinorType>CASH</altSecurityMinorType>
      <settlementCurrency>EUR</settlementCurrency>
      <incomeCurrency>USD</incomeCurrency>
      <quantityType>PAR</quantityType>
      <priceMultiplier>103.103</priceMultiplier>
      <quantityScale>104.104</quantityScale>
      <issueCountryCode>US</issueCountryCode>
      <countryOfRiskCode>US</countryOfRiskCode>
      <issueTaxType>EQ</issueTaxType>
    </genericSMF>
  </referenceTransaction>
  <referenceTransaction>
    <header>
      <objectType>GenericCA</objectType>
    </header>
    <genericCA>
      <objectType>GenericCA</objectType>
      <objectId>GenericCA</objectId>
      <objectDescription>REFERENCE</objectDescription>
      <updateTimestamp>2015-08-21T12:00:00-01:00</updateTimestamp>
      <sourceName>EAGLE PACE</sourceName>
      <updateSource>MCADMIN</updateSource>
      <effectiveDate>2016-10-05</effectiveDate>
      <primaryAssetId>TST_TEST_SMF</primaryAssetId>
      <primaryAssetType>INTERNAL</primaryAssetType>
      <issueName>TST_TEST_SMFISSUENAME</issueName>
      <issueDescription>TST_TEST_SMFSSUEDESC</issueDescription>
      <primaryExchangeCode>ALL</primaryExchangeCode>
      <assetCurrency>USD</assetCurrency>
      <clientSpecificCAId>1</clientSpecificCAId>
      <transType>CASHDIV</transType>
      <sweepDate>2015-10-05</sweepDate>
      <exDate>2015-10-05</exDate>
      <cpStatus>PD</cpStatus>
      <cpReason>Cash Dividend</cpReason>
      <cpInstance>6259</cpInstance>
      <splitRatio>333.592</splitRatio>
      <prevSecId>TSTR820A</prevSecId>
      <divAmt>833.412</divAmt>
      <cpAnnounceDate>2015-10-05</cpAnnounceDate>
      <cpAcquiredBy>CP_ACRED_BY</cpAcquiredBy>
      <cpAddedAmt>768.954</cpAddedAmt>
      <cpAddress>CP_ADDRESS</cpAddress>
      <cpAgent>CP_AGENT</cpAgent>
      <toAsset>
        <primaryAssetId>TST_TEST_SMF</primaryAssetId>
        <primaryAssetType>INTERNAL</primaryAssetType>
        <issueName>TST_TEST_SMFISSUENAME</issueName>
        <issueDescription>TST_TEST_SMFSSUEDESC</issueDescription>
        <primaryExchangeCode>ALL</primaryExchangeCode>
        <assetCurrency>USD</assetCurrency>
      </toAsset>
      <corpActId>754107</corpActId>
      <fromFmvPrice>502.536</fromFmvPrice>
      <toFmvPrice>126.918</toFmvPrice>
      <cpChangeDate>2015-10-05</cpChangeDate>
      <cpNotes>CP_NOTES</cpNotes>
      <batchEventId>TSR80A</batchEventId>
      <costAmtMethod>M</costAmtMethod>
      <consentRt>364.514</consentRt>
      <consentDt>2014-07-22</consentDt>
      <proRation>345.541</proRation>
      <cpCrncy1>USD</cpCrncy1>
      <roundingPoint>170.374</roundingPoint>
      <limitGlIndicator>I</limitGlIndicator>
      <trmtCashGl>I</trmtCashGl>
      <preference>687795</preference>
      <sequence>316684</sequence>
      <matchingFlag>Y</matchingFlag>
      <cpReinvestRatio>723.068</cpReinvestRatio>
      <cxlEventType>CXL_EVYPE</cxlEventType>
      <princLossFactor>446.15</princLossFactor>
      <qdiPercent>0.4</qdiPercent>
      <qdiHldPeriodOvrd>I</qdiHldPeriodOvrd>
      <drdHldPeriodOvrd>I</drdHldPeriodOvrd>
      <qdiDrdOvrdComment>QDI_DRDENT</qdiDrdOvrdComment>
      <qdiPercentEffectiveDate>2015-10-05</qdiPercentEffectiveDate>
      <qdiEligibleFlag>Y</qdiEligibleFlag>
      <caDetailModel>
        <actionPreference>7735</actionPreference>
        <actionSequence>9163</actionSequence>
        <cahdAmount>453.977</cahdAmount>
        <cpTkr>CP_TKR</cpTkr>
      </caDetailModel>
    </genericCA>
  </referenceTransaction>
  <referenceTransaction>
    <header>
      <objectType>CreateTSR</objectType>
    </header>
    <createTSR>
      <objectType>CreateTSR</objectType>
      <JMSDeliveryStream>tst_jms_stream_tsr</JMSDeliveryStream>
    </createTSR>
  </referenceTransaction>
</EagleML>

2. Single EagleML message processing step. This is done via the same core stream eagle_ml-2-0_default_in_xml_reference, but with Splitting parameter set to No. This setting will assure that within Single EagleML message (generated on prior step) objects will go in proper order. However, due to the fact that ProcessInStream element was used in the prior step, different messages will still be processed concurrently. The rsf was modified to be able to execute GenericSMF objects as well. Finally, new CreateTSR object is also processed and a trigger is sent to eagle_ml-2-0_default_cm_task_reporter stream to create a TSR.

Implementation details

NOTE: For now developed functionality is supported only in ORACLE environments.

In rule for processing incoming RTR it is necessary introduce:

Check for correlation id uniqueness
Check for correlation id length limit
Parse task parameterss
Creating orch state
Send Ack (if switched on) , generate ACK, sent to configured delivery method (if NACK Update Proc Status)
Introduce task parameter for turn on/off ACK – EnableACKSwitch. In case EnableACKSwitch is Y or omitted then send ACK, in case EnableACKSwitch = N – do not send ACK.

Code Block

language	xml

<taskParameter> 
	<name>EnableACKSwitch</name>
	<dataType>S</dataType>
	<value>Y</value>
</taskParameter>

Ref Inbound Stream (No Split)

In rules for load (Ref Inbound Stream No Split) it is necessary introduce:

Add to mapping for Correlation Id alias (reference_common_header_aliases.inc)
Collect transaction id value for each record into orch state. The transaction_id field was used as it is an index field of msgcenter_dbo.msg_detail table.

New include should be created for get_w_state for current correlation id, and updating state with update list of transaction ids. The new include will be called from set_transaction_id_content.inc

In rules for eagle_ml-2-0_default_cm_task_reporterit is necessary introduce:

New include similar to task_reporter_generate_task_unit.inc include in order to be able to collect msg_details by list of transaction id.
Update logic of collecting TSR
Update report_dist_methods,inc include in order to send message instead of file for Message Queue Delivery methods.

IssuePcPrice

In memory rule has been updated and new usage case has been added, currently it works only for IssuePcPrice with one referenceTransaction in the file. If the incoming file has more than one IssuePcPrice record or another objectType, it will be resend to the default eagle_ml-2-0_default_cm_control_message stream.

This behavior can be overridden by option:

W_RESEND_TO_REQUEST_STREAM

If file has 1 IssuePcPrice transaction it will be sent to the stream: eagle_ml-2-0_default_in_xml_reference as it happens usually, but right after load TSR message will be generated and delivered in the PcPrice rule (xml-ref_issuepcprice.xml). As a result you will not see any runs of eagle_ml-2-0_default_cm_task_reporter.

Versions Compared

Old Version 3

New Version Current

Key

Implementation details

IssuePcPrice