draft-edit: Upload CSV Data File into a Region

There are several ways to upload (ingest) your data file into the region. We are going to cover two of them in this article.

Option 1: Using Run button.

1. Click Run button in the ribbon.

2. Select data file from a local folder by pressing ellipsis button (...) in the Run dialog

       3. Click Run button in the dialog.

4. View load details in the EJM Processes tab.

After you click Run button, a new tab "EJM Processes" will automatically appear. It will contain the processing details of the solution.

You may need to click "refresh" button to update the information during the execution.

You will be able to access this information later by running Monitoring Tool application. It can be launched from Solution ribbon in IWS.

Option 2. Using file name in the Run Task Request (RTR) message.

Option 1, described above is very useful for testing the workflow in development stages.

After your workflow was developed, tested and deployed to PROD region, the way you deliver data to your application will change.

It is expected that data files will be uploaded to the region automatically, and then the load gets triggered by sending a control message (a.k.a Run Task Request message) to Message Center.

The default location for data files in this case is tpe/data/msgcenter/cmw/in ,

1. To upload your data file into this folder you can use helper stream eagle_ml-2-0_default_cm_data_load_cmw_in in Automation Center:

2. Now we need to tell Message Center to start processing this file.

You can use same send data feature, but now for the eagle_ml-2-0_default_cm_control_message stream

The name of the file containing RTR message does not matter.

The content of the RTR message should look like follows:

<EagleML xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="RunTaskRequestSync" xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0" eaglemlVersion="2-0" xsi:schemaLocation="http://www.eagleinvsys.com/2011/EagleML-2-0 eagleml-main-2-0.xsd" eaglemlType="RunTaskRequest">
  <header>
    <messageId>5EDA26591AB784</messageId>
    <sentBy>http://www.client.com/eagleadmin</sentBy>
    <sendTo>http://www.eagleinvsys.com/eagle_ml-2-0_default_cm_control_message</sendTo>
    <replyTo>http://www.client.com/Eagle Control Messages</replyTo>
    <creationTimestamp>2020-07-06T11:43:08-04:00</creationTimestamp>
  </header>
  <taskIdentifier>
    <correlationId correlationIdScheme="correlationIdScheme">5EDA26591AB784a</correlationId>
    <businessTaskId correlationIdScheme="businessTaskIdScheme">5EDA26591AB784a</businessTaskId>
  </taskIdentifier>
  <taskTypeEnum>NEW</taskTypeEnum>
  <taskParameters>
    <taskParameter>
      <name>StreamName</name>
      <dataType>S</dataType>
      <value>eagle_ml-2-0_default_cm_execwkfl</value>
    </taskParameter>
    <taskParameter>
      <name>ActionType</name>
      <dataType>S</dataType>
      <value>EXECUTE</value>
    </taskParameter>
    <taskParameter>
      <name>WorkflowName</name>
      <dataType>S</dataType>
      <value>testvendor_feed_1_raw_data</value>
    </taskParameter>
    <taskParameter>
      <name>WorkflowData</name>
      <dataType>S</dataType>
      <value>example_buildingtable.csv</value>
    </taskParameter>
  </taskParameters>
</EagleML>

There are three values which you may need to adjust

• correlationId. This should be a unique value for the region. Messages with duplicate correlation Ids will be rejected by the system.
• workflowName. This value should reflect the workflow name. You can see it in the publish screen.
• WorkflowData. This is the name of data file which you uploaded to cmw/in folder.