About Data Loading Patterns

This article helps outline what is the best Data Loading Pattern to choose based on your particular use case. 

The common data loading patterns include

1. Loading many individual small data messages (non-compressed, non-encoded, order doesn't matter):

   • Loading thru FILE, FTP, JMS data delivery channels

   • Loading thru WS delivery channel

2. Loading accumulated messages (non-compressed, non-encoded):

   • Loading accumulated messages in the parallel mode (simultaneous loading)

   • Loading accumulated messages in the sequential mode (loading based on order)

3. Loading messages in the encoded archive (compressed, encoded) 

Factors to Consider - Data Delivery Channels

The Data Delivery Channel is important when choosing the Data Loading Pattern.

The common channels are:

• FILE (FileSystem) channel - sending files directly to "incoming" folder of the stream using FileSystem. This is the same if you use the UI (Message Center Console, Message Center Editor, EJM Monitoring Tool) to drop files to streams.

• FTP/SFTP channel - sending files to FTP/SFTP folders. In this case to take files the related stream must be set to that FTP/SFTP folder.

  • You can read more about FTP/SFTP stream setup at  Configure Delivery Method Options. 

  • Also, it is possible to delivery files from FTP/SFTP using Sitesee-EJM functionality (more info at Sitesee-EJM Configuration Steps).

• JMS/MQ channel - sending files to queue, stream is connected to that queue and reads the data

  • You can read more about setup JMS/MQ streams in the following topics: at Creating Outbound JMS Streams and Creating Inbound JMS Streams.

• WS (Web Service call) - sending files using Web Protocols (like HTTP). Web services may use SOAP over HTTP protocol. 

  • You can read more about EagleML Web Service and setup at  About the EagleML Web Service and Configure the EagleML Web Service.