EagleML API Client - Calling the EagleML API

The EagleMLApi library is using SLF4J/LOG4J for logging. The logging can be configured in a log4j.properties file.
The EagleMLApi library can execute the requests in Asynchronous or Synchronous mode.
The Asynchronous mode will receive the reply from EagleML API thru a callback. The EagleML API client sends a WS request to EJM and waits for the Task Status Reply and the extract data to be delivered asynchronously when ready via a callback.
The Synchronous mode will execute the request and will wait for the TSR/reply on the same connection.

Initialization

Before using the EagleML API it should be initialized as described here:

// This is the initialization and shutdown code

// Create the eaglemlApi object
String endpoint = "http://url.to.mcservice:port";
String login = "eagleuser";
String password = "eaglepassword";
com.eagleinvsys.eagleapi.client.EagleApi eaglemlApi = new EagleApi(endpoint, login, password, null /* by default use java.io.tmpdir temp folder, or specify the path to the temporary folder for big extracts */);

// Optional - Add this to enable the CSV metrics reporting
String csvmetricsdir="/apps/eagle/csvmetrics";
eaglemlApi.enableCsvMetricReporter(csvmetricsdir);

// Optional - disable caching
//eaglemlApi.setCachingEnabled(false);

// Optional - set the threshold on the extract size, after which the data is saved to a temporary file. Default is 1MB
//eaglemlApi.setTempFileThreshold(10000000);

// Optional - set the starting callback port number and the number of ports to try. 
// If not specified the operating system will use a random port number.
// You need to do this only if there is a firewall between the EaglemlApi object and Message Center, preventing the callback to complete
//eaglemlApi.setStartingCallbackPort(40000);
// Optional - set the number of ports to try. In this example eaglemlApi will attempt to use ports in the range of 40000 to 40020
// If it can not listen to any of these ports it will fail to start.
//eaglemlApi.setNumberOfPortsToTry(20);


// Optional - set the published host address for callbacks. If not set the local host address (one of the interfaces) will be used.
// This is needed only if there is a firewall/NAT between EaglemlApi and MessageCenter and Message Center needs a specific IP to execute the callback.
//eaglemlApi.setPublishedHostAddress("192.168.1.104");

// Start the EagleApi listener and initialize the objects required to parse the results.
// Pass true as a parameter to wait for initialization to complete.
// Pass false as a parameter to complete the initialization asynchronously
eaglemlApi.start(true);

The above should be executed only once. After that multiple requests can be executed concurrently on the eaglemlApi object.

To stop the eaglemlApi instance and clean up the resources run:

eaglemlApi.stop();

Executing the Request Asynchronously

The requests can be executed only after EagleML Client API was started using the EagleApi.start() method.
The asynchronous requests can be executed in two modes - with an asynchronous callback or by waiting on Future<IEagleApiResult> object.

Waiting on a Future<IEagleApiResult> Object

// THIS IS A SAMPLE OF THE REQUEST EXECUTION. REQUESTS CAN BE EXECUTED ONLY AFTER THE eaglemlApi.start() is called.

Map<String, String> taskParams = new HashMap<String, String>();

// Add the parameters required to build a Control Message
// Generate a unique correlationId
// Generate a business task id
Future<IEagleApiResult> futureResult = eaglemlApi.executeRequestAsync(correlationId, businessTaskId, taskParams, 600 /* timeout in seconds */);
try(IEagleApiResult result = futureResult.get()) {
    switch(result.getResultStatus()) {
        case EagleApiResultStatus.SUCCESS:
            List<IEagleApiResultDataFile> files = result.getDataFiles();
            for (IEagleApiResultDataFile file : files) {
                // See the java doc for the IEagleApiResultDataFile. The file will represent one extract.
                // Currently EaglemlApi will produce only one file.
                // You can get a stream of uncompressed data:
                try(InputStream stream = file.getStream()) {
                    // Parse the extract and execute any transformations - for example to JSON
                }
                // If needed you can get the compressed data and get access to the file with the file.isInMemory(), file.getData(), file.getFile()
                // The temporary files (if any) created while processing the extracts will be removed when you close the IEagleApiResult instance.
                // If you want to preserve you'll have to move/copy them here.
            }
            break;
        case EagleApiResultStatus.TIMEOUT:
            // TIMEOUT - report an error
            break;
        case EagleApiResultStatus.ERROR:
            // ERROR - use result.getErrorMessage to get the error message
            break;
        case EagleApiResultStatus.STOPPED:
            // The request execution was stopped, when eaglemlApi.stop() is called
            break;
        case EagleApiResultStatus.NO_DATA:
            // The extract resulted in no data retrieved. Report this back.
            break;
    }
}

Asynchronous Execution with a Callback

// THIS IS A SAMPLE OF THE REQUEST EXECUTION. REQUESTS CAN BE EXECUTED ONLY AFTER THE eaglemlApi.start() is called.

Map<String, String> taskParams = new HashMap<String, String>();

// Add the parameters required to build a Control Message
// Generate a unique correlationId
// Generate a business task id
eaglemlApi.executeRequestAsync(correlationId, businessTaskId, taskParams, 600 /* timeout in seconds */, (result, exception) -> {
    if(exception != null) {
     // There was an exception while executing the request. Process it.
    } else {
        switch(result.getResultStatus()) {
            case EagleApiResultStatus.SUCCESS:
                List<IEagleApiResultDataFile> files = result.getDataFiles();
                for (IEagleApiResultDataFile file : files) {
                    // See the java doc for the IEagleApiResultDataFile. The file will represent one extract.
                    // Currently EaglemlApi will produce only one file.
                    // You can get a stream of uncompressed data:
                    try(InputStream stream = file.getStream()) {
                        // Parse the extract and execute any transformations - for example to JSON
                    }
                    // If needed you can get the compressed data and get access to the file with the file.isInMemory(), file.getData(), file.getFile()
                    // The temporary files (if any) created while processing the extracts will be removed when you close the IEagleApiResult instance.
                    // If you want to preserve you'll have to move/copy them here.
                }
                break;
            case EagleApiResultStatus.TIMEOUT:
                // TIMEOUT - report an error
                break;
            case EagleApiResultStatus.ERROR:
                // ERROR - use result.getErrorMessage to get the error message
                break;
            case EagleApiResultStatus.STOPPED:
                // The request execution was stopped, when eaglemlApi.stop() is called
                break;
            case EagleApiResultStatus.NO_DATA:
                // The extract resulted in no data retrieved. Report this back.
                break;
        }
    }
});

Executing the Request Synchronously

// THIS IS A SAMPLE OF THE REQUEST EXECUTION. REQUESTS CAN BE EXECUTED ONLY AFTER THE eaglemlApi.start() is called.

Map<String, String> taskParams = new HashMap<String, String>();

// Add the parameters required to build a Control Message
// Generate a unique correlationId
// Generate a business task id
try(IEagleApiResult result = eaglemlApi.executeRequestSync(String correlationId, String businessTaskId, Map<String, String> taskParams, int timeout))
{
        switch(result.getResultStatus()) {
            case EagleApiResultStatus.SUCCESS:
                List<IEagleApiResultDataFile> files = result.getDataFiles();
                for (IEagleApiResultDataFile file : files) {
                    // See the java doc for the IEagleApiResultDataFile. The file will represent one extract.
                    // Currently EaglemlApi will produce only one file.
                    // You can get a stream of uncompressed data:
                    try(InputStream stream = file.getStream()) {
                        // Parse the extract and execute any transformations - for example to JSON
                    }
                    // If needed you can get the compressed data and get access to the file with the file.isInMemory(), file.getData(), file.getFile()
                    // The temporary files (if any) created while processing the extracts will be removed when you close the IEagleApiResult instance.
                    // If you want to preserve you'll have to move/copy them here.
                }
                break;
            case EagleApiResultStatus.TIMEOUT:
                // TIMEOUT - report an error
                break;
            case EagleApiResultStatus.ERROR:
                // ERROR - use result.getErrorMessage to get the error message
                break;
            case EagleApiResultStatus.STOPPED:
                // The request execution was stopped, when eaglemlApi.stop() is called
                break;
            case EagleApiResultStatus.NO_DATA:
                // The extract resulted in no data retrieved. Report this back.
                break;
        }
} catch(EagleApiException e) {
  // Process the exception
};

Configuration

The EagleML Client API supports a set of configuration options on the EagleAPI class, as well as system properties to control the location of the temporary files.
The EagleAPI configuration should complete before the EagleAPI.start() method is called.

Main Configuration Options on the EagleAPI Class

Metric Reporters

EagleML Client API uses the Dropwizard Metrics library to collect and report metrics. By default it reports the metrics to the SLF4J log.

You can use EagleAPI.addMetricReporter method to add any DropWizard compatible ScheduledReporter. You can enable a CSV metric reporter by calling EagleAPI.enableCsvMetricReporter(directory, pollingInterval) to enable the reporter of metrics in a separate folder. The default metric reporting interval can be changed by calling EagleAPI.setDefaultReportingInterval(pollingIntervalSeconds).

Caching

EagleML Client API will cache the results of a list of preconfigured requests. To disable caching you can call EagleApi.setCachingEnabled(false).
The caching is controlled by a JSON configuration file. The configuration file name and/or location can be redefined using the eagle.eaglemlapi.config property. The file should be on the classpath.
Currently only exact matches are supported. If the request has a parameter with a value defined in the configuration file the request will be cached.

The config file format is:

{
  "exactMatchRules": [
    {
      "parameterName": "FeedType",
      "matchRules": [
        {
          "parameterValue": "ENTITYEXTRACT",
          "expirationTimeSeconds": 720
        },
        {
          "parameterValue": "SMFEXTRACT"
        },
        {
          "parameterValue": "ISSUEEXTRACT"
        }
      ]
    }
  ]
}

In the example above the requests with FeedType set to ENTITYEXTRACT, SMFEXTRACT and ISSUEEXTRACT will be cached. The ENTITYEXTRACT requests will be cached for 720 seconds, the SMFEXTRACT and ISSUEEXTRACT requests will be cached for the default expiration time of 14400 seconds (4 hours).

Temporary File Threshold

The EagleML Client API caches the responses exceeding a predefined threshold to disk. By default the threshold is 1MB. To change the threshold you can call EagleApi.setTempFileThreshold(sizeInBytes).

Temporary File Location

By default the EagleML Client API will store the responses exceeding the threshold to the location defined in java.io.tmpdir system property. To redefine the location specify the folder as the stagingDirectory path in EagleApi constructor.

Callback Parameters

The EagleML Client API opens a listener port to listen for replies in the asynchronous execution mode. By default it uses a ephemeral (allocated by the operating system) port. To control what port the EagleML Client API should use you can use:

EagleApi.setStartingCallbackPort(int portNumber)

If the port is used by another program the EagleML Client API will attempt to increment the port number and try again. The number of attempts to listen on a port can be configured with EagleApi.setNumberOfPortsToTry(int numberOfPorts).