MC2 Rest Services

MC2 provides various Rest interfaces to support features such as generic load, extract, health check, pace-reportservices etc. It is a multi-tier architecture for security, scalability, and fault tolerances. Before understand how to troubleshoot MC2's ReST services, it is essential to understand all components involved from configuration to run time. If you already familiar all these components, please skip to troubleshooting section.

To enable MC2 ReST services, it includes the following components: 

• Eagle Web Server
• MC2 extractservice lb
• MC2 extractservice worker
• MC2 pyruleservice
• Pace-reportservice

High Level Diagram

Following picture depicts the high-level flows of current MC2 Rest Services. 

MC2 ReST services deployment and enabling

MC2 can be enabled by the following steps:

• Install Eagle/MC2 "all-services" packages from EagleML Installer. Reference EagleML / MC2 Release Deployment for detail steps of the deployment.
• After deploy, users needs to further configure Eagle Web Services Frontend to enable ReST and SOAP services. Reference Installation additional configurations for MC2 for configuration details.
• After frontend configuration, make sure restart Eagle WebServer and/or IIS to ensure configuration taking effect.

MC2 Rest Endpoints Routing

MC2 currently supports the following Rest Services:

• MC2 Rest Endpoints for Generic Load/Extract
• EBS Endpoints (SOAP only)
• Health Check Endpoints (backend access only)
• Pace report service (additional services outside MC2) per OLAP configurations.

MC2 Rest Service Routes

For MC2 Generic Load/Extract, it travels through the following route:

Rest Client → Web Server (Security Validation) → MC2 Load Balancer → MC2 Worker.

For MC2 EBS Request, it travels through the following route:

Rest Client → Web Server (Security Validation) → MC2 Load Balancer → Pyruleservice.

For Pace Report Service Rest Request, it will take the route below:

Rest Client → Web Server (Security Validation) → MC2 Load Balancer → Pace Report Service.

When encounter errors, depends on the type of request and error code, identify the route and check each endpoint of the route to diagnose the trouble endpoint(s).

General Troubleshooting guide

When invoke MC2 rest services, it returns a status code which follows the standard of HTTP respond codes:

• 200+   OK
• 400+   bad client request
• 500+   internal server error

Possible client-side requests issues:

When Rest Invocation responds with error code int range of 400+, it usually indicates that the error is on the client end with exception of when server endpoint is down.

Frequently bad request error codes are:

• 400 - Bad request
• 401 - Unauthorized
• 403 - Forbidden
• 404 - Not Found

Checklist when encountering client-side errors are:

• If response code = 400 or 404, check if request URL is correct or server URL is accessible.
• If response code = 401, check if correct authentication parameters, user name or password, is specified correctly as part of request.
• If response code = 403, the specified URL is not allowed.

Possible server endpoints services issues:

When Rest Invocation failed with error code >= 500, it indicates as a server site errors.

• Is all corresponding backend endpoints are up and running?
• Is System configurations
• Is all deployed Endpoints versions compatibility
• others ...

There are multiple ways to check if all backend services are up and running. It is easier to log in to backend servers or with backend accesses to perform the following checks.

• Check MC2 health using healthcheck url

From the browser, check the health check url http://{appserver}:{healthcheckport}/health. On returning json, check if all services are "UP".

• From the backend, check if MC2 services (extractservicelb, extractserviceworker, kafkaservice and pyruleservices) are up and running:

Please reference MC2 Troubleshooting to check if all required services endpoint is up and running.

• Check version incompatibilities

It is strongly recommended that all MC2 components need to be from the same release. Please reference Installed Version Check and verify all installed versions.

For an issue related to the pace-reportservice endpoint, check the following:

• Is the pace-reportservice deployed?

"pace-reportservice" is deployed on $EAGLE_ROOT_TO_APP/estar/tpe/servers/pace-reportservice. Ensure there is a deployed version in the path.

• Is pace-reportservice up and running?

Login to backend, check "ps -ef | grep pace-reportserivce" and see if the process is up and running.

Frequently Encountered Issues and Troubleshooting Steps

Receive Eagle landing page when invoke Rest service

When submit Rest request, receiving the landing page contents instead of expected JSON data response, usually due to configuration and/or issues on Eagle WebServer. Following the following steps to diagnose the issue:

Checking configuration

Check ${EagleAppToRoot}/tpe/cfg/starsecuity.ini configurations specified in Installation additional configurations for MC2 and validate all corresponding configuration from all WebServer nodes.

Repeat above actions for all WebServer nodes and ensure changes has been made for all WebServer nodes.

Checking status of WebServer(s)

From System Management Console (sample image below), check all listed services type equals to  "WEBSERVER" have the "last recycle" time greater than the timestamp of starsecurity.ini and status are "UP".

URL for a quick check of the rest endpoints

Following URL can be a quick diagnose of the health of MC2 rest endpoints

http://{webserver}/eagle/v2/api-doc

https://{webserver}/mc2/swagger-ui.html or http://{webserver}/mc2/swagger-ui.html

or more extract specific URLs such as:

https://{webserver}/eagle/v2/accounting-basis?streamName=eagle_ml-2-0_default_out_extract_service&outputFormat=json"