MC2 REST API Troubleshooting

MC2 REST Services

MC2 provides various REST interfaces to support available features including the generic load, extract, health check, and pace report services. It is a multi-tier architecture for security, scalability, and fault tolerances. Before understanding how to troubleshoot MC2's REST services, it is essential to review all the components involved from the configuration to run time.

The MC2 REST services includes the following components:

Eagle Web Server
MC2 extractservice lb
MC2 extractservice worker
MC2 pyruleservice
Pace-reportservice

On this page

High Level Diagram

The diagram below depicts the high-level flows of current MC2 REST Services.

MC2 REST Services Deployment and Enablement

MC2 is enabled by the following steps:

Install the Eagle/MC2 "all-services" packages from the EagleML Installer. Reference EagleML / MC2 Release Deployment for detailed steps of the deployment.
After deploying, configure the Eagle Web Services front-end to enable REST and SOAP services. Reference the Installation additional configurations for MC2 for configuration details.
After front-end configuration, restart the Eagle WebServer and/or IIS to ensure the configuration changes take effect.

MC2 REST Endpoints Routing

MC2 currently supports the following REST Services:

MC2 REST Endpoints for Simple Extracts
MC2 REST Endpoints for the Generic Extractor (POST only)
EDS/EQL Endpoints (POST only)
Health Check Endpoints (backend access only)
PACE report service (additional services outside MC2) to run OLAP reports (both POST and GET requests).

MC2 Rest Service Routes

For the MC2 Generic Load/Extract, it travels through the following route:

REST Client → Web Server (Security Validation) → MC2 Load Balancer → MC2 Worker.

For an MC2 EDS Request, it travels through the following route:

REST Client → Web Server (Security Validation) → MC2 Load Balancer → Pyruleservice.

For PACE Report Service REST Request, it takes the route below:

REST Client → Web Server (Security Validation) → MC2 Load Balancer → PACE Report Service.

If an error occurs, it depends on the type of request and error code to resolve. Review and identify the route and then check each endpoint of the route to diagnose the trouble endpoint(s).

General Troubleshooting

When invoking MC2 REST services, it returns a status code which follows the standard of HTTP response codes:

200+ OK
400+ bad client request
500+ internal server error

Possible Client-Side Requests Issues

When REST Invocation responds with an error code int range of 400+, it usually indicates that the error is on the client end with the exception of when server endpoint is down.

Frequently bad request error codes are:

400 - Bad request
401 - Unauthorized
403 - Forbidden
404 - Not Found

When encountering client-side errors:

If the response code = 400 or 404, check if the requested URL is correct or the server URL is accessible.
If the response code = 401, check if the correct authentication parameters, user name or password, is specified correctly as part of request.
If the response code = 403, the specified URL is not allowed.

Possible Server Endpoint Services Issues

When the REST Invocation fails with error code >= 500, it indicates a server side error.

Review the following items:

Are all corresponding backend endpoints up and running?
Are are all System configurations accurate?
Are all deployed Endpoints versions compatible?

There are multiple ways to check if all backend services are up and running. It is easier to log in to the backend servers with backend access to perform the following checks:

Check MC2 health using the Healthcheck Url

From the browser, check the health check url http://{appserver}:{healthcheckport}/health. On the returning JSON, check if all services are "UP".

From the backend, check if the MC2 services (extractservicelb, extractserviceworker, kafkaservice and pyruleservices) are up and running:

Please reference the MC2 Troubleshooting to check if all the required services endpoint are up and running.

Check version incompatibilities

It is strongly recommended that all MC2 components are from the same release. Please reference the Installed Version Check and verify all installed versions.

For an issue related to the pace-reportservice endpoint, check the following:

Is the pace-reportservice deployed?

"pace-reportservice" is deployed on $EAGLE_ROOT_TO_APP/estar/tpe/servers/pace-reportservice. Ensure there is a deployed version in the path.

Is the pace-reportservice up and running?

Login to backend, run the command "ps -ef | grep pace-reportserivce" and review if the process is up and running.

Frequently Encountered Issues and Troubleshooting Steps

Receive Eagle Landing Page When Invoking REST Service

When submitting the REST request, the request produces the landing page contents instead of the expected JSON data response. This is usually due to a configuration issue and/or issues on the Eagle WebServer. Use the following steps to diagnose the issue:

Check Configuration

Check the ${EagleAppToRoot}/tpe/cfg/starsecuity.ini configurations specified in the Installation additional configurations for MC2 and validate all corresponding configuration from all WebServer nodes.

Repeat the above actions for all the WebServer nodes and ensure the changes have been made for all WebServer nodes.

Check the Status of WebServer(s)

From the System Management Console (sample image below), check all of the listed services "Type" equal to "WEBSERVER" have the "last recycle" time greater than the timestamp of the starsecurity.ini and the "Status Message" reflects the value of "UP".

URLs for a Quick Check of Rest Endpoints

The following URLs can be used for a quick diagnosis of the health of MC2 rest endpoints. Replace webserver with webserver hostname or IP address.

http://{webserver}/eagle/v2/api-doc

https://{webserver}/mc2/swagger-ui.html or http://{webserver}/mc2/swagger-ui.html

or more extract specific URLs such as:

https://{webserver}/eagle/v2/entities?maxrows=1