FAQ for EagleML / MC2 Installation

Owned by Pamela Totas (Deactivated)
Last updated: Jul 03, 2024
4 min read

System Requirements

  • What is the Disk space requirement?
    The installer checks for 5GB of free disk space on the estar/tpe/dynamic and data/msgcenter folders

  • How much Memory is needed?
    There is approximately a 7GB memory footprint per server on top of what Eagle App Servers are using. If some of the load is shifted from the MC to MC2, this may offset the need in the memory increase.

  • What is the CPU requirement?
    This depends on the intended usage. There are no specific requirements except what is required for the Eagle App Server

  • What is the File system requirement?
    MC2 will be using the data/msgcenter for shared data and the dynamic folder for the rules. The MC2 Kafka Service writes local topic data to the estar/tpe/servers/kafkaservice/kafka/kafka-logs.  Despite the name, these are not logs, as this is data. The MC2 Kafka Service writes the Apache Zookeeper data under the estar/tpe/servers/kafkaservice/zookeeper/ directory.

  • Are there any files updated (file list), version, components, and services installed and enabled, new events (any continuous streams or events that need to be enabled and running as part of deployment) including new purge rules?
    There are not currently any continuous streams or events to be enabled or specific purge rules that are required to be created.

Eagle Instance Impact

The packages installs:

  • A MC2 specific JRE in the estar/tpe/java/ folder

  • A MC2 specific Python deployment under the estar/tpe/pypy and estar/tpe/python folders

  • MC2 Extract Service (MC2 Core) in the estar/tpe/servers/eaglemc2s and configuration in the estar/tpe/cfg/eaglemc2s/extractservice. Logs are stored in the eagle/logs/sarweb/eaglemc2s/extractservice/

  •  MC2 Eagle Kafka Service in the estar/tpe/servers/kafkaservice folder and configuration in the estar/tpe/cfg/kafkaservice. Logs are stored in the eagle/logs/starweb/kafkaservice

  •  MC2 Redis Server distribution in the estar/tpe/servers/redis and configuration in the estar/tpe/cfg/eaglemc2s/redis. Logs are stored in the eagle/logs/stdstreams/redis*.(stderr|stdout)

  • MC2 Python Rule Service in the estar/tpe/dynamic/pyrules with the logs in eagle/logs/starweb/pyruleservice

  •  MC2 Eagle Plugin (to implement the SOAP and REST interface thru standard Eagle IIS interfaces) in the eagle/estar/tpe/servers/plugins/plugin_mc2.so and configuration in the estar/tpe/cfg/plugin_mc2.ini. The logs will be in the eagle/logs/starweb/plugins/mc2/

  •  The installer updates (or creates) the eagle/eaglemgr/config/processguard_custom.xml and the eagle/cfg/system.xml to register the new services.

Configuration Impact

How to scale?
The framework runs one loadbalancer and two MC2 workers on each server. To scale additional servers may be added.

What are the timeouts?
The timeouts are defined in the configuration files. 

What are the configuration parameters and where are they stored?
The configuration template is in the eagle/estar/tpe/cfg/eaglemc2s/extractservice/application.yml.template. This file has defaults for all parameters supported by MC2. To change a parameter value, create a new application.yml file in the same folder and redefine only the specific parameters.

Common Errors/Troubleshooting

What are the Log location and errors for alerting?
Logs are located at the /apps/eagle/logs/starweb/eaglemc2s. The alerts are based on the sitescope monitors for the worker and loadbalancer processes. If they go down, an alert if the monitors have been configured, will be generated.

How do you kill a process?
Generally, it is not recommended to kill a process because pypy runs many threads. When MC2 is started, as of the latest release, there are 3 java processes that start (extractserviceworker x2 and extractservicelb) as well as a number of pypy processes that is equivalent to the number of CPU cores a Unix VM has. For example, if there are 16 cores on the CPU, there will be 16 pypy processes that start. All 19 processes in this scenario would need to be killed manually for a hard stop. 
For a graceful stop, run from /apps/eagle/eaglemgr, ./stop starweb extractserviceworker extractservicelb

How to restart a process?
From /apps/eagle/eaglemgr, run ./restart starweb extractserviceworker extractservicelb. The client cannot restart these processes from the front end because there is no entry for them in SMC. This is expected to be addressed at some point in a future release. 

Where to check the running events?
This may be monitored the same way as MessageCenter. Via the Message Center Console, go to the EJM Tasks on the top bar and search for any in-progress events.  There is no way currently to distinguish an extractservice job from a message center job.

Service Health-Checks

What is the service status? What are the service process counts? What is the memory/CPU thresholds?

MC2 provides a simple, REST based health check API for accessing the health status of all types such as MC/MC2 streams states, connections status, and many others default builtin system health info. It may also be a quick way to validate the basic running status after each release deployment.

Please reference the MC2 Healthcheck Utility document for more details.

Third Party Software Requirements

List any third party requirements installed or being deployed as part of the installation      

The following third party software will be installed while deploying MC2:

  • java - starting with July 2019 release we install Open JRE 11 on supported environments and JDK 1.8 on older versions, usually 13.1.2 and 2015R2. JRE 11 requires GLIBC2.14 in order to run.

  • kafka

  • Redis

  • pypy

Service Pre-Requisites

Auto installer, What is the version of EagleML? What are the Dependencies on core Eagle?

Please check the prerequisites of the document

What is the recommended minimum version of core Eagle?
The minimum version is v13.1.2. There is support for Linux only.

Install Instructions

What are the install instructions?

There are different options for installation based on the different environments and client configurations:

The installation document is available on Confluence

Application Architecture

What is the Application architecture?
The application is Java and python. There are different monitoring requirements based upon the application architecture. 

DB connections - - number and if it is configurable
The number of DB connections is configurable in the connections.xml for the service type estarmc.

API Settings

Is the module accessible via API? If so how? Link to connectivity details/documentation? 
MC2 is accessible thru SOAP and REST API. The REST API exposes the swagger. 

The installation document has a description on how to activate/expose the APIs