Versions Compared

Old Version 2

changes.mady.by.user Marius Navickas

Saved on

compared with

New Version Current

changes.mady.by.user Pamela Totas

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Starting with December 2020 release of MC2 the EQL Rest requests are being executed without running an EJM Task. The requests are executed in MC2 Load Balancer instance, without involving the EJM engine which is running in MC2 Worker instances. This results in the EQL Rest request not being reflected in the EJM Tasks window in Message Center Console.


The default configuration provided with MC2 will add the EQL Rest request execution information to the database so that it shows in Message Center Console in:

  • Tasks Status

  • File Statistics

  • Message Status if the request failed.

This can be controlled by changing the MC2 configuration file. The estar/tpe/dynamic/mc2/cfg/extractservice.yml file can be created to change the configuration for all instances of MC2 in a region. The configuration parameters should be in the extractservice-lb section. The default configuration is:

On this page

Table of Contents
minLevel1
maxLevel3
outlinefalse
styledisc
typelist
printabletrue
Code Block
---

...


spring:

...


  profiles: extractservice-lb

...


  application:

...


    name: extractservice-lb

...



eagle.eaglemlrestapi.services.EQL.directexecution: true

...


eagle.pyservice.executors: 4

...


eagle.eaglemlrestapi.services.EQL.messagedetails.level: errorsonly

...


eagle.eaglemlrestapi.services.EQL.tasks: true

...


eagle.eaglemlrestapi.services.EQL.filestatistics: true
Info

...

The --- is important and has to be added as shown to the configuration file. Only one extractservice-lb section should be defined in the configuration file.

Increase the number of EDS executors

To increase the concurrency of request execution the number of python service executors can be increased by using the eagle.pyservice.executors parameter. By default MC2 starts four executors per instance. If the number of concurrent requests is big, this parameter can be adjusted. This will increase the memory and CPU usage. Having 32 Python Service Executors will lead to about 7GB of additional memory usage.

Change the Message Status Updates

The eagle.eaglemlrestapi.services.EQL.messagedetails.level controls what is shown in Message Status tab in Message Center Console. The information added to Message Center Console is registered for stream eagle_default_ruleservice. The eagle.eaglemlrestapi.services.EQL.messagedetails.level parameter can have the following values:

  • errorsonly - only errors will be shown in Message Center Console/Message Status

  • statusonly - each request will result in one record shown in Message Center Console/Message Status. The Incoming Message and Status Detail will be empty

  • allinformation - each request will result in one record shown in Message Center Console/Message Status. The Incoming Message and Status Detail will contain the information related to the request. The Outgoing Message will be empty.

Change the Tasks Status

If parameter eagle.eaglemlrestapi.services.EQL.tasks is set to false, the Tasks Status information will not show the execution of the EQL Rest Requests.

...

This can improve the performance of the EQL REST requests execution if the database update is slow.

Change the File Statistics status update

If parameter eagle.eaglemlrestapi.services.EQL.filestatistics is set to false, the File Statistics information will not show the execution of the EQL Rest Requests.

...

This can potentially improve the performance of the EQL REST requests execution if the database update is slow.

Execute EQL Requests Using The EJM Engine

Setting parameter eagle.eaglemlrestapi.services.EQL.directexecution to false will result in the EQL requests being executed on MC2 Workers using EJM. As a result the EQL Rest requests will be reflected in EJM Monitor and in Message Center Console EJM Tasks window.

...

Currently, this can add overhead and decrease the EQL Rest requests performance under a big load/concurrency by a factor of 5.