View the IWS Solution for XML

Owned by Natalie Gesin (Deactivated)
Last updated: Jul 29, 2024 by Pamela Totas (Deactivated)
2 min read

The process for XML feed is composed of two steps.

Step 1 - Preprocessing Logic

First step is the preprocess logic for transforming XML data to CSV data for selected models in Raw Service:

For example, if you select 5 models in the the XML feed then Raw Service will generate (it generates those diagrams automatically) 5 sequence logics for transforming each models to xmldbapi format for load this data into Data Base and each model will call the own load process.

Step 2 - Default Load Workflow

The second step is the default load Raw Service workflow:

There are four steps in the workflow:

  1. Raw_data_header_load – load model (structure) in Data Base with unique Batch_id. Global columns loads in this step as well;

  2. Raw_data_load – load incoming data in XMLDBAPI format;

  3. Raw_create_view – step of create view tables for loaded data by loaded structure;

  4. Raw_create_dimension_table – create dimension table for current batch_id

In this diagram, you can see in the EJM Monitoring Tool after run your feed:

The following processes will start to run:

Process dd_xml2csv_t_(name or alias of model): this process transforms XML message to CSV before the mail load process. Process is started for every model in the XML data file.

Process load_(name or alias of model): the full load process for CSV file (for XML load the file is transformed from XML to CSV). Process is started for every model in the XML data file.

Contains the following tasks:

  • Task raw_data_header_load: this task loads header and defines types of fields. The Batch Id is defined in this task – the unique identifier of load process.

  • Task eagle_wrf_raw_data_load: this task loads data into DB:

  • Task dd_csv2xmldbapi_generic: this task transforms CSV message to XMLDBAPI format for the following load.

  • Task eagle_ml20_default_in_xml_warehouse loads the data records in XMLDBAPI format to a database.

  • Task raw_common_values_load loads GLOBAL VALUES into DB by Batch Id.

  • Task raw_create_view creates the view for current process by Batch Id.

  • Task raw_create_dimension_table created the dimension table with specified in Raw Service name.