Purge and Archive Parameters

Owned by Natalie Gesin (Deactivated)
Last updated: Oct 22, 2024 by Kinga Huszno
3 min read

Purge and Archive parameters should be set in the following format:

[storage_type];[clean_option];[path_to_process];[files_mask];[clean_type: PURGE or ARCHIVE];[archive_folder];[is_need_compress];[older_than_time_in_hours];[include_subfolders]|

The following table lists descriptions of these parameters.

Name

Required or Optional

Possible values

Description

Name

Required or Optional

Possible values

Description

[storage_type]

Required

FTP or FILE

Storage type. If set to FTP, files to process (purge or archive) are located on the FTP box, if set to FILE, files to process are located on the APP box.

[clean_option]

Required

FOLDER or MCSTREAMFOLDER or ALLMCSTREAMFOLDERS

Clean option.

  • FOLDER value - if set to FOLDER then files in the specified folder will be processed. The folder is selected by the [path_to_process] parameter.This FOLDER value is can be set both for FTP and FILE storage types;

  • MCSTREAMFOLDER value – if set to MCSTREAMFOLDER then files in the specified stream folder will be processed. The stream and its folder are selected by [path_to_process] parameter.The MCSTREAMFOLDER value can be set only for the FILE storage type;

  • ALLMCSTREAMFOLDERS value – if set to ALLMCSTREAMFOLDERS then files in the specified folder of all enabled streams will be processed. The folder, which must exist for all streams, is selected by [path_to_process] parameter. The ALLMCSTREAMFOLDERS value is available to set only for the FILE storage type.

[path_to_process]

Required

String value (example: CMW/inbound_tst)

The path to the folder containing files to be processed.

Note: If set to “FILE”, the folder path starts with  […]/estar/tpe/data/msgcenter/

[files_mask]

Required

String value (example: *.csv)

Mask of files to process.

Note: For FTP there is currently no support of wildcards. To process all files in a directory, use the mask "*" or "*. *"

[clean_type]

Required

PURGE or ARCHIVE

Clean type.

If PURGE is set then selected files will be removed. If ARCHIVE is set then selected files will be moved or archived into the folder set in the [archive_folder] parameter.

[archive_folder]

Required if [clean_type]= ARCHIVE

String value (example: data/msgcenter/archive/)

The path to a folder on the APP box, to which processed files will be moved OR in which the archive file containing processed files will be created.

Note: Files can’t be archived/moved to a FTP folder, only folders on the APP box can be chosen for these actions.

Note: Folder path starts with […]/estar/tpe/

[is_need_compress]

Required if [clean_type]= ARCHIVE

Y or N

Compression flag.

If set to Y, the files to process will be archived into an archive file. If set to N, the files to process will be just moved.

[older_than_time_in_hours]

Required

Int value starting from 0 (examples: 0,1,2,3...24...48 and so on)

Must be a multiple of 24 in WIN environments:

24, 48, 72 etc

This parameter selects files to be processed by checking their creation time.
Please check if the clock on FTP server is synchronized with the clock on your box and take this difference into account.
If the clock on FTP server is AHEAD the time on your box, this parameter could be negative.

[include_subfolders]

Required

Y or N

Toggles recursive processing.

When set to Y, files in all subfolders of the processing directory will be purged (folder structure remains intact).

If set to N - only files in the processing directory will be removed.

PURGE ALL FILES Feature

Available since June 2017 EagleML release

Makes the P&A rule purge ALL files and directories inside the folder specified and ignore all other CM parameters. There are two ways of using this option:

  • File mask is empty or not specified. In this case all other CM parameters will be ignored;

Please note, that the option with empty file mask is not implemented for IWS P&A Workflow – use the second one.

  • File mask is set to * or *.*, [older_than_time_in_hours] parameter is set to 0 and [include_subfolders] parameter is set to “Y”, like in the example below:
    FILE;FOLDER;cmw/reply;*.*;PURGE;;;0;Y|

In this case ALL files inside the folder specified in the parameter will be purged.

Examples of Purge and Archive Parameter Values

  • FTP;FOLDER;CMW/Inbound;*;ARCHIVE;data/msgcenter/archive/inbound;Y;72;N|
    Description: all files in the CMW/Inbound FTP folder should be archived to the data/msgcenter/archive/inbound folder on the APP box. Only files older than 3 days (72 hours) will be selected for archivation. The resulting archive file will be created in data/msgcenter/archive/inbound on the APP box and will contain files selected above. These files will be removed from the CMW/Inbound FTP folder.

  • FILE;FOLDER;cmw/reply;*.xml;PURGE;;;48;N|
    Description: all files with the “xml” extension on the cmw/reply APP box folder older than 2 days (48 hours) will be removed.

  • FILE;FOLDER;cmw/reply;*;PURGE;;;48;Y|
    Description: all files and subfolders inside the cmw/reply processing folder older than 2 days (48 hours) will be removed recursively

  • FILE;ALLMCSTREAMFOLDERS;processing;*;PURGE;;;0;N|
    Description: all files in the processing folders of all enabled streams on the APP box will be removed.

  • FILE,MCSTREAMFOLDER,cfs_flow_control/outgoing,*.csv,ARCHIVE,data/msgcenter/archive,N,24;N|
    Description: all CSV files older than 1 day (24 hours) from the “outgoing” APP box folder of the cfs_flow_control stream will be moved to the data/msgcenter/archive/ folder of the APP box.

  • FILE;FOLDER;cmw/reply;*;PURGE;;;-2;N|
    Description: in case when FTP time is greater than time on box on 2 hours: all files on the cmw/reply FTP folder will be removed