ctdam.proc.procedure module

class ctdam.proc.procedure.Procedure(configuration, seabird_exe_directory=None, available_hex_converters=['datcnv', 'hex2py'], auto_run=True, procedure_fingerprint_directory=None, file_type_dir=None, plot=False, verbose=False, timeout=60)

Bases: object

Runs a couple of processing steps in sequence on one or more CTD data source files.

It can use seabird internal processing modules, as well as custom ones. These can be in the form of independent windows exes or just pure python code. The input data can be .hex, .cnv or python data representations, predominantly CTDData. The input and all module and extra information is stored in a dict that usually will be generated by the settings Configuration module that reads a toml config.

Parameters:

• configuration (dict | Configuration) – The information necessary to run a processing procedure.

• seabird_exe_directory (Path | str | None) – The path to the directory where the Sea-Bird exes reside in.

• available_hex_converters (list[str]) – A list of the known hex converters.

• auto_run (bool) – Whether to autopilot the whole procedure.

• procedure_fingerprint_directory (Path | str | None) – A path to a directory where the fingerprint are meant to be stored in. If none given, this option is considered to be turned off.

• file_type_dir (Path | str | None) – A path to a directory where the individual Sea-Bird file types are differentiated into respective directories. If none given, this option is considered to be turned off.

• plot (bool) – Whether to create .html plots of the output data

• verbose (bool) – Sets whether the Sea-Bird modules are run silently or not.

• timeout (int) – The time in seconds after which individual processing steps will be killed automatically.

run(file='')

Runs given file or uses the one inside of the config.

A ‘run’ consists of the application of all the given modules to a given file. It is the structure that can be represented by a fingerprint file.

Parameters:

files (CnvFile | Path | str) –

The input file.

(Default value = ‘’)

Return type:

CTDData

load_config()

Thorough input/format check of the processing configuration, that either stems from a .toml config file, or is a self-build dictionary. Checks for the presence of certain keys, and then, depending on their importance, either fails or sets default values.

check_config_entry(key, default_value)

Handles configuration file entries.

is_seabird_module(module)

Answers the simple boolean question, whether the module in question is a seabird module or not.

Does that by checking for the presence of a certain key ‘psa’. All modules that are meant to run as a standalone executable should follow this principle and set their config file to the psa key.

Parameters:

module (dict) – The specific module parameters.

Return type:

bool

create_seabird_module(module_info)

Create instance of ProcessingModule to run Sea-Bird module.

Parameters:

module_info (dict) – The module information

Return type:

ProcessingModule

create_seabird_step(module, input_path, output_name=None)

Create instance of ProcessingStep from ProcessingModule.

Parameters:

• module (ProcessingModule) – A module to create a step from

• input_path (Path) – Path to target file

• output_name (Path | str | None) – New name of output file

Return type:

ProcessingStep

convert(hex_path, hex_converter)

Runs ‘datcnv’ on target data.

Parameters:

• hex_path (Path) – The path to the target hex file.

• hex_converter (dict) – The module parameters for the conversion.

Return type:

CTDData

new_file_path(file=PosixPath('.'))

Creates the new output file path.

Takes the file type directory or the given output directory and joins them with the given output name.

Parameters:

file (Path) – The current path to the target file.

Return type:

Path

go()

Performs the processing on all target files.

This is the ‘main’ method of the procedure. All previous methods prepare data for this method to then finally transform the input files into the wanted format. The main purpose of this method is the coordination of the two different forms of processing modules: standalone executables with config files, mainly Sea-Bird processing modules, and python-internal classes that implement the Module interface. The caveats are mainly the switching from one form to the other. This for example results in a in or out parsing of a CnvFile object.

Return type:

CTDData

procedure_fingerprint()

Handles the creation of individual processing procedure fingerprints.

A fingerprint is a ‘receipt’ of one invocation of this class to one or more target files or data. It shall serve the purpose of an easy to understand proof of what exactly has been done with the data to retreive the given result. The especially neat thing is, that they are at the very same time plain configuration files, allowing the easy re-running of processing procedures. What distinguishes them from the usual configuration files, is one, the exact target file list, that can be ommitted upon invocation, and a timestamp that prefixes the source file name and adheres to ISO 8601.

Return type:

Configuration | None

handle_fill_type_dir_creation()

Move target files to corresponding file type directories.