uval package

Subpackages

• uval.stages package
  • Submodules
  • uval.stages.combine_files module
  • uval.stages.dataset_specification module
  • uval.stages.hdf5 module
  • uval.stages.metrics module
  • uval.stages.stage module
  • uval.stages.stage_data module
  • Module contents
• uval.utils package
  • Submodules
  • uval.utils.hdf5_format module
  • uval.utils.hdf5_io module
  • uval.utils.hdf5_verification module
  • uval.utils.hdf5_virtual module
  • uval.utils.label_naming module
  • uval.utils.log module
  • uval.utils.yaml_io module
  • Module contents

Submodules

uval.context module

This module provides the Context class. The user of uval shall create a single instance of this class and call all the stages as needed.

For an example on how to use this, please that a look at the examples folder.

class uval.context.CachedContextManager(ctx: uval.context.Context)

Bases: contextlib.AbstractContextManager

This context manager will be returned by Context.cached() and can be used in a with statement. All

class uval.context.Context(max_workers=None)

Bases: object

cached()

set_cache_folder(folder_path: str)

uval.context.get_context(max_workers)

uval.main module

uval.main.main()

uval.main.single(cfg)

Module contents

class uval.Metrics(dataset: uval.stages.stage_data.SupportedDatasetSpecificationData, metrics_settings, output_settings)

Bases: object

This class implements all the evaluation metrics.

basic_metric(iou_threshold: float = None, confidence_threshold: float = None) → List[dict]

Get the TP, FP, Precision and recall.

Parameters

• iou_threshold (float, optional) – Threshold for IOU. Defaults to None.

• confidence_threshold (float, optional) – Threshold for confidence. Defaults to None.

Returns

A list of dictionaries. Each dictionary contains information and metrics of each class.

Return type

List[dict]

static calculate_average_precision(rec: List[float], prec: List[float]) → List[Any]

property confidence_threshold

data_preparations()

static eleven_point_interpolated_ap(rec: List[float], prec: List[float]) → List[Any]

evaluate()

evaluate_range(iou_range: List[float], confidence_threshold: Optional[float] = None)

generate_report(results_cluttered: dict) → None

get_average_precision(basic_metrics: List[dict], method: str = None) → List[dict]

Get the average precision. This will be used in multiple other metrics such as COCO or pascal voc.

Parameters

• basic_metrics (List[dict]) – [description]

• method (str, optional) – choice between precise (EveryPointInterpolation or None)

• estimation (or) –

Returns

adds ap to the each class of the output dictionaries.

Return type

List[dict]

get_average_recall(recalls: Dict[float, list], iou_range: List) → List

get_fscore(basic_metrics: List[dict]) → List[dict]

Get the f score metrics.

Parameters

• basic_metrics (List[dict]) – output of basic_metric method.

• method. (needs to be called before this) –

Returns

adds dict[‘F score’] and dict[‘F score soft’] to the inputs.

Return type

List[dict]

get_pascal_voc2007_metric(confidence_threshold=None) → list

get_pascal_voc2012_metric(confidence_threshold=None) → list

static iou(start_a: List[float], shape_a: List[float], start_b: List[float], shape_b: List[float]) → float

Calculates the intersection over union of the two cubes A and B.

Parameters

• start_a (List[float]) – bottom left corner of the cube A.

• shape_a (List[float]) – size of each dimension in the cube A.

• start_b (List[float]) – bottom left corner of the cube B.

• shape_b (List[float]) – size of each dimension in the cube B.

Returns

3D IOU of these cubes.

Return type

float

property iou_threshold

property output_path

plot_precision_recall_curve(pascal_voc_metrics: List[dict], show_ap: bool = True, show_interpolated_precision: bool = True, show_graphic: bool = False) → None

Plot the Precision x Recall curve for a given class.

Parameters

• pascal_voc_metrics (List[dict]) – Output of some pascal voc metric. needs to be

• method. (called before this) –

• show_ap (bool, optional) – if True, the average precision value will be shown

• False. (the interpolated precision. Defaults to) –

• show_interpolated_precision (bool, optional) – if True, it will show in the plot

• False. –

• show_graphic (bool, optional) – if True, the plot will be shown. Defaults to False.

Raises

IOError – [description]

plot_recall_iou_curve(recalls: Dict[float, List[float]], iou_thresholds: List[float], classes: List[str], show_ar: bool = True) → None

Plot the Recall x IOU curve for a given class.

Parameters

• (Dict[float (recalls) – list]): keys are iou thresholds. value is a list of recall for each class.

• iou_thresholds – list of iou thresholds.

• classes – list containing names of all classes.

plot_roc_curves(roc_metrics: List[dict], show_graphic: bool = False) → None

Plot the ROC curve for every class.

Parameters

• roc_metrics (List[dict]) – Output of some basic_metric. needs to be

• method. (called before this) –

• show_graphic (bool, optional) – if True, the plot will be shown. Defaults to False.

Raises

IOError – [description]

property templates_path

worker(iou_threshold)

uval.default_argument_parser(epilog=None)

Create a parser with some common arguments used by uval users. :param epilog: epilog passed to ArgumentParser describing the usage. :type epilog: str

Return type

argparse.ArgumentParser

uval.get_context(max_workers)

uval.load_datasplit(file_path: str, subsets=None, output=None) → uval.stages.stage_data.DatasetSpecificationData

This stage will load a data split file in YAML format and keep the id lists for train, val and test set.

Parameters

• file_path (str) – path to the yaml file.

• subsets ([str], optional) – which subsets of the yaml files are to be loaded,

• None (if set to) –

• None. (all subsets are used. Defaults to) –

• output – The output file to be generated, if needed

Returns

returns the dataset.

Return type

DatasetSpecificationData

uval.load_evaulation_files(folder_path: str, recursive=False, dataset=None, ignore_missing_files=False, max_workers=None) → Tuple[uval.stages.stage_data.Hdf5FilesData, uval.stages.stage_data.Hdf5FilesData, uval.stages.stage_data.Hdf5FilesData]

This is a wrapper that calls underlying functions and loads positive detections, GTs and negative detections.

Parameters

• folder_path (str) – path to the data folder.

• recursive (bool, optional) – set to True if subdirectories should also be searched. Defaults to False.

• dataset ([DatasetSpecificationData], optional) – The input dataset

• None. (loaded from YAML file. Defaults to) –

• ignore_missing_files – If a HDF5 file is missing, no error is raised

Returns

Tuple including all three parts of the dataset.

Return type

Hdf5FilesData

uval.setup_from_args(args)

Create configs and perform basic setups.

uval.support_dataset_with_file_paths(gt_files: uval.stages.stage_data.Hdf5FilesData, det_files: uval.stages.stage_data.Hdf5FilesData, soc_files: uval.stages.stage_data.Hdf5FilesData)

Given a dataset and sets of Hdf5 files found on disk, try to combine everything to create a supported dataset.