dispel.processing.trace module

Module to inspect definitions of processing steps.

class dispel.processing.trace.DataSetTrace

Bases: _LevelTraceBase

The trace of a data set.

data_set_id

The id of the data set traced.

Type:

str

definition

The raw data set definition.

Type:

dispel.data.raw.RawDataSetDefinition | None

__init__(level, data_set_id, definition=None)

Parameters:

• level (LevelFilter) –

• data_set_id (str) –

• definition (RawDataSetDefinition | None) –

Return type:

None

data_set_id: str

definition: RawDataSetDefinition | None = None

trace_type: TraceType = 'data_set'

class dispel.processing.trace.EpochTrace

Bases: _LevelTraceBase

The trace of an epoch.

epoch

The definition of the epoch

Type:

dispel.data.epochs.EpochDefinition

__init__(level, epoch)

Parameters:

• level (LevelFilter) –

• epoch (EpochDefinition) –

Return type:

None

epoch: EpochDefinition

trace_type: TraceType = 'epoch'

class dispel.processing.trace.MeasureTrace

Bases: _LevelTraceBase

The trace of a measure.

measure

The definition of the measure

Type:

dispel.data.values.ValueDefinition

__init__(level, step, measure)

Parameters:

• level (LevelFilter) –

• step (MeasureDefinitionMixin) –

• measure (ValueDefinition) –

Return type:

None

measure: ValueDefinition

step: MeasureDefinitionMixin

trace_type: TraceType = 'measure'

class dispel.processing.trace.OriginTrace

Bases: Trace

The origin of inspection.

This trace is the node to which all other nodes have a path and is added at the start of inspection.

__init__()

Return type:

None

trace_type: TraceType = 'origin'

class dispel.processing.trace.StepGroupTrace

Bases: StepTrace

The trace of a processing group step.

__init__(step)

Parameters:

step (ProcessingStep) –

Return type:

None

trace_type: TraceType = 'step-group'

class dispel.processing.trace.StepTrace

Bases: Trace

The trace of a generic processing step.

step

The traced processing step.

Type:

dispel.processing.core.ProcessingStep

__init__(step)

Parameters:

step (ProcessingStep) –

Return type:

None

step: ProcessingStep

trace_type: TraceType = 'step'

class dispel.processing.trace.Trace

Bases: ABC

A trace of a processing element from the inspection.

trace_type

The type of the trace. This is one of TraceType and set in the derived trace classes.

Type:

dispel.processing.trace.TraceType

__init__()

Return type:

None

trace_type: TraceType

class dispel.processing.trace.TraceRelation

Bases: Enum

An enum to denote the relationship between nodes in edges.

GROUP = 'group'

INPUT = 'input'

OUTPUT = 'output'

SEQUENCE = 'sequence'

class dispel.processing.trace.TraceType

Bases: Enum

An enum to indicate type of trace from inspection.

DATA_SET = 'data_set'

EPOCH = 'epoch'

MEASURE = 'measure'

ORIGIN = 'origin'

STEP_GENERIC = 'step'

STEP_GROUP = 'step-group'

dispel.processing.trace.collect_measure_value_definitions(steps, **kwargs)

Collect all measure value definitions from a list of processing steps.

Parameters:

• steps (Iterable[ProcessingStep]) – The steps from which to collect the measure value definitions.

• kwargs – See inspect().

Yields:

Tuple[ProcessingStep, ValueDefinition] – The processing step in question along with the value definition.

Return type:

Iterator[Tuple[MeasureDefinitionMixin, ValueDefinition]]

dispel.processing.trace.get_ancestor_source_graph(graph, trace)

Get a subgraph of all sources leading to the extraction of a measure.

Parameters:

• graph (MultiDiGraph) – The graph containing all traces.

• trace (MeasureTrace) – A trace of a measure for which to return the source graph.

Returns:

A subgraph of graph comprised of nodes and edges only from data sets and measures leading to the extraction of the provided measure through trace.

Return type:

networkx.MultiDiGraph

dispel.processing.trace.get_ancestors(graph, trace, rel_filter=None)

Get all ancestors of a trace.

Parameters:

• graph (MultiDiGraph) – The graph containing the relationship between the traces derived from inspect().

• trace (Trace) – The trace for which to find all ancestors, i.e., all predecessors of all predecessors until the origin.

• rel_filter (Callable[[MultiDiGraph, Trace, Trace], bool] | None) – If provided, predecessors will only be considered if the return value is True. The function needs to accept the graph, the current trace, and the potential predecessor to be considered as ancestor.

Returns:

A set of traces of all predecessors to trace.

Return type:

Set[Trace]

dispel.processing.trace.get_edge_parameters(graph)

Get parameters defined in steps attributes of edges.

Parameters:

graph (MultiDiGraph) –

Return type:

Dict[ProcessingStep, Set[Parameter]]

dispel.processing.trace.get_traces(graph, trace_type)

Get all traces being an instance of a specific type.

Parameters:

• graph (MultiDiGraph) – The graph providing the nodes/traces

• trace_type (Type[TraceTypeT]) – The object type the trace has to be of.

Returns:

Returns an iterable of traces of type trace_type.

Return type:

Iterable[TraceTypeT]

dispel.processing.trace.inspect(steps, **kwargs)

Inspect the relationships defined in processing steps.

Parameters:

• steps (ProcessingStep | Iterable[ProcessingStep]) – The processing steps to be inspected.

• kwargs – Any additional processing arguments typically passed to the processing steps.

Returns:

A graph representing the processing elements found in steps.

Return type:

networkx.MultiDiGraph