Scope Hierarchy for Live Context

Module: openhcs.pyqt_gui.widgets.shared.parameter_form_manager Status: STABLE

—

Overview

Multiple orchestrators can exist simultaneously (different plates, different pipelines). The scope hierarchy system prevents cross-orchestrator parameter bleed-through by ensuring step and function editors only see parameters from their own orchestrator context.

Problem Context

Without scope isolation, parameter forms collect live context from all open windows:

# Orchestrator A: Plate 1, Step 1
step_editor_A.get_parameter('gaussian_sigma')  # Should see Step 1 values

# Orchestrator B: Plate 2, Step 2 (different plate!)
step_editor_B.get_parameter('gaussian_sigma')  # Should NOT see Step 1 values

# Without scope isolation:
# Both editors see each other's parameters → incorrect placeholder text

Scope isolation ensures each editor only sees parameters from its own orchestrator.

Solution: Hierarchical Scope IDs

Scope IDs create a hierarchy that matches the orchestrator → step relationship.

Current Implementation (uses :: separator and plate_path):

# Orchestrator/Plate scope (unique per plate)
plate_scope = str(orchestrator.plate_path)
# Example: "/data/plates/plate_001"

# Step scope (inherits plate scope)
step_token = getattr(step, '_pipeline_scope_token', step.name)
step_scope = f"{plate_scope}::{step_token}"
# Example: "/data/plates/plate_001::step_0"

Note: The :: (double colon) separator is used for hierarchical scoping, not . (period).

Editors with matching scope prefixes share live context. Editors with different scopes are isolated.

Scope Hierarchy Architecture

Two-Level Hierarchy

# Level 1: Plate/Orchestrator (uses actual plate_path)
scope_id = "/data/plates/plate_001"
# Shared by: All config editors for this plate

# Level 2: Step (inherits plate scope with :: separator)
scope_id = "/data/plates/plate_001::step_0"
# Shared by: Step editor and its function editor for this specific step

Real Examples from Code:

# From dual_editor_window.py:240-245
def _build_step_scope_id(self, fallback_name: str) -> str:
    plate_scope = getattr(self.orchestrator, 'plate_path', 'no_orchestrator')
    token = getattr(self.editing_step, '_pipeline_scope_token', None)
    if token:
        return f"{plate_scope}::{token}"
    return f"{plate_scope}::{fallback_name}"

# From plate_manager.py:419, 1141
scope_id = str(orchestrator.plate_path)  # Plate-level config editing

Key insight: Step and function editors share the same scope prefix (including step token), enabling them to see each other’s live parameters while remaining isolated from other orchestrators and other steps.

Scope Matching Logic

Actual Implementation (from parameter_form_manager.py:375-393):

@staticmethod
def _is_scope_visible_static(manager_scope: str, filter_scope) -> bool:
    """
    Check if scopes match (prefix matching for hierarchical scopes).
    Supports generic hierarchical scope strings like 'x::y::z'.
    """
    # Convert filter_scope to string if it's a Path
    filter_scope_str = str(filter_scope) if not isinstance(filter_scope, str) else filter_scope

    return (
        manager_scope == filter_scope_str or
        manager_scope.startswith(f"{filter_scope_str}::") or
        filter_scope_str.startswith(f"{manager_scope}::")
    )

Examples:

# Same plate: plate scope matches step scope (parent-child)
_is_scope_visible_static(
    "/data/plates/plate_001::step_0",     # manager_scope (step)
    "/data/plates/plate_001"               # filter_scope (plate)
)
# → True (step scope starts with "plate_scope::")

# Different plates: no match
_is_scope_visible_static(
    "/data/plates/plate_001::step_0",     # manager_scope
    "/data/plates/plate_002"               # filter_scope
)
# → False (different plate prefixes)

This ensures step and function editors see each other’s parameters (same plate/step) while remaining isolated from other orchestrators.

Implementation Patterns

Dual Editor Window

Step and function editors share scope to enable parameter synchronization.

Actual Implementation (from dual_editor_window.py:240-267):

class DualEditorWindow(BaseFormDialog):
    def _build_step_scope_id(self, fallback_name: str) -> str:
        plate_scope = getattr(self.orchestrator, 'plate_path', 'no_orchestrator')
        token = getattr(self.editing_step, '_pipeline_scope_token', None)
        if token:
            return f"{plate_scope}::{token}"
        return f"{plate_scope}::{fallback_name}"

    def create_step_tab(self):
        step_name = getattr(self.editing_step, 'name', 'unknown_step')
        scope_id = self._build_step_scope_id(step_name)
        # Result: "/data/plates/plate_001::step_0"

        # Step editor uses step scope
        self.step_editor = StepParameterEditorWidget(
            self.editing_step,
            scope_id=scope_id
        )

    def create_function_tab(self):
        step_name = getattr(self.editing_step, 'name', 'unknown_step')
        scope_id = self._build_step_scope_id(step_name)
        # Same scope as step editor!

        # Function editor uses same step scope
        self.func_editor = FunctionListEditorWidget(
            scope_id=scope_id  # Same as step editor
        )

Why same scope? Function editor needs to see step parameters (e.g., processing_config.group_by) for placeholder resolution.

Function List Editor

class FunctionListEditor(QWidget):
    def __init__(self, step, scope_id):
        self.step = step
        self.scope_id = scope_id  # Inherited from dual editor

    def refresh_from_step_context(self):
        """Refresh function editor when step parameters change."""
        # Collect live context from step editor (same scope)
        live_context = self._collect_live_context()

        # Resolve group_by placeholder using step's processing_config
        group_by_value = self._resolve_group_by_placeholder(live_context)

        # Update UI
        self.group_by_selector.setCurrentValue(group_by_value)

Config Window

Config windows use plate-scoped or global scope depending on the config being edited.

Actual Implementation (from config_window.py:59,77,117 and plate_manager.py:1141-1148):

class ConfigWindow(BaseFormDialog):
    def __init__(self, config_class, current_config, ...,
                 scope_id: Optional[str] = None):
        # scope_id passed from caller
        self.scope_id = scope_id

        self.form_manager = ParameterFormManager.from_dataclass_instance(
            dataclass_instance=current_config,
            scope_id=self.scope_id  # Plate-scoped or None for global
        )

# From plate_manager.py - creating plate-scoped config window
scope_id = str(orchestrator.plate_path) if orchestrator else None
config_window = ConfigWindow(
    config_class,
    current_config,
    scope_id=scope_id  # "/data/plates/plate_001" or None
)

Scope Semantics:

• scope_id=None: Global config (GlobalPipelineConfig) - visible to all orchestrators

• scope_id="/data/plates/plate_001": Plate-scoped config (PipelineConfig) - only visible to this plate’s editors

Scope Isolation Examples

Isolated Orchestrators

# Orchestrator A: Plate 1
orchestrator_A = PipelineOrchestrator(plate_path=Path("/data/plates/plate_001"))
scope_A = str(orchestrator_A.plate_path)  # "/data/plates/plate_001"

# Orchestrator B: Plate 2
orchestrator_B = PipelineOrchestrator(plate_path=Path("/data/plates/plate_002"))
scope_B = str(orchestrator_B.plate_path)  # "/data/plates/plate_002"

# Step editors are isolated (using actual implementation)
step_editor_A = StepParameterEditor(step_A, scope_id=f"{scope_A}::step_0")
step_editor_B = StepParameterEditor(step_B, scope_id=f"{scope_B}::step_0")
# step_editor_A: "/data/plates/plate_001::step_0"
# step_editor_B: "/data/plates/plate_002::step_0"

# step_editor_A does NOT see step_editor_B's parameters
# Different scope prefixes: "/data/plates/plate_001" vs "/data/plates/plate_002"

Shared Step/Function Context

# Same plate, same step
plate_scope = "/data/plates/plate_001"
step_scope = f"{plate_scope}::step_0"  # "/data/plates/plate_001::step_0"

# Step editor
step_editor = StepParameterEditor(step, scope_id=step_scope)

# Function editor (same scope!)
function_editor = FunctionListEditor(step, scope_id=step_scope)

# function_editor DOES see step_editor's parameters
# Same scope: "/data/plates/plate_001::step_0"

Cross-Window Synchronization

The scope system enables selective cross-window synchronization:

# User edits step parameter in step editor
step_editor.update_parameter('processing_config.group_by', GroupBy.WELL)

# Triggers cross-window refresh
ParameterFormManager.trigger_global_cross_window_refresh()

# Function editor receives refresh signal (same scope)
function_editor.refresh_from_step_context()

# Function editor sees updated group_by value
# Updates group_by selector to match step editor

Key insight: Scope matching ensures only related windows refresh, preventing unnecessary updates.

Recursive Live Context Collection

The collect_live_context() method recursively collects values from all managers AND their nested managers via _collect_from_manager_tree():

@classmethod
def _collect_from_manager_tree(cls, manager, result: dict, scoped_result: dict = None):
    """Recursively collect values from manager and all nested managers."""
    if manager.dataclass_type:
        result[manager.dataclass_type] = manager.get_user_modified_values()
        if scoped_result is not None and manager.scope_id:
            scoped_result.setdefault(manager.scope_id, {})[manager.dataclass_type] = result[manager.dataclass_type]

    # Recurse into nested managers
    for nested in manager.nested_managers.values():
        cls._collect_from_manager_tree(nested, result, scoped_result)

This enables sibling inheritance: when live_context contains both LazyStepWellFilterConfig and LazyWellFilterConfig values, _find_live_values_for_type() can use issubclass() matching to find StepWellFilterConfig values when resolving WellFilterConfig placeholders.

Example: Step form has two nested config managers:

• step_well_filter_config → LazyStepWellFilterConfig(well_filter=123)

• well_filter_config → LazyWellFilterConfig(well_filter=None)

Old behavior: Only root manager’s values collected (nested values missed).

New behavior: Both nested managers’ values collected, enabling:

1. well_filter_config.well_filter needs placeholder

2. _find_live_values_for_type(LazyWellFilterConfig, live_context) called

3. Finds LazyStepWellFilterConfig via issubclass(StepWellFilterConfig, WellFilterConfig)

4. Returns step_well_filter_config values with well_filter=123

5. Placeholder shows “Pipeline default: 123”

See Field Change Dispatcher Architecture for how changes trigger sibling refresh.

Implementation Notes

🔬 Source Code:

• Scope matching: openhcs/pyqt_gui/widgets/shared/parameter_form_manager.py

• Recursive collection: _collect_from_manager_tree() in same file

• Dual editor scope setup: openhcs/pyqt_gui/windows/dual_editor_window.py

• Function editor scope: openhcs/pyqt_gui/widgets/function_list_editor.py

🏗️ Architecture:

• Field Change Dispatcher Architecture - Unified field change handling

• Context Management System - Configuration context and inheritance

📊 Performance:

• Scope matching is O(n) where n = number of active form managers

• Typically < 10 managers active, so overhead is negligible

• Scope string comparison is fast (prefix matching)

• Recursive collection adds minimal overhead (tree depth typically < 5)

Key Design Decisions

Why use plate_path for orchestrator scope instead of object ID?

• Plate path is semantically meaningful (shows which plate the editor is for)

• Plate path is stable across sessions (object ID changes each run)

• Plate path enables future scope persistence/serialization

• In practice, only one orchestrator per plate is active at a time

Why use ``::`` separator instead of ``.``?

• Avoids conflicts with file paths (which use . for extensions)

• More visually distinct in logs and debugging

• Consistent with other path-like separators in the codebase

Why share scope between step and function editors?

Function editor needs step parameters for placeholder resolution (e.g., group_by selector). Sharing scope enables this without manual parameter passing.

Why use strings for scope IDs?

Scope IDs are strings, enabling serialization and comparison without object reference issues.

Common Gotchas

• Don’t use global scope (``None``) for plate-specific editors: Each plate must have unique scope (plate_path) to prevent parameter bleed-through

• Step and function editors must share scope: Function editor needs step parameters for placeholder resolution

• Scope IDs are immutable: Don’t change scope_id after form manager creation

• Scope matching uses ``::`` separator: "/data/plates/plate_001" matches "/data/plates/plate_001::step_0" but not "/data/plates/plate_002"

• Separator matters: Use :: (double colon), not . (period) or : (single colon)

Debugging Scope Issues

Symptom: Function Editor Not Syncing

Cause: Step and function editors have different scopes

Diagnosis:

# Check scope IDs
logger.debug(f"Step editor scope: {step_editor.form_manager.scope_id}")
logger.debug(f"Function editor scope: {function_editor.scope_id}")
# Should be identical (including step token)

Fix: Ensure both editors use same scope_id from _build_step_scope_id()

Symptom: Cross-Orchestrator Bleed-Through

Cause: Multiple orchestrators sharing same scope prefix

Diagnosis:

# Check orchestrator/plate scopes
logger.debug(f"Orchestrator A scope: {scope_A}")
logger.debug(f"Orchestrator B scope: {scope_B}")
# Should have different plate_path prefixes

Fix: Ensure each orchestrator uses unique plate_path as scope prefix

Pipeline Editor Preview Labels

The pipeline editor displays real-time preview labels (MAT, NAP, FIJI, FILT) that show which configurations are enabled for each step. These labels must update immediately when fields are changed in the step editor, including when fields are reset to None.

Critical Implementation Details

Problem: Preview labels were not updating when resetting fields that had concrete saved values.

Root Cause: The pipeline editor was resolving config objects from the original saved step instead of from the merged step with live values.

When a step is saved with a concrete value (e.g., napari_streaming_config.enabled=True), and then reset to None in the step editor:

1. The live form manager has enabled=None in its current values

2. The pipeline editor collects live context and merges it into a new step object

3. BUG: The config object being resolved was from the ORIGINAL saved step, not the merged step

4. Result: Lazy resolution sees the saved concrete value instead of the live None value

Solution: Resolve config from merged step, not original step

# WRONG: Resolve from original step's config
config = getattr(step, 'napari_streaming_config')  # Has saved enabled=True
# ... merge live values into step ...
resolved = config.enabled  # Still resolves to True!

# CORRECT: Resolve from merged step's config
step_to_use = merge_live_values(step, live_values)  # Has enabled=None
config = getattr(step_to_use, 'napari_streaming_config')  # Has live enabled=None
resolved = config.enabled  # Correctly resolves to None, walks up context

Implementation (openhcs/pyqt_gui/widgets/pipeline_editor.py):

# Build merged step with live values
step_to_use = step
if step_live_values:
    step_to_use = self._merge_live_values(step, step_live_values)

# CRITICAL: Get config from merged step, not original step!
config_to_resolve = getattr(step_to_use, config_attr_name, config)

# Now resolve through context stack
with config_context(global_config):
    with config_context(pipeline_config):
        with config_context(step_to_use):
            resolved_value = config_to_resolve.enabled

This ensures that when a field is reset to None in the step editor, the pipeline editor sees the None value and correctly resolves it through the context hierarchy (GlobalPipelineConfig → PipelineConfig → Step).

Scope Matching for Step Editors

The pipeline editor must match step editors by scope_id to collect the correct live values:

# Build step-specific scope
step_scope = f"{plate_path}::{step.name}"

# Only collect live context from:
# 1. Global scope (None)
# 2. Exact plate scope match
# 3. Exact step scope match (for THIS specific step)
is_visible = (
    manager.scope_id is None or
    manager.scope_id == plate_scope or
    manager.scope_id == step_scope
)

This prevents collecting live values from other step editors in the same plate, ensuring each step’s preview labels only reflect its own editor’s state.