HydroPilot

Architecture

Architecture

This page introduces HydroPilot's current architecture from the implementation point of view.

Two major chains

HydroPilot has two connected but distinct chains:

  • Config chain: YAML config -> validation and template expansion -> RunConfig
  • Runtime chain: SimModel -> Session -> Workspace -> Executor -> execution services -> reporting

The config chain turns user input into a validated runtime configuration. The runtime chain uses that configuration to execute model runs, extract outputs, evaluate metrics, and persist results.

Config chain

At a high level, configuration preparation follows this path:

text
YAML
  -> prepare_config()
  -> optional template expansion
  -> general validation
  -> RunConfig.from_raw()
  -> load_config()

Key behavior:

  • version: general means the YAML already describes the runtime config directly.
  • Template modes such as version: swat, version: swatplus, or version: xaj are expanded into the general runtime form before execution.
  • Validation happens on the expanded config, so model-template issues surface early.
  • When runtime loading succeeds, HydroPilot can emit a resolved _general.yaml beside the source config for inspection.

This chain is shared across CLI validation and runtime loading. The difference is that CLI commands convert failures into user-facing diagnostics, while the Python API raises exceptions.

Main modules

The runtime-facing modules are organized around:

  • api for public entry points such as SimModel
  • cli for command-line interfaces
  • config for config loading, path resolution, and schema conversion
  • evaluation for objectives, constraints, diagnostics, and derived values
  • io for runners, readers, and writers
  • models for template-specific model logic
  • params for parameter space and parameter application
  • reporting for archived outputs and run artifacts
  • runtime for session and workspace lifecycle
  • series for simulation and observation extraction planning
  • validation for user-facing config diagnostics

Runtime chain

The runtime path starts from SimModel and then delegates to a set of orchestration components:

text
SimModel
  -> Session
  -> Workspace
  -> Executor
     -> ExecutionServices
        -> ParamSpace
        -> ParamWritePlan
        -> ParamApplier
        -> SeriesPlan
        -> ObsStore
        -> SubprocessRunner
        -> SeriesExtractor
        -> Evaluator
  -> RunReporter

Each layer has a clear role:

  • SimModel is the public facade for Python users.
  • Session owns lifecycle and cleanup for one loaded runtime.
  • Workspace creates isolated run directories and manages reusable model instances.
  • Executor runs one or more evaluations sequentially or in parallel.
  • ExecutionServices assembles the concrete runtime dependencies from RunConfig.
  • RunReporter persists artifacts, summaries, and archived outputs.

Workspace and execution

HydroPilot executes model runs inside isolated working directories under basic.workPath. For each evaluation path it can:

  1. create or acquire a prepared project copy
  2. transform design values into physical parameters
  3. write parameters into model input files
  4. run the configured model command
  5. extract simulation outputs and observation slices
  6. evaluate objectives, constraints, and diagnostics
  7. archive the results and release the instance

By default, temporary instance_* directories are cleaned after the session closes. If basic.keepInstances: true is enabled, those directories are preserved for debugging.

Why templates matter

Templates are one of HydroPilot's main architectural advantages. They let users work with model-facing concepts such as SWAT parameters or XAJ CSV mappings without writing every low-level reader and writer manually.

In practice, this means:

  • model-specific knowledge stays close to the template layer
  • the runtime still executes a single general-form config
  • CLI, Python API, and UQPyL integration all share the same execution foundation

Why this matters

This split is what makes HydroPilot useful as an engineering layer:

  • configuration stays declarative
  • model-specific details stay close to templates and I/O handlers
  • runtime orchestration stays reusable for CLI, Python API, and UQPyL integration