Skip to content

Add evolving myopic mode#278

Open
idelder wants to merge 12 commits intoTemoaProject:unstablefrom
idelder:feature/myopic_evolving
Open

Add evolving myopic mode#278
idelder wants to merge 12 commits intoTemoaProject:unstablefrom
idelder:feature/myopic_evolving

Conversation

@idelder
Copy link

@idelder idelder commented Mar 13, 2026

Creates two versions of myopic mode: normal and evolving.

  • Adds these settings to the config
  • Adds a boilerplate for the evolution script
  • Updates iteration logic
  • Updates myopic sequencing test to test both modes
  • Updates some guard rails and logging

Evolving myopic:

We call an "evolution script" between iterations, which may be used to update the myopic database (e.g., to "surprise" the optimisation as it moves forward, or to update parameters based on past decisions). In this mode, the problem may change going forward and so we need to "shrink" the view depth as we approach the end of the horizon, so that we make decisions at every defined step.

Normal (non-evolving) myopic:

We do not call the evolution script. As a result, the period which views to the end of the horizon should be the final planning period. There is no need to shrink the view depth and step forward as the model should make identical decisions, as this is just a subproblem of the previous problem.

NOTE: Evolving myopic (with no evolution script) is currently the default in Temoa, but this would make normal (non-evolving) the default behaviour as it saves unnecessary compute.

Utopia testing for all possible view depths and step sizes:

Old behaviour:

log_baseline.txt

New normal (non-evolving) behaviour for all possible view depths and step sizes:

log_new_non-evolving.txt

New evolving behaviour for all possible view depths and step sizes:

log_new_evolving.txt

Summary by CodeRabbit

  • New Features

    • Added optional evolution step between myopic iterations with support for user-provided evolution scripts.
    • Progress output now shows evolution stages during runs.
  • Improvements

    • Enforced positive-integer validation for myopic parameters.
    • Revised iteration planning to avoid redundant near-horizon repeats and surface runtime errors for invalid plans.
  • Configuration

    • New myopic settings: evolving (bool) and evolution_script (string).
  • Tests

    • Expanded tests to cover evolving vs non-evolving scenarios and expected iteration outcomes.

@coderabbitai
Copy link

coderabbitai bot commented Mar 13, 2026

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: a69d8443-3061-4ed7-8817-c18d7933e75b

📥 Commits

Reviewing files that changed from the base of the PR and between 4788834 and dfe5d49.

📒 Files selected for processing (1)
  • temoa/extensions/myopic/myopic_sequencer.py
 ________________________________
< Overly attached code reviewer. >
 --------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       /   づ

Walkthrough

This PR introduces evolving iteration functionality to the myopic optimization framework. It adds configuration parameters and execution hooks that allow custom evolution scripts to update the database during myopic iterations, alongside modifications to progress reporting and iteration sequencing logic.

Changes

Cohort / File(s) Summary
Configuration and Display
temoa/core/config.py, temoa/tutorial_assets/config_sample.toml, tests/testing_configs/config_utopia_myopic.toml
Extended TemoaConfig.__repr__ to display evolving mode and evolution script path. Updated sample configuration to set step_size: 2, evolving: false, and evolution_script path. Added evolving configuration to test config.
Evolution Framework
temoa/extensions/myopic/evolution_updater.py
New module providing a top-level iterate() function as a hook executed after each myopic iteration. Accepts iteration context (base year, previous year, instance status) and database connection.
Progress Reporting
temoa/extensions/myopic/myopic_progress_mapper.py
Extended MyopicProgressMapper.report() to accept 'evolve' status and emit "EVLV" markers with consistent formatting during evolution steps.
Core Iteration Logic
temoa/extensions/myopic/myopic_sequencer.py
Added evolving and evolution_script attributes. Tightened validation for view_depth and step_size as positive integers. Inserted evolution step at the beginning of each non-first iteration. Added run_evolution_script() method to dynamically load and invoke evolution modules. Refactored characterize_run() iteration planning with base_years computation and redundancy pruning for non-evolving scenarios.
Tests
tests/test_myopic_sequencer.py
Expanded parameterization to test evolving vs non-evolving scenarios (single_step_evolving, triple_step_evolving). Added expected_last_base_year assertions and improved failure messages for last_year and last_demand_year validation.

Sequence Diagram(s)

sequenceDiagram
    participant Seq as MyopicSequencer
    participant EvUp as EvolutionUpdater
    participant DB as SQLite Database
    
    loop Each Myopic Iteration
        Seq->>Seq: Log iteration index
        alt evolving == true AND not first iteration
            Seq->>EvUp: run_evolution_script(idx, prev_base_year, status)
            EvUp->>DB: Update via iterate() hook
            DB-->>EvUp: Changes committed
            EvUp-->>Seq: Evolution complete
            Seq->>Seq: Report 'evolve' status
        end
        Seq->>Seq: Load demand data
        Seq->>Seq: Construct TEMOA instance
        Seq->>DB: Solve optimization
        Seq->>Seq: Report 'solve' status
        Seq->>Seq: Check results
        Seq->>Seq: Report 'report' status
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related PRs

  • TemoaProject/temoa#186: Modifies TemoaConfig class in temoa/core/config.py, overlapping with this PR's configuration display enhancements.
  • TemoaProject/temoa#158: Affects temoa/core/config.py and temoa/extensions/myopic/myopic_sequencer.py, sharing core myopic module modifications.

Suggested labels

enhancement

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 58.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Add evolving myopic mode' accurately reflects the main feature addition: a new evolving myopic mode variant to the Temoa system.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@idelder idelder changed the title Adds evolving myopic mode Add evolving myopic mode Mar 13, 2026
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@temoa/extensions/myopic/myopic_sequencer.py`:
- Around line 213-221: Check that the path in self.evolution_script exists and
is a readable file (use os.path.exists/isfile) before calling
util.spec_from_file_location; after creating spec verify spec is not None and
spec.loader is not None before calling spec.loader.exec_module; once the module
is loaded verify hasattr(evolution_module, "iterate") and
callable(evolution_module.iterate) and only then call
evolution_module.iterate(...)—otherwise log/raise a clear, descriptive error
mentioning the missing file, spec, loader, or non-callable iterate so failures
are explicit and debuggable.

In `@temoa/tutorial_assets/config_sample.toml`:
- Line 157: The sample config uses a Windows-style path for the evolution_script
setting; update the evolution_script value (the "evolution_script" key) to use a
cross-platform path (POSIX-style forward slashes) so it resolves on Linux/macOS
and is safe to copy/paste (e.g., change the value from
"temoa\\extensions\\myopic\\evolution_updater.py" to a forward-slash form like
"temoa/extensions/myopic/evolution_updater.py").

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 18339bab-ae1c-43c9-ad62-14122d735c76

📥 Commits

Reviewing files that changed from the base of the PR and between 7f7ff86 and ea95e06.

📒 Files selected for processing (8)
  • temoa/core/config.py
  • temoa/data_io/hybrid_loader.py
  • temoa/extensions/myopic/evolution_updater.py
  • temoa/extensions/myopic/myopic_progress_mapper.py
  • temoa/extensions/myopic/myopic_sequencer.py
  • temoa/tutorial_assets/config_sample.toml
  • tests/test_myopic_sequencer.py
  • tests/testing_configs/config_utopia_myopic.toml

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
temoa/tutorial_assets/config_sample.toml (1)

156-156: ⚠️ Potential issue | 🟠 Major

Set sample evolving default to non-evolving.

evolving = true in the sample template enables the compute-heavier path by default for copy/paste users, which contradicts the PR goal of defaulting to normal (non-evolving) myopic behavior.

🔧 Proposed fix
-evolving = true
+evolving = false
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@temoa/tutorial_assets/config_sample.toml` at line 156, The sample config
currently sets evolving = true which makes the heavier evolving path the
default; update the sample template to set the flag to non-evolving by changing
the setting evolving = true to evolving = false (update the config key
"evolving" in the sample file/template so copy/paste users get the normal myopic
behavior by default).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@temoa/extensions/myopic/evolution_updater.py`:
- Around line 8-12: The code allows idx: MyopicIndex | None but later uses
idx.base_year unguarded; add an explicit check in the function (the constructor
or initializer that accepts idx) to validate idx is not None before accessing
idx.base_year (or handle the None case deterministically), e.g., if idx is None
raise a clear ValueError or provide a documented fallback; update any logic that
reads idx.base_year to run only after this guard so failures are explicit and
deterministic (refer to the parameter name idx and the type MyopicIndex and the
property base_year when locating the usage).

---

Duplicate comments:
In `@temoa/tutorial_assets/config_sample.toml`:
- Line 156: The sample config currently sets evolving = true which makes the
heavier evolving path the default; update the sample template to set the flag to
non-evolving by changing the setting evolving = true to evolving = false (update
the config key "evolving" in the sample file/template so copy/paste users get
the normal myopic behavior by default).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 96b3e5d7-d97a-4301-b805-bacc6ad8bbbd

📥 Commits

Reviewing files that changed from the base of the PR and between ea95e06 and 563fb1b.

📒 Files selected for processing (3)
  • temoa/extensions/myopic/evolution_updater.py
  • temoa/extensions/myopic/myopic_sequencer.py
  • temoa/tutorial_assets/config_sample.toml

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@temoa/tutorial_assets/config_sample.toml`:
- Around line 149-155: Clarify that step_size is only meaningful when evolving
mode is enabled by updating the comment near the step_size setting: mention that
step_size controls how many periods the planner advances on each myopic
iteration when evolving mode (and evolution_script) is enabled, and that it must
be <= view_depth; also add a short note that when evolving mode is false,
step_size has no effect. Reference the existing symbols: step_size, evolving
mode (e.g., evolution_script), and view depth in the updated comment so users
understand the relationship.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 1d335026-e71d-4fbb-bf8d-1b8ef1ed4823

📥 Commits

Reviewing files that changed from the base of the PR and between 563fb1b and 1133ff4.

📒 Files selected for processing (2)
  • temoa/tutorial_assets/config_sample.toml
  • tests/testing_configs/config_utopia_myopic.toml

@ParticularlyPythonicBS ParticularlyPythonicBS added Feature Additional functionality enhancement labels Mar 21, 2026
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
temoa/extensions/myopic/evolution_updater.py (1)

32-36: 🧹 Nitpick | 🔵 Trivial

Use lazy logging format instead of f-string.

The f-string is evaluated eagerly even if the log level is disabled. Use %-style formatting for deferred evaluation.

♻️ Suggested cleanup
-    logger.info(f"Running myopic iteration updater for base year {idx.base_year}")
-    
-    # Update your myopic database here.
-
-    return
+    logger.info('Running myopic iteration updater for base year %s', idx.base_year)
+
+    # Update your myopic database here.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@temoa/extensions/myopic/evolution_updater.py` around lines 32 - 36, The log
call in evolution_updater.py uses an eager f-string (logger.info(f"Running
myopic iteration updater for base year {idx.base_year}")), which should be
replaced with lazy %-style logging to avoid unnecessary formatting when the log
level is disabled; update the call to use logger.info("Running myopic iteration
updater for base year %s", idx.base_year) (referencing the existing logger and
idx variables in the same scope).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@temoa/extensions/myopic/myopic_sequencer.py`:
- Around line 360-366: Change the type annotation of the run_evolution_script
parameter idx from int | None to MyopicIndex | None and update any related type
hints/usages accordingly (e.g., the progress_mapper.report call) so it accepts a
MyopicIndex; ensure MyopicIndex is imported or available in the module and
adjust function signature in def run_evolution_script(self, idx: MyopicIndex |
None, ...) and any callers that rely on that annotation to match the actual
MyopicIndex object passed from the main loop.

---

Duplicate comments:
In `@temoa/extensions/myopic/evolution_updater.py`:
- Around line 32-36: The log call in evolution_updater.py uses an eager f-string
(logger.info(f"Running myopic iteration updater for base year
{idx.base_year}")), which should be replaced with lazy %-style logging to avoid
unnecessary formatting when the log level is disabled; update the call to use
logger.info("Running myopic iteration updater for base year %s", idx.base_year)
(referencing the existing logger and idx variables in the same scope).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 5e846fbf-fa5e-431e-a80e-986f27c1a72f

📥 Commits

Reviewing files that changed from the base of the PR and between 1133ff4 and 4788834.

📒 Files selected for processing (7)
  • temoa/core/config.py
  • temoa/extensions/myopic/evolution_updater.py
  • temoa/extensions/myopic/myopic_progress_mapper.py
  • temoa/extensions/myopic/myopic_sequencer.py
  • temoa/tutorial_assets/config_sample.toml
  • tests/test_myopic_sequencer.py
  • tests/testing_configs/config_utopia_myopic.toml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Feature Additional functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants