feat: add flexibility for solvers besides Gurobi

[danielolsen]

Purpose

Add flexibility for solvers besides Gurobi. Closes #97

What is the code doing

In loop.jl:

• we add a function symbolize (unrelated refactor to clean up the manual conversion from a Dict of strings to a NamedTuple within interval_loop).
• we add a function new_model, which returns a new JuMP model. This function can take either a Gurobi environment, or an optimizer factory.
• in interval_loop:
  • we change the inputs, so that we can pass either a Gurobi environment or any other optimizer factory.
  • instead of hardcoding that we create a new model from the passed Gurobi environment, we use the new_model function.
  • we only activate `BarHomogeneous if we are using Gurobi
  • we use our new symbolize function for succinctness (unrelated)

In REISE.jl, we add a new optional input optimizer_factory

• if this is not given, we start a Gurobi environment, just like before. We also put Gurobi-specific stuff within if block.
• if this is given, we pass this to interval_loop

Testing

This has been tested with both Clp.jl and GLPK.jl, as well as in the default mode with Gurobi.

with GLPK.jl installed:

import REISE
import GLPK
REISE.run_scenario(; interval=1, n_interval=3, start_index=1, inputfolder="path_to_your_local_input_files", optimizer_factory=GLPK.Optimizer)

with Clp.jl installed:

import REISE
import Clp
REISE.run_scenario(; interval=1, n_interval=3, start_index=1, inputfolder="path_to_your_local_input_files", optimizer_factory=Clp.Optimizer)

Time to review

30 minutes.

[BainanXia]

We don't need to do similar thing for other solvers or this is something to explore？

[danielolsen]

I don't know that other solvers have a similar 'environment' feature that needs to be managed.

[BainanXia]

Again, this implies JuMP treats Gurobi differently but all the other solvers in the same way?

[danielolsen]

We need to use the direct_model feature to be able to pass the Gurobi environment to the Gurobi.Optimizer.

[danielolsen]

@BainanXia we can have JuMP treat Gurobi the same as all of the other optimizers if we call

import REISE
import Gurobi
REISE.run_scenario(; interval=1, n_interval=3, start_index=1, inputfolder="path_to_your_local_input_files", optimizer_factory=Gurobi.Optimizer)

[BainanXia]

@BainanXia we can have JuMP treat Gurobi the same as all of the other optimizers if we call

import REISE
import Gurobi
REISE.run_scenario(; interval=1, n_interval=3, start_index=1, inputfolder="path_to_your_local_input_files", optimizer_factory=Gurobi.Optimizer)

Good to know. I think it is better to make Gurobi one of the options instead of one special option?

[danielolsen]

Good to know. I think it is better to make Gurobi one of the options instead of one special option?

The reason that we have the 'special' code for Gurobi is because we want to have the ability to reuse a single Gurobi environment for all of the solves, for performance reasons. I don't think there's any convenient way to pass this to run_scenario though, unless we handle the creation of the Gurobi environment and the related cleanup section outside of this function.

For example, we could refactor run_scenario/interval_loop/new_model to take either an optimizer factory, or an optimizer factory + a list of arguments + solver settings, and then we would call some new function run_scenario_gurobi_env which would create an env, pass it with the solver_kwargs and Gurobi.Optimizer to run_scenario etc., and then clean up afterwards. Then run_scenario wouldn't have anything Gurobi-specific in it, although interval_loop still requires us to know which solver we are using to be able to know which solver settings are available and how to modify them as we run into infeasibilities.

[BainanXia]

Good to know. I think it is better to make Gurobi one of the options instead of one special option?

The reason that we have the 'special' code for Gurobi is because we want to have the ability to reuse a single Gurobi environment for all of the solves, for performance reasons. I don't think there's any convenient way to pass this to run_scenario though, unless we handle the creation of the Gurobi environment and the related cleanup section outside of this function.

For example, we could refactor run_scenario/interval_loop/new_model to take either an optimizer factory, or an optimizer factory + a list of arguments + solver settings, and then we would call some new function run_scenario_gurobi_env which would create an env, pass it with the solver_kwargs and Gurobi.Optimizer to run_scenario etc., and then clean up afterwards. Then run_scenario wouldn't have anything Gurobi-specific in it, although interval_loop still requires us to know which solver we are using to be able to know which solver settings are available and how to modify them as we run into infeasibilities.

For other solvers, if possible, we also want to reuse the environment setup for all of the solves to improve the performance, right?

[danielolsen]

For other solvers, if possible, we also want to reuse the environment setup for all of the solves to improve the performance, right?

It depends. Some solvers have that capability, some do not. Some solvers don't even let you reassign values for constraints, etc. Using JuMP.Model is the most general way to interface with a variety of different solvers. direct_model lets you do some things that can improve performance (if they are supported by the solver), with some drawbacks (see https://jump.dev/JuMP.jl/stable/solvers/#JuMP.direct_model). I don't think there are any commands in JuMP to make the 'reuse the same environment' calls generalizeable across many different solvers.

[BainanXia]

Given @danielolsen has tested with optimizations, I think for now this is good enough to have such capability only. We may explore in the future on how to support other solvers better, but given we are focusing on Gurobi, it is reasonable to say our framework is optimized based on Gurobi, which is obviously the suggested solver.

[dmuldrew]

I'm wondering about the term factory_like...I'm not sure exactly what it communicates. Maybe something like factory_spec or factory_env or optimization_factory_env, etc?

[danielolsen]

It's either an optimization factory or a Gurobi environment.