Skip to content

Fix small issues in everest docs #12598

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
SAKavli merged 2 commits into from
Jan 12, 2026
Merged

Fix small issues in everest docs #12598

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4c5de6f
Small fixups in Everest documentation
SAKavli Jan 8, 2026
018361a
Consistently wrap all long lines
SAKavli Jan 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 48 additions & 40 deletions docs/everest/example.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,22 @@
Example
*******

In this section we will go through the setup and running of a simple everest optimization case.
In this section we will go through the setup and running of a simple Everest optimization
case.

Problem definition
##################

The optimization problem we will solve using everest will be to minimize the distance between an initial 3D point
and another given target 3D point.
The optimization problem we will solve using Everest will be to minimize the distance
between an initial 3D point and another given target 3D point.

Everest config
##################

The main element an Everest optimization experiment requires is a configuration file. Everest supports the `YAML <https://en.wikipedia.org/wiki/YAML>`_
format for the configuration file. The configuration file is build up of multiple sections and next we will go through it section
by section.
The main element an Everest optimization experiment requires is a configuration file.
Everest supports the `YAML <https://en.wikipedia.org/wiki/YAML>`_ format for the
configuration file. The configuration file is built up of multiple sections and next we
will go through it section by section.

Creating the optimization folder::

Expand Down Expand Up @@ -77,11 +79,12 @@ everest_config.yml::
min: -1.0
max: 1.0

``perturbation_magnitude`` controls the size of perturbations (standard deviation of a normal distribution) of controls used to approximate the gradient.
The value depends on the type of control and magnitude of the variables.
For continuous controls smaller values should give a better gradient,
whilst for more discrete controls larger values should give a better
result. However, this is a balance as too large or too small values also cause issues.
``perturbation_magnitude`` controls the size of perturbations (standard deviation of a
normal distribution) of controls used to approximate the gradient. The value depends on
the type of control and magnitude of the variables. For continuous controls smaller values
should give a better gradient, whilst for more discrete controls larger values should give
a better result. However, this is a balance as too large or too small values also cause
issues.

``variables``: list of control variables.

Expand All @@ -107,17 +110,18 @@ everest_config.yml::
**Details**

``algorithm``: the algorithm used for the optimization
currently supported algorithms: optpp_q_newton, conmin_mfd, conmin_frcg, mesh_adaptive_search.
currently supported algorithms: optpp_q_newton, conmin_mfd, conmin_frcg,
mesh_adaptive_search.

``convergence_tolerance``: provides a real value for controlling the termination of
iteration. In most cases, it is a relative convergence tolerance for the
objective function; i.e., if the change in the objective function between
successive iterations divided by the previous objective function is less than
the amount specified by convergence_tolerance, then this convergence criterion is satisfied on the
current iteration. Since no progress may be made on one iteration followed by significant progress
on a subsequent iteration, some libraries require that the convergence tolerance
be satisfied on two or more consecutive iterations prior to termination of
iteration (from the Dakota Manual.)
iteration. In most cases, it is a relative convergence tolerance for the objective
function; i.e., if the change in the objective function between successive iterations
divided by the previous objective function is less than the amount specified by
convergence_tolerance, then this convergence criterion is satisfied on the current
iteration. Since no progress may be made on one iteration followed by significant progress
on a subsequent iteration, some libraries require that the convergence tolerance be
satisfied on two or more consecutive iterations prior to termination of iteration (from
the Dakota Manual.)

Install jobs definition
-----------------------
Expand All @@ -132,7 +136,8 @@ everest_config.yml::

**Details**

The custom job can be *installed* by adding information regarding it in the **install_jobs** section. Each custom job entry will contain:
The custom job can be *installed* by adding information regarding it in the
**install_jobs** section. Each custom job entry will contain:

``name``: the name the job, the same name will be used in the forward model section.

Expand All @@ -149,8 +154,8 @@ everest_config.yml::

**Details**

``realizations``: list of realizations to use in optimization ensemble. Typically, this is a
list [0, 1, ..., n-1] of all realizations in the ensemble.
``realizations``: list of realizations to use in optimization ensemble. Typically, this is
a list [0, 1, ..., n-1] of all realizations in the ensemble.

Forward model definition
------------------------
Expand All @@ -162,11 +167,12 @@ everest_config.yml::

**Details**

``distance3d``: the name of the custom job installed in the ``install_jobs`` section of the config file.
The job name is followed by named arguments the job is designed to accept.
``distance3d``: the name of the custom job installed in the ``install_jobs`` section of
the config file. The job name is followed by named arguments the job is designed to
accept.

``point.json``: file generated by the Everest optimization experiment. It contains the list of control variables with
the values used for the current forward model evaluation.
``point.json``: file generated by the Everest optimization experiment. It contains the
list of control variables with the values used for the current forward model evaluation.

Example::

Expand All @@ -176,8 +182,8 @@ Example::
"z" : 0
}

Everest creates the file with the name ``point.json`` because we have added to the ``controls`` section the control with
the name ``point``.
Everest creates the file with the name ``point.json`` because we have added to the
``controls`` section the control with the name ``point``.

``distance``: file required for the optimization experiment
to succeed.
Expand All @@ -187,7 +193,8 @@ One of the jobs in the forward model needs to create the file ``distance``.
``distance`` needs to contain the value for the objective function
evaluation.

The files needs to be called ``distance`` because we have defined the objective function named ``distance``.
The files needs to be called ``distance`` because we have defined the objective function
named ``distance``.

Optimization environment definition
-----------------------------------
Expand Down Expand Up @@ -252,14 +259,14 @@ everest_config.yml::
output_folder: everest_optimization
random_seed: 999

More information regarding all the available section options and additional sections not covered in the current example
can be found in :ref:`cha_config` section
More information regarding all the available section options and additional sections not
covered in the current example can be found in :ref:`cha_config` section

Creating the custom job
#######################

Before we are able to start the optimization experiment we need to create the script for the custom job
``distance3d`` used in the forward model.
Before we are able to start the optimization experiment we need to create the script for
the custom job ``distance3d`` used in the forward model.

Creating the folders and files::

Expand All @@ -268,8 +275,8 @@ Creating the folders and files::
~/everest_example/jobs$: touch distance3d.py
~/everest_example/jobs$: chmod a+x distance3d.py

``chmod +x distance3d.py``: command is used to change the access permissions for the file ``distance3d.py``, such that execution of the file is
allowed.
``chmod +x distance3d.py``: command is used to change the access permissions for the file
``distance3d.py``, such that execution of the file is allowed.

distance3d.py::

Expand Down Expand Up @@ -318,13 +325,14 @@ distance3d.py::
if __name__ == "__main__":
main(sys.argv[1:])

More information about creating custom jobs can be found in the :ref:`cha_creating_custom_jobs` section
More information about creating custom jobs can be found in the
:ref:`cha_creating_custom_jobs` section

Running everest
Running Everest
###############

Now we have all the components needed to start the optimization experiment,
which can be done using the following command::
Now we have all the components needed to start the optimization experiment, which can be
done using the following command::

~/everest_example$: everest run everest_config.yml

Expand Down
92 changes: 71 additions & 21 deletions docs/everest/introduction.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,38 +9,75 @@ Background and motivation

Everest is a tool, which uses ensembles of models to deliver a robust optimization.

While the result of an data assimilation process is an ensemble of models, the result of the robust optimization process is one optimal strategy, for example for well order or drainage strategy. With this optimised strategy we can, e.g., maximize net present value (NPV) with respect to the geological uncertainty in the models. Note that Everest is a generic tool that can be applied to other optimisation problems (also outside the domain of Oil & Gas).
While the result of a data assimilation process is an ensemble of models, the result of
the robust optimization process is one optimal strategy, for example for well order or
drainage strategy. With this optimized strategy we can, e.g., maximize net present value
(NPV) with respect to the geological uncertainty in the models. Note that Everest is a
generic tool that can be applied to other optimization problems (also outside the domain
of Oil & Gas).


Everest
=======

The objectives (what to optimise) can, for example, be the Net Present Value (NPV), the recovery factor, or carbon emissions. The controls (what to adjust in order to optimise the objective) can, for example, be the drilling order of the wells and production and/or injection rates (drainage strategy). Additionally, constraints on these controls can be handled both within Everest and/or in flow simulator.
The objectives (what to optimize) can, for example, be the Net Present Value (NPV), the
recovery factor, or carbon emissions. The controls (what to adjust in order to optimize
the objective) can, for example, be the drilling order of the wells and production and/or
injection rates (drainage strategy). Additionally, constraints on these controls can be
handled both within Everest and/or in flow simulator.

The primary goal of the Everest tool is to find an "optimal" strategies by utilizing an ensemble of reservoir models (e.g., an ensemble of geologically-consistent models). This will help make robust decisions about drilling schedule or drainage strategy in an efficient and assisted manner.
The primary goal of the Everest tool is to find an "optimal" strategy by utilizing an
ensemble of reservoir models (e.g., an ensemble of geologically-consistent models). This
will help make robust decisions about drilling schedule or drainage strategy in an
efficient and assisted manner.

To this end, an optimization problem is formulated and solved by the *Ensemble Optimization* (EnOpt) method described here: :ref:`en-opt-label`.
To this end, an optimization problem is formulated and solved by the
*Ensemble Optimization* (EnOpt) method described here: :ref:`en-opt-label`.

A single configuration file is required to define controls, constraints, objective functions, and other items related to gradient estimation for optimization. This file should also include items related to simulations themselves, such as manipulating schedule files according to the controls and calculating the values of objective functions and constraints.
A single configuration file is required to define controls, constraints, objective
functions, and other items related to gradient estimation for optimization. This file
should also include items related to simulations themselves, such as manipulating schedule
files according to the controls and calculating the values of objective functions and
constraints.

Everest will, then, generate new set of controls as a result of the simulated results, e.g., ensemble objective function values.
The flow chart below provides a detailed description of the elements involved in running an optimization experiment.
Everest will, then, generate new set of controls as a result of the simulated results,
e.g., ensemble objective function values. The flow chart below provides a detailed
description of the elements involved in running an optimization experiment.


Considerations regarding the usage of Everest
-----------------------------------------------

Everest can assist the decision makers when there are many outcomes or decisions that need to be made. If the decision and outcome is limited or easy, then Everest may not be needed as one can evaluate the outcomes with a heuristic approach. However, it is unlikely that one can easily find few outcomes or real optimised, robust strategies using this “manual” approach when uncertainty is involved. In the following sections we will discuss the use of Everest in different cases and some of the "ifs and buts " associated with the application of the tool.

The power of the Everest tool is to assist in optimising strategies while capturing the underlying reservoir uncertainty, which is modelled by an ensemble of models. A crucial factor for the success of the optimisation is to have good enough model quality and the uncertainty well represented - i.e., to span the current understanding or knowledge of the reservoir uncertainty. Moreover, the greater the reservoir complexity, the more important it is to have many model realisations in the ensemble, i.e., to span the actual reservoir uncertainty. The alternative might lead to a situation where the strategy is not representative. The essential take away is that Everest cannot improve your model's quality and your uncertainty representation, and that the quality of the Everest optimisation results is dependent on the quality of the underlying model and uncertainty representation.

When doing experiments that involve tuning specific parameters, it is advisable to begin with a coarse-grained approach when adjusting control variables. Initially, one might consider modifying the parameters at long intervals, such as every several years or at key points in the process timeline. This sets a foundational structure for the optimization without overwhelming the algorithm with too many variables.

In general, the following elements should be reflected upon when consider applying Everest for decision support and decision making:

* the need for, or value of, robustness of optimisation results;

* the value of time saving from using the tool (heuristic approach versus assisted optimisation);
Everest can assist the decision makers when there are many outcomes or decisions that need
to be made. If the decision and outcome are limited or easy, then Everest may not be
needed as one can evaluate the outcomes with a heuristic approach. However, it is unlikely
that one can easily find few outcomes or real optimized, robust strategies using a
“manual” approach when uncertainty is involved. In the following sections we will discuss
the use of Everest in different cases and some of the "ifs and buts" associated with the
application of the tool.

The power of the Everest tool is to assist in optimizing strategies while capturing the
underlying reservoir uncertainty, which is modelled by an ensemble of realizations. A
crucial factor for the success of the optimization is to have good enough model quality
and the uncertainty well represented - i.e., to span the current understanding or
knowledge of the reservoir uncertainty. Moreover, the greater the reservoir complexity,
the more important it is to have many model realizations in the ensemble, i.e., to span
the actual reservoir uncertainty. The alternative might lead to a situation where the
strategy is not representative. The essential takeaway is that Everest's results depend
on — but does not improve a model's quality or uncertainty representation.

When doing experiments that involve tuning specific parameters, it is advisable to begin
with a coarse-grained approach when adjusting control variables. Initially, one might
consider modifying the parameters at long intervals, such as every several years or at key
points in the process timeline. This sets a foundational structure for the optimization
without overwhelming the algorithm with too many variables.

In general, the following elements should be reflected upon when consider applying Everest
for decision support and decision making:

* the need for, or value of, robustness of optimization results;

* the value of time saving from using the tool (heuristic approach versus assisted optimization);

* the quality of the underlying reservoir models and uncertainty span;

Expand All @@ -55,6 +92,19 @@ Everest workflow
:width: 700px
:alt: Everest workflow

The figure shows the Everest workflow. The workflow starts with defining the initial controls (e.g., drilling order), objective function (e.g., NPV), constraints (e.g., upper injection rate) and related information, such as optimization algorithm, perturbation magnitude, and ensemble realizations used in optimization. Based on these information, Everest will run simulations in order to evaluate either objective function or gradient with respect to the current control variables. Also, it prepares the input files for the simulator, such as the schedule file.

Normally, different schedule files will be generated for different control values. For example, in case the drilling order is the control, with different drilling orders the well open dates can be totally different, which leads to different schedule files. Once all necessary simulator files are set up, the ensemble of simulations will be run. Based on the simulation results, Everest will calculate the objective function value (e.g. NPV) or constraint values. Everest will then update the controls based on the selected optimization algorithm and run a new experiment until the convergence criteria is satisfied.
The figure shows the Everest workflow. The workflow starts with defining the initial
controls (e.g., drilling order), objective function (e.g., NPV), constraints (e.g., upper
injection rate) and related information, such as optimization algorithm, perturbation
magnitude, and ensemble realizations used in optimization. Based on this information,
Everest will run simulations in order to evaluate either objective function or gradient
with respect to the current control variables. Also, it prepares the input files for the
simulator, such as the schedule file.

Normally, different schedule files will be generated for different control values. For
example, in case the drilling order is the control, with different drilling orders the
well open dates can be totally different, which leads to different schedule files. Once
all necessary simulator files are set up, the ensemble of simulations will be run. Based
on the simulation results, Everest will calculate the objective function value (e.g. NPV)
or constraint values. Everest will then update the controls based on the selected
optimization algorithm and run a new experiment until the convergence criteria is
satisfied.
Loading