Observation positional data filtered out when combining with responses

[oyvindeide]

In

ert/src/ert/storage/local_ensemble.py

Line 986 in 5fe1b25

def get_observations_and_responses(

we filter out the location data from the resulting dataframe, meaning that the locations is not available to the update algorithm. In the data loaded from: In

ert/src/ert/storage/local_ensemble.py

Line 1004 in 5fe1b25

, we have the positional data, but it is since filtered out.

We should include it so that the update algorithm also has access to it, to reproduce:

Modify the snake oil observation config to only have:

SUMMARY_OBSERVATION WOPR_OP1_9
{
    VALUE   = 0.1;
    ERROR   = 0.05;
    DATE    = 2010-03-31;  -- (RESTART = 9)
    KEY     = WOPR:OP1;
    LOCATION_X = 50;
    LOCATION_Y = 50;
    LOCATION_RANGE = 2;
};

[SAKavli]

Including the 3 location keywords in the DataFrame will not concat with other observation types which does not have the same DataFrame width. This failed on my branch:
For example:

SUMMARY_OBSERVATION WOPR_OP1_9
{
    VALUE   = 0.1;
    ERROR   = 0.05;
    DATE    = 2010-03-31;  -- (RESTART = 9)
    KEY     = WOPR:OP1;
    LOCATION_X = 50;
    LOCATION_Y = 50;
    LOCATION_RANGE = 2;
};
GENERAL_OBSERVATION WPR_DIFF_1 {
   DATA       = SNAKE_OIL_WPR_DIFF;
   INDEX_LIST = 400,800,1200,1800;
   DATE       = 2015-06-13;  -- (RESTART = 199)
   OBS_FILE   = wpr_diff_obs.txt;
};

[SAKavli]

I suppose we do not want to add localization keywords to GENERAL_OBSERVATIONS, but we do not want to keep localization keywords in it's own DataFrame. Do you see some solution to this @oyvindeide ?

[SAKavli]

[SAKavli]

Perhaps we could allow localization keywords to be Null for gen obs?

[oyvindeide]

Perhaps we could allow localization keywords to be Null for gen obs?

I think this is a good suggestion! To make that part of the default for observations, that localization is not activated, and it is up to the observation type to implement a way to activate that feature.

That means that a future CSVObservation could allow the user to give a table of:

value	error	custom_prop_1	custom_prop_2	localization_x	localization_y
10	0.1	prop_a	1020	10.34	1045.50

this would give the response a primary key of: ["custom_prop_1", "custom_prop_2", "localization_x", "localization_y"]

and a response per realization of:

value	custom_prop_1	custom_prop_2	localization_x	localization_y
10	prop_a	1040	10.34	1045.50
10	prop_b	1020	12f0.34	1045.50
10	prop_c	1020	14320.34	1045.50
12	prop_a	1020	1.34	1045.50
10	prop_d	1020	1453.34	1045.50

[SAKavli]

I will follow the lead of RFT Observations and rename localization keywords:

• location_x -> east
• location_y -> north
  RFT does not have location_range implemented yet, but as we are stepping away from the location_ format, I suggest renaming it to influence_range