parquet part 3 : parquet for legacy solver by guilpier-code · Pull Request #3586 · AntaresSimulatorTeam/Antares_Simulator

guilpier-code · 2026-04-21T16:03:57Z

As we choose to break ticket ANT-4633 into several parts, this is part 3.
This PR intends to write simulation tables for legacy solver.

Please note : list of what was done can be found below, but some github comments were left to ease review.
Caution : many review comments made here are addressed in this PR 3657.

What was done :

remove the aggregation of all years' simulation tables into only 2 blocks (one per optimization), as well as printing them into 2 CSV files only. Instead, make each year print its own simulation table.
class OptimisationsSimulationTable was (physically) moved to where it more likely belongs (next to class SimulationTable)
class OptimisationsSimulationTable was very much simplified : it now only contains data (could now be simple struct) and the writing part was transferred to class LegacySimulationTablesWriter.
New class LegacySimulationTablesWriter now uses intensively the table writers developed on modeler side (in PR 3565), instead of keeping old code from class OptimisationsSimulationTable

What remains to be done (if item is checked, then it's done) :

continue simplifying class SimulationTable (especially there is code duplication between class SimulationTable [writeToBuffer(), buffer(), buffer_] and class CsvTableWriter [make_line(...), ...])
We removed unit tests on printing simulation tables into CSV format. They may be replaced by tests printing into files. We could even add more tests for thorough checks.
OptimisationsSimulationTable could be renamed into LegacySimulationTables, maybe more clear.
Consider removing class FileWriter. Indeed, it does little now, and calls another polymorphic writer (csv or parquet). So what FileWriter does could be done in the parent (abstract) class of the polymorphic writer.
Cucumber scripts : I made changes in class solver_input_handler and solver_output_handler, in the name of the simu table files name. are this files useful ? please check.
target parquet_writer was reintroduced by a merge from develop, it used to be named simulation-table-writers, much better (more accurate). Let's rename this target back.
(maybe fixed with a fresh clone) in debug (at least on Win), zlibd.lib not copied into build folder at configuration step. And dll like parquet, arrows + dependencies are not copied next to .exe needing them (even if they exist somewhere in the build folder after cmake config step).
what else ?

…ature/parquet-new-start-after-long-pause # Conflicts: # src/io/outputs/CMakeLists.txt

When vcpkg is used (e.g. for Arrow/Parquet), it prepends its paths to CMAKE_PREFIX_PATH. This causes find_dependency(absl) inside ortoolsConfig.cmake to resolve vcpkg's absl (lts_20240722) instead of the version OR-Tools was compiled against (lts_20250512), leading to thousands of linker errors due to ABI incompatibility. Fix: before find_package(ortools), detect the OR-Tools install prefix and set absl_DIR/re2_DIR to the co-installed versions. Supports both lib/ (Ubuntu, Windows) and lib64/ (Oracle Linux 8) layouts.

Apply the same absl_DIR/re2_DIR pinning to Config.cmake.in so that downstream consumers (e.g. api_client_example) using vcpkg also get the correct OR-Tools co-installed absl/re2 instead of vcpkg's version.

On Windows, vcpkg builds Arrow as a shared library (DLLs), so only Arrow::arrow_shared and Parquet::parquet_shared targets exist. Use the ARROW_BUILD_STATIC generator expression (as recommended by Arrow's own CMake usage docs) to select the correct target based on how Arrow was built.

…t-deps

…here it's needed

…s code

…ing part) + comment for later

sonarqubecloud · 2026-05-20T15:35:57Z

Quality Gate failed

Failed conditions
52.6% Coverage on New Code (required ≥ 60%)

See analysis details on SonarQube Cloud

guilpier-code · 2026-05-21T09:12:29Z

Answers to the big comment above :

Was addressed in merged PR 3654
dynamic_cast chain with no registration mechanism : addressed in separate PR 3657
OptimisationsSimulationTable no longer accumulates data across write() calls : it's a warning (we change of behavior), not a fix requirement.
OptimisationsSimulationTable header has no namespace and a TODO comment : addressed in separate PR 3657
ITableWriter::output_file_ is a non-const reference : Addressed in separate PR 3657
makeTableWriter takes std::filesystem::path& (non-const reference) : addressed in separate PR 3657
LegacySimulationTablesWriter uses std::invalid_argument for filesystem error : it's done in this PR
LegacySimulationTablesWriter constructor takes const fs::path& but stores by copy : addressed in separate PR 3657

Signed-off-by: GitHub Copilot <copilot@example.com>

JasonMarechal25 · 2026-05-27T07:04:02Z

    // Options parsing
    std::filesystem::path studyPath(argv[1]);
-    Writer::TableFormat tableFormat = getTableFormat(argc, argv);
+    bool parquetFormat = parquetFormatRequired(argc, argv);


Please keep format enum and not a boolean.

I knew you would say that, and I think you're right, I should have left it.
I wanted to make it as simple as possible by introducing a boolean, but it's less clear.

This reverts commit f0825a6.

This reverts commit b265d5b.

Co-authored-by: guilpierre <guillaume.pierre_externe@rte-france.com>

…ests NR and make it point to the right commit

…for-legacy-solver # Conflicts: # src/packaging/CMakeLists.txt # src/solver/simulation/economy.cpp # src/tests/resources/Antares_Simulator_Tests_NR

Signed-off-by: GitHub Copilot <copilot@example.com>

Co-authored-by: guilpierre <guillaume.pierre_externe@rte-france.com>

sonarqubecloud · 2026-06-01T07:03:42Z

Quality Gate failed

Failed conditions
48.7% Coverage on New Code (required ≥ 60%)

See analysis details on SonarQube Cloud

JasonMarechal25 and others added 30 commits October 6, 2025 11:29

Wip CSV/Parquet

f94d570

Add parquet

008ad11

Implement parquet table writer

431265b

Use writer

114c932

Sort

e55efde

Merge remote-tracking branch 'remotes/origin/feature/parquet' into fe…

56f980b

…ature/parquet-new-start-after-long-pause # Conflicts: # src/io/outputs/CMakeLists.txt

Fix parquet build

0f56acf

Merge remote-tracking branch 'origin/develop' into fix/parquet-deps

808e264

Fix absl/re2 version conflict in installed AntaresConfig.cmake

bb519fe

Apply the same absl_DIR/re2_DIR pinning to Config.cmake.in so that downstream consumers (e.g. api_client_example) using vcpkg also get the correct OR-Tools co-installed absl/re2 instead of vcpkg's version.

Static linkage for absl & re2 to avoid windows conflicts

6081c84

Fix missing DLL on windows

11cf1cc

[skip ci] ANT-4633 - parquet : shrink license headers

e678bf4

ANT-4633 - parquet : add comments in code + small simplification

ef4625b

ANT-4633 - parquet : code formatting

c748a24

ANT-4633 - parquet : change a lambda to free function

b940a91

[skip ci] ANT-4633 - parquet : clarifications

46d3dee

Merge remote-tracking branch 'remotes/origin/develop' into fix/parque…

1abfc98

…t-deps

ANT-4633 - parquet : renaming

ed7ddd7

ANT-4633 - parquet : handle --parquet cmd line option

da94ac6

ANT-4633 - parquet (modeler side) : bring down the output format to w…

28ca2c2

…here it's needed

ANT-4633 - parquet (modeler side) : remove code not required.

98b2c7f

ANT-4633 - parquet : fix cmake config

d366790

ANT-4633 - parquet : fix a crash

75373ad

ANT-4633 - parquet : code formatting

86f7a2b

ANT-4633 - parquet > solver : define --parquet option + remove useles…

ad7fd91

…s code

ANT-4633 - parquet : remove useless lock guard (we're not in a thread…

b539331

…ing part) + comment for later

ANT-4633 - parquet : make comment more clear

c22240a

ANT-4633 - parquet : fix in modeler + simplification + comments in code

3fceb46

guilpier-code mentioned this pull request May 22, 2026

fix: for review of PR 3586 [ANT-4633] #3657

Merged

6 tasks

Parquet - fix review of PR 3586 : revert changes on submodule vcpkg

d35a11c

guilpier-code requested a review from flomnes May 26, 2026 09:57

JasonMarechal25 and others added 8 commits May 26, 2026 13:10

test: update sha1

c5ff2cc

test: update sha1

be39c27

fix: remove test NR checkout

0882fc6

Merge branch 'develop' into feature/parquet-for-legacy-solver

5bfcd63

fix: e2e tags

131216f

Signed-off-by: GitHub Copilot <copilot@example.com>

fix: throw dedicated errors

8530bd7

Signed-off-by: GitHub Copilot <copilot@example.com>

fix: move OptimisationsSimulationTable in IO::Outputs namespace

b265d5b

Signed-off-by: GitHub Copilot <copilot@example.com>

format

f0825a6

Signed-off-by: GitHub Copilot <copilot@example.com>

JasonMarechal25 requested changes May 27, 2026

View reviewed changes

guilpierre and others added 8 commits May 27, 2026 10:35

Revert "format"

f24a27b

This reverts commit f0825a6.

Revert "fix: move OptimisationsSimulationTable in IO::Outputs namespace"

8a0ead6

This reverts commit b265d5b.

fix: for review of PR 3586 [ANT-4633] (#3657)

89da1ad

Co-authored-by: guilpierre <guillaume.pierre_externe@rte-france.com>

parquet part 3 - parquet for legacy solver #3586 : update submodule t…

947137d

…ests NR and make it point to the right commit

parquet part 3 - parquet for legacy solver #3586 : restore table format

05f32f4

ANT-4633 - parquet (legacy solver) : fix coding error

f835350

Merge remote-tracking branch 'official/develop' into feature/parquet-…

a352a27

…for-legacy-solver # Conflicts: # src/packaging/CMakeLists.txt # src/solver/simulation/economy.cpp # src/tests/resources/Antares_Simulator_Tests_NR

fix: build

ceed9b0

Signed-off-by: GitHub Copilot <copilot@example.com>

JasonMarechal25 approved these changes May 29, 2026

View reviewed changes

JasonMarechal25 and others added 4 commits May 29, 2026 15:54

Merge branch 'develop' into feature/parquet-for-legacy-solver

bfc4c08

Remove table_writer sources & headers from result_writer (#3677)

bf4efd5

[skip ci] Modeler - trial for removing FileWriter : add some doc

d8e8377

refacto: removing useless class FileWriter (#3675)

443ef77

Co-authored-by: guilpierre <guillaume.pierre_externe@rte-france.com>

flomnes merged commit e51c283 into develop Jun 1, 2026
10 of 11 checks passed

flomnes deleted the feature/parquet-for-legacy-solver branch June 1, 2026 08:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

parquet part 3 : parquet for legacy solver#3586

parquet part 3 : parquet for legacy solver#3586
flomnes merged 141 commits into
developfrom
feature/parquet-for-legacy-solver

guilpier-code commented Apr 21, 2026 •

edited by flomnes

Loading

Uh oh!

sonarqubecloud Bot commented May 20, 2026

Uh oh!

guilpier-code commented May 21, 2026 •

edited

Loading

Uh oh!

JasonMarechal25 May 27, 2026

Uh oh!

guilpier-code May 27, 2026 •

edited

Loading

Uh oh!

guilpier-code May 27, 2026

Uh oh!

sonarqubecloud Bot commented Jun 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

guilpier-code commented Apr 21, 2026 • edited by flomnes Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sonarqubecloud Bot commented May 20, 2026

Quality Gate failed

Uh oh!

guilpier-code commented May 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JasonMarechal25 May 27, 2026

Choose a reason for hiding this comment

Uh oh!

guilpier-code May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

guilpier-code May 27, 2026

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud Bot commented Jun 1, 2026

Quality Gate failed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

guilpier-code commented Apr 21, 2026 •

edited by flomnes

Loading

guilpier-code commented May 21, 2026 •

edited

Loading

guilpier-code May 27, 2026 •

edited

Loading