Skip to content

Commit f8bf6a6

Browse files
rberlichdenisbertiniagarcia-gmfJonas-Wessner
authored
New Release Geneva 1.11 (#36)
* Removed surplus boost/optional headers * Replaced boost::filesystem by std::filesystem * Removed boost::any and boost::variant (plus some boost::serialization::variant headers) * Removed some more boost::variant headers * Enforced CMake 3.10 and higher and simplified changing of CMake minimum version (now only in the top level CMakeLists.txt and in the CommonGenevaBuild.cmake-files) * Moved demo processing containers to the courtier main directories * Moved courtier benchmarks to its own benchmark directory * Moved geneva benchmarks to their own benchmark directory * Updated CHANGES file * Fixing a bug in a demo * Adding explicitely the stdc++fs library for gcc < 9.0 using proper Cmake Module (#3) * Add explicit stdc++fs library for gcc version < 9.0 * add explictely the stdc++fs library for gcc version < 9.0 using IdentifySystemParameter module * build: improve setting the stdc++fs library. Replace CMAKE_CXX_STANDARD_LIBRARIES with CMAKE_SHARED_LINKER_FLAGS as used in the rest of the function, since it should also "not be set by project code" according to docu. Also fix indentation and update the copies of IdentifySystemParameters.cmake in the example folders. * 12 mpi consumer (#17) * #12 add build directories to .gitignore * #12 add boilerplate for MPIConsumerT * #12 create initial example for MPIConsumerT * #12 reformat file examples/geneva/16_GMPIConsumer/GMPIConsumer.cpp * #12 add todos and remove unnecessary variables * #12 create and start implement class 'GMPIConsumerT' * #12 make GMPIConsumerT boilerplate syntactically correct and let it inherit from both classes: GBaseConsumerT and GBaseClientT. Override all virtual required methods with empty implementation. * #12 implement all trivial but required methods in GMPIConsumerT * #12 remove deprecated code from GMPIConsumerT.hpp and forward function calls of class 'GMPIConsumerT' to the classes 'GMPIConsumerMasterNodeT' and 'GMPIConsumerWorkerNodeT' * #12 reconstruate basic GAsioConsumerT and GAsioConsumerClientT workflow as a comment * #12 pass worldSize and worldRank to GMPIConsumerWorkerNodeT and GMPIConsumerMasterNodeT for more flexible access to the information about the processes position in cluster * #12 fix: make compilation of GMPIConsumerT.hpp conditional to whether the CMake variable BUILD_WITH_MPI_CONSUMER is set * #12 todos and implementation draft in form of comments * #12 enhance in-code documentation and make number of IO-threads for GMPIConsumerMasterNodeT configurable * #12 send and receive test messages between master node and worker nodes * #12 create new header and source files for helper functions regarding MPI * #12 remove deprecated todo * #12 implement processing of requests in GMPIConsumerWorkerNodet and GMPIConsumerMasterNodeT. So far compilation works but runtime errors occur. * #12 remove deprecated todos * #12 log more descriptive error messages in catch-blocks * #12 fix: resolve runtime bug regarding serialization and not sending correct result from worker node * #12 GMPIConsumerMasterNodeT: run receiver in dedicated thread outside of the thread group which handles single connections * #12 use buffer allocation on heap with smart pointer for incoming requests on GMPIConsumerMasterNodeT * #12 GMPIConsumerWorkerNodeT: use buffer on the heap managed by smart pointer for incoming messages (work items) * #12 add todo * #12 use NODATA variant in case broker has no work items instead of holding the connection until there is a work item availible. The disadvantage of the old approach is that we possibly hold connecitons for too long, which might be problematic if worker nodes crash or get stopped during the connection is alive * #12 use appropriate amount of heap memory for incoming messsages * #12 fix: initialize MPI in multithreaded mode which prevents concurrency error inside of the MPI library that had occurred inside of the GMPIConsumerMasterNodeT * #12 clean up 16_GMPIConsumer/ example * #12 adjust addCLOptions_ * #12 GMPIConsumerT: extract constants for poll intervals and timouts * #12 update in-code documentation * #12 add todos * #12 improve start up terminal messages * #12 GMPIConsumerT: use appropriate logging levels for messages * #12 GMPIConsumerT: add todos * #12 GMPIConsumerT: do not pass this by reference but only required methods as lambdas to fulfil principle of least priviledge * #12 GMPIConsumerT: use GFILE logging level instead of STDOUT logging level for less important non-erroneous messages * #12 GMPIConsumerT: always immediately initialize all variables * #12 GMPIConsumerT: add todos * #12 GMPIConsumerWorkerNodeT: shut down workers properly by using async communication. Use double buffering to retrieve next work item while previous work item is still being processed * #12 GMPIConsumerT.cpp: add todos * #12 GMPIConsumer: feat: add timeout for send operations in GMPIConsumerSessionT; refactor: extract configuration structs for the configuration of GMPIMasterNodeT and GMPIWorkerNodeT * #12 remove already resolved 'TODO's * #12 GMPIConsumerMasterNodeT: calculate recommendation for number of threads in thread pool handling open connections * #12 GMPIConsumerWorkerNodeT: use member variables for mpi request handles, because program logic ensures exclusive access to those resources * #12 GMPIConsumerWorkerNodeT: cancel active asynchronous requests in case an error occures * #12 GMPIConsumerMasterNodeT: cancel pending requests in case of receiving a stop signal from caller on the main thread * #12 GMPIConsumerMasterNodeT: remove unnecessary call to boost::asio::io_service::work * #12 GMPIConsumerT: only specify logging level and let the user specify whether to log to file or to console by setting logging target * #12 GMPIConsumer.cpp: make the target of log messages configurable through a command line argument. * #12 GMPIConsumerT: make all constants configurable and update in-code documentation * #12 incorporate GMPIConsumerT into Go2 * #12 GMPIConsumerMasterNodeT: move the log message which informs the user that the master node has started up back into the constructor, because now we already now the final size of threads in the constructor. This is because we no longer need to act on the command line options in GMPIConsumerWorkerNodeT and GMPIConsumerMasterNodeT, because that is done by the wrapping GMPIConsumerT * #12 examples: link MPI into examples if those examples use the Go2 class and the current build of geneva includes the MPI consumer * #12 docs: update READMEs: reference example 16_GMPIConsumer/ in the examples XX_DirectXX/. Reference examples using the Go2 class in example 16_GMPIConsumer/ * #12 link MPI to all tests, benchmarks and examples in case the current build of geneva includes the mpi consumer * #12 use singleton access to GMPIConsumerT in Go2.cpp in order to not call MPI_Init or MPI_Finalize more than once * #12 uncomment instantiation of GMPIConsumerT * #12 replace boost::asio::detail::thread_group with Gem::Common::GThreadPool * #12 remove already resolved 'TODO' * #12 create script to run command multiple times * #12 make output of scripts/repeatedRun.sh prettier * #12 fix non-deterministic bug when calling glogger from GConsumerSessionT * #12 clarify comment * #12 include string in file that uses string although string * #12 create docker setup * #12 also install 'which' in build-image * #12 use multithreaded building for docker images * #12 make it possible to set receivePollIntervalUSec for GMPIConsumerMasterNodeT to 0; add todo * #12 link MPI in case of building with MPI to all examples and tests that require it * #12 create common base image 'GenevaPrerequisites.Dockerfile' for geneva docker images * #12 set up a docker image for the completed geneva build * #12 add todo * #12 fix various bugs in the docker setup; change file names; set up interactive shell in geneva_builder for developers * #12 improve output of geneva/scripts/Testing/repeatedRun.sh * #12 calling docker-compose down when running 'run-geneva-builder-interactive' * #12 improve output of geneva/scripts/Testing/repeatedRun.sh * #12 write readme for docker and docker-compose setup * #12 fix local slurm setup, write information regarding this to the README.md * #12 add todos * #12 use std primitive types instead of boost primitive types * #12 use std::chrono to measure timeouts and use std::this_thread::sleep_for for waiting * #12 add boilerplate for example 17_GMPISubClients * #12 make MPI subclients possible - example 17_GMPISubClients still contains bugs * #12 do not finalize mpi if it has never been initialized * #12 fix example 17_GMPISubClients * #12 fix sub-communicator grouping in example 17_GMPIEvaluatedIndividual * #12 use one cleanup thread to wait for completion of sessions in order to more quickly enable threads in thread-pool to be ready for new sessions * #12 update in-code documentation for GMPIConsumerT * #12 correct typos in comments * #12 add comment * #12 correct return value of initializeMPI method * #12 increase maximum message size and add check if maximum message size is sufficient for runtime requirements * #13 move docker setup to a seperate branch `13-docker-and-docker-compse-setup` * #12 fix external iteration counter exception in benchmarks/geneva/GParallelisationOverhead * #12 start implementation of performance benchmark for GAsioConsumerT and GMPIConsumerT * #12 implement benchmark for asio-consumer and mpi-consumer. benchmarking asio-consumer is not impelmented yet * #12 implement asio benchmark * #12 draw multiple curves in one plot with different colors * #12 plot legends * #12 also plot graphs with nClients on x axis * #12 set y-axis limits correctly in graph with subplots * #12 extract method to set axis limits to GBasePlotter and use it also for Graph2ED in the GAsioMPIBenchmark * #15 extract GAsioMPIBenchmark to branch 15 * #12 add init method to GBaseConsumerT * #12 add todo in example 17 * #12 remove already resolved 'TODO', add new 'TODO' * #12 remove already resolved 'TODO', add new 'TODO' * #12 close sessions correctly on client side * #12 fix: use m_handlerThreadPool to schedule tasks to pool instead of asio::post * #12 remove deprecated todos * #12 show worker node shutdown message also in release mode, because that is an important information which the user might want to see and use to adjust the workers timeout-configuration * #12 try to synchronize all proceses before starting consumer in order to not trigger timeouts on worker side if master node is not up and running yet * #12 change unit of worker poll timeout to seconds, because any amount below one second is not useful and configuring it this way is more convenient * #12 update in-code documentation for GMPIConsumerT * #12 update examples/geneva/16_GMPIConsumer/README.md * #12 make synchronization configurable * #12 leave synchronization open to user, because synchronizing on startup causes issues in some edge cases like sub-client optimization * #12 change all clocks to std::chrono::steady_clock in GMPIConsumerT * #12 improve code readability by making the time since a session is finished a property of the session class itself * #12 correct two mistakes in CMD messages which must have happened when copy pasting * #12 fix bug in prepare build script which occured because of wrong variable name * #12 wrap access to GMPIConsumer in standard consumers in compile time macro ifdef to make sure it is only compiled if building with MPI-Consumer * Added a thank you message for Jonas' contribution Co-authored-by: Jonas Weßner <jonas.wessner@gmx.de> Co-authored-by: rberlich <r.berlich@gemfony.eu> * 16 gmpisubclientoptimizer (#18) * #12 add build directories to .gitignore * #12 add boilerplate for MPIConsumerT * #12 create initial example for MPIConsumerT * #12 reformat file examples/geneva/16_GMPIConsumer/GMPIConsumer.cpp * #12 add todos and remove unnecessary variables * #12 create and start implement class 'GMPIConsumerT' * #12 make GMPIConsumerT boilerplate syntactically correct and let it inherit from both classes: GBaseConsumerT and GBaseClientT. Override all virtual required methods with empty implementation. * #12 implement all trivial but required methods in GMPIConsumerT * #12 remove deprecated code from GMPIConsumerT.hpp and forward function calls of class 'GMPIConsumerT' to the classes 'GMPIConsumerMasterNodeT' and 'GMPIConsumerWorkerNodeT' * #12 reconstruate basic GAsioConsumerT and GAsioConsumerClientT workflow as a comment * #12 pass worldSize and worldRank to GMPIConsumerWorkerNodeT and GMPIConsumerMasterNodeT for more flexible access to the information about the processes position in cluster * #12 fix: make compilation of GMPIConsumerT.hpp conditional to whether the CMake variable BUILD_WITH_MPI_CONSUMER is set * #12 todos and implementation draft in form of comments * #12 enhance in-code documentation and make number of IO-threads for GMPIConsumerMasterNodeT configurable * #12 send and receive test messages between master node and worker nodes * #12 create new header and source files for helper functions regarding MPI * #12 remove deprecated todo * #12 implement processing of requests in GMPIConsumerWorkerNodet and GMPIConsumerMasterNodeT. So far compilation works but runtime errors occur. * #12 remove deprecated todos * #12 log more descriptive error messages in catch-blocks * #12 fix: resolve runtime bug regarding serialization and not sending correct result from worker node * #12 GMPIConsumerMasterNodeT: run receiver in dedicated thread outside of the thread group which handles single connections * #12 use buffer allocation on heap with smart pointer for incoming requests on GMPIConsumerMasterNodeT * #12 GMPIConsumerWorkerNodeT: use buffer on the heap managed by smart pointer for incoming messages (work items) * #12 add todo * #12 use NODATA variant in case broker has no work items instead of holding the connection until there is a work item availible. The disadvantage of the old approach is that we possibly hold connecitons for too long, which might be problematic if worker nodes crash or get stopped during the connection is alive * #12 use appropriate amount of heap memory for incoming messsages * #12 fix: initialize MPI in multithreaded mode which prevents concurrency error inside of the MPI library that had occurred inside of the GMPIConsumerMasterNodeT * #12 clean up 16_GMPIConsumer/ example * #12 adjust addCLOptions_ * #12 GMPIConsumerT: extract constants for poll intervals and timouts * #12 update in-code documentation * #12 add todos * #12 improve start up terminal messages * #12 GMPIConsumerT: use appropriate logging levels for messages * #12 GMPIConsumerT: add todos * #12 GMPIConsumerT: do not pass this by reference but only required methods as lambdas to fulfil principle of least priviledge * #12 GMPIConsumerT: use GFILE logging level instead of STDOUT logging level for less important non-erroneous messages * #12 GMPIConsumerT: always immediately initialize all variables * #12 GMPIConsumerT: add todos * #12 GMPIConsumerWorkerNodeT: shut down workers properly by using async communication. Use double buffering to retrieve next work item while previous work item is still being processed * #12 GMPIConsumerT.cpp: add todos * #12 GMPIConsumer: feat: add timeout for send operations in GMPIConsumerSessionT; refactor: extract configuration structs for the configuration of GMPIMasterNodeT and GMPIWorkerNodeT * #12 remove already resolved 'TODO's * #12 GMPIConsumerMasterNodeT: calculate recommendation for number of threads in thread pool handling open connections * #12 GMPIConsumerWorkerNodeT: use member variables for mpi request handles, because program logic ensures exclusive access to those resources * #12 GMPIConsumerWorkerNodeT: cancel active asynchronous requests in case an error occures * #12 GMPIConsumerMasterNodeT: cancel pending requests in case of receiving a stop signal from caller on the main thread * #12 GMPIConsumerMasterNodeT: remove unnecessary call to boost::asio::io_service::work * #12 GMPIConsumerT: only specify logging level and let the user specify whether to log to file or to console by setting logging target * #12 GMPIConsumer.cpp: make the target of log messages configurable through a command line argument. * #12 GMPIConsumerT: make all constants configurable and update in-code documentation * #12 incorporate GMPIConsumerT into Go2 * #12 GMPIConsumerMasterNodeT: move the log message which informs the user that the master node has started up back into the constructor, because now we already now the final size of threads in the constructor. This is because we no longer need to act on the command line options in GMPIConsumerWorkerNodeT and GMPIConsumerMasterNodeT, because that is done by the wrapping GMPIConsumerT * #12 examples: link MPI into examples if those examples use the Go2 class and the current build of geneva includes the MPI consumer * #12 docs: update READMEs: reference example 16_GMPIConsumer/ in the examples XX_DirectXX/. Reference examples using the Go2 class in example 16_GMPIConsumer/ * #12 link MPI to all tests, benchmarks and examples in case the current build of geneva includes the mpi consumer * #12 use singleton access to GMPIConsumerT in Go2.cpp in order to not call MPI_Init or MPI_Finalize more than once * #12 uncomment instantiation of GMPIConsumerT * #12 replace boost::asio::detail::thread_group with Gem::Common::GThreadPool * #12 remove already resolved 'TODO' * #12 create script to run command multiple times * #12 make output of scripts/repeatedRun.sh prettier * #12 fix non-deterministic bug when calling glogger from GConsumerSessionT * #12 clarify comment * #12 include string in file that uses string although string * #12 create docker setup * #12 also install 'which' in build-image * #12 use multithreaded building for docker images * #12 make it possible to set receivePollIntervalUSec for GMPIConsumerMasterNodeT to 0; add todo * #12 link MPI in case of building with MPI to all examples and tests that require it * #12 create common base image 'GenevaPrerequisites.Dockerfile' for geneva docker images * #12 set up a docker image for the completed geneva build * #12 add todo * #12 fix various bugs in the docker setup; change file names; set up interactive shell in geneva_builder for developers * #12 improve output of geneva/scripts/Testing/repeatedRun.sh * #12 calling docker-compose down when running 'run-geneva-builder-interactive' * #12 improve output of geneva/scripts/Testing/repeatedRun.sh * #12 write readme for docker and docker-compose setup * #12 fix local slurm setup, write information regarding this to the README.md * #12 add todos * #12 use std primitive types instead of boost primitive types * #12 use std::chrono to measure timeouts and use std::this_thread::sleep_for for waiting * #12 add boilerplate for example 17_GMPISubClients * #12 make MPI subclients possible - example 17_GMPISubClients still contains bugs * #12 do not finalize mpi if it has never been initialized * #12 fix example 17_GMPISubClients * #12 fix sub-communicator grouping in example 17_GMPIEvaluatedIndividual * #12 use one cleanup thread to wait for completion of sessions in order to more quickly enable threads in thread-pool to be ready for new sessions * #12 update in-code documentation for GMPIConsumerT * #12 correct typos in comments * #12 add comment * #12 correct return value of initializeMPI method * #12 increase maximum message size and add check if maximum message size is sufficient for runtime requirements * #13 move docker setup to a seperate branch `13-docker-and-docker-compse-setup` * #12 fix external iteration counter exception in benchmarks/geneva/GParallelisationOverhead * #12 start implementation of performance benchmark for GAsioConsumerT and GMPIConsumerT * #12 implement benchmark for asio-consumer and mpi-consumer. benchmarking asio-consumer is not impelmented yet * #12 implement asio benchmark * #12 draw multiple curves in one plot with different colors * #12 plot legends * #12 also plot graphs with nClients on x axis * #12 set y-axis limits correctly in graph with subplots * #12 extract method to set axis limits to GBasePlotter and use it also for Graph2ED in the GAsioMPIBenchmark * #15 extract GAsioMPIBenchmark to branch 15 * #12 add init method to GBaseConsumerT * #12 add todo in example 17 * #12 remove already resolved 'TODO', add new 'TODO' * #12 remove already resolved 'TODO', add new 'TODO' * #12 close sessions correctly on client side * #12 fix: use m_handlerThreadPool to schedule tasks to pool instead of asio::post * #12 remove deprecated todos * #16 implement first version of GMPISubClientOptimizer * #16 set default callback which throws an exception indicating that setting callback has been forgotten * #16 create GMPISubClientIndividual * #16 implement GMPISubClient example * #16 rename and adjust config file for GMPISubClientOptimizer example * #16 fix bug related to passing nullptr to scatter and gather as root process * #16 use async calls to implement timeout * #16 update readme * #12 show worker node shutdown message also in release mode, because that is an important information which the user might want to see and use to adjust the workers timeout-configuration * #16 make retriving client mode possible * #16 do not initialize MPI if using a sub communicator * #16 delete deprecated unused file * #16 delete deprecated unused file * #16 make sub-client job part of the individual * #16 add comment * #16 move GMPISubClientIndividual from geneva-individuals to geneva, because it is not a complete individual but an abstract base class for user defined individuals * #16 make rank 0 of each subgroup be the Geneva client and the other ranks be the sub-client. Before this commit the rank n-1 of each subgroup was the geneva client, which is not intuitive * #12 try to synchronize all proceses before starting consumer in order to not trigger timeouts on worker side if master node is not up and running yet * #12 change unit of worker poll timeout to seconds, because any amount below one second is not useful and configuring it this way is more convenient * #12 update in-code documentation for GMPIConsumerT * #12 update examples/geneva/16_GMPIConsumer/README.md * #16 extend documenation for examples/geneva/17_GMPISubClients/ * #16 correct typo in documenation for examples/geneva/17_GMPISubClients/ * #16 change std::cout to glogger * #16 update examples/geneva/README * #12 make synchronization configurable * #16 correct typo * #16 improve code readability * #12 leave synchronization open to user, because synchronizing on startup causes issues in some edge cases like sub-client optimization * #16 fix: parse GMPISubClientOptimizer configuration options from file * #16 rewrite example 17_GMPISubClients to actually parallelize the work and measure the total wall time * #16 rename class and associated files * #16 change configuration for geneva example 17 * #16 fix: use shutdown signal instead of timeout for sub-client termination * #12 change all clocks to std::chrono::steady_clock in GMPIConsumerT * #12 improve code readability by making the time since a session is finished a property of the session class itself * #16 reformat file GMPISubClientIndividual.cpp * #16 reformat file GMPISubClientIndividual.hpp * #16 fix: make client mode accessible from GMPISubClientIndividual and use this to correct error when calling getClientStatus() in client mode, because request object was not set * #12 correct two mistakes in CMD messages which must have happened when copy pasting * #12 fix bug in prepare build script which occured because of wrong variable name * #12 wrap access to GMPIConsumer in standard consumers in compile time macro ifdef to make sure it is only compiled if building with MPI-Consumer Co-authored-by: Jonas Weßner <jonas.wessner@gmx.de> * 2023 01 06 compilation fix develop (#20) * #12 add build directories to .gitignore * #12 add boilerplate for MPIConsumerT * #12 create initial example for MPIConsumerT * #12 reformat file examples/geneva/16_GMPIConsumer/GMPIConsumer.cpp * #12 add todos and remove unnecessary variables * #12 create and start implement class 'GMPIConsumerT' * #12 make GMPIConsumerT boilerplate syntactically correct and let it inherit from both classes: GBaseConsumerT and GBaseClientT. Override all virtual required methods with empty implementation. * #12 implement all trivial but required methods in GMPIConsumerT * #12 remove deprecated code from GMPIConsumerT.hpp and forward function calls of class 'GMPIConsumerT' to the classes 'GMPIConsumerMasterNodeT' and 'GMPIConsumerWorkerNodeT' * #12 reconstruate basic GAsioConsumerT and GAsioConsumerClientT workflow as a comment * #12 pass worldSize and worldRank to GMPIConsumerWorkerNodeT and GMPIConsumerMasterNodeT for more flexible access to the information about the processes position in cluster * #12 fix: make compilation of GMPIConsumerT.hpp conditional to whether the CMake variable BUILD_WITH_MPI_CONSUMER is set * #12 todos and implementation draft in form of comments * #12 enhance in-code documentation and make number of IO-threads for GMPIConsumerMasterNodeT configurable * #12 send and receive test messages between master node and worker nodes * #12 create new header and source files for helper functions regarding MPI * #12 remove deprecated todo * #12 implement processing of requests in GMPIConsumerWorkerNodet and GMPIConsumerMasterNodeT. So far compilation works but runtime errors occur. * #12 remove deprecated todos * #12 log more descriptive error messages in catch-blocks * #12 fix: resolve runtime bug regarding serialization and not sending correct result from worker node * #12 GMPIConsumerMasterNodeT: run receiver in dedicated thread outside of the thread group which handles single connections * #12 use buffer allocation on heap with smart pointer for incoming requests on GMPIConsumerMasterNodeT * #12 GMPIConsumerWorkerNodeT: use buffer on the heap managed by smart pointer for incoming messages (work items) * #12 add todo * #12 use NODATA variant in case broker has no work items instead of holding the connection until there is a work item availible. The disadvantage of the old approach is that we possibly hold connecitons for too long, which might be problematic if worker nodes crash or get stopped during the connection is alive * #12 use appropriate amount of heap memory for incoming messsages * #12 fix: initialize MPI in multithreaded mode which prevents concurrency error inside of the MPI library that had occurred inside of the GMPIConsumerMasterNodeT * #12 clean up 16_GMPIConsumer/ example * #12 adjust addCLOptions_ * #12 GMPIConsumerT: extract constants for poll intervals and timouts * #12 update in-code documentation * #12 add todos * #12 improve start up terminal messages * #12 GMPIConsumerT: use appropriate logging levels for messages * #12 GMPIConsumerT: add todos * #12 GMPIConsumerT: do not pass this by reference but only required methods as lambdas to fulfil principle of least priviledge * #12 GMPIConsumerT: use GFILE logging level instead of STDOUT logging level for less important non-erroneous messages * #12 GMPIConsumerT: always immediately initialize all variables * #12 GMPIConsumerT: add todos * #12 GMPIConsumerWorkerNodeT: shut down workers properly by using async communication. Use double buffering to retrieve next work item while previous work item is still being processed * #12 GMPIConsumerT.cpp: add todos * #12 GMPIConsumer: feat: add timeout for send operations in GMPIConsumerSessionT; refactor: extract configuration structs for the configuration of GMPIMasterNodeT and GMPIWorkerNodeT * #12 remove already resolved 'TODO's * #12 GMPIConsumerMasterNodeT: calculate recommendation for number of threads in thread pool handling open connections * #12 GMPIConsumerWorkerNodeT: use member variables for mpi request handles, because program logic ensures exclusive access to those resources * #12 GMPIConsumerWorkerNodeT: cancel active asynchronous requests in case an error occures * #12 GMPIConsumerMasterNodeT: cancel pending requests in case of receiving a stop signal from caller on the main thread * #12 GMPIConsumerMasterNodeT: remove unnecessary call to boost::asio::io_service::work * #12 GMPIConsumerT: only specify logging level and let the user specify whether to log to file or to console by setting logging target * #12 GMPIConsumer.cpp: make the target of log messages configurable through a command line argument. * #12 GMPIConsumerT: make all constants configurable and update in-code documentation * #12 incorporate GMPIConsumerT into Go2 * #12 GMPIConsumerMasterNodeT: move the log message which informs the user that the master node has started up back into the constructor, because now we already now the final size of threads in the constructor. This is because we no longer need to act on the command line options in GMPIConsumerWorkerNodeT and GMPIConsumerMasterNodeT, because that is done by the wrapping GMPIConsumerT * #12 examples: link MPI into examples if those examples use the Go2 class and the current build of geneva includes the MPI consumer * #12 docs: update READMEs: reference example 16_GMPIConsumer/ in the examples XX_DirectXX/. Reference examples using the Go2 class in example 16_GMPIConsumer/ * #12 link MPI to all tests, benchmarks and examples in case the current build of geneva includes the mpi consumer * #12 use singleton access to GMPIConsumerT in Go2.cpp in order to not call MPI_Init or MPI_Finalize more than once * #12 uncomment instantiation of GMPIConsumerT * #12 replace boost::asio::detail::thread_group with Gem::Common::GThreadPool * #12 remove already resolved 'TODO' * #12 create script to run command multiple times * #12 make output of scripts/repeatedRun.sh prettier * #12 fix non-deterministic bug when calling glogger from GConsumerSessionT * #12 clarify comment * #12 include string in file that uses string although string * #12 create docker setup * #12 also install 'which' in build-image * #12 use multithreaded building for docker images * #12 make it possible to set receivePollIntervalUSec for GMPIConsumerMasterNodeT to 0; add todo * #12 link MPI in case of building with MPI to all examples and tests that require it * #12 create common base image 'GenevaPrerequisites.Dockerfile' for geneva docker images * #12 set up a docker image for the completed geneva build * #12 add todo * #12 fix various bugs in the docker setup; change file names; set up interactive shell in geneva_builder for developers * #12 improve output of geneva/scripts/Testing/repeatedRun.sh * #12 calling docker-compose down when running 'run-geneva-builder-interactive' * #12 improve output of geneva/scripts/Testing/repeatedRun.sh * #12 write readme for docker and docker-compose setup * #12 fix local slurm setup, write information regarding this to the README.md * #12 add todos * #12 use std primitive types instead of boost primitive types * #12 use std::chrono to measure timeouts and use std::this_thread::sleep_for for waiting * #12 add boilerplate for example 17_GMPISubClients * #12 make MPI subclients possible - example 17_GMPISubClients still contains bugs * #12 do not finalize mpi if it has never been initialized * #12 fix example 17_GMPISubClients * #12 fix sub-communicator grouping in example 17_GMPIEvaluatedIndividual * #12 use one cleanup thread to wait for completion of sessions in order to more quickly enable threads in thread-pool to be ready for new sessions * #12 update in-code documentation for GMPIConsumerT * #12 correct typos in comments * #12 add comment * #12 correct return value of initializeMPI method * #12 increase maximum message size and add check if maximum message size is sufficient for runtime requirements * #13 move docker setup to a seperate branch `13-docker-and-docker-compse-setup` * #12 fix external iteration counter exception in benchmarks/geneva/GParallelisationOverhead * #12 start implementation of performance benchmark for GAsioConsumerT and GMPIConsumerT * #12 implement benchmark for asio-consumer and mpi-consumer. benchmarking asio-consumer is not impelmented yet * #12 implement asio benchmark * #12 draw multiple curves in one plot with different colors * #12 plot legends * #12 also plot graphs with nClients on x axis * #12 set y-axis limits correctly in graph with subplots * #12 extract method to set axis limits to GBasePlotter and use it also for Graph2ED in the GAsioMPIBenchmark * #15 extract GAsioMPIBenchmark to branch 15 * #12 add init method to GBaseConsumerT * #12 add todo in example 17 * #12 remove already resolved 'TODO', add new 'TODO' * #12 remove already resolved 'TODO', add new 'TODO' * #12 close sessions correctly on client side * #12 fix: use m_handlerThreadPool to schedule tasks to pool instead of asio::post * #12 remove deprecated todos * #16 implement first version of GMPISubClientOptimizer * #16 set default callback which throws an exception indicating that setting callback has been forgotten * #16 create GMPISubClientIndividual * #16 implement GMPISubClient example * #16 rename and adjust config file for GMPISubClientOptimizer example * #16 fix bug related to passing nullptr to scatter and gather as root process * #16 use async calls to implement timeout * #16 update readme * #12 show worker node shutdown message also in release mode, because that is an important information which the user might want to see and use to adjust the workers timeout-configuration * #16 make retriving client mode possible * #16 do not initialize MPI if using a sub communicator * #16 delete deprecated unused file * #16 delete deprecated unused file * #16 make sub-client job part of the individual * #16 add comment * #16 move GMPISubClientIndividual from geneva-individuals to geneva, because it is not a complete individual but an abstract base class for user defined individuals * #16 make rank 0 of each subgroup be the Geneva client and the other ranks be the sub-client. Before this commit the rank n-1 of each subgroup was the geneva client, which is not intuitive * #12 try to synchronize all proceses before starting consumer in order to not trigger timeouts on worker side if master node is not up and running yet * #12 change unit of worker poll timeout to seconds, because any amount below one second is not useful and configuring it this way is more convenient * #12 update in-code documentation for GMPIConsumerT * #12 update examples/geneva/16_GMPIConsumer/README.md * #16 extend documenation for examples/geneva/17_GMPISubClients/ * #16 correct typo in documenation for examples/geneva/17_GMPISubClients/ * #16 change std::cout to glogger * #16 update examples/geneva/README * #12 make synchronization configurable * #16 correct typo * #16 improve code readability * #12 leave synchronization open to user, because synchronizing on startup causes issues in some edge cases like sub-client optimization * #16 fix: parse GMPISubClientOptimizer configuration options from file * #16 rewrite example 17_GMPISubClients to actually parallelize the work and measure the total wall time * #16 rename class and associated files * #16 change configuration for geneva example 17 * #16 fix: use shutdown signal instead of timeout for sub-client termination * #12 change all clocks to std::chrono::steady_clock in GMPIConsumerT * #12 improve code readability by making the time since a session is finished a property of the session class itself * #16 reformat file GMPISubClientIndividual.cpp * #16 reformat file GMPISubClientIndividual.hpp * #16 fix: make client mode accessible from GMPISubClientIndividual and use this to correct error when calling getClientStatus() in client mode, because request object was not set * #12 correct two mistakes in CMD messages which must have happened when copy pasting * #12 fix bug in prepare build script which occured because of wrong variable name * #12 wrap access to GMPIConsumer in standard consumers in compile time macro ifdef to make sure it is only compiled if building with MPI-Consumer * Added a thank you message for Jonas' contribution * Fixing compiler problems * Fixing compiler problems * Update of CHANGES to reflect new minimal cmake version * Fixing compiler problems Co-authored-by: Jonas Weßner <jonas.wessner@gmx.de> Co-authored-by: Jonas-Wessner <81162699+Jonas-Wessner@users.noreply.github.com> * #21 add 'cpp' file extension in Cmake files, which is a requirement after raising the Cmake version in #20 (#22) Co-authored-by: Jonas Weßner <jonas.wessner@gmx.de> * Added <optional> to some files to fix g++ 12.2 compilation problem with MPI consumer and examples (#23) * Mpi consumer close all requests (#24) * #0 make sure to always free all open MPI requests either by calling MPI_Cancel and MPI_Request_free or by calling MPI_Test until the status is completed * #0 cancel MPI_IBarrier request if timeout triggered * exclude all Clion build directories from git * decrease CMAKE_MINIMUM_REQUIRED to 3.13 because this works as well and it is the version available at GSI's virgo cluster * make it possible to set path to MPI installation through config file * GMPIConsumerT.hpp: only emit finalized twice warning if it has been initialized. * Use MPI_Request_free instead of MPI_Test to free request that is guaranteed to be completed * #0 update logging output --------- Co-authored-by: Jonas Weßner <jonas.wessner@gmx.de> * 25 no mpi cancel for isend (#26) * #25 begin restructuring to remove MPI_Cancel for ISend calls * #25 continue restructure GMPIConsumerT to remove MPI_Cancel for MPI_ISend * #25 shut down clients when receiving stop requests * #25 pass correct arguments to instantiate nodes * #25 add debug output * #25 use MPI test instead of MPI Wait to await receive on worker * #25 fix bug regarding uninitialized serialization mode in mpi sessions * #25 change const to constexpr where possible * #25 comment out uncecessary and deprecated code * #25 simplify configuration struct and store const references to it in the worker and the master node classes to avoid copying and avoid unintentional modifications; determine the hardware recommendation for handler threads inside config struct constructor * #25 fix deadlock at shutdown; do not await another response. The response has been received already after sending the request * #25 clean up code: remove prints, remove commented out code * #25 use MPI wait instead of test if no meaningful work to do on worker node * #25 remove timeout in synchronization method * #25 remove unused code segments * #25 remove unused CLI args * #25 clean up code and update documentation * #25 use MPI_Wait intead of MPI_Request_free * #25 capture shared pointer by value in lambda to make sure its does not go out of scope when the lambda function is executed (race condition) * #25 remove unused members * #25 remove debug statements * Added CUDA example, restarted work on Geneva (#34) * #12 add build directories to .gitignore * #12 add boilerplate for MPIConsumerT * #12 create initial example for MPIConsumerT * #12 reformat file examples/geneva/16_GMPIConsumer/GMPIConsumer.cpp * #12 add todos and remove unnecessary variables * #12 create and start implement class 'GMPIConsumerT' * #12 make GMPIConsumerT boilerplate syntactically correct and let it inherit from both classes: GBaseConsumerT and GBaseClientT. Override all virtual required methods with empty implementation. * #12 implement all trivial but required methods in GMPIConsumerT * #12 remove deprecated code from GMPIConsumerT.hpp and forward function calls of class 'GMPIConsumerT' to the classes 'GMPIConsumerMasterNodeT' and 'GMPIConsumerWorkerNodeT' * #12 reconstruate basic GAsioConsumerT and GAsioConsumerClientT workflow as a comment * #12 pass worldSize and worldRank to GMPIConsumerWorkerNodeT and GMPIConsumerMasterNodeT for more flexible access to the information about the processes position in cluster * #12 fix: make compilation of GMPIConsumerT.hpp conditional to whether the CMake variable BUILD_WITH_MPI_CONSUMER is set * #12 todos and implementation draft in form of comments * #12 enhance in-code documentation and make number of IO-threads for GMPIConsumerMasterNodeT configurable * #12 send and receive test messages between master node and worker nodes * #12 create new header and source files for helper functions regarding MPI * #12 remove deprecated todo * #12 implement processing of requests in GMPIConsumerWorkerNodet and GMPIConsumerMasterNodeT. So far compilation works but runtime errors occur. * #12 remove deprecated todos * #12 log more descriptive error messages in catch-blocks * #12 fix: resolve runtime bug regarding serialization and not sending correct result from worker node * #12 GMPIConsumerMasterNodeT: run receiver in dedicated thread outside of the thread group which handles single connections * #12 use buffer allocation on heap with smart pointer for incoming requests on GMPIConsumerMasterNodeT * #12 GMPIConsumerWorkerNodeT: use buffer on the heap managed by smart pointer for incoming messages (work items) * #12 add todo * #12 use NODATA variant in case broker has no work items instead of holding the connection until there is a work item availible. The disadvantage of the old approach is that we possibly hold connecitons for too long, which might be problematic if worker nodes crash or get stopped during the connection is alive * #12 use appropriate amount of heap memory for incoming messsages * #12 fix: initialize MPI in multithreaded mode which prevents concurrency error inside of the MPI library that had occurred inside of the GMPIConsumerMasterNodeT * #12 clean up 16_GMPIConsumer/ example * #12 adjust addCLOptions_ * #12 GMPIConsumerT: extract constants for poll intervals and timouts * #12 update in-code documentation * #12 add todos * #12 improve start up terminal messages * #12 GMPIConsumerT: use appropriate logging levels for messages * #12 GMPIConsumerT: add todos * #12 GMPIConsumerT: do not pass this by reference but only required methods as lambdas to fulfil principle of least priviledge * #12 GMPIConsumerT: use GFILE logging level instead of STDOUT logging level for less important non-erroneous messages * #12 GMPIConsumerT: always immediately initialize all variables * #12 GMPIConsumerT: add todos * #12 GMPIConsumerWorkerNodeT: shut down workers properly by using async communication. Use double buffering to retrieve next work item while previous work item is still being processed * #12 GMPIConsumerT.cpp: add todos * #12 GMPIConsumer: feat: add timeout for send operations in GMPIConsumerSessionT; refactor: extract configuration structs for the configuration of GMPIMasterNodeT and GMPIWorkerNodeT * #12 remove already resolved 'TODO's * #12 GMPIConsumerMasterNodeT: calculate recommendation for number of threads in thread pool handling open connections * #12 GMPIConsumerWorkerNodeT: use member variables for mpi request handles, because program logic ensures exclusive access to those resources * #12 GMPIConsumerWorkerNodeT: cancel active asynchronous requests in case an error occures * #12 GMPIConsumerMasterNodeT: cancel pending requests in case of receiving a stop signal from caller on the main thread * #12 GMPIConsumerMasterNodeT: remove unnecessary call to boost::asio::io_service::work * #12 GMPIConsumerT: only specify logging level and let the user specify whether to log to file or to console by setting logging target * #12 GMPIConsumer.cpp: make the target of log messages configurable through a command line argument. * #12 GMPIConsumerT: make all constants configurable and update in-code documentation * #12 incorporate GMPIConsumerT into Go2 * #12 GMPIConsumerMasterNodeT: move the log message which informs the user that the master node has started up back into the constructor, because now we already now the final size of threads in the constructor. This is because we no longer need to act on the command line options in GMPIConsumerWorkerNodeT and GMPIConsumerMasterNodeT, because that is done by the wrapping GMPIConsumerT * #12 examples: link MPI into examples if those examples use the Go2 class and the current build of geneva includes the MPI consumer * #12 docs: update READMEs: reference example 16_GMPIConsumer/ in the examples XX_DirectXX/. Reference examples using the Go2 class in example 16_GMPIConsumer/ * #12 link MPI to all tests, benchmarks and examples in case the current build of geneva includes the mpi consumer * #12 use singleton access to GMPIConsumerT in Go2.cpp in order to not call MPI_Init or MPI_Finalize more than once * #12 uncomment instantiation of GMPIConsumerT * #12 replace boost::asio::detail::thread_group with Gem::Common::GThreadPool * #12 remove already resolved 'TODO' * #12 create script to run command multiple times * #12 make output of scripts/repeatedRun.sh prettier * #12 fix non-deterministic bug when calling glogger from GConsumerSessionT * #12 clarify comment * #12 include string in file that uses string although string * #12 create docker setup * #12 also install 'which' in build-image * #12 use multithreaded building for docker images * #12 make it possible to set receivePollIntervalUSec for GMPIConsumerMasterNodeT to 0; add todo * #12 link MPI in case of building with MPI to all examples and tests that require it * #12 create common base image 'GenevaPrerequisites.Dockerfile' for geneva docker images * #12 set up a docker image for the completed geneva build * #12 add todo * #12 fix various bugs in the docker setup; change file names; set up interactive shell in geneva_builder for developers * #12 improve output of geneva/scripts/Testing/repeatedRun.sh * #12 calling docker-compose down when running 'run-geneva-builder-interactive' * #12 improve output of geneva/scripts/Testing/repeatedRun.sh * #12 write readme for docker and docker-compose setup * #12 fix local slurm setup, write information regarding this to the README.md * #12 add todos * #12 use std primitive types instead of boost primitive types * #12 use std::chrono to measure timeouts and use std::this_thread::sleep_for for waiting * #12 add boilerplate for example 17_GMPISubClients * #12 make MPI subclients possible - example 17_GMPISubClients still contains bugs * #12 do not finalize mpi if it has never been initialized * #12 fix example 17_GMPISubClients * #12 fix sub-communicator grouping in example 17_GMPIEvaluatedIndividual * #12 use one cleanup thread to wait for completion of sessions in order to more quickly enable threads in thread-pool to be ready for new sessions * #12 update in-code documentation for GMPIConsumerT * #12 correct typos in comments * #12 add comment * #12 correct return value of initializeMPI method * #12 increase maximum message size and add check if maximum message size is sufficient for runtime requirements * #13 move docker setup to a seperate branch `13-docker-and-docker-compse-setup` * #12 fix external iteration counter exception in benchmarks/geneva/GParallelisationOverhead * #12 start implementation of performance benchmark for GAsioConsumerT and GMPIConsumerT * #12 implement benchmark for asio-consumer and mpi-consumer. benchmarking asio-consumer is not impelmented yet * #12 implement asio benchmark * #12 draw multiple curves in one plot with different colors * #12 plot legends * #12 also plot graphs with nClients on x axis * #12 set y-axis limits correctly in graph with subplots * #12 extract method to set axis limits to GBasePlotter and use it also for Graph2ED in the GAsioMPIBenchmark * #15 extract GAsioMPIBenchmark to branch 15 * #12 add init method to GBaseConsumerT * #12 add todo in example 17 * #12 remove already resolved 'TODO', add new 'TODO' * #12 remove already resolved 'TODO', add new 'TODO' * #12 close sessions correctly on client side * #12 fix: use m_handlerThreadPool to schedule tasks to pool instead of asio::post * #12 remove deprecated todos * #16 implement first version of GMPISubClientOptimizer * #16 set default callback which throws an exception indicating that setting callback has been forgotten * #16 create GMPISubClientIndividual * #16 implement GMPISubClient example * #16 rename and adjust config file for GMPISubClientOptimizer example * #16 fix bug related to passing nullptr to scatter and gather as root process * #16 use async calls to implement timeout * #16 update readme * #12 show worker node shutdown message also in release mode, because that is an important information which the user might want to see and use to adjust the workers timeout-configuration * #16 make retriving client mode possible * #16 do not initialize MPI if using a sub communicator * #16 delete deprecated unused file * #16 delete deprecated unused file * #16 make sub-client job part of the individual * #16 add comment * #16 move GMPISubClientIndividual from geneva-individuals to geneva, because it is not a complete individual but an abstract base class for user defined individuals * #16 make rank 0 of each subgroup be the Geneva client and the other ranks be the sub-client. Before this commit the rank n-1 of each subgroup was the geneva client, which is not intuitive * #12 try to synchronize all proceses before starting consumer in order to not trigger timeouts on worker side if master node is not up and running yet * #12 change unit of worker poll timeout to seconds, because any amount below one second is not useful and configuring it this way is more convenient * #12 update in-code documentation for GMPIConsumerT * #12 update examples/geneva/16_GMPIConsumer/README.md * #16 extend documenation for examples/geneva/17_GMPISubClients/ * #16 correct typo in documenation for examples/geneva/17_GMPISubClients/ * #16 change std::cout to glogger * #16 update examples/geneva/README * #12 make synchronization configurable * #16 correct typo * #16 improve code readability * #12 leave synchronization open to user, because synchronizing on startup causes issues in some edge cases like sub-client optimization * #16 fix: parse GMPISubClientOptimizer configuration options from file * #16 rewrite example 17_GMPISubClients to actually parallelize the work and measure the total wall time * #16 rename class and associated files * #16 change configuration for geneva example 17 * #16 fix: use shutdown signal instead of timeout for sub-client termination * #12 change all clocks to std::chrono::steady_clock in GMPIConsumerT * #12 improve code readability by making the time since a session is finished a property of the session class itself * #16 reformat file GMPISubClientIndividual.cpp * #16 reformat file GMPISubClientIndividual.hpp * #16 fix: make client mode accessible from GMPISubClientIndividual and use this to correct error when calling getClientStatus() in client mode, because request object was not set * #12 correct two mistakes in CMD messages which must have happened when copy pasting * #12 fix bug in prepare build script which occured because of wrong variable name * #12 wrap access to GMPIConsumer in standard consumers in compile time macro ifdef to make sure it is only compiled if building with MPI-Consumer * Added a thank you message for Jonas' contribution * Fixing compiler problems * Fixing compiler problems * Fixing compiler problems * Added <optional> to some files to fix g++ 12.2 compilation problem with MPI consumer and examples * Added some missing cstind headers to make code compile on new gcc in Ubuntu 23.10 * Working towards reviving the OpenCL code (OpenCL still does not compile, program does) * Working towards reviving the OpenCL code (OpenCL still does not compile, program does) * Removed old CUDA example * Moved example to CUDA * Debugging CUDA evaluation calculation * Debugging CUDA evaluation calculation * Centralized workers for local consumers * Fixed triangle-alignment with width and height * Moving all data definitions to float. RGB (unsigned char) is only involved when loading / saving images. * Example is now fully float-based. RGB only occurs when reading/writing images from/to disc. * float-based code again produces images * Hunting a bug where evaluation values do not fit the evaluation * Working on fixing process(), so that we can externally inject a result through the custom channels. * Eradicating problems with external evaluation * Some refactoring. process() can now accept external evaluations. * Now only a range of best individuals is added to the priority queue * Fixed a bug * Working towards random number generation on the GPU * Efficiency improvements * Removed atomic adds * Further fine-tuning * Started to refactor Geneva * Moving to C++20 and CMake 3.27 * Code now compiles with Boost 1.87 * Allowed for random initialization of triangle radius * Update of CHANGES to reflect new minimal cmake version * Removed some obsolete test code * Removed some obsolete test code * Working to revive tests * Working to revive tests * Fixing the build system to comply with Boost 1.90 * Fixing the build system to comply with Boost 1.90 * Dealing with linker issues related to CUDA --------- Co-authored-by: Jonas Weßner <jonas.wessner@gmx.de> Co-authored-by: Jonas-Wessner <81162699+Jonas-Wessner@users.noreply.github.com> * Update CHANGES file with current issues (#35) Document current issues with the development release. --------- Co-authored-by: denisbertini <d.bertini@gsi.de> Co-authored-by: Ariel Garcia <a.garcia@gemfony.eu> Co-authored-by: Jonas-Wessner <81162699+Jonas-Wessner@users.noreply.github.com> Co-authored-by: Jonas Weßner <jonas.wessner@gmx.de>
1 parent 15f1046 commit f8bf6a6

File tree

425 files changed

+49713
-69430
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

425 files changed

+49713
-69430
lines changed

.clang-format

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# This .clang-format replicates the style shown in GBoundedBufferT.hpp
2+
Language: Cpp
3+
BasedOnStyle: LLVM
4+
IndentWidth: 4
5+
TabWidth: 4
6+
UseTab: Never
7+
AccessModifierOffset: -4
8+
BreakBeforeBraces: BS_Attach
9+
BraceWrapping:
10+
AfterClass: true
11+
AfterControlStatement: false
12+
AfterEnum: true
13+
AfterFunction: false
14+
AfterNamespace: true
15+
AfterStruct: true
16+
AfterUnion: true
17+
IndentBraces: false
18+
AlwaysBreakAfterDefinitionReturnType: All
19+
AlwaysBreakAfterDeclarationReturnType: All
20+
BinPackParameters: false
21+
AlignAfterOpenBracket: BlockIndent
22+
SpaceBeforeParensControlStatements: Always
23+
SpaceBeforeParensFunctionDecl: false
24+
SpaceBeforeParensFunctionCall: false
25+
AllowShortIfStatementsOnASingleLine: false
26+
AllowShortFunctionsOnASingleLine: Empty
27+
AllowShortBlocksOnASingleLine: false
28+
ColumnLimit: 120
29+
AlignTrailingComments: true
30+
31+

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,6 @@
55
/.cproject
66
.DS_Store
77
.com.apple.*
8+
build/
9+
cmake-build-*/
10+
.vscode

CHANGES

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,38 @@
1+
//------------------------------------------------------------------------------------------------------
2+
* Version 1.11, "Hendaye" - March 16, 2026
3+
4+
We have restarted development of the Geneva library!
5+
6+
Recommended versions:
7+
- Boost version 1.90 or higher is required. You may try with Boost 1.89 (untested). Note that the
8+
Boost.System library linkage has been removed from the build-system, as that library is now header-only. So
9+
in order to get Geneva to compiler with older Boost versions you may have to modify the build-system (see
10+
the various instances of CommonGenevaBuild.cmake).
11+
- CMake version 3.27 or higher is required
12+
- GCC is recommended to be at least version 13, clang should be at least version 18.
13+
Earlier versions might work if C++20 is sufficiently supported
14+
- We suggest to use Ubuntu 24.04 LTS as the host platform if you plan to use the CUDA example.
15+
16+
********************** INCOMPATIBILITIES / PLEASE NOTE *************************
17+
18+
- Geneva now uses C++20 constructs, hence you need a compiler that supports this.
19+
It is thus also recommended to compile Boost with C++20 support, to be consistent
20+
21+
************************* Misc *************************************************
22+
23+
- Where CUDA is available, random numbers will now be calculated on the GPU
24+
- A new CUDA example has been added
25+
- boost::optional, boost::any, boost::filesystem, boost::variant have been replaced
26+
by their C++17 counterparts
27+
- Benchmarks for courtier and geneva have been moved to their own benchmarks top-level
28+
directory and are no longer considered to be tests
29+
- An extensive CUDA example has been implemented
30+
31+
********************************************************************************
32+
33+
Current issues with this development release:
34+
- The C++ standard is not set centrally, but individually for each library
35+
136
//------------------------------------------------------------------------------------------------------
237
* Version 1.10, "Genova" - March 10, 2020
338

CMakeLists.txt

Lines changed: 26 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -34,11 +34,11 @@
3434
#
3535
################################################################################
3636

37-
CMAKE_MINIMUM_REQUIRED(VERSION 3.1)
37+
CMAKE_MINIMUM_REQUIRED(VERSION 3.27 FATAL_ERROR)
3838

3939
MESSAGE ("")
4040

41-
PROJECT (Geneva_Library_Collection)
41+
PROJECT (Geneva_Library_Collection LANGUAGES CXX CUDA)
4242

4343
################################################################################
4444
# We create some examples and test programs as part of the build process.
@@ -50,9 +50,13 @@ PROJECT (Geneva_Library_Collection)
5050
SET(GENEVA_FULL_TREE_BUILD TRUE)
5151

5252
################################################################################
53-
# Set this variable to make all examples run as part of the testing with CTest.
53+
# Set these variables to make all examples and/or benchmarks run as part of the
54+
# testing with CTest.
5455

55-
#SET(GENEVA_RUN_EXAMPLES_AS_TESTS TRUE)
56+
# NOTE: Not yet functional
57+
58+
SET(GENEVA_RUN_EXAMPLES_AS_TESTS FALSE)
59+
SET(GENEVA_RUN_BENCHMARKS_AS_TESTS FALSE)
5660

5761
################################################################################
5862
# Different subcomponents of the Geneva binary tree can be installed in
@@ -77,6 +81,12 @@ ENDIF()
7781

7882
SET(CMAKE_MODULE_PATH "${PROJECT_SOURCE_DIR}/CMakeModules")
7983

84+
################################################################################
85+
# Set preprocessor define to use in source code if building with MPI consumer
86+
IF (GENEVA_BUILD_WITH_MPI_CONSUMER)
87+
ADD_COMPILE_DEFINITIONS(GENEVA_BUILD_WITH_MPI_CONSUMER)
88+
ENDIF ()
89+
8090
###############################################################################
8191
# Include the shared functionality module for any Geneva build
8292

@@ -87,7 +97,7 @@ INCLUDE(CommonGenevaBuild)
8797
# versions while even minor numbers stand for production releases.
8898

8999
SET(VERSION_MAJOR "1")
90-
SET(VERSION_MINOR "10")
100+
SET(VERSION_MINOR "11")
91101
SET(VERSION_PATCH "0")
92102

93103
SET(GENEVA_VERSION ${VERSION_MAJOR}.${VERSION_MINOR}.${VERSION_PATCH})
@@ -128,16 +138,24 @@ IF( GENEVA_BUILD_TESTS )
128138
ENDIF()
129139

130140
# Examples are also run as tests if enabled with 'GENEVA_RUN_EXAMPLES_AS_TESTS'
131-
ADD_SUBDIRECTORY (examples)
141+
IF( GENEVA_BUILD_EXAMPLES )
142+
ADD_SUBDIRECTORY (examples)
143+
ENDIF()
144+
145+
# Benchmarks are also run as tests if enabled with GENEVA_RUN_BENCHMARKS_AS_TESTS
146+
IF( GENEVA_BUILD_BENCHMARKS )
147+
ADD_SUBDIRECTORY (benchmarks)
148+
ENDIF()
132149

133-
# Holds an incubator facility for new optimization algorihms
150+
# Holds an incubator facility for new optimization algorithms
134151
# ADD_SUBDIRECTORY (incubator)
135152

136153
################################################################################
137154
# Custom target for all compilable items in the distribution
138155
ADD_CUSTOM_TARGET(
139156
"gemfony-build-all" DEPENDS
140157
core
158+
benchmarks
141159
examples
142160
tests
143161
# incubator
@@ -240,7 +258,7 @@ SET(CPACK_RPM_PACKAGE_LICENSE "Apache License 2.0")
240258
SET(CPACK_RPM_PACKAGE_GROUP "Sciences/Libraries")
241259
SET(CPACK_RPM_PACKAGE_REQUIRES "boost-devel >= ${GENEVA_MIN_BOOST_VERSION}")
242260

243-
# Detect the distribution to autoselect the package generator...
261+
# Detect the distribution to autos-elect the package generator...
244262
EXECUTE_PROCESS(COMMAND lsb_release -is
245263
OUTPUT_VARIABLE CPACK_AUX_DISTRO
246264
)

CMakeModules/CMakeLists.txt

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,38 @@
1-
CMAKE_MINIMUM_REQUIRED(VERSION 3.1)
1+
################################################################################
2+
#
3+
# This file is part of the Geneva library collection. The following license
4+
# applies to this file:
5+
#
6+
# ------------------------------------------------------------------------------
7+
# Licensed under the Apache License, Version 2.0 (the "License");
8+
# you may not use this file except in compliance with the License.
9+
# You may obtain a copy of the License at
10+
#
11+
# http://www.apache.org/licenses/LICENSE-2.0
12+
#
13+
# Unless required by applicable law or agreed to in writing, software
14+
# distributed under the License is distributed on an "AS IS" BASIS,
15+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16+
# See the License for the specific language governing permissions and
17+
# limitations under the License.
18+
# ------------------------------------------------------------------------------
19+
#
20+
# Note that other files in the Geneva library collection may use a different
21+
# license. Please see the licensing information in each file.
22+
#
23+
################################################################################
24+
#
25+
# Geneva was started by Dr. Rüdiger Berlich and was later maintained together
26+
# with Dr. Ariel Garcia under the auspices of Gemfony scientific. For further
27+
# information on Gemfony scientific, see http://www.gemfomy.eu .
28+
#
29+
# The majority of files in Geneva was released under the Apache license v2.0
30+
# in February 2020.
31+
#
32+
# See the NOTICE file in the top-level directory of the Geneva library
33+
# collection for a list of contributors and copyright information.
34+
#
35+
################################################################################
236

337
SET ( CMAKEFILES
438
README

CMakeModules/CommonGenevaBuild.cmake

Lines changed: 20 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@
4141

4242
################################################################################
4343

44-
CMAKE_MINIMUM_REQUIRED(VERSION 3.1 FATAL_ERROR)
44+
CMAKE_MINIMUM_REQUIRED(VERSION 3.27 FATAL_ERROR)
4545

4646
# Include guard
4747
IF(NOT COMMON_GENEVA_BUILD_INCLUDED)
@@ -99,13 +99,14 @@ FLAG_UNSUPPORTED_SETUPS(
9999
################################################################################
100100
# Set the C++ standard to be used
101101

102-
# Geneva requires at least the C++14 Standard. The user may force another
103-
# value at its own risk by setting the variable CMAKE_CXX_STANDARD.
102+
# Geneva requires at least the C++20 Standard. The user may force another
103+
# value at his own risk by setting the variable CMAKE_CXX_STANDARD.
104104
IF( NOT DEFINED CMAKE_CXX_STANDARD )
105-
SET( CMAKE_CXX_STANDARD "14" )
105+
SET( CMAKE_CXX_STANDARD "20" )
106106
ENDIF()
107107

108-
SET_CXX_STANDARD_FLAG()
108+
SET(CMAKE_CXX_STANDARD_REQUIRED ON)
109+
set(CMAKE_CXX_EXTENSIONS OFF)
109110

110111
################################################################################
111112
# Set the compiler and linker flags
@@ -159,16 +160,8 @@ ENDIF ()
159160

160161
SET (Boost_USE_MULTITHREAD ON)
161162
SET (Boost_ADDITIONAL_VERSIONS
162-
"1.70"
163-
"1.70.0"
164-
"1.71"
165-
"1.71.0"
166-
"1.72"
167-
"1.72.0"
168-
"1.73"
169-
"1.73.0"
170-
"1.74"
171-
"1.74.0"
163+
"1.90.0"
164+
"1.90"
172165
)
173166

174167
IF ( GENEVA_STATIC )
@@ -192,7 +185,7 @@ ELSE () # Dynamic libraries
192185
ENDIF ()
193186

194187
# The minimum Boost version required for building Geneva and Geneva applications
195-
SET (GENEVA_MIN_BOOST_VERSION 1.70)
188+
SET (GENEVA_MIN_BOOST_VERSION 1.90)
196189

197190
# These are the libraries required for any Geneva build
198191
SET (
@@ -201,7 +194,6 @@ SET (
201194
filesystem
202195
regex
203196
serialization
204-
system
205197
program_options
206198
)
207199

@@ -238,13 +230,14 @@ MESSAGE("")
238230

239231
INCLUDE_DIRECTORIES(${Boost_INCLUDE_DIRS})
240232

241-
# Optionally search for OpenCL
242-
IF(GENEVA_BUILD_WITH_OPENCL_EXAMPLES)
243-
FIND_PACKAGE(OpenCL REQUIRED)
244-
MESSAGE("")
245-
IF(OpenCL_FOUND)
246-
INCLUDE_DIRECTORIES(${OpenCL_INCLUDE_DIRS})
247-
ENDIF()
233+
# Optionally Search for MPI
234+
IF(GENEVA_BUILD_WITH_MPI_CONSUMER)
235+
MESSAGE("Searching for MPI...\n")
236+
FIND_PACKAGE(MPI REQUIRED)
237+
MESSAGE("")
238+
IF(MPI_FOUND)
239+
INCLUDE_DIRECTORIES(${MPI_INCLUDE_PATH})
240+
ENDIF()
248241
ENDIF()
249242

250243
# Add compile-time debug information about Boost's linked libraries
@@ -444,6 +437,9 @@ IF(NOT CMAKE_CONFIGURATION_TYPES)
444437
STRING (REGEX REPLACE "[ \t]+" "\n\t\t\t\t\t " CMAKE_CXX_FLAGS_SEP ${CMAKE_CXX_FLAGS_STRIPPED})
445438
MESSAGE ("\twith C++ compiler flags:\t ${CMAKE_CXX_FLAGS_SEP}")
446439
ENDIF ()
440+
441+
MESSAGE("\tUsing C++ standard ${CMAKE_CXX_STANDARD}")
442+
447443
# Don't print linker options if empty
448444
STRING (STRIP "${CMAKE_EXE_LINKER_FLAGS}" CMAKE_L_FLAGS_STRIPPED)
449445
IF(NOT "${CMAKE_L_FLAGS_STRIPPED}" STREQUAL "")

CMakeModules/IdentifySystemParameters.cmake

Lines changed: 14 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -206,25 +206,6 @@ FUNCTION (
206206

207207
ENDFUNCTION()
208208

209-
################################################################################
210-
# Sets the C++ standard flags for this compiler
211-
#
212-
MACRO (
213-
SET_CXX_STANDARD_FLAG
214-
)
215-
216-
#--------------------------------------------------------------------------
217-
# Set the C++-standard switch for our compiler
218-
IF (${CMAKE_VERSION} VERSION_LESS 3.1)
219-
ADD_COMPILE_OPTIONS("${${CMAKE_CXX_COMPILER_ID}_DEF_CXX${CMAKE_CXX_STANDARD}_STANDARD_FLAG}")
220-
ELSE()
221-
# Let CMake take care of the C++ standard flag, using CMAKE_CXX_STANDARD
222-
SET(CMAKE_CXX_STANDARD_REQUIRED ON)
223-
ENDIF()
224-
#--------------------------------------------------------------------------
225-
226-
ENDMACRO()
227-
228209
################################################################################
229210
# Sets the compiler flags for this platform and compiler
230211
#
@@ -319,19 +300,30 @@ FUNCTION (
319300
# For Clang on MacOSX we require the standard C++ library
320301
IF(${GENEVA_OS_NAME_IN} STREQUAL "MacOSX")
321302
SET (CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -stdlib=libc++" PARENT_SCOPE)
322-
IF( NOT GENEVA_STATIC )
303+
IF(NOT GENEVA_STATIC)
323304
SET (CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -stdlib=libc++" PARENT_SCOPE)
324305
SET (MACOSX_RPATH 1)
325306
ENDIF()
326307
ELSEIF(${CMAKE_CXX_COMPILER_VERSION} VERSION_LESS 3.7)
327308
# Avoid https://llvm.org/bugs/show_bug.cgi?id=18402
328309
# when using older libstdc++ versions
329310
SET (CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -stdlib=libc++" PARENT_SCOPE)
330-
IF( NOT GENEVA_STATIC )
311+
IF(NOT GENEVA_STATIC)
331312
SET (CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -stdlib=libc++" PARENT_SCOPE)
332313
ENDIF()
333314
ENDIF()
334315

316+
#*****************************************************************
317+
ELSEIF(CMAKE_CXX_COMPILER_ID MATCHES ${GNU_DEF_IDENTIFIER})
318+
319+
# For GCC version < 9.0 add the filesystem library explicitely
320+
IF(${CMAKE_CXX_COMPILER_VERSION} VERSION_LESS 9.0)
321+
SET (CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -lstdc++fs" PARENT_SCOPE)
322+
IF(NOT GENEVA_STATIC)
323+
SET (CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -lstdc++fs" PARENT_SCOPE)
324+
ENDIF()
325+
ENDIF()
326+
335327
ENDIF()
336328
#--------------------------------------------------------------------------
337329

@@ -378,7 +370,7 @@ FUNCTION (
378370
#--------------------------------------------------------------------------
379371
IF(${GENEVA_OS_NAME_IN} STREQUAL "MacOSX")
380372
# Only clang is currently supported on MacOS
381-
IF(NOT ${CMAKE_CXX_COMPILER_ID} STREQUAL ${CLANG_DEF_IDENTIFIER} AND NOT CMAKE_CXX_COMPILER_ID STREQUAL ${APPLECLANG_DEF_IDENTIFIER})
373+
IF(NOT ${CMAKE_CXX_COMPILER_ID} STREQUAL ${CLANG_DEF_IDENTIFIER} AND NOT CMAKE_CXX_COMPILER_ID STREQUAL ${APPLECLANG_DEF_IDENTIFIER})
382374
MESSAGE("####################################################################################")
383375
MESSAGE("# Compiler ${CMAKE_CXX_COMPILER_ID} is not supported on MacOSX. Use Clang instead. #")
384376
MESSAGE("####################################################################################")

0 commit comments

Comments
 (0)