Change PyEmu's RNG to be mostly locally scoped#689
Merged
briochh merged 10 commits intopypest:developfrom Mar 18, 2026
Merged
Conversation
added 10 commits
March 10, 2026 17:34
…anigans for en.py
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #689 +/- ##
===========================================
+ Coverage 75.14% 75.15% +0.01%
===========================================
Files 37 37
Lines 19234 19253 +19
===========================================
+ Hits 14453 14470 +17
- Misses 4781 4783 +2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR changes PyEmu’s RNG to be local rather than global, based on the discussion in #688. This protects both the user and PyEmu. When PyEmu’s RNG is dependent on the global seed, it is prone to manipulation by both the user and other package imports that may be altering
np.random.seed(). Local RNG protects PyEmu and ensures that reproducible results occur regardless of the state of the script that’s calling PyEmu functions. i.e. If the global seed is changed by the user or another package, PyEmu’s draws will not change because nearly all draw functions now rely on local rng objects.The goal of this PR is moving RNG to a local scope while also maintaining the hard-sought reproducibility in draws that PyEmu creates for ensembles and other cases for which values must be drawn from a distribution. Draws made by PyEmu should not change post-merge, and the fact that all tests pass give me confidence in this. Some tests check PyEmu output from a pickled state, and as PyEmu output is still matching those saved base cases, this tells me that the RNG state has remained intact even though the implementation has changed.
The main changes occur in
en.py, withnp.random.seed(SEED)now replaced withrng = np.random.RandomState(SEED).RandomStateis the older NumPy generator class (identical to manipulating rng usingnp.random.seed()) and ensures consistency regardless of the NumPy version that’s in the environment that is using PyEmu. By default, functions or methods that rely on rng point to the RandomState object created at the start ofen.py. Functions inen.py,helpers.py,geostats.pyandpst_from.py, are also altered to accept a local “rng” object. This allows tests that previously manipulated the global seed to still pass, and additionally allows users that want a different generator, for whatever reason, to draw using their own generator. For the vast majority of users, the rng argument will remain set to None for all draw functions, resulting in the use of the RandomState object at the top ofen.py(which produces the same sequences as the current development branch).An exhaustive list of methods that have had their rng code altered to be local are listed below. If methods are missing, please let me know. There are two special cases that I did not alter:
draw_conditional”which was already set up to accept a seed number, has not been altered. This results in a slight discontinuity in how rng is implemented in draw_conditional vs. other methods (seed number vs. rng object), but I don’t want to cause issues for current users.The other culprit is DSIVC’s
AutoEncoderindsiae.py. Due to how potentially entangled tensorflow’s and numpy’s RNGs may be under the hood of some of these ML functions, I have left the current global seed code untouched. I am unsure of the order of operations for these functions, but the current implementation leavesprepare_dsivcuntouched, so that its draw function still points to the RNG fromen.py. Once the AutoEncoder is fit, the global seed for both tensorflow and numpy will be set to the user-defined or default random_state value, at which point anynp.randomortf.randomoperations performed afterward in the script will rely on that newly defined seed. Again, this is the same behavior as before. I am less worried about this particular case because this global seed definition is nestled away inside a function definition, rather than activated on import.en.py:Rng at top of file has been changed to a RandomState object.
reseednow points to the new RandomState object.If a function below has a definition for both ObservationEnsemble and ParameterEnsemble, both versions have been altered.
_guassian_draw_draw_new_ensemblefrom_guassian_drawdraw_new_ensemblefrom_triangular_drawfrom_uniform_drawfrom_mixed_drawsgeostats.py:draw_arraysgrid_par_ensemble_helperhelpers.py:autcorrelated_drawdraw_by_groupgeostatistical_drawsget_maha_obs_summarypst_from.py:draw(In various other locations and example notebooks, small
np.randomfunctions pop up. These have been changed to point to the rng object inen.pyinstead.)I know this is a substantial change and will require some thorough review. I appreciate your time. I am investigating the large diffs in dsiae.py and transformers.py. I presume a rogue space or something has caused GitHub's diff to freak out a bit.
Runners are now all passing.