Implement HER

docs/misc/changelog.rst

@@ -25,6 +25,7 @@ New Features:
 - Refactored opening paths for saving and loading to use strings, pathlib or io.BufferedIOBase (@PartiallyTyped)
 - Added ``DDPG`` algorithm as a special case of ``TD3``.
 - Introduced ``BaseModel`` abstract parent for ``BasePolicy``, which critics inherit from.
+- Added Hindsight Experience Replay ``HER``. (@megan-klaiber)
 
 Bug Fixes:
 ^^^^^^^^^^
@@ -356,4 +357,4 @@ And all the contributors:
 @Miffyli @dwiel @miguelrass @qxcv @jaberkow @eavelardev @ruifeng96150 @pedrohbtp @srivatsankrishnan @evilsocket
 @MarvineGothic @jdossgollin @SyllogismRXS @rusu24edward @jbulow @Antymon @seheevic @justinkterry @edbeeching
 @flodorner @KuKuXia @NeoExtended @PartiallyTyped @mmcenta @richardwu @kinalmehta @rolandgvc @tkelestemur @mloo3
-@tirafesi @blurLake @koulakis @joeljosephjin @shwang
+@tirafesi @blurLake @koulakis @joeljosephjin @shwang @megan-klaiber

stable_baselines3/her/goal_selection_strategy.py

@@ -0,0 +1,31 @@
+from enum import Enum
+
+
+class GoalSelectionStrategy(Enum):
+    """
+    The strategies for selecting new goals when
+    creating artificial transitions.
+    """
+
+    # Select a goal that was achieved
+    # after the current step, in the same episode
+    FUTURE = 0
+    # Select the goal that was achieved
+    # at the end of the episode
+    FINAL = 1
+    # Select a goal that was achieved in the episode
+    EPISODE = 2
+    # Select a goal that was achieved
+    # at some point in the training procedure
+    # (and that is present in the replay buffer)
+    RANDOM = 3
+
+
+# For convenience
+# that way, we can use string to select a strategy
+KEY_TO_GOAL_STRATEGY = {
+    "future": GoalSelectionStrategy.FUTURE,
+    "final": GoalSelectionStrategy.FINAL,
+    "episode": GoalSelectionStrategy.EPISODE,
+    "random": GoalSelectionStrategy.RANDOM,
+}