hw_submission(李昀迪): add hw4_20230316

Q1/README.md

@@ -0,0 +1,8 @@
+## RND主要创新点
+1、采用随机生成的固定网络作为目标网络，另一网络不断最小化与其的误差，从而达到评估观察的新颖性</br>
+2、介绍了一种灵活地结合内在和外在奖励的方法
+
+## RND结果分析
+1、little_RND_net和small_RND_net版本，一个little指标都接近于0，一个small指标很高，模型的目标就是最小化MSE，但是small版本的MSE很高，感觉这两个版本都没有训练成功。</br>
+2、standard_RND_net这个版本是最好的版本。</br>
+3、large_RND_net和very_large_RND_net版本，很明显，reward_min值开始提高，min值就是代表网络开始过拟合，开始往训练集过度靠拢，所以这两个版本出现了过拟合的情况。</br>

Q2/q2.py

@@ -0,0 +1,30 @@
+# Please install latest DI-engine's main branch first
+from ding.bonus import PPOF
+
+
+def acrobot():
+    # Please install acrobot env first, `pip3 install gym`
+    # You can refer to the env doc (https://di-engine-docs.readthedocs.io/zh_CN/latest/13_envs/acrobot_zh.html) for more details
+    agent = PPOF(env='acrobot', exp_name='./acrobot_demo')
+    agent.train(step=int(1e5))
+
+
+def metadrive():
+    # Please install metadrive env first, `pip install metadrive-simulator`
+    # You can refer to the env doc (https://di-engine-docs.readthedocs.io/zh_CN/latest/13_envs/metadrive_zh.html) for more details
+    agent = PPOF(env='metadrive', exp_name='./metadrive_demo')
+    agent.train(step=int(1e6), context='spawn')
+
+
+def minigrid_fourroom():
+    # Please install minigrid env first, `pip install gym-minigrid`
+    # Note: minigrid env doesn't support Windows platform
+    # You can refer to the env doc (https://di-engine-docs.readthedocs.io/zh_CN/latest/13_envs/minigrid_zh.html) for more details
+    agent = PPOF(env='minigrid_fourroom', exp_name='./minigrid_fourroom_demo')
+    agent.train(step=int(3e6))
+
+
+if __name__ == "__main__":
+    # acrobot()
+    # metadrive()
+    minigrid_fourroom()