强化学习系列文章

第一章 强化学习入门

目录

1.简介

2.Gym游戏环境的安装

3.Gym所含的游戏

3.1 打砖块 Breakout-v0或Breakout-v4

3.2 倒立摆游戏 CartPole-v0或CartPole-v1

3.3 小车翻沟游戏

3.4 火箭降落小游戏LunarLander-v2

3.5 所有支持的游戏如下

1.简介

安装破解Pycharm:https://www.shuopython.com/archives/2339

OpenAI的官网地址：https://www.openai.com/

OpenAI教学网址：https://spinningup.openai.com/en/latest/

OpenAI的强化学习源代码：https://github.com/openai/baselines

                                                              图1 2019年7月OpenAI团队和他们的合作伙伴的户外合照

大事记：

2.Gym游戏环境的安装

超简单，直接pip install gym

我是用pycharm，所以在project interpreter中安装了gym。整个安装过程没有遇到bug。

验证运行的代码：

import gym
env = gym.make('CartPole-v0')
for i_episode in range(20):observation = env.reset()for t in range(100):env.render()print(observation)action = env.action_space.sample()observation, reward, done, info = env.step(action)if done:print("Episode finished after {} timesteps".format(t+1))break

运行如下画面：

3.Gym所含的游戏

从gym的安装目录下envs即可看出有哪些游戏，里面的py程序可以直接运行。听说还有各个游戏的排行榜，不知道在哪看。

Gym包含游戏的查看网址：https://gym.openai.com/envs/#classic_control

3.1 打砖块 Breakout-v0或Breakout-v4

第一步：安装

pip3 install keyboard
pip3 install atari-py

第二步：开始玩游戏

import gym
import keyboard
import numpy as np
import timetotal_reward = 0
env = gym.make('Breakout-v0')
state = env.reset()action = 0def preprocess(img):img_temp = img.mean(axis = 2)x = -1y = -1if len(np.where((img_temp[100:189,8:152])!= 0)[0]) != 0:x = np.where((img_temp[100:189,8:152])!= 0)[0][0]y = np.where((img_temp[100:189,8:152])!= 0)[1][0]if len(np.where((img_temp[193:,8:152])!= 0)[0]) != 0:x = -2y = -2p = int(np.where(img_temp[191:193,8:152])[1].mean() - 7.5)#return img_tempreturn (x,y,p)# 实际按键中只检测0，1，2，3 游戏中需要的按键
# 我添加了一个按键4，用于暂停
def abc(x):global actionif x.event_type == "down" and x.name == '0':action = 0elif x.event_type == "down" and x.name == '1':#发球action = 1elif x.event_type == "down" and x.name == '2':#左移action = 3elif x.event_type == "down" and x.name == '3':#右移action = 2elif x.event_type == "down" and (action == 4 or x.name == '4'):action = 4elif action != 4:action = 0# 添加hook，以检测用户的按键
keyboard.hook(abc)
total_reward = 0
for j in range(1000):env.render()while action == 4:time.sleep(0.1)if action == 4:action = 0next_state,reward,done,_ = env.step(action)total_reward += reward(x2,y2,p2) = preprocess(next_state)print(action,total_reward,done,x2,y2,p2)time.sleep(0.1)if done:break
keyboard.wait()

3.2 倒立摆游戏 CartPole-v0或CartPole-v1

import gym
env = gym.make('CartPole-v0')
env.reset()
for _ in range(1000):env.render()env.step(env.action_space.sample()) # take a random action
env.close()

3.3 小车翻沟游戏

## 基于强化学习实现小车自适应翻越小沟import gym
import timeclass BespokeAgent:def __init__(self, env):passdef decide(self, observation):position, velocity = observationlb = min(-0.09 * (position + 0.25) ** 2 + 0.03, 0.3 * (position + 0.9) ** 4 - 0.008)ub = -0.07 * (position + 0.38) ** 2 + 0.06if lb < velocity < ub:action = 2else:action = 0return action  # 返回动作def learn(self, *args):  # 学习passdef play_ones(self, env, agent, render=False, train=False):episode_reward = 0  # 记录回合总奖励，初始值为0observation = env.reset()  # 重置游戏环境，开始新回合while True:  # 不断循环，直到回合结束if render:  # 判断是否显示env.render()  # 显示图形界面，可以用env.close()关闭action = agent.decide(observation)next_observation, reward, done, _ = env.step(action)  # 执行动作episode_reward += reward  # 搜集回合奖励if train:  # 判断是否训练智能体breakobservation = next_observationreturn episode_reward  # 返回回合总奖励if __name__ == '__main__':env = gym.make('MountainCar-v0')env.seed(0)  # 设置随机数种子，只是为了让结果可以精确复现，一般情况下可以删除agent = BespokeAgent(env)for _ in range(100):episode_reward = agent.play_ones(env, agent, render=True)print('回合奖励={}'.format(episode_reward))time.sleep(10)  # 停顿10senv.close()  # 关闭图形化界面

3.4 火箭降落小游戏LunarLander-v2

第一步，安装box2d

pip install box2d

第二步，开始玩

import gymenv = gym.make('LunarLander-v2')# print env.observation_space
# print env.action_spacefor i_episode in range(100):observation = env.reset()for t in range(100):env.render()print(observation)action = env.action_space.sample()observation, reward, done, info = env.step(action)if done:print("Episode finished after {} timesteps".format(t + 1))break

3.5 所有支持的游戏如下

(Acrobot-v1),
(AirRaid-ram-v0),
(AirRaid-ram-v4),
(AirRaid-ramDeterministic-v0),
(AirRaid-ramDeterministic-v4),
(AirRaid-ramNoFrameskip-v0),
(AirRaid-ramNoFrameskip-v4),
(AirRaid-v0),
(AirRaid-v4),
(AirRaidDeterministic-v0),
(AirRaidDeterministic-v4),
(AirRaidNoFrameskip-v0),
(AirRaidNoFrameskip-v4),2d飞机
(Alien-ram-v0),
(Alien-ram-v4),
(Alien-ramDeterministic-v0),
(Alien-ramDeterministic-v4),
(Alien-ramNoFrameskip-v0), 2d迷宫小游戏
(Alien-ramNoFrameskip-v4),
(Alien-v0),小游戏迷宫打怪
(Alien-v4),
(AlienDeterministic-v0),
(AlienDeterministic-v4),
(AlienNoFrameskip-v0),
(AlienNoFrameskip-v4),
(Amidar-ram-v0), 、
(Amidar-ram-v4),
(Amidar-ramDeterministic-v0),
(Amidar-ramDeterministic-v4), 小游戏迷宫打怪
(Amidar-ramNoFrameskip-v0),
(Amidar-ramNoFrameskip-v4), 小游戏迷宫打怪
(Amidar-v0),
(Amidar-v4),
(AmidarDeterministic-v0),
(AmidarDeterministic-v4),小游戏迷宫
(AmidarNoFrameskip-v0),
(AmidarNoFrameskip-v4), 2d迷宫
(Ant-v2),
(Assault-ram-v0), 2d飞行小游戏
(Assault-ram-v4),
(Assault-ramDeterministic-v0), 小游戏2d飞行打怪
(Assault-ramDeterministic-v4),
(Assault-ramNoFrameskip-v0),
(Assault-ramNoFrameskip-v4), 小游戏打怪
(Assault-v0),
(Assault-v4),
(AssaultDeterministic-v0),
(AssaultDeterministic-v4),
(AssaultNoFrameskip-v0),
(AssaultNoFrameskip-v4),
(Asterix-ram-v0),
(Asterix-ram-v4),
(Asterix-ramDeterministic-v0),
(Asterix-ramDeterministic-v4),小游戏飞行打怪
(Asterix-ramNoFrameskip-v0),
(Asterix-ramNoFrameskip-v4),
(Asterix-v0),
(Asterix-v4),
(AsterixDeterministic-v0),
(AsterixDeterministic-v4),
(AsterixNoFrameskip-v0),
(AsterixNoFrameskip-v4),
(Asteroids-ram-v0),
(Asteroids-ram-v4), 小游戏
(Asteroids-ramDeterministic-v0),
(Asteroids-ramDeterministic-v4), 小游戏打怪
(Asteroids-ramNoFrameskip-v0),
(Asteroids-ramNoFrameskip-v4),
(Asteroids-v0),
(Asteroids-v4),
(AsteroidsDeterministic-v0),
(AsteroidsDeterministic-v4),
(AsteroidsNoFrameskip-v0),
(AsteroidsNoFrameskip-v4),
(Atlantis-ram-v0),
(Atlantis-ram-v4),
(Atlantis-ramDeterministic-v0), 打水怪
(Atlantis-ramDeterministic-v4),
(Atlantis-ramNoFrameskip-v0),小游戏打水怪
(Atlantis-ramNoFrameskip-v4),
(Atlantis-v0),
(Atlantis-v4),
(AtlantisDeterministic-v0),飞行打怪小游戏
(AtlantisDeterministic-v4), 2D飞机打水怪
(AtlantisNoFrameskip-v0),
(AtlantisNoFrameskip-v4),
(BankHeist-ram-v0),
(BankHeist-ram-v4),
(BankHeist-ramDeterministic-v0), 2d迷宫
(BankHeist-ramDeterministic-v4),
(BankHeist-ramNoFrameskip-v0),
(BankHeist-ramNoFrameskip-v4),
(BankHeist-v0),
(BankHeist-v4),
(BankHeistDeterministic-v0),
(BankHeistDeterministic-v4),
(BankHeistNoFrameskip-v0),
(BankHeistNoFrameskip-v4),
(BattleZone-ram-v0),
(BattleZone-ram-v4),
(BattleZone-ramDeterministic-v0),
(BattleZone-ramDeterministic-v4),
(BattleZone-ramNoFrameskip-v0),
(BattleZone-ramNoFrameskip-v4), 2D坦克
(BattleZone-v0),
(BattleZone-v4),
(BattleZoneDeterministic-v0), 坦克小游戏
(BattleZoneDeterministic-v4),
(BattleZoneNoFrameskip-v0),
(BattleZoneNoFrameskip-v4),
(BeamRider-ram-v0),
(BeamRider-ram-v4),
(BeamRider-ramDeterministic-v0),
(BeamRider-ramDeterministic-v4),
(BeamRider-ramNoFrameskip-v0),
(BeamRider-ramNoFrameskip-v4),
(BeamRider-v0),
(BeamRider-v4),
(BeamRiderDeterministic-v0),
(BeamRiderDeterministic-v4), 小游戏2d飞行打怪
(BeamRiderNoFrameskip-v0),
(BeamRiderNoFrameskip-v4),
(Berzerk-ram-v0),
(Berzerk-ram-v4),
(Berzerk-ramDeterministic-v0),小游戏迷宫打怪
(Berzerk-ramDeterministic-v4), 小游戏迷宫打怪
(Berzerk-ramNoFrameskip-v0),
(Berzerk-ramNoFrameskip-v4),
(Berzerk-v0),
(Berzerk-v4),
(BerzerkDeterministic-v0), 2d迷宫
(BerzerkDeterministic-v4), 2d飞行小游戏
(BerzerkNoFrameskip-v0),
(BerzerkNoFrameskip-v4), 小游戏迷宫打怪
(BipedalWalker-v2),
(BipedalWalkerHardcore-v2),
(Blackjack-v0), 错误
(Bowling-ram-v0),
(Bowling-ram-v4), 弹球类小游戏
(Bowling-ramDeterministic-v0),
(Bowling-ramDeterministic-v4),
(Bowling-ramNoFrameskip-v0),
(Bowling-ramNoFrameskip-v4),
(Bowling-v0),
(Bowling-v4), 小游戏弹球
(BowlingDeterministic-v0),
(BowlingDeterministic-v4),
(BowlingNoFrameskip-v0),
(BowlingNoFrameskip-v4),
(Boxing-ram-v0),
(Boxing-ram-v4),
(Boxing-ramDeterministic-v0),小游戏博弈
(Boxing-ramDeterministic-v4),
(Boxing-ramNoFrameskip-v0),
(Boxing-ramNoFrameskip-v4),
(Boxing-v0), 小游戏博弈
(Boxing-v4),
(BoxingDeterministic-v0),
(BoxingDeterministic-v4), 小游戏博弈
(BoxingNoFrameskip-v0),
(BoxingNoFrameskip-v4),
(Breakout-ram-v0), 乒乓球
(Breakout-ram-v4),
(Breakout-ramDeterministic-v0),
(Breakout-ramDeterministic-v4), 小游戏乒乓球
(Breakout-ramNoFrameskip-v0),
(Breakout-ramNoFrameskip-v4),
(Breakout-v0),
(Breakout-v4), 小游戏打乒乓球
(BreakoutDeterministic-v0),
(BreakoutDeterministic-v4),
(BreakoutNoFrameskip-v0),
(BreakoutNoFrameskip-v4),
(CNNClassifierTraining-v0),
(CarRacing-v0),
(Carnival-ram-v0), 2d飞机打怪
(Carnival-ram-v4),
(Carnival-ramDeterministic-v0),
(Carnival-ramDeterministic-v4),
(Carnival-ramNoFrameskip-v0), 小游戏2d飞行打怪
(Carnival-ramNoFrameskip-v4),
(Carnival-v0),
(Carnival-v4),
(CarnivalDeterministic-v0),
(CarnivalDeterministic-v4),
(CarnivalNoFrameskip-v0),
(CarnivalNoFrameskip-v4),
(CartPole-v0),
(CartPole-v1),
(Centipede-ram-v0),
(Centipede-ram-v4),
(Centipede-ramDeterministic-v0),
(Centipede-ramDeterministic-v4),
(Centipede-ramNoFrameskip-v0),
(Centipede-ramNoFrameskip-v4),
(Centipede-v0), 小游戏弹球打怪
(Centipede-v4),
(CentipedeDeterministic-v0),
(CentipedeDeterministic-v4),
(CentipedeNoFrameskip-v0),
(CentipedeNoFrameskip-v4),
(ChopperCommand-ram-v0),
(ChopperCommand-ram-v4),小游戏打怪
(ChopperCommand-ramDeterministic-v0),
(ChopperCommand-ramDeterministic-v4),
(ChopperCommand-ramNoFrameskip-v0),
(ChopperCommand-ramNoFrameskip-v4),小游戏打怪
(ChopperCommand-v0),
(ChopperCommand-v4),
(ChopperCommandDeterministic-v0), 小游戏两边射出子弹打怪
(ChopperCommandDeterministic-v4),
(ChopperCommandNoFrameskip-v0),
(ChopperCommandNoFrameskip-v4),
(CliffWalking-v0),
(ConvergenceControl-v0),
(Copy-v0),
(CrazyClimber-ram-v0),
(CrazyClimber-ram-v4),
(CrazyClimber-ramDeterministic-v0),
(CrazyClimber-ramDeterministic-v4),
(CrazyClimber-ramNoFrameskip-v0),
(CrazyClimber-ramNoFrameskip-v4),
(CrazyClimber-v0), 小游戏2d攀爬
(CrazyClimber-v4),
(CrazyClimberDeterministic-v0),
(CrazyClimberDeterministic-v4),
(CrazyClimberNoFrameskip-v0),
(CrazyClimberNoFrameskip-v4),
(DemonAttack-ram-v0),
(DemonAttack-ram-v4),
(DemonAttack-ramDeterministic-v0),
(DemonAttack-ramDeterministic-v4),
(DemonAttack-ramNoFrameskip-v0),
(DemonAttack-ramNoFrameskip-v4),
(DemonAttack-v0),
(DemonAttack-v4),
(DemonAttackDeterministic-v0),
(DemonAttackDeterministic-v4),
(DemonAttackNoFrameskip-v0),
(DemonAttackNoFrameskip-v4),
(DoubleDunk-ram-v0),
(DoubleDunk-ram-v4),
(DoubleDunk-ramDeterministic-v0),
(DoubleDunk-ramDeterministic-v4),
(DoubleDunk-ramNoFrameskip-v0),
(DoubleDunk-ramNoFrameskip-v4),
(DoubleDunk-v0),
(DoubleDunk-v4),
(DoubleDunkDeterministic-v0),
(DoubleDunkDeterministic-v4),
(DoubleDunkNoFrameskip-v0),
(DoubleDunkNoFrameskip-v4), 投篮小游戏
(DuplicatedInput-v0),
(ElevatorAction-ram-v0),
(ElevatorAction-ram-v4),
(ElevatorAction-ramDeterministic-v0),
(ElevatorAction-ramDeterministic-v4),
(ElevatorAction-ramNoFrameskip-v0),
(ElevatorAction-ramNoFrameskip-v4),
(ElevatorAction-v0),
(ElevatorAction-v4),
(ElevatorActionDeterministic-v0),
(ElevatorActionDeterministic-v4),
(ElevatorActionNoFrameskip-v0),
(ElevatorActionNoFrameskip-v4),
(Enduro-ram-v0),
(Enduro-ram-v4),
(Enduro-ramDeterministic-v0), 小游戏赛车
(Enduro-ramDeterministic-v4),
(Enduro-ramNoFrameskip-v0),
(Enduro-ramNoFrameskip-v4),
(Enduro-v0),
(Enduro-v4), 2d赛车
(EnduroDeterministic-v0),
(EnduroDeterministic-v4),
(EnduroNoFrameskip-v0),
(EnduroNoFrameskip-v4),
(FishingDerby-ram-v0),
(FishingDerby-ram-v4),
(FishingDerby-ramDeterministic-v0),
(FishingDerby-ramDeterministic-v4),
(FishingDerby-ramNoFrameskip-v0),
(FishingDerby-ramNoFrameskip-v4),
(FishingDerby-v0),小游戏钓鱼
(FishingDerby-v4),
(FishingDerbyDeterministic-v0),
(FishingDerbyDeterministic-v4),
(FishingDerbyNoFrameskip-v0),
(FishingDerbyNoFrameskip-v4),
(Freeway-ram-v0),
(Freeway-ram-v4),
(Freeway-ramDeterministic-v0),
(Freeway-ramDeterministic-v4),
(Freeway-ramNoFrameskip-v0),
(Freeway-ramNoFrameskip-v4),
(Freeway-v0),
(Freeway-v4),
(FreewayDeterministic-v0),
(FreewayDeterministic-v4),
(FreewayNoFrameskip-v0),
(FreewayNoFrameskip-v4),
(Frostbite-ram-v0), 小游戏类似跳水
(Frostbite-ram-v4),小游戏类似跳水
(Frostbite-ramDeterministic-v0),
(Frostbite-ramDeterministic-v4),
(Frostbite-ramNoFrameskip-v0),
(Frostbite-ramNoFrameskip-v4),
(Frostbite-v0),
(Frostbite-v4),
(FrostbiteDeterministic-v0),
(FrostbiteDeterministic-v4),
(FrostbiteNoFrameskip-v0), 小游戏
(FrostbiteNoFrameskip-v4),
(FrozenLake-v0),
(FrozenLake8x8-v0),
(Go19x19-v0),
(Go9x9-v0),
(Gopher-ram-v0), 小游戏鱼儿
(Gopher-ram-v4), 小游戏鱼儿
(Gopher-ramDeterministic-v0),小游戏鱼儿
(Gopher-ramDeterministic-v4),
(Gopher-ramNoFrameskip-v0),
(Gopher-ramNoFrameskip-v4), 小游戏鱼儿
(Gopher-v0),
(Gopher-v4),
(GopherDeterministic-v0),
(GopherDeterministic-v4),
(GopherNoFrameskip-v0),
(GopherNoFrameskip-v4),
(Gravitar-ram-v0),
(Gravitar-ram-v4),
(Gravitar-ramDeterministic-v0),
(Gravitar-ramDeterministic-v4),
(Gravitar-ramNoFrameskip-v0),
(Gravitar-ramNoFrameskip-v4),
(Gravitar-v0),
(Gravitar-v4), 2d射击类小游戏
(GravitarDeterministic-v0),
(GravitarDeterministic-v4),
(GravitarNoFrameskip-v0), 小游戏打怪
(GravitarNoFrameskip-v4),
(GuessingGame-v0),
(HalfCheetah-v2), 错误
(Hero-ram-v0),
(Hero-ram-v4),
(Hero-ramDeterministic-v0),
(Hero-ramDeterministic-v4),
(Hero-ramNoFrameskip-v0), 小游戏类似挖宝藏
(Hero-ramNoFrameskip-v4),
(Hero-v0),
(Hero-v4),
(HeroDeterministic-v0),
(HeroDeterministic-v4), 小游戏类似挖宝藏
(HeroNoFrameskip-v0),
(HeroNoFrameskip-v4),
(Hex9x9-v0),
(Hopper-v2), 错误
(HotterColder-v0),
(Humanoid-v2), 错误
(HumanoidStandup-v2)])
(IceHockey-ram-v0),
(IceHockey-ram-v4),
(IceHockey-ramDeterministic-v0),
(IceHockey-ramDeterministic-v4),2d小游戏
(IceHockey-ramNoFrameskip-v0), 打冰球
(IceHockey-ramNoFrameskip-v4),
(IceHockey-v0),
(IceHockey-v4),
(IceHockeyDeterministic-v0), 冰球
(IceHockeyDeterministic-v4),
(IceHockeyNoFrameskip-v0),
(IceHockeyNoFrameskip-v4),
(InvertedDoublePendulum-v2),
(InvertedPendulum-v2),
(Jamesbond-ram-v0),
(Jamesbond-ram-v4), 射击打怪小游戏
(Jamesbond-ramDeterministic-v0),
(Jamesbond-ramDeterministic-v4),
(Jamesbond-ramNoFrameskip-v0),
(Jamesbond-ramNoFrameskip-v4),
(Jamesbond-v0),
(Jamesbond-v4),
(JamesbondDeterministic-v0), 2d射击打怪
(JamesbondDeterministic-v4),
(JamesbondNoFrameskip-v0),
(JamesbondNoFrameskip-v4),
(JourneyEscape-ram-v0),
(JourneyEscape-ram-v4),
(JourneyEscape-ramDeterministic-v0),
(JourneyEscape-ramDeterministic-v4),
(JourneyEscape-ramNoFrameskip-v0),
(JourneyEscape-ramNoFrameskip-v4),
(JourneyEscape-v0),
(JourneyEscape-v4),
(JourneyEscapeDeterministic-v0),
(JourneyEscapeDeterministic-v4),飞行射击类小游戏
(JourneyEscapeNoFrameskip-v0),
(JourneyEscapeNoFrameskip-v4),
(Kangaroo-ram-v0),
(Kangaroo-ram-v4),
(Kangaroo-ramDeterministic-v0),
(Kangaroo-ramDeterministic-v4),
(Kangaroo-ramNoFrameskip-v0),
(Kangaroo-ramNoFrameskip-v4),
(Kangaroo-v0),
(Kangaroo-v4), 小游戏寻宝
(KangarooDeterministic-v0),
(KangarooDeterministic-v4),
(KangarooNoFrameskip-v0),
(KangarooNoFrameskip-v4),
(KellyCoinflip-v0),
(KellyCoinflipGeneralized-v0),
(Krull-ram-v0),小游戏
(Krull-ram-v4),
(Krull-ramDeterministic-v0), 小游戏
(Krull-ramDeterministic-v4),
(Krull-ramNoFrameskip-v0),小游戏打怪
(Krull-ramNoFrameskip-v4),
(Krull-v0),
(Krull-v4),
(KrullDeterministic-v0),
(KrullDeterministic-v4), 小游戏
(KrullNoFrameskip-v0),
(KrullNoFrameskip-v4),
(KungFuMaster-ram-v0),
(KungFuMaster-ram-v4),
(KungFuMaster-ramDeterministic-v0),
(KungFuMaster-ramDeterministic-v4),
(KungFuMaster-ramNoFrameskip-v0),
(KungFuMaster-ramNoFrameskip-v4),
(KungFuMaster-v0),
(KungFuMaster-v4),
(KungFuMasterDeterministic-v0),
(KungFuMasterDeterministic-v4),
(KungFuMasterNoFrameskip-v0),
(KungFuMasterNoFrameskip-v4),
(LunarLander-v2),
(LunarLanderContinuous-v2),
(MontezumaRevenge-ram-v0),
(MontezumaRevenge-ram-v4),
(MontezumaRevenge-ramDeterministic-v0),
(MontezumaRevenge-ramDeterministic-v4),
(MontezumaRevenge-ramNoFrameskip-v0),
(MontezumaRevenge-ramNoFrameskip-v4),
(MontezumaRevenge-v0),
(MontezumaRevenge-v4),
(MontezumaRevengeDeterministic-v0),
(MontezumaRevengeDeterministic-v4),
(MontezumaRevengeNoFrameskip-v0),
(MontezumaRevengeNoFrameskip-v4),
(MountainCar-v0),
(MountainCarContinuous-v0),
(MsPacman-ram-v0),
(MsPacman-ram-v4), 小游戏走迷宫
(MsPacman-ramDeterministic-v0),
(MsPacman-ramDeterministic-v4),
(MsPacman-ramNoFrameskip-v0),
(MsPacman-ramNoFrameskip-v4),
(MsPacman-v0),
(MsPacman-v4),
(MsPacmanDeterministic-v0),
(MsPacmanDeterministic-v4),
(MsPacmanNoFrameskip-v0),
(MsPacmanNoFrameskip-v4),
(NChain-v0),
(NameThisGame-ram-v0),
(NameThisGame-ram-v4),
(NameThisGame-ramDeterministic-v0),
(NameThisGame-ramDeterministic-v4),
(NameThisGame-ramNoFrameskip-v0),2d打怪
(NameThisGame-ramNoFrameskip-v4),
(NameThisGame-v0),
(NameThisGame-v4),
(NameThisGameDeterministic-v0),
(NameThisGameDeterministic-v4),
(NameThisGameNoFrameskip-v0),
(NameThisGameNoFrameskip-v4),
(OffSwitchCartpole-v0),
(OffSwitchCartpoleProb-v0),
(OneRoundDeterministicReward-v0),
(OneRoundNondeterministicReward-v0),
(Pendulum-v0),
(Phoenix-ram-v0),
(Phoenix-ram-v4),
(Phoenix-ramDeterministic-v0),
(Phoenix-ramDeterministic-v4),
(Phoenix-ramNoFrameskip-v0),
(Phoenix-ramNoFrameskip-v4),
(Phoenix-v0),
(Phoenix-v4),
(PhoenixDeterministic-v0),
(PhoenixDeterministic-v4), 2d移动打怪小游戏
(PhoenixNoFrameskip-v0),
(PhoenixNoFrameskip-v4),
(Pitfall-ram-v0),
(Pitfall-ram-v4),
(Pitfall-ramDeterministic-v0),
(Pitfall-ramDeterministic-v4),
(Pitfall-ramNoFrameskip-v0),
(Pitfall-ramNoFrameskip-v4),
(Pitfall-v0),
(Pitfall-v4), 小游戏
(PitfallDeterministic-v0),
(PitfallDeterministic-v4),
(PitfallNoFrameskip-v0),
(PitfallNoFrameskip-v4),
(Pong-ram-v0),
(Pong-ram-v4),
(Pong-ramDeterministic-v0),
(Pong-ramDeterministic-v4),
(Pong-ramNoFrameskip-v0),
(Pong-ramNoFrameskip-v4),
(Pong-v0),
(Pong-v4), 小游戏乒乓球
(PongDeterministic-v0),
(PongDeterministic-v4),
(PongNoFrameskip-v0),
(PongNoFrameskip-v4),
(Pooyan-ram-v0),
(Pooyan-ram-v4), 小游戏
(Pooyan-ramDeterministic-v0),
(Pooyan-ramDeterministic-v4),
(Pooyan-ramNoFrameskip-v0),
(Pooyan-ramNoFrameskip-v4),
(Pooyan-v0), 2d飞行小游戏
(Pooyan-v4),
(PooyanDeterministic-v0),
(PooyanDeterministic-v4),
(PooyanNoFrameskip-v0), 2d小游戏打怪
(PooyanNoFrameskip-v4), 2d打怪小游戏
(PredictActionsCartpole-v0),
(PredictObsCartpole-v0),
(PrivateEye-ram-v0),
(PrivateEye-ram-v4),
(PrivateEye-ramDeterministic-v0),
(PrivateEye-ramDeterministic-v4),
(PrivateEye-ramNoFrameskip-v0),
(PrivateEye-ramNoFrameskip-v4),
(PrivateEye-v0),
(PrivateEye-v4),
(PrivateEyeDeterministic-v0),
(PrivateEyeDeterministic-v4),
(PrivateEyeNoFrameskip-v0),
(PrivateEyeNoFrameskip-v4),
(Pusher-v2),
(Qbert-ram-v0),
(Qbert-ram-v4),
(Qbert-ramDeterministic-v0),
(Qbert-ramDeterministic-v4),
(Qbert-ramNoFrameskip-v0),
(Qbert-ramNoFrameskip-v4),
(Qbert-v0),
(Qbert-v4),
(QbertDeterministic-v0),
(QbertDeterministic-v4),
(QbertNoFrameskip-v0),
(QbertNoFrameskip-v4),小游戏跳塔
(Reacher-v2),
(RepeatCopy-v0),
(Reverse-v0),
(ReversedAddition-v0),
(ReversedAddition3-v0),
(Riverraid-ram-v0),
(Riverraid-ram-v4),
(Riverraid-ramDeterministic-v0),
(Riverraid-ramNoFrameskip-v0),
(Riverraid-ramNoFrameskip-v4),
(Riverraid-v0),
(Riverraid-v4),
(RiverraidDeterministic-v0),
(RiverraidDeterministic-v4),
(RiverraidNoFrameskip-v0),
(RiverraidNoFrameskip-v4),
(RoadRunner-ram-v0), 2d博弈
(RoadRunner-ram-v4),
(RoadRunner-ramDeterministic-v0),
(RoadRunner-ramDeterministic-v4),
(RoadRunner-ramNoFrameskip-v0),
(RoadRunner-ramNoFrameskip-v4),
(RoadRunner-v0), 小游戏博弈
(RoadRunner-v4),
(RoadRunnerDeterministic-v0),
(RoadRunnerDeterministic-v4),
(RoadRunnerNoFrameskip-v0),
(RoadRunnerNoFrameskip-v4),
(Robotank-ram-v0), 小游戏打靶
(Robotank-ram-v4),
(Robotank-ramDeterministic-v0),
(Robotank-ramDeterministic-v4),
(Robotank-ramNoFrameskip-v0),
(Robotank-ramNoFrameskip-v4),
(Robotank-v0),
(Robotank-v4),
(RobotankDeterministic-v0),
(RobotankDeterministic-v4),
(RobotankNoFrameskip-v0),
(RobotankNoFrameskip-v4),
(Roulette-v0), 错误
(Seaquest-ram-v0), 小游戏2d左右打怪
(Seaquest-ram-v4),
(Seaquest-ramDeterministic-v0),
(Seaquest-ramDeterministic-v4),
(Seaquest-ramNoFrameskip-v0),
(Seaquest-ramNoFrameskip-v4),
(Seaquest-v0),
(Seaquest-v4),
(SeaquestDeterministic-v0),
(SeaquestDeterministic-v4),
(SeaquestNoFrameskip-v0),
(SeaquestNoFrameskip-v4),
(SemisuperPendulumDecay-v0),
(SemisuperPendulumNoise-v0),
(SemisuperPendulumRandom-v0),
(Skiing-ram-v0),
(Skiing-ram-v4),
(Skiing-ramDeterministic-v0),
(Skiing-ramDeterministic-v4),
(Skiing-ramNoFrameskip-v0),
(Skiing-ramNoFrameskip-v4),
(Skiing-v0), 小游戏滑雪
(Skiing-v4),
(SkiingDeterministic-v0),
(SkiingDeterministic-v4),
(SkiingNoFrameskip-v0),
(SkiingNoFrameskip-v4),
(Solaris-ram-v0),
(Solaris-ram-v4),
(Solaris-ramDeterministic-v0),
(Solaris-ramDeterministic-v4),
(Solaris-ramNoFrameskip-v0),
(Solaris-ramNoFrameskip-v4),
(Solaris-v0),
(Solaris-v4),
(SolarisDeterministic-v0),
(SolarisDeterministic-v4),
(SolarisNoFrameskip-v0),
(SolarisNoFrameskip-v4),2d飞行类似小游戏
(SpaceInvaders-ram-v0),
(SpaceInvaders-ram-v4),
(SpaceInvaders-ramDeterministic-v0),
(SpaceInvaders-ramDeterministic-v4),
(SpaceInvaders-ramNoFrameskip-v0),
(SpaceInvaders-ramNoFrameskip-v4),
(SpaceInvaders-v0),
(SpaceInvaders-v4),
(SpaceInvadersDeterministic-v0),
(SpaceInvadersDeterministic-v4),
(SpaceInvadersNoFrameskip-v0),
(SpaceInvadersNoFrameskip-v4),
(StarGunner-ram-v0),
(StarGunner-ram-v4),
(StarGunner-ramDeterministic-v0),
(StarGunner-ramDeterministic-v4),
(StarGunner-ramNoFrameskip-v0),
(StarGunner-ramNoFrameskip-v4),
(StarGunner-v0),
(StarGunner-v4),
(StarGunnerDeterministic-v0),小游戏2d飞行
(StarGunnerDeterministic-v4),
(StarGunnerNoFrameskip-v0),
(StarGunnerNoFrameskip-v4),
(Striker-v2),
(Swimmer-v2),
(Taxi-v2),
(Tennis-ram-v0),
(Tennis-ram-v4),
(Tennis-ramDeterministic-v0),
(Tennis-ramDeterministic-v4),
(Tennis-ramNoFrameskip-v0),
(Tennis-ramNoFrameskip-v4), 网球小游戏
(Tennis-v0), 网球
(Tennis-v4),
(TennisDeterministic-v0), 小游戏打网球
(TennisDeterministic-v4), 网球小游戏
(TennisNoFrameskip-v0),
(TennisNoFrameskip-v4),
(Thrower-v2),
(TimePilot-ram-v0),
(TimePilot-ram-v4),
(TimePilot-ramDeterministic-v0), 小游戏飞云打怪
(TimePilot-ramDeterministic-v4),
(TimePilot-ramNoFrameskip-v0),
(TimePilot-ramNoFrameskip-v4),
(TimePilot-v0),
(TimePilot-v4),
(TimePilotDeterministic-v0), 小游戏飞云打怪
(TimePilotDeterministic-v4),
(TimePilotNoFrameskip-v0),
(TimePilotNoFrameskip-v4),
(Tutankham-ram-v0),
(Tutankham-ram-v4), 小游戏
(Tutankham-ramDeterministic-v0),
(Tutankham-ramDeterministic-v4),
(Tutankham-ramNoFrameskip-v0),
(Tutankham-ramNoFrameskip-v4),
(Tutankham-v0), 小游戏打怪
(Tutankham-v4),
(TutankhamDeterministic-v0),
(TutankhamDeterministic-v4),
(TutankhamNoFrameskip-v0),
(TutankhamNoFrameskip-v4),
(TwoRoundDeterministicReward-v0),
(TwoRoundNondeterministicReward-v0),
(UpNDown-ram-v0),
(UpNDown-ram-v4),
(UpNDown-ramDeterministic-v0), 小路上开车小游戏
(UpNDown-ramDeterministic-v4),
(UpNDown-ramNoFrameskip-v0),
(UpNDown-ramNoFrameskip-v4),
(UpNDown-v0), 小路上开车小游戏
(UpNDown-v4),
(UpNDownDeterministic-v0),
(UpNDownDeterministic-v4),
(UpNDownNoFrameskip-v0),
(UpNDownNoFrameskip-v4),
(Venture-ram-v0),
(Venture-ram-v4),
(Venture-ramDeterministic-v0),
(Venture-ramDeterministic-v4),
(Venture-ramNoFrameskip-v0),
(Venture-ramNoFrameskip-v4),
(Venture-v0),
(Venture-v4),
(VentureDeterministic-v0),
(VentureDeterministic-v4),
(VentureNoFrameskip-v0),
(VentureNoFrameskip-v4),
(VideoPinball-ram-v0),
(VideoPinball-ram-v4),
(VideoPinball-ramDeterministic-v0),
(VideoPinball-ramDeterministic-v4),
(VideoPinball-ramNoFrameskip-v0),
(VideoPinball-ramNoFrameskip-v4),
(VideoPinball-v0),
(VideoPinball-v4),小游戏弹球
(VideoPinballDeterministic-v0),
(VideoPinballDeterministic-v4),
(VideoPinballNoFrameskip-v0),
(VideoPinballNoFrameskip-v4),
(Walker2d-v2), 错误
(WizardOfWor-ram-v0),
(WizardOfWor-ram-v4),
(WizardOfWor-ramDeterministic-v0),
(WizardOfWor-ramDeterministic-v4),
(WizardOfWor-ramNoFrameskip-v0),
(WizardOfWor-ramNoFrameskip-v4),
(WizardOfWor-v0),
(WizardOfWor-v4), 小游戏迷宫
(WizardOfWorDeterministic-v0),
(WizardOfWorDeterministic-v4),
(WizardOfWorNoFrameskip-v0),
(WizardOfWorNoFrameskip-v4),
(YarsRevenge-ram-v0),
(YarsRevenge-ram-v4),
(YarsRevenge-ramDeterministic-v0),
(YarsRevenge-ramDeterministic-v4),
(YarsRevenge-ramNoFrameskip-v0),
(YarsRevenge-ramNoFrameskip-v4),
(YarsRevenge-v0),
(YarsRevenge-v4), 小游戏2d飞行打怪
(YarsRevengeDeterministic-v0),
(YarsRevengeDeterministic-v4),
(YarsRevengeNoFrameskip-v0),
(YarsRevengeNoFrameskip-v4), 2d打怪小游戏
(Zaxxon-ram-v0),
(Zaxxon-ram-v4), 小游戏打怪
(Zaxxon-ramDeterministic-v0),
(Zaxxon-ramDeterministic-v4),
(Zaxxon-ramNoFrameskip-v0),小游戏越过墙打怪
(Zaxxon-ramNoFrameskip-v4),
(Zaxxon-v0),
(Zaxxon-v4),
(ZaxxonDeterministic-v0),
(ZaxxonDeterministic-v4),
(ZaxxonNoFrameskip-v0),小游戏打怪
(ZaxxonNoFrameskip-v4),

让AI先在人类的操作下学习，然后再自行学习，效果会不会好点？？？