本文内容源自百度强化学习 7 日入门课程学习整理
感谢百度 PARL 团队李科浇老师的课程讲解

强化学习算法 Sarsa 解迷宫游戏

文章目录

  • 一、安装依赖库
  • 二、导入依赖库
  • 三、智能体 Agent 的算法：Sarsa
  • 四、训练和测试语句
  • 五、创建环境，实例化Agent，启动训练和测试
  • 五、结果分析

一、安装依赖库

安装强化学习算法中环境库 Gym

pip install gym

二、导入依赖库

import gym
import numpy as np
import time # 用于延时程序，方便渲染画面

三、智能体 Agent 的算法：Sarsa

  • 智能体 Agent 是和环境 environment 交互的主体
    • 包含了观察当前状态
    • 根据当前状态作出动作选择
    • 根据选择后的结果更新 Q 值表
  • predict() 方法：输入观察值 observation（或者说状态state），输出 “预测” 动作 action （最优动作）
    • 观察当前状态下，所有可以采用的 action 对应的 Q 值
    • 在其中选取最大的，组成一个列表
    • 该列表对应可能选取的最优动作列表
    • 在最优动作列表中随机选取一个动作
  • sample() 方法：在 predict() 方法基础上使用 ε-greedy 增加探索，输出 “实际” 动作 action
    • 采用 epsilon greedy 算法
    • 90% 概率选择最优动作
    • 10% 概率选择随机动作
  • learn() 方法：输入训练数据，完成一轮Q表格的更新
    • 更新的是之前状态 obs 下采取动作 action 后的 Q 值
    • 如果游戏结束，则 reward 为新的 Q 值
    • 如果游戏没有结束，则 reward 和下一步的 Q 值结合产生新的 Q 值
    • 同时用学习速率 lr 做更新约束
class SarsaAgent(object):def __init__(self, obs_n, act_n, learning_rate=0.01, gamma=0.9, e_greed=0.1):self.act_n = act_n      # 动作维度，有几个动作可选self.lr = learning_rate # 学习率self.gamma = gamma      # 后面的 Q 值对前面的影响self.epsilon = e_greed  # 按一定概率随机选动作self.Q = np.zeros((obs_n, act_n))# 根据输入观察值，采样输出的动作值（带 10% 的探索）def sample(self, obs):if (np.random.uniform(0, 1) < 1 - self.epsilon): # 这里是 90% 可能性action = self.predict(obs) # 执行最优动作else: # 10% 的概率action = np.random.choice(self.act_n) # 执行随机动作return action# 根据输入观察值，预测输出的动作值def predict(self, obs):Q_list = self.Q[obs, :] # 获取当前状态下，作出所有动作，对应的 Q 值列表maxQ = np.max(Q_list) # 求列表中的最大值action_list = np.where(Q_list == maxQ)[0] # 最大 Q 值对应的动作即最优动作action = np.random.choice(action_list) # 随机选择一个最优动作return action# 学习方法，也就是更新Q-table的方法def learn(self, obs, action, reward, next_obs, next_action, done):""" on-policyobs: 交互前的obs, s_taction: 本次交互选择的action, a_treward: 本次动作获得的奖励rnext_obs: 本次交互后的obs, s_t+1next_action: 根据当前Q表格, 针对next_obs会选择的动作, a_t+1done: episode是否结束"""predict_Q = self.Q[obs,action] # 交互前的状态下，选择的动作所对应 Q 值if (done): # 游戏结束target_Q = reward # 新的 Q 值为 rewardelse: # 游戏没有结束target_Q = reward + self.gamma * self.Q[next_obs, next_action]# 用 reward 和 交互后状态下，选择的下一个动作对应的 Q 值，综合得到新的 Q 值self.Q[obs,action] += self.lr * (target_Q - predict_Q) # 使用 lr 做修正更新的幅度# 保存Q表格数据到文件def save(self):npy_file = './q_table.npy'np.save(npy_file, self.Q)print(npy_file + ' saved.')# 从文件中读取数据到Q表格中def restore(self, npy_file='./q_table.npy'):self.Q = np.load(npy_file)print(npy_file + ' loaded.')

四、训练和测试语句

每一局游戏，记录下步数 total_steps 和总奖励 total_reward

每一步都更新 Q 值表

def run_episode(env, agent, render=False):total_steps = 0 # 记录每个episode走了多少steptotal_reward = 0 # 记录每一局游戏的总奖励obs = env.reset() # 重置环境, 重新开一局（即开始新的一个episode）action = agent.sample(obs) # 根据算法选择一个动作while True:next_obs, reward, done, _ = env.step(action) # 与环境进行一个交互，执行动作next_action = agent.sample(next_obs) # 根据算法选择下一个动作# 训练 Sarsa 算法agent.learn(obs, action, reward, next_obs, next_action, done)# obs 执行动作前的状态，action 执行的动作，得到预测的 Q0# reward 执行动作后的奖励，next_obs 执行动作后的状态，next_action 选择的下一个动作，得到更新的 Q0# done 判断游戏是否结束action = next_action # 迭代新的动作obs = next_obs  # 存储上一个观察值，迭代新的状态total_reward += reward # 累计奖励total_steps += 1 # 计算step数if render: # 判断是否需要渲染图形显示env.render() #渲染新的一帧图形if done: # 游戏结束break # 跳出循环，即结束本局游戏return total_reward, total_steps # 返回总的奖励和总的步数def test_episode(env, agent):total_reward = 0 # 记录总的奖励obs = env.reset() # 重置环境，obs 初始观察值，即初始状态while True:action = agent.predict(obs) # greedy，每次选择最优动作next_obs, reward, done, _ = env.step(action) # 交互后，获取新的状态，奖励，游戏是否结束total_reward += reward # 累计奖励obs = next_obs # 迭代更新状态time.sleep(0.5) # 休眠，以便于我们观察渲染的图形env.render() # 渲染图形显示if done: # 游戏结束break # 跳出循环return total_reward # 返回最终累计奖励

五、创建环境，实例化Agent，启动训练和测试

使用 Gym 库创建我们需要的环境

实例化 SarsaAgent 类，创建一个 Agent 对象，同时设定超参数

训练 500 局游戏，查看每一局游戏的结果

训练结束后进行测试

# 使用gym创建迷宫环境，设置is_slippery为False降低环境难度
env = gym.make("FrozenLake-v0", is_slippery=False)  # 0 left, 1 down, 2 right, 3 up
# 使用 make 方法创建需要的环境# 创建一个agent实例，输入超参数
agent = SarsaAgent(obs_n=env.observation_space.n, # 16 个状态代表这个环境中 4*4 一共 16 个格子act_n=env.action_space.n, # 4 种动作选择：0 left, 1 down, 2 right, 3 uplearning_rate=0.1, # 学习速率gamma=0.9, # 下一步的影响率e_greed=0.1) # 随机选择概率# 训练500个episode，打印每个episode的分数
for episode in range(500):ep_reward, ep_steps = run_episode(env, agent, False)print('Episode %s: steps = %s , reward = %.1f' % (episode, ep_steps, ep_reward))# 全部训练结束，查看算法效果
test_reward = test_episode(env, agent)
print('test reward = %.1f' % (test_reward))

运行结果：

Episode 0: steps = 6 , reward = 0.0
Episode 1: steps = 17 , reward = 0.0
Episode 2: steps = 9 , reward = 0.0
Episode 3: steps = 2 , reward = 0.0
Episode 4: steps = 8 , reward = 0.0
Episode 5: steps = 8 , reward = 0.0
Episode 6: steps = 14 , reward = 0.0
Episode 7: steps = 7 , reward = 0.0
Episode 8: steps = 7 , reward = 0.0
Episode 9: steps = 2 , reward = 0.0
Episode 10: steps = 3 , reward = 0.0
Episode 11: steps = 8 , reward = 0.0
Episode 12: steps = 3 , reward = 0.0
Episode 13: steps = 8 , reward = 0.0
Episode 14: steps = 6 , reward = 0.0
Episode 15: steps = 5 , reward = 0.0
Episode 16: steps = 5 , reward = 0.0
Episode 17: steps = 7 , reward = 0.0
Episode 18: steps = 2 , reward = 0.0
Episode 19: steps = 7 , reward = 0.0
Episode 20: steps = 2 , reward = 0.0
Episode 21: steps = 7 , reward = 0.0
Episode 22: steps = 6 , reward = 0.0
Episode 23: steps = 3 , reward = 0.0
Episode 24: steps = 4 , reward = 0.0
Episode 25: steps = 4 , reward = 0.0
Episode 26: steps = 17 , reward = 0.0
Episode 27: steps = 11 , reward = 0.0
Episode 28: steps = 4 , reward = 0.0
Episode 29: steps = 9 , reward = 0.0
Episode 30: steps = 3 , reward = 0.0
Episode 31: steps = 11 , reward = 0.0
Episode 32: steps = 7 , reward = 0.0
Episode 33: steps = 3 , reward = 0.0
Episode 34: steps = 16 , reward = 0.0
Episode 35: steps = 10 , reward = 0.0
Episode 36: steps = 2 , reward = 0.0
Episode 37: steps = 9 , reward = 0.0
Episode 38: steps = 9 , reward = 0.0
Episode 39: steps = 19 , reward = 1.0
Episode 40: steps = 6 , reward = 0.0
Episode 41: steps = 6 , reward = 0.0
Episode 42: steps = 7 , reward = 0.0
Episode 43: steps = 4 , reward = 0.0
Episode 44: steps = 4 , reward = 0.0
Episode 45: steps = 5 , reward = 0.0
Episode 46: steps = 4 , reward = 0.0
Episode 47: steps = 22 , reward = 1.0
Episode 48: steps = 2 , reward = 0.0
Episode 49: steps = 2 , reward = 0.0
Episode 50: steps = 2 , reward = 0.0
Episode 51: steps = 17 , reward = 0.0
Episode 52: steps = 14 , reward = 0.0
Episode 53: steps = 6 , reward = 0.0
Episode 54: steps = 8 , reward = 0.0
Episode 55: steps = 18 , reward = 0.0
Episode 56: steps = 5 , reward = 0.0
Episode 57: steps = 2 , reward = 0.0
Episode 58: steps = 8 , reward = 0.0
Episode 59: steps = 4 , reward = 0.0
Episode 60: steps = 10 , reward = 0.0
Episode 61: steps = 2 , reward = 0.0
Episode 62: steps = 11 , reward = 0.0
Episode 63: steps = 21 , reward = 0.0
Episode 64: steps = 4 , reward = 0.0
Episode 65: steps = 2 , reward = 0.0
Episode 66: steps = 3 , reward = 0.0
Episode 67: steps = 3 , reward = 0.0
Episode 68: steps = 18 , reward = 1.0
Episode 69: steps = 6 , reward = 0.0
Episode 70: steps = 8 , reward = 0.0
Episode 71: steps = 8 , reward = 0.0
Episode 72: steps = 4 , reward = 0.0
Episode 73: steps = 13 , reward = 0.0
Episode 74: steps = 3 , reward = 0.0
Episode 75: steps = 7 , reward = 0.0
Episode 76: steps = 8 , reward = 0.0
Episode 77: steps = 3 , reward = 0.0
Episode 78: steps = 7 , reward = 0.0
Episode 79: steps = 8 , reward = 0.0
Episode 80: steps = 7 , reward = 0.0
Episode 81: steps = 10 , reward = 1.0
Episode 82: steps = 6 , reward = 1.0
Episode 83: steps = 9 , reward = 1.0
Episode 84: steps = 6 , reward = 0.0
Episode 85: steps = 6 , reward = 1.0
Episode 86: steps = 3 , reward = 0.0
Episode 87: steps = 7 , reward = 1.0
Episode 88: steps = 6 , reward = 1.0
Episode 89: steps = 7 , reward = 1.0
Episode 90: steps = 6 , reward = 1.0
Episode 91: steps = 6 , reward = 1.0
Episode 92: steps = 10 , reward = 1.0
Episode 93: steps = 6 , reward = 1.0
Episode 94: steps = 8 , reward = 1.0
Episode 95: steps = 6 , reward = 1.0
Episode 96: steps = 7 , reward = 1.0
Episode 97: steps = 6 , reward = 1.0
Episode 98: steps = 6 , reward = 1.0
Episode 99: steps = 8 , reward = 1.0
Episode 100: steps = 6 , reward = 1.0
Episode 101: steps = 8 , reward = 1.0
Episode 102: steps = 6 , reward = 1.0
Episode 103: steps = 6 , reward = 1.0
Episode 104: steps = 6 , reward = 1.0
Episode 105: steps = 8 , reward = 1.0
Episode 106: steps = 6 , reward = 1.0
Episode 107: steps = 6 , reward = 1.0
Episode 108: steps = 6 , reward = 1.0
Episode 109: steps = 6 , reward = 1.0
Episode 110: steps = 4 , reward = 0.0
Episode 111: steps = 6 , reward = 1.0
Episode 112: steps = 6 , reward = 1.0
Episode 113: steps = 6 , reward = 1.0
Episode 114: steps = 6 , reward = 1.0
Episode 115: steps = 7 , reward = 1.0
Episode 116: steps = 7 , reward = 1.0
Episode 117: steps = 10 , reward = 1.0
Episode 118: steps = 5 , reward = 0.0
Episode 119: steps = 6 , reward = 1.0
Episode 120: steps = 3 , reward = 0.0
Episode 121: steps = 6 , reward = 1.0
Episode 122: steps = 6 , reward = 1.0
Episode 123: steps = 9 , reward = 1.0
Episode 124: steps = 6 , reward = 1.0
Episode 125: steps = 5 , reward = 0.0
Episode 126: steps = 6 , reward = 1.0
Episode 127: steps = 6 , reward = 1.0
Episode 128: steps = 8 , reward = 1.0
Episode 129: steps = 6 , reward = 1.0
Episode 130: steps = 6 , reward = 1.0
Episode 131: steps = 8 , reward = 1.0
Episode 132: steps = 8 , reward = 1.0
Episode 133: steps = 6 , reward = 1.0
Episode 134: steps = 6 , reward = 1.0
Episode 135: steps = 6 , reward = 1.0
Episode 136: steps = 6 , reward = 1.0
Episode 137: steps = 6 , reward = 1.0
Episode 138: steps = 6 , reward = 1.0
Episode 139: steps = 4 , reward = 0.0
Episode 140: steps = 6 , reward = 1.0
Episode 141: steps = 6 , reward = 1.0
Episode 142: steps = 6 , reward = 1.0
Episode 143: steps = 9 , reward = 1.0
Episode 144: steps = 6 , reward = 1.0
Episode 145: steps = 6 , reward = 1.0
Episode 146: steps = 6 , reward = 1.0
Episode 147: steps = 7 , reward = 1.0
Episode 148: steps = 7 , reward = 1.0
Episode 149: steps = 6 , reward = 1.0
Episode 150: steps = 6 , reward = 1.0
Episode 151: steps = 6 , reward = 1.0
Episode 152: steps = 7 , reward = 1.0
Episode 153: steps = 6 , reward = 1.0
Episode 154: steps = 6 , reward = 1.0
Episode 155: steps = 7 , reward = 1.0
Episode 156: steps = 7 , reward = 1.0
Episode 157: steps = 7 , reward = 1.0
Episode 158: steps = 6 , reward = 1.0
Episode 159: steps = 6 , reward = 1.0
Episode 160: steps = 6 , reward = 1.0
Episode 161: steps = 4 , reward = 0.0
Episode 162: steps = 6 , reward = 1.0
Episode 163: steps = 5 , reward = 0.0
Episode 164: steps = 6 , reward = 1.0
Episode 165: steps = 6 , reward = 1.0
Episode 166: steps = 6 , reward = 1.0
Episode 167: steps = 6 , reward = 1.0
Episode 168: steps = 9 , reward = 1.0
Episode 169: steps = 6 , reward = 1.0
Episode 170: steps = 8 , reward = 1.0
Episode 171: steps = 6 , reward = 1.0
Episode 172: steps = 6 , reward = 1.0
Episode 173: steps = 6 , reward = 1.0
Episode 174: steps = 6 , reward = 1.0
Episode 175: steps = 6 , reward = 1.0
Episode 176: steps = 6 , reward = 1.0
Episode 177: steps = 6 , reward = 1.0
Episode 178: steps = 8 , reward = 1.0
Episode 179: steps = 6 , reward = 1.0
Episode 180: steps = 6 , reward = 1.0
Episode 181: steps = 3 , reward = 0.0
Episode 182: steps = 6 , reward = 1.0
Episode 183: steps = 6 , reward = 1.0
Episode 184: steps = 6 , reward = 1.0
Episode 185: steps = 8 , reward = 1.0
Episode 186: steps = 10 , reward = 1.0
Episode 187: steps = 8 , reward = 1.0
Episode 188: steps = 6 , reward = 1.0
Episode 189: steps = 6 , reward = 1.0
Episode 190: steps = 6 , reward = 1.0
Episode 191: steps = 6 , reward = 1.0
Episode 192: steps = 7 , reward = 1.0
Episode 193: steps = 6 , reward = 1.0
Episode 194: steps = 6 , reward = 1.0
Episode 195: steps = 8 , reward = 1.0
Episode 196: steps = 6 , reward = 1.0
Episode 197: steps = 4 , reward = 0.0
Episode 198: steps = 5 , reward = 0.0
Episode 199: steps = 6 , reward = 1.0
Episode 200: steps = 6 , reward = 1.0
Episode 201: steps = 6 , reward = 1.0
Episode 202: steps = 4 , reward = 0.0
Episode 203: steps = 8 , reward = 1.0
Episode 204: steps = 8 , reward = 1.0
Episode 205: steps = 7 , reward = 1.0
Episode 206: steps = 6 , reward = 1.0
Episode 207: steps = 6 , reward = 1.0
Episode 208: steps = 6 , reward = 1.0
Episode 209: steps = 8 , reward = 1.0
Episode 210: steps = 7 , reward = 1.0
Episode 211: steps = 6 , reward = 1.0
Episode 212: steps = 6 , reward = 1.0
Episode 213: steps = 10 , reward = 1.0
Episode 214: steps = 6 , reward = 1.0
Episode 215: steps = 6 , reward = 1.0
Episode 216: steps = 6 , reward = 1.0
Episode 217: steps = 6 , reward = 1.0
Episode 218: steps = 6 , reward = 1.0
Episode 219: steps = 6 , reward = 1.0
Episode 220: steps = 6 , reward = 1.0
Episode 221: steps = 7 , reward = 1.0
Episode 222: steps = 6 , reward = 1.0
Episode 223: steps = 6 , reward = 1.0
Episode 224: steps = 6 , reward = 1.0
Episode 225: steps = 6 , reward = 1.0
Episode 226: steps = 6 , reward = 1.0
Episode 227: steps = 6 , reward = 1.0
Episode 228: steps = 7 , reward = 1.0
Episode 229: steps = 6 , reward = 1.0
Episode 230: steps = 6 , reward = 1.0
Episode 231: steps = 10 , reward = 1.0
Episode 232: steps = 6 , reward = 1.0
Episode 233: steps = 6 , reward = 1.0
Episode 234: steps = 6 , reward = 1.0
Episode 235: steps = 8 , reward = 1.0
Episode 236: steps = 6 , reward = 1.0
Episode 237: steps = 6 , reward = 1.0
Episode 238: steps = 6 , reward = 1.0
Episode 239: steps = 8 , reward = 1.0
Episode 240: steps = 6 , reward = 1.0
Episode 241: steps = 6 , reward = 1.0
Episode 242: steps = 8 , reward = 1.0
Episode 243: steps = 2 , reward = 0.0
Episode 244: steps = 6 , reward = 1.0
Episode 245: steps = 6 , reward = 1.0
Episode 246: steps = 6 , reward = 1.0
Episode 247: steps = 6 , reward = 1.0
Episode 248: steps = 6 , reward = 1.0
Episode 249: steps = 6 , reward = 1.0
Episode 250: steps = 7 , reward = 1.0
Episode 251: steps = 6 , reward = 1.0
Episode 252: steps = 2 , reward = 0.0
Episode 253: steps = 6 , reward = 1.0
Episode 254: steps = 6 , reward = 1.0
Episode 255: steps = 6 , reward = 1.0
Episode 256: steps = 8 , reward = 1.0
Episode 257: steps = 6 , reward = 1.0
Episode 258: steps = 6 , reward = 1.0
Episode 259: steps = 7 , reward = 1.0
Episode 260: steps = 6 , reward = 1.0
Episode 261: steps = 6 , reward = 1.0
Episode 262: steps = 7 , reward = 1.0
Episode 263: steps = 6 , reward = 1.0
Episode 264: steps = 6 , reward = 1.0
Episode 265: steps = 6 , reward = 1.0
Episode 266: steps = 6 , reward = 1.0
Episode 267: steps = 7 , reward = 1.0
Episode 268: steps = 6 , reward = 1.0
Episode 269: steps = 6 , reward = 1.0
Episode 270: steps = 6 , reward = 1.0
Episode 271: steps = 6 , reward = 1.0
Episode 272: steps = 6 , reward = 1.0
Episode 273: steps = 7 , reward = 1.0
Episode 274: steps = 3 , reward = 0.0
Episode 275: steps = 8 , reward = 1.0
Episode 276: steps = 7 , reward = 1.0
Episode 277: steps = 4 , reward = 0.0
Episode 278: steps = 6 , reward = 1.0
Episode 279: steps = 4 , reward = 0.0
Episode 280: steps = 7 , reward = 1.0
Episode 281: steps = 6 , reward = 1.0
Episode 282: steps = 6 , reward = 1.0
Episode 283: steps = 6 , reward = 1.0
Episode 284: steps = 6 , reward = 1.0
Episode 285: steps = 7 , reward = 1.0
Episode 286: steps = 8 , reward = 1.0
Episode 287: steps = 6 , reward = 1.0
Episode 288: steps = 5 , reward = 0.0
Episode 289: steps = 8 , reward = 1.0
Episode 290: steps = 7 , reward = 1.0
Episode 291: steps = 8 , reward = 1.0
Episode 292: steps = 4 , reward = 0.0
Episode 293: steps = 6 , reward = 1.0
Episode 294: steps = 9 , reward = 1.0
Episode 295: steps = 6 , reward = 1.0
Episode 296: steps = 6 , reward = 1.0
Episode 297: steps = 6 , reward = 0.0
Episode 298: steps = 6 , reward = 1.0
Episode 299: steps = 6 , reward = 1.0
Episode 300: steps = 6 , reward = 1.0
Episode 301: steps = 5 , reward = 0.0
Episode 302: steps = 6 , reward = 1.0
Episode 303: steps = 7 , reward = 1.0
Episode 304: steps = 6 , reward = 1.0
Episode 305: steps = 8 , reward = 1.0
Episode 306: steps = 6 , reward = 1.0
Episode 307: steps = 6 , reward = 1.0
Episode 308: steps = 6 , reward = 1.0
Episode 309: steps = 6 , reward = 1.0
Episode 310: steps = 4 , reward = 0.0
Episode 311: steps = 7 , reward = 1.0
Episode 312: steps = 8 , reward = 1.0
Episode 313: steps = 7 , reward = 1.0
Episode 314: steps = 6 , reward = 1.0
Episode 315: steps = 6 , reward = 1.0
Episode 316: steps = 7 , reward = 1.0
Episode 317: steps = 6 , reward = 1.0
Episode 318: steps = 6 , reward = 1.0
Episode 319: steps = 6 , reward = 1.0
Episode 320: steps = 6 , reward = 1.0
Episode 321: steps = 6 , reward = 1.0
Episode 322: steps = 7 , reward = 1.0
Episode 323: steps = 6 , reward = 1.0
Episode 324: steps = 6 , reward = 1.0
Episode 325: steps = 6 , reward = 1.0
Episode 326: steps = 6 , reward = 1.0
Episode 327: steps = 6 , reward = 1.0
Episode 328: steps = 6 , reward = 1.0
Episode 329: steps = 6 , reward = 1.0
Episode 330: steps = 6 , reward = 1.0
Episode 331: steps = 6 , reward = 1.0
Episode 332: steps = 6 , reward = 1.0
Episode 333: steps = 6 , reward = 1.0
Episode 334: steps = 3 , reward = 0.0
Episode 335: steps = 6 , reward = 1.0
Episode 336: steps = 6 , reward = 1.0
Episode 337: steps = 4 , reward = 0.0
Episode 338: steps = 6 , reward = 1.0
Episode 339: steps = 8 , reward = 1.0
Episode 340: steps = 6 , reward = 1.0
Episode 341: steps = 6 , reward = 1.0
Episode 342: steps = 6 , reward = 1.0
Episode 343: steps = 6 , reward = 1.0
Episode 344: steps = 6 , reward = 1.0
Episode 345: steps = 6 , reward = 1.0
Episode 346: steps = 6 , reward = 1.0
Episode 347: steps = 6 , reward = 1.0
Episode 348: steps = 6 , reward = 1.0
Episode 349: steps = 6 , reward = 1.0
Episode 350: steps = 6 , reward = 1.0
Episode 351: steps = 7 , reward = 1.0
Episode 352: steps = 6 , reward = 1.0
Episode 353: steps = 10 , reward = 1.0
Episode 354: steps = 3 , reward = 0.0
Episode 355: steps = 7 , reward = 1.0
Episode 356: steps = 7 , reward = 1.0
Episode 357: steps = 6 , reward = 1.0
Episode 358: steps = 2 , reward = 0.0
Episode 359: steps = 6 , reward = 1.0
Episode 360: steps = 6 , reward = 1.0
Episode 361: steps = 6 , reward = 1.0
Episode 362: steps = 7 , reward = 1.0
Episode 363: steps = 8 , reward = 1.0
Episode 364: steps = 6 , reward = 1.0
Episode 365: steps = 2 , reward = 0.0
Episode 366: steps = 6 , reward = 1.0
Episode 367: steps = 5 , reward = 0.0
Episode 368: steps = 6 , reward = 1.0
Episode 369: steps = 6 , reward = 1.0
Episode 370: steps = 6 , reward = 1.0
Episode 371: steps = 6 , reward = 1.0
Episode 372: steps = 6 , reward = 1.0
Episode 373: steps = 6 , reward = 1.0
Episode 374: steps = 8 , reward = 1.0
Episode 375: steps = 9 , reward = 1.0
Episode 376: steps = 6 , reward = 0.0
Episode 377: steps = 6 , reward = 1.0
Episode 378: steps = 6 , reward = 1.0
Episode 379: steps = 8 , reward = 1.0
Episode 380: steps = 6 , reward = 1.0
Episode 381: steps = 6 , reward = 1.0
Episode 382: steps = 6 , reward = 1.0
Episode 383: steps = 6 , reward = 1.0
Episode 384: steps = 6 , reward = 1.0
Episode 385: steps = 6 , reward = 1.0
Episode 386: steps = 8 , reward = 1.0
Episode 387: steps = 6 , reward = 1.0
Episode 388: steps = 6 , reward = 1.0
Episode 389: steps = 2 , reward = 0.0
Episode 390: steps = 6 , reward = 1.0
Episode 391: steps = 6 , reward = 1.0
Episode 392: steps = 6 , reward = 1.0
Episode 393: steps = 6 , reward = 1.0
Episode 394: steps = 7 , reward = 1.0
Episode 395: steps = 6 , reward = 1.0
Episode 396: steps = 6 , reward = 1.0
Episode 397: steps = 6 , reward = 1.0
Episode 398: steps = 6 , reward = 1.0
Episode 399: steps = 7 , reward = 1.0
Episode 400: steps = 6 , reward = 1.0
Episode 401: steps = 6 , reward = 1.0
Episode 402: steps = 6 , reward = 1.0
Episode 403: steps = 6 , reward = 1.0
Episode 404: steps = 8 , reward = 1.0
Episode 405: steps = 6 , reward = 1.0
Episode 406: steps = 6 , reward = 1.0
Episode 407: steps = 6 , reward = 1.0
Episode 408: steps = 6 , reward = 1.0
Episode 409: steps = 6 , reward = 1.0
Episode 410: steps = 6 , reward = 1.0
Episode 411: steps = 6 , reward = 1.0
Episode 412: steps = 6 , reward = 1.0
Episode 413: steps = 6 , reward = 1.0
Episode 414: steps = 6 , reward = 1.0
Episode 415: steps = 9 , reward = 1.0
Episode 416: steps = 6 , reward = 1.0
Episode 417: steps = 4 , reward = 0.0
Episode 418: steps = 6 , reward = 1.0
Episode 419: steps = 6 , reward = 1.0
Episode 420: steps = 7 , reward = 1.0
Episode 421: steps = 6 , reward = 1.0
Episode 422: steps = 6 , reward = 1.0
Episode 423: steps = 10 , reward = 1.0
Episode 424: steps = 6 , reward = 1.0
Episode 425: steps = 6 , reward = 1.0
Episode 426: steps = 8 , reward = 1.0
Episode 427: steps = 6 , reward = 1.0
Episode 428: steps = 9 , reward = 1.0
Episode 429: steps = 6 , reward = 1.0
Episode 430: steps = 4 , reward = 0.0
Episode 431: steps = 6 , reward = 1.0
Episode 432: steps = 6 , reward = 1.0
Episode 433: steps = 6 , reward = 1.0
Episode 434: steps = 6 , reward = 1.0
Episode 435: steps = 8 , reward = 1.0
Episode 436: steps = 6 , reward = 1.0
Episode 437: steps = 6 , reward = 1.0
Episode 438: steps = 6 , reward = 1.0
Episode 439: steps = 8 , reward = 1.0
Episode 440: steps = 2 , reward = 0.0
Episode 441: steps = 6 , reward = 1.0
Episode 442: steps = 10 , reward = 1.0
Episode 443: steps = 6 , reward = 1.0
Episode 444: steps = 6 , reward = 1.0
Episode 445: steps = 8 , reward = 1.0
Episode 446: steps = 6 , reward = 1.0
Episode 447: steps = 6 , reward = 1.0
Episode 448: steps = 5 , reward = 0.0
Episode 449: steps = 6 , reward = 1.0
Episode 450: steps = 8 , reward = 1.0
Episode 451: steps = 6 , reward = 1.0
Episode 452: steps = 8 , reward = 1.0
Episode 453: steps = 8 , reward = 1.0
Episode 454: steps = 7 , reward = 1.0
Episode 455: steps = 5 , reward = 0.0
Episode 456: steps = 6 , reward = 1.0
Episode 457: steps = 6 , reward = 1.0
Episode 458: steps = 8 , reward = 1.0
Episode 459: steps = 8 , reward = 1.0
Episode 460: steps = 10 , reward = 1.0
Episode 461: steps = 8 , reward = 1.0
Episode 462: steps = 7 , reward = 1.0
Episode 463: steps = 7 , reward = 1.0
Episode 464: steps = 6 , reward = 1.0
Episode 465: steps = 6 , reward = 1.0
Episode 466: steps = 6 , reward = 1.0
Episode 467: steps = 6 , reward = 1.0
Episode 468: steps = 6 , reward = 1.0
Episode 469: steps = 6 , reward = 1.0
Episode 470: steps = 3 , reward = 0.0
Episode 471: steps = 7 , reward = 1.0
Episode 472: steps = 6 , reward = 1.0
Episode 473: steps = 6 , reward = 1.0
Episode 474: steps = 7 , reward = 1.0
Episode 475: steps = 6 , reward = 1.0
Episode 476: steps = 8 , reward = 1.0
Episode 477: steps = 6 , reward = 1.0
Episode 478: steps = 6 , reward = 1.0
Episode 479: steps = 6 , reward = 1.0
Episode 480: steps = 6 , reward = 1.0
Episode 481: steps = 6 , reward = 1.0
Episode 482: steps = 6 , reward = 1.0
Episode 483: steps = 6 , reward = 1.0
Episode 484: steps = 5 , reward = 0.0
Episode 485: steps = 6 , reward = 1.0
Episode 486: steps = 9 , reward = 1.0
Episode 487: steps = 7 , reward = 1.0
Episode 488: steps = 6 , reward = 1.0
Episode 489: steps = 6 , reward = 1.0
Episode 490: steps = 6 , reward = 1.0
Episode 491: steps = 6 , reward = 1.0
Episode 492: steps = 9 , reward = 1.0
Episode 493: steps = 6 , reward = 1.0
Episode 494: steps = 6 , reward = 1.0
Episode 495: steps = 9 , reward = 1.0
Episode 496: steps = 6 , reward = 1.0
Episode 497: steps = 6 , reward = 1.0
Episode 498: steps = 6 , reward = 1.0
Episode 499: steps = 7 , reward = 1.0(Down)
SFFF
FHFH
FFFH
HFFG(Down)
SFFF
FHFH
FFFH
HFFG(Right)
SFFF
FHFH
FFFH
HFFG(Down)
SFFF
FHFH
FFFH
HFFG(Right)
SFFF
FHFH
FFFH
HFFG(Right)
SFFF
FHFH
FFFH
HFFG
test reward = 1.0

五、结果分析

我们可以查看下最终训练完成的 Q 表：

print(agent.Q)

运行结果：

[[0.27140285 0.4364344  0.09145568 0.15201279][0.26813138 0.         0.         0.00945424][0.         0.         0.         0.        ][0.         0.         0.         0.        ][0.26636559 0.51632351 0.         0.13684245][0.         0.         0.         0.        ][0.         0.         0.         0.        ][0.         0.         0.         0.        ][0.33346755 0.         0.68004322 0.31572772][0.26970648 0.77477987 0.35436455 0.        ][0.04662094 0.73217092 0.         0.        ][0.         0.         0.         0.        ][0.         0.         0.         0.        ][0.         0.39939922 0.88159607 0.11581402][0.4472322  0.72976712 1.         0.40947544][0.         0.         0.         0.        ]]

16 个格子对应的情况：

SFFF
FHFH
FFFH
HFFG

其中 S 代表起点，F 代表平地，H 代表陷阱（掉进去游戏结束），G 代表终点（到达则获胜）

每个格子的排序序号：

0  1  2  3
4  5  6  7
8  9  10 11
12 13 14 15

所以测试开始后，首先在第 0 格，这个时候的 4 个动作对应的 Q 值是：

[0.27140285 0.4364344  0.09145568 0.15201279]

这 4 个 Q 值对应：0 left，1 down，2 right，3 up

所以最大值 0.4364344 对应的是 1，即动作为往下走一格

这个时候到达了第 4 个格子：

[0.26636559 0.51632351 0.         0.13684245]

选择 1，动作：down，到达第 8 个格子：

[0.33346755 0.         0.68004322 0.31572772]

选择 2，动作：right，到达第 9 个格子：

[0.26970648 0.77477987 0.35436455 0.        ]

选择 1，动作：down，到达第 13 个格子：

[0.         0.39939922 0.88159607 0.11581402]

选择 2，动作 right，到达第 14 个格子：

[0.4472322  0.72976712 1.         0.40947544]

选择 2，动作 right，到达第 15 个格子：终点！

查看全文
如若内容造成侵权/违法违规/事实不符，请联系编程学习网邮箱：809451989@qq.com进行投诉反馈，一经查实，立即删除！

相关文章

  1. 前端学习记录2-HTML

    前端学习记录2-HTML绝对路径与相对路径base标签锚点定位pre文本预格式化标签表格table标签table标签下 tr(行) td(单元格) th(表头单元格)table标签下 caption(表格标题)table标签下的单元格合并 rowspan & colspan表格的结构划分 thead tbody tfoot 绝对路径与相对路径相对…...

    2024/4/15 3:34:50
  2. HTTP及HTTPS协议

    HTTP协议协议概述HTTP（Hypertext Transfer Protocol）中文 “超文本传输协议”，是一种为分布式，合作式，多媒体信息系统服务，面向应用层的协议，是 Internet 上目前使用最广泛的应用层协议，它基于传输层的TCP协议进行通信，HTTP协议是通用的、无状态的协议。HTTP是一个应用…...

    2024/5/2 16:35:19
  3. 支付宝开放平台_沙箱环境_第三方 API _SpringBoot

    文章目录一、支付宝接口1-1】官网设置配置信息1-2】创建项目1-3】测试 及 Bug 排除一、支付宝接口 1-1】官网设置配置信息未登录 —— 百度【支付宝开放平台】-【支付宝开发者中心】-【沙箱】查看官方提示文档登录【支付宝开放平台】-【开发者中心】-【研发服务】 / 【沙箱当面…...

    2024/5/2 16:31:44
  4. music

    能够算的上听音乐应该是初二那时候吧，因为小学那时候都是父母在听，我只是被动听的份。初中那时候才算真正开始听流行音乐，有线电视台天天放点播歌曲，印象中第一首听的感觉比较好的应该是一首《中华民谣》和《九月九。。》，学校的广播也天天放。后来初三才买了随身听，开始…...

    2024/4/24 13:46:44
  5. 电气器件系列八：交流继电器（交流接触器）

    http://m.elecfans.com/article/962803.htmlhttps://haokan.baidu.com/v?vid=12816294309604270819&pd=bjh&fr=bjhauthor&type=video交流继电器和交流接触器的区别电磁式的继电器和接触器，它们的工作原理应该说是一样的。有时就是同一个器件，用在这个电路作为接…...

    2024/4/24 13:46:36
  6. 1. Git的配置与快速入门（手把手教学）

    目录你先要准备好这些东西1. Git的简介2. Git的配置3. Git本地常用工作指令3.1 status指令3.2 add指令3.3 commit指令3.4 代码版本更新与指令综合3.5 log指令3.6 reset指令4. Git与Gitee4.1 Gitee创建仓库4.2 SSH公钥关联4.3 Git配置远程仓库4.4 push到Gitee4.5 从Gitee中clone…...

    2024/4/24 13:46:35
  7. 虚拟机Ubuntu18.04连接网络

    之前因为工作需要，将虚拟机的Ubuntu18.04设置成静态IP，今天刚改为动态获取IP连接外网，记录一下。1、首先虚拟机先设置NAT模式，启动时连接，因为我已经开启虚拟机，所以有已连接状态。2、使用命令 sudo gvim /etc/network/interfaces 编辑该文件，第5行的dhcp即自动获取IP。…...

    2024/4/24 13:46:34
  8. 服务器概念、组成和架构详解

    前言：服务器是构建云计算的最核心基础设备，在“新基建”加快推进、公有云持续放量的背景下，服务器行业正迎来景气拐点。本文围绕4个核心问题，由浅入深对服务器进行深入剖析：1、服务器是什么？2、服务器的构成？3、服务器的分类？4、X86/ARM之争？一、服务器是什么？服务器…...

    2024/4/24 13:46:33
  9. ip数据报的转发

    IP数据包经由路由转发的时候源IP，目的ip是否改变？ 这是个搞网络的基础问题，答案是不能改变的，除非做了nat转换才能改变。否则，数据包在整个传输过程中，源IP和目的IP不会发生改变。 不过MAC地址是变化的，因为发送端开始不知道目的主机的MAC地址，所以每经过一个路由器，M…...

    2024/4/24 13:46:35
  10. 不要过度追求时髦，而忘了一些不变的道理

    1. 你是不是因为担心自己落伍而拼命奔跑，觉得凡是和变化、新技术有关的都是好的，都需要追随？2. 其实你大可不必这样。通过这次疫情，我对变化有不同的理解，有的变化其实没那么重要，比如这场疫情，有的人看到了倒卖口罩的机会，在疫情期间小赚一笔，但疫情过后，口罩价格断…...

    2024/5/2 15:18:31
  11. SpringMVC基础学习

    SpringMVC 流程图 [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-SnycNcz4-1592493397355)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191206165911747.png)] springmvc是将原本web服务器一体化的操作，改变为多个…...

    2024/4/24 13:46:30
  12. 踏入大数据的第一天，我先入入门

    目录一、大数据特点二、大数据能干啥三、业务流程分析四、大数据部分组织结构【转行原因】目前数据每天已亿计算，作为 Java程序员的我已经无法使用常规工具对数据进行捕捉、管理和处理这些数据集合。MySQL的存储容量也只有500-1000行数据。而且数据每天还在告诉增长，还不带重…...

    2024/4/24 13:46:30
  13. KGB知识图谱帮助金融机构进行风险预判

    由于金融行业自身存在的高风险性，使得金融行业的发展也存在着各种潜在威胁，一些小的风险累计也会在不定期产生巨大的危机，因此对金融行业的风险及时发现和采取有效的预防措施就显得极为重要。 金融风险又分为信用风险和欺诈风险。信用风险指的是由于客户的信用引起的不想或者…...

    2024/4/15 3:35:09
  14. 0x16 文件系统接口

    一、文件 文件概念 计算机中信息存储的基本组织形式； 相关信息结合； 具有文件名。 文件名 按名存取：文件名 -> 存储位置 文件名由一串ASCII码或（和）汉字构成； 名字长度8.3规则：文件名8个字符，类型3个字符，之间由“.”分割 长文件名：可以最多输入255多个字符作为文…...

    2024/4/20 15:41:22
  15. 图像分割，深度学习做图像分割，适合医学图像分割的Unet网络（理清逻辑关系）

    参考：https://www.zhihu.com/question/269914775?sort=created一.什么是图像分割？ 《数字图像处理》一书中给出：把图像细分为其组成区域或物体。 也可以用函数的知识去理解： 给出一张图像 I，这个问题就是求一个函数，从I映射到Mask。至于怎么求这个函数有多种方法。 举个…...

    2024/4/15 3:35:14
  16. Spring Cloud 配置中心 Spring Cloud Config 入门

    1. 概述Spring Cloud Config 是由 Spring Cloud 官方推出，基于 Spring Cloud 体系的配置中心。相比 Nacos、Apollo 等其它配置中心来说，Spring Cloud Config 是一个轻量级的配置中心，和 Spring Cloud 的集成度会更好，不过功能上会薄弱一些。例如说，灰度发布、运维界面、配…...

    2024/4/17 22:58:55
  17. VUE简单介绍

    Vue.js是一个轻巧、高性能、可组件化的MVVM库，同时拥有非常容易上手的API； Vue.js是一个构建数据驱动的Web界面的库。 Vue.js是一套构建用户界面的 渐进式框架。与其他重量级框架不同的是，Vue 采用自底向上增量开发的设计。Vue的核心库只关注视图层，并且非常容易学习，非常…...

    2024/4/17 3:17:43
  18. “数字经济”助力汽车业快速复苏

    受新冠肺炎疫情影响，旅游、餐饮、汽车等行业遭遇前所未有的寒冬。疫情防控常态化背景下，数字经济的优势进一步凸显，为各行业发展提供了转型的契机。 以机械制造为主体的传统汽车产业，纷纷从危机中寻找转机，突破传统经营模式，拥抱数字经济, 找业务增长点，助推汽车产业在数…...

    2024/5/2 16:25:43
  19. 中文版MathType如何批量修改公式的字体大小

    1、在Word中双击任一个公式，打开mathtype编辑器，然后点击菜单栏的“大小”——>"定义"进入定义字体大小界面。2、在第一个输入框设定你想要的字体大小，在此中文版的mathtype下是“完整”后面的输入框，一般五号对应10pt，小四对应12pt，设置完之后点击“确定”…...

    2024/4/15 3:35:03
  20. 入门 Android Lint

    Lint ​ Android Lint 是Android Studio 在ADT(Android Developer Tools)16提供的代码扫描工具，可以帮助我们发现和更正代码结构质量的问题。系统会报告该工具检测到的每个问题并提供问题的描述消息和严重级别，以便快速确定需要优先进行的修改。此外，我们还可以通过降低问题…...

    2024/4/15 3:35:04

最新文章

  1. 【工具类安装教程】IDEA Ui设计器JFormDesigner

    1、下载插件 File->Settings->Plugins->JFormDesigner 2、注册教程 2.1注册机下载 链接&#xff1a;https://pan.baidu.com/s/1Rb1EMva5HIYbyBcYgCxIsw 提取码&#xff1a;6666 2.2找到目录 1、找到idea目录 2.3进入plugins目录 找到在JFormDesigner文件夹下lib文…...

    2024/5/2 16:57:42
  2. 梯度消失和梯度爆炸的一些处理方法

    在这里是记录一下梯度消失或梯度爆炸的一些处理技巧。全当学习总结了如有错误还请留言&#xff0c;在此感激不尽。 权重和梯度的更新公式如下&#xff1a; w w − η ⋅ ∇ w w w - \eta \cdot \nabla w ww−η⋅∇w 个人通俗的理解梯度消失就是网络模型在反向求导的时候出…...

    2024/3/20 10:50:27
  3. 【python】Flask Web框架

    文章目录 WSGI(Web服务器网关接口)示例Web应用程序Web框架Flask框架创建项目安装Flask创建一个基本的 Flask 应用程序调试模式路由添加变量构造URLHTTP方法静态文件模板—— Jinja2模板文件（Template File）<...

    2024/5/2 2:33:45
  4. 【Godot4自学手册】第三十五节摇杆控制开门

    本节主要实现&#xff0c;在地宫墙壁上安装一扇门&#xff0c;在核实安装一个开门的摇杆&#xff0c;攻击摇杆&#xff0c;打开这扇门&#xff0c;但是只能攻击一次&#xff0c;效果如下&#xff1a; 一、添加完善节点 切换到underground场景&#xff0c;先将TileMap修改一下…...

    2024/5/1 13:37:32
  5. 【外汇早评】美通胀数据走低，美元调整

    原标题：【外汇早评】美通胀数据走低，美元调整昨日美国方面公布了新一期的核心PCE物价指数数据，同比增长1.6%，低于前值和预期值的1.7%，距离美联储的通胀目标2%继续走低，通胀压力较低，且此前美国一季度GDP初值中的消费部分下滑明显，因此市场对美联储后续更可能降息的政策…...

    2024/5/1 17:30:59
  6. 【原油贵金属周评】原油多头拥挤，价格调整

    原标题：【原油贵金属周评】原油多头拥挤，价格调整本周国际劳动节，我们喜迎四天假期，但是整个金融市场确实流动性充沛，大事频发，各个商品波动剧烈。美国方面，在本周四凌晨公布5月份的利率决议和新闻发布会，维持联邦基金利率在2.25%-2.50%不变，符合市场预期。同时美联储…...

    2024/5/2 16:16:39
  7. 【外汇周评】靓丽非农不及疲软通胀影响

    原标题：【外汇周评】靓丽非农不及疲软通胀影响在刚结束的周五，美国方面公布了新一期的非农就业数据，大幅好于前值和预期，新增就业重新回到20万以上。具体数据： 美国4月非农就业人口变动 26.3万人，预期 19万人，前值 19.6万人。 美国4月失业率 3.6%，预期 3.8%，前值 3…...

    2024/4/29 2:29:43
  8. 【原油贵金属早评】库存继续增加，油价收跌

    原标题：【原油贵金属早评】库存继续增加，油价收跌周三清晨公布美国当周API原油库存数据，上周原油库存增加281万桶至4.692亿桶，增幅超过预期的74.4万桶。且有消息人士称，沙特阿美据悉将于6月向亚洲炼油厂额外出售更多原油，印度炼油商预计将每日获得至多20万桶的额外原油供…...

    2024/5/2 9:28:15
  9. 【外汇早评】日本央行会议纪要不改日元强势

    原标题：【外汇早评】日本央行会议纪要不改日元强势近两日日元大幅走强与近期市场风险情绪上升，避险资金回流日元有关，也与前一段时间的美日贸易谈判给日本缓冲期，日本方面对汇率问题也避免继续贬值有关。虽然今日早间日本央行公布的利率会议纪要仍然是支持宽松政策，但这符…...

    2024/4/27 17:58:04
  10. 【原油贵金属早评】欧佩克稳定市场，填补伊朗问题的影响

    原标题：【原油贵金属早评】欧佩克稳定市场，填补伊朗问题的影响近日伊朗局势升温，导致市场担忧影响原油供给，油价试图反弹。此时OPEC表态稳定市场。据消息人士透露，沙特6月石油出口料将低于700万桶/日，沙特已经收到石油消费国提出的6月份扩大出口的“适度要求”，沙特将满…...

    2024/4/27 14:22:49
  11. 【外汇早评】美欲与伊朗重谈协议

    原标题：【外汇早评】美欲与伊朗重谈协议美国对伊朗的制裁遭到伊朗的抗议，昨日伊朗方面提出将部分退出伊核协议。而此行为又遭到欧洲方面对伊朗的谴责和警告，伊朗外长昨日回应称，欧洲国家履行它们的义务，伊核协议就能保证存续。据传闻伊朗的导弹已经对准了以色列和美国的航…...

    2024/4/28 1:28:33
  12. 【原油贵金属早评】波动率飙升，市场情绪动荡

    原标题：【原油贵金属早评】波动率飙升，市场情绪动荡因中美贸易谈判不安情绪影响，金融市场各资产品种出现明显的波动。随着美国与中方开启第十一轮谈判之际，美国按照既定计划向中国2000亿商品征收25%的关税，市场情绪有所平复，已经开始接受这一事实。虽然波动率-恐慌指数VI…...

    2024/4/30 9:43:09
  13. 【原油贵金属周评】伊朗局势升温，黄金多头跃跃欲试

    原标题：【原油贵金属周评】伊朗局势升温，黄金多头跃跃欲试美国和伊朗的局势继续升温，市场风险情绪上升，避险黄金有向上突破阻力的迹象。原油方面稍显平稳，近期美国和OPEC加大供给及市场需求回落的影响，伊朗局势并未推升油价走强。近期中美贸易谈判摩擦再度升级，美国对中…...

    2024/4/27 17:59:30
  14. 【原油贵金属早评】市场情绪继续恶化，黄金上破

    原标题：【原油贵金属早评】市场情绪继续恶化，黄金上破周初中国针对于美国加征关税的进行的反制措施引发市场情绪的大幅波动，人民币汇率出现大幅的贬值动能，金融市场受到非常明显的冲击。尤其是波动率起来之后，对于股市的表现尤其不安。隔夜美国股市出现明显的下行走势，这…...

    2024/5/2 15:04:34
  15. 【外汇早评】美伊僵持，风险情绪继续升温

    原标题：【外汇早评】美伊僵持，风险情绪继续升温昨日沙特两艘油轮再次发生爆炸事件，导致波斯湾局势进一步恶化，市场担忧美伊可能会出现摩擦生火，避险品种获得支撑，黄金和日元大幅走强。美指受中美贸易问题影响而在低位震荡。继5月12日，四艘商船在阿联酋领海附近的阿曼湾、…...

    2024/4/28 1:34:08
  16. 【原油贵金属早评】贸易冲突导致需求低迷，油价弱势

    原标题：【原油贵金属早评】贸易冲突导致需求低迷，油价弱势近日虽然伊朗局势升温，中东地区几起油船被袭击事件影响，但油价并未走高，而是出于调整结构中。由于市场预期局势失控的可能性较低，而中美贸易问题导致的全球经济衰退风险更大，需求会持续低迷，因此油价调整压力较…...

    2024/4/26 19:03:37
  17. 氧生福地 玩美北湖（上）——为时光守候两千年

    原标题：氧生福地 玩美北湖（上）——为时光守候两千年一次说走就走的旅行，只有一张高铁票的距离～ 所以，湖南郴州，我来了～ 从广州南站出发，一个半小时就到达郴州西站了。在动车上，同时改票的南风兄和我居然被分到了一个车厢，所以一路非常愉快地聊了过来。 挺好，最起…...

    2024/4/29 20:46:55
  18. 氧生福地 玩美北湖（中）——永春梯田里的美与鲜

    原标题：氧生福地 玩美北湖（中）——永春梯田里的美与鲜一觉醒来，因为大家太爱“美”照，在柳毅山庄去寻找龙女而错过了早餐时间。近十点，向导坏坏还是带着饥肠辘辘的我们去吃郴州最富有盛名的“鱼头粉”。说这是“十二分推荐”，到郴州必吃的美食之一。 哇塞！那个味美香甜…...

    2024/4/30 22:21:04
  19. 氧生福地 玩美北湖（下）——奔跑吧骚年！

    原标题：氧生福地 玩美北湖（下）——奔跑吧骚年！让我们红尘做伴 活得潇潇洒洒 策马奔腾共享人世繁华 对酒当歌唱出心中喜悦 轰轰烈烈把握青春年华 让我们红尘做伴 活得潇潇洒洒 策马奔腾共享人世繁华 对酒当歌唱出心中喜悦 轰轰烈烈把握青春年华 啊……啊……啊 两…...

    2024/5/1 4:32:01
  20. 扒开伪装医用面膜，翻六倍价格宰客，小姐姐注意了！

    原标题：扒开伪装医用面膜，翻六倍价格宰客，小姐姐注意了！扒开伪装医用面膜，翻六倍价格宰客！当行业里的某一品项火爆了，就会有很多商家蹭热度，装逼忽悠，最近火爆朋友圈的医用面膜，被沾上了污点，到底怎么回事呢？ “比普通面膜安全、效果好！痘痘、痘印、敏感肌都能用…...

    2024/4/27 23:24:42
  21. 「发现」铁皮石斛仙草之神奇功效用于医用面膜

    原标题：「发现」铁皮石斛仙草之神奇功效用于医用面膜丽彦妆铁皮石斛医用面膜|石斛多糖无菌修护补水贴19大优势： 1、铁皮石斛：自唐宋以来，一直被列为皇室贡品，铁皮石斛生于海拔1600米的悬崖峭壁之上，繁殖力差，产量极低，所以古代仅供皇室、贵族享用 2、铁皮石斛自古民间…...

    2024/4/28 5:48:52
  22. 丽彦妆\医用面膜\冷敷贴轻奢医学护肤引导者

    原标题：丽彦妆\医用面膜\冷敷贴轻奢医学护肤引导者【公司简介】 广州华彬企业隶属香港华彬集团有限公司，专注美业21年，其旗下品牌： 「圣茵美」私密荷尔蒙抗衰，产后修复 「圣仪轩」私密荷尔蒙抗衰，产后修复 「花茵莳」私密荷尔蒙抗衰，产后修复 「丽彦妆」专注医学护…...

    2024/4/30 9:42:22
  23. 广州械字号面膜生产厂家OEM/ODM4项须知！

    原标题：广州械字号面膜生产厂家OEM/ODM4项须知！广州械字号面膜生产厂家OEM/ODM流程及注意事项解读： 械字号医用面膜，其实在我国并没有严格的定义，通常我们说的医美面膜指的应该是一种「医用敷料」，也就是说，医用面膜其实算作「医疗器械」的一种，又称「医用冷敷贴」。 …...

    2024/5/2 9:07:46
  24. 械字号医用眼膜缓解用眼过度到底有无作用？

    原标题：械字号医用眼膜缓解用眼过度到底有无作用？医用眼膜/械字号眼膜/医用冷敷眼贴 凝胶层为亲水高分子材料，含70%以上的水分。体表皮肤温度传导到本产品的凝胶层，热量被凝胶内水分子吸收，通过水分的蒸发带走大量的热量，可迅速地降低体表皮肤局部温度，减轻局部皮肤的灼…...

    2024/4/30 9:42:49
  25. 配置失败还原请勿关闭计算机,电脑开机屏幕上面显示，配置失败还原更改 请勿关闭计算机 开不了机 这个问题怎么办...

    解析如下&#xff1a;1、长按电脑电源键直至关机&#xff0c;然后再按一次电源健重启电脑&#xff0c;按F8健进入安全模式2、安全模式下进入Windows系统桌面后&#xff0c;按住“winR”打开运行窗口&#xff0c;输入“services.msc”打开服务设置3、在服务界面&#xff0c;选中…...

    2022/11/19 21:17:18
  26. 错误使用 reshape要执行 RESHAPE，请勿更改元素数目。

    %读入6幅图像&#xff08;每一幅图像的大小是564*564&#xff09; f1 imread(WashingtonDC_Band1_564.tif); subplot(3,2,1),imshow(f1); f2 imread(WashingtonDC_Band2_564.tif); subplot(3,2,2),imshow(f2); f3 imread(WashingtonDC_Band3_564.tif); subplot(3,2,3),imsho…...

    2022/11/19 21:17:16
  27. 配置 已完成 请勿关闭计算机,win7系统关机提示“配置Windows Update已完成30%请勿关闭计算机...

    win7系统关机提示“配置Windows Update已完成30%请勿关闭计算机”问题的解决方法在win7系统关机时如果有升级系统的或者其他需要会直接进入一个 等待界面&#xff0c;在等待界面中我们需要等待操作结束才能关机&#xff0c;虽然这比较麻烦&#xff0c;但是对系统进行配置和升级…...

    2022/11/19 21:17:15
  28. 台式电脑显示配置100%请勿关闭计算机,“准备配置windows 请勿关闭计算机”的解决方法...

    有不少用户在重装Win7系统或更新系统后会遇到“准备配置windows&#xff0c;请勿关闭计算机”的提示&#xff0c;要过很久才能进入系统&#xff0c;有的用户甚至几个小时也无法进入&#xff0c;下面就教大家这个问题的解决方法。第一种方法&#xff1a;我们首先在左下角的“开始…...

    2022/11/19 21:17:14
  29. win7 正在配置 请勿关闭计算机,怎么办Win7开机显示正在配置Windows Update请勿关机...

    置信有很多用户都跟小编一样遇到过这样的问题&#xff0c;电脑时发现开机屏幕显现“正在配置Windows Update&#xff0c;请勿关机”(如下图所示)&#xff0c;而且还需求等大约5分钟才干进入系统。这是怎样回事呢&#xff1f;一切都是正常操作的&#xff0c;为什么开时机呈现“正…...

    2022/11/19 21:17:13
  30. 准备配置windows 请勿关闭计算机 蓝屏,Win7开机总是出现提示“配置Windows请勿关机”...

    Win7系统开机启动时总是出现“配置Windows请勿关机”的提示&#xff0c;没过几秒后电脑自动重启&#xff0c;每次开机都这样无法进入系统&#xff0c;此时碰到这种现象的用户就可以使用以下5种方法解决问题。方法一&#xff1a;开机按下F8&#xff0c;在出现的Windows高级启动选…...

    2022/11/19 21:17:12
  31. 准备windows请勿关闭计算机要多久,windows10系统提示正在准备windows请勿关闭计算机怎么办...

    有不少windows10系统用户反映说碰到这样一个情况&#xff0c;就是电脑提示正在准备windows请勿关闭计算机&#xff0c;碰到这样的问题该怎么解决呢&#xff0c;现在小编就给大家分享一下windows10系统提示正在准备windows请勿关闭计算机的具体第一种方法&#xff1a;1、2、依次…...

    2022/11/19 21:17:11
  32. 配置 已完成 请勿关闭计算机,win7系统关机提示“配置Windows Update已完成30%请勿关闭计算机”的解决方法...

    今天和大家分享一下win7系统重装了Win7旗舰版系统后&#xff0c;每次关机的时候桌面上都会显示一个“配置Windows Update的界面&#xff0c;提示请勿关闭计算机”&#xff0c;每次停留好几分钟才能正常关机&#xff0c;导致什么情况引起的呢&#xff1f;出现配置Windows Update…...

    2022/11/19 21:17:10
  33. 电脑桌面一直是清理请关闭计算机,windows7一直卡在清理 请勿关闭计算机-win7清理请勿关机,win7配置更新35%不动...

    只能是等着&#xff0c;别无他法。说是卡着如果你看硬盘灯应该在读写。如果从 Win 10 无法正常回滚&#xff0c;只能是考虑备份数据后重装系统了。解决来方案一&#xff1a;管理员运行cmd&#xff1a;net stop WuAuServcd %windir%ren SoftwareDistribution SDoldnet start WuA…...

    2022/11/19 21:17:09
  34. 计算机配置更新不起,电脑提示“配置Windows Update请勿关闭计算机”怎么办？

    原标题&#xff1a;电脑提示“配置Windows Update请勿关闭计算机”怎么办&#xff1f;win7系统中在开机与关闭的时候总是显示“配置windows update请勿关闭计算机”相信有不少朋友都曾遇到过一次两次还能忍但经常遇到就叫人感到心烦了遇到这种问题怎么办呢&#xff1f;一般的方…...

    2022/11/19 21:17:08
  35. 计算机正在配置无法关机,关机提示 windows7 正在配置windows 请勿关闭计算机 ，然后等了一晚上也没有关掉。现在电脑无法正常关机...

    关机提示 windows7 正在配置windows 请勿关闭计算机 &#xff0c;然后等了一晚上也没有关掉。现在电脑无法正常关机以下文字资料是由(历史新知网www.lishixinzhi.com)小编为大家搜集整理后发布的内容&#xff0c;让我们赶快一起来看一下吧&#xff01;关机提示 windows7 正在配…...

    2022/11/19 21:17:05
  36. 钉钉提示请勿通过开发者调试模式_钉钉请勿通过开发者调试模式是真的吗好不好用...

    钉钉请勿通过开发者调试模式是真的吗好不好用 更新时间：2020-04-20 22:24:19　浏览次数：729次 区域： 南阳 > 卧龙 列举网提醒您：为保障您的权益，请不要提前支付任何费用！ 虚拟位置外设器!!轨迹模拟&虚拟位置外设神器 专业用于：钉钉，外勤365，红圈通，企业微信和…...

    2022/11/19 21:17:05
  37. 配置失败还原请勿关闭计算机怎么办,win7系统出现“配置windows update失败 还原更改 请勿关闭计算机”，长时间没反应，无法进入系统的解决方案...

    前几天班里有位学生电脑(windows 7系统)出问题了&#xff0c;具体表现是开机时一直停留在“配置windows update失败 还原更改 请勿关闭计算机”这个界面&#xff0c;长时间没反应&#xff0c;无法进入系统。这个问题原来帮其他同学也解决过&#xff0c;网上搜了不少资料&#x…...

    2022/11/19 21:17:04
  38. 一个电脑无法关闭计算机你应该怎么办,电脑显示“清理请勿关闭计算机”怎么办？...

    本文为你提供了3个有效解决电脑显示“清理请勿关闭计算机”问题的方法&#xff0c;并在最后教给你1种保护系统安全的好方法&#xff0c;一起来看看&#xff01;电脑出现“清理请勿关闭计算机”在Windows 7(SP1)和Windows Server 2008 R2 SP1中&#xff0c;添加了1个新功能在“磁…...

    2022/11/19 21:17:03
  39. 请勿关闭计算机还原更改要多久,电脑显示:配置windows更新失败，正在还原更改，请勿关闭计算机怎么办...

    许多用户在长期不使用电脑的时候&#xff0c;开启电脑发现电脑显示&#xff1a;配置windows更新失败&#xff0c;正在还原更改&#xff0c;请勿关闭计算机。。.这要怎么办呢&#xff1f;下面小编就带着大家一起看看吧&#xff01;如果能够正常进入系统&#xff0c;建议您暂时移…...

    2022/11/19 21:17:02
  40. 还原更改请勿关闭计算机 要多久,配置windows update失败 还原更改 请勿关闭计算机，电脑开机后一直显示以...

    配置windows update失败 还原更改 请勿关闭计算机&#xff0c;电脑开机后一直显示以以下文字资料是由(历史新知网www.lishixinzhi.com)小编为大家搜集整理后发布的内容&#xff0c;让我们赶快一起来看一下吧&#xff01;配置windows update失败 还原更改 请勿关闭计算机&#x…...

    2022/11/19 21:17:01
  41. 电脑配置中请勿关闭计算机怎么办,准备配置windows请勿关闭计算机一直显示怎么办【图解】...

    不知道大家有没有遇到过这样的一个问题&#xff0c;就是我们的win7系统在关机的时候&#xff0c;总是喜欢显示“准备配置windows&#xff0c;请勿关机”这样的一个页面&#xff0c;没有什么大碍&#xff0c;但是如果一直等着的话就要两个小时甚至更久都关不了机&#xff0c;非常…...

    2022/11/19 21:17:00
  42. 正在准备配置请勿关闭计算机,正在准备配置windows请勿关闭计算机时间长了解决教程...

    当电脑出现正在准备配置windows请勿关闭计算机时&#xff0c;一般是您正对windows进行升级&#xff0c;但是这个要是长时间没有反应&#xff0c;我们不能再傻等下去了。可能是电脑出了别的问题了&#xff0c;来看看教程的说法。正在准备配置windows请勿关闭计算机时间长了方法一…...

    2022/11/19 21:16:59
  43. 配置失败还原请勿关闭计算机,配置Windows Update失败，还原更改请勿关闭计算机...

    我们使用电脑的过程中有时会遇到这种情况&#xff0c;当我们打开电脑之后&#xff0c;发现一直停留在一个界面&#xff1a;“配置Windows Update失败&#xff0c;还原更改请勿关闭计算机”&#xff0c;等了许久还是无法进入系统。如果我们遇到此类问题应该如何解决呢&#xff0…...

    2022/11/19 21:16:58
  44. 如何在iPhone上关闭“请勿打扰”

    Apple’s “Do Not Disturb While Driving” is a potentially lifesaving iPhone feature, but it doesn’t always turn on automatically at the appropriate time. For example, you might be a passenger in a moving car, but your iPhone may think you’re the one dri…...

    2022/11/19 21:16:57