半月刊

ISSN 1000-1026

CN 32-1180/TP

+高级检索 English
基于深度强化学习的自适应不确定性经济调度
作者:
作者单位:

武汉大学电气与自动化学院,湖北省 武汉市 430072

作者简介:

彭刘阳(1995—),男,硕士研究生,主要研究方向:人工智能在电力系统中的应用。E-mail:leftimepelly@whu.edu.cn

通讯作者:

孙元章(1953—),男,通信作者,教授,博士生导师,主要研究方向:电力系统分析与控制。E-mail:yzsun@mail.tsinghua.edu.cn

基金项目:

国家重点研发计划资助项目(2016YFB0900100);湖北省杰出青年基金资助项目(2018CFA080);国家自然科学基金资助项目(51707136)。


Self-adaptive Uncertainty Economic Dispatch Based on Deep Reinforcement Learning
Author:
Affiliation:

School of Electrical Engineering and Automation, Wuhan University, Wuhan 430072, China

Fund Project:

This work is supported by National Key R&D Program of China (No. 2016YFB0900100), Excellent Youth Foundation of Hubei Province of China (No. 2018CFA080) and National Natural Science Foundation of China (No. 51707136).

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
    摘要:

    当风电、光伏等间歇性电源大规模接入电力系统时,为应对其出力的不确定性,电力系统经济调度模型需建立在对不确定性建模的基础上,建模精确度将直接影响调度结果的精确度。但当系统同时包含风电、光伏和负荷复杂的不确定性时,对系统整体不确定性进行精确建模显得尤为困难。针对这一问题,引入深度强化学习中深度确定性策略梯度算法,避免对复杂的不确定性进行建模,利用其与环境交互、根据反馈学习改进策略的机制,自适应不确定性的变化。为确保算法适用性,进行了模型泛化方法的设计,针对算法稳定性问题进行了感知-学习比例调整和改进经验回放的机制设计。算例结果表明,所提方法能在自适应系统不确定性的基础上,实现任意场景下的电力系统动态经济调度。

    Abstract:

    When the large-scale intermittent generation sources including wind and photovoltaic generation connects to the power system, in order to cope with their uncertainty, the economic dispatch model for power system needs to be built based on the uncertainty modeling. The accuracy of modeling will directly affect the accuracy of the dispatch results. However, when considering the complex uncertainty of both load and intermittent generation sources such as wind and photovoltaic generation, it is particularly difficult to accurately model the overall uncertainty of the system. In view of this problem, the deep deterministic policy gradient (DDPG) algorithm in the deep reinforcement learning is introduced. The work of uncertainty modeling is avoided. Instead, the uncertainty is adapted by the DDPG algorithm relied on the mechanism of interacting with the environment and improving the strategy based on feedbacks. In order to guarantee the applicability of the algorithm, the generalization method for the DDPG model is proposed. Aiming at the stability problem of the algorithm, two mechanisms are designed, including perception-learning ratio adjustment and experience replay improvement. The result of example shows that the dynamic economic dispatch problem of power systems in any scenario can be solved by the proposed method based on adapting the system uncertainty.

    表 1 调度结果经济性对比Table 1 Economic comparison of dispatch results
    图1 DDPG算法流程Fig.1 Flow chart of DDPG algorithm
    图1 DDPG算法流程Fig.1 Flow chart of DDPG algorithm
    图2 训练过程中的总回报平均值Fig.2 Average total reward during training process
    图3 典型日调度结果Fig.3 Dispatching results for a typical day
    图4 系统备用对不确定性的平衡Fig.4 Power uncertainty balanced by system reserve
    图5 功率缺额概率分布与系统预留备用情况Fig.5 Situation of power shortage probability distribution and system reserve
    图6 机制设计对DDPG算法收敛性的影响Fig.6 Effect of mechanism design on convergence of DDPG algorithm
    图 Critic网络及其目标网络结构Fig. Structure of critic network and its target network
    图 Actor网络及其目标网络结构Fig. Structure of actor network and its target network
    图 Critic网络及其目标网络结构Fig. Structure of critic network and its target network
    图 过拟合时第一隐层网络参数Fig. The parameters of the first hidden layer when overfitted
    图 网络收敛情况(损失函数)Fig. Network convergence (Loss function)
    图 过拟合时第一隐层网络参数Fig. The parameters of the first hidden layer when overfitted
    图 网络收敛情况(损失函数)Fig. Network convergence (Loss function)
    图 损失函数曲线发散Fig. Loss function curve diverges
    图 学习停滞Fig. The stagnation of learning
    图 训练时间随机组规模的变化Fig. The training time varies with the scale of power units
    参考文献
    相似文献
    引证文献
引用本文

彭刘阳,孙元章,徐箭,等.基于深度强化学习的自适应不确定性经济调度[J].电力系统自动化,2020,44(9):33-42. DOI:10.7500/AEPS20190706003.
PENG Liuyang,SUN Yuanzhang,XU Jian,et al.Self-adaptive Uncertainty Economic Dispatch Based on Deep Reinforcement Learning[J].Automation of Electric Power Systems,2020,44(9):33-42. DOI:10.7500/AEPS20190706003.

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2019-07-06
  • 最后修改日期:2019-12-20
  • 录用日期:
  • 在线发布日期: 2020-05-10
  • 出版日期: