This output is listed in the Digital Public Goods Alliance Registry and contributes to the following UN Sustainable Development Goals (SDGs):

United Nations Sustainable Development Goal 7: Affordable and Clean Energy United Nations Sustainable Development Goal 11: Sustainable Cities and Communities United Nations Sustainable Development Goal 13: Climate Action

CityLearn is an open source OpenAI Gym environment for the implementation of Multi-Agent Reinforcement Learning (RL) for building energy coordination and demand response in cities [23, 25]. A major challenge for RL in demand response is the ability to compare algorithm performance [27]. Thus, CityLearn facilitates and standardizes the evaluation of RL agents such that different algorithms can be easily compared with each other.

demand response

Districts and cities have periods of high demand for electricity, which raise electricity prices and the overall cost of the power distribution networks. Flattening, smoothening, and reducing the overall curve of electrical demand helps reduce operational and capital costs of electricity generation, transmission, and distribution networks. CityLearn allows the easy implementation of reinforcement learning agents in a single or multi-agent setting to reshape their aggregated curve of electrical demand by controlling active energy storage for load shifting and heat pump or electric heater power for load shedding.


CityLearn has been utilized in the following projects and publications:



Benchmarking reinforcement learning algorithms

Dhamankar et al. [4], Khayatian et al. [7], Nweye et al. [10], Pinto et al. [17], Qin et al. [19]

Coordinated energy management

Deltetto [2], Glatt et al. [5], Kathirgamanathan et al. [6], Pinto et al. [16], Pinto et al. [18], Qin et al. [20], Vazquez-Canteli et al. [26], Yang et al. [28]

Incentive based demand-response

Deltetto et al. [3], Tolovski [21]

Independent energy management

Chen et al. [1], Nweye et al. [11]


Zhang et al. [29]

Model predictive control

Zhan, Sicheng et al. [30]

Transfer learning

Nweye et al. [12]

Voltage regulation

Pigott et al. [15]

Cite CityLearn

   author = {V\'{a}zquez-Canteli, Jos\'{e} R. and K\"{a}mpf, J\'{e}r\^{o}me and Henze, Gregor and Nagy, Zoltan},
   title = {CityLearn v1.0: An OpenAI Gym Environment for Demand Response with Deep Reinforcement Learning},
   year = {2019},
   isbn = {9781450370059},
   publisher = {Association for Computing Machinery},
   address = {New York, NY, USA},
   url = {},
   doi = {10.1145/3360322.3360998},
   booktitle = {Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation},
   pages = {356–357},
   numpages = {2},
   keywords = {Smart Grid, Building Energy Control, Smart Buildings},
   location = {New York, NY, USA},
   series = {BuildSys '19}

Indices and tables