Open In Colab

QuickStart

Install the latest CityLearn version from PyPi with the :code:pip command:

[ ]:
!pip install CityLearn

No Control (Baseline)

Run the following to simulate an environment where the storage systems and heat pumps are not controlled (baseline). The storage actions prescribed will be 0.0 and the heat pump will have no action, i.e. None, causing it to deliver the ideal load in the building time series files:

[1]:
from citylearn.agents.base import DummyAgent as Agent
from citylearn.citylearn import CityLearnEnv

dataset_name = 'citylearn_challenge_2023_phase_2_local_evaluation'
env = CityLearnEnv(dataset_name, central_agent=True)
model = Agent(env)
model.learn(episodes=1)

# print cost functions at the end of episode
kpis = model.env.evaluate()
kpis = kpis.pivot(index='cost_function', columns='name', values='value').round(3)
kpis = kpis.dropna(how='all')
display(kpis)
/Users/kingsleyenweye/Desktop/INTELLIGENT_ENVIRONMENT_LAB/citylearn/CityLearn/env/lib/python3.7/site-packages/gym/spaces/box.py:128: UserWarning: WARN: Box bound precision lowered by casting to float32
  logger.warn(f"Box bound precision lowered by casting to {self.dtype}")
name Building_1 Building_2 Building_3 District
cost_function
annual_normalized_unserved_energy_total 0.019 0.018 0.018 0.018
annual_peak_average NaN NaN NaN 1.000
carbon_emissions_total 1.000 1.000 1.000 1.000
cost_total 1.000 1.000 1.000 1.000
daily_one_minus_load_factor_average NaN NaN NaN 1.000
daily_peak_average NaN NaN NaN 1.000
discomfort_delta_average -1.544 0.538 -0.543 -0.517
discomfort_delta_maximum 5.384 5.847 3.835 5.022
discomfort_delta_minimum -4.741 -1.772 -3.466 -3.326
discomfort_proportion 0.370 0.034 0.085 0.163
discomfort_too_cold_proportion 0.360 0.000 0.082 0.147
discomfort_too_hot_proportion 0.010 0.034 0.003 0.016
electricity_consumption_total 1.000 1.000 1.000 1.000
monthly_one_minus_load_factor_average NaN NaN NaN 1.000
one_minus_thermal_resilience_proportion 0.333 0.571 0.133 0.346
power_outage_normalized_unserved_energy_total 0.723 0.692 0.637 0.684
ramping_average NaN NaN NaN 1.000
zero_net_energy 1.000 1.000 1.000 1.000

Centralized RBC

Run the following to simulate an environment controlled by centralized RBC agent for a single episode:

[2]:
from citylearn.agents.rbc import BasicRBC as Agent
from citylearn.citylearn import CityLearnEnv

dataset_name = 'citylearn_challenge_2023_phase_2_local_evaluation'
env = CityLearnEnv(dataset_name, central_agent=True)
model = Agent(env)
model.learn(episodes=1)

# print cost functions at the end of episode
kpis = model.env.evaluate()
kpis = kpis.pivot(index='cost_function', columns='name', values='value').round(3)
kpis = kpis.dropna(how='all')
display(kpis)
name Building_1 Building_2 Building_3 District
cost_function
annual_normalized_unserved_energy_total 0.017 0.016 0.016 0.016
annual_peak_average NaN NaN NaN 1.179
carbon_emissions_total 1.997 1.936 1.737 1.890
cost_total 1.930 1.877 1.709 1.839
daily_one_minus_load_factor_average NaN NaN NaN 0.721
daily_peak_average NaN NaN NaN 1.352
discomfort_delta_average -9.724 -3.425 -3.153 -5.434
discomfort_delta_maximum 1.693 4.220 3.269 3.061
discomfort_delta_minimum -13.562 -9.930 -5.399 -9.630
discomfort_proportion 0.975 0.899 0.957 0.944
discomfort_too_cold_proportion 0.975 0.892 0.953 0.940
discomfort_too_hot_proportion 0.000 0.007 0.003 0.004
electricity_consumption_total 1.993 1.962 1.751 1.902
monthly_one_minus_load_factor_average NaN NaN NaN 0.863
one_minus_thermal_resilience_proportion 0.667 0.429 0.267 0.454
power_outage_normalized_unserved_energy_total 0.781 0.759 0.711 0.750
ramping_average NaN NaN NaN 0.958
zero_net_energy 2.058 1.993 1.769 1.940

Decentralized-Independent SAC

Run the following to simulate an environment controlled by decentralized-independent SAC agents for 1 training episode:

[3]:
from citylearn.agents.sac import SAC as Agent
from citylearn.citylearn import CityLearnEnv

dataset_name = 'citylearn_challenge_2023_phase_2_local_evaluation'
env = CityLearnEnv(dataset_name, central_agent=False)
model = Agent(env)
model.learn(episodes=2, deterministic_finish=True)

# print cost functions at the end of episode
kpis = model.env.evaluate()
kpis = kpis.pivot(index='cost_function', columns='name', values='value').round(3)
kpis = kpis.dropna(how='all')
display(kpis)
name Building_1 Building_2 Building_3 District
cost_function
annual_normalized_unserved_energy_total 0.013 0.013 0.012 0.013
annual_peak_average NaN NaN NaN 0.947
carbon_emissions_total 0.939 0.943 0.940 0.941
cost_total 0.902 0.905 0.907 0.905
daily_one_minus_load_factor_average NaN NaN NaN 0.947
daily_peak_average NaN NaN NaN 0.925
discomfort_delta_average -1.587 -0.276 -0.522 -0.795
discomfort_delta_maximum 4.775 6.418 4.015 5.069
discomfort_delta_minimum -6.335 -4.455 -2.931 -4.574
discomfort_proportion 0.468 0.474 0.256 0.400
discomfort_too_cold_proportion 0.422 0.302 0.223 0.316
discomfort_too_hot_proportion 0.046 0.172 0.033 0.084
electricity_consumption_total 0.949 0.960 0.954 0.955
monthly_one_minus_load_factor_average NaN NaN NaN 0.997
one_minus_thermal_resilience_proportion 0.733 0.500 0.133 0.456
power_outage_normalized_unserved_energy_total 0.627 0.641 0.593 0.620
ramping_average NaN NaN NaN 0.883
zero_net_energy 0.964 0.962 0.955 0.960

Decentralized-Cooperative MARLISA

Run the following to simulate an environment controlled by decentralized-cooperative MARLISA agents for 1 training episodes:

[4]:
from citylearn.agents.marlisa import MARLISA as Agent
from citylearn.citylearn import CityLearnEnv

dataset_name = 'citylearn_challenge_2023_phase_2_local_evaluation'
env = CityLearnEnv(dataset_name, central_agent=False)
model = Agent(env)
model.learn(episodes=2, deterministic_finish=True)

kpis = model.env.evaluate()
kpis = kpis.pivot(index='cost_function', columns='name', values='value').round(3)
kpis = kpis.dropna(how='all')
display(kpis)
name Building_1 Building_2 Building_3 District
cost_function
annual_normalized_unserved_energy_total 0.013 0.013 0.012 0.013
annual_peak_average NaN NaN NaN 0.949
carbon_emissions_total 0.954 0.972 0.957 0.961
cost_total 0.916 0.934 0.925 0.925
daily_one_minus_load_factor_average NaN NaN NaN 0.939
daily_peak_average NaN NaN NaN 0.933
discomfort_delta_average -1.703 -0.357 -0.569 -0.876
discomfort_delta_maximum 4.678 6.237 3.967 4.961
discomfort_delta_minimum -6.464 -4.542 -2.989 -4.665
discomfort_proportion 0.495 0.470 0.266 0.410
discomfort_too_cold_proportion 0.454 0.315 0.236 0.335
discomfort_too_hot_proportion 0.041 0.155 0.030 0.075
electricity_consumption_total 0.964 0.990 0.971 0.975
monthly_one_minus_load_factor_average NaN NaN NaN 0.993
one_minus_thermal_resilience_proportion 0.733 0.500 0.133 0.456
power_outage_normalized_unserved_energy_total 0.630 0.646 0.593 0.623
ramping_average NaN NaN NaN 0.882
zero_net_energy 0.980 0.993 0.972 0.982

Stable Baselines3 Reinforcement Learning Algorithms

Install the latest version of Stable Baselines3:

[ ]:
%pip install shimmy==0.2.1
%pip install stable-baselines3==2.1.0

Before the environment is ready for use in Stable Baselines3, it needs to be wrapped. Firstly, wrap the environment using the NormalizedObservationWrapper (see docs) to ensure that observations served to the agent are min-max normalized between [0, 1] and cyclical observations e.g. hour, are encoded using the cosine transformation.

Next, we wrap with the StableBaselines3Wrapper (see docs) that ensures observations, actions and rewards are served in manner that is compatible with Stable Baselines3 interface.

For the following Stable Baselines3 example, the baeda_3dem dataset that support building temperature dynamics is used.

⚠️ NOTE: central_agent in the env must be True when using Stable Baselines3 as it does not support multi-agents.

[5]:
from stable_baselines3.sac import SAC as Agent
from citylearn.citylearn import CityLearnEnv
from citylearn.wrappers import NormalizedObservationWrapper, StableBaselines3Wrapper

dataset_name = 'citylearn_challenge_2023_phase_2_local_evaluation'
env = CityLearnEnv(dataset_name, central_agent=True)
env = NormalizedObservationWrapper(env)
env = StableBaselines3Wrapper(env)
model = Agent('MlpPolicy', env)
episodes = 2
model.learn(total_timesteps=env.time_steps*episodes)

# evaluate
observations = env.reset()

while not env.done:
    actions, _ = model.predict(observations, deterministic=True)
    observations, _, _, _ = env.step(actions)

kpis = env.evaluate()
kpis = kpis.pivot(index='cost_function', columns='name', values='value').round(3)
kpis = kpis.dropna(how='all')
display(kpis)
/Users/kingsleyenweye/Desktop/INTELLIGENT_ENVIRONMENT_LAB/citylearn/CityLearn/env/lib/python3.7/site-packages/gym/core.py:330: DeprecationWarning: WARN: Initializing wrapper in old step API which returns one bool instead of two. It is recommended to set `new_step_api=True` to use new step API. This will be the default behaviour in future.
  "Initializing wrapper in old step API which returns one bool instead of two. It is recommended to set `new_step_api=True` to use new step API. This will be the default behaviour in future."
/Users/kingsleyenweye/Desktop/INTELLIGENT_ENVIRONMENT_LAB/citylearn/CityLearn/env/lib/python3.7/site-packages/stable_baselines3/common/vec_env/patch_gym.py:50: UserWarning: You provided an OpenAI Gym environment. We strongly recommend transitioning to Gymnasium environments. Stable-Baselines3 is automatically wrapping your environments in a compatibility layer, which could potentially cause issues.
  "You provided an OpenAI Gym environment. "
name Building_1 Building_2 Building_3 District
cost_function
annual_normalized_unserved_energy_total 0.015 0.012 0.014 0.014
annual_peak_average NaN NaN NaN 0.834
carbon_emissions_total 0.388 0.366 0.503 0.419
cost_total 0.368 0.341 0.473 0.394
daily_one_minus_load_factor_average NaN NaN NaN 1.301
daily_peak_average NaN NaN NaN 0.691
discomfort_delta_average 9.481 7.166 7.980 8.209
discomfort_delta_maximum 16.788 14.865 13.923 15.192
discomfort_delta_minimum -0.124 -0.581 -0.372 -0.359
discomfort_proportion 0.982 0.978 0.978 0.979
discomfort_too_cold_proportion 0.000 0.000 0.000 0.000
discomfort_too_hot_proportion 0.982 0.978 0.978 0.979
electricity_consumption_total 0.393 0.374 0.510 0.425
monthly_one_minus_load_factor_average NaN NaN NaN 1.126
one_minus_thermal_resilience_proportion 1.000 1.000 1.000 1.000
power_outage_normalized_unserved_energy_total 0.571 0.510 0.553 0.545
ramping_average NaN NaN NaN 0.905
zero_net_energy 0.276 0.349 0.455 0.360