citylearn.agents.sac module

class citylearn.agents.sac.SAC(*args, **kwargs)[source]

Bases: citylearn.agents.rlc.RLC

add_to_buffer(observations: List[List[float]], actions: List[List[float]], reward: List[float], next_observations: List[List[float]], done: bool)[source]

Update replay buffer.

Parameters
  • observations (List[List[float]]) – Previous time step observations.

  • actions (List[List[float]]) – Previous time step actions.

  • reward (List[float]) – Current time step reward.

  • next_observations (List[List[float]]) – Current time step observations.

  • done (bool) – Indication that episode has ended.

get_encoded_observations(index: int, observations: List[float]) numpy.ndarray[Any, numpy.dtype[numpy.float64]][source]
get_exploration_actions(observations: List[List[float]]) List[List[float]][source]

Return randomly sampled actions from action_space multiplied by action_scaling_coefficient.

Returns

actions – Action values.

Return type

List[List[float]]

get_normalized_observations(index: int, observations: List[float]) numpy.ndarray[Any, numpy.dtype[numpy.float64]][source]
get_normalized_reward(index: int, reward: float) float[source]
get_post_exploration_actions(observations: List[List[float]], deterministic: bool) List[List[float]][source]

Action sampling using policy, post-exploration time step

select_actions(observations: List[List[float]], deterministic: Optional[bool] = None)[source]

Provide actions for current time step.

Will return randomly sampled actions from action_space if end_exploration_time_step >= time_step else will use policy to sample actions.

Parameters
  • observations (List[List[float]]) – Environment observations

  • deterministic (bool, default: False) – Wether to return purely exploitatative deterministic actions.

Returns

actions – Action values

Return type

List[float]

set_encoders() List[List[citylearn.preprocessing.Encoder]][source]

Get observation value transformers/encoders for use in agent algorithm.

The encoder classes are defined in the preprocessing.py module and include PeriodicNormalization for cyclic observations, OnehotEncoding for categorical obeservations, RemoveFeature for non-applicable observations given available storage systems and devices and Normalize for observations with known minimum and maximum boundaries.

Returns

encoders – Encoder classes for observations ordered with respect to active_observations.

Return type

List[List[Encoder]]

set_networks(internal_observation_count: Optional[int] = None)[source]
class citylearn.agents.sac.SACBasicBatteryRBC(*args, **kwargs)[source]

Bases: citylearn.agents.sac.SACBasicRBC

class citylearn.agents.sac.SACBasicRBC(*args, **kwargs)[source]

Bases: citylearn.agents.sac.SACRBC

class citylearn.agents.sac.SACOptimizedRBC(*args, **kwargs)[source]

Bases: citylearn.agents.sac.SACBasicRBC

class citylearn.agents.sac.SACRBC(*args, **kwargs)[source]

Bases: citylearn.agents.sac.SAC

get_exploration_actions(states: List[float]) List[float][source]

Return actions using RBC.

Returns

actions – Action values.

Return type

List[float]

property rbc: citylearn.agents.rbc.RBC

RBC or child class, used to select actions during exploration.