Ray
Returns True if the initial state is empty.
By default, RLlib assumes that the module is not recurrent if the initial state is an empty dict and recurrent otherwise. This behavior can be overridden by implementing this method.