Abstract: As we interact with the world around us, we experience a constant stream of sensory inputs, and must generate a constant stream of behavioral actions. What makes brains more than simple input-output machines is their capacity to integrate sensory inputs with an animal’s own internal motivational state to produce behavior that is flexible and adaptive. Working with neural recordings from subcortical structures involved in regulation of survival behaviors, we show how the dynamical properties of neural populations give rise to motivational states that change animal behavior on a timescale of minutes, while neuromodulation can alter these dynamics to change behavior on timescales of hours to days. Using methods from control theory and reinforcement learning, we demonstrate that different sites of modulation within a neural circuit produce different resulting effects on behavior and neural activity. We then show how theoretical models can reveal unexpected effects of neuromodulation on the dynamic regimes of recurrent neural networks, illuminating the ways in which the brain might use small molecules to reshape its activity and thus modify behavior.