Description
The replay of neuronal activity associated with previous behavior and experience is a well reported phenomenon that has been linked to memory, learning and planning. For instance, the replay of neuronal activity representing spatial trajectories in the hippocampus of rodents is known to support spatial learning. If the hippocampus was damaged or replay was suppressed, spatial learning is impaired. Furthermore, the content of replay seems to be subject to prioritization, e.g., a location is preferably replayed over another one, and may exhibit generative properties, e.g., replay of short-cut trajectories which were never seen during behavior.
We study the mechanisms that generate replays and the functions that these replays serve in the framework of reinforcement learning (RL). RL describes the interaction of an agent, which seeks to maximize collected rewards, with its environment from which it receives feedback in the form of rewards and observations. This closed-loop nature of RL makes it a perfect tool to study the dynamic interaction between behavior and memory in simulation.
We offer projects at bachelor and master level focused on the development and use of RL algorithms driven by experience replay to model memory and dynamics of learning in both spatial and non-spatial settings. All projects are conducted using CoBeL-RL, which is a neuroscience-oriented simulation framework written in Python that was developed by the Computational Neuroscience group.
Required skills:
Very good programming skills in Python.
Basic knowledge of machine learning, especially reinforcement learning, is beneficial.
Literature:
Diekmann, N., Vijayabaskaran, S., Zeng, X., Kappel, D., Menezes, M. C., & Cheng, S.. (2023). CoBeL-RL:
A neuroscience-oriented simulation framework for complex behavior and learning. Frontiers in
Neuroinformatics, 17. http://doi.org/10.3389/fninf.2023.1134405
Diekmann, N., & Cheng, S.. (2023). A model of hippocampal replay driven by experience and
environmental structure facilitates spatial learning. eLife, 12, e82301.
http://doi.org/10.7554/eLife.82301
Zeng, X., Diekmann, N., Wiskott, L., & Cheng, S.. (2023). Modeling the function of episodic memory in
spatial learning. Frontiers in Psychology, 14. http://doi.org/10.3389/fpsyg.2023.1160648
Supervisors: