Congratulations Dr Nicolas Diekmann!
Our warmest congratulations to Nicolas on the defense of his thesis "Modeling mechanisms and functions of neuronal replay in a
reinforcement learning framework". You can find more about his thesis work here >>.
- RUB
- INI
- Research Groups
- Computational Neuroscience
Computational Neuroscience
News
Are you a student and would either like to pursue a Bachelor/Master thesis or internship in our group? Read our frequently asked questions sheet. Our main language of communication is English.
Our Research:
A particular focus of our research is understanding the function of a brain region, called the hippocampus. It is involved in storing and retrieving episodic memories and in generating representations of space. However, it remains unclear why these two functions go together and how these functions are implemented in the hippocampus. To address these two questions, we employ a number of computational and theoretical approaches, including
- biologically realistic neural network models, which nonetheless are highly simplified, that capture the essence of the neural circuit mechanism underlying learning and memory,
- algorithmic models of the storage and retrieval of episodic memories,
- conceptual models of the nature of episodic memory, and
- virtual reality simulations of spatial navigation and learning.
The two projects described below exemplify the approach of our group.
Modeling spatial learning and navigation
Rodents are known to perform extremely well when navigating complex environments and to adapt rapidly to novel situations. They maneuver through dark sewers while looking for food or fleeing from predators and seem to know exactly where they are and where to go. We model the spatial behavior of rodents using agent-based simulations. A particularly interesting question is: how do rodents learn to navigate novel environments and to perform new tasks? We study this question using a machine learning approach called "reinforcement learning". Within this framework, we implement the neural circuitry of the hippocampus to study the neural mechanisms underlying spatial learning and navigation. For more details see the "Projects" tab above.
Neural sequences in the hippocampus
The Nobel Prize in Physiology or Medicine in 2014 was awarded to John O’Keefe for the discovery of place cells and to May-Britt Moser and Edvard Moser for the discovery of grid cells. Place cells are found in the hippocampus and they respond, or “spike”, when an animal is located in a particular place, called a “place field”. The timing of spiking is such that, during one cycle of the theta oscillation (8 – 12Hz), groups of place cells spike in a sequence that corresponds to the order of their place fields in space (theta sequences). Intriguingly, the same place cells also spike in sequential order afterwards, when the animal is resting or sleeping (replay sequences). We are developing neural network models to help us understand how theta and replay sequences are generated and what role they play in learning and memory.
Affiliations:
 |
 |
 |
|
Speaker & projects P0, P2, and P5 |
Projects A14 & F01 | Executive board |
German collaborations:
- Gerald Echterhoff, University of Münster
- Onur Güntürkün, Ruhr University Bochum
- Denise Manahan-Vaughan, Ruhr University Bochum
- Ricarda Schubotz, University of Münster
- Dagmar Timmann-Braun, University of Duisburg-Essen
- Markus Werning, Ruhr University Bochum
- Laurenz Wiskott, Ruhr University Bochum
- Oliver Wolf, Ruhr University Bochum
International collaborations:
- Paulo de Almeida Ribeiro, Federal University of Maranhão, Brazil
- Yuri Dabaghian, The University of Texas Health Science Center at Houston, USA
- Kamran Diba, University of Michigan, USA
- Areolino Neto, Federal University of Maranhão, Brazil
- Alexandre de Oliveira, Federal University of Maranhão, Brazil
- Henry Otgaar, Maastricht University, Netherlands
- Edison Pignaton de Freitas, Federal University of Rio Grande do Sul, Brazil
- Bailu Si, School of Systems Science, Beijing Normal University, China
- Thomas Suddendorf, University of Queensland, Australia
Group Leader
Members
Sandhiya Vijayabaskaran, M.Sc.
Former Members
Dr. Mohammadreza Mohagheghi Nejad
Affiliated
Gaia Braccia
Vincent Dekorsy
Jessica Kühne
Amany Omar
Nicolas Robayo
Camila Sanchez, B.Sc.
CoBeL-RL
CoBeL-RL, a closed-loop simulator of complex behavior and learning based on Reinforcement Learning (RL) and deep neural networks, provides a neuroscience-oriented framework for efficiently setting up and running simulations. CoBeL-RL offers a set of virtual environments, e.g., T-maze and Morris water maze, which can be simulated at different levels of abstraction, e.g., a simple gridworld or a 3D environment with complex visual stimuli, and set up using intuitive GUI tools. A range of RL algorithms, e.g., Dyna-Q and deep Q-network algorithms, is provided and can be easily extended. CoBeL-RL provides tools for monitoring and analyzing behavior and unit activity, and allows for fine-grained control of the simulation via interfaces to relevant points in its closed-loop. For more information please read CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning
Link: https://github.com/sencheng/CoBeL-RL
License: GNU General Public License v3.0
Publications:
Diekmann, N., Vijayabaskaran, S., Zeng, X., Kappel, D., Menezes, M. C., & Cheng, S. (2023). CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning. Frontiers in Neuroinformatics, 17, 1134405. https://doi.org/10.3389/fninf.2023.1134405
Diekmann, N., & Cheng, S. (2023). A Model of Hippocampal Replay Driven by Experience and Environmental Structure Facilitates Spatial Learning. eLife, 12:e82301. https://doi.org/10.7554/eLife.82301
Zeng, X., Diekmann, N., Wiskott, L., & Cheng, S. (2023). Modeling the function of episodic memory in spatial learning. Frontiers in Psychology, 14, 1160648. https://doi.org/10.3389/fpsyg.2023.1160648
Vijayabaskaran, S., & Cheng, S. (2022). Navigation task and action space drive the emergence of egocentric and allocentric spatial representations. PLOS Computational Biology, 18(10), e1010320. https://doi.org/10.1371/journal.pcbi.1010320
Walther, T., Diekmann, N., Vijayabaskaran, S., Donoso, J. R., Manahan-Vaughan, D., Wiskott, L., & Cheng, S. (2021). Context-dependent extinction learning emerging from raw sensory inputs: A reinforcement learning approach. Scientific Reports, 11(1), Article 1. https://doi.org/10.1038/s41598-021-81157-z
Diekmann, N., Walther, T., Vijayabaskaran, S., & Cheng, S. (2019, September 15). Deep reinforcement learning in a spatial navigation task: Multiple contexts and their representation [Poster presentation]. Conference on Cognitive Computational Neuroscience, Berlin, Germany. https://ccneuro.org/2019/Papers/ViewPapers.asp?PaperNum=1151
CoBeL-Spike
CoBel-Spike is a closed-loop simulator of complex behavior and learning based on spiking neural networks. The CoBeL-spike tool-chain consists of three main components: the artificial agent, the environment, and a bidirectional interface between behavior and neuronal activity. The agent consists of a spiking neural network that receives sensory inputs and generates motor commands, which control the behavior of the agent in the simulated environment.
If you would like to dive deeper and see how it works, you can find the open-source code on github.
Publications:
Ghazinouri, B., Nejad, M.M. & Cheng, S. Navigation and the efficiency of spatial coding: insights from closed-loop simulations. Brain Struct Funct (2023). https://doi.org/10.1007/s00429-023-02637-8
Data Analysis
⁃ Pyka-Parametric-Anatomical-Modeling-2014
Parametric Anatomical Modeling is a method to translate large-scale anatomical data into spiking neural networks. PAM is implemented as a Blender addon.
LICENSE: GNU GPL v2.0
DOI: 10.5281/zenodo.3298590
PUBLICATIONS: Pyka, M., Klatt, S., & Cheng, S. (2014). Parametric Anatomical Modeling: a method for modeling the anatomical layout of neurons and their projections. Frontiers in Neuroanatomy, 8, 91. http://doi.org/10.3389/fnana.2014.00091
⁃ Pyka-Pam-Utils--2014
This is a module with some helpful functions to process the data generated by PAM
License: GNU GPL v2.0
DOI: 10.5281/zenodo.3298825
PUBLICATIONS: Pyka, M., Klatt, S., & Cheng, S. (2014). Parametric Anatomical Modeling: a method for modeling the anatomical layout of neurons and their projections. Frontiers in Neuroanatomy, 8, 91. http://doi.org/10.3389/fnana.2014.00091
Neural Networks (empty)
Cognitive Models
⁃ Dynamics of Disease States in Depression
Major depressive disorder (MDD) is a disabling condition that adversely affects a person general health, work or school life, sleeping and eating habits, and person's family. Despite intense research efforts, the response rate of antidepressant treatments are relatively low and the etiology and progression of MDD remain poorly understood. To advance our understanding of MDD, we use computational modelling as described in our article.
The model to simulate the dynamics of disease states in depression can be found below.
License: GNU GPL v3.0
DOI: 10.5281/zenodo.3299247
PUBLICATIONS: Demic, S. & Cheng, S. (2014): Modeling the Disease States in Depression. PLoS ONE 9(10): e110358. https://doi.org/10.1371/journal.pone.0110358
⁃ Episodic Memory Deficits in Depression
License: GNU GPL v3.0
DOI: 10.5281/zenodo.3299871
PUBLICATIONS: Fang, J., Demic, S., & Cheng, S. (2018) The reduction of adult neurogenesis in depression impairs the retrieval of new as well as remote episodic memory, PLOS ONE, 13(6), e0198406
Reinforcement Learning (empty)
High-quality figures
Our group aims to provide neuroscientific community with a collection of high-quality SVG-figures for free use in publications, presentations, websites etc. via GitHub.
All SVG-files in the repository are distributed under the terms of the Create Commons Attribution 4.0 International License.
Current available images include:
-
neuron density during mouse brain development focused on the comparison of neuron density in the dorsal pallidum/isocortex compared to the hippocampal formation https://github.com/MartinPyka/NeuroSVG/tree/master/Mammals/Mouse .
- detailed diagram of the crossection of a rat's hippocampus entorhinal cortexhttps://github.com/MartinPyka/NeuroSVG/tree/master/Mammals/Rat .
Dynamics of extinction learning in behavior and neural activity
How does learning unfold over time? This question can be studied in experimental data and with computational modeling. We analyze behavioral and neural activity data that was collected by collaborating labs. In our theoretical work, we employ simple associative models as well as deep reinforcement learning, which allows us to study the emerging representations and to correlate them to experimental data.
Emergent behavior and neural representations in spatial learning
Spatial navigation might appear to be a simple behavior, but closer inspection reveals that it is the complex result of many interacting sub-processes. We use deep reinforcement learning to understand how goal-directed behavior emerges in an artificial agent, how the deep neural network represents spatial information, and how the model's representations are related to neural codes for space in the hippocampus.
Episodic memory interactions with sensory and semantic systems
We study, on the one hand, how episodic memories are stored and retrieved in interaction with the sensory and semantic systems. On the other hand, we investigate how episodic memories influence these sensory and semantic systems.
Nature and function of episodic memory
There is still great uncertainty as to what episodic memory is and what function it might serve. Within the research unit FOR 2812, we are working on an interdisciplinary framework for episodic memory.
Neural mechanisms underlying spatial navigation
A large number of cell types in the mammalian brain code for various types of spatial information, e.g., head direction cells, place cells, and grid cells. We study how networks of these cell types could support spatial navigation by combining computational modeling and data analysis.
Sequence memory in the hippocampus
The CRISP theory suggests that episodic memories are best represented by neuronal sequences and specific mechanisms by which sequences are stored and retrieved from the hippocampal circuit. Using neural network models, we investigate under which conditions the hippocampal circuit can perform the hypothesized functions reliably and robustly.
- RUB news: https://news.rub.de/english/2024-09-13-interview-understanding-brain-thanks-artificial-intelligence
- Episodic memory deficits in depression Labor Praxis, Freie Presse, Evangelische Pressedienst (dpd)
- Parametric Anatomical Modeling: Sciencedaily, Neurosciencenews.com, R&D, MedGadget, Bionity.com, PsyPost
-
Gedächtnisverbesserung: Möglichkeiten und kritische Betrachtung