Tim Nisslbeck

I am a PhD candidate at BIASlab, TU Eindhoven, supervised by Wouter Kouw. My research focuses on sample-efficient robot learning — designing agents that learn to control physical robots from far fewer interactions than current reinforcement learning methods require. I build probabilistic world models that maintain a full Bayesian posterior over their parameters and update exactly at each timestep, and I develop model-based agents that use expected free energy (EFE) as their planning objective — a variational quantity from the active inference framework that replaces the hand-designed reward function and jointly drives goal attainment and uncertainty reduction, similar in spirit to information-seeking, curiosity-driven model-based RL. I validate these methods on real robotic hardware.

Before my PhD I worked as a researcher at Fraunhofer IIS, where I applied deep reinforcement learning to autonomous driving, led an interpretable RL project, and built large-scale HPC training pipelines. For my MSc thesis I collected real RC car motion-capture data, trained an LSTM-based world model on it, and used that learned simulator to train a DDQN agent entirely in simulation before deploying the policy on the physical car.

I am broadly interested in the following areas:

Probabilistic world models more

I design dynamics models that maintain a full Bayesian posterior over their parameters and update exactly at each timestep, giving calibrated uncertainty estimates without gradient descent or replay buffers. This uncertainty is what makes the downstream agent able to reason about what it does and does not yet know.

Online Bayesian system identification · conjugate autoregressive models · factor graphs · uncertainty quantification
MARX (Entropy 2025)

Decision-making under uncertainty more

I develop model-based agents whose planning objective decomposes into a goal-seeking exploitation term and an information-seeking exploration term, with no hand-designed reward function. The balance between the two emerges automatically as the agent's model uncertainty evolves over time.

Active inference · expected free energy · model-based RL with information-seeking objectives · exploration vs exploitation
MARX-EFE (preprint), MP-MARX-EFE (IWAI 2025), CARX-EFE (IWAI 2024)

Real-robot learning more

A key motivation for probabilistic world models is making sample-efficient learning tractable on physical hardware. I deploy learned controllers directly on real robots — without simulation pre-training — to validate that sample efficiency gains hold outside the simulator.

Sample efficiency on physical hardware · sim-to-real transfer · quadruped locomotion · learning from scratch without demonstrations
Bittle quadruped, RC car MSc thesis

Publications and preprints

Papers sorted by recency. Representative papers are highlighted. Click a thumbnail to enlarge; click "more" for a short summary.

Open Source

Simulation environments, tools, and reproducible implementations.