modellrl

modellrl is an open-source software library designed to support model-based reinforcement learning workflows. It provides a framework for building, training, and evaluating agents that use explicit models of their environment, including learned dynamics and reward models. The library is intended for research and applied use, supporting both simulated environments and real-world data by enabling rapid experimentation with different modeling and planning approaches.

The project emphasizes modularity and interoperability, allowing users to combine neural dynamics models with planning and policy optimization. modellrl integrates with popular machine-learning ecosystems such as PyTorch and JAX, and it provides adapters for common environments and data formats, including OpenAI Gym and custom simulators.

Key components include dynamics models, reward models, and uncertainty estimators; ensembles for robust predictions; planners that use simulated rollouts, model-predictive control, or Dyna-style lookahead; policy networks and actor-critic methods; data buffers and replay mechanisms; and evaluation suites to compare agents under varying conditions.

The library supports a range of model-based algorithms, from world-model based planning and offline model-based RL to hybrid approaches that integrate learned models with contemporary policy optimization. It also includes tooling for experiment tracking, hyperparameter sweeps, and benchmarking across environments.

Typical workflows involve collecting data from an environment or simulator, training predictive models of dynamics and rewards, applying planners to select actions, and evaluating agent performance. modellrl aims to be compatible with standard RL interfaces, easing integration with existing environments and datasets.

modellrl is developed openly by a community of researchers and practitioners. It is released under a permissive license and maintained with documentation, tutorials, and community contributions on its official repository.

Related topics include model-based reinforcement learning, planning in RL, and comparative RL libraries such as RLlib and Stable Baselines, which modellrl complements by providing explicit world-model capabilities and planning tools.