The purpose of this paper is to introduce a new framework for training and testing Reinforcement Learning (RL) algorithms for robotic unscrewing tasks. The paper investigates current disassembly technologies through a state-of-the-art analysis, and the basic concepts of reinforcement learning are studied. A comparable framework exists as an extension for OpenAI gym called Gym-Gazebo, which is tested and analysed. Based on this analysis, a design for a new framework is made to specifically support unscrewing operations in robotics disassembly of electronics waste. The proposed simulation architecture uses ROS as data middleware, Gazebo (with the ODE physics solver) for simulating the robot environment, and MoveIt as a controller. The Gazebo simulation consists of a minimalistic setup in order to stay focused on the architecture and usability of the framework. The simulation world interfaces with the RL-agent, using OpenAI Gym and ROS-topics, which can be adapted to interface with a real robot. Lastly, the work demonstrates the functionality of the system by implementing an application example using a Q-learning algorithm, and the results of this are presented. ; The purpose of this paper is to introduce a new framework for training and testing Reinforcement Learning (RL) algorithms for robotic unscrewing tasks. The paper investigates current disassembly technologies through a state-of-the-art analysis, and the basic concepts of reinforcement learning are studied. A comparable framework exists as an extension for OpenAI gym called Gym-Gazebo, which is tested and analysed. Based on this analysis, a design for a new framework is made to specifically support unscrewing operations in robotics disassembly of electronics waste. The proposed simulation architecture uses ROS as data middleware, Gazebo (with the ODE physics solver) for simulating the robot environment, and MoveIt as a controller. The Gazebo simulation consists of a minimalistic setup in order to stay focused on the architecture and usability of the ...