Creating intelligent artificial systems, and in particular robots, that improve themselves just like humans do is one of the most ambitious goals in robotics and machine learning. The concept of robot experience exists for some time now, but has up to now not fully found its way into autonomous robots. This thesis is devoted to both, analyzing the underlying requirements for enabling robot learning from experience and actually implementing it on real robot hardware. For effective robot learning from experience I present and discuss three main requirements: (a ) Clearly expressing what a robot should do, on a vague, abstract level I introduce Generalized Plans as a means to express the intention rather than the actual action sequence of a task, removing as much task specific knowledge as possible. (a ) Defining, collecting, and analyzing robot experiences to enable robots to improve I present Episodic Memories as a container for all collected robot experiences for any arbitrary task and create sophisticated action (effect) prediction models from them, allowing robots to make better decisions. (a ) Properly abstracting from reality and dealing with failures in the domain they occurred in I propose failure handling strategies, a failure taxonomy extensible through experience, and discuss the relationship between symbolic/discrete and subsymbolic/continuous systems in terms of robot plans interacting with real world sensors and actuators. I concentrate on the domain of human-scale robot activities, specifically on doing household chores. Tasks in this domain offer many repeating patterns and are ideal candidates for abstracting, encapsulating, and modularizing robot plans into a more general form. This way, very similar plan structures are transformed into parameters that change the behavior of the robot while performing the task, making the plans more flexible. While performing tasks, robots encounter the same or similar situations over and over again. Albeit humans are able to benefit from this and improve at what they do, robots in general lack this ability. This thesis presents techniques for collecting and making robot experiences accessible to robots and outside observers alike, answering high level questions such as "What are good spots to stand at for grasping objects from the fridge?" or "Which objects are especially difficult to grasp with two hands while they are in the oven?". By structuring and tapping into a robot's memory, it can make more informed decisions that are not based on manually encoded information, but self-improved behavior. To this end, I present several experience-based approaches to improve a robot's autonomous decisions, such as parameter choices, during execution time. Robots that interact with the real world are bound to deal with unexpected events and must properly react to failures of any kind of action. I present an extensible failure model that suits the structure of Generalized Plans and Episodic Memories and make clear how each module should deal with their own failures rather than directly handing them up to a governing cognitive architecture. In addition, I make a distinction between discrete parametrizations of Generalized Plans and continuous low level components, and how to translate between the two.