Humans are capable of pursuing multiple objectives simultaneously in everyday life, thereby flexibly prioritizing motion tasks and constraints among others. A major goal of current robotics research is to equip highly redundant robots with similar adaptive behavior. The aim of this thesis is to endow such robots with the abilities to optimize priorities for possibly conflicting objectives, to smoothly rearrange such priorities and to decouple the objectives related to motion generation from the objective of simultaneous interaction with the environment through forces. A commonly adopted approach is to tune priorities by hand once which then remain constant over time. In contrast to such constant prioritization, this thesis promotes a different view: A framework for automated learning soft priorities is introduced. Employing state-of-the-art optimization, different sets of priorities are evaluated offline and improved until the prioritized superposition of all underlying controllers satisfies a desired high-level goal. As an alternative approach to soft prioritization with scalar weights, the well-known Stack-of-Tasks prioritization scheme relies on projectors and enforces strict priorities. The second part of this thesis performs a thorough formal analysis of projectors and proposes to smoothly shape between idempotent matrix operators. A novel prioritization scheme is presented with the help of this method, which generalizes previous approaches implementing either strict or soft priorities. It enables to insert new or remove outdated objectives if necessary and allows to rearrange priorities continuously online without inertia coupling while offering the ability to control the interference between objectives. Finally, underactuated robots subject to contact constraints are studied. A projection operator is derived which enables to decouple contact wrench control from constraint-consistent motion generation for robots with passive or virtual joints. This is essential for controlling legged robots which are represented by a floating-base. Moreover, this thesis shows that the same principle holds for underactuated grasping scenarios. The approach is validated with a large panel of experiments on various robot platforms both in simulation and the real-world. This thesis provides numerous tools that endows existing and future robot systems with ever-increasing degrees of freedom to create complex motion aiming to interact with their environment effectively.