Cerebral palsy (CP) is the most common motor disorder in children, and is caused by injuries of the brain shortly before, during, or after birth. With traditional methods, a reliable diagnosis of CP can generally not be made until after the first year of life or even later. Based on the discovery that the condition of the nervous system of young infants is reflected in the quality of their spontaneous movements, the general movement assessment (GMA) was developed. GMA lets trained experts detect CP in infants as young as four months. However, regular practice and re-calibration are required, and GMA suffers from human variability. CP manifests itself in a variety of symptoms, which complicates finding the right parameters for predicting the risk of CP. A cheap, automated system would allow the widespread screening of infants in order to concentrate early intervention efforts on affected children. We divide the system into the motion capture stage and the motion analysis stage. Although the extraction of motion, respectively body pose, from images and videos is a very active area of computer vision research, most of the proposed approaches focus on adults and can not be directly transferred to infants. In this thesis, we present multiple contributions towards an unobtrusive system for medical motion analysis of infants. First, we propose an approach for estimating 3D pose from depth images, speeding up the training procedure by nearly two orders of magnitude compared to previous approaches. Second, we develop a model-based approach for markerless full-body tracking. We learn the Skinned Multi-Infant Linear model (SMIL) from low-quality RGB-D data and accurately register it to sequences of moving infants, capturing shape and pose while handling self-occlusions and fast movements. We show that our method captures enough motion detail for GMA. Third, we develop an approach towards predicting the GMA class from motion sequences, which is based on features of motion complexity and variation to capture characteristics of general movements. Our methods enable accurate tracking of infant shape and pose, and enable applications like monitoring of therapy/disease progression, tracking growth or motor development, and detection of malnutrition. By publicly releasing the learned model and a synthetic, but realistic data set of moving infants, we hope to foster research focused on infants.