Aging is inevitably associated with a decline in physical abilities and can pose challenges to the social lives of elderly individuals. In long-term care facilities, group exercise is instrumental for keeping elderly residents physically and socially healthy. Accommodating these needs in elderly care can be challenging due to staff shortages and other lacking resources. A robotic exercise coach could be helpful in such contexts. Intelligent human–robot interaction requires accurate and efficient human activity recognition. Several solutions focusing on human activity recognition in healthcare robotics have been proposed. However, multiperson activity recognition remains a challenging task in case of using vision-based or wearable sensors data, and past research has mainly focused on single-person rather than multiperson or group activity recognition. Moreover, the existing state-of-the-art methods for activity recognition mainly use heavyweight Convolutional Neural Network (CNN) models to achieve good accuracy. However, these models have certain drawbacks, such as requiring significant computational resources, higher memory and storage needs, and slower inference times. Another challenge is the limited number of publicly available datasets containing few activities for physical activity recognition. In this work, we propose a lightweight, deep learning-based, multiperson activity recognition system for group exercise training of elderly persons. Considering the limited publicly available datasets, we curated a new dataset named the Routine Exercise Dataset (RED), comprising 19 routine exercise activities recommended for elderly persons. The RED dataset has 14,440 samples collected from 19 participants and is one of the most extensive datasets of its kind. We evaluated our proposed activity recognition method based on proposed feature extraction modules and a one-dimensional multilayer long short-term memory network on 16 datasets, including 10 publicly available benchmark activity recognition datasets, an RED ...