The Unmanned Aerial Vehicle (UAV) has over the last years become an increasingly prevalent technology in several sectors of modern society. Many UAVs are today used in a wide series of applications, from disaster relief to surveillance. A recent initiative by the Swedish Sea Rescue Society (SSRS) aims to implement UAVs in their emergency response. By quickly deploying drones to an area of interest, an assessment can be made, prior to personnel getting there, thus saving time and increasing the likelihood of a successful rescue operation. An aircraft like this, that will travel great distances, have to rely on a navigation system that does not require an operator to continuously see the vehicle. To travel to its goal, or search an area, the operator should be able to define a travel route that the UAV follows, by feeding it a series of waypoints. As an initial step towards that kind of system, this thesis has developed and tested the concept of waypoint navigation on a small and slow airship/blimp, in a simulated indoor environment. Mainly, two commonly used navigation algorithms were tested and compared. One is inspired by a sub-category of machine learning: reinforcement learning (RL), and the other one is based on the rapidly exploring random tree (RRT) algorithm. Four experiments were conducted to compare the two methods in terms of travel distance, average speed, energy efficiency, as well as robustness towards changes in the waypoint configurations. Results show that when the blimp was controlled by the best performing RL-based version, it generally travelled a more optimal (distance-wise) path than the RRT-based method. It also, in most cases, proved to be more robust against changes in the test tracks, and performed more consistently over different waypoint configurations. However, the RRT approach usually resulted in a higher average speed and energy efficiency. Also, the RL algorithm had some trouble navigating tracks where a physical obstacle was present. To sum up, the choice of algorithm depends on ...