A key challenge in the use of drones for an aerial manipulation task such as cutting tree branches is the control problem, especially in the presence of an unpredictable and nonlinear environment. While prior work focused on simplifying the problem by modeling a simple interaction with branches and controlling the system with nonlinear and non-robust control schemes, the current work deals with the problem by designing novel robust nonlinear controllers for aerial manipulation systems that are appropriate for vegetation sampling. In this regard, two different potential control schemes are proposed: nonlinear disturbance observer-based control (NDOBC) and adaptive sliding mode control (ASMC). Each considers the external disturbances and unknown parameters in controller design. The proposed control scheme in both methods employs a decoupled architecture that treats the unmanned aerial vehicle and the manipulator arm of the sampler payload as separate units. In the proposed control structures, controllers are designed after comprehensively investigating the dynamics of both the aerial vehicle and the robotic arm. Each system is then controlled independently in the presence of external disturbances, unknown parameter changes, and the nonlinear coupling between the aerial vehicle and robotic arm. In addition, fully actuated and underactuated aerial platforms are examined, and their stability and controllability are compared so as to choose the most practical framework. Finally, the simulation findings verify and compare the performance and effectiveness of the proposed control strategies for a custom aerial manipulation system that has been designed and developed for field trials.