Limited work has been done on examining deep learning for ultrasound robotic navigation, and there is no existing literature found that explores object detection within an ultrasound-guided navigation environment. Our work addresses this gap by exploring a solution to automated, robotic breast tumor detection using reinforcement learning. We are successfully able to train a multi-model agent for the task for breast tumor detection within a 3D Slicer simulated environment by combining deep Q-learning along with an image classifier, and a classification accuracy of 96% is obtained. Future work includes exploring more diverse simulation environments as well as investigating the effectiveness of simulation-to-real-world transfer learning