Object Detection and Image Categorization by Transferring Commonsense Knowledge with Premises and Quantifiers
Date of Award
Master of Science (MS)
College of Science and Mathematics
Thesis Sponsor/Dissertation Chair/Project Chair
Domestic, or household robots, are autonomous robots designed to make our home-life easier by performing chores and mundane tasks such as cleaning, or cooking. Currently domestic robots are specialized to complete a specific task and, therefore, are confined by factors such as mobility, size, and complexity. With the fast development of computer vision and robotics, the need for more compact, advanced and multi-task robots has emerged. Therefore, the robot needs to be multi-functional, able to discern the environment and the tasks. The aim of this paper is to categorize images in domestic robots as relevant to the culinary, laundry, vacuum class or non-relevant at all. The traditional approach in computer vision involves manual annotation of the large number of images and training the model. The most widespread model training techniques comprise of methods such as convolutional neural networks, regression and support vector machine algorithms. We propose an approach that takes a different route by incorporating commonsense knowledge into the algorithm. Our approach, a Commonsense Knowledge-Detector, or a CSK-Detector, performs the basic object detection on a few household objects via Mask R-CNN, and then utilizes commonsense knowledge clauses obtained from a state-of-the-art Knowledge Base “Dice” for large scale image categorization. The live web-camera object detection is also implemented into the model, allowing the CSK-Detector to classify room environment in real time. In addition, our model is a white-box algorithm that returns explainable results in the form of a decision tree. Moreover, it reaches accuracy scores higher than 90% on the whole, which is similar to the black- box core deep learning models in literature. The CSK-Detector Model refinement and expansion to other, non-domestic domains can potentially aid human-robot collaboration and next-generation robotics.
Chernyavsky, Irina, "Object Detection and Image Categorization by Transferring Commonsense Knowledge with Premises and Quantifiers" (2023). Theses, Dissertations and Culminating Projects. 1225.
Available for download on Saturday, January 20, 2024