Information technologies and computer sciences

Consolidated Ranking of Modern Cloud-Based Online Object Recognition Services on Images

Authors

O. Ya. Shulhin
Vasyl’ Stus Donetsk National University, Vinnytsia
S. D. Shtovba
Vasyl’ Stus Donetsk National University, Vinnytsia; Vinnytsia National Technical University
DOI https://doi.org/10.31649/1997-9266-2023-171-6-39-45

Keywords

object recognition cloud services metric ranking images

Abstract

Automating the object recognition in images is a widely encountered task with evident practical applications in industrial production, medicine, transportation, security, safety, and other fields. Today, there are several cloud services that offer online tools for solving various image recognition tasks. They have several advantages over traditional tools. Today, there are no methods that allow users to identify which cloud service is best suited for their tasks. Moreover, the identification should be concise and based on small set of profile images. Accordingly, a typical brute-force method that requires uploading and analyzing a large number of images is unacceptable. The method should be based on a detailed analysis of the recognition results using a small set of test images, taking into account the features of cloud services. This paper proposes a method for ranking cloud services using small test datasets. In this case, the user forms test datasets that take into account the profile of his or her object recognition tasks. The proposed method is based on three particular metrics, each of which takes into account some features of cloud services. The first metric is the difference between the sum of the confidence levels of correctly recognized objects and the sum of the confidence levels of incorrectly recognized objects. The second metric is the accuracy. The third metric is the median of the length of the service output until the first misrecognized object. The objects should be sorted in descending order of confidence. The first two metrics are traditional, the third metric is new. The final decision is made based on a consolidated score that aggregates the 3 metrics. The application of the method is illustrated by the task of ranking the following cloud services: Microsoft Azure AI Vision Studio, Amazon Rekognition, Google Cloud Vision, and Imagga.

628 132

How to Cite

[1]
“Consolidated Ranking of Modern Cloud-Based Online Object Recognition Services on Images”, Вісник ВПІ, no. 6, pp. 39–45, Dec. 2023, doi: 10.31649/1997-9266-2023-171-6-39-45.

Author Biographies

O. Ya. Shulhin, Vasyl’ Stus Donetsk National University, Vinnytsia

Graduate Student of the Department of Information and Applied Technologies

S. D. Shtovba, Vasyl’ Stus Donetsk National University, Vinnytsia; Vinnytsia National Technical University

Dr. Sc. (Eng.), Professor, Professor of the Chair of Information Technology of Vasyl’ Stus Donetsk National University;   Professor, of the Chair of Computer Control Systems of  Vinnytsia National Technical University

References

Z. Zou, K. Chen, Z. Shi, Y. Guo, and J. Ye, “Object Detection in 20 Years: A Survey,” Proceedings of the IEEE, no. 111 (3), pp. 257-276,2023). https://doi.org/10.1109/JPROC.2023.3238524 .
N. Pinto, D. D. Cox, and J. J. DiCarlo, “Why is real-world visual object recognition hard?” PLoS Computational Biology, no. 4 (1), pp. 0151-0156, 2008. https://doi.org/10.1371/journal.pcbi.0040027 .
A. Berg, and M. Nelimarkka, “Do you see what I see? Measuring the semantic differences in image-recognition services’ outputs,” Journal of the Association for Information Science and Technology, no. 74(11), pp. 1307-1324, 2023. https://doi.org/10.1002/asi.24827 .
T. Araujo, I. Lock, and B. van de Velde, “Automated Visual Content Analysis (AVCA) in Communication Research: a Protocol for Large Scale Image Classification with Pre-Trained Computer Vision Models,” Communication Methods and Measures, no. 14(4), pp. 239-265, 2020. https://doi.org/10.1080/19312458.2020.1810648 .
Ethan Shafer, Jesse Wood, Sheyla Street, Enoch Crow, Calvin Lu, “Social Bias and Image Tagging: Evaluation of Progress in State-of-the-Art Models,” in book Advances in Visual Computing https://doi.org/10.1007/978-3-031-47966-3_26 .
How To Calculate the Confidence Interval. [Electronic resource]. Available: https://www.indeed.com/career-advice/career-development/how-to-calculate-confidence-interval .
R. Padilla, W. L. Passos, T. L. B. Dias, S. L. Netto, and E. A. B. Da Silva, “A comparative analysis of object detection metrics with a companion open-source toolkit,” Electronics (Switzerland), no. 10 (3), pp. 1-28, 2021. https://doi.org/10.3390/electronics10030279 .

Most read articles by the same author(s)

<< < 1 2 3