Fusion of Deep Features and Weighted VLAD Vectors based on Multiple Features for Image Retrieval

¹ School of Computer and Information Technology, Beijing Jiaotong University, 100081, Beijing, China
² Key Laboratory of Advanced Information Science and Network Technology of Beijing, 100081, Beijing, China
³ Institute of Electronic Commerce, Guangdong University of Finance & Economics, 510320, Guangzhou, China
⁴ College of Mechanical Engineering, Guizhou University, 550025, Guiyang, China
⁵ Department of Radio-electronic systems, Don State Technical University, 346500, Rostov-on-Don, Russia
⁶ Faculty of Technical Sciences University of Kragujevac, 32000, Cacak, Serbia

^* Corresponding author: lianglq@gdufe.edu.cn

Abstract

In traditional vector of locally aggregated descriptors (VLAD) method, the final VLAD vector is reshaped by summing up the residuals between each descriptor and its corresponding visual word. The norm of the residuals varies significantly, and it can make “visual burst”. This is caused by a fact that the contribution of each descriptor to VLAD vector is not the same. To address this problem, we add a different weight to each residual such that the contribution of each descriptor to the VLAD vector becomes even to a certain degree. Also, traditional VLAD method only uses the local gradient features of images. Thus it has a low discrimination. In this paper, local color features are extracted and used to the VLAD method. Moreover, we fuse deep features and the multiple VLAD vectors based on local gradient and color information. Also, in order to reduce running time and improve retrieval accuracy, PCA and whitening operations are used for VLAD vectors. Our proposed method is evaluated on three benchmark datasets, i.e., Holidays, Ukbench and Oxford5k. Experimental results show that our proposed method achieves good performance.