IMAGE CAPTIONING USING CNN AND DEEP STACKED LSTM

Abstract

In image captioning, the content describing an image are generated automatically which involves computer vision and NLP (Natural Language Processing). Researches in the field of the computer vision and natural processing language has sky rocketed in past few years resulting remarkable contributions and achievements. This has resulted positive impacts in the field of image captioning as well. Numbers of the models and methods have been developed to attain maximum accuracy in the caption. In this work, deep neural network using Encoder-decoder approach was implemented for image captioning. In Encoder-decoder approach, VGG16 was used as encoder to extract features from images in form of dense vectors, Bidirectional Stacked LSTM as decoder uses extracted dense vectors as input for the sentence generation. The caption of the image generated mainly focused in describing the object’s gestures and action including some information about the environment. Both English And Nepali caption was generated .The BLEU score obtained from the generated captioning was 0.576