Classification of Tourist Photo for Intelligent Tourism Service

doi:10.16879/jkca.2019.19.3.087

All Issue

2019 Vol.19, Issue 3 Preview Page Next Page

Research Article

Classification of Tourist Photo for Intelligent Tourism Service 지능형 관광 서비스를 위한 관광 사진 분류체계 개발: 조나혜¹, 강영옥², 윤지영³, 박소연⁴
Nahye Cho¹, Youngok Kang², Jiyoung Yoon³, Soyeon Park⁴; ¹이화여자대학교, 사회과교육과, 박사후 연구원
²이화여자대학교, 사회과교육과, 교수
³이화여자대학교 일반대학원 사회과교육과 지리학전공 석사과정
⁴이화여자대학교 일반대학원 사회과교육과 지리학전공 석사과정

¹PostDoc., Department of Social Studies, Ewha Womans University
²Professor, Department of Social Studies, Ewha Womans University
³Master’s Student, Department of Social Studies Education, Ewha Womans University
⁴Master’s Student, Department of Social Studies Education, Ewha Womans University

31 December 2019. pp. 87~101

PDF

Abstract

In recent years technology of Convolutional Neural Network (CNN) among the technologies of deep learning has evolved dramatically and has shown an outstanding performance in the analysis of image data. First of all, the training of deep learning model is prerequisite to classify the photos posted by the tourists on Web by applying CNN technology. In this study we aim to develop the photo classification system in view of travel purpose in order to classify the photos posted by tourists on Flickr. We developed the category for photo classification by reviewing around 38,000 photos posted by tourists as well as by analysing literatures and web sites, and then verified the category by classifying 8,400 photos one by one manually according to the category developed. The category we developed has 3 hierarchical levels such as 13 major classification, 64 medium classification and 164 minor classification. We expect that our study can applied in base material when one tries to classify the photos for travel purpose by using the CNN deep learning model.

Keywords

Flickr

Social Network Service

Photo Classification for Tourist purpose

Deep Learning

Convolutional Neural Network

Korea Tour

최근 딥러닝 기술 가운데 이미지데이타 분석에 뛰어난 성능을 보이는 합성곱신경망 기술의 발전은 이미지 분석 영역에서 다양한 가능성을 제시하고 있다. 관광객이 게시한 사진을 딥러닝 기술을 이용하여 분류하기 위해서는 관광사진에 대한 분류와 목적에 맞는 딥러닝 모델의 훈련작업이 필수적으로 선행되어야 한다. 본 연구에서는 관광객이 플리커에 게시한 사진을 효율적으로 분류하기 위해 관광목적으로 사진이 어떻게 분류되어야 하는지 관광목적 사진분류 체계를 개발하고자 하였다. 관광목적 사진 분류 카테고리 개발을 위해 문헌분석, 웹사이트 분석, 관광객이 게시한 약 38,000장 사진의 검토과정을 거쳐 사진 분류 카테고리를 개발하였으며, 약 8400장의 사진을 개발된 카테고리에 맞춰 분류해 봄으로써 개발된 카테고리의 검증과정을 거쳤다. 이 과정을 거쳐 최종으로 제안된 카테고리는 13개 대분류, 64개 중분류, 164개의 세분류 체계를 갖으며, 본 연구 결과는 향후 관광목적 사진을 딥러닝 모델을 이용하여 분류하고자 할 때 기초자료로 활용될 것으로 기대된다.

키워드

플리커

SNS

관광목적 사진 분류체계

딥러닝

합성곱신경망

한국 관광

References

강영옥･조나혜･이주윤･윤지영･이혜진, 2019, “경험적 모델과 머신러닝 기법을 활용한 SNS사용자 분류방법 비교: 플리커 데이터의 관광객 분류방법,” 대한공간정보학회지, 27(4), 29-37.10.7319/kogsis.2019.27.4.029
김나연･강영옥, 2019, “지오태깅된 사진 데이터를 활용한 서울방문 관광객의 주요 관광지 분석,” 한국지도학회지, 19(1), 35-46.10.16879/jkca.2019.19.1.035
관계부처합동, 2018, ｢지역관광 활성화 방안｣.
김기환, 2005, “도시이미지의 경쟁력,” 대한건축학회 부산 경남지회 의장색채 심포지엄 발표자료.
박송･심현정, 2017, “CNN을 이용한 영화 포스터의 장르 분류,” 2017년 한국컴퓨터종합학술대회 논문집, 한국정보과학회, 890-892.
박종준･윤현위･권혜정･정원욱･박종화, 2012, “도시 이미지의 지리적 시각화:서울시 대표 이미지 요소 평가를 중심으로,” 서울도시연구, 13(1), 167-180.
이주윤･강영옥･김나연･김동은･박예림, 2018, “궤적 데이터 마이닝을 통한 서울방문 관광객의 이동특성 분석,” 한국지도학회지, 18(3), 117-129.10.16879/jkca.2018.18.3.117
이태민･이인권, 2017, “딥러닝을 이용한 애니메이션 캐릭터 인식,” 한국컴퓨터그래픽스학회 학술대회 논문집, 한국컴퓨터그래픽스학회, 37-38.
이태숙･김철원･안덕수, 2017, “중국 개별자유여행객(FIT) 이 인식한 한국관광에 대한 연구: SNS 빅데이터 분석을 중심으로”. 관광레저연구, 29(7), 91-113.
장현웅･조수선, 2016, “CNN을 이용한 소셜 이미지 자동 태깅,” 정보과학회논문지, 43(1), 47-53.10.5626/JOK.2016.43.1.47
조광익, 1997, “국립공원방문자의 이미지 구성요인,” 관광학연구, 21(1), 231-249.
최승담·박경렬, 2005, “관광목적지로서 도시관광이미지 측정척도 개발,” 서울도시연구, 6(1), 93-106.
한국관광공사, 2017, ｢2016 한국관광 키워드 트렌드 보고서｣.
홍택은･신주현, 2016, “이미지와 텍스트 정보의 카테고리 분류에 의한 SNS 팔로잉 추천 방법,” 스마트미디어저널, 5(3), 54-61.
황미진, 2014, ｢체험형 관광서비스의 소비자권익 증진방안 연구｣, 정책연구보고서.
Baloglu. S. and Brinberg, D., 1997, Affective image of tourism destinations, Journal of Travel Research, 35(4), 11-15.10.1177/004728759703500402
Ciresan, D.C., Meier, U., Masci, J., Maria Gambardella, L., and Schmidhuber, J., 2011, Flexible, high performance convolutional neural networks for image classification, Proceedings, International Joint Conference on Artificial Intelligence (IJCAI), 22(1), 1237-1242.
Deepak, S.S. and Chavan, M.S., 2012, Content Based Image Retrieval: Review, International Journal of Emerging Technology and Advanced Engineering, 2(9), 2250-2459.
Donaire, J.A., Camprubi, R., and Gali, N., 2014, Tourism clusters from Flickr travel photography, Tourism Management Perspectives, 11, 26-33.10.1016/j.tmp.2014.02.003
Embacher, J. and F. Buttler., 1989, A Repertory Grid Analysis of Austria’s Image as a Summer Vacation Destination, Journal of Travel Research, 27(3), 3-7.10.1177/004728758902700302
Géron, A., 2017, Hands-On Machine Learning with Scikit-Learn and TensorFlow, O'Reilly Media.
Hunter, W.C., 2016, The social construction of tourism online destination image: A comparative semiotic analysis of the visual representation of Seoul, Tourism Management, 54, 221-229.10.1016/j.tourman.2015.11.012
Kádár, B., 2014, Measuring tourist activities in cities using geotagged photography, Tourism Geographies, 16(1), 88-104.10.1080/14616688.2013.868029
Kagaya, H. and Aizawa, K., 2015, Highly accurate food/non-food image classification based on a deep convolutional neural network, International Conference on Image Analysis and Processing, Springer, 350-357.10.1007/978-3-319-23222-5_43
Kaneko, T., and Yanai, K., 2013, Visual event mining from geo-tweet photos, Proceedings, 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), IEEE, 1-6.10.1109/ICMEW.2013.6618407PMC3648607
Kim, D., Kang, Y., Park, Y., Kim, N., and Lee, J., 2019, Understanding tourists’ urban images with geotagged photos using convolutional neural networks, Spatial Information Research, https://doi.org/10.1007/s41324- 019-00285-x.10.1007/s41324- 019-00285-x.
Kisilevich, S., Rohrdantz, C., Maidel, V., and Keim, D., 2013, What do you think about this photo? A novel approach to opinion and sentiment analysis of photo comments, International Journal of Data Mining, Modelling and Management, 5(2), 138-157.10.1504/IJDMMM.2013.053693
Krishnan, K.R., M., Midhila, M., and Sudhakar, R., 2018, Tensor flow based analysis and classification of liver disorders from ultrasonography images, in Hemanth, D. and Smys, S., Computational Vision and Bio Inspired Computing, Cham, Switzerland: Springer, 734-743.10.1007/978-3-319-71767-8_64
Lynch, K., 1960, The Image of the City, Cambridge: MIT Press.
Mak, A.H., 2017, Online destination image: Comparing national tourism organisation's and tourists' perspectives, Tourism Management, 60, 280-297.10.1016/j.tourman.2016.12.01
Motlagh, N.H., Jannesary, M., Aboulkheyr, H.R., Khosravi, P., Elemento, O., Totonchi, M., and Hajirasouliha, I., 2018, Breast cancer histopathological image classification: A deep learning approach, bioRxiv, https://doi.org/10.1101/242818.10.1101/242818.
Okuyama, K. and Yanai, K., 2013, A travel planning system based on travel trajectories extracted from a large number of geotagged photos on the web, in Jin, J.S., Xu, C., and Xu, M. eds, The Era of Interactive Media, New York: Springer, 657-670.10.1007/978-1-4614-3501-3_54
Russell, J.A., Ward, L.M., and Pratt, G., 1981, Affective Quality Attributed to Environments: A Factor Analytic Study, Environment and Behavior, 13(3), 259–288.10.1177/0013916581133001
Saito, N., Ogawa, T., Asamizu, S., and Haseyama, M., 2018, Tourism category classification on image sharing services through estimation of existence of reliable results, Proceedings, 2018 ACM on International Conference on Multinedia Retrieval, 493-496.10.1145/3206025.3206085
Sawant, A., Bhandari, M., Yadav, R., Yele, R., and Bendale, S., 2018, Brain cancer detection from mri: A machine learning approach (tensorflow), International Research Journal of Engineering and Technology, 5(4), 2089-2094.
Stern, E. and S. Krakover, 1993, The formation of a composite urban image, Geographical Analysis, 25(2), 130-146. 10.1111/j.1538-4632.1993.tb00285.x
Xu, G., Zhu, X., Fu, D., Dong, J., and Xiao, X., 2017, Automatic land cover classification of geo-tagged field photos by deep learning, Environmental Modelling & Software, 91, 127-134.10.1016/j.envsoft.2017.02.004
Yuan, Y., and Medel, M., 2016, Characterizing international travel behavior from geotagged photos: A case study of flickr, PloS one, 11(5), e0154885.10.1371/journal.pone.015488527159195PMC4861279
Zheng, Y.T., Zha, Z.J., and Chua, T.S., 2012, Mining travel patterns from geotagged photos, ACM Transactions on Intelligent Systems and Technology, 3(3), 56–73.10.1145/2168752.2168770
외박닷컴, http://me2.do/F0buqBpc
Amazon AWS, https://aws.amazon.com
EarthTory, https://www.earthtory.com/ko/city/seoul_310
ImageNet, http://image-net.org
Things to Do: Japan Travel JNTO, https://www.japan. travel/en/destinations/kanto/tokyo
Utrip, https://utrip.com

Information

Publisher :The Korean Cartographic Association
Publisher(Ko) :한국지도학회
Journal Title :Journal of the Korean Cartographic Association
Journal Title(Ko) :한국지도학회지
Volume : 19
No :3
Pages :87~101
DOI :https://doi.org/10.16879/jkca.2019.19.3.087

[1] 강영옥･조나혜･이주윤･윤지영･이혜진, 2019, “경험적 모델과 머신러닝 기법을 활용한 SNS사용자 분류방법 비교: 플리커 데이터의 관광객 분류방법,” 대한공간정보학회지, 27(4), 29-37.10.7319/kogsis.2019.27.4.029

[2] 김나연･강영옥, 2019, “지오태깅된 사진 데이터를 활용한 서울방문 관광객의 주요 관광지 분석,” 한국지도학회지, 19(1), 35-46.10.16879/jkca.2019.19.1.035

[3] 관계부처합동, 2018, ｢지역관광 활성화 방안｣.

[4] 김기환, 2005, “도시이미지의 경쟁력,” 대한건축학회 부산 경남지회 의장색채 심포지엄 발표자료.

[5] 박송･심현정, 2017, “CNN을 이용한 영화 포스터의 장르 분류,” 2017년 한국컴퓨터종합학술대회 논문집, 한국정보과학회, 890-892.

[6] 박종준･윤현위･권혜정･정원욱･박종화, 2012, “도시 이미지의 지리적 시각화:서울시 대표 이미지 요소 평가를 중심으로,” 서울도시연구, 13(1), 167-180.

[7] 이주윤･강영옥･김나연･김동은･박예림, 2018, “궤적 데이터 마이닝을 통한 서울방문 관광객의 이동특성 분석,” 한국지도학회지, 18(3), 117-129.10.16879/jkca.2018.18.3.117

[8] 이태민･이인권, 2017, “딥러닝을 이용한 애니메이션 캐릭터 인식,” 한국컴퓨터그래픽스학회 학술대회 논문집, 한국컴퓨터그래픽스학회, 37-38.

[9] 이태숙･김철원･안덕수, 2017, “중국 개별자유여행객(FIT) 이 인식한 한국관광에 대한 연구: SNS 빅데이터 분석을 중심으로”. 관광레저연구, 29(7), 91-113.

[10] 장현웅･조수선, 2016, “CNN을 이용한 소셜 이미지 자동 태깅,” 정보과학회논문지, 43(1), 47-53.10.5626/JOK.2016.43.1.47

[11] 조광익, 1997, “국립공원방문자의 이미지 구성요인,” 관광학연구, 21(1), 231-249.

[12] 최승담·박경렬, 2005, “관광목적지로서 도시관광이미지 측정척도 개발,” 서울도시연구, 6(1), 93-106.

[13] 한국관광공사, 2017, ｢2016 한국관광 키워드 트렌드 보고서｣.

[14] 홍택은･신주현, 2016, “이미지와 텍스트 정보의 카테고리 분류에 의한 SNS 팔로잉 추천 방법,” 스마트미디어저널, 5(3), 54-61.

[15] 황미진, 2014, ｢체험형 관광서비스의 소비자권익 증진방안 연구｣, 정책연구보고서.

[16] Baloglu. S. and Brinberg, D., 1997, Affective image of tourism destinations, Journal of Travel Research, 35(4), 11-15.10.1177/004728759703500402

[17] Ciresan, D.C., Meier, U., Masci, J., Maria Gambardella, L., and Schmidhuber, J., 2011, Flexible, high performance convolutional neural networks for image classification, Proceedings, International Joint Conference on Artificial Intelligence (IJCAI), 22(1), 1237-1242.

[18] Deepak, S.S. and Chavan, M.S., 2012, Content Based Image Retrieval: Review, International Journal of Emerging Technology and Advanced Engineering, 2(9), 2250-2459.

[19] Donaire, J.A., Camprubi, R., and Gali, N., 2014, Tourism clusters from Flickr travel photography, Tourism Management Perspectives, 11, 26-33.10.1016/j.tmp.2014.02.003

[20] Embacher, J. and F. Buttler., 1989, A Repertory Grid Analysis of Austria’s Image as a Summer Vacation Destination, Journal of Travel Research, 27(3), 3-7.10.1177/004728758902700302

[21] Géron, A., 2017, Hands-On Machine Learning with Scikit-Learn and TensorFlow, O'Reilly Media.

[22] Hunter, W.C., 2016, The social construction of tourism online destination image: A comparative semiotic analysis of the visual representation of Seoul, Tourism Management, 54, 221-229.10.1016/j.tourman.2015.11.012

[23] Kádár, B., 2014, Measuring tourist activities in cities using geotagged photography, Tourism Geographies, 16(1), 88-104.10.1080/14616688.2013.868029

[24] Kagaya, H. and Aizawa, K., 2015, Highly accurate food/non-food image classification based on a deep convolutional neural network, International Conference on Image Analysis and Processing, Springer, 350-357.10.1007/978-3-319-23222-5_43

[25] Kaneko, T., and Yanai, K., 2013, Visual event mining from geo-tweet photos, Proceedings, 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), IEEE, 1-6.10.1109/ICMEW.2013.6618407PMC3648607

[26] Kim, D., Kang, Y., Park, Y., Kim, N., and Lee, J., 2019, Understanding tourists’ urban images with geotagged photos using convolutional neural networks, Spatial Information Research, https://doi.org/10.1007/s41324- 019-00285-x.10.1007/s41324- 019-00285-x.

[27] Kisilevich, S., Rohrdantz, C., Maidel, V., and Keim, D., 2013, What do you think about this photo? A novel approach to opinion and sentiment analysis of photo comments, International Journal of Data Mining, Modelling and Management, 5(2), 138-157.10.1504/IJDMMM.2013.053693

[28] Krishnan, K.R., M., Midhila, M., and Sudhakar, R., 2018, Tensor flow based analysis and classification of liver disorders from ultrasonography images, in Hemanth, D. and Smys, S., Computational Vision and Bio Inspired Computing, Cham, Switzerland: Springer, 734-743.10.1007/978-3-319-71767-8_64

[29] Lynch, K., 1960, The Image of the City, Cambridge: MIT Press.

[30] Mak, A.H., 2017, Online destination image: Comparing national tourism organisation's and tourists' perspectives, Tourism Management, 60, 280-297.10.1016/j.tourman.2016.12.01

[31] Motlagh, N.H., Jannesary, M., Aboulkheyr, H.R., Khosravi, P., Elemento, O., Totonchi, M., and Hajirasouliha, I., 2018, Breast cancer histopathological image classification: A deep learning approach, bioRxiv, https://doi.org/10.1101/242818.10.1101/242818.

[32] Okuyama, K. and Yanai, K., 2013, A travel planning system based on travel trajectories extracted from a large number of geotagged photos on the web, in Jin, J.S., Xu, C., and Xu, M. eds, The Era of Interactive Media, New York: Springer, 657-670.10.1007/978-1-4614-3501-3_54

[33] Russell, J.A., Ward, L.M., and Pratt, G., 1981, Affective Quality Attributed to Environments: A Factor Analytic Study, Environment and Behavior, 13(3), 259–288.10.1177/0013916581133001

[34] Saito, N., Ogawa, T., Asamizu, S., and Haseyama, M., 2018, Tourism category classification on image sharing services through estimation of existence of reliable results, Proceedings, 2018 ACM on International Conference on Multinedia Retrieval, 493-496.10.1145/3206025.3206085

[35] Sawant, A., Bhandari, M., Yadav, R., Yele, R., and Bendale, S., 2018, Brain cancer detection from mri: A machine learning approach (tensorflow), International Research Journal of Engineering and Technology, 5(4), 2089-2094.

[36] Stern, E. and S. Krakover, 1993, The formation of a composite urban image, Geographical Analysis, 25(2), 130-146. 10.1111/j.1538-4632.1993.tb00285.x

[37] Xu, G., Zhu, X., Fu, D., Dong, J., and Xiao, X., 2017, Automatic land cover classification of geo-tagged field photos by deep learning, Environmental Modelling & Software, 91, 127-134.10.1016/j.envsoft.2017.02.004

[38] Yuan, Y., and Medel, M., 2016, Characterizing international travel behavior from geotagged photos: A case study of flickr, PloS one, 11(5), e0154885.10.1371/journal.pone.015488527159195PMC4861279

[39] Zheng, Y.T., Zha, Z.J., and Chua, T.S., 2012, Mining travel patterns from geotagged photos, ACM Transactions on Intelligent Systems and Technology, 3(3), 56–73.10.1145/2168752.2168770

[40] 외박닷컴, http://me2.do/F0buqBpc

[41] Amazon AWS, https://aws.amazon.com

[42] EarthTory, https://www.earthtory.com/ko/city/seoul_310

[43] ImageNet, http://image-net.org

[44] Things to Do: Japan Travel JNTO, https://www.japan. travel/en/destinations/kanto/tokyo

[45] Utrip, https://utrip.com

Journal of the Korean Cartographic Association ISSN:1598-6160(Print) 한국지도학회지

All Issue