Takafumi Nakanishi

Through technology, it is essential to seamlessly bridge the divide between diverse speaking communities (including the signer (the sign language speaker) community). In order to realize communication that successfully conveys emotions, it is necessary to recognize not only verbal information but also non-verbal information. In the case of signers, there are two main types of behavior: verbal behavior and emotional behavior. This paper presents a sign language recognition method by similarity measure with emotional expression specific to signers. We focus on recognizing the sign language conveying verbal information itself and on recognizing emotional expression. Our method recognizes sign language by time-series similarity measure on a small amount of model data, and at the same time, recognizes emotion expression specific to signers. Our method extracts time-series features of the body, arms, and hands from sign language videos and recognizes them by measuring the similarity of the time-series features. In addition, it recognizes the emotional expressions specific to signers from the time-series features of their faces.

In this paper, we present a realization method of discovery for burst topic transition using the topic change point detection method for time-series text data. In our method, we focus on the topic change point detection method for time-series text data. By similarity measure using the topic change point detection method for time-series text data, we can discover for burst topic transition. In general, when we would like to understand the outline or main points of an event, we often read articles written by people who know information about the event or ask others who are aware of the event to tell us about it. However, the information obtained by these means is hearsay from others and subject to third-party bias, it is difficult to comprehend the events objectively. In our paper, we focus on the topic change and extract the topic change point detection It enables us to discover burst topic transitions. In this paper, we describe an evaluation experiment of a prototype system using our discovery for burst topic transition to verify the effectiveness of our method. We also implement an application by the user interface that provides some crews of a trendy word.

In this paper, we present an emotion estimation method using heart rate variability parameters of vital data. Recently, as sensors have become more precise and smaller, it has been possible to obtain users' vital data in real-time quickly. In our method, ECG (electrocardiogram) data are measured beforehand while listening to a story with voice narration that evokes emotions and based on the trends obtained through the measurement, the emotions that have a high correlation with the newly acquired ECG data are estimated to be the emotions expressed in the ECG data. With the implementation of our method, it is possible to estimate the user's emotions based on ECG data. In this paper, we also represent the application of our method to chat icons that see users' emotions in real-time. By realizing this application, users will see the changes in their emotions and control their mental health.

In general, when a place depicted in a novel exists, users may like to visit the site that appears in the story. There is a need to efficiently visit tourist spots in or around the places depicted in the novel when visiting them. When we realize to retrieve appropriate tourist spots in areas in stories, it is possible to increase the number of new sightseeing opportunities for users. This paper presents a tourist spot recommendation method corresponding to place names appearing in novel contents. Our system recommends some actual tourist spots corresponding to a novel selected by a user. The feature of our proposed method is to recommend tourist spots within the range that users can visit in a day, based on the place names that appear in the novel. We apply this method to realize seamless linking media between the creative world like a novel and the real-world.

This paper presents a recipe recommendation method by similarity measure with food image recognition. In general, it is difficult for users with little cooking experience to find out what kind of dishes they can make from the ingredients they currently have. Therefore, we propose a system that recommends recipes based on the ingredients in the user's current inventory, thereby increasing the number of dishes in the user's cooking repertoire. This system uses camera images of foodstuffs as input, recognizes the foodstuffs, and searches for recipes. In the experiment, we conducted a questionnaire survey of the recognized food ingredients and a questionnaire survey of recipe suggestions, and the results showed that more than 3/4 of the respondents answered that the recognition results and recipe contents were correct for some of the images. In this way, possible for users to search for recipes with fewer steps.

In this paper, we present a method for discovering of historical transition in aesthetic notions of waka poetry over by using changes in co-occurrence words. The structure and the words used in waka poetry change as time passes. By analyzing the chronological changes in the structures and words of waka poetry, we can clarify the historical changes in the aesthetic notions of Japanese people. In this paper, we focus on the three major anthologies of Japanese poetry, Manyoshu, Kokin Wakashu and Shin Kokin Wakashu, and on Kago, which is the central elements in Japanese poetry. Kago is a word and expression often used in Japanese poetry. In our method, we derive differences in the frequency of occurrence of Kago categorized according to their meanings in each collection and extract important co-occurrences of Kago as context words. By our method, we clarify the differences in the context words of Kago and show the historical transition of aesthetic notions.

In this study, we propose an automatic mashup creation method based on the similarity of musical features. The mashup is one of the techniques of uninterrupted composition of music with similar sound quality and characteristics. When mashups can be created algorithmically and automatically, there are more opportunities to listen to music corresponding to one’s sensibilities using streaming services with large amounts of musical content. Our method automatically creates mashup works by decomposing existing music into components defined by their respective instruments. These components are subsequently recombined with each other depending on the evaluation of their similarity. In this study, we also implement an automated mashup creation system using our proposed method and validate it using a questionnaire. Our method can change the way people listen to music, and enable them to easily consume the large amount of music available, especially through the Internet.

This paper presents a weather map prediction method using RGB metaphorical feature extraction for atmospheric pressure patterns. In the field of meteorological science, predicting weather based on the analysis of observational data and the knowledge of weather experts is crucial. Weather experts draw weather maps based on air pressure distribution; hence, we believe that weather maps entail the interpretations of weather experts. In this study, we improved the prediction accuracy by using machine learning to recognize patterns of qualitative expert interpretations that cannot be predicted by analyzing observed data alone. The proposed method can be realized via two steps. The first is developing a module for extracting pressure pattern features from a weather map. Certain features, such as tropical cyclones or atmospheric high/low pressure distributions, are emphasized in weather maps to facilitate better understanding of the weather features. Therefore, we can predict weather features based on the knowledge of weather experts using data that contain their interpretations, particularly weather maps. The developed module extracts the atmospheric pressure features from the current weather map as an RGB metaphorical gradation map. The second step is developing a module to design a predicted weather map using the extracted features. The weather map of the following day is predicted using pix2pix. To the best of our knowledge, our method for extracting features from weather maps is the first to create a predicted weather map automatically.

Herein, we present a novel topic variation detection method that combines a topic extraction method and a change-point detection method. It extracts topics from time-series text data as the feature of each time and detects change points from the changing patterns of the extracted topics. We applied this method to analyze the valuable, albeit underutilized, text dataset containing the Japanese Prime Minister’s (PM’s) detailed daily activities for over 32 years. The proposed method and data provide novel insights into the empirical analyses of political business cycles, which is a classical issue in economics and political science. For instance, as our approach enables us to directly observe and analyze the PM’s actions, it can overcome the empirical challenges encountered by previous research owing to the unobservability of the PM’s behavior. Our empirical observations are primarily consistent with recent theoretical developments regarding this topic. Despite limitations, by employing a completely novel method and dataset, our approach enhances our understanding and provides new insights into this classic issue.

In this paper, we present gaze-music transformation realization method by realizing gaze emotion detection system and interconnecting the detection system and an automatic music media creation system. This method automatically creates a music media contents by a detected emotion from human gaze. When it is possible to realize a system that acquires human gaze data from a camera device, detects emotions from the data, and creates media content in response to those emotions, it will contribute to creative activities corresponding to human emotion. Our method connects a gaze emotion detection system and an automatic music creation system by the similarity of words that represent emotions and impressions. It is possible to compare emotions by comparing the emotions expressed by each different vocabulary using the similarity of words.