Protein phosphorylation, a key regulator of cellular processes, plays a central role in brain function and is implicated in neurological disorders. Information on protein phosphorylation is expected to be a clue for understanding various neuropsychiatric disorders and developing therapeutic strategies. Nonetheless, existing databases lack a specific focus on phosphorylation events in the brain, which are crucial for investigating the downstream pathway regulated by neurotransmitters. To overcome the gap, we have developed a web-based database named “Kinase-Associated Neural PHOspho-Signaling (KANPHOS).” This paper presents the design concept, detailed features, and a series of improvements for KANPHOS. KANPHOS is designed to support data-driven research by fulfilling three key objectives: (1) enabling the search for protein kinases and their substrates related to extracellular signals or diseases; (2) facilitating a consolidated search for information encompassing phosphorylated substrate genes, proteins, mutant mice, diseases, and more; and (3) offering integrated functionalities to support pathway and network analysis. KANPHOS is also equipped with API functionality to interact with external databases and analysis tools, enhancing its utility in data-driven investigations. Those key features represent a critical step toward unraveling the complex landscape of protein phosphorylation in the brain, with implications for elucidating the molecular mechanisms underlying neurological disorders. KANPHOS is freely accessible to all researchers at https://kanphos.jp.
Frontiers in Psychiatry, 14, Jun 27, 2023 Peer-reviewedLast authorCorresponding author
Background
Phenotyping analysis that includes time course is useful for understanding the mechanisms and clinical management of postoperative delirium. However, postoperative delirium has not been fully phenotyped. Hypothesis-free categorization of heterogeneous symptoms may be useful for understanding the mechanisms underlying delirium, although evidence is currently lacking. Therefore, we aimed to explore the phenotypes of postoperative delirium following invasive cancer surgery using a data-driven approach with minimal prior knowledge.
Methods
We recruited patients who underwent elective invasive cancer resection. After surgery, participants completed 5 consecutive days of delirium assessments using the Delirium Rating Scale-Revised-98 (DRS-R-98) severity scale. We categorized 65 (13 questionnaire items/day × 5 days) dimensional DRS-R-98 scores using unsupervised machine learning (K-means clustering) to derive a small set of grouped features representing distinct symptoms across all participants. We then reapplied K-means clustering to this set of grouped features to delineate multiple clusters of delirium symptoms.
Results
Participants were 286 patients, of whom 91 developed delirium defined according to Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, criteria. Following the first K-means clustering, we derived four grouped symptom features: (1) mixed motor, (2) cognitive and higher-order thinking domain with perceptual disturbance and thought content abnormalities, (3) acute and temporal response, and (4) sleep–wake cycle disturbance. Subsequent K-means clustering permitted classification of participants into seven subgroups: (i) cognitive and higher-order thinking domain dominant delirium, (ii) prolonged delirium, (iii) acute and brief delirium, (iv) subsyndromal delirium-enriched, (v) subsyndromal delirium-enriched with insomnia, (vi) insomnia, and (vii) fit.
Conclusion
We found that patients who have undergone invasive cancer resection can be delineated using unsupervised machine learning into three delirium clusters, two subsyndromal delirium clusters, and an insomnia cluster. Validation of clusters and research into the pathophysiology underlying each cluster will help to elucidate the mechanisms of postoperative delirium after invasive cancer surgery.
<title>Abstract</title>Scintillators emit visible luminescence when irradiated with X-rays. Given the unlimited tissue penetration of X-rays, the employment of scintillators could enable remote optogenetic control of neural functions at any depth of the brain. Here we show that a yellow-emitting inorganic scintillator, Ce-doped Gd3(Al,Ga)5O12 (Ce:GAGG), can effectively activate red-shifted excitatory and inhibitory opsins, ChRmine and GtACR1, respectively. Using injectable Ce:GAGG microparticles, we successfully activated and inhibited midbrain dopamine neurons in freely moving mice by X-ray irradiation, producing bidirectional modulation of place preference behavior. Ce:GAGG microparticles are non-cytotoxic and biocompatible, allowing for chronic implantation. Pulsed X-ray irradiation at a clinical dose level is sufficient to elicit behavioral changes without reducing the number of radiosensitive cells in the brain and bone marrow. Thus, scintillator-mediated optogenetics enables minimally invasive, wireless control of cellular functions at any tissue depth in living animals, expanding X-ray applications to functional studies of biology and medicine.
Paul Rossener Regonia, Masahiro Takamura, Takashi Nakano, Naho Ichikawa, Alan Fermin, Go Okada, Yasumasa Okamoto, Shigeto Yamawaki, Kazushi Ikeda, Junichiro Yoshimoto
Frontiers in Psychiatry, 12 780997-780997, Nov 25, 2021 Peer-reviewedLast authorCorresponding author
Our current understanding of melancholic depression is shaped by its position in the depression spectrum. The lack of consensus on how it should be treated—whether as a subtype of depression, or as a distinct disorder altogethe—interferes with the recovery of suffering patients. In this study, we analyzed brain state energy landscape models of melancholic depression, in contrast to healthy and non-melancholic energy landscapes. Our analyses showed significant group differences on basin energy, basin frequency, and transition dynamics in several functional brain networks such as basal ganglia, dorsal default mode, and left executive control networks. Furthermore, we found evidences suggesting the connection between energy landscape characteristics (basin characteristics) and depressive symptom scores (BDI-II and SHAPS). These results indicate that melancholic depression is distinguishable from its non-melancholic counterpart, not only in terms of depression severity, but also in brain dynamics.
<title>Abstract</title>Neurofeedback (NF) aptitude, which refers to an individual’s ability to change its brain activity through NF training, has been reported to vary significantly from person to person. The prediction of individual NF aptitudes is critical in clinical NF applications. In the present study, we extracted the resting-state functional brain connectivity (FC) markers of NF aptitude independent of NF-targeting brain regions. We combined the data in fMRI-NF studies targeting four different brain regions at two independent sites (obtained from 59 healthy adults and six patients with major depressive disorder) to collect the resting-state fMRI data associated with aptitude scores in subsequent fMRI-NF training. We then trained the regression models to predict the individual NF aptitude scores from the resting-state fMRI data using a discovery dataset from one site and identified six resting-state FCs that predicted NF aptitude. Next we validated the prediction model using independent test data from another site. The result showed that the posterior cingulate cortex was the functional hub among the brain regions and formed predictive resting-state FCs, suggesting NF aptitude may be involved in the attentional mode-orientation modulation system’s characteristics in task-free resting-state brain activity.
Frontiers in Psychiatry, 12, Aug 18, 2021 Peer-reviewedLast author
Recently, the dimensional approach has attracted much attention, bringing a paradigm shift to a continuum of understanding of different psychiatric disorders. In line with this new paradigm, we examined whether there was common functional connectivity related to various psychiatric disorders in an unsupervised manner without explicitly using diagnostic label information. To this end, we uniquely applied a newly developed network-based multiple clustering method to resting-state functional connectivity data, which allowed us to identify pairs of relevant brain subnetworks and subject cluster solutions accordingly. Thus, we identified four subject clusters, which were characterized as major depressive disorder (MDD), young healthy control (young HC), schizophrenia (SCZ)/bipolar disorder (BD), and autism spectrum disorder (ASD), respectively, with the relevant brain subnetwork represented by the cerebellum-thalamus-pallidum-temporal circuit. The clustering results were validated using independent datasets. This study is the first cross-disorder analysis in the framework of unsupervised learning of functional connectivity based on a data-driven brain subnetwork.
Neural Networks, 142 269-287, May, 2021 Peer-reviewedLast author
In neuroscience, the functional magnetic resonance imaging (fMRI) is a vital tool to non-invasively access brain activity. Using fMRI, the functional connectivity (FC) between brain regions can be inferred, which has contributed to a number of findings of the fundamental properties of the brain. As an important clinical application of FC, clustering of subjects based on FC recently draws much attention, which can potentially reveal important heterogeneity in subjects such as subtypes of psychiatric disorders. In particular, a multiple clustering method is a powerful analytical tool, which identifies clustering patterns of subjects depending on their FC in specific brain areas. However, when one applies an existing multiple clustering method to fMRI data, there is a need to simplify the data structure, independently dealing with elements in a FC matrix, i.e., vectorizing a correlation matrix. Such a simplification may distort the clustering results. To overcome this problem, we propose a novel multiple clustering method based on Wishart mixture models, which preserves the correlation matrix structure without vectorization. The uniqueness of this method is that the multiple clustering of subjects is based on particular networks of nodes (or regions of interest, ROIs), optimized in a data-driven manner. Hence, it can identify multiple underlying pairs of associations between a subject cluster solution and a ROI sub-network. The key assumption of the method is independence among sub-networks, which is effectively addressed by whitening correlation matrices. We applied the proposed method to synthetic and fMRI data, demonstrating the usefulness and power of the proposed method.
Frontiers in psychiatry, 11 400-400, 2020 Peer-reviewedLast authorCorresponding author
Resting-state fMRI has the potential to help doctors detect abnormal behavior in brain activity and to diagnose patients with depression. However, resting-state fMRI has a bias depending on the scanner site, which makes it difficult to diagnose depression at a new site. In this paper, we propose methods to improve the performance of the diagnosis of major depressive disorder (MDD) at an independent site by reducing the site bias effects using regression. For this, we used a subgroup of healthy subjects of the independent site to regress out site bias. We further improved the classification performance of patients with depression by focusing on melancholic depressive disorder. Our proposed methods would be useful to apply depression classifiers to subjects at completely new sites.
We propose a novel method for multiple clustering, which is useful for analysis of high-dimensional data containing heterogeneous types of features. Our method is based on non-parametric Bayesian mixture models in which features are automatically partitioned (into views) for each clustering solution. This feature partition works as feature selection for a particular clustering solution, which screens out irrelevant features. To make our method applicable to high-dimensional data, a co-clustering structure is newly introduced for each view. Further, the outstanding novelty of our method is that we simultaneously model different distribution families, such as Gaussian, Poisson, and multinomial distributions in each cluster block, which widens areas of application to real data. We apply the proposed method to synthetic and real data, and show that our method outperforms other multiple clustering methods both in recovering true cluster structures and in computation time. Finally, we apply our method to a depression dataset with no true cluster structure available, from which useful inferences are drawn about possible clustering structures of the data.
In diagnostic applications of statistical machine learning methods to brain imaging data, common problems include data high-dimensionality and co-linearity, which often cause over-fitting and instability. To overcome these problems, we applied partial least squares (PLS) regression to resting-state functional magnetic resonance imaging (rs-fMRI) data, creating a low-dimensional representation that relates symptoms to brain activity and that predicts clinical measures. Our experimental results, based upon data from clinically depressed patients and healthy controls, demonstrated that PLS and its kernel variants provided significantly better prediction of clinical measures than ordinary linear regression. Subsequent classification using predicted clinical scores distinguished depressed patients from healthy controls with 80% accuracy. Moreover, loading vectors for latent variables enabled us to identify brain regions relevant to depression, including the default mode network, the right superior frontal gyrus, and the superior motor area.
Background: Advance in high-throughput technologies in genomics, transcriptomics, and metabolomics has created demand for bioinformatics tools to integrate high-dimensional data from different sources. Canonical correlation analysis (CCA) is a statistical tool for finding linear associations between different types of information. Previous extensions of CCA used to capture nonlinear associations, such as kernel CCA, did not allow feature selection or capturing of multiple canonical components. Here we propose a novel method, two-stage kernel CCA (TSKCCA) to select appropriate kernels in the framework of multiple kernel learning.
Results: TSKCCA first selects relevant kernels based on the HSIC criterion in the multiple kernel learning framework. Weights are then derived by non-negative matrix decomposition with L1 regularization. Using artificial datasets and nutrigenomic datasets, we show that TSKCCA can extract multiple, nonlinear associations among high-dimensional data and multiplicative interactions among variables.
Conclusions: TSKCCA can identify nonlinear associations among high-dimensional data more reliably than previous nonlinear CCA methods.
Trends in pharmacological sciences, 37(10) 858-871, Oct, 2016 Peer-reviewedInvited
Dopamine signaling in the brain is a complex phenomenon that strongly contributes to emotional behaviors. Medium spiny neurons (MSNs) play a major role in dopamine signaling through dopamine D1 receptors (D1Rs) or dopamine D2 receptors (D2Rs) in the striatum. cAMP/protein kinase A (PKA) regulates phosphorylation signals downstream of D1Rs, which affects the excitability of MSNs, leading to reward-associated emotional expression and memory formation. A combination of phosphoproteomic approaches and the curated KANPHOS database can be used to elucidate the physiological and pathophysiological functions of dopamine signaling and other monoamines. Emerging evidence from these techniques suggests that the Rap1 pathway plays a crucial role in the excitability of MSNs, leading to the expression of emotional behaviors.
Intelligent Automation and Soft Computing, 17(1) 71-94, 2011 Peer-reviewedLead author
This paper presents a variational Bayes (VB) method for normalized Gaussian network, which is a mixture model of local experts. Based on the Bayesian framework, we introduce a meta-learning mechanism to optimize the prior distribution and the model structure. In order to search for the optimal model structure efficiently, we also develop a hierarchical model selection method. The performance of our method is evaluated by using function approximation problems and an system identification problem of a nonlinear dynamical system. Experimental results show that our Bayesian framework results in the reduction of generalization error and achieves better function approximation than existing methods within the finite mixtures of experts family whim the number of training data is fairly small.
Corticostriatal synapse plasticity of medium spiny neurons is regulated by glutamate input from the cortex and dopamine input from the substantia nigra. While cortical stimulation alone results in long-term depression (LTD), the combination with dopamine switches LTD to long-term potentiation (LTP), which is known as dopamine-dependent plasticity. LTP is also induced by cortical stimulation in magnesium-free solution, which leads to massive calcium influx through NMDA-type receptors and is regarded as calcium-dependent plasticity. Signaling cascades in the corticostriatal spines are currently under investigation. However, because of the existence of multiple excitatory and inhibitory pathways with loops, the mechanisms regulating the two types of plasticity remain poorly understood. A signaling pathway model of spines that express D1-type dopamine receptors was constructed to analyze the dynamic mechanisms of dopamine- and calcium-dependent plasticity. The model incorporated all major signaling molecules, including dopamine- and cyclic AMP-regulated phosphoprotein with a molecular weight of 32 kDa (DARPP32), as well as AMPA receptor trafficking in the post-synaptic membrane. Simulations with dopamine and calcium inputs reproduced dopamine- and calcium-dependent plasticity. Further in silico experiments revealed that the positive feedback loop consisted of protein kinase A (PKA), protein phosphatase 2A (PP2A), and the phosphorylation site at threonine 75 of DARPP-32 (Thr75) served as the major switch for inducing LTD and LTP. Calcium input modulated this loop through the PP2B (phosphatase 2B)-CK1 (casein kinase 1)-Cdk5 (cyclin-dependent kinase 5)-Thr75 pathway and PP2A, whereas calcium and dopamine input activated the loop via PKA activation by cyclic AMP (cAMP). The positive feedback loop displayed robust bi-stable responses following changes in the reaction parameters. Increased basal dopamine levels disrupted this dopamine-dependent plasticity. The present model elucidated the mechanisms involved in bidirectional regulation of corticostriatal synapses and will allow for further exploration into causes and therapies for dysfunctions such as drug addiction.
IPSJ SIG technical reports, 2006(64) 35-40, Jun 15, 2006
In a computational approach to life science, computer simulation is essential for verifying hypothetical kinetic models of intracellular molecular mechanisms. However, kinetic parameters used in simulation are often unidentified or unreliable. It is highly desired to develop a method that can automatically estimate the parameters from accessible experimental measurements with high confidence. In this report, we address this estimation problem and present a Bayesian method for solving it. Our contribution is to unify the following three issues in the framework of Bayesian inference: 1) identifying unknown parameters to reproduce experimental measurements; 2) representing confidence intervals of estimated parameters; and 3) visualizing the parameter space in which the kinetic behaviors are similar to experimental measurements. Existing methods rarely dealt with the last two issues. The effectiveness of our method is demonstrated in three benchmark problems.
IEICE technical report, 106(101) 31-36, Jun 8, 2006
In a computational approach to life science, computer simulation is essential for verifying hypothetical kinetic models of intracellular molecular mechanisms. However, kinetic parameters used in simulation are often unidentified or unreliable. It is highly desired to develop a method that can automatically estimate the parameters from accessible experimental measurements with high confidence. In this report, we address this estimation problem and present a Bayesian method for solving it. Our contribution is to unify the following three issues in the framework of Bayesian inference: 1) identifying unknown parameters to reproduce experimental measurements; 2) representing confidence intervals of estimated parameters; and 3) visualizing the parameter space in which the kinetic behaviors are similar to experimental measurements. Existing methods rarely dealt with the last two issues. The effectiveness of our method is demonstrated in three benchmark problems.
IEICE technical report, 105(659) 19-24, Mar 17, 2006
Because reinforcement learning (RL) methods have an advantage such that a control rule can be obtained autonomously without any knowledge of the target system. RL methods have been successfully applied to automatic control of various robots such as balancing control of a cart-pole. However, most real control problems have non-linear dynamics with a large number of degrees of freedom, therefore it is necessary to develop an RL method to deal with such situations. This difficulty is called "curse of dimensionality" in the context of RL. We formarly proposed an RL method that switches controllers which had been developed in the field of control theory, and applied our method to an automatic control problem of an acrobot. The results showed that a good controller for a simulator can be obtained autonomously but the control of the real acrobot was not stable. In the current study, we propose an RL method which is robust against the system identification error, and show that a good controller for a real robot can be obtained by our method.
IEICE technical report, 105(657) 91-96, Mar 15, 2006
Self-localization is one of the important topics in the field of mobile robotics. Method using probabilistic models such as Kalman filter (KF) and Monte Carlo localization (MCL) are widely-used because they can effectively deal with uncertainty of the environment. In this report, we propose a self-localization method based on mixture Kalman filters, which is a hybrid of KF and MCL to exploit complementary advantages of the two methods. The effectiveness of our method is demonstrated through computer simulations and real experiments with a rat-like mobile robot Cyber Rodent.
YUKINAWA NAOTO, YOSHIMOTO JUN-ICHIRO, OBA SHIGEYUKI, ISHII SHIN
46(10) 57-65, Jun 15, 2005
Several methods based on state space models have been proposed for analyzing dynamics of gene expression. Existing analysis methods can detect false noisy internal variables which seem to have no dynamics in state space because the methods do not assume any dynamics with system noise and observation noise. In this study, we propose a linear dynamical system model in which state variables and observation variables are generated by Gaussian white noise process and provide a variational Bayes inference for the model. We first show effectiveness of our method when applied to a synthesized noisy time-series data set. We also applied our method to a published yeast cell-cycle gene expression data set, then our method could select a simpler and more plausible model than existing method did. In addition, the resultant model parameters well matched the biological considerations.
Synthetic approaches have been popular to uncover mechanisms for the emergence of communication. One of the computational problems for the emergence of communication is that both a message sender and a receiver need to finish acquiring behavioral functions which enable communication to happen. This report presents a mechanism that deals with this computational problem in a competitive multi-agent system. The key point of the mechanism is there exists some environmental and fatal factors which agents have to cope with immediately. Once acquiring a function to cope with the environmental factors, agents divert it to another issue and then communication emerges. Simulation results supported our insights : (1) a fatal factor in environment helps agents to emerge communication with others ; and, (2) communication brings higher fitness to agents.
IEICE technical report. Artificial intelligence and knowledge-based processing, 104(727) 19-24, Mar 8, 2005
Synthetic approaches have been popular to uncover mechanisms for the emergence of communication. One of the computational problems for the emergence of communication is that both a message sender and a receiver need to finish acquiring behavioral functions which enable communication to happen. This report presents a mechanism that deals with this computational problem in a competitive multi-agent system. The key point of the mechanism is there exists some environmental and fatal factors which agents have to cope with immediately. Once acquiring a function to cope with the environmental factors, agents divert it to another issue and then communication emerges. Simulation results supported our insights : (1) a fatal factor in environment helps agents to emerge communication with others ; and, (2) communication brings higher fitness to agents.
YUKINAWA NAOTO, YOSHIMOTO JUN-ICHIRO, OBA SHIGEYUKI, ISHII SHIN
IPSJ SIG Notes, 51(92) 13-16, Sep 13, 2004 Peer-reviewed
Several methods based on state space models have been proposed for analyzing dynamics of gene expression. Existing analysis methods can detect false noisy internal variables which seem to have no dynamics in state space because the methods don't assume any dynamics with system noise and observation noise. In this study, we propose a linear dynamical system model in which state variables and observation variables are generated by Gaussian white noise process and we provide a variational Bayes inference for the model. We first show effectiveness of our method for synthesized noisy time-series data set. We also apply our method to a published yeast cell-cycle gene expression data set. then show that our method could select simpler and more plausible model than existing method does. In addition, the resultant model parameters well match the biological considerations.
IEICE technical report. Neurocomputing, 103(734) 61-66, Mar 12, 2004
For controlling mobile robots within the framework of statistical inference, precise models of state transition and observation processes are crucial. Although these models can be obtained if the robot makes use of external information, such a setting makes the robot's applicability narrow and the implementation costly. In this study, we propose a method to identify these models from the robot's own observation. In this method, an environment is formulated as a parameterized probabilistic model in consideration of a geometric property and sensory non-linearity. The unknown parameters are determined based on the maximum likelihood estimation. We evaluated the performance of the proposed method using synthetic and real datasets. The results showed that the method successfully obtained the environmental model with a fairly good precision only from the robot's own sensory information.
IEICE technical report. Neurocomputing, 103(734) 97-102, Mar 12, 2004
A brain needs to detect an environmental change and to quickly learn internal representations necessary in a new environment. This report presents a theoretical model of cortical representational learning that can adapt to dynamic environments, incorporating the results by previous studies on a functional role of acetylcholine (ACh). We adopt the probabilistic principal component analysis (PPCA) as a functional model of cortical representational learning, and present an on-line learning method for PPCA according to Bayesian inference. Our approach is examined in two types of simulations with synthesized and realistic datasets, in which our model is able to re-learn new representation bases after environmental changes. We suggest that the function of ACh corresponds to the learning rate in our learning model and describe both biological and computational studies related to our model.
IEICE technical report. Neurocomputing, 103(734) 115-120, Mar 12, 2004
In this report, we propose a novel approach to acquiring the optimal policy for continuous Markov decision processes. Based on an analogy from statistical mechanics, we introduce a variational free energy over a policy. An exactly or approximately optimal policy can be obtained by minimizing the variational free energy. According to our approach, the optimal policy in linear quadratic regulator problems can be obtained by using Kalman filtering and smoothing techniques. Even in non-linear problems, a semi-optimal policy can be obtained by Monte Carlo technique with a Gaussian process method.
IEICE technical report. Neurocomputing, 103(465) 45-50, Nov 14, 2003
Reinforcement learning(RL) has recently been applied to contracting a controller for nonlinear systems. Merits of the RL are that an exact dynamics model of the controlled object is not required and adaptability to non-stationary environments. If the dynamics has strong nonlinearity and/or high dimensionality, however, the RL suffers from unstable learning and/or a huge number of learning episodes. For this reason, a naive RL method is not suitable for the tasks in a real world. In order to overcome the disadvantage of the RL, we propose a new RL scheme using multiple incomplete controllers, each of which is constructed based on a locally linear dynamics in a specific subspace. An inverted-pendulum task and an acrobot control task on computer simulation showed that our method was able to select an appropriate incomplete controller for each subspace, so that a fairly good nonlinear control was realized.
IEICE technical report. Neurocomputing, 102(731) 131-136, Mar 12, 2003
In this report, we present an on-line variational Bayes (VB) method for system identification based on linear state space models. The learning algorithm is implemented as the maximization of an on-line free energy, which can be used for determining the dimension of the internal state. We also propose a reinforcement learning (RL) method using this system identification method. Our RL method is applied to a simple automatic control problem with hidden state variables. The result shows that our method is able to determine correctly the dimension of the internal state and to acquire a good control, even in a partially observable environment.
IEICE technical report. Neurocomputing, 102(731) 137-142, Mar 12, 2003
Actor-critic methods in reinforcement learning (RL) consist of two modules; one is a critic that estimates a value function and the other is an actor that improves an action policy. Unlike other RL methods such as Q-learning and SARSA, the actor-critic methods can select an action with a low computational cost because the action policy is represented explicitly by the actor. This enables the methods to be applied to various problems with continuous action spaces. However, the training of the methods is often unstable because the actor and the critic are dependent on each other. In order to solve this difficulty, an actor-critic method based on a stochastic policy gradient method, whose convergence is guaranteed, has been proposed recently. In this report, we apply that method to an automatic control problem of a continuous dynamical system and discuss the performance.
NAKAMURA Yutaka, OBA Shigeyuki, YOSHIMOTO Junichiro, ISHII Shin
IEICE technical report. Neurocomputing, 102(729) 149-153, Mar 10, 2003
Human motions have been studied for gesture recognition, computer graphics, robot control with referring human motions, and so on. A linear state space model is often used for modeling and analyzing such motions. In this report, we present an system identification study of human pointing motions from observation by a motion capture system. We assume the pointing motion can be approximated by a linear state space model, and the system identification is done by an online Bayes method. Based on a Bayesian criterion, we find that the effective dimensionality of human pointing motions is not high.
IEICE technical report. Artificial intelligence and knowledge-based processing, 100(88) 29-36, May 18, 2000 Lead author
In this report, we propose a new reinforcement learning(RL)method for continuous dynamical systems by using function approximation and stochastic learning. Our RL method has an architecture like the actor-critic model. The critic tries to approximate the Q-function, which is the expected future return for the current state-action pair. The actor tries to approximate a stochastic soft-max policy defined by the Q-function. The soft-max policy is more likely to select an action that has a higher Q-function value. The on-line EM algorithm is used to train the critic and the actor. We apply this method to two control problems. Computer simulations show that our method is able to acquire faurly good control in the two tasks after a few learning trials.
IEICE technical report. Neurocomputing, 99(684) 165-172, Mar 13, 2000 Lead author
In this report, we propose a new reinforcement learning(RL)method for dynamical systems that have continuous state and action spaces. Our RL method has an architecture like the actor-critic model. The critic tries to approximate the Q-function, which is the expected future retun for the current state-action pair. The actor tries to approximate a stochastic soft-max policy defined by the Q-function. The soft-max policy is more likely to select an action that has a higher Q-function value. The on-line EM algorithm is used to train the critic and the actor. We apply this method to two control problems. Computer simulations show that our method is able to acquire fairly good control in the two tasks after a few learning trials.
Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research on Innovative Areas (Research in a proposed research area), Japan Society for the Promotion of Science, Apr, 2010 - Mar, 2015
ISHII Shin, OBA Shigeyuki, MAEDA Shinichi, YOSHIMOTO Junichiro, SAKUMURA Yuichi