Lab Research Themes


   The main research interests of our lab are Computational Intelligence including neural networks, fuzzy systems and genetic algorithm, especially deep learning and its applications. Our researches mainly focus on how to apply those technologies to prediction, classification, clustering and optimization in various real applications, such as pattern recognition and classification, system identification and control, data mining, financial analysis and so on. Based on the information extracted automatically from application domain, we are trying to develop application specific new efficient computational intelligent technologies, for examples, Quasi-linear support vector machines, Quasi-linear ARX neural networks, Deep neural networks, Deep kernel learning.

Self-Organizing Function Localization Neural Networks


   In the book of Organization of Behavior, D.O. Hebb proposed two radically new theories about how the brain worked. The first idea is later known as Hebbian learning; the second one is known as Cell assemblies. In the Cell assemblies theory, neurons would form many groups thanks to Hebbian learning. Specific groups of neurons would be activated corresponding to specific sensory information the brain receives. Additionally, the formed groups would have neuron overlapped between the other groups. That is, neurons would be mutually connected ‘functionally' rather than ‘structurally', and the connections would vary appropriately according to the sensory information. We call this as a capability of function localization. If we consider the formed loops as modules, the brain may be seen to consist of many overlapping modules.
   On the other hand, it is recently suggested that the three parts of brain: cerebellum, cerebral cortex and basal ganglia are specialized, respectively, in supervised, unsupervised and reinforcement learning. Although our brain is a highly complicated structure and has many capabilities which are not entirely clear, we believe that our brain is not working with a single learning paradigm, but working with a combination of different learning paradigms. From these known knowledge of brain, we are expecting to develop a more sophisticated learning system by combining different learning algorithms.
   Inspired by the above knowledge of brain, in this research we present a self-organizing function localization neural network consisting of three parts: supervised learning (SL) part, unsupervised learning (UL) part and reinforcement learning (RL) part. The SL part is a main part learning input-output mapping; the UL part is a competitive network dividing input space into subspaces and realizes the capability of function localization by controlling firing strength of neurons in the SL part based on input patterns; the RL part is a reinforcement learning scheme, which optimizes system performance by adjusting the parameters in the UL part.
  The idea of self-organizing function localization neural network structure are further extended and applied to various applications in our researches.
  • T. Sasakawa, J. Hu and K. Hirasawa, "A Brainlike Learning  System with Supervised, Unsupervised, and Reinforcement Learning", Electrical Engineering in Japan, Vol.162, No.1, pp.32-39, 2008.
  • 笹川・古月・平澤, "教師あり学習・教師なし学習・強化学習を複合したbrain- like学習システム", 電気学会論文誌C, Vol.126, No.9, 1165-1172, 2006.
  • 笹川・古月・平澤, "自己組織化機能局在型ニューラルネットワーク", 計測自動制御学会論文集, Vol.41, No.1, pp.67-74, 2005.
  • T.Sasakawa, J.Hu, K.Isono and K.Hirasawa, "Effective Training Methods for Function Localization Neural Networks", in Proc. of  International Joint Conference on Neural Networks (Vancouver), 7, 2006, pp.9535-9540
  • T.Sasakawa, J.Hu and K.Hirasawa, "Performance Optimization of Function Localization Neural Network by Using Reinforcement Learning", in Proc. of International Joint Conference on Neural Networks (Montreal), 8, 2005, pp.1314-1319
  • J.Hu, K.Hirasawa and Q.Xiong, " Overlapped Multi-Neural-Network and Its Training Algorithm ", Trans. of the Institute of Electrical Engineers of Japan, Vol.121-C, No.12, 1949-1956, 2001.

Quasi-ARX Modeling and Identification


   Neural networks (NNs) and neurofuzzy networks (NFs) have been proved to have universal approximation ability. They can learn any nonlinear mapping. Many nonlinear ARMAX models have been proposed based directly on NNs and NFs. However, system identification is always followed by certain applications such as system control and fault diagnosis. From a user's point of view, NNs and NFs are not user-friendly since they do not have structures favorable to the applications of system control and fault diagnosis. To solve this problem, it is natural to consider a modeling scheme to construct models consisting of two parts: macro-part and kernel-part. The macro-part is a user-friendly interface constructed using application specific knowledge and the nature of network structure; efforts in this part are made to introduce some properties favorable to certain applications, while to embed the resulted model complexity in the coefficients. The kernel-part is a flexible multi-input multi-output (MIMO) nonlinear model such as NN and NF, etc. which is used to represent the complicated coefficients of macro-parts. Nonlinear models constructed in this way are expected to be user-friendly and to have excellent presentation ability.
   It is obviously that the above modeling scheme is application oriented because application specific knowledge is used to construct the macro-part. For different application interests, different user-friendly models are to be developed. In this research, we proposed a class of quasi-ARX models for nonlinear systems. Similar to ordinary nonlinear ARX models, the quasi-ARX models are flexible black-box models, but they have various linearity properties similar to those of linear ARX model.
   It is shown that the proposed quasi-ARX models have both good approximation ability and some easy-to-use properties. The proposed models have been successfully applied to prediction, fault detection and adaptive control of nonlinear systems.
  • L. Wang, Y. Cheng and J. Hu, "A Quasi-ARX Neural Network with Switching Mechanism to Adaptive Control of Nonlinear Systems", SICE Journal of Control, Measurement, and System Integration, Vol.3, No.4, 2010.
  • L. Wang and J. Hu, "An Improvement of Quasi-ARX Predictor to Control of Nonlinear Systems Using Nonlinear PCA Network", in Proc. of ICROS-SICE International Joint Conference 2009 (Fukuoka), 8, 2009, pp.5095-5099.
  • 古月 敬之, "第二章 線形特性を有するニューラルネットワーク",  渡辺桂吾編著, 「ニューラルネットワーク計算知能」, pp.27-49, 森北出版株式会社, 東京, 2006
  • J.Hu and K.Hirasawa, "A Method for Applying Neural Networks to Control of Nonlinear Systems", in book entitled Neural Information Processing: Research and Development,  J.C.Rajapakse and L.Wang, Eds, pp..351-369, Springer, Berlin, GERMANY, 2004.
  • 古月・平澤, "非線形システムの制御のためのニューラルネットワーク予測モデル", 計測自動制御学会論文集, Vol.39, No.2, pp.168-175, 2003
  • J.Hu, K.Kumamaru and K.Hirasawa, " A Quasi-ARMAX Approach to the Modeling of Nonlinear Systems ", International Journal of Control, Vol.74, No.18, pp.1754-1766, 2001.
  • J.Hu, K.Kumamaru and K.Hirasawa, "A Neurofuzzy Approach to Fault Detection of Nonlinear Systems ",  Journal of Advanced Intelligence, Vol.3, No.6, pp.524-531,1999.
  • J.Hu, K.Hirasawa and K.Kumamaru, " A Neurofuzzy-Based Adaptive Predictor for Control of Nonlinear Systems ", Trans. of the Society of Instrument and Control Engineering, Vol.35, No.8, pp.1060-1068,1999.
  • J.Hu, K.Kumamaru, K.Inoue and K.Hirasawa, " KDI-Based Robust Fault Detection in Presence of Nonlinear Undermodeling", Trans. of the Society of Instrument and Control Engineering, Vol.35, No.2, pp.200-207,1999
  • J.Hu, K.Kumamaru, K.Inoue and K.Hirasawa, " A Hybrid Quasi-ARMAX Modeling Scheme for Identification of Nonlinear Systems ", Trans. of the Society of Instrument and Control Engineering, Vol.34, No.8, pp.977-985,1998.

Yin-Yang EAs Balancing Adaptivity and Diversity


   Evolutionary algorithms (EAs) are search and optimization algorithms based on the principles of natural evolution, which have found successful applications in biogenetics, computer science, engineering, economics, chemistry, manufacturing, mathematics, physics and other fields.
   In applying EAs to solve large-scale real world problems, however, confronted with the conflict between accuracy and speed, EAs often result in an unsatisfactory compromise. Furthermore, one of the commonest difficulties encountered is premature convergence.
   In a conventional EA, individuals are selected through a fitness-based procedure, the decrease of population diversity leads to premature convergence. To solve this problem, the population diversity should be incorporated into the selection mechanism. If the fitness is pursued excessively, we are actually conducting a local search, where the diversity declines sharply so that the global optimization becomes more difficult and even impossible; However, when we care about the diversity too much, it turns out to be a blindness random search which does not result in any rise in fitness, leaving us far from the global optimum. Therefore, we should seek for an appropriate equilibrium point between individual fitness and population diversity, which seems to be the key to a successful global optimization.
   This reminds us of Yin-Yang theory. Yin and Yang are the dual concepts originated in ancient Chinese philosophy and metaphysics, which describe two opposing but complementary principles in all non-static objects and processes in the universe. Yin and Yang are mutually coupled in equilibrium or even a harmony to jointly face a same world with shared tasks. It is shown that the balance between Yin and Yang is crucial to various complex systems such as human body in terms of health and robustness.
   Genetic Algorithms (GAs) with reserve selection mechanism and Adaptive niching Estimation Distribution Algorithms (EDAs) were developed to balance the adaptivity and diversity. Strategies were introduced to seek for an optimal equilibrium point between individual fitness and population diversity.
  • B. Chen and J. Hu, "An Adaptive Niching EDA Based on Clustering Analysis", in Proc. of 2010 IEEE Congress on Evolutionary Computation (CEC'10) (Barcelona), July 2010
  • Y. Chen, J. Hu, K. Hirasawa and S. Yu, "Performance Tuning of Genetic Algorithm with Reserve Selection", in Proc. of 2007 IEEE Congress on Evolutionary Computation (CEC2007) (Sigapore), 9, 2007, pp.2202-2209.
  • Y. Chen, J. Hu, K. Hirasawa and S. Yu, "Optimizing Reserve Size in Genetic Algorithms with Reserve Selection Using Reinforcement Learning", in Proc. of SICE Annual Conference 2007 (Kagawa), 9, 2007, pp.1341-1347.
  • Y. Chen, J. Hu, K. Hirasawa and S. Yu, "GARS: An Improved Genetic Algorithm with Reserve Selection for Global Optimization", 2007 Genetic and Evolutionary Computation Conference (GECCO2007),  University College London, London, England, 7, 2007, pp. 1173-1178.

Support Vector Machines for Real-world Pattern Recognition


   SVM is a nonlinear pattern recognition algorithm based on kernel methods. In contrast to linear methods, kernel methods map the original parameter vectors into a higher (possibly infinite) dimensional feature space through a nonlinear kernel function. Without need to compute the nonlinear mapping explicitly, dot-products can be computed efficiently in higher dimensional space. The dominant feature which makes SVM very attractive is that classes which are nonlinearly separable in the original space can be linearly separated in the higher dimensional feature space. Thus SVM is capable to solve complex nonlinear pattern recognition problems. Important characteristics of SVM are its ability to solve pattern recognition problems by means of convex quadratic programming (QP), and also the sparseness resulting from this QP problem.
  Although SVM-based methods show advantages in terms of generalization performance and the recognition accuracy, the pattern recognition of real-world data using SVM still faces several challenges that we are trying to solve.
  Huge training data: To make the training of classifier executable, most learning algorithms require a suitable amount of training data which scales with the number of inputs. SVM is one of the kernel methods and formulated as quadratic programming (QP) problems. The training time and space complexities have exponential relationships with the size of training data. Hence, a major stumbling block in SVM is the high training time and space complexities for large datasets, which is commonly encountered in real-world pattern recognition applications.
  High dimensional input space: Depending on the acquisition resolution, many real-world databases consist of hundreds to thousands of measurements. While the higher dimension of this input potentially makes classifier a unique and powerful technique for a certain application, on the other hand it complicates the computation and the design of an appropriate method to handle it.
  Noise and interaction: Since the acquisitions (training data and test data) are usually obtained from real-world, thus these databases are usually affected by interaction and many kinds of noises between classes. In most actual designs of classifiers, noises and interaction make the boundary between classes not clear. Many noise reduction approaches have proposed. However, it is expected that the classifier should be robust against these imperfections.
  Imbalance in database: In most actual classification applications, databases are usually unbalanced. That is, the size of one class is commonly much larger than the others. This phenomenon widely exists in the real-world and it is the main reason for causing the excursion of separation boundaries in SVM classifiers. Thus, it is required to construct a classifier which can modify the separation boundaries and overcome the excursion.
  Furthermore, based on the principle of ‘divide-and-conquer’, we are also developing modular SVM regression systems to solve the complicated problems of time series prediction and multiple SVM classifier systems to solve the complicated problems of data classification.
  • Boyang Li, "Study on Multi-SVM Systems and Their Application to Pattern Recognition", Waseda University, 7, 2010
  • B. Li, Q. Wang and J. Hu, "Fast SVM Training Using Edge Detection on Very Large Datasets", IEEJ Trans. on Electrical and Electronic Engineering, Vol.8, No.3, pp.229-237, May, 2013.
  • B. Li, Q. Wang and J. Hu, "Multi-SVM Classifier Systems with Piecewise Interpolation", IEEJ Trans. on Electrical and Electronic Engineering, Vol.8, No.2, pp.132-138, March, 2013.
  • B. Li, Q. Wang and J. Hu, "Feature Subset Selection: A Correlation-Based SVM Filter Approach", IEEJ Trans. on Electrical and Electronic Engineering, Vol.6, No.2, pp.173-179, March, 2011.
  • Q. Wang, B. Li and J. Hu, "Human Resource Selection Based on Performance Classification Using Weighted Support Vector Machine", Journal of Advanced Computational Intelligence and Intelligent Informatics, Vol.13, No.4, pp.407-417,  2009.
  • B. Li, Q. Wang and J. Hu, "A Fast SVM Training Method for Very Large Datasets", in Proc. of International Joint Conference on Neural Networks (Atlanta), June 2009, pp.1784-1789.
  • B. Li, J. Hu and K. Hirasawa, "Support Vector Machine Classifier with WHM Offset for Unbalanced Data",  Journal of Advanced Computational Intelligence and Intelligent Informatics, Vol.12, No.1, pp.94-101, 2008.
  • B. Li, J. Hu and Hirasawa, "Financial Time Series Prediction Using a Support Vector Regression Network",  in Proc. of  International Joint Conference on Neural Networks (IJCNN2008) (Hongkong), 6, 2008, pp.622-628.
  • B. Li,  J. Hu and K. Hirasawa, "An Improved Support Vector Machine with Soft Decision-Making Boundary", in Proc. of 2008 the IASTED  International Conference on Artificial Intelligence and Applications (AIA 2008) (Innsbruck),  2, 2008,  pp.40-45.

Machine Learning Methods in Bioinformatics Applications


   The proteomics is an important domain where machine learning techniques are applied in bioinformatics. In the proteomics, two main applications of computational methods are protein structure prediction and protein function prediction. Generally, the first is an optimization problem and the second is a classification problem. Evolutionary algorithm (EA) based methods are the main optimization technologies for protein structure prediction, such as genetic algorithm (GA), estimation distribution algorithm (EDA), etc. Supervised and unsupervised classification methods are often used to predict protein function, such as Clustering, SVM, NN, etc.
 How to extract the useful information from biological data is one of the main challenges in computational biology. For the complicated problems in proteomics, such as the long protein sequence structure prediction and multi-label protein function classification, applying machine learning methods simply usually cannot obtain expectable results. In our research, improved machine learning methods are explored for solving complicated proteomic applications. The information of the biology data is extracted and used to guide the training procedure of machine learning methods, and some delicate techniques are also combined according to the characteristics of application problem.
 Protein Structure Prediction is to predict protein three-dimensional structure (tertiary structure) from its amino acid sequence (primary structure). The EDA based methods are explored to solve the protein HP model structure prediction and side chain placement problems. A hybrid EDA with composite fitness function and local search is proposed to solve protein lattice HP model folding problem. And a niching EDA method based on clustering analysis and balance searching is also explored to solve another important structure prediction problem, the protein side-chain prediction.
 Protein Function Classification is to classify the protein functions according to its sequence data. The hierarchical multi-label protein function classification is a very difficult task in this domain. An improved multi-label classification method based on SVM with delicate decision boundary is proposed to solve multi-label protein function classification. In current research, more efficient machine learning methods will be explored to solve the hierarchical multi-label FunCat classification problem.
  • B. Chen and J. Hu, "Hierarchical Multi-label Classification Based on Over-sampling and Hierarchy Constraint for Gene Function Prediction", IEEJ Trans. on Electrical and Electronic Engineering, Vol.7, No.2, pp.183-189, March, 2012.
  • B. Chen and J. Hu, "An Adaptive Niching EDA with Balance Searching Based on Clustering Analysis", IEICE Trans on Fundamentals of Electronics, Communications and Computer Sciences, Vol.E93-A, No.10, Oct. 2010.
  • B. Chen, W. Gu and J. Hu, "An Improved Multi-label Classification Method and Its Application to Functional Genomics", Int. J. Computational Bilogy and Drug Design, Vol.3, No.2, pp.133-145, 2010.
  • B. Chen and J. Hu, "An Adaptive Niching EDA Based on Clustering Analysis", in Proc. of 2010 IEEE Congress on Evolutionary Computation (CEC'10) (Barcelona), July, 2010
  • B. Chen and J. Hu, "A Hybrid EDA for Protein Folding Based on HP Model", IEEJ Trans. on Electrical and Electronic Engineering, Vol.5, No.4, pp.459-466, July 2010.
  • B. Chen, L. Ma and J. Hu, "An Improved Multi-label Classification Method Based on SVM with Delicate Decision Boundary", International Journal of Innovative Computing, Information and Control, Vol.6, No.4, pp.1605-1614, April 2010.
  • B. Chen and J. Hu, "A Novel Clustering Based Niching EDA for Protein Folding",  in Proc. of World Congress on Nature and Biologically Inspired Computing (NaBIC 2009) (India), 12, 2009, pp.748-753.

Modeling, Identification and Control of Nonlinear Systems


  Modeling, identification and control are the art and science of dynamic system analysis and application. They can be seen as the interfaces between the real world of applications and the mathematical world of control theory and model abstractions. Till now, there are many scientific communities are dedicated to the research of this area, as one of the most active groups, our works are mainly on the theory and technology development by means of Machine Learning.
 Constructing models and identification from observed data are fundamental elements in system science. However, it is not an easy job due to the noisy and complexity for the real world dynamic systems. Consider the fact that machine learning technologies such as Genetic Algorithm and Neural Networks provide powerful tools to automatically evolve behaviors and make intelligent decisions based on empirical data; it is natural to combine these algorithms with traditional methods.
 Nonlinear system control is the area of control engineering specifically involved with the systems that are nonlinear, time-variant, or both. A lot of techniques that are used for nonlinear systems come from linear systems, because: 1) Nonlinear systems can (sometime) be approximated by linear systems. 2) Nonlinear systems can (sometime) be “transformed” into linear systems. 3) The tools are generalized and extended. However, linearity is idealization and a lot of phenomena are only present in nonlinear systems. The design and analysis of nonlinear control systems requires more sophisticated mathematical tools than are typically used for linear systems
   Model structure selection: Model structure selection for nonlinear systems is a difficult problem because, in nonlinear system identification and function (signal) approximation, there are a large number of terms or basis functions existed, and the number increased with the number of input variables exponentially. Even with state of art heuristic methods, the processing is with low efficiency. In our research, a hierarchical selection method was proposed, which deal with this problem with two steps. In the first step, traditional selection algorithms with correlation analysis and orthogonal least square are applied to reduce the candidate pool effectively. In the second step, multi-objective evolutionary algorithm is implemented to refine the selection result. The idea of hierarchical is proved to select model structure efficiency, and machine learning technology could be well complemented to the traditional identification method.
   System identification based on Qusai-ARX models: Qusai-ARX models are a class of nonlinear models developed in our Lab, in which a group of nonlinear nonparametric models (NNMs) are embedded into the coefficients of a linear ARX structure. Thanks to the NNMs, it is possible to identify the models with some extra freedoms. Some works have been done to find a better compromise to the trade-off between model flexibility and simplicity by incorporating different NNMs such as fuzzy system, RBF networks, neural networks and wavelet networks.
   Control based on Q-ARX model: An adaptive control law is proposed for nonlinear dynamical systems based on the characteristic of quasi-ARX neural network structure, and the stability of control system is proved. The quasi-ARX neural network is divided into two parts: the linear part is used to ensure the nonlinear control stability, and the nonlinear part is utilized to improve the control accuracy, which is realized by introducing a switching law based on system input-output variables and prediction errors.
  • L. Wang, Y. Cheng and J. Hu, "Stabilizing Switching Control for Nonlinear System Based on Quasi-ARX Model", IEEJ Trans. on Electrical and Electronic Engineering, Vol.7, No.4, pp.390-396, July, 2012.
  • L. Wang, Y. Cheng and J. Hu, "A Quasi-ARX Neural Network with Switching Mechanism to Adaptive Control of Nonlinear Systems", SICE Journal of Control, Measurement, and System Integration, Vol.3, No.4, pp.245-252, 2010.
  • Y. Cheng, L. Wang and J. Hu, "A Two-step Scheme for Polynomial NARX Model Identification Based on MOEA with Pre-screening Process", IEEJ Trans. on Electrical and Electonic Engineering, Vol.6, No.3, 2011.
  • L. Wang, Y.  Cheng and J. Hu, "Nonlinear Adaptive Control Using a Fuzzy Switching Mechanism Based on Improved Quasi-ARX Neural Network", in Proc of 2010 IEEE International Joint Conference on Neural Networks (IJCNN'10)(Barcelona), July 2010
  • J.Hu and K.Hirasawa, "A Method for Applying Neural Networks to Control of Nonlinear Systems", in book entitled Neural Information Processing: Research and Development,  J.C.Rajapakse and L.Wang, Eds, pp..351-369, Springer, Berlin, GERMANY, 2004.
  • 古月・平澤, "非線形システムの制御のためのニューラルネットワーク予測モデル", 計測自動制御学会論文集, Vol.39, No.2, pp.168-175, 2003.
  • J.Hu, K.Hirasawa and K.Kumamaru, " A Neurofuzzy-Based Adaptive Predictor for Control of Nonlinear Systems ", Trans. of the Society of Instrument and Control Engineering, Vol.35, No.8, pp.1060-1068,1999.

Medical Image Processing


   Medical images from computer tomography (CT), magnetic resonance imaging (MRI), microscope and other imaging modalities are playing increasingly important roles in many fields of medical and laboratory research and clinical practice. Accordingly, analyzing these images for computer-aided diagnosis and therapy planning is becoming a more challenging research task.
   Kidney disease is any disease or disorder that affects the function of the kidneys, which may aggravate quality of life or even affect mortality of patients. A renal biopsy (also known as a kidney biopsy) is an important test to diagnose and determine the severity of a kidney disorder. However, analysis consistency is quite difficult to be maintained due to individual difference of clinicians, staining and specimen quality. So our purpose is to develop a method for analyzing renal biopsy images automatically.
   The genetic algorithm (GA) based methods and neural network (NN) based methods have been proposed for segment the region of interest (ROI) from the renal biopsy images. To provide medical doctors a visual representation of information, a three dimensional (3D) reconstruction for renal biopsy image sequence is under consideration.
  • J. Zhang, J. Hu and H. Zhu, "Contour Extraction of Glomeruli by Using Genetic Algorithm for Edge Patching", IEEJ Trans. on Electrical and Electonic Engineering, Vol.6, No.3, 2011.
  • J. Zhang and J. Hu, "Color Quantization Based on Hierarchical Frequency Sensitive Competitive Learning", Journal of Advanced Computational Intelligence and Intelligent Informatics, Vol.14, No.5,  2010.
  • J. Zhang, J. Hu and H. Zhu, "Extraction of Glomeruli Using a Canny Operator with a Feedback Strategy", JAMIT Medical Imaging Technology, Vol.28. No.2, pp.127-134, March 2010.
  • J. Zhang and J. Hu, "Renal Biopsy Image Segmentation Based on 2-D Otsu Method with Histogram Analysis", JAMIT Medical Imaging Technology, Vol.27, No.3, pp.185-192, May, 2009.

Motor Imagery Based BCI


Brain-Computer Interface (BCI) is a system provides an alternative communication and control channel between the human brain and computer. In other word, subjects can communicate with computers and equipments without the participation of their muscles and peripheral nerves. For a long time, electroencephalogram (EEG) has been chosen to capture brain wave for BCI applications because of its safety, simplicity, and high temporal resolution. Motor imagery can modify the neuronal activity in the primary sensorimotor areas in a very similar way as observable with a real executed movement. Motor Imagery has been one of the most effective methodologies employed in EEG-based BCIs.
 Our current research on EEG-based BCIs is focused on (1) Feature extraction based on Common Spatial Pattern (CSP) and (2) Feature selection and classification for Motor Imagery based BCI.

  • G. Sun, J. Hu and G. Wu, "A Novel Frequency Band Selection Method for Common Spatial Pattern in Motor Imagery Based Brain Computer Interface", in Proc of 2010 IEEE International Joint Conference on Neural Networks (IJCNN'10)(Barcelona), July 2010

Human Resource Selection System with AI


 Artificial Intelligence (AI) is a branch of computer science that deals with intelligent behavior, learning, and adaptation in machines. Artificial Intelligence has successfully been used in a wide range of fields, especially in industry. However, Artificial Intelligence is also an effective instrument in economics and management. Its application in stock trading shows this ability well. More success using Artificial Intelligence technologies to solve economics and management problems are expected.
  Our research aims to develop a human resource selection system which can further improve the effect for selecting human resource for the organizations. Human resource selection aims at finding the right person for the right position. Traditional selection method is based on linear model. Job performance is the measurement for human resource selection. Traditional linear method recognizes good job performance subjectively and in advance. Sometimes it can not find the right person. In fact, the selection decision is influenced by many factors. It is very intricate. Therefore, human resource selection system can be seen as a nonlinear system. AI technologies are applied for this system.
  Three main problems in human resource selection system are selected as research items. The description of these problems, the research target and the solutions corresponding are list in the following.
  (1) Model the nonlinear classification with Support Vector Machine (SVM): Selection appraisal criterions are used as system input and job performance is system output. The selection process is seen as a performance classification problem. Candidates are expected to be predicted in the real job performance level. The key classification method here is SVM. It gives the feedback of learning result to the system to modify the learning process. It will get better performance than traditional linear method.
  (2) Improve the classification accuracy with Weighted SVM (WSVM): In human resource selection problem, samples are not equaled important different selection criterions have different contribution for target job. Therefore, we need to reduce the effect of outliers and noise and to emphasis some important criterions. The key method here is WSVM. Each data point is assigned a different weight according to the different import. In this item, a new weight generation method is proposed.
   (3) Make feature selection for human resource selection system: High classification accuracy is always the purpose for classification problem. However, this process often brings high computational complexity which takes more resource. The appraisal of selection criterions cost a lot for organizations. It is needed to find the crucial features and get rid of the noisy features for classification. we propose a new feature selection method. This is an integrative hybrid method using Affinity Propagation and SVM sensitivity analysis, and then use forward selection and backward elimination method to optimize the feature subset.
  • Q. Wang, B. Li and J. Hu, "Human Resource Selection Based on Performance Classification Using Weighted Support Vector Machine", Journal of Advanced Computational Intelligence and Intelligent Informatics, Vol.13, No.4, pp.407-417,  2009.
  • Q. Wang, J. Hu and Y. Zhou, "Weighted Support Vector Machine with Combination Weighting Method for Human Resource Selection", in Proc. of  the 3rd International Symposium on Computational Intelligence and Industrial Applications (Dali), 11, 405-413, 2008
  • Q. Wang, B. Li and J. Hu, "Feature Selection for Human Resource Selection Based on Affinity Propagation and SVM Sensitivity", in Proc. of World Congress on Nature and Biologically Inspired Computing (NaBIC 2009) (India), 12, 2009, pp.31-36.

Stock Evaluation System Based on Quasi-Linear Regression Models


 Stock investment is one of the most important investment activities in the world. There are many researches focusing on stock market. Building stock evaluation system is an important one among them, which can provide more reasonable information for investors. The expert questionnaire method is commonly used method to build the stock evaluation system in traditional methods. The stock experts’ experiences are utilized to determine the importance stock indexes in the expert questionnaire method. However, since each expert comes from different fields, and each of them has different experience which might give different importance levels of stock indexes, this may affect the performance of the stock evaluation system built.   
   On the other hand, since there are available a large number of stock datasets (stock database), it is possible to develop stock evaluation systems by using machine learning methods. In this research we first propose a quasi-linear regression model consisting of linear part and nonlinear part, then apply the model to develop an improved stock evaluation system. Instead of using expert questionnaire, machine learning method is applied to estimate the model parameters from the past stock datasets. Hierarchical algorithm is developed to identify the quasi-linear regression model in such a way that the linear part describes the importance of stock indexes and the nonlinear part improves the accuracy of the forecast. Therefore our stock evaluation system can help investors to judge the importance of stock indexes and select the stocks of higher return. It can solve the problems of traditional expert system and also increase the accuracy of the forecast.
  • Y. Lin, W. Shih and J. Hu, "Development of Stock Evaluation System Based on Quasi-Linear Regression Model", International Journal of Electronic Business Management, Vol.11, No.1, pp.23-32, 2013.

Quasi-linear Support Vector Machines for Nonlinear Classification


  Quasi-linear ARX models are a class of nonlinear models with easy-to-use structure. A quasi-linear ARX model can be seen as a nonlinear model consisting of multiple local linear models with interpolation and is shown to have good generalization performance in identifying nonlinear systems. In this research, we extend the quasi-linear ARX modeling idea to solve classification problem by applying the quasi-linear ARX model to approximate the nonlinear separation hyperplane. Consequently, a quasi-linear support vector machine (SVM) is proposed, which is an SVM with a composite quasi-linear kernel.
  In the quasi-linear SVM model, the nonlinear separation hyperplane is approximated by multiple local linear models with interpolation. Instead of building multiple local SVM models separately, the quasi-linear SVM realizes the multi local linear model approach in the kernel level. That is, it is built exactly in the same way as a single SVM model, by composing a quasi-linear kernel.
  Guided partitioning methods are proposed to obtain the local partitions for the composition of quasi-linear kernel function.
  If viewed from the original input space, an SVM with a RBF kernel may structurally be seen as a RBF network, which means that it approximates the nonlinear separation hyperplane by a set of RBF. On the other hand, a quasi-linear SVM approximates the nonlinear separation hyperplane by M local linear models with interpolation of RBF; its classification performance is shown to be less sensitive to the parameters of RBF, especially when M is small.
  • B. Zhou, C. Hu, B. Chen and J. Hu, "A Transductive Support Vector Machine with Adjustable Quasi-linear Kernel for Semi-Supervised Data Classification", in Proc. of 2014 IEEE International Joint Conference on Neural Networks (IJCNN'2014) (Beijing), July, 2014.
  • Y. Lin, Y. Fu and J. Hu, "Support Vector Machine with SOM-based Quasi-linear Kernel for Nonlinear Classification", in Proc. of 2014 IEEE International Joint Conference on Neural Networks (IJCNN'2014) (Beijing), July, 2014.
  • B. Zhou, B. Chen and J. Hu, "Quasi-linear Support Vector Machine for Nonlinear Classification", IEICE Trans. on Fundamentals of Electronics, communications and Computer Sciences, Vol.E97-A, No.7, July, 2014.
  • Y. Lin, H. Guo and J. Hu, "A SVM-based Approach for Stock Market Trend Prediction", in Proc. of 2013 IEEE International Joint Conference on Neural Networks (IJCNN'2013) (Dallas), Aug., 2013, pp.237-242.
  • B. Zhou, C. Yang, H. Guo and J. Hu, "A Quasi-linear SVM Combined with Assembled SMOTE for Imbalanced Data Classification", in Proc. of 2013 IEEE International Joint Conference on Neural Networks (IJCNN'2013) (Dallas), Aug., 2013, pp.2351-2357.
  • B. Chen F. Sun and J. Hu, "Local Linear Multi-SVM Method for Gene Function Classification", in Proc. of the World Congress on Nature and Biologically Inspired Computing (NaBIC2010)(Kitakyushu), Dec., 2010, pp.183-188.

Fast SVM for Classification of Large Scale Data

Credit Risk Evaluation and Default Prediction

Text Categorazition