Ioannis Vlahavas

Aristidis Likas

Georgios Paliouras

Department of Informatics, Aristotle University of Thessaloniki

Department of Computer Science, University of Ioannina

Institute of Informatics and Telecommunications, National Centre for Scientific Research “Demokritos”

Introduction

Machine Learning is a subfield of Artificial Intelligence that is concerned with algorithms and techniques that allow computer systems to “learn from experience” to successfully solve artificial intelligence problems. Experience is usually provided in the form of problem-specific “examples” (organized in “datasets”) that allow the learning system to discover new knowledge and improve its performance on a particular task. If the training examples are not available at the beginning of the learning process, but they are collected during training we have the case of “on-line” or incremental learning. If the system is already provided some knowledge about the domain and/or the task, we have the case of knowledge refinement or analytical learning.

Machine Learning problems are generally distinguished into three main categories depending on the nature of the datasets: In supervised learning, we are given a set of labeled examples and the aim is to discover the knowledge required for labeling new examples. Typical tasks that fall within the paradigm of supervised learning are classification where a class label is predicted for each input example and regression, where a numerical value is predicted for each input example. In unsupervised learning we are given a set of unlabeled examples and the aim is to identify the underlying structure of the data and extract it in the form of actionable knowledge. Typical unsupervised learning tasks are clustering, where the goal is to identify commonalities among the examples and form interesting clusters, and model estimation, where the knowledge model that generated the data is being sought, e.g. in the form of probability density functions. In semi-supervised learning only partial supervision is provided to the learning machine. Partial supervision can take the form of partial labeling of the examples, i.e., providing a set of labeled and a set of unlabeled examples. In particular, semi-supervised learning methods that involve the user in exploring the set of unlabelled examples are called active learning methods, while those that learn from data labeled with the same label are often called one-class or descriptive learning methods . Another form of partial supervision appears in reinforcement learning, where a label (reward/penalty) is provided for a complete sequence of actions and needs to be distributed to the individual actions, in order to let the system learn how to select actions. Reinforcement learning is particularly useful for control problems, games and sequential decision making.

Machine Learning problems are also distinguished according to the format of the training data. In most cases the data take the form of propositional assertions or equivalently feature vectors in the space of example attributes. In such cases, the knowledge to be discovered is of similar expressive power, e.g. propositional rules, decision trees etc. On the other hand, there are cases where we want to learn from relational data, e.g. first-order predicates or relational databases. In such cases the discovered knowledge is also necessarily of higher expressive power, e.g. logic programs. Inductive logic programming and statistical relational learning are two very active approaches to relational learning. In between first-order predicates and propositional assertions, there is a range of structured data, for which specialized and efficient learning methods have been developed: (a) sequential data, such as character strings, have given rise to a variety of sequence learning methods, which try to identify interesting sequential patterns, such as grammatical rules that could have generated the data, (b) graph data, i.e. data that are related with a single type of binary relation, and tree data have also led to specialized graph and tree learning methods.

In addition to the representation of the training examples, machine learning methods need to deal with the quality of the data provided. Most of the work in this area has focused on the manipulation of the attribute or feature space, i.e. the characteristics that are chosen for describing the examples. The effort to reduce the set of those descriptive features to the subset of useful ones has led to research in dimensionality reduction and feature selection. A variety of methods have been developed for this purpose, differing significantly between supervised and unsupervised learning problems. Apart from manipulating the feature space, work has been done in identifying and reducing the noise in the data, undersampling or oversampling the data, in particular when the given sample is considered to misrepresent the distribution of examples in the true world.

A great variety of models have been developed to tackle the learning problems. A well-studied category concerns neural networks, that are non-linear models inspired from the biological way of information processing and learning. The most popular neural architectures are the feedforward neural networks such as Multilayer Perceptrons (MLP) and Radial Basis Function (RBF) networks that have been successfully used in various learning problems. More recently, kernel models have emerged that provide state-of-the-art performance in many supervised and unsupervised learning tasks.

In this chapter we aim to provide a summary of the recent activity of researchers in Greece in the areas of “Machine Learning – Neural Networks”. This summary covers a wide range of research work, however it is by no means a complete description, since it contains information provided only by the research groups who responded to our invitation and submitted their contribution. The chapter also excludes people who are now active outside Greece and includes the past activity of people who are now in Greece. The chapter organization is group-based and, for each research group, a brief description of important proposed methods is provided along with an indication of the machine learning topic in which the method falls (e.g. supervised learning - neural networks). All references are given in the end of the chapter in alphabetical order. Finally it should be mentioned that this chapter focuses on general “machine learning – neural network” methods and algorithms and does not cover straigthforward applications of such algorithms to specific problem domains.