Designing neural networks for tackling hard classification problems

• Authors: Georgios Lappas
• Venue: WSEAS TRANSACTIONS on SYSTEMS
• Year: 2009

When designing neural networks for tackling hard classification problems researchers face the trivial problem of deciding the appropriate size of the neural network. The problem of optimizing the size of a neural network for obtaining high classification accuracy in datasets is indeed a hard problem in the literature. Existing studies provide theoretical upper bounds on the size of neural networks that are unrealistic to implement. Alternatively, optimizing empirically the neural network size may need a large number of experiments, which due to a considerable number of free parameters may become a real hard task in time and effort to accomplish. Hard classification problems are usually large in size datasets. Such datasets derive from collection of real world data like from multimedia content and are usually rich in training samples and rich in features that describe each collected sample. Working with neural networks and hard classification datasets will make even harder the task to optimize the neural network size. This work highlights on a mathematical formula for a priori calculating the size of a neural network for achieving high classification accuracy rate. The formula estimates neural networks size based only on the number of available training samples, resulting in sizes of neural networks that are realistic to implement. Using this formula in hard classification datasets aims to fix the size of an accurate neural network and allows researchers to concentrate on other aspects of their experiments. The focus on this approach turns to the number of available data for training the neural network, which is a new perspective in the neural network theory and the characteristics of this perspective are discussed in this article for designing neural networks for tackling hard classification problems.