Machine learning and GDV images: A case study


Igor Kononenko, Tatjana Zrimec, Aleš Doganoc, Mitja Krebelj, Marjan Simčič*
University of Ljubljana, Faculty of Computer and Information Science, Tržaška 25
* Biotechnical Faculty, Dept. of Food Science and Technology, Jamnikarjeva 101
SI-1001 Ljubljana, Slovenia
tel: +386-61-1768390, fax: +386-61-1264647
e-mail: {igor.kononenko;tatjana.zrimec}@fri.uni-lj.si; marjan.simcic@bf.uni-lj.si

 

Abstract

Machine learning technology is well suited for the induction of diagnostic and prognostic rules and solving of small and specialized classification, diagnostic and prognostic problems. The classification knowledge can be automatically derived from the description of cases solved in the past. Recently developed technology for recording the human bioelectric field using the Gas Discharge Visualisation (GDV) technique provides possibly useful information about the biophysical and/or psyhical state of the object/patient. However, the problem is the interpretation of the GDV images. By using machine learning we could alleviate that problem by means of automatically generating classification rules from the description of GDV images. By using machine learning we are able to detect useful information from parameters that describe the GDV images. We ilustrate the methodology with a case study: recording coronas of apples. In conclusion we briefly describe ongoing studies with Kirlian camera.


1. Introduction

Machine learning technology (Mitchell, 1997; Dietterich and Shavlik, 1990) is well suited for the induction of diagnostic and prognostic rules and solving of small and specialized classification, diagnostic and prognostic problems. Data about correct classification is often available together with the description of objects. All that has to be done is to arrange the data, i.e. the records of the objects with known correct class, in the appropriate form and run the learning algorithm. This is of course an oversimplification, but in principle, the classification knowledge can be automatically derived from the description of cases solved in the past. The derived classifier can then be used either to assist the expert when classiffying new objects in order to improve the classification speed, accuracy and/or reliability, or to train the students or beginners non-specialists to classify the objects in some special classification problem.

We actually applied machine learning algorithms in several problem domains, mostly in medical diagnosis (Kononenko, 1993; Kononenko et al. 1998; Kukar et al., 1996, Zelič et al. 1998). When applying a machine learning system there are several specific requirements that the system must meet, such as reliability and transparency of decisions. Typically, the automatically generated diagnostic rules achieve the same or slightly better diagnostic accuracy than human experts.

Recently developed technology for recording human's bioelectric field by using the Gas Discharge Visualisation (GDV) technique (Korotkov, 1998a;b) provide a possibly useful information about the biophysical and psyhical state of the object/patient that could in some cases improve the classification/diagnostic process. However, the problem is the interpretation of the GDV images. By using machine learning we could alleviate that problem by means of automatically generating classification rules from GDV images. For that purpose the Kirlian images need to be described with a set of parameters.

 

2. Kirlian effect for studying the object's / human's bioelectric field

All over the world various researchers study unusual phenomena, most of them under the name of parapsychology, and probably the most experienced researchers are in Russia (Lee, 1996). Most widely are studied telephaty, telekinetics, and extrasensory perception. Quite some people are known that are able to see the bioelectric field, also known as aura (Chalko, 1997). With the developement of technology it has become possible to scientificaly study some aspects of the bioelectric field phenomenon (Korotkov, 1998b).

Korotkov (1998a; 1998b) and his team in St. Petersburg, Russia developed the instrument, Crown-TV, that can be routinely used. This opens practical possibilities to study the effects of GDV. Besides studying non-living objects, such as water, various liquids, (Korotkov and Korotkina, 1998), and minerals, the most widely studied are living organisms: plants (leafs, seeds, etc.), (Korotkov and Kouznetsov, 1996; Korotkov and Ratman, 1998), animals, (Krashenuk et al., 1998) and of course humans. For humans, most widely are recorded bioelectric fields of fingers (Kraweck, 1994; Korotkov, 1998b), and GDV records of blood excerpts (Korotkov, 1998b). (Voeikov, 1998). Principal among these are studies of the psycho-physiological state and energy of the human being, diagnosis (Gurvits and Korotkov, 1998), reactions to some medicines, (Magidov et al., 1998; Filippova et al., 1998) reactions to various substances, food (Kraweck, 1994), dental treatment (Lee, 1998), alternative healing treatment, such as accupuncture, 'bioenergy', homeopaty, various relaxation and massage techniques (Korotkov and Chalko, 1998), GEM therapy (Shah, 1998), applied kineziology and flower essence treatment (Hein and Bedoya, 1998), leech therapy (Korotkov, 1998b), etc.), and even studying the GDV images after death (Korotkov, 1998a).

An important part of the image analysis is to formulate and describe visual information, such as what type of image features we can extract from the images, what properties those features are expected to have, and how they are related to each other. However, a lot of knowledge is required to read bioelectric field (Korotkov, 1998b). The image analysis of those images requires preprocessing, segmentation into meaningful image structures and transformation the structures into a suitable representation. In our study we used The GDV Analysis software developed by Korotkov's team and which is provided together with the Crown-TV equipment. The parameters that are used to describe the GDV images are:

1.. Area of GDV-gram.
2.. Noise, deleted from the picture (depends on the first setting in the program).
3.. Form coefficient K = L2/2p S .
4.. Fractal dimension.
5.. Brightness coefficient.
6.. Brightness deviation.
7.. Number of separated fragments in the image.
8.. Average area of the fragment.
9.. Deviation of fragments' areas.
10.. Form coefficient II (ratio) KII = (Simage + Soval)/Soval.
11.. Relative coefficient of glow inside the inner oval.
12.. Relative coefficient of image glow for 25% area (from the whole area). Quantile
13.. Relative coefficient of image glow for 50% area (from the whole area).
14.. Relative coefficient of image glow for 75% area (from the whole area).
15.. Relative coefficient of image glow for 100% the whole area.
16.. Number of sectors (varies from 6 to 9).
17.. Areas in the sectors (17.1; 17.2; up to 17.9 ).
18.. Quantile coefficients for the sectors. (18.1.1 up to 18.9.4)

 

3. Selecting the appropriate ML system

In recent years many different machine learning algorithms were developed (Dietterich & Schavlik, 1990; Mitchell, 1997). They can be classified into three major groups (Michie et al., 1994): statistical or pattern recognition methods (such as the K-nearest neighbours, discriminant analysis, and Bayesian classifiers), inductive learning of symbolic rules (such as top-down induction of decision trees, decision rules and induction of logic programs), and artificial neural networks (such as the multilayered feedforward neural network with backpropagation learning, the Kohonen's self-organizing network and the Hopfield's associative memory). However, not all the systems are equally appropriate. When applying a machine learning system there are several specific requirements that the system must meet.

In this section we give a description of specific requirements that any machine learning system has to accomplish in order to be used in the developement of applications. Three learning algorithms that were used in our study are then briefly described.


3.1. Specific requirements for ML systems

For ML system to be useful in solving classification tasks the following features are desired: good perfomance, the transparency of generated knowledge, the ability to explain decisions, and the ability to appropriately deal with missing data .

Good performance: The algorithm has to be able to extract significant information from the available data. The diagnostic accuracy on new cases has to be as high as possible. In majority of learning problems, various approaches typically achieve similar performance in terms of classification accuracy although in some cases some algorithms may perform significantly better than the others (Michie et al., 1994) . Therefore, apriori almost none of the algorithms can be excluded with respect to the performance criterion. Rather, all available algorithms should be tested on the available data and one or few with best estimated performance should be considered for the developement of the application.

Transparency of diagnostic knowledge: The generated knowledge and the explanation of decisions should be transparent to the human. She should be able to analyse and understand the generated knowledge. Ideally, the automatically generated knowledge will provide a novel point of view on the given problem and, eventually may reveal new interrelations and regularities that the human expert did not see before in an explicit form.

Explanation ability: The system must be able to explain and argue about decisions. When faced with the curious solution of the new problem the human shall require further explanations, otherwise she will not seriously consider the system's suggestions.

Dealing with missing data: In real world very often the description of problems lacks certain data. ML algorithms have to be able to appropriately deal with such incomplete descriptions.


3.2. Description of various algorithms

In this subsection we briefly describe three algorithms that were used in our experiments: Assistant-R, Assistant-I, and the naive Bayesian classifier.

Assistant-R: is a reimplementation of the Assistant learning system for top down induction of decision trees (Cestnik et al., 1987). The basic algorithm goes back to CLS (Concept Learning System) developed by Hunt et al. (1966) and reimplemented by several authors (see (Quinlan, 1986) for an overview). The main features of Assistant are binarization of attributes, decision tree prepruning and postpruning, incomplete data handling, and the use of the naive Bayesian classifier to calculate the classification in "null leaves".

The main difference between Assistant and its reimplementation Assistant-R is that ReliefF is used for attribute selection (Kononenko et al., 1997). ReliefF is an extended version of RELIEF, developed by Kira and Rendell (1992), which is a non-myopic heuristic measure that is able to estimate the quality of attributes even if there are strong conditional dependencies between attributes. For example, RELIEF can efficiently estimate the quality of attributes in parity problems. In addition, wherever appropriate, instead of the relative frequency, Assistant-R uses the $m$-estimate of probabilities, which was shown to often improve the performance of machine learning algorithms (Cestnik, 1990).

Assistant-I: A variant of Assistant-R that instead of ReliefF uses information gain for the selection criterion, as does original Assistant. However, the other differences to original Assistant remain (m-estimate of probabilities).

Naive Bayesian Classifier: A classifier that uses the naive Bayesian formula to calculate the probability of each class C given the values Vi of all attributes for a given instance to be classified and assuming the conditional independence of the attributes given the class:

A new instance is classified into the class with maximal calculated probability. We use the m-estimate of probabilities (Cestnik, 1990). For prior probabilities the Laplace's law of succession is used:

These prior probabilities are then used in the m-estimate of conditional probabilities:

 

The parameter m trades off between the contributions of the relative frequency and the prior probability. In our experiments, the parameter m was set to 2.0 (this setting is usually used as default and, empirically, gives satisfactory results (Cestnik, 1990)).

The relative performance of the naive Bayesian classifier can serve as an estimate of the conditional independence of attributes.

 

4. A case study: Recording coronas of apples

The aim of the study was to determine, whether Kirlian camera can record any useful information by recording coronas of apples. We decided to record the coronas of peels that were cut off from apples in a standardized way (four circular peels with diameter of 18mm, cut off from equatorial part of the apple skin and all equally oriented). We used four sorts of apples of two different ages (one year difference). The central parts of coronas were manuallly deleted in order to adapt the records for the available GDV Analysis software. The parameters of the program were set as follows: remove all fragments up to 40 and background 230.

We tried to solve three different problems: classification of apples according to sort (four classes, 40 training examples of each class), according to age (two classes, 40 examples of each class), and according to the sun/shadow part of apple (two classes, 32 examples of each class). We measured the classification accuracy and information score (Kononenko & Bratko, 1991). This measure eliminates the influence of prior probabilities and appropriately treats probabilistic answers of the classifier. The average information score is defined as:

where the information score of the classification of i-th testing instance is defined by:

Cli is the class of the i-th testing instance, P(Cl) is the prior probability of class Cl and P'(Cl) the probability returned by a classifier.

For each problem we randomly split the available examples into 70% for training and 30% for testing set. We repeated this proces 10 times and the resulta were averaged and the standard deviation calculate. The results are presented in Tables 1 (accuracy) and 2 (information score).

Table 3 Classification accuracy (%) of learning systems on three classification problems

problem

assistant-I

assistant-R

naďve Bayes

sort

51.1± 6.9

55.6± 4.0

57.1± 3.1

age

61.3± 10.7

62.9± 6.3

69.9± 8.9

sun/shadow

48.2± 10.5

57.2± 10.8

55.8± 4.7

 

Table 4 Average information score (bit) of learning systems on three classification problems

problem

assistant-I

assistant-R

naďve Bayes

sort

0.73± 0.14

0.80± 0.07

1.01± 0.06

age

0.21± 0.15

0.21± 0.11

0.40± 0.16

sun/shadow

-0.01± 0.14

0.10± 0.16

0.12± 0.10

The average information score shows that parameters contain useful information for the first two problems while for the problem of separating sun/shadow sides of the apple there is practically no useful information. The classification accuracy is rather low in all three cases but is in the first two problems significantly higher than if the classifier would be random. Among the three classifiers the naive Bayesian classifier achieves the best results.

 

5. Conclusion and ongoing studies

The analysis described in this paper shows that machine learning algorithms can be used to detect whether the GDV records, described with a set of parameters, contain any useful information or it is only noise. In the case of apples it was clearly demonstrated, that for the two problems (determening the sort and the age of apples) the poarameters that describe coronas of apple peels contain useful information while for the sun/shadow problem the parameters are not better than noise.

Currently we are performing several studies with the Crown-TV:

One of our colleagues always had normal (single) coronas. Once he recorded double coronas on his fingers after one short treatment with color essences. After an hour the double corona disapeared and additional treatments with essences didn't show this effect anymore.

 

References

Cestnik B., Kononenko I.& Bratko I. (1987) ASSISTANT 86 : A knowledge elicitation tool for sophisticated users, in: I.Bratko, N.Lavrač (eds.): Progress in Machine learning, Wilmslow: Sigma Press.

Cestnik B. (1990) Estimating Probabilities: A Crucial Task in Machine Learning, Proc. European Conf. on Artificial Intelligence, Stockholm, August, 1990, pp. 147-149.

T.J.Chalko (1997) How to see and read the Aura, Melbourne, Australia, http://www.thiaoouba.com/seeau.htm

T.G.Dietterich, J.W.Shavlik (eds.) (1990) Readings in machine learning, Morgan Kaufmann.

B.Gurvits and K.Korotkov (1998) A new concept of the early diagnosis of cancer, Consciousness and Physical Reality, 1(1)84--89.

M.Hein and R.R.Bedoya (1998) Synergessences: a new flower essence paradigm, Proc. Int. Scientific Conf. Kirlionics, White Nights 98 (Abstracts), Federal Technical University SPIFMO, St. Petersburg, Russia, June 1998.

Hunt E., Martin J & Stone P. (1966) Experiments in Induction, New York, Academic Press.

Kira K. & Rendell L. (1992) A practical approach to feature selection, Proc. Intern. Conf. on Machine Learning (Aberdeen, July 1992) D.Sleeman & P.Edwards (eds.), Morgan Kaufmann, pp.249-256.

I.Kononenko (1993) Inductive and Bayesian learning in medical diagnosis, Applied Artificial Intelligence, 7:317-337.

Kononenko I. & Bratko I. (1991) Information based evaluation criterion for classifier's performance, Machine Learning, Vol.6, No.1, pp.67-80.

I.Kononenko, E.šimec, M.Robnik (1997) Overcoming the myopia of inductive learning algorithms, Applied Intelligence, 7:39-55

I.Kononenko, I.Bratko, M.Kukar (1998) Application of machine learning to medical diagnosis. In: R.S.Michalski, I.Bratko, and M.Kubat (eds.): Machine Learning, Data Mining and Knowledge Discovery: Methods and Applications, John Wiley & Sons.

K.Korotkov (1998a) Light after Life: A scientific Journey into the Spiritual World, Fair Lawn, USA: Backbone Publ. Comp.

K.Korotkov (1998b) Aura and Consciousness: A New Stage of Scientific Understanding, St.Petersburg, Russia: State Editing & Publishing Unit ``Kultura''.

K.Korotkov and T.Chalko (1998) Conscious control of human bio-energy field (Aura) – Application to self healing. In: K.Korotkov: Aura and Consciousness: A New Stage of Scientific Understanding, St.Petersburg, Russia: State Editing & Publishing Unit ``Kultura'', pp. 173-181. (also: http://www/thiaooba.com/kirc.htm.)

A.Kraweck (1994) Life's Hidden Forces: A Personal Journey into Kirlian Photography, Edmont, Canada: Triune-Being Research Organization Ltd.

M.Kukar, I.Kononenko, T.Silvester (1996) Machine learning in prognostics of the femoral neck fracture recovery, Artificial Intelligence in Medicine, 8:431-451.

A.G.Lee (1996) The Selected Works in the Region of Parapsychology, Moscow, Russia: L.L.Vasilyev Fund of Parapsychology.

S.D.Lee (1998) The application of kirlian photography in dentistry, Proc. Int. Scientific Conf. Kirlionics, White Nights 98 (Abstracts), Federal Technical University SPIFMO, St. Petersburg, Russia, June 1998.

D.Michie , D.J.Spiegelhalter , C.C.Taylor (eds.) (1994) Machine learning, neural and statistical classification, Ellis Horwood.

T.Mitchell (1997) Machine Learning, McGraw Hill.

Quinlan J.R. (1986) Induction of Decision Trees. Machine Learning. Vol. 1, No. 1, pp. 81-106.

J.M.Shah (1998) Preventing heart problems by Kirlian photography and Gems therapy, Proc. Int.

Scientific Conf. Kirlionics, White Nights 98 (Abstracts), Federal Technical University SPIFMO, St. Petersburg, Russia, June 1998.

I.Zelič, I.Kononenko, N.Lavrač, V.Vuga (1997) Diagnosis of sport injuries with machine learning: Explanation of induced decisions, Journal of Medical Systems, 21(6):429-444.