**LECTURING**

**Courses:
**

**Undergraduate:**

- Algorithms
and Data Structures I (in Slovene)
- Artificial
Intelligence (In Slovene)
- Intelligent
Systems (also In Slovene)

**Postgraduate (Master’s):**

*Course title:* **Intelligent
Systems (BUNI)**

*Lecturers*: prof. dr. Igor
Kononenko and prof. dr. Marko Robnik Šikonja

*Assistant*: dr. Petar Vračar

*The course purpose*: The goal of the course is the students to become acquainted
with the field of intelligent systems, which includes a collection of tools and
approaches for solving problems which are difficult or unpractical to tackle
with other methods.

· on time finished and positively graded homeworks (web quizes and other reports)

· on time finished and positively graded two seminar works

· written exam

Exercises

The exercises grade is a joint grade for two seminar works, each
seminar work has to be finished on time and graded positively. The precondition for positive exercises grade
is also that you achieve at least half of the points from homeworks (web quizes
and other reports). We take into account only on time finished homeworks. The
exercises grade is valid only the current year. If in the current year the
student does not pass the exam, the next year he/she has to do the exercises
(homeworks and seminar works) all over again.

**Exam****
**Exam is written (and optionally the oral part). The precondition for
the written exam is the positive grade from exercises. At the written part of
the exam, it is allowed to use an A4 sheet of paper, hand written with an
ordinary pencil (that can be rub out - photocopies and prints are not allowed)
and signed with a ball-point pen (name, surname and the inscription
student number). At the end of the exam,
this sheet of paper has to be given to assistant together with the written
exam. The precondition for the (optional) oral exam is positive grade from
written exam.

Final
grade

Final grade is combined from the exercise grade (50%) and the
exam grade (50%). Both parts need to be positive.

**Approximate Course Contents****:**

1. Intelligence,
artificial intelligence (AI) and human-machine interaction: basic philosophical
questions about intelligence and AI, the role of AI

2. Machine
learning and data mining, overview of basic algorithms.

3. Data
preprocessing, discretization, visualization.

4. Intelligent data
analysis

5. Basic
principles of machine learning (ML), evaluation of learning, combining ML algorithms

6. Parallel
distributed processing and artificial neural networks

7. Evolutionary
computation and genetic algorithms

8. Basic principles of
modelling: learning as modelling, model quality, model evaluation

9. Statistical
modelling: Bayesian reasoning, linear models, regression models, multivariate
models, causal reasoning, non-parametric models, stochastic processes

10. Decision support
systems: classical decision theory, utility functions, game theory,
multi-parameter decision models, uncertainty and risk management, group
decision making, quality of decision models

11. intelligent robots
and agents: overview and state-of-the-art, agent architectures and agent
theory, software agents, learning agents, mobile agents, multiagent
systems

12. Natural language
processing: deterministic and stochastic grammars, information extraction,
machine translation

13. Cognitive
modelling: humans as intelligent systems, cognitive architectures

14. Reinforcement
learning: basic approaches and algorithms, Q learning, TD learning

Basic
literature:

SLOVENE:

I. Kononenko
in M. Robnik Šikonja: *Inteligentni
sistemi*. Založba FE in FRI, Ljubljana, 2010

ENGLISH:

I. Kononenko, M. Kukar:
Machine Learning and Data Mining, Horwood publ., 2007.

S.J. Russell, P. Norvig:
Artificial Intelligence: A Modern Approach, 3rd ed. Prentice Hall, 2009.

Additional literature:

· Bratko: Prolog Programming for Artificial Intelligence, Third edition, Addison-Wesley, 2000.

· G. Luger: Artificial Intelligence: Structures and Strategies for Complex Problem Solving (6th ed.), Addison-Wesley Pearson Education, Boston, 2009

· D. Jurafsky, J. H. Martin. Speech and language processing: An introduction to natural language processing, computational linguistics, and speech recognition. Pearson Education Inc., 2009.

· Y. Shoham, K. Leyton-Brown. Multiagent systems: algorithmic, game-theoretic, and logical foundations. Cambridge University Press, 2009.

· L. Busoniu, R. Babuska, B. De Schutter, D. Ernst. Reinforcement Learning and Dynamic Programming using Function Approximators. Taylor & Francis CRC Press, 2010

· W.N. Venables, B.D. Ripley: Modern Applied Statistics with S, 4th edition. Springer, 2002.

· R.O. Duda, P.E. Hart, D.E. Stork: Pattern Classification, 2nd Edition, Wiley 2001.

· E. Turban, J.E. Aronson: Decision support systems and intelligent systems. Prentice-Hall, 2004.

*Course title:* **Machine Learning**

*Lecturer:* Prof. Dr. Igor kononenko

Assistant: Dr. Petar Vračar

*Basic goal of the course*: To know the basic principles and methods of
machine learning (ML)

*Contents*:

- Overview of ML methods
- What is learning and relation between learning and intelligence
- ML basics
- the basic principles of ML (MDL, multiple explanations, incremental learning, m-estimate of probabilities)
- Estimating performance of ML (classification accuracy, CONFUSION MATRIX, Brier score, information score, sensitivity, specificity, ROC curve, MSE, RMSE, MAE, RMAE, cross-validation)
- Attribute evaluation measures (impurity measures, information gain, gain ratio, distance measure, gini-index, ReliefF, variance difference)
- Data preprocessing, visualization
- Comparing algorithms, statistical tests, Bonferroni correction
- Combining ML algorithms: bagging, boosting, random forests
- Building and prunning of decision and regression trees, Naive Bayes, K-nearest neighbors, locally weighted regression
- Artificial neural networks, backpropagation of error
- Search algorithms for ML subproblems (hypothesis search, binarization, discretization, feature subset selection, parameter tuning)
- Advanced attribute evaluation measures: MDL, relation between gini-index and ReliefF, Regressional ReliefF, Weight of evidence, ORT
- J-measure and learning decision rules
- Advanced methods for estimating performance of ML: misclassification cost, recall and precision, correlation coefficient, bias and variance, bootstrapping
- Advanced visualization methods: Vizrank
- Combining ML algorithms: ensemble learning, stacking, inverse transduction, cost-sensitive learning, learning from imbalanced datasets, function immitation, error-correcting codes
- Bayesian learning: Bayesian classifier, Naïve and non-naive Bayes, semi-naïve Bayes, Bayesian nets, TAN
- Calibration of probabilities
- Explanation of individual predictions
- Numerical ML methods (LR, Logistic regression, SVM)
- Artificial neural networks: Hopfield NN, RBF, Deep NN
- Unsupervised learning: clustering, Association rules
- Estimating the reliability of individual predictions
- Text mining
- Matrix factorization, Arcehtypal analysis
- ML as data compression
- Active learning
- Introduction to learning theory
- History of ML;
- ML approaches, not covered by this course (multitarget learning, semi-supervised learning, reinforcement learning, incremental learning, learning from data streams, constructive induction, learning of plans, equation model discovery, image mining, Graph and network mining, COLT, time-series analysis, spatial DM, user porfiling and recommendation systems, Inductive logic programming)

**Exercises: **

**During exercises you will practice methods and
techniques, described in lectures. Students also have to solve homeworks (web
quiz) and every student gets a mark for homeworks and the mark has to be at
least 50% in order to be able to get the mark for the course.**

**Research works and MLDM (Machine
Learning and Data Mining) Workshop:**

Each student (or at most two in a group) autonomously implements a research
work, writes a report in a form of a workshop paper (in English) and defends
the research work to his supervisor and presents the paper at the **MLDM
Workshop**, which is organized last two week of the semester during the exercises.
The mark for the research work counts as the mark for the exercise OC (provided
that the homeworks were positively (>=50%) completed). Positive mark for the
exercises is a necessary condition for the student to be able to apply for the
exam. The mark for the exercises is valid only the current year. If in the same
year the student does not pass the exam, the next year he/she has to complete
the exercises and homeworks all over again.

Every student (or a group of two), has to prepare a report in a form of the scientific paper (6-8 pages) and the format is specified with the MLDM workshop paper writing guide. The student submits the PDF before the deadline and then the review period begins. Every student receives three papers to review, completes a review form for each of them where the necessary corrections of each paper are also required. The reviews are then distributed to the authors of the papers and the final version of the paper has to be submitted one week before the workshop begins. At the workshop, each student presents the paper in a short lecture (10 min + 5 min discussion).

All the students have to be present at the workshop.

__The final mark is the average of the marks for the exam (I) and the
exercises (V): __

**O
= (I+V)/2, where I >= 50% and V >= 50%.**

- V =
OC, if (DN> = 50%), otherwise V
= insufficient

Necessary condition to apply for the exam: V >= 50% - I = the mark of the written AND the oral exam

*Exam*:

Exam is divided into written and the oral part. At the written part of the exam, from the literature it is allowed to use an A4 sheet of paper, hand written with an ordinary pencil (that can be rub out) and signed with a ball-point pen (name, surname and the inscription student number) (photocopies and prints are not allowed). At the end of the exam, this sheet of paper has to be given to assistant together with the written exam.

*Literature:*

Basic:

BOOK:

- Igor
Kononenko, Matjaž Kukar:
*Machine Learning and Data Mining: Introduction to Principles and Algorithms*, Horwood publ., 2007

PAPERS:

·
Reliability of single predictions in
classification and regression (paper: Comparison
of approaches for estimating reliability of individual regression predictions
)

- Calibration of probabilities (paper: Predicting
Good Probabilities With Supervised Learning)

·
Explaining classification of a
single instance (paper: Explaining
instance classifications with interactions of subsets of feature values )

·
Deep neural networks (paper: Exploring
strategies for training deep neural networks )

·
Learning from impalanced
data sets (Book chapter: Use of
Prediction Reliability Estimates onImbalanced Datasets )

·
Learning as compression (PhD by
Andrej Bratko: Text mining
using data compression models )

·
Web user profiling (paper: Web
User Profiles with Time-Decay and Prototyping )

·
Recommendation systems (paper: Towards
the Next Generation of Recommender Systems )

·
Archetypal Analysis (PhD by Ercan
Canhasi: Graph-based
models for multi-document summarization)

·
Active Learning (paper: Active Learning
Literature survey )

Additional:

- H. Witten and E. Frank.
*Data Mining: Practical Machine Learning Tools and Techniques*. Morgan Kaufmann, 2*nd*edition, 2005.