Evolving Granular Modeling from Uncertain Data Streams Daniel Leite Federal University of Lavras – UFLA, Brazil

Plenary talk at the IEEE Conference on Evolving and Adaptive Intelligent Systems EAIS’16 – Natal, May 2016

Summary Part I – Granular Computing Part II – Evolving Granular Systems Part III – Applications Examples

Part I – Granular Computing

What is Granular Computing? Dictionary • Granular: composed of granules • Granule: a small amount of something • Granularity: the extent to which a system is subdivided in smaller parts

Granular Computing (GrC) • Problem solving based on different levels of granularity (detail/abstraction) • Different levels of granularity are essential for humans to solve problems

Example: boiling egg Rule Drop the egg in boiling water for 5 minutes

Problem Sometimes egg too soft Sometimes egg too hard

Suggestion IF

THEN

5 min

IF

THEN

5 min

Solution Higher granularity of the attribute ‘size’

Example: packing bag

Solution Lower granularity of the attribute ‘temperature’

Granularity and abstraction “We look at the world under various grain sizes and abstract from it only those things that serve our present interest”. Hobbs, 1985. “Abstraction allows people to consider what is relevant and to forget irrelevant details which would get in the way of what they are trying to do”. Giunchiglia, 1992.

Reflection We • perceive and represent the world in different levels of granularity • understand problems and solutions in different levels of abstraction • adopt appropriate levels of granularity and change granularity easily

GrC inspiration • Capture basic principles used by humans to solve problems • Hierarchical notion - Low level → detailed/precise concepts - High level → abstract/imprecise concepts

Necessary and sufficient granularities

GrC • Theories and methods that make use of granules to solve problems • Subset of a universe → granule • GrC basic ingredients - Subsets - Classes - Clusters

Fundamental concepts • Granule: interval, fuzzy set, rough set, cluster, … • Granulation - Decomposition: coarse → fine (boiling egg) - Construction: fine → coarse (packing bag)

• Granular structure: a set of granules • Relationship, hierarchy

Multidisciplinarity • There is no unified framework for GrC • Different names in correlated areas: - Fuzzy and rough sets - Interval mathematics - Divide and conquer - Quotient space theory - Information fusion, ...

Research • GrC is a research area by itself: it has its own principles, theory and applications • Zadeh, 1997, 2005: “GrC is a superset of the theory of fuzzy information granulation, rough set theory and interval computations” “A generalized theory of uncertainty”

Historical notes • 1979, Zadeh discusses information granulation • 1997, T. Y. Lin suggests the term GrC. A group, BISCGrC, is formed • 2004, IEEE CIS Task Force on GrC is proposed • 2005, 1st IEEE International Conference on GrC • 2009, Journal of GrC, Rough Sets and Intelligent Systems • 2016, Springer Journal of GrC

GrC Publications • IEEExplore, ScienceDirect • Search (metadata only) - Granular computing - Granular fuzzy - Information granulation/granularity

Bargiela, A.; Pedrycz, W. Granular Computing: An Introduction. Kluwer Academic Pub. Boston, 2004

Pedrycz, W.; Gomide, F. Fuzzy Systems Engineering: Toward Human-Centric Computing. Wiley-Hoboken, 2007

Pedrycz, W.; Skowron, A.; Kreinovich, V. Handbook of Granular Computing. Wiley-Chichester, 2008

Part I: final remarks • GrC: a searchable term in bibliographic databases • GrC shares generalities of many domains • Research grows → long way ahead → requires interaction, dissemination

Part II – Evolving granular systems

Challenges in data analysis

• Data increase in volume, speed, uncertainty • Analyze data as it occurs • Make sense of uncertain data • Online learning human-centered systems

Motivation State-of-the-art machine learning needs methods and algorithms to: • build models of complex systems • process time-varying data streams • recursively adapt parameters and structures • deal with different types of data

Evolving granular systems Features • Incremental learning from online data streams • Gradual structural development • Models: rule base, neural network, ... • A GrC aspect: representation, parameter, data, …

How do granular data arise? 1 - Measurements e.g. from unreliable sensors

→ true value, certainly included

2 - Expert judgement e.g. medical diagnosis, risk analysis

Patient Interview

Physical Exam

Signs, symptoms, complaint

Past, present

Diagnosis Medical tests

Medical history

Past, present

Patient’s, family’s

3 - Imprecision in pre-processing steps Missing data, outliers Data cleaning, integration, transformation

4 - Summaries of numeric data over the time 1

1

1

1.5

1.5 Time

Central tendency, dispersion, dependence

Time and space granulation

Singular approximation

Granular approximation

Learning from granular data streams Begin Do 1: Read data sample 2: Fit the data 2.1: Create new granule 2.2: Adapt granules and associated information 3: Delete data sample 4: Update the granular structure End

1. Interval Based evolving Modeling IBeM features: • Processes interval data • Develops and adapts interval rules • Does not allow granules overlapping

Examples of interval data • Temperature between 20 and 27oC • Speed over 40Km/h and lower than 60Km/h • Body mass index between 18 and 25Kg/m2

IBeM rule:

IBeM learning algorithm Creates and adapts granules and rules Refines existing granules Sets model granularity Covers gaps Merges granules Deletes inactive rules Approximates functions

IBeM learning algorithm Creates and adapts granules and rules Refines existing granules Sets model granularity Covers gaps Merges granules Deletes inactive rules Approximates functions

IBeM learning algorithm Creates and adapts granules and rules Refines existing granules Sets model granularity Covers gaps Merges granules Deletes inactive rules Approximates functions

IBeM learning algorithm Creates and adapts granules and rules Refines existing granules Sets model granularity Covers gaps Merges granules Deletes inactive rules Approximates functions

IBeM learning algorithm Creates and adapts granules and rules Refines existing granules Sets model granularity Covers gaps Merges granules Deletes inactive rules Approximates functions

Contracts

Expands

IBeM learning algorithm Creates and adapts granules and rules Refines existing granules Sets model granularity Covers gaps Merges granules Deletes inactive rules Approximates functions

IBeM learning algorithm Creates and adapts granules and rules Refines existing granules Sets model granularity

Gap

Covers gaps Merges granules Deletes inactive rules Approximates functions

Neighbors

IBeM learning algorithm Creates and adapts granules and rules Refines existing granules Sets model granularity Covers gaps Merges granules Deletes inactive rules Approximates functions

IBeM learning algorithm Creates and adapts granules and rules Refines existing granules Sets model granularity Covers gaps Merges granules Deletes inactive rules Approximates functions

IBeM learning algorithm Creates and adapts granules and rules Refines existing granules Sets model granularity Covers gaps Merges granules Deletes inactive rules Approximates functions

2. Evolving fuzzy modeling FBeM main features: • Learn from fuzzy data • Union of local fuzzy models • Accurate and linguistic outputs

*FBeM: Fuzzy set Based evolving Modeling

Examples of fuzzy data • Gasoline price around R$3.50 • Atmospheric pressure approximately 101325Pa • Electrical current of 10A more or less 0.2A

FBeM rule:

Granulation of fuzzy data

3. Evolving neuro-fuzzy modeling eGNN features: • Learn from fuzzy data • Adaptation of granules, neurons, connections • Information fusion: aggregation neurons

*eGNN: evolving Granular Neural Network

Fuzzy aggregation neuron

Example:

is a uninorm

Uninorm: Umin,max, e = 0.3, v = 0

Structure eGNN

Structure eGNN

Structure eGNN

Structure eGNN

Structure eGNN

Structure eGNN

Structure eGNN

Part II: final remarks • IBeM, FBeM and eGNN • Deal with current challenges in data analysis • Granular data stream modeling

Part III – Applications Examples

1. Semi-supervised classification Twin Gaussians rotation (concept drift)

• (x,C)[h], h = 1, ... 1 ≤ h ≤ 200 201 ≤ h ≤ 400

Data stream Stationary concept Gradual rotation of 90o

eGNN (before) Right/Wrong rate 189/11 (94.5%) 5 granules

eGNN (after) Right/Wrong rate 195/5 (97.5%) 5 granules

ROC analysis

New class (concept shift)

• (x,C)[h], h = 1, ... 1 ≤ h ≤ 199 h = 200 201 ≤ h ≤ 400

Data stream Two stationary classes Concept shift Three stationary classes

eGNN (before) Right/Wrong rate 189/11 (94.5%) 6 granules

eGNN (after) Right/Wrong rate 185/15 (92.5%) 8 granules

Semi-supervised classification Rotating Gaussians problem

New class problem

2. Time series prediction Weather prediction • monthly data in 5 metheorological stations • period: Jan 1871 – Dec 2010 • min, average, max temperature • inputs: x[h-11], ..., x[h] • outputs: y[h+1]

FBeM prediction for Ottawa

Granular prediction

Comparing predictors

3. Function approximation Parkinson telemonitoring • 5875 biomedical voice measurements • 42 early-stage Parkinson’s diseased individuals • 16 inputs: shimmer/jitter attributes • 1 output: total UPDRS

Attribute selection • Leave-one-variable-out method: progressively eliminates input variables

FBeM

Performance comparison

4. Model-free control Autonomous navigation

IBeM

FBeM

eGNN

FBeM granular output

5. Model-based control Self-tuning PDC evolving granular regulator

Fuzzy granular control rule

• Assume fuzzy Lyapunov function • Proof of a theorem for closed-loop stability with bounded inputs: an LMI feasibility problem

Lorenz attractor

Stabilization of chaos

Part III: final remarks Evolving granular systems applied to: • Semi-supervised classification • Time series prediction • Function approximation • Control

Conclusion • Evolving granular models and controllers • Efficient to deal with time-varying data streams • Granular and/or numerical data • Provide precise and linguistic outputs

Open issues 1 - New time and space granulation methods 2 - Similarity measures and aggregation operators for granular data 3 - Incremental feature selection and data imputation 4 - Convenience of granular parameters 5 - Optimal granularity 6 - Big data problems and complexity issues 7 - Evolving approximation theorems