Lumping Markov Chains with Silent Steps Jasen Markovski, Nikola Trˇcka
[email protected],
[email protected]
Formal Methods Group Department of Mathematics and Computer Science Technische Universiteit Eindhoven
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 1/40
Contents 1. Markov Chains (MC’s) - Definition, Performance Analysis 2. Interactive Markov Chains - Definition, Performance Analysis 3. Motivation / Method 4. Lumping of MC’s - Introduction, Definition, Examples 5. (Discontinuous) Markov Processes - Introduction, Definition 6. MC’s with Fast Transitions - Introduction, Definition, Examples 7. τ -lumping of MC’s with Fast Transitions - Definition, Soundness, Examples 8. MC’s with Silent Steps - Introduction, Definition, Examples 9. τ∼ -lumping of MC’s with Silent Steps - Definition, Soundness, Examples 10. Markovian weak bisimulation vs. τ∼ -lumping - Divergence / Conclusion
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 2/40
Exponential Distribution • X ∈ E(λ) iff
P (X ≤ t) = 1 − e−λt .
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 3/40
Exponential Distribution • X ∈ E(λ) iff
P (X ≤ t) = 1 − e−λt . • The only memoryless continuous distribution:
P (X > t + ∆t | X > t) = P (X > ∆t).
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 3/40
Exponential Distribution • X ∈ E(λ) iff
P (X ≤ t) = 1 − e−λt . • The only memoryless continuous distribution:
P (X > t + ∆t | X > t) = P (X > ∆t). • Closed under minimum:
X ∈ E(λ) and Y ∈ E(µ) implies min(X, Y ) ∈ E(λ + µ).
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 3/40
Exponential Distribution • X ∈ E(λ) iff
P (X ≤ t) = 1 − e−λt . • The only memoryless continuous distribution:
P (X > t + ∆t | X > t) = P (X > ∆t). • Closed under minimum:
X ∈ E(λ) and Y ∈ E(µ) implies min(X, Y ) ∈ E(λ + µ). • Expected value: E(X) =
1 λ,
for X ∈ E(λ).
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 3/40
Continuous-time Markov Chains - MC’s Visual representation and a generator matrix Q=
1
−λ λ 0 0
Q = (0)
1
λ
2
Q=
1 λ
−(λ+µ) λ µ 0 0
1
0 0 0 0
µ
2
3
T
µ
λ
Q=
−λ λ µ −µ
2
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 4/40
Performance Measures of MC’s • Transient probabilities (Examples): ◦ Reaching a state before a given time. ◦ Reaching a state for a first time. ◦ Leaving a state before a given time.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 5/40
Performance Measures of MC’s • Transient probabilities (Examples): ◦ Reaching a state before a given time. ◦ Reaching a state for a first time. ◦ Leaving a state before a given time. • Stationary probabilities ◦ Obtained after an (infinitely) long run
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 5/40
Performance Measures of MC’s • Transient probabilities (Examples): ◦ Reaching a state before a given time. ◦ Reaching a state for a first time. ◦ Leaving a state before a given time. • Stationary probabilities ◦ Obtained after an (infinitely) long run • Transition matrix of a MC with generator Q
P (t) = eQt =
∞ X Qn tn
n=0
n!
.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 5/40
Performance Measures of MC’s • Transient probabilities (Examples): ◦ Reaching a state before a given time. ◦ Reaching a state for a first time. ◦ Leaving a state before a given time. • Stationary probabilities ◦ Obtained after an (infinitely) long run • Transition matrix of a MC with generator Q
P (t) = eQt =
∞ X Qn tn
n=0
n!
.
• All (relevant) information is in the transition matrix.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 5/40
Performance Measures of MC’s • Transient probabilities (Examples): ◦ Reaching a state before a given time. ◦ Reaching a state for a first time. ◦ Leaving a state before a given time. • Stationary probabilities ◦ Obtained after an (infinitely) long run • Transition matrix of a MC with generator Q
P (t) = eQt =
∞ X Qn tn
n=0
n!
.
• All (relevant) information is in the transition matrix. • We abstract from the initial probability vector.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 5/40
Interactive Markov Chains (IMC’s) - Hermanns, 2002 • Compositional generation of MC’s.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 6/40
Interactive Markov Chains (IMC’s) - Hermanns, 2002 • Compositional generation of MC’s. • Action information added to enable interaction.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 6/40
Interactive Markov Chains (IMC’s) - Hermanns, 2002 • Compositional generation of MC’s. • Action information added to enable interaction. • Actions interleaved with rates.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 6/40
Interactive Markov Chains (IMC’s) - Hermanns, 2002 • Compositional generation of MC’s. • Action information added to enable interaction. • Actions interleaved with rates. • Many standard operators defined.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 6/40
Interactive Markov Chains (IMC’s) - Hermanns, 2002 • Compositional generation of MC’s. • Action information added to enable interaction. • Actions interleaved with rates. • Many standard operators defined.
Example: 1 1 a
µ
T
2
λ
2
[ ρ
a
3
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 6/40
Interactive Markov Chains (IMC’s) - Hermanns, 2002 • Compositional generation of MC’s. • Action information added to enable interaction. • Actions interleaved with rates. • Many standard operators defined.
Example:
1,1
B 1 1 a
λ
2
µ
[
1,2
µ
T
k{a}
\
2
ρ ρ
λ
a
=
2,3
a
3 1,3
λ
77 77ρ 77
2,1
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 6/40
Performance Analysis of IMC’s - Example 1
IMC Open system 1 a
λ
2
µ
/ 3
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 7/40
Performance Analysis of IMC’s - Example 1
IMC Open system
IMC Closed system
1
1
a
λ
2
µ
/ 3
Renaming
/
τ
λ
2
µ
/ 3
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 7/40
Performance Analysis of IMC’s - Example 1
IMC Open system
IMC Closed system 1
1 a
λ
2
µ
/ 3
Renaming
/
τ
λ
2
µ
/ 3
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 8/40
Performance Analysis of IMC’s - Example 1
IMC Open system
IMC Closed system
Corresp. MC
1
1,2
1 a
λ
2
µ
/ 3
Renaming
/
τ
2
µ
λ
weak bisim.
τ −elim.+lump.
/ 3
/
µ
2
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 8/40
Performance Analysis of IMC’s - Example 2
IMC Open system 1
O
a
b
2
µ
3
λ
4
~
λ
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 9/40
Performance Analysis of IMC’s - Example 2
IMC Open system
IMC Closed system
1
1
O
a
2
µ
λ
4
~
O
τ
b Renaming
/
τ
3
2
λ
λ
µ
3
4
~
λ
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 9/40
Performance Analysis of IMC’s - Example 2
IMC Open system
IMC Closed system 1
1
O
a
2
µ
λ
4
~
O
τ
b Renaming
/
τ
3
2
λ
λ
µ
3
4
~
λ
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 10/40
Performance Analysis of IMC’s - Example 2
IMC Open system
IMC Closed system 1
1
O
a
µ
3
O
τ
b
2
Corresp. MC
Renaming
/
τ
2
µ
3
1,2,3
T
weak bisim.
/
τ −elim.+lump.
µ
λ
λ
4
~
λ
λ
4
~
λ
2
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 10/40
Motivation and Goal • Example: 0 B B B B B B Perf B B B B B
1
1
O
τ
τ
µ
2 λ
"
3
4
|
λ
C C C C C C C C C C C A
0
≈
B B B Perf B
1
1
T µ
λ
C C C C A
.
2
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 11/40
Motivation and Goal • Example: 0 B B B B B B Perf B B B B B
1
1
O
τ
τ
µ
2 λ
"
3
4
|
λ
C C C C C C C C C C C A
0
≈
B B B Perf B
1
1
T µ
λ
C C C C A
.
2
• What is ≈?
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 11/40
Motivation and Goal • Example: 0 B B B B B B Perf B B B B B
1
1
O
τ
τ
µ
2 λ
"
3
4
|
λ
C C C C C C C C C C C A
0
≈
B B B Perf B
1
1
T µ
λ
C C C C A
.
2
• What is ≈? • Goal: Obtain the (probabilistic) connection between the
original process (with τ -steps) and the minimized process (in general without τ -steps).
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 11/40
Method Based on lumping of MC’s; treat τ as a large exponential rate.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 12/40
Method Based on lumping of MC’s; treat τ as a large exponential rate. 1. We assign a weight to each τ .
33 33λ aτ 3
Intermediate model: MC with fast transitions.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 12/40
Method Based on lumping of MC’s; treat τ as a large exponential rate. 1. We assign a weight to each τ .
33 33λ aτ 3
Intermediate model: MC with fast transitions.
2. We define lumping for MC’s with fast transitions.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 12/40
Method Based on lumping of MC’s; treat τ as a large exponential rate. 1. We assign a weight to each τ .
33 33λ aτ 3
Intermediate model: MC with fast transitions.
2. We define lumping for MC’s with fast transitions. 3. We abstract from weights (by taking all possible weights).
33 33λ 3
τ
=
8 < :
33 33λ 3
aτ
9 =
|a>0 ;
New model: MC with silent steps.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 12/40
Method Based on lumping of MC’s; treat τ as a large exponential rate. 1. We assign a weight to each τ .
33 33λ aτ 3
Intermediate model: MC with fast transitions.
2. We define lumping for MC’s with fast transitions. 3. We abstract from weights (by taking all possible weights).
33 33λ 3
τ
=
8 < :
33 33λ 3
aτ
9 =
|a>0 ;
New model: MC with silent steps.
4. We define lumping for MC’s with silent steps.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 12/40
Method Based on lumping of MC’s; treat τ as a large exponential rate. 1. We assign a weight to each τ .
33 33λ aτ 3
Intermediate model: MC with fast transitions.
2. We define lumping for MC’s with fast transitions. 3. We abstract from weights (by taking all possible weights).
33 33λ 3
τ
=
8 < :
33 33λ 3
aτ
9 =
|a>0 ;
New model: MC with silent steps.
4. We define lumping for MC’s with silent steps. 5. We show that this lumping is a proper lifting of the lumping of MC’s with fast transitions up to equivalence classes. DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 12/40
Progress 1. Markov Chains (MC’s) - Definition, Performance Analysis 2. Interactive Markov Chains - Definition, Performance Analysis 3. Motivation / Method 4. Lumping of MC’s - Introduction, Definition, Examples 5. (Discontinuous) Markov Processes - Introduction, Definition 6. MC’s with Fast Transitions - Introduction, Definition, Examples 7. τ -lumping of MC’s with Fast Transitions - Definition, Soundness, Examples 8. MC’s with Silent Steps - Introduction, Definition, Examples 9. τ∼ -lumping of MC’s with Silent Steps - Definition, Soundness, Examples 10. Markovian weak bisimulation vs. τ∼ -lumping - Divergence / Conclusion
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 13/40
Lumping of MC’s • A way to simplify models.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 14/40
Lumping of MC’s • A way to simplify models. • Abstraction from states; states become classes of states.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 14/40
Lumping of MC’s • A way to simplify models. • Abstraction from states; states become classes of states. • Relationship between the original and the lumped process
is intuitive: Probability of being in a class is the same as the sum of probabilities of being in each state of the class.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 14/40
Lumping of MC’s • A way to simplify models. • Abstraction from states; states become classes of states. • Relationship between the original and the lumped process
is intuitive: Probability of being in a class is the same as the sum of probabilities of being in each state of the class. • Performance characteristics are the same (albeit
abstracted).
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 14/40
Lumping of MC’s • A way to simplify models. • Abstraction from states; states become classes of states. • Relationship between the original and the lumped process
is intuitive: Probability of being in a class is the same as the sum of probabilities of being in each state of the class. • Performance characteristics are the same (albeit
abstracted). • Properties like utilization, throughput, stationary
probabilities, etc. are preserved.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 14/40
Lumping of MC’s - Example
4 1 j ρ
λ
µ
2 E E λ 2
4
ρ
3
EE EE EE λ λ EE 2 "
5
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 15/40
Lumping of MC’s - Example
4 1 j
1 µ
λ
Z
λ+µ ρ
2 E E λ 2
4
3
EE EE EE λ λ EE 2 "
5
ρ
{{1},{2,3},{4,5}}
/
2,3
ρ
λ
4,5
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 15/40
Lumping of MC’s - Theory •
Partitioning ◦
◦
◦
P = {C1 , . . . , CN }. The collector matrix V [i, j] =
8 < 0,
i 6∈ Cj
: 1,
i ∈ Cj
A distributor matrix U ≥ 0, U V = I.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 16/40
Lumping of MC’s - Theory •
Partitioning ◦
◦
◦ •
P = {C1 , . . . , CN }. The collector matrix V [i, j] =
8 < 0,
i 6∈ Cj
: 1,
i ∈ Cj
A distributor matrix U ≥ 0, U V = I.
Lumping condition: V U QV = QV - rows of QV are the same for the states belonging to the same partitioning class, i.e. The accumulative rate of going to another class is the same.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 16/40
Lumping of MC’s - Theory •
Partitioning ◦
◦
◦
P = {C1 , . . . , CN }. The collector matrix V [i, j] =
8 < 0,
i 6∈ Cj
: 1,
i ∈ Cj
A distributor matrix U ≥ 0, U V = I.
•
Lumping condition: V U QV = QV - rows of QV are the same for the states belonging to the same partitioning class, i.e. The accumulative rate of going to another class is the same.
•
ˆ = U QV . The lumped process: Q
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 16/40
Lumping of MC’s - Theory •
Partitioning ◦
◦
◦
P = {C1 , . . . , CN }. The collector matrix V [i, j] =
8 < 0,
i 6∈ Cj
: 1,
i ∈ Cj
A distributor matrix U ≥ 0, U V = I.
•
Lumping condition: V U QV = QV - rows of QV are the same for the states belonging to the same partitioning class, i.e. The accumulative rate of going to another class is the same.
•
ˆ = U QV . The lumped process: Q
Example (cf.):
0 −(λ+µ)
Q=
0 0 ρ ρ
λ −λ 0 0 0
µ 0 −λ 0 0
0
0
λ 2
λ 2
0 λ −ρ 0 0 −ρ
1
V =
A
1 0 0 0 0
1
U=
0 1 1 0 0
0 0 0 1 1
!
0 0 1 0 1 2 2 0 0 0
0 0 0 0
ˆ= Q
−(λ+µ) λ+µ 0 0 −λ λ ρ 0 −ρ
1 1 2 2
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 17/40
Progress 1. Markov Chains (MC’s) - Definition, Performance Analysis 2. Interactive Markov Chains - Definition, Performance Analysis 3. Motivation / Method 4. Lumping of MC’s - Introduction, Definition, Examples 5. (Discontinuous) Markov Processes - Introduction, Definition 6. MC’s with Fast Transitions - Introduction, Definition, Examples 7. τ -lumping of MC’s with Fast Transitions - Definition, Soundness, Examples 8. MC’s with Silent Steps - Introduction, Definition, Examples 9. τ∼ -lumping of MC’s with Silent Steps - Definition, Soundness, Examples 10. Markovian weak bisimulation vs. τ∼ -lumping - Divergence / Conclusion
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 18/40
(Discontinuous) Markov Processes (MP) •
Generalization of MC’s.
•
Contains instantaneous states.
•
Infinitely many transitions in finite time.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 19/40
(Discontinuous) Markov Processes (MP) •
Generalization of MC’s.
•
Contains instantaneous states.
•
Infinitely many transitions in finite time. Discontinuous Markov process
Markov Chain 1
Y
2
"
3
4
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 19/40
(Discontinuous) Markov Processes (MP) •
Generalization of MC’s.
•
Contains instantaneous states.
•
Infinitely many transitions in finite time. Discontinuous Markov process
Markov Chain 1
Y
2
"
3
4
•
No (unique) visual representation.
•
Represented by a pair (Π, Q); transition matrix P (t) = ΠeQt (cf.)
•
When Π = I then MP is a MC. DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 19/40
MP - Example
0 0 Π= 0 0
b a+b b a+b b a+b
a a+b a a+b a a+b
0
0
0 0 0 1
0 0 Q= 0 0
bλ − a+b bλ − a+b bλ − a+b
aλ − a+b aλ − a+b aλ − a+b
0
0
• States 1, 2 and 3 are instantaneous (Π[i, i] < 1).
λ λ λ 0
• State 1 is an evanescent state (Π[1, 1] = 0). • State 4 is a regular state (Π[4, 4] = 1).
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 20/40
Lumping of MP’s • Lumping condition must hold for both Π and Q:
V U ΠV = ΠV
and
V U QV = QV
• For Π = I coincides with the lumping of MC’s (cf.).
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 21/40
Lumping of MP’s • Lumping condition must hold for both Π and Q:
V U ΠV = ΠV
V U QV = QV
and
• For Π = I coincides with the lumping of MC’s (cf.).
Example:
Π=
b a+b b a+b
0
a a+b a a+b
0
P = {{1, 2}, {3}}
0 0 1 ˆ = Π
Q=
bλ − a+b bλ − a+b
aλ − a+b aλ − a+b
0
! 1 0 0 1
ˆ= Q
0
λ λ 0
−λ λ 0 0
!
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 21/40
Progress 1. Markov Chains (MC’s) - Definition, Performance Analysis 2. Interactive Markov Chains - Definition, Performance Analysis 3. Motivation / Method 4. Lumping of MC’s - Introduction, Definition, Examples 5. (Discontinuous) Markov Processes - Introduction, Definition 6. MC’s with Fast Transitions - Introduction, Definition, Examples 7. τ -lumping of MC’s with Fast Transitions - Definition, Soundness, Examples 8. MC’s with Silent Steps - Introduction, Definition, Examples 9. τ∼ -lumping of MC’s with Silent Steps - Definition, Soundness, Examples 10. Markovian weak bisimulation vs. τ∼ -lumping - Divergence / Conclusion
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 22/40
Markov Chains with Fast Transitions •
Parameterized MC (Qr , Qs ); generator of the form Q = Qr + τ Qs .
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 23/40
Markov Chains with Fast Transitions •
Parameterized MC (Qr , Qs ); generator of the form Q = Qr + τ Qs .
aτ
1 j
*
2
Qr =
bτ
λ
3
~
λ
Qs =
−λ
−a
0 λ 0 −λ λ 0 0 0 a 0 b −b 0 0 0 0
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 23/40
Markov Chains with Fast Transitions •
Parameterized MC (Qr , Qs ); generator of the form Q = Qr + τ Qs .
aτ
1 j
2
Qr =
bτ
λ
3
•
* ~
λ
Qs =
−λ
−a
0 λ 0 −λ λ 0 0 0 a 0 b −b 0 0 0 0
Thm: When τ → ∞, MC with fast transitions goes to a MP.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 23/40
Markov Chains with Fast Transitions •
Parameterized MC (Qr , Qs ); generator of the form Q = Qr + τ Qs .
aτ
1 j
−a
0 λ 0 −λ λ 0 0 0
Qr =
2
−λ
bτ
λ
3
•
* ~
λ
a 0 b −b 0 0 0 0
Qs =
Thm: When τ → ∞, MC with fast transitions goes to a MP.
Π=
b a a+b a+b a b a+b a+b
0
Q=
0
0
!
0 1
bλ − a+b bλ − a+b
aλ − a+b aλ − a+b
0
0
λ
!
λ 0 DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 23/40
Markov Chains with Fast Transitions •
Parameterized MC (Qr , Qs ); generator of the form Q = Qr + τ Qs . 1 aτ
1 j
*
−λ
0 0 −λ λ 0 0 0
Qr =
2
bτ
λ
3
~
λ
λ
−a
0
a b −b 0 0 0 0
Qs =
O
aτ
2
µ
Qr =
3
λ
4 •
bτ
0
~
λ
0 0 −λ 0 0 µ 0
0 0 −λ 0
−(a+b) a
Qs =
0 0 0
0 λ λ −µ
b 0 0 0 0 0 0
0 0 0 0
Thm: When τ → ∞, MC with fast transitions goes to a MP.
Π=
b a a+b a+b a b a+b a+b
0
Q=
0
0
!
0 1
bλ − a+b bλ − a+b
aλ − a+b aλ − a+b
0
0
λ
!
λ 0 DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 23/40
Markov Chains with Fast Transitions •
Parameterized MC (Qr , Qs ); generator of the form Q = Qr + τ Qs . 1 aτ
1 j
*
−λ
0 0 −λ λ 0 0 0
Qr =
2
bτ
λ
3
~
λ
λ
−a
0
a b −b 0 0 0 0
Qs =
O
aτ
bτ
2
µ
λ
~
λ
0 0 −λ 0 0 µ 0
Qr =
3
4 •
0
0 0 −λ 0
−(a+b) a 0 0 0
Qs =
0 λ λ −µ
b 0 0 0 0 0 0
0 0 0 0
Thm: When τ → ∞, MC with fast transitions goes to a MP.
Π=
b a a+b a+b a b a+b a+b
0
Q=
0
0
!
0 1
bλ − a+b bλ − a+b
aλ − a+b aλ − a+b
0
0
λ λ 0
!
Π=
a b 0 a+b 0 a+b 0 1 0 0 0 0 1 0 0 0 0 1
Q=
aλ − bλ λ 0 − a+b a+b 0 −λ 0 λ 0 0 −λ λ µ 0 0 −µ
!
!
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 23/40
Classification of States of MC’s with Fast Transitions • Reachability of states: we can get from one state to another
by doing only fast transitions.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 24/40
Classification of States of MC’s with Fast Transitions • Reachability of states: we can get from one state to another
by doing only fast transitions. • Communicating classes: all states in a class can reach
each other.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 24/40
Classification of States of MC’s with Fast Transitions • Reachability of states: we can get from one state to another
by doing only fast transitions. • Communicating classes: all states in a class can reach
each other. • States that perform a fast transition become instantaneous
when τ → ∞.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 24/40
Classification of States of MC’s with Fast Transitions • Reachability of states: we can get from one state to another
by doing only fast transitions. • Communicating classes: all states in a class can reach
each other. • States that perform a fast transition become instantaneous
when τ → ∞. • Closed communicating classes are formed of ergodic states
(ergodic classes).
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 24/40
Classification of States of MC’s with Fast Transitions • Reachability of states: we can get from one state to another
by doing only fast transitions. • Communicating classes: all states in a class can reach
each other. • States that perform a fast transition become instantaneous
when τ → ∞. • Closed communicating classes are formed of ergodic states
(ergodic classes). • Other instantaneous states are transient (evanescent).
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 24/40
Classification of States of MC’s with Fast Transitions • Reachability of states: we can get from one state to another
by doing only fast transitions. • Communicating classes: all states in a class can reach
each other. • States that perform a fast transition become instantaneous
when τ → ∞. • Closed communicating classes are formed of ergodic states
(ergodic classes). • Other instantaneous states are transient (evanescent). • Regular states are considered as ergodic classes with one
element.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 24/40
τ -lumping - Example 1
T
aτ
bτ
2 λ
3
Qr = Qs =
0
−a
0 0 0 −λ λ 0 0 0
a 0 b −b 0 0 0 0
τ →∞
(Qr , Qs ) → (Π, Q)
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 25/40
τ -lumping - Example 1
T
aτ
bτ
{{1,2},{3}}
2
/
λ
3
Qr = Qs =
0
0 0 0 −λ λ 0 0 0
−a
a 0 b −b 0 0 0 0
τ →∞
V =
(Qr , Qs ) → (Π, Q)
1 0 1 0 0 1
V U ΠV = ΠV V U QV = QV
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 25/40
τ -lumping - Example 1
T
aτ
1,2
bτ
{{1,2},{3}}
2
/
a λ a+b
3
λ
3
Qr = Qs =
0
0 0 0 −λ λ 0 0 0
−a
a 0 b −b 0 0 0 0
τ →∞
V =
(Qr , Qs ) → (Π, Q)
1 0 1 0 0 1
W =
0
0
0 1
ˆs = ( 0 0 ) Q 0 0
V U ΠV = ΠV V U QV = QV
b a a+b a+b
ˆr = Q
a λ a λ − a+b a+b 0 0
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 25/40
τ -lumping of MC’s with Fast Transitions • Partitioning is a lumping of a MC with fast transitions if it is a
lumping of its limit MP.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 26/40
τ -lumping of MC’s with Fast Transitions • Partitioning is a lumping of a MC with fast transitions if it is a
lumping of its limit MP. • The lumped process:
ˆr = W QV Q
ˆs = W Qs V. Q
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 26/40
τ -lumping of MC’s with Fast Transitions • Partitioning is a lumping of a MC with fast transitions if it is a
lumping of its limit MP. • The lumped process:
ˆr = W QV Q
ˆs = W Qs V. Q
• W - instance of U (cf.) such that
ΠV W Π = ΠV W.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 26/40
τ -lumping Is Sound Thm: The following diagram commutes: MC with f.t.
τ →∞
/ MP
ordinary lumping
τ -lumping
lumped MC with f.t.
τ →∞
/ lumped
MP
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 27/40
Markov Chains with Silent Steps • Abstraction from "weights" in MC’s with fast transitions.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 28/40
Markov Chains with Silent Steps • Abstraction from "weights" in MC’s with fast transitions. • Def: MC with silent steps (Qr , [Qs ]∼ ).
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 28/40
Markov Chains with Silent Steps • Abstraction from "weights" in MC’s with fast transitions. • Def: MC with silent steps (Qr , [Qs ]∼ ). • Equivalence ∼ of matrices with the same grammar. a b c 0
∼
d e f 0
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 28/40
Markov Chains with Silent Steps • Abstraction from "weights" in MC’s with fast transitions. • Def: MC with silent steps (Qr , [Qs ]∼ ). • Equivalence ∼ of matrices with the same grammar. a b c 0
d e f 0
∼
Example:
1 τ
2
1
O
τ
µ
λ
4
~
3 λ
O
aτ
={
µ
2 λ
bτ
4
3
|
a, b > 0
}
λ
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 28/40
Progress 1. Markov Chains (MC’s) - Definition, Performance Analysis 2. Interactive Markov Chains - Definition, Performance Analysis 3. Motivation / Method 4. Lumping of MC’s - Introduction, Definition, Examples 5. (Discontinuous) Markov Processes - Introduction, Definition 6. MC’s with Fast Transitions - Introduction, Definition, Examples 7. τ -lumping of MC’s with Fast Transitions - Definition, Soundness, Examples 8. MC’s with Silent Steps - Introduction, Definition, Examples 9. τ∼ -lumping of MC’s with Silent Steps - Definition, Soundness, Examples 10. Markovian weak bisimulation vs. τ∼ -lumping - Divergence / Conclusion
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 29/40
Lumping of MC’s with Silent Steps - Main Requirement
• Should be a proper lifting of τ -lumping to the equivalence
classes of ∼.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 30/40
Lumping of MC’s with Silent Steps - Main Requirement
• Should be a proper lifting of τ -lumping to the equivalence
classes of ∼. • Lumping of (Qr , [Qs ]∼ ) should:
1. be a τ -lumping of any (Qr , Qs′ ), where Qs′ ∼ Qs ; and 2. not dependent on the choice for Qs′ .
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 30/40
Lumping of MC’s with Silent Steps - Example Intuitively: 1 O
τ
2
τ
µ
3
1,2,3 {{1,2,3},{4}}
/
T λ
µ
3 λ
4
~
λ
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 31/40
Lumping of MC’s with Silent Steps - Example This is because: 1 O
aτ
2
bτ
µ
1,2,3 {{1,2,3},{4}}
3
/
T λ
µ
3 λ
4
~
λ
• Correct for every value of the parameters; the lumped
process does not depend on them.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 31/40
τ∼ -lumping - Counterexample 1
1
T
1,2
τ
τ
2 λ
N OT
{{1,2},{3}}
/
λ
3
3
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 32/40
τ∼ -lumping - Counterexample 1 Because:
1 aτ
1,2
bτ
{{1,2},{3}}
2 λ
T
/
a λ a+b
3
3 •
τ -lumping works for any a, b, but the result depends on a, b.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 32/40
τ∼ -lumping - Counterexample 2
1 E E τ
2
EE zz z EE z EE zzzτ τ zzEEE z EE zz EE z |zzz E"
3
4
1,2 τ
N OT
{{1,2},{3},{4}}
/ 3
τ
44 44τ 44
4
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 33/40
τ∼ -lumping - Counterexample 2
Even though it works for arbitrary a, b, a, b: 1 E E aτ
2
EE zz z EE z z E z bτ EEzz aτ zz EEE z EE z z EE |zzz "
3
4
1,2 bτ
{{1,2},{3},{4}}
/
aτ
3
44 44bτ 44
4
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 33/40
τ∼ -lumping - Counterexample 2
It does not work for a, b, a, c: 1 E E aτ
2
EE zz z EE z z bτ EE zz aτ zzEEE z EE zz EE z |zzz E"
3
4
1,2 cτ
N OT
{{1,2},{3},{4}}
/
aτ
3
44 44?τ 44
4
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 33/40
τ∼ -lumping - Definition 1. For every class from the partitioning at least one of the following holds (cf.):
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 34/40
τ∼ -lumping - Definition 1. For every class from the partitioning at least one of the following holds (cf.): (a) Every ergodic class that can be reached by doing silent steps belongs to the same partitioning class.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 34/40
τ∼ -lumping - Definition 1. For every class from the partitioning at least one of the following holds (cf.): (a) Every ergodic class that can be reached by doing silent steps belongs to the same partitioning class. (b) All ergodic states that can be reached by doing silent steps belong to the same ergodic class.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 34/40
τ∼ -lumping - Definition 1. For every class from the partitioning at least one of the following holds (cf.): (a) Every ergodic class that can be reached by doing silent steps belongs to the same partitioning class. (b) All ergodic states that can be reached by doing silent steps belong to the same ergodic class. (c) The class contains only transient states and only one state can reach another class by doing a silent step.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 34/40
τ∼ -lumping - Definition 1. For every class from the partitioning at least one of the following holds (cf.): (a) Every ergodic class that can be reached by doing silent steps belongs to the same partitioning class. (b) All ergodic states that can be reached by doing silent steps belong to the same ergodic class. (c) The class contains only transient states and only one state can reach another class by doing a silent step. and 2. All ergodic states from one class reach other classes with the same accumulative rate.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 34/40
Soundness of τ∼ -lumping
Thm: The following holds: MC with f.t.
τ -lumping
/ τ -lumped
MC with f.t.
≀
≀
MC with f.t.
τ -lumping
/ τ -lumped
MC with f.t.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 35/40
τ∼ -lumping - Examples 1
O
τ
τ
2
µ
1,2,3
{{1,2,3},{4}}
3
cond: 1a, 2
/
T λ
µ
3
λ
4
~
λ
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 36/40
τ∼ -lumping - Examples 1
O
τ
τ
2
µ
1,2,3
{{1,2,3},{4}}
3
cond: 1a, 2
T /
λ
µ
3
λ
4
τ
λ
~
1,2
1 `
T τ
{{1,2},{3},{4}}
2 τ
4
4 3 λ
τ
τ
cond: 1b, 2
/
3 λ
4 DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 36/40
τ∼ -lumping - Examples 2 1 1,2
τ
τ
3
{{1,2},{3},{4}}
22
22 22τ 2
cond: 1c, 2
/ 3
τ
66 66τ 66
4
4
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 37/40
τ∼ -lumping - Examples 2 1 1,2
τ
22
cond: 1c, 2
22 22τ 2
τ
3
{{1,2},{3},{4}}
1
t
/ 3
τ
66 66τ 66
4
4
τ
22 τ 22 λ 22
3
4 2 λ
1,2 {{1,2},{3}} cond: 1a, 2
/
λ
3
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 37/40
Progress 1. Markov Chains (MC’s) - Definition, Performance Analysis 2. Interactive Markov Chains - Definition, Performance Analysis 3. Motivation / Method 4. Lumping of MC’s - Introduction, Definition, Examples 5. (Discontinuous) Markov Processes - Introduction, Definition 6. MC’s with Fast Transitions - Introduction, Definition, Examples 7. τ -lumping of MC’s with Fast Transitions - Definition, Soundness, Examples 8. MC’s with Silent Steps - Introduction, Definition, Examples 9. τ∼ -lumping of MC’s with Silent Steps - Definition, Soundness, Examples 10. Markovian weak bisimulation vs. τ∼ -lumping - Divergence / Conclusion
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 38/40
Markovian weak bsm. vs. τ∼ -lumping – Divergence Markovian weak bsm.
τ∼ -lumping •
•
Silent steps always have priority over rates.
Silent steps have priority over rates only if they are not part of a closed τ -loop (i.e. form an ergodic class).
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 39/40
Markovian weak bsm. vs. τ∼ -lumping – Divergence Markovian weak bsm. •
Silent steps always have priority over rates.
•
Divergence denoted by ⊥ in case of closed τ loops.
τ∼ -lumping •
Silent steps have priority over rates only if they are not part of a closed τ -loop (i.e. form an ergodic class).
•
No divergence at all.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 39/40
Markovian weak bsm. vs. τ∼ -lumping – Divergence Markovian weak bsm.
τ∼ -lumping
•
Silent steps always have priority over rates.
•
Divergence denoted by ⊥ in case of closed τ loops. In IMC:
1
•
Silent steps have priority over rates only if they are not part of a closed τ -loop (i.e. form an ergodic class).
•
No divergence at all.
T
τ
1
τ
weak bisim.
/
τ −elim.+lump.
2
τ
T
weak bisim. τ
/
τ −elim.+lump.
⊥
λ
2
3
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 39/40
Markovian weak bsm. vs. τ∼ -lumping – Divergence Markovian weak bsm.
τ∼ -lumping
•
Silent steps always have priority over rates.
•
Divergence denoted by ⊥ in case of closed τ loops. In IMC:
1
•
Silent steps have priority over rates only if they are not part of a closed τ -loop (i.e. form an ergodic class).
•
No divergence at all.
T
τ
1
τ
weak bisim.
/
τ −elim.+lump.
2
τ
T
weak bisim. τ
/
τ −elim.+lump.
⊥
λ
2
3 •
The process observed as MC with silent steps is not τ∼ -lumpable. (cf.) DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 39/40
Present and Future Work • Lumping with rewards (problem when rewards depend on
τ ). • Introduction of probabilistic choice (Qr , Qp , [Qs ]∼ ). • Addition of actions and parallel composition. • Axiomatization. • Algorithms.
DSSG, Saarbrucken ¨ - Feb. 1; PAM, CWI Amsterdam - Feb. 22; PROSE - TU/e, Apr. 13; MOVES, Aachen, May 10 – p. 40/40
