Estimating Copulas from Scarce Observations, Expert Opinions and Regulatory Guidelines: A Bayesian Approach Philipp Arbenz www.math.ethz.ch/∼arbenz/
RiskDay 2010, ETH Z¨ urich, 17. September 2010 Joint work with Davide Canestraro Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
1 / 21
Outline
1
Copulas and associated problems in practice
2
Different sources of information for copula estimation
3
Psychological aspects in expert judgement
4
Conclusion
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
2 / 21
Copulas and associated problems in practice
Copulas - a tool to represent dependence The cdf H(x, y ) = P[X ≤ x, Y ≤ y ] of a bivariate random vector (X , Y ) can be written as H(x, y ) = C (FX (x), FY (y )) , where C : [0, 1]2 → [0, 1] is the copula function. The copula captures all aspects of dependence between X and Y .
Parametric families for modeling: Gaussian, t, Clayton, Gumbel, Frank, ... The same is valid in multidimensional settings
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
3 / 21
Copulas and associated problems in practice
Example: Two copula scatter plots Two copula sample sets (N = 500) with • same Spearman rank correlation = 0.6 • very different tail behaviour Gaussian Copula
Flipped Clayton Copula
1
1
0.5
0.5
0 0
Philipp Arbenz (ETH Z¨ urich, SCOR)
0.5
1
0 0
Estimating Copulas from Data and Experts
0.5
1
17.9.2010 - Z¨ urich
4 / 21
Copulas and associated problems in practice
Diversification and dependence • Insurance companies are pooling risks: After aggregation, risks which are not bearable individually are bearable for an insurance. Risks diversify!
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
5 / 21
Copulas and associated problems in practice
Diversification and dependence • Insurance companies are pooling risks: After aggregation, risks which are not bearable individually are bearable for an insurance. Risks diversify!
• Measure diversification → correctly assess marginal distributions and dependence of the risks!
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
5 / 21
Copulas and associated problems in practice
Diversification and dependence • Insurance companies are pooling risks: After aggregation, risks which are not bearable individually are bearable for an insurance. Risks diversify!
• Measure diversification → correctly assess marginal distributions and dependence of the risks!
• Estimate dependence/copula from data - Joint observations of the risks are needed. Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
5 / 21
Copulas and associated problems in practice
Assessing dependence: Problem 1
Problem 1: Joint observations are often inexistent, scarce or even unobservable
Example: Risk managers wants to estimate the dependence for joint 1-in-250 (or even 1-in-1000) year events, but often only few years of data are available.
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
6 / 21
Copulas and associated problems in practice
Assessing dependence: Problem 2 Problem 2: Existing joint observations can be very misleading
Example: Using the last 50 years for estimation, emprical correlation of government defaults of Ukraine and Romania is = -2%! But dependence is certainly • positive, • very strong in the tails.
Historic defaults: Romania 1982 Ukraine 1998 Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
7 / 21
Different sources of information for copula estimation
Problem Setting Problem Setting: • We want to estimate the copula C of (X, Y) • Assume C belongs to a given parametric family • But joint observations are scarce → it is not sensible to use maxiumum-likelihood or method-of-moments alone as estimation uncertainty would be too high.
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
8 / 21
Different sources of information for copula estimation
Problem Setting Problem Setting: • We want to estimate the copula C of (X, Y) • Assume C belongs to a given parametric family • But joint observations are scarce → it is not sensible to use maxiumum-likelihood or method-of-moments alone as estimation uncertainty would be too high.
Instead, we seek to estimate C from • the scarce observations, • expert opinions, • prior information from regulators. Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
8 / 21
Different sources of information for copula estimation
First Source: Scarce Observations
First Source: Scarce Observations Observations of (X , Y ) =⇒ Observations of C (Ui , Vi ) = (FX (Xi ), FY (Yi )) ∼ C Pareto Marginals
Copula (Pseudo−)Samples 1
8 6 4
0.5
2 0 0
2
4
Philipp Arbenz (ETH Z¨ urich, SCOR)
6
8
0 0
Estimating Copulas from Data and Experts
0.5
1 17.9.2010 - Z¨ urich
9 / 21
Different sources of information for copula estimation
Second Source: Expert opinions
Second Source: Expert opinions
Let ρ(·, ·) be a dependence measure, e.g. • (Rank) correlation • Tail dependence Experts can provide subjective estimates of ρ(X , Y ).
Attention! Strong psychological effects involved.
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
10 / 21
Different sources of information for copula estimation
Third Source: Regulatory Guidelines
Third Source: Regulatory Guidelines Excerpt from the correlation matrix for current year risk of different lines of business in the SST standard-model: (SST = Swiss Solvency Test) Motor Liability Motor Liability 1 0 Property General Liability 0.25 0 Aviation
Property 0 1 0.25 0
General Liability 0.25 0.25 1 0
Aviation 0 0 0 1
Possible alternatives for this third source: • Industry standards • Prior year estimates Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
11 / 21
Different sources of information for copula estimation
Combining Information through Bayesian Inference
Combining Information through Bayesian Inference
Bayesian approach: 1
Suppose ρ(X , Y ) is a realization of a random variable θ
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
12 / 21
Different sources of information for copula estimation
Combining Information through Bayesian Inference
Combining Information through Bayesian Inference
Bayesian approach: 1
Suppose ρ(X , Y ) is a realization of a random variable θ
2
Calculate a posterior density πpost (θ) of θ, given all three sources of information
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
12 / 21
Different sources of information for copula estimation
Combining Information through Bayesian Inference
Combining Information through Bayesian Inference
Bayesian approach: 1
Suppose ρ(X , Y ) is a realization of a random variable θ
2
Calculate a posterior density πpost (θ) of θ, given all three sources of information Infer an estimate θb of ρ(X , Y ) from πpost (θ)
3
(e.g. posterior mean)
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
12 / 21
Different sources of information for copula estimation
Combining Information through Bayesian Inference
Combining Information through Bayesian Inference
Bayesian approach: 1
Suppose ρ(X , Y ) is a realization of a random variable θ
2
Calculate a posterior density πpost (θ) of θ, given all three sources of information Infer an estimate θb of ρ(X , Y ) from πpost (θ)
3
(e.g. posterior mean) 4
\ Calibrate the copula C according to θb = ρ(X ,Y)
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
12 / 21
Different sources of information for copula estimation
Combining Information through Bayesian Inference
Bayesian Inference in Detail Bayesian inference gives a posterior density πpost (θ) of θ: πpost (θ) ∝ πprior (θ)
N Y
c FX (Xn ), FY (Yn )|θ
| {z } |n=1
A
{z
B
K Y
ek (ϕk |θ)
} k=1 | {z
C
}
A Prior density: fitted to estimate in regulatory guidelines
(uninformative if not available). B Copula likelihood function: product of copula density conditioned on ρ=θ C Expert opinion: product of the conditional expert densities (Assuming experts are independent, conditionally unbiased, and have a certain variance) Full mathematical details: see paper Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
13 / 21
Different sources of information for copula estimation
Combining Information through Bayesian Inference
Bayesian Inference in Detail Bayesian inference gives a posterior density πpost (θ) of θ: πpost (θ) ∝ πprior (θ)
N Y
c FX (Xn ), FY (Yn )|θ
| {z } |n=1
A
{z
B
K Y
ek (ϕk |θ)
} k=1 | {z
C
}
A Prior density: fitted to estimate in regulatory guidelines (uninformative if not available). B Copula likelihood function: product of copula density
conditioned on ρ = θ C Expert opinion: product of the conditional expert densities (Assuming experts are independent, conditionally unbiased, and have a certain variance) Full mathematical details: see paper Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
14 / 21
Different sources of information for copula estimation
Combining Information through Bayesian Inference
Bayesian Inference in Detail Bayesian inference gives a posterior density πpost (θ) of θ: πpost (θ) ∝ πprior (θ)
N Y
c FX (Xn ), FY (Yn )|θ
| {z } |n=1
A
{z
B
K Y
ek (ϕk |θ)
} k=1 | {z
C
}
A Prior density: fitted to estimate in regulatory guidelines (uninformative if not available). B Copula likelihood function: product of copula density conditioned on ρ=θ C Expert opinion: product of the conditional expert
densities (Assuming experts are independent, conditionally unbiased, and have a certain variance) Full mathematical details: see paper Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
15 / 21
Different sources of information for copula estimation
Combining Information through Bayesian Inference
An Example Illustrating the Bayesian Inference Example: Gaussian copula • uninformative prior (no information from regulator) • 24 observations • Expert estimates of Spearman rank correlation: 0.35, 0.6, 0.7
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
16 / 21
Different sources of information for copula estimation
Combining Information through Bayesian Inference
An Example Illustrating the Bayesian Inference Example: Gaussian copula • uninformative prior (no information from regulator) • 24 observations • Expert estimates of Spearman rank correlation: 0.35, 0.6, 0.7 8 6 4 2 0 −1
θ θ|Experts θ|Observations θ|Observations, Experts
−0.5
Philipp Arbenz (ETH Z¨ urich, SCOR)
0 Spearman rank correlation
Estimating Copulas from Data and Experts
0.5
1
17.9.2010 - Z¨ urich
16 / 21
Different sources of information for copula estimation
Combining Information through Bayesian Inference
An Example Illustrating the Bayesian Inference Example: Gaussian copula • uninformative prior (no information from regulator) • 24 observations • Expert estimates of Spearman rank correlation: 0.35, 0.6, 0.7 8 6 4 2 0 −1
θ θ|Experts θ|Observations θ|Observations, Experts
−0.5
0 Spearman rank correlation
0.5
1
Estimate: θb = 0.43 = E[θ|Observations, Experts] Uncertainty: std(θ|Observations, Experts) = 0.056 90%-Credible-Interval = [0.33, 0.51] Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
16 / 21
Psychological aspects in expert judgement
First Example: Representativeness
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
17 / 21
Psychological aspects in expert judgement
First Example: Representativeness Linda is 31 years old, single, outspoken, bright and majored in philosophy. She is deeply concerned with issues of discrimination and social justice. Which is more likely? A Linda is a bank teller B Linda is a bank teller who is active in the feminist movement. (bank teller = cashier)
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
17 / 21
Psychological aspects in expert judgement
First Example: Representativeness Linda is 31 years old, single, outspoken, bright and majored in philosophy. She is deeply concerned with issues of discrimination and social justice. Which is more likely? A Linda is a bank teller B Linda is a bank teller who is active in the feminist movement. (bank teller = cashier)
In a study most people answered P(B) > P(A) BUT: P(B) < P(A) as B ⊂ A!
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
17 / 21
Psychological aspects in expert judgement
First Example: Representativeness Linda is 31 years old, single, outspoken, bright and majored in philosophy. She is deeply concerned with issues of discrimination and social justice. Which is more likely? A Linda is a bank teller B Linda is a bank teller who is active in the feminist movement. (bank teller = cashier)
In a study most people answered P(B) > P(A) BUT: P(B) < P(A) as B ⊂ A!
In the context of dependence: the strength of dependence between risks is not necessarily linked to homogeneity of the risks. Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
17 / 21
Psychological aspects in expert judgement
Second Example: Availablity
Which hazard claims more lives in the United States: lightning or tornadoes?
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
18 / 21
Psychological aspects in expert judgement
Second Example: Availablity
Which hazard claims more lives in the United States: lightning or tornadoes? Most people deem tornadoes to be more deadly due to larger media coverage. BUT: Lightning kills 73 per year, Tornados 68 per year!
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
18 / 21
Psychological aspects in expert judgement
Second Example: Availablity
Which hazard claims more lives in the United States: lightning or tornadoes? Most people deem tornadoes to be more deadly due to larger media coverage. BUT: Lightning kills 73 per year, Tornados 68 per year!
In the context of dependence: easily recallable risk factors might not be the most important for dependence
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
18 / 21
Conclusion
Conclusion
• Copulas can account for all aspects of dependency • Scarce observations can lead to misleading and wrong estimates of dependence • Estimating copulas by combining different sources of information allows a - reduced estimation uncertainty, - prudent, defendable dependence estimation. • Expert judgement procedures must be planned carefully. A badly designed elicitation procedure may result in worthless answers.
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
19 / 21
Conclusion
Thank you for your attention!
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
20 / 21
References
• Arbenz, P. and Canestraro, D. (2010): Estimating Copulas for Insurance from Scarce Observations, Expert Opinions and Prior Information: A Bayesian Approach. Submitted. • Lambrigger, D., Shevchenko, P. and W¨ uthrich, M. (2007) The quantification of operational risk using internal data, relevant external data and expert opinions. Journal of Operational Risk 2(3), 3-27. • O’Hagan et al. (2006) Uncertain Judgements: Eliciting Experts’ Probabilities. Wiley, Chichester. • Kynn, M. (2008) The ”Heuristics and Biases” bias in expert elicitation. Journal of the Royal Statistical Society 171(1), 239-264. • Kahneman, D. and Tversky, A. (1982): On the study of statistical intuitions. Cognition 11, 123-141.
Philipp Arbenz (ETH Z¨ urich, SCOR)
Estimating Copulas from Data and Experts
17.9.2010 - Z¨ urich
21 / 21
