2013 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, SEPT. 22–25, 2013, SOUTHAPMTON, UK

ESTIMATION OF CAUSAL STRUCTURES IN LONGITUDINAL DATA USING NON-GAUSSIANITY Kento Kadowaki, Shohei Shimizu∗ , Takashi Washio † The Institute of Scientific and Industrial Research, Osaka University, Japan previous methods [6] cannot estimate causal structures correctly when causal structures change over time. Therefore, we Recently, there is a growing need for statistical learning of take a new approach to overcome this problem. To be more causal structures in data with many variables. A structural precise, we propose a new model that describes the features equation model called Linear Non-Gaussian Acyclic Model of longitudinal data and attempt to more accurately estimate (LiNGAM) has been extensively studied to uniquely estimate causal structures in longitudinal data. To check the validity causal structures in data. The key assumptions are that exof our approach, we conduct experiments with artificial data ternal influences are independent and follow non-Gaussian and simulated Functional MRI (FMRI) data. There are many distributions. However, LiNGAM does not capture temporal observed data that have temporal structural changes, such as structural changes in observed data. In this paper, we conFMRI data. Previous studies have used AR-LiNGAM algosider learning causal structures in longitudinal data that colrithm [6]. However, AR-LiNGAM cannot estimate causal lects samples over a period of time. In previous studies of structures which change over time because it can accept only LiNGAM, there was no model specialized to handle longione time-series data, e.g., a data set from single subject in tudinal data with multiple samples. Therefore, we propose the case of FMRI. However, we often obtain more than one a new model called longitudinal LiNGAM and a new estitime-series data set, e.g., a data set from multiple subjects in mation method using the information on temporal structural the case of FMRI. Therefore, we propose a model that capchanges and non-Gaussianity of data. The new approach retures causal structures changing over time and its estimation quires less assumptions than previous methods. algorithm using the information from multiple subjects. This paper is structured as follows. We first define ’lonIndex Terms— structural equation models, non-Gaussianity, gitudinal data’ and review LiNGAM model, and also discuss autoregressive model their relations to previous models and estimation methods in Section 2. Next, we propose a new model called longitudinal 1. INTRODUCTION LiNGAM, and its estimation algorithm. In section 4, we conduct experiments on artificial data to evaluate our estimation In the field of machine learning, there has been an increase method. We also conduct an experiment on simulated FMRI in research to study causal structures in data [1]. Strucdata and discuss the result. Section 6 concludes the paper. tural equation models [2] and Bayesian networks [3, 4] have been used to analyze causal structures and widely applied in 2. BACKGROUND many fields. A structural equation model called Linear NonGaussian Acyclic Model (LiNGAM) [5] has been extensively 2.1. LONGITUDINAL DATA studied to uniquely estimate causal structures in data. The key assumptions are that external influences are independent and Longitudinal data are defined as data set that is provided follow non-Gaussian distributions. However, LiNGAM does by observing variables from multiple samples over a period not capture temporal structural changes in observed data. of time. It is very important that variables are arranged in In our study, we focus on estimating causal structures order in time-series. It is also important that the number in ’longitudinal data’ that is provided by observing variof time points is infinite and their values are discrete, and ables from many samples repeatedly over a period of time. the time interval is set arbitrarily by the observer. These Previously, a non-Gaussian learning method that combines variables observed over a period of time are defined as LiNGAM with autoregressive models (AR-models) [6] have xm i (t) (i = 1, . . . , p, m = 1, . . . , n, t = 1, . . . , T ) where been used. The method is called AR-LiNGAM. However, i is an index of data dimension, m is an index of sample ∗ Thanks to KAKENHI 24700275. size and t is an index of time points. Specifying one sam† Thanks to KAKENHI 22300054. ple number ’m’ provides us one time-series data sets with ABSTRACT

c 978-1-4673-1026-0/12/$31.00 ⃝2013 IEEE

sample number ’m’ that are expressed by p-dimensional vectors xm (1), . . . , xm (T ) (m = 1, . . . , n) where xm (t) = m t [xm 1 (t), . . . , xp (t)] . Similarly, specifying one time point ’t’ provides us one data matrix X(t) = [x1 (t), . . . , xn (t)] whose size is p × n. This means that a matrix X(t) is observed in n different samples at time point t. Observed data set called ”longitudinal data” is a data tensor whose size is p × n × T . In addition, longitudinal data X(t) is obtained by arranging T data matrices X(t) in time ordering. 2.2. A LINEAR NON-GAUSSIAN ACYCLIC MODEL: LiNGAM In [5], a non-Gaussian variant of structural equation models and Bayesian networks, which is called LiNGAM, was proposed. Assume that observed data are generated from a process represented graphically by a directed acyclic graph, that is, DAG. Let us represent this DAG by a p × p adjacency matrix B = {bij } where every bij represents the direct causal effect (connection strength) from a variable xj to another xi . Moreover, let us denote by k(i) a causal order of variable xi so that no later variable has a directed path1 to any earlier variable in the DAG. In other words, k(i) < k(j) means that xi could have a directed path to xj , but not vice versa. Further, assume that the relations between variables are linear. Without loss of generality, each variable xi is assumed to have zero mean. Then we have ∑ bij xi + ei , (1) xi =

x(t) =

where ei is an external influence. All external influences ei are continuous random variables having non-Gaussian distributions with zero means and non-zero variances. The ei are independent of each other assuming that there is no unobserved confounding variable [4]. We rewrite the model (1) in a matrix form as follows: (2)

where x is a p-dimensional variable vector, and B could be permuted by a simultaneous equal row and column permutation to be lower triangular with all zeros on the diagonal due to the acyclicity assumption [2]. Now we can uniquely estimate a causal order k(i) and an adjacency matrix B via LiNGAM analysis [5]. For instance, a typical estimation method for LiNGAM analysis is DirectLiNGAM that is proposed in [7]. In a part of our estimation algorithm, we use DirectLiNGAM in order to estimate causal structures. 2.3. A STRUCTURAL VECTOR AUTOREGRESSIVE MODEL: SVAR MODEL In [8], a structural vector autoregressve model so called SVAR model that requires one vectorial time-series data as input 1 A directed path from x to x is a sequence of directed edges such that i j xj is reachable from xi .

l ∑

Bτ x(t − τ ) + e(t),

(3)

τ =1

where l is the number of time-delays, that is, the order of ARmodel and Bτ , τ = 1, . . . , l are p × p matrices. In addition, e(t) is the vector of whose components are external influences ei (t). In many applications, the influences between xi (t) can be instantaneous and lagged [6, 8]. However, AR-model does not capture the instantaneous causal structures. As a solution, SVAR model was proposed in the econometrics literature [8], which is a combination of AR models and structural equation models. In SVAR model, lagged influences are characterized by (3), and instantaneous influences are by (2). Thus, SVAR model: x(t) =

k(j)
x = Bx + e,

was proposed and has been widely used to estimate causal structures of the time-series data, especially in the field of econometrics. We first explain about a typical time-series model which is called autoregressive model (AR-model) before mentioning the SVAR model. We denote variables that are observed over a period of time as xi (t), i = 1, . . . , p, t = 1, . . . , T . Each xi (t) has zero mean without loss of generality. All the time-series variables are collected into a single column vector x(t) = [x1 (t), . . . , xp (t)]t . The AR-model is defined as follows:

l ∑

Bτ x(t − τ ) + e(t),

(4)

τ =0

where Bτ of which size is p×p matrices express causal effects between the variables xi with time lag τ (τ = 0, . . . , l). For τ > 0, the effects are ordinary lagged effects from the past to the present, while for τ = 0, the effects are instantaneous. However, it is quite difficult to estimate SVAR-model since B0 is not identifiable in many cases if Gaussianity of external influences is assumed. In [6], the following assumptions were added to ei (t) in order to estimate causal structures of SVAR model via LiNGAM analysis. 1. The ei (t) are mutually independent, both of each other and over time. 2. The ei (t) are non-Gaussian． 3. The matrix modelling instantaneous effects, B0 , corresponds to a directed acyclic graph． These assumptions were the same as in LiNGAM [9]. They were typical assumptions in this field. We call SVAR model with these LiNGAM assumptions AR-LiNGAM model [6] and its estimation algorithm, that is AR-LiNGAM algorithm, in the next section.

2.4. AR-LiNGAM ALGORITHM In [7], it was shown that a causal order of a residual in regression analysis is equal to a causal order of the variable before regression analysis. So it would be possible to apply LiNGAM analysis to residuals which was obtained by eliminating effects from the past and to estimate instantaneous causal effects B0 in AR-LiNGAM model. The basic idea in AR-LiNGAM algorithm is as follows. First, we estimate autoregressive coefficient matrices Mτ by classic least-squares methods. Second, we compute the residual vector n(t) via Mτ . Third, we apply LiNGAM analysis to n(t) to obtain instantaneous causal effects in B0 . ARLiNGAM algorithm makes it possible to estimate both effects from the past to the present and instantaneous [6]. The following Algorithm 1 is AR-LiNGAM algorithm [6]. Algorithm 1: AR-LiNGAM algorithm Input: p × T data matrix XAR ,2 (p ≪ T ), p variables，T time points 1. Estimate a classic autoregressive model for the input data l ∑ x(t) = Mτ x(t − τ ) + n(t) (5) τ =1

3. A NEW APPROACH TO ESTIMATE CAUSAL STRUCTURES OF LONGITUDINAL DATA 3.1. DEFINITION OF OUR NEW MODEL: LONGITUDINAL LiNGAM In SVAR model (4), input data is limited to one vectorial timeseries data, for the sample size n = 1. However, more than one time-series data, that is, longitudinal data, are often obtained in many applications. Such longitudinal data enables us to eliminate the restriction of AR-LiNGAM model and to propose such a new model. By the definition of longitudinal data in Sec. 2.1 , we denote X at a time point t by X(t), where X is a p × n × T tensor and X(t) is a p × n matrix. The longitudinal LiNGAM model is defined as follows: X(t) =

l ∑

B(t, t − τ )X(t − τ ) + E(t).

(6)

τ =0

This model (6) has the following features:

using any conventional implementation of a least squares method. Note that n(t) collects residuals in AR model．Denote that the least squares estimates of ˆ τ. the autoregressive matrices by M 2. From (5), compute the following residuals n(t): ˆ (t) = x(t) − n

data is too small to estimate causal structures in longitudinal data that may temporally change. This restriction causes a problem that AR-LiNGAM is not able to estimate causal structures correctly in cases where causal structures change over a period of time.

l ∑

ˆ τ x(t − τ ). M

τ =1

3. Perform the LiNGAM analysis on the residuals n(t). This gives the estimate of the matrix B0 as instantaneous causal effect, since n(t) follows LiNGAM (2). ˆ (t) = B0 n ˆ (t) + e(t). n 4. Finally, compute the estimates of the causal effect matrices Bτ for τ > 0 as ˆ τ = (I − Bˆ0 )M ˆ τ (τ > 0).3 B ˆ 0 which represents instantaneous effects Output: A matrix B ˆ τ (τ > 0) which characterize lagged effects. and matrices B This existing method requires that the input data is only one time-series data XAR and has strong restriction that maˆ 0 and matrices B ˆ τ (τ > 0) do not change with time trix B point t. The information amount such as only one time-series 2X AR means one vectorial time-series data which arranges pdimensional variable vectors in order of time. 3 Refer to [6] to derive relational expression between B , B and τ 0 Mτ (τ > 0).

• A component eij (t) of a p×n matrix E(t) is an external influence. Each eij (t) follows a non-Gaussian distribution. They are mutually independent both of each other and over time. The independence assumption is also made in LiNGAM and AR-LiNGAM. • B(t, t − τ ) are p × p matrices. B(t, t − τ ) represent the effects from variables τ time points before if τ is positive (τ > 0) and effects from instantaneous variables if τ equals to zero. In addition, B(t, t − τ ) depend not only on time lag τ but also on time point t, so this model characterizes complex causal structures that change over a period of time. • l means the number of time-delays used. In addition, we give examples of AR-LiNGAM model and longitudinal LiNGAM model in Fig.1. AR-LiNGAM model describes causal structures that are always fixed in time. In contrast, longitudinal LiNGAM model describes causal structures that may change over a period of time. 3.2. LONGITUDINAL LiNGAM ALGORITHM Now we propose a new algorithm to estimate causal structures of a longitudinal LiNGAM model. First, we explain the basic idea of a step in the method. From (6), we have (I − B(t, t))X(t) =

l ∑ τ =1

B(t, t − τ )X(t − τ ) + E(t).

Algorithm 2: Longitudinal LiNGAM algorithm

AR-LiNGAM model A5 :u;

A5 :t;

A5 :s; T5 :s;

T5 :t;

T5 :u;

T6 :s;

T6 :t;

T6 :u;

‰‰‰

Input: p × n × T data tensor X，(p ≪ n) 1. Estimate matrices M(t, t − τ ) (τ > 0) of (6) applying typical regression methods such as least square regression in which X(t) are set to response variables and X(t − 1), . . . , X(t − l) are set to explanatory variables. Here, the matrices obtained by a least square regression ˆ t − τ ). method are denoted by M(t,

‰‰‰

A6 :s;

A6 :u;

A6 :t;

Longitudinal LiNGAM model A5 :u;

A5 :t;

A5 :s; T5 :s;

T5 :t;

T5 :u;

T6 :s;

T6 :t;

T6 :u;

2. Compute the residuals N(t) based on (6) as follows: ‰‰‰

‰‰‰

A6 :s;

A6 :t;

ˆ N(t) = X(t) −

Multiplying (I−B(t, t))−1 from the left on the both sides, we have l ∑ τ =1

∑l Denote X(t) − τ =1 (I − B(t, t))−1 B(t, t − τ )X(t − τ ) by N(t). 4 I − B(t, t) is invertible since B(t, t) is a lower triangular matrix whose diagonal elements are zero. Therefore, we have N(t) = (I−B(t, t))−1 E(t) ⇐⇒ N(t) = B(t, t)N(t)+E(t). This means N(t) follows LiNGAM (2). Denote the matrix (I − B(t, t))−1 B(t, t − τ ) by M(t, t − τ ). The matrix M(t, t − τ ) depends not only on time lag τ but also on time point t unlike AR-LiNGAM model. Then we obtain: X(t) =

M(t, t − τ )X(t − τ ) + N(t).

(7)

τ =1

Now we can estimate M(t, t − τ ) by using any typical method such as least-square regression, where the response variables are X(t) and the explanatory variables are X(t − 1), . . . , X(t − l). Then we can compute the instantaneous effects B(t, t) by applying LiNGAM analysis [6] to the residuals N(t) where lagged effects have already been removed by the regression above. The actual algorithm is given in Algorithm 2.

4 N(t)

is a p × n matrix of which size equals to X(t)

3. Estimate matrices B(t, t) (t = 2, . . . , T ) applying LiNGAM analysis such as DirectLiNGAM because the residuals follow LiNGAM: ˆ ˆ N(t) = B(t, t)N(t) + E(t). 4. Finally, estimate B(t, t − τ ) (τ > 0) as follows. ˆ t − τ ) = (I − B(t, ˆ t))M(t, ˆ t − τ ) (τ > 0). B(t, ˆ t) that represent instantaneous causal Output: A matrix B(t, ˆ t − τ ) (τ > 0) that represent lagged effects and matrices B(t, effects.

(I − B(t, t))−1 B(t, t − τ )X(t − τ )

= (I − B(t, t))−1 E(t).

l ∑

ˆ t − τ )X(t − τ ). M(t,

τ =1

A6 :u;

Fig. 1. Example graphs of AR-LiNGAM model and longitudinal LiNGAM model where p = 2, T = 3, l = 1.

X(t) −

l ∑

4. ARTIFICIAL DATA EXPERIMENTS We conducted experiments on artificial data sets. In this experiment, we compared estimation accuracy of longitudinal LiNGAM algorithm, AR-LiNGAM algorithm and AR-model that is a typical time series analysis. 4.1. EXPERIMENTAL SETTING We generated artificial data sets as follows. 1. We generated lower triangular matrices whose diagonal components are zero and lower triangular components are not zero. We labeled the matrices as B(t, t) (t = 1, . . . , T ) that characterize instantaneous causal effects. 2. We replaced non-zero elements of B(t, t) (t = 1, . . . , T ) with the values randomly generated from the uniform distribution between [−1.5, −0.5] ∪ [0.5, 1.5]. We set l to 1, and randomly chose zero or non-zero pattern in B(t, t − 1) (t = 2, . . . , T ). Then, we generated each non-zero element in B(t, t − 1) (t = 2, . . . , T ) from the interval [−0.5, 0] ∪ [0, 0.5]. 3. We randomly selected variances of the external influence eij (t) from the interval [−0.5, 0] ∪ [0, 0.5]. The distributions of external influences eij (t) (t =

1, . . . , T, i = 1, . . . , p, j = 1, . . . , n) were randomly selected from various eighteen non-Gaussian distributions in [10]. In case of t = 1, we generated a matrix X(1) via following equation. X(1) = B(1, 1)X(1) + E(1). 4. We generated observed matrices X(t) via (6). Finally, we randomly permuted the rows of each X(t). We evaluated estimation accuracy of instantaneous influences and lagged influences. Thus, we estimated B(t, t) and B(t, t − 1) applying AR-LiNGAM algorithm, AR-model and longitudinal LiNGAM algorithm to the generated data and compared its estimation accuracy. In the case of AR-model, we estimated only lagged influences. To apply AR-LiNGAM algorithm and AR-model to the data, we chose one vectorial time-series data XAR as an input data by randomly specifying one sample number. Then we obtained instantaneous matrix B0 and lagged matrix B1 by AR-LiNGAM algorithm and also obtained lagged matrix Φ1 by AR-model. Next, we input aforementioned data tensor X into longitudinal LiNGAM algorithm and obtained B(t, t) (t = 1, . . . , T ) and B(t, t − 1) (t = 2, . . . , T ). We finally set redundant causal effects bij to zero using adaptive lasso [11] similarly to [7]. 4.2. EXPERIMENTAL RESULT We compared Root Mean Square Error (RMSE) of each maˆ 0, B ˆ 1, Φ ˆ 1 , B(t, ˆ t) (t = 1, . . . , T ) and B(t, ˆ t − 1) (t = trix B 2, . . . , T ) estimated by the two methods. The RMSE of B(t, t) (t = 1, . . . , T ) defined as follows:

RM SE(B(t, t)) = √ ∑T ∑p 1 p2 (T −1)

t=1

i,j=1

(

)2 ˆ t)(i, j) . Btrue (t, t)(i, j) − B(t,

ˆ 0, B ˆ 1 ,Φ ˆ 1 and B(t, ˆ t − 1) (t = We calculated RMSE of B 2, . . . , T ) in the same manner.5 Then we estimated instantaneous matrices repeatedly by each algorithm. We computed mean values of RMSE of each method. The number of triˆ 0, B ˆ 1, als was 10. Table 1 shows the means of RMSEs of B ˆ 1 , B(t, ˆ t) (t = 1, . . . , T ) and B(t, ˆ t − 1) (t = 2, . . . , T ) in Φ the case of T = 20, and Table 2 shows those in the case of T = 50. The tables show that the estimation accuracy of longitudinal LiNGAM improves as sample size n gets larger, but those of AR-LiNGAM and AR-model do not improve. AR-LiNGAM does not estimate causal structures correctly since it doesn’t meet the assumption that causal structures can change over a 5 In this case, B ˆ 0, B ˆ 1 and Φ ˆ 1 does not depend on time point t (t = 1, . . . , T ).

Table 1. Root mean square errors in the case of T = 20.

AR-LiNGAM algorithm (Used only one sample) AR-model (Used only one sample and estimated only lagged infuences) Longitudinal LiNGAM algorithm

p=2 p=3 p=5 p=2 p=3 p=5 p=2 p=3 p=5

100 0.54 0.88 0.77 0.40 0.61 0.55 0.36 0.43 0.51

sample size n 200 500 0.52 0.49 0.67 0.72 0.80 1.02 0.43 0.34 0.46 0.40 0.50 0.67 0.35 0.24 0.29 0.20 0.37 0.40

1000 0.51 0.67 0.83 0.45 0.54 2.87 0.12 0.22 0.11

Table 2. Root mean square errors in the case of T = 50.

AR-LiNGAM algorithm (Used one sample) AR-model (Used only one sample and estimated only lagged infuences) Longitudinal LiNGAM algorithm

p=2 p=3 p=5 p=2 p=3 p=5 p=2 p=3 p=5

100 0.59 0.71 0.83 0.31 0.37 0.93 0.37 0.44 0.48

sample size n 200 500 0.52 0.51 0.68 0.67 1.10 1.03 0.37 0.27 0.36 0.34 0.60 1.61 0.31 0.29 0.33 0.30 0.41 0.31

1000 0.54 0.62 0.82 0.32 0.43 1.40 0.12 0.21 0.26

period of time. In addition, the differences in estimation accuracy between these two methods become significant as the dimension p gets larger. These results show that our proposed method is much better than the conventional AR-LiNGAM method when estimating causal structures in longitudinal data.

5. SIMULATED FMRI DATA EXPERIMENTS We also conducted experiments on simulated Functional Magnetic Resonance Imaging (FMRI) data presented in [12]. FMRI is an MRI procedure that measures brain activity by detecting associated changes in blood flow. The simulated FMRI data were generated based on a mathematical brain model called dynamic causal modeling [13]. In this mathematical model, relations between variables are non-linear. We used Simulation 22 data since the causal effects are nonstationary over a period of time. This dataset consisted of 5 variables whose instantaneous causal structure is shown in Fig. 2. The session duration was 2.5 minutes which meant 50 time points. The sample size n is 50. Fig. 2 shows the causal structure of the simulated FMRI data. By this figure, matrix

Hence, we would conduct experiments on real data to evaluate our approach in future work.

T5 T9

T6

References [1] J. Han and M. Kamber. Data mining: concepts and techniques. Morgan Kaufmann, 2006.

T7

T8

[2] K. A. Bollen. Structural Equations with Latent Variables. John Wiley & Sons, 1989.

Fig. 2. Causal structure of simulated FMRI data． Table 3. Root mean square errors in the simulated FMRI experiment. method Longitudinal LiNGAM algorithm AR-LiNGAM algorithm (Only one sample used)

RMSE 0.549 0.663

B(t, t) is as follows.    Btrue (t, t) =   

 0 0 0 0 0 ∗ 0 0 0 0   0 ∗ 0 0 0  . 0 0 ∗ 0 0  ∗ 0 0 ∗ 0

(8)

In the matrix above, we can not give exact values at ∗ due to the non-linearity. In [12], lagged effects exist but do not clearly specified. We didn’t apply AR-model to simulated FMRI data since AR-model captures not instantaneous effects but lagged effects. We applied to the data set longitudinal LiNGAM algorithm and AR-LiNGAM algorithm, and compared estimation accuracy of these two methods. We evaluated whether the two methods correctly estimated the presence and absence of causal effects between variables, that is, directed edges. In other words, we first replaced non-zero components of estimated matrices and Btrue (t, t) to 1. Then we evaluated estimation accuracy by RMSE. Results in this experiment are shown in Table 3. Our longitudinal LiNGAM algorithm gave a smaller RMSE and was able to estimate causal structures closer to the true structure in FMRI data. 6. CONCLUSIONS We proposed a new model and its estimation algorithm for deriving causal structures that may change over a period of time. We demonstrated superior performance of our method by conducting experiments on artificial data and simulated FMRI data. In FMRI studies, many cases seem to have characteristics that their causal structures change over a period of time. We expect that our proposed method is useful for accurately analyzing causal structures in real longitudinal data.

[3] J. Pearl. Causality: Models, Reasoning, and Inference. Cambridge University Press, 2000. [4] P. Spirtes, C. Glymour, and R. Scheines. Causation, Prediction, and Search. Springer Verlag, 1993. (2nd ed. MIT Press 2000). [5] S. Shimizu, P. O. Hoyer, A. Hyv¨arinen, and A. Kerminen. A linear non-Gaussian acyclic model for causal discovery. Journal of Machine Learning Research, Vol. 7, pp. 2003–2030, 2006. [6] A. Hyv¨arinen, K. Zhang, S. Shimizu, and P.O. Hoyer. Estimation of a structural vector autoregression model using non-gaussianity. Journal of Machine Learning Research, Vol. 11, pp. 1709–1731, 2010. [7] S. Shimizu, T. Inazumi, Y. Sogawa, A. Hyvarinen, Y. Kawahara, T. Washio, P.O. Hoyer, and K. Bollen. Directlingam: A direct method for learning a linear nongaussian structural equation model. Journal of Machine Learning Research, Vol. 11, pp. 1225–1248, 2011. [8] N.R. Swanson and C.W.J. Granger. Impulse response functions based on a causal approach to residual orthogonalization in vector autoregressions. Journal of the American Statistical Association, Vol. 92, No. 437, pp. 357–367, 1997. [9] A. Hyv¨arinen, J. Karhunen, and E. Oja. Independent component analysis. Wiley, New York, 2001. [10] F. R. Bach and M. I. Jordan. Kernel independent component analysis. Journal of Machine Learning Research, Vol. 3, pp. 1–48, 2002. [11] Hui Zou. The adaptive lasso and its oracle properties. Journal of the American statistical association, Vol. 101, No. 476, pp. 1418–1429, 2006. [12] S.M. Smith, K.L. Miller, G. Salimi-Khorshidi, M. Webster, C.F. Beckmann, T.E. Nichols, J.D. Ramsey, and M.W. Woolrich. Network modelling methods for fmri. Neuroimage, Vol. 54, No. 2, pp. 875–891, 2011. [13] K.J. Friston, L. Harrison, and W. Penny. Dynamic causal modelling. Neuroimage, Vol. 19, No. 4, pp. 1273–1302, 2003.