Conditional Independence

Given mutually exclusive sets of random variables \(\mathbf{X},\mathbf{Y},\mathbf{Z}\) we say \(\mathbf{X}\) is conditionally independent of \(\mathbf{Y}\) given \(\mathbf{Z}\), denoted \(\mathbf{X} -||- \mathbf{Y}|\mathbf{Z}\) or \(\mathbb{I}_\mathbb{P}(X,Y|Z)\) if \(\mathbb{P}(\mathbf{X},\mathbf{Y}|\mathbf{Z}) = \mathbb{P}(\mathbf{X}|\mathbf{Z})\mathbb{P}(\mathbf{Y}|\mathbf{Z})\).

Conditional independence (and dependence) can be graphically represented in Bayesian Networks (BNs). Let \(X,Y,Z\) be random variables, we then have:

• The chains \(X \leftarrow Z \leftarrow Y\), \(X\rightarrow Z \rightarrow Y\) represent \(\mathbb{I}_\mathbb{P}(X,Y|Z)\). To see this, without loss of generality take the chain \(X \rightarrow Z \rightarrow Y\), then \(\mathbb{P}(X,Y|Z)=\frac{\mathbb{P}(X,Y,Z)}{\mathbb{P}(Z)}=\frac{\mathbb{P}(Y|Z)\mathbb{P}(X|Z)\mathbb{P}(Z)}{\mathbb{P}(Z)} =\mathbb{P}(X|Z)\mathbb{P}(Y|Z)\). Here, we use Bayes theorem in the first equality and the Markov property in the second one.
• The fork/common cause \(X \leftarrow Z \rightarrow Y\) represents \(\mathbb{I}(X,Y|Z)\). The reasoning is exactly similar to above.
• The v-structure/immorality \(X \rightarrow Z \leftarrow Y\) represents the conditional dependence relation \(\not \mathbb{I}_\mathbb{P}(X,Y|Z)\). To see this, \(\mathbb{P}(X,Y|Z)=\frac{\mathbb{P}(X,Y,Z)}{\mathbb{P}(Z)}=\frac{\mathbb{P}(Y)\mathbb{P}(X)\mathbb{P}(Z|X,Y)}{\mathbb{P}(Z)} \neq \mathbb{P}(X|Z)\mathbb{P}(Y|Z)\).