Disclaimer: Dieser Thread wurde aus dem alten Forum importiert. Daher werden eventuell nicht alle Formatierungen richtig angezeigt. Der ursprüngliche Thread beginnt im zweiten Post dieses Threads.
Question regarding variable order in Bayesian Networks
Hello,
unfortunately I can’t attend the Q/A-Lecture this week so I hope someone here can help me.
My questions is about slide 643 and 644 in the lecture notes notes-2018-19.pdf on page 407 of 627. There are two Bayesian Networks with a different variable order.
There is something that I really don’t understand: We follow the algorithm from slide 642 for constructing a BN, so when inserting the node Earthquake on slide 643, we ask ourselves: “Is Mary Calls independent from Earthquake?”. If I’m not wrong, we say “Yes it is independent, so we don’t add this edge”.
Now on to slide 643, we now have a different variable ordering. But also in this case, when we insert Earthquake, we check dependencies with all previously inserted nodes. So again, we ask ourselves “Is MaryCalls independent from Earthquake?”. This time however, we say “No, it is not independent, so we insert the edge MaryCalls → Earthquake”.
Why have we decided differently in these two situations? I don’t understand why this is because of the variable ordering, since in both situations we ask ourselves the same question. On slide 642, we don’t add the edge, on slide 643 however, we insert the edge.
I hope I could make my question clear and I hope someone can explain this.
Have a nice day!
Correction. We choose a minimal set A of previously inserted Nodes such that „Earthquake“ is conditionally independent given A of the remaining nodes.
In the case of slide 643, Earthquake is conditionally independent of „Mary Calls“ given „Alarm“ and „Burglary“. In other words: If we already know whether there was a Burglary and whether (or not) the Alarm went off, then whether or not Mary calls (or not) does not give us any new information.
Formally: P(Earthquake | Alarm, Burglary) = P(Earthquake | MaryCalls, Alarm, Burglary)
In the second example we ask the same question, but when we insert „Earthquake“, the only variables already inserted are „John Calls“ and „Mary Calls“. Both of these give us new information on the question of whether or not there was an earthquake. If only Mary calls, then the probability of there being an earthquake is slightly lower than if both Mary and John call, so „Earthquake“ is not conditionally independent of either given the other.
Formally: P(Earthquake | MaryCalls) =/= P(Earthquake | JohnCalls, MaryCalls) =/= P(Earthquake | JohnCalls)
In other words: I think you’re confusing “(statistical/stochastic) independence” with “conditional indepenence given X”.
The two (maybe surprisingly) do not imply each other at all: In general, if A and B are (stochastically) independent, then A and B are not necessarily also conditionally independent given some C, and if A and B are conditionally independent given C, then they are not necessarily also stochastically independent.
MaryCalls and Earthquake are quite clearly not independent, but they are conditionally independent given Alarm and Burglary.
For the other direction, imagine you have a stock portfolio containing both (e.g.) Microsoft stock and (e.g.) BMW stock. It’s probably safe to say that they’re pretty independent of each other, i.e. what happens to the Microsoft stock tells us nothing about BMW and vice versa. But given that you have a portfolio that contains both, Microsoft and BMW are not conditionally independent given the Portfolio: For example, if the portfolio increases in value, then knowing that the Microsoft stock has not changed in value increases the probability that BMW stock has gone up.
Thank you very much for the thorough explanation, this answered all my questions I had so far. I somehow got confused and forgot that we have to check for conditional independence and not ‘general’ independence.
That was the example that I first found when I looked this up years ago and it stuck with me 
The most general example would probably: Let A,B be independent and C=A XOR B, then clearly A and B are not conditionally independent given C. But that’s not very illuminating 