Markov chains - with one-step trainsition matrices

jjgoth · **Joined:** Sun, 6 May 2012 01:08:03 UTC **Posts:** 5

Hi all hoping you can help me with a couple of questions:

Firstly I have a two-state transition probability matrix:
$\bf{ T} = \bordermatrix{ & 0 & 1 \cr 0 & (1-a) & a \cr 1 & b & (1-b) \cr}$
where 0<a,b<1

. I need to prove that
$P(X_{n} = 0 | X_{n-1} =0,X_{n+1} =0) = \frac{(1-a)^{2}}{ (1-a)^{2} + ab}$

I know that to go from $0 \rightarrow 0 \rightarrow 0$ would be $(1-a)^{2}$ and the only other option is $0 \rightarrow 1 \rightarrow 0$ which is

so is it simply that this probability is just $\frac{P(0 \rightarrow 0 \rightarrow 0)} {P(0 \rightarrow 0 \rightarrow 0) + P(0 \rightarrow 1 \rightarrow 0)} = \frac{(1-a)^{2}}{ (1-a)^{2} + ab}$ . This seems far too straightforward!

Secondly for another two-state transition probability matrix:
$\bf{ T} = \bordermatrix{ & 0 & 1 \cr 0 & \frac{2}{3} & \frac{1}{3} \cr 1 & \frac{1}{4} & \frac{1}{4} \cr}$
Given the chain is in state

at time

, what is the probability it was in state

at time

?

I wasn't sure at all about this either, the only thing I could think of was calculating $\frac{\frac{2}{3}}{\frac{2}{3}+\frac{1}{4}}=\frac{8}{11}$ which sounds reasonable but could be a complete fallacy :lol:

Could anyone please give me a thumbs up (or down) on this too?

Many thanks!

Shadow · **Posted:** Sun, 6 May 2012 02:13:02 UTC

jjgoth wrote:

Hi all hoping you can help me with a couple of questions:

Firstly I have a two-state transition probability matrix:
$\bf{ T} = \bordermatrix{ & 0 & 1 \cr 0 & (1-a) & a \cr 1 & b & (1-b) \cr}$
where 0<a,b<1

. I need to prove that
$P(X_{n} = 0 | X_{n-1} =0,X_{n+1} =0) = \frac{(1-a)^{2}}{ (1-a)^{2} + ab}$

I know that to go from $0 \rightarrow 0 \rightarrow 0$ would be $(1-a)^{2}$ and the only other option is $0 \rightarrow 1 \rightarrow 0$ which is

so is it simply that this probability is just $\frac{P(0 \rightarrow 0 \rightarrow 0)} {P(0 \rightarrow 0 \rightarrow 0) + P(0 \rightarrow 1 \rightarrow 0)} = \frac{(1-a)^{2}}{ (1-a)^{2} + ab}$ . This seems far too straightforward!

Secondly for another two-state transition probability matrix:
$\bf{ T} = \bordermatrix{ & 0 & 1 \cr 0 & \frac{2}{3} & \frac{1}{3} \cr 1 & \frac{1}{4} & \frac{1}{4} \cr}$
Given the chain is in state

at time

, what is the probability it was in state

at time

?

I wasn't sure at all about this either, the only thing I could think of was calculating $\frac{\frac{2}{3}}{\frac{2}{3}+\frac{1}{4}}=\frac{8}{11}$ which sounds reasonable but could be a complete fallacy :lol:

Could anyone please give me a thumbs up (or down) on this too?

Many thanks!

That seems pretty reasonable to me, why do you think it isn't what you should do?

jjgoth · **Joined:** Sun, 6 May 2012 01:08:03 UTC **Posts:** 5

It's just I haven't answered questions of this form so just used intuition so could easily be wrong! That's cool though if it's right, thanks

Shadow · **Posted:** Sun, 6 May 2012 02:34:55 UTC

outermeasure · **Posted:** Mon, 7 May 2012 10:23:38 UTC

Shadow wrote:

jjgoth wrote:

Hi all hoping you can help me with a couple of questions:

Firstly I have a two-state transition probability matrix:
$\bf{ T} = \bordermatrix{ & 0 & 1 \cr 0 & (1-a) & a \cr 1 & b & (1-b) \cr}$
where 0<a,b<1

. I need to prove that
$P(X_{n} = 0 | X_{n-1} =0,X_{n+1} =0) = \frac{(1-a)^{2}}{ (1-a)^{2} + ab}$

I know that to go from $0 \rightarrow 0 \rightarrow 0$ would be $(1-a)^{2}$ and the only other option is $0 \rightarrow 1 \rightarrow 0$ which is

so is it simply that this probability is just $\frac{P(0 \rightarrow 0 \rightarrow 0)} {P(0 \rightarrow 0 \rightarrow 0) + P(0 \rightarrow 1 \rightarrow 0)} = \frac{(1-a)^{2}}{ (1-a)^{2} + ab}$ . This seems far too straightforward!

Secondly for another two-state transition probability matrix:
$\bf{ T} = \bordermatrix{ & 0 & 1 \cr 0 & \frac{2}{3} & \frac{1}{3} \cr 1 & \frac{1}{4} & \frac{1}{4} \cr}$
Given the chain is in state

at time

, what is the probability it was in state

at time

?

I wasn't sure at all about this either, the only thing I could think of was calculating $\frac{\frac{2}{3}}{\frac{2}{3}+\frac{1}{4}}=\frac{8}{11}$ which sounds reasonable but could be a complete fallacy :lol:

Could anyone please give me a thumbs up (or down) on this too?

Many thanks!

That seems pretty reasonable to me, why do you think it isn't what you should do?

The second one is wrong --- you don't even have a stochastic matrix, and you should use Bayes' theorem.

Shadow · **Posted:** Mon, 7 May 2012 13:27:32 UTC

outermeasure wrote:

Shadow wrote:

jjgoth wrote:

Hi all hoping you can help me with a couple of questions:

Firstly I have a two-state transition probability matrix:
$\bf{ T} = \bordermatrix{ & 0 & 1 \cr 0 & (1-a) & a \cr 1 & b & (1-b) \cr}$
where 0<a,b<1

. I need to prove that
$P(X_{n} = 0 | X_{n-1} =0,X_{n+1} =0) = \frac{(1-a)^{2}}{ (1-a)^{2} + ab}$

I know that to go from $0 \rightarrow 0 \rightarrow 0$ would be $(1-a)^{2}$ and the only other option is $0 \rightarrow 1 \rightarrow 0$ which is

so is it simply that this probability is just $\frac{P(0 \rightarrow 0 \rightarrow 0)} {P(0 \rightarrow 0 \rightarrow 0) + P(0 \rightarrow 1 \rightarrow 0)} = \frac{(1-a)^{2}}{ (1-a)^{2} + ab}$ . This seems far too straightforward!

Secondly for another two-state transition probability matrix:
$\bf{ T} = \bordermatrix{ & 0 & 1 \cr 0 & \frac{2}{3} & \frac{1}{3} \cr 1 & \frac{1}{4} & \frac{1}{4} \cr}$
Given the chain is in state

at time

, what is the probability it was in state

at time

?

I wasn't sure at all about this either, the only thing I could think of was calculating $\frac{\frac{2}{3}}{\frac{2}{3}+\frac{1}{4}}=\frac{8}{11}$ which sounds reasonable but could be a complete fallacy :lol:

Could anyone please give me a thumbs up (or down) on this too?

Many thanks!

That seems pretty reasonable to me, why do you think it isn't what you should do?

The second one is wrong --- you don't even have a stochastic matrix, and you should use Bayes' theorem.

Right, didn't check on the second one, should have said that for the op's benefit.

jjgoth · **Joined:** Sun, 6 May 2012 01:08:03 UTC **Posts:** 5

My bad, lack of sleep is getting to me...

$\bf{ T} = \bordermatrix{ & 0 & 1 \cr 0 & \frac{2}{3} & \frac{1}{3} \cr 1 & \frac{1}{4} & \frac{3}{4} \cr}$
Given the chain is in state

at time

, what is the probability it was in state

at time

?

So using Bayes' theorem I get:
$P(n-1=0|n=0) = \dfrac{P(n=0|n-1=0) \times P (n-1=0)}{P(n=0)}= \dfrac{\frac{2}{3} \times \frac{\frac{2}{3}}{\frac{2}{3} + \frac{3}{4}}}{1} = \frac{16}{51} \approx 0.3137$

Is that better?

outermeasure · **Posted:** Tue, 8 May 2012 06:49:32 UTC

jjgoth wrote:

My bad, lack of sleep is getting to me...

$\bf{ T} = \bordermatrix{ & 0 & 1 \cr 0 & \frac{2}{3} & \frac{1}{3} \cr 1 & \frac{1}{4} & \frac{3}{4} \cr}$
Given the chain is in state

at time

, what is the probability it was in state

at time

?

So using Bayes' theorem I get:
$P(n-1=0|n=0) = \dfrac{P(n=0|n-1=0) \times P (n-1=0)}{P(n=0)}= \dfrac{\frac{2}{3} \times \frac{\frac{2}{3}}{\frac{2}{3} + \frac{3}{4}}}{1} = \frac{16}{51} \approx 0.3137$

Is that better?

No.

Unless you assume your starting distribution $\mathbb{P}(X_{n-1}=0)=$ ..., you won't get $\mathbb{P}(X_{n-1}=0\mid X_n=0)=\frac{16}{51}$ . However, there is no reason why you would want, a priori, $X_{n-1}$ to have that distribution (and another problem is whether that is reachable from a distribution of X_0

, for all n). On the other hand, if you start with the invariant distribution, and solve for your reverse time Markov chain ...

S.O.S. Mathematics CyberBoard

Markov chains - with one-step trainsition matrices

Who is online