[ad_1]

If your worked in a data team maybe you heard about Markov Chains, for instance:

- When they apoke about a bayesian process, maybe they mentioned the
**Markov Chain Montecarlo (MCMC)**method. - Maybe your heard about a kind of model called
**Hidden Markov Chain**. - Finally, maybe they mentioned that some time series have M
**arkov Properties**.

These are a few examples, the point is that Markov Chains are used for a lot of different purposes, but the question is, what are Markov Chains about?

**During this article we will give an explanation about what Markov Chains are, some properties, and give some examples. The idea of this article is to be the base for other more advanced articles which will come, showing real application you can do using this. There are a lot of powerful applications, believe me.**

Suppose that you want to predict the weather for the ninth day as in the picture below.

The possible options are sunny, cloudy and rain. In theory, you could build a model which considers the weather of all the past days. In this case:

The weather day 8+ The weather day 7 + … + The weather day 1

I think you got the idea.

Now, the question is, are all these days equally meaningful for your prediction? Well, it’s likely that this won’t be the case. Indeed,** it could be that the last day will be the most important for our prediction**. In our case to predict the weather for day 9, day 8 would be the most important.

*This is something you could make out using PACF (Partial autocorrelation function), used often in timeseries analysis.*

Let’s assume that day 8 is much more important than any other day to predict the next day, **could we not just use that last day to make the prediction and make our model simpler**? Well, we can, and if we take only the current day to predict the next, we are seeing a Markov Chain.

In the context of Markov Chains, we speak about states. Our current weather would be our current state, the next state would be the weather of tomorrow.

**One of the most important properties of a Markov Chain is that the next state only depends on the current one**. In other words, that the weather of tomorrow only depends on the weather of today. If we can make that assumption, we can treat this case as a Markov Chain.

In this case the weather of Day 2 depends only on Day 1, the weather of Day 3 depends only on Day 2, and so on…

But, given that we said that we are working with a Markov Chain, there is an efficient way to graph the transitions from one weather to another.

## Markov Chain graph

At the graph below we can see the probabilities for the weather at the next day given the current weather (The probabilities are random selected by me for the example).

How to read this? Well, let’s assume that today it’s sunny, which are the probabilities for the weather tomorrow?

Let’s highlight the related lines at the graph. There are 3 lines which start at the sunny symbol, these represents the probability for the weather of tomorrow given that today it’s sunny. We have:

- 30% probability that it will be sunny again.
- 30% probability that it will rain.
- 40% probability that tomorrow it will be cloudy.

Also, **notice that the sum of the probabilities of all the lines which come out from one state sum together 100%**, this is a property.

We could write this down in probability terms following the same example, that the current weather is sunny (*If you didn’t studied probabilities don’t worry, it’s another way to express what I said above*):

- P(S/S): 0.3
- P(R/S): 0.3
- P(C/S): 0.4

So, now we know how to read that graph we can make out many things from it, for instance: Given that today it’s raining, what it the probability that tomorrow it will becloudy? As we can see the probability is relatively high, it’s 60%.

Seems like we are pretty done, right? Given the current weather we can make out which will be the weather tomorrow. Well, no.

Somebody could come to use and ask: “If I don’t know the current weather, because I live far away, how could I know the probability that it will be raining?”

Basically, this person is asking us, **given any day, which is the probability for each weather**? In terms of probabilities they are asking us:

Let’s do some maths to get that.

A transition is the change from one state to the next in the context of Markov Chains.

To calculate this, we will need a transition matrix, this is **a matrix which have the values for all the possible transitions **, from any weather to any weather in our case.

## Transition Matrix

At this matrix I have the exact same values as in the graph above, to read it start selecting a row, which represents the current weather. We would say, **given that today it’s raining (Second row) the probability that tomorrow it will be sunny is 30%, that it will be again cloudy is 40% and that it will rain is 30%**. Take your time to make sure you understand it until now.

Now, we will jump to python to make some calculations. You will need for this some basic understanding of matrix operations. If not don’t worry, I will try to make it clear for you.

What we will do is select randomly the weather for the first day, let’s say it’s cloudy. As with the transition matrix, the second column and row correspond to the cloudy weather, so we will use the following one row matrix to indicate this weather.

**By calculating the product of the first day matrix and the transition matrix we will get the probabilities for each weather for the next day.** Let’s do this with python to verify it.

*If you don’t know about matrices operation don’t worry. What I’m about to do below is the dot product between two matrices, one of many ways to multiply two matrices.*

## Get the general probabilities

[ad_2]

Source link