Markov Properties

1) Transition probabilities are dependent only on the current state of the system i.e. provided that the current state is recognized; the conditional probability of the next state is independent of the states, prior to the current state. (This is known as property of no memory).

2) The transition probabilities are constant over time.

3) The transition probabilities of moving to alternative states in the next time period, given a state in the current time period, must sum to one.

These properties are quite restrictive and hence the application of Markov Analysis is limited to few real-world problems.

Given that a set of possible states in Markov Chain is finite, a square matrix, P, made up of all Pij's of Markov Chain can be formed.

The following table represents the commonly used transition probability matrix.

Where n is the number of exhaustive and mutually exclusive states.

Pij is the transition probability of going from the present i th state to the next j th state.

Thus the rows represent the possible present states (i's) and columns, represent the possible future states (j's).

By definition     P11 +    P12 +    P13 +........ P1n = 1
Similarly,         P21 +    P22 +    P23 +........  P2n = 1 etc.