Probability – Markov chains
Markov Chains First assignment
Solutions should be handed in in one pdf document, well-written and readable. Elucidate the line of thought by plain English – so not only formulas!
Exercise 1 – Olinick 7
A particle moves along a line from an initial position 2 feet to the right of the origin. Each minute it moves one foot to the right with probability 1/2 or 1 foot to
the left. There are barriers at the origin and 4 feet to the right of the origin; if the particle reaches the barrier at the origin, it must move one foot to the right
in the next minute, while if it hits the other barrier, it must move one foot to the left in the next minute. Determine the transition matrix for the associated Markov
chain and draw the state diagram.
Exercise 2
Assume that a persons work can be classified as professional, skilled labor, or unskilled labor. Assume that of the children of professionals, 80% are professional,
10% are skilled laborers, and 10% are unskilled laborers. In the case of children of skilled laborers, 60% are skilled laborers, 20% are professionals, and 20% are
unskilled laborers. Finally, in the case of unskilled laborers, 50% of the children are unskilled laborers, and 25% each are in the other two categories. Assume that
every person has a child, and form a Markov chain by following a given family through several generations. In commenting on the society described, the famed
sociologist Harry Perlstadt has written, ”No matter what the initial distribution of the labor force is, in the long run the majority of the workers will be
professionals”. Is he correct? Why?
Exercise 3 – drunken walk
In a busy street, there are three bars next to each other. Bar 1 one is next to bar 2, which is in the middle, so bar 3 is on the other side of bar 2. A man goes for a
drink, and after each drink he randomly chooses with equal probabilities to either stay and have another drink at the same bar, or go outside and chooses a
neighbouring bar to have a drink.
1. What is the long-term probability that he is in each of the bars?
2. Supposethepriceofadrinkinbar1is 1,inbar2itis2,andinbar3youcan get a drink for 3. Suppose the man starts in bar 1. What is the expected amount of money spent,
if the man decides to have 4 drinks?
The city council has decided to fight the abuse of drinking. So they built a jail next to bar 1, and a detox clinic next to bar 3. To be precise, the street now
contains a row of five buildings: a jail, bar 1, bar 2, bar 3, and the detox clinic. Furthermore, they decided that after one drink in a bar, each visitor must get
out, and there should be no more cheap bars: all prices become equal to 3. One night the same man enters one of the bars, and after having a drink, he randomly selects
one of the neighbouring buildings to enter. If that is the jail or the detox clinic, he has to stay there.
3. Suppose the man has again money for four drinks. What is the probability that he ends in jail?
4. Suppose he starts in bar 1, and he has endless money. What is the probability that he ends in jail?
5. Suppose he has endless money. How many drinks will he have had before he ends up in jail or detox? Does it matter in which bar he started?
