The Bellman Equation and The Expected Value of The Game Question

You play a game where you throw a fair four-sided die numbered 1 to 4, starting with a total of 0 points. On each turn of the game, you can choose to stop or roll. If you stop, you end the game with your current point total. If you roll, and you roll a 4,your point total is set to 0 and the game ends. Otherwise, if you roll a 1, 2, or 3, the number you roll is added to your point total, and the game continues to the next turn. Additionally, if your point total is 9 or greater, you must then stop the game with your current number of points; there is no choice to continue rolling. You wish to maximize the expected value of the number of points that you end with.

(a) Write a Bellman equation for this problem. Include the boundary conditions.

(b) Solve the Bellman equation from the previous part to determine the expected value of the game under optimal play (in terms of the number of points). Give the answer as a decimal rounded to one decimal place.

(c) Determine all optimal strategies for this game.