Exercise 2.9

Demonstration that the soft-max distribution is equal to the logistic and sigmoid distribution for the case of two actions.

First we show that the soft-max is a logistic function.

Secondly, a sigmoid function refers to the special case of the logistic function. From the following relation

it is clear that the logistic function is also a sigmoid function.

Exercise 2.10

case
A 0.1 0.2
B 0.9 0.8

If you are not able to tell which case you face at any step, what is the best expectation of success you can achieve and how should you behave to achieve it.

Allways choose action

a1

a2

Either choosing or leads to the same expected value.