Counterexamples In Markov Decision Processes

Markov Decision Processes (MDPs) form a cornerstone of applied probability, with over 50 years of rich research history. Throughout this time, numerous foundational books and thousands of journal articles have shaped the field. The central objective of MDP theory is to identify the optimal control strategy for Markov random processes with discrete time. Interestingly, the best control strategies often display unexpected or counterintuitive behaviors, as documented by a wide array of studies.This