In this paper we consider the first passage process of a spectrally negative Markov additive process (MAP). The law of this process is uniquely characterized by a certain matrix function, which plays ...

In this paper, it is shown that the Foster-Lyapunov criterion is sufficient to ensure the existence of an invariant probability measure for both discrete- and continuous-time Markov processes without ...

APPM 4560/5560 Markov Processes, Queues, and Monte Carlo Simulations Brief review of conditional probability and expectation followed by a study of Markov chains, both discrete and continuous time.

Markov decision process (MDP): A mathematical framework used to model decision making in situations where outcomes are partly random and partly under the control of a decision-maker.