Singapore University of Social Sciences

Stochastic Processes I

Stochastic Processes I (MTH361)


MTH361 Stochastic Processes I introduces the concept of modelling dependence and will focus on discrete-time Markov chains. The purpose of the course is to develop and analyse probability models that capture the salient features under study to predict the short and long term effects that this randomness will have on the systems under consideration. Additionally, computer simulation with Python will also be taught.

Level: 3
Credit Units: 5
Presentation Pattern: EVERY JAN


  • Stochastic Processes
  • Random Walk
  • Discrete-Time Markov Chains
  • Chapman-Kolmogorov Equations
  • Convergence Theorems
  • Stationary Distributions
  • Classification of States
  • Irreducible Markov Chains
  • Periodic Markov Chain
  • Limiting Probabilities
  • Recurring States
  • Transient States

Learning Outcome

  • Compute probabilities of events of a Markov chain expectation or expectation of random variables.
  • Show the validity of given mathematical statements in stochastic processes.
  • Calculate the stationary distribution in a given Markov chain with finite and irreducible state space.
  • Determine which states of a given Markov chain are recurrent and which are transient.
  • Solve for the asymptotic behaviour of Markov chains.
  • Formulate Markov chain models from word problems.
Back to top
Back to top