INTRODUCTION TO STOCHASTIC PROCESSES, AN

An Introduction to Stochastic Processes provides a clear and rigorous introduction to the theory and applications of stochastic processes. The book begins with an introductory chapter that reviews essential probability tools, including computing by conditioning and the law of large numbers, while introducing classical processes such as random walks, gambler's ruin, and branching processes. Key concepts like renewal processes and stopping times are also presented, providing a foundation for the rest of the text.

Markov chains and continuous-time Markov processes are treated on both finite and countable state spaces, with elementary proofs of central results such as limiting distributions and ergodic theorems. Differences between finite and countable settings are highlighted to enhance understanding, and martingales are introduced as a powerful framework for analyzing stochastic processes. The presentation remains accessible to students with a background in basic probability and linear algebra, without requiring measure theory.

Poisson processes are developed beyond the traditional one-dimensional case to include multi-dimensional processes, expanding applications while maintaining clarity. The book also provides a simple algorithm for generating non-homogeneous Poisson processes in any dimension. Packed with examples and exercises of varying difficulty, this text bridges theory and practice, making it an essential resource for students, instructors, and anyone seeking a solid foundation in stochastic processes.

Contents:

• About the Author
• Acknowledgments
• Introduction to Stochastic Processes:
  • What Is a Stochastic Process?
  • Computing by Conditioning
  • Renewal Processes: Applications of the LLN and CLT
  • Stopping Times and Wald's Identities
  • Limits of Expected Values
  • Exercises
• Markov Chains:
  • The Markov Property
  • Constructing a Markov Chain
  • Multi-Step Transition Probabilities
  • Hitting Probabilities
  • Expected Hitting Times
  • Stationary Distributions
  • Stationary Distributions: Special Cases
  • Limiting Distributions
  • Ergodic Limit Theorems
  • Irreducibility and Decomposition of theState Space
  • Periodicity
  • Recurrence, Transience, and the Strong Markov Property
  • Proof of Limit Theorems I: Ergodic Limits
  • Proof of Limit Theorems II: Limiting Distributions
  • Concluding Remarks
  • Exercises
• Poisson Processes:
  • The Poisson Process as a Renewal Process
  • Properties of the One-Dimensional Homogeneous Poisson Process
  • Non-Homogeneous and Multi-Dimensional PoissonPoint Processes
  • Transformations of Poisson Processes
  • Construction of Poisson Processes
  • Exercises
• Continuous Time Markov Processes:
  • The Continuous-Time Markov Property
  • Ingredients of a CTMP
  • Computing Transition Probabilities and the Kolmogorov Differential Equations
  • The Generator Matrix and Jump Rates
  • Rigorous Derivation of the Kolmogorov Equations
  • Stationary Distributions
  • Limit Theorems
  • Proofs of the Limit Theorems for CTMP
  • Hitting Probabilities and Hitting Times
  • Exercises
• Martingales:
  • The Martingale Property
  • Submartingales and Supermartingales
  • The Optional Stopping Theorem
  • The Martingale Convergence Theorem
  • Exercises
• Review of Basic Probability:
  • Jensen's s Inequality
  • Notions of Convergence in Probability Theory
  • Exercises
• Limit Theorems for Expectations:
  • Exercises
• Bibliography
• Index

Readership: This book is intended as a textbook for advanced undergraduate and beginning graduate courses in stochastic processes. It may also interest engineers, computer scientists, and readers in related fields seeking an approach to stochastic processes that combines theory and practical applications.