The objective of this book is to describe the fundamental theory of birth-death processes and Markov chains and to present the developments in this field in recent years. The so-called Markov chain here refers to a Markov pro-cess which has continuous time parameters, countably many states and is time-homogeneous. Chains of this kind are important not only because its comparatively complete and unified theory can be used for reference in general Markov chains and other stochastic processes, but als0 because there is a steady increase in the applications to natural sciences and practical prob-lems such as physics, biology, chemistry, programming theory and queueing theory. For these the reader is referred to the works by K. L. Chung, Hou Zhen-ting, Guo Qing-feng, Bharucha-Reid, quoted at the end of this book.

Preface Chapter I General Concepts of Stochastic Processes 1.1 Definition of Stochastic Processes 1.2 Separability of Stochastic Processes 1.3 Measurability of Stochastic Processes 1.4 Conditional Probabilities and Conditional Mathematical Expectations 1.5 Markov Property 1.6 Transition Probabilities Chapter II Analytic Theory of Markov Chains 2.1 General Properties of Measurable Transition Matrices 2.2 Differentiability of Standard Transition Matrices 2.3 Backward and Forward Differential Equations 2.4 Standard Generalized Transition Matrices 2.5 Resolvent Matrices 2.6 The Minimal Q-Resolvent Matrices 2.7 Properties of the Minimal Q-Resolvent Matrix 2.8 Exit Families and Entrance Families 2.9 General Forms of Q-Resolvent Matrices 2.10 Construction of Q-Resolvent Matrices in Simple Cases Chapter III Properties of Sample Functions 3.1 Sets of Constancy and Intervals of Constancy 3.2 Right Lower Semi-Continuity; Canonical Chains 3.3 Strong Markov Property Chapter IV Some Topics in Markov Chains 4.1 Zero-One Laws 4.2 Recurrence and Excessive Functions 4.3 Distributions of Stochastic Functionals of Integral Type 4.4 Embedding Problem Chapter V Basic Theory of Birth and Death Processes 5.1 Probabilistic Meanings of Numerical Characteristics 5.2 Random Functionals of Upward Integral Type 5.3 First Entrance Time and Sojourn Time 5.4 Random Functionals of Downward Integral Type 5.5 Solutions of Several Types of Kolmogorov Equations and Stationary Distributions 5.6 Some Applications of Birth and Death Processes Chapter VI Construction Theory of Birth and Death Processes 6.1 Transformations of Doob Processes 6.2 The Necessary and Sufficient Condition that a Process Cannot Enter Continuously 6.3 Transformation of General Q- Processes into Doob Processes 6.4 Construction of Q-Processes for ? 6.5 Characteristic Sequences and Classification of Birth and Death Processes 6.6 Fundamental Theorem 6.7 Another Construction of Q-Processes in Case ? 6.8 Ergodic Property and Zero-One Laws Chapter VII Analytical Construction of Birth and Death Processes 7.1 Natural Scale and Canonical Measure 7.2 Second Order Difference Operators 7.3 The Solution to the Equation ? 7.4 Construction of the Minimal Solution 7.5 Some Lemmas 7.6 Construction of the Q-Resolvent Matrix of B-Type 7.7 Construction of the Q-Resolvent Matrix of F-Type 7.8 Construction of Q-Resolvent Matrices Neither of B-Type nor of F-Type: In the Case of Linear Dependence 7.9 Construction of Q-Resolvent Matrices Neither of B-Type nor of F-Type: In the Case of Linear Independence 7.10 The Conditions for? ! 7.11 Birth and Death Processes of Probability 7.12 Relation Between Probabilistic and Analytic Constructions 7.13 Properties of a Process on Its First Point of Flying Leap 7.14 Invariant Measure Chapter VIII Bilateral Birth and Death Processes 8.1 Numerical Characteristics and Classification of Boundary Points 8.2 Solutions to the Equation ? 8.3 The Minimal Solution 8.4 Representations for Exit and Entrance Families 8.5 r1 is Entrance or Natural and r2 is Regular or Exit 8.6 Both r1 and r2 are Regular or Exit 8.7 Conditions for? 8.8 Properties of Boundaries 8.9 Recurrence and Ergodicity Appendix I Excessive Functions of Markov Chains with Discrete Time 0.1 Potential and Excessive Functions 0.2 Limit Theorems on Excessive Functions Appendix II A-Systems and the Z-System Method Annotations on the History of the Contents of Each Section Bibliography Index