Dynamic Probabilistic Systems - Volume II

Semi-Markov and Decision Processes

Ronald A. Howard

576 pages, parution le 01/06/2007

Ajouter à une liste

Indisponible

Résumé

This book is an integrated work published in two volumes. The first volume treats the basic Markov process and its variants; the second, semi-Markov and decision processes. It equips readers to formulate, analyze, and evaluate simple and advanced Markov models of systems, ranging from genetics to space engineering to marketing. 1971 edition.

Sommaire

The Discrete-Time Semi-Markov Process
The Formal Model
The Interval Transition Probabilities
Transform Analysis
An Alternate Formulation-Conditional Transition Probabilities
Flow Graph Analysis
Counting Transitions
Entrance and Destination Probabilities
Transient Processes
Duration
First Passage Times
State Occupancies
The General Discrete-Time Semi-Markov Process
Random Starting
Basic Markovian Equivalents
Conclusion
The Continuous-Time Semi-Markov Process
The Formal Model
The Car Rental Example
The Interval Transition Probabilities
The Exponential Transform
Transform Analysis of the Continuous-Time Semi-Markov Process
Alternate Formulations
Flow Graph Analysis
Counting Transitions
Entrance and Destination Probabilities
Transient Processes
First Passage Times
State Occupancies
The General Continuous-Time Semi-Markov Process
Random Starting
The Continuous-Time Renewal Process
Conclusion
Continuous-Time Markov Processes
Defining Relationships
Interval Transition Probabilities of the Continuous-Time Markov Process
A Continuous-Time Taxicab Problem and Other Examples
Flow Graph Analysis
Interpretation as Competing Exponential Processes
Continuous-Time Birth and Death Processes
Transient Processes and First Passage Times
The Infinite-State Continuous-Time Markov Process
Counting Transitions
Processes with Partial Information: Inference
The Continuous-Time Chapman-Kolmogorov Equations
Conclusion
Rewards
The Reward Structure for Continuous-Time Processes
The Continuous-Time Semi-Markov Reward Process with Discounting
The Continuous-Time Semi-Markov Reward Process without Discounting
The Car Rental Example Again
The Reward Structure for Discrete-Time Processes
The Discrete-Time Semi-Markov Reward Process with Discounting
The Discrete-Time Semi-Markov Reward Process without Discounting
The Discrete-Time Car Rental Example
The Effect of Lapsed Time on Expected Rewards
Rewards in Transient Processes
Concluding Remarks
Dynamic Programming
The Structure of Sequential Decision Processes
The Solution Concepts of Dynamic Programming
A Production Scheduling Example
An Action-Timing Problem
The Dynamic Programming Formalism
Multiplicative Rewards
Conclusion
Semi-Markov Decision Processes
The Decision Structure
Value Iteration
Policy Iteration
Policy Iteration with Discounting
Policy Iteration in Transient Processes
Examples of Infinite Duration Processes
Examples of Transient Processes
Conclusion
Notation
Appendix
Properties of Congruent Matrix Multiplication
References
Index

Voir tout

Replier

Caractéristiques techniques

	PAPIER
Éditeur(s)	Dover
Auteur(s)	Ronald A. Howard
Parution	01/06/2007
Nb. de pages	576
Format	16 x 23
Couverture	Relié
Poids	1640g
Intérieur	Noir et Blanc
EAN13	9780486458724
ISBN13	978-0-4864-5872-4