Finite State Continuous Time Markov Decision Processes with a Finite Planning Horizon.

Becky Miller

Expert InsightsPublished 1967

The system considered may be in one of [n] states at any point in time; its probability law is a Markov process that depends on the policy (control) chosen. The return to the system over a given planning horizon is the integral over that horizon of a return rate that depends on both the policy and the sample path of the process. A necessary and sufficient condition is obtained for a policy that maximizes the expected return over the given planning horizon. A constructive proof is given that there is a piecewise constant policy that is optimal. A bound on the number of switches (points where the piecewise constant policy jumps) is obtained for the case in which there are two possible states and they communicate for one element. 26 pp. Ref.

Document Details

  • Copyright: RAND Corporation
  • Availability: Web Only
  • Year: 1967
  • Pages: 26
  • Document Number: P-3569

Citation

Chicago Manual of Style

Miller, Becky, Finite State Continuous Time Markov Decision Processes with a Finite Planning Horizon. Santa Monica, CA: RAND Corporation, 1967. https://www.rand.org/pubs/papers/P3569.html.
BibTeX RIS

This publication is part of the RAND paper series. The paper series was a product of RAND from 1948 to 2003 that captured speeches, memorials, and derivative research, usually prepared on authors' own time and meant to be the scholarly or scientific contribution of individual authors to their professional fields. Papers were less formal than reports and did not require rigorous peer review.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.