The system states are described by the states of the components, the failures and their repairs by component state transitions, and the system state transitions by continuous time Markov chain(CTMC).

Integrated Semigroups of Linear Operators and Their Applications to Continuous-Time Markov Chains;

Motivated by the needs of optimization and control of practical engineering systems with uncertain parameters, we considered, through the Markov performance potential theory, the robust control problems for a class of continuous time Markov control processes with uncertain transition rates that are constrained on compact sets.

In this paper,we study the approximation of transition functions in continuous-time Markov chains by means of semigroups of linear operators.

The authors have studied the integral property of the continuous-time Markov chains.

An algorithm for estimating the policy gradient is presented for the performance optimization of continuous-time partially observable Markovian decision processes(CTPOMDPs).