Added to Favorites

Related Searches

Definitions

A Markov process, named after the Russian mathematician Andrey Markov, is a mathematical model for the random evolution of a memoryless system. Often the property of being 'memoryless', the Markov property, is expressed such that conditional on the present state of the system, its future and past are independent.## See also

## References

Mathematically, the Markov process is expressed as for any n and $t\_1,\; math>$

- $P[x(t\_n)\; le\; x\_n~|~x(t)~forall~t\; le\; t\_\{n-1\}]\; =\; P[x(t\_n)\; le\; x\_n~|~x(t\_\{n-1\})].,!$

Often, the term Markov chain is used to mean a discrete-time Markov process. Also see continuous-time Markov process.

Mathematically, if X(t), t > 0, is a stochastic process, the Markov property states that

- $mathrm\{Pr\}big[X(t+h)\; =\; y\; ,|,\; X(s)\; =\; x(s),\; forall\; s\; leq\; tbig]\; =\; mathrm\{Pr\}big[X(t+h)\; =\; y\; ,|,\; X(t)\; =\; x(t)big],\; quad\; forall\; h\; >\; 0.$

Markov processes are typically termed (time-) homogeneous if

- $mathrm\{Pr\}big[X(t+h)\; =\; y\; ,|,\; X(t)\; =\; xbig]\; =\; mathrm\{Pr\}big[X(h)\; =\; y\; ,|,\; X(0)\; =\; x(0)big],\; quad\; forall\; t,\; h\; >\; 0,$

In some cases, apparently non-Markovian processes may still have Markovian representations, constructed by expanding the concept of the 'current' and 'future' states. For example, let X be a non-Markovian process. Then define a process Y, such that each state of Y represents a time-interval of states of X, i.e. mathematically,

- $Y(t)\; =\; big\{\; X(s):\; s\; in\; [a(t),\; b(t)]\; ,\; big\}.$

An example of a non-Markovian process with a Markovian representation is a moving average time series.

- Examples of Markov chains
- Memorylessness
- Semi-Markov process
- Markov chain
- Markov decision process
- Dynamics of Markovian particles
- Conditional Probability

Wikipedia, the free encyclopedia © 2001-2006 Wikipedia contributors (Disclaimer)

This article is licensed under the GNU Free Documentation License.

Last updated on Saturday August 30, 2008 at 15:44:51 PDT (GMT -0700)

View this article at Wikipedia.org - Edit this article at Wikipedia.org - Donate to the Wikimedia Foundation

This article is licensed under the GNU Free Documentation License.

Last updated on Saturday August 30, 2008 at 15:44:51 PDT (GMT -0700)

View this article at Wikipedia.org - Edit this article at Wikipedia.org - Donate to the Wikimedia Foundation

Copyright © 2015 Dictionary.com, LLC. All rights reserved.