This may sound paradoxical; how can one implement a language in that language if it doesn't yet exist? However there is little mystery here. Interpreters are always "mocked up" in some other existing language and then later converted to the language they interpret. In these cases the early mockups can be used to develop the source code of the interpreter. Once the system is bootstrapped new versions of the interpreter can be developed in the language itself.
A definition of a computer language is usually made in relation to an abstract machine (so-called operational semantics) or as a mathematical function (denotational semantics). A language can also be defined by an interpreter, whereby the semantics of the language in which the interpreter is defined is usually considered as given. The definition of a language by a self-interpreter is not well-founded (i.e., cannot be used to define a language), but a self-interpreter tells a reader a lot about the expressiveness and elegance of a language. It also enables the interpreter to interpret its own source code, the first step towards a reflective interpreter.
There is an important design dimension in the implementation of a self-interpreter, namely whether a language feature in the interpreted language is implemented by using the same feature in the implementation language of the interpreter (the host language). A typical example is whether a closure in a Lisp-like language is implemented using closures in the interpreter language or implemented "manually" using a data structure that stores the environment explicitly. The more features are implemented by the same feature in the interpreter language, the less control the programmer of the interpreter has. For example, a different behavior for dealing with number overflows cannot be realized if the arithmetic operations are just delegated to the corresponding operations in the host language.
There are some languages that have a particularly nice and elegant self-interpreter, such as Lisp or Prolog. Much research on self-interpreters, in particular reflective interpreters, has been conducted in the context of the Scheme programming language, a dialect of Lisp. In general, however, any Turing-complete language allows the writing of its own interpreter.
Clive Gifford introduced a measure quality of self-interpreter - eigenratio. Eigenratio is a ratio between computer time spent to run a stack of N self-interpreters and time spent to run a stack of N-1 self-interpreters, when N goes to infinity. This value does not depend on top-level program being run.