Usually such a language is limited in one or several ways. By this, it is meant that one of its major limitations is its number of programming constructs or concepts supported. The language can still be complete in the eyes of the creator or its community of users. The grade of completeness in this context is related to real life use, as opposed to its mathematical completeness. Another typical limitation of toy languages is that they do not necessarily have a set of support libraries/functions/utilities that are considered a requirement for creating production quality programs.
A toy language has many uses. One such use is in computer languages research. Some uses are as frameworks for researching new programming constructs or as a prototype for new language concepts or paradigms. Another notable use is as a learning or demonstration tool, e.g. in universities, for programming constructs and techniques not available in main stream languages. As such there is a considerable amount of personal feelings for the language by both its opponents and proponents.
As an example, many would argue that the Scheme programming language is a toy language, as it is mostly used in academia. A view that many share is that even though it is complete in its own right, it has limited capability in real life projects. As such, other languages with similar properties but extended capabilities are preferred, such as Lisp. Another, somewhat similar example, is the Lava programming language, which is purely experimental.
Another test for a language to be considered not a toy is for the language to be powerful enough to implement an operating system. Whether this means that all purely interpreted or virtual machine languages are toys is left as an exercise for the reader.