A newsgroup is a repository usually within the Usenet system, for messages posted from many users in different locations. The term may be confusing to some, because it is usually a discussion group. Newsgroups are technically distinct from, but functionally similar to, discussion forums on the World Wide Web. Newsreader software is used to read newsgroups.
Newsgroups generally come in either of two types, binary or text. There is no technical difference between the two, but the naming differentiation allows users and servers with limited facilities the ability to minimize network bandwidth usage. Generally, Usenet conventions and rules are enacted with the primary intention of minimizing the overall amount of network traffic and resource usage.
Newsgroups are much like the public message boards on old bulletin board systems. For those readers not familiar with this concept, envision an electronic version of the corkboard in the entrance of your local grocery store.
Newsgroups frequently become cliquish and are subject to sporadic flame wars and trolling, but they can also be a valuable source of information, support and friendship, bringing people who are interested in specific subjects together from around the world.
Back when the early community was the pioneering computer society, the common habit seen with many articles was a notice at the end disclosed if the author was free of, or had a conflict of interest, or had any financial motive, or axe to grind, in posting about any product or issue. This is seen much less now, and the reader must read skeptically, just like in society, besides all the privacy or phishing issues.
There are currently well over 100,000 Usenet newsgroups, but only 20,000 or so of those are active. Newsgroups vary in popularity, with some newsgroups only getting a few posts a month while others get several hundred (and in a few cases a couple of thousand) messages a day.
A website called Deja News has begun archiving Usenet in the 1990s. DejaNews also provided a searchable web interface. Google bought the archive from them and made efforts to buy other Usenet archives to attempt to create a complete archive of Usenet newsgroups and postings from its early beginnings. Like DejaNews, Google has a web search interface to the archive, but Google also allows newsgroup posting.
Non-Usenet newsgroups are possible and do occur, as private individuals or organizations set up their own nntp servers. Examples include the newsgroups Microsoft runs to allow peer-to-peer support of their products and those at [news://news.grc.com news://news.grc.com].
Every host of a news server maintains agreements with other news servers to regularly synchronize. In this way news servers form a network. When a user posts to one news server, the message is stored locally. That server then shares the message with the servers that are connected to it if both carry the newsgroup, and from those servers to servers that they are connected to, and so on. For newsgroups that are not widely carried, sometimes a carrier group is used for crossposting to aid distribution. This is typically only useful for groups that have been removed or newer alt.* groups. Crossposts between hierarchies, outside of the Big 8 and alt.* hierarchies, are failure prone.
The most commonly known hierarchies is the usenet hierarchies. So for instance newsgroup rec.arts.sf.starwars.games would be in the rec.* top-level usenet hierarchy, where the asterisk (*) is defined as a wildcard character. There were seven original major hierarchies of usenet newsgroups, known as the "Big 7":
These were all created in the Great Renaming of 1986–1987, prior to which all of these newsgroups were in the net.* hierarchy. At that time there was a great controversy over what newsgroups should be allowed. Among those that the usenet cabal (who effectively ran the Big 7 at the time) did not allow were those concerning recipes, drugs, and sex.
This resulted in the creation of an alt.* (short for "alternative") usenet hierarchy where these groups would be allowed. Over time the laxness of rules on newsgroup creation in alt.* compared to the Big 7 meant that many new topics could, given time, gain enough popularity to get a Big 7 newsgroup. This resulted in a rapid growth of alt.* which continues to this day. Due to the anarchistic nature with which the groups sprung up, some jokingly referred to ALT standing for "Anarchists, Lunatics and Terrorists" (a backronym).
In 1995, humanities.* was created for the discussion of the humanities (e.g. literature, philosophy), and the Big 7 became the Big 8.
The alt.* hierarchy has discussion of all kinds of topics, and many hierarchies for discussion specific to a particular geographical area or in a language other than English.
Before a new Big 8 newsgroup can be created, an RFD (Request For Discussion) must be posted into the newsgroup [news:news.announce.newgroups news.announce.newgroups], which is then discussed in [news:news.groups.proposals news.groups.proposals]. Once the proposal has been formalized with a name, description, charter, the Big-8 Management Board will vote on whether to create the group. If the proposal is approved by the Big-8 Management Board, the group is created. Groups are removed in a similar manner.
Creating a new group in the alt.* hierarchy is not subject to the same rules; anybody can create a newsgroup, and anybody can remove them, but most news administrators will ignore these requests unless a local user requests the group by name.
Additionally, there is the free.* hierarchy, which can be considered "more alt than alt.*". There are many local sub-hierarchies within this hierarchy, usually for specific countries or cultures (such as free.it.* for Italy).
There were originally a number of obstacles to the transmission of binary files over Usenet. Firstly, Usenet was designed with the transmission of text in mind. Due to this, for a long period of time, it was impossible to send binary data as it was. So, a workaround, Uuencode (and later on Base64 and yEnc), was developed which mapped the binary data from the files to be transmitted (e.g. sound or video files) to text characters which would survive transmission over Usenet. At the receiver's end, the data needed to be decoded by the user's news client. Additionally, there was a limit on the size of individual posts such that large files could not be sent as single posts. To get around this, Newsreaders were developed which were able to split long files into several posts. Intelligent newsreaders at the other end could then automatically group such split files into single files, allowing the user to easily retrieve the file. These advances have meant that Usenet is used to send and receive many terabytes of files per day.
There are two main issues that pose problems for transmitting binary files over newsgroups. The first is completion rates and the other is retention rates. The business of premium news servers is generated primarily on their ability to offer superior completion and retention rates, as well as their ability to offer very fast connections to users. Completion rates are significant when users wish to download large files that are split into pieces; if any one piece is missing, it is impossible to successfully download and reassemble the desired file. To work around this, a redundancy scheme known as PAR is commonly used.
A number of websites exist for the purpose of keeping an index of the files posted to binary newsgroups.