In Terminology of the Imperium, the glossary of the novel Dune, Herbert provides the following definition:
WAR OF ASSASSINS: the limited form of warfare permitted under the Great Convention and the Guild Peace. The aim is to reduce involvement of innocent bystanders. Rules prescribe formal declarations of intent and restrict permissible weapons.
The term describes the use of professional assassins in a feud between two Great Houses of the Landsraad in the pre-Atreides Imperium. Atomics were outlawed by the Great Convention, and Holtzman shields nullified projectile weapons and non-nuclear explosives. All-out warfare was also considered too destructive and a threat to peace in the Empire. Hence the Great Houses were obliged to resort to treachery and professional assassination in order to carry on with their perennial vendettas (known as kanly). To facilitate this form of conflict, Great Houses customarily hired a master assassin — usually one with Mentat training — to supervise the house's corps of assassins. Such Mentats were supposed to act as intelligence and counter-intelligence experts, advising and acting for their masters, both during their own campaigns, and on foiling of opposing plots and schemes.
Under the terms of the Great Convention, an outright war of assassins always required a formal declaration of intent to be filed with the Imperial Court, the offices of the Spacing Guild, and the Secretary of the Landsraad. Imperial judges were appointed to monitor the conflict and, particularly, to observe any instances where innocent bystanders might be harmed. Such supervision did not always ensure the safety of noncombatants, nor a fair treatment of the two houses involved, for the Emperor was rarely without vested interest in the outcome of the conflict. The regulations and supervision made the feuding houses wary of harming or even involving outsiders, though.
The glossary also describes the Assassin's Handbook:
ASSASSINS' HANDBOOK: Third-century compilation of poisons commonly used in a War of Assassins. Later expanded to include those deadly devices permitted under the Guild Peace and Great Convention.
The first recorded instance of a war of assassins is the conflict between House Pardee and House Harkonnen in 3367 - 3375 A.G., a conflict which virtually obliterated House Pardee. This war involved an extensive use of poison, but it was climaxed by the famous Lasgun Massacre, when Harkonnen mercenaries ambushed Duke Ira Pardee and his family.
Numerous other famous wars of assassins occurred in Imperial history, often altering the balance of power among the Great Houses of the Landsraad. For instance, House Galloway was powerful in the third millennium, but was nearly destroyed in its war of assassins with House Albonite. Some other well-known wars of assassins were the Steinhauser-Boudreau feud (8193 - 8195 A.G.) and the ChoiDwyer-Ferguson war (8787 - 8843 A.G.), a three-cornered bloodbath from which each house emerged weakened. This latter conflict was unusually bitter, and ended only when the Emperor himself interceded with the families in conflict.
The important wars of assassins dominated political history of the last century of the pre-Atreides Imperium. The first of these was the feud of House Moritani of Grumman with House Ginaz, the latter an ally of Duke Leto Atreides I. It took place in 10175 - 10181 A.G., and ended with the total defeat of House Ginaz, after the Ginaz Duke and several of his family were poisoned. The Corrino Emperor's wariness concerning Duke Leto Atreides and his house was a great aid to House Moritani in this affair.
Shaddam Corrino IV's favor also helped House Harkonnen immeasurably in its feud with House Atreides, culminating in the famous massacre of the Atreides on Arrakis in 10,191 A.G. In this war, the Harkonnens used Suk Doctor Wellington Yueh, the family physician of House Atreides, as their piece of treachery, and Piter De Vries as their Mentat-Assassin. According to the Oral History and other sources, Dr. Yueh cut off the power, thereby lowering the shield defenses of the Atreides palace. The Atreides residence was then overrun by Harkonnen mercenaries and Imperial Sardaukar. The conflict cost the Harkonnens their Mentat, but the Atreides Mentat-Assassin, Thufir Hawat, was compelled to join the Harkonnens after being poisoned with a substance that required a daily dose of antidote that only Baron Vladimir Harkonnen could provide.