It was originally a record manager that was published by SoftCraft at around the same time as the release of the first IBM PCs. After gaining market share and popularity, it was purchased by Novell for integration into their Netware operating system. The product failed to gain significant market share and, after some reorganization within Novell, the product was spun off to be developed by a new company known as Btrieve Technologies, Inc. (BTI).
Btrieve was modularized starting with version 6.15 and became one of two database backends that plugged into a standard software interface called the Micro-Kernel Database Engine. The other product is Scalable SQL, a relational database product that uses Structured Query Language, otherwise known as SQL. After several new versions were released the company was renamed to Pervasive Software and they now sell the product as Pervasive PSQL.
Btrieve is not a relational database management system (RDBMS). Early descriptions of Btrieve referred to it as a record manager (though Pervasive initially used the term navigational database but later changed this to transactional database) because it only deals with the underlying record creation, data retrieval, record updating and data deletion primitives. It uses ISAM as its underlying indexing and storage mechanism. A key part of Pervasive's architecture is the use of a MicroKernel Database Engine, which allows different database backends to be modularised and integrated easily into their DBMS package, Pervasive.SQL. This has allowed them to support both their Btrieve navigational database engine and an SQL-based engine, Scalable SQL.
Current versions of Btrieve support system transactions and user transactions, where system transactions are a bundle of non-transactional operations and/or user transactions, while user transactions are transactions that work on actual data in the database. System transactions were developed to allow multiple transactions to be done in a batch and to allow the ability to recover data more easily.
The Btrieve file format consists entirely of pages, which are the data that moves between memory and storage media when the engine performs an I/O operation. Versions prior to 6.0 merely used data pages, index pages and a file control record (FCR). The file had an index for searching that linked to physical pages. Beginning with version 6.0 logical pages started to be used, which are pages that are mapped to physical pages (pages at a fixed location in the file) on the disk through the use of a set of page allocation tables (PATs). The FCR is a record that contains important information about Btrieve files, such as the number of pages in current use. In order to avoid corruption in the database Btrieve uses two methods of updating records: pre-image paging in Btrieve versions before 6.0 and shadow paging in subsequent versions. It was mainly the change-over from pre-image paging to shadow-paging that caused radical file format changes that broke compatibility between previous versions of Btrieve and version 6.x of the product.
Although Btrieve was fairly popular, it was not strongly differentiated from the killer-app database on the PC, dBase, and never gained the same sort of popularity. However, the known developer base had grown to over 5,000 users and it was widely used in the financial area. The company took some time to create a user-interface for the product, however in 1984 they released Xtrieve, a menu-driven program that used a new .DDF data dictionary to enforce relational database rules.
In 1987 Novell started diversifying and buying companies to add to their NetWare operating system. One of the companies they purchased was SoftCraft. Nancy Woodward became the Vice-President and General Manager of Novell's Austin operations while Doug Woodward became the Vice-President of Advanced Database Technologies. Early the next year Btrieve 5.0 was released to run as a native NetWare application, or VAP (Value Added Process). According to Jim Kyle, "it had auto-increment key types, the BROUTER network process server, data-only and key-only files, and optional data compression". Version 5.1 was released in 1990 with increased file-handling transaction capability, logging and roll-forward operations, along with several API enhancements. Several versions were created for DOS, OS/2 and Microsoft Windows. Version 6.0 was released in June 1992, however it was not promoted extensively by Novell, and due to enhancements (such as the change from pre-imaging to shadow-paging) it was incompatible with previous versions of Btrieve. The market did not increase much for Btrieve and it did not see wide adoption due to these issues.
When the company was acquired by Novell, SoftCraft had been working on a product called XQL, which was an SQL interpreter that was designed to better deal with industry standard SQL, which the Xtrieve package was not fully compliant with. This became the basis for NetWare SQL, which was initially released in 1989, and was a bare-bones SQL interpreter which implemented the base IBM version of SQL.
By 1994 Novell had largely given up on attempting to make NetWare into a complete alternative operating system, and started selling off many of the companies it had acquired only a few years earlier. They had also done minimal promotion of Btrieve, largely due to the long time (24 months) it took to release version 6. Negotiations between Nancy and Doug Woodward with Novell were entered into and after two years Novell announced (on January 26, 1994) that it was going to transfer ownership of Btrieve to Btrieve Technologies, Incorporated (also known as BTI). On April 29, 1994 the transfer was completed and Nancy Woodward became the Chairman of BTI and Doug Woodward was made the Chief Technical Officer. The CEO position was given to Ron Harris, a former employee of Texas Instruments, and one of the founding employees of Citrix Systems, Inc. where he was employed first as Director of Strategic Planning, then as Vice-President of Marketing, and finally as the Product Group Vice President.
Btrieve was totally rewritten and on July 1, 1994 Btrieve 6.15 was released for DOS, Windows and OS/2. Novell SQL was renamed to Scalable SQL to reflect the change in ownership of the company. In 1995 version 6.15 was released for Novell NetWare, Windows NT Server and for Windows NT/95, and thus became a cross-platform database product. The concept of a Micro Kernel Database Engine (MKDE) was introduced in this version.
In 2000, Novell was criticized after it ceased bundling Pervasive.SQL with NetWare (5.1 was the first version affected). Instead, it shipped with a trial version that shut down after 90-days. The latest version, Pervasive PSQL Summit v10, was released in October 2007.
BSERVER, on the file-sharing server and this managed data I/O in conjunction with the network file system. The server process was first implemented as a Netware Value Added Process (VAP) called
BSERVER.VAP, but was switched to a Netware NetWare Loadable Module (NLM) soon after. Basically,
BSERVERwas the database engine that dealt with access to records, however it also accepted requests from the transmittal of requested data to another server via the
Btrieve used requesters to make database I/O requests from the client workstation. These requesters were available for DOS, OS/2, Microsoft Windows, and UnixWare. The program
BREQUEST.EXE accepted I/O requests via the Btrieve API and relayed them to
BSERVER. It then handled the responses from
BSERVER and relayed them back to the appropriate applications.
BROUTER process allowed for incoming requests to be "routed" to a copy of the database on another server. It was loaded on the Netware server and dealt with communication between multiple server processes running on the one file-server through the use of two File Server Tables (FSTs). According to Pervasive, these provide a list of "server names and addresses, and the Server Routing Table (SRT)". BROUTER also allowed communication requests to be routed to the correct server via SPX by looking up the
BSPXCOM NLM and coordinated locks and other mechanisms that controlled access to the data in the Btrieve database.
Btrieve for DOS used the SEFS and MEFS modes for file sharing, and because it was able to run on a network it was able to use exclusive and concurrent transactions.
Btrieve for Windows could run as a client to the database that utilized SEFS or MEFS modes, or it could directly access the Btrieve server.
The client-based version of Btrieve has all the database files either directly on the local computer or via a mapped network drive (setup using DOS’s
NET USE command).
Applications make a function call to
WBTRCALL.DLL, a loader and requester interface. The loader and requester module checks the
BTI.INI configuration file is correctly setup to load the client-based Btrieve engine. In turn, this loads the local interface to the btrieve engine (
WBTRLOCL.DLL). If necessary, this local interface loads the Btrieve engine (
WBTR32.EXE) into memory and sends the necessary database requests to it. The database engine then calls various Win32 system libraries to perform file operations on the database files.
As with the client-based interface, the Btrieve-based application makes a call to the
WBTRCALL.DLL loader and requester interface library. This library checks the
BTI.INI file to see if it needs to access data on the local system or whether it needs to access data on a remote server. If it needs to access the server then it uses the Windows version of DPMI to access a DOS-based requester named
BREQUEST.EXE. The requester then establishes a network connection to the server, which processes the request and passes back a message to the requester when the database request is completed.
There are two configurations of Btrieve for Windows NT/95: standalone workstation and client/server.
When using the standalone workstation configuration of Btrieve, all processing of records is done on the local workstation. The workstation relies on the underlying mechanisms of Windows to allow the MKDE (the
W32MKDE.EXE program) to gain direct access to the database files, and uses lock files to deal with concurrency issues.
In this configuration the application makes calls to the Btrieve API, or Microkernel Interface (
WBTRV32.DLL). The call is then processed by this interface and passed along to the MKDE (
W32MKDE.EXE) which then uses the underlying operating system file system (whether it be network or local) to directly access the database files.
This leads to some peculiar issues. If Btrieve uses Windows file sharing and has the database engine open files directly on a file share, for instance, and there is network instability (or even if a network cable is unplugged) during an update the fields used to link one Btrieve file to another can become unsynchronized (to all intents and purposes the data loses its relationships or links to other data) and the database file itself can get corrupted (though the chance of this is reduced due to pre-image paging).
When using the client/server (or Server edition) configuration of Btrieve, processing of records is generally done on a Windows file server via a mapped drive (a way of mapping a file share to a "virtual" disk drive in Windows via the
NET USE command). It utilises the permissions that you are assigned when authenticating, either from when logging on or via the permissions given for the
NET USE is utilised.
On Windows 95 the MKDE interface (a Windows dynamic link library (DLL) called
WBTRV32.DLL) actually determines what database access method is in use via the configuration file. If it detects that both the client/server and workstation engines are installed on the one machine it checks whether the target is set to workstation or server. If running on Windows NT and the server process
NTMKDE.EXE is running along with the standalone workstation process
W32MKDE.EXE it looks in the registry to determine if the target is either server or workstation. In both cases, if the MKDE interface is set to workstation (the "Standalone workstation" configuration) it uses the MKDE (
W32MKDE.EXE) to directly access the file. If it is set to server then the MKDE interface on the client uses a communications module (on Windows 95 this is
W32BTICM.DLL, on Windows NT this is
NTBTICM.DLL) that "talks" to the server. The server itself has its own matching communications module (again either
NTBTICM.DLL) that resides on the mapped drive. The server DLL then communicates with the server MKDE (
NTMKDE.EXE) which updates records, then sends a confirmation that the operation succeeded back through the communications module to the client.
The advantage of this system is that if a network connection failure occurs the MKDE on the server will be able to detect this and recover in a more graceful manner than the workstation configuration is able to.
A configuration utility was included with Btrieve to alter MKDE settings. The settings that could be changed were:
PVSW.LOGand that had a unified and enhanced log file format. They also improved their error messages and error message reporting mechanisms.
The MKDE was retained in Pervasive.SQL 7. However, due to the new component architecture's dynamic binding, the internal architecture was modified. The application using Btrieve calls a services manager which then searches through various configured directories for specific encoded filename. The file name loaded for Btrieve files in Backus-Naur form is:
::= "W1" | "W2" | "W3" | "W9" | "WT" | "NW" | "O3"
::= ::= "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9"
|W1||Windows 3.1x, incl. Windows for Workgroups (Win16)|
|W2||Extended Windows (32-bit Watcom Extender)|
|W3||Windows 95, Windows NT (Win32)|
|NW||Netware 3.x and 4.x|
The "glue" module, which is a DLL, is loaded into memory and becomes the interface to the MKDE. The MKDE then determines whether it is configured to be a workstation based configuration or a server based configuration. It then passes requests via its communications "requester" module onto the database server, or directly modifies the database files if configured in workstation mode.
The V8 Security Feature Pack (a mid-release product update designated 8.5) added important new security features designed to lock down Pervasive.SQL data files. Prior to 8.5, access to Btrieve data was controlled by the operating system's security mechanism. This meant that any user who needed read/write access to the database, also needed read/write access to the underlying data files. 8.5 introduced new security models, which allow administrators to control access to the Btrieve data using database security. Once activated, database security no longer requires that the user has access to the underlying files. In addition, client/server configurations no longer require the use of network shares or mapped drives. Applications can reference secure Btrieve data using a URI connection string.
In conjunction with PSQL v9 Pervasive reintroduced the DDF Builder utility and added support for text searching with the Full Text Search (FTS) add-on, which was later removed from the product line. DDF Builder provides a mechanism for Btrieve users to define the meta data for existing Btrieve files, thus allowing Btrieve data to be accessible via SQL tools and utilities.
All versions of the MKDE retain full backward read-level compatibility with earlier versions of Btrieve, including those that pre-date introduction of the MKDE itself, and do not change the file's version unless specifically requested to do so. Btrieve files that are in the 5.x or older file formats MUST be rebuilt (using the GUI or command line Rebuild utilities) to 6.x or newer format to support database writes from the 9.0 or newer database engine.