Quake Command Processor (Quake Console)

Synopsis

Provides information to assist in mimicking the behavior of the Quake command processor in other programs. Presents the execution model of the Quake command processor, including buffering, parsing, and interpretation. The command processor is separate from the console; the latter is the presentational frontend to the former.

Motivation: To generalize the operations of the Quake command processor to assist in implementing a similar command processor for snes9x.

Caveat: The primary source of information is from the GPL'd release of Quake 2. The sources to Quake 1 have not been used, nor those of Quake 3.

The Command Buffer

The command buffer consists of a flat storage area of characters. In Quake 2, the buffer is 8192 characters. In Quake 3, the buffer appears to be 16384 characters. The command buffer stores all text sent to the command processor. The buffer is a simple cache of all incoming text, without parsing. In this respect, the command buffer resembles a String I/O object (file i/o interface to a string).

Operations:

These operations are the public interface to the command buffer. Other components of the program may feed command text to the command buffer in this manner. Two such components in Quake 2 are the game console (e.g. "/quit") and the system-level argument handler (e.g. "quake2 +set game wf").

The Executor

The executor can be decomposed into two parts: the parser, the dispatcher. Intriguingly, the executor does not alter the command buffer. This separation permits the command processor to send a command text directly to the executor, bypassing the command buffer.

Parser

The parser receives a text string, and fills in three variables. One variable is an integer counter `cmd_argc' (argument count), which starts at zero (0) and increments with each parsed word. The second variable is an array/list of strings, `cmd_argv' (argument vector), which holds the parsed words. The third variable is a flat string, `cmd_args' (argument string), which contains the unparsed text starting from the second word. This set of variables parallels the Unix notion of command-line parameters.

The parser does not care about the meaning of any particular word, just the chopping up (parsing) of the text into tokens (words). The dispatcher attaches meaning to words.

This document does not present an in-depth topic on the techniques of parsing.

Token types:

Terminator
Either an end-of-string condition, or a newline character. Causes the parser to terminate.
Separators
A character or sequence of characters that separates words. Typically whitespaces (space, tab, form feed, etc.). Multiple separator characters in a row are treated as a single separator for parsing purposes. In Quake 2, any character with an ASCII code of 32 or less, with the exception of newline (which is terminator) is treated as a separator. The separator is ignored (skipped over) by the parser.
Comment
C++ style comment. Two forward slashes in sequence ('//') start a comment, and continue until a terminator. While the comment token may arguably be considered a line terminator, the comment token consumes characters from the text, whereas a terminator stops consuming. This difference is significant if parsing a multi-line text (you want parsing to continue after the comment, not inside it).
Quoted-word
Starts with double-quotes ("), and ends with double-quotes ("). Any text within are preserved verbatim (to wit, multiple spaces remain intact). The double-quotes are discarded, the content is treated and returned as a word (notice this stretches the colloquial notion of "word"). The parser does not support escape characters, so a double quote cannot be escaped
Word
Starts with a character without any other special meaning, consists of a sequence of such non-special characters, and ends on a token with special meaning (separator, terminator, comment, quoted-word).

Dispatcher

The dispatcher takes parsed arguments, and treats the first word as a command. Based on the command, the dispatcher branches to appropriate command handler code. In Quake 2, this involves a lookup table of command names and function pointers (this relates to commands listing and command completion); the dispatcher loops through the lookup table for all the command names, comparing to the first word (case-insensitive), then using the associated function pointer on a match.

The command parameters are already marshalled into the variables `cmd_argc', `cmd_argv', and `cmd_args'. In Quake 2, these are global variables. Alternatively, these variables may be members of an object of some sort; the encapsulting object itself may then be global, or explicitly passed to handlers. The command handlers access the command parameters via the parameter variables.

Built-in Commands

The command processor in Quake 2 comes with 4 built-in commands: exec, echo, alias, wait. The other commands in Quake 2 are added as extensions to the command processor.

exec

This command is the "include" mechanism of the command processor. The command processor opens the filename specified as the command parameter (the first argument), and reads in commands from the file to append to the command buffer. In the simplest form, the "exec" command opens a file, dumps the entire content to the command buffer, and closes the file. This form limits the size of file to the size of the command buffer.

echo

Simply echoes the command's parameter to the output. In Quake 2, this means printing to the console, using the Com_Printf() function. If you use a similar wrapper print function, you can maintain better control of output display.

alias

Creates a command alias. Aliases is a more advanced topic of the command processor; in Quake 3, aliases have been dropped in favor of the more explicit "vstr" command.

wait

Waits one cycle ("cycle" covered later) before executing the next command. This command introduces a short delay within a chain of commands. In Quake 3, the "wait" command has been expanded to take an integer parameter, indicating a number of successive wait cycles (e.g. "wait 3" == "wait; wait; wait")

The Command Cycle

The command processor needs frequent prodding to proceed with processing of the command buffer. This prodding may be implemented in the form of an explicit call to the command processor from a parent function (e.g. once per for loop), or via a timer callback. In any case, due to the necessary semantics of the "wait" command, the time interval between proddings should be as consistent as possible.

At each prodding, the command buffer extracts one line's worth of command from the command buffer (consuming them), then hands off the extracted text to the command executor. The buffer extraction has a limited parsing ability. Extraction from the command buffer terminates on an unquoted semicolon (;), or on a newline -- meaning the extractor must understand quoting rules, and preserve semicolons within quotes. The trailing terminator (semicolon or newline) is consumed, but not stored (when passed to the executor). Any double quotes in the extracted text are preserved; the extractor needs only be aware whether a semicolon is within double quotes or not.

The command processor continually extracts additional commands from the command buffer, passing them to the command executor, in a loop, until either the command buffer is emptied, or a "wait" command is encountered. In the case of Quake 2, merely stopping this loop fulfills the semantics of the "wait" command (extraction resumes on the next prodding). Quake 3 semantics (multiple wait cycles) requires additional logic; a simple countdown counter ought to be sufficient.

In Quake 2, the command processor is given a timeslice by calling Cbuf_Execute() once per game frame.

Aliases (vstr)

Some cvars may contain a series of commands, which are designed to be run at some future time. Quake 2 supports the notion of aliases to copy the contents of cvars into the command buffer. In Quake 3, the same action is carried out with the "vstr" command.

alias

Quake 2 aliases involves two parts. First, the "alias" command binds an alias (a sequence of commands separated by semicolons (;)) to a command name: alias cmdname "content"
Due to parsing, the content is often within double-quotes ("), so that any embedded semicolons is stored with the alias, rather than treated as a command separator as the alias command is being parsed. The "alias" command creates a new entry in the commands lookup table, specially tagged to indicate an alias rather than a binding to an internal function. Once established, the alias may be used as a normal built-in command. Whenever the alias is executed, it appends a copy of its contents into the command buffer.

vstr

Quake 3 eliminates the "alias" command in favor of the "vstr" command. The "vstr" command is a built-in command, which treats its first parameter as the name of a cvar, then appends a copy of the content of the cvar into the command buffer. The nature of the "vstr" command means the "alias" command can be removed, and the "set" command be used to create alias-type commands. Also, the command lookup table is not cluttered with each set of custom scripted commands.

Extending

The Quake 2 command processor only recognizes a few built-in commands. The other commands in the game are established at runtime by other components. For example, the command "+forward" (the Q2 bind for moving your avatar forward) is explicitly added to the command processor when the client (rendering) subsystem is initalized. The client subsystem tells the command processor to bind the command "+forward" to a client subsystem function called IN_ForwardDown(). The command processor has no inherent knowledge of the command "+forward", it merely provides a way of relating/translating/resolving the command name (from the console/keybind) to a C function in the client game subsystem. Two operations assist in changing commands recognized by the command processor: AddCommand, RemoveCommand.

AddCommand

This operation adds a new entry for the command processor to recognize. The operation also binds the (new) command name to a command handler. In Quake 2, the command lookup table is dynamically reallocated with each new command, each of which is bound to a C function (function pointer). In Quake 3, the non-builtin-command lookup table resides in the VM space, statically allocated from startup, and (as with Quake 2) each entry is a name/function pointer pair.

RemoveCommand

This operation does the reverse of AddCommand, removes a command entry so that the command processor no longer recognizes the command. In Quake 2, the command lookup table is dynamically reallocated with each removal.


Sample implementation: cmdproc-20030924.c, cmdproc-20030924.h

Python implementation (documentation inside): cmdproc-20040222.py


Created 2003.09.08
Updated 2004.02.22

-- PhaethonH (phaethon@linux.ucla.edu)