“But That’s Impossible in Pascal”

or

Systems Programming in a High-Level Language

E. W. (Wayne) Sewell Software Engineering Specialist E-Systems, Garland Division Mail Stop 53730, P. O. Box 660023 Dallas, Texas 75266–0023

Abstract

Popular wisdom has always held that systems programming can be done only in MACRO or Bliss. While this is true for code running in kernel mode and/or directly accessing the system memory space, it does not automatically follow that it is not possible to use high level languages at all.

This session illustrates techniques for systems programming in Pascal, including utilization of user-written system services and accessing the system memory space. Examples include a simple program to display various system control blocks and portions of code from an Ancillary Control Process (ACP) written in Pascal. While the examples are in Pascal, the techniques illustrated can be used for any high-level language which supports a record structure, pointers, and the VAX standard calling mechanism.

Special Requirements of Systems Programming

Before we can begin a discussion of how to do systems programming in a high-level language, we first must determine what systems programming is. There are many different definitions for a systems program. Virtually everyone agrees that the VMS Scheduler is a systems program and that a payroll system is an application program, but many other cases are not so clearly cut. For example, a compiler is logically a systems program, but the processing it does, although complex, makes no special demands upon the operating system and can easily be implemented in a high-level language. For the purposes of this discussion, we will ignore the function performed by a program, and instead define a system program to be any program which requires access to the internals of the VMS operating system not provided by the standard libraries (system services, run-time libraries, RMS, etc.). Using this definition, systems programs are those which must do the following:

• Access the system address space, including the nonpaged pool. The system space is simply not accessible from user mode. An attempt to reference a system address from a non-privileged mode has the same result as a reference to a non-existent virtual address—an access violation, followed by termination of the process.

• Access VMS routines in system space. Since they are not addressable, for the reasons mentioned above, they cannot be called by a user mode program.

• Raise IPL (Interrupt Priority Level, not to be confused with the priority of a process). Increasing IPL is a VMS mechanism for synchronization. A program wishing to lock the I/O data base for exclusive access raises its IPL to a predefined level, effectively locking out all other processes in the system attempting to access the data base. Programs in user mode run at IPL 0, and cannot issue the privileged instructions necessary to change IPL.

• Use MUTEXes (MUTual EXclusion—the VMS version of a semaphore). The MUTEX structures themselves and the system routines that manipulate them are in system space.

All of these functions require processing in kernel mode. Unfortunately, much as I hate to admit it, kernel mode processing in a high-level language really is impossible. There are too many constraints placed on code which is meant to run in kernel mode. For instance, all references to memory must be to valid addresses, because an access violation in kernel mode will crash the entire system instead of just the offending process (in MACRO, the PROBE instruction is utilized to test addresses for validity before actually referencing them). Many calls to system routines require the parameters to be passed in registers rather than pushed onto the stack as specified in the VMS calling standard. In fact, these procedures require that the JSB instruction be used rather than the normal CALLS or CALLG. There are many other special instructions in the VAX architecture which are restricted to usage in kernel mode. The high-level language programmer has very little control over any code generated by the compiler, especially with optimization enabled. This is as it should be—one of the main advantages of using high-level languages is a lack of involvement with the actual machine code, but it does make it difficult to generate the special instructions needed by kernel mode such as JSB or PROBE.

So why not do systems programming in MACRO and be done with it? The primary reason is that it is tedious to write entire programs in MACRO. Textual output, so easy with the Pascal WRITELN, is especially tiresome. Without the inherent benefits of structured programming, MACRO programs tend to be harder to read without conscientious effort on the part of the programmer, and the control mechanisms supplied by a high-level language have to be performed manually. Another problem with MACRO is that some classes of system programs only need kernel mode at distinct points—the rest of the processing can be done in user mode. However, most system programs written in MACRO change to kernel mode when the program begins and remain in that state until termination. During development of such a program, the most trivial of logic errors can crash the entire system.

So what’s the best way? My personal preference for all types of programming is to code the main program in a high-level language, supplemented by MACRO subroutines for whatever functions cannot be performed by that language. In the case of a systems program, the MACRO subroutines would exist in a change-mode-to-kernel dispatcher (described below). The subroutines can be made short and modular, and if coded generically they can be reused for other programs performing a similar function.

Systems programs which are likely candidates for high-level languages are programs which need to run in kernel mode only briefly, during distinct phases such as input or output, and most processing can be done in user mode. An example of such a program is an Ancillary Control Process (ACP). Systems programs which are poor candidates for high-level languages are exemplified by device drivers and other code loaded into system space, programs which cannot be implemented as a standard VMS process, or programs which must run in kernel mode most of the time.

The change-mode-to-kernel dispatcher

A change-mode-to-kernel dispatcher is the VMS mechanism by which a normal user-mode process can execute privileged code. VMS system services are implemented this way. The dispatcher can be considered a special form of subroutine library. The subroutine definition in the calling program is exactly the same as for any other externally-defined subroutine (in Pascal, with the “EXTERNAL” designator). As with ordinary entry points in a shareable image, control is passed to a transfer vector rather than the actual entry point of the subroutine. For user mode routines, the transfer vector contains a simple jump instruction to the actual subroutine address. However, vectors for kernel mode routines instead issue a change-mode-to-kernel instruction, specifying the unique index number of the target subroutine. When control is transferred to the subroutine, the instructions are executed in kernel mode with all of the privileges thereof. When the subroutine finishes processing, it issues a normal return (RET) instruction. When control is returned to the calling program, the process is once again executing in user mode. The percentage of the time the process spends in kernel mode is minimized; during development, exception conditions crash only the process. After the various routines in the dispatcher are debugged, the system crashes may stop entirely, since these error-free procedures can be reused by the next system program developed and only the new user-mode code must be debugged.

This paper does not discuss the implementation of a change-mode-to-kernel dispatcher; this has been covered in other presentations at various Symposia. For an example of implementation, see the sample dispatcher in

                                                    SYS$EXAMPLES:USSDISP.MAR or the one supplied with this

session in the Languages and Tools Session Notes.

The most important advantage to using home-grown system services is that it minimizes the percentage of the code executing in kernel mode. In a given program, some operations can be performed only in kernel mode, but many others can be performed equally well in user mode. For instance, the sample program described below, which does nothing more than dump the PCB of the current process, must access the non-paged pool to get its input data. This can only be done in kernel mode. However, there is no reason for the actual formatting of the PCB fields, which is done with ordinary Pascal write lines, to execute in kernel mode. The kernel mode processing is restricted to the few instructions necessary to get the address of the PCB and copy its contents into normal process memory space.

One drawback to systems programming in Pascal is that the overhead of the change-mode-to-kernel dispatcher is a factor in time-critical code. There is more overhead for a user-written change-mode-to-kernel dispatcher than for a simple procedure call (the amount of overhead is exactly the same as that caused by system service calls, since system services also utilize a dispatcher). If a process is continuously switching between user mode and kernel mode, this technique is not the optimum approach and the all-MACRO version should probably be used.

The System Control Blocks

The system control blocks are defined in Pascal as standard record types and are referenced with standard Pascal pointer variables. There are two basic types of system control blocks—structured and unstructured. In a structured block, the definition of the first 11 bytes is the same for all block types: the first two longwords are forward and backward pointers, used to place the block into a queue; the word following the queue pointers (at offset 8) contains the actual length of the block in bytes; the byte following the length (at offset 10) contains a binary code uniquely identifying the block type.

As an example of a structured block, the I/O Request Packet (IRP) is defined in Pascal as:

    irp_ptr=  ^irp  ;
    irp = packed record
    irp$l_ioqfl :  irp_ptr ;
    irp$l_ioqbl :  irp_ptr ;
    irp$w_size :  word ;
    irp$b_type :  byte ;
    irp$b_rmod :  byte ;
    .
    .
    .
    end ;

The unstructured system control blocks have no header because they are never queued (they are referenced by other means) and the length of block is fixed or stored outside of the block. The best known example of an unstructured block is the Process Header (PHD)—the length is fixed, and the PHD address is contained in the PCB.

The major disadvantage of systems programming in Pascal is that code does not track changes to VMS data structures automatically; creating the record layouts for the system control blocks is a manual operation. In MACRO, the definition of the system blocks is provided with VMS, contained in the system macro library SYS$LIBRARY:LIB.MLB. When a new version of VMS is received, a new version of the library comes with it, and recompiling and relinking is the only action required to pick up any changes to the definitions of the control blocks (provided there are no major functional changes such as deletion of fields currently in use; in this case, MACRO and high-level language programmers are in the same boat). Pascal, since it is not expected to need this type of information (this is impossible in Pascal, remember?), has no corresponding definitions supplied with the system. The record layout must be edited by hand if a new version of VMS changes the system control blocks. (To minimize this activity, it is wise to place the record layouts in an environment or include file rather than duplicating them in every source file.)

When creating the record layout, it is possible to accelerate the process by dumping the macro definitions contained in the macro library to a temporary file and editing this file rather than starting from scratch. This file can be created by entering:

                                                        $ library/macro/extract=$xyzdef -
                                                                /out=tempfile.dat       -
                                                                sys$library:lib

where XYZ is the name of the system definition desired, such as PCB. The result of the above command is a file containing lines such as:

                                                        $equ pcb$w_size 8
                                                        $equ pcb$b_type 10
                                                        $equ pcb$b_pri  11
                                                        $equ pcb$l_pid  96

$EQU is the name of the macro which defines a symbol—it can be edited out. The middle column is the symbol itself and the number on the right is the offset (in bytes) of the field from the beginning of the record. The $L, $W, and $B within the name is the size of the field (longword, word, and byte, respectively). After editing, the above text should look like this:

                                                        pcb$w_size :  word;
                                                        pcb$b_type :  byte;
                                                        pcb$b_pri :  byte;
                                                        pcb$l_pid :  integer;

A great deal of care must be taken ensuring that the Pascal records match the system control blocks exactly. For this reason, the PACKED designator must be used on the record statements if fields identified as words or bytes are present. If left unpacked, these fields, which are defined as subranges (word = 0..65535 ; byte = 0..255), will be allocated space according to their base type (integer, 32 bits). This will cause all following fields to have incorrect offsets.

There are several places in the various VMS control blocks where privilege masks are used, notably in the PCB. A VMS privilege mask is an array of 64 bits, where each bit set to one represents the ownership of a particular privilege. The array is packed, so the mask is exactly a quadword (8 bytes) in size. In Pascal, privilege masks can be implemented either as a packed array of type boolean or as a set. Either approach can be used as long as the resultant variable is exactly a quadword in size.

A privilege mask defined using the boolean method would be coded like this:

                                                        type  privmask = packed array [0..63]
                                                                       of boolean ;

SCANPCB—a trivial sample program

The following program provides an example of accessing the non-paged pool. Its sole function is to dump the contents of the process control block (PCB), process header (PHD), and Job Information Block (JIB) of the current process to the standard output file. The fact that most of this information can be more easily obtained via $GETJPI is irrelevant, since this is only an example of the technique of accessing system control blocks; most of the other information in the system space cannot be acquired by using system services provided by VMS. One important concept illustrated in this example is block chaining. Once we have the address of the PCB, we can easily acquire the address of the other blocks from the associated pointers in the PCB.

Declarations

There are three routines contained in the example changemode-to-kernel dispatcher:

• GET MY PCB, which simply returns the address of the process control block (PCB) of the current process

• COPY SYSTEM CONTROL BLOCK, copies one of the sys-

tem control blocks from the nonpaged-pool to process memory space or vice versa using the lengths contained within the control blocks themselves—used for structured blocks

• COPY NONPAGED BUFF, which performs the same copy

operation as COPY SYSTEM CONTROL BLOCK, except that

the lengths are explicitly specified—used for unstructured blocks, which have no embedded length field

To access the fields in the control blocks, we define two pointer variables, a local pointer and a non-paged-pool pointer, for each of the three control blocks used by the program (PCB, JIB, and PHD). The local pointer contains the address of the control block created in the process virtual memory space, a standard Pascal record allocated on the heap and accessed by normal Pascal references. The nonpaged pointer cannot be used as a pointer in the standard way; the block it points to is in the non-paged pool and therefore inaccessible to a user mode program. If it is used as a pointer, an access violation will result and the program will abort. This variable is instead used to simply store the non-paged address until it can be used as one of the parameters to the COPY SYSTEM CONTROL BLOCK procedure.

var
nonpaged_pcb_pointer,
local_pcb_pointer
:  pcb_ptr ;

The three procedures in the change-mode-to-kernel dispatcher are defined as Pascal EXTERNAL procedures. It is transparent to Pascal whether the procedures called reside in an object library, a shareable image, or a changemode-to-kernel dispatcher. The calling mechanism is the same—the only concern is ensuring that the parameter definitions agree. Note that the copy-from and copy-to block addresses for the two copy procedures (formal variables f and t) are defined as integers, although they are actually pointers. These parameters would normally be defined as the pointer type for the block being copied. However, these procedures are called using pointers to three different block types (pcb ptr, jib ptr, and phd ptr). In fact, since these procedures are intended for become the nucleus of a general purpose dispatcher to support a multitude of systems programs, the blocks to be copied could be of any type. The problem is that the procedure cannot specify multiple types on a formal parameter definition (actually, there is a method to accomplish this using variant records, but it can become very messy). The solution is to define the parameter as a neutral type of the same size as a pointer, such as an integer, and then using the type cast operator (::), which allows a temporary override of the rigid type checking of Pascal, when the procedure is actually called.

    function copy_system_control_block
              (%immed f , t :  integer)    :
              integer ; extern ;
    function copy_nonpaged_buff
              (%immed flen, tlen:  integer;
              %immed f , t :  integer)
              :  integer ; extern ;
    function get_my_pcb
              (var p :  pcb_ptr)
              :  integer ; extern ;

SCANPCB processing

The first action performed by the SCANPCB program is to get the pointer to the current PCB, which is located in the nonpaged pool. At the same time, a local PCB is allocated in the normal process address space using the Pascal NEW function. The contents of the local PCB are initially undefined.

    return_code := get_my_pcb
           (nonpaged_pcb_pointer) ;
    new(local_pcb_pointer) ;

After the two pointer variables are loaded the system control block copy routine (the one used for structured blocks) is used to copy the inaccessible real PCB to the local duplicate. The type cast operator (::) is used to temporarily redefine nonpaged pcb pointer and local pcb pointer from type pcb ptr to integer, which is the type specified on the formal parameters in the procedure definition, without affecting these variables elsewhere in the program.

return_code := copy_system_control_block
(nonpaged_pcb_pointer ::  integer,
local_pcb_pointer ::  integer) ;

Now that the local PCB is a mirror-image of the one in the nonpaged pool, the fields within it can be accessed via normal Pascal mechanisms. In this sample program, we simply print the fields in decimal or hex on the standard output device.

with local_pcb_pointer^do
writeln ('pcb$l_pid ' ,
'(process id) = ' ,
hex (pcb$l_pid)) ;

The individual privileges in a privilege mask can easily be accessed as boolean variables in the packed array of bits.

with local_pcb_pointer^do
if pcb$q_priv[prv$v_oper] then
writeln('this process has oper priv');

Since the field PCB$L JIB contains the address of the JIB in the non-paged pool, we can initialize the non-paged pointer to it with a simple Pascal assignment statement.

nonpaged_jib_pointer :=
local_pcb_pointer^.pcb$l_jib ;

Now that we have the address of the JIB, we can

use COPY SYSTEM CONTROL BLOCK again to create a local

JIB and print its contents as we did for the PCB. The procedure to dump the PHD is the same except that COPY NONPAGED BUFF is used, since the PHD is an unstructured block.

Procedures in the dispatcher

Within the change-mode-to-kernel dispatcher itself, the following MACRO code returns the address of the Process Control Block of the current process:

$ent     get_my_pcb
movl     r4,@4(ap)
movl     #ss$_normal,r0
ret

One of the key points illustrated by this subroutine is its brevity, which is enhanced by the fact that the address of the current PCB is preloaded into register 4 by the dispatcher; however, the active PCB can also be found via the absolute memory location sch$gl curpcb.

The following code copies a system control block from the process virtual address space to system space or vice versa. The lengths of the blocks are contained within them.

         $ent     copy_system_control_block
         movl     4(ap),r2
         movl     8(ap),r3
         movc5    pcb$w_size(r2),(r2),#0, -
                        pcb$w_size(r3),(r3)
         movl     #ss$_normal,r0
         ret

This subroutine is almost as short as the first one. It would be slightly longer in practice, since for clarity this sample routine does not include any validity checking of the input parameters, which would be essential in a real routine. The other block-copy routine, COPY NONPAGED BUFF, is not shown here because it is virtually identical to COPY SYSTEM CONTROL BLOCK; the only difference is that the lengths of the two blocks are passed as parameters instead of being retrieved from the blocks themselves.

A Pascal ACP

A VMS device driver operates in a limited context. The driver code does not execute as a normal process, and therefore has no user memory space, no event flags, no global sections, etc. It cannot issue system services, receive AST’s, hibernate, or wait for event flags. Sometimes the functions being performed by a driver are complex enough to require one of these forbidden operations. Or, one I/O operation may cause several lower-level operations to occur, requiring the driver to issue I/O requests to itself or another driver via the QIO mechanism. An example of this type of operation would be a disk read of a virtual block, which would be transformed to multiple logical and physical operations (SEEK the physical block, READ the physical block, etc.).

An Ancillary Control Process (ACP) is a standard VMS mechanism to expand the capabilities of a device driver. It can be considered an extension of the driver to handle functions which cannot be performed easily by the driver. Since the ACP is a full VMS process, it can hibernate, issue system services, and do all of the other things a driver cannot.

The ACP described here is used with a simulated terminal driver, similar in function to the Decnet REMACP. It allows remote login over a proprietary local area network using non-VT100 terminals.

The functions performed by the ACP include:

• Creating and initializing the ACP Queue Block (AQB) and Volume Control Block (VCB) in the non-paged pool.

• Reading the I/O Request Packets (IRPs)

• Posting the I/O complete Declarations

In addition to the two block-copy routines listed above for the example program, the following additional routines are contained in the change-mode-to-kernel dispatcher for the ACP:

• GET IRP FROM QUEUE, which gets the next IRP for processing

• POST IO COMPLETION, which places an IRP which has returned from the link into the I/O postprocessing queue

  • ALLOCATE ACP QUEUE BLOCK, which is used during

process initialization

  • CREATE NEW UCB and KILL UCB, which are used to

create and delete virtual terminals

Initializing the AQB

This section of code allocates and initializes an ACP Queue Block (AQB). The ALLOCATE ACP QUEUE BLOCK procedure is similar to the GET IRP FROM QUEUE procedure, in that it returns the address of a block in the non-paged pool, except that in this case, the block is actually created by the procedure.

    return_code :=
         allocate_acp_queue_block
               (nonpaged_aqb_ptr);

All of the fields of the local block must be initialized, since this block will be copied to the pool to become the real AQB.

    new(local_aqb_ptr);
    with local_aqb_ptr^do begin
         aqb$w_size := aqb$c_length;
         aqb$b_type := dyn$c_aqb;
         .
         .
         .
    As before, the COPY SYSTEM CONTROL BLOCK proce-

dure is used. However, this time the local block is copied to the non-paged pool.

    return_code :=
         copy_system_control_block
               (local_aqb_ptr ::  integer;
               nonpaged_aqb_ptr ::  integer);

The main program

The following code is the main processing loop of the ACP; during each iteration, it either gets another IRP from the ACP queue or another Remote Data Packet (RDP) returning from the link. If either process link data or process request packet is executed during an iteration, the io performed flag is set. If neither is executed, then the process will hibernate until new data is received from either side of the interface and a $WAKE is issued.

                                                        repeat
                                                             io_performed := false;
                                                             if link_data_present
                                                                  then process_link_data;
                                                             if irp_present
                                                                  then process_request_packet;
                                                             if not io_performed  then
                                                                  hibernate_until_new_input;
                                                        until dead;

Reading the IRPs

I/O Request Packets (IRPs) are passed from the simulated terminal driver to the ACP via a linked list (queue) in the non-paged pool. Since the user-mode ACP cannot access the pool directly, a subroutine in the dispatcher is used to get the address of the next IRP in the queue.

                                                        get_irp_from_queue(nonpaged_aqb_ptr,
                                                                       nonpaged_irp_ptr);

Using a technique similar to that of the PCB processing described earlier, the ACP creates a local IRP in the normal program space and uses the COPY SYSTEM CONTROL BLOCK procedure to copy the inaccessible real IRP to the local duplicate.

                                                        new(local_irp_ptr);
                                                        return_code :=
                                                             copy_system_control_block
                                                                  (nonpaged_irp_ptr ::  integer,
                                                                  local_irp_ptr ::  integer);

The I/O buffer associated with the IRP, which must be transmitted across the link and is in the form of a Remote Data Packet (RDP), must also be copied from the non-paged pool to the process address space. Since the RDP is not a structured block, the COPY NONPAGED BUFF procedure must be used and the actual length of the buffer must be specified (the address and length are contained

                                                    in the IRP fields irp$l svapte and irp$w boff).
                                                        with local_irp_ptr^do begin
                                                             from_ptr := irp$l_svapte;
                                                             to_ptr := local_io_buff_ptr;
                                                             return_code := copy_nonpaged_buff
                                                                  (irp$w_boff,irp$w_boff,
                                                                  from_ptr ::  integer,
                                                                  to_ptr ::  integer);

Since multiple I/O operations may be in progress simultaneously and they may not complete in the same order they were initiated, we must save the IRP in a local table, identified by sequence number (this number is guaranteed by VMS to be unique). The RDP is then transmitted to the remote system where the I/O will actually be performed.

    put_irp_on_local_queue(irp$l_seqnum);
    send_rdp_across_link;

Posting the I/O complete

When the response RDP returns from the link, now in a different format consisting of the completion code of the I/O and possibly containing data (in the case of a read operation), the original QIO must be posted complete. Using the sequence number, we locate the IRP associated with this RDP. COPY NONPAGED BUFF is again used, this time to overlay the original RDP in the system space with the response RDP just received (the block was originally allocated by the driver to be large enough for either form of an RDP, so we do not have to worry about overwriting system space).

Finally, we call a routine in the dispatcher to post the I/O operation complete.

    return_code :=
        post_io_completion(nonpaged_irp_ptr,
        nonpaged_ucb,
        interrupt_routine_address);

Procedures in the dispatcher

The following code in the change-mode-to-kernel dispatcher for the ACP returns the address of the next IRP in the ACP queue, if one is present; if the queue is empty, a warning-level return code is issued.

            $ent    get_irp_from_queue
            movl    4(ap),r5
            remque  @(r5),@8(ap)
            bvc     10$
            movl    #queue_is_empty,r0
            ret
    10$:    movl    #ss$_normal,r0
            ret

It cannot be over-emphasized that the process is only in kernel mode for the amount of time needed to execute these tiny procedures in the dispatcher. The time window where the VAX is vulnerable to a full-system crash is very small, and procedures this size are comparatively easy to debug.

Conclusions

• Some classes of systems programs can be implemented in a high-level language

• Systems programs can gain the benefits of structured programming and can be made more readable

• Development time can be shortened

• System crashes can be reduced during development • Kernel mode routines (in a change-mode-to-kernel dispatcher) developed for one system program can be used again for another