HP OpenVMS Guide to System Security > Appendix C Running an OpenVMS System in a C2 Environment

Trusted Computing Base (TCB) for C2 Systems

  Table of Contents



The federal government's evaluation of a computer system measures the trusted computing base (TCB) against the criteria summarized in “Definition of the C2 Environment”. The TCB is a combination of computer hardware and an operating system that enforces a security policy.

Hardware in the TCB

The architectural design of VAX processors prevent competing programs from interfering with the data of another program. VAX hardware prevents one program from interfering with the memory of another program.

The security features described in this guide apply to any VAX processor in the evaluated hardware configurations and to all supported mass storage and communications devices. The Final Evaluation Report, Digital Equipment Corporation, OpenVMS VAX and SEVMS Version 6.1 provides a full listing of the evaluated hardware.

Software in the TCB

In OpenVMS operating systems, the TCB encompasses much of the operating system. It includes the entire executive and file system, all other system components that do not execute in user mode (such as device drivers, RMS, and DCL), most system programs installed with privilege, and a variety of other utilities used by system managers to maintain data relevant to the TCB.

As a convenience to customers, the OpenVMS operating system ships with more than the base operating system. The software package includes save sets and supportive images for layered products typically run on OpenVMS operating systems. Yet only the base operating system was evaluated as a C2 system. Layered products, such as DECwindows software and Display PostScript support, were not part of the evaluation. For this reason, the C2 rating does not extend to OpenVMS VAX systems running the software listed in Table C-1 “Software Not Included in the C2-Evaluated System”. The exclusion of these software components in no way implies they are insecure; it only means that they were not part of the evaluated system. After the introduction of any such software, the base system must be accredited for its particular usage.

Table C-1 Software Not Included in the C2-Evaluated System

Software Function Description

DECwindows software

Windowing interface

DECwindows is a layered product. Although DECwindows has been designed to meet the C2 requirements, it has not been evaluated.

DECdns distributed name service

Client support

DECdns software requires server software, which is a layered product. A cluster can make DECnet connections independently of DECdns.

HP DECamds software

Monitoring and diagnostics

HP DECamds software is outside the domain of the evaluated configuration.

LASTport and LASTport/DISK protocols

Protocol support

HP's Infoserver products, which are outside the security domain of a clustered system, depend on these protocols.

LAT protocol

Protocol support

The LAT protocol is used for connections to DECserver terminal servers, which are outside the domain of the evaluated configuration.

DECnet/OSI Full Names

Protocol support

Support of the use of DECnet/OSI (Phase V) node names within the OpenVMS operating system. Use of this feature is not in the C2 evaluated configuration.

HSM (Hierarchial Shelving Manager)

Storage Support

File Shelving is a layered product. Use of the File Shelving facility (HSM) is not supported in the C2 evaluated configuration.

MME (Media Management Extension)

Client Support

Media Management Extension (MME) allows the use of storage media programs. Use of media management is outside of the domain of the C2 evaluated configuration.

OpenVMS Management Station


The OpenVMS Management Station provides PC-based system management tools for OpenVMS. The OpenVMS Management Station has not been validated in a C2 evaluated configuration.

Access control strings

File access on a remote node

You should use proxy accounts instead of access control strings in an evaluated configuration.


Protecting Objects

The OpenVMS operating system controls access to objects that contain information. Protected objects include ODS-2 or ODS-5 disk files, common event flag clusters, devices, all group and system global sections, logical name tables, queues, resource domains, and ODS-2 or ODS-5 disk volumes. The capability object and the security class object enjoy full discretionary access protection but they are not objects according to the C2 evaluation criteria.

Chapter 4 “Protecting Data”Chapter 4 and Chapter 5 “Descriptions of Object Classes”Chapter 5 describe object protection and explain how the operating system provides template profiles so all new objects have UICs, protection codes and, possibly, ACLs. “Establishing an Inheritance Scheme for Files”“Establishing an Inheritance Scheme for Files” on page 78, “Providing a Default Protection Code for a Directory Structure”See “Providing a Default Protection Code for a Directory Structure” on page 85, and “Setting Default Protection and Ownership”, in particular, explain how to set default protection for newly created objects.

The default protections assigned to global section and mailbox objects are less restrictive than those assigned to other objects. This is due to the fact that certain software products assume that mailbox and global section objects are created, by default, with the less restrictive protections. You can modify the template profiles for these objects so they have more stringent protection, but do keep in mind that some software products may be adversely affected.

To change the default protection, you need to modify both the template profile for the object and any existing object. For example, the following command modifies the MAILBOX template for the device class:


The operating system applies this value to all new mailboxes. The protection on each existing mailbox still has to be made more restrictive using the SET SECURITY command. For example:

_$ /PROTECTION=(S:RWPL,O:RWPL,G,W) mailbox_name

The default object protections specified in security templates survive system shutdown and reboot, so rebooting the system automatically ensures that all objects created after the reboot are created with the new default protections unless an object's creator specifies an alternate protection.

Protecting the TCB

The code and data that make up the OpenVMS TCB reside in files and, in part, in the address space of the running operating system. They are protected by the use of file access controls and memory page protection. Memory page protection is set up by the operating system as it executes and is normally not of concern to the system manager.

Protecting Files

The files that make up the TCB are correctly protected when the operating system is installed; however, sufficiently privileged users can alter the protection. Appendix B “Protection for OpenVMS System Files” of this guide describes the correct file protection of operating system files.

When installing an OpenVMS operating system, avoid modifying any system files except those specific to your site. You want to maintain the security of the base operating system.

Privileges for Trusted Users

Certain privileges allow the holder to bypass normal file and memory access controls directly or indirectly and, therefore, must not be granted to persons other than the system manager, security administrator, or other trusted users. Privileges in four categories are appropriate only for trusted users: Objects, All, System, and Group. Refer to Table 8-2 “OpenVMS Privileges” for the privileges belonging to each of these categories. The privileges themselves are described in detail in Appendix A “Assigning Privileges”.

Privileges in the Objects and All categories allow the holder to violate the isolation of the TCB from untrusted users. Privileges in the System category allow the holder to interfere with normal system operation and cause denial of service, but they do not allow the holder to actually violate object access controls. Some privileges in the System category also allow access controls to be ultimately bypassed.

Privileges in the Group category permit the holder to interfere with the operations of others in the same group. The GRPPRV privilege, in particular, permits the holder to violate normal access controls within that holder's group because it grants access (through the system field of the protection code) to objects owned by subjects sharing the same group UIC.

All trusted users should be familiar with all the effects of any operations they perform. In particular, they need to know all software products an operation might use because a trusted user's privileges can allow untrusted software to perform operations that OpenVMS security policy would otherwise preclude.

Privileges for Untrusted Users

Untrusted users can hold any privilege in the Normal and Devour category with the exception of GRPNAM. Exercise caution in granting privileges from the Devour category, however, for they permit the holder to consume resources without limit, thereby causing possible denial of service and interference with the operations of other users on the system. Table C-2 “Privileges for Untrusted Users” lists privileges allowed to untrusted users.

Table C-2 Privileges for Untrusted Users

Category Privilege Activity Permitted



Create network connections Create temporary mailbox



Disable accounting Allocate spooled devices Make bugcheck error log entries Exceed disk quotas Create/delete permanent common event flag clusters Create permanent global sections Create permanent mailboxes Create/delete structures in shared memory


Physical Security

Physical and environmental security are critical to the secure operation of the system. All physical components of the TCB require adequate protection, or unauthorized people can jeopardize the system's security. Because the following practices and features jeopardize the security of the TCB, they must not be used in a C2 environment:

  • Do not put the console terminal in a public area. The console terminal must always be physically secured because it controls operation of the CPU and, consequently, operation of the system.

  • Do not leave the console password disabled if the console has the password feature. (It is available on some VAXstation 3100s, most later models, and the evaluated Alpha models.) The console password prevents unauthorized personnel from using commands to boot from alternate media, to perform a conversational boot, or to modify memory.

  • Do not allow modems. Modems provide an avenue into the trusted system, and the possibilities for compromising system security are enormous.

  • Do not leave remote diagnostics enabled. Remote diagnostics provide another avenue into the trusted system. Disable remote diagnostics by placing the diagnostics switch in the off position.

  • Do not allow authentication cards. These devices are not supported in a C2 evaluated configuration.

  • Do not permit physical access to cluster communication media. Intruders can penetrate the system if they have physical access to any processor or cable.

    The operating system protects all communications interfaces against world access by default. This includes the CI and local area network (LAN) devices, such as the Ethernet, DSSI, FDDI, and SCSI. The CI interface is a trusted interface among members of a CI cluster and is inaccessible to unprivileged users. Unprivileged users should not be granted access to LAN devices.

  • Do not allow untrusted users to access the HSC console. Place the console in an area where only authorized personnel can use it. You do not want untrusted users to perform sensitive operations, such as backing up and restoring disk volumes.

  • Do not allow users to read printer output of other users. Protect printer output so users have access only to their own data.

  • Do not leave storage media, such as disks, tapes, and compact discs, where unauthorized users can access it. Once users have the media in their possession, they can read and modify its contents.

Configuring a C2 System

This section discusses C2 constraints on the use of OpenVMS features. It includes the following topics:

  • Requirements for maintaining individual accountability

  • Correct management of the audit log file

  • Correct use of terminals, volumes, and printers

  • Cluster requirements

  • Required settings for system parameters

  • Commands and software excluded from system operation

Keeping Individuals Accountable

The proper use of names, UICs, and passwords ensures that individual accountability is enforced by the OpenVMS operating system. As a general practice, HP recommends that you use generated passwords on privileged accounts. Because the following practices and features result in the loss of individual accountability, they must not be used in a C2 environment:

  • Do not assign the same UIC to more than one user. The UIC is used as the universal internal user identifier; therefore, unique UICs must be assigned to all users.

  • Do not allow open accounts. Lack of a password makes an account available to all users aware of its identity. The system manager can prevent open accounts by never setting null passwords with the Authorize utility (AUTHORIZE) and by ensuring that all accounts are set up with a nonzero minimum password length.

  • Do not allow group accounts. Individual accountability is lost when more than one person shares an account. Each user must be given a unique account.

  • Do not allow guest accounts because they allow multiple users access to resources on your system through a common account. Most needs for a guest account can be handled by special proxy login accounts.

  • Do not enable autologin. The automatic login facility (ALF) associates an account with a particular terminal instead of a particular person and, therefore, causes a loss of individual accountability.

  • Do not initiate network proxy accounts for groups. In order to preserve individual accountability, each individual in a network must be given a unique network proxy account on each node to which that user has access. Assign the same user name and UIC on all applicable nodes, and then set up individual proxies among the corresponding accounts.

  • Do not grant privileged access to proxy accounts.

  • Do not grant the DBG$ENABLE_SERVER identifer in the rights database unless it is needed to run the debug server.

  • Do not log operator HSC activities to a video terminal. You must use a hardcopy printer to log operator activities so it is possible to associate a specific system operation with the person performing it.

  • Ensure users are familiar with the restrictions on the use of access control strings in the evaluated configuration. (See page 3-15 in the SFUG.) Specifically, the use of access control strings is not permitted in an evaluated configuration. The proxy login accounts should be used in the evaluated configuration.

  • Do not allow operators to perform any task from the HSC console without signing the operator log. The sign-in log is required to track who performed HSC console operations and when. Together with the hardcopy output, the log provides a record of HSC operations.

Managing the Auditing Trail

The security-auditing system lets you track security-relevant activity on the system provided you manage it correctly. To follow a trail of activity in the audit logs, you must have complete and accurate records. Security event messages can be recorded in the security audit log file and on any terminal designated to receive security-class event messages. Because the following practices jeopardize a site's ability to track security-relevant events in the system, they must not be used in a C2 environment:

  • Do not disable the audit server or OPCOM. The audit server must be running to process audit event messages, and OPCOM is required to deliver alarms.

  • Do not use multiple audit log files in a cluster. You must use the clusterwide audit log file, which the system establishes by default. Without this clusterwide file, it is difficult to show the precise relationship among events that occur on various cluster nodes during any given time period.

  • Do not use a video terminal as a security operator terminal. You must enable a hardcopy terminal to receive security event messages.

  • Do not place the security operator terminal in a public location. Physically secure the terminal so that only authorized personnel have access to it.

  • Do not ignore the audit log file. You must review the security audit log file regularly for all audit events. In particular, notice whether any auditing modifications have been made. (Any use of the SET AUDIT command indicates some modification has taken place.) The audit log file is normally protected against reading or modification by unauthorized users.

  • Do not allow tampering with the audit log file. Always place security-auditing ACEs on the system security audit log file to enable auditing of all attempts to modify or delete the audit log file.

    For example:


    The operating system audits ACL events by default. You can verify this setting with the DCL command SHOW AUDIT. If necessary, reenable ACL alarms and audits with the following command:

  • Do not allow trusted users to operate without supervision. You should audit the actions of trusted users (such as operators, managers, and security administrators) by enabling auditing of changes to the authorization database. Also place security-auditing ACEs on captive login command procedures and the directories containing them so you can detect modifications.

Reusing Objects

Before allocating memory or protected objects like volumes and devices to new users, sites must ensure that they are free of old data. The memory management subsystem protects against the reuse of system memory pages, and it cannot be defeated. Because the following practices jeopardize the clearing of old data from volumes and terminals before reallocation, they must not be followed in a C2 environment:

  • Do not disable high-water marking on system disk volumes. The high-water marking and erase-on-delete features of the operating system protect against reuse of disk blocks (see “Protecting Disks”).

  • Do not allow users to leave their terminals on after logging out. They must turn off their terminals so the logout message is erased. The logout message reveals a user name and sometimes a node name. Moreover, by turning off the terminal, terminal characteristics are reset, and memory buffers are cleared. Some Trojan horse attacks use hardware frame buffers and the answerback capabilities that are built into newer terminals.

  • Do not recycle tape volumes to new users until the tapes have been erased externally by operations personnel. The operating system provides no protection against reuse of tape volumes. (This is because the OpenVMS operating system considers tape drives to be single-user devices. It provides tape protection only at the volume level; an entire volume can be assigned ownership and protection but individual files on the volume cannot.)

HP recommends that sites clear printers between jobs to ensure that print jobs do not interfere with one another. A security administrator can reset printers automatically at the start or end (or both) of each job by associating a device control library with the print queue. Consult the documentation supplied with your printer to determine the appropriate reset sequence, and then refer to the HP OpenVMS System Manager's Manual for directions on adding that sequence to a library and associating the library with the queue.

Configuring Clusters

All valid cluster configurations, when configured as common environment clusters, fully support the OpenVMS security features. Because the following practices and features result in the loss of a common environment cluster, they must not be used in a C2 environment.

NOTE: OpenVMS clusters can consist of VAX and Alpha nodes.
  • Do not operate with multiple authorization databases or audit log files. A clustered system is considered a single security and management domain and must operate with a shared authorization database and a single audit log file. If you have multiple system disks for performance reasons, system managers should ensure that the system files are identical.

    The following files must be shared across all cluster members:















  • Do not attach nodes to the cluster that are not part of the evaluated system. The evaluated OpenVMS configuration includes DECnet software bounded to the cluster environment that is a single security domain. All physically attached nodes must be part of the evaluated system.

Starting Up and Operating the System

A C2 system is the shipped system that has been configured according to the guidelines in this appendix. When configuring your system, you must observe the following guidelines:

  • Set security-sensitive parameters to the following values:

    System Parameter Setting Description



    Disables use of LOGINOUT callouts



    Disables site-specific password filters



    Sets the maximum UIC value for the system category to single-digit UICs



    Disables use of a conversational system bootstrap



    Sets a default protection code for user's files of S:RWED,O:RWED,G,W



    Disables certain unevaluated operating system components



    Disables the minimum sequence of the startup procedure

  • Do not use the CONNECT CONSOLE command to connect to a console storage device, except on a VAX 9000 system. On a VAX 9000 system, use the console command SET SPU_UPDATE OFF to isolate the storage device. Some console subsystems support a storage device, such as a tape or disk, that is used to load system and diagnostic programs; however, the operating system also supports the capability to read and write data on a console storage device, so it is neccessary to isolate the console storage device from the system. This command is not available on the evaluated Alpha platforms.

  • Do not enable console operations by booting with FYDRIVER. FYDRIVER would make two DCL commands operative:

    • SET HOST/HSC allows a user to initiate certain HSC console operations from an OpenVMS node

    • SET HOST/DUP is used for configuring DSSI devices

    If you need to install FYDRIVER during system startup to configure your HSC devices and disks or perform necessary diagnostics, then perform a minimum boot and install FYDRIVER so you can configure devices and so on. Then shut down the system and reboot without FYDRIVER.

Forcing Immediate Reauthentication of a Specified Subject After a Change in Access Rights

A system or security administrator may force untrusted subjects to reauthenticate themselves at any time. This might be necessary when the subject's access rights have been modified. The procedure is as follows and can be performed only by a trusted subject.

NOTE: This procedure assumes that there are no privileged applications present on the system that would enable an untrusted user to create a detached process.

Additionally, this procedure is not suitable for forcing reauthentication of trusted or privileged users, or where privileged applications are used. In these cases, a system reboot is required to adequately force reauthentication.

  1. Make the changes to the subject's authorization record in the authorization file.

  2. Obtain the owner's UIC of the subject from the authorization file.

  3. Enter the SYSMAN utility.

  4. Use the SYSMAN utility to identify all processes owned by the subject.

    1. In an OpenVMS Cluster environment, set the SYSMAN environment clusterwide. If you are not in an OpenVMS Cluster environment, skip this step.

    2. Use SYSMAN DO SHOW SYSTEM/FULL to obtain a listing of all processes on the system or OpenVMS cluster. This command also lists the owner UIC and system PID of each process. Record this information.

  5. From SYSMAN, stop every process on every system that is owned by the subject.

    Note: Any process created by the subject after Step 4 is bound by the new access rights and does not need to be deleted. Therefore, this is not a recursive procedure.

    1. In the OpenVMS Cluster environment, set the SYSMAN environment to point to only one node. If you are not in the OpenVMS Cluster environment, skip this step.

    2. For each process on the system to be deleted, identify the PID from Step 2 and use the SYSMAN DO STOP/ID=pid command to stop the job.

    3. Repeat Steps a and b until all desired processes on all nodes of the cluster have been stopped.