HP OpenVMS systems documentation

Each DSSI interconnect can have up to eight nodes attached; four can be systems and the rest can be storage devices. Figure 10-8, Figure 10-9, and Figure 10-10 show a progression from a two-node DSSI OpenVMS Cluster to a four-node DSSI OpenVMS Cluster.

10.4.1 Two-Node DSSI OpenVMS Cluster

In Figure 10-8, two nodes are connected to four disks by a common DSSI interconnect.

Figure 10-8 Two-Node DSSI OpenVMS Cluster

The advantages and disadvantages of the configuration shown in Figure 10-8 include:

Advantages

• Both nodes have shared, direct access to all storage.
• The Ethernet LAN ensures failover capability if the DSSI interconnect fails.

Disadvantages

• The amount of storage that is directly accessible to all nodes is limited.
• A single DSSI interconnect can become a single point of failure.

If the OpenVMS Cluster in Figure 10-8 required more processing power, more storage, and better redundancy, this could lead to a configuration like the one shown in Figure 10-9.

10.4.2 Four-Node DSSI OpenVMS Cluster with Shared Access

In Figure 10-9, four nodes have shared, direct access to eight disks through two DSSI interconnects. Two of the disks are shadowed across DSSI interconnects.

Figure 10-9 Four-Node DSSI OpenVMS Cluster with Shared Access

The advantages and disadvantages of the configuration shown in Figure 10-9 include:

Advantages

• All nodes have shared, direct access to all storage.
• The Ethernet LAN ensures failover capability if the DSSI interconnect fails.
• Shadowing across DSSI interconnects provides increased performance and availability.

Disadvantage

• The amount of storage that is directly accessible to all nodes is limited.

If the configuration in Figure 10-9 required more storage, this could lead to a configuration like the one shown in Figure 10-10.

10.4.3 Four-Node DSSI OpenVMS Cluster with Some Nonshared Access

Figure 10-10 shows an OpenVMS Cluster with 4 nodes and 10 disks. This model differs from Figure 10-8 and Figure 10-9 in that some of the nodes do not have shared, direct access to some of the disks, thus requiring these disks to be MSCP served. For the best performance, place your highest-priority data on disks that are directly connected by common DSSI interconnects to your nodes. Volume shadowing across common DSSI interconnects provides the highest availability and may increase read performance.

Figure 10-10 DSSI OpenVMS Cluster with 10 Disks

The advantages and disadvantages of the configuration shown in Figure 10-10 include:

Advantages

• All nodes have shared, direct access to most of the storage.
• The MSCP server is enabled to allow failover to the alternate DSSI interconnect if one of the DSSI interconnects fails.
• Shadowing across DSSI interconnects provides increased performance and availability.
• The SCSI storage connected through the HSZ controllers provides good performance and scalability.

Disadvantages

• The amount of storage that is directly accessible to all nodes is limited.
• Shadow set 2 requires MSCP serving to coordinate shadowing activity.
• Some nodes do not have direct access to storage. For example, Alpha 2 and Alpha 4 do not have direct access to disks connected to Alpha 1 and Alpha 3.

Each MEMORY CHANNEL (MC) interconnect can have up to four nodes attached to each MEMORY CHANNEL hub. For two-hub configurations, each node must have two PCI adapters, and each adapter must be attached to a different hub. In a two-node configuration, no hub is required because one of the PCI adapters serves as a virtual hub.

Figure 10-11, Figure 10-12, and Figure 10-13 show a progression from a two-node MEMORY CHANNEL cluster to a four-node MEMORY CHANNEL cluster.

Reference: For additional configuration information and a more detailed technical summary of how MEMORY CHANNEL works, see Appendix B.

10.5.1 Two-Node MEMORY CHANNEL Cluster

In Figure 10-11, two nodes are connected by a MEMORY CHANNEL interconnect, a LAN (Ethernet, FDDI, or ATM) interconnect, and a Fibre Channel interconnect.

Figure 10-11 Two-Node MEMORY CHANNEL OpenVMS Cluster

The advantages and disadvantages of the configuration shown in Figure 10-11 include:

Advantages

• Both nodes have shared, direct access to all storage.
• The Ethernet/FDDI/ATM interconnect enables failover if the MEMORY CHANNEL interconnect fails.
• The limit of two MEMORY CHANNEL nodes means that no hub is required; one PCI adapter serves as a virtual hub.

Disadvantages

• The amount of storage that is directly accessible to all nodes is limited.
• A single SCSI interconnect or HSZ controller can become a single point of failure.

If the OpenVMS Cluster in Figure 10-11 required more processing power and better redundancy, this could lead to a configuration like the one shown in Figure 10-12.

10.5.2 Three-Node MEMORY CHANNEL Cluster

In Figure 10-12, three nodes are connected by a high-speed MEMORY CHANNEL interconnect, as well as by a LAN (Ethernet, FDDI, or ATM) interconnect. These nodes also have shared, direct access to storage through the Fibre Channel interconnect.

Figure 10-12 Three-Node MEMORY CHANNEL OpenVMS Cluster

The advantages and disadvantages of the configuration shown in Figure 10-12 include:

Advantages

• All nodes have shared, direct access to storage.
• The Ethernet/FDDI/ATM interconnect enables failover if the MEMORY CHANNEL interconnect fails.
• The addition of a MEMORY CHANNEL hub increases the limit on the number of nodes to a total of four.

Disadvantage

• The amount of storage that is directly accessible to all nodes is limited.

If the configuration in Figure 10-12 required more storage, this could lead to a configuration like the one shown in Figure 10-13.

10.5.3 Four-Node MEMORY CHANNEL OpenVMS Cluster

Figure 10-13, each node is connected by a MEMORY CHANNEL interconnect as well as by a CI interconnect.

Figure 10-13 MEMORY CHANNEL Cluster with a CI Cluster

The advantages and disadvantages of the configuration shown in Figure 10-13 include:

Advantages

• All nodes have shared, direct access to all of the storage.
• This configuration has more than double the storage and processing capacity as the one shown in Figure 10-12.
• If the MEMORY CHANNEL interconnect fails, the CI can take over internode communication.
• The CIPCA adapters on two of the nodes enable the addition of Alpha systems to a CI cluster that formerly comprised VAX (CIXCD-based) systems.
• Multiple CIs between processors and storage provide twice the performance of one path. Bandwidth further increases because MEMORY CHANNEL offloads internode traffic from the CI, enabling the CI to be devoted only to storage traffic. This improves the performance of the entire cluster.
• Volume shadowed, dual-ported disks increase data availability.

Disadvantage

• This configuration is complex and requires the care of an experienced system manager.

SCSI-based OpenVMS Clusters allow commodity-priced storage devices to be used directly in OpenVMS Clusters. Using a SCSI interconnect in an OpenVMS Cluster offers you variations in distance, price, and performance capacity. This SCSI clustering capability is an ideal starting point when configuring a low-end, affordable cluster solution. SCSI clusters can range from desktop to deskside to departmental and larger configurations.

Note the following general limitations when using the SCSI interconnect:

• Because the SCSI interconnect handles only storage traffic, it must always be paired with another interconnect for node-to-node traffic. In the figures shown in this section, MEMORY CHANNEL is the alternate interconnect; but CI, DSSI, Ethernet, and FDDI could also be used.
• Total SCSI cable lengths must take into account the system's internal cable length. For example, an AlphaServer 1000 rackmount uses 1.6 m of internal cable to connect the internal adapter to the external connector. Two AlphaServer 1000s joined by a 2 m SCSI cable would use 1.6 m within each system, resulting in a total SCSI bus length of 5.2 m.
  Reference: For more information about internal SCSI cable lengths as well as highly detailed information about clustering SCSI devices, see Appendix A.
• This capability is available only for older SCSI adapters. The newest SCSI adapters, such as the KZPEA, the A6828A, the A6829A, and the A7173A, do not support this feature.

The figures in this section show a progression from a two-node SCSI configuration with modest storage to a four-node SCSI hub configuration with maximum storage and further expansion capability.

10.6.1 Two-Node Fast-Wide SCSI Cluster

In Figure 10-14, two nodes are connected by a 25-m, fast-wide differential (FWD) SCSI bus, with MEMORY CHANNEL (or any) interconnect for internode traffic. The BA356 storage cabinet contains a power supply, a DWZZB single-ended to differential converter, and six disk drives. This configuration can have either narrow or wide disks.

Figure 10-14 Two-Node Fast-Wide SCSI Cluster

The advantages and disadvantages of the configuration shown in Figure 10-14 include:

Advantages

• Low cost SCSI storage is shared by two nodes.
• With the BA356 cabinet, you can use a narrow (8 bit) or wide (16 bit) SCSI bus.
• The DWZZB converts single-ended signals to differential.
• The fast-wide SCSI interconnect provides 20 MB/s performance.
• MEMORY CHANNEL handles internode traffic.
• The differential SCSI bus can be 25 m.

Disadvantage

• Somewhat limited storage capability.

If the configuration in Figure 10-14 required even more storage, this could lead to a configuration like the one shown in Figure 10-15.

10.6.2 Two-Node Fast-Wide SCSI Cluster with HSZ Storage

In Figure 10-15, two nodes are connected by a 25-m, fast-wide differential (FWD) SCSI bus, with MEMORY CHANNEL (or any) interconnect for internode traffic. Multiple storage shelves are within the HSZ controller.

Figure 10-15 Two-Node Fast-Wide SCSI Cluster with HSZ Storage

The advantages and disadvantages of the configuration shown in Figure 10-15 include:

Advantages

• Costs slightly more than the configuration shown in Figure 10-14, but offers significantly more storage. (The HSZ controller enables you to add more storage.)
• Cache in the HSZ, which also provides RAID 0, 1, and 5 technologies. The HSZ is a differential device; no converter is needed.
• MEMORY CHANNEL handles internode traffic.
• The FWD bus provides 20 MB/s throughput.
• Includes a 25 m differential SCSI bus.

Disadvantage

• This configuration is more expensive than the one shown in Figure 10-14.

In Figure 10-16, three nodes are connected by two 25-m, fast-wide (FWD) SCSI interconnects. Multiple storage shelves are contained in each HSZ controller, and more storage is contained in the BA356 at the top of the figure.

Figure 10-16 Three-Node Fast-Wide SCSI Cluster

The advantages and disadvantages of the configuration shown in Figure 10-16 include:

Advantages

• Combines the advantages of the configurations shown in Figure 10-14 and Figure 10-15:
  • Significant (25 m) bus distance and scalability.
  • Includes cache in the HSZ, which also provides RAID 0, 1, and 5 technologies. The HSZ contains multiple storage shelves.
  • FWD bus provides 20 MB/s throughput.
  • With the BA356 cabinet, you can use narrow (8 bit) or wide (16 bit) SCSI bus.

Disadvantage

• This configuration is more expensive than those shown in previous figures.

Figure 10-17 shows four nodes connected by a SCSI hub. The SCSI hub obtains power and cooling from the storage cabinet, such as the BA356. The SCSI hub does not connect to the SCSI bus of the storage cabinet.

Figure 10-17 Four-Node Ultra SCSI Hub Configuration

The advantages and disadvantages of the configuration shown in Figure 10-17 include:

Advantages

• Provides significantly more bus distance and scalability than the configuration shown in Figure 10-15.
• The SCSI hub provides fair arbitration on the SCSI bus. This provides more uniform, predictable system behavior. Four CPUs are allowed only when fair arbitration is enabled.
• Up to two dual HSZ controllers can be daisy-chained to the storage port of the hub.
• Two power supplies in the BA356 (one for backup).
• Cache in the HSZs, which also provides RAID 0, 1, and 5 technologies.
• Ultra SCSI bus provides 40 MB/s throughput.

Disadvantage

• You cannot add CPUs to this configuration by daisy-chaining a SCSI interconnect from a CPU or HSZ to another CPU.
• This configuration is more expensive than those shown in Figure 10-14 and Figure 10-15.
• Only HSZ storage can be connected. You cannot attach a storage shelf with disk drives directly to the SCSI hub.

The number of satellites in an OpenVMS Cluster and the amount of storage that is MSCP served determine the need for the quantity and capacity of the servers. Satellites are systems that do not have direct access to a system disk and other OpenVMS Cluster storage. Satellites are usually workstations, but they can be any OpenVMS Cluster node that is served storage by other nodes in the OpenVMS Cluster.

Each Ethernet LAN segment should have only 10 to 20 satellite nodes attached. Figure 10-18, Figure 10-19, Figure 10-20, and Figure 10-21 show a progression from a 6-satellite LAN to a 45-satellite LAN.

10.7.1 Six-Satellite OpenVMS Cluster

In Figure 10-18, six satellites and a boot server are connected by Ethernet.

Figure 10-18 Six-Satellite LAN OpenVMS Cluster

The advantages and disadvantages of the configuration shown in Figure 10-18 include:

Advantages

• The MSCP server is enabled for adding satellites and allows access to more storage.
• With one system disk, system management is relatively simple.
  Reference: For information about managing system disks, see Section 11.2.

Disadvantage

• The Ethernet is a potential bottleneck and a single point of failure.

If the boot server in Figure 10-18 became a bottleneck, a configuration like the one shown in Figure 10-19 would be required.

10.7.2 Six-Satellite OpenVMS Cluster with Two Boot Nodes

Figure 10-19 shows six satellites and two boot servers connected by Ethernet. Boot server 1 and boot server 2 perform MSCP server dynamic load balancing: they arbitrate and share the work load between them and if one node stops functioning, the other takes over. MSCP dynamic load balancing requires shared access to storage.

Figure 10-19 Six-Satellite LAN OpenVMS Cluster with Two Boot Nodes

The advantages and disadvantages of the configuration shown in Figure 10-19 include:

Advantages

• The MSCP server is enabled for adding satellites and allows access to more storage.
• Two boot servers perform MSCP dynamic load balancing.

Disadvantage

• The Ethernet is a potential bottleneck and a single point of failure.

If the LAN in Figure 10-19 became an OpenVMS Cluster bottleneck, this could lead to a configuration like the one shown in Figure 10-20.

10.7.3 Twelve-Satellite LAN OpenVMS Cluster with Two LAN Segments

Figure 10-20 shows 12 satellites and 2 boot servers connected by two Ethernet segments. These two Ethernet segments are also joined by a LAN bridge. Because each satellite has dual paths to storage, this configuration also features MSCP dynamic load balancing.

Figure 10-20 Twelve-Satellite OpenVMS Cluster with Two LAN Segments

The advantages and disadvantages of the configuration shown in Figure 10-20 include:

Advantages

• The MSCP server is enabled for adding satellites and allows access to more storage.
• Two boot servers perform MSCP dynamic load balancing.
  From the perspective of a satellite on the Ethernet LAN, the dual paths to the Alpha and VAX nodes create the advantage of MSCP load balancing.
• Two LAN segments provide twice the amount of LAN capacity.

Disadvantages

• This OpenVMS Cluster configuration is limited by the number of satellites that it can support.
• The single HSJ controller is a potential bottleneck and a single point of failure.

If the OpenVMS Cluster in Figure 10-20 needed to grow beyond its current limits, this could lead to a configuration like the one shown in Figure 10-21.

10.7.4 Forty-Five Satellite OpenVMS Cluster with FDDI Ring

Figure 10-21 shows a large, 51-node OpenVMS Cluster that includes 45 satellite nodes. The three boot servers, Alpha 1, Alpha 2, and Alpha 3, share three disks: a common disk, a page and swap disk, and a system disk. The FDDI ring has three LAN segments attached. Each segment has 15 workstation satellites as well as its own boot node.

Figure 10-21 Forty-Five Satellite OpenVMS Cluster with FDDI Ring

The advantages and disadvantages of the configuration shown in Figure 10-21 include:

Advantages

• Decreased boot time, especially for an OpenVMS Cluster with such a high node count.
  Reference: For information about booting an OpenVMS Cluster like the one in Figure 10-21 see Section 11.2.4.
• The MSCP server is enabled for satellites to access more storage.
• Each boot server has its own page and swap disk, which reduces I/O activity on the system disks.
• All of the environment files for the entire OpenVMS Cluster are on the common disk. This frees the satellite boot servers to serve only root information to the satellites.
  Reference: For more information about common disks and page and swap disks, see Section 11.2.
• The FDDI ring provides 10 times the capacity of one Ethernet interconnect.

Disadvantages

• The satellite boot servers on the Ethernet LAN segments can boot satellites only on their own segments.
• FDDI is not supported on OpenVMS I64 systems.
• FDDI has largely been superseded by Gigabit Ethernet, which is significantly faster and less expensive.

Figure 10-22 shows an OpenVMS Cluster configuration that provides high performance and high availability on the FDDI ring.

Figure 10-22 High-Powered Workstation Server Configuration 1995

In Figure 10-22, several Alpha workstations, each with its own system disk, are connected to the FDDI ring. Putting Alpha workstations on the FDDI provides high performance because each workstation has direct access to its system disk. In addition, the FDDI bandwidth is higher than that of the Ethernet. Because Alpha workstations have FDDI adapters, putting these workstations on an FDDI is a useful alternative for critical workstation requirements. FDDI is 10 times faster than Ethernet, and Alpha workstations have processing capacity that can take advantage of FDDI's speed. (The speed of Fast Ethernet matches that of FDDI, and Gigabit Ethernet is 10 times faster than Fast Ethernet and FDDI.)

10.7.6 High-Powered Workstation OpenVMS Cluster (2004 Technology)

Figure 10-23 shows an OpenVMS Cluster configuration that provides high performance and high availability using Gigabit Ethernet for the LAN and Fibre Channel for storage.

Figure 10-23 High-Powered Workstation Server Configuration 2004

In Figure 10-23, several Alpha workstations, each with its own system disk, are connected to the Gigabit Ethernet LAN. Putting Alpha workstations on the Gigabit Ethernet LAN provides high performance because each workstation has direct access to its system disk. In addition, the Gigabit Ethernet bandwidth is 10 times higher than that of the FDDI. Alpha workstations have processing capacity that can take advantage of Gigabit Ethernet's speed.

10.7.7 Guidelines for OpenVMS Clusters with Satellites

The following are guidelines for setting up an OpenVMS Cluster with satellites:

• Extra memory is required for satellites of large LAN configurations because each node must maintain a connection to every other node.
• Configure network to eliminate bottlenecks (that is, allocate sufficient bandwidth within the network cloud and on server connections).
• Maximize resources with MSCP dynamic load balancing, as shown in Figure 10-19 and Figure 10-20.
• Keep the number of nodes that require MSCP serving minimal for good performance.
  Reference: See Section 10.8.1 for more information about MSCP overhead.
• To save time, ensure that the booting sequence is efficient, particularly when the OpenVMS Cluster is large or has multiple segments. See Section 11.2.4 for more information about how to reduce LAN and system disk activity and how to boot separate groups of nodes in sequence.
• Use multiple LAN adapters per host, and connect to independent LAN paths. This enables simultaneous two-way communication between nodes and allows traffic to multiple nodes to be spread over the available LANs. In addition, multiple LAN adapters increase failover capabilities.