| Title and Copyright Information |
| About This Manual |
| Audience |
| New and Changed Features |
| Organization |
| Related Documents |
| Reader's Comments |
| Conventions |
| 1 | Managing Clusters Overview |
| 1.1 | Commands and Utilities for Clusters |
| 1.2 | Commands and Features That Are Different in a Cluster |
| 2 | Tools for Managing Clusters |
| 2.1 | Overview of Available Management Tools and Interfaces |
| 2.1.1 | Cluster Tools Quick Start |
| 2.1.2 | HP Insight Manager Integration |
| 2.1.3 | HP Insight Manager XE Integration |
| 2.1.4 | Available Management Tools and Interfaces |
| 2.2 | Cluster Configuration Tools and Related User Interfaces |
| 2.3 | Overview of SysMan Components |
| 2.3.1 | Introduction to SysMan Menu |
| 2.3.2 | Introduction to SysMan Station |
| 2.3.3 | Introduction to the SysMan Command Line |
| 2.4 | Using SysMan Menu in a Cluster |
| 2.4.1 | Getting in Focus |
| 2.4.2 | Specifying a Focus on the Command Line |
| 2.4.3 | Invoking SysMan Menu |
| 2.5 | Using SysMan Station in a Cluster |
| 2.5.1 | Invoking SysMan Station |
| 2.6 | Using the SysMan Java Applets in a Cluster |
| 2.6.1 | Invoking the SysMan Java Applets |
| 2.7 | Using SysMan Java PC Applications in a Cluster |
| 2.7.1 | Invoking SysMan Java Applications on a PC |
| 2.8 | Using SysMan Command-Line Interface in a Cluster |
| 2.9 | Using HP Insight Manager in a Cluster |
| 2.9.1 | Invoking HP Insight Manager |
| 2.10 | Using Tru64 UNIX Configuration Report |
| 3 | Managing the Cluster Alias Subsystem |
| 3.1 | Summary of Cluster Alias Features |
| 3.2 | Configuration Files |
| 3.3 | General Planning Considerations |
| 3.4 | Things to Do Before You Create an Alias |
| 3.5 | Specifying and Joining a Cluster Alias |
| 3.6 | Modifying Cluster Alias and Service Port Attributes |
| 3.7 | Monitoring Cluster Aliases |
| 3.8 | Leaving a Cluster Alias |
| 3.9 | Deleting a Cluster Alias |
| 3.10 | Application Load Balancing |
| 3.11 | Extending Clusterwide Port Space |
| 3.12 | Enabling Cluster Alias vMAC Support |
| 3.13 | Routing Configuration Guidelines |
| 3.14 | Routing Daemon and Static Routing Options |
| 3.15 | Cluster Alias and NFS |
| 3.16 | Cluster Alias and Cluster Application Availability |
| 4 | Managing Cluster Membership |
| 4.1 | Overview of Connection Manager |
| 4.2 | Overview of Quorum and Votes |
| 4.2.1 | How a System Becomes a Cluster Member |
| 4.2.2 | Expected Votes |
| 4.2.3 | Current Votes |
| 4.2.4 | Node Votes |
| 4.2.5 | Quorum Disk Votes |
| 4.3 | Overview of Calculating Cluster Quorum |
| 4.4 | Connection Manager Example |
| 4.5 | Using a Quorum Disk |
| 4.5.1 | Replacing a Failed Quorum Disk |
| 4.6 | Using the clu_quorum Command to Display Cluster Vote Information |
| 4.7 | Cluster Vote Assignment Examples |
| 4.8 | Monitoring the Connection Manager |
| 4.9 | Connection Manager Panics |
| 4.10 | Troubleshooting Unfortunate Expected Vote and Node Vote Settings |
| 4.10.1 | Joining a Cluster After a Cluster Member or Quorum Disk Fails and Cluster Loses Quorum |
| 4.10.2 | Forming a Cluster When Members Do Not Have Enough Votes to Boot and Form a Cluster |
| 5 | Managing Cluster Members |
| 5.1 | Managing Configuration Variables |
| 5.2 | Managing Kernel Attributes |
| 5.3 | Managing Remote Access Within and From the Cluster |
| 5.4 | Shutting Down the Cluster |
| 5.5 | Shutting Down and Starting One Cluster Member |
| 5.5.1 | Identifying a Critical Voting Member |
| 5.5.2 | Preparing to Halt or Delete a Critical Voting Member |
| 5.5.3 | Halting a Noncritical Member |
| 5.5.4 | Shutting Down a Hosting Member |
| 5.6 | Shutting Down a Cluster Member to Single-User Mode |
| 5.7 | Rebooting Cluster Members |
| 5.8 | Deleting a Cluster Member |
| 5.9 | Removing a Cluster Member and Restoring It as a Standalone System |
| 5.10 | Moving a Cluster to Another IP Subnet |
| 5.10.1 | File Edit Tables |
| 5.10.1.1 | Changing External IP Addresses Only |
| 5.10.1.2 | Changing External IP Addresses and Host Names |
| 5.10.1.3 | Changing External and Internal IP Addresses and Host Names |
| 5.10.2 | Attribute and Checklist Tables |
| 5.10.2.1 | External Host Names and IP Addresses |
| 5.10.2.2 | Cluster Name and Cluster Aliases |
| 5.10.2.3 | Interface IP Aliases |
| 5.10.2.4 | External Servers |
| 5.10.2.5 | Checklist |
| 5.10.3 | Verifying Success |
| 5.10.4 | Troubleshooting |
| 5.11 | Changing the Cluster Name or IP Address |
| 5.11.1 | Changing the Cluster Name |
| 5.11.2 | Changing the Cluster IP Address |
| 5.12 | Changing the Member Name, IP Address, or Cluster Interconnect Address |
| 5.12.1 | Check Quorum Before Removing Member |
| 5.12.2 | Check CAA Restricted Placement Policies Before Removing Member |
| 5.12.3 | Remove and Add the Member |
| 5.13 | Managing Software Licenses |
| 5.14 | Installing and Deleting Layered Applications |
| 5.15 | Managing Accounting Services |
| 6 | Managing Networks in a Cluster |
| 6.1 | Updating |
| 6.2 | Providing Failover for Network Interfaces |
| 6.3 | Running IP Routers |
| 6.4 | Configuring the Network |
| 7 | Managing Network Services |
| 7.1 | Configuring DHCP |
| 7.2 | Configuring NIS |
| 7.2.1 | Configuring an NIS Master in a Cluster with Enhanced Security |
| 7.3 | Configuring Printing |
| 7.4 | Configuring DNS/BIND |
| 7.5 | Managing Time Synchronization |
| 7.5.1 | Configuring NTP |
| 7.5.2 | Using the Same External NTP Servers for All Members |
| 7.5.2.1 | Time Drift |
| 7.6 | Managing NFS |
| 7.6.1 | Configuring NFS |
| 7.6.2 | Considerations for Using NFS in a Cluster |
| 7.6.2.1 | CFS Support of NFS File Systems |
| 7.6.2.2 | Clients Must Use a Cluster Alias |
| 7.6.2.3 | Using CDSLs to Mount NFS File Systems |
| 7.6.2.4 | Loopback Mounts Not Supported |
| 7.6.2.5 | Do Not Mount Non-NFS File Systems on NFS-Mounted Paths |
| 7.6.3 | Using AutoFS in a Cluster |
| 7.6.3.1 | Forcibly Unmounting File Systems |
| 7.6.3.1.1 | Determining Whether a Forced Unmount Is Required |
| 7.6.3.1.2 | Correcting the Problem |
| 7.6.4 | Migrating from Automount to AutoFS |
| 7.6.4.1 | Migrating Without a Reboot |
| 7.6.4.2 | Migrating When Rebooting a Cluster Member |
| 7.6.4.3 | Migrating When Rebooting the Cluster |
| 7.7 | Managing inetd Configuration |
| 7.8 | Managing Mail |
| 7.8.1 | Configuring Mail |
| 7.8.1.1 | Mail Files |
| 7.8.1.2 | The Cw Macro (System Nicknames List) |
| 7.8.1.3 | Configuring Mail at Cluster Creation |
| 7.8.1.4 | Configuring Mail After the Cluster Is Running |
| 7.8.2 | Distributing Mail Load Among Cluster Members |
| 7.9 | Configuring a Cluster for RIS |
| 7.10 | Displaying X Window Applications Remotely |
| 8 | Managing Highly Available Applications |
| 8.1 | Learning the Status of a Resource |
| 8.1.1 | Learning the State of a Resource |
| 8.1.2 | Learning Status of All Resources on One Cluster Member |
| 8.1.3 | Learning Status of All Resources on All Cluster Members |
| 8.1.4 | Getting Number of Failures and Restarts and Target States |
| 8.2 | Reporting Resource Availability Measurements |
| 8.3 | Relocating Applications |
| 8.3.1 | Manual Relocation of All Applications on a Cluster Member |
| 8.3.2 | Manual Relocation of a Single Application |
| 8.3.3 | Manual Relocation of Dependent Applications |
| 8.4 | Starting and Stopping Application Resources |
| 8.4.1 | Starting Application Resources |
| 8.4.2 | Stopping Application Resources |
| 8.4.3 | No Multiple Instances of an Application Resource |
| 8.4.4 | Using caa_stop to Reset UNKNOWN State |
| 8.5 | Balancing Application Resources |
| 8.5.1 | Balancing All Applications on a Cluster |
| 8.5.2 | Balancing All Applications on a Member |
| 8.5.3 | Balancing Specific Applications |
| 8.5.4 | Balancing an Application at Specific Time |
| 8.6 | Registering and Unregistering Resources |
| 8.6.1 | Registering Resources |
| 8.6.2 | Unregistering Resources |
| 8.6.3 | Updating Registration |
| 8.7 | Network, Tape, and Media Changer Resources |
| 8.8 | Using SysMan to Manage CAA |
| 8.8.1 | Managing CAA with SysMan Menu |
| 8.8.1.1 | CAA Management Dialog Box |
| 8.8.1.2 | Start Dialog Box |
| 8.8.1.3 | Setup Dialog Box |
| 8.8.2 | Managing CAA with SysMan Station |
| 8.8.2.1 | Starting an Application with SysMan Station |
| 8.8.2.2 | Resource Setup with SysMan Station |
| 8.9 | CAA Considerations for Startup and Shutdown |
| 8.10 | Managing caad |
| 8.10.1 | Determining Status of the Local CAA Daemon |
| 8.10.2 | Restarting the CAA Daemon |
| 8.10.3 | Monitoring CAA Daemon Messages |
| 8.11 | Using EVM to View CAA Events |
| 8.11.1 | Viewing CAA Events |
| 8.11.2 | Monitoring CAA Events |
| 8.12 | Troubleshooting with Events |
| 8.13 | Troubleshooting a Command-Line Message |
| 9 | Managing File Systems and Devices |
| 9.1 | Working with CDSLs |
| 9.1.1 | Making CDSLs |
| 9.1.2 | Maintaining CDSLs |
| 9.1.3 | Kernel Builds and CDSLs |
| 9.1.4 | Exporting and Mounting CDSLs |
| 9.2 | Managing Devices |
| 9.2.1 | Managing the Device Special File |
| 9.2.2 | Determining Device Locations |
| 9.2.3 | Adding a Disk to the Cluster |
| 9.2.4 | Managing Third-party Storage |
| 9.2.5 | Tape Devices |
| 9.2.6 | Formatting Diskettes in a Cluster |
| 9.2.7 | CD-ROM and DVD-ROM |
| 9.3 | Managing the Cluster File System |
| 9.3.1 | When File Systems Cannot Fail Over |
| 9.3.2 | Direct Access Cached Reads |
| 9.3.3 | CFS Load Balancing |
| 9.3.3.1 | Starting and Stopping |
| 9.3.3.2 | EVM Events |
| 9.3.3.3 | Modifying the |
| 9.3.3.4 | Understanding |
| 9.3.3.5 | Automatic Relocations |
| 9.3.3.6 | Relationship to CAA Resources |
| 9.3.3.7 | Balancing CFS Load Without |
| 9.3.3.8 | Distributing CFS Server Load via |
| 9.3.4 | Distributing File Systems Via the |
| 9.3.5 | Freezing a Domain Before Cloning |
| 9.3.5.1 | Determining Whether a Domain Is Frozen |
| 9.3.6 | Optimizing CFS Performance |
| 9.3.6.1 | Changing the Number of Read-Ahead and Write-Behind Threads |
| 9.3.6.2 | Taking Advantage of Direct I/O |
| 9.3.6.2.1 | Differences Between Cluster and Standalone AdvFS Direct I/O |
| 9.3.6.2.2 | Cloning a Fileset with Files Open in Direct I/O Mode |
| 9.3.6.2.3 | Gathering Statistics on Direct I/O |
| 9.3.6.3 | Adjusting CFS Memory Usage |
| 9.3.6.4 | Using Memory Mapped Files |
| 9.3.6.5 | Avoiding Full File Systems |
| 9.3.6.6 | Other Strategies |
| 9.3.7 | MFS and UFS File Systems Supported |
| 9.3.8 | Partitioning File Systems |
| 9.3.9 | Block Devices and Cache Coherency |
| 9.3.10 | CFS Restrictions |
| 9.4 | Managing the Device Request Dispatcher |
| 9.4.1 | Direct-Access I/O and Single-Server Devices |
| 9.4.1.1 | Devices Supporting Direct-Access I/O |
| 9.4.1.2 | Replacing RZ26, RZ28, RZ29, or RZ1CB-CA as Direct-Access I/O Disks |
| 9.4.1.3 | HSZ Hardware Supported on Shared Buses |
| 9.5 | Managing AdvFS in a Cluster |
| 9.5.1 | Integrating AdvFS Files from a Newly Added Member |
| 9.5.2 | Create Only One Fileset in Cluster Root Domain |
| 9.5.3 | Adding Filesets to a Member's Boot Partition Not Recommended |
| 9.5.4 | Do Not Add a Volume to a Member's Root Domain |
| 9.5.5 | Using the addvol and rmvol Commands in a Cluster |
| 9.5.6 | User and Group File System Quotas Are Supported |
| 9.5.6.1 | Quota Hard Limits |
| 9.5.6.2 | Setting the |
| 9.5.7 | Storage Connectivity and AdvFS Volumes |
| 9.6 | Considerations When Creating New File Systems |
| 9.6.1 | Verifying Disk Connectivity |
| 9.6.2 | Looking for Available Disks |
| 9.6.2.1 | Looking for the Location of the Quorum Disk |
| 9.6.2.2 | Looking for the Location of Member Boot Disks and Clusterwide AdvFS File Systems |
| 9.6.2.3 | Looking for Member Swap Areas |
| 9.6.3 | Editing /etc/fstab |
| 9.7 | Managing CDFS File Systems |
| 9.8 | Backing Up and Restoring Files |
| 9.8.1 | Suggestions for Files to Back Up |
| 9.9 | Managing Swap Space |
| 9.9.1 | Locating Swap Device for Improved Performance |
| 9.10 | Fixing Problems with Boot Parameters |
| 9.11 | Using the verify Utility in a Cluster |
| 9.11.1 | Using the verify Utility on Cluster Root |
| 10 | Using Logical Storage Manager in a Cluster |
| 10.1 | Differences Between Managing LSM in Clusters and in Standalone Systems |
| 11 | Troubleshooting Clusters |
| 11.1 | Resolving Problems |
| 11.1.1 | Booting Systems Without a License |
| 11.1.2 | Shutdown Leaves Members Running |
| 11.1.3 | Environmental Monitoring and Cluster Shutdowns |
| 11.1.4 | Dealing with CFS Errors at Boot |
| 11.1.5 | Backing Up and Repairing a Member's Boot Disk |
| 11.1.5.1 | Example of Recovering a Member's Boot Disk |
| 11.1.6 | Specifying cluster_root at Boot Time |
| 11.1.7 | Recovering the Cluster Root File System to a Disk Known to the Cluster |
| 11.1.8 | Recovering the Cluster Root File System to a New Disk |
| 11.1.9 | Dealing with AdvFS Problems |
| 11.1.9.1 | Responding to Warning Messages from addvol or rmvol |
| 11.1.9.2 | Resolving AdvFS Domain Panics Due to Loss of Device Connectivity |
| 11.1.9.3 | Forcibly Unmounting an AdvFS File System or Domain |
| 11.1.9.4 | Avoiding Domain Panics |
| 11.1.10 | Accessing Boot Partitions on Down Systems |
| 11.1.11 | Booting a Member While Its Boot Disk Is Already Mounted |
| 11.1.12 | Generating Crash Dumps |
| 11.1.12.1 | Generating a Crash Dump When a Member Is Hung |
| 11.1.13 | Fixing Network Problems |
| 11.1.14 | Running routed in a Cluster |
| 11.2 | Hints for Managing Clusters |
| 11.2.1 | Moving /tmp |
| 11.2.2 | Running the MC_CABLE Console Command |
| 11.2.3 | Korn Shell Does Not Record True Path to Member-Specific Directories |
| A | Cluster Events |
| B | Configuration Variables |
| C | clu_delete_member Log |
| D | LAN Cluster Administration |
| D.1 | Configuring a NetRAIN Virtual Interface for a Cluster LAN Interconnect |
| D.2 | Tuning the LAN Interconnect |
| D.2.1 | Improving Cluster Interconnect Performance by Setting Its ipmtu Value |
| D.2.2 | Set Ethernet Switch Address Aging to 15 Seconds |
| D.3 | Obtaining Network Adapter Configuration Information |
| D.4 | Monitoring LAN Interconnect Activity |
| D.5 | Migrating from Memory Channel to LAN |
| D.6 | Migrating from LAN to Memory Channel |
| D.7 | Migrating from Fast Ethernet LAN to Gigabit Ethernet LAN |
| D.8 | Troubleshooting |
| D.8.1 | Many Broadcast Errors on Booting or Booting New Member Panics Existing Member |
| D.8.2 | Cannot Manually Fail Over Devices in a NetRAIN Virtual Interface |
| D.8.3 | Applications Unable to Map to Port |
| Examples |
| 2-1 | Example |
| 9-1 | Sample |
| Figures |
| 2-1 | SysMan Menu Hierarchy |
| 2-2 | SysMan Menu Interfaces |
| 2-3 | SysMan Station Graphical Interface |
| 2-4 | SysMan Station Initial Cluster View |
| 2-5 | Sample SysMan Station Cluster Hardware View |
| 2-6 | Displaying Available Actions in SysMan Station |
| 2-7 | HP Insight Manager Display |
| 2-8 | Sample Configuration Report Display |
| 4-1 | Three-Member deli Cluster |
| 4-2 | Three-Member deli Cluster Loses a Member |
| 4-3 | Two-Member deli Cluster Without a Quorum Disk |
| 4-4 | Two-Member deli Cluster with Quorum Disk Survives Member Loss |
| 8-1 | CAA Branch of SysMan Menu |
| 8-2 | CAA Management Dialog Box |
| 8-3 | Start Dialog Box |
| 8-4 | Setup Dialog Box |
| 8-5 | SysMan Station CAA_Applications_(active) View |
| 8-6 | SysMan Station CAA_Applications_(all) View |
| 8-7 | SysMan Station CAA Setup Screen |
| 9-1 | SysMan Station Display of Hardware Configuration |
| 9-2 | Cluster with Semi-private Storage |
| 9-3 | Four Node Cluster |
| Tables |
| 1-1 | Cluster Commands |
| 1-2 | File Systems and Storage Differences |
| 1-3 | Networking Differences |
| 1-4 | Printing Differences |
| 1-5 | Security Differences |
| 1-6 | General System Management Differences |
| 1-7 | Features Not Supported |
| 2-1 | Cluster Tools Quick Start |
| 2-2 | Available Management Tools and Interfaces |
| 2-3 | Cluster Management Tools |
| 2-4 | Invoking SysMan Menu |
| 2-5 | Invoking SysMan Station |
| 4-1 | Effects of Various Member cluster_expected_votes Settings and Vote Assignments in a Two- to Four-Member Cluster |
| 4-2 | Examples of Resolving Quorum Loss in a Cluster with Failed Members or Quorum Disk |
| 4-3 | Examples of Repairing a Quorum Deficient Cluster by Booting a Member with Sufficient Votes to Form the Cluster |
| 5-1 | /etc/rc.config* Files |
| 5-2 | Kernel Attributes Not to Decrease |
| 5-3 | Configurable TruCluster Server Subsystems |
| 8-1 | Target and State Combinations for Application Resources |
| 8-2 | Target and State Combinations for Network Resources |
| 8-3 | Target and State Combinations for Tape and Media Changer Resources |
| 9-1 | Sources of Information of Storage Device Management |
| 11-1 | Member Boot Disk Partitions |
| B-1 | Cluster Configuration Variables |
| Index |