<<< DOCD$:[NOTES$LIBRARY]SCT-GRYPHON.NOTE;1 >>> -< sct-gryphon >- ================================================================================ Note 2247.0 BUGFIX: Fix for backup saveset corruption (low A No replies EVMS::EWOODS "{VDE SCT}" 210 lines 12-SEP-1996 21:14 -------------------------------------------------------------------------------- Project: RMS Project Leader: Elinor M. Woods Reviewed by: Stu Davidson Development Stream: GRYPHON_FT2 Checkin Type: BugFix NOTE: Karen, this is the fix I spoke to you about that you asked me to add a note to indicate that you had approved it for checkin to GRYPHON_FT2. Problem Symptom: Backup saveset contains zeroed out blocks (missing VBNs). In other words, the saveset is corrupted. If the verify option is not used, no error is reported by backup and the user will not learn the saveset is corrupted until a restore is attempted at some later date. If the verify option is used, one or more of the following errors will be reported during the verification pass: %BACKUP-E-VBNMISSING, has missing blocks through Diagnosis: It took the field several months to identify the condition that triggers this problem: low ASTLM. One other factor that is associated with consistent reproduction of this problem, is an explicit blocksize specified for the backup that is equal to the RMS intermediate buffer size, which is determined by the multiblock count). In this case, /BLOCK=8192 was used for the backup, and the default RMS multiblock count of 16 blocks was used. In general, backup uses $QIOs. However, it does use an RMS $PUT for creating savesets. If an error is returned on a $PUT, it checks whether the system error returned by RMS in the RAB$L_STV is an SS$_EXQUOTA or SS$_EXASTLM error. If so, it sets up a timer and waits for it to complete, assuming that with a little idle time the AST quota will be replenished. It then reissues the $PUT for the record for which the error was returned. While the RMS $PUT only uses one AST, backup can easily exhaust a process's ASTLM because it issues a large number of asynchronous $QIO reads without any throttle. What makes the /BLOCK=8192 important is that the default size of the intermediate buffer used by RMS is also 8192 (16 blocks). Backup uses the blocksize as the size of the record it $PUTs. This results in this case in each record $PUT by backup exactly filling the RMS intermediate buffer. When the buffer is exactly full at the end of a $PUT, if the file is not being shared (exclusive), RMS defers the write to disk until the beginning of the next $PUT. However, at the end of the current $PUT, it advances the end-of-file and internal pointers for positioning the next record. When the next record is $PUT and the actual write ($QIO) is done of the full buffer, if the write fails due to no available ASTLM, backup tries to recover from the EXQUOTA error. It reissues the $PUT with the failing record. Since all the pointers have been advanced, the next record is actually put in the file where it should be but a gap is left out on disk for the previous record that filled buffer which failed to be written. The error being returned is associated with the previous record $PUT -- not the current one. In other words, by deferring the write back to disk of the full buffer, the error is out of sync with the $PUT. Cure: If the buffer is full at the end of a $PUT, handle exclusive case the same as currently done for shared case: release bucket instead of deferring release until the beginning of the next $PUT. If there is an error on the write ($QIO), this change results in the error being in sync with the current $PUT that filled the buffer. And very importantly, when there is an error, RMS will not advance the end-of-file and it will restore the next record position to what it was at the beginning of the $PUT. One outcome of this change will be that BACKUP's current EXQUOTA recovery strategy (one of its own making -- not sanctioned by RMS) will now work regardless of what record size backup uses and what size the RMS intermediate buffer is. Platforms affected: Alpha and VAX Modules: [RMS]RM1PUTBLD.MAR Affected Images: [SYS$LDR]RMS.EXE Impact/Risks: This problem raised larger issues with RMS's current handling of modified, dirty buffers -- particularly in the case of a write to disk failing due to a quota exhaustion like ASTLM. However, to minimize the risk for Gryphon, a conservative approach was taken to make a small, reasonable change that would solve the specific backup corruption problem and not break any other application. Overall performance will not be changed by this fix. However, the timing of when the write of a buffer that is exactly full at the end of a $PUT in the case of an exclusive access will shift from being done at the beginning of the next $PUT to the end of the current $PUT. Our assessment is that no application should be affected by this change. How this change was Tested: A new RMS.EXE was built with the changed modules and booted successfully on a Gryphon test system. Relevant sequential file tests (both exclusive and shared) were run; no regression was found. A reproducer of the backup problem with a process with low ASTLM now succeeded. Associated QARs,CLDs,SPRs: CFS.38206; CFS.44599 Comments: (optional) DIFFERENCES: ************ File ROOT:[RMS.SRC]RM1PUTBLD.MAR;2 1 $BEGIN RM1PUTBLD,46,RM$RMS1,, 2 ****** File ROOT:[RMS.SRC]RM1PUTBLD.MAR;1 1 $BEGIN RM1PUTBLD,45,RM$RMS1,, 2 ************ ************ File ROOT:[RMS.SRC]RM1PUTBLD.MAR;2 37 ; X-46 EMW0115 Elinor M. Woods 12-Sep-1996 38 ; If buffer is full at end of a $PUT, handle exclusive 39 ; case the same as shared case: release bucket instead 40 ; of deferring release until the beginning of the next 41 ; $PUT. If there is an error on the write ($QIO), this 42 ; change results in the error being in sync with the 43 ; last (current) $PUT that filled the buffer. The 44 ; end-of-file will not be advanced and the next record 45 ; pointer will be restored to its starting position. 46 ; 47 ; X-45 EMW0097 Elinor M. Woods 23-Apr-1996 ****** File ROOT:[RMS.SRC]RM1PUTBLD.MAR;1 37 ; X-45 EMW0097 Elinor M. Woods 23-Apr-1996 ************ ************ File ROOT:[RMS.SRC]RM1PUTBLD.MAR;2 1067 ; This buffer is full; if disk, release bucket. In the case of a shared 1068 ; file, write-thru is forced to minimize the exposure to data loss (and 1069 ; hence file corruption). In the case of an unshared (exclusive) file, 1070 ; release is no longer deferred until next $PUT so that if an error 1071 ; occurs on the write ($QIO), the error will be in sync with the current 1072 ; $PUT that filled the buffer. This results in the end-of-file not being 1073 ; advanced and the next record pointer being restored to its position 1074 ; at the start of the current $PUT. 1075 ; 1076 1077 BLK_FULL: ****** File ROOT:[RMS.SRC]RM1PUTBLD.MAR;1 1057 ; This block is full or at least the next record can't possibly 1058 ; fit in it, so change to next block. 1059 ; 1060 1061 BLK_FULL: ************ ************ File ROOT:[RMS.SRC]RM1PUTBLD.MAR;2 1080 MOVL #@16 - ; force write this (full) buffer 1081 + CSH$M_LOCK!CSH$M_NOREAD- ; flag no read required, reuse likely 1082 !CSH$M_REUSE_LIKELY,R3 ; (NOTE: Not used in NORECLK case) 1083 .SET_REGISTERS READ=,WRITTEN= 1084 BSBW RM$RELBLK1 ; Release this bucket 1085 BLBC R0,ZERO_RFA ; Branch if error; else drop 1086 ; through to PUT01_BR 1087 CLRL R1 ; So next $PUT does not shortcut 1088 ; call to RM$GETBLKNRP (NOTE: PUT01 1089 ; path moves R1 into IRB$L_CURBLKADR) 1090 PUT01_BR: ****** File ROOT:[RMS.SRC]RM1PUTBLD.MAR;1 1064 BBC #IFB$V_NORECLK,(R10),FORCE_WRT ; Ignore if sharing. 1065 INCL BDB$L_REL_VBN(R4) ; increment relative vbn 1066 BICB2 #BDB$M_VAL,BDB$B_FLGS(R4) ; make invalid 1067 CLRL R1 ; don't shortcut getblknrp 1068 1069 PUT01_BR: ************ ************ File ROOT:[RMS.SRC]RM1PUTBLD.MAR;2 1093 ; ****** File ROOT:[RMS.SRC]RM1PUTBLD.MAR;1 1072 ; Here a $PUT has filled a buffer on a shared file. We will force write-thru 1073 ; to minimize the exposure to data loss (and hence file corruption). 1074 ; 1075 FORCE_WRT: 1076 MOVL #@16 - ; force write this (full) buffer 1077 + CSH$M_LOCK!CSH$M_NOREAD- ; flag no read required, reuse 1078 !CSH$M_REUSE_LIKELY,R3 ; likely 1079 .SET_REGISTERS READ=,WRITTEN= 1080 BSBW RM$RELBLK1 ; Finally, release this bucket. 1081 BRW PUT01 ; rejoin code 1082 ; ************ Number of difference sections found: 5 Number of difference records found: 41 DIFFERENCES /IGNORE=()/MERGED=1/OUTPUT=ROOT:[RMS.SRC]RM1PUTBLD.DIF;1- ROOT:[RMS.SRC]RM1PUTBLD.MAR;2- ROOT:[RMS.SRC]RM1PUTBLD.MAR;1