From:	CSBVAX::CSBVAX::MRGATE::"SMTP::CRVAX.SRI.COM::RELAY-INFO-VAX" 20-JAN-1989 22:50
To:	MRGATE::"ARISIA::EVERHART"
Subj:	long term scheduler - part 5 of 5


Received: From KL.SRI.COM by CRVAX.SRI.COM with TCP; Thu, 19 JAN 89 20:27:58 PDT
Received: from RELAY.CS.NET by KL.SRI.COM with TCP; Thu, 19 Jan 89 20:24:54 PST
Received: from relay2.cs.net by RELAY.CS.NET id az21989; 19 Jan 89 20:44 EST
Received: from a.cs.okstate.edu by RELAY.CS.NET id ae18099; 19 Jan 89 19:45 EST
Received: from nemo.math.okstate.edu by a.cs.okstate.edu id ac10453;
          19 Jan 89 9:51 CST
Date: Thu, 19 Jan 89 09:40 CDT
From: Gregg Wonderly <GREGG%nemo.math.okstate.edu@RELAY.CS.NET>
Subject: long term scheduler - part 5 of 5
To: info-vax@KL.SRI.COM
X-VMS-To: in%"info-vax@kl.sri.com"

...................... Cut between dotted lines and save ......................
$!.............................................................................
$! VAX/VMS archive file created by VMS_SHAR V-5.03 07-Oct-1987
$! which was written by Michael Bednarek (U3369429@ucsvc.dn.mu.oz.au)
$! To unpack, simply save and execute (@) this file.
$!
$! This archive was created by GREGG
$! on Thursday 19-JAN-1989 09:18:20.54
$!
$! It contains the following 7 files:
$! MAKE.COM SCHED.RNO DO.COM MAKEIT.COM SCHED.COM SCHED_SETUP.COM SCHED.CLD
$!=============================================================================
$ Set Symbol/Scope=(NoLocal,NoGlobal)
$ Version=F$GetSYI("VERSION") ! See what VMS version we have here:
$ If Version.ges."V4.4" then goto Version_OK
$ Write SYS$Output "Sorry, you are running VMS ",Version, -
                ", but this procedure requires V4.4 or higher."
$ Exit 44
$Version_OK: CR[0,8]=13
$ Pass_or_Failed="failed!,passed."
$ Goto Start
$Convert_File:
$ Read/Time_Out=0/Error=No_Error1/Prompt="creating ''File_is'" SYS$Command ddd
$No_Error1: Define/User_Mode SYS$Output NL:
$ Edit/TPU/NoSection/NoDisplay/Command=SYS$Input/Output='File_is' -
        VMS_SHAR_DUMMY.DUMMY
set(Informational, off); set (success, off);
f:=Get_Info(Command_Line,"File_Name"); b:=Create_Buffer("",f);
o:=Get_Info(Command_Line,"Output_File"); Set(Output_File,b,o);
Position(Beginning_of(b)); Loop x:=Erase_Character(1);
Loop ExitIf x<>"V"; Move_Vertical(1); x:=Erase_Character(1);
Append_Line; Move_Horizontal(-Current_Offset);
EndLoop; Move_Vertical(1); ExitIf Mark(None)=End_of(b)
EndLoop; Position(Beginning_of(b)); Loop
x:=Search("`",Forward,Exact); ExitIf x=0; Position(x);
Erase_Character(1); If Current_Character='`' then Move_Horizontal(1);
else Copy_Text(ASCII(INT(Erase_Character(3)))); EndIf; EndLoop; Exit;
$ Delete VMS_SHAR_DUMMY.DUMMY;*
$ Checksum 'File_is
$ Success=F$Element(Check_Sum_is.eq.CHECKSUM$CHECKSUM,",",Pass_or_Failed)+CR
$ Read/Time_Out=0/Error=No_Error2/Prompt=" CHECKSUM ''Success'" SYS$Command ddd
$No_Error2: Return
$Start:
$ File_is="MAKE.COM"
$ Check_Sum_is=86124512
$ Copy SYS$Input VMS_SHAR_DUMMY.DUMMY
X$ makeit="@sched$root:[exe]makeit"
X$ do="@sched$root:[exe]do"
X$ if f$search("SCHED.HLB") .eqs. "" then do library/create/help sched
X$ makeit sched rno runoff "/output=sched.hlp" "" "" "" "sched.hlp"
X$ if $status .eq. 3 then goto done
X$ do library/help/replace sched sched
X$!
X$! makeit sched hlp library "/help/replace sched " "" "" "" "sched.hlb"
X$! if $status .eq. 3 then goto done
X$!
X$ done:
X$ exit
X$!
$ GoSub Convert_File
$ File_is="SCHED.RNO"
$ Check_Sum_is=185641675
$ Copy SYS$Input VMS_SHAR_DUMMY.DUMMY
X.NO FLAGS ALL
X.P0
X.ap
X.NO NUMBER
X.NPA
X.LM1.RM70
X.I-1
X1 SCHED
X.s
XSCHED is a program used to monitor the CPU utilization.  It allows you
Xto control CPU bound processes so that requests for small amounts of CPU
Xhave higher priority.  The "Scheduler" process runs as a detached process
Xunder the [1,4] UIC.  This process is controlled by a separate program
Xwhich communicates with the "Scheduler" process via a writeable/shareable
Ximage, and a permanent mailbox.  The controlling program, SCHCMD.FOR, can
Xbe installed with privileges so that all users can run SCHCMD to see what
Xthe scheduler is doing.  SCHCMD requires that the executing process have
XOPER privilege before it will allow any parameters to the "Scheduler"
Xprocess to be altered.  SCHCMD must be installed with BYPASS privilege
Xin order for it to be able to READ the shareable/writeable image.
X.s
XIf you do not desire other users to have access to the SCHCMD program, then
Xyou can remove the INSTALLation of the program with privileges from the
X[.SRC]MAKE.COM command procedure.
X.I-1
X2 Author
X.s
XSCHED was written by Gregg Wonderly at the Mathematics Department at
XOklahoma State University <gregg@nemo.math.okstate.edu>.
X.I-1
X2 Background
X.s
XSCHED represents an attempt at controlling interactive CPU use under VMS.  VMS
Xprovides a very handy queuing facility for running programs that have
Xresource requirements different from normal interactive processes.  Certain
Xproblems surface when these facilities are not used, or are used improperly.
X.s
XWhenever a program is being executed by a user, he/she must share the CPU
Xresources with other users.  Users using programs such as editors, mail, or
Xother programs which require only occassional CPU use should have priority
Xover users which need prolonged amounts of CPU.  This allows the occasional
Xuser to progress with their business in a timely fashion.
X.s
XThe VMS operating system does not allow this type of prioritization though.
XInstead, the operating system has a very short term memory of the behavior
Xof a process.  A process which completes its time quantum without asking for
XI/O will be lowered in priority, to a minimum of its base priority.  A process
Vthat is rescheduled because of a request for I/O will be given a small priorit
Xy
Vboost for purposes of rescheduling.  Typically, a process whose actual priorit
Xy
Vis some number, n, greater than its base priority, will return to base priorit
Xy
Xafter n time quantums.
X.s
XIt turns out that this is not a very good guess of a processes long term
Xbehavior.  This method allows users at interactive priority to disrupt other
Xusers by running CPU bound processes that almost always complete a time
Xquantum without asking for I/O.  On the other hand, there are several programs
Xthat require large amounts of CPU and should be run interactively for
Xconvienence.
X.s
XSCHED was written to resolve this problem by providing a much longer term
Xanalyis of the processes running in the system.  SCHED does two things.
X.s
X.lm+5
X1) It adjusts the priority of selected processes
X.br
X2) It pauses batch queues based on load averages
X.lm-5
X.s
XThese two things alone can make a tremendous change in the response time on
Xa heavily loaded system.
X.s
XTo carry out 1), SCHED maintains a list of processes that meet certain
Xcriteria.  In general, the processes of most concern are interactive
Xprocesses being ran by normal users which are using large amounts of CPU.
XFor this reason, SCHED collects processes which are not in the SYSTEM groups
X(group UIC of the process is less than or equal to the /SYSTEM parameter),
Xare interactive, and do not have BYPASS privilege enabled.
XThere are some other criteria which are selectable, such as CPU time that
Xmust be consumed before a process can be considered (this allows users
Xto logon to check mail or do something else requiring a small amount of time).
X.s
XThis list of processes is then reviewed to determine what processes are using
Xthe CPU time CONSUMED BY THIS GROUP OF PROCESSES, and which are not.  The
Xprocesses who are using CPU are candidates for a decrease in base priority
X(providing they are not completely adjusted down), and those that are not
Xusing CPU are candidates for an increase in base priority (providing they
Xhave previously been adjusted down).  I say candidates because each process
Xmust meet certain criteria, some of which can be changed dynamically.
X.s
XA process which is consuming more than the maximum percentage of CPU as
Xindicated by the /MAX_PERCENT= value will have its base priority adjusted
Xdown to the /DOWN_PRIORITY= value if it
X.s
X.lm+5
X1) Has not invoked a new image
X.br
X2) Does not have a current priority greater than the /IO_DIFFERENCE= value
Xplus its base priority.
X.s
X.lm-5
XAlso, whenever more than 95 percent of the /INTERVAL= time is consumed by the
Xprocesses being considered, then the process consuming the most CPU for that
Xinterval will have its base priority lowered to the /DOWN_PRIORITY value.
XThis exception allows all of the CPU bound processes to be weeded out if
Xthe system suddenly becomes loaded.
X.s
XProcesses consuming less than /MIN_PERCENT percent of the CPU which have
Xpreviously had their base priority lowered to the /DOWN_PRIORITY value will
Xhave their base priority raised to the /UP_PRIORITY value if their base
Xpriority has been lowered for at least /DOWN_COUNT intervals, or the process
Xenvokes a new image.  The /DOWN_COUNT value is necessary to prevent two CPU
Xbound processes from juggling between the /UP_PRIORITY value and the
X/DOWN_PRIORITY value.  A processes envoking a new image is given special
Xconsideration to help make that transition faster.  If the new image requires
Xa lot of CPU, then the process will be treated accordingly. 
X.s
XIn addition to the /DOWN_PRIORITY adjustment, a process can be classified as
Xa CPU HOG.  The value of the /HOG_PRIORITY parameter becomes the base priority
Xof any process who stays at the /DOWN_PRIORITY setting for /HOG_COUNT 
Xobservations.  If the /INTERVAL value is 7 seconds, then a /HOG_COUNT value
Xof 20 would cause a process that is CPU bound for 140 seconds to be further
Xreduced in priority.  It will be raised to the /DOWN_PRIORITY value only
Xafter it becomes idle, does not have a state of COM during a particular
Xobservation, or invokes a new image.
X.s
XThe /IDLE_COUNT value is used to control how long a process sets idle because
Xof higher priority processes consumming CPU.  When at HOG_PRIORITY, if a
Xprocess is receiving less than /MIN_PERCENT of the CPU, for /IDLE_COUNT
Xconsecutive observations, then it will be raised back to the /DOWN_PRIORITY
Xvalue.
X.I-1
X2 Command_Qualifiers
X.s
X.I-1
X/DISPLAY
X.br
X/DISPLAY [ = terminal name]
X.s
XCauses the SCHCMD image to monitor the information kept by SCHED.
XAt regular intervals the information available will be summarized on the
Xterminal.  If a string value is given, that device will be used for output.
XOtherwise, the logical device, TT:, is used for output.
X.s
X.I-1
X/DOWN_COUNT
X.br
X/DOWN_COUNT = nn
X.s
XChanges the number of intervals that a process must have its priority down
Xgraded before it can be set back up.  This value should be at least 3 to
Xmake sure that priority thrashing is kept to a minimum.
X.s
XWithout this value, 2 processes that do not go CPU bound simultaneously
Vwill start switching between priorities because one will be at UP_PRIORITY, an
Xd
Xget all of the CPU for an interval, causing the one at DOWN_PRIORITY to appear
Xidle.  This will cause the UP_PRIORITY process(es) to be set to DOWN_PRIORITY,
Vand the DOWN_PRIORITY processes set to UP_PRIORITY.  This will continue as lon
Xg
Vas the processes are CPU bound.  Therefore, you should always set DOWN_COUNT t
Xo
Xat least 3. 
X.s
X.I-1
X/DOWN_PRIORITY
X.br
X/DOWN_PRIORITY = nn
X.s
XChanges the base priority value used by SCHED to lower a process's priority.
XThis qualifier causes SCHED to unadjust the priority of all adjusted processes
Xso that the proper results can be expected.
X.s
X.I-1
X/HOG_COUNT
X.br
X/HOG_COUNT=nn
X.s
XSets the maximum number of consecutive observations that a process can
Xconsume excessive CPU before it is reduced in priority to the /HOG_PRIORITY
Xvalue.
X.s
X.I-1
X/HOG_PRIORITY
X.br
X/HOG_PRIORITY=nn
X.s
XThis is the base priority value that will be applied to a process that has
Xusing /MAX_PERCENT of the CPU, or more during /HOG_COUNT consecutive
Xobservations.
X.s
X.I-1
X/IDLE_COUNT
X.s
XControls the maximum number of observations that a process can set idle at
X/HOG_PRIORITY, BECAUSE OF HIGHER PRIORITY CPU USE.  After a process is
Xset to /HOG_PRIORITY, it must receive /MIN_PERCENT CPU during at least
Xone observation during /IDLE_COUNT
Xconsecutive observations, or it will have its base priority elevated back to
X/DOWN_PRIORITY.
X.s
X.I-1
X/INTERVAL
X.br
X/INTERVAL = nn
X.s
XChanges the processing interval used by SCHED.  The number is in seconds, and
Xcan be arbitrarily large.  A default interval is compiled into the SCHED image
Xand is located in the file SCHED$ROOT:[SRC]PARAMS.INC.  Usually 6-10 seconds
Xcovers a typically loaded system.  Longer periods will causes users to notice
Xthe downgrade in priority whenever they switch images.  Shorter periods
Xwill consume unnecessary amounts of CPU.
X.s
X.I-1
X/IO_DIFFERENCE
X.br
X/IO_DIFFERENCE = nn
X.s
XProcesses consumming large amounts of CPU but who also have a scheduling
Xpriority boost (due to I/O completion rates) greater than or equal to this
Xnumber will not be considered CPU bound.  This number can be used to control
Xprocesses that consume a very large amount of CPU while retaining an I/O boost
Xdue to VMS's scheduling habits.
X.s
X.I-1
X/MAX_ACTIVE
X.br
X/MAX_ACTIVE = nn
X.s
XSets the maximum number of intervals that a process may use MAX_PERCENT of
Xthe CPU before its priority is adjusted down.  The default value is zero
Xso that process are immediately considered.
X.s
X.I-1
X/MAX_PERCENT
X.br
X/MAX_PERCENT = nn
X.s
XSets the maximum percentage of the /INTERVAL period that a process can consume
Xbefore being considered CPU bound.
X.s
X.I-1
X/MIN_PERCENT
X.br
X/MIN_PERCENT = nn
X.s
XSets the minimum percentage of the /INTERVAL period that an adjusted process
Xmust get CPU before it is no longer considered CPU bound. 
X.s
X.I-1
X/MIN_TIME
X.br
X/MIN_TIME = nn.nn
X.s
XSets the value of CPUTIM that a process must attain before it is considered by
XSCHED.  This value is in seconds (a decimal point is allowed to specify values
Xless than 1 second).  See the $GETJPI system service description for a
Xdefinition of the CPUTIM value (Note that CPUTIM as returned by $GETJPI is in
Xone hundredths of a second).
X.s
X.I-1
X/PRIORITY
X.br
X/PRIORITY = n
X.s
XChanges the priority that SCHED executes under.  You should set this value to
Xthe same value as /UP_PRIORITY.
X.s
X.I-1
X/STOP
X.br
X/STOP
X.s
VCauses any currently executing SCHED program to exit after the current interva
Xl
Xhas elasped.  
X.s
XNote that there are certain cases where SCHED/STOP will fail to stop the
Xprogram.  If you /DELETE and the reinstall the SCHED$GDATA image while the
Xscheduler is running, then the memory used to communicate the fact that the
Xscheduler should stop will no long be shared between the SCHED image and the
XSCHCMD image. 
X.s
XIn this case you must use STOP/ID= to stop the scheduler, and then manually
Xcorrect any processes and/or queues that where currently adjusted/paused when
Xyou stopped the scheduler. 
X.s
VJust remember to SCHED/STOP the scheduler before messing with the images.  Not
Xe
Xthat SCHED$ROOT:[SRC]MAKE.COM does just that before performing any /DELETE
Xoperations with INSTALL, or before relinking the images.
X.s
X.I-1
X/SUBMIT
X.br
X/SUBMIT
X.s
VCauses a new detached process to be created that runs SCHED.  If the program i
Xs
Valready executing, then you must use the /STOP qualifier with the /SUBMIT.  No
Xt
Vusing the /STOP quailfier when SCHED is already running will cause the message
Xs
X.s
X.lm+5
X%SCHED-W-ACTIVE, scheduler process is currently running
X.br
X.lm+5
X`009and
X.lm-5
X.br
X%SYSTEM-F-DUPLNAM, duplicate name
X.lm-5
X.s
Xto be displayed.  The second is a result of the failure of $CREPRC to work
Xbecause a process with the name "Scheduler" already exists.
X.s
X.I-1
X/UP_PRIORITY
X.br
X/UP_PRIORITY = n
X.s
XThis is the priority that a process will be returned to if/when it is
Xdetermined that it should no longer be adjusted.  This should be the priority
Xof normal interactive processes. 
X.I-1
X2 Alternate strategies
X.s
XSome processes do not fall under the categories described previously.  A
Xsimple mechanism is therefore included to allow operators to control the
Xbehavior of SCHED towards certain processes.  The /STRATEGY qualifier
Xcontrols this behavior
X.I-1
X/STRATEGY
X.s
X/STRATEGY=NONE or IGNORE or SMASH
X.s
X.LT
XNONE        Releases a process from one of the previously
X            established strategies so that it will, once
X            again be a candidate for priority management by
X            SCHED.
X            
XIGNORE      The process will be adjusted to /UP_PRIORITY,
X            and then ignored.
X            
XSMASH       The process will be set to the priority given by 
X            the /BIAS parameter, and then ignored.
X
X.EL
X.I-1
X/BIAS=priority
X.s
X/BIAS is used by the /STRATEGY=SMASH qualifier to specify
Xthe priority that the process will be set to.
X.I-1
X/IDENTIFIER=pid
X.s
X/IDENTIFIER is used to specify which process the /STRATEGY
Xvalue chosen is to be applied to.
X.I-1
X2 Load_Averages
X.s
XSCHED monitors the load averages of the system, and can pause selected batch
Xqueues when the load average reaches a set maximum level.  The queues will be
Xresumed when the load average reaches a set minimum level.  The /QUEUE
Xqualifier allows you to refer to the queue you wish to control or release
Xcontrol of.
X.s
X.I-1
X/QUEUE
X.br
X/QUEUE = batch_queue_name
X.s
VThis qualfier must be used with either the /AVERAGES or the /RELEASE qualifier
X. 
XIt specifies the name of the queue to be used with these qualifiers.
X.s
X.I-1
X2 Positional_Parameters
X.s
X.I-1
X/AVERAGES
X.br
X/AVERAGES=(1 minute_min, 5 minute_min, 15 minute_min,
X.br
X.lm+11
X1 minute_max, 5 minute_max, 15 minute_max)
X.lm-11
X.s
XMust be used with the /QUEUE qualifier to set the load average values used
Vto determine when to pause and resume a batch queue.  There must be six number
Xs
Xgiven as the argument list.  The first three are used for resuming a paused
Xqueue, and the last three are used for pausing an active queue.  When the
Vload average, for a specific interval, reaches the specified maximum value, th
Xe
Xbatch queue specified with the /QUEUE qualifier will be paused.  When the
Xload average, for a specific interval, decreases back to one of the specified
Xminimum values, and none of the maximum values are out of range, a paused
Xqueue will be resumed.
X.s
X.I-1
X/RELEASE
X.br
X/RELEASE
X.s
XCauses the queue given in the /QUEUE qualifier to be removed from the list of
Xqueues to monitor.  If it has been paused, it will be resumed.
$ GoSub Convert_File
$ File_is="DO.COM"
$ Check_Sum_is=388123797
$ Copy SYS$Input VMS_SHAR_DUMMY.DUMMY
X$ v=f$verify(0)
X$!
X$!`009DO.COM
X$!
X$!`009Written by Gregg Wonderly 30-Oct-1986
X$!
X$! write sys$output "$ ''P1' ''P2' ''P3' ''P4' ''P5' ''P6' ''P7' ''P8'"
X$ write sys$output "$ ",P1," ",P2," ",P3," ",P4," ",P5," ",P6," ",P7," ",P8
X$ 'P1' 'P2' 'P3' 'P4' 'P5' 'P6' 'P7' 'P8'
X$ EXIT $status+('f$verify(v)'*0)
$ GoSub Convert_File
$ File_is="MAKEIT.COM"
$ Check_Sum_is=1947675016
$ Copy SYS$Input VMS_SHAR_DUMMY.DUMMY
X$ @pub:makeit "''p1'" "''p2'" "''p3'" "''p4'" "''p5'" "''p6'" "''p7'" "''p8'"
$ GoSub Convert_File
$ File_is="SCHED.COM"
$ Check_Sum_is=1279087681
$ Copy SYS$Input VMS_SHAR_DUMMY.DUMMY
X$ CREATE SCHED$ROOT:[000000]SCHED.LOG
X$ SCHED/SUBMIT-
X`009/INTERVAL=7/MIN_PERCENT=10/MAX_PERCENT=25 -
X`009/PRIO=7/UP_PRIORITY=5/DOWN_PRIORITY=4/IO=3 -
X`009/MIN_TIME=20/DOWN_COUNT=3/HOG_COUNT=60/HOG_PRIORITY=2 -
X`009/IDLE_COUNT=8/SYSGRP=51
X$ SCHED/QUEUE=SLOW$BATCH/AVERAGES=(2.60,2.40,2.20,4.5,4.2,4.0)
X$ SCHED/QUEUE=SYS$BATCH/AVERAGES=(3.8,3.70,3.50,5.7,5.5,5.3)
X$ EXIT
$ GoSub Convert_File
$ File_is="SCHED_SETUP.COM"
$ Check_Sum_is=1074479888
$ Copy SYS$Input VMS_SHAR_DUMMY.DUMMY
X$!
X$!  Command file to set up and start the scheduler.
X$!
X$!  Written by Gregg Wonderly Apr-15-1987
X$!
X$! Define a rooted directory
X$ DEFINE/SYSTEM/NOLOG/TRANS=CONCEAL/EXEC SCHED$ROOT MATH0:[PUBLIC.SCHED.]
X$!
X$! Install the images
X$!
X$ set noon
X$ @SCHED$ROOT:[SRC]MAKE IMAGE
X$ set on
X$!
X$! Define global section file
X$!
X$ DEFINE/SYSTEM/NOLOG/EXEC SCHED$GDATA SCHED$ROOT:[EXE]SCHED$GDATA.EXE
X$!
X$! Set the shared image writeable so that normal users may run
X$! SCHED
X$!
X$ SET FILE/PROT=(W:REW) SCHED$GDATA
X$!
X$! Start the scheduler running
X$!
X$ @SCHED$ROOT:[EXE]SCHED
X$ EXIT
$ GoSub Convert_File
$ File_is="SCHED.CLD"
$ Check_Sum_is=2102424870
$ Copy SYS$Input VMS_SHAR_DUMMY.DUMMY
X!
Xdefine type strat_type
X    keyword NEWIMAGE
X    keyword NONE
X    keyword IGNORE
X    keyword SMASH
X!
Xdefine verb sched
Ximage sched$root:[exe]schcmd.exe
Xqualifier display value
Xqualifier down_count, value(required)
Xqualifier down_priority, value(required)
Xqualifier hog_count, value(required)
Xqualifier hog_priority, value(required)
Xqualifier identification, value(required)
Xqualifier idle_count, value(required)
Xqualifier interval, value(required)
Xqualifier io_difference, value(required)
Xqualifier max_active, value(required)
Xqualifier max_percent, value(required)
Xqualifier min_percent, value(required)
Xqualifier min_time, value(required)
Xqualifier priority, value(required)
Xqualifier bias, value(required)
Xqualifier strategy, value(required,type=strat_type)
Xqualifier queue, value(required)
Xqualifier averages, value(required,list)
Xqualifier sysgrp, value(required)
Xqualifier release
Xqualifier stop
Xqualifier submit
Xqualifier up_priority, value(required)
Vdisallow (strategy and not (identification)) or (queue and not (averages or re
Xlease)) or (not queue and (averages or release))
$ GoSub Convert_File
$ Exit

