CMU-TEK FTP V3.0 RELEASE NOTES
Table of Contents
1. Overview
2.1 New and Changed Features
2.1.1 Security
2.1.2 Max write size
2.1.3 New logical symbols
2.1.4 New modes/structures/types
2.1.5 New server Functions
2.1.6 New client Functions
2.1.7 Alternate FTP defaults.
2.1.8 New files
2.1.9 New File Parsing
2.2 Anonymous accounts
2.2.1 Creation
2.2.2 Restriction
2.2.3 Unrestriction
3. Variance from RFC959
4. Unresolved Problems
1. Overview
The FTP utility is used to implement the File Transfer Protocol.
This protocol is based on top of the IP and TCP protocols. The
CMU-TEK IP/TCP software is used to implement these lower levels.
The File Transfer Protocol is the language spoken between the FTP utility
and an FTP Server running on a remote machine. The local process is the
master and the remote server is the slave.
Through a series of commands and replies the local FTP utility tries to
send or receive files and other information.
Since the File Transfer Protocol is a negotiation language, not all
requests that the FTP utility asks for can always be satisfied by the
remote server. These requests that are unsatisfied will be signalled as
errors to the FTP utility.
This version conforms to RFC959. Most of the RFC has been implemented.
In addition a number of features that are commonly supported by other
FTP versions have been added.
2.1 New and Changed Features
Essentially major sections of FTP have been rewritten, and old bugs fixed
while hopefully there are no new bugs. Functionality has been dramatically
increased. This covers some of the fixes and enhancements, but by no means
all are included here.
2.1.1 Security
The old login mechanism has been discarded along with its problems and
virtues. Only the code for checking on load and primedays for anonymous
login, was kept from the RPI enhancements.
FTP now uses DEC login security. The normal LOGIN.COM is run for the user
who is logged in, with MODE "NETWORK". If the password is incorrect it is
logged, along with the remote node number of the offending FTP client,
and the port number. The user will receive a message that there were n
non interactive login failures, at the next login.
In addition the SYS$REM_ID shows the remote node number of the client.
This can be used by programs such as FINGER to reveal the user.
FTP_LOGIN uses sysgen parameter LGI_RETRY_LIM to limit attempts.
This parameter is normally set to 3.
IF you wish an FTP anonymous log then the anonymous account
must have write access to the anonymous log directory.
FTP_SERVER now produces a log file in the target user directory. This will
be named FTP_SERVER.LOG unless you have defined system logical FTP_SERVER.
Sample: ANONYMOUS_LOGIN.COM prevents all other access other than FTP.
It is now possible using logicals FTP_RESTRICT, FTP_DIRS to create
secure accounts to be used as FTP servers. This supplements the
ANONYMOUS account.
2.1.2 Max write size
Previously it was 2048. Now it is 2048 for stream mode,
2050 for block mode, and about 2064 for compressed mode.
Note Bene: SYSGEN parameter MAXBUF must be at least 8192.
2.1.3 New logical symbols
The following user logical symbols were added for FTP usage,
They may be defined system wide or each user may define the logical:
User Logical Result
---- ------- ------
FTP_DIRS Restricts FTP access to user directories.
It is similar to FTP_ANONYMOUS_DIRS
FTP_RESTRICT Restricts FTP login see section on Anonymous
FTP_TIMEOUT Is the user timeout in seconds.
This may be defined by user in his LOGIN.COM
If zero no FTP login is possible.
FTP_LOG Produces detailed log if it is 1-7 and not
if 0. See the FTP help for more information.
The following system logical symbols were added for FTP usage,
They may be defined system wide by including them in the IP_STARTUP.COM
File location:
System Logical Result
------ ------- ------
FTP_SERVER Points to the FTP_SERVER.COM file
This allows you to put it in a different place.
Login messages:
System Logical Result
------ ------- ------
FTP_ANNOUNCE Message when first connect.
FTP_WELCOME Message after normal login.
FTP_ANONYMOUS_WELCOME Message after anonymous login
The system logical may contain either:
1. The text of the message (NO CR-LF in text please)
2. @FILENAME - The location of a file containing text.
Time zone:
This is normally defined by DEC startup files.
System Logical Result
------ ------- ------
SYS$TIMEZONE_NAME Is now the "official" time zone name
This is the DEC name used by DECnet extensions.
Login process definitions:
System Logical Result
------ ------- ------
FTP_LOGIN_LOG System logical to produce detailed log
of login. Values=1-7
FTP_LOGIN_TIMEOUT Is the login timeout in seconds.
If zero FTP connections are killed immediately.
2.1.4 New modes/structures/types
Types:
1. Fortran, Print, carriage control files are now converted on
fly for STRU=FILE. This guarantees that they are correctly
transferred.
2. TYPE=ASCII:CONTROL For Fortran carriage control.
3. TYPE=ASCII:TELNET
Structures:
1. STRU=Record This allows you to transfer files with record
structure kept intact.
2. Correct EOF for STRU=FILE, TYPE=IMAGE. This fixes a
longstanding problem that binary files transferred from other
systems had an incorrect number of bytes.
Modes:
1. MODE=Compress, (Not avail in STRU=Page)
Supported by Multinet, not by Wollengong or Unix.
This allows you to slightly compress files on the fly for
transmission to remote sites.
2. MODE=Block, (Not avail in STRU=Page)
Supported by Unix, not by Wollengong or Multinet.
This mode allows multiple file transfers without reopening
connections. This gets around the max number of connections
problem.
2.1.5 New server Functions
There are many new server features, and commands.
Only a selection are mentioned here in no particular order..
1. Recursive directory, file transfers
2. APPE file - Append file has been added.
3. HELP topic - Remote help on topics.
4. SITE CHMOD (SET PROTECTION) for remote files.
Supported by UNIX.
5. SITE UMASK (SET PROTECTION/DEF) for remote process.
Supported by UNIX.
6. SITE BLOCK sets the block size for binary transfers.
Unique to CMU.
7. SITE PRIV shows the current priveleges.
Supported by MULTINET.
8. STAT param (SHOW FILE) Gives an almost "FULL" file listing.
Including file type, owner, prot....
9. STOU Store with Unique file name has been added.
10. SYST command returns the correct system type.
11. The FTP_SERVER has changed the default behavior of parsing when
an NLST command has been received:
If 'file.ext' is specified,
return 'file.ext' with no version.
If 'file.ext;' is specified,
return 'file.ext;n', where n is the highest version number of
the file.
If 'file.ext;*' is specified,
return all file specifications with versions attached.
While we're at it, convert the text to lower case.
This should keep DOS and UNIX systems happy.
Well, maybe not...
12. The FTP_SERVER will transfer the contents of an NLST command in
ASCII (meaning <text><cr><lf>) regardless of what the TYPE has
been set to.
Reported by Daren Critelli, Great Plains Region, USBR.
This has caused a problem with the Persoft FTP client.
2.1.6 New client Functions
There are many new client features, commands and qualifiers.
Only a selection are mentioned here in no particular order.
1. APPEND Command has been added
2. /APPEND has been added to RECEIVE (GET) command.
3. Many DCL COPY qualifiers have been added to SEND (PUT)
Such as /SINCE/BEFORE...
4. The FTP command has a second parameter to allow
execution of commands inline.
5. /USERNAME/PASSWORD qualifiers are now available on FTP command
line and for OPEN command.
6. QUIET mode makes FTP behave like DCL copy command, without
all of the extraneous FTP messages. This is invoked by
SET QUIET ON command of /QUIET qualifier.
7. /INITIALIZATION qualifier on FTP command line executes
a file before FTP brings up the command prompt.
8. Most functions now have the following qualifiers.
/CONFIRM asks you for yes or no before doing it.
/LOG Shows you the result of operation.
9. Both ^A and ^T can be pressed to show the progress of file
transfers. These reveal the file name and number of bytes
transferred.
10. /BLOCKSIZE qualifier controls the blocks size on a binary
GET of a file.
11. SHOW SUMMARY shows a summary of all transmissions to date.
12. /RECURSIVE may be used to transfer directory trees.
It should work with both VMS and UNIX sites.
This will even work with BSD systems that violate the RFC
when returning file specs from an NLST command.
13. SHOW SYSTEM_TYPE shows the remote system type.
14. Send of a "directory" will create a remote directory.
15. A number of standard "DCL" commands have been added or fixed.
SET HOST, SET DEFAULT, CREATE/DIR ...
2.1.7 Alternate FTP defaults.
The system manager is supplied with 3 possible CLD files
Depending on which one is used FTP will default to different behavior.
FTP.CLD Similar to the current one
These are highly recommended.
---> FTP_NOREPLY.CLD Gives a resonably quiet FTP ala Multinet
---> FTP_QUIET.CLD Gives an extremely quiet version that
may disturb some users, but is very
pleasing!!!
To update the DCL tables on a homogenious cluster:
$ SET COMMAND FTPxxx.CLD -
/TABLE=SYS$COMMON:[SYSLIB]DCLTABLES -
/OUTPUT=SYS$COMMON:[SYSLIB]DCLTABLES
To install the new DCL tables on all nodes of the cluster:
$ RUN SYS$SYSTEM:SYSMAN
SYSMAN> SET ENVIRONMENT /CLUSTER
SYSMAN> DO INSTALL REPLACE SYS$LIBRARY:DCLTABLES
2.1.8 New files
There are some changes that have been made to IP_STARTUP.COM. This is
necessary since there are 2 executables for FTP.
1. FTP_LOGIN.EXE Initially starts up and then gets the username
and password.
2. FTP_SERVER.COM Is executed after login to invoke the FTP_SERVER.EXE.
This version of FTP.EXE should work with older versions of CMUIP,
but the server modules FTP_LOGIN.EXE and FTP_SERVER.EXE require a new
version if IPACP and a new IPDRIVER.
Assuming the following line is already in INET$CONFIG:
WKS:21:FTPSRV:TCP$FTPSRV:NETWRK:*:::::4:5
You must have following in IP_STARTUP.COM
$!
$! Define the location of the FTP login server
$!
$ Define/System/NoLog TCP$FTPSRV CMUIP_ROOT:[SYSEXE]FTP_LOGIN.EXE
$!
$! Grant necessary privs to the user server.
$! Note: PHY_IO is not necessary for most clients except for
$! NCSA Telnet, which uses low number ports.
$!
$ if (f$search("CMUIP_ROOT:[SYSEXE]FTP_SERVER.EXE") .nes. "") then -
if (.not. f$file("CMUIP_ROOT:[SYSEXE]FTP_SERVER.EXE","KNOWN")) then -
install add CMUIP_ROOT:[SYSEXE]FTP_SERVER.EXE -
/open/header/share -
/priv=(phy_Io,share)
In addition the new file FTP_SERVER.COM should reside in
CMUIP_ROOT:[SYSEXE]
2.1.9 New file parsing
The server now parses file and directory names in a fairly intellignet
manner. If they "look" like VMS names they are untouched. If they look
like UNIX names, they are converted to equivalent VMS names.
2.2 Anonymous accounts
2.2.1 Creation
To setup an anonymous account:
1. Create an account named "anonymous" with password guest. This is
different from the previous version of FTP in that a fixed password
must be used.
2. Restrict it to just NETWORK login. This is different from the previous
version, which turned off NETWORK login also.
3. Set the default anonymous login file to ANONYMOUS_LOGIN.COM
Previous versions did not use a login file. The sample file is
designed to prevent unauthorized access.
5. Disable incoming mail
6. Set the restricted flag. This is just an added precaution, that
ensures that the ANONYMOUS_LOGIN.COM is executed.
7. Create the directories to which ANONYMOUS will have access. Do not
permit ANONYMOUS to own any of the files or be in the same group
as the owner of the files. Set WORLD:R protection on all files
and directories to be accessible, or use an ACL to grant access
specifically to ANONYMOUS.
8. Setup a directory for the anonymous log files. You must also
define a logical to point to that directory. in the
ANONYMOUS_LOGIN.COM
9. You may further restrict the anonymous account using logicals.
Example:
$ SET DEF SYS$SYSTEM
$ RUN SYS$SYSTEM:AUTHORIZE
ADD ANONYMOUS/PASSWORD=GUEST/NOBATCH/NOLOCAL/NODIALUP/NOREMOTE -
/FLAG=(DISMAIL,RESTRICTED)/UIC=[666,666] -
/DEV=USER$DISK/DIR=[PUBLIC] -
/LGICMD=CMUIP_ROOT:[SYSMGR]ANONYMOUS_LOGIN.COM
$ CREATE /DIRECTORY USER$DISK:[ANONYMOUS_LOG]/OWNER=ANONYMOUS
$ CREATE /DIRECTORY USER$DISK:[PUBLIC]
You may check on the DECNET security with the following:
$ DIR 0"anonymous guest"::
you should get the following message:
-RMS-E-FND, ACP file or directory lookup failed
-SYSTEM-F-LINKEXIT, network partner exited
2.2.2 Restriction
Anonymous FTP is restricted to only reading files.
You may further restrict the anonymous account using logical symbols.
All FTP_ANON logical names should now be placed in the logical name
table FTP_NAME_TABLE, to get them out of the system name table. To
do this, the following lines may be in your IP_STARTUP.COM:
$!
$! Create the anonymous table.
$!
$ CREATE/NAME_TABLE/EXEC/PROT=(S:RWED,O:RWED,G:R,W:R)-
/PARENT=LNM$SYSTEM_DIRECTORY FTP_NAME_TABLE
$ FTPDEF := DEFINE/TABLE=FTP_NAME_TABLE/EXEC/NOLOG
then use FTPDEF to define the FTP_ANON... logical names, for example:
$!
$! Restrict anonymous to particular directories
$!
$ FTPDEF FTP_ANONYMOUS_DIRS USER$DISK:[PUBLIC...]
$!
$! Prevent login when load is too great on prime days
$! some floating-point number between 0.0 and 1.0.
$!
$ FTPDEF FTP_ANON_LOAD_THRESHOLD "0.5"
$!
$! Define prime days
$!
$ FTPDEF FTP_ANON_PRIME_DAYS "2,3,4" ! Tuesday, Wednesday, Thursday
$!
$! Define prime hours
$!
$ FTPDEF FTP_ANON_PRIMETIME_START 8:00 ! 8:00 am
$ FTPDEF FTP_ANON_PRIMETIME_END 16:00 ! 4:00 pm
The only required logical is FTP_ANON_LOAD_THRESHOLD. If that logical name
exists and the LAV0 device exists, the load checking code is used. The
code does the following:
If FTP ANON_PRIME_DAYS is defined, it is translated. The comma-separated
list of numbers (where 1=Monday, 2=Tuesday, etc.) is used to identify
the days in which "prime time" is effective. If it does not exist,
"prime time" is assumed to be in effect Monday through Friday.
Note: Use ONLY numbers 1 through 7, and NO SPACES in the string. Surround
the string with quotation marks when defining!
If FTP_ANON_PRIMETIME_START is defined, it is translated and converted
into a system date-time value using LIB$CONVERT_DATE_STRING. If not,
then 09:00 is used as the start of "prime time".
If FTP_ANON_PRIMETIME_END is defined, it is translated and converted
into a system date-time value using LIB$CONVERT_DATE_STRING. If not,
then 17:00 (5:00 pm) is used as the end of "prime time".
If the current time is between the prime-time start and end times,
then the current load averages are read from the LAV device. The
current load is computed using the following formula:
load = M15 * (P15 / 4.0)
where M15 is the average load over the last 15 minutes, and P15 is
the average priority over the last 15 minutes. Thus, the average load
is normalized against typical interactive priority to guard against
low-priority batch jobs preventing guest login access.
If the load is greater than or equal to the LOAD_THRESHOLD value, then
the guest login is denied with a reason of "system too busy".
If the threshold is not exceeded, then the guest login is accepted, but
the user is warned to minimize access during prime time (with the
start and end times displayed along with the time zone information
[if FTP_ANON_TIME_ZONE is defined]).
If the current time does not fall within prime time, no load checking
is performed.
You can restrict the directories to which ANONYMOUS has access by
defining the system-wide logical name FTP_ANONYMOUS_DIRS to
a search list of device/directory specifications. Any RETR,
LIST, or NLST will check against this list before going through normal
system access checks. This prevents ANONYMOUS from gaining access
to people's files via WORLD access. If you do not define
FTP_ANONYMOUS_DIRS, the extra access checks do not take place.
You can use [directory...] notation to allow access to the entire
subdirectory tree below the specified directory.
Make sure it is defined before allowing Anonymous access.
Make sure that the default device/directory in the UAF is accessible
(not strictly necessary, but easier on the users).
Example: $ DEFINE/SYSTEM FTP_ANONYMOUS_DIRS -
USER$DISK:[PUBLIC...],- ! public files
USER$DISK:[NEWS...],- ! news archives
USER$DISK:[MAIL] ! mail archives
While these logicals were meant to enhance the security of Anonymous
FTP, neither the author nor his employer (nor anyone else for that matter)
guarantees that the software is secure.
2.2.3 Unrestriction
A new logical has been added to remove any Anonymous restrictions.
Normally anonymous can read files, but not write then. To remove this
restriction: The following is added to the ANONYMOUS_LOGIN.COM
$!Sample: Give anonymous users all access to account.
$!
$ DEFINE FTP_RESTRICT 0
If you want only partial restriction you may define a different value,
by adding together all the values you wish:
Logical Name Value Description
------- ---- ----- -----------
FTP_RESTRICT number Puts restrictions on FTP actions
1 No read (RETR)
2 No write (STOR,STOU,APPE,MKD)
4 No Control (SITE)
8 No Delete (DELE,RMD)
16 No List (LIST,NLST,STAT param)
3. Variance from RFC959
The following from RFC959 are unsupported.
1. PASV - This is probably only significant for 3'rd party FTP tranfers.
2. REIN - Reinitialize server. This is difficult to implement, but it
could be done. It is not generally supported by other VMS
implementations of FTP.
3. SMNT - Mount command. This is potentially useful, and may be
implemented in the future.
4. TYPE=EBCDIC
1. MODE=Compress. This mode is supposed to signal EOF by an escape code
of <Null><64>. Unfortunately Multinet does not recognize this for
STRU=VMS or STRU=FILE!!!
For compatability CMU also closes the connection to signal EOF.
CMU adheres to the standard on receive, and recognizes <Null><64>.
All of the proposed mail enhancements to FTP which are not in RFC959 are
not supported. In addition the proposed 3'rd party FTP which is not in
the RFC is likewise omitted.
4. Unresolved Problems
Some servers do not recognize the SYST command. If you have REPLY=ON
mode, this will generate an error message. This is known to happen with
the SUN Ftp server. The message may be safely ignored, or use /NOREPLY.