CMU-TEK FTP V3.0 RELEASE NOTES Table of Contents 1. Overview 2.1 New and Changed Features 2.1.1 Security 2.1.2 Max write size 2.1.3 New logical symbols 2.1.4 New modes/structures/types 2.1.5 New server Functions 2.1.6 New client Functions 2.1.7 Alternate FTP defaults. 2.1.8 New files 2.1.9 New File Parsing 2.2 Anonymous accounts 2.2.1 Creation 2.2.2 Restriction 2.2.3 Unrestriction 3. Variance from RFC959 4. Unresolved Problems 1. Overview The FTP utility is used to implement the File Transfer Protocol. This protocol is based on top of the IP and TCP protocols. The CMU-TEK IP/TCP software is used to implement these lower levels. The File Transfer Protocol is the language spoken between the FTP utility and an FTP Server running on a remote machine. The local process is the master and the remote server is the slave. Through a series of commands and replies the local FTP utility tries to send or receive files and other information. Since the File Transfer Protocol is a negotiation language, not all requests that the FTP utility asks for can always be satisfied by the remote server. These requests that are unsatisfied will be signalled as errors to the FTP utility. This version conforms to RFC959. Most of the RFC has been implemented. In addition a number of features that are commonly supported by other FTP versions have been added. 2.1 New and Changed Features Essentially major sections of FTP have been rewritten, and old bugs fixed while hopefully there are no new bugs. Functionality has been dramatically increased. This covers some of the fixes and enhancements, but by no means all are included here. 2.1.1 Security The old login mechanism has been discarded along with its problems and virtues. Only the code for checking on load and primedays for anonymous login, was kept from the RPI enhancements. FTP now uses DEC login security. The normal LOGIN.COM is run for the user who is logged in, with MODE "NETWORK". If the password is incorrect it is logged, along with the remote node number of the offending FTP client, and the port number. The user will receive a message that there were n non interactive login failures, at the next login. In addition the SYS$REM_ID shows the remote node number of the client. This can be used by programs such as FINGER to reveal the user. FTP_LOGIN uses sysgen parameter LGI_RETRY_LIM to limit attempts. This parameter is normally set to 3. IF you wish an FTP anonymous log then the anonymous account must have write access to the anonymous log directory. FTP_SERVER now produces a log file in the target user directory. This will be named FTP_SERVER.LOG unless you have defined system logical FTP_SERVER. Sample: ANONYMOUS_LOGIN.COM prevents all other access other than FTP. It is now possible using logicals FTP_RESTRICT, FTP_DIRS to create secure accounts to be used as FTP servers. This supplements the ANONYMOUS account. 2.1.2 Max write size Previously it was 2048. Now it is 2048 for stream mode, 2050 for block mode, and about 2064 for compressed mode. Note Bene: SYSGEN parameter MAXBUF must be at least 8192. 2.1.3 New logical symbols The following user logical symbols were added for FTP usage, They may be defined system wide or each user may define the logical: User Logical Result ---- ------- ------ FTP_DIRS Restricts FTP access to user directories. It is similar to FTP_ANONYMOUS_DIRS FTP_RESTRICT Restricts FTP login see section on Anonymous FTP_TIMEOUT Is the user timeout in seconds. This may be defined by user in his LOGIN.COM If zero no FTP login is possible. FTP_LOG Produces detailed log if it is 1-7 and not if 0. See the FTP help for more information. The following system logical symbols were added for FTP usage, They may be defined system wide by including them in the IP_STARTUP.COM File location: System Logical Result ------ ------- ------ FTP_SERVER Points to the FTP_SERVER.COM file This allows you to put it in a different place. Login messages: System Logical Result ------ ------- ------ FTP_ANNOUNCE Message when first connect. FTP_WELCOME Message after normal login. FTP_ANONYMOUS_WELCOME Message after anonymous login The system logical may contain either: 1. The text of the message (NO CR-LF in text please) 2. @FILENAME - The location of a file containing text. Time zone: This is normally defined by DEC startup files. System Logical Result ------ ------- ------ SYS$TIMEZONE_NAME Is now the "official" time zone name This is the DEC name used by DECnet extensions. Login process definitions: System Logical Result ------ ------- ------ FTP_LOGIN_LOG System logical to produce detailed log of login. Values=1-7 FTP_LOGIN_TIMEOUT Is the login timeout in seconds. If zero FTP connections are killed immediately. 2.1.4 New modes/structures/types Types: 1. Fortran, Print, carriage control files are now converted on fly for STRU=FILE. This guarantees that they are correctly transferred. 2. TYPE=ASCII:CONTROL For Fortran carriage control. 3. TYPE=ASCII:TELNET Structures: 1. STRU=Record This allows you to transfer files with record structure kept intact. 2. Correct EOF for STRU=FILE, TYPE=IMAGE. This fixes a longstanding problem that binary files transferred from other systems had an incorrect number of bytes. Modes: 1. MODE=Compress, (Not avail in STRU=Page) Supported by Multinet, not by Wollengong or Unix. This allows you to slightly compress files on the fly for transmission to remote sites. 2. MODE=Block, (Not avail in STRU=Page) Supported by Unix, not by Wollengong or Multinet. This mode allows multiple file transfers without reopening connections. This gets around the max number of connections problem. 2.1.5 New server Functions There are many new server features, and commands. Only a selection are mentioned here in no particular order.. 1. Recursive directory, file transfers 2. APPE file - Append file has been added. 3. HELP topic - Remote help on topics. 4. SITE CHMOD (SET PROTECTION) for remote files. Supported by UNIX. 5. SITE UMASK (SET PROTECTION/DEF) for remote process. Supported by UNIX. 6. SITE BLOCK sets the block size for binary transfers. Unique to CMU. 7. SITE PRIV shows the current priveleges. Supported by MULTINET. 8. STAT param (SHOW FILE) Gives an almost "FULL" file listing. Including file type, owner, prot.... 9. STOU Store with Unique file name has been added. 10. SYST command returns the correct system type. 11. The FTP_SERVER has changed the default behavior of parsing when an NLST command has been received: If 'file.ext' is specified, return 'file.ext' with no version. If 'file.ext;' is specified, return 'file.ext;n', where n is the highest version number of the file. If 'file.ext;*' is specified, return all file specifications with versions attached. While we're at it, convert the text to lower case. This should keep DOS and UNIX systems happy. Well, maybe not... 12. The FTP_SERVER will transfer the contents of an NLST command in ASCII (meaning ) regardless of what the TYPE has been set to. Reported by Daren Critelli, Great Plains Region, USBR. This has caused a problem with the Persoft FTP client. 2.1.6 New client Functions There are many new client features, commands and qualifiers. Only a selection are mentioned here in no particular order. 1. APPEND Command has been added 2. /APPEND has been added to RECEIVE (GET) command. 3. Many DCL COPY qualifiers have been added to SEND (PUT) Such as /SINCE/BEFORE... 4. The FTP command has a second parameter to allow execution of commands inline. 5. /USERNAME/PASSWORD qualifiers are now available on FTP command line and for OPEN command. 6. QUIET mode makes FTP behave like DCL copy command, without all of the extraneous FTP messages. This is invoked by SET QUIET ON command of /QUIET qualifier. 7. /INITIALIZATION qualifier on FTP command line executes a file before FTP brings up the command prompt. 8. Most functions now have the following qualifiers. /CONFIRM asks you for yes or no before doing it. /LOG Shows you the result of operation. 9. Both ^A and ^T can be pressed to show the progress of file transfers. These reveal the file name and number of bytes transferred. 10. /BLOCKSIZE qualifier controls the blocks size on a binary GET of a file. 11. SHOW SUMMARY shows a summary of all transmissions to date. 12. /RECURSIVE may be used to transfer directory trees. It should work with both VMS and UNIX sites. This will even work with BSD systems that violate the RFC when returning file specs from an NLST command. 13. SHOW SYSTEM_TYPE shows the remote system type. 14. Send of a "directory" will create a remote directory. 15. A number of standard "DCL" commands have been added or fixed. SET HOST, SET DEFAULT, CREATE/DIR ... 2.1.7 Alternate FTP defaults. The system manager is supplied with 3 possible CLD files Depending on which one is used FTP will default to different behavior. FTP.CLD Similar to the current one These are highly recommended. ---> FTP_NOREPLY.CLD Gives a resonably quiet FTP ala Multinet ---> FTP_QUIET.CLD Gives an extremely quiet version that may disturb some users, but is very pleasing!!! To update the DCL tables on a homogenious cluster: $ SET COMMAND FTPxxx.CLD - /TABLE=SYS$COMMON:[SYSLIB]DCLTABLES - /OUTPUT=SYS$COMMON:[SYSLIB]DCLTABLES To install the new DCL tables on all nodes of the cluster: $ RUN SYS$SYSTEM:SYSMAN SYSMAN> SET ENVIRONMENT /CLUSTER SYSMAN> DO INSTALL REPLACE SYS$LIBRARY:DCLTABLES 2.1.8 New files There are some changes that have been made to IP_STARTUP.COM. This is necessary since there are 2 executables for FTP. 1. FTP_LOGIN.EXE Initially starts up and then gets the username and password. 2. FTP_SERVER.COM Is executed after login to invoke the FTP_SERVER.EXE. This version of FTP.EXE should work with older versions of CMUIP, but the server modules FTP_LOGIN.EXE and FTP_SERVER.EXE require a new version if IPACP and a new IPDRIVER. Assuming the following line is already in INET$CONFIG: WKS:21:FTPSRV:TCP$FTPSRV:NETWRK:*:::::4:5 You must have following in IP_STARTUP.COM $! $! Define the location of the FTP login server $! $ Define/System/NoLog TCP$FTPSRV CMUIP_ROOT:[SYSEXE]FTP_LOGIN.EXE $! $! Grant necessary privs to the user server. $! Note: PHY_IO is not necessary for most clients except for $! NCSA Telnet, which uses low number ports. $! $ if (f$search("CMUIP_ROOT:[SYSEXE]FTP_SERVER.EXE") .nes. "") then - if (.not. f$file("CMUIP_ROOT:[SYSEXE]FTP_SERVER.EXE","KNOWN")) then - install add CMUIP_ROOT:[SYSEXE]FTP_SERVER.EXE - /open/header/share - /priv=(phy_Io,share) In addition the new file FTP_SERVER.COM should reside in CMUIP_ROOT:[SYSEXE] 2.1.9 New file parsing The server now parses file and directory names in a fairly intellignet manner. If they "look" like VMS names they are untouched. If they look like UNIX names, they are converted to equivalent VMS names. 2.2 Anonymous accounts 2.2.1 Creation To setup an anonymous account: 1. Create an account named "anonymous" with password guest. This is different from the previous version of FTP in that a fixed password must be used. 2. Restrict it to just NETWORK login. This is different from the previous version, which turned off NETWORK login also. 3. Set the default anonymous login file to ANONYMOUS_LOGIN.COM Previous versions did not use a login file. The sample file is designed to prevent unauthorized access. 5. Disable incoming mail 6. Set the restricted flag. This is just an added precaution, that ensures that the ANONYMOUS_LOGIN.COM is executed. 7. Create the directories to which ANONYMOUS will have access. Do not permit ANONYMOUS to own any of the files or be in the same group as the owner of the files. Set WORLD:R protection on all files and directories to be accessible, or use an ACL to grant access specifically to ANONYMOUS. 8. Setup a directory for the anonymous log files. You must also define a logical to point to that directory. in the ANONYMOUS_LOGIN.COM 9. You may further restrict the anonymous account using logicals. Example: $ SET DEF SYS$SYSTEM $ RUN SYS$SYSTEM:AUTHORIZE ADD ANONYMOUS/PASSWORD=GUEST/NOBATCH/NOLOCAL/NODIALUP/NOREMOTE - /FLAG=(DISMAIL,RESTRICTED)/UIC=[666,666] - /DEV=USER$DISK/DIR=[PUBLIC] - /LGICMD=CMUIP_ROOT:[SYSMGR]ANONYMOUS_LOGIN.COM $ CREATE /DIRECTORY USER$DISK:[ANONYMOUS_LOG]/OWNER=ANONYMOUS $ CREATE /DIRECTORY USER$DISK:[PUBLIC] You may check on the DECNET security with the following: $ DIR 0"anonymous guest":: you should get the following message: -RMS-E-FND, ACP file or directory lookup failed -SYSTEM-F-LINKEXIT, network partner exited 2.2.2 Restriction Anonymous FTP is restricted to only reading files. You may further restrict the anonymous account using logical symbols. All FTP_ANON logical names should now be placed in the logical name table FTP_NAME_TABLE, to get them out of the system name table. To do this, the following lines may be in your IP_STARTUP.COM: $! $! Create the anonymous table. $! $ CREATE/NAME_TABLE/EXEC/PROT=(S:RWED,O:RWED,G:R,W:R)- /PARENT=LNM$SYSTEM_DIRECTORY FTP_NAME_TABLE $ FTPDEF := DEFINE/TABLE=FTP_NAME_TABLE/EXEC/NOLOG then use FTPDEF to define the FTP_ANON... logical names, for example: $! $! Restrict anonymous to particular directories $! $ FTPDEF FTP_ANONYMOUS_DIRS USER$DISK:[PUBLIC...] $! $! Prevent login when load is too great on prime days $! some floating-point number between 0.0 and 1.0. $! $ FTPDEF FTP_ANON_LOAD_THRESHOLD "0.5" $! $! Define prime days $! $ FTPDEF FTP_ANON_PRIME_DAYS "2,3,4" ! Tuesday, Wednesday, Thursday $! $! Define prime hours $! $ FTPDEF FTP_ANON_PRIMETIME_START 8:00 ! 8:00 am $ FTPDEF FTP_ANON_PRIMETIME_END 16:00 ! 4:00 pm The only required logical is FTP_ANON_LOAD_THRESHOLD. If that logical name exists and the LAV0 device exists, the load checking code is used. The code does the following: If FTP ANON_PRIME_DAYS is defined, it is translated. The comma-separated list of numbers (where 1=Monday, 2=Tuesday, etc.) is used to identify the days in which "prime time" is effective. If it does not exist, "prime time" is assumed to be in effect Monday through Friday. Note: Use ONLY numbers 1 through 7, and NO SPACES in the string. Surround the string with quotation marks when defining! If FTP_ANON_PRIMETIME_START is defined, it is translated and converted into a system date-time value using LIB$CONVERT_DATE_STRING. If not, then 09:00 is used as the start of "prime time". If FTP_ANON_PRIMETIME_END is defined, it is translated and converted into a system date-time value using LIB$CONVERT_DATE_STRING. If not, then 17:00 (5:00 pm) is used as the end of "prime time". If the current time is between the prime-time start and end times, then the current load averages are read from the LAV device. The current load is computed using the following formula: load = M15 * (P15 / 4.0) where M15 is the average load over the last 15 minutes, and P15 is the average priority over the last 15 minutes. Thus, the average load is normalized against typical interactive priority to guard against low-priority batch jobs preventing guest login access. If the load is greater than or equal to the LOAD_THRESHOLD value, then the guest login is denied with a reason of "system too busy". If the threshold is not exceeded, then the guest login is accepted, but the user is warned to minimize access during prime time (with the start and end times displayed along with the time zone information [if FTP_ANON_TIME_ZONE is defined]). If the current time does not fall within prime time, no load checking is performed. You can restrict the directories to which ANONYMOUS has access by defining the system-wide logical name FTP_ANONYMOUS_DIRS to a search list of device/directory specifications. Any RETR, LIST, or NLST will check against this list before going through normal system access checks. This prevents ANONYMOUS from gaining access to people's files via WORLD access. If you do not define FTP_ANONYMOUS_DIRS, the extra access checks do not take place. You can use [directory...] notation to allow access to the entire subdirectory tree below the specified directory. Make sure it is defined before allowing Anonymous access. Make sure that the default device/directory in the UAF is accessible (not strictly necessary, but easier on the users). Example: $ DEFINE/SYSTEM FTP_ANONYMOUS_DIRS - USER$DISK:[PUBLIC...],- ! public files USER$DISK:[NEWS...],- ! news archives USER$DISK:[MAIL] ! mail archives While these logicals were meant to enhance the security of Anonymous FTP, neither the author nor his employer (nor anyone else for that matter) guarantees that the software is secure. 2.2.3 Unrestriction A new logical has been added to remove any Anonymous restrictions. Normally anonymous can read files, but not write then. To remove this restriction: The following is added to the ANONYMOUS_LOGIN.COM $!Sample: Give anonymous users all access to account. $! $ DEFINE FTP_RESTRICT 0 If you want only partial restriction you may define a different value, by adding together all the values you wish: Logical Name Value Description ------- ---- ----- ----------- FTP_RESTRICT number Puts restrictions on FTP actions 1 No read (RETR) 2 No write (STOR,STOU,APPE,MKD) 4 No Control (SITE) 8 No Delete (DELE,RMD) 16 No List (LIST,NLST,STAT param) 3. Variance from RFC959 The following from RFC959 are unsupported. 1. PASV - This is probably only significant for 3'rd party FTP tranfers. 2. REIN - Reinitialize server. This is difficult to implement, but it could be done. It is not generally supported by other VMS implementations of FTP. 3. SMNT - Mount command. This is potentially useful, and may be implemented in the future. 4. TYPE=EBCDIC 1. MODE=Compress. This mode is supposed to signal EOF by an escape code of <64>. Unfortunately Multinet does not recognize this for STRU=VMS or STRU=FILE!!! For compatability CMU also closes the connection to signal EOF. CMU adheres to the standard on receive, and recognizes <64>. All of the proposed mail enhancements to FTP which are not in RFC959 are not supported. In addition the proposed 3'rd party FTP which is not in the RFC is likewise omitted. 4. Unresolved Problems Some servers do not recognize the SYST command. If you have REPLY=ON mode, this will generate an error message. This is known to happen with the SUN Ftp server. The message may be safely ignored, or use /NOREPLY.