[next] [previous] [contents] [full-page]6.1 - "Securing" The Site
6.1.1 - Directory Listings
6.1.2 - Server Reports
6.1.3 - Scripting
6.1.4 - Server Side Includes
6.1.5 - Authorization
6.1.6 - Miscellaneous Issues
6.2 - Content-Type Configuration
6.3 - Virtual Services
6.4 - Error Reporting
6.4.1 - Basic and Detailed
6.4.2 - Site Specific
6.5 - Access Logging
6.5.1 - Log Format
6.5.2 - Log Period
6.5.3 - Log Per-Service
6.5.4 - Log Naming
6.5.5 - Access Tracking
Server configuration basically concerns itself the very fundamental behaviour of the server process. Requirements such as buffer and cache sizes, timeout values, scripting limits, content-types, icons, number and type of services offerted, etc., are determined from configuration information.
By default, the system-table logical name HTTPD$CONFIG locates a common
configuration file, unless an individual configuration file is specified using
a job-table logical name. Simple editing of this file changes the
configuration. Comment lines may be included by prefixing them with the hash
"#" character. Configuration file directives are not case-sensitive. Any
changes to the configuration file can only be enabled by restarting the HTTPd
process using the following command on the server system.
$ HTTPD /DO=RESTART
A server's currently loaded configuration can be interrogated. See
15 - Server Administration for further information.
6.1 - "Securing" The Site
This section does not pretend to be a complete guide to keeping the "bad guys" out. It does provide a short guide to making a site more-or-less liberal in the way the server supplies information about the site and itself. The reader is also strongly recommended to a number of hard and Web based resources on this topic.
The WASD package had it's genesis in making the VMS operating system and
associated resources, in a development environment, available via Web
technology. For this reason the default configuration is fairly liberal,
providing information of use in a technical environment, but that may be
superfluous or less-than-desirable in other, possibly commercial environments.
For instance, directory listings can contain VMS file system META information,
error reports can be generated with similar references along with reporting
source code module and line information. This section refers to configuration
directives discussed in more detail later in this chapter and in the next.
6.1.1 - Directory Listings
Published guidelines for securing a Web site generally advise against automatic directory listing generation. Where a home page is not available this may leak information on other directory contents, provide parent and child directory access, etc. Compounding this is the WASD facility to force a listing by providing a directory URL with file wildcards (not to decry it's usefulness in some environments).
Reports are pages generated by the server, usually to indicate an error or other non-success condition, but sometimes to indicate success (e.g. after a successful file upload). Reports provide either basic or detailed information about the situation. Sometimes the detailed information includes VMS file system details, system status codes etc. To limit this information to a minimum indication adjust the following directives.
Scripting has been a notorious source of server compromise, particularly within Unix environments where script process shell command-line issues require special attention. The WASD CGI scripting interface does not pass any arguments on the command line, and is careful not to allow substitution when constructing it's CGI environment. Never-the-less, script behaviours cannot be guranteed and care should be exercised in their deployment.
SSI documents are pages containing special markup directives interpreted by the server and replaced with dynamic content. This can include detail about the server, the file or files making up the document, and can even include DCL commands and procedure activation for supplying content into the page. All this by anyone who can author on the site.
Authorization issues imply controlling access to various resources and actions and therefore require careful planning and implementation if compromise is to be avoided. WASD has a quite capable and versatile authorization and authentication environment, with a significant number of considerations. The reader refered to the chapter on this topic, 12 - Authentication and Authorization.
WASD authorization cannot be enabled without the administrator configuring at least three resources, and so therefore cannot easily be "accidentally" activated. One of these is the addition of a startup qualifier controlling where authentication information may be sourced. Another the server configuration file. The third, mapping paths against authorization configuration.
For sites that may be particularly sensitive about inadvertant access to
some resources it is possible to use the authorization configuration file as a
type of cross-check on the mapping configuration file. The server
/AUTHORIZATION=ALL startup qualifier forces all access to be authorized (even
if some are marked "none"). This means that if something
"escapes" via the mapping file it will very likely be "caught" by
an absence in the authorization file.
6.1.6 - Miscellaneous Issues
Although it is of limited usefulness because server identity may be deduced from behaviour and other indicators the exact server and version may be obscured by using the otherwise undocumented /SOFTWARE= qualifier to change the server identification string to (basically) whatever the administrator desires. This identification is included as part of all HTTP response headers.
Historically and by default server configuration and authorization sources
are contained within the server package tree. There is no reason why they
cannot be located anywhere the site prefers. Generally all that is required is
a change to logical name definition and server startup.
Package Tree
On many sites there may be little need to access the full, or any of the WASD package tree. A combination of mapping and/or authorization rules can relatively simply block or control access to it. These examples can be easily tailored to suit a site's specific requirements.
This example shows blocking all access to the /ht_root/ tree, except for
documentation, source code, examples and exercise (performance results) areas.
# HTTPD$MAP
pass /ht_root/doc/*
pass /ht_root/src/*
pass /ht_root/example/*
pass /ht_root/exercise/*
fail /ht_root/*
The next example forbids all access to the package tree unless authorized
(the authorization detail would vary according to the site). It also allows
modify access for the Admin Menu and to the /ht_root/local/ area.
# HTTPD$MAP
pass /ht_root/*
# HTTPD$AUTH
[WASD_WEB_ADMIN=id]
/httpd/-/admin/* r+w
/ht_root/local/* r+w
/ht_root/* r
6.2 - Content-Type Configuration
HTTP uses a quasi-standard implementation of the MIME (Multi-purpose
Internet Mail Extensions) specification for identifying the type of data
returned in a response. A MIME content-type consists of a plain text string
describing the data as a type and slash-separated
subtype, as illustrated in the following examples:
text/html
text/plain
image/gif
image/jpeg
application/octet-stream
The content-type is returned to the client as part of the HTTP response,
the client then using this information to correctly process and present the
data contained in that response.
In common with most HTTP servers WASD uses a file's type (extension, suffix, e.g. ".HTML", ".TXT", ".GIF") to identify the data type within the file. The [AddType] directive (see below) is used in configuration to bind a file type to a MIME content-type. To make the server recognise and return specific content-types the [AddType] directives matches file types to content-types.
NOTE
When adding a totally new content-type to the configuration be sure also to bind an icon to that type using the [AddIcon] directive (see below). When this is not done a directory listing shows "[?]" in place of an icon.
With the VMS file system there is no effective file characteristic or algorithm for identifying a file's content without an exhaustive examination of the data contained there-in ... a very expensive process (and probably still inconclusive in many cases), hence the reliance on the file type.
If a file type is not recognised (i.e. no [AddType] corresponding to the
file type) then by default WASD identifies its data as
application/octet-stream (i.e. essentially binary data). Most
browsers respond to this content-type with a download dialog, allowing the
data to be saved as a file. Most commonly these unknown types manifest
themselves when authors use "interesting" file names to indicate their
purpose. Here are some examples the author has encountered:
README.VMS
README.1ST
READ-ME.FIRST
BUILD.INSTRUCTIONS
MANUAL.PT1 (.PT2, ...)
If the site administrator would prefer another default content-type,
perhaps "text/plain" so that any unidentified files default to plain
text, then this may be configured by specifying that content-type as the
description of the catch-all file type entry. Examples (use one
of):
[AddType]
* internal/x-unknown -
* internal/x-unknown - application/octet-stream
* internal/x-unknown - text/plain
* internal/x-unknown - something/else-entirely
It is the author's opinion that unidentified file types should remain as
binary downloads, not "text" documents, which they are probably more
often not, but it's there if it's wanted.
Explicitly Specifying Content-Type
When accessing files it is possible to explicitly specify the identifying content-type to be returned to the browser in the HTTP response header. Of course this does not change the actual content of the file, just the header content-type! This is primarily provided to allow access to plain-text documents that have obscure, non-"standard" or non-configured file extensions.
It could also be used for other purposes, "forcing" the browser to accept a particular file as a particular content-type. This can be useful if the extension is not configured (as mentioned above) or in the case where the file contains data of a known content-type but with an extension conflicting with an already configured extension specifying data of a different content-type.
Enter the file path into the browser's URL specification field ("Location:",
"Address:"). Then, for plain-text, append the following query string:
?httpd=content&type=text/plain
For another content-type substitute it appropriately.
For example, to retrieve a text file in binary (why I can't imagine :^) use
?httpd=content&type=application/octet-stream
This is an example:
file.unknown file.unknown?httpd=content&type=text/plain
It is posssible to "force" the content-type for all files in a particular
directory. Enter the path to the directory and then add
?httpd=index&type=text/plain
(or what-ever type is desired). Links to files in the listing will contain the appropriate "?httpd=content&type=..." appended as a query string.
This is an example:
*.* *.*?httpd=index&type=text/plain
The WASD server is capable of concurrently supporting the same host name on different port numbers and a number of different host names (DNS aliased or multi-homed) using the same port number. This capability is generally known as a virtual server. Virtual services offer versatile and powerful multi-site capabilities using the one system and server process. Service determination is based on the contents of the request's "Host:" header field. If none is present it defaults to base service for the interface's IP address and port.
The same mechanism also effectively allows a single instance of the configuration files to support multiple server processes (using the /SERVICE qualifier), either on the one system or across multiple systems, as in a cluster (i.e. supports "virtual" and "real" servers.) See STARTUP_SERVER.COM for further information on startup support for these configurations.
WASD provides server process run-time parameters via the HTTPD$CONFIG
configuration file. These provide settings for logging, scripting, timeouts,
file content-type mappings, etc. The HTTPD$MSG file provides configurable
system messages.
[Service]
Using the [Service] HTTPD$CONFIG configuration parameter or the /SERVICE qualifier the server creates an HTTP service for each specified. If the host name is omitted it defaults to the local host name. If the port is omitted it defaults to 80. The first port specified in the service list becomes the "administration" port of the server, using the local host name, appearing in administration reports, menus, etc. This port is also that specified when sending control commands via the /DO= qualifier (see 5.3.2 - Server Command Line Control).
This rather contrived example shows a server configured to provide four
services over two host names.
[Service]
alpha.wasd.dsto.defence.gov.au
alpha.wasd.dsto.defence.gov.au:8080
beta.wasd.dsto.defence.gov.au
beta.wasd.dsto.defence.gov.au:8000
Note that both the HTTPD$SERVICE configuration file (see
8 - Service Configuration and the /SERVICE= command-line
qualifier (see 5.3 - HTTPd Command Line) override this directive.
HTTPD$SERVICE
If the logical name HTTPD$SERVICE is defined (and it is optional) the HTTPD$CONFIG [Service] directive is not used. As mentioned, this configuration file is optional. For simple sites, those containing one or two services, the use of a separate service configuration file is probably not warranted. Once the number begins to grow this file offers an easier-to-manage interface for those services.
See 8 - Service Configuration for further detail.
[[virtual-server]]
The essential profile of a site is established by it's mapped resources and any authorization controls, the HTTPD$MAP and HTTPD$AUTH configuration files respectively, and these two files support directives that allow configuration rules to be applied to all virtual services (i.e. a default), to a host name (all ports), or to a single specified service (host name and specific port).
To restrict rules to a specified server (virtual or real) add a line
containing the server host name, and optionally a port number, between
double-square brackets. All following rules will be applied only to that
service. If a port number is not present it applies to all ports for that
service name, otherwise only to the service using that port. To resume
applying rules to all services use a single asterisk instead of a host name. In
this way default (all service) and server-specific rules may be interleaved to
build a composite environment, server-specific yet with defaults. Note that
service-specific and service-common rules may be mixed in any order allowing
common rules to be shared. This descriptive example shows a file with one rule
per line.
# just an example
this rule applies to all services
so does this
and this one
[[alpha.wasd.dsto.defence.gov.au]]
this one however applies only to ALPHA, but to all ports
as indeed does this
[[beta.wasd.dsto.defence.gov.au:8000]]
now we switch to the BETA service, but only port 8000
another one only applying to BETA
and a third
[[*]]
now we have a couple default rules
that again apply to all servers
Both the mapping and authorization modules report if rules are provided
for services that are not configured for the particular server process (i.e.
not in the server's [Service] or /SERVICE parameter list). This provides
feedback to the site administrator about any configuration problems that
exist, but may also appear if a set of rules are shared between multiple
processes on a system or cluster where processes deliver differing services.
In this latter case the reports can be considered informational, but should be
checked initially and then occasionally for misconfiguration.
Unknown Virtual Server
If a service is not configured for the particular host address and port of a request one of two actions will be taken.
[ServiceNotFoundURL] //server.host.name/httpd/-/servicenotfound.html
pass /*/-/admin/* pass /*/-/* /ht_root/runtime/*/* exec /cgi-bin/* /cgi-bin/* [[virtual1.host.name]] /* /web/virtual1/* / /web/virtual1/ [[virtual2.host.name]] /* /web/virtual2/* / /web/virtual2/ [[virtual3.host.name]] /* /web/virtual3/* / /web/virtual3/ [[*]] /* /web/servicenotfound.html
This applies to dotted-decimal addresses as well as alpha-numeric. Therefore if there is a requirement to connect via a numeric IP address such a service must have been configured.
Note also that the converse is possible. That is, it's possible to
configure a service that the server cannot ever possibly respond to because it
does not have an interface using the IP address represented by the service
host.
6.4 - Error Reporting
By default the server provides it's own internal error reporting facility.
These reports may be configured as basic or detailed on
a per-path basis, as well as determining the basic "look-and-feel". For
more demanding requirements the [ErrorReportPath] configuration directive
allows a redirection path to be specified for error reporting, permitting the
site administrator to tailor both the nature and format of the information
provided. A Server Side Include document, CGI script or even standard HTML
file(s) may be specified. Generally an SSI document would be recommended for
it's simplicity yet versatility.
6.4.1 - Basic and Detailed
Internally generated error reports are the most efficient. These can be delivered with two levels of error information. The default is more detailed.
ERROR 404 - The requested resource could not be found.
Document not found ... /ht_root/index.html
(document, bookmark, or reference requires revision)
Additional information: 1xx, 2xx, 3xx, 4xx, 5xx, Help
WASD/7.0.0 Server at wasd.dsto.defence.gov.au Port 80
There is also the more basic.
ERROR 404 - The requested resource could not be found.
Additional information: 1xx, 2xx, 3xx, 4xx, 5xx, Help
WASD/7.0.0 Server at wasd.dsto.defence.gov.au Port 80
These can be set per-server using the [ReportBasicOnly] configuration directive, or on a per-path basis in the HTTPD$MAP configuration file. The basic report is intended for environments where traditionally a minimum of information might be provided to the user community, both to reduce site configuration information leakage but also where a general user population may only need or want the information that a document was either found or not found. The detailed report often provides far more specific information as to the nature of the event and so may be more appropriate to a more technical group of users. Either way it is relatively simple to provide one as the default and the other for specific audiences. Note that the detailed report also includes in page <META> information the code module and line references for reported errors.
To default to a basic report for all but selected resource paths introduce
the following to the top of the HTTPD$MAP configuration file.
# default is basic reports
set /* report=basic
set /internal-documents/* report=detailed
set /other/path/* report=detailed
To provide the converse, default to a detailed report for all but selected
paths use the following.
# default is detailed reports
set /web/* report=basic
Other Customization
The additional reference information included in the report may be disabled using the appropriate HTTPD$MSG [status] message item. Emptying this message results in an error report similar to the following.
ERROR 404 - The requested resource could not be found.
WASD/7.0.0 Server at wasd.dsto.defence.gov.au Port 80
The server signature may be disabled using the HTTPD$CONFIG [ServerSignature] configuration directive. This results in a minimal error report.
ERROR 404 - The requested resource could not be found.
A simple approach to providing a site-specific "look-and-feel" to
server reports is to customize the [ServerReportBodyTag] HTTPD$CONFIG
configuration directive. Using this directive report page background colour,
background image, text and link colours, etc., may be specified for all
reports. It is also possible to more significantly change the report format
and contents (within some constraints), without resorting to the site-specific
mechansims refered to below, by changing the contents of the appropriate
HTTPD$MSG [status] item. This should be undertaken with care.
6.4.2 - Site Specific
Site-specific error reporting works by internal redirection. When an error
is reported the original request is concluded and the request reconstructed
using the error report path before internally being reprocessed. Error
information becomes available via a specially-built query string, and from
that as CGI variables in the error report context. One implication is the
original request path and query string are no longer available. All error
information must be obtained from the error information in the new query
string.
Using an SSI Document
Three example SSI error report documents are provided.
It is suggested with any use of this facility the reporting
document(s) be located somewhere local, probably HT_ROOT:[LOCAL], and then
enabled by placing the appropriate path into the [ErrorReportPath]
configuration directive, as shown for the examples here
[ErrorReportPath] /ht_root/local/reporterror.shtml
Note that virtual services can subsequently have this path mapped to other
documents (or even scripts) so that some or all services may have custom error
reports. For instance the following arrangement provides each host (service)
with an customized error report.
# HTTPD$CONFIG
[ErrorReportPath] /errorreport.shtml
# HTTPD$MAP
[[alpha.wasd.dsto.gov.au]]
pass /errorreport.shtml /ht_root/local/alphareport.shtml
[[beta.wasd.dsto.gov.au]]
pass /errorreport.shtml /ht_root/local/betareport.shtml
[[gamma.wasd.dsto.gov.au]]
pass /errorreport.shtml /ht_root/local/gammareport.shtml
The following SSI variables are available specifically for generating error reports. The <!--#printenv --> statement near the top of the file may be uncommented to view all SSI and CGI variables available.
Once the SSI report document has been satifactorily developed and is to be
put into production it is recommended it be SCRUNCHED, making
it's processing significantly more efficient. See 20.4 - Scrunch Utility.
Using a Script
It is also possible to report using a script. This is discouraged dues to
it's relative complexity, and should only be undertaken for verify specific
reasons of limitation within the SSI implementation. The source code
HT_ROOT:[SRC.MISC]REPORTERROR.C
provides such an implementation example.
Using HTML Documents
SSI documents are the obvious choice for reporting errors but they do take
time and CPU for processing, and this may be a significant consideration on
busy sites. If prefered, static HTML documents may be used to report errors.
Generally one per possible response error status code is required. When
providing a error report path including a "!UL" introduces the response
status code into the file path, providing a report path that includes a three
digit number representing the error. A file for each possible number must then
be provided.
[ErrorReportPath] /ht_root/local/reporterror!UL.html
This mapping will generate paths such as the following, and require at
least these to respond to general errors.
/ht_root/local/reporterror400.html
/ht_root/local/reporterror401.html
/ht_root/local/reporterror403.html
/ht_root/local/reporterror404.html
/ht_root/local/reporterror409.html
/ht_root/local/reporterror500.html
/ht_root/local/reporterror501.html
/ht_root/local/reporterror502.html
/ht_root/local/reporterror503.html
6.5 - Access Logging
WASD provides a versatile access log, allowing data to be collected in
Web-standard common and combined formats, as well as
allowing customization of the log record format. It is also possible to
specify a log period. If this is done log files are automatically changed
according to the period specified. Exclude requests from specified hosts
using the [LogExcludeHosts] configuration parameter.
6.5.1 - Log Format
The configuration parameter [LogFormat] and the server qualifier /FORMAT specifies one of three pre-defined formats, or a user-definable format. Most log analysis tools can process the three pre-defined formats. There is a small performance impost when using the user-defined format, as the log entry must be specially formatted for each request.
The user-defined format allows customised log formats to be specified using a selection of commonly required data. The specification must begin with a character that is used as a substitute when a particular field is empty (use "\0" for no substitute, as in the "windows log format" example below).
Two different "escape" characters introduce the following parameters:
A "!" followed by
A "\" followed by
Any other character is directly inserted into the log entry.
Note on "PA" and "RQ"
The "PA" and "RQ" have distinct roles. In general the
"RQ" (request) directive will always be used as this is the full
request string; script component (if any), path string and query string
component (if any). The "PA" directive is merely the path string after
any script and query string components have been removed.
Examples
-!CN - !AU [!TC] \q!RQ\q !RS !BY
-!CN - !AU [!TC] \q!RQ\q !RS !BY \q!RF\q \q!UA\q
\0!TC\t!CA\t!SN\t!AR\t!AU\t!ME\t!PA\t!RQ\t!EM\t!UA\t!RS\t!BB\t
-!CN - !AU [!TC] \q!RQ\q !RS !BY !ES
The access log file may have a period specified against it, producing an automatic generation of log file based on that period. This allows logs to be systematically named, ordered and kept to a managable size. The period specified can be one of
The log file changes on the first request after midnight of the new period. When using a weekly period the new log file comes into effect on the first request following midnight on the specified day.
When using a periodic log file, the file name specified by HTTPD$LOG or the configuration parameter [LogFileName] is partially ignored, only partially because the directory component of it is used to located the generated file name. The periodic log file name generated comprises
HT_LOGS:WASD_80_19971013_ACCESS.LOG
For the daily period the date represents the request date. For the weekly
period it is the date of the previous (or current) day specified. That is, if
the request occurs on the Wednesday for a weekly period specified by Monday
the log date show the last Monday's. For the monthly period it uses the first.
6.5.3 - Log Per-Service
By default a single access log file is created for each HTTP server
process. Using the [LogPerService] configuration directive a log file for each
service provided by the HTTPd is generated (see 6.3 - Virtual Services).
The [LogNaming] format can be any of "NAME" (default) which names the log file
using the first period-delimited component of the IP host name, "HOST" which
uses as much of the IP host name as can be accomodated within the maximum 39
character filename limitation (of ODS-2), or "ADDRESS" which uses the full IP
host address in the name. Both HOST and ADDRESS have hyphens substituted for
periods in the string. If these are specified then by default the service port
follows the host name component. This may be suppressed using the
[LogPerServiceHostOnly] directive, allowing a minimum extra 3 characters in the
name, and combining entries for all ports associated with the host name
(for example, a standard HTTP service on port 80 and an SSL service on port 443
would have entries in the one file).
6.5.4 - Log Naming
When per-period or per-service logging is enabled the access log file has a name generated. Part of this name is the host's name or IP address. By default the host name is used, however if the host IP address is specified the dot-numeric address is used, hyphens being substituted for the periods. Accepted values for the [LogNaming] configuration directive are:
Examples of generated per-service (non-per-period) log names:
HT_LOGS:131-185-250-202_80_ACCESS.LOG
HT_LOGS:WASD-DSTO-DEFENCE-GOV-AU_80_ACCESS.LOG
HT_LOGS:WASD_80_ACCESS.LOG
Examples of generated per-period (with/without per-service) log names:
HT_LOGS:131-185-250-202_80_19971013_ACCESS.LOG
HT_LOGS:WASD-DSTO-DEFENCE-GO_80_19971013_ACCESS.LOG
HT_LOGS:WASD_80_19971013_ACCESS.LOG
6.5.5 - Access Tracking
The term access tracking describes the ability to follow a single user's accesses through a particular site or group of related sites. This is accomplished by setting a unique cookie in a user's browser. This cookie is then sent with all requests to that site. The site detects the cookie's unique identifier, or token, and includes it the access log, allowing the user's route through the site or sites to be reviewed. Note that a browser must have cookies enabled for this mechanism to operate.
WASD access tracking is controlled using the [Track...] directives. The tracking cookie uses an opaque, nineteen character string as the token (e.g. "ORoKJAOef8sAAAkuACc"). This token is spatially and temporally completely unique, generated the first time a user's browser accesses the site. This token is by default added to the server access log in the common format "remote-ID" location. It can also be placed into custom logs. From this identifier in the logs a session's progress may be easily tracked. Note that the token contains nothing related to the user's actual identity! It is merely a unique identifier that tags a single browser's access trail through a site.
The [Track] directive enables access tracking on a per-server basis. By default all non-proxy services will then have tracking enabled. Individual services may be then be disabled (or enabled in the case of proxy services) using the per-service ";notrack" and ";track" parameters.
By default a session track token expires when the user closes the browser. To encourage the browser to keep this token between uses enable multi-session tracking using the [TrackMultiSession] directive. Note that browsers may dispose of any cookie at any time resources become scarce, and that users can also remove them.
Session tracking can be extended from the default of the local server
(virtual if applicable) to a group of servers within a local domain. This
means the same, initial identifier appears in the logs of all WASD servers in a
related group of hosts. Of course tracking must be enabled on all servers.
The host grouping is specified using the [TrackDomain] directive (this follows
the general rules governing cookie domain behaviour - see RFC2109). Most host
grouping require a minimum of three dots in the specification.
For example (note the leading dot)
.site.org.domain
which would match the following servers, "curly.site.org.domain", "larry.site.org.domain", "moe.site.org.domain", etc. Sites in top-level domains (e.g. "edu", "com", "org") need only specify a minimum of two periods.