Documentation for HTTP:EXPORT-URL

HTTP:EXPORT-URL [function]: (URL EXPORT-TYPE &REST ARGS)

HTTP:EXPORT-URL is the primary method that exports URLS to make them accessible via the http
server. URL is either a string or an interned URL to be exported. EXPORT-TYPE is the method to use
in exporting URL.

I. Basic Export Types: These involve making the contents of a file accessible via a URL. These types
require URLs that are object (i.e., have a name and extension).

:HTML-FILE (&key pathname)
:TEXT-FILE (&key pathname)
:XML-FILE (&key pathname)
:CSS-FILE (&key pathname)
:RTF-FILE (&key pathname)
:LISP-FILE (&key pathname)

:GIF-IMAGE (&key pathname)
:JPEG-IMAGE (&key pathname)
:X-BITMAP-IMAGE (&key pathname)
:PICT-IMAGE (&key pathname)
:TIFF-IMAGE (&key pathname)
:PNG-IMAGE (&key pathname)
:ICO-IMAGE (&key pathname)

:BASIC-AUDIO (&key pathname)
:AIFF-AUDIO (&key pathname)
:WAV-AUDIO (&key pathname)
:REAL-AUDIO (&key pathname)

:MPEG-VIDEO (&key pathname)
:QUICKTIME-VIDEO (&key pathname)

:VRML-WORLD (&key pathname)
:SHOCKWAVE-FILE (&key pathname)

:DIRECTORY (&key pathname immediate-export recache recursive-p)
:HTML-DIRECTORY (&key pathname immediate-export recache recursive-p)
:TEXT-DIRECTORY (&key pathname immediate-export recache recursive-p)
:LISP-DIRECTORY (&key pathname immediate-export recache recursive-p)
:IMAGE-DIRECTORY (&key pathname immediate-export recache recursive-p)
:AUDIO-DIRECTORY (&key pathname immediate-export recache recursive-p)
:VIDEO-DIRECTORY (&key pathname immediate-export recache recursive-p)
:WORLD-DIRECTORY (&key pathname immediate-export recache recursive-p)
:APPLICATION-DIRECTORY (&key pathname immediate-export recache recursive-p)

:DIRECTORY exports all files whose resource type is known. Others export a specific content type,
and ignore other file types. When recursive-p is non-null, all subdirectories are also exported.
Otherwise, subdirectories are ignored.

When *AUTO-EXPORT* is non-null, new files are automatically exported when they are scoped by one of
these directory export types. Auto-export occurs on demand for the GET and HEAD methods. If
*AUTO-EXPORT* is :ON-DEMAND, files are exported when they are first requested rather than at the
time the directory is exported. When exporting a directory, a non-null argument to :IMMEDIATE-EXPORT
overrides lazy directory exporting. In general, on-demand directory exports trade faster server
start up for a slightly slower first access to a file URL within the directory. When :RECACHE is
non-null, a directory export updates the export parameters for every inferior. This parameter forces
traversal of the entire structure, like a non-null :IMMEDIATE-EXPORT would.

A directory-style list in HTML is the default content returned for the get method on a URL directory
path. This behavior is customized by providing a response function via the :DIRECTORY-WRITER
keyword. This response function is called with the arguments (URL STREAM) and must return non-null
when it handles the response. If it declines to handle the response, it may return NIL, and the
standard method will list the directory as usual. HTTP:WRITE-INDEXED-DIRECTORY-LISTING is a
directory writer that will serve the contents of an index.html file found in the directory. Other
computed returns are possible.

Note that the presence in file or directory names of escaped characters (see *ESCAPED-CHARACTERS*)
will lead to inconsistent results, and possibly errors. Space and question mark are examples.

:PDF-FILE (&key pathname)
:POSTSCRIPT-FILE (&key pathname)

:BINHEX-FILE (&key pathname)
:STUFFIT-FILE (&key pathname)
:COMPRESSED-FILE (&key pathname)
:MAC-BINARY-FILE (&key pathname)

:WORD-FILE (&key pathname)
:POWER-POINT-FILE (&key pathname)
:EXCEL-FILE (&key pathname)

The Java language provides applets that execute on the client. This kind of mobile code is
supported with the following export types and the HMTL generation macro WITH-JAVA-APPLET.

:JAVA-FILE (&key pathname)
:JAVA-BINARY (&key pathname)

:JAVA-FILE exports the source code whereas :JAVA-BINARY provides the byte compiled binary to the
client running the applet.

Various scripting languages can be exported for use in browsers

:JAVA-SCRIPT-FILE (&key pathname)
:TCL-FILE (&key pathname)
:VISUAL-BASIC-SCRIPT-FILE (&key pathname)

:JAVA-SCRIPT-FILE exports the browser scripting language, JavaScript, which is simpler, and easier
to use than Java itself. :TCL-FILE export the popular TCL user interface language while
:VISUAL-BASIC-SCRIPT-FILE exports a MicroSoft Visual Basic Script. Such scripts can be emitted in
HTML 4.0.1 as client-side mobile code using the WITH-SCRIPT macro.

The following export types support inline plug-ins on the client side. Plug-ins can be referenced
using NS2.0:WITH-EMBEDDED-SCRIPT (deprecated).

Inline speech synthesis is available using a macintalk plug-in
from http://www.mvpsolutions.com/PlugInSite/Talker.html

:TALK-AUDIO (&key pathname)

II. Redirect Export Types: These export types inform the client to look elsewhere for a URL. They
work for all operations, including GET and HEAD. The exported URL can be either an HTTP object or
an HTTP path. ALTERNATE-URLS is a list of one or more target URLs where the resource may be
obtained. When more than one target URL is provided, the client is free to choose among them.

:REDIRECT (&key alternate-urls)
:TEMPORARY-REDIRECT (&key alternate-urls)

Alternatively, a computed response may call REDIRECT-REQUEST to issue a redirect rather than serving
content itself.

III. Search Export Types: these involve performing searches using the search index or map search
facilities in HTTP. Search URLs must end with ? so that the system can composed the right
combination of classes. In all cases, the response function must compute the answer and return HTML
over the http stream to the client.

General Purpose Searches

:SEARCH (&key response-function search-parser search-database)

This exports a URL that performs searches by calling RESPONSE-FUNCTION with the arguments
URL and STREAM. The search database and search parameters are cached on the URL and
accessible via URL:SEARCH-DATABASE and URL:SEARCH-KEYS.

The optional export argument SEARCH-PARSER is the parser that obtains URL:SEARCH-KEYS from
the suffix of a search URL. Several parsers are predefined:

URL:STANDARD-PARSE-SEARCH-INFO: This standard parser for search URLs. It tests
whether the search info encodes form or list data and calls the appropriate of the
next two parsers.

URL:PARSE-SEARCH-INFO: This normal parser for search URLs produces a list of search
parameters using + as the delimiter.

URL:PARSE-SEARCH-INFO-AS-QUERY-ALIST: This parser for URL encoded form values
returns an alist just like posting a form would. This parser should be used when an
HTML form is returned to a search URL. However, this method of returning form
values is limited to 1024 characters total in the URL, and therefore, it's use in
new code is deprecated.

URL:PARSE-SEARCH-INFO-AS-XML-SAFE-QUERY-ALIST: This parser is recommend for HTML 4.0
and XHTML documents becauase it uses semi-colon to delimit form fields, thus
avoiding interactions with SGML's use of ampersand.

Users may provide other specialized parsers. They should accept the arguments (url-string
start end) and need not located the ? suffix delimiter.

The export argument, SEARCH-WRITER, allows a URL to specialize how the parsed presentation
on URL:SEARCH-KEYS is written. Several writers are predefined.

URL:STANDARD-WRITE-SEARCH-INFO: This standard writer for search URLs. It tests
whether the search info encodes form or list data and calls the appropriate of the
next two writers.

URL:WRITE-SEARCH-INFO: This normal writer for search URLs produces a list of search
parameters using + as the delimiter.

URL:WRITE-SEARCH-INFO-AS-QUERY-ALIST: This writer for URL encoded form values that
prints alist values as name value pairs using the urlencoding syntax.

URL:WRITE-SEARCH-INFO-AS-XML-SAFE-QUERY-ALIST: This writer is recommend for HTML 4.0
and XHTML documents becauase it uses semi-colon to delimit form fields, thus
avoiding interactions with SGML's use of ampersand.

The export argument, HEADER-FUNCTION, allows a search URL to specialize how it responds to
the HEAD method based on runtime knowledge. HEADER-FUNCTION is called with the argument URL
and returns values used to determine respoonse to HEAD. The returned values are:
(CONTENT-TYPE CONTENT-LENGTH LAST-MODIFICATION VERSION CHARSET PUBLIC ALLOW)

CONTENT-TYPE (required) is a keyword or content-type. Default is HTML.

CONTENT-LENGTH (optional) is the length in bytes of the entity body. Default is
none.

LAST-MODIFICATION (optional) is a universal time indicating when the entity was last
changed. Default is now.

VERSION (optional) is a number or string that distinguishes different versions of
the entity. Default is none.

CHARSET (optional) is a keyword indicating the character set of the entity. Default
is :ISO-8859-1.

PUBLIC (optional) is a boolean indicating whether the resource is available to the
public. Default is none.

ALLOW (optional) is a list of HTTP method keywords available on the resource.
Default is none.

Image Searches

Image Maps

:IMAGE-MAP (&key pathname export-type map-format map-pathname
search-parser search-writer header-function)

This exports the image in PATHNAME as IMAGE-EXPORT-TYPE and establishes a response function
based on the image map in MAP-PATHNAME whose MAP-FORMAT is either :CERN or :NCSA.
EXPORT-TYPE is the appropriate image search export type (see below).

Direct Searches

These provide direct control over the response function for image searches.

:GIF-IMAGE (&key pathname response-function search-database
search-parser search-writer header-function)
:JPEG-IMAGE (&key pathname response-function search-database
search-parser search-writer header-function)
:PNG-IMAGE (&key pathname response-function search-database
search-parser search-writer header-function)
:X-BITMAP-IMAGE (&key pathname response-function search-database
search-parser search-writer header-function)
:PICT-IMAGE (&key pathname response-function search-database)
search-parser search-writer header-function)
:TIFF-IMAGE (&key pathname response-function search-database
search-parser search-writer header-function)

These export types allow the client's user to click on images and invoke a response from the
server. These URL are both objects and searchable. When they are requested without the ?
suffix, the contents of their associate image file is returned. When the ? suffix appears,
their RESPONSE-FUNCTION is called on the arguments URL and STREAM. See the macro
HTTP:WITH-IMAGE-COORDINATES automatically binds the X and Y coordinates.

IV. Computed Export Types: These compute responses returned to clients.

:COMPUTED (&key response-function header-function) RESPONSE-FUNCTION is called with the
arguments URL and STREAM and is responsible for returning HTML to the client. :COMPUTED has
an optional pathname so that the computation may reference a file, if necessary.
HEADER-FUNCTION is documented section III.

:HTML-FORM (&key response-function pathname durable-form-values server) :HTML-FORM returns
the contents of PATHNAME when it is requested via GET. When there is a POST, its
RESPONSE-FUNCTION is called with URL, STREAM, and FORM-ALIST. FORM-ALIST is an alist of
(QUERY RAW-VALUE) for all the queries in the form that the client returns. QUERY is a
keyword. When a client returns multiple raw-values for a QUERY, these are aggregated into a
list of the values associated with the query in a single, order-preserving entry.
DURABLE-FORM-VALUES is a boolean and controls whether query values are new, copied strings
or indirect arrays pointing into a volitile form data buffer. High-volume applications or
forms large query values can gain efficiency by using durable form values, all operations on
these values must be completed within the context of the form's response function. When the
form uses the MIME multipart encoding, the MAXIMUM-UPLOAD-FILE-SIZE may supply the maximum
allowable file size, if it diverges from the default HTTP:*FILE-UPLOAD-MAXIMUM-SIZE*.

:COMPUTED-FORM (&key form-function response-function header-functiondurable-form-values
server) :COMPUTED-FORM is a cross between :COMPUTED and :HTML-FORM that provides
FORM-FUNCTION to generate html just like :COMPUTED and RESPONSE-FUNCTION to handle the post
method when form values are returned. FORM-FUNCTION is called with the arguments URL and
STREAM and is responsible for returning HTML to the client. RESPONSE-FUNCTION takes the same
arguments as :HTML-FORM.

:SCRIPT (&key script header-function) Script is a client-side script defined with
HTML:DEFINE-SCRIPT. These scripts may merely deliver a static copy of the client-side
script, or they may perform a computation that emits an appropriate script.

:SHTML-FILE (&key pathname header-function) This is a computed response that is inserted in
a static HTML file containing server-parsed HTML. When an SHTML element is found by the
server, it inserts the result of a computation in place of the SHTML element. SHTML
elements are delimted by  and look like:

INCLUDE is an SHTML operation that requires a FILE. For security reasons, FILE must be in
the same directory as the STHML file. Access may be controlled by providing the optional
SUBNETS parameter, which is a comma-separated string of IP addresses without whitespace.

EVAL is a more general SHTML operation that requires an ACTION parameter. ACTION is a string
denoting an SHTML action. Specific parameters may be required by individual actions. Here,
DATE-TIME is the operation and FORMAT is a parameter.

Predefined actions are documented by HTTP:SHOW-SHTML-ACTIONS. New SHTML actions are defined
with HTTP:DEFINE-SHTML-ACTION. Files with the extention SHTML are autoexported by HTML
directory export types.

V. Reverse Proxy Export Type

This provides a reverse proxying facility for servers, the reverse of normal client proxying. In a
reverse proxy, when a client requests a specific URL, the server proxies the request to an origin
server and returns the resulting HTTP message to the client. Thus, the reverse proxy behaves as if
it was the origin server.

To perform this task, the reverse proxy needs to know the origin server to which a particular
incoming client HTTP request is mapped. This information is provided at URL export time by the
application. Although all requests on a particular port may be reverse proxied, this facility allows
different segments of the URL space to be delegated to different origin servers according to the
local URL provided to EXPORT-URL. Applications may choose to either run with local proxy caching or
not, depending on the value of *PROXY-CACHING-P*. Normally, local proxy caching will improve
performance. Note that this facility is only available when the proxy server is loaded.

:REVERSE-PROXY (&key proxy-forwardings) proxy-forwardings is a list of (DOMAIN-NAME PORT
PROTOCOL). If more than one host is provided, forwarding distributed over the set of origin
servers. Only the most general URL needs to be exported because the facility automatically
forwards requests to inferior URLs. Although the reverse proxy enforces all security
parameters on the root reverse-proxy URL, all other issues concerning the URL are delegated
to the origin server, for example, method availability and URL existence. When desired, a
proxy may be placed between the reverse proxy and the origin server using
DEFINE-CLIENT-PROXY-MAPPING.

To handle remapping internal url contexts to external contexts, see URL:*RELATIVIZE-URLS* and
DEFINE-URL-CONTEXT-REMAPPINGS.

VI. Export Parameters and Issues

1. Portable Exports: The #u reader macro merges a partial URL specification against the default for
the local host. Use this when you be able to load the same exports file on different hosts. The #u
reader macro has an extended syntax that allows you to overview the default host and port specified
by the server configuration. The syntax is

#u(url-string :host host-string :port port-number)

URL-STRING is a relative URI. HOST-STRING is the domain name or IP string for the host. PORT-NUMBER
is an integer.

2. Security:

A. Subnet Security: Secure subnets are a list of IP addresses, where 0 is a wildcard. For example,
128.52.0.0 matches all the subnets at the AI lab. The following export arguments allow control over
which clients can perform HTTP methods such as GET, HEAD, POST, :OPTIONS, or :TRACE (read access)
versus PUT or DELETE (write access).

:READ-SUBNETS allows read access to be specified at the level of URLs as they are exported.

:WRITE-SUBNETS allows write access to be specified at the level of URLs as they are
exported.

DEFINE-READ-SUBNETS restricts read access globally to the server.

DEFINE-WRITE-SUBNETS restricts write access globally to the server.

Write access presumes read access, and consequently, IP addresses from the write subnets need not be
included in the read subnets. To select the global authentication policy for write methods,
HTTP:*ACCEPT-WRITE-METHODS*.

DEFINE-SUBNET can be used to specify named subnet for use in subnet specifications. Note that named
subnets are resolved when they are used, for example by a call to EXPORT-URL, and therefore, changes
to named subnets require re-export of URL referring to them.

B. User Authentication: URL authentication can be specified using the following export arguments.

:AUTHENTICATION-REALM a realm obtainable via INTERN-REALM. These can be created with
ADD-REALM.

:CAPABILITIES a URL access control obtainable via INTERN-ACCESS-CONTROL. These can be
created with ADD-ACCESS-CONTROL-GROUP.

See also: ADD-GROUP, ADD-USER, and SAVE-AUTHENTICATION-DATA.

C. Connection Timeouts: URL-specific timeouts can override the global timeouts controlling when an
HTTP connection can be kill

:LIFE-TIME is the number of milliseconds that an HTTP connection can live before it is
scavengeable. The gloabl default is *SERVER-LIFE-TIME*.

:TIMEOUT is 60ths of a second and controls the amount idle time before an HTTP connection is
scavengeable. The gloabl default is *SERVER-TIMEOUT*.

These value supplied for :LIFE-TIME or :TIMEOUT must be an integer, null or a function. An integer
is interpreted as timeout value in the appropriate units. When the the value is null, the field is
is cleared on the URL and the global default applies. If the value is a function, the function is
called with the argument (URL) and must return an integer.

3. Expiration: The expiration time for a url is issued as an EXPIRES header so that proxy servers
can determine when they need to refresh their cache.

Expiration is controlled by providing the :EXPIRATION when exporting any URL. If expiration is not
provided, the default is no expiration date.

The :EXPIRATION keyword takes one argument, which is either keyword or a list of (keyword &rest
arguments).

Arguments Meaning

:NO-EXPIRATION-HEADER -- No EXPIREs header is issued.
:NEVER -- EXPIRES header indicates one year from now.
(:INTERVAL <universal-time>) -- EXPIRES header indicates now + <universal-time>.
(:TIME <universal-time>) -- EXPIRES header indicates an <universal-time>.
(:FUNCTION <function-spec>) -- EXPIRES header indicates universal time computed by
applying <function-spec> to URL. <function-spec>
should return a universal for use in the EXPIRES header
or NIL, in which case no EXPIRES header is issued.

4. Character Sets: The :CHARACTER-SET keyword allows any URLs whose content type is TEXT (e.g.,
text/plain, text/html) to be exported with character sets other than the HTTP default :ISO-8859-1,
or subsets. The valid character sets for HTTP are: :US-ASCII, :ISO-8859-1, :ISO-8859-2, :ISO-8859-3,
:ISO-8859-4, :ISO-8859-5, :ISO-8859-6, :ISO-8859-7, :ISO-8859-8, :ISO-8859-9, :ISO-2022-JP,
:ISO-2022-JP, :ISO-2022-KR, :UNICODE-1-1, :UNICODE-2-2-UTF-7, :UNICODE-2-2-UTF-7. Whenever TEXT
content types use a character set other than :ISO-8859-1, the HTTP requires explicit specification
via this export keyword.

6. Languages: The :LANGUAGE keyword may be supplied for any exported URL. The value is a sequence of
keywords denoting the language(s) in which the resource is written. HTTP 1.1 defines these keywords
in section 3.10 to conform with RFC 1766. They can be a two-letter ISO 639 language abbreviation,
optionally with a two-letter ISO 3166 country code as a subtag.

7. Documentation: Keywords and a descriptive string can be attached to URLs at export time. For
directory exports, note that these arguments apply to ALL URLs exported during the directory walk.

:KEYWORDS A list of keywords.
:DOCUMENTATION A string describing the URL.

8. Virtual Hosts: HTTP 1.1 requires a virtual hosting facility, which this server implements. You
can define a virtual host (or vanity name) that will be served by the physical server from the same
IP address. Any URIs served by a virtual host must be exported by specifying the absolute URI,
including host and port number. The #u reader macro may be useful here. The following operators
are available for use with virtual hosts:

ADD-VIRTUAL-HOST: Adds a virtual host on a virtual port and and makes URLs served by that
host available to HTTP 1.1 or greater clients.

REMOVE-VIRTUAL-HOST: Makes the virtual host unavailable, but does not remove any URLs it
exports.

9. HTTP 1.1 Cache Control: The keywords below may be supplied when exporting any URL in order to
statically control how downstream proxies and caches handle the content associated with a URL.

:PUBLIC -- If the value is T, the entire message is cachable by any cache even if it would
not normally be cached.

:PRIVATE -- If the value is T, the entire message is intended for a single user and must not
be cached by a shared cache. If the value is a list of keywords denoting specific header,
then only these headers should be considered private.

:NO-CACHE -- If the value is T, the entire message must not be cached by any cache along the
request chain. If the value is a list of keywords denoting specific headers, then only
these headers should be discarded before caching.

:NO-STORE -- If the value is T, the entire message must not be stored in any non-volatile
storage by any cache along the request chain.

:MAX-AGE -- The value is the number of seconds for which the response is valid, after which
it should be revalidated. This directive overrides the expiration header, thus allowing
HTTP 1.1 server to provide a longer expiration time to HTTP 1.1 servers and proxies. This
defaults to a value derived from the expiration time.

:MUST-REVALIDATE -- If the value is T, caches and proxies must not serve the resource
without revalidating it once it becomes stale, even if configured to operate with state
data. Servers should send this directive if and only if failure to revalidate will result in
incorrect operation. Recipients must not take any automated action that violates this
directive.

:PROXY-REVALIDATE -- If the value is T, this directive is the same as :MUST-REVALIDATE
except that it applies only to proxies and not non-shared user caches. It can be used on
response to an authenticated request to permit the user's cache to store and later return
the response without needing to revalidate it.

:NO-TRANSFORM -- If the value is T, caches and proxies must not change the body of the
message or change headers describing the content.

10. File Data Caching: When certain files are accessed with exceptionally high frequency, caching them in
memory improves performance and reduces disk contention. This assumes sufficient physical memory is
available or operating system paging algorithms perform better than simple disk access. Both of these
assumption are often true. This caching approach is relevant when your operating system does not already
provide in-memory caching for the file system (many do) or fine-grain control is desired.

When exporting any URL with static content on disk (any export type requiring a :PATHNAME argument), the
following arguments work for both objects and directories, wether export is immediate or deferred.

:CACHE-FILE-DATA -- Controls whether file data is cached in memory or not. The value is either a
DATA-UNIVERSE, a STRING naming a data universe, or T (which uses the default
*url-file-data-universe-name*). If the value is changed to NIL during a subsequent EXPORT-URL,
Dfile data caching will be turned off for the URL, or the URL structure below in the case of
directories.

:FILE-DATA-REVALIDATION -- The number of seconds between checks of the data on disk to see if the
original data has changed.

:FILE-DATA-WIRED -- Prevents file data from being dropping from the in-memory when its access
frequency falls and in-memory cache space is needed for other more frequently accessed data.

If file names are exported under other URLs, turning file data caching off for one URL will turn it off
for any other URLs servicing the same physical files.

See DEFINE-CACHED-DATA for an alternative interface for directly specifying files data to cache in
memory. *DATA-UNIVERSE-MAXIMUM-CACHE-SIZE* controls the maximum size of data caches.
*DATA-UNIVERSE-MAXIMUM-CACHE-ELEMENTS* controls the maximum number of items retained in the cache.
Facilities are also available for auditing the runtime behavior of data caches to support tuning the
performance of applications.

Advanced users can define their own data universes and develop cache management policies appropriate for
their applications, or develop specialized data universe classes for handling dynamic application data.

11. Property List: The PROPERTY-LIST keyword adds a property list of alternating keywords and value
to a URL. These properties can be read and set via GET-VALUE.

12. New static export types for data stored in files can be defined with DEFINE-URL-EXPORT-TYPE.

CL-HTTP/70.216 (LispWorks; 2.1.7)