MCL CL-HTTP Release Notes

MCL Versions: This release runs best in the latest version of Macintosh Common Lisp (4.2 PPC or greater). The server runs also on 68K machines under 3.3. It works with earlier versions of MCL from 3.0 onwards, but some functionality may be missing. If you have versions of MCL earlier than MCL 3.0, MCL CL-HTTP will not be suitable for production use but will work fine for development work with single threaded clients like MacWeb. If you encounter bugs, unclear documentation, or other shortcomings, please report them to: bug-mcl-cl-http@nospam.cl-http.org . Since development of this server is a community effort, please try to send fixes along with your bug reports.

Questions: If you have questions about the configuration, operation, or design of the server, please send mail to www-cl@csail.mit.edu WWW-CL@CSAIL.MIT.EDU, which you can join.

---

Release 70.190

Mobility support: Mobility support enables remapping running servers when IP addresses change. See: http:move-server-to-new-host

Mac OS X:CL-HTTP now runs over Mac OS X under MCL 5.0.

Release 70.156

Server Window:A new CLIM-base server window is available under MCL.

CL-HTTP Menu:This has been upgraded and featurized.

Proxy:The caching HTTP proxy now works robustly under MCL.

HTTP 1.1 Robustness: A series of bugs in HTTP 1.1 chunked transfer encoding were fixed and serveral efficiency improvements added. Proxy transfers were fixed. Open transport reliability was improved.

Efficiency Improvements: Numerous efficiency enhancements have been made, including specialization of stream method, low-level connection handling, time parsing, OpenTransport process waiting, error signalling when reading TCP buffers.

Specializations: The pathname data caching facility, server-side includes (SHTML), idle server scavenging, and more now work under MCL.

Release 67.99

CGI Facility: Terje Norderhaug (terje@nospam.in-progress.com) has contributed a CGI facility that allows MCL CL-HTTP to benefit from the hundreds of CGI applications written for Mac Web servers. Read Me

Multiple Reader Locks: Performance has been improved for loaded servers using shtml templates, passwords, or file data caching by introducing multiple simultaneous readers of lock protected data.

System Management Rationalized: The system loading facility has been upgraded. It knows how to find unloaded CL-HTTP systems and perform load or compile operations on them. System definitions and loading file now appear exclusively under http:mcl; All MCL-specific files now reside within this directory. All systems have a subdirectory there that contains their system definition.

Proxy Service: The MCL port has been upgraded to support HTTP 1.1 on the client side and allow chunked PUTs. Consequently, the new caching proxy implementation runs on the MAC.

Full Text Search: The Lambda-Vista system runs under MCL and has been integrated with the mail hyperarchive.

Release 63.58

ESMTP Supported:The SMTP package for sending email now talks to SMTP mailers. Useful for automatic bug reporting and the HyperArchive.

MenuBar Control: A control item on the menubar now makes operating the server and related tools much easier. (Courtesy Rainer Joswig).

Dramatic Stability Improvements: Concerted efforts between Digitool and MIT have resulted in the most stable and highest performance release for the Power Macintosh.

OpenTransport: Native OpenTransport networking is available for PowerPC MCL (4.1) using the Digitool OpenTransport interface.

TCP: A TCP level bug that could cause crashes was fixed. A timing bug on TCP-level process waits was fixed. The default TCP timeout was lengthened to eliminate premature connection closure with slow clients and fast servers.

Release 60.57

Bug Fixes: A number of bugs in the MCL port have been fixed. Daily log file rotation now works. TCP behaves more robustly under loading. 60.57 is MCL 4.0 & 3.1 ready.

TCP Stability: An extensive upgrade of of the mactcp.lisp interface has resulted in dramatic improvements in the stability of server running on both 68k and PowerPC machines.

HTTP 1.1 Compliance: The server complies with the HTTP 1.1 IETF draft standard dated July 4, 1996 for all platforms (except Lucid). Connections are persistent for static and dynamic resources for all content types. Static files with content type text are transferred using a CRLF (binary format on several architectures), which is cached automatically. Virtual hosts are available for HTTP 1.1 clients, as well as some late HTTP 1.0 clients (e.g., Netscape 2.0).

W3P Presentation System: A light-weight and fast presentation system has been integrated with the server for use in generating form queries and parsing form returns. W3p implements a subset of the Common Lisp Interface Manager, version 2.1. In a later release, a Dynamic Form Processing layer will be added above W3p. Chris Vincent Beta-Test

Web Server Configuration: The server can be configured over the Web using W3P and a preference facility. This paves the way for booting binary images containing only the server and configured them from a Web interface rather than from Lisp.

VRML Generation: A set of generation macros for the Virtual Reality Modeling Language support VRML 1.0. A brief tutorial introduces Lisp programmers to generating 3 dimensional graphics on the fly under program control. Rainer Joswig

W4 Constraint-Guided Web Walker: A web walker is included in the distribution, but must be loaded separately. It provides a constraint posting architecture control and action. The basic HTTP client has been ported to all platforms. Persistent connections, performance tuning, and general hardening have enhanced usability. W4 runs on all platforms except Lucid, most robustly on the Lisp Machine. Beta-Test. An HTML parser has been integrated as a url generator. Alpha-Test.

Distributed AI Applications: A MIME type, Application/Lisp-SExp, has been defined to allow Lisp programs to Post values from the basic client to the server. A response function on the server may execute and return Lisp values to the client. This clean facility for remote procedure call provides a foundation for building distributed AI applications. See the example in http:examples;client.lisp

Put Method: Netscape Gold 3.0 clients and the basic HTTP client can use the PUT method to publish Web documents to the server.

Client-Side Cookies: Facilities for setting and using cookies are now available. See computing response functions.

Netscape 3.0 HTML Generation: HTML generation has been extended to support Netscape 3.0 extensions, which include HTML 3.2 tables.

Contributions Directory: A directory for general-purpose contributions by users has be added at HTTP:CONTRIB; See also http;mcl;CONTRIB;

Fast Access Logging: A dedicated process now handles writing access logs for each port served. This allows threads to be reused faster because they do not wait for their log entry to be written. Disk contention is reduced because multiple logs are written in sequence while keeping the log file open.

Automatic GIF Image Sizing: Netscape 1.1 & 2.0 image macros automatically default the size of GIF images when http:*image-sizes-default-automatically* is non-null. This speeds the display of images on major clients. Shannon Spires

Mail Archive Facility: Archives of email discussions can be exported using the :MAIL-ARCHIVE export type, found in http:examples;mail-archive.lisp. This facility provides several views on the archive, is always current, and can be extended via subclassing. New mail formats beyond the current Lisp Machine, Eudora, UNIX Rmail formats are easily added.

Charsets: Character sets other than the HTTP default of :ISO-8859-1 may be specified for all text content types at export time, including directories. Similarly, dynamically computed pages may specify alternate character sets.

Languages: ISO language abbreviations can be associated with all resources, whether static or dynamic.

Computed Directory Listings: The standard url-oriented listing of directory paths may be over-ridden by a computed response. A built-in function is available to serve index.html files instead of standard listings.

Release 58.12

Inline Speech: Support is available for inline speech synthesis using a Netscape plug-in and the Macintalk. See discussion on the features page.

JavaScript Emission: HTML generation macros incorporate client-side events. Events are handled by client-side code. ns2.0:define-script provides a facility for encapsulating mobile code and emitting it from lisp during HTML generation with html:accept-input, html:with-event-handlers, and the simplified html:fast-format.

Persistent Connections: HTTP 1.1 persistent connections are available for exports where the content length is known in advance. This produces more efficient HTTP service by reducing the overhead of tearing down and reestablishing TCP connections for each request.

HTTP 1.1 Methods: The POST, PUT, and DELETE have been updated to the HTTP 1.1 specification.

Lower Server Latency: Server latency should be noticeably less due to the following improvements:

• More efficient header parsing.
• Significantly less consing parsing headers.
• Faster URL parsing.

Faster HTML Generation: Consing during HTML generation is virtually eliminated, and i stack consed where unavoidable.

Access Control Management: An example file implements an access control manger over the Web and prevents unauthorized access with digest authentication.

Netscape 2.0: HTML generation is now available for the range of 2.0 extensions, including frames and client-side image maps. Java and JavaScript can be exported and invoked from synthesized HTML. (see the CL-HTTP Features.

Digest Authentication: The MD5 Digest Authentication scheme is implemented. It provides substantially better passwords than the Basic Authentication scheme.

Modular Logging: Logging is now extensible. The variable http:*log-access-log-class* controls the class of log used. A variety of log formats are predefined.

ASCII Translation: The MCL HTTP stream now translates #\Return as #\Return #\Linefeed like other platforms. This corrects a bug with MIME multipart transmissions, and makes text files server from the MAC more manageable on other platforms.

Netscape Package: In preparation for the release of Netscape 2.0, the Netscape package has been renamed to Netscape1.1 and no longer has HTML3 as a nickname.

Server Push is now implemented for the Netscape 1.1N synthesis package by ns1.1:with-server-push-response. This complements client pull which is implemented by ns1.1:declare-refresh-rate. Choose Color Schemes illustrates server push, tables, and multipart MIME generation.

Access Control: In addition to secure subnets, the server now supports user authentication with passwords.

Email Interface: If you set smtp:*network-mail-host*, the server and response functions will be able to send mail using functions like www-utils:send-mail-from and www-utils:report-bug. Best results are obtained by providing alternate store-and-forward mailers on smtp:*store-and-forward-mail-hosts*. A simple SMTP package implements this mail sending facility in Lisp.

Server Interface: The server now has documentation on how to write response functions, and includes two interface specifications for accessing server state, CL-HTTP Server Interface 1.0 and Common Gateway Interface 1.1. Common Gateway Interface Standard 1.1.. Use the macro HTTP:WITH-CGI-ENVIRONMENT to make CGI variables available in response functions or form generation functions.

Image Maps: The server now exports image maps and supports both the NCSA and CERN file formats.

HTML 2.0 Conformance: The server's HTML generation facility fully conforms to HTML 2.0 with the upgrading of form processing in 0.5.0. Please report any divergence from the specification.

Netscape 1.1 HTML Extensions: The major Netscape 1.1 HTML extensions to HTML are implemented within the Netscape package. HTTP:USER-AGENT allows generation code to determine which client and version is being served.

Local Talk Operation: The server can run in a Local Talk Configuration for standalone o network service.

Clients: The server has been tested in MCL 3.0b1 with the following MAC clients: MacWeb 1.00A3.2, MacWeb.News 1.00A3d23d, Mac Mosaic 1.0.3, Mac Mosaic 2.0.0b7 without appreciable problems. UNIX Netscape 1.1N works better than MAC Netscape 1.1N with CL-HTTP. Netscape appears to get confused sometimes when it gets a 304 (conditional get code indicated data not modified).

Multithreaded Clients: MAC CL-HTTP is multithreaded in MCL 3.0, and so it can handle not just multiple simultaneous connections but also multiple connections from a single client like Netscape.

HTTP Thread Resource: Listening threads and HTTP server threads are resourced in order to reduce latency starting up threads and consing with heavy HTTP traffic. The number of threads defaults to HTTP:*NUMBER-OF-LISTENING-PROCESSES* . A keyword argument to HTTP:ENABLE-HTTP-SERVICE overrides this default.

Improved File Transfer Rates: The server now copies both ascii and binary files to HTTP transmission streams in blocks of buffer size., which defaults to HTTP:*HTTP-STREAM-BUFFER-SIZE* .

Arithmetic Operations Declared: Many arithmetic operations have been declared for faster server operation. Keep an eye out for any missing or erroneous declarations.

Lisp Machine Source Compatibility: The current MAC version is source compatible with the Lisp Machine version.

MAC Export Types: HTTP:EXPORT-URL now supports export types for a number of popular MAC file formats: :AIFF-AUDIO, :COMPRESSED-FILE, :EXCEL-FILE, :MAC-BINARY-FILE, :PDF-FILE, :PICT-IMAGE, :POWER-POINT-FILE, :QUICKTIME-VIDEO, :RTF-FILE, :STUFFIT-FILE, :TIFF-IMAGE, :WAV-AUDIO, :WORD-FILE. New export types can be defined for static, file-based data with HTTP:DEFINE-URL-EXPORT-TYPE.

Directory Export Types: HTTP:EXPORT-URL now supports an extensive variety of directory exports types, including single level and recursive exports. Additionally, :DIRECTORY will export all files for which pathname extension to export type mappings are defined, and recursively descend directories when :RECURSIVE-P is non-null. The variable HTTP:*AUTO-EXPORT* controls whether new files added to exported directories are exported on demand or not. Presently, URL-level security is not preserved for auto-exporting of new subdirectories, and so, new subdirectories should rely on manual export or global subnet security.

MCL 2.0.1 CLOS Method Combination Bug: A longstanding bug in CLOS method combination can cause trouble for CL-HTTP running in MCL 2.0.1. Digitool has provided a patch for MCL 2.0.1 that fixes the bug (not needed for MCL 3.0). MCL 2.0.1 users should load this patch prior to loading CL-HTTP. The patch can be found in in http;mcl;contributions;MCL-2-0-1-kernel-patches.hqx