Chapter 5 CL as an Internet Application Server
In this chapter
we will extend the personal-accountant Defpart we developed in
the previous chapter. As we left it, the only obvious way to interact with the
application was by evaluating Lisp expressions at a Command Prompt. Now we will
convert it into a Web application, accessible by a web browser (e.g. Netscape
or MS Internet Explorer ) on any machine on the network connected to the
machine running the CL process.
To accomplish
this, we will make use of some readily-available CL-based tools:
AllegroServe is a webserver developed for use with Common Lisp, under open-source licensing. It is guaranteed to work with Allegro Common Lisp. The main purpose of AllegroServe is to serve dynamic pages using an html1 generator. Franz Inc.'s stated goal in developing this module was to make it easy to publish complex web pages and add a web interface to a Common Lisp application.
GWL Implements an approach to building Web applications using a tree of dynamic Common Lisp Objects to represent the tree of pages in a Web application. It runs atop ICAD/IDL or GDL, and CL_HTTP or AllegroServe.
Traditional
webservers work by transmitting prepared static files from a Server computer
over a network to a web browser running on a Client computer. Beyond this,
scripts can be run, as separate processes from the webserver itself, through
the Common Gateway Interface, or CGI, to enable a web server to generate the
contents of a page based on computed information, or in response to a
"fillout-form" submitted by the user.
CL webservers
like AllegroServe and CL_HTTP take the concept of dynamically generated content
to a new level by embedding the actual webserver in a running CL process,
and each request is handled by running a Function or Method which responds by
sending output to a Stream connected to the requesting web browser. In this
scenario, each client request is usually handled by a separate thread of
execution in the CL process.
AllegroServe includes a special Macro, called simply html, which makes the generation of HTML output very convenient. Since HTML is already very similar to Lists in structure, the mapping is straightforward. Here are some examples:
NET.HTML.GENERATOR(26):
(html "hello")
hello
NET.HTML.GENERATOR(27):
(html (:html "hello"))
<html>hello</html>
NET.HTML.GENERATOR(28):
(html (:html (:body "hello")))
<html><body>hello</body></html>
NET.HTML.GENERATOR(29):
(html (:html (:body
(:table
:newline
(:tr (:td "hello"))))))
<html><body><table>
<tr><td>hello</td></tr></table></body></html>
NET.HTML.GENERATOR(30):
(html (:html (:body
((:table :width "100%")
:newline
(:tr (:td "hello"))))))
<html><body><table
WIDTH="100%">
<tr><td>hello</td></tr></table></body></html>
NET.HTML.GENERATOR(31):
(setq name "Joe")
"Joe"
NET.HTML.GENERATOR(32):
(html (:html (:body
((:table :width "100%")
:newline
(:tr (:td (:princ name)))))))
<html><body><table
WIDTH="100%">
<tr><td>Joe</td></tr></table></body></html>
As you can see,
the html macro allows you to write List expressions which contain keywords
corresponding to the usual HTML tags, and it follows a simple set of rules for
converting these expressions into actual HTML. Any kind of normal Lisp
expressions are also allowed within a call to the html Macro (e.g.
conditionals, Function calls, iteration, etc.), so you can compute and generate
any kind of required HTML.
Working with
the html Macro is much more convenient than working with actual HTML, not only
because you have the full programmatic capabilities of CL, but also because you
have the automatic nesting of the List expressions. There is no need to grapple
with named closing tags (e.g. </TABLE> or </HTML>).
For complete details
of working with htmlgen, see documentation in the file htmlgen.html which is
included in the AllegroServe distribution (http://opensource.franz.com).
GWL, or
Generative Web Language, designed to work with either ICAD/IDL or GDL (covered in
the previous chapter), consists mainly of a primitive Defpart called base-html-sheet. To use GWL,
you first identify those parts in your Object hierarchy which are to be
available as Web pages in your application. Those parts should then mix in
(i.e. inherit from) this base-html-sheet primitive part, and should
define a Method called :write-html-sheet which generates the actual
HTML corresponding to that "page," or Object.
GWL also
supports several built-in Messages to facilitate the creation of hyperlinks
from one "page" (Object) to another, as well as the ability for the user
to interact with the Object hierarchy by submitting values from "fillout-forms."
Form submittals have the effect of modifying ("bashing") the Object
hierarchy, so that the dependency tracking inherent in the language will
automatically update any dependent elements on other "pages" on-demand as
the user visits those pages.
For a more
detailed description of GWL, please see the documentation which accompanies its
distribution.
Figure 5.1 shows an example of a simple part which mixes in base-html-sheet. When this part is instantiated in a web browser (using a URL like "make?part=hello"), the following HTML would be returned:
<head><title>Greeting
Page</title></head>
<body>Hello,
Ben Franklin</body>
This simple example consists of a single root-level page. A typical application will consist of a tree of pages, which we can represent with :parts contained within the part corresponding to our root-level page. These :parts can have their own :parts, which have their own :parts, and so on, resulting in a "tree," or hierarchy, of arbitrary depth, breadth, and complexity. Figure 5.2 presents
an example of
our "hello" page with some child parts: Notice that we have added two :parts to this page,
each of which presumably are themselves some kind of specialization of base-html-sheet, as well as a
call to the Method :write-child-links. This is a built-in
Method of base-html-sheet which automatically creates hyperlinks pointing to
each of the child parts in the page.
When
instantiated in the browser, this part now produces the following HTML:
<head><title>Greeting
Page</title></head>
<body>Hello,
Ben Franklin
<p><ul>
<li><a
HREF="answer?respondant=diary+root(569)">Diary</a></li>
<li><a
HREF="answer?respondant=finances+root(569)">Finances</a></li>
</ul></p></body>
The hyperlink ("HREF") Strings in the above HTML will be recognized by the response Method in the webserver (generically defined by GWL), and will allow the server to respond by selecting the correct Object hierarchy and individual Object within that hierarchy. The :write-html-sheet Method of the appropriate Object Instance will then be used to produce the actual response page.
Sometimes it is
helpful to think of a "core" Object, which computes results, accesses
databases, etc., separate from various "views" on that Object, such as an
HTML view, a plain text view, or perhaps an XML view. One way to achieve this
would be to add a new "writer-method" for each desired view to the main
Object. But this would clutter up the main Object with Methods which may never be
needed.
A better
approach, supported by ICAD/IDL and GDL, is to associate "writer-methods"
with the core Object through the use of output companions. The details
of working with Companions are beyond the scope of this chapter, but you should
be aware that the technique exists.
Companions
allow you to attach Methods to a particular combination of a core Defpart
and a desired output format, for example, the Defpart hello and the output
format HTML.
This represents
a step beyond the basic Message-passing paradigm, because we are now
associating Methods with two specialized arguments (i.e. the core Defpart and the output
format), rather than just the one specialized argument as in basic
Message-passing. Pure CLOS, of course, allows Methods to be specialized on any number of
arguments - so the use of Companions still only scratches the surface of the
full power of CLOS.
Our web-enabled Personal Accounting Application will consist of a single "root-level" web page which will contain the following elements:
As our first step, let us prepare the root-level web page containing the two hyperlinks for our two "fillout-forms." For this, we will make use of a pre-defined GWL Defpart called html-sql-table. This Defpart takes a :table-name as its required Input, and will produce a generic data maintenance form for this table, providing basic Insert, Update, and Delete capabilities. When instantiated, the html-sql-table Defpart will contain an instance of a Table Object corresponding to the specified :table-name, accessible through the Message :table. Therefore, we can produce our initial rootlevel page using just a slight variation on the personal-accountant Defpart we prepared in Figure 4.7, as shown in Figure 5.3. Observe the following about this Defpart, as compared with the one in Figure 4.7:
Figure 5.4: Home Page of Personal Accounting Application
Figure 5.4 shows the top level page, which at this point only contains two hyperlinks. Clicking the "Accounts" hyperlink will display the table of Accounts, as seen in Figure 5.5.
Figure 5.5: Table of Accounts
This table of Accounts is generated automatically by the generic html-sql-table Defpart we are using. This page supports Inserting, Updating, and Deleting the individual rows in the table. Updating and Deleting is accomplished by clicking form elements or hyperlinks associated with each row, and Inserting is accomplished by clicking an insert-form hyperlink at the bottom of the page. Figure 5.6 shows the "fillout-form" which results from clicking this link.
Figure 5.6: Form for Inserting a New Account
Once a new Account has been added, the table of Accounts looks like Figure 5.7. Figures 5.8, 5.9, and 5.10 show the parallel set of pages for the Transaction table.
Figure 5.7: Table of Accounts after Inserting a New One
As our application currently stands, inserting and updating Accounts and especially Transactions is not very easy. The user must enter all the data into the forms exactly as it will be stored in the database table.
Figure 5.8: Table of Transactions
Figure 5.9: Form for Inserting a New Transaction
Figure 5.10: Table of Transactions after Inserting a New One
For example,
Transactions are defined as coming "from" one Account "to" another
account, where the "from" and "to" must be entered in the form of the
:gen_id index Numbers. The :gen_id is actually intended as a hidden
"internal" column of the table, not really for human consumption.
A human using
this program really wants to select from a List of existing Accounts by name. For this
purpose, the html-sql-table Defpart supports the :columns-and-names :optional-input, as seen in
the snippet of the personal-accountant Defpart in Figure 5.11.
The :columns-and-names is a List of Lists, one for each column in the
table. The first of each sublist is the Column name, and the second is the desired
Prompt name. These are followed optionally by the :choices Keyword, which
can specify that the choices for the field come from a column in a related table.
Note the use of
the backquote (`) in this snippet. This, together with the comma, is a convenient mechanism
for building a literal List, while selectively evaluating certain elements of
it (the elements following the commas are evaluated).
Figure 5.12
shows the updated Form, with selection Lists in place of the old type-in
fields.
Similarly, the
Account table contains the field :acct_type which is supposed to be a
String with a value of either "income/expense" or "asset/liability."
The user will want to pick one of these two choices from a List, not have to
type the String exactly. This is accomplished as illustrated in Figure 5.13,
which shows just the :accounts part specification from personal-accountant. This gives a similar
result, as seen in Figure 5.14.
Many other
customizations are of course possible. But already, with 70 lines of
application code, we have prepared a functioning web application which handles
the fundamental database table maintenance that we require.
Looking back at our initial requirements at the beginning of this section, we have yet to place the Current Balances, Net Worth, and Cash Flow on the root-level page. We can accomplish this by extending the :write-html-sheet Method of our personal-accountant Defpart. First we add an HTML table to display the individual Account Balances (one row per Account). In Figure 5.15 is the relevant portion from the :write-html-sheet Method after this is done. Note that the value is negated for accounts of type "Income/Expense," in order to make Income appear positive and
Figure 5.11: Code to Produce Transaction Entry Form with Selection Lists
Figure 5.12: Transaction Entry Form with Selection Lists for "From"
and "To" Accounts
Figure 5.13: Code to Produce Account Entry Form with Selection List
Figure 5.14: Account Entry Form with Selection List for Account Type
Figure 5.15: Displaying Account Balances
Expenses appear
negative.
Finally, we add
a two-row HTML table to display the current values for Net Worth and Cash Flow.
Figure 5.16 shows the relevant section of the :write-html-sheet: Method. Notice
the use
Figure 5.16: Displaying Net Worth and Cash Flow
of the format directive $, which formats
Numbers in a normal currency format (e.g. with two places after the decimal
point by default).
The root-level
screen now appears as in Figure 5.17
We now have a
usable, web-enabled, SQL-database-driven, reasonably maintainable, highly
extensible and customizable Personal Accounting Application, written with
approximately 95 lines of application code.
This guide has
given you a rapid tour of the world of Common Lisp, from the basics of starting
and working with CL, through the fundamental parts of the CL language itself,
and finally building a working
application on top of CL. As you begin your journey of creating your own CL
applications, you will experience constant "surprise and delight" as you
discover new ways to use its power to accomplish your problem-solving goals.
Just as CL is
an extensible, scalable language, so is the learning process involved. We hope
you now have the confidence and motivation to begin immediately putting CL to
work for you, solving practical, "every day" programming challenges - and
the foundation to move your applications to ever-higher levels.
Figure 5.17: Home Page with Basic Reports Added
Appendix A
This Bibliography should also be considered as a Recommended Reading list.
Appendix B
B.1 Lisp Mode
When you edit a
Lisp file in Emacs, Emacs should automatically put you into Lisp Mode or Common
Lisp Mode. You will see this in the status bar near the bottom of the screen.
Lisp Mode gives you many convenient commands for working with List expressions,
otherwise known as Symbolic Expressions, or s-expressions, or sexp's for short.
For a complete
overview of Lisp Mode, issue the command M-x describe-mode in Emacs. Table
B.1 describes a sampling of the commands available in Lisp Mode.
B.2 Making Your Own Keychords
You may wish to automate some commands which do not already have keychords associated with them. The usual way to do this is to invoke some global-set-key commands in the .emacs file (Emacs' initialization file) in your home directory. Here are some example entries you could put in this file, along with comments describing what they do:
;; Make C-x
& switch to the *common-lisp* buffer
(global-set-key
"¡¬C-x&" '(lambda() (interactive)
(switch-to-buffer
"*common-lisp*")))
Table B.1: Common Commands of the Emacs-Lisp Interface
;; Make
function key 5 (F5) start or visit an OS shell.
(global-set-key
[f5] '(lambda() (interactive) (shell)))
;; Make sure
M-p functions to scroll through previous commands.
(global-set-key
"¡¬M-p" 'fi:pop-input)
;; Make sure
M-n functions to scroll through next commands.
(global-set-key
"¡¬M-n" 'fi:push-input)
;; Enable the
hilighting of selected text to show up.
(transient-mark-mode
1)
B.3 Keyboard Mapping
If you plan to spend significant time working with your keyboard, consider investing some time in setting it up so you can use it effectively. As a minimum, you should make sure your Control and Meta keys are set up so that you can get at them without moving your hands away from the "home keys" on the keyboard. This means that:
If you are working on an XWindow terminal (e.g. a Unix or Linux machine, or a PC running an X Server such as Hummingbird eXceed) you can customize your keyboard with the xmodmap Unix command. To use this command, prepare a file called .Xmodmap with your desired customizations, then invoke this file with the command
xmodmap .Xmodmap
Here is an example .Xmodmap file for a PC keyboard, which makes the Caps Lock key function as Control, and ensures that both Alt keys will function as Meta:
remove Lock =
Caps_Lock
keysym
Caps_Lock = Control_L
add Control =
Control_L
keysym Alt_L = Meta_L Alt_L
For PCs running Microsoft Windows, the Windows Control Panel should contain options for customizing the keyboard in a similar manner.
Appendix C
C.1 About This Book
This book was typeset using an IBM Thinkpad 1451i running Red Hat Linux and Allegro Common Lisp Professional Edition, using a Lisp-based markup language. For the printed output, the Lispbased markup language generated LATEX code and was typeset using LATEX by Leslie Lamport, which is built atop TEX by Donald Knuth.
C.2 Acknowledgements
I have received inspiration and instruction from many enabling me to write this book. I would like to thank, in no particular order, John McCarthy, Erik Naggum of Naggum Software and comp.lang.lisp, John Foderaro of Franz Inc., Greg Burek, Jeff Moffa, Steve Seippel, and Mark Geoffrey of Ford Motor Company; Stanley Knutson, Garreth Evans, David Howe, Brian Grogan, and the rest of the staff at Knowledge Technologies International; Professor John Laird who taught me CL in college; John Mallery and Paul Graham, who taught me that CL is the platform to be using for server applications, James Russell, AndrewWolven, Dan Fagan,Wuinnie Jimenez, Vassilios Theodoracatos, Craig Lienard, Tom Johnson, and many others whom I am sure I am overlooking here. I would especially like to thank my father, David J. Cooper Sr., Lisa Fettner, Dr. Sheng-Chuan Wu, and Scott Smith for reviewing and commenting on the manuscripts, Peter Karp from Stanford Research Institute for creating the original outline, and Hannu Koivisto for helping to improve the PDF output. "Thank you" to my fiancée Kai Yan, for putting up with my antics while I was writing this (and in general). I would also like to thank Fritz Kunze of Franz Inc. for telling me to "go write a book."
C.3 About the Author
David J. Cooper has been Chief Engineer with Genworks International since November 1997. His principal activity is working with large manufacturing companies, helping them with their transformation into Knowledge-based Organizations (KBO's). He has led instruction courses in all aspects of KBE, including of course CL, at General Motors, Visteon Automotive, Raytheon Aircraft, Ford Motor Company, Knowledge Technologies International, and others. Prior to his career with Genworks, Mr. Cooper worked with Ford Motor Company, mostly on deployment of CL-based systems in the field of mechanical engineering and manufacturing. He also has software development experience in the CAD and database industries. He received his M.S. degree in Computer Science from the University of Michigan in 1993, and holds a B.S. in Computer Science and a B.A. in German from Michigan as well. His current interests include promoting the use of CL for all kinds of server applications.