CL as a CASE tool

Chapter 4 CL as a CASE tool

In the following two chapters, we will illustrate a rudimentary web-based Personal Accounting Application, complete with SQL database table definitions and User Interface. We will implement this application in less than 100 lines of code, using CL and a simple CASE-like Macro extension language embedded in CL.

4.1 Executable Object Model

First, we will describe the Macro language, which allows us to define Object Classes, their Methods, and their relationships in a simplified, concise manner. This language captures many of the central concepts of a general-purpose "Object modeling" language such as the Unified Modeling Language (UML), with the crucial difference being that the Object model is directly executable as an application.
   In the next chapter, we will layer an HTML-based web user interface on top of the Object model, making use of Franz Inc.'s "AllegroServe" (CL-based web server) together with their "HTMLgen" (HTML generation system).
   The language described in this chapter allows the developer to work using an Object-oriented approach called the message-passing paradigm, which is somewhat simplified (and limited) compared to what pure CLOS can do. However, for some types of problems, working in the Message-passing style is sufficient and appropriate, and can simplify the programming process. Remember, when we use Macro extensions on top of CL, we are working in a superset of CL, so we do not give up any functionality - we always have the option of dropping back into pure CLOS or CL.
   In the Message-passing paradigm of Object-oriented programming, Methods "belong" to Classes - that is, the Methods of a Class can be defined right along with the Class definition, as in Java or C++. This is not possible in pure CLOS, since a CLOS Method may be associated with any arbitrary number of Classes.
   Methods which "belong" to particular Classes are also known as messages, thus the term "Messagepassing."

4.2 Knowledge Base

In addition to implementing a Message-passing style, the Macro extension we illustrate in this chapter also implements some knowledge base features, which mainly consist of caching, dependency tracking, and a non-procedural style of programming.

Caching means that the return-value of a call to a Method (Message) is automatically "memoized" by the system, so that if the Method is called again during the same running session, the system can directly return the value, and does not have to do the actual computation again.

Dependency Tracking is a necessary complement to Caching - this feature allows the system to detect if a cached value depends on another value somewhere else in the system. This way, if the "depended-upon" value is modified ("bashed," in KB parlance), then the cached value is known to be stale and must be recomputed (and re-cached) the next time it is demanded.

Non-procedural refers to a style of programming, also known as declarative programming, in which the programmer need not be concerned with the order of evaluation, i.e. there is no explicit "begin" and "end" to a program. There is just a high-level Object model which specifies what the solution to a problem is, not procedurally how the problem is to be solved. The language handles the details of exactly which Methods to call, and when.

This general concept of a Knowledge Base has seen a multitude of implementations, mainly in CL. An example of a successful commercial implementation is The ICAD System^® from Knowledge Technologies International, whose Allegro CL-based language is known as the ICAD Design Language, or ICAD/IDL. ICAD/IDL is specifically geared towards mechanical engineering and geometric work, and so it has very deep and broad support for geometric Objects (wireframe, surfaces, and solids), a graphical user interface with geometric display capability, as well as integrations with various Computer-Aided Design (CAD) systems, built on top of the basic KB Object description language.
An example of an open-source KB language implementation is the General-purpose Declarative Language, or GDL, which also implements a cached, dependency-tracked, message-passing programming style. GDL is intended as a teaching tool and as a substrate for implementing certain kinds of Object-oriented applications in Common Lisp.
ICAD/IDL is available from Knowledge Technologies International at
http://www.ktiworld.com

as part of their Knowledge Based Organization (KBO) suite. GDL is available in unsupported form through Sourceforge at
http://www.sourceforge.net/projects/gdl/

and in a supported package form from Genworks International^® at
http://www.genworks.com

The Personal Accounting example presented in this and the next chapter should function similarly with either ICAD/IDL or GDL.

4.3 IDL and GDL Syntax

4.3.1 Defpart

Defpart is the basic Macro for defining Classes in IDL and GDL. The original meaning of "part" refers to physical geometric parts, but here the meaning is extended to include any kind of Functional Object. A part definition (Defpart) maps directly into a Lisp Class definition.
The defpart Macro takes three basic arguments:

a name, which is a Symbol.
a mixin-list, which is a List of Symbols naming other Defparts from which the current part will inherit characteristics.
a specification-plist, which is spliced in (i.e. doesn't have its own surrounding parentheses) after the Mixin-list, and describes the Object model by specifying Methods, contained Objects (aggregates), etc. The Specification-plist typically makes up the bulk of the Defpart.

Here are descriptions of the most common keywords making up the Specification-plist:

Inputs specify information to be passed into the Object instance when it is created.

Attributes are really cached Methods, with expressions to compute and return a value.

Parts specify other Instances to be "contained" within this instance.

Methods are (uncached) Methods "of" the Defpart, and can also take other non-specialized arguments, just like a normal Function.

Figure 4.1 shows a simple example, which contains two Inputs, :first-name and :last-name, and a single Attribute, :greeting. As you can see, a defpart is analogous in some ways to a defun,

where the Inputs are like arguments to the Function, and the Attributes are like return-values. But seen another way, each Attribute in a defpart is like a Function in its own right.
The referencing Macro "the" shadows CL's "the" (which is a seldom-used type declaration operator). "The" in IDL and GDL is a Macro which is used to reference the value of other Messages within the same Defpart or within contained Defparts. In the above example, we are using "the" to refer to the values of the Messages (Inputs) named :first-name and :last-name.

4.3.2 Making Instances and Sending Messages

Once we have defined a Defpart such as the example above, we can use the constructor Function make-part in order to create an instance of it. This Function is very similar to the CLOS make-instance Function. Here we create an instance of hello with specified values for :first-name and :last-name (the required Inputs), and assign this instance as the value of the Symbol mypart:

GDL-USER(16): (setq mypart
(make-part 'hello :first-name "John"
:last-name "Doe"))
#<HELLO @ #x218f39c2>

As you can see, the return value is an Instance of Class hello. Now that we have an Instance, we can use the Macro the-object to send Messages to this Instance:

GDL-USER(17): (the-object mypart :greeting)
"Hello, John Doe!!"

The-object is similar to the, but as its first argument you must give an expression which evaluates to a part Instance. The, by contrast, assumes that the part instance is the lexical variable self, which is automatically set within the lexical context of a Defpart.
For convenience, you can also set self manually at the CL Command Prompt, and use the instead of the-object for referencing:

GDL-USER(18): (setq self
(make-part 'hello :first-name "John"
:last-name "Doe"))
#<HELLO @ #x218f406a>

GDL-USER(19): (the :greeting)
"Hello, John Doe!!"

In actual fact, (the ...) simply expands into (the-object self ...).

4.3.3 Parts

The "Parts" keyword specifies a List of "contained" Instances, where each instance is considered to be a "child" part of the current part. Each child Part is of a specified Type, which itself must be defined as a Defpart before the child part can be instantiated.
Inputs to each instance are specified as a Plist of Keywords and Value expressions, spliced in after the part's name and type specification, which must match the Inputs protocol of the Defpart being instantiated. Figure 4.2 shows an example of a Defpart which contains some Parts. In this

example, hotel and bank are presumed to be already (or soon) defined as Defparts themselves, which each answer the :water-usage Message. The reference chains:

(the :hotel :water-usage)

and

(the :bank :water-usage)

provide the mechanism to access Messages within the child part Instances.
These child parts become instantiated on demand, meaning that the first time these instances or any of their Messages are referenced, the actual instance will be created and cached for future reference.

4.3.4 Quantified Parts

Parts may be quantified, to specify, in effect, an Array or List of part Instances. The most common type of Quantification is called :series quantification. See Figure 4.3 for an example of a Defpart

which contains a quantified set of Instances representing U.S. presidents. Each member of the quantified set is fed inputs from a List of Plists, which simulates a relational database table (essentially a "List of Rows").
Note the following from this example:

in order to quantify a Part, the additional Input keyword :quantify is added, with a List consisting of the keyword :series followed by an expression which must evaluate to a Number.
The special Parts keyword :parameters allows you to pass a List of Plists directly as the Inputs to a quantified set of child parts. This will also work for passing a single Plist into a non-quantified Part.
The Keyword :optional-inputs is used, which is a hybrid of :attributes and :inputs, allowing a default expression as with :attributes, but allowing a value to be passed in on instantiation or from the parent, as with :inputs. A passed-in value will override the Default Expression.

4.4 Personal Accounting Application

Now that we have a basic overview of the language, let's examine some of the Objects required to implement a rudimentary Personal Accounting Application. The Application is expected to keep track of Accounts and Transactions, and to compute Balances, Cash Flows, etc. The Accounts will be Objects which answer the following Messages:

Name
Description
Account Number
Account Type (income/expense or asset/liability)
Beginning Balance
Current Balance

The Transactions represent movements of money from one account to another. Therefore they are Objects which must answer the following Messages:

Date
Source Account
Target Account
Payee
Amount

Now let's see how these Objects come together in order to compute the desired results.

4.4.1 Persistent Objects

Since this is largely a database application, we can start by looking at which Messages in our two Defparts might represent columns in a database table:
A basic financial Account should store its Name, Description, Account Number, Account Type, and Beginning Balance. Its Current Balance is computed (derived) information and is not required to be stored. A Transaction does not contain derived information for our purposes. All its Messages are basically columns in a database table.
So let us start by defining some database Objects. Because database table definition requires some datatype information to be specified in advance, and because we would like to produce more than one Defpart to represent each table, we will use a special definition syntax for database tables.
We have prepared the simple Macro deftable for this purpose, as seen in Figure 4.4. As you can see, deftable accepts a description of an SQL-style database table. If you are familiar with Standard Query Language (SQL), you will recognize the data types in the above definitions, e.g. varchar2 for character strings.

Deftable then processes the description into two Defparts: one representing an instance of the table itself, and another representing the Rows of the table instance. The "table" Defpart will contain a quantified set of "Row" Instances. The Values in the individual Rows are represented by ordinary Messages supported by the "Row" Instances. The Table Object supports Methods for Inserting and Deleting Rows from the table, and the Row Object supports Methods for Updating the values in each Row. (Methods, in this context, are Messages which take arguments in addition to the implicit self argument present in all Messages).
As a side-effect, the Deftable Macro also communicates through a database connection Object to a live SQL database, and either

Creates the table in SQL if it does not exist
Alters the table (at least attempts to alter the table) if it already exists

This second of the possible side-effects represents a convenient way to do schema evolution on the live tables in the database.

4.4.2 Adding Computed Messages

Now we can extend the Account definition to compute the current balance. For this purpose, we can take advantage of the fact that the "Row" Object generated by deftable will have been defined to inherit from a (initially empty) custom Class, or "mixin." If the SQL table is named ACCOUNT, the Defpart representing Rows in this table will be named account-sql-row, and the Custom Mixin will be named custom-account-sql-row-mixin. Therefore, we can redefine the custom-account-sql-row-mixin Class as shown in Figure 4.5 . Instances of the account-sql-row will now answer the :current-balance Message. The Message is being computed as follows:

First notice that :transaction-list is a required Input. This represents a List of Objects representing all Transactions currently in the database.
The :current-balance Message starts by initializing a local variable, result, to the beginning balance of the Account (remember that the Row Object represents one particular Account in the Database, i.e. Row in the Account table).

Next, an iteration is established which executes once for each transaction in the :transaction-list, setting the local variable transaction to each Object in this List.
Each time through the body of the dolist, a conditional is evaluated:

If the current Account Object is the same as the "source" account of the Transaction (i.e. the :from_acct), the amount is subtracted from the result.
If the current Account Object is the same as the "target" account of the Transaction (i.e. the :to_acct), the amount is added to the result.
If the current Account Object matches neither the "source" nor the "target" of the Transaction, none of the conditional tests will match, and nothing will happen for that particular Transaction.

Finally, after all transactions have been processed, the dolist will return result, which will have the correct value.

Note also the following:

We use the :gen_id Message for matching Accounts and Transactions. This is the default Unique Key for the database table, a column automatically generated by the deftable Macro.
We are using the SQL Database just as "dumb" data storage, not relying on it at all for doing
ltering, joining, etc. For very large datasets we will start to use the database's indexing and joining capabilities - but for now we can conveniently do much of our work right here in CL.

4.4.3 Building the Object Hierarchy

Now that we have some basic persistent building blocks, let us put together a simple application. At this point the only user interface to the application will be the CL Command Prompt. In the next chapter, we will attach a simple web-based interface.
At this point we need a "container" Object to hold our collections of Accounts and Transactions. Figure 4.6 shows a first version of such an Object definition. This part has one Object for Accounts

and one Object for Transactions. It also computes :transaction-list as the List of all rows from the Transactions table, and specifies this Message as descendant. Descendant-attributes provides a mechanism for ensuring that this Message will be available in all descendant parts from the part in which it is defined, as if it had been explicitly passed as an Input into each of these parts. This way, we are ensuring that :transaction-list will be available in our Account Row Objects (children of the :accounts), since our custom-mixin requires : transaction-list as an Input.
   Descendant-attributes should be used sparingly, as their overuse can lead to confusing situations (i.e. Messages coming from unexpected places in the Object hierarchy).
   Now, we will add two Messages to our container, one to compute Net Worth and one to compute Cash Flow. Net Worth is the sum of Balances of all accounts of type "Asset/Liability," and Cash Flow is the negation of the sum of Balances of all accounts of type "Income/Expense" (the sign is reversed so that Income appears positive and Expenses appear negative).
   Figure 4.7 shows our personal-accountant Defpart with these two Messages added. Each Message is computed in basically the same manner: a dolist is used to iterate through all accounts, accumulating the balance of each account of the applicable type into the result.
   Now our application consists of one root Object, personal-accountant, with two child Objects, one for the Accounts and one for the Transactions. These three Objects answer Messages as follows:

personal-accountant will answer :net-worth and :cash-flow Messages.
Each Row Object in account-sql-table will answer a :current-balance Message.
The Row Objects in each of account-sql-table and transaction-sql-table will answer an :update! Message (with arguments).
account-sql-table and transaction-sql-table will each answer :insert! and :delete! Messages (with arguments).

4.4.4 Interacting with the Model

At this point we have just a CL Command Prompt interface to the system. For purposes of illustration, we will enter two Rows into the Accounts table and a transaction into the Transactions table, then query the application for some of our supported Messages:

GENACC(20): (setq self (make-part 'personal-accountant))
#<PERSONAL-ACCOUNTANT @ #x20f678e2>

GENACC(21): (the :accounts :row-list)
NIL

GENACC(22): (the :accounts (:insert!
                                               (list :name "Checking"
                                             :description "ACME National Back Checking"
                                             :acct_number 01
                                             :acct_type "asset/liability"
                                             :begin_balance 500)))

GENACC(23): (the :accounts (:insert!
                                               (list :name "Utilities"
                                                     :description "Home Utilities -- electric, etc."
                                                     :acct_number 55
                                                     :acct_type "income/expense"
                                                     :begin_balance 0)))

The two calls to the :insert! Method each added a Row to the database, as well as updating the current Objects so that any dependent Objects or Messages will see the change immediately:

GENACC(24): (the :accounts :row-list)
(#<ACCOUNT-SQL-ROW @ #x20f67954> #<ACCOUNT-SQL-ROW @ #x20f67982>)

GENACC(25): (mapsend (the :accounts :row-list) :index)
(1 2)

GENACC(26): (the :accounts (:rows 1) :current-balance)
500

GENACC(27): (the :accounts (:rows 2) :current-balance)
0

The mapsend Function above takes a List of Objects and sends a given Message to each of them, returning a List of the results. The (:rows 1) and (:rows 2) syntax is a way of referencing particular members of a quantified set.
Now let us add a transaction, and see that it updates the relevant Messages:

GENACC(28): (the :transactions (:insert!
                                                   (list :trans_date "25-aug-00"
                                                         :from_acct 1
                                                         :to_acct 2
                                                         :payee "Detroit Edison"
                                                         :amount 75)))

GENACC(29): (the :accounts (:rows 1) :current-balance)
425

GENACC(30): (the :accounts (:rows 2) :current-balance)
75

GENACC(31): (the :net-worth)
425

GENACC(32): (the :cash-flow)
-75

GENACC(33): (the :transactions (:insert!
                                                   (list :trans_ date "25-aug-00"
                                                         :from_acct 1
                                                         :to_acct 2
                                                         :payee "Michcon Gas"
                                                         :amount 125)))

GENACC(34): (the :accounts (:rows 1) :current-balance)
300

GENACC(35): (the :accounts (:rows 2) :current-balance)
200

GENACC(36): (the :cash-flow)
-200

GENACC(37): (the :net-worth)
300

So our main Messages are working, through a simple command-line interface for now. A well-rounded personal accounting program would, of course, require certain extensions. Here are a few examples:

computing cash ow for a specified time period
allowing expense accounts to have "sub-accounts"
allowing the creation of budget targets for expense accounts

With a robust Object Model, features such as these can be added and tested in an incremental manner. We would simply add new Messages and possibly additional Objects to our Object Hierarchy, building steadily toward the desired functionality.
As new "required functionality" becomes apparent (as it invariably does), our exible and extensible Object Model stands ready to comply.

4.5 Conclusion

In about 45 lines of code, we have written a basic runnable account-balancing program, including the creation of the required SQL database tables and associated Methods for transacting with the database. Done in pure Common Lisp (i.e. without using any extension Macros), this application would have taken slightly more code, but still not an unreasonable amount.
The main objective of the example is to show what really is possible with Common Lisp by judiciously selecting and/or developing some strategic extension Macros. The Defpart Macro is but one example of how CL can automate the programming process; many other approaches certainly exist, and others have yet to be discovered.