Using ROFM CGI

Serving Your Own Database

The Basics

ROFM CGI allows you to find, get, add, modify and delete records in your databases. Most of these actions may be disabled; so it is easy to prevent users from (for instance) changing anything in a database. ROFM CGI will not even touch a database until you enter its name and permissions in ROFM's Security window. Forgetting to do this is a common mistake, but the resulting protection is well worth the hassle. For more information on security, see the Security section.

Before getting into details, let's follow a transaction from beginning to end. To start things off a user connects to your WWW server and receives a page containing a "form". The user fills out the form and pushes the Submit button. The user's browser packages up the data on the form and sends it to your server, which passes it along to ROFM CGI. The CGI parses the data, talks to your database (which must already be open in FileMaker Pro), formats a reply page and returns it to your server. Finally, the server sends the reply page to the user. Note: it is also possible to command ROFM CGI using links (URLs) instead of forms; that is a detail discussed elsewhere.

Forms may contain text boxes, menus, radio buttons and other such things for the user to fill out or select. These allow the user to specify what to search for, what field to sort by, and so on. Forms may also contain hidden information the user is not expected to change. This usually includes such things as the name of the database and what to do (e.g. find records or add a record). Be warned that hidden information is not truly secret; any user can view the a form as plain text ("source") and see and change anything they like. That is why it's important not to permit adding, modifying or deleting records in a database unless you really mean it!

Though forms may look complicated, at their heart they are very simple. Forms simply consist of a number of fields each of which has a name and a value. Text boxes, menus, radio buttons and hidden fields are all just different kinds of fields. The only information sent when a form is submitted are the field names and values. ROFM has no idea what the form looked like, nor what kind of fields were on it.

If a field's name begins with an underscore (_) then ROFM CGI treats it specially. Such fields are called parameters and are used to tell ROFM CGI what to do and how to do it. All other fields are database fields; each of these must have a name identical to a field in the database (but case doesn't matter). For more information about forms (and links, and alternate way to command ROFM CGI) see the section on forms.

Let's look at an example. Suppose a form contains the following fields:

<input type="hidden" name="_database" value="bb.db">
<input type="hidden" name="_action" value="Find">
<input type="hidden" name="_field" value="HTMLLink">
<input type="text" name="Author">

And the user has typed the value "Joe" into the "Author" field.

This tells ROFM CGI to talk to database "bb.db", finding all records for which the field "Author" contains a word beginning with "Joe". ROFM CGI then reads the field "HTMLLink" from each found record, formats it into a return page (as discussed next) and returns the result to the user.

Let's look at the return page more closely. The main body of the reply consists of data read from one field in the database, the one specified by the _field parameter. For searches this field is read from every found record (and is blank if no records are found) and the results are concatenated together, one after another. Typically this field is a calculation field that combines data from many other fields in the record to produce a nicely formatted result. For examples of this see the fields "HTMLSummary", "HTMLLink" and "HTMLRecord" in the sample database "bb.db".

The returned page may also include a header at the beginning and a footer at the end. Custom headers and footers may be created in ROFM CGI using the "Headers" and "Footers" window. These can then be specified by name using the _header and _footer parameters. If you do not specify a custom header or footer then a default (which you cannot change) is used. Headers and footers may contain more than just plain text; they may also include certain variables to display the current date, the number of records found, etc.

At this point I encourage you to skim the rest of this section of the manual and then try searching one of your own databases. Start by modifying a copy of "bbFind.html". Initially set it to return some existing field in your database. Once you have that working, try creating a simple calculation field in your database analogous to "HTMLSummary" or "HTMLLink" in "bb.db", and return that. Feel free to use whatever you can from the sample forms and database.

Finding and adding records are fairly straightforward to set up and fairly safe, since existing data cannot be corrupted or destroyed. Modification and deletion of records are significantly more difficult and dangerous. I strongly encourage you to postpone working with them until you have mastered Find and Add and carefully read all of this chapter of the manual.

Prepare Your Database

There are a few reserved field names you should know about. If any of the following fields exist in your database they will be set automatically on "Add". If you are already using these field names for another purpose then you must rename the existing fields!

Every field you want to access via the CGI must obey certain rules:

Create calculation fields to format the returned data. You will probably need several of the following types of fields:

These fields may include HTML tags, pointers to images, pointers to other documents, even URLs that access this (or any other) CGI.

Modify and Delete forms must refer to the record by record ID number, a number that is unique to that record for the life of the database. Unfortunately, FileMaker Pro does not allow one to obtain this useful number within a calculation field. To get around this problem, fields may include the variable "__RecID" (which starts with two underscores). This string will be replaced by the record ID number before it is returned to the user. Warning: due to speed considerations this substitution does not (yet) occur for data returned by "Find" and "FindLinks".

Specify Security Information

You must tell ROFM CGI about each database you wish to serve. Select "Security..." from the "Configure" menu, then create a new entry for each of your databases. Specify the name of the database file (without directory information), and whether to permit reading information and adding records. Note that you can change the information any time the CGI is running; it will take effect as soon as you click the "Save" button. For additional security considerations see the section on security.

Create The Forms

Set up at least one form for each database and action. Forms for finding and adding records are usually text files like the ones supplied, though some users serve them from a database. I suggest using the sample forms as templates, modifying them to suit yourself. Be sure to examine "bbFind.html" because it is liberally commented. Forms for modifying and deleting records usually must be created in your database using a calculation field. For example see the field "HTMLModifyRecord" in the sample database "bb.db". Once you set up your calculation fields in the database to create modify and/or delete forms, you may access them via a search form such as "bbFindToEdit.html". For more general information about forms I suggest that you browse the World Wide Web and/or buy a good book (please don't ask me for recommendations). Most HTML editors contain tools to help you create forms.

Your forms should contain the following elements:

If you misspell the name of a database field, or include one that does not exist in the database, you will get erratic behavior. Blank fields are ignored by the CGI (except for Modify), so you will only trigger the problem when the misnamed field is actually filled in. Such problems can be very frustrating and hard to track down, so please always test your forms by filling in every single field. If you can successfully submit the form with all field names filled you can be sure all database field names on the form are spelled correctly!

Data in double quotes must have certain characters encoded. Please see special characters for more details.

Constants (information the user cannot change) may be specified using a hidden input field or placed after the ? in the action, or any combination of the two. The form bbFind.html shows both. The format for specifying constant data after the ? is as follows:

<form action="ROFM.acgi?fieldName1=value&amp;fieldName2=value..." method="post">

Data after the ? needs special encoding. Please see special characters for more details. In particular, note that to send & or = as data to the CGI you must encode them as %2526 and %253D, respectively.

If all data is constant you may also send requests to ROFM CGI using URLs. This is how the links returned by bbFind.html work, and is also handy for "fixed" searches (such as a link that always returns the most recent entry). But be careful when using URLs; they can only pass about 2k characters of data, whereas forms can pass over 20k characters. URLs are formatted as follows:

<a href="ROFM.acgi?_param1=value&amp;_param2=value&amp;...fieldName1=value&amp;fieldName2=value...">text of link here</a>

For example (from Default.html):

<a href="ROFM.acgi?_action=Find&_database=bb.db&_field=HTMLLink">Get every record from the bulletin board</a>

The rules for encoding are similar to those for action data (as discussed above). In particular, note that the two characters "&" and "=" must be specially encoded as %2526 and %253D if they are used as data. Please see special characters for details.

Parameter Fields

Parameter fields are special fields in a form used to pass data to the CGI, rather than to the database. Every parameter field's name starts with underscore (_), and in fact any field whose name starts with underscore is assumed to be a parameter field. Many of these fields are optional, and some are relevant only for certain kinds of forms (and are ignored if present on other forms). The parameter fields are as follows:

Tips for Best Performance

Troubleshooting

If the "next page" link fails (especially for searches involving special characters), you are probably using the wrong version of ROFM CGI. ROFM CGI version 4.2 and later should be used with WebSTAR 2, QuidProQuo 2, and other browsers that do not hex-decode the HTTP search arguments. Version 4.1-B should be used with WebSTAR 1, QuidProQuo 1 and other browsers that hex-decode the HTTP search arguments.

The error: "ROFM CGI is not allowed to access database ..." indicates that the database has not been entered in ROFM CGI's Security window. Only databases listed in the security window may be accessed.

A complaint about needing "FaceSpan Extension" at startup probably indicates that QuickTime and/or (if you have a PowerMac) QuickTime PowerPlug are not installed. It can also occur if some component of AppleScript 1.1 is missing (if in doubt about this you can easily install AppleScript from your system installer disks).

If ROFM CGI pauses while starting up and then gives an error "Finder got an error: Apple Event timed out" and if you are using MacOS 7.5 or 7.6 then the file "Finder Scripting Extension" is probably missing from your Extensions folder (in your System Folder). Run the Extensions Manager control panel to make sure this file is present and enabled. This file is a standard component of MacOS 7.5 and 7.6, but is not part of MacOS 8.

The error: ''you must specify a database" may mean that you put data after a "$" instead of a "?" in your <form action...> field or URL. The character was a "$" for ROFM CGI 4.0, but is a "?" in 4.1 and beyond (as well as older versions of ROFM).

If your search form returns all records in the database, no matter what you enter into the various fields, you probably have two fields with the same name in your form. Thanks for Jon L. Gardner for reporting this.

The error: "database ... does not understand the 'exists' message" is caused by a bug in Apple's ObjectSupportLib. The solution is to obtain ObjectSupportLib 1.2 (or newer) from Apple, Version 1.2 was released after System 7.6, but presumably will be included with MacOS 7.6.1 and beyond.

If an operation return just a header and footer with no data from the record(s), make sure that you have specified _field. If you don't specify _field then no data from the database is returned (this can be a feature).

Email has very poor error handling. If the email address is bad or the server cannot be found, Eudora will display a dialog box and stop responding to apple events. The best you can do for now is to check "Say OK to alerts after 2 minutes" in the "Getting Attention" settings panel. I am considering alternatives to Eudora and have also reported the problem to Qualcomm as a bug in their apple event handling.

If a field is claimed to not exist on layout "0", this means that the field does not exist in the database. Perhaps you mis-spelled the field name, or perhaps the field is a related field instead of one in the database. Layout 0 is the default layout and contains every field present in the database, but no fields from other related databases. To access fields from related databases you must specify a layout containing the field.

If acesses are very slow, please see Tips for Best Performance.

If you are getting mysterious failures, please check the following:

Security

Serving a database puts it at great risk. A user can write and submit their own forms, so users can do anything the CGI allows regardless of whether you provide a form for it. For example, just because your forms don't access a particular field in the database doesn't mean that field cannot be read. Please read this section carefully.

ROFM CGI offers three types of security:

  1. The security window (required): the security window tells ROFM CGI which databases it can access, and which actions are allowed on that database. If a database is not listed in the security window then it cannot be accessed.
  2. Field protection (optional): fields whose names begin with underscore (for example: "_username") cannot be read; specifying such fields for parameters _field, _emailField, etc. will result in an error. Note that all other fields in the database can be read by anybody. If you have sensitive data in the database be sure to rename the relevant fields with a leading underscore, and be sure that this data isn't available in some other field (such as a calculation field).
  3. Password protection (optional): if your database includes the special field names "_username" and/or "_password" then the same field(s) must be present on any Modify or Delete form, and the value(s) must match. If these fields are not present in the database (and Modify or Delete privilege is permitted in the security window), than anyone can modify or delete any record! Be very careful to spell these field names correctly in the database, and test the results!

Your WWW server probably allows you to restrict access to various CGIs (restricting access to your forms is not good enough; users can generate their own forms.) Read your server's documentation for help with this. This feature (if present) can be used to offer different sets of users different levels of access to your databases, as follows:

Be warned that you should not allow heavy use of any but one copy of ROFM CGI; the other copies should only be for occasional high-security access. The problem is that FileMaker Pro can only handle one request at a time. If one copy of ROFM CGI is accessing a database and another copy attempts to do anything with FileMaker Pro (even in an entirely different database), the second access will fail with a "database busy" error message. You may be able to avoid this problem by running multiple copies of FileMaker Pro (each with its own name), but I am not sure FileMaker Pro supports this.

FileMaker Pro allows you to password-protect various elements of a database, including adding records and reading specific fields. But once a database is opened using a secure username and password it becomes vulnerable to access by the CGI, so this mechanism must be used with great care. Also note that a protected database will ask for a username and password when it is first opened; so automatic startup you will probably require a special script to open your databases. For help setting up usernames and passwords, please read the FileMaker Pro documentation or contact Claris technical support.

Please be sure to test all security features before relying on them, especially for sensitive data. Don't just assume things work! For instance if the fields _username or _password are spelled wrong in the database then that protection will be inactive. Also, ROFM CGI (like most software) is not guaranteed to work correctly. Your best defense is to try to break the system yourself. Feel free to read the source code and see what ROFM CGI is actually doing. If in doubt then don't serve your data!

Support

If you have questions or comments about this CGI, or wish to be notified of new versions, please subscribe to the FileMaker Pro CGI Mailing List. I do not usually have time to respond to personal email about this CGI. Please do not try to contact me by phone regarding this CGI; I have a day job and would like to keep it (though am open to alternate possibilities).

If you improve the script, please let me know, and if possible email me a copy of the improved script. I would be especially interested in hearing of a translation to a faster language.