Introduction

 Holon
 Forth
 Papers

The Beauty of Separate Systems

Wolf Wejgaard

Presented at the EuroFORML conference 1990

Abstract

A concept of separate systems is proposed for programming in Forth. The application is built in its own code space independent of the development system. This approach relieves space limits and delivers compact applications. The full interactivity of Forth can be retained.
  

1. Introduction

The conventional way to build an application in Forth is to extend a given system with the functionality of the application. This is the simplest way and it gives good results for small applications.

However, for larger applications there soon is a space conflict between development system and application. This conflict is solved by all sorts of clever means: using a lean development system, loading editor and assembler in high memory locations with some re-linking scheme (as long as space is still available), coding code words by hand in order to dismiss the need for the assembler, and so on.

Or solving the conflict by brute force: converting to a 32-bit system or using an extended addressing scheme as in F-PC.

The conflict could easily be solved if the two systems were separated. Then each system could be optimized for its purpose. On the one side we have the development system with all the necessary tools (editor, assembler, compiler, dictionary, debugger, ...; no need to economize on space) and on the other side we have the application program consisting only of the necessary code to fulfill its function.

This approach is already in use in the form of meta- or cross-compilers. Here we have a Forth system building an application in a separate code space. The application is separated by necessity since its code is built for a different processor.

A cross-compiler can be just as interactive as a conventional Forth system. This has been shown very nicely by the "umbilical-link approach" [ROB89].

From this it is only a small step to propose an interactive cross-compiler also for the development of applications for the same processor (host system).

There ought to be a separation since both systems serve different purposes. The host serves to build a target system using the best available tools and software engineering knowledge. The target serves to fulfill the clients needs; be robust, serviceable, trustworthy, and easily enhanced.
  

2. Separate systems

Let us take a look at the memory space of the separated systems.

During development the target is contained as an independent program in the code space of the development system. The target is separated through a simple turnkey process.
  

Development system (Host)

The development system is a conventional Forth system extended for cross-compilation. It consists of:

• Start-code to crank the engine.
• An enhanced Forth system.
• An elegant editor, possibly one of the new browser implementations, which are coupled with the dictionary [RAS89, WEJ89].
• Room for the applications dictionary. The host dictionary is not needed (with the exception of the words used for cross-compilation) and may be absent.
• A cross-assembler and a cross-compiler, which lay down code in the application space.
• A cross-interpreter which calls (executes) the application code. All it takes is to jump to the code field in the application space and back again. (The problem is to jump back. This mechanism is added to the application in the form of a little monitor.)
• A cross-debugger, which examines the execution of code in the target.
• Space for new cross-compiler words generated in (by) the application.
    

Application system (Target)

The application system consists of the bare minimum for the application:

• The start code will be linked to the main routine of the application.
• A tiny monitor handles the communication with the host. In the general case of a different target processor the application code is downloaded to the target system and the interactivity is provided by a serial link to the monitor. The monitor only needs to receive and send memory data and execute a word given by its code address. Before and after execution the applications stack is copied from/to the development system (ROB89).
• A standard Forth kernel comprises the core word set without interpreter and compiler, i.e. just the necessary words to run the application. The kernel can be modified as needed.
• The standard extensions are loaded as needed. Examples are file-i/o, interpreter/compiler, double number arithmetic.
• Useful modules, like a string package, menu and window routines, are either provided or may be present from a former project. They are also loaded as needed. (Modules in source form are what we need.)
• Finally the actual application is built on this foundation.
    

3. Discussion

• The whole application program exists in source form. Kernel, extensions and utility modules are delivered as source code and can be shaped to the delights of the programmer.

• During development the target is part of the host. I.e. whenever the host is saved the target is also saved in its present state. Thus the host is dedicated to this application. To this end a copy of the development system is used to start a new application.

• There is no space limit to the target.

• The target can have a different organization from the host (direct threading, 32 Bits, extended addressing, ...).

• The host can be extended without fear of code contention. All conceivable goodies can be incorporated.

• The application system is as lean as possible. The code is header less. The applications dictionary is managed in the host.

• The programmer can apply all decent Forth tricks to the application. He can build new compiler and definer words.

• The application is built as a turnkey program. No need to strip parts of the system, no copyright problems. A commercial development system is never compromised.

• A vendor can deliver a closed development system (header less). His proprietary code is excellently shielded.

• For a given widely distributed development system it is interesting to provide utility modules. A closed development system is an ideal base for the development of third party utility modules.

• A vendor can deliver versions of the development system for separate target processors.

• No interpreter is needed in the target. The target is tested interactively via the host. The turnkey program will work exactly the same, since it is exactly the same program.

• If test words are needed in the target they could be a part of the application (call them with special function keys). An interpreter can certainly be added - we can build complete Forth systems, even our own enhanced development system, if the need arises.

• The price to pay for the separation of systems is a cross-compiler. This is no problem since the process of cross-compiling is well understood.
    

4. Enhancements

The handling of text files can be simplified if we use only one text file for the whole project. Then the dedicated development system and the text file carry the same name (with different extensions). Therefore the development system can automatically handle the text file; no file handling is needed by the programmer. (Contrast this with the elaborate file handling facilities of modern development systems.)

The utility modules are loaded into the text file (imported) once at the beginning of the project. Likewise modules may be exported as module files for use in another project.

It is quite feasible to develop a Forth program incrementally word by word in source and code. When we change a word only this word needs to be recompiled. The code of the word can be re-linked automatically with the rest of the program (the obvious way is to 'defer' every word, but there are more subtle schemes in use).

Incremental development really shines with separated systems. Programming becomes sculpturing. Words are crafted, added to the target, modified and tested immediately. When we work this way, compiling speed is an issue no more. The complete program is rarely recompiled. Most of the time we sculpture. 

To test the application we call it from the cross-interpreter. A useful addition to the host is a function key that calls the main word of the application directly from the browser. Then we can switch from the host to the target and back again with a stroke of a key. The momentary main word of the target could be linked to the host via a cross-compiler directive like "program: <targetmain>", included in the target source.
  

5. Conclusion

The concepts described in this paper have been realized and tested in my own system, Holon. This system is still experimental although heavily used to build applications. It evolved rather painlessly, once the basic structure was right [WEJ89].

The ideal development system ought to be transparent, putting as little in the way as possible between the programmer and his program. I have found the concept of separate systems helpful to approach this goal.
  

6. References

RAS89: F.Raschke et al., A Databased Forth, FORML 1989, p.348-353

ROB89: A.M.Robertson, Interactive Remote Compilation for Development and Machine Integration, FORML 1989, p.354-372

WEJ89: W.Wejgaard, Not Screens Nor Files But Words, FORML 1989, p.425-430 (reprinted in SIGForth Newsletter, Volume 1, No.4, p.13-15)

Copyright © 1999 by Wolf Wejgaard