One Definition of Forth

The first document published on the Forth programming language was Forth - A Language for Interactive Computing in 1970 (in HTML and PDF formats.) It states that the Forth was first implemented in 1968 and makes a number of points about the language. Amoung these points are:

1. Forth provides a complete software interface in contrast to the "multi-level nightmare" which separate assembler, supervisor (OS), compiler, and application programs. (p. 1)

2. Forth provides an order of magnitude improvement in cost, effort, and efficiency over the multi-level nightmare. (p. 1)

3. Everything in Forth is a word and each word. Words specify code that is to be executed whenever the word is encountered. (p. 6)

4. In Forth there are no reserved words or reserved concepts. The user is completely free to redefine any aspect of Forth. (p. 7)

5. "Forth encourages compact programs. Definitions make the programmers job so much easier that he is strongly motivated to isolate the significant features of his problem and to factor out those constructs that occur repeatedly. Proof of the effectivness of this technique is the compactness of Forth itself."(p. 16) Both core and source program size are orders of magnitude less than the conventional systems that they replace.(p. 17)

6. Forth is written in Forth.(p. 16)

These six points describe some of the aspects of Forth that set it appart from conventional programming enviroments where a hierachy of structure is present separating BIOS, device drivers, OS, GUI, DLL, application libraries, and application programs. In this multi-level nightmare the user is not free to change anything except the application program and the user is not free to change the reserved words or the syntax of the language even when it might greatly simplify an application program.

The item that is perhaps least clear is item number four, that the user is completely free to redefine any aspect of Forth. I think that the definition of Forth must be qualified by the other five statements because without any qualification, with the ability to change anything, one could declare anything to be Forth.

Chuck Moore has said that Forth is more an approach to problem solving than a specification for a programming language. The approach involves an analysis of the problem and an effort to eliminate any work that is not required to reach the desired goal. He has said, "Forth doesn't need to be complicated. Classic Forth started out simple, it gradually accreted layers of complexity. At Forth Inc. that kind of became the company culture. We had this package and we were selling it and we were exploiting it and we were stuck with it." (1X Forth C.M. 1999)

"My contention is that every application that I have seen that I didn't code has ten times as much code in it as it needs. And I see Forth programmers writing applications with ten times as much code as necessary. The concern that I have, the problem that I have been pondering for the last few years is: How can I pursuade these people to write good Forth? How can I pursuade them that it's possible to write good Forth? Why would anyone want to write ten times as much as they would need to write?" (1X Forth C.M. 1999)

From the points that Chuck has made I think that it should be clear that Chuck's claims that orders of magnitude of improvement are available are closely tied to an application of the approach taken in Forth. However in the thirty years of Forth's history, and in particular in the last fifteen years, there has been a widespread effort by a great number of people to take Forth in nearly the opposite direction as what Chuck described as Forth.

Instead of seeing Forth as an approach to problem solving rather than a specification for a programming language there is now a formal ANSI specification for the Forth programming language. Some of the people responsible for standardizing Forth now feel that Chuck Moore should stop using his definition of Forth and thus stop using the term Forth to describe his programming. They feel that they now own the term and acknowledge that their definition is not the same as the definition used by the inventor of the language.

In Chuck's definition Forth provides a complete software interface and thus enables the programmer to make custom changes to the operating system code, the device drivers, the compiler, the user interface, or anything they want because it is part of Forth. In a conventional environment the programmer must accept many of these things as part of the problem. They may choose to use a fixed operating system that they didn't write, or a compiler with fixed syntax and reserved words, or other people's device drivers and DLLs, or the standard GUI interfaces that other people use, or other things that they are not free to change. In short they may choose to take a conventional programming approach rather than Forth.

But many programmers have choosen to use the term Forth for what they do but to reject the main defining principles of Forth. Many programmers choose to reject the idea that Forth is the OS. Instead they prefer to implement Forth as a conventional programming language on top of the operating system of their choice. By doing so they violate several of the original definitions of Forth. What they have is not longer a complete system, instead they have choosen to return to the multi-level nightmare. In doing so they have given up much of the power that Forth offers to reduce the cost, effort and improve the efficiency by an order of magnitude.

There are programmers who choose to extend Forth to look more like other conventional programming languages by providing extenstions that look like 'C' or OOP. They prefer fixed and standardized approaches rather than the open approach that Forth offers with its ability to let the programmer solve a problem with what they consider the best approach for that problem rather than forcing things into the mold of 'C' structures or OOP methods.

There are programmers who have choosen to use the term Forth for what they do. According to the inventor of the language Forth has no reserved words or even reserved concepts. There are programmers who will say that Forth allows you to change anything you want to change, but then they add that there are hundreds of pages of documentation about all the things that you cannot change. In addition to the hundreds of pages of things that the programmer must NOT change in Forth for it to be a "standard" Forth many users also insist on pages of rules for naming files, naming words, spelling words, layout and indentation of source etc. These programmers will say that a programmer is free to change anything in Forth except for the hundreds of pages in the official language specification for Forth. People who have personal preferences about which programming language they prefer will often want to use the ability to change Forth to make it look more like the programming language of their choice. They choose to add words and concepts from other languages, they choose to adopt the same syntax and concepts from other languages until it may be difficult to distinguish their Forth from some other language. In particular when people add extenstions to let them use the syntax from other languages, then use this to make copies of programs in these other languages using the same factoring, function names, syntax, filenames, header files, etc., they have abandoned most of the features of the Forth approach and also give up the benefits associated with that approach. Often the result has some features from Forth grafted onto a poor pale implementation of the language of their choice.

The programmers who have choosen to accept the restrictions of fixed BIOS, fixed OS, fixed GUI, fixed language specification, fixed DLLs, fixed application libraries, fixed OOP packages, etc. will say that this approach makes their job easier because they can just focus on the application program. But they are also forced to deal with huge problems in the process. They are no longer able to create simple or compact programs. They are no longer able to write programs ten times faster by making them ten times smaller. They are no longer able to optimize the bottlenecks because they are not allowed to change most things.

The Microsoft Windows (tm) GUI API is a hundred times more complex than Forth itself. A program resembling Forth, but which uses the conventional BIOS, OS, GUI, DLL, will be one hundred times more complex than a Forth that as the original definition of Forth states provides a complete environment. In this environment it is no longer possible to create compact or efficient programs. The 10% of the programs that are Forth can be optimized to be compact and efficient and may be more compact and efficient than other programs that also start with a 90% overhead.

In Forth it is easy to profile an application and identify the bottlenecks that limit performance. In Forth things get factored such that it is easy to focus on the performance bottlenecks and improve them. In Forth the device drivers, OS and GUI functions and compiler constructs are part of Forth and thus available for optimization by the programmer.

In conventional environments like Windows (TM) it is well known that the OS and GUI will demand a large percentage of the system resources. It was commonly accepted that Windows would use 90% of the power of machine leaving only 10% of the power available for applications in early implementations and on early machines. Now that machines have increased speeds and available memory the percentage of resources used by the GUI may be less than 90%. But in any case this is clearly the bottleneck to performance in this environment. Yet the major Forth vendors have put years of effort into optimizing the code they compile. They have ignored the bottleneck and focused on optimizing the 10% available to their code, they can do nothing about the 90% given overhead. They tend to focus their promotional activities on the results of benchmarks on the 10% that they are free to optimize and don't talk about the real restrictions to performance since they are a given. Programmers in these enviroments just accept the overhead as a given, rather than taking the Forth approach to empower the programmer to improve whatever needs improvement.

I can personally relate my experince in moving from a 120Mhz Pentium machine with 24MB of memory to a machine with a 360Mhz Pentium II and 128MB of memory. I foolishly expected to see about a 4x improvement in speed because of the faster processor and increased memory. However the machine also included an upgrade to a later version of Windows that imposes a much greater overhead. The result was more or less a wash. Internally the Forth programs compiled by popular commerical Forth for Windows products do show code speedup, but as I said before that is not the bottleneck. When you compare the overall program performance including load times and GUI calls the increased overhead is clearly apparent.

Forth is written in Forth. There are programmers who prefer to write programs in other languages. It is possible to write something resembling a Forth interpreter or compiler in some other language. I like to say if it is written in 'C' that it is 'C' and that if it is written in Java that it is Java. A "Forth in C" may provide Forth scripting ability in 'C' but it is still 'C'. It will not have the ability we have in Forth to change whatever we want, it will have to comply with the reserved words and fixed syntax of 'C'.

Forth is the interpeter/compiler not the application. An application may be written in Forth but it is not Forth. There must be some line drawn between where Forth ends and an application begins. Many Forths today include things that are extentions to Forth into the direction of an application in case a user chooses to build an application that might head in that direction. The line between Forth and application may be a tough line to draw. Where do you start?

I think one must start with the idea that Forth is simple and fairly general purpose. When you begin to specialize your focus on a particular problem domain you will need to define extentions to Forth. These extensions are Forth code but not Forth itself. One of Chuck's definitions of Forth was that it is simple, compact, and a methodology of focusing on solving specific problems rather than providing specifications for general purpose solutions to all problems. I think to be consistent with the concept of Forth a Forth system must be simple and compact.

Over the years Chuck has focused on making his Forths simpler, faster, and more compact. He has done this because he wants the minimal amount of things that cannot be changed between him and the problem he wants to solve. His Forths have shown a consistent methodology and approach and a series of smaller, simpler, more efficient implementations of his language. In sharp contrast the majority of Forth programmers have gone in nearly the opposite direction. They have thrown out most of the Forth methodology, their Forths are not complete systems, they rely on other people's BIOS, OS, GUI, Languages, Compilers, and DLL and are thus not able to reduce many of the problems they must face to construct a solution.

Forth has been most sucessful in the embedded programming arena where severe restrictions in computing power and memory size dictated by minimal cost and power consumption requirments demand compact and efficient programs. In these enviroments Forth provides and enviroment where the programmer is free to change anything. They are stuck with an OS or GUI that demands that many megabytes of memory overhead and a high computational overhead force the use of more expensive hardware. In these environments it is quite natural that Forth be its own operating system and GUI.

When I worked at the iTV Corporation I worked on their 4OS which provides a large number of Operating System Services Because it was written in Forth it was remarkably small, compact and fast. Some of the features that it supports are fast multitasking, memory management with garbage collection and file caching, compressed boot code, flash file system, a rich variety of network protocols including SLIP, PPP, TCP/IP, UDP, TFTP, Telnet, and HTTP, GUI with windowing, multiple fonts and font attributes, GIF and JPEG image decode and display, and object oriented programming extensions. One advantage of the system was that because it was small and efficient it could run the email and browser programs with about 1/5 of the memory of the next smallest compeditor's products. Another advantage was that it was designed to run on custom Forth hardware with 1/1000 the cost and power consumption of conventional systems.

Naturally I was inclined to make comparisons between the performace of the big machines with many very big and very expensive components running Windows to the small machines we made when they perform the same functions. Both systems had email and browser programs and could thus download and display the same images in HTML documents. Providing overall higher performance on a machine with 1/1000 the resources was sometimes tricky. But many of the functions in Windows (tm) are so badly designed and coded that providing higher performance is no problem. In particular GUI and OS features showed the biggest differences. The fun was not just getting higher performance than a Windows machine with 1000 times more resources but providing much higher performance. I always found it fun when we could do something a million times faster than the Windows machines.

The design and performance of Windows is an industry joke. However the alternatives that people use my be far superior to Windows but also have serious code bloat and performance problems. In our enviroments we are used to doing things quickly and simply. Even on simple processors we are used to fast multitasking and were able to get task switches in 20 to 200ns. My windows machine takes from 20 milliseconds to 20 seconds. In contrast Unix systems usually fall in the middle around 20 microseconds. The latest release from Bell Labs, Plan 9, was designed to improve this number by a factor of ten, still far short of the performace we are used to in Forth.

Today there are many people who do not write in Forth, but write Forths in something else to get some of Forths features in their environments. I think there should be a special term to describe them. Today most Forths have gone very far in nearly the opposite direction described by the inventor of Forth as Forth. There are many Forths today that are as big, as slow, and as complex as everything else. This is true because they include the requirement to interface to everything the same way as everone else does in other languages. They do not reflect the Forth approach of minimizing the complexity to be able to get the most out of the programmer and the computer.

"Forth is what Forth programmers do. I would like to think of it as what Forth programmers ought to do." (1X Forth C.M. 1999)

Jeff Fox 9/2/00