Recognizers

The goal of a recognizer is to dynamically extent the Forth command interpreter and make it understand and handle new data formats as well as new synatax’s. The present, 2nd generation recognizers achieve this by generalizing the classic interpreter with an API to factor the components. Recognizers are portable across different forth’s.

Recognizers are not a new concept for forth. They have been discussed earlier.

More recognizer examples are available at The Forth Net.

Version 4

Version 4 (pdf), (txt)

Forth source code and test code require Stack.frt and tester.fs.

Most changes are only wording changes. Most importandly many of the key words are renamed to better ones. They now describe what they do in a less confusing way (I hope). No changes were made to the stack effect so all existing code continues to work with a simple search and replace.

v3 v4
recognizer rec-stack
recognizer: dt-token:
do-recognizer recognize
r>* dt>*
r:fail dt:null

non-normative wording changes

v3 v4
rec:word rec:find
r:word dt:xt
r:num dt:num
r:dnum dt:dnum
r:float dt:float
r:name dt:nt

The data type token DT is not new actually. It got only a more prominent role since it is one result of the parsing process but is not related to recognizers themselves. It provides information about the data type and how to handle a certain data type (hence the name) inside the interpreter. It is not relevant where it comes from.

The gforth team suggests that the DT>x words should have a different stack effect. DT>COMP ( i*x DT:TOKEN -- j*y XT )

The input is what comes from the recognizer parsing words, the result should be feedable directly to EXECUTE to achieve the desired semantics. The words may change the result set too. In the reference implementation this is not used however.

The primary purpose of this seems to be an optimization and an unification of the name token and the data type token for named words), so that DT>COMP becomes identical to NAME>COMPILE for name tokens.

Outdated

The 1st formal RFD (pdf), (txt) was published at october, 3 2014. Version 2 (pdf), (txt) has been published on september, 20 2015. It improves the proposed standard section and adds a long chapter discussing the recognizer design based on feedback from version 1. The 3rd version has been started immediately after v2 due to a suggestion changing the POSTPONE action. Version 3 (pdf), (txt).

The Sourcecode requires Stack.frt. In the Recognizer-Test are many tests and example implementation for gforth, MPE’s vfxlin and Swift-Forth from Forth Inc.

The papers linked below give some historical background information.

Namespace RFD

An inofficial Namespace RFD.