Looking for cliparse.zip / Buscando cliparse.zip

[Antonio Linares]

When the Harbour project started, at the very early beginings, I published a first initial file named cliparse.zip

Do you have a copy of it for any chance ? thanks

Cuendo el proyecto Harbour comenzó, en su inicio, yo publiqué un pequeño prototipo inicial llamado cliparse.zip

Tienes una copia por causalidad ? gracias

ftp://ftp.iag.net/pub/clipper/general/cliparse.zip

this url is no longer working. Este enlace ya no funciona.

[Antonio Linares]

This was the start for Harbour

https://groups.google.com/forum/#!searchin/comp.lang.clipper/cliparse%7Csort:relevance/comp.lang.clipper/rcvDzVQ3xcY/ZCsG81PxEV0J

clipper.y (yacc rules)

Code: Select all  Expand view  RUN

%{

// Clipper yacc rules.
// Developed by Antonio Linares (alin...@fivetech.com, http://www.fivetech.com)
// A first step for a GNU Clipper
// Build 1: March 99
// Usage: byacc.exe -d -v clipper.y
// You may find byacc.exe at ftp://ftp.iag.net/pub/clipper/utils/byacc19.zip

#include <alloc.h>
#include <stdio.h>
#include <stdlib.h>

extern FILE * yyin;

void yyerror( char * );
int yylex( void );

%}

%union
{
   char * string;
   int    number;
   float  fNumber;
   void * pVoid;
};

%token FUNCTION IDENTIFIER RETURN NIL NUMBER INASSIGN
%token LOCAL STATIC IF ELSE END ENDIF LITERAL TRUE FALSE
%token INCLUDE EXTERN INIT EXIT AND OR NOT PUBLIC EQ NE1 NE2
%token INC DEC ALIAS DOCASE CASE OTHERWISE ENDCASE
%token WHILE EXIT LOOP END FOR NEXT TO DOWNTO LE GE
%token PLUSEQ MINUSEQ MULTEQ DIVEQ POWER EXPEQ MODEQ

%nonassoc NOT
%left  AND OR
%left  INASSIGN '='
%left  PLUSEQ MINUSEQ
%left  MULTEQ DIVEQ POWER
%left  EXPEQ MODEQ
%left  '+' '-' EQ NE1 NE2 INC DEC ALIAS
%left  '*' '/' '<' '>' LE GE
%right '\n' ';'

%type <string>  IDENTIFIER
%type <fNumber> NUMBER

%%

Main       : Source            { printf( "syntax ok\n" ); }
           ;

Source     : Crlf
           | Include
           | Extern
           | StaticsDef
           | Function
           | Source Crlf
           | Source Include
           | Source Extern
           | Source StaticsDef
           | Source Function
           ;

Include    : NE1 INCLUDE LITERAL
           ;

Extern     : EXTERN ExtList
           ;

ExtList    : IDENTIFIER
           | ExtList ',' IDENTIFIER
           ;

StaticsDef : STATIC VarList
           ;

Function   : FunHead Statements
           | FunHead VarDefs Statements
           | FunScope FunHead Statements
           | FunScope FunHead VarDefs Statements
           ;

FunHead    : FUNCTION IDENTIFIER Crlf
           | FUNCTION IDENTIFIER '(' ')' Crlf
           | FUNCTION IDENTIFIER '(' ParamList ')' Crlf
           ;

ParamList  : IDENTIFIER
           | ParamList ',' IDENTIFIER
           ;

Statements : Statement
           | Statements Statement
           ;

Statement  : ExecFlow Crlf
           | FunCall Crlf
           | ObjectMethod Crlf
           | ObjectData INC Crlf
           | INC ObjectData Crlf
           | Variable Crlf
           | VarAssign Crlf
           | RETURN Crlf
           | RETURN Expression Crlf
           | PUBLIC VarList Crlf
           ;

FunScope   : INIT
           | EXIT
           | STATIC
           ;

FunCall    : IDENTIFIER '(' ')'
           | IDENTIFIER '(' ExpList ')'
           ;

ObjectMethod : IDENTIFIER ':' IDENTIFIER '(' ')'
           | IDENTIFIER   ':' IDENTIFIER '(' ExpList ')'
           | FunCall      ':' IDENTIFIER '(' ')'
           | FunCall      ':' IDENTIFIER '(' ExpList ')'
           | ObjectMethod ':' IDENTIFIER '(' ')'
           | ObjectMethod ':' IDENTIFIER '(' ExpList ')'
           | ObjectData   ':' IDENTIFIER '(' ')'
           | ObjectData   ':' IDENTIFIER '(' ExpList ')'
           ;

Expression : NIL
           | NUMBER
           | LITERAL
           | Variable
           | Logical
           | Operators
           | FunCall
           | Array
           | CodeBlock
           | ObjectMethod
           | '&' '(' Expression ')'
           | AliasExp
           | '(' ExpList ')'
           ;

AliasExp   : IDENTIFIER ALIAS IDENTIFIER
           | IDENTIFIER ALIAS '(' FunCall ')'
           ;

Variable   : IDENTIFIER
           | IDENTIFIER INC
           | INC IDENTIFIER
           | IDENTIFIER DEC
           | DEC IDENTIFIER
           | ObjectData
           | ObjectData INC
           | INC ObjectData
           | ObjectData DEC
           | DEC ObjectData
           ;

VarAssign  : Variable '='      Expression
           | Variable INASSIGN Expression
           | Variable PLUSEQ   Expression
           | Variable MINUSEQ  Expression
           | Variable MULTEQ   Expression
           | Variable DIVEQ    Expression
           | Variable EXPEQ    Expression
           ;

Operators  : Expression '+'    Expression
           | Expression '-'    Expression
           | Expression '*'    Expression
           | Expression '/'    Expression
           | Expression '<'    Expression
           | Expression '>'    Expression
           | Expression LE     Expression
           | Expression GE     Expression
           | Expression AND    Expression
           | Expression OR     Expression
           | Expression EQ     Expression
           | Expression NE1    Expression
           | Expression NE2    Expression
           | Expression POWER  Expression
           | NOT Expression
           | VarAssign
           ;

ObjectData : IDENTIFIER   ':' IDENTIFIER
           | FunCall      ':' IDENTIFIER
           | ObjectData   ':' IDENTIFIER
           | ObjectMethod ':' IDENTIFIER
           ;

Logical    : TRUE
           | FALSE
           ;

Array      : '{' '}'
           | '{' ExpList '}'
           ;

CodeBlock  : '{' '|' '|' ExpList '}'
           | '{' '|' ParamList '|' ExpList '}'
           ;

ExpList    : ','
           | Expression
           | ExpList ','
           | ExpList ',' Expression
           ;

VarDefs    : VarScope VarList Crlf
           | VarDefs VarScope VarList Crlf
           ;

VarScope   : LOCAL
           | STATIC
           ;

VarList    : VarDef
           | VarList ',' VarDef
           ;

VarDef     : IDENTIFIER
           | IDENTIFIER INASSIGN Expression
           ;

ExecFlow   : IfEndif
           | DoCase
           | DoWhile
           | ForNext
           ;

IfEndif    : IfBegin EndIf
           | IfBegin IfElse EndIf
           ;

IfBegin    : IF Expression Crlf
           | IF Expression Crlf Statements
           ;

IfElse     : ELSE Crlf
           | ELSE Crlf Statements
           ;

EndIf      : ENDIF
           | END
           ;

DoCase     : DOCASE Crlf
                Cases
             ENDCASE

           | DOCASE Crlf
                Otherwise
             ENDCASE

           | DOCASE Crlf
                Cases
                Otherwise
             ENDCASE
           ;

Cases      : CASE Expression Crlf
           | CASE Expression Crlf Statements
           | Cases CASE Expression Crlf
           | Cases CASE Expression Crlf Statements
           ;

Otherwise  : OTHERWISE Crlf
           | OTHERWISE Crlf Statements
           ;

DoWhile    : WHILE Expression Crlf
                WhileStatements
             END
           ;

WhileStatements : Statement
           | EXIT Crlf
           | LOOP Crlf
           | WhileStatements Statement
           | WhileStatements EXIT Crlf
           | WhileStatements LOOP Crlf
           ;

ForNext    : FOR IDENTIFIER ForAssign Expression UpTo Expression Crlf
                ForStatements
           ;

ForAssign  : '='
           | INASSIGN
           ;

UpTo       : TO
           | DOWNTO
           ;

ForStatements : Statements NEXT
           | NEXT
           ;

Crlf       : '\n'
           | ';'
           | '\n' Crlf
           | ';'  Crlf
           ;

%%

void yyerror( char * s )
{
   printf( s );
   exit( 1 );
}

int main( int argc, char * argv[] )
{
   printf( "Clipper parser\n" );
   printf( "build 1 March 99\n\n" );

   if( argc > 1 )
   {
      yyin = fopen( argv[ 1 ], "r" );
      yyparse();
   }
   else
      printf( "Use: cliparse.exe <file.prg>\n" );

   return 0;
}

///------------ EOF -------------------------------

Clipper.l (Clipper lex rules)

Code: Select all  Expand view  RUN

%{

// Clipper lex rules.
// Developed by Antonio Linares (alin...@fivetech.com, http://www.fivetech.com)
// A first step for a GNU Clipper
// Build 1: March 99
// Usage: lex -i clipper.l
// You may find lex.exe at ftp://ftp.iag.net/pub/clipper/utils/byacc19.zip

#include <alloc.h>
#include <dos.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "y_tab.h"

%}

SpaceTab      [ \t]+
Number        ([0-9]+)|([0-9]+\.[0-9]+)|(\.[0-9]+)
Identifier    @?(([a-zA-Z])|([_a-zA-Z][_a-zA-Z0-9]+))
String        \"(([^\"]*)|([\!]*))\"

%x            COMMENT1 COMMENT2

%%

"//"|"&&"      BEGIN COMMENT1;
<COMMENT1>.*   BEGIN 0;

"/*"           BEGIN COMMENT2;
<COMMENT2>\n   ;
<COMMENT2>.*   ;
<COMMENT2>"*/" BEGIN 0;

{SpaceTab}     ;

\n+            return '\n';

"function"     return FUNCTION;
"do"[ ]+"case" return DOCASE;
("do"[ ]+"while")|"while" return WHILE;
"case"         return CASE;
"downto"       return DOWNTO;
"otherwise"    return OTHERWISE;
"end"          return END;
"endcase"      return ENDCASE;
"for"          return FOR;
"if"^\(        return IF;
"include"      return INCLUDE;
"init"         return INIT;
"else"         return ELSE;
"end"          return END;
"endif"        return ENDIF;
"exit"         return EXIT;
"extern"       return EXTERN;
"local"        return LOCAL;
"loop"         return LOOP;
"next"         return NEXT;
"public"       return PUBLIC;
"return"       return RETURN;
"static"       return STATIC;
"to"           return TO;
"nil"          return NIL;
"#"            return NE1;
"<>"|"!="      return NE2;
":="           return INASSIGN;
"=="           return EQ;
"++"           return INC;
"--"           return DEC;
"->"           return ALIAS;
"<="           return LE;
">="           return GE;
"+="           return PLUSEQ;
"-="           return MINUSEQ;
"*="           return MULTEQ;
"/="           return DIVEQ;
"^="           return EXPEQ;
"%="           return MODEQ;
".t."          return TRUE;
".f."          return FALSE;
".and."        return AND;
".or."         return OR;
"!"|".not."    return NOT;
[\(\)]         return yytext[ 0 ];
[,;+\-\*\/\{\}\|\#\&\:\<\>\=] return yytext[ 0 ];


{Number}       yylval.fNumber = atof( yytext ); return NUMBER;
{Identifier}   yylval.string = strdup( yytext ); return IDENTIFIER;
{String}       yylval.string = strdup( yytext + 1 ); yylval.string[ yyleng -
2 ] = 0; return LITERAL;

%%

///------------ EOF -------------------------------

clipper.mak

Code: Select all  Expand view  RUN

cliparse.exe : y_tab.c lexyy.c
   bcc -ml -ecliparse.exe y_tab.c lexyy.c

y_tab.c : clipper.y
   byacc -d -v clipper.y

lexyy.c : clipper.l
   lex -i clipper.l

///------------ EOF -------------------------------

To build it:

make -fclipper.mak

[Antonio Linares]

https://groups.google.com/forum/#!searchin/comp.lang.clipper/antonio$20pcode%7Csort:relevance/comp.lang.clipper/LWSLdJx1MA0/Gk6XB0XFjWcJ

FIVE 1.0


Index:

a.. Overview of Five 1.0.
b.. The Clipper virtual Machine (CVM).
c.. The compiler.
d.. The Run Time Library.

a.. OVERVIEW OF FIVE 1.0


Five 1.0 is a 32 bit Clipper-compatible compiler. It is divided in two
main parts: The compiler itself that generates OBJ files from PRG files and
the libraries to be linked to build the final application (EXE file).

The first part, the compiler, reads your Clipper code, and generates a
file with the PCode instructions necessary for being interpreted by the
Clipper Virtual Machine in order to execute your program. In order to do so,
the compiler reads each instruction and generates for it one or more PCode
instructions.

The compiler has been written using a lexical analyzer (Lex: FLEX
www.ecs.soton.ac.uk/support/gnu/flex ) and a syntax parser (Yacc: Berkeley
University BYACC htttp://www.first.gmd.de/cogent/catalog/lexparse.html )
which produce a lexer and a parser, respectively. Both tools, Flex and Byacc
may be free downloaded from Internet and they contain full source code. We
strongly recommend the reading of the book lex & yacc from By John Levine,
Tony Mason & Doug Brown published by Published by O'Reilly & Associates and
associates. You may order this book directly from Internet. This book will
teach you in deep about how to use these two powerful tools and this will
help you to understand how we have used them to write Five 1.0.

The second part, the Clipper Virtual Machine (CVM), is included in the
program as a Run Time Library, called Five.lib. Thus, all your Clipper
executable programs will include portions of Five.lib inside them. The CVM
uses the same technical principles as the Java Virtual machine. We recommend
the reading of Java Virtual Machine Published by O'Reilly & Associates
www.ora.com to get a deep understanding of this technology. The CVM uses
certain subsystems that are vital components in the final application. We
are going to analyze in deep these components to get a perfect understanding
of the CVM.

b.. THE CLIPPER VIRTUAL MACHINE


a.. Overview of the Clipper Virtual Machine (CVM)

The CVM is formed by the main execution loop and several subsystems, each of
which could be theoretically replaced, supposing that you respect the
interface of each subsystem.

The main execution loop is defined in the C function named CVM (the
equivalent to Plankton in the CA-Clipper compiler), which receives two
parameters: the pcode instructions to execute and the local symbol table (a
portion of the static symbol table) used by that pcode:

CVM( pcode, local symbols )

The CVM may invoke the CVM (itself) again. This let the Clipper language to
access Clipper functions and methods and external C language functions again
and again. The CVM organizes these multiple accesses in a ordered and full
controlled way and implements services to access these multiple execution
levels (ProcName(), ProcLine(), debugging, and stack variables access).

The CVM subsystems are continuously used by the main execution loop. Lets
review these CVM subsystems:


a.. The startup: Controls the initialization of the different CVM
subsystems and it is invoked at the beginning of the application. It also
controls the exiting of the application.

b.. The stack: The CVM does not use the stack of the computer
directly, it uses instead its own stack for manipulating values (parameters,
returned values, and symbols) as a hardware stack does.
c.. The static symbol table: Created by the compiler at compile time
and grouped by the linker, this subsystem is responsible for an immediate
access to functions location and it is highly related to the dynamic symbol
table at runtime. It contains many duplicated symbols that will be optimized
by the dynamic symbol table.
d.. The dynamic symbol table: Dynamically generated from the startup
subsystem at the beginning of the application. It organizes in an efficient
way the static symbol table creating an alphabetical index that allows a
dicotomic search of symbols. This subsystem is responsible for quick access
to symbols (functions, variables, fields and workareas aliases).
e.. The static and public variables: Responsible for storing public
and static variables. The CVM uses two different Clipper arrays for this
purpose.
f.. The memory: Responsible for allocating, reallocating, locking,
unlocking and freeing memory. It also organizes garbage collection. It
directly connects to the Windows API memory management functions. This
subsystem is one of those to be easily modified to port Five 1.0 to another
computer platform.
g.. The extend system: Defines the interface (_parc(), ...,
_retc() ) from low level (C language) to high level (Clipper language). This
subsystem is responsible for connecting in a proper way C language functions
to the entire application.
h.. Multidimensional arrays:This subsystem allows the creation of
arrays, and the services to manipulate these arrays in all ways. The arrays
are extensively used by the Clipper language and also they are the
foundation of the Objects (Objects are just arrays related to a specific
Class). These arrays are also used as a storage system by several subsystems
(static and public variables).
i.. The Objects engine:Responsible for the creation of Classes and
Objects. It also defines the way to access a specific Class method to be
invoked by the CVM and provides all kind of Classes information that may be
requested at runtime.
j.. The macro subsystem: it implements a reduced compiler that may
be used at runtime to generate pcode to be used by the application. In fact
it is a portion of the yacc Five 1.0 specifications.
k.. The workareas subsystem: Responsible for databases management.
It defines the locations where the used workareas will be stored and
provides all the functions to access those workareas. It also implements the
interface to the replaceable database drivers.


In the next chapters we are going to review in deep these subsystems to
fully understand each of them.

[Antonio Linares]

The Clipper pcode model

Lets consider the following Clipper sample Test.prg:

function Main()

? "Hello world!"

return nil


Once it gets compiled into a OBJ, what is there inside it ?
In fact, what we get is the equivalent to the following C language
application:


SYMBOL symbols[] = { ... };

void MAIN( void )
{
BYTE pcode[] = { ... };

VirtualMachine( pcode, symbols );
}


Basically, Test.prg source code has been converted into a sequence
of pcode bytes contained into the array pcode[] = { ... } and all our MAIN()
function does is invoke, at run-time, a Clipper VirtualMachine() that will
process those pcode bytes.

Lets review the Test.prg pcode structure in more detail:

0000 (2A) LINE 0 2A 00 00
0003 (2A) LINE 3 2A 03 00
0006 (13) SYMF [QOUT] 13 02 00
0009 (01) PUSHC "Hello world!" 01 ...
0018 (27) DO(1) 27 01 00
001B (2A) LINE 5 2A 05 00
001E (7B) UNDEF 7B
001F (79) SAVE_RET 79
0020 (1E) JMP 0023 1E 00 00
0023 (60) ENDPROC 60


We could define a pcode.h file to better read that pcode:

pcode.h

#define LINE 0x2A
#define SYMF 0x13
#define PUSHC 0x01
#define DO 0x27
#define UNDEF 0x7B
...


So finally it may looks like:

BYTE pcode[] = { LINE, 0, 0,
LINE, 3, 0,
SYMF, 2, 0,
PUSHC, 'H', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l',
'd', '!', '0',
DO, 1, 0,
LINE, 5, 0,
UNDEF,
SAVE_RET,
JMP, 0, 0,
ENDPROC };

And what is it SYMBOL symbols[] ? Clipper creates a symbol table into
the OBJ that later on will be used to create a dynamic symbol table
shared by the entire application. Each of those symbols have the following
structure:

typedef struct
{
char * szName; // Clipper infact keeps an array here (11 bytes).
BYTE bScope;
LPVOID pVoid;
} SYMBOL;

#define PUBLIC 0 // the scope of the function!

SYMBOL symbols[] = { { "MAIN", PUBLIC, MAIN },
{ "QQOUT", PUBLIC, QQOUT } };


Lets remember that the name of a function (MAIN, QQOUT) it is the address of
the
function, so our symbol table will be ready to use it to jump and execute
any
linked function.

In fact, the pcode SYMF 2, 0 in our sample, will instruct the
VirtualMachine()
to use the 2 symbol which it is QQOUT.

Lets read the pcode:

LINE 0, 0 => We are located at line 0
LINE 3, 0 => We are located at line 3
SYMF 2, 0 => We are going to call QQOUT from our symbol table (symbol nº
2)
PUSHC ... => This string is going to be used as a parameter
DO 1, 0 => ok, jump to QQOUT and remember we have just suplied 1
parameter
LINE 5, 0 => We are back from QQOUT and we are located at line 5
UNDEF => we are going to return this value (NIL)
SAVE_RET => Ok, return it
JMP 0 => We don't jump to elsewhere, just continue to next pcode byte
ENDPROC => This is the end. We have completed this function execution

All these instructions will be evaluated from our VirtualMachine() function
(Clipper names it _plankton()). All functions end using ENDPROC, so when
the VirtualMachine() finds ENDPROC it knows it has reached the end of a
function pcode.

Now that we clearly understand this basic model we are ready to start
implementing 'production rules' on our yacc (clipper.y) syntax to generate
the specific output file (test.c) with the above structure (or we could
easily just generate the OBJ file for it).

[Antonio Linares]

This is our pcode C model. It compiles fine.
Test.c

typedef unsigned char BYTE;
typedef void * LPVOID;

typedef struct
{
char * szName;
BYTE bScope;
LPVOID pVoid;
} SYMBOL;
#define PUBLIC 0

void MAIN( void );
void QOUT( void );
void VirtualMachine( BYTE * pcode, SYMBOL * symbols );

SYMBOL symbols[] = { { "MAIN", PUBLIC, MAIN },
{ "QOUT", PUBLIC, QOUT } };
void MAIN( void )
{
BYTE pcode[] = { 0, 1, 2, 3 };

VirtualMachine( pcode, symbols );
}

[Enrico Maria Giordano]

Fascinating...

EMG

[cnavarro]

Incredible, extraordinary