ScriptBasic Developers Manual

by Peter Verhas

Table of Contents

[Contents]

1. Introduction

ScriptBasic is a scripting implementation of the programming language BASIC. The interpreter has several features that make it a considerable choice when program designers want to choose an interpreter to embed into an application.

The aim of the implementation of this interpreter was to deliver a programming language that can be powerfully used to program several tasks by programmers without learning a new system, a new programming language. Thus the language BASIC was chosen and was implemented in a interpreter with the following features:

[Contents]

1.1. Chapters

This documentation has four main chapters. These are

[Contents]

2. Interpreter Architecture

This chapter tells you the architecture of the interpreter. It is not a must to read this chapter, and you may find that some topic is irrelevant or not needed to learn to embed or to extend ScriptBasic. However understanding the internal working order of ScriptBasic should help you understand why some of the extending or embedding interfaces work the way they actually do. So I recommend that you read on and not skip this chapter.

To read this chapter and to understand the internal working of the interpreter is vital when you decide to write an internal preprocessor. Internal preprocessors interact with not only the execution system but also the reader, lexer, syntaxer and builder modules thus internal preprocessor writers have to understand how these modules work.

ScriptBasic is not a real scripting language. This is a mix of a scripting language and compiled languages. The language is scripting in the sense that it is very easy to write small programs, there is no need for long variable and function declarations. On the other hand the language is compiled into an internal code that it executed afterwards. This is the technique for example the implementations of the language Java.

ScriptBasic as a language is a BASIC dialect that implements most BASIC features that BASIC implementations usually do. However ScriptBasic variables are not typed, and dynamic storage, like arrays are automatically allocated and released. A ScriptBasic variable can store a string, an integer, a real value or an array. Naturally a variable can not store more than one of any of these types of values. But you need not declare a variable to be INTEGER or REAL, or STRING. A variable may store a string at a time and the next assignment command may release the original value and store a different value in the variable.

When a program is executed it goes through several steps. The individual steps are implemented in different modules each being coded in a separate C language source file. The modules are

The following sections detail these modules and also some other modules that help these modules to perform their actual tasks.

[Contents]

2.1. External Preprocessor

The module that implements the external preprocessor handling is coded in the C source file `epreproc.c' (Note that there is a file `ipreproc.c' that handles internal preprocessors.) The sole function in this file is named epreproc. This is quite a complex function that gets a lot of arguments listing all the preprocessors that have to be executed one after the other on the source file or if not the first one then on the file created by the previous one. The function gets a series of preprocessors to be executed in the command line but in case there is no preprocessor defined in the arguments it executes the preprocessors that have to be executed due to the BASIC program file name extension.

Only the preprocessors that are configured can be executed. Each preprocessor has a symbolic name that is used to name it in the configuration file or on the command line using the option `-p'

An external preprocessor is a program that reads a text file and produces a text file. ScriptBasic executes the external preprocessor in a separate process and supplies it with the input and the output file name on the command line. For example the `scriba.conf.lsp' configuration file may contain the following lines

preproc (
  extensions (
     heb "heb"
     )
  external (
    heb (
      executable "/usr/bin/scriba /usr/share/scriba/source/heber.bas"
      directory "/var/cache/scriba/hebtemp/"
      )    
    )
  )

The key executable defines the executable image of the preprocessor. In this case because the HEB preprocessor is written in BASIC this is an executable image and an argument. The other arguments, namely the input file name and the output file name is appended after this string separated by space. When the preprocessor is executed the actual command line is

/usr/bin/scriba /usr/share/scriba/source/heber.bas myprog.heb /var/cache/scriba/hebtemp/ADBKPLNBDELMMNKGNBBLHAHGBKKJFIMN

The final argument is a file name that is automatically generated by ScriptBasic using MD5 calculation of the input file name full path (may and presumably should not be correct in this example above as I just typed some random 32 characters). The preprocessor should read the file myprog.heb and should generate /var/ca@dots{MN} (forgive me for not typing that long file name again).

If there are more preprocessor to be executed on the generated result then the next is executed on the generated file as input file and another output file is generated.

The preprocessor program should gracefully exit the process it runs in and exit with the code 0. Any other process exit code is treated as error and the further processing of the BASIC program is aborted.

[Contents]

2.2. Reader

The module reader is implemented in the C source file `reader.c' This is a very simple module it just reads the lines from the source code files and stores the actual text in memory using linked lists. There is not too much possibility to configure this module except that the memory handling functions and the file opening, closing and reading functions are used via function pointers that can be altered by the caller.

Like any other module in ScriptBasic the reader module uses a module object. This is like a class definition except that the interpreter is coded in C and thus there is nothing like VTABLE or inheritance. Otherwise the code is object oriented. Here we list the actual definition of the reader object. Note however that this is actually a copy of the actual definition from the file `reader.c' and it may have been changed since I wrote this manual. So the reader object by the time I wrote this manual was:

#define BUFFER_INITIAL_SIZE 1024
#define BUFFER_INCREMENT 1024
 typedef struct _ReadObject {
  void * (*fpOpenFile)(char *, void *);
  int (*fpGetCharacter)(void *, void *);
  void (*fpCloseFile)(void *, void *);
  void *pFileHandleClass;

void *(*memory_allocating_function)(size_t, void *); void (*memory_releasing_function)(void *, void *); void *pMemorySegment;

ptConfigTree pConfig;

char *Buffer; long dwBuffer; long cBuffer;

pSourceLine Result; pSourceLine CurrentLine; long NextCharacterPosition; char fForceFinalNL;

pReportFunction report; void *reportptr; int iErrorCounter; unsigned long fErrorFlags;

pImportedFileList pImportList;

char *FirstUNIXline; struct _PreprocObject *pPREP; } ReadObject, *pReadObject ;

The pointers fpOpenFile, fpGetCharacter and fpCloseFile point to functions that are used to open the input file. The pointer pFileHandleClass set by the higher code using the module reader is passed to these functions without caring its meaning. This is not used by the standard file input/output functions that are used by the command line version of the program, but can be useful for program environment when the source file is stored in some other forms and not in a file. An example of such use can be seen in the function scriba_LoadProgramString implemented in the file `scriba.c'.

The linked list of source lines is stored in the structure named SourceLine The definition of this structure is

typedef struct _SourceLine {
  char *line;
  long lLineNumber;
  long LineLength;
  char *szFileName;
  struct _SourceLine *next;
  } SourceLine, *pSourceLine;

You can see that each source line is pointed by the field line and the length of the line is also stored. The reason for this extra field is that the line itself may contain zero character although this is rare for a program source file to contain zero character inside.

Before the file is read the function reader_InitStructure should be called. This is usual for the ScriptBasic modules. This function initializes the reader object to the usual values that actually ScriptBasic needs.

The reader provides function reader_ReadLines that actually reads the lines and also processes all lines that contain an include or import directive to include a line.

The reader has some extra functions that are specific to ScriptBasic or generally saying are specific to program source reading.

Source programs under UNIX usually start with a line

#! /usr/bin/scriba

that tells the operating system how to start the code.

((((Some very old and totally outdated version of some UNIX systems check the first four characters to look for #! /. There is a space between the ! and the /. So if you want to be look a real code geek put this extra space before the executable path. To be honest I have never encountered this issue since 1987 when I met my first UNIX at TU Delft, Hollandia. OK that's for the story, get back to the reader!))) Always close the parentheses you open!)

The reader recognizes this line if this is the very first line of the program and unlinks it from the list. Instead you can reach this line via the reader object variable FirstUNIXline. This is specific to program source reading and not general file reading. But the reader module is a program source reader or more specific: a BASIC, even ScriptBasic program source reader, though it was coded to be as general as possible.

Another specific issue is the new line at the end of the last line. Lines are usually terminated by new-line. This line terminating character is included in the string at the end of each element of the linked list the reader creates. However when the last line is terminated by the pure EOF some syntax analyzers may fail (ScriptBasic syntax analyzer is also an example, but the reader is kind to care about this). For the reason if the variable fForceFinalNL is set to be TRUE this line gets the extra new-line when read.

This module contains the functions that read a source file.

Script basic has several passes until it can start to execute the code. The very first pass is to read the source lines from the files. The routines in this module perform this task and build up a linked list that contains the ascii values of the lines.

The input functions are parametrized, and the caller should support. If you have different system dependent file reading functions, or if you have the input file in some format in memory or in any other data holding space you can support these routines with character fetch functions.

[Contents]

2.2.1. reader_IncreaseBuffer()

When the reader encounters a line which is longer than the currently allocated input buffer it calls this function to increase the size of the input buffer. The input buffer is linearly increased by BUFFER_INCREMENT size (defined in the header section of reader.c

When a new buffer is allocated the bytes from the old buffer are copied to the new and the old buffer is released. It is vital that the buffer is always referenced via the pRo->buffer pointer because resizing buffer does change the location of the buffer.

If the memory allocation fails the function return READER_ERROR_MEMORY_LOW error. Otherwise it returns zero.

int reader_IncreaseBuffer(pReadObject pRo
  ){

[Contents]

2.2.2. reader_gets()

This function reads a newline terminated line from the file. The file is identified by function pRo->fpGetCharacter and the pointer fp.

When the input buffer is too small it automatically increases the buffer. The terminating new line is included in the buffer. If the last line of the file is not terminated by newline an extra newline character is added to this last line.

The addition of this extra newline character can be switched off setting pRo->fForceFinalNL to false. Even if this variable is false the normal newline characters which are present in the file are included in the buffer.

int reader_gets(pReadObject pRo,
                 void *fp
  ){

[Contents]

2.2.3. reader_ReadLines()

This function calls reader_ReadLines_r() to read the lines of the file given by the file name szFileName into pRo->Result. For further information see reader_ReadLines_r().
int reader_ReadLines(pReadObject pRo,
                     char *szFileName
  ){
The function returns zero or the error code.

[Contents]

2.2.4. reader_ReadLines_r()

This function reads the lines of a file and creates a linked list of the read lines.
int reader_ReadLines_r(pReadObject pRo,
                       char *szFileName,
                       pSourceLine *pLine
  ){
The file is identified by its name given in the string variable szFileName. The file is opened by the function pointed by pRo->fpOpenFile This function should return a void pointer and this void pointer is passed to reader_gets() (reader_gets) to get a single character.

The argument pLine is a pointer to a SourceLine pointer. The linked list lines read will be chained into this pointer. The last read line will be followed by the line pointed by *pLine and *pLine will point to the first line.

This design makes it easy to use and elegant to perform file inclusions. The caller has to pass the address of the pointer field next of the source line after which the file is to be inserted.

See also ReadLines that calls this function.

[Contents]

2.2.5. reader_ProcessIncludeFiles()

This function is called from reader_ReadLines() after calling reader_ReadLines_r().

This function goes through all the lines and checks if there is any line containing an include directive.

An include directive is a line starting with a word INCLUDE (case insensitive) and is followed by the file name. The file name can be enclodes between double quotes.

Note that the processing of the include directives are done on the characters on the line, because they are processed before any tokenization of the lexer module. This can cause some problem only when there is an include like line inside a multiline string. For example:

a = """Hey this is a multiline string
include "subfile.txt"
"""

This will include the file subfile.txt and its content will become part of the string. This becomes more complicated when the file subfile.txt contains strings.

The file name may not be enclosed between double quotes. In this case the file is tried to be found in predefined system directories.

If the programmer uses the command IMPORT instead of INCLUDE the file will only be included if it was not included yet into the current program.

void reader_ProcessIncludeFiles(pReadObject pRo,
                                pSourceLine *pLine
  ){

The file read is inserted into the plce where the include statement was.

[Contents]

2.2.6. reader_LoadPreprocessors()

Preprocessors are not part of ScriptBasic. They can be implemented as external DLLs and should be configured in the configuration file.

When a line contains

PREPROCESS preprocessorname

this reader module loads the preprocessor DLL or SO (dll under unix) file.

void reader_LoadPreprocessors(pReadObject pRo,
                                 pSourceLine *pLine
  ){

[Contents]

2.2.7. reader_StartIteration()

The package supports functions that help upper layer modules to iterate through the lines read. This function should be called to start the iteration and to set the internal iteration pointer to the first line.
void reader_StartIteration(pReadObject pRo
  ){

[Contents]

2.2.8. reader_NextLine()

This function returns a string which is the next line during iteration. This function does NOT read anything from any file, only returns a pointer to a string that was already read.

This function can be used together with reader_NextCharacter(). When a line was partially passed to an upper layer that uses reader_NextCharacter this function will only return the rest of the line.

char *reader_NextLine(pReadObject pRo
  ){

[Contents]

2.2.9. reader_NextCharacter()

This function gets the next character from the actual line, or gets the first character of the next line.

This function does NOT read anything from any file, only returns a character from a string that was already read.

When the last character of the last line was passed it return EOF

int reader_NextCharacter(void *p
  ){

[Contents]

2.2.10. reader_FileName()

This function returns the file name of the actual line. This is the string that was used to name the file when it was opened. This can be different for different lines when the reader is called several times to resolve the "include" statements.
char *reader_FileName(void *p
  ){

[Contents]

2.2.11. reader_LineNumber()

This function returns the line number of the current line durig iteration. This number identifies the line in the file where it was read from.

long reader_LineNumber(void *p
  ){

[Contents]

2.2.12. reader_InitStructure()

This function should be called to initialize the reader structure. It sets the file handling routines to the standard fopen, fclose and getc functions, and also sets the function pointers so that the module uses malloc and free.

void reader_InitStructure(pReadObject pRo
  ){

[Contents]

2.2.13. reader_RelateFile()

This function gets a file name, which is either absolute or relative to the current working directory and another file name which is absolute or relative to the first one.

The return value of the function is a file name which is either absolute or relative to the current woring directory.

The return value is dynamically allocated and is to be release by the caller. The allocation function is taken from the class function and the segment is pMemorySegment.

char *reader_RelateFile(pReadObject pRo,
                       char *pszBaseFile,
                       char *pszRelativeFile
  ){

[Contents]

2.2.14. reader_DumpLines()

This is a debug function that prints the lines into a debug file.
void reader_DumpLines(pReadObject pRo,
                      FILE *fp
  ){

[Contents]

2.3. Lexer

The module lexer is implemented in the C source file `lexer.c' This is a module that converts the read characters to a list of tokens. The lexer recognizes the basic lexical elements, like numbers, strings or keywords. It starts to read the characters provided by the reader and group it p into lexical elements. For example whenever the lexical analyzer sees a " character it starts to process a string until it finds the closing ". When it does the module creates a new token, links it to the end of the list and goes on.

To do this the lexical analyzer has to know what is a keyword, string or number.

Because general purpose, table driven lexical analyzers are usually rather slow ScriptBasic uses a proprietary lexical analyzer that is partially table driven, but not so general purpose as one created using the program LEX.

There are some rules that are coded into the C code of the lexical analyzer, while other are defined in tables. Even the rules coded into the C program are usually parameterized in the module object.

Lets see the module object definition from the file `lexer.c' (Note that the C .h header files are extracted from the .c files thus there is no need to double maintain function prototypes.)

Note however that this is actually a copy of the actual definition from the file `lexer.c' and it may have been changed since I wrote this manual. So the lexer object by the time I wrote this manual was:

typedef struct _LexObject {
  int (*pfGetCharacter)(void *);
  char * (*pfFileName)(void *);
  long (*pfLineNumber)(void *);
  void *pvInput;
  void *(*memory_allocating_function)(size_t, void *);
  void (*memory_releasing_function)(void *, void *);
  void *pMemorySegment;

char *SSC; char *SCC; char *SFC; char *SStC; char *SKIP;

char *ESCS; long fFlag;

pReportFunction report; void *reportptr; int iErrorCounter; unsigned long fErrorFlags;

char *buffer; long cbBuffer;

pLexNASymbol pNASymbols; int cbNASymbolLength;

pLexNASymbol pASymbols;

pLexNASymbol pCSymbols; pLexeme pLexResult; pLexeme pLexCurrentLexeme; struct _PreprocObject *pPREP; }LexObject, *pLexObject;

This struct contains the global variables of the lexer module. In the first "section" of the structure you can see the variables that may already sound familiar from the module reader. These parameterize the memory allocation and the input source for the module. The input functions are usually set so that the characters come from the module reader, but there is no principal objection to use other character source for the purpose.

The variable pvInput is not altered by the module. It is only passed to the input functions. The function pointer name pfGetCharacter speaks for itself. It is like getc returns the next character. However when this function pointer is set to point to the function reader_NextCharacter the input is already preprocessed a bit. Namely the include and import directives were processed.

This imposes some interesting feature that you may recognize now if you read the reader module and this module definition carefully. include and import works inside multi-line strings. (OK I did not talk about multi-line strings so far so do not feel ashamed if you did not realize this.)

The function pointers pfFileName and pfLineNumber should point to functions that return the file name and the line number of the last read character. This is something that a getc will not provide, but the reader functions do. This will allow the lexical analyzer to store the file name and the line number for each token.

The next group of variables seems to be frightening and unreadable at first, but here is this book to explain them. These variables define what is a string, a symbol, what has to be treated as unimportant space and so on. Usually symbols start with alpha character and are continued with alphanumeric characters in most programming languages. But what is an alpha character? Is _ one or is $ a valid alphanumeric character. Well, for the lexer module if any of these characters appear in the variable SSC then the answer is yes. The name stands for Symbol Start Characters. But lets go through all these variables one by one.

The default values for these variables are set in the function lex_InitStructure. Interestingly these default values are perfectly ok for ScriptBasic.

The field pNASymbols points to an array that contains the non-alpha symbols list. Each element of this array contains a string that is the textual representation of the symbol and a code, which is the token code of the symbol. For example the table NASYMBOLS in file `syntax.c' is:

LexNASymbol NASYMBOLS[] = { { "@\\" , CMD_EXTOPQN } , { "@`" , CMD_EXTOPQO } , { "@'" , CMD_EXTOPQP } , { "@" , CMD_EXTOPQQ } ,

...

{ "@" , CMD_EXTOPQ } , { "^" , CMD_POWER } , { "*" , CMD_MULT } , { NULL, 0 } };

When the lexical analyzer finds something that is not a string, number or alphanumeric symbol it tries to read forward and recognize any of the non-alpha tokens listed in this table. It is extremely important that the symbols are ordered in this table so that the longer symbols come first thus a symbol abc is not presented before abcd. Otherwise abcd will never be found!

The variable cbNASymbolLength is nothing to care about. This is used internally and is calculated automatically by the lexical analyzer.

The variable pASymbols is similar to the variable pNASymbols pointing to a same kind of table. This variable however should point to an array that contains the alphanumeric symbols. You can find the array ASYMBOLS in file `syntax.c' that is pointed by this variable for ScriptBasic.

The order of the words in this array is not important except that more frequent words being listed earlier result faster compilation.

The field pCSymbols points to an array that is used only for debugging purposes. I mean debugging ScriptBasic code itself and not debugging BASIC programs.

The rest of the variables are used by the functions that iterate through the list of tokens when the syntax analyzer reads the token list or to report errors during lexical analysis. Error reporting is detailed in a separate section.

The tables that list the lexical elements are not maintained "by hand". The source for ScriptBasic syntax is maintained in the file `syntax.def' and the program `syntaxer.pl' creates the C syntax file `syntax.c' from the syntax definition.

The program `syntaxer.pl' is so complex that after two years I wrote it I had hard time to understand it and I rather treat it as a holly code: blessed and untouchable. (Ok: see: that code is quite compound, but if there was any bug found in that I could understand what I did in a few hours. Anyway, the brain created that code once belonged to me.)

[Contents]

2.3.1. Functions implemented in this module

The following subsections list the functions that are implemented in this module. The source text of this documentation was extracted from the source Documentation embedded in the C source as comment. Treat these subsections more as reference documentation and less tutorial like.

This module contains the functions and structures that are used by ScriptBasic to perform lexical analysis of the source code. The module was originally developed for ScriptBasic but was developed to be general enough to be used in other projects.

[Contents]

2.3.1.1. lex_SymbolicName()

This function usually is for debug purposes. This searches the table of the predefined symbols and returns the string which is the predefined symbols for which the code was passsed.
char *lex_SymbolicName(pLexObject pLex,
                       long OpCode
  ){

[Contents]

2.3.1.2. lex_HandleContinuationLines()

This function is called from the main function before syntax analysis is started. This function handles the usual basic continuation lines. If the last character on a line is a _ character, which is either recognised during lexical analysis as a character or as a symbol then this lexical element and the following new-line character token is removed from the list of tokens.

void lex_HandleContinuationLines(pLexObject pLex
  ){

[Contents]

2.3.1.3. lex_RemoveSkipSymbols()

This function is called from lex_DoLexicalAnalysis() to remove the lexical elements from the list of tokens that were denoted by the preprocessors to be deleted.

Some lexical elements are used to give information to some of the preprocessors. These tokens should be deleted, because later processing can not deal with them and confuses syntax analysis.

In those cases the preprocessor should set the type of the token to be LLEX_T_SKIP or LEX_T_SKIP_SYMBOL. The type LEX_T_SKIP should be used in case the token is handled due to ProcessLexSymbol preprocessor command and LEX_T_SKIP otherwise.

When the type is set LEX_T_SKIP_SYMBOL the lexical analyzer knows to release the string holding the symbol. If the type is LEX_T_SKIP only the token record is released.

If the symbol string is not released due to erroneously setting the type to LEX_T_SKIP instead LEX_T_SKIP_SYMBOL the memory will not be released until the interpreter finishes pre execution steps. So usually if you do not know how to set the type to skip a token LEX_T_SKIP is safe.

void lex_RemoveSkipSymbols(pLexObject pLex
  ){

[Contents]

2.3.1.4. lex_RemoveComments()

This function called from the function lex_DoLexicalAnalysis() function to remove the comments before the syntax analysis starts.

It should be called before calling the continuation line handling because usually REM lines are not continuable

void lex_RemoveComments(pLexObject pLex
  ){

[Contents]

2.3.1.5. lex_NextLexeme()

Use this function during iteration to get the next lexeme from the list of lexemes.

void lex_NextLexeme(pLexObject pLex
  ){

[Contents]

2.3.1.6. lex_SavePosition()

Use this function to save the current position of the iteration. This is neccessary during syntactical analysis to return to a certain position when syntactical analysis fails and the program has to go back and try a different command syntax.

void lex_SavePosition(pLexObject pLex,
                      pLexeme *ppPosition
  ){

The second argument is a pLexeme * type variable that holds the position and should be passed as argument to the function lex_RestorePosition().

[Contents]

2.3.1.7. lex_RestorePosition()

Use this function to restore the lexeme position that was saved calling the function lex_SavePosition()

void lex_RestorePosition(pLexObject pLex,
                         pLexeme *ppPosition
  ){

[Contents]

2.3.1.8. lex_StartIteration()

You should call this function when the list of lexemes was built up before starting the iteration of the syntax analyzer. This function sets the iteration pointer to the first lexeme.

void lex_StartIteration(pLexObject pLex
  ){

[Contents]

2.3.1.9. lex_EOF()

Call this function to check if the iteration has reached the last lexeme.

int lex_EOF(pLexObject pLex
  ){

[Contents]

2.3.1.10. lex_Type()

During lexeme iteration this function can be used to retrieve the typeof the current lexeme. The type of a lexeme can be:

int lex_Type(pLexObject pLex
  ){

[Contents]

2.3.1.11. lex_Double()

When the type of the current lexeme is LEX_T_DOUBLE during the lexeme iteration this function should be used to retrieve the actual value of the current lexeme.

double lex_Double(pLexObject pLex

[Contents]

2.3.1.12. lex_String()

When the type of the current lexeme is LEX_T_STRING during the lexeme iteration this function should be used to retrieve the actual value of the current lexeme.
char *lex_String(pLexObject pLex
  ){

[Contents]

2.3.1.13. lex_StrLen()

When the type of the current lexeme is LEX_T_STRING during the lexeme iteration this function should be used to retrieve the length of the current lexeme. This is more accurate than calling strlen on the actual string because the string itself may contain zero characters.
long lex_StrLen(pLexObject pLex
  ){

[Contents]

2.3.1.14. lex_Long()

When the type of the current lexeme is LEX_T_LONG during the lexeme iteration this function should be used to retrieve the actual value of the current lexeme.

long lex_Long(pLexObject pLex
  ){

[Contents]

2.3.1.15. lex_LineNumber()

This function returns the line number that the actual lexeme is in the source file. This function is needed to print out syntax and lexical error messages.

See also lex_FileName().

long lex_LineNumber(pLexObject pLex
  ){

[Contents]

2.3.1.16. lex_FileName()

This function returns a pointer to a constant string which is the file name that the lexeme was read from. Use this function to print out error messages when syntax or lexical error occures.

See also lex_LineNumber().

char *lex_FileName(pLexObject pLex
  ){

[Contents]

2.3.1.17. lex_XXX()

These access functions are implemented as macros and are put into <lexer.h> by the program headerer.pl

The macros access Int, Symbol, Float etc values of the current lexeme. However these are strored in a location which is named a bit different. For example the string of a symbol is stored in the string field of the lexeme. To be readable and to be compatible with future versions use these macros to access lexeme values when lexeme has any of these types.

/*
TO_HEADER:
#define lex_Int(x) lex_Long(x)
#define lex_Symbol(x) lex_String(x)
#define lex_Float(x) lex_Double(x)
#define lex_Char(x) lex_Long(x)
#define lex_Token(x) lex_Long(x)
#define lex_Code(x) lex_Long(x)
*/
/*

[Contents]

2.3.1.18. lex_Finish()

Call this functionto release all memory allocated by the lexical analyzer.

void lex_Finish(pLexObject pLex
  ){

[Contents]

2.3.1.19. lex_DumpLexemes()

Us this function for debugging. This function dumps the list of lexemes to the file psDump.

void lex_DumpLexemes(pLexObject pLex,
                     FILE *psDump
  ){

[Contents]

2.3.1.20. lex_ReadInput()

Call this function after proper initialization to read the input file. This function performs the laxical analysis and builds up an internal linked list that contains the lexemes.

int lex_ReadInput(pLexObject pLex
  ){

[Contents]

2.3.1.21. lex_InitStructure()

You may but need not call this function to initialize a LexObject. You may also call this function to use the settings of the function and set some variables to different values after the function returns.

void lex_InitStructure(pLexObject pLex
  ){

[Contents]

2.4. Syntax Analyzer

The syntax analyzer is a module that reads the token stream delivered by the lexer module and builds a memory data structure containing the syntactically analyzed and built program. The syntax analyzer is contained in the source file `expression.c' The name of this module come from the fact that the most important and most complex task of syntax analysis is the analysis of the expressions.

For the syntax analyzer the program is a series of commands. A command is a series of symbols. There is nothing like command blocks, or one command embedding another command. Therefore the syntax definition is quite simple and yet still powerful enough to define a BASIC like language.

Because syntax analysis is quite a complex task and the syntax analyzer built for ScriptBasic is quite a complex one I recommend that you first read the tutorial from the ScriptBasic web site that talks about the syntax analysis. This is a series of slides together with real audio voice explaining the structure of the syntax analyzer of ScriptBasic.

The syntax analyzer should be configured using a structure containing the configuration parameters and the “global variables” for the syntactical analyzer. This structure contains the pointer to the array containing the syntax definition. Each element of the array defines command syntax. Command syntax is the list of the symbols that construct the command. When the syntactical analyzer tries to analyze a line it tries the array elements until it finds one matching the line. When checking a line against a syntax definition the syntactical analyzer takes the lexical elements on the line and checks that they match the next symbol in the syntax definition. A symbol can be as simple as a reserved word, like if else, endif. Such a syntax element is matched by the specific keyword. On the other hand a symbol on the syntax definition can be as complex as an expression, which is matched by a whole expression.

The syntax analyzer has some built in assumption about the language, but the actual syntax is defined in tables. This way it is possible to analyze different languages using different tables in the same program even in the same process in separate threads.

When the syntax analyzer reads the syntax definition of a line and matches the tokens from the lexer against the syntax element it may do several things:

The first is the case when the syntax element is a constant symbol. For example it is a command keyword. In this case there is nothing to do with the keyword except that the syntax analyzer has to recognize that this is the statement identified by the keyword. The actual code will be generated later when non-constant syntactical elements are found.

When the syntax analyzer sees that the next syntax element is some variable, non-constant syntax element it matches the coming tokens and creates the nodes that hold the actual value for the tokens. For example when the syntax element is string the syntax analyzer checks that the coming token is a string and creates a node that holds the string. The most important example is the syntax element expression. In this case the syntax analyzer checks that the coming tokens form an expression and not only "consumes" these tokens, but creates several nodes that hold the structure of the expression.

We can distinguish between constant and variable symbolic definition elements.

There are some special symbols that are always matched whenever they are checked by the syntax analyzer. They do not consume any lexical element from the line, and generate values in the memory structure that the analyzer builds.

The symbolic definition elements are:

Note that you can find other syntax definition elements in the file syntax.def. However these are converted to a character value by the Perl script tool syntaxer.pl These pseudo syntax definition elements are:

[Contents]

2.4.1. Name Space

The scriba syntax analyzer implements a simple name space handling. A variable, label or function name always belongs to a name space. The default name space is main. When the syntax analyzer processes a non-absolute symbol it converts the name to contain the name space. A variable named var in the name space main has the name main::var. The syntax analyzer automatically converts the name to contain the name space, therefore main::var and var are equivalent when used in the name space main.

When a variable contains double colon it is treated as an absolute name, and no name space is prepended to it. If you are in the name space module and use the variable name main::var it will NOT be converted to module::main::var. The reason is that it already contains the characters :: and therefore scriba assumes that it already contains the name space.

If you are in the name space module and want to refer to the variable module::main::var you can use the names module::main::var or ::main::var. The first format contains all nested name spaces to the variable. The second version tells the syntax analyzer that the variable is relative, altough it contains double colon. This is because it starts with double colons.

If you are in name space module::submodul and want to refer to the variable module::var you can either write module::var or _::var. The first format contains all nested name spaces to the variable. The second version tells the syntax analyzer that the variable is relative and the base module is the embedding module of the current one.

If you are familiar with the UNIX or DOS/Windows directory notation you can find similarities of file names and name space in scriba. In file names double dot means the parent directory. In scriba underscore character means the parent name space. You can use the _ character not only in front of a name, but also within :: characters. For example

Main::module::var
Main::module::submodule::_::var
Main::_::Main::module::var

are equivalent.

Name spaces help to separate variables and to develop scripts cooperatively, but does not prohibit one name space to access variables or symbols of other name spaces.

[Contents]

2.4.2. Expression

The formal description of an expression syntax is:

 tag ::= UNOP tag
         NUMBER
         STRING
         '(' expression ')'
         VARIABLE { '[' expression_list ']' }
         FUNC '(' expression_list ')'
         .

expression_list ::= expression [ ',' expression_list ] . expression_i(1) ::= tag . expression_i(i) := expression_i(i-1) [ OP(i) expression_i(i) ] . expression ::= expression_i(MAX_PREC) .

where

The syntax analyzer is written to match an expression whenever an expression syntax definition element is to be matched according to these rules. The list of built-in function, unary operators and binary operators are defined in the module "global" variables, BuiltInFunctions, Unaries, and Binaries.

[Contents]

2.4.3. Functions implemented in this module

The following subsections list the functions that are implemented in this module. The source text of this documentation was extracted from the source Documentation embedded in the C source as comment. Treat these subsections more as reference documentation and less tutorial like.

The functions in this file compile a ScriptBasic expression into internal form. The functions are quite general, and do NOT depend on the actual operators that are implemented in the actual version.

This means that you can define extra operators as well as extra built-in functions easily adding entries into tables and need not modify the compiling code.

[Contents]

2.4.3.1. What is an expression in ScriptBasic

Altough the syntax defintion in script basic is table driven and can easily be modified expressions are not. The syntax of an expression is somewhat fix. Here we formally define what the program thinks to be an expression. This restriction should not cause problem in the usage of this module because this is the usual syntax of an expression. Any altering to this would result in an expression syntax which is unusual, and therefore difficult to use for the common users. The operators and functions along with therir precedence values are defined in tables anyway, so you have flexibility.

The formal description of an expression syntax:

 tag ::= UNOP tag
         NUMBER
         STRING
         '(' expression ')'
         VARIABLE { '[' expression_list ']' }
         VARIABLE '{' expression_list '}'
         FUNC '(' expression_list ')'
         .

expression_list ::= expression [ ',' expression_list ] .

expression_i(1) ::= tag .

expression_i(i) := expression_i(i-1) [ OP(i) expression_i(i) ] .

expression ::= expression_i(MAX_PREC) .

left_value ::= variable { '[' expression_list ']' } variable '{' expression_list '}' .

[Contents]

2.4.3.2. ex_DumpVariables()

This function dumps the variables stored in the symbol table to the file pointed by fp

void ex_DumpVariables(SymbolTable q,
                      FILE *fp
  ){

Note that this function is a debug function.

[Contents]

2.4.3.3. expression_PushNameSpace()

When a module name instruction is encountered the name space is modified. However the old name space should be reset when an end module statement is reached. As the modules can be nested into each other the name spaces are stored in a name space stack during syntax analysis.

This function pushes the current name space onto the stack. After calling this function the caller can put the new string into the pEx->CurrentNameSpace variable and later calling ex_PopNameSpace() can be called to retrive the saved name space.

int expression_PushNameSpace(peXobject pEx
  ){

[Contents]

2.4.3.4. ex_CheckUndefinedLabels()

This function traverses the label symbol table and reports all undefined labels as error. Undefined labels reference the node with node-number zero. Jumping on a label like that caused the program to stop instead of compile time error in previous versions.

void ex_CheckUndefinedLabels(peXobject pEx
  ){

[Contents]

2.4.3.5. ex_CleanNameSpaceStack()

This function cleans the name space stack. This cleaning does not need to be done during syntax analysis. It is needed after the analysis has been done to detect unclosed modules.

Note that the main:: module is implicit and can not and should not be closed unless it was explicitly opened.

The function calls the report function if the name space is not empty when the function is called.

void ex_CleanNameSpaceStack(peXobject pEx
  ){

[Contents]

2.4.3.6. expression_PopNameSpace()

When a module name instruction is encountered the name space is modified. However the old name space should be reset when an end module statement is reached. As the modules can be nested into each other the name spaces are stored in a name space stack during syntax analysis.

This function pops the name space from the name space stack and copies the value to the pEx->CurrentNameSpace variable. This should be executed when a name space is closed and we want to return to the embedding name space.

int expression_PopNameSpace(peXobject pEx
  ){

[Contents]

2.4.3.7. ex_PushWaitingLabel()

This function is used to define a label.

int ex_PushWaitingLabel(peXobject pEx,
                         pSymbolLABEL pLbl
  ){

When a label is defined the eNode_l that the label is going to belong still does not exists, and therefore the NodeId of that eNode_l is not known. This function together with ex_PopWaitingLabel() maintains a stack that can store labels which are currently defined and still need a line to be assigned to them. These labels all point to the same line. Altough it is very rare that many labels point to the same line, it is possible. The number of labels that can point the same line is defined by the constant MAX_SAME_LABELS defined in expression.c

To make it clear see the following BASIC code:

this_is_a_label: REM this is a comment PRINT "hello word!!"

The label is defined on the first line of the example. However the label belongs to the third line containing the statement PRINT. When the label is processed the compiler does not know the node number of the code segment which is generated from the third line. Therefore this function maintains a label-stack to store all labels that need a line. Whenever a line is compiled so that a label can be assigned to that very line the stack is emptied and all labels waiting on the stack are assigned to the line just built up. (Or the line is assigned to the labels if you like the sentence the other way around.)

Note that not only labels given by a label defining statement are pushed on this stack, but also labels generated by commands like 'while/wend' of 'if/else/endif'.

[Contents]

2.4.3.8. ex_PopWaitingLabel()

This function is used to get a label out of the waiting-label-stack.

pSymbolLABEL ex_PopWaitingLabel(peXobject pEx
  ){

To get some description of waiting labels see the description of the function ex_PushWaitingLabel().

[Contents]

2.4.3.9. _ex_PushLabel()

This function is used to push an unnamed label on the compile time stack. For more detailed defintion of the unnamed labels and this stack see the documentation of the function ex_PopLabel().

int _ex_PushLabel(peXobject pEx,
                  pSymbolLABEL pLbl,
                  long Type,
                  void *pMemorySegment
  ){

The argument Type is used to define the type of the unnamed label. This is usually defined in the table created by the program syntaxer.pl

=bold Do NOT get confused! This stack is NOT the same as the waiting label stack. That is usually for named labels. =nobold

However the non-named labels are also pushed on that stack before they get value.

[Contents]

2.4.3.10. _ex_PopLabel()

This function is used to pop an unnamed label off the compile stack.

When a construct, like IF/ELSE/ENDIF or REPEAT/UNTIL or WHILE/WEND is created it is defined using compile time label stack.

For example analyzing the instruction WHILE pushes a "go forward" value on the compile time label stack. When the instruction WEND is analyzed it pops off the value and stores NodeId for the label. The label itself is not present in the global label symbol table, because it is an unnamed label and is referenced during compile time by the pointer to the label structure.

The value of the AcceptedType ensures that a WEND for example do not matches an IF.

pSymbolLABEL _ex_PopLabel(peXobject pEx,
                          long *pAcceptedType
  ){

The array pAcceptedType is an array of long values that have MAX_GO_CONSTANTS values. This is usually points to a static table element which is generated by the program syntaxer.pl.

=bold Do NOT get confused! This stack is NOT the same as the waiting label stack. That is for named labels. =nobold

[Contents]

2.4.3.11. _ex_CleanLabelStack()

This function is used to clean the unnamed label stack whenever a locality is left. This helps to detect when an instruction like FOR or WHILE is not closed within a function.
void _ex_CleanLabelStack(peXobject pEx
  ){

[Contents]

2.4.3.12. Some NOTE on SymbolXXX functions

The functions named SymbolXXX like SymbolLABEL, or SymbolUF do NOT store the names of the symbols. They are named SymbolXXX because they are natural extensions of the symbol table system. In other compilers the functionality to retrieve the arguments of a symbol is part of the symbol table handling routines.

In script basic the symbol table handling routines were developed to be general purpose. Therefore all the arguments the symbol table functions bind toa symbol is a void * pointer. This pointer points to a struct that holds the arguments of the symbols, and the functions SymbolXXX allocate the storage for the arguments.

This way it is possible to allocate arguments for non-existing symbols, as it is done for labels. Script basic uses non-named labels to arrange the "jump" instructions for IF/ELSE/ENDIF constructs. (And for some other constructs as well.) The label and jump constructs look like:

IF expression Then

ELSE label1:

END IF label2:

The labels label1 and label2 do not have names in the system, not even autogenerated names. They are referenced via pointers and their value (the NodeId of the instruction) get into the SymbolLABEL structure and later int o the cNODE during build.

[Contents]

2.4.3.13. _new_SymbolLABEL()

This function should be used to create a new label. The label can be named or unnamed. Note that this structure does NOT contain the name of the label.

pSymbolLABEL _new_SymbolLABEL(peXobject pEx
  ){

Also note that all labels are global in a basic program and are subject to name space decoration. However the same named label can not be used in two different functions in the same name space.

A label has a serial value, which is not actually used and a number of the node that it points to.

See the comments on ex_symbols().

[Contents]

2.4.3.14. _new_SymbolVAR()

This function should be used to create a new variable during compile time. A variable is nothing else than a serial number. This serial number starts from 1.

pSymbolVAR _new_SymbolVAR(peXobject pEx,
                          int iLocal
  ){

The second argument should be true for local variables. The counting of local variables are reset whenever the program enters a new locality. Localities can not be nested.

Also note that local variables are allocated in a different segment because they are deallocated whenever the syntax analyzer leaves a locality.

[Contents]

2.4.3.15. _new_SymbolUF()

This function should be used to create a new user defined function symbol.
pSymbolUF _new_SymbolUF(peXobject pEx
  ){

A user function is defined by its serial number (serial number is actually not used in the current sytsem) and by the node number where the function actually starts.

The number of arguments and the number of local variables are defined in the generated command and not in the symbol table. This way these numbers are available as they should be during run time.

[Contents]

2.4.3.16. _new_eNODE()

This function should be used to create a new eNODE.

peNODE _new_eNODE(peXobject pEx
  ){

Each eNODE and eNODE_l structure has a serial number. The eNODEs are referencing each other using pointers. However after build these pointers become integer numbers that refer to the ordinal number of the node. Nodes are stored in a single memory block after they are packed during build.

An eNODE is a structure that stores a unit of compiled code. For example an addition in an expression is stored in an eNODE containing the code for the addition operator and containing pointers to the operands.

[Contents]

2.4.3.17. _new_eNODE_l()

This function should be used to create a new eNODE list. This is nothing else than a simple structure having two pointers. One pointer points to an eNODE while the other points to the next eNODE_l struct or to NULL if the current eNODE_l is the last of a list.

peNODE_l _new_eNODE_l(peXobject pEx,
                      char *pszFileName,
                      long lLineNumber
  ){

Note that eNODE and eNODE_l are converted to the same type of structure during build after the syntactical analysis is done.

[Contents]

2.4.3.18. ex_free()

This function releases all memory that was allocated during syntactical analysis.

void ex_free(peXobject pEx
  ){

[Contents]

2.4.3.19. ex_init()

This function should be called before starting syntactical analysis. This function

int ex_init(peXobject pEx
  ){

[Contents]

2.4.3.20. ex_CleanNamePath()

This function created a normalized name space name from a non normalized. This is a simple string operation.

Think of name space as directories and variables as files. A simple variable name is in the current name space. If there is a 'path' before the variable or function name the path has to be used. This path can either be relative or absolute.

File system:

../ is used to denote the parent directory in file systems.

Name space:

_:: is used to denote the parent name space.

File system:

mydir/../yourdir is the same as yourdir

Name space:

myns::_::yourns is the same as yourns

This function removes the unneccesary downs and ups from the name space and creates the result in the same buffer as the original. This can always be done as the result is always shorter. (Well, not longer.)

void ex_CleanNamePath(char *s
  ){

[Contents]

2.4.3.21. ex_ConvertName()

Use this function to convert a relative name to absolute containing name space.

This function checks if the variable or function name is relative or absolute. If the name is relative it creates the absolute name using the current name space as a base.

The result is always put into the Buffer.

A name is relative if it does NOT contain :: at all (implicit relative), if it starts with :: or is it starts with _:: (explicit relative).

int ex_ConvertName(char *s,          /* name to convert            */
                   char *Buffer,     /* buffer to store the result */
                   size_t cbBuffer,  /* size of the buffer         */
                   peXobject pEx     /* current expression object  */
  ){

The error value is EX_ERROR_SUCCESS (zero) menaing succesful conversion or EX_ERROR_TOO_LONG_VARIABLE meaning that the variable is too long for the buffer.

Note that the buffer is allocated in ex_init() according to the size value given in the class variable cbBuffer, which should be set by the main function calling syntax analysis.

[Contents]

2.4.3.22. ex_IsBFun()

This function checks if the current lexeme is a built-in function and returns pointer to the function in the table BuiltInFunctions or returns NULL if the symbol is not a built-in function.

pBFun ex_IsBFun(peXobject pEx
  ){

[Contents]

2.4.3.23. ex_IsUnop()

This function checks if the current lexeme is an unary operator and returns the op code or zero if the lexem is not an unary operator.

unsigned long ex_IsUnop(peXobject pEx
  ){

[Contents]

2.4.3.24. ex_IsBinop()

This function checks if the current lexeme is a binary operator of the given precedence and returns the op code or zero.

unsigned long ex_IsBinop(peXobject pEx,
               unsigned long precedence
  ){

[Contents]

2.4.3.25. ex_LeftValueList()

This function works up a leftvalue_list pseudo terminal and creates the nodes for it.

peNODE_l ex_LeftValueList(peXobject pEx
  ){

[Contents]

2.4.3.26. ex_ExpressionList()

This function works up an expression_list pseudo terminal and creates the nodes for it.
peNODE_l ex_ExpressionList(peXobject pEx
  ){

[Contents]

2.4.3.27. ex_Local()

This function work up a local pseudo terminal. This does not create any node.

int ex_Local(peXobject pEx
  ){
The return value is 0 if no error happens.

1 means sytax error (the coming token is not a symbol)

2 means that there is no local environment (aka. the local var is not inside a function)

[Contents]

2.4.3.28. ex_PredeclareGlobalLongConst()

This function is used to declare the global constants that are given in the syntax defintinon, and should be defined before the program is started to be analized.

int ex_PredeclareGlobalLongConst(peXobject pEx,
                                 char *pszConstName,
                                 long lConstValue
  ){

[Contents]

2.4.3.29. ex_Command_r()

This function finds the matching sytax line for the actual line in a loop. It starts with the first syntax definition and goes on until there are no more syntax defintions, a fatal error has happened or the actual line is matched.

void ex_Command_r(peXobject pEx,
                  peNODE *Result,
                  int *piFailure
  ){
pEx is the execution object.

Result is the resulting node.

piFailure is the error code.

[Contents]

2.4.3.30. ex_Command_l()

This function goes over the source lines and performs the syntax analysis. This function calls the function ex_Command_r(). When that function returns it allocated the list nodes that chain up the individual lines. It also defines the labels that are waiting to be defined.

int ex_Command_l(peXobject pEx,
                  peNODE_l *Result
  ){
When all the lines are done this function cleans the name space stack, check for undefined labels that remained undefined still the end of the source file.

[Contents]

2.4.3.31. ex_IsCommandCALL()

Because the syntax of a call statement is very special here is a special function to analyze the CALL statement.

A call statement is a keyword CALL followed by a function call.

If the function or sub is already defined then the keyword CALL can be missing.

When the function or sub is called this way and not inseide an expression the enclosing parentheses can be missing.

peNODE ex_IsCommandCALL(peXobject pEx,
                        pLineSyntax p,
                        int *piFailure
  ){

To get some description of waiting labels see the description of the function ex_PushWaitingLabel().

[Contents]

2.4.3.32. ex_IsCommandOPEN()

The open statement is a simple one. The only problem is that the last parameter defining the length of a record is optional. This can only be handled using a separate function

peNODE ex_IsCommandOPEN(peXobject pEx,
                        pLineSyntax p,
                        int *piFailure
  ){

'open' expression 'for' absolute_symbol 'as' expression 'len' '=' expression nl

[Contents]

2.4.3.33. ex_IsCommandSLIF()

If syntax analysis gets to calling this function the command is surely not single line if, because the command SLIF is recognised by IsCommandIF.

The syntax of the command IF is presented in the syntax table before SLIF and therefore if the syntax analyser gets here it can not be SLIF.

The original function IsCommandThis could also do failing automatically, but it is simpler just to fail after the function call, so this function is just a bit of speedup.

peNODE ex_IsCommandSLIF(peXobject pEx,
                        pLineSyntax p,
                        int *piFailure
  ){

[Contents]

2.4.3.34. ex_IsCommandIF()

The statement IF is quite simple. However there is another command that has almost the same syntax as the IF statement. This is the SLIF, single line IF.

The difference between the command IF and SLIF is that SLIF does not have the new line character after the keyword THEN.

peNODE ex_IsCommandIF(peXobject pEx,
                        pLineSyntax p,
                        int *piFailure
  ){

IF/IF: 'if' * expression 'then' go_forward(IF) nl SLIF/SLIF: 'slif' * expression 'then'

[Contents]

2.4.3.35. ex_IsCommandLET()

peNODE ex_IsCommandLET(peXobject pEx,
                       pLineSyntax p,
                       int *piFailure
  ){

[Contents]

2.5. Builder

The module is implemented in the file `builder.c'.

The rule of the builder is to compile the code created by the syntax analyzer into a continuous memory area. This compilation phase has two advantages. First of all it results a compact code which can easily saved into external file and can also be loaded making recompilation unnecessary before each execution. The second advantage is that the resulting code is smaller and saves memory.

When the syntax analyzer starts its work the final size of the code is not known. Therefore the syntax analyzer starts to build up a memory structure using pointers, linked lists allocating memory step by step as the code grows. Each command, expression element is stored in an internal structure, which is called a node. For example a node containing the operation "plus" contains the node type, which says it is an operator "plus" and contains two pointers to the two operands. The operands are also nodes. If the operand is a number the node contains the value of the number and the node type telling that it is a long or double number. If the operand is a variable the node type tells that the node is a variable and the node contains the serial number of the variable. If the operand needs further evaluation then the node is probably an operation having arguments pointed by pointers.

The structure that the builder creates is same as that of the syntax analyzer but it is allocated in a single memory chunk and instead of pointers it uses indices to refer a node from another. These indices are numbered from 1 and not from zero. This is because the index zero is used for NULL pointer if you know what I mean.

[Contents]

2.5.1. Node Structure

Have a look at the C definition of a node:

typedef struct _cNODE {
  long OpCode; // the code of operation
  union {
    struct {// when the node is a command
      unsigned long next;
      union {
        unsigned long pNode;// node id of the node
        long lLongValue;
        double dDoubleValue;
        unsigned long szStringValue;
        }Argument;
      }CommandArgument;
    struct {//when the node is an operation
      unsigned long Argument;//node id of the node list head
      }Arguments;
    union {// when the node is a constant
      double dValue;        
      long   lValue;        
      unsigned long sValue; // serial value of the string from the string table       
      }Constant;
    struct {// when the node is a variable
      unsigned long Serial;// the serial number of the variable
      }Variable;
    struct {// when node is a user functions
      unsigned long NodeId; // the entry point of the function
      unsigned long Argument; // node id of the node list head
      }UserFunction;
    struct {// when the node is a node list head
      unsigned long actualm; //car
      unsigned long rest;    //cdr
      }NodeList;
    }Parameter;
  } cNODE,*pcNODE;

The field OpCode is the same as the code used in the lexer or the syntax analyzer. In case of an IF statement it is CMD_IF. This field can, should and is used to identify which part of the union Parameter is to be used.

The individual lines of the BASIC program that create code are chained into a list. Each line has a head node. The OpCode of the head nodes is eNTYPE_LST. This type of node contains NodeList structure. The field NodeList.actualm contains the index of the first node of the actual line and the field NodeList.rest contains the index of the next header node.

This type of node is used to gather expression lists into a linked list.

Note that usually not the first node in the byte-code is the first head node, where the code is to be started. The nodes generated from a line are created before the head node is allocated in the syntax analyzer and the head node thus gets a larger serial number. The builder uses the serial numbers counted by the syntax analyzer and does not rearrange the nodes.

The command node that the field NodeList.actualm "points" contains the opcode of the command. For example if the actual command is IF then the OpCode is CMD_IF.

In case of command nodes the Parameter is CommandArgument. If the command has only a single argument the field next is zero. Otherwise this field contains the node index of the node holding the next argument.

The Parameter.CommandArgument.Argument union contains the actual argument of the command. There is no indication in the data structure what type the argument is. The command has to know what kind of arguments it gets, and should not interpret the union different.

The field pNode is the node index of the parameter. This is the case for example when the parameter is an expression or a label to jump to.

The fields lLongValue, dDoubleValue and szStringValue contain the constant values in case the argument is a constant. However this is actually not the string that is stored in the field szStringValue but the index to the string table where the string is started. (Yes, here is some inconsistency in naming.)

Strings are stored in a string table where each string is stored one after the other. Each string is terminated with a zero character and each string is preceded by a long value that indicates the length of the string. The zero character termination eases the use of the string constants when they have to be passed to the operating system avoiding the need to copy the strings in some cases.

The field Parameter.CommandArgument.next is zero in case there are no more arguments of the command, or the index of the node containing the next argument. The OpCode field of the following arguments is eNTYPE_CRG.

When the node is part of an expression and represents an operation or the call of a built-in function then the Arguments structure of the Parameter union is to be used. This simply contains Argument that "points" to a list of "list" nodes that list the arguments in a list. In this case the OpCode is the code of the built-in function or operation.

When the node represents a string or a numeric constant the Constant union field of the union Parameter should be used. This stores the constant value similar as the field CommandArgument except that it can only be long, double or a string. In case of constant node the OpCode is eNTYPE_DBL for a double, eNTYPE_LNG for a long and eNTYPE_STR for a string.

When the node represents a variable the field Variable has to be used. In this case the field Serial contains the serial number of the variable. To distinguish between local and global variables the OpCode is either eNTYPE_LVR for local variables or eNTYPE_GVR for global variables.

When the node is a user defined function call the field UserFunction is used. Note that this is not the node that is generated from the line sub/function myfunc but rather when the function or subroutine is called. The OpCode is eNTYPE_FUN.

The field NodeId is the index of the node where the function or subroutine starts. The field Argument is the index of the list node that starts the list of the argument expressions.

[Contents]

2.5.2. Binary File Format

The built code is usually saved to a cache file and this file is used later to load the already compiled code into memory for subsequent execution. The format of this file can be read from the function build_SaveCode in file `builder.c'. This function is quite linear it just saves several structures and is well commented so you should not have problem to understand it. However here I also give some description on the format of the binary file.

The binary file may or may not start with a textual line. This line is the usual UNIX #! /usr/bin/scriba string telling the operating system how to execute the text file. Altough we are talking now about a binary file, from the operating system point of view this is just a file, like a ScriptBasic source file, a Perl script or a bash script. The operating system starts to read the file and if the start of the file is something like

#! /usr/bin/scriba\n

with a new-line character at the end then it can be executed from the command line if some other permission related constraints are met.

When ScriptBasic saves the binary format file it uses the same executable path that was given in the source file. If the source file starts with the line #! /usr/bin/mypath/scriba and the basic progam `myprog.bas' was started using the command line

/usr/bin/scriba -o myprog.bbf myprog.bas

then the file `myprog.bbf' will start with the line #! /usr/bin/mypath/scriba.

The user's guide lists a small BASIC program that reads and writes the binary file and alters this line.

Having this line on the first place in the binary format BASIC file makes it possible to deliver programs in compiled format. For example you may develop a CGI application and deliver it as compiled format to protect your program from the customer. You can convert your source issuing the command line

/usr/bin/scriba -o outputdirectory/myprog.bas myprog.bas

and deliver the binary `myprog.bas' to the customer. ScriptBasic does not care the file extension and does not expect a file with the extension .bas to be source BASIC. It automatically recognizes binary format BASIC programs and thus you need no alter even the URLs that refer to CGI BASIC programs.

The next byte in the file following this optional opening line is the size of a long on the machine the code was created. The binary code is not necessarily portable from one machine to another. It depends on pointer and long size as well as byte ordering. We experienced Windows NT and Linux to create the same binary file but this is not a must, may change.

The size of a long is stored in a single character as sizeof(long)+0x30 so the ASCII character is either '4' or '8' on 32 and 64 bit machines.

This byte is followed by the version information. This is a struct:

  unsigned long MagicCode;
  unsigned long VersionHigh, VersionLow;
  unsigned long MyVersionHigh,MyVersionLow;
  unsigned long Build;
  unsigned long Date;
  unsigned char Variation[9];

The MagicCode is 0x1A534142. On DOS based system this is the characters 'BAS' and ^Z which means end of text file. Thus if you issue the command

C:\> type mybinaryprogram.bbf

you will get

4BAS

without scrambling your screen. If you use UNIX system then be clever enough not to cat a binary program to the terminal.

The values VersionHigh and VersionLow are the version number of ScriptBasic core code. This is currently 1 and 0. The fields MyVersionHigh and MyVersionLow are reserved for developers who develop a variation of ScriptBasic. The variation may alter some features and still is based on the same version of the core code. These two version fields are reserved here to distinguish between different variation versions based on the same core ScriptBasic code. To maintain these version numbers is essential for those who embed ScriptBasic into an application, especially if the different versions of the variations alter the binary file format which I doubt is really needed.

The field Build is the build of the core ScriptBasic code.

The Date is date when the file `builder.c' was compiled. The date is stored in a long in a tricky way that ensures that no two days result the same long number. In case you want to track how this is coded see the function build_MagicCode in file `builder.c'. This is really tricky.

The final field is Variation which is and should be an exactly 8 character long string and a zero character.

If you want to compile a different variation then alter the #define directives in the file `builder.c'

#define VERSION_HIGH 0x00000001
#define VERSION_LOW  0x00000000
#define MYVERSION_HIGH 0x00000000
#define MYVERSION_LOW  0x00000000
#define VARIATION "STANDARD"

To successfully load a binary format file to run in ScriptBasic the long size, the magic code, version information including the build and the variation string should match. Date may be different.

The following foru long numbers in the binary file define

This is followed by the nodes themselves and the stringtable.

This is the last point that has to exist in a binary format file of a BASIC program. The following bytes are optional and may not be present in the file.

The optional part contains the size of the function table defined on a long and the function table. After this the size of the global variable table is stored in a long and the global variable table.

The global variable and function symbol table are list of elements, each containing a long followed by the zero character terminated symbolic name. The long stores the serial number of the variable or the entry point of the function (the node index where the function starts).

These two tables are not used by ScriptBasic by itself, ScriptBasic does not need any symbolic information to execute a BASIC program. Programmers embedding ScriptBasic however demanded access global variables by name and the ability to execute individual functions from a BASIC program. If this last part is missing from a binary format BASIC program you will not be able to use in an application that uses these features.

[Contents]

2.5.3. Functions implemented in this module

The following subsections list the functions that are implemented in this module. The source text of this documentation was extracted from the source Documentation embedded in the C source as comment. Treat these subsections more as reference documentation and less tutorial like.

This module can and should be used to create the memory image for the executor module from the memory structure that was created by the module expression.

The memory structure created by expression is segmented, allocated in many separate memory chunks. When the module expression has been finished the size of the memory is known. This builder creates a single memory chunk containing all the program code.

Note that the function names all start with the prefix build_ in this module.

The first argument to each function is a pointer to a BuildObject structure that contains the "global" variables for the module. This technique is used to ensure multithread usage. There are no global variables which are really global within the process.

The functions in this module are:

=toc

[Contents]

2.5.3.1. The structure of the string table

The string table contains all string contansts that are used in the program. This includes the single and multi line strings as well as symbols. (note that even the variable name after the keyword next is ignored but stored in the string table).

The strings in the string table are stored one after the other zero character terminated. Older version of ScriptBasic v1.0b21 and before stored string constants zero character terminated. Because of this string constants containing zero character were truncated (note that \000 creates a zero character in a string constant in ScriptBasic).

The version v1.0b22 changed the way string constants are stored and the way string table contains the strings. Each string is stored with its length. The length is stored as a long on sizeof(long) bytes. This is followed by the string. Whenever the code refers to a string the byte offset of the first character of the string is stored in the built code. For example the very first string starts on the 4. byte on 32 bit machines.

Altough the string length and zero terminating characters are redundant information both are stored to avoid higher level mistakes causing problem.

[Contents]

2.5.3.2. build_AllocateStringTable()

This function allocates space for the string table. The size of the string table is already determined during syntax analysis. The determined size should be enough. In some cases when there are repeated string constants the calculated sizte is bigger than the real one. In that case the larger memory is allocated and used, but only the really used part is written to the cache file.

If the program does not use any string constants then a dummy string table of length one byte is allocated.

void build_AllocateStringTable(pBuildObject pBuild,
                          int *piFailure
  ){

The first argument is the usual pointer to the "class" structure. The second argument is the result value. It can have two values:

The string table is allocated using the function alloc_Alloc. The string table is pointed by the class variable StringTable. The size of the table is stored in cStringTable

[Contents]

2.5.3.3. build_StringIndex()

In the built code all the strings are references using the offset of the string from the string table (See build_AllocateStringTable()). This function calculates this value for the string.

This function is used repetitively during the code building. Whenever a string index is sought that is not in the string table yet the string is put into the table and the index is returned.

If there is not enough space in the string table the function calls the system function exit and stops the process. This is rude especially in a multithread application but it should not ever happen. If this happens then it is a serious internal error.

unsigned long build_StringIndex(pBuildObject pBuild,
                                char *s,
                                long sLen
  ){

[Contents]

2.5.3.4. build_Build_l()

This function converts an eNODE_l list to cNODE list in a loop. This function is called from build_Build() and from build_Build_r().

int build_Build_l(pBuildObject pBuild,
                  peNODE_l Result
  ){
The function returns the error code, or zero in case of success.

[Contents]

2.5.3.5. build_Build_r()

This function builds a single node. This actually means copiing the values from the data structure created by the module expression. The major difference is that the pointers of the original structure are converted to unsigned long. Whenever a pointer pointed to a eNODE the unsigned long will contain the NodeId of the node. This ID is the same for the eNODE and for the cNODE that is built from the eNODE.

int build_Build_r(pBuildObject pBuild,
                  peNODE Result
  ){

The node to be converted is passed by the pointer Result. The return value is the error code. It is zero (BU_ERRROR_SUCCESS) in case of success.

When the node pointed by Result references other nodes the function recursively calls itself to convert the referenced nodes.

[Contents]

2.5.3.6. build_Build()

This is the main entry function for this module. This function initializes the class variable pointed by pBuild and calls build_Build_l() to build up the command list.
int build_Build(pBuildObject pBuild
  ){

[Contents]

2.5.3.7. build_MagicCode()

This is a simple and magical calculation that converts any ascii date to a single unsigned long. This is used as a magic value in the binary format of the compiled basic code to help distinguish incompatible versions.

This function also fills in the sVersion static struct that contains the version info.

unsigned long build_MagicCode(pVersionInfo p
  ){

[Contents]

2.5.3.8. build_SaveCCode()

This function saves the binary code of the program into the file given by the name szFileName in C programming language format.

The saved file can be compiled using a C compiler on the platform it was saved. The generated C file is not portable.

void build_SaveCCode(pBuildObject pBuild,
                    char *szFileName
  ){

[Contents]

2.5.3.9. build_SaveCode()

This function saves the binary code of the program into the file given by the name szFileName.

This version is hard wired saving the code into an operating system file because it uses fopen, fclose and fwrite. Later versions may use other compatible functions passed as argument and thus allowing output redirection to other storage media (a database for example).

However I think that this code is quite simple and therefore it is easier to rewrite the whole function along with build_LoadCode() for other storage media than writing an interface function.

The saved binary code is NOT portable. It saves the internal values as memory image to the disk. It means that the size of the code depends on the actual size of long, char, int and other types. The byte ordering is also system dependant.

The saved binary code can only be loaded with the same version, and build of the program, therefore it is vital to distinguish each compilation of the program. To help the recognition of the different versions, the code starts with a version structure.

The very first byte of the code contains the size of the long on the target machine. If this is not correct then the code was created on a different processor and the code is incompatible.

The version info structure has the following fileds:

int build_SaveCode(pBuildObject pBuild,
                    char *szFileName
  ){
The function returns zero on success (BU_ERROR_SUCCESS) and BU_ERROR_FAIL if the code could not be saved.

[Contents]

2.5.3.10. build_LoadCode()

For detailed definition of the binary format see the code and the documentation of build_SaveCode()

In case the file is corrupt the function reports error.

void build_LoadCode(pBuildObject pBuild,
                   char *szFileName
  ){

[Contents]

2.5.3.11. build_IsFileBinaryFormat()

This function test a file reading its first few characters and decides if the file is binary format of a basic program or not.

int build_IsFileBinaryFormat(char *szFileName
  ){

[Contents]

2.5.3.12. build_pprint()

This is a debug function that prints the build code into a file.

This function is not finished and the major part of it is commented out using #if 0 construct.

void build_pprint(pBuildObject pBuild,
                  FILE *f
  ){

[Contents]

2.5.3.13. build_CreateFTable()

When the binary code of the BASIC program is saved to disk the symbol table of the user defined functions and the symbol table of global variables is also saved. This may be needed by some applications that embed ScriptBasic and want to call specific function or alter global variables of a given name from the embedding C code. To do this they need the serial number of the global variable or the entry point of the function. Therefore ScriptBasic v1.0b20 and later can save these two tables into the binary code.

The format of the tables is simple optimized for space and for simplicity of generation. They are stored first in a memory chunk and then written to disk just as a series of bytes.

The format is

long      serial number of variable or entry point of the function
zchar     zero character terminated symbol

This is easy to save and to load. Searching for it is a bit slow. Embedding applications usually have to search for the values only once, store the serial number/entry point value in their local variable and use the value.

The function CreateFTable converts the symbol table of user defined function collected by symbolic analysis into a single memory chunk.

The same way build_CreateVTable() converts the symbol table of global variables collected by symbolic analysis into a single memory chunk.

int build_CreateFTable(pBuildObject pBuild
  ){

[Contents]

2.5.3.14. build_CreateVTable()

When the binary code of the BASIC program is saved to disk the symbol table of the user defined functions and the symbol table of global variables is also saved. This may be needed by some applications that embed ScriptBasic and want to call specific function or alter global variables of a given name from the embedding C code. To do this they need the serial number of the global variable or the entry point of the function. Therefore ScriptBasic v1.0b20 and later can save these two tables into the binary code.

The format of the tables is simple optimized for space and for simplicity of generation. They are stored first in a memory chunk and then written to disk just as a series of bytes.

The format is

long      serial number of variable or entry point of the function
zchar     zero character terminated symbol

This is easy to save and to load. Searching for it is a bit slow. Embedding applications usually have to search for the values only once, store it in their local variable and use the value.

The function build_CreateFTable() converts the symbol table of user defined function collected by symbolic analysis into a single memory chunk.

The same way CreateVTable converts the symbol table of global variables collected by symbolic analysis into a single memory chunk.

int build_CreateVTable(pBuildObject pBuild
  ){

[Contents]

2.5.3.15. build_LookupFunctionByName()

long build_LookupFunctionByName(pBuildObject pBuild,
                          char *s
  ){

[Contents]

2.5.3.16. build_LookupVariableByName()

long build_LookupVariableByName(pBuildObject pBuild,
                          char *s
  ){

[Contents]

2.6. Executor

The executor kills the code. Oh, no! I was just kidding. It executes the code, which means something different. Starts with the first node and goes on.

The execution of the code starts calling the function execute_Execute implemented in the file `execute.c'

This function initializes the execution environment that was not initialized before calling the function execute_InitStructure

It allocates global variables, it fills command and instruction parameters and finalizer function pointers with NULL and starts the function execute_Execute_r

[Contents]

2.6.1. Command parameters

Each command type has a pointer that it can use for its own purpose. This is to avoid using global variables. The file commands for example use the pointer available for the command OPEN. To access this pointer the macro PARAMPTR is used defined in the file `command.c'

[Contents]

2.6.2. Instrunction parameters

Not only the commands have a pointer for their use, but there are pointers available for each instruction. To make it clear:

If there are three OPEN statements in a program they share a common command pointer, but each have its own instruction pointer.

The code fragments implementing the different commands are free to use their own or any other related command or instruction pointer.

[Contents]

2.6.3. Finalizer function

Finalizer function pointers are available for each command type. This way they are similar to command parameters. There can be many OPEN statements in a program they share a common finalizer pointer. Each finalizer pointer is initialized to NULL.

The code fragments may put a function entry address in the finalizer pointer. When the execution of a program is finished the executing function calls each function that has a non NULL pointer in the finalizer array.

The _r in the function name tells that this is a recursive function that may call itself when an expression evaluation performs a function or subroutine call.

[Contents]

2.6.4. Functions implemented in this module

The following subsections list the functions that are implemented in this module. The source text of this documentation was extracted from the source Documentation embedded in the C source as comment. Treat these subsections more as reference documentation and less tutorial like.

This module contain the functions that execute the code resuled by the builder.

[Contents]

2.6.4.1. execute_GetCommandByName()

The op-code of a command can easily be identified, because syntax.h contains symbolic constant for it. This function can be used by external modules to get this opcode based on the name of the function. The argument pszCommandName should be the name of the command, for example "ONERRORRESUMENEXT". The third argument is the hint for the function to help to find the value. It should always be the opcode of the command. The return value is the actual opcode of the command. For example:

i = execute_GetCommandByName(pEo,"ONERRORRESUMENEXT",CMD_ONERRORRESUMENEXT);

will return CMD_ONERRORRESUMENEXT.

Why is this function all about then?

The reason is that the external module may not be sure that the code CMD_ONERRORRESUMENEXT is the same when the external module is compiled and when it is loaded. External modules negotiate the interface version information with the calling interpreter, but the opcodes may silently changed from interpreter version to the next interpreter version and still supporting the same extension interface version.

When an external module needs to know the opcode of a command of the calling interpreter it first calls this function telling:

I<I need the code of the command ONERRORRESUMENEXT. I think that the code is CMD_ONERRORRESUMENEXT, but is it the real code?>

The argument lCodeHint is required only, because it speeds up search.

If there is no function found for the given name the returnvalue is zero.

long execute_GetCommandByName(pExecuteObject pEo,
                              char *pszCommandName,
                              long lCodeHint
  ){

[Contents]

2.6.4.2. execute_CopyCommandTable()

The command table is a huge table containing pointers to functions. For example the CMD_LET-th element of the table points to the function COMMAND_LET implementing the assignment command.

This table is usually treated as constant and is not moduified during run time. In case a module wants to reimplement a command it should alter this table. However the table is shared all concurrently running interpreter threads in a multi-thread variation of ScriptBasic.

To avoid altering the command table of an independent interpreter threadthe module wanting altering the command table should call this function. This function allocates memory for a new copy of the command table and copies the original constant value to this new place. After the copy is done the ExecuteObject will point to the copied command table and the extension is free to alter the table.

In case the function is called more than once for the same interpreter thread only the first time is effective. Later the function returns without creating superfluous copies of the command table.

int execute_CopyCommandTable(pExecuteObject pEo
  ){

[Contents]

2.6.4.3. execute_InitStructure()

int execute_InitStructure(pExecuteObject pEo,
                          pBuildObject pBo
  ){

[Contents]

2.6.4.4. execute_ReInitStructure()

This function should be used if a code is executed repeatedly. The first initialization call is execute_InitStructure() and consecutive executions should call this function.

int execute_ReInitStructure(pExecuteObject pEo,
                            pBuildObject pBo
  ){

[Contents]

2.6.4.5. execute_Execute_r()

This function executes a program fragment. The execution starts from the class variable ProgramCounter. This function is called from the execute_Execute() function which is the main entry point to the basic main program. This function is also called recursively from the function execute_Evaluate() when a user defined function is to be executed.

void execute_Execute_r(pExecuteObject pEo,
                       int *piErrorCode
  ){

[Contents]

2.6.4.6. execute_InitExecute()

void execute_InitExecute(pExecuteObject pEo,
                        int *piErrorCode
  ){

[Contents]

2.6.4.7. execute_FinishExecute()

void execute_FinishExecute(pExecuteObject pEo,
                           int *piErrorCode
  ){

[Contents]

2.6.4.8. execute_Execute()

This function was called from the basic main function. This function performs inititalization that is needed before each execution of the code and calls execute_Execute_r() to perform the execution.

Note that execute_Execute_r() is recursively calls itself.

This function is obsolete and is not used anymore. This is kept in the source for the shake of old third party variations that may depend on this function.

Use of this function in new applications is discouraged.

void execute_Execute(pExecuteObject pEo,
                     int *piErrorCode
  ){

[Contents]

2.6.4.9. execute_ExecuteFunction()

This function is used by the embedding layer (aka scriba_ functions) to execute a function. This function is not directly called by the execution of a ScriptBasic program. It may be used after the execution of the program by a special embeddign application that keeps the code and the global variables in memory and calls functions of the program.

The function takes pEo as the execution environment. StartNode should be the node where the sub or function is defined. cArgs should give the number of arguments. pArgs should point to the argument array. pResult will point to the result. If pResult is NULL the result is dropped. Otherwise the result is a mortal variable.

Note that this code does not check the number of arguments you provide. There can be more arguments passed to the SUB than it has declared, therefore you can initialize the local variables of the sub. (You should know that arguments are local variables in ScriptBasic just as any other non-argument local variable.)

The arguments should be normal immortal variables. They are passed to the SUB by reference and in case they are modified the old variable is going to be released.

piErrorCode returns the error code of the execution which is zero in case of no error.

void execute_ExecuteFunction(pExecuteObject pEo,
                             unsigned long StartNode,
                             long cArgs,
                             pFixSizeMemoryObject *pArgs,
                             pFixSizeMemoryObject *pResult,
                             int *piErrorCode
  ){

[Contents]

2.6.4.10. execute_Evaluate()

This function evaluates an expression. You should not get confused! This is not syntax analysis, caring operator precedences and grouping by nested parentheses. That has already been done during syntax analysis. This code performs the code that was generated from an expression.

The result is usually a mortal memory value which is the final result of the expression. However this piece of code assumes that the caller is careful enough to handle the result as read only, and sometimes the return value is not mortal. In this case the return value is a memory object that a variable points to. Whenever the caller needs this value to perform an operation that does not alter the value it is OK. Duplicating the structure to create a mortal would be waste of time and memory. On the other hand sometimes operations modify their operands assuming that they are mortal values. They should be careful.

Operators are actually created in the directory commands and they use the macros defined in command.h (created by headerer.pl from command.c). They help to avoid pitfalls.

The argument iArrayAccepted tells the function whether an array as a result is accepted or not. If a whole array is accepted as a result of the expression evaluation the array is returned. If the array is not an acceptable result, then the first element of the array is retuned in case the result is an array. If the result is NOT an array this parameter has no effect.

pFixSizeMemoryObject execute_Evaluate(pExecuteObject pEo,
                                      unsigned long lExpressionRootNode,
                                      pMortalList pMyMortal,
                                      int *piErrorCode,
                                      int iArrayAccepted
  ){

[Contents]

2.6.4.11. execute_LeftValue()

This function evaluate a left value. A left value is a special expression that value can be assigned, and therefore they usually stand on the left side of the assignment operator. That is the reason for the name.

When an expression is evaluates a pointer to a memory object is returned. Whenever a left value is evaluated a pointer to the variable is returned. If any code assignes value to the variable pointed by the return value of this function it should release the memory object that the left value points currently.

pFixSizeMemoryObject *execute_LeftValue(pExecuteObject pEo,
                                        unsigned long lExpressionRootNode,
                                        pMortalList pMyMortal,
                                        int *piErrorCode,
                                        int iArrayAccepted
  ){

[Contents]

2.6.4.12. execute_EvaluateArray()

This function should be used to evaluate an array access to get the actual value. This is called by execute_Evaluate().

An array is stored in the expression as an operator with many operands. The first operand is a local or global variable, the rest of the operators are the indices.

Accessing a variable holding scalar value with array indices automatically converts the variable to array. Accessing an array variable without indices gets the "first" element of the array.

pFixSizeMemoryObject execute_EvaluateArray(pExecuteObject pEo,
                                      unsigned long lExpressionRootNode,
                                      pMortalList pMyMortal,
                                      int *piErrorCode
  ){

[Contents]

2.6.4.13. execute_EvaluateSarray()

This function should be used to evaluate an array access to get the actual value. This is called by execute_Evaluate().

An array is stored in the expression as an operator with many operands. The first operand is a local or global variable, the rest of the operators are the indices.

Associative arrays are normal arrays, only the access mode is different. When accessing an array using the fom a{key@} then the access searches for the value key in the evenly indexed elements of the array and gives the next index element of the array. This if

a[0] = "kakukk"
a[1] = "birka"
a[2] = "kurta"
a[3] = "mamus"

then a{"kakukk"@} is "birka". a{"birka"@} is undef. a{"kurta"@} is "mamus".

pFixSizeMemoryObject execute_EvaluateSarray(pExecuteObject pEo,
                                      unsigned long lExpressionRootNode,
                                      pMortalList pMyMortal,
                                      int *piErrorCode
  ){

[Contents]

2.6.4.14. execute_LeftValueArray()

This function evaluates an array access left value. This function is also called by execute_EvaluateArray() and the result pointer is dereferenced.

pFixSizeMemoryObject *execute_LeftValueArray(pExecuteObject pEo,
                                             unsigned long lExpressionRootNode,
                                             pMortalList pMyMortal,
                                             int *piErrorCode
  ){

[Contents]

2.6.4.15. execute_LeftValueSarray()

This function evaluates an associative array access left value. This function is also called by execute_EvaluateSarray() and the result pointer is dereferenced.

pFixSizeMemoryObject *execute_LeftValueSarray(pExecuteObject pEo,
                                              unsigned long lExpressionRootNode,
                                              pMortalList pMyMortal,
                                              int *piErrorCode
  ){

[Contents]

2.6.4.16. execute_Convert2String()

This functionconverts a variable to string. When the variable is already a string then it returns the pointer to the variable. When the variable is long or double sprintf is used to convert the number to string.

When the conversion from number to string is done the result is always a newly allocated mortal. In other words this conversion routine is safe, not modifying the argument memory object.

pFixSizeMemoryObject execute_Convert2String(pExecuteObject pEo,
                                          pFixSizeMemoryObject pVar,
                                          pMortalList pMyMortal
  ){

[Contents]

2.6.4.17. execute_Convert2Long()

This function should be used to convert a variable to long. The conversion is usually done in place. However strings can not be converted into long in place, because they have different size. In such a case a new variable is created. If the mortal list pMyMortal is NULL then the new variable in not mortal. In such a case care should be taken to release the original variable.

Usually there is a mortal list and a new mortal variable is generated. In such a case the original value is also a mortal and is automatically released after the command executing the conversion is finished.

Note that strings are converted to long in two steps. The first step converts the string to double and then this value is converted to long in-place.

pFixSizeMemoryObject execute_Convert2Long(pExecuteObject pEo,
                                          pFixSizeMemoryObject pVar,
                                          pMortalList pMyMortal
  ){

[Contents]

2.6.4.18. execute_Convert2LongS()

This is the safe version of the conversion function execute_Convert2Long().

This function ALWAYS create a new variable and does NOT convert a double to long in place. This function is called by the extensions, because extensions tend to be more laisy regarding conversion and many converts arguments in place and thus introduce side effect.

To solve this problem we have introduced this function and have set the support table to point to this function.

pFixSizeMemoryObject execute_Convert2LongS(pExecuteObject pEo,
                                           pFixSizeMemoryObject pVar,
                                           pMortalList pMyMortal
  ){

[Contents]

2.6.4.19. execute_Convert2Double()

This function should be used to convert a variable to double. The conversion is usually done in place. However strings can not be converted into double in place, because they have different size. In such a case a new variable is created. If the mortal list is NULL then the new variable in not mortal. In such a case care should be taken to release the original variable.

Usually there is a mortal list and a new mortal variable is generated. In such a case the original value is also a mortal and is automatically released after the command executing the conversion is finished.

pFixSizeMemoryObject execute_Convert2Double(pExecuteObject pEo,
                                            pFixSizeMemoryObject pVar,
                                            pMortalList pMyMortal
  ){

[Contents]

2.6.4.20. execute_Convert2DoubleS()

This is the safe version of the conversion function execute_Convert2Double().

This function ALWAYS create a new variable and does NOT convert a long to double in place. This function is called by the extensions, because extensions tend to be more laisy regarding conversion and many converts arguments in place and thus introduce side effect.

To solve this problem we have introduced this function and have set the support table to point to this function.

pFixSizeMemoryObject execute_Convert2DoubleS(pExecuteObject pEo,
                                             pFixSizeMemoryObject pVar,
                                             pMortalList pMyMortal
  ){

[Contents]

2.6.4.21. execute_Convert2Numeric()

This function should be used to convert a variable to numeric type.

The conversion results a double or long variable. If the source variable was already a long or double the function does nothing but results the source variable.

undef is converted to long zero.

The function calls execute_Convert2Long and execute_Convert2Double thus all other parameters are treated according to that.

pFixSizeMemoryObject execute_Convert2Numeric(pExecuteObject pEo,
                                             pFixSizeMemoryObject pVar,
                                             pMortalList pMyMortal
  ){

[Contents]

2.6.4.22. execute_Dereference()

This function recursively follows variable references and returns the original variable that was referenced by the original variable.

A reference variable is a special variable that does not hold value itself but rather a pointer to another variable. Such reference variables are used when arguments are passed by reference to BASIC subroutines.

Calling this function the caller can get the original variable and the valéue of the original variable rather than a reference.

pFixSizeMemoryObject execute_Dereference(pExecuteObject pEo,
                                         pFixSizeMemoryObject p,
                                         int *piErrorCode
  ){
See also execute_DereferenceS().

[Contents]

2.6.4.23. execute_DereferenceS()

This function does the same as execute_Dereference() except that it has different arguments fitted to support external modules and besXXX macros.

int execute_DereferenceS(unsigned long refcount,
                         pFixSizeMemoryObject *p
  ){
See also execute_Dereference().

[Contents]

2.6.4.24. execute_GetDoubleValue()

Use this function whenever you want to access the value of a variable as a double. Formerly ScriptBasic in such situation converted the variable to double calling execute_Convert2Double() and then used the macro DOUBLEVALUE. This method is faster because this does not create a new mortal variable but returns directly the double value.

The macro GETDOUBLEVALUE can be used to call this function with the default execution environment variable pEo

Note however that the macro GETDOUBLEVALUE and DOUBLEVALUE are not interchangeable. GETDOUBLEVALUE is returnig a double while DOUBLEVALUE is a left value available to store a double.

double execute_GetDoubleValue(pExecuteObject pEo,
                              pFixSizeMemoryObject pVar
  ){

[Contents]

2.6.4.25. execute_GetLongValue()

Use this function whenever you want to access the value of a variable as a long. Formerly ScriptBasic in such situation converted the variable to long calling execute_Convert2Long() and then used the macro LONGVALUE. This method is faster because this does not create a new mortal variable but returns directly the long value.

The macro GETLONGVALUE can be used to call this function with the default execution environment variable pEo

Note however that the macro GETLONGVALUE and LONGVALUE are not interchangeable. GETLONGVALUE is returnig a long while LONGVALUE is a left value available to store a long.

long execute_GetLongValue(pExecuteObject pEo,
                          pFixSizeMemoryObject pVar
  ){

Please also note that the result of converting a string variable to LONG and then accessing its longvalue may not result the same number as calling this function. The reason is that conversion of a string to a LONG variable is done in two steps. First it converts the string to a double and then it rounds the double value to long. On the other hand this function converts a string diretly to long.

For example the string "3.7" becomes 4 when converted to long and 3 when getting the value as a long.

[Contents]

2.6.4.26. execute_IsStringInteger()

This function should be used to check a string before converting it to numeric value. If the string contains only digits it should be converted to long. If the string contains other characters the it should be converted to double. This function decides what characters the string contains.

int execute_IsStringInteger(pFixSizeMemoryObject pVar
  ){

[Contents]

2.6.4.27. execute_IsInteger()

This function checks that a variable being long, double or string can be converted to long without loosing information.

int execute_IsInteger(pFixSizeMemoryObject pVar
  ){

[Contents]

2.7. Configuration File Handling

ScriptBasic contains a fairly sophisticated configuration handling module. The configuration information is read each time the interpreter starts, therefore it is vital that the information can be processed fast even if the configuration data is complex. This can be the case because the configuration information may also contain data for external modules that the interpreter loads when it starts and the external modules can access it any time they run.

The configuration information for ScriptBasic has to be maintained in textual format in a file that has more or less LISP syntax.

The format of the text file version of the configuration information is simple. It contains the keys and the corresponding values separated by one or more spaces and new lines. Usually a key and the assigned value is written on a line. Lines starting with the character ; is comment.

The values can be integer numbers, real numbers, strings and sub-configurations. Strings can either be single line or multi-line strings starting and ending with three """ characters, just like in the language ScriptBasic or in the language Python.

Sub-configurations start with the character ( and are closed with the character ). The list between the parentheses are keys and corresponding values.

This text file has to be converted to binary format. The ScriptBasic interpreter loads this binary format into memory without processing its content, thus loading speed of the configuration information is limited only by IO.

When the interpreter or an external module needs some configuration information there are functions in this module that can search and read information from the configuration file.

[Contents]

2.7.1. Functions implemented in this module

The following subsections list the functions that are implemented in this module. The source text of this documentation was extracted from the source Documentation embedded in the C source as comment. Treat these subsections more as reference documentation and less tutorial like.

[Contents]

2.7.1.1. cft_init()

Before calling any other configuration handling function the caller has to prepare a tConfigTree structure. To do this it has to call this function.

The first argument has to point to an allocated and uninitialized tConfigTree structure. The second argument has to point to a memory allocating function. The third argument has to point to the memory releasing function that is capable releasing the memory allocated by the memory allocating function.

The argument pMemorySegment should be the segment pointer to the memory handling functions. All memory allocation will be performed calling the memory_allocating_function and passing the pMemorySegment pointer as second argument to it. All memory releasing will be done via the function memory_releasing_function passing pMemorySegment pointer as second argument. This lets the caller to use sophisticated memory handling architecture.

On the other hand for the simple use all these three arguments can be NULL. In this case the configuration management system will use its own memory allocating and releasing function that simply uses malloc and free. In this case pMemorySegment is ignored.

For a ready made module that delivers more features see the alloc module of the ScriptBasic project at http://scriptbasic.com

int cft_init(ptConfigTree pCT,
              void *(*memory_allocating_function)(size_t, void *),
              void (*memory_releasing_function)(void *, void *),
              void *pMemorySegment
  ){
Note that suggested convention is to use the '.' character as separator for hierarchical key structures, but this is only a suggestion. In other words the module writers advice is to use key.subkey.subsubkey as key string for hierarchical strings. On the other hand you can use any character as separator except the zero character and except the characters that are used as key characters. You can write

key\subkey\subsubkey

if you are a windows geek. To do this you have to change the character saying

    pCT->TC = '\\';

after calling the initialization function. You can change this character any time, this character is not used in the configuration structure. The only point is that you have to use the actual character when you have changed it. The best practice is to use the dot ever.

[Contents]

2.7.1.2. GetConfigFileName()

This function tries to locate the configuration file. The working of this function is system dependant. There are two different implementations: one for UNIX and one for Win32.

WIN32

On Win32 systems the function tries to read the system registry. The value of the key given in the argument env is used and returned as the config file name. For example if the argument env is Software\myprog\conf then the registry value of the key HKEY_LOCAL_MACHINE\Software\myprog\conf will be returned as configuration file name. The program does not check that the file really exists. It only checks that the registry key exists, it is a string and has some value.

If the registry key does not exists the program tries to locate the system directory getting the environment variable windir, then systemroot and finally taking c:\WINDOWS. The argument DefaultFileName is appended to the directory name and is returned.

UNIX

On UNIX it is more simple. The environment variable env is used as a file name. If this does not exists the DefaultFileName is used and returned.

BOTH

The return value of the function is zero if no error has happened. A pointer to the resulting file name is returned in the variable ppszConfigFile. The space to hold the resulting file name is allocated via the allocation function given by the tConfigTree structure pointed by pCT.

/**/
static int GetConfigFileName(ptConfigTree pCT,
                             char **ppszConfigFile,
                             char *env,/* environment variable or registry key on win32 */
                             char *DefaultFileName
  ){
This function is static and can not be called from outside of this module.

[Contents]

2.7.1.3. cft_start()

When writing real applications you usually want to call this function. This function initializes the tConfigTree structure pointed by pCT, searches for the configuration file and reads it.

When trying to allocate the configuration file the static internal function GetConfigFileName is used.

The argument Envir is the registry key under HKLM, eg Software\Myprog\conf under Win32 or the environment variable to look for the configuration file name. The argument pszDefaultFileName is the file name searched on WIN32 in the system directories or the full path to the default configuration file nam eunder UNIX. The argument pszForcedFileName can overrride the file name search or has to be NULL to let the reader search the environment and registry for file name.

int cft_start(ptConfigTree pCT,
              void *(*memory_allocating_function)(size_t, void *),
              void (*memory_releasing_function)(void *, void *),
              void *pMemorySegment,
              char *Envir,
              char *pszDefaultFileName,
              char *pszForcedFileName
  ){

[Contents]

2.7.1.4. strmyeq()

This is an internal static function that compares two strings and returns true iff they are equal. The string terminator is the usual zero character or the dot. Both are legal terminators for this functions and their difference in the compared strings is not treated as difference in the result. If one string is terminated by zero character and the other is terminated by a dot but they are the same in any other character then the return value is true.

This function is used find a sub-key when the caller has specified a dot separated hierarchical key.

Note that the dot is only a convention and the default value for the separator and the caller has

/**/
static int strmyeq(ptConfigTree pCT,char *a, char *b){
This function is static and can not be called from outside of this module.

[Contents]

2.7.1.5. cft_FindNode()

Find a node starting from the start node lStartNode and searching for the key.

The function returns zero if the key is not found in the configuration information tree pCT or returns the node id of the key. This node can either be an internal node or leaf.

Note that the string key may contain dot characters. In this case the key is searched down in the configuration tree. (You can set the separator character different from the dot character.)

CFT_NODE cft_FindNode(ptConfigTree pCT,
                      CFT_NODE lStartNode,
                      char *key
  ){
You need this function when you want to iterate over the sub-keys of a node. You get the node id for the key and then you can call cft_EnumFirst to start the loop and then cft_EnumNext to iterate the loop over the sub-keys.

If you just want to get the value of a single key you can call the function cft_GetEx that uses this function.

[Contents]

2.7.1.6. cft_GetEx()

Get the value associated with the key key from the configuration structure pCT, or get the values of a node.

The arguments:

Note that any of ppszValue, plValue, pdValue can point to a variable or to NULL in case the caller does not need the actual value.

int cft_GetEx(ptConfigTree pCT,
              char *key,
              CFT_NODE *plNodeId,
              char **ppszValue,
              long *plValue,
              double *pdValue,
              int *type
  ){

The function returns CFT_ERROR_SUCCESS if no error happens. The value CFT_ERROR_SUCCESS is zero.

If an error happens the error code is returned. These error codes are:

[Contents]

2.7.1.7. cft_GetString()

This is the simplest interface function to retrieve a configuration string. This assumes that you exactly know the name of the key and you are sure that the value is a string. The function returns the pointer to the constant string or returns NULL if the configuration key is not present in the tree or the value is not a string.

The use of this function is not recommended. This function is present in this package to ease porting of programs that use simpler configuration information management software.

char *cft_GetString(ptConfigTree pCT,
                    char *key
  ){
This function calls cft_GetEx.

[Contents]

2.7.1.8. cft_EnumFirst()

Whenever you need to enumerate the sub-keys of a key you have to get the node associated with the key (see cft_GetEx or cft_FindNode). When you have the node associated with the key you can get the node of the first sub-key calling this function.

The function needs the node id lNodeId of the key for which we need to enumerate the sub keys and returns the node id of the first sub key.

If the key is associated with a leaf node the function returns zero.

If the key is associated with a branch node that has no sub-keys the function returns zero.

CFT_NODE cft_EnumFirst(ptConfigTree pCT,
                       CFT_NODE lNodeId
  ){

[Contents]

2.7.1.9. cft_EnumNext()

Whenever you need to enumerate the sub-keys of a key you have to get the node associated with the key (see cft_GetEx or cft_FindNode). When you have the node associated with the key you can get the node of the first sub-key calling the function cft_EnumFirst. Later on you can enumerate the sub keys stepping from node to node calling this function.

The function needs the node id lNodeId returned by cft_EnumFirst or by previous call of this function.

The function returns the node id of the next sub key.

If the enumeration has ended, in other words there is no next sub-key the function returns zero.

long cft_EnumNext(ptConfigTree pCT,
                  long lNodeId
  ){

[Contents]

2.7.1.10. cft_GetKey()

This function returns a pointer to the constant zchar string that holds the key of the node defined by the id lNodeId.
char *cft_GetKey(ptConfigTree pCT,
                 CFT_NODE lNodeId
  ){

[Contents]

2.7.1.11. cft_ReadConfig()

int cft_ReadConfig(ptConfigTree pCT,
                   char *pszFileName
  ){

[Contents]

2.7.1.12. cft_WriteConfig()

int cft_WriteConfig(ptConfigTree pCT,
                    char *pszFileName
  ){

[Contents]

2.7.1.13. cft_DropConfig()

void cft_DropConfig(ptConfigTree pCT
  ){

[Contents]

2.8. Memory Allocation

This module is a general purpose memory allocation module, which can be used in any project that needs heavy and sophisticated memory allocation. Originally the module was developed for the ScriptBasic project. Later we used it for Index.hu Rt AdEgine project and multi-thread features were introduced.

The major problem with memory allocation is that memory should be released. Old programs depend on the operating system to release the memory when the process exists and do not release the memory before program termination. Such programs are extremely difficult to port to multi-thread operation. In multi thread operation a thread my exit, but the memory still belongs to the process that goes on.

This module provides a bit of abstraction that helps the programmer to release the memory. The abstraction is the following:

A piece of memory is always allocated from a segment. A segment is logical entity and you should not think of a segment in this content as a continuous memory area. I could also say that: whenever a piece of memory is allocated it is assigned to a segment. When a piece of memory is released it is removed from the segment. A segment is an administrative entity that keep track of the memory pieces that were allocated and assigned to the segment.

To explain segment to the fines details: segments are implemented as linked lists. Each element of the list contains the allocated memory piece as well as a pointer to the next and previous list members.

Whenever the programmer starts a sophisticated task that allocates several memory pieces it has to create a new segment and allocate the memory from that segment. When the memory is to be release the programmer can just say: release all the memory from the segment. This way he or she does not need keep track of the allocated memory structures, and walk through the memory pointers of his or her program which are designed to the program function instead of releasing the memory.

The overhead is the space allocated by two pointers for each memory piece and the size of the three pointers for each segment.

[Contents]

2.8.1. Functions implemented in this module

The following subsections list the functions that are implemented in this module. The source text of this documentation was extracted from the source Documentation embedded in the C source as comment. Treat these subsections more as reference documentation and less tutorial like.

[Contents]

2.8.1.1. Multi-thread use of this module

You can use this module in multi threaded environment. In this case the module depend on the module thread.c which contains the thread and mutex interface functions that call the operating system thread and mutex functions on UNIX and on Windows NT.

In single thread environment there is no need to use the locking mechanism. To get a single-thread version either you can edit this file (myalloc.c) or compile is using the option -DMTHREAD=0 The default compilation is multi threaded.

Multi thread implementation has two levels. One is that the subroutines implemented in this module call the appropriate locking functions to ensure that no two concurrent threads access and modify the same data at a time and thus assure that the data of the module is correct. The other level is that you can tell the module that the underlying memory allocation and deallocation modules are mot thread safe. There are global variables implementing global mutexes that are locked and unlocked if you use the module that way. This can be useful in some environment where malloc and free are not thread safe.

Note that this should not be the case if you call malloc and free or you linked the wrong versio of libc. However you may use a non-thread safe debug layer for example the one that ScriptBasic uses.

[Contents]

2.8.1.2. alloc_InitSegment()

Call this function to get a new segment. You should specify the functions that the segement should use to get memory from the operating system, and the function the segment should use to release the memory to the operating system. These functions should be like malloc and free.

If the second argument is NULL then the function will treat the first argument as an already allocated and initialized memory segment and the memory allocation and freeing functions will be inherited from that segment.

void *alloc_InitSegment(void * (*maf)(size_t), /* a 'malloc' and a 'free' like functions */
                        void   (*mrf)(void *)
  ){

The return value is a void* pointer which identifies the segment and should be passed to the other functions as segment argument.

The first argument is the malloc like function and the second if the free like function.

[Contents]

2.8.1.3. alloc_GlobalUseGlobalMutex()

Some installation use memory allocation function that are not thread safe. On some UNIX installations malloc is not thread safe. To tell the module that all the allocation function primitives are not thread safe call this function before initializing any segment.

void alloc_GlobalUseGlobalMutex(
  ){

[Contents]

2.8.1.4. alloc_SegmentLimit()

You can call this function to set a segment limit. Each segment keeps track of the actual memory allocated to the segment. When a new piece of memory allocated in a segment the calculated segment size is increased by the size of the memory chunk. When a piece of memory is release the calculated size of the segment is decreased.

Whenever a segment approaches its limit the next allocation function requesting memory that would exceed the limit returns NULL and does not allocate memory.

The value of the limit is the number of bytes allowed for the segment. This is the requested number of bytes without the segment management overhead.

Setting the limit to zero means no limit except the limits of the underlying memory allocation layers, usually malloc.

You can dynamically set the limit during handling the memory at any time except that you should not set the limit to zero unless the segment is empty and you should not set the limit to a positive value when the actual limit is zero (no limit) and the segment is not empty. This restriction is artificial in this release but is needed to be followed to be compatible with planned future developments.

This function sets the limit for the segment pointed by p and returns the old value of the segment.

long alloc_SegmentLimit(void *p,
                        unsigned long NewMaxSize
  ){

[Contents]

2.8.1.5. alloc_FreeSegment()

Use this function to release all the memory that was allocated to the segment p. Note that after calling this function the segment is still usable, only the memory that it handled was released. If you do not need the segment anymore call the function alloc_FinishSegment() that calls this function and then releases the memory allocated to store the segment information.

Sloppy programmers may pass NULL as argument, it just returns.

void alloc_FreeSegment(void *p
  ){

[Contents]

2.8.1.6. alloc_FinishSegment()

Use this function to release all the memory that was allocated to the segment p. This function also releases the memory of the segment head and therefore the segment pointed by p is not usable anymore.

void alloc_FinishSegment(void *p
  ){

[Contents]

2.8.1.7. alloc_Alloc()

Use this function to allocate a memory piece from a segment.

void *alloc_Alloc(size_t n,
                  void *p
  ){

The first argument is the size to be allocated. The second argument is the segment which should be used for the allocation.

If the memory allocation fails the function returns NULL.

[Contents]

2.8.1.8. alloc_Free()

You should call this function whenever you want to release a single piece of memory allocated from a segment. Note that you also have to pass the segment pointer as the second argument, because the segment head pointed by this void pointer contains the memory releasing function pointer.

Sloppy programmers may try to release NULL pointer without harm.

void alloc_Free(void *pMem, void *p
  ){

[Contents]

2.8.1.9. alloc_Merge()

Call this function in case you want to merge a segment into another. This can be the case when your program builds up a memory structure in several steps.

This function merges the segment p2 into p1. This means that the segment p1 will contain all the memory pieces that belonged to p2 before and p2 will not contain any allocated memory. However the segment p2 is still valid and can be used to allocated memory from. If you also want to finish the segment p2 call the function alloc_MergeAndFinish().

void alloc_Merge(void *p1, void *p2
  ){

Note that the two segments SHOULD use the same, or at least compatible system memory handling functions! You better use the same functions for both segments.

Example:

ScriptBasic builds up a sophisticated memory structure during syntactical analysis. This memory structure contains the internal code generated from the program lines of the basic program. When ScriptBasic analyses a line it tries several syntax descriptions. It checks each syntax defintion against the tokens of the line until it finds one that fits. These checks need to build up memory structure. However if the check fails and ScriptBasic should go for the next syntac definition line to check the memory allocated during the failed checking should be released. Therefore these memory pieces are allocated from a segment that the program calls pMyMemorySegment. If the syntax check fails this segment if freed. If the syntax check succedes this segment is merged into another segement that contains the memory structures allocated from the previous basic program lines.

[Contents]

2.8.1.10. alloc_MergeAndFinish()

Use this function in case you not only want to merge a segment into another but you also want to finish the segment that was merged into the other.

See also alloc_Merge()

void alloc_MergeAndFinish(void *p1, void *p2
  ){

[Contents]

2.8.1.11. alloc_InitStat()

This function initializes the global statistical variables. These variables can be used in a program to measure the memory usage.

This function should be called before any other memory handling function.

void alloc_InitStat(
  ){

[Contents]

2.8.1.12. alloc_GlobalGetStat()

From period to period the code using this memory management layer may need to know how much memory the program is using.

Calling this function from time to time you can get the minimum and maximum memory that the program used via this layer since the last call to this function or since program start in case of the first call.

void alloc_GlobalGetStat(unsigned long *pNetMax,
                         unsigned long *pNetMin,
                         unsigned long *pBruMax,
                         unsigned long *pBruMin,
                         unsigned long *pNetSize,
                         unsigned long *pBruSize
  ){

[Contents]

2.8.1.13. alloc_GetStat()

From period to period the code using this memory management layer may need to know how much memory the program is using.

Calling this function from time to time you can get the minimum and maximum memory that the program used via this layer since the last call to this function or since program start in case of the first call.

void alloc_GetStat(void *p,
                   unsigned long *pMax,
                   unsigned long *pMin,
                   unsigned long *pActSize
  ){

[Contents]

2.9. Variable Allocation

This module implemented in the source file `memory.c' provides functions to allocate and deallocate memory for BASIC variables. This module itself allocates memory calling the underlying allocation module implemented in the file `myalloc.c'. The role of this module is to help ScriptBasic to reuse the allocated memory used for BASIC variable value store effectively.

[Contents]

2.9.1. Functions implemented in this module

The following subsections list the functions that are implemented in this module. The source text of this documentation was extracted from the source Documentation embedded in the C source as comment. Treat these subsections more as reference documentation and less tutorial like.

[Contents]

2.9.1.1. memory_InitStructure()

Each execution context should have its own memory object responsible for the administration of the variables and the memory storing the values assigned to variables.

This function initializes such a memory object.

int memory_InitStructure(pMemoryObject pMo
  ){

[Contents]

2.9.1.2. memory_RegisterType()

This function should be used to register a variable type. The return value is the serial number of the type that should later be used to reference the type.

int memory_RegisterType(pMemoryObject pMo,
                        unsigned long SizeOfThisType
  ){
The argument of the function is the size of the type in terms of bytes. Usually this is calculated using the C structure sizeof.

If the type can not be registered -1 is returned.

[Contents]

2.9.1.3. memory_RegisterTypes()

This function should be used to initialize the usual FixSizeMemoryObject types. This sets some usual string sizes, but the caller may not call this function and set different size objects.

void memory_RegisterTypes(pMemoryObject pMo
  ){
This function registers the different string sizes. In the current implementation a string has at least 32 characters. If this is longer that that (including the terminating zchar) then a 64 byte fix size object is allocated. If this is small enough then a 128 byte fix size memory object is allocated and so on up to 1024 bytes. If a string is longer that that then a LARGE_OBJECT_TYPE is allocated.

The reason to register these types is that this memory management module keeps a list for these memory pieces and when a new short string is needed it may be available already without calling malloc. On the other hand when a LARGE_OBJECT_TYPE value is released it is always passed back to the operating system calling free.

[Contents]

2.9.1.4. memory_DebugDump()

This is a debugging function that dumps several variable data to the standard output. The actual behavior of the function may change according to the actual debug needs.

void memory_DebugDump(pMemoryObject pMo
  ){

[Contents]

2.9.1.5. memory_NewVariable()

This function should be used whenever a new variable is to be allocated. The function returns a pointer to a FixSizeMemoryObject structure that holds the variable information and pointer to the memory that stores the actual value for the memory.

If there is not engough memory or the calling is illegal the returned value is NULL

pFixSizeMemoryObject memory_NewVariable(pMemoryObject pMo,
                                        int type,
                                        unsigned long LargeBlockSize
  ){

The second argument gives the type of the memory object to be allocated. If this value is LARGE_BLOCK_TYPE then the third argument is used to determine the size of the memory to be allocated. If the type if NOT LARGE_BLOCK_TYPE then this argument is ignored and the proper size is allocated.

If the type has memory that was earlier allocated and released it is stored in a free list and is reused.

[Contents]

2.9.1.6. memory_ReleaseVariable()

This function should be used to release a memory object.

int memory_ReleaseVariable(pMemoryObject pMo,
                           pFixSizeMemoryObject p
  ){

[Contents]

2.9.1.7. memory_NewString()

This function should be used to allocate string variable.
pFixSizeMemoryObject memory_NewString(pMemoryObject pMo,
                                      unsigned long StringSize
  ){

The second argument specifies the length of th required string including.

The function checks the desired length and if this is small then is allocates a fix size object. If this is too large then it allocates a LARGE_BLOCK_TYPE

[Contents]

2.9.1.8. memory_NewCString()

This function should be used to allocate variable to store a constant string.
pFixSizeMemoryObject memory_NewCString(pMemoryObject pMo,
                                       unsigned long StringSize
  ){

The second argument specifies the length of the required string.

[Contents]

2.9.1.9. memory_SetRef()

Set the variable ppVar to reference the variable ppVal.

int memory_SetRef(pMemoryObject pMo,
                   pFixSizeMemoryObject *ppVar,
                   pFixSizeMemoryObject *ppVal
  ){

[Contents]

2.9.1.10. memory_NewRef()

pFixSizeMemoryObject memory_NewRef(pMemoryObject pMo
  ){

[Contents]

2.9.1.11. memory_IsUndef()

int memory_IsUndef(pFixSizeMemoryObject pVar
  ){

[Contents]

2.9.1.12. memory_SelfOrRealUndef()

pFixSizeMemoryObject memory_SelfOrRealUndef(pFixSizeMemoryObject pVar
  ){

[Contents]

2.9.1.13. memory_NewUndef()

pFixSizeMemoryObject memory_NewUndef(pMemoryObject pMo
  ){

[Contents]

2.9.1.14. memory_ReplaceVariable()

int memory_ReplaceVariable(pMemoryObject pMo,
                           pFixSizeMemoryObject *Lval,
                           pFixSizeMemoryObject NewValue,
                           pMortalList pMortal,
                           int iDupFlag
  ){

[Contents]

2.9.1.15. memory_NewLong()

pFixSizeMemoryObject memory_NewLong(pMemoryObject pMo
  ){

[Contents]

2.9.1.16. memory_NewDouble()

pFixSizeMemoryObject memory_NewDouble(pMemoryObject pMo
  ){

[Contents]

2.9.1.17. memory_CopyArray

pFixSizeMemoryObject memory_CopyArray(pMemoryObject pMo,
                                      pFixSizeMemoryObject p
  ){

[Contents]

2.9.1.18. memory_NewArray()

This function should be used whenever a new array is to be allocated.

pFixSizeMemoryObject memory_NewArray(pMemoryObject pMo,
                                     long LowIndex,
                                     long HighIndex
  ){
The index variables define the indices that are to be used when accessing an array element. The index values are inclusive.

[Contents]

2.9.1.19. memory_ReDimArray()

This function should be used when an array needs redimensioning. If the redimensioning is succesful the function returns the pointer to the argument p. If memory allocation is needed and the memory allocation fails the function returns NULL. In this case the original array is not changed.

If the redimensioned array is smaller that the original no memory allocation takes place, only the array elements (pointers) are moved.

pFixSizeMemoryObject memory_ReDimArray(pMemoryObject pMo,
                                       pFixSizeMemoryObject p,
                                       long LowIndex,
                                       long HighIndex
  ){

[Contents]

2.9.1.20. memory_CheckArrayIndex()

This function should be called before accessing a certain element of an array. The function checks that the index is within the index limitsof the array and in case the index is outside the index limits of the array it redimensionate the array.

The function returns the pointer passed as parameter p or NULL in case there is a memory allocation error.

pFixSizeMemoryObject memory_CheckArrayIndex(pMemoryObject pMo,
                                            pFixSizeMemoryObject p,
                                            long Index
  ){

[Contents]

2.9.1.21. memory_Mortalize()

This function should be used when a variable is to be put in a mortal list.

void memory_Mortalize(pFixSizeMemoryObject p,
                      pMortalList pMortal
  ){
Note that care should be taken to be sure that the variable is NOT on a mortal list. If the variable is already on a mortal list calling this function will break the original list and therefore may loose the variables that follow this one.

[Contents]

2.9.1.22. memory_Immortalize()

Use this function to immortalize a variable. This can be used when the result of an expression evaluation gets into a mortal variable and instead of copiing the value from the mortal variable to an immortal variable the caller can immortalize the variable. However it should know which mortal list the variable is on.

void memory_Immortalize(pFixSizeMemoryObject p,
                        pMortalList pMortal
  ){

[Contents]

2.9.1.23. memory_NewMortal()

When an expression is evaluated mortal variables are needed to store the intermediate results. These variables are called mortal variables. Such a variable is is allocated using this function and specifying a variable of type MortalList to assign the mortal to the list of mortal variables.

When the expression is evaluated all mortal variables are to be released and they are calling the function memory_ReleaseMortals (see memory_ReleaseMortals()).

pFixSizeMemoryObject memory_NewMortal(pMemoryObject pMo,
                                      BYTE type,
                                      unsigned long LargeBlockSize,
                                      pMortalList pMortal
  ){
If the parameter pMortal is NULL the generated variable is not mortal.

[Contents]

2.9.1.24. memory_DupImmortal()

This function creates a new mortal and copies the argument pVar into this new mortal.

pFixSizeMemoryObject memory_DupImmortal(pMemoryObject pMo,
                                        pFixSizeMemoryObject pVar,
                                        int *piErrorCode
  ){

[Contents]

2.9.1.25. memory_DupVar()

This function creates a new mortal and copies the argument pVar into this new mortal.

pFixSizeMemoryObject memory_DupVar(pMemoryObject pMo,
                                   pFixSizeMemoryObject pVar,
                                   pMortalList pMyMortal,
                                   int *piErrorCode
  ){
This function is vital, when used in operations that convert the values to long or double. Expression evaluation may return an immortal value, when the expression is a simple variable access. Conversion of the result would modify the value of the variable itself. Therefore functions and operators call this function to duplicate the result to be sure that the value they convert is mortal and to be sure they do not change the value of a variable when they are not supposed to.

Note that you can duplicate long, double and string values, but you can not duplicate arrays! The string value is duplicated and the characters are copied to the new location. This is perfect. However if you do the same with an array the array pointers will point to the same variables, which are not going to be duplicated. This result multiple reference to a single value. This situation is currently not supported by this system as we do not have either garbage collection or any other solution to support such memory structures.

[Contents]

2.9.1.26. memory_DupMortalize()

This function creates a new mortal and copies the argument pVar into this new mortal only if the value is immortal. If the value is mortal the it returns the original value.

pFixSizeMemoryObject memory_DupMortalize(pMemoryObject pMo,
                                         pFixSizeMemoryObject pVar,
                                         pMortalList pMyMortal,
                                         int *piErrorCode
  ){

[Contents]

2.9.1.27. memory_ReleaseMortals()

This function should be used to release the mortal variables.

When an expression is evaluated mortal variables are needed to store the intermediate results. These variables are called mortal variables. Such a variable is is allocated using this function and specifying a variable of type MortalList to assign the mortal to the list of mortal variables.

void memory_ReleaseMortals(pMemoryObject pMo,
                           pMortalList pMortal
  ){

[Contents]

2.9.1.28. memory_DebugDumpVariable()

This function is used for debugging purposes. (To debug ScriptBasic and not to debug a BASIC program using ScriptBasic. :-o )

The function prints the content of a variable to the standard output.

void memory_DebugDumpVariable(pMemoryObject pMo,
                              pFixSizeMemoryObject pVar
  ){

[Contents]

2.9.1.29. memory_DebugDumpMortals()

This function is used for debugging purposes. (To debug ScriptBasic and not to debug a BASIC program using ScriptBasic. :-o )

The function prints the content of the mortal list to the standard output.

void memory_DebugDumpMortals(pMemoryObject pMo,
                             pMortalList pMortal
  ){

[Contents]

2.9.1.30. memory_NewMortalString()

pFixSizeMemoryObject memory_NewMortalString(pMemoryObject pMo,
                                            unsigned long StringSize,
                                            pMortalList pMortal
  ){
If the parameter pMortal is NULL the generated variable is not mortal.

[Contents]

2.9.1.31. memory_NewMortalCString()

pFixSizeMemoryObject memory_NewMortalCString(pMemoryObject pMo,
                                             unsigned long StringSize,
                                             pMortalList pMortal
  ){
If the parameter pMortal is NULL the generated variable is not mortal.

[Contents]

2.9.1.32. memory_NewMortalLong()

pFixSizeMemoryObject memory_NewMortalLong(pMemoryObject pMo,
                                            pMortalList pMortal
  ){
If the parameter pMortal is NULL the generated variable is not mortal.

[Contents]

2.9.1.33. memory_NewMortalRef()

This function was never used. It was presented in the code to allow external modules to create mortal reference variables. However later I found that the variable structure design does not allow mortal reference variables and thus this function is nonsense.

Not to change the module interface defintion the function still exists but returns NULL, like if memory were exhausted.

pFixSizeMemoryObject memory_NewMortalRef(pMemoryObject pMo,
                                         pMortalList pMortal
  ){
If the parameter pMortal is NULL the generated variable is not mortal.

[Contents]

2.9.1.34. memory_NewMortalDouble()

pFixSizeMemoryObject memory_NewMortalDouble(pMemoryObject pMo,
                                            pMortalList pMortal
  ){
If the parameter pMortal is NULL the generated variable is not mortal.

[Contents]

2.9.1.35. memory_NewMortalArray()

pFixSizeMemoryObject memory_NewMortalArray(pMemoryObject pMo,
                                           pMortalList pMortal,
                                           long IndexLow,
                                           long IndexHigh
  ){
If the parameter pMortal is NULL the generated variable is not mortal.

[Contents]

2.10. Error Reporting

[Contents]

2.10.1. Functions implemented in this module

The following subsections list the functions that are implemented in this module. The source text of this documentation was extracted from the source Documentation embedded in the C source as comment. Treat these subsections more as reference documentation and less tutorial like.

This file contains a simple error report handling function that prints the error to the standard error.

This is a default reporting function used by most variations of ScriptBasic. However some variations like the ISAPI one needs to implements a function having the same interface.

[Contents]

2.10.1.1. report_report()

This function implements the default error reporting function for both run-time and parse time errors and warnings.

void report_report(void *filepointer,
                   char *FileName,
                   long LineNumber,
                   unsigned int iErrorCode,
                   int iErrorSeverity,
                   int *piErrorCounter,
                   char *szErrorString,
                   unsigned long *fFlags
  ){
Aguments:

[Contents]

2.11. The Logger Module

The logger module is included in ScriptBasic though it is not a critical part of it. The command line version itself does not use the functions implemented in this module, though the functions are available for external modules. The Eszter SB Application Engine uses this module to asynchronously log hits and other events to ASCII text log files.

This module can be used to log events. The module implements two type of logs.

Synchronous logs are just the normal plain logging technic writing messages to a log file. This is low performance, because the caller has to wait until the logging is performed and written to a file. On the other hand this is a safe logging.

Asynchronous logging is a fast performance logging method. In this case the caller passes the log item to the logger. The logger puts the item on a queue and sends it to the log file in another thread when disk I/O bandwith permits. This is high performance, because the caller does not need to wait for the log item written to the disk. On the other hand this logging is not safe because the caller can not be sure that the log was written to the disk.

The program using this module should use asynchronous logging for high volume logs and synchronous logging for low volume logging. For example a panic log that reports configuration error has to be written synchronously.

Using this module you can initialize a log specifying the file where to write the log, send logs and you can tell the log to shut down. When shutting down all waiting logs are written to the file and no more log items are accepted. When all logs are written the logging thread terminates.

[Contents]

2.11.1. log_state()

This function safely returns the actual state of the log. This can be:

int log_state(ptLogger pLOG
  ){

[Contents]

2.11.2. log_init()

Initialize a log. The function sets the parameters of a logging thread. The parameters are the usual memory allocation and deallocation functions and the log file name format string. This format string can contain at most four %d as formatting element. This will be passed to sprintf with arguments as year, month, day and hour in this order. This will ease log rotating.

Note that log file name calculation is a CPU consuming process and therefore it is not performed for each log item. The log system recalculates the log file name and closes the old log file and opens a new one whenever the actual log to be written and the last log wrote is in a different time interval. The time interval is identified by the time stamp value divided (integer division) by the time span value. This is 3600 when you want to rotate the log hourly, 86400 if you want to rotate the log daily. Other rotations, like monthly do not work correctly.

To do this the caller has to set the TimeSpan field of the log structure. There is no support function to set this.

For example:

if( log_init(&ErrLog,alloc_Alloc,alloc_Free,pM_AppLog,CONFIG("log.err.file"),LOGTYPE_NORMAL) ) return 1; if( cft_GetEx(&MyCONF,"log.err.span",&ConfNode,NULL,&(ErrLog.TimeSpan),NULL,NULL) ) ErrLog.TimeSpan = 0;

as you can see in the file ad.c Setting TimeSpan to zero results no log rotation.

Note that it is a good practice to set the TimeSpan value to positive (non zero) even if the log is not rotated. If you ever delete the log file while the logging application is running the log is not written anymore until the log file is reopened.

The log type can be LOGTYPE_NORMAL to perform asynchronous high performance logging and LOGTYPE_SYNCHRONOUS for syncronous, "panic" logging. Panic logging keeps the file continously opened until the log is shut down and does not perform log rotation.

int log_init(ptLogger pLOG,
             void *(*memory_allocating_function)(size_t, void *),
             void (*memory_releasing_function)(void *, void *),
             void *pMemorySegment,
             char *pszLogFileName,
             int iLogType
  ){

[Contents]

2.11.3. log_printf()

This function can be used to send a formatted log to the log file. The function creates the formatted string and then puts it onto the log queue. The log is actually sent to the log file by the asynchronous logger thread.

int log_printf(ptLogger pLOG,
               char *pszFormat,
               ...
  ){

[Contents]

2.11.4. log_shutdown()

Calling this function starts the shutdown of a log queue. This function allways return 0 as success. When the function returns the log queue does not accept more log items, however the queue is not completely shut down. If the caller wants to wait for the queue to shut down it has to wait and call the function log_state to ensure that the shutdown procedure has been finished.

int log_shutdown(ptLogger pLOG
  ){

[Contents]

2.12. Hook Functions

@c Hook functions

This file contains the hook functions that are called by the commands whenever a command wants to access the operating system functions. The hook functions implemented here are transparent, they call the operating system. However these hook functions are called via the HookFunctions function pointer table and external modules may alter this table supplying their own hook functions.

There are some hook functions, which do not exist by default. In this case the hook functions table points to NULL. These functions, if defined are called by ScriptBasic at certain points of execution. For example the function HOOK_ExecBefore is called each time before executing a command in case an external module defines the function altering the hook function table.

The hook functions have the same arguments as the original function preceeded by the pointer to the execution object pExecuteObject pEo.

[Contents]

2.12.1. hook_Init

@c Allocate and initialize a hook function table

This function allocates a hook function table and fills the function pointers to point to the original transparent hook functions.

int hook_Init(pExecuteObject pEo,
              pHookFunctions *pHookers
  ){

[Contents]

2.12.2. hook_file_access

@c file_access

This function gets a file name as an argument and return an integer code that tells the caller if the program is allowed to read, write or both read and write to the file. The default implementation just dumbly answers that the program is allowed both read and write. This function is called by each other hook functions that access a file via the file name. If a module wants to restrict the basic code to access files based on the file name the module does not need to alter all hook functions that access files via file name.

The module has to write its own file_access hook function instead, alter the hook function table to point to the module's function and all file accessing functions will ask the module's hook function if the code may access the file.

The argument pszFileName is the name of the file that the ScriptBasic program want to do something. The actual file_access hook function should decide if the basic program is

The default implementation of this function just allows the program to do anything. Any extension module may have its own implementation and restrict the basic program to certain files.

int hook_file_access(pExecuteObject pEo,
                     char *pszFileName
  ){

[Contents]

2.12.3. hook_fopen

@c fopen
FILE *hook_fopen(pExecuteObject pEo,
                 char *pszFileName,
                 char *pszOpenMode
  ){

[Contents]

2.12.4. hook_fclose

@c fclose
void hook_fclose(pExecuteObject pEo,
                  FILE *fp
  ){

[Contents]

2.12.5. hook_size

@c size
long hook_size(pExecuteObject pEo,
               char *pszFileName
  ){

[Contents]

2.12.6. hook_time_accessed

@c time_accessed

long hook_time_accessed(pExecuteObject pEo,
                        char *pszFileName
  ){

[Contents]

2.12.7. hook_time_modified

@c time_modified

long hook_time_modified(pExecuteObject pEo,
                        char *pszFileName
  ){

[Contents]

2.12.8. hook_time_created

@c time_created

long hook_time_created(pExecuteObject pEo,
                        char *pszFileName
  ){

[Contents]

2.12.9. hook_isdir

@c isdir

int hook_isdir(pExecuteObject pEo,
               char *pszFileName
  ){

[Contents]

2.12.10. hook_isreg

@c isreg

int hook_isreg(pExecuteObject pEo,
               char *pszFileName
  ){

[Contents]

2.12.11. hook_fileexists

@c fileexists

int hook_exists(pExecuteObject pEo,
                char *pszFileName
  ){

[Contents]

2.12.12. hook_truncate

@c truncate
int hook_truncate(pExecuteObject pEo,
                  FILE *fp,
                  long lNewFileSize
  ){

[Contents]

2.12.13. hook_fgetc

@c fgetc
int hook_fgetc(pExecuteObject pEo,
               FILE *fp
  ){

[Contents]

2.12.14. hook_ferror

@c ferror
int hook_ferror(pExecuteObject pEo,
               FILE *fp
  ){

[Contents]

2.12.15. hook_fread

@c fread
int hook_fread(pExecuteObject pEo,
               char *buf,
               int size,
               int count,
               FILE *fp
  ){

[Contents]

2.12.16. hook_setmode

@c Set the mode of a file stream to binary or to ASCII

void hook_setmode(pExecuteObject pEo,
                  FILE *fp,
                  int mode
  ){

[Contents]

2.12.17. hook_binmode

@c Set a file stream to binary mode
void hook_binmode(pExecuteObject pEo,
                  FILE *fp
  ){

[Contents]

2.12.18. hook_textmode

@c Set a file stream to text mode
void hook_textmode(pExecuteObject pEo,
                   FILE *fp
  ){

[Contents]

2.12.19. hook_fwrite

@c fwrite
int hook_fwrite(pExecuteObject pEo,
               char *buf,
               int size,
               int count,
               FILE *fp
  ){

[Contents]

2.12.20. hook_fputc

@c fputc

int hook_fputc(pExecuteObject pEo,
               int c,
               FILE *fp
  ){

[Contents]

2.12.21. hook_flock

@c flock

int hook_flock(pExecuteObject pEo,
               FILE *fp,
               int iLockType
  ){

[Contents]

2.12.22. hook_lock

@c lock

int hook_lock(pExecuteObject pEo,
              FILE *fp,
              int iLockType,
              long lStart,
              long lLength
  ){
  return file_lock(fp,iLockType,lStart,lLength);

[Contents]

2.12.23. hook_feof

@c feof

int hook_feof(pExecuteObject pEo,
              FILE *fp
  ){

[Contents]

2.12.24. hook_mkdir

@c mkdir

int hook_mkdir(pExecuteObject pEo,
               char *pszDirectoryName
  ){

[Contents]

2.12.25. hook_rmdir

@c rmdir

int hook_rmdir(pExecuteObject pEo,
               char *pszDirectoryName
  ){

[Contents]

2.12.26. hook_remove

@c remove

int hook_remove(pExecuteObject pEo,
                char *pszFileName
  ){

[Contents]

2.12.27. hook_deltree

@c deltree

int hook_deltree(pExecuteObject pEo,
                 char *pszDirectoryName
  ){

[Contents]

2.12.28. hook_MakeDirectory

@c MakeDirectory

int hook_MakeDirectory(pExecuteObject pEo,
                       char *pszDirectoryName
  ){

[Contents]

2.12.29. hook_opendir

@c opendir

DIR *hook_opendir(pExecuteObject pEo,
                  char *pszDirectoryName,
                  tDIR *pDirectory
  ){

[Contents]

2.12.30. hook_readdir

@c readdir

struct dirent *hook_readdir(pExecuteObject pEo,
                            DIR *pDirectory
  ){

[Contents]

2.12.31. hook_closedir

@c closedir

void hook_closedir(pExecuteObject pEo,
                   DIR *pDirectory
  ){

[Contents]

2.12.32. hook_sleep

@c sleep

void hook_sleep(pExecuteObject pEo,
                long lSeconds
  ){

[Contents]

2.12.33. hook_curdir

@c curdir

int hook_curdir(pExecuteObject pEo,
                char *Buffer,
                unsigned long cbBuffer
  ){

[Contents]

2.12.34. hook_chdir

@c chdir

int hook_chdir(pExecuteObject pEo,
               char *Buffer
  ){

[Contents]

2.12.35. hook_chown

@c chown

int hook_chown(pExecuteObject pEo,
               char *pszFileName,
               char *pszOwner
  ){

[Contents]

2.12.36. hook_SetCreateTime

@c SetCreateTime

int hook_SetCreateTime(pExecuteObject pEo,
                       char *pszFileName,
                       long lTime
  ){

[Contents]

2.12.37. hook_SetModifyTime

@c SetModifyTime

int hook_SetModifyTime(pExecuteObject pEo,
                       char *pszFileName,
                       long lTime
  ){

[Contents]

2.12.38. hook_SetAccessTime

@c SetAccessTime

int hook_SetAccessTime(pExecuteObject pEo,
                       char *pszFileName,
                       long lTime
  ){

[Contents]

2.12.39. hook_gethostname

@c gethostname

int hook_gethostname(pExecuteObject pEo,
                     char *pszBuffer,
                     long cbBuffer
  ){

[Contents]

2.12.40. hook_gethost

@c gethost
int hook_gethost(pExecuteObject pEo,
                 char *pszBuffer,
                 struct hostent *pHost
  ){

[Contents]

2.12.41. hook_tcpconnect

@c tcpconnect
int hook_tcpconnect(pExecuteObject pEo,
                    SOCKET *sClient,
                    char *pszRemoteSocket
  ){

[Contents]

2.12.42. hook_tcpsend

@c tcpsend
int hook_tcpsend(pExecuteObject pEo,
                 SOCKET sClient,
                 char *pszBuffer,
                 long cbBuffer,
                 int iFlags
  ){

[Contents]

2.12.43. hook_tcprecv

@c tcprecv
int hook_tcprecv(pExecuteObject pEo,
                 SOCKET sClient,
                 char *pszBuffer,
                 long cbBuffer,
                 int iFlags
  ){

[Contents]

2.12.44. hook_tcpclose

@c tcpclose
int hook_tcpclose(pExecuteObject pEo,
                  SOCKET sClient
  ){

[Contents]

2.12.45. hook_killproc

@c killproc
int hook_killproc(pExecuteObject pEo,
                  long pid
  ){

[Contents]

2.12.46. hook_getowner

@c getowner
int hook_getowner(pExecuteObject pEo,
                  char *pszFileName,
                  char *pszOwnerBuffer,
                  long cbOwnerBuffer
 ){

[Contents]

2.12.47. hook_fcrypt

@c fcrypt
char *hook_fcrypt(pExecuteObject pEo,
                  char *buf,
                  char *salt,
                  char *buff
  ){

[Contents]

2.12.48. hook_CreateProcess

@c CreateProcess
long hook_CreateProcess(pExecuteObject pEo,
                         char *pszCommandLine
  ){

[Contents]

2.12.49. hook_CreateProcessEx

@c CreateProcessEx
long hook_CreateProcessEx(pExecuteObject pEo,
                          char *pszCommandLine,
                          long lTimeOut,
                          unsigned long *plPid,
                          unsigned long *plExitCode
  ){

[Contents]

2.12.50. hook_CallScribaFunction

@c Start to execute a scriba function

This is a hook function that performs its operation itself without calling underlying file_ function. This function is called by external modules whenever the external module wants to execute certain ScriptBasic function.

The external module has to know the entry point of the ScriptBasic function.

int hook_CallScribaFunction(pExecuteObject pEo,
                            unsigned long lStartNode,
                            pFixSizeMemoryObject *pArgument,
                            unsigned long NumberOfPassedArguments,
                            pFixSizeMemoryObject *pFunctionResult
  ){

[Contents]

2.13. Handle Pointers in External Modules Support Functions

@c Handling handle pointer conversion

The functions in this file help the various ScriptBasic extension modules to avoid crashing the system even if the BASIC programs use the values passed by the module in a bad way.

For example a database handling module opens a database and allocates a structure describing the connection. The usual way to identify the structure is to return a BASIC string variable to the BASIC code that byte by byte holds the value of the pointer. This works on any machine having 32bit or 64bit pointers because strings can be arbitrary length in ScriptBasic.

When another external module function need access to the structure it needs a pointer to it. This is easily done by passing the string variable to the module. The module converts the string variable back byte by byte to a pointer and all is fine.

Is it?

The issue is that the BASIC program may alter the pointer and pass a string containg garbage back to the module. The module has no way to check the correctness tries to use it and crashes the whole interpreter. (Even the other interpreters running in the same process in different threads.)

=bold ScriptBasic external modules should never ever pass pointers in strings back to the BASIC code. =nobold

(Even that some of the modules written by the ScriptBasic developers followed this method formerly.)

The better solution is to store these module pointers in arrays and pass the index of the pointer in the array to the basic application. This way the BASIC program will get INTEGER values instead of STRING and will not be able to alter the pointer value and crash the program.

To store the pointer and get the index (we call it a handle) these functions can be used.

Whenever a pointer needs a handle the module has to call GetHandle. This function stores the pointer and returns the handle to it. When the BASIC program passes the handle back to the module and the module needs the pointer associated with the handle it has to call GetPointer.

When a pointer is not needed anymore the handle should be freed calling FreeHandle.

This implementation uses arrays to hold the pointers. The handles are the indexes to the array. The index 0 is never used. Handle value zero is returned as an invalid handle value whenever some error occures, like out of memory condition.

[Contents]

2.13.1. handle_GetHandle

@c GetHandle

Having a pointer allocate a handle. This function stores the pointer and returns the handle.

The handle is a small positive integer.

If any error is happened (aka out of memory) zero is returned.

unsigned long handle_GetHandle(void **pHandle,
                               void *pMEM,
                               void *pointer
  ){

Note that NULL pointer can not be stored in the array.

The pointer to the handle array pHandle should be initialized to NULL before the first call to handle_GetHandle. For example:

   void *Handle = NULL;
     ....
   if( !handle_GetHandle(&Handle,pMEM,pointer) )return ERROR_CODE;

[Contents]

2.13.2. handle_GetPointer

@c GetPointer

This function is the opposite of GetHandle. If a pointer was stored in the handle array this function can be used to retrieve the pointer knowing the handle.

void *handle_GetPointer(void **pHandle,
                        unsigned long handle
  ){

If there was not pointer registered with that handle the return value of the function is NULL.

[Contents]

2.13.3. handle_FreeHandle

@c FreeHandle

Use this function when a pointer is no longer valid. Calling this function releases the handle for further pointers.

void handle_FreeHandle(void **pHandle,
                       unsigned long handle
  ){

[Contents]

2.13.4. handle_DestroyHandleArray

@c DestroyHandleArray

Call this function to release the handle array after all handles are freed and there is no need for the handle heap.

Use the same memory head pMEM that was used in GetHandle.

void handle_DestroyHandleArray(void **pHandle,
                               void *pMEM
  ){

[Contents]

2.14. Thread Support Functions

@c thread handling routines

This file implements global thread handling functions. If the programmer uses these functions instead of the operating system provided functions the result will be Windows NT and UNIX portable program. These routines handling thread and mutex locking functions had been extensively tested in commercial projects.

[Contents]

2.14.1. thread_CreateThread

@c Create a new thread

This is a simplified implementation of the create thread interface.

The function creates a new detached thread. If the thread can not be created for some reason the return value is the error code returned by the system call pthread_start on UNIX or GetLastError on NT.

If the thread was started the return value is 0.

int thread_CreateThread(PTHREADHANDLE pThread,
                      void *pStartFunction,
                      void *pThreadParameter
  ){
The arguments

[Contents]

2.14.2. thread_ExitThread

@c Exit from a thread

Exit from a thread created by CreateThread. The implementation is simple and does not allow any return value from the thread.

void thread_ExitThread(
  ){

[Contents]

2.14.3. thread_InitMutex

@c Initialize a mutex object

This function initializes a MUTEX variable. A MUTEX variable can be used for exclusive access. If a mutex is locked another lock on that mutex will wait until the first lock is removed. If there are several threads waiting for a mutex to be released a random thread will get the lock when the actually locking thread releases the mutex. In other words if there are several threads waiting for a mutex there is no guaranteed order of the threads getting the mutex lock.

Before the first use of a MUTEX variable it has to be initialized calling this function.

void thread_InitMutex(PMUTEX pMutex
  ){
Arguments:

[Contents]

2.14.4. thread_FinishMutex

@c Delete a mutex object

When a mutex is not used anymore by a program it has to be released to free the system resources allocated to handle the mutex.

void thread_FinishMutex(PMUTEX pMutex
  ){
Arguments:

[Contents]

2.14.5. thread_LockMutex

@c Lock a mutex object

Calling this function locks the mutex pointed by the argument. If the mutex is currently locked the calling thread will wait until the mutex becomes available.

void thread_LockMutex(PMUTEX pMutex
  ){
Arguments:

[Contents]

2.14.6. thread_UnlockMutex

@c Unlock a mutex object

Calling this function unlocks the mutex pointed by the argument. Calling this function on a mutex currently not locked is a programming error and results undefined result. Different operating system may repond different.

void thread_UnlockMutex(PMUTEX pMutex
  ){
Arguments:

[Contents]

2.14.7. thread_shlckstry

@c Shared locks

The following functions implement shared locking. These functions do not call system dependant functions. These are built on the top of the MUTEX locking functions.

A shareable lock can be READ locked and WRITE locked. When a shareable lock is READ locked another thread can also read lock the lock.

On the other hand a write lock is exclusive. A write lock can appear when there is no read lock on a shareable lock and not write lock either.

@cr

The story to understand the workings:

Imagine a reading room with several books. You can get into the room through a small entrance room, which is dark. To get in you have to switch on the light. The reading room has a light and a switch as well. You are not expected to read in the dark. The reading room is very large with several shelves that easily hide the absent minded readers and therefore the readers can not easily decide upon leaving if they are the last or not. This actually led locking up late readers in the dark or the opposite: lights running all the night.

To avoid this situation the library placed a box in the entrance room where each reader entering the room have to place his reader Id card. When they leave they remove the card. The first reader coming switches the light on, and the last one switches the light off. Coming first and leaving last is easily determined looking at the box after dropping the card or after taking the card out. If there is a single card after dropping the reader card into you are the first coming and if there is no card in it you took your one then you are the last.

To avoid quarreling and to save up energy the readers must switch on the light of the entrance room when they come into and should switch it off when they leave. However they have to do it only when they go into the reading room, but not when leaving. When someone wants to switch a light on, but the light is already on he or she should wait until the light is switched off. (Yes, this is a MUTEX.)

When the librarian comes to maintain ensures that no one is inside, switches the light of the entrance room on, and then switches the reading room light on. If someone is still there he cannot switch the light on as it is already switched on. He waits until the light is switched off then he switches it on. When he has switched the light of the reading room on he switches the light of the entrance room off and does his job in the reading room. Upon leaving he switches off the light of the reading room.

Readers can easily enter through the narrow entrance room one after the other. They can also easily leave. When the librarian comes he can not enter until all readers leave the reading room. Before getting into the entrance room he has equal chance as any of the readers.

[Contents]

2.14.8. thread_InitLock

@c Initialize a shareable lock

void shared_InitLock(PSHAREDLOCK p
  ){

[Contents]

2.14.9. thread_FinishLock

@c Finish a shareable lock

void shared_FinishLock(PSHAREDLOCK p
  ){

[Contents]

2.14.10. thread_LockRead

@c Lock a shareable lock for shared (read) lock

void shared_LockRead(PSHAREDLOCK p
  ){

[Contents]

2.14.11. thread_LockWrite

@c Lock a shareable lock for exclusive locking

void shared_LockWrite(PSHAREDLOCK p
  ){

[Contents]

2.14.12. thread_UnlockRead

@c Unlock a sharebale lock that was locked shared

void shared_UnlockRead(PSHAREDLOCK p
  ){

[Contents]

2.14.13. thread_UnlockWrite

@c Unlock a sharebale lock that was locked exclusive

void shared_UnlockWrite(PSHAREDLOCK p
  ){

[Contents]

2.15. Dynamic Library Handling Support Functions

@c Handling Dynamic Load Libraries

The Dynamic Load Libraries are handled different on all operating systems. This file implements a common functional base handling the DLLs for ScriptBasic. All other modules of ScriptBasic that want to use DLLs should call only the functions implemented in this file.

=toc

[Contents]

2.15.1. dynlolib_LoadLibrary

@c Load a library

This function loads a library and returns a pointer that can be used in other functions referencing the loaded library.

void *dynlolib_LoadLibrary(
  char *pszLibraryFile
  ){
The argument pszLibraryFile is the ZCHAR file name.

The file name is either absolute or relative. When a relative file name is specified the directories searched may be different on different operating systems.

[Contents]

2.15.2. dynlolib_FreeLibrary

@c Release a library

This function releases the library that was loaded before using dynlolib_LoadLibrary

void dynlolib_FreeLibrary(
  void *pLibrary
  ){
The argument pLibrary is the pointer, which was returned by the function dynlolib_LoadLibrary

[Contents]

2.15.3. dynlolib_GetFunctionByName

@c Get the entry point of a function by its name

This function can be used to get the entry point of a function of a loaded module specifying the name of the function.

void *dynlolib_GetFunctionByName(
  void *pLibrary,
  char *pszFunctionName
  ){
The argument pLibrary is the pointer, which was returned by the function dynlolib_LoadLibrary

The argument pszFunctionName is the ZCAR function name.

[Contents]

2.16. Other System Dependant Functions

@c Handling system specific file operations =abstract The file filesys.h contains file handling primitive functions. The reason for this module is to have all system specific file handling functions to be separated in a single file. All other modules use these functions that behave the same on Win32 platform as well as on UNIX. =end These functions are to be used by other parts of the program. They implement system specific operations, and other levels need not care about these system specific stuff.

The function names are prefixed usually with file_, some are prefixed with sys_.

=toc

[Contents]

2.16.1. file_fopen

@c Open a file

This is same as fopen.

VMS has some specialities when writing a file.

FILE *file_fopen(
  char *pszFileName,
  char *pszOpenMode
  ){

[Contents]

2.16.2. file_fclose

@c Close a file

This is same as fclose. Nothing special. This is just a placeholder.

void file_fclose(FILE *fp
  ){

[Contents]

2.16.3. file_size

@c return the size of a file

long file_size(char *pszFileName
  ){

[Contents]

2.16.4. file_time_accessed

@c return the time the file was last accessed

long file_time_accessed(char *pszFileName
  ){

[Contents]

2.16.5. file_time_modified

@c return the time the file was modified

long file_time_modified(char *pszFileName
  ){

[Contents]

2.16.6. file_time_created

@c return the time the file was created

long file_time_created(char *pszFileName
  ){

[Contents]

2.16.7. file_isdir

@c return true if the file is a directory

int file_isdir(char *pszFileName
  ){

[Contents]

2.16.8. file_isreg

@c return true if the file is a regular file (not directory)

int file_isreg(char *pszFileName
  ){

[Contents]

2.16.9. file_fileexists

@c return true if the file exists

int file_exists(char *pszFileName
  ){

[Contents]

2.16.10. file_truncate

@c truncate a file to a given length

It return 0 on success and -1 on error.

int file_truncate(FILE *fp,
                  long lNewFileSize
  ){

[Contents]

2.16.11. file_fgetc

@c Get a single character from a file

Nothing special, it is just a placeholder.

int file_fgetc(FILE *fp
  ){

[Contents]

2.16.12. file_ferror

@c ferror

Nothing special, it is just a placeholder.

int file_ferror(FILE *fp
  ){

[Contents]

2.16.13. file_fread

@c fread

Nothing special, it is just a placeholder.

int file_fread(char *buf,
               int size,
               int count,
               FILE *fp
  ){

[Contents]

2.16.14. file_fwrite

@c fwrite

Nothing special, it is just a placeholder.

int file_fwrite(char *buf,
               int size,
               int count,
               FILE *fp
  ){

[Contents]

2.16.15. file_fputc

@c Get a single character from a file

Nothing special, it is just a placeholder.

int file_fputc(int c, FILE *fp
  ){

[Contents]

2.16.16. file_setmode

@c Set the mode of a file stream to binary or to ASCII

Nothing special, it is just a placeholder. On UNIX this is doing nothing transparently.

void file_setmode(FILE *fp,
                  int mode
  ){

[Contents]

2.16.17. file_binmode

@c Set a file stream to binary mode
void file_binmode(FILE *fp
  ){

[Contents]

2.16.18. file_textmode

@c Set a file stream to text mode
void file_textmode(FILE *fp
  ){

[Contents]

2.16.19. file_flock

@c Lock a file

int file_flock(FILE *fp,
               int iLockType
  ){

[Contents]

2.16.20. file_lock

@c Lock a range of a file

int file_lock(FILE *fp,
              int iLockType,
              long lStart,
              long lLength
  ){

[Contents]

2.16.21. file_feof

@c Check end of file condition

Nothing special, it is just a placeholder.

int file_feof(FILE *fp
  ){

[Contents]

2.16.22. file_mkdir

@c Create a directory

This is the usual UNIX mkdir function. The difference is that the access code is always 0777 on UNIX which means that the user, group and others can read, write and execute the directory. If the permission needed is different from that you have to call the file_chmod function as soon as it becomes available.

The argument of the function is the name of the desired directory.

int file_mkdir(char *pszDirectoryName
  ){

[Contents]

2.16.23. file_rmdir

@c Remove a directory

This is the usual UNIX rmdir function.

The argument of the function is the name of the directory to be deleted.

int file_rmdir(char *pszDirectoryName
  ){

[Contents]

2.16.24. file_remove

@c Remove a file

Nothing special, it is just a placeholder. This function performs the UNIX remove functionality. This function also exists under WIN32, therefore this function is only a placeholder.

int file_remove(char *pszFileName
  ){

[Contents]

2.16.25. file_deltree

@c Delete a directory tree

int file_deltree(char *pszDirectoryName
  ){

[Contents]

2.16.26. file_MakeDirectory

@c Create a directory

This function is a bit out of the line of the other functions in this module. This function uses the file_mkdir function to create a directory. The difference is that this function tries to create a directory recursively. For example you can create the directory

/usr/bin/scriba

with a simple call and the function will create the directories /usr if it did not exist, then /usr/bin and finally /usr/bin/scriba The function fails if the directory can not be created because of access restrictions or because the directory path or a sub path already exists, and is not a directory.

The argument of the function is the name of the desired directory.

The function alters the argument replacing each \ character to /

In case of error, the argument may totally be destroyed.

int file_MakeDirectory(char *pszDirectoryName
  ){

[Contents]

2.16.27. file_opendir

@c Open a directory for listing

This function implements the opendir function of UNIX. The difference between this implementation and the UNIX version is that this implementation requires a DIR structure to be passed as an argument. The reason for this is that the Windows system calls do not allocate memory and pass return values in structures allocated by the caller. Because we did not want to implement memory allocation in these routines we followed the Windows like way.

The first argument pszDirectoryName is a ZCAR directory name to be scanned. The second argument is an allocated DIR structure that has to be valid until the file_closedir is called.

The second parameter under UNIX is not used. However to be safe and portable to Win32 the parameter should be handled with care.

DIR *file_opendir(char *pszDirectoryName,
                  tDIR *pDirectory
  ){

[Contents]

2.16.28. file_readdir

@c Read next item from a directory

This function is the implementation of the UNIX readdir

struct dirent *file_readdir(DIR *pDirectory
  ){

[Contents]

2.16.29. file_closedir

@c Close a directory opened for listing

void file_closedir(DIR *pDirectory
  ){

[Contents]

2.16.30. file_sleep

@c Sleep the process

void sys_sleep(long lSeconds
  ){

[Contents]

2.16.31. file_curdir

@c Get the current working directory

The first argument should point to a buffer having space for at least cbBuffer characters. The function will copy the name of the current directory into this buffer.

Return value is zero on success. If the current directory can not be retrieved or the buffer is too short the return value is -1.

int file_curdir(char *Buffer,
                unsigned long cbBuffer
  ){

[Contents]

2.16.32. file_chdir

@c Change the current working direcory

int file_chdir(char *Buffer
  ){

[Contents]

2.16.33. file_chown

@c Change owner of a file

This function implements the chown command of the UNIX operating system on UNIX and Windows NT. The first argument is the ZCHAR terminated file name. No wild card characters are allowed.

The second argument is the name of the desired new user. The function sets the owner of the file to the specified user, and returns zero if the setting was succesful. If the setting fails the function returns an error code. The error codes are:

COMMAND_ERROR_CHOWN_NOT_SUPPORTED COMMAND_ERROR_CHOWN_INVALID_USER COMMAND_ERROR_CHOWN_SET_OWNER

int file_chown(char *pszFile,
               char *pszOwner
  ){

[Contents]

2.16.34. file_getowner

@c Get the owner of a file

int file_getowner(char *pszFileName,
                  char *pszOwnerBuffer,
                  long cbOwnerBuffer
 ){

[Contents]

2.16.35. file_SetCreateTime

@c Set the creation time of a file

Note that this time value does not exist on UNIX and therefore calling this function under UNIX result error.

The argument to the function is the file name and the desired time in number of seconds since the epoch. (January 1, 1970. 00:00)

If the time was set the return value is zero. If there is an error the return value is the error code.

int file_SetCreateTime(char *pszFile,
                       long lTime
  ){

[Contents]

2.16.36. file_SetModifyTime

@c Set the modification time of a file

The argument to the function is the file name and the desired time in number of seconds since the epoch. (January 1, 1970. 00:00)

If the time was set the return value is zero. If there is an error the return value is the error code.

int file_SetModifyTime(char *pszFile,
                       long lTime
  ){

[Contents]

2.16.37. file_SetAccessTime

@c Set the access time of a file

The argument to the function is the file name and the desired time in number of seconds since the epoch. (January 1, 1970. 00:00)

If the time was set the return value is zero. If there is an error the return value is the error code.

int file_SetAccessTime(char *pszFile,
                       long lTime
  ){

[Contents]

2.16.38. file_gethostname

@c Get the name of the actual host

This function gets the name of the host that runs the program. The result of the function is positive if no TCP/IP protocol is available on the machine or some error occured.

In case of success the return value is zero.

int file_gethostname(char *pszBuffer,
                     long cbBuffer
  ){
The first argument should point to the character buffer, and the second argument should hold the size of the buffer in bytes.

[Contents]

2.16.39. file_gethost

@c Get host by name or by address

This function gets the struct hostent entry for the given address. The address can be given as a TQDN or as an IP octet tuple, like www.digital.com or 16.193.48.55

Optionally the address may contain a port number separated by : from the name or the IP number. The port number is simply neglected.

int file_gethost(char *pszBuffer,
                 struct hostent *pHost
  ){
pszBuffer should hold the name or the address of the target machine. This buffer is not altered during the function.

pHost should point to a buffer ready to hold the hostent information.

[Contents]

2.16.40. file_tcpconnect

@c Connect a socket to a server:port

This function tries to connect to the remote port of a remote server. The first argument of the function should be a pointer to SOCKET variable as defined in filesys.h or in the Windows header files. The second argument is a string that contains the name of the remote host, or the IP number of the remote host and the desired port number following the name separated by a colon. For example index.hu:80 tries to connect to the http port of the server index.hu. You can also write 16.192.80.33:80 to get a connection. The function automatically recognizes IP numbers and host names. The socket is created automatically calling the system function socket.

If the function successfully connected to the remote server the return value is zero. Otherwise the return value is the error code.

int file_tcpconnect(SOCKET *sClient,
                    char *pszRemoteSocket
  ){

[Contents]

2.16.41. file_tcpsend

@c send bytes to remote server via socket

int file_tcpsend(SOCKET sClient,
                 char *pszBuffer,
                 long cbBuffer,
                 int iFlags
  ){

[Contents]

2.16.42. file_tcprecv

@c receive bytes from remote server via socket

int file_tcprecv(SOCKET sClient,
                 char *pszBuffer,
                 long cbBuffer,
                 int iFlags
  ){

[Contents]

2.16.43. file_tcpclose

@c close a tcp connection

int file_tcpclose(SOCKET sClient
  ){

[Contents]

2.16.44. file_killproc

@c Kill a process

This function kills a process identified by the process ID (PID).

If the process is killed successfully the return value is zero, otherwise a positive value.

int file_killproc(long pid
  ){

[Contents]

2.16.45. file_fcrypt

@c Calculate encrypted password

This function implements the password encryption algorithm using the DES function. The first argument is the clear text password, the second argument is the two character salt value. This need not be zero terminated. The third argument should point to a 13 characters char array to get the encoded password. buff[13] will contain the terminating zchar upon return.

char *file_fcrypt(char *buf, char *salt, char *buff
  ){

[Contents]

2.16.46. file_CreateProcess

@c Run a new program

This function creates a new process using the argument as command line. The function does NOT wait the new process to be finished but returns the pid of the new process.

If the new process can not be started the return value is zero.

The success of the new process however can not be determined by the return value. On UNIX this value is generated by the fork system call and it still may fail to replace the executeable image calling exevp. By that time the new program creation is already in the newprocess and is not able to send back any error information to the caller.

The caller of this function should also check other outputs of the created process that of the pid is returned. For example if the execv call failed the process exit code is 1. This is usually an error information of a process.

long file_CreateProcess(char *pszCommandLine
  ){

[Contents]

2.16.47. file_CreateProcessEx

@c Run a new program and wait for it

This function starts a new process and starts to wait for the process. The caller can specify a timeout period in seconds until the function waits.

When the process terminates or the timeout period is over the function returns.

int file_CreateProcessEx(char *pszCommandLine,
                          long lTimeOut,
                          unsigned long *plPid,
                          unsigned long *plExitCode
  ){
Arguments:

The return value indicates the success of the execution of the new process:

Note that the behaviour of this function is slightly different on Windows NT and on UNIX. On Windows NT the function will return FILESYSE_NOTSTARTED when the new process can not be started. Under UNIX the process performs a fork() and then an execv. The fork() does not return an error value. When the execvp fails it is already in the new process and can not return an error code. It exists using the exit code 1. This may not be distinguished from the program started and returning an exit code 1.

[Contents]

2.17. Module Management

@c Module management

This file contains all the functions that handle external module management.

Note that all function names are prepended by modu_

[Contents]

2.17.1. modu_Init

@c Initialize the module management

This function allocates memory for the external module interface table and initializes the function pointers.

If the interface already exists and the function is called again it just silently returns.

The second argument can be zero or 1. The normal operation is zero. If iForce is true the function sets each function pointer to its initial value even if an initialization has already occured before.

This can be used in a rare case when a module modifies the interface table and want to reinitialize it to the original value. Be carefule with such constructions.

int modu_Init(pExecuteObject pEo,
              int iForce
  ){

[Contents]

2.17.2. modu_Preload

@c Preload the modules configured in the configuration file

int modu_Preload(pExecuteObject pEo
  ){

[Contents]

2.17.3. modu_GetModuleFunctionByName

@c Get a function entry point from a module

This function gets the entrypoint of a module function. This module can either be statically or dynamically linked to ScriptBasic. This function is one level higher than GetStaticFunctionByName or dynlolib_GetFunctionByName. The first argument to this function is not the module handle as returned by dynlolib_LoadLibrary but rather the pointer to the module description structure that holds other information on the modula. Namely the information that the module is loaded from dll or so, or if the module is linked to the interpreter static.

void *modu_GetModuleFunctionByName(
  pModule pThisModule,
  char *pszFunctionName
  ){

[Contents]

2.17.4. modu_GetStaticFunctionByName

@c Get a function entry point from a statically linked library

Get the entry point of a function that was linked to the ScriptBasic environment statically.

This is the counterpart of the function dynlolib_GetFunctionByName for functions in library linked static. This function searches the SLFST table for the named function and returns the entry point or NULL if there is no functions with the given name defined.

void *modu_GetStaticFunctionByName(
  void *pLibrary,
  char *pszFunctionName
  ){

[Contents]

2.17.5. modu_LoadModule

@c Load a module

This function loads a module and returns the module pointer to in the argument pThisModule. If the module is already loaded it just returns the module pointer.

When the function is called first time for a module it loads the module, calls the version negotiation function and the module initializer.

If module file name given in the argument pszLibrary file name is an absolute file name this is used as it is. Otherwise the different configured module directories are seached for the module file, and the operating system specific extension is also appended to the file name automatically.

If the caller does not need the pointer to the module the argument pThisModule can be NULL.

int modu_LoadModule(pExecuteObject pEo,
                    char *pszLibraryFile,
                    pModule **pThisModule
  ){

[Contents]

2.17.6. modu_GetFunctionByName

This function can be called to get the entry point of a function from an external module. If the module was not loaded yet it is automatically loaded.

int modu_GetFunctionByName(pExecuteObject pEo,
                           char *pszLibraryFile,
                           char *pszFunctionName,
                           void **ppFunction,
                           pModule **pThisModule
  ){

[Contents]

2.17.7. modu_UnloadAllModules

@c Unload all loaded modules

This function unloads all modules. This is called via the command finalizer mechanizm. If ever any module was loaded via a "declare sub" statement the command execution sets the command finalizer function pointer to point to this function.

int modu_UnloadAllModules(pExecuteObject pEo
  ){

In a multi-threaded environment this function calls the keeper function of the module and in case the keeper returns 1 the module is kept in memory, though the module finalizer function is called. This lets multi-thread external modules to keep themselfs in memory even those times when there is not any interpreter thread using the very module running.

In that case the module is put on the module list of the process SB object. That list is used to shut down the modules when the whole process is shut down.

If there is no process SB object (pEo->pEPo is NULL) then the variation is a single process single thread implementation of ScriptBasic. In this case this function first calls the module finalizer function that is usally called in multi-threaded environment every time an interpreter thread is about to finish and after this the module shutdown function is called, which is called in a multi-thread environment when the whole process is to be shut down. After that the module is unloaded even if the keeper function said that the module wants to stay in memory.

Don't worry about this: it is not abuse. The keeper function saying 1 means that the module has to stay in memory after the actual interpreter thread has finished until the process finishes. However in this very case the process also terminates.

Note: A one-process one-thread implementation may also behave like a multi thread implementation allocating a separate process SB object and a program object to run. Then it should inherit the support table and the execution object of the process SB object to the runnable program object. After running finish the runned program object and call the shutdown process for the process SB object. But that is tricky for a single thread implementation.

[Contents]

2.17.8. modu_UnloadModule

@c Unload the named module

This function unloads the named module. Note that this function is not called unless some extension module calls it to unload another module.

Currently there is no support for a module to unload itself.

int modu_UnloadModule(pExecuteObject pEo,
                      char *pszLibraryFile
  ){

[Contents]

2.17.9. modu_ShutdownModule

@c Shut down a module

This function calls the shutdown function of a module.

If the shutdown function performs well and returns SUCCESS this function also returns success. If the shutdown function returns error code it means that the module has running thread and thus can not be unloaded.

int modu_ShutdownModule(pExecuteObject pEo,
                        pModule pThisModule
  ){

[Contents]

2.18. Run Time Options Handling

@c Setting and getting option values

Each BASIC interpreter maintains a symbol table holding option values. These option values can be set using the BASIC command OPTION and an option value can be retrieved using the function OPTION().

An option has an integer value (long). Options are usually used to alter the behaviour of some commands or modules, altough BASIC programs are free to use any string to name an option. For example the option compare may alter the behavior of the string comparision function to be case sensitive or insensitive:

OPTION compare 1

Unitialized options are treated as being zero. There is no special option value for uninitialized options. In other words BASIC programs can not distinguish between unitialized options and options having the value zero.

This file contains the functions that handle the option symbol table. The option symbol tableis pointed by the field OptionsTable of the execution object. This pointer is initialized to be NULL, which means no options are available, or in other words all options are zero.

[Contents]

2.18.1. options_Reset

@c Clear an option data

Calling this function resets an option. This means that the memory holding the long value is released and the pointer that was pointing to it is set NULL.

int options_Reset(pExecuteObject pEo,
                  char *name
  ){

[Contents]

2.18.2. options_Set

@c Set option data

This function sets a long value for an option. If the option did not exist before in the symbol table it is inserted. If the symbol table was empty (aka OptionsTable pointed NULL) the symbol table is also created.

If the symbol already existed with some long value then the new value is stored in the already allocated place and thus the caller may store the pointer to the long returned by GetR and access possibly updated data without searching the table again and again.

int options_Set(pExecuteObject pEo,
                char *name,
                long value
  ){
The function returns zero if the option was set or 1 if there was a memory failure.

[Contents]

2.18.3. options_Get

@c Get option data

This function retrieves and returns the value of an option data.

long options_Get(pExecuteObject pEo,
                 char *name
  ){
The return value is the option value or zero in case the option is not set.

[Contents]

2.18.4. options_GetR

@c Get option data

This function retrieves and returns the value of an option data.

long *options_GetR(pExecuteObject pEo,
                 char *name
  ){
The return value is a long * pointer to the option value or NULL if the option is not set. If the caller sets the long variable pointed by the returned pointer the value of the option is changed directly.

[Contents]

2.19. Simple Pattern Matching

@c Simple Pattern Matching

=abstract A simple, non-regular expression pattern matching module mainly to perform file name pattern matching, like *.txt or file0?.bin and alikes. =end

This is a simple and fast pattern matching algorithm. This can be used when the matching does not require regular expression complexity and the processign on the other hand should be fast.

There are two major tasks implemented here. One is to match a string against a pattern. The second is to create a replacement string. When a pattern is matched by a string an array of string values are created. Each contains a substring that matches a joker character. Combining this array and a format string a replacement string can be created.

For example:

String = "mortal combat" Pattern = "mo?tal co*"

the joker characters are the ?, the space (matching one or more space) and the * character. They are matched by r, two spaces and mbat. If we use the format string

Format string = "$1u$2"

we get the result string rumbat. The format string can contain $n placeholders where n starts with 1 and is replaced by the actual value of the n-th joker character.

[Contents]

2.19.1. match_index

@c Return the joker index of the character

There are a few characters that can be used as joker character. These are

*#$@?&%!+/|<>

match_index returns the serial number of the character.

unsigned long match_index(char ch
){

[Contents]

2.19.2. InitSets

@c Initialize a set collection

Call this function to initialize a set collection. The argument should point to a MatchSets structure and the function fills in the default values.

void match_InitSets(pMatchSets pMS
  ){

[Contents]

2.19.3. ModifySet

@c Modify a joker set

This function can be used to modify a joker set. The first argument pMS points to the joker set collection. The second argument JokerCharacter specifies the joker character for which the set has to be modified.

The argument nChars and pch give the characters that are to be modified in the set. nChars is the number of characters in the character array pointed by pch.

The last argument fAction specifies what to do. The following constants can be used in logical OR.

TO_HEADER:

#define MATCH_ADDC 0x0001 //add characters to the set #define MATCH_REMC 0x0002 //remove characters from the set #define MATCH_INVC 0x0004 //invert the character #define MATCH_SNOJ 0x0008 //set becomes no-joker #define MATCH_SSIJ 0x0010 //set becomes single joker #define MATCH_SMUJ 0x0020 //set becomes multiple joker #define MATCH_NULS 0x0040 //nullify the set #define MATCH_FULS 0x0080 //fullify the set

*/

The function first checks if it has to modify the state of the joker character. If any of the bits MATCH_SNOJ, MATCH_SSIJ or MATCH_SMUJ is set in the field fAction the type of the set is modified.

If more than one bit of these is set then result is undefined. Current implementation checks these bits in a specific order, but later versions may change.

If the bit MATCH_NULS is set all the characters are removed from the set. If the bit MATCH_FULS is set all characters are put into the set.

If more than one bit of these is set then result is undefined. Current implementation checks these bits in a specific order, but later versions may change.

MATCH_NULS or MATCH_FULS can be used in a single call to initialize the set before adding or removing the specific characters.

The bits MATCH_ADDC, MATCH_REMC and MATCH_INVC can be used to add characters to the set, remove characters from the set or to invert character membership. The characters are taken from the character array pointed by the function argument pch.

If more than one bit of these is set then result is undefined. Current implementation checks these bits in a specific order, but later versions may change.

If none of these bits is set the value of the pointer pch is ignored.

It is no problem if a character is already in the set and is added or if it is not member of the set and is removed. Although it has no practical importance the array pointed by pch may contain a character many times.

void match_ModifySet(pMatchSets pMS,
                     char JokerCharacter,
                     int nChars,
                     unsigned char *pch,
                     int fAction
  ){

[Contents]

2.19.4. match

@c Match pattern to string FUNCTION:

match checks if pszString matches the pattern pszPattern. pszPattern is a string containing joker characters. These are:

 * matches one or more any character
 # matches one or more digit
 $ matches one or more alphanumeric character
 @ matches one or more alpha character
   (space) matches one or more spaces
 ? matches a single character

~x matches x even if x is pattern matching character or tilde

x matches character x unless it is a joker character

RETURN VALUE:

The function returns zero if no error occures and returns an error code in case some of the memory buffer does not have enough space. (Either pszBuffer or ParameterArray)

PARAMETERS:

pszPattern IN the pattern to match

--

cbPattern IN the number of characters in the pattern

--

pszString IN the string which is compared to the pattern

--

cbString IN the number of characters in the string

--

ParameterArray OUT is an uninitialized character pointer array. Upon return ParameterArray[i] points the string that matches the i-th joker character.

--

pcbParameterArray OUT is an uninititalized unsigned long array. Upon return pcbParameterArray[i] contains the length of the output parameter ParameterArray[i].

--

pszBuffer OUT should point to a buffer. The size of the buffer should be specified by cbBufferSize. A size equal

             cbString
is a safe size. The actual strings matching the joker characters will get into this buffer zero terminated one after the other:

--

cArraySize IN number of elements in the array ParameterArray

--

cbBufferSize IN size of the buffer pointed by pszBuffer

--

fCase IN pattern matching is performed case sensitive if this value if TRUE.

--

iResult OUT TRUE if pszString matches the pattern pszPattern. FALSE otherwise.

NOTE:

pszPattern and pszString are NOT changed.

If the function returns non-zero (error code) none of the output variables can be reliably used.

int match_match(char *pszPattern,
                unsigned long cbPattern,
                char *pszString,
                unsigned long cbString,
                char **ParameterArray,
                unsigned long *pcbParameterArray,
                char *pszBuffer,
                int cArraySize,
                int cbBufferSize,
                int fCase,
                pMatchSets pThisMatchSets,
                int *iResult
  ){

[Contents]

2.19.5. count

@c Count the joker characters in a pattern

This function counts the number of jokers in the string and returns it. This function should be used to calculate the safe length of the pszBuffer given as a parameter to match.

int match_count(char *pszPattern,
                unsigned long cbPattern
  ){

[Contents]

2.19.6. parameter

@c Fill parameters into format string

This function takes a format string and a string array and copies the format string replacing $0, $1 ... $n values with the appropriate string values given in the array pointed by ParameterArray.

RETURN VALUE:

The function returns zero if no error occures and returns an error code in case some of the memory buffer does not have enough space or invalid parameter is referenced.

PARAMETERS: pszFormat IN The format string containing the $i placeholders.

--

cbFormat IN The number of characters in the format string

--

ParameterArray IN string array so that ParameterArray[i] is to be inserted in place of the $i placeholders

--

pcbParameterArray IN array of unsigned long values. pcbParameterArray[i] gives the length of the i-th string parameter.

--

pszBuffer OUT buffer to put the result

--

cArraySize IN Number of parameters given in the ParameterArray

--

pcbBufferSize IN/OUT Available bytes in buffer pointed by pszBuffer. Upon return it contains the number of characters that were placed in the buffer.

--

NOTE:

If the function returns non-zero (error code) none of the output variables can be reliably used.

int match_parameter(char *pszFormat,
                    unsigned long cbFormat,
                    char **ParameterArray,
                    unsigned long *pcbParameterArray,
                    char *pszBuffer,
                    int cArraySize,
                    unsigned long *pcbBufferSize
  ){

[Contents]

2.19.7. size

@c Calculate the neccessary buffer size

Calculate the size of the output. The IN/OUT parameter cbBufferSize is increased by the number of needed characters.

The return value is zero if no error occured or the error code.

NOTE: cbBuffer size should be initialized to 0 if you want to get the size of the buffer needed.

int match_size(char *pszFormat,
               unsigned long cbFormat,
               unsigned long *pcbParameterArray,
               int cArraySize,
               int *cbBufferSize
  ){

[Contents]

2.20. Symbol Table Handling

The functions in this module implement a general purpose symbol table handling.

Generally a symbol table is a binding functionality that associates symbols with attributes. Symbols in this implementation is a zero terminated string, and the attribute is a void * pointer. This is a general approach that can be used to store and retrieve any kind of symbols.

The symbol table handling functions usually always return a void ** that can be modified to point to the actual structure storing the attributes of the symbol.

The internal structure of a symbol table is a hash table of PRIME elements (211). Each hash stores a binary table sorting the symbols.

[Contents]

2.20.1. sym_NewSymbolTable()

This function creates a new symbol table. Later this symbol table should be used to store and retrieve symbol information.
SymbolTable sym_NewSymbolTable(
  void* (*memory_allocating_function)(size_t,void *),
  void *pMemorySegment
  ){
The second argument should point to the memory allocating function that the symbol table creation process should use. The last argument is an pointer to a memory segment which is passed to the memory allocation function. The actual arguments of the memory allocation function fits the allocation function from the package alloc. However the defintion is general enough to use any other function.

[Contents]

2.20.2. sym_FreeSymbolTable()

This function should be used to release the memory allocated for a symbol table. This function releases all the memory that was allocated during symbol table creation and during symbol insertion.

Note that the memory allocated outside the symbol table handling routines is not released. This means that it is the caller responsibility to relase all memory that holds the actual values associated with the symbols.

void sym_FreeSymbolTable(
  SymbolTable table,
  void (*memory_releasing_function)(void *,void *),
  void *pMemorySegment
  ){

[Contents]

2.20.3. sym_TraverseSymbolTable()

This function can be used to traverse through all the symbols stored in a symbol table. The function starts to go through the symbols and for each symbol calls the function call_back_function.
void sym_TraverseSymbolTable(
  SymbolTable table,
  void (*call_back_function)(char *SymbolName, void *SymbolValue, void *f),
  void *f
  ){
The first argument is the symbol table to traverse. The second argument is the function to be called for each symbol. This function gets three arguments. The first is a pointer to the symbol string. The second is the pointer to the symbol arguments. The third argument is a general pointer which is passed to the function sym_TraverseSymbolTable.

Note that the call back function gets the pointer to the symbol arguments and not the pointer to the pointer to the symbol arguments, and therefore call back function can not change the actual symbol value pointer.

[Contents]

2.20.4. sym_LookupSymbol()

This function should be used to search a symbol or to insert a new symbol.

void **sym_LookupSymbol(
  char *s,                 /* zero terminated string containing the symbol                 */
  SymbolTable hashtable,   /* the symbol table                                             */
  int insert,              /* should a new empty symbol inserted, or return NULL instead   */
  void* (*memory_allocating_function)(size_t, void *),
  void (*memory_releasing_function)(void *, void *),
  void *pMemorySegment
  ){

This function usually returns a pointer to the void * pointer which is supposed to point to the structure, which actually holds the parameters for the symbol. When a symbol is not found in the symbol table the parameter insert is used to decide what to do. If this parameter is zero the function returns NULL. If this parameter is 1 the function creates a new symbol and returns a pointer to the void * pointer associated with the symbol.

If a new symbol is to be inserted and the function returns NULL means that the memory allocation function has failed.

If the new symbol was created and a pointer to the void * pointer is returned the value of the pointer is NULL. In other words:

void **a;

a = sym_LookupSymbol(s,table,1,mymema,mymemr,p);

if( a == NULL )error("memory releasing error"); if( *a == NULL )error("symbol not found");

[Contents]

2.20.5. sym_DeleteSymbol()

This function should be used to delete a symbol from the symbol table

int sym_DeleteSymbol(
  char *s,                 /* zero terminated string containing the symbol                 */
  SymbolTable hashtable,   /* the symbol table                                             */
  void (*memory_releasing_function)(void *, void *),
  void *pMemorySegment
  ){

This function searches the given symbol and if the symbol is found it deletes it from the symbol table. If the symbol was found in the symbol table the return value is zero. If the symbol was not found the return value is 1. This may be interpreted by the caller as an error or as a warning.

Note that this function only deletes the memory that was part of the symbol table. The memory allocated by the caller and handled via the pointer value usually returned by sym_LookupSymbol() should be released by the caller.

[Contents]

3. Embedding the Interpreter

ScriptBasic was designed from the very start to be embeddable. This means that C programmers having their own application can fairly easy compile and link ScriptBasic together with their application and have ScriptBasic as a built in scripting language in their application.

To do this the C programmer should use the C api implemented in the file `scriba.c'. In this chapter we detail the C API as a reference listing all callable function, but before that there are some sections that describe a bit the overall model of ScriptBasiC. The next section will talk about what object orientation means for ScriptBasic and how to follow this object oriented approach when programming a ScriptBasic extended application in C.

[Contents]

3.1. Object Oriented Model of ScriptBasic

Although ScriptBasic is implemented in pure C the coding and developing concept is rather object oriented. Such is the concept of the C API calling interface. This means that you have to deal with an abstract ScriptBasic program object when you want to execute a program. The structure of this program object is totally private to ScriptBasic and as a programmer embedding the interpreter you need not worry about it. The only action you have to do is to create such an object before doing any other function call calling scriba_new() and to destroy it after the BASIC program was executed and is not going to be used any more calling the function scriba_destroy().

The object is stored in memory and this piece of memory is allocated by ScriptBasic. The function scriba_new() allocates this memory and returns a pointer to this "object". Later this pointer has to be used to refer to this object.

Because C is not object oriented the functions called should explicitly get this pointer as first argument. When programming C++ the class pointer is used to access the class methods, and that also implicitly passes the object pointer to the method. The pointer passing code is generated by the C++ compiler. When calling ScriptBasic API the programmer has to store the "object" pointer and pass it as first argument to any function.

The type of the object pointer is pSbProgram.

[Contents]

3.2. Sample Embedding

The best way of learning is learning by example. Therefore here we will discuss the most obvious embedding application: the command line variation of ScriptBasic. The command line variation of ScriptBasic can be found in the file `scribacmd.c' in the source directory `variations/standard'. You may also find there a file named `basiccmd.c' that contains the code that was used before the scriba_ C API was introduced. Have a look at it and bless God you have the new API.

In this section we will present the code from the file, but for brevity some code will not be copied to here. Also note that the code is copied and as the program develops the actual code may change while the one copied here most probably remains the same. (The API definitions are not "hand" copied, but rather taken from the C file when the documentation is compiled, so whenever the API changes the new documentation recompiled reflects the change.)

[Contents]

3.2.1. Include Header Files

The main program of the standalone variation is implemented in the file `scribacmd.c'. If you look at the start of this program you can see that it start including the file `scriba.h'. This file contains all definitions that are needed by the C compiler to compile the code calling the scriba_ API functions.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#include "../../getopt.h" #include "../../scriba.h"

#include "../../basext.h"

In case you miss the file `scriba.h' then you should generate it using the program `headerer.pl' or `headerer.bas' which are also part of the distribution. C header files in ScriptBasic are not maintained by themselves. The text is rather maintained inside the C code, and is extracted using one of these scripts. (They do the same task.) The reason for this is to eliminate the double maintenance of function prototypes in the C and in the header file.

The file `basext.h' is also #included by the main program. This is not usually needed by other main programs. The standalone version needs it for the sole reason to be able to print out on the usage screen the extension module interface version that the actual interpreter support. This version is defined in the macro INTERFACE_VERSION. Other than that there is no reason or need to include any other file than `scriba.h'.

[Contents]

3.2.2. Function main(), Variable Declarations

After some macro definitions that I do not list here the start of the function main comes:

main(int argc, char *argv[], char *env[]){

. . . . variable declarations . . . . .

pSbProgram pProgram;

. . . . variable declarations . . . . .

This piece of code defines the local variables. The variable pProgram will hold the pointer to the program object or simply saying pProgram is the program object variable. The other variables are not described here, their names and the code where they are used should make their usage and purpose clear.

[Contents]

3.2.3. Command Line handling

The next piece of code is the command line option handling. This is not described here, because this is nothing special to ScriptBasic.

[Contents]

3.2.4. Creating Program Object

The real interesting piece of code starts at this fragment:

  pProgram = scriba_new(malloc,free);

This code first of all creates a new program object calling scriba_new(). The two arguments to this function should be two pointers to two functions that will be used by the interpreter to allocate and release memory. The two functions should exhibit behavior like malloc and free. All lower layers inherit these functions and call these functions to get and to release memory. In case there are more than one program objects used they may one after the other or in different threads at the same time use the same functions or different functions. The only requirement is that the functions pointed by the arguments should be thread safe if different threads are started executing ScriptBasic interpreters. Thread safeness also may play important role when some extension module is multi-thread.

[Contents]

3.2.5. Loading Configuration

  scriba_LoadConfiguration(pProgram,pszForcedConfigurationFileName);

The next function call is scriba_LoadConfiguration(). This function gets the configuration information from the compiled configuration file. The details of configuration files is detailed in section Configuration File Handling.

It is not a must to call this function from the embedding application, but with some few exceptions most embedding applications will do. Most ScriptBasic interpreter program objects need configuration information to be executed successfully.

To get the configuration for a program object there are two ways:

The second approach is necessary to gain performance in case there are several interpreters running one after the other or in parallel threads in the same process. If all these interpreters need the same configuration information they can use the same memory data, because the configuration information is not alterable by the interpreter. This way you can save the successive file loading and the extra memory space.

When the configuration is inherited it is very important that the program object holding the configuration is not destroyed while any of the inheriting program objects are alive. Therefore such applications usually create a program object that does not run any program but holds the configuration information loaded calling scriba_LoadConfiguration() and that exists for the life of the whole process.

For more information how to inherit the configuration information see the function scriba_InheritConfiguration().

[Contents]

3.2.6. Loading Internal Preprocessors

The next function call loads the internal preprocessors. This is not necessary needed for the applications. This is needed only if there is some way to load an internal preprocessor before the program code is loaded. In the case of the standalone variation the command line option `-i' can specify internal preprocessor names.

  iError = scriba_LoadInternalPreprocessor(pProgram,pszIPreproc);
  if( iError ){
    report_report(stderr,"",0,iError,REPORT_ERROR,&iErrorCounter,NULL,&fErrorFlags);
    ERREXIT;
    }

If this function is not called by the main program for the program object the interpreter will still process internal preprocessors that are referenced by the program code using the statement use.

[Contents]

3.2.7. Setting the File Name

To successfully load a program there is a need for the name of the file that holds the source code. To specify this file name for a program object the function scriba_SetFileName() should be used.

  scriba_SetFileName(pProgram,szInputFile);

This is the usual way getting the program into the program object, but there is another way. Some applications do not store the BASIC program string in text files, but in some other mass media, like relation database. In that case the application has to load the program string into the application memory and call scriba_LoadProgramString() to compile and execute the BASIC program. However in this case the program should not contain any include or import statement.

[Contents]

3.2.8. Using the Cache

ScriptBasic is powerful being able to store the compiled version of BASIC programs in binary format in file. The next piece of code checks if there is already a compiled version of the BASIC program in the configured cache directory calling the function scriba_UseCacheFile()

  if( scriba_UseCacheFile(pProgram) == SCRIBA_ERROR_SUCCESS ){
    if( (iError = scriba_LoadBinaryProgram(pProgram)) != 0 ){
      ERREXIT;
      }

The function scriba_LoadBinaryProgram() loads the binary program.

[Contents]

3.2.9. Run External Preprocessors

If the cache file is not usable then the source text has to be loaded and compiled. Before that the external preprocessors has to do their jobs if there is any. To do this the function scriba_RunExternalPreprocessor() is called.

    iError=scriba_RunExternalPreprocessor(pProgram,pszEPreproc);

It is not a must for an application to call this function. Some application may require the user to write their program in pure BASIC and not to use any preprocessor.

[Contents]

3.2.10. Loading the Source Program

When the external preprocessors are done the program has to be loaded calling the function scriba_LoadSourceProgram().

    if( scriba_LoadSourceProgram(pProgram) )ERREXIT;

This call loads the program source and processes it up to the point of execution. All interpreter processing, file reading, inclusion of other files, lexical, syntactical analysis and program building is done calling this function.

[Contents]

3.2.11. Saving Binary File

Before executing the loaded program the application may save the compiled binary version of the program. This can be done calling the function scriba_SaveCode() to save the binary code into a specific file, or calling the function scriba_SaveCacheFile() to save the binary code into the cache with an automatically calculated name.

    if( szOutputFile ){
      if( isCoutput )
        scriba_SaveCCode(pProgram,szOutputFile);
      else
        scriba_SaveCode(pProgram,szOutputFile);
      if( !execute )exit(0);
      }
     if( ! nocache )scriba_SaveCacheFile(pProgram);
 

[Contents]

3.2.12. Execute the BASIC Program

This is the step that all the previous steps were done. This function call to scriba_Run() executes the program.

  if( iError=scriba_Run(pProgram,CmdLinBuffer) ){
 report_report(stderr,"",0,iError,REPORT_ERROR,&iErrorCounter,NULL,&fErrorFlags);
    ERREXIT;
    }

[Contents]

3.2.13. Destroying the Program Object

Although the program is executed and has finished still there is something extremely important that the application has to do: clean-up.

  scriba_destroy(pProgram);

The call to scriba_destroy releases all resources (memory) that were allocated to execute the BASIC program. Calling this function is not mandatory in a single process application that executes only one interpreter and exits from the process. In that case not calling this function the operating system is going to release the resources. However this is a bad practice.

[Contents]

3.3. ScriptBasic C API

[Contents]

3.3.1. scriba_new()

To create a new SbProgram object you have to call this function. The two arguments should point to malloc and free or similar functions. All later memory allocation and releasing will be performed using these functions.

Note that this is the only function that does not require a pointer to an SbProgram object.

pSbProgram scriba_new(void * (*maf)(size_t),
                      void   (*mrf)(void *)
  ){

[Contents]

3.3.2. scriba_destroy()

After a ScriptBasic program was successfully execued and there is no need to run it anymore call this function to release all memory associated with the code.

void scriba_destroy(pSbProgram pProgram
  ){

[Contents]

3.3.3. scriba_NewSbData()

Allocate and return a pointer to the allocated SbData structure.

This structure can be used to store ScriptBasic variable data, long, double or string. This function is called by other functions from this module. Usually the programmer, who embeds ScriptBasic will rarely call this function directly. Rather he/she will use scriba_NewSbLong() (as an example) that creates a variable capable holding a long, sets the type to be SBT_LNG and stores initial value.

See also scriba_NewSbLong(), scriba_NewSbDouble(), scriba_NewSbUndef(), scriba_NewSbString(), scriba_NewSbBytes(), scriba_DestroySbData().

pSbData scriba_NewSbData(pSbProgram pProgram
  ){

[Contents]

3.3.4. scriba_InitSbData()

This function initializes an SbData structure to hold undef value. This function should be used to initialize an allocated SbData memory structure. This function internally is called by scriba_NewSbData().

See also scriba_NewSbLong(), scriba_NewSbDouble(), scriba_NewSbUndef(), scriba_NewSbString(), scriba_NewSbBytes(), scriba_DestroySbData().

void scriba_InitSbData(pSbProgram pProgram,
                         pSbData p
  ){

[Contents]

3.3.5. scriba_UndefSbData()

This function sets an SbData structure to hold the undefined value.

This function should should not be used instead of scriba_InitSbData(). While that function should be used to inititalize the memory structure this function should be used to set the value of an alreasdy initialized and probably used SbData variable to undef.

The difference inside is that if the SbData structure is a string then this function releases the memory occupied by the string, while scriba_InitSbData() does not.

See also scriba_NewSbLong(), scriba_NewSbDouble(), scriba_NewSbUndef(), scriba_NewSbString(), scriba_NewSbBytes(), scriba_DestroySbData().

void scriba_UndefSbData(pSbProgram pProgram,
                        pSbData p
  ){

[Contents]

3.3.6. scriba_NewSbLong()

This function allocates and returns a pointer pointing to a structure of type SbData holding a long value. If the allocation failed the return value is NULL. If the memory allocation was successful the allocated structure will have the type SBT_LONG and will hold the initial value specified by the argument lInitValue.

pSbData scriba_NewSbLong(pSbProgram pProgram,
                         long lInitValue
  ){
See also scriba_NewSbLong(), scriba_NewSbDouble(), scriba_NewSbUndef(), scriba_NewSbString(), scriba_NewSbBytes(), scriba_DestroySbData().

[Contents]

3.3.7. scriba_NewSbDouble()

This function allocates and returns a pointer pointing to a structure of type SbData holding a double value. If the allocation failed the return value is NULL. If the memory allocation was successful the allocated structure will have the type SBT_DOUBLE and will hold the initial value specified by the argument dInitValue.

pSbData scriba_NewSbDouble(pSbProgram pProgram,
                           double dInitValue
  ){
See also scriba_NewSbLong(), scriba_NewSbDouble(), scriba_NewSbUndef(), scriba_NewSbString(), scriba_NewSbBytes(), scriba_DestroySbData().

[Contents]

3.3.8. scriba_NewSbUndef()

This function allocates and returns a pointer pointing to a structure of type SbData holding an undef value. If the allocation failed the return value is NULL. If the memory allocation was successful the allocated structure will have the type SBT_UNDEF.

pSbData scriba_NewSbUndef(pSbProgram pProgram
  ){
See also scriba_NewSbLong(), scriba_NewSbDouble(), scriba_NewSbUndef(), scriba_NewSbString(), scriba_NewSbBytes(), scriba_DestroySbData().

[Contents]

3.3.9. scriba_NewSbString()

This function allocates and returns a pointer pointing to a structure of type SbData holding a string value. If the allocation failed the return value is NULL. If the memory allocation was successful the allocated structure will have the type SBT_STRING and will hold the initial value specified by the argument pszInitValue.

pSbData scriba_NewSbString(pSbProgram pProgram,
                           char *pszInitValue
  ){
Note on ZCHAR termination:

The init value pszInitValue should be a zchar terminated string. Note that ScriptBasic internally stores the strings as series byte and the length of the string without any terminating zchar. Therefore the length of the string that is stored should have been strlen(pszInitValue). This does not contain the terminating zchar.

In reality however we allocate an extra byte that stores the zchar, but the size of the string is one character less. Therefore ScriptBasic routines will recognize the size of the string correct and also the caller can use the string using the macro scriba_GetString as a zchar terminated C string. This requires an extra byte of storage for each string passed from the embedding C application to ScriptBasic, but saves a lot of hedeache and also memory copy when the string has to be used as a zchar terminated string.

See also scriba_NewSbLong(), scriba_NewSbDouble(), scriba_NewSbUndef(), scriba_NewSbString(), scriba_NewSbBytes(), scriba_DestroySbData().

[Contents]

3.3.10. scriba_NewSbBytes()

This function allocates and returns a pointer pointing to a structure of type SbData holding a string value. If the allocation failed the return value is NULL. If the memory allocation was successful the allocated structure will have the type SBT_STRING and will hold the initial value specified by the argument pszInitValue of the length len.

pSbData scriba_NewSbBytes(pSbProgram pProgram,
                          unsigned long len,
                          unsigned char *pszInitValue
  ){
This function allocates len+1 number of bytes data and stores the initial value pointed by pszInitValue in it.

The extra plus one byte is an extra terminating zero char that may help the C programmers to handle the string in case it is not binary. Please also read the not on the terminating ZChar in the function scriba_NewSbString().

See also scriba_NewSbLong(), scriba_NewSbDouble(), scriba_NewSbUndef(), scriba_NewSbString(), scriba_NewSbBytes(), scriba_DestroySbData().

[Contents]

3.3.11. scriba_DestroySbData()

Call this function to release the memory that was allocated by any of the NewSbXXX functions. This function releases the memory and also cares to release the memory occupied by the characters in case the value had the type SBT_STRING.

void scriba_DestroySbData(pSbProgram pProgram,
                          pSbData p
  ){
See also scriba_NewSbLong(), scriba_NewSbDouble(), scriba_NewSbUndef(), scriba_NewSbString(), scriba_NewSbBytes(), scriba_DestroySbData().

[Contents]

3.3.12. scriba_PurgeReaderMemory()

Call this function to release all memory that was allocated by the reader module. The memory data is needed so long as long the lexical analyzer has finished.
void scriba_PurgeReaderMemory(pSbProgram pProgram
  ){

[Contents]

3.3.13. scriba_PurgeLexerMemory()

void scriba_PurgeLexerMemory(pSbProgram pProgram
  ){

[Contents]

3.3.14. scriba_PurgeSyntaxerMemory()

void scriba_PurgeSyntaxerMemory(pSbProgram pProgram
  ){

[Contents]

3.3.15. scriba_PurgeBuilderMemory()

void scriba_PurgeBuilderMemory(pSbProgram pProgram
  ){

[Contents]

3.3.16. scriba_PurgePreprocessorMemory()

This function purges the memory that was needed to run the preprocessors.

void scriba_PurgePreprocessorMemory(pSbProgram pProgram
  ){

[Contents]

3.3.17. scriba_PurgeExecuteMemory()

This function purges the memory that was needed to execute the program, but before that it executes the finalization part of the execution.

void scriba_PurgeExecuteMemory(pSbProgram pProgram
  ){

[Contents]

3.3.18. scriba_SetFileName()

Call this function to set the file name where the source informaton is. This file name is used by the functions scriba_LoadBinaryProgram() and scriba_LoadSourceProgram as well as error reporting functions to display the location of the error.

int scriba_SetFileName(pSbProgram pProgram,
                       char *pszFileName
  ){
The argument pszFileName should be zchar terminated string holding the file name.

[Contents]

3.3.19. scriba_GettingConfiguration()

scriba_LoadConfiguration() and scriba_InheritConfiguration() can be used to specify configuration information for a ScriptBasic program. Here we describe the differences and how to use the two functions for single-process single-basic and for single-process multiple-basic applications.

To execute a ScriptBasic program you usually need configuration information. The configuration information for the interpreter is stored in a file. The function scriba_LoadConfiguration() reads the file and loads it into memory into the SbProgram object. When the object is destroyed the configuration information is automatically purged from memory.

Some implementations like the Eszter SB Engine variation of ScriptBasic starts several interpreter thread within the same process. In this case the configuration information is read only once and all the running interpreters share the same configuration information.

To do this the embedding program has to create a pseudo SbProgram object that does not run any ScriptBasic program, but is used only to load the configuration information calling the function scriba_LoadConfiguration(). Other SbProgram objects that do intepret ScriptBasic program should inherit this configuration calling the function scriba_InheritConfiguration(). When a SbProgram object is destroyed the configuration is not destroyed if that was inherited belonging to a different object. It remains in memory and can later be used by other intrepreter instances.

Inheriting the configuration is fast because it does not require loading the configuration information from file. This is essentially sets a pointer in the internal interpreter structure to point to the configuration information held by the other object and all the parallel running interpreters structures point to the same piece of memory holding the common configuration information.

See the configuration handling functions scriba_LoadConfiguration() and scriba_InheritConfiguration().

[Contents]

3.3.20. scriba_LoadConfiguration()

This function should be used to load the configuration information from a file.

The return value is zero on success and the error code when error happens.

int scriba_LoadConfiguration(pSbProgram pProgram,
                             char *pszForcedConfigurationFileName
  ){

[Contents]

3.3.21. scriba_InheritConfiguration()

Use this function to get the configuration from another program object.

The return value is zero on success and error code if error has happened.

int scriba_InheritConfiguration(pSbProgram pProgram,
                                pSbProgram pFrom
  ){

[Contents]

3.3.22. scriba_InitModuleInterface()

Initialize the Support Function Table of a process level ScriptBasic program object to be inherited by other program objects. If you read it first time, read on until you understand what this function really does and rather how to use it!

This is going to be a bit long, but you better read it along with the documentation of the function scriba_InheritModuleInterface().

This function is needed only for programs that are

You most probably know that modules can access system and ScriptBasic fucntions via a call-back table. That is a huge struct containing pointers to the functions that ScriptBasic implements. This is the ST (aka support table).

This helps module writers to write system independent code as well as to access ScriptBasic functions easily. On the other hand modules are also free to alter this table and because many functions, tough not all are called via this table by ScriptBasic itself a module may alter the core behavior of ScriptBasic.

For this reason each interpreter has its own copy of ST. This means that if an interpreter alters the table it has no effect on another interpreter running in the same process in anther thread.

This is fine so far. How about modules that run asynchronous threads? For example the very first interpter thread that uses the module mt starts in the initialization a thread that later deletes all sessions that time out. This thread lives a long life.

The thread that starts the worker thread is an interpreter thread and has its own copy of the ST. The thread started asynchronous however should not use this ST because the table is purged from memory when the interpreter instance it blelonged to finishes.

To have ST for worker threads there is a need for a program object that is not purged from memory so long as long the process is alive. Fortunately there is such an object: the configuration program object. Configuration is usually read only once by multi-thread implementations and the same configuration information is shared by the serveral threads. The same way the several program objects may share a ST.

The difference is that configuration is NOT altered by the interpreter or by any module in any way but ST may. Thus each execution object has two pointers: pST and pSTI. While pST points to the support table that belongs to the interpreter instance the secondpointer pSTI points to a ST that is global for the whole process and is permanent. This ST is to be used by worker threads and should not be altered by the module without really good reason.

Thus: Don't call this function for normal program objects! For usualy program objects module interface is automatically initialized when the first module function is called. Call this function to initialize a ST for a pseudo program object that is never executed but rather used to inherit this ST for worker threads.

int scriba_InitModuleInterface(pSbProgram pProgram
  ){

[Contents]

3.3.23. scriba_InheritModuleInterface()

Inherit the support function table (ST) from another program object.

Note that the program object is going to initialize its own ST the normal way. The inherited ST will only be used by worker threads that live a long life and may exist when the initiating interpreter thread already exists.

For further information please read the description of the function scriba_InitModuleInterface().

int scriba_InheritModuleInterface(pSbProgram pProgram,
                                  pSbProgram pFrom
  ){

[Contents]

3.3.24. scriba_InheritExecuteObject()

int scriba_InheritExecuteObject(pSbProgram pProgram,
                                  pSbProgram pFrom
  ){

[Contents]

3.3.25. scriba_SetProcessSbObject()

Use this program in multi-thread environment to tell the actual interpreter which object is the process level pseudo object that

If the embeddingprogram calls this function there is no need to call scriba_InheritConfiguration() and scriba_InheritModuleInterface(). This function call does all those tasks and also other things.

int scriba_SetProcessSbObject(pSbProgram pProgram,
                              pSbProgram pProcessObject
  ){

[Contents]

3.3.26. scriba_ShutdownMtModules()

A multi threaded application should call this function for the process SB object when the process finishes. Calling this function will call each of the shutdown functions of those external modules that decided to keep in memory and export the shutdown function named shutmodu. This allows these modules to gracefully shut down their operation. As an example cached data can be written to disk, or database connections can be closed.

int scriba_ShutdownMtModules(pSbProgram pProgram
  ){

[Contents]

3.3.27. scriba_SetCgiFlag()

You can call this function to tell the reporting subsystem that this code runs in a CGI environment and therefore it should format error messages according to the CGI standard sending to the standard output including HTTP headers and HTML code pieces.

void scriba_SetCgiFlag(pSbProgram pProgram
  ){

[Contents]

3.3.28. scriba_SetReportFunction()

This function should be used to set the report function for a program. The report function is used to send info, warning, error, fatal and internal error messages to the user.

In case you want to implement a specific report function see the sample implementation in the file report.c. The documentation of the function report_report describes not only the details of the sample implementation but also the implementation requests for other reporting functions.

void scriba_SetReportFunction(pSbProgram pProgram,
                              void *fpReportFunction
  ){

[Contents]

3.3.29. scriba_SetReportPointer()

This pointer will be passed to the reporting function. The default reporting uses this pointer as a FILE * pointer. The default value for this pointer is stderr.

Other implementations of the reporting function may use this pointer according their needs. For example the WIN32 IIS ISAPI implementation uses this pointer to point to the extension controll block structure.

void scriba_SetReportPointer(pSbProgram pProgram,
                             void *pReportPointer
  ){

[Contents]

3.3.30. scriba_SetStdin()

You can call this function to define a special standard input function. This pointer should point to a function that accepts a void * pointer as argument. Whenever the ScriptBasic program tries to read from the standard input it calls this function pasing the embedder pointer as argument.

If the stdin function is not defined or the parameter is NULL the interpreter will read the normal stdin stream.

void scriba_SetStdin(pSbProgram pProgram,
                     void *fpStdinFunction
  ){

[Contents]

3.3.31. scriba_SetStdout()

You can call this function to define a special standard output function. This pointer should point to a function that accepts a (char, void *) arguments. Whenever the ScriptBasic program tries to send a character to the standard output it calls this function. The first parameter is the character to write, the second is the embedder pointer.

If the standard output function is not defined or the parameter is NULL the interpreter will write the normal stdout stream.

void scriba_SetStdout(pSbProgram pProgram,
                      void *fpStdoutFunction
  ){

[Contents]

3.3.32. scriba_SetEmbedPointer()

This function should be used to set the embed pointer.

The embed pointer is a pointer that is not used by ScriptBasic itself. This pointer is remembered by ScriptBasic and is passed to call-back functions. Like the standard input, output and environment functions that the embedding application may provide this pointer is also available to external modules implemented in C or other compiled language in DLL or SO files.

The embedder pointer should usually point to the struct of the thread local data. For example the Windows NT IIS variation of ScriptBasic sets this variable to point to the extension control block.

If this pointer is not set ScriptBasic will pass NULL pointer to the extensions and to the call-back function.

void scriba_SetEmbedPointer(pSbProgram pProgram,
                            void *pEmbedder
  ){

[Contents]

3.3.33. scriba_SetEnvironment()

You can call this function to define a special environment query function. This pointer should point to a function that accepts a (void *, char *, long ) arguments.

Whenever the ScriptBasic program tries to get the value of an enviroment variable it calls this function. The first argument is the embedder pointer.

The second argument is the name of the environment variable to retrieve or NULL.

The third argument is either zero or is the serial number of the environment variable.

ScriptBasic never calls this function with both specifying the environment variable name and the serial number.

The return value of the function should either be NULL or should point to a string that holds the zero character terminated value of the environment variable. This string is not changed by ScriptBasic.

If the special environment function is not defined or is NULL ScriptBasic uses the usual environment of the process calling the system functiongetenv.

void scriba_SetEnvironment(pSbProgram pProgram,
                           void *fpEnvirFunction
  ){
For a good example of a self-written environment function see the source of the Eszter SB Engine that alters the environment function so that the ScriptBasic programs feel as if they were executed in a real CGI environment.

[Contents]

3.3.34. scriba_LoadBinaryProgram()

Use this function to load ScriptBasic program from a file that is already compiled into internal form.

The return value is the number of errors (hopefully zero) during program load.

int scriba_LoadBinaryProgram(pSbProgram pProgram
  ){
Before calling this function the function scriba_SetFileName() should have been called specifying the file name.

[Contents]

3.3.35. scriba_InheritBinaryProgram()

Use this function in application that keeps the program code in memory.

int scriba_InheritBinaryProgram(pSbProgram pProgram,
                                pSbProgram pFrom
  ){

The function inherits the binary code from the program object pFrom. In server type applications the compiled binary code of a BASIC program may be kept in memory. To do this a pseudo program object should be created that loads the binary code and is not destroyed.

The program object used to execute the code should inherit the binary code from this pseudo object calling this function. This is similar to the configuration inheritance.

[Contents]

3.3.36. scriba_LoadInternalPreprocessor()

This function can be used by embedding applications to load an internal preprocessor into the interpereter. Note that preprocessors are usually loaded by the reader module when a preprocess statement is found. However some preprocessors in some variation of the interpreter may be loaded due to configuration or command line option and not because the source requests it.

The preprocessors that are requested to be loaded because the source contains a preprocess line usually implement special language fetures. The preprocessors that are loaded independent of the source because command line option or some other information tells the variation to call this function are usually debuggers, profilers.

(To be honest, by the time I write it there is no any internal preprocessors developed except the test one, but the statement above will become true.)

int scriba_LoadInternalPreprocessor(pSbProgram pProgram,
                            char *ppszPreprocessorName[]
  ){
The first argument is the program object. If the program object does not have a preprocessor object the time it is called the preprocessor object is created and initiated.

The second argument is the array of names of the preprocessor as it is present in the configuration file. This is not the name of the DLL/SO file, but rather the symbolic name, which is associated with the file. The final element of the array has to be NULL.

The return value is zero or the error code.

[Contents]

3.3.37. scriba_ReadSource()

Loads the source code of a ScriptBasic program from a text file.

The return code is the number of errors happened during read.

int scriba_ReadSource(pSbProgram pProgram
  ){
Do not get confused! This function only reads the source. Does not compile it. You will usually need scriba_LoadSourceProgram() that does reading, analyzing, building and all memory releases leaving finally a ready-to-run code in memory.

Before calling this function the function scriba_SetFileName() should have been called specifying the file name.

See also scriba_ReadSource(), scriba_DoLexicalAnalysis(), scriba_DoSyntaxAnalysis(), scriba_BuildCode().

[Contents]

3.3.38. scriba_DoLexicalAnalysis()

This function performs lexical analysis after the source file has beed read.

This function is rarely needeed by applicationdevelopers. See scriba_LoadSourceProgram() instead.

int scriba_DoLexicalAnalysis(pSbProgram pProgram
  ){
See also scriba_ReadSource(), scriba_DoLexicalAnalysis(), scriba_DoSyntaxAnalysis(), scriba_BuildCode().

[Contents]

3.3.39. scriba_DoSyntaxAnalysis()

This function performs syntax analysis after the lexical analysis has been finished.

This function is rarely needeed by applicationdevelopers. See scriba_LoadSourceProgram() instead.

int scriba_DoSyntaxAnalysis(pSbProgram pProgram
  ){
See also scriba_ReadSource(), scriba_DoLexicalAnalysis(), scriba_DoSyntaxAnalysis(), scriba_BuildCode().

[Contents]

3.3.40. scriba_BuildCode()

This function builds the finall ready-to-run code after the syntax analisys has been finished.

This function is rarely needeed by applicationdevelopers. See scriba_LoadSourceProgram() instead.

int scriba_BuildCode(pSbProgram pProgram
  ){
See also scriba_ReadSource(), scriba_DoLexicalAnalysis(), scriba_DoSyntaxAnalysis(), scriba_BuildCode().

[Contents]

3.3.41. scriba_IsFileBinaryFormat()

This function decides if a file is a correct binary format ScriptBasic code file and returns true if it is binary. If the file is a ScriptBasic source file or an older version binary of ScriptBasic or any other file it returns zero.

This function just calls the function build_IsFileBinaryFormat

int scriba_IsFileBinaryFormat(pSbProgram pProgram
  ){

[Contents]

3.3.42. scriba_GetCacheFileName()

Calculate the name of the cache file for the given source file name and store the calculated file name in the program object.

int scriba_GetCacheFileName(pSbProgram pProgram
  ){
The program returns zero or the error code. It returns SCRIBA_ERROR_FAIL if there is no cache directory configured.

The code uses a local buffer of length 256 bytes. The full cached file name should fit into this otherwise the program will return SCRIBA_ERROR_BUFFER_SHORT.

The code does not check if there exists an appropriate cache directory or file. It just calculates the file name.

[Contents]

3.3.43. scriba_UseCacheFile()

Call this function to test that the cache file is usable. This function calls the function scriba_GetCacheFileName() to calculate the cache file name.

If

then this function alters the source file name property (pszFileName) of the program object so that the call to scriba_LoadBinaryProgram() will try to load the cache file.

int scriba_UseCacheFile(pSbProgram pProgram
  ){
The function returns zero or the error code. The function returns SCRIBA_ERROR_FAIL in case the cache file is old, or not valid. Therefore returning a positive value does not neccessarily mean a hard error.

[Contents]

3.3.44. scriba_SaveCacheFile()

Call this function to generate a cache file after a successful program compilation.

int scriba_SaveCacheFile(pSbProgram pProgram
  ){
The function returns zero (SCRIBA_ERROR_SUCCESS) if there was no error. This does not mean that the cache file was saved. If there is no cache directory configured doing nothing is success.

Returning any positive error code means that ScriptBasic tried to write a cache file but it could not.

[Contents]

3.3.45. scriba_RunExternalPreprocessor()

This function should be called to execute external preprocessors.

This function does almost nothing else but calls the function epreproc().

int scriba_RunExternalPreprocessor(pSbProgram pProgram,
                                   char **ppszArgPreprocessor
  ){
The argument ppszArgPreprocessor should point to a string array. This string array should contain the configured names of the preprocessors that are applied one after the other in the order they are listed in the array.

Note that this array should contain the symbolic names of the preprocessors. The actual preprocessor executable programs, or command lines are defined in the configuration file.

After calling this function the source file name property of the program object (pszFileName) is also modified so that it points to the result of the preprocessor. This means that after the successful return of this function the application may immediately call scriba_LoadSourceProgram().

If there is any error during the preprocessor execution the function returns some error code (returned by epreproc) otherwise the return value is zero.

[Contents]

3.3.46. scriba_SaveCode()

Call this function to save the compiled byte code of the program into a specific file. This function is called by the function scriba_SaveCacheFile().
int scriba_SaveCode(pSbProgram pProgram,
                     char *pszCodeFileName
  ){
The function does nothing else, but calls build_SaveCode.

The return code is zero or the error code returned by build_SaveCode.

[Contents]

3.3.47. scriba_SaveCCode()

void scriba_SaveCCode(pSbProgram pProgram,
                      char *pszCodeFileName
  ){

[Contents]

3.3.48. scriba_LoadSourceProgram()

Call this function to load a BASIC program from its source format after optionally checking that there is no available cache file and after executing all required preprocessors. This function calls scriba_ReadSource(), scriba_DoLexicalAnalysis(), scriba_DoSyntaxAnalysis(), scriba_BuildCode(), and also releases the memory that was needed only for code building calling scriba_PurgeReaderMemory(), scriba_PurgeLexerMemory(), scriba_PurgeSyntaxerMemory().

After the successful completion of this program the BASIC program is in the memory in the ready-to-run state.

int scriba_LoadSourceProgram(pSbProgram pProgram
  ){
Before calling this function the function scriba_SetFileName() should have been called specifying the file name.

The return value is zero (SCRIBA_ERROR_SUCCESS) or the error code returned by the underlying layer that has detected the error.

[Contents]

3.3.49. scriba_LoadProgramString()

Use this function to convert a string containing a BASIC program that is already in memory to ready-to-run binary format. This function is same as scriba_LoadSourceProgram() except that this function reads the source code from a string instead of a file.
int scriba_LoadProgramString(pSbProgram pProgram,
                             char *pszSourceCode,
                             unsigned long cbSourceCode
  ){
The argument pProgram is the program object. The argument pszSourceCode is the BASIC program itself in text format. Because the source code may contain ZCHAR just for any chance the caller has to provide the number of characters in the buffer via the argument cbSourceCode. In case the source program is zero terminated the caller can simply say strlen(pszSourceCode) to give this argument.

Before calling this function the function scriba_SetFileName() may be called. Altough the source code is read from memory and thus there is no source file the BASIC program may use the command include or import that includes another source file after reading the code. If the program does so the reader functions need to know the actual file name of the source code to find the file to be included. To help this process the caller using this function may set the file name calling scriba_SetFileName(). However that file is never used and need not even exist. It is used only to calculate the path of included files that are specified using relative path.

The return value is zero (SCRIBA_ERROR_SUCCESS) or the error code returned by the underlying layer that has detected the error.

[Contents]

3.3.50. scriba_Run()

Call this function to execute a program. Note that you can call this function many times. Repetitive execution of the same program will execute the ScriptBasic code again and again with the global variables keeping their values.

If you want to reset the global variables you have to call scriba_ResetVariables().

There is no way to keep the value of the local variables.

The argument pszCommandLineArgument is the command part that is passed to the BASIC program.

int scriba_Run(pSbProgram pProgram,
               char *pszCommandLineArgument
  ){
The return value is zero in case of success or the error code returned by the underlying execution layers.

Note that you can not call BASIC subroutines or functions without initializations that scriba_Run performs. You also can not access global variables. Therefore you either have to call scriba_Run or its brother scriba_NoRun() that performs the initializations without execution.

You also have to call scriba_NoRun() if you want to execute a program with some global variables having preset values that you want to set from the embedding C program. In that case you have to call scriba_NoRun() then one or more times scriba_SetVariable() and finally Run.

[Contents]

3.3.51. scriba_NoRun()

In case the embedding program want to set global variables and execute subroutines without or before starting the main program it has to call this function first. It does all the initializations that are done by scriba_Run() except that it does not actually execute the program.

After calling this function the main program may access global variables and call BASIC functions.

int scriba_NoRun(pSbProgram pProgram
  ){
See also scriba_Run().

[Contents]

3.3.52. scriba_ResetVariables()

Call this function if you want to execute a program object that was already executed but you do not want the global variables to keep their value they had when the last execution of the BASIC code finished.

Global variables in ScriptBasic are guaranteed to be undef before they get any other value and some programs depend on this.

void scriba_ResetVariables(pSbProgram pProgram
  ){
See also scriba_SetVariable(), scriba_Run(), scriba_NoRun().

[Contents]

3.3.53. scriba_Call()

This function can be used to call a function or subroutine. This function does not get any arguments and does not provide any return value.

int scriba_Call(pSbProgram pProgram,
                unsigned long lEntryNode
  ){
The return value is zero or the error code returned by the interpreter.

Note on how to get the Entry Node value:

The argument lEntryNode should be the node index of the subroutine or function that we want to execute. This can be retrieved using the function scriba_LookupFunctionByName() if the name of the function or subroutine is know. Another method is that the BASIC program stored this value in some global variables. BASIC programs can access this information calling the BASIC function Address( f() ).

[Contents]

3.3.54. scriba_CallArg()

This function can be used to call a function or subroutine with arguments passed by value. Neither the return value of the SUB nor the modified argument variables are not accessible via this function. CallArg is a simple interface to call a ScriptBasic subroutine or function with argument.

int scriba_CallArg(pSbProgram pProgram,
                   unsigned long lEntryNode,
                   char *pszFormat, ...
  ){
Arguments

The format string is case insensitive. The characters u, i, r, b and s have meaning. All other characters are ignored. The format characters define the type of the arguments from left to right.

Note that this SUB calling function is a simple interface and has no access to the modified values of the argument after the call or the return value.

If you need any of the functionalities that are not implemented in this function call a more sophisticated function.

Example:

iErrorCode = scriba_CallArg(&MyProgram,lEntry,"i i s d",13,22,"My string.",54.12);

[Contents]

3.3.55. scriba_DestroySbArgs()

This function can be used to release the memory used by arguments created by the function scriba_NewSbArgs().

void scriba_DestroySbArgs(pSbProgram pProgram,
                          pSbData Args,
                          unsigned long cArgs
  ){

Arguments:

[Contents]

3.3.56. scriba_NewSbArgs()

Whenever you want to handle the variable values that are returned by the scriba subroutine you have to call scriba_CallArgEx(). This function needs the arguments passed in an array of SbDtata type.

This function is a usefuly tool to convert C variables to an array of SbData

pSbData scriba_NewSbArgs(pSbProgram pProgram,
                         char *pszFormat, ...
  ){
The arguments passed are

The format string is case insensitive. The characters u, i, r, b and s have meaning. All other characters are ignored. The format characters define the type of the arguments from left to right.

Example:

pSbData MyArgs;

MyArgs = scriba_NewSbArgs(pProgram,"i i r s b",13,14,3.14,"string",2,"two character string"); if( MyArgs == NULL )error("memory alloc");

scriba_CallArgEx(pProgram,lEntry,NULL,5,MyArgs);

This example passes five arguments to the ScriptBasic subroutine. Note that the last one is only two character string, the rest of the characters are ignored.

[Contents]

3.3.57. scriba_CallArgEx()

This is the most sophisticated function of the ones that call a ScriptBasic subroutine. This function is capable handling parameters to scriba subroutines, and returning the modified argument variables and the return value.

int scriba_CallArgEx(pSbProgram pProgram,
                     unsigned long lEntryNode,
                     pSbData ReturnValue,
                     unsigned long cArgs,
                     pSbData Args
  ){
The arguments:

[Contents]

3.3.58. scriba_LookupFunctionByName()

This function should be used to get the entry point of a function knowing the name of the function. The entry point should not be treated as a numerical value rather as a handle and to pass it to functions like scriba_CallArgEx().

long scriba_LookupFunctionByName(pSbProgram pProgram,
                                 char *pszFunctionName
  ){
The return value of the function is the entry node index of the function named or zero if the function is not present in the program.

[Contents]

3.3.59. scriba_LookupVariableByName()

This function can be used to get the serial number of a global variable knowing the name of the variable.

Note that all variables belong to a name space. Therefore if you want to retrieve the global variable foo you have to name it main::foo.

long scriba_LookupVariableByName(pSbProgram pProgram,
                                 char *pszVariableName
  ){
The return value is the serial number of the global avriable or zero if there is no variable wit that name.

[Contents]

3.3.60. scriba_GetVariableType()

Get the type of the value that a variable is currently holding. This value can be

long scriba_GetVariableType(pSbProgram pProgram,
                            long lSerial
  ){
The argument lSerial should be the serial number of the variable as returned by scriba_LookupVariableByName().

If there is no variable for the specified serian mumber (lSerial is not positive or larger than the number of variables) the function returns SBT_UNDEF.

[Contents]

3.3.61. scriba_GetVariable()

This function retrieves the value of a variable. A new SbData object is created and the pointer to it is returned in pVariable. This memory space is automatically reclaimed when the program object is destroyed or the function DestroySbData can be called.

int scriba_GetVariable(pSbProgram pProgram,
                       long lSerial,
                       pSbData *pVariable
  ){
The argument lSerial should be the serial number of the global variable as returned by scriba_LookupVariableByName().

The funtion returns SCRIBA_ERROR_SUCCESS on success,

SCRIBA_ERROR_MEMORY_LOW if the data cannot be created or

SCRIBA_ERROR_FAIL if the parameter lSerial is invalid.

[Contents]

3.3.62. scriba_SetVariable()

This function sets the value of a global BASIC variable. You can call this function after executing the program before it is reexecuted or after successfull call to scriba_NoRun().

int scriba_SetVariable(pSbProgram pProgram,
                       long lSerial,
                       int type,
                       long lSetValue,
                       double dSetValue,
                       char *pszSetValue,
                       unsigned long size
  ){
The argument lSerial should be the serial number of the global variable as returned by scriba_LookupVariableByName().

The argument type should be one of the followings:

The function uses one of the arguments lSetValue, dSetValue or pszSetValue and the other two are ignored based on the value of the argument type.

If the value of the argument type is SBT_UNDEF all initialization arguments are ignored and the global variable will get the value undef.

If the value of the argument type is SBT_DOUBLE the argument dSetValue will be used and the global variable will be double holding the value.

If the value of the argument type is SBT_LONG the argument lSetValue will be used and the global variable will be long holding the value.

If the value of the argument type is SBT_STRING the argument pszSetValue will be used and the global variable will be long holding the value. The length of the string should in this case be specified by the variable size.

If the value of the argument type is SBT_ZCHAR the argument pszSetValue will be used and the global variable will be long holding the value. The length of the string is automatically calculated and the value passed in the variable size is ignored. In this case the string pszSetValue should be zero character terminated.

The funtion returns SCRIBA_ERROR_SUCCESS on success,

SCRIBA_ERROR_MEMORY_LOW if the data cannot be created or

SCRIBA_ERROR_FAIL if the parameter lSerial is invalid.

[Contents]

4. Extension Modules

Extension modules are written usually using the language C and implement functions that can not be efficiently implemented in BASIC. These functions can be called from the BASIC programs just as if they were written in BASIC.

[Contents]

4.1. How Extension Modules are Used

To write external modules it is a good practice to learn first how ScriptBasic uses the modules.

External functions and external commands in ScriptBasic are declared using the declare sub or declare command statements. An example of such a statement is

declare sub alma alias "trial" lib "ext_tial"
or
declare command iff alias "iff" lib "ext_tial"

Following this declaration the function or the command can be used just as it were implemented in BASIC.

call alma(1,2,3)

The difference between external functions and external commands is the way ScriptBasic handles the arguments passed to them. Both external functions and external commands are implemented as C functions in the extension module compiled into a DLL or shareable object. Both of them look like user defined functions in the BASIC source code.

The difference is that external functions are called after the actual arguments are evaluated, while external commands are called without evaluating the arguments. Because of this external functions and external commands are implemented in C functions that have different prototypes. There is a prototype for external functions and a different one for external commands.

When ScriptBasic compiles this line the function or subroutine alma is defined just as a normal function or subroutine defined using the instructions of the basic language. Note that there are no differences other than syntax between subroutines and functions in ScriptBasic. When the program calls the function alma the ScriptBasic run-time system performs a function call to the basic function alma. In other words there is no difference from the caller point of view between the line above and the line:

Sub alma(a,b)
End sub
The function call can be performed in two different ways. One way is when the function appears in an expression. The other way is when the function is called using the call statement. There is no difference between the two calling possibilities from the internal operation point of view of the interpreter. This is because CALL statement is implemented in a very simple way to simply evaluate the expression after the call statement and drop the result.

The calling code does not evaluate the expressions passed to the function as arguments. This is usually the task of the functions. The functions get the node pointer to the expression list where the actual values for the arguments are and they can evaluate them.

The two different declarations declare sub and declare command differ in the way ScriptBasic interpreter handles the arguments. When an external function is declared using the command declare sub the arguments are evaluated by the interpreter before the function implemented in the external module is called. When an external command is declared using the command declare command the arguments are NOT evaluated by the ScriptBasic interpreter before calling the function implemented in the external module. In the latter case the external function has to decide if it wants certain arguments to be evaluated and can call the ScriptBasic function execute_Evaluate via the extension call-back table to evaluate the arguments. Also the prototype of a function declared using the statement declare command is different from the prototype of a function declared using the command declare sub.

When a function is implemented externally ScriptBasic sees a declare sub statement instead of a function or sub statement and starts to execute this statement calling the code implemented in the file `external.c' in the source directory `commands'.

The name of the example function is alma as declared in the statement above. However this is only a symbolic name that exists only during syntax analysis and is not available when the code is executed. The alias for the function is trial. This is the name of the function as it is implemented in the external module. When the interpreter executes this line the function name trial is used in the system call to locate the entry point. The module that contains the function is ext_trial. The actual file name is `ext_trial.dll' or `ext_trial.so' or some other name containing the given name and an extension specific to the operating system. During module load ScriptBasic automatically appends the appropriate extension as defined in the configuration file of ScriptBasic. (For further information on ScriptBasic configuration file syntax and location see the ScriptBasic Users' Guide!) ScriptBasic searches the module in the directories defined in the configuration file and tries to load it using absolute file name. This way the system specific search paths are not taken into account.

When function implemented in an external module is first called the interpreter checks if the module is loaded or not. If the module is not loaded the interpreter loads the module and calls module initialization functions. If the module was already loaded it locates the entry point of the function and calls the function.

During module load ScriptBasic appends the appropriate dynamic load library extension and tries to load the module from the directories defined in the configuration file. It takes the directories in the order they are specified in the configuration file and in case it can load the module from a directory listed it stops trying.

When the module is loaded ScriptBasic locates the function versmodu and calls it. The task of this function is to negotiate the interface version between the external module and ScriptBasic. The current interface version is defined in the file `basext.c' with the C macro INTERFACE_VERSION. ScriptBasic calls this function to tell the module what version ScriptBasic supports. The function can decide if the module can work with the indicated version and can answer: yes it is OK, no it is not OK or yes, but I can support only version X. This is a negotiation process that finally result some agreement or the module is abandoned if no agreement can be reached.

The function versmodu gets three arguments:

int versmodu(int Version, char *pszVariation, void **ppModuleInternal)

The first argument is the version of the interface. The second argument is the ZCHAR terminated 8-character string of the variation. This is “STANDARD” for the standard, stand alone, command line version of ScriptBasic. The ppModuleInternal pointer points to the module pointer initialized to NULL. This pointer is hardly ever used in this function, but its address is passed as a third argument in case some application needs it. The role of this pointer will be discussed later.

The function should check the parameters passed and return either zero in case it can not accept the interface or the highest interface it can handle. If this is the same as the version passed in the first argument the module should be accepted. If this is smaller than the interface version offered by ScriptBasic the interpreter can decide if it can support the older interface required by the module.

If the function versmodu returns a version larger than the version offered the interpreter will interpret this as a negotiation failure and will treat the module as not loaded.

If there is no function named versmodu in the library ScriptBasic crosses the fingers and hopes the best and assumes that the module will be able to work with the interface ScriptBasic offers. (should we change it to be configurable to disallow such modules?)

After the successful version negotiation the interpreter calls the function named bootmodu. This function gets four arguments.

int bootmodu(pSupportTable pSt, void **ppModuleInternal, pFixSizeMemoryObject pParameters, pFixSizeMemoryObject *pReturnValue)

The first parameter is a pointer to the interface structure. This interface structure can and should be used to communicate with ScriptBasic. The second parameter points to the module pointer. The last two parameters are NULL for this function. The reason to pass two NULL pointers is that this is the prototype of each function callable by ScriptBasic implemented in the module. The last two parameters point to the parameters of the function and to the left value where the function value is to be returned. bootmodu actually does not get any parameter and should not pass any value back.

This function can be used to initialize the module, to allocate memory for the common storage if the functions implemented in the module keep some state information. If there is no function named bootmodu in the library file ScriptBasic assumes that the module does not need initialization. If the function bootmodu exists it should return zero indicating success or an error code. If an error code is returned the module is treated as failed. And an error is raised. (Errors can be captured using the BASIC ON ERROR GOTO command.

When this function returns the ScriptBasic interpreter evaluates the arguments and performs a call to the function named trial in our example.

When the program has finished the interpreter tries to locate the function finimodu in the module. This function may exist and should have the same prototype as any other function (except versmodu):

int finimodu(pSupportTable pSt,
             void **ppModuleInternal,
             pFixSizeMemoryObject pParameters,
             pFixSizeMemoryObject *pReturnValue)

This function can be used to perform clean-up tasks. The interpreter may call the finimodu functions of different modules in different threads asynchronously. (However it does not currently.)

Note that there is no need to release the allocated memory in case the module allocates memory using the memory allocation methods provided by the interface. Other resources may need release; for example files may need closing.

[Contents]

4.2. A Simple Sample Module

After you have got a slight overview how ScriptBasic handles the modules get a jump-start looking at the simplest sample module trial.c!

This module was the very first module the developers wrote to test the module handling functionality of ScriptBasic. This does almost nothing, but prints out some trace messages so that you can see how the different functions are called in the module. There is only one function in this module that the basic code can call. This is trial. This function increments a long value and returns the actual value of this state variable.

Here is the whole code:

#include <stdio.h>

#include "../../basext.h"

besVERSION_NEGOTIATE

printf("The function bootmodu was started and the requested version is %d\n",Version); printf("The variation is: %s\n",pszVariation); printf("We are returning accepted version %d\n",(int)INTERFACE_VERSION);

return (int)INTERFACE_VERSION;

besEND

besSUB_START long *pL;

besMODULEPOINTER = besALLOC(sizeof(long)); if( besMODULEPOINTER == NULL )return 0; pL = (long *)besMODULEPOINTER; *pL = 0L;

printf("The function bootmodu was started.\n");

besEND

besSUB_FINISH printf("The function finimodu was started.\n"); besEND

besFUNCTION(trial) long *pL;

printf("Function trial was started...\n"); pL = (long *)besMODULEPOINTER; (*pL)++; besRETURNVALUE = besNEWMORTALLONG; LONGVALUE(besRETURNVALUE) = *pL;

printf("Module directory is %s\n",besCONFIG("module")); printf("dll extension is %s\n",besCONFIG("dll")); printf("include directory is %s\n",besCONFIG("include"));

besEND

besCOMMAND(iff) NODE nItem; VARIABLE Op1; long ConditionValue;

USE_CALLER_MORTALS;

/* evaluate the parameter */ nItem = besPARAMETERLIST; if( ! nItem ){ RESULT = NULL; RETURN; } Op1 = besEVALUATEEXPRESSION(CAR(nItem)); ASSERTOKE;

if( Op1 == NULL )ConditionValue = 0; else{ Op1 = besCONVERT2LONG(Op1); ConditionValue = LONGVALUE(Op1); }

if( ! ConditionValue ) nItem = CDR(nItem);

if( ! nItem ){ RESULT = NULL; RETURN; } nItem = CDR(nItem);

RESULT = besEVALUATEEXPRESSION(CAR(nItem)); ASSERTOKE; RETURN; besEND_COMMAND

As you can see there is a lot of code hidden behind the macros. You can not see versmodu, bootmodu or finimodu, because they are implemented using the macro besVERSION_NEGOTIATE, besSUB_START and bes_SUB_FINISH. These macros are provided in the header file `basext.h', along with other type definitions and include statements that are needed to compile a module. To have a deeper understanding feel free to have a look at the file `basext.c' containing the source for basext.h.

All the macros defined in the header file for the extensions start with the three letters bes. These stand for basic extension support.

The version negotiation function prototype and function start is created using the macro besVERSION_NEGOTIATE. This macro generates the function head with the parameters named Version, pszVariation and ppModuleInternal. The accepted version is returned using the macro INTERFACE_VERSION. The reason to use this macro is to ease maintainability.

The current version of the interface is 10 (or more). Later interfaces may probably support more callback interface functions, but it is unlikely that the interfaces become incompatible on the source level. When the module is recompiled in an environment that uses a newer interface it will automatically return the interface version that it really supports. If the interfaces become incompatible in source level the compilation phase will most probably fail.

The bootmodu function is created using the macro besSUB_START. In this example this allocates a state variable, which is a long. The memory allocation is performed using a callback function and with the aid of the macro besALLOC.

Here we can stop a bit and examine, how the callback functions work. The ScriptBasic interpreter has several functions that are available for the extensions. To access these functions the module should know the entry point (address) of the functions. To get the entry points ScriptBasic creates a table of the callback functions. A pointer to this table is passed as the first argument to each module function except the version negotiation function versmodu. In C syntax this table is a struct named SupportTable.

There are numerous functions that an extension can and should use to communicate with ScriptBasic. One of the most important functions is memory allocation. The field of the SupportTable named Alloc is initialized to point to the function alloc_Alloc defined in the file `myalloc.c'. This allocation function needs the size of the needed memory block and a pointer to a so-called memory segment. The memory segment pointer to be used is available via the SupportTable. The first member of the table is a pointer to another table containing the current interpreter execution environment. This environment is also a &code{struct} and contains the memory segment pointer.

This is a bit complicated and you can get confused. To ease coding use the macros available. These will hide all these nifty details. However know that these macros assume that you use them together. In other words besALLOC can only be used in a function when the function head was created using the macro besFUNCTION (or besSUB_START or besSUB_FINISH in case of bootmodu and finimodu). This is because the macros assume certain variable names in the arguments.

The function finimodu is created using the macro besSUB_FINISH. This function in this example does nothing but prints a test message. There is no need to release the memory allocated using besALLOC, because the memory is administered to belong to the segment of the interpreter execution environment and is released by the interpreter before the exits.

The real function of the module is named trial and is defined using the macro besFUNCTION. It gets the state variable via the module pointer. The module pointer is always passed to the module functions as the second argument. The functions can access it using the macro besMODULEPOINTER. Using this macro it looks like a local variable.

To return the counter value the function needs a basic variable. This variable should hold a long value and should be mortal. This is created using the macro besNEWMORTALLONG. The variable is actually a struct and the field containing the long value can be accessed using the macro LONGVALUE.

You can notice that this macro does not start with the letters bes. The reason is that this macro comes from a different header file `command.h'. This header file is included by `basext.h' and the definitions in that file are mainly for implementing internal commands linked statically. However some of the macros can be used for dynamic modules as well.

The function trial finally writes out some configuration data. On one hand this is another example of a callback function used via a macro named besCONFIG. But this is more important than that. This shows you that the modules can access any configuration data.

There is no need for any module to process separate configuration files. ScriptBasic reads the configuration file and stores each key and value pair it finds. It stores even those that do not mean anything for ScriptBasic itself, because they may be meaningful and needed by modules. The example module trial does not have its own data, therefore we print out the configuration data that ScriptBasic surely has.

[Contents]

4.3. Compiling a Module

Compiling a module is easy and straightforward. Just do it as you would do for any dynamic load library. On Linux you have to compile the source code to object file, saying
cc -c -o trial.o trial.c
ld -d -warn-section-align -sort-comon -shared -o trial.so trial.o
assuming that the name of the file to compile is `trial.c'. On other Unix operating systems you have to issue similar commands.

On Windows NT you can use Visual C++ and create an empty project using the work space wizard to create a new dll file. You have to add your source code to the project, select release version and compile the program.

Note that Windows NT dll files do not automatically export each non-static function of a program. They have to be declared in a DEF file or the functions should be denoted with a special type casting keyword. If you use the predefined macros available including the file `basext.h' your functions will be exported without creating a def file.

[Contents]

4.4. Installing a Module

A module is usually composed of two files. One file is the binary library module with the extension .dll under Windows NT or .so under Unix. The other file is an include file, which contains the declare sub statement for each external function.

To install a module you have to copy or move the module binary to one of the directories specified in the configuration file as ScriptBasic module directory and you have to copy or move the include file into one of the directories specified in the configuration file as ScriptBasic include directory.

The basic program that uses the module includes the include file and it is ready to call the functions declared in that file. Currently there are no install programs available that place the file at the appropriate locations.

[Contents]

4.5. Module Support Functions

Module support functions and macros are to ease the life of the extension programmers. They are defined in the file `basext.c' and in the file `basext.h' generated from `basext.c' using the tool `headerer.pl'.

It is highly recommended that the extensions use these functions and that the extensions use the macros to call the functions the way the documentation suggests. The reason for this is to create readable code and to provide maintainability.

[Contents]

4.5.1. Basic Extension Support functions, structures and macros

[Contents]

4.5.2. besALLOC(X)

Use this macro to allocate memory in an extension like you would use malloc in a normal, old fashioned C program. The argument is the size of the memory in byte count to be allocated. The result is the pointer to the allocated memory or NULL if there is not enough memory available.

The allocated memory is assigned to the memory segment of the execution thread and thus this memory is released automatically when the module is unloaded from the interpreter. In other words if a module uses besALLOC to allocate memory there is no need to call besFREE(X).

Modules written for multi-thread variations of ScriptBasic should also be aware of the fact that the memory allocated by this macro is released whent he calling interpreter thread finishes.

This macro calls the function alloc_Alloc()

[Contents]

4.5.3. besPROCALLOC(X)

Use this macro in multi-thread supporting modules to allocate memory that is not freed when the actual interpreter finishes but stays in memory so long as long the process is alive.

[Contents]

4.5.4. besFREE(X)

Use this macro to release memory that was allocated by besALLOC(X).

Altough all memory allocated by besALLOC() is automatically released when the interpreter thread calling besALLOC() finishes it is advised to release all memory chunks, especially large onces when they are not needed anymore.

This macro also NULLifies the argument.

This macro calls the function alloc_Free()

[Contents]

4.5.5. besPROCFREE(X)

This is the counterpart of besPROCALLOC releasing memory allocated for process life time.

[Contents]

4.5.6. besPROCMEMORYSEGMENT

Use this macro in case you need to pass the process level memory segment to a function.

[Contents]

4.5.7. besNEWMORTALSTRING(X)

Create a new mortal string and return a pointer to it. The argument should be the number of characters in the string. This macro calls the function memory_NewMortalString().

See also

[Contents]

4.5.8. besNEWMORTALLONG

Create a new mortal long and return the pointer to it. This macro calls the function memory_NewMortalLong().

See also

[Contents]

4.5.9. besNEWMORTALREF

Create a new mortal reference and return the pointer to it. This macro calls the function memory_NewMortalRef().

See also

[Contents]

4.5.10. besNEWMORTALDOUBLE

Create a new mortal double and return the pointer to it. This macro calls the function memory_NewMortalDouble().

See also

[Contents]

4.5.11. besNEWMORTALARRAY(X,Y)

Create a new mortal array and return the pointer to it. The arguments define the array low index and the array high index. This macro calls the function memory_NewMortalArray().

See also

[Contents]

4.5.12. besNEWSTRING(X)

Create a new string and return the pointer to it. The argument defines the number of bytes in the string. This macro calls the function memory_NewString().

See also

[Contents]

4.5.13. besNEWLONG

Create a new long and return the pointer to it. This macro calls the function memory_NewLong().

See also

[Contents]

4.5.14. besNEWREF

Create a new reference variable and return the pointer to it. This macro calls the function memory_NewRef().

See also

[Contents]

4.5.15. besNEWDOUBLE

Create a new double and return the pointer to it. This macro calls the function memory_NewDouble().

See also

[Contents]

4.5.16. besNEWARRAY(X,Y)

Create a new array and return the pointer to it. The arguments define the array low index and the array high index. This macro calls the function memory_NewArray().

See also

[Contents]

4.5.17. besRELEASE(X)

Use this macro to release a non-mortal variable. This macro calls the function memory_ReleaseVariable() with the appropriate memory segment arguments.

See also

[Contents]

4.5.18. besCONFIG(X)

Get a configuration value that is supposed to be a string. This macro uses the configuration of the callign interpreter thread and it calls the function cft_GetString.

[Contents]

4.5.19. besCONFIGFINDNODE(X,Y,Z)

This macro calls the function cft_FindNode. Use this macro for sophisticated configuration handling that needs configuration key enumeration.

[Contents]

4.5.20. besCONFIGEX(CT,CS,NS,CSS,LS,DS,IS)

This macro calls the function cft_GetEx. Use this macro to retrieve configuration information that is not a string or the type is not known.

[Contents]

4.5.21. besCONFIGENUMFIRST(X,Y)

This macro calls the function cft_EnumFirst.

[Contents]

4.5.22. besCONFIGENUMNEXT(X,Y)

This macro calls the function cft_EnumNext.

[Contents]

4.5.23. besCONFIGGETKEY(X,Y)

This macro calls the function cft_GetKey.

[Contents]

4.5.24. besNEWSYMBOLTABLE

This macro allocates a new symbol table and returns the handle pointer to it. This macro calls the function sym_NewSymbolTable. The allocation function and the memory segment used to allocate the symbol table is the default one that is used by the interpreter.

[Contents]

4.5.25. besFREESYMBOLTABLE(X)

This macro releases a symbol table calling the function sym_FreeSymbolTable. The allocation function and the memory segment used to allocate the symbol table is the default one that is used by the interpreter.

[Contents]

4.5.26. besTRAVERSESYMBOLTABLE(X,Y,Z)

This macro calls the function sym_TraverseSymbolTable.

[Contents]

4.5.27. besLOOKUPSYMBOL(X,Y,Z)

This macro calls the function sym_LookupSymbol.

[Contents]

4.5.28. besDeleteSymbol(X,Y,Z)

This macro calls the function sym_DeleteSymbol.

[Contents]

4.5.29. besLOADLIBRARY(X)

This macro calls the function dynlolib_LoadLibrary().

[Contents]

4.5.30. besFREELIBRARY(X)

This macro calls the function dynlolib_FreeLibrary().

[Contents]

4.5.31. besGETFUNCTIONBYNAME(X)

This macro calls the function dynlolib_GetFunctionByName().

[Contents]

4.5.32. besFOPEN

This macro calls the function file_fopen.

[Contents]

4.5.33. besFCLOSE

This macro calls the function file_fclose.

[Contents]

4.5.34. besSIZE

This macro calls the function file_size.

[Contents]

4.5.35. besTIME_ACCESSED

This macro calls the function file_time_accessed.

[Contents]

4.5.36. besTIME_MODIFIED

This macro calls the function file_time_modified.

[Contents]

4.5.37. besTIME_CREATED

This macro calls the function file_time_created.

[Contents]

4.5.38. besISDIR

This macro calls the function file_isdir.

[Contents]

4.5.39. besISREG

This macro calls the function file_isreg.

[Contents]

4.5.40. besEXISTS

This macro calls the function file_fileexists.

[Contents]

4.5.41. besTRUNCATE

This macro calls the function file_truncate.

[Contents]

4.5.42. besFGETC

This macro calls the function file_fgetc.

[Contents]

4.5.43. besFREAD

This macro calls the function file_fread.

[Contents]

4.5.44. besFWRITE

This macro calls the function file_fwrite.

[Contents]

4.5.45. besSETMODE

This macro calls the function file_setmode.

[Contents]

4.5.46. besBINMODE

This macro calls the function file_binmode.

[Contents]

4.5.47. besTEXTMODE

This macro calls the function file_textmode.

[Contents]

4.5.48. besFERROR

This macro calls the function file_ferror.

[Contents]

4.5.49. besFPUTC

This macro calls the function file_fputc.

[Contents]

4.5.50. besFLOCK

This macro calls the function file_flock.

[Contents]

4.5.51. besLOCK

This macro calls the function file_lock.

[Contents]

4.5.52. besFEOF

This macro calls the function file_feof.

[Contents]

4.5.53. besMKDIR

This macro calls the function file_mkdir.

[Contents]

4.5.54. besRMDIR

This macro calls the function file_rmdir.

[Contents]

4.5.55. besREMOVE

This macro calls the function file_remove.

[Contents]

4.5.56. besDELTREE

This macro calls the function file_deltree.

[Contents]

4.5.57. besMAKEDIRECTORY

This macro calls the function file_MakeDirectory.

[Contents]

4.5.58. besOPENDIR

This macro calls the function file_opendir.

[Contents]

4.5.59. besREADDIR

This macro calls the function file_readdir.

[Contents]

4.5.60. besCLOSEDIR

This macro calls the function file_closedir.

[Contents]

4.5.61. besOPTION(X)

Get the long value of the option X. This macro calls the function options_Get. The macro uses the default execution context.

[Contents]

4.5.62. besSETOPTION(x,y)

Set the y long value of the option x. This macro calls the function options_Set. The macro uses the default execution context.

[Contents]

4.5.63. besRESETOPTION(X)

Reset the option X. This macro calls the function options_Reset. The macro uses the default execution context.

[Contents]

4.5.64. besCONVERT2STRING(x)

Use this macro to convert a mortal VARIABLE to string. The macro calls the function execute_Convert2String and uses the global mortal list.

[Contents]

4.5.65. besCONVERT2LONG(x)

Use this macro to convert a mortal VARIABLE to long. The macro calls the function execute_Convert2Long and uses the global mortal list.

[Contents]

4.5.66. besGETLONGVALUE(x)

Use this macro to get the long value of a variable. This macro is not the same as LONGVALUE. The macro LONGVALUE simply accesses the long value of a variable and thus can also be used as a left value assigning value to. On the other hand besGETLONGVALUE is a function call that returns a long value when the argumentum variable is NULL, double, string or some other value. In such situation using LONGVALUE would be erroneous.

The macro LONGVALUE should be used to access the long value of a variable that is known to hold a long value or when the long value of a variable is to be set.

The macro besGETLONGVALUE has to be used in a situation when we want to use a variable for its value being long. It is faster and consumes less memory than converting the variable to long allocating a new mortal just to access the long value of the new mortal using LONGVALUE.

The same statements hold for DOUBLEVALUE and besGETDOUBLEVALUE.

[Contents]

4.5.67. besCONVERT2DOUBLE(x)

Use this macro to convert a mortal VARIABLE to double. The macro calls the function execute_Convert2Double and uses the global mortal list.

[Contents]

4.5.68. besGETDOUBLEVALUE(x)

Use this macro to get the double value of a variable.

For comparision of DOUBLEVALUE and this macro see the explanation in besGETLONGVALUE.

[Contents]

4.5.69. besISSTRINGINTEGER(x)

Use this macro to decide if a string contains caharacters copnvertible to an integer value. This macro calls the function execute_IsStringInteger.

[Contents]

4.5.70. besCONVERT2ZCHAR(x)

Use this macro to convert a VARIABLE that is already STRING type to zero character terminated string. This is needed many times when a BASIC string has to be passed to operating system functions.

The macro argument x is converted to zero character terminated string and the result will be pointed by y. y has to be a (char *) C variable.

If there is not enough memory the macro returns from the function with the error code COMMAND_ERROR_MEMORY_LOW thus there is no need to check the value of the variable y if it is NULL.

The memory allocated to store the ZCHAR string should be released by the macro besFREE.

Note that the macro does not check wheter the string already contains a zero character or not. It simply allocates a buffer that has length n+1 bytes to store the n bytes of the original BASIC string and an extra zero character.

[Contents]

4.5.71. besREINITINTERFACE

This macro calls the function modu_Init to reset the module interface to point to the original functions.

External modules are allowed to alter the support function table implementing their own functions and thus altering the behavior of other extensions. These extensions usually set some of the entries of the support function table to point to functions implemented in the external module. However these functions are available only so long as long the extension is in memory. When the extension is exiting the pointers should be restored. Otherwise it may happen that the extension is already unloaded and the function is used by another module exiting later.

Because there is no guaranteed order of extension module unload the modules should call this function to restore the support function table and should not rely on altered function calls during module exit code is executed.

[Contents]

4.5.72. besLOADMODULE(x,y)

This macro calls the function modu_LoadModule. The execution environment (ExecuteObject) used by the macro while calling the function is the default execution environment.

[Contents]

4.5.73. besGETMODULEFUNCTIONBYNAME

This macro calls the function modu_GetFunctionByName. The execution environment (ExecuteObject) used by the macro while calling the function is the default execution environment.

[Contents]

4.5.74. besUNLOADALLMODULES

This macro calls the function modu_UnloadAllModules. This will unload all modules that are not active. Because this macro is called by an extension that is a module itself the calling module will not be unloaded.

A module is active if there is a function called in the module and the module did not return from the function call yet. Typically there can be only one active module at a time unless some modules call each other in tricky ways through the ScriptBasic module calling functions. Therefore calling this macro practically unloads all modules but the very one using this macro.

See also besUNLOADMODULE.

[Contents]

4.5.75. besUNLOADMODULE(x)

This macro calls the function modu_UnloadModule to unload the named module. Note that the module name x should be the same string that was used to load the module and it can not be the actual module or any other module that is active.

See also besUNLOADALLMODULES.

[Contents]

4.5.76. besSLEEP(x)

This macro calls file_sleep to sleep x number of seconds.

[Contents]

4.5.77. besCURDIR(x,y)

This function calls file_curdir to get the current working directory.

[Contents]

4.5.78. besCHDIR(x)

This function calls file_chdir to set the current working directory. Be careful changing the working directory because it may prevent the extension module being used in multi-thread variation of ScriptBasic.

[Contents]

4.5.79. besCHOWN(x,y)

This macro calls the function file_chown.

[Contents]

4.5.80. besSETCREATETIME(x,y)

This macro calls the function file_SetCreateTime.

[Contents]

4.5.81. besSETMODIFYTIME(x,y)

This macro calls the function file_SetModifyTime.

[Contents]

4.5.82. besSETACCESSTIME(x,y)

This macro calls the function file_SetAccessTime.

[Contents]

4.5.83. besGETHOSTNAME(x,y)

This macro calls the function file_GetHostName.

[Contents]

4.5.84. besGETHOST(x,y)

This macro calls the function file_GetHost.

[Contents]

4.5.85. besTCPCONNECT(x,y)

This macro calls the function file_TcpConnect.

[Contents]

4.5.86. besTCPSEND(x,y,z)

This macro calls the function file_TcpSend.

[Contents]

4.5.87. besTCPRECV(x,y,z)

This macro calls the function file_TcpRecv.

[Contents]

4.5.88. besTCPCLOSE(y)

This macro calls the function file_TcpClose.

[Contents]

4.5.89. besKILLPROC(x)

This macro calls the function file_KillProc.

[Contents]

4.5.90. besGETOWNER(x,y,z)

This macro calls the function file_GetOwner.

[Contents]

4.5.91. besCRYPT(x,y,z)

This macro calls the function file_fcrypt.

[Contents]

4.5.92. besMD5INIT(C)

This macro calls the function MD5Init.

[Contents]

4.5.93. besMD5UPDATE(C,I,L)

This macro calls the function MD5Update.

[Contents]

4.5.94. besMD5FINAL(D,C)

This macro calls the function MD5Final.

[Contents]

4.5.95. besCREATEPROCESS(X)

This macro calls the function file_CreateProcess.

[Contents]

4.5.96. besCOPYCOMMANDTABLE(X)

This macro calls the function execute_CopyCommandTable.

[Contents]

4.5.97. besGETCOMMANDBYNAME(X,Y)

This macro calls the function execute_GetCommandByName.

[Contents]

4.5.98. besEVALUATEEXPRESSION(X)

This macro evaluates an expression and returns the result VARIABLE. This should usually be used only in extension besCOMMAND commands and not in besFUNCTION. The result is duplicated and mortalized and therefore the command is free to modify the VARIABLE.

[Contents]

4.5.99. _besEVALUATEEXPRESSION(X)

This macro evaluates an expression and returns the result VARIABLE. This should usually be used only in extension besCOMMAND commands and not in besFUNCTION. The result is NOT duplicated and NOT mortalized and therefore the command modifying the VARIABLE may happen to modify a BASIC variable.

This should not be done! There are other ways to modify the value of a variable.

[Contents]

4.5.100. _besEVALUATEEXPRESSION_A(X)

This macro evaluates an expression and returns the result VARIABLE. This macro is the same as _besEVALUATEEXPRESSION except that this macro may result a whole array.

[Contents]

4.5.101. besEVALUATELEFTVALUE(X)

This macro evaluates a left value. This left-value is a pointer to a VARIABLE. If this is not NULL it has to be released first using the macro besRELEASE and a new immortal value may be assigned to it afterwards.

[Contents]

4.5.102. besEVALUATELEFTVALUE_A(X)

This macro evaluates a left value. This macro is the same as besEVALUATELEFTVALUE except that this macro may result a whole array.

[Contents]

4.5.103. besIMMORTALIZE(x)

This macro calls the function memory_Immortalize(). Use this macro to immortalize a mortal variable before assigning it to a BASIC variable.

[Contents]

4.5.104. besDEREFERENCE(X)

This macro calls execute_DereferenceS.

[Contents]

4.5.105. besMatchIndex(X)

This macro calls match_index.

[Contents]

4.5.106. besMatchIniSets(X)

This macro calls match_InitSets.

[Contents]

4.5.107. besMatchModifySet(X,Y,Z,W,Q)

This macro calls match_ModifySet.

[Contents]

4.5.108. besMatchMatch(P1,P2,P3,P4,P5,P6,P7,P8,P9,P10,P11,P12)

This macro calls match_match.

[Contents]

4.5.109. besMatchCount(X,Y)

This macro calls match_count.

[Contents]

4.5.110. besMatchParameter(P1,P2,P3,P4,P5,P6,P7)

This macro calls match_parameter.

[Contents]

4.5.111. besMatchSize(P1,P2,P3,P4,P5)

This macro calls match_size().

[Contents]

4.5.112. besCreateThread(X,Y,Z)

This macro calls thread_CreateThread.

[Contents]

4.5.113. besExitThread

This macro calls thread_ExitThread.

[Contents]

4.5.114. besInitMutex(X)

This macro calls thread_InitMutex.

[Contents]

4.5.115. besFinishMutex(X)

This macro calls thread_FinishMutex.

[Contents]

4.5.116. besLockMutex(X)

This macro calls thread_LockMutex.

[Contents]

4.5.117. besUnlockMutex(X)

This macro calls thread_UnlockMutex.

[Contents]

4.5.118. besInitSharedLock(X)

This macro calls thread_InitLock.

[Contents]

4.5.119. besFinishSharedLock(X)

This macro calls thread_FinishLock.

[Contents]

4.5.120. besLockSharedRead(X)

This macro calls thread_LockRead.

[Contents]

4.5.121. besLockSharedWrite(X)

This macro calls thread_LockWrite.

[Contents]

4.5.122. besUnlockSharedRead(X)

This macro calls thread_UnlockRead.

[Contents]

4.5.123. besUnlockSharedWrite(X)

This macro calls thread_UnlockWrite.

[Contents]

4.5.124. besScribaNew(F0,F1)

This macro calls scriba_new

[Contents]

4.5.125. besScribaDestroy(F0)

This macro calls scriba_destroy

[Contents]

4.5.126. besScribaNewSbData(F0)

This macro calls scriba_NewSbData

[Contents]

4.5.127. besScribaNewSbLong(F0,F1)

This macro calls scriba_NewSbLong

[Contents]

4.5.128. besScribaNewSbDouble(F0,F1)

This macro calls scriba_NewSbDouble

[Contents]

4.5.129. besScribaNewSbUndef(F0)

This macro calls scriba_NewSbUndef

[Contents]

4.5.130. besScribaNewSbString(F0,F1)

This macro calls scriba_NewSbString

[Contents]

4.5.131. besScribaNewSbBytes(F0,F1,F2)

This macro calls scriba_NewSbBytes

[Contents]

4.5.132. besScribaDestroySbData(F0,F1)

This macro calls scriba_DestroySbData

[Contents]

4.5.133. besScribaPurgeReaderMemory(F0)

This macro calls scriba_PurgeReaderMemory

[Contents]

4.5.134. besScribaPurgeLexerMemory(F0)

This macro calls scriba_PurgeLexerMemory

[Contents]

4.5.135. besScribaPurgeSyntaxerMemory(F0)

This macro calls scriba_PurgeSyntaxerMemory

[Contents]

4.5.136. besScribaPurgeBuilderMemory(F0)

This macro calls scriba_PurgeBuilderMemory

[Contents]

4.5.137. besScribaPurgeExecuteMemory(F0)

This macro calls scriba_PurgeExecuteMemory

[Contents]

4.5.138. besScribaSetFileName(F0,F1)

This macro calls scriba_SetFileName

[Contents]

4.5.139. besScribaLoadConfiguration(F0,F1)

This macro calls scriba_LoadConfiguration

[Contents]

4.5.140. besScribaInheritConfiguration(F0,F1)

This macro calls scriba_InheritConfiguration

[Contents]

4.5.141. besScribaSetCgiFlag(F0)

This macro calls scriba_SetCgiFlag

[Contents]

4.5.142. besScribaSetReportFunction(F0,F1)

This macro calls scriba_SetReportFunction

[Contents]

4.5.143. besScribaSetReportPointer(F0,F1)

This macro calls scriba_SetReportPointer

[Contents]

4.5.144. besScribaSetStdin(F0,F1)

This macro calls scriba_SetStdin

[Contents]

4.5.145. besScribaSetStdout(F0,F1)

This macro calls scriba_SetStdout

[Contents]

4.5.146. besScribaSetEmbedPointer(F0,F1)

This macro calls scriba_SetEmbedPointer

[Contents]

4.5.147. besScribaSetEnvironment(F0,F1)

This macro calls scriba_SetEnvironment

[Contents]

4.5.148. besScribaLoadBinaryProgram(F0)

This macro calls scriba_LoadBinaryProgram

[Contents]

4.5.149. besScribaInheritBinaryProgram(F0,F1)

This macro calls scriba_InheritBinaryProgram

[Contents]

4.5.150. besScribaReadSource(F0)

This macro calls scriba_ReadSource

[Contents]

4.5.151. besScribaDoLexicalAnalysis(F0)

This macro calls scriba_DoLexicalAnalysis

[Contents]

4.5.152. besScribaDoSyntaxAnalysis(F0)

This macro calls scriba_DoSyntaxAnalysis

[Contents]

4.5.153. besScribaBuildCode(F0)

This macro calls scriba_BuildCode

[Contents]

4.5.154. besScribaIsFileBinaryFormat(F0)

This macro calls scriba_IsFileBinaryFormat

[Contents]

4.5.155. besScribaGetCacheFileName(F0)

This macro calls scriba_GetCacheFileName

[Contents]

4.5.156. besScribaUseCacheFile(F0)

This macro calls scriba_UseCacheFile

[Contents]

4.5.157. besScribaSaveCacheFile(F0)

This macro calls scriba_SaveCacheFile

[Contents]

4.5.158. besScribaRunExternalPreprocessor(F0,F1)

This macro calls scriba_RunExternalPreprocessor

[Contents]

4.5.159. besScribaSaveCode(F0,F1)

This macro calls scriba_SaveCode

[Contents]

4.5.160. besScribaSaveCCode(F0,F1)

This macro calls scriba_SaveCCode

[Contents]

4.5.161. besScribaLoadSourceProgram(F0)

This macro calls scriba_LoadSourceProgram

[Contents]

4.5.162. besScribaRun(F0,F1)

This macro calls scriba_Run

[Contents]

4.5.163. besScribaNoRun(F0)

This macro calls scriba_NoRun

[Contents]

4.5.164. besScribaResetVariables(F0)

This macro calls scriba_ResetVariables

[Contents]

4.5.165. besScribaCall(F0,F1)

This macro calls scriba_Call

[Contents]

4.5.166. besScribaCallArg(F0,F1,F2,F3)

This macro calls scriba_CallArg

[Contents]

4.5.167. besScribaDestroySbArgs(F0,F1,F2)

This macro calls scriba_DestroySbArgs

[Contents]

4.5.168. besScribaNewSbArgs(F0,F1,F2)

This macro calls scriba_NewSbArgs

[Contents]

4.5.169. besScribaCallArgEx(F0,F1,F2,F3,F4)

This macro calls scriba_CallArgEx

[Contents]

4.5.170. besScribaLookupFunctionByName(F0,F1)

This macro calls scriba_LookupFunctionByName

[Contents]

4.5.171. besScribaLookupVariableByName(F0,F1)

This macro calls scriba_LookupVariableByName

[Contents]

4.5.172. besScribaGetVariableType(F0,F1)

This macro calls scriba_GetVariableType

[Contents]

4.5.173. besScribaGetVariable(F0,F1,F2)

This macro calls scriba_GetVariable

[Contents]

4.5.174. besScribaSetVariable(F0,F1,F2,F3,F4,F5,F6)

This macro calls scriba_SetVariable

[Contents]

4.5.175. besLogState(X)

This macro calls log_state

[Contents]

4.5.176. besLogInit(F0,F1,F2,F3,F4,F5)

This macro calls log_init

[Contents]

4.5.177. besLogPrintf(pLOG,FORMAT, ...)

This macro calls log_printf

[Contents]

4.5.178. besLogShutdown(pLOG)

This macro calls log_shutdown

[Contents]

4.5.179. besHandleGetHandle(X,Y)

This macro calls handle_GetHandle. The memory segment used by the macro is the default.

Example:

  void *H;
  int i;
    ....
  i = besHandleGetPointer(H,pointer);

[Contents]

4.5.180. besHandleGetPointer(X,Y)

This macro calls handle_GetPointer. The memory segment used by the macro is the default.

Example:

  void *H,*pointer;
    ....
  pointer = besHandleGetPointer(H,handle);

[Contents]

4.5.181. besHandleFreeHandle(X,Y)

This macro calls handle_FreeHandle

[Contents]

4.5.182. besHandleDestroyHandleArray(X)

This macro calls handle_DestroyHandleArray

[Contents]

4.5.183. besINIT_SEGMENT(MAF,MRF)

This macro calls the function alloc_InitSegment

[Contents]

4.5.184. besSEGMENT_LIMIT(PMS,L)

This macro calls the function alloc_SegmentLimit

[Contents]

4.5.185. besFREE_SEGMENT(PMS)

This macro calls the function alloc_FreeSegment

[Contents]

4.5.186. besFINISH_SEGMENT(PMS)

This macro calls the function alloc_FinishSegment

[Contents]

4.5.187. besFUNCTION(X)

Use this macro to start an extension module interface function. The macro argument X is the name of the function as it is used in the BASIC statement

declare sub ::Function alias "X" lib "module"

This macro handles all system dependant function decoration declarations and declares the argument variables. Altough it is possible to name the argument variables in a different way it is strongly recommended that the programmer writing external module uses this macro and thus uses the argument names, because the besXXX macros rely on these variable names.

[Contents]

4.5.188. besASSERT_FUNCTION

Use this macro to check inside a besFUNCTION that the function was called as a sub and not command. If the include file declares the function as

declare command XXX alias "xxx" lib "library"

instead of

declare sub XXX alias "xxx" lib "library"

then you can not execute the rest of the code safely. This macro returns with the error code COMMAND_ERROR_BAD_CALL if the function was declared the wrong way.

[Contents]

4.5.189. besCOMMAND(X)

Use this macro to start an extension module interface command. The macro argument X is the name of the command as it is used in the BASIC statement

declare command ::Function alias "X" lib "module"

This macro handles all system dependant function decoration declarations and declares the argument variables. Altough it is possible to name the argument variables in a different way it is strongly recommended that the programmer writing external module uses this macro and thus uses the argument names, because the besXXX macros rely on these variable names.

In addition to the arguments this macro also declares some mandatory local variables thatshould be used in most of the module implemented commands and are used by some macros.

Note that interface functions get their arguments already evaluated while interface commands may decide if an argument is evaluated or not, or even evaluated multiple times.

[Contents]

4.5.190. besASSERT_COMMAND

Use this macro to check inside a besCOMMAND that the command was called as a command. If the include file declares the function as

declare sub XXX alias "xxx" lib "library"

instead of

declare command XXX alias "xxx" lib "library"

then you can not execute the rest of the code safely. This macro returns with the error code COMMAND_ERROR_BAD_CALL if the function was declared the wrong way.

[Contents]

4.5.191. besEND_COMMAND

Use this macro to finish an extension module command.

Note that this macro uses the macro FINISH that is empty by default. However a command may decide to perform some action after the mortals are released. To do so the macro FINISH can be redefined.

[Contents]

4.5.192. besARGNR

This macro returns the number of arguments passed to the extension module interface function.

[Contents]

4.5.193. besARGUMENT(X)

To access the function arguments the module can use the macro. The argument is the ordinal number of the argument starting from 1. You can overindex the arguments. In that case the macro value is NULL.

Note that this macro can only be used in interface functions and not in interface commands.

[Contents]

4.5.194. besPARAMETERLIST

Get the ordinal number of the node, where the parameter list starts for the extenal module interface command. This macro should not be used in interface functions only in interface commands.

[Contents]

4.5.195. besLEFTVALUE(X,Y)

Use this macro to evaluate an argument as left value in an interface command. This macro should not be used in interface functions.

[Contents]

4.5.196. besVERSION_NEGOTIATE

Use this macro to start the module interface version negotiation function. The simplest example is:

besVERSION_NEGOTIATE
  return (int)INTERFACE_VERSION;
besEND

[Contents]

4.5.197. besSUB_START

Use this macro to start the module initialization function.

besSUB_START
  ....
besEND

[Contents]

4.5.198. besSUB_FINISH

Use this macro to start the module finalization function.

besSUB_FINISH
  ....
besEND

[Contents]

4.5.199. besSUB_ERRMSG

Use this macro to start the error message function of an external module.

besSUB_ERRMSG
  ....
besEND

[Contents]

4.5.200. besSUB_PROCESS_START

Use this macro to start the function that will be invoked when the module is loaded by the operating system. This is _init under UNIX and DllMain under Windows NT. Using this macro the programmer can hide the OS dependant code.

[Contents]

4.5.201. besSUB_PROCESS_FINISH

Use this macro to start the function that will be invoked when the module is unloaded by the operating system. This is _fini under UNIX and DllMain under Windows NT. Using this macro the programmer can hide the OS dependant code.

[Contents]

4.5.202. besSUB_KEEP

Use this macro to start the module kepper function. This function should return 1 when the module wants to remain in memory and 0 when the module can be unloaded.

[Contents]

4.5.203. besSUB_SHUTDOWN

Use this macro to start a module shutdown function.

This shutdown function is called before a module is unloaded from the process space. The function is similar to besSUB_FINISH. That function is called when the interpreter finishes. When there are many interpreter threads in a single process that uses the module the function besSUB_FINISH is called each time an interpreter finishes. The function besSUB_SHUTDOWN is called only once, before the interpreter unloads the extesion from memory.

The difference between besSUB_SHUTDOWN and besSUB_PROCESS_FINISH is that besSUB_SHUTDOWN is called by the interpreter, besSUB_PROCESS_FINISH is called by the operating system. besSUB_SHUTDOWN can access the support functions because it gets the pSt argument, besSUB_PROCESS_FINISH can not access these functions.

When a single thread interpreter finishes it first calls the function besSUB_FINISH to unload the module and after that it calls besSUB_SHUTDOWN.

This is not an error if a module does not implement these functions.

The function should return COMMAND_ERROR_SUCCESS if the module has no remaining activity and is ready to be unloaded.

The function should return COMMAND_ERROR_STAYS_IN_MEMORY if there are unstopped threads that use the module code. In this case unloading the module would cause segmentation fault that would interfere with the still running shutdown procedures. In that case the module is not unloaded by the program, but only when the process finishes by the operating system.

[Contents]

4.5.204. besSUB_AUTO

Use this macro to start the external module autoloader function.

[Contents]

4.5.205. besEND

Use this macro to close an extension module interface function.

[Contents]

4.5.206. besRETURNVALUE

Use this macro to access the extension function or command return value pointer. Assign allocated mortal variable to this pointer.

[Contents]

4.5.207. besMODULEPOINTER

Use this macro to access the extension module pointer.

[Contents]

4.5.208. besALLOC_RETURN_STRING(X)

Use this macro to allocate a string as a return value. If there is not enough space to store the result the macro returns from the function with the error code COMMAND_ERROR_MEMORY_LOW.

The argument should be the number of bytes of the return value. After using this macro the macro STRINGVALUE(besRETURNVALUE) can be used to access the byte-buffer of the return value. Usually the program uses memcpy to copy the bytes there.

[Contents]

4.5.209. besALLOC_RETURN_LONG

Use this macro to allocate a long as a return value. If there is not enough space to store the result the macro returns from the function with the error code COMMAND_ERROR_MEMORY_LOW.

After using this macro the macro LONGVALUE(besRETURNVALUE) can be used to access the long value of the return value.

[Contents]

4.5.210. besALLOC_RETURN_DOUBLE

Use this macro to allocate a double as a return value. If there is not enough space to store the result the macro returns from the function with the error code COMMAND_ERROR_MEMORY_LOW.

After using this macro the macro DOUBLEVALUE(besRETURNVALUE) can be used to access the double value of the return value.

[Contents]

4.5.211. besSETCOMMAND(X,Y)

Use this macro to alter the command table. X is the command code and Y is the new function implementing the command.

[Contents]

4.5.212. besGETCOMMAND(X)

Use this macro to get the command function currently assigned to the command X.

[Contents]

4.5.213. INTERFACE_VERSION

The current external module interface version. This is an integer number.

[Contents]

4.5.214. besHOOK_FILE_ACCESS

This macro calls the function hook_file_access

[Contents]

4.5.215. besHOOK_FOPEN

This macro calls the function hook_fopen

[Contents]

4.5.216. besHOOK_FCLOSE

This macro calls the function hook_fclose

[Contents]

4.5.217. besHOOK_SIZE

This macro calls the function hook_size

[Contents]

4.5.218. besHOOK_TIME_ACCESSED

This macro calls the function hook_time_accessed

[Contents]

4.5.219. besHOOK_TIME_MODIFIED

This macro calls the function hook_time_modified

[Contents]

4.5.220. besHOOK_TIME_CREATED

This macro calls the function hook_time_created

[Contents]

4.5.221. besHOOK_ISDIR

This macro calls the function hook_isdir

[Contents]

4.5.222. besHOOK_ISREG

This macro calls the function hook_isreg

[Contents]

4.5.223. besHOOK_EXISTS

This macro calls the function hook_fileexists

[Contents]

4.5.224. besHOOK_TRUNCATE

This macro calls the function hook_truncate

[Contents]

4.5.225. besHOOK_FGETC

This macro calls the function hook_fgetc

[Contents]

4.5.226. besHOOK_FREAD

This macro calls the function hook_fread

[Contents]

4.5.227. besHOOK_FWRITE

This macro calls the function hook_fwrite

[Contents]

4.5.228. besHOOK_FERROR

This macro calls the function hook_ferror

[Contents]

4.5.229. besHOOK_PUTC

This macro calls the function hook_fputc

[Contents]

4.5.230. besHOOK_FLOCK

This macro calls the function hook_flock

[Contents]

4.5.231. besHOOK_LOCK

This macro calls the function hook_lock

[Contents]

4.5.232. besHOOK_FEOF

This macro calls the function hook_feof

[Contents]

4.5.233. besHOOK_MKDIR

This macro calls the function hook_mkdir

[Contents]

4.5.234. besHOOK_RMDIR

This macro calls the function hook_rmdir

[Contents]

4.5.235. besHOOK_REMOVE

This macro calls the function hook_remove

[Contents]

4.5.236. besHOOK_DELTREE

This macro calls the function hook_deltree

[Contents]

4.5.237. besHOOK_MAKEDIRECTORY

This macro calls the function hook_MakeDirectory

[Contents]

4.5.238. besHOOK_OPENDIR

This macro calls the function hook_opendir

[Contents]

4.5.239. besHOOK_READDIR

This macro calls the function hook_readdir

[Contents]

4.5.240. besHOOK_CLOSEDIR

This macro calls the function hook_closedir

[Contents]

4.5.241. besHOOK_SLEEP

This macro calls the function hook_sleep

[Contents]

4.5.242. besHOOK_CURDIR

This macro calls the function hook_curdir

[Contents]

4.5.243. besHOOK_CHDIR

This macro calls the function hook_chdir

[Contents]

4.5.244. besHOOK_CHOWN

This macro calls the function hook_chown

[Contents]

4.5.245. besHOOK_SETCREATETIME

This macro calls the function hook_SetCreateTime

[Contents]

4.5.246. besHOOK_SETMODIFYTIME

This macro calls the function hook_SetModifyTime

[Contents]

4.5.247. besHOOK_SETACCESSTIME

This macro calls the function hook_SetAccessTime

[Contents]

4.5.248. besHOOK_GETHOSTNAME

This macro calls the function hook_GetHostName

[Contents]

4.5.249. besHOOK_GETHOST

This macro calls the function hook_GetHost

[Contents]

4.5.250. besHOOK_TCPCONNECT

This macro calls the function hook_TcpConnect

[Contents]

4.5.251. besHOOK_TCPSEND

This macro calls the function hook_TcpSend

[Contents]

4.5.252. besHOOK_TCPRECV

This macro calls the function hook_TcpRecv

[Contents]

4.5.253. besHOOK_TCPCLOSE

This macro calls the function hook_TcpClose

[Contents]

4.5.254. besHOOK_KILLPROC

This macro calls the function hook_KillProc

[Contents]

4.5.255. besHOOK_GETOWNER

This macro calls the function hook_GetOwner

[Contents]

4.5.256. besHOOK_CREATEPROCESS

This macro calls the function hook_CreateProcess

[Contents]

4.5.257. besHOOK_CALLSCRIBAFUNCTION

This macro calls the function hook_CallScribaFunction

[Contents]

4.5.258. besSETHOOK(X,Y)

Use this macro to alter the hook function table.

[Contents]

4.5.259. besDLL_MAIN

process header macro makes UNIX like dll loading and unloading on Win32. This just defines a wrapper DllMain that calls _init() and _fini() that a the default library loading and unloading functions under UNIX.

Use of this macro may be needed for modules that serve multi thread interpreters and share resources on the process level

[Contents]

4.5.260. INITLOCK

Whent he process first time loads an extension and the extension wants to decide whether to unload it or keep in memory it needs a counter and a mutex to access the counter to declare the counter the extension should use the macro SUPPORT_MULTITHREAD and to initialize it it should use the macro INIT_MULTITHREAD

Note that the call-back functions to handle the mutexes OS independant are not available by the time when the OS calls DllMain on NT or _init on UNIX, thus the code should call system dependant functions directly

IsThisTheVeryFirstThreadCallingTheModule <- name of the function

[Contents]

4.5.261. basext_GetArgsF()

This function can be used to get arguments simple and fast in extension modules. All functionality of this function can be individually programmed using the besXXX macros. Here it is to ease the programming of extension modules for most of the cases.

This function should be called like

  iError = besGETARGS "ldz",&l1,&d1,&s besGETARGE

The macro besGETARGS (read GET ARGument Start) hides the complexity of the function call and the macro besGETARGE (read Get ARGument End) simply closes the function call.

The first argument is format string. Each character specifies how the next argument should be treated. The following characters are recognized:

int basext_GetArgsF(pSupportTable pSt,
                    pFixSizeMemoryObject pParameters,
                    char *pszFormat,
                    ...
  ){

[Contents]

5. Preprocessors

ScriptBasic is capable handling two kind of preprocessors. One is external preprocessors, the other one is internal preprocessor. The names external and the internal distinguish between the execution type of these preprocessors. External preprocessors are executed in a separate process. Internal preprocessors run in the interpreter thread.

Because of this external preprocessors are standalone command line tools, which may be written for any application and not specifically for ScriptBasic. You can edit the ScriptBasic configuration file so that you can use the C preprocessor, m4 or jamal as a preprocessor. It is fairly easy to write an external preprocessor compared to internal preprocessors. External preprocessor reads a file and creates an output file. It need not know anything about the internal structures of ScriptBasic. Then only thing a ScriptBasic external preprocessor writer has to know is what it wants to do and how to read and write files.

Internal preprocessors are implemented in dynamic link libraries, work closely together with ScriptBasic and can not be started standalone. Internal preprocessors are written specifically for ScriptBasic and can and should access many of ScriptBasic internal structures.

From this you can see that external preprocessors are much easier to write, while internal preprocessors have much more possibilities. An internal preprocessor can never started as external and vice versa.

Before starting to write a preprocessor you have to carefully check what you want to gain and decide if you want to write an external preprocessor or an internal.

Because external preprocessors are just standalone programs and there is even a sample preprocessor HEB written in BASIC this chapter talks about the internal preprocessor capabilities. External Preprocessor

[Contents]

5.1. Loading Preprocessor

Loading a preprocessor depends on the embedding application. To load an internal preprocessor the function ipreproc_LoadInternalPreprocessor is called (implemented in the file `ipreproc.c').

This function gets the name of an external preprocessor to load. The function searches the configuration information for the named preprocessor, loads the DLL/SO and invokes the initiation function of the preprocessor.

int ipreproc_LoadInternalPreprocessor(pPreprocObject pPre,
                                      char *pszPreprocessorName);

The first argument is the pointer to the ScriptBasic preprocessor object to access the configuration information and the list of loaded preprocessors to put the actual one on the list.

The second argument is the name of the preprocessor as named in the configuration file, for example

preproc (
  internal (
    sample "C:\\ScriptBasic\\bin\\samplepreprocessor.dll"
    )

The return value is zero or the error code.

(Note that this documentation may not be up-to date about the precise functioning of this function. For most up-to date information see the source documentation that is extracted from the source comment using the tool `esd2html.pl'.)

In the code base of ScriptBasic this function is called by the reader `reader.c' in the function reader_LoadPreprocessors. This is called automatically when the source is read and include files were also included. This function (reader_LoadPreprocessors) goes through all the lines and searches for lines that start with the word preprocess and name a preprocessor. The function loads the preprocessor and deletes the source line.

The function ipreproc_LoadInternalPreprocessor is also called from the function scriba_LoadInternalPreprocessor in source file `scriba.c'

This function can and should be used by the embedding programs to load all internal preprocessors that are to be loaded based on some external conditions. For example the VARIATION STANDARD of ScriptBasic (aka. the command line embedding variation) loads all internal preprocessors that were named after the command line option `-i'.

Other embedding application may get the information of desired preprocessors from different sources, like environment variables, configuration files and so on.

Note that internal preprocessors are not used whenever an already compiled version of the program is executing. If there is a preprocess line in the source code and the program has generated a cache or any other binary format BASIC program and that file is used to execute the program the preprocessor will no effect. The interpreter will not load the preprocessor and thus it will act as it did not exist.

There are two kind of preprocessors:

Altering preprocessors are BASIC program specific and are usually invoked because the program contains a line preprocess.

Debugger type preprocessors are loaded the way of the execution of the program requests it. For example the user uses the option `-i'. In this case the command line version of ScriptBasic does not use cache file to ensure that the program really does using the preprocessor and starts the debugger, for example.

[Contents]

5.2. Skeleton of a Preprocessor

An internal preprocessor implemented as a .dll or .so file has to export a single function named preproc. The declaration of this function should look like this:

int DLL_EXPORT preproc(pPrepext pEXT,
                       long *pCmd,
                       void *p);

The first argument of the function is the preprocessor pointer. This pointer points to a structure. For each preprocessor loaded there is a separate structure of this type. This structure hold information on the preprocessor and there is some storage pointer in the structure that the preprocessor can access.

The definition of the structure is:

typedef struct _Prepext {
  long lVersion;
  void *pPointer;
  void *pMemorySegment;
  struct _SupportTable *pST;
  } Prepext, *pPrepext;

The field lVersion is the version of the interface that ScriptBasic wants to use when communicating with the preprocessor. If this version is not the same as the interface version that the preprocessor was compiled for then the preprocessor has to ask ScriptBasic not to use it and may return an error code or zero indicating that no error has happened. This may be useful in some cases when the preprocessor is optional and in case the preprocessor can not be loaded the program still may function. Read on how the preprocessor has to do this.

The field pPointer is initialized to NULL and is never changed by ScriptBasic. This is a pointer that can be used by the preprocessor to access its own thread local variables.

The field pMemorySegment is initialized to point to a memory segment that can be used via the memory allocation routines of ScriptBasic. These routines, however need not be linked to the DLL, because they are already in the process loaded as part of the executable and can be reached via the function support table.

This support table is pointed by the field pST and is the same type of struct as the support table available for the extension modules. Although this support table is the same type of struct it is not the same struct. The fields pointing to the different functions are eventually pointing to the same function, but when the program starts to execute a new support table is allocated and initialized. Thus there is no reason for the preprocessor to alter this table, because altering the table will have no effect on the execution of the program. Also the preprocessor if decides to stay in memory while the program is executed (for example a debugger), may rely on this support table even if a module altering the run-time support table is used.

In case there are more than one internal preprocessors used they will share the same support table. This way a preprocessor altering the preprocessor support table may alter the behavior of another internal preprocessor. However doing that need deep and detailed information of both ScriptBasic code and the other preprocessor and may result code that closely depends on the different versions of the different programs that work together.

The next argument to the function pCmd is an input and output variable. When the function is called by ScriptBasic it contains the command that the preprocessor is expected to perform. In other words this value defines the reason why the preprocessor was loaded. There are numerous points when ScriptBasic calls the preprocessor and at each point it sets this variable differently.

When the function returns it is supposed to set the variable *pCmd to one of the following values:

The preprocessors function preproc may at any call return an int value. This value will be used by the interpreter as error code. This code is zero in case there was no error. This value has to be returned to ensure that the interpreter goes on. The error code 1 is used at any code in ScriptBasic to signal memory allocation problems. The symbolic constant COMMAND_ERROR_PREPROCESSOR_ABORT can be used to tell the interpreter to stop. For example the sample debugger preprocessor uses this error code when the user gives the command q to quit debugging.

[Contents]

5.3. Preprocessor Entry Points

Lets recall the prototype of the preprocessor function, which has to be implemented in each preprocessor:

int DLL_EXPORT preproc(pPrepext pEXT,
                       long *pCmd,
                       void *p);

This function has to be implemented in each internal preprocessor and is called when the preprocessor is loaded or when some the processing of the source program has reached a certain point. To inform the function about this point the argument pCmd is used. This argument points to a long that holds the a constant identifying the reason why the preprocessor function was called. The following subsections list these identifiers.

[Contents]

5.3.1. PreprocessorLoad

This entry point is used when the preprocessor is loaded. The pointer p is NULL.

When the preprocessor function is called with this argument it can be sure that this is the very first call to the function within the actual interpreter thread. It also can depend on the support function table and on the preprocessor memory segment pointer being initialized and ready to allocate memory.

It has to check that the version the preprocessor was designed and compiled for is appropriate and works together with the ScriptBasic interpreter that invoked the preprocessor. This can easily be done checking the version information in the preprocessor structure. Sample code:

      if( pEXT->lVersion != IP_INTERFACE_VERSION ){
        *pCmd = PreprocessorUnload;
        return 0;
        }

This code is the appropriate place to allocate space for the preprocessor structure that hold the thread local variables. For example:

      pDO = pEXT->pST->Alloc(sizeof(DebuggerObject),pEXT->pMemorySegment);
      *pCmd = PreprocessorUnload;
      if( pDO == NULL )return 1;

Note that this example is a simplified version of the one that you can find in the sample debugger preprocessor. This example uses the Alloc support function that is usually points to the function alloc_Alloc implemented in the file `myalloc.c'. When coding an external preprocessor you can rely on ScriptBasic that as soon as the preprocessor is unloaded the memory allocated using this function with the memory segment initiated for the preprocessor (like above) will be released.

The preprocessor has to return the error code or zero and may alter the value of the parameter *pCmd to PreprocessorContinue or PreprocessorUnload.

Although the calling code ignores the value returned in *pCmd unless it is PreprocessorUnload it is required by principle to set a value in this variable. The value PreprocessorDone can not be used when returning from this entry.

[Contents]

5.3.2. PreprocessorReadStart

This entry point is used before the source file reading starts. The pointer p points to the ReadObject used to read the source files. There is few uses of this entry point. You may alter the file handling function pointers or the memory allocation function pointers in the ReadObject.

This entry point is invoked only when the preprocessor load was initiated by external conditions, like command line option `-i' and never if the source code contained the preprocessor loading directive. This is because when the reader realizes that the preprocessor has to be loaded it is far over this point.

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.3. PreprocessorReadDone0

This entry point is used when the reader has done the reading of the source file, but did not do any processing further. The parameter p points to the ReadObject. The preprocessor at this point can access the lines of the core BASIC program file without the included files and the very first line of the program that may be a Windows NT or UNIX special line (like #!/usr/bin/scriba) is still in the input lines.

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.4. PreprocessorReadDone1

This entry point is used when the reader has done the reading the source file and unhooked the optional first Windows NT or UNIX start line (like #!/usr/bin/scriba), but did not process the included files. The preprocessor at this point can access the lines of the core BASIC program file without the included files. The parameter p points to the ReadObject.

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.5. PreprocessorReadDone2

This entry point is used when the reader has done the reading the source file and unhooked the optional first Windows NT or UNIX start line (like #!/usr/bin/scriba), and has processed the included files. The preprocessor at this point can access the lines of the full BASIC program file with the included files. The parameter p points to the ReadObject.

This is the last point before the source code directive loaded preprocessors are invoked.

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.6. PreprocessorReadDone3

This point is used when the reader has done all reading tasks, processed and linked the include files, has removed the first Windows NT or UNIX specific line, and loaded the preprocessors that were to be loaded by program directive. The parameter p points to the ReadObject.

At this point the preprocessor may check the source code and alter it according to its need. All processing should be done here that needs the characters of the source file. Careful decision has to be made whether using this point of entry to alter the source file or the entry point PreprocessorLexDone when the source file is already tokenized.

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.7. PreprocessorLexInit

This entry point is used when the lexer object was allocated and initialized. The pointer p points to the LexObject.

The preprocessor at this point may alter the LexObject parameters. Here is a copy of the function lex_InitStructure from ScriptBasic v1.0build26. (other versions may slightly differ).

void lex_InitStructure(pLexObject pLex
  ){
/*noverbatim
CUT*/
  pLex->pfGetCharacter = NULL;
  pLex->pfFileName = _MyFileName;
  pLex->pfLineNumber = _MyLineNumber;
  pLex->SSC = "QWERTZUIOPASDFGHJKLYXCVBNMqwertzuiopasdfghjklyxcvbnm_:$";
  pLex->SCC = "QWERTZUIOPASDFGHJKLYXCVBNMqwertzuiopasdfghjklyxcvbnm_1234567890:$";
  pLex->SFC = "QWERTZUIOPASDFGHJKLYXCVBNMqwertzuiopasdfghjklyxcvbnm_1234567890$";
  pLex->SStC = "\"";
  pLex->ESCS = "\\n\nt\tr\r\"\"\'\'";
  pLex->fFlag = LEX_PROCESS_STRING_NUMBER       |
                LEX_PROCESS_STRING_OCTAL_NUMBER |
                LEX_PROCESS_STRING_HEX_NUMBER   |
                0;
  pLex->SKIP = " \t\r"; /* spaces to skip 
                           \r is included to ease compilation of DOS edited 
                           binary transfered files to run on UNIX */
  pLex->pNASymbols = NULL;
  pLex->pASymbols  = NULL;
  pLex->pCSymbols  = NULL;
  pLex->cbNASymbolLength = 0; /* it is to be calculated */

pLex->buffer = lexALLOC(BUFFERINCREASE*sizeof(char));

if( pLex->buffer ) pLex->cbBuffer = BUFFERINCREASE; else pLex->cbBuffer = 0;

CALL_PREPROCESSOR(PreprocessorLexInit,pLex); }

(Note CALL_PREPROCESSOR is a macro that call the preprocessor with appropriate arguments.)

The preprocessor may decide for example to alter the string SSC that contains all characters that may start a symbol, or SCC that contains all characters that can part a symbol or SFC that contains all characters that can be the final character of a symbol. This way for example a preprocessor may set these strings that allows Hungarian programmers to use ISO-Latin-2 accented letters in their variables. (However those characters are going to be case sensitive.)

The preprocessor may also set the pointers that point to the tables that contains the alphanumeric symbols (pASymbols), non-alpha symbols (pNASymbols) and the table used for some ScriptBasic internal debugging purpose (pCSymbols).

The preprocessor may also release and reallocate a smaller or larger buffer if wishes. (I personally see no reason.)

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.8. PreprocessorLexDone

This entry point is used when the lexer has finished the lexical analysis and the list of tokens is already in the memory. The pointer p points to the LexObject.

At this point the preprocessor may alter the tokenized form of the BASIC program. The list of tokens still contains the comment lines (also tokenized although it may make no sense), and the continuation lines are still being split containing the _ character and the new-line token.

The preprocessor may gain information from comments in case the some comments provide information for the preprocessor.

If the preprocessor uses some special symbols that drive the preprocessor processing but should be removed from the token list the preprocessor at this point may unlink the token from the list or just set the type of the token to LEX_T_SKIP or LEX_T_SKIP_SYMBOL.

If you do not or do not want to understand the difference between the two possibilities described soon then the rule of thumb is to use LEX_T_SKIP and you are safe.

The type LEX_T_SKIP should be used in case the token is handled due to ProcessLexSymbol preprocessor command and LEX_T_SKIP otherwise.

When the type is set LEX_T_SKIP_SYMBOL the lexical analyzer knows to release the string holding the symbol. If the type is LEX_T_SKIP only the token record is released.

If the symbol string is not released due to erroneously setting the type to LEX_T_SKIP instead LEX_T_SKIP_SYMBOL the memory will not be released until the interpreter finishes pre execution steps. So usually if you do not know how to set the type to skip a token LEX_T_SKIP is safe.

When processing comments the preprocessor should either use only the comments starting with the keyword rem or should carefully detect the comments starting with '.

For more information how to do it you really have to look at the function lex_RemoveComments in the file `lexer.c'.

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.9. PreprocessorLexNASymbol

This entry point is used when the lexer has found a non alpha symbol. Non alpha symbols are predefined character strings, like <> or <= that contain more than one character but are not alpha characters.

When this entry point is called the pointer p points to a pointer that point to a pointer pointing to the last processed lexeme. This seems to be quite complex and uselessly complex to pass such a pointer at the firstglance. However it is not.

When the lexer builds the list of the lexemes reading the characters it creates a linked list of structures of type Lexeme. The lexer object field pLexResult points to the first element of this list. The lexer code uses a local variable named plexLastLexeme that first points to this pointer, and later it always points to the forward link pointer of the last element of the list.

When the preprocessor is called using this entry point this variable passed in the argument p "by value". Through this pointer you can

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.10. PreprocessorLexASymbol

This entry point is used when the lexer has found an alpha symbol. Alpha symbols are the keywords that are predefined in ScriptBasic. When this entry point is called the pointer p points to a pointer that point to a pointer pointing to the last processed lexeme. This seems to be quite complex and uselessly complex to pass such a pointer at the firstglance. However it is not.

When the lexer builds the list of the lexemes reading the characters it creates a linked list of structures of type Lexeme. The lexer object field pLexResult points to the first element of this list. The lexer code uses a local variable named plexLastLexeme that first points to this pointer, and later it always points to the forward link pointer of the last element of the list.

When the preprocessor is called using this entry point this variable passed in the argument p "by value". Through this pointer you can

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.11. PreprocessorLexSymbol

This entry point is used when the lexer finds a symbol that is alphanumeric but is not predefined by ScriptBasic. These are the variables and symbols (like the statement OPEN opening modes.)

When this entry point is called the pointer p points to a pointer that point to a pointer pointing to the last processed lexeme. This seems to be quite complex and uselessly complex to pass such a pointer at the firstglance. However it is not.

When the lexer builds the list of the lexemes reading the characters it creates a linked list of structures of type Lexeme. The lexer object field pLexResult points to the first element of this list. The lexer code uses a local variable named plexLastLexeme that first points to this pointer, and later it always points to the forward link pointer of the last element of the list.

When the preprocessor is called using this entry point this variable passed in the argument p "by value". Through this pointer you can

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.12. PreprocessorLexString

This entry point is used when the preprocessor has processed a single-line string.

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.13. PreprocessorLexMString

This entry point is used when the preprocessor has processed a multi-line string. When this entry point is called the pointer p points to a pointer that point to a pointer pointing to the last processed lexeme. This seems to be quite complex and uselessly complex to pass such a pointer at the firstglance. However it is not.

When the lexer builds the list of the lexemes reading the characters it creates a linked list of structures of type Lexeme. The lexer object field pLexResult points to the first element of this list. The lexer code uses a local variable named plexLastLexeme that first points to this pointer, and later it always points to the forward link pointer of the last element of the list.

When the preprocessor is called using this entry point this variable passed in the argument p "by value". Through this pointer you can

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.14. PreprocessorLexInteger

This entry point is called when the lexer has processed an integer.

When this entry point is called the pointer p points to a pointer that point to a pointer pointing to the last processed lexeme. This seems to be quite complex and uselessly complex to pass such a pointer at the firstglance. However it is not.

When the lexer builds the list of the lexemes reading the characters it creates a linked list of structures of type Lexeme. The lexer object field pLexResult points to the first element of this list. The lexer code uses a local variable named plexLastLexeme that first points to this pointer, and later it always points to the forward link pointer of the last element of the list.

When the preprocessor is called using this entry point this variable passed in the argument p "by value". Through this pointer you can

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.15. PreprocessorLexReal

This entry point is called when the lexer has processed an real number.

When this entry point is called the pointer p points to a pointer that point to a pointer pointing to the last processed lexeme. This seems to be quite complex and uselessly complex to pass such a pointer at the firstglance. However it is not.

When the lexer builds the list of the lexemes reading the characters it creates a linked list of structures of type Lexeme. The lexer object field pLexResult points to the first element of this list. The lexer code uses a local variable named plexLastLexeme that first points to this pointer, and later it always points to the forward link pointer of the last element of the list.

When the preprocessor is called using this entry point this variable passed in the argument p "by value". Through this pointer you can

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.16. PreprocessorLexCharacter

This entry point is called when the lexer has processed a character.

When this entry point is called the pointer p points to a pointer that point to a pointer pointing to the last processed lexeme. This seems to be quite complex and uselessly complex to pass such a pointer at the firstglance. However it is not.

When the lexer builds the list of the lexemes reading the characters it creates a linked list of structures of type Lexeme. The lexer object field pLexResult points to the first element of this list. The lexer code uses a local variable named plexLastLexeme that first points to this pointer, and later it always points to the forward link pointer of the last element of the list.

When the preprocessor is called using this entry point this variable passed in the argument p "by value". Through this pointer you can

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.17. PreprocessorExStart

This entry point is used when the syntax analyzer starts.

The argument p points to the actual peXobject syntax analysis object structure.

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.18. PreprocessorExStartLine

This entry point is used when the syntax analyzer starts to analyze a program line. The argument p points to the actual peXobject syntax analysis object structure. The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.19. PreprocessorExEnd

This entry point is used when the syntax analyzer has finished analyzing the basic program. The argument p points to the actual peXobject syntax analysis object structure. The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.20. PreprocessorExFinish

This entry point is called from the function scriba_DoSyntaxAnalysis implemented in the file `scriba.c' when the syntax analyzer has finished.

The only difference between this entry point and the entry point PreprocessorExEnd is that at this point the field pCommandList in the peXobject object structure already points to the list of nodes.

The argument p points to the actual peXobject syntax analysis object structure. The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.21. PreprocessorExStartLocal

This entry point is used when the syntax analyzer starts a new local scope. This is when a function or a sub starts.

The argument p points to the actual peXobject syntax analysis object structure. The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.22. PreprocessorExEndLocal

This entry point is used when the syntax analyzer exists a local scope. This is when a function or a sub ends.

The argument p points to the actual peXobject syntax analysis object structure. The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.23. PreprocessorExLineNode

This entry point is used when the syntax analyzer creates a new node for a basic program line.

The argument p points to the actual peXobject syntax analysis object structure. The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.24. PreprocessorExeStart

This entry point is used from the function scriba_Run implemented in the file `scriba.c' before the execution of the basic program starts.

The argument p points to the execution context of the basic program.

By this time most of the preprocessors should have asked the basic interpreter to unload. Only preprocessors implementing debugger, profiler or other development support functions may remain in memory and active.

At this very point debugger or other development support preprocessors may and should access the execution hook functions, like the sample debugger preprocessor does:

  case PreprocessorExeStart:

{ pExecuteObject pEo = p; pDebuggerObject pDO = pEXT->pPointer; pEo->pHookers->hook_pointer = pEXT; pDO->CallStackDepth = 0; pDO->DbgStack = NULL; pDO->StackTop = NULL; pEo->pHookers->HOOK_ExecBefore = MyExecBefore; pEo->pHookers->HOOK_ExecAfter = MyExecAfter; pEo->pHookers->HOOK_ExecCall = MyExecCall; pEo->pHookers->HOOK_ExecReturn = MyExecReturn; *pCmd = PreprocessorContinue; return 0; }

(Note that this is an example only and not the actual code. The actual code performs other tasks as well.)

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.25. PreprocessorExeFinish

This entry point is used from the function scriba_Run implemented in the file `scriba.c' after the execution of the basic program finished.

The argument p points to the execution context of the basic program.

By this time most of the preprocessors should have asked the basic interpreter to unload. Only preprocessors implementing debugger, profiler or other development support functions may remain in memory and active.

The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.

[Contents]

5.3.26. PreprocessorExeNoRun

This entry point is used from the function scriba_NoRun implemented in the file `scriba.c' before the execution of the basic program is not started. (Note that scriba_NoRun performs initialization for the execution but does not start the execution.)

By this time most of the preprocessors should have asked the basic interpreter to unload. Only preprocessors implementing debugger, profiler or other development support functions may remain in memory and active. The function has to return zero or the error code and should set the parameter *pCmd to PreprocessorContinue, PreprocessorDone, or PreprocessorUnload.