From:	CRDGW2::CRDGW2::MRGATE::"SMTP::CRVAX.SRI.COM::RELAY-INFO-VAX" 26-AUG-1989 03:24
To:	MRGATE::"ARISIA::EVERHART"
Subj:	Re: Flex (latest 2.1 beta) on VMS

Received: From KL.SRI.COM by CRVAX.SRI.COM with TCP; Fri, 25 AUG 89 23:49:46 PDT
Received: from ucbvax.Berkeley.EDU by KL.sri.com with TCP; Fri, 25 Aug 89 23:38:36 PDT
Received: by ucbvax.Berkeley.EDU (5.61/1.37)
	id AA25478; Fri, 25 Aug 89 23:36:31 -0700
Received: from USENET by ucbvax.Berkeley.EDU with netnews
	for info-vax@kl.sri.com (info-vax@kl.sri.com)
	(contact usenet@ucbvax.Berkeley.EDU if you have questions)
Date: 25 Aug 89 17:26:57 GMT
From: naucse!jdc@arizona.edu  (John Campbell)
Organization: Northern Arizona University, Flagstaff, AZ
Subject: Re: Flex (latest 2.1 beta) on VMS
Message-Id: <1664@naucse.UUCP>
References: <13603@bloom-beacon.MIT.EDU>
Sender: info-vax-request@kl.sri.com
To: info-vax@kl.sri.com

From article <13603@bloom-beacon.MIT.EDU>, by scs@adam.pika.mit.edu (Steve Summit):
: In article <1629@naucse.UUCP> jdc@naucse.UUCP (John Campbell) writes:
:>The new flex reads a large chunck at a
:>time.  With VMS STREAM-LF files this works just fine--but with "normal"
:>VFC editor text files (darn these RMS things) the VMS 'C' rtl will only
:>return at most 1 record full of characters for any large read() byte
:>request.
:>During processing on a flex input file, flex complains of a "NULL in
:>input."  This seems to be because yyunput() wants to "shift things up
:>to make room" and assumes that the end of the valid buffer is around
:>YY_BUF_SIZE deep.
: 
: This sounds like a bug in flex.  If I understand the complaint
: correctly, the code gets confused when the buffer is not (?)
: substantially full.  (This sounds odd; code usually fails when
: buffers fill up, not when they stay relatively empty.)

I'm the original poster of the first article.  To date I have not proven
that there is a bug in flex.  I still believe this is true, but I haven't
had the time to make the obvious tests in other environments.  I still
believe that yyunput() is in error in some way, but I would like to have
some follow up information before I bug Vern Paxson with my worries.

I can tell you that using VMS fread() instead of read() for initscan.c
seems to work just fine on all format of input files:  

#define YY_INPUT(buf,result,max_size) \
        if ( (result = fread(buf, 1, max_size, yyin)) == 0 ) \
            if (ferror(yyin))\
               YY_FATAL_ERROR( "fread() in flex scanner failed" );


I'm a little confused because, with no change to the yyunput() routine, I 
can run one of my old programs which has to replace YY_INPUT() with my
own read routine to toss out '\0's.  I find that this routine, which does
not try to fill up to max_size works for the regular expressions that
I analyze.  At this point I assume there is something more complicated
going on in the lexer for flex itself than in the small ditty I wrote.

Anyway, if anyone would make some YY_INPUT() substitutions (like use
getc()) and see what happens when they recompile and test flex itself
we'd all be very happy.  I work for a university that is about to start
classes so I'm overwhelmed right now.

-- 
	John Campbell               ...!arizona!naucse!jdc
                                    CAMPBELL@NAUVAX.bitnet
	unix?  Sure send me a dozen, all different colors.