yy_scan_string

[never2di]

I'm a newbie to Bison + Flex. and I'm having trouble in getting yy_scan_string to work. I have tried the following

YY_BUFFER_STATE my_string_buffer = yy_scan_string(my_string);
yyparse();
yy_delete_buffer(my_string_buffer );

the parser errors out with a syntax error at the first token. I have verified that the grammar and the content of 'my_string' works using yyrestart(yyin) and also with yy_create_buffer() through a file.

Is there something special state that I have to set flex in before I call yyparse()?
if not what is that I'm doing wrong and what needs to be done?

bison (GNU Bison) 2.4  

Thanks

[Philip Herron]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

never2di wrote:

> I'm a newbie to Bison + Flex. and I'm having trouble in getting
> yy_scan_string to work. I have tried the following
>
> YY_BUFFER_STATE my_string_buffer = yy_scan_string(my_string);
> yyparse(); yy_delete_buffer(my_string_buffer );
>
> the parser errors out with a syntax error at the first token. I
> have verified that the grammar and the content of 'my_string' works
> using yyrestart(yyin) and also with yy_create_buffer() through a
> file.
>
> Is there something special state that I have to set flex in before
> I call yyparse()? if not what is that I'm doing wrong and what
> needs to be done?
>
> bison (GNU Bison) 2.4
>
> Thanks :handshake:
>
>
>

Hey

You might find this useful i do this in my lexer:

int parse( FILE* fin )
{
  int err= 0;
  if( fin == stdin )
    {
#ifdef HAVE_LIBREADLINE
      char *line, *prompt= ">>> ";
#else
      char line[256];
#endif
      YY_BUFFER_STATE bp;

      while( 1 )
    {
#ifdef HAVE_LIBREADLINE
      line= readline(prompt);
#else
      fgets(line, sizeof(line), stdin);
#endif
      if( line == 0 )
        break;
      else if( strlen(line) > 0)
        {
#ifdef HAVE_READLINE_HISTORY
          add_history(line);
#endif
          bp= yy_scan_string( line );
          yy_switch_to_buffer(bp);
          err= yyparse();
          yy_delete_buffer(bp);
        }
    }
    }
  else
    {
      yyin= fin;
      err= yyparse();
    }
  return err;
}

The bit your interested is:

bp= yy_scan_string( line );
yy_switch_to_buffer(bp);
err= yyparse();
yy_delete_buffer(bp);

You need to switch to the buffer you created from the scan string.

- --Phil
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkrVtyQACgkQAhcOgIaQQ2FudACfUfM1LKEf/RmzmterEOd41Gbz
dukAn3x9VI01C4RUc/7M3KKEIedOKahW
=GXnJ
-----END PGP SIGNATURE-----



_______________________________________________
help-bison@... http://lists.gnu.org/mailman/listinfo/help-bison

[never2di]

Philip Herron wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

never2di wrote:
> I'm a newbie to Bison + Flex. and I'm having trouble in getting
> yy_scan_string to work. I have tried the following
>
> YY_BUFFER_STATE my_string_buffer = yy_scan_string(my_string);
> yyparse(); yy_delete_buffer(my_string_buffer );
>
> the parser errors out with a syntax error at the first token. I
> have verified that the grammar and the content of 'my_string' works
> using yyrestart(yyin) and also with yy_create_buffer() through a
> file.
>
> Is there something special state that I have to set flex in before
> I call yyparse()? if not what is that I'm doing wrong and what
> needs to be done?
>
> bison (GNU Bison) 2.4
>
> Thanks :handshake:
>
>
>

Hey

You might find this useful i do this in my lexer:

int parse( FILE* fin )
{
  int err= 0;
  if( fin == stdin )
    {
#ifdef HAVE_LIBREADLINE
      char *line, *prompt= ">>> ";
#else
      char line[256];
#endif
      YY_BUFFER_STATE bp;

      while( 1 )
    {
#ifdef HAVE_LIBREADLINE
      line= readline(prompt);
#else
      fgets(line, sizeof(line), stdin);
#endif
      if( line == 0 )
        break;
      else if( strlen(line) > 0)
        {
#ifdef HAVE_READLINE_HISTORY
          add_history(line);
#endif
          bp= yy_scan_string( line );
          yy_switch_to_buffer(bp);
          err= yyparse();
          yy_delete_buffer(bp);
        }
    }
    }
  else
    {
      yyin= fin;
      err= yyparse();
    }
  return err;
}

The bit your interested is:

bp= yy_scan_string( line );
yy_switch_to_buffer(bp);
err= yyparse();
yy_delete_buffer(bp);

You need to switch to the buffer you created from the scan string.

- --Phil
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkrVtyQACgkQAhcOgIaQQ2FudACfUfM1LKEf/RmzmterEOd41Gbz
dukAn3x9VI01C4RUc/7M3KKEIedOKahW
=GXnJ
-----END PGP SIGNATURE-----

_______________________________________________
[e-mail] http://lists.gnu.org/mailman/listinfo/help-bison

Hi Phil

Thanks for the reply.

The documentation for yy_scan_string says that the switching of the buffer will be done by yy_scan_string itself and I verified that same in the code.

http://flex.sourceforge.net/manual/Multiple-Input-Buffers.html#index-yy_005fscan_005fstring-210

I though if it works for you then why not give it a try but unfortunately it didn't work. I read some where that I will have to set the initial state for the lexer, not sure what that means.

Thanks

[Luca-63]

never2di ha scritto:

> I'm a newbie to Bison + Flex. and I'm having trouble in getting
> yy_scan_string to work. I have tried the following
>
> YY_BUFFER_STATE my_string_buffer = yy_scan_string(my_string);
> yyparse();
> yy_delete_buffer(my_string_buffer );
>
> the parser errors out with a syntax error at the first token. I have
> verified that the grammar and the content of 'my_string' works using
> yyrestart(yyin) and also with yy_create_buffer() through a file.
>
> Is there something special state that I have to set flex in before I call
> yyparse()?
> if not what is that I'm doing wrong and what needs to be done?
>
> bison (GNU Bison) 2.4  
>
> Thanks :handshake:
>
>
>
>  

       

The problem concerns Flex and not Bison.
I anyway think you're right, you have to call yy_switch_to_buffer;
the following code should be right:

YY_BUFFER_STATE b=yy_scan_string(my_string);
yy_switch_to_buffer(b);       //buffer change
yyparse();  
yylex_destroy();             //to avoid leakage

You can read how "Multiple Input Buffers" works in flex help:
http://flex.sourceforge.net/manual/Multiple-Input-Buffers.html#Multiple-Input-Buffers

Luca





_______________________________________________
help-bison@... http://lists.gnu.org/mailman/listinfo/help-bison

[never2di ()]

never2di ha scritto:
> I'm a newbie to Bison + Flex. and I'm having trouble in getting
> yy_scan_string to work. I have tried the following
>
> YY_BUFFER_STATE my_string_buffer = yy_scan_string(my_string);
> yyparse();
> yy_delete_buffer(my_string_buffer );
>
> the parser errors out with a syntax error at the first token. I have
> verified that the grammar and the content of 'my_string' works using
> yyrestart(yyin) and also with yy_create_buffer() through a file.
>
> Is there something special state that I have to set flex in before I call
> yyparse()?
> if not what is that I'm doing wrong and what needs to be done?
>
> bison (GNU Bison) 2.4  
>
> Thanks :handshake:
>
>
>
>  
       

The problem concerns Flex and not Bison.
I anyway think you're right, you have to call yy_switch_to_buffer;
the following code should be right:

YY_BUFFER_STATE b=yy_scan_string(my_string);
yy_switch_to_buffer(b);       //buffer change
yyparse();  
yylex_destroy();             //to avoid leakage

You can read how "Multiple Input Buffers" works in flex help:
http://flex.sourceforge.net/manual/Multiple-Input-Buffers.html#Multiple-Input-Buffers

Luca


The documentation says that there is no need to call yy_switch_to_buffer after calling yy_scan_string.
below is the exact text from the link the Luca sent.  But any way it still doesn't work for me.

I have also tried to set BEGIN(0)before and after yy_scan_string, not sure what that does though.

any help is greatly appreciated, and thanks for you time.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

The following routines are available for setting up input buffers for scanning in-memory strings instead of files. All of them create a new input buffer for scanning the string, and return a corresponding YY_BUFFER_STATE handle (which you should delete with yy_delete_buffer() when done with it). They also switch to the new buffer using yy_switch_to_buffer(), so the next call to yylex() will start scanning the string.

— Function: YY_BUFFER_STATE yy_scan_string ( const char *str )

scans a NUL-terminated string.

— Function: YY_BUFFER_STATE yy_scan_bytes ( const char *bytes, int len )

scans len bytes (including possibly NULs) starting at location bytes.

Note that both of these functions create and scan a copy of the string or bytes. (This may be desirable, since yylex() modifies the contents of the buffer it is scanning.) You can avoid the copy by using:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

_______________________________________________
[e-mail] http://lists.gnu.org/mailman/listinfo/help-bison