[ragel-users] A generic eof action from a ragel scanner

[gonzalo diethelm]

I am playing with a typical toy calculator written with ragel and lemon, in C. The scanner is defined like this:

main := |*
number => num_tok;
'(' => lparen_tok;
')' => rparen_tok;
'+' => plus_tok;
'-' => minus_tok;
'*' => mul_tok;
'/' => div_tok;
'^' => pow_tok;
ws;
*|;

Each of the *_tok actions calls the lemon-generated parser. For example:

action pow_tok { CalcParser(parser, TOKEN_POW, 0); }
action num_tok { CalcParser(parser, TOKEN_NUM, get_num(ts, te)); }

The idea is to pass each argv[] character as a separate expression to compute. So this execution:

./calc 2+3*4 7-5 "2 + 4"

Should output three lines:
14
2
6

Since the lemon parser has to be informed about the EOF condition, what I am doing is manually informing the parser of this condition right when I have finished processing each one of the argv[] arguments:

calc_exec(argv[i], strlen(argv[i]));
...
void calc_exec (const char* data, int len)
{
const char* p = data;
const char* pe = data + len; // Should this be len+1?
const char* eof = pe;

%% write exec;

/* Make sure we inform the parser we saw the whole input */
CalcParser(parser, 0, 0);
}

But what I would really like to do is to add a "generic EOF action" to the scanner, that issued this final call to the parser:

??? %eof(CalcParser(parser, 0, 0);)

I have not been able to found how to specify this "generic EOF action" to my ragel scanner. Is this even possible? Is it the right approach?

Thanks for any help. And thanks for the great tool that ragel is.

--
Gonzalo Diethelm
DCV Chile





-----------------------------------------
Declaraci?n de confidencialidad: Este Mensaje esta destinado para
el uso de la o las personas o entidades a quien ha sido dirigido y
puede contener informaci?n reservada y confidencial que no puede
ser divulgada, difundida, ni aprovechada en forma alguna. El uso no
autorizado de la informaci?n contenida en este correo podr? ser
sancionado de conformidad con la ley chilena.
Si usted ha recibido este correo electr?nico por error, le pedimos
eliminarlo junto con los archivos adjuntos y avisar inmediatamente
al remitente, respondiendo este mensaje.

"Before printing this e-mail think if is really necesary".
Disclosure: This Message is to be used by the individual,
individuals or entities that it is addressed to and may include
private and confidential information that may not be disclosed,
made public nor used in any way at all. Unauthorized use of the
information in this electronic mail message may be subject to the
penalties set forth by Chilean law.
If you have received this electronic mail message in error, we ask
you to destroy the message and its attached file(s) and to
immediately notify the sender by answering this message.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.complang.org/pipermail/ragel-users/attachments/20130219/65caeb39/attachment.html>

[Adrian Thurston]

There is no native mechanism to express EOF within a scanner. Since your
grammar doesn't require a scanner (no backtracking will ever take
place), you can kleene star a union of tokens and then wrap that in an
eof action. Like this:

main := ( ' ' %{} | '\t' %{} | [0-9]+ %{})** $eof {};

Just note you will be forced to use one character of lookahead. The
scanner eliminates lookahead whenever possible.

Another option is to send the null character to the state machine, then
write scanner pattern for that.

Regards,
Adrian

Post by gonzalo diethelm
I am playing with a typical toy calculator written with ragel and lemon,
main := |*
number => num_tok;
'(' => lparen_tok;
')' => rparen_tok;
'+' => plus_tok;
'-' => minus_tok;
'*' => mul_tok;
'/' => div_tok;
'^' => pow_tok;
ws;
*|;
action pow_tok { CalcParser(parser, TOKEN_POW, 0); }
action num_tok { CalcParser(parser, TOKEN_NUM, get_num(ts, te)); }
The idea is to pass each argv[] character as a separate expression to
./calc 2+3*4 7-5 ?2 + 4?
14
2
6
Since the lemon parser has to be informed about the EOF condition, what
I am doing is manually informing the parser of this condition right when
calc_exec(argv[i], strlen(argv[i]));
?
void calc_exec (const char* data, int len)
{
const char* p = data;
const char* pe = data + len; // Should this be len+1?
const char* eof = pe;
%% write exec;
/* Make sure we inform the parser we saw the whole input */
CalcParser(parser, 0, 0);
}
But what I would really like to do is to add a ?generic EOF action? to
??? %eof(CalcParser(parser, 0, 0);)
I have not been able to found how to specify this ?generic EOF action?
to my ragel scanner. Is this even possible? Is it the right approach?
Thanks for any help. And thanks for the great tool that ragel is.
--
Gonzalo Diethelm
DCV Chile
------------------------------------------------------------------------
Declaraci?n de confidencialidad: Este Mensaje esta destinado para el uso
de la o las personas o entidades a quien ha sido dirigido y puede
contener informaci?n reservada y confidencial que no puede ser
divulgada, difundida, ni aprovechada en forma alguna. El uso no
autorizado de la informaci?n contenida en este correo podr? ser
sancionado de conformidad con la ley chilena. Si usted ha recibido este
correo electr?nico por error, le pedimos eliminarlo junto con los
archivos adjuntos y avisar inmediatamente al remitente, respondiendo
este mensaje.
Disclosure: This Message is to be used by the individual, individuals or
entities that it is addressed to and may include private and
confidential information that may not be disclosed, made public nor used
in any way at all. Unauthorized use of the information in this
electronic mail message may be subject to the penalties set forth by
Chilean law. If you have received this electronic mail message in error,
we ask you to destroy the message and its attached file(s) and to
immediately notify the sender by answering this message.
_______________________________________________
ragel-users mailing list
ragel-users at complang.org
http://www.complang.org/mailman/listinfo/ragel-users