![[BACK]](/cvsweb/icons/back.gif){kind=icon}
Return to lex.1 CVS log ![[TXT]](/cvsweb/icons/text.gif){kind=icon}
![[DIR]](/cvsweb/icons/dir.gif){kind=icon}
Up to [CSRG BSD Unix] / 43BSDReno / pgrm / lex|
Return to lex.1 CVS log | Up to [CSRG BSD Unix] / 43BSDReno / pgrm / lex |
1.1 root 1: .\" Copyright (c) 1990 The Regents of the University of California.
2: .\" All rights reserved.
3: .\"
4: .\" Redistribution and use in source and binary forms are permitted provided
5: .\" that: (1) source distributions retain this entire copyright notice and
6: .\" comment, and (2) distributions including binaries display the following
7: .\" acknowledgement: ``This product includes software developed by the
8: .\" University of California, Berkeley and its contributors'' in the
9: .\" documentation or other materials provided with the distribution and in
10: .\" all advertising materials mentioning features or use of this software.
11: .\" Neither the name of the University nor the names of its contributors may
12: .\" be used to endorse or promote products derived from this software without
13: .\" specific prior written permission.
14: .\" THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED
15: .\" WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF
16: .\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
17: .\"
18: .\" @(#)lex.1 5.10 (Berkeley) 7/24/90
19: .\"
20: .Dd July 24, 1990
21: .Dt LEX 1
22: .Sh NAME
23: .Nm lex
24: .Nd fast lexical analyzer generator
25: .Sh SYNOPSIS
26: .Nm lex
27: .Ob
28: .Op Fl bcdfinpstvFILT8
29: .Cx Fl C
30: .Op efmF
31: .Cx
32: .Cx Fl S
33: .Ar skeleton
34: .Cx
35: .Oe
36: .Nm lex
37: .Ar
38: .Sh DESCRIPTION
39: .Nm Lex
40: is a tool for generating
41: .Ar scanners :
42: programs which recognized lexical patterns in text.
43: .Nm Lex
44: reads
45: the given input files, or its standard input if no file names are given,
46: for a description of a scanner to generate. The description is in
47: the form of pairs
48: of regular expressions and C code, called
49: .Em rules .
50: .Nm Lex
51: generates as output a C source file,
52: .Pa lex.yy.c ,
53: which defines a routine
54: .Fn yylex .
55: This file is compiled and linked with the
56: .Fl lfl
57: library to produce an executable. When the executable is run,
58: it analyzes its input for occurrences
59: of the regular expressions. Whenever it finds one, it executes
60: the corresponding C code.
61: .Pp
62: For full documentation, see
63: .Em Lexdoc .
64: This manual entry is intended for use as a quick reference.
65: .Sh OPTIONS
66: .Nm Lex
67: has the following options:
68: .Tw Ds
69: .Tp Fl b
70: Generate backtracking information to
71: .Va lex.backtrack .
72: This is a list of scanner states which require backtracking
73: and the input characters on which they do so. By adding rules one
74: can remove backtracking states. If all backtracking states
75: are eliminated and
76: .Fl f
77: or
78: .Fl F
79: is used, the generated scanner will run faster.
80: .Tp Fl c
81: is a do-nothing, deprecated option included for POSIX compliance.
82: .Pp
83: .Ar NOTE :
84: in previous releases of
85: .Nm Lex
86: .Op Fl c
87: specified table-compression options. This functionality is
88: now given by the
89: .Fl C
90: flag. To ease the the impact of this change, when
91: .Nm lex
92: encounters
93: .Fl c,
94: it currently issues a warning message and assumes that
95: .Fl C
96: was desired instead. In the future this "promotion" of
97: .Fl c
98: to
99: .Fl C
100: will go away in the name of full POSIX compliance (unless
101: the POSIX meaning is removed first).
102: .Tp Fl d
103: makes the generated scanner run in
104: .Ar debug
105: mode. Whenever a pattern is recognized and the global
106: .Va yy_Lex_debug
107: is non-zero (which is the default), the scanner will
108: write to
109: .Li stderr
110: a line of the form:
111: .Pp
112: .Dl --accepting rule at line 53 ("the matched text")
113: .Pp
114: The line number refers to the location of the rule in the file
115: defining the scanner (i.e., the file that was fed to lex). Messages
116: are also generated when the scanner backtracks, accepts the
117: default rule, reaches the end of its input buffer (or encounters
118: a NUL; the two look the same as far as the scanner's concerned),
119: or reaches an end-of-file.
120: .Tp Fl f
121: specifies (take your pick)
122: .Em full table
123: or
124: .Em fast scanner .
125: No table compression is done. The result is large but fast.
126: This option is equivalent to
127: .Fl Cf
128: (see below).
129: .Tp Fl i
130: instructs
131: .Nm lex
132: to generate a
133: .Em case-insensitive
134: scanner. The case of letters given in the
135: .Nm lex
136: input patterns will
137: be ignored, and tokens in the input will be matched regardless of case. The
138: matched text given in
139: .Va yytext
140: will have the preserved case (i.e., it will not be folded).
141: .Tp Fl n
142: is another do-nothing, deprecated option included only for
143: POSIX compliance.
144: .Tp Fl p
145: generates a performance report to stderr. The report
146: consists of comments regarding features of the
147: .Nm lex
148: input file which will cause a loss of performance in the resulting scanner.
149: .Tp Fl s
150: causes the
151: .Ar default rule
152: (that unmatched scanner input is echoed to
153: .Ar stdout )
154: to be suppressed. If the scanner encounters input that does not
155: match any of its rules, it aborts with an error.
156: .Tp Fl t
157: instructs
158: .Nm lex
159: to write the scanner it generates to standard output instead
160: of
161: .Pa lex.yy.c .
162: .Tp Fl v
163: specifies that
164: .Nm lex
165: should write to
166: .Li stderr
167: a summary of statistics regarding the scanner it generates.
168: .Tp Fl F
169: specifies that the
170: .Em fast
171: scanner table representation should be used. This representation is
172: about as fast as the full table representation
173: .Pq Fl f ,
174: and for some sets of patterns will be considerably smaller (and for
175: others, larger). See
176: .Em Lexdoc
177: for details.
178: .Pp
179: This option is equivalent to
180: .Fl CF
181: (see below).
182: .Tp Fl I
183: instructs
184: .Nm lex
185: to generate an
186: .Em interactive
187: scanner, that is, a scanner which stops immediately rather than
188: looking ahead if it knows
189: that the currently scanned text cannot be part of a longer rule's match.
190: Again, see
191: .Em Lexdoc
192: for details.
193: .Pp
194: Note,
195: .Fl I
196: cannot be used in conjunction with
197: .Em full
198: or
199: .Em fast tables ,
200: i.e., the
201: .Fl f , F , Cf ,
202: or
203: .Fl CF
204: flags.
205: .Tp Fl L
206: instructs
207: .Nm lex
208: not to generate
209: .Li #line
210: directives in
211: .Pa lex.yy.c .
212: The default is to generate such directives so error
213: messages in the actions will be correctly
214: located with respect to the original
215: .Nm lex
216: input file, and not to
217: the fairly meaningless line numbers of
218: .Pa lex.yy.c .
219: .Tp Fl T
220: makes
221: .Nm lex
222: run in
223: .Em trace
224: mode. It will generate a lot of messages to
225: .Li stdout
226: concerning
227: the form of the input and the resultant non-deterministic and deterministic
228: finite automata. This option is mostly for use in maintaining
229: .Nm lex .
230: .Tp Fl 8
231: instructs
232: .Nm lex
233: to generate an 8-bit scanner.
234: On some sites, this is the default. On others, the default
235: is 7-bit characters. To see which is the case, check the verbose
236: .Pq Fl v
237: output for "equivalence classes created". If the denominator of
238: the number shown is 128, then by default
239: .Nm lex
240: is generating 7-bit characters. If it is 256, then the default is
241: 8-bit characters.
242: .Tc Fl C
243: .Op Cm efmF
244: .Cx
245: controls the degree of table compression. The default setting is
246: .Fl Cem .
247: .Pp
248: .Tw Ds
249: .Tp Fl C
250: A lone
251: .Fl C
252: specifies that the scanner tables should be compressed but neither
253: equivalence classes nor meta-equivalence classes should be used.
254: .Tp Fl \&Ce
255: directs
256: .Nm lex
257: to construct
258: .Em equivalence classes ,
259: i.e., sets of characters
260: which have identical lexical properties.
261: Equivalence classes usually give
262: dramatic reductions in the final table/object file sizes (typically
263: a factor of 2-5) and are pretty cheap performance-wise (one array
264: look-up per character scanned).
265: .Tp Fl \&Cf
266: specifies that the
267: .Em full
268: scanner tables should be generated -
269: .Nm lex
270: should not compress the
271: tables by taking advantages of similar transition functions for
272: different states.
273: .Tp Fl \&CF
274: specifies that the alternate fast scanner representation (described in
275: .Em Lexdoc )
276: should be used.
277: .Tp Fl \&Cm
278: directs
279: .Nm lex
280: to construct
281: .Em meta-equivalence classes ,
282: which are sets of equivalence classes (or characters, if equivalence
283: classes are not being used) that are commonly used together. Meta-equivalence
284: classes are often a big win when using compressed tables, but they
285: have a moderate performance impact (one or two "if" tests and one
286: array look-up per character scanned).
287: .Tp Fl Cem
288: (default)
289: Generate both equivalence classes
290: and meta-equivalence classes. This setting provides the highest
291: degree of table compression.
292: .Tp
293: .Pp
294: Faster-executing scanners can be traded off at the cost of larger tables with
295: the following generally being true:
296: .Pp
297: .Ds C
298: slowest & smallest
299: -Cem
300: -Cm
301: -Ce
302: -C
303: -C{f,F}e
304: -C{f,F}
305: fastest & largest
306: .De
307: .Pp
308: .Fl C
309: options are not cumulative; whenever the flag is encountered, the
310: previous -C settings are forgotten.
311: .Pp
312: The options
313: .Fl \&Cf
314: or
315: .Fl \&CF
316: and
317: .Fl \&Cm
318: do not make sense together - there is no opportunity for meta-equivalence
319: classes if the table is not being compressed. Otherwise the options
320: may be freely mixed.
321: .Tc Fl S
322: .Ar skeleton_file
323: .Cx
324: overrides the default skeleton file from which
325: .Nm lex
326: constructs its scanners. Useful for
327: .Nm lex
328: maintenance or development.
329: .Sh SUMMARY OF Lex REGULAR EXPRESSIONS
330: The patterns in the input are written using an extended set of regular
331: expressions. These are:
332: .Pp
333: .Dw 8n
334: .Di L
335: .Dp Li x
336: match the character 'x'
337: .Dp Li \&.
338: any character except newline
339: .Dp Op Li xyz
340: a "character class"; in this case, the pattern
341: matches either an 'x', a 'y', or a 'z'
342: .Dp Op Li abj-oZ
343: a "character class" with a range in it; matches
344: an 'a', a 'b', any letter from 'j' through 'o',
345: or a 'Z'
346: .Dp Op \&Li ^A-Z
347: a "negated character class", i.e., any character
348: but those in the class. In this case, any
349: character EXCEPT an uppercase letter.
350: .Dp Op \&Li ^A-Z\en
351: any character EXCEPT an uppercase letter or
352: a newline
353: .Dp Li r*
354: zero or more r's, where r is any regular expression
355: .Dp Li r+
356: one or more r's
357: .Dp Li r?
358: zero or one r's (that is, "an optional r")
359: .Dp Li r{2,5}
360: anywhere from two to five r's
361: .Dp Li r{2,}
362: two or more r's
363: .Dp Li r{4}
364: exactly 4 r's
365: .Dp Li {name}
366: the expansion of the "name" definition
367: (see above)
368: .Dc Op Li xyz
369: .Li \&\e"foo"
370: .Cx
371: the literal string:
372: [xyz]"foo
373: .Dp Li \&\eX
374: if X is an 'a', 'b', 'f', 'n', 'r', 't', or 'v',
375: then the ANSI-C interpretation of \ex.
376: Otherwise, a literal 'X' (used to escape
377: operators such as '*')
378: .Dp Li \&\e123
379: the character with octal value 123
380: .Dp Li \&\ex2a
381: the character with hexadecimal value 2a
382: .Dp Li (r)
383: match an r; parentheses are used to override
384: precedence (see below)
385: .Dp Li rs
386: the regular expression r followed by the
387: regular expression s; called "concatenation"
388: .Dp Li rs
389: either an r or an s
390: .Dp Li r/s
391: an r but only if it is followed by an s. The
392: s is not part of the matched text. This type
393: of pattern is called as "trailing context".
394: .Dp Li \&^r
395: an r, but only at the beginning of a line
396: .Dp Li r$
397: an r, but only at the end of a line. Equivalent
398: to "r/\en".
399: .Dp Li <s>r
400: an r, but only in start condition s (see
401: below for discussion of start conditions)
402: .Dp Li <s1,s2,s3>r
403: same, but in any of start conditions s1,
404: s2, or s3
405: .Dp Li <<EOF>>
406: an end-of-file
407: .Dp Li <s1,s2><<EOF>>
408: an end-of-file when in start condition s1 or s2
409: .Dp
410: The regular expressions listed above are grouped according to
411: precedence, from highest precedence at the top to lowest at the bottom.
412: Those grouped together have equal precedence.
413: .Pp
414: Some notes on patterns:
415: .Pp
416: Negated character classes
417: .Ar match newlines
418: unless "\en" (or an equivalent escape sequence) is one of the
419: characters explicitly present in the negated character class
420: (e.g., " [^A-Z\en] ").
421: .Pp
422: A rule can have at most one instance of trailing context (the '/' operator
423: or the '$' operator). The start condition, '^', and "<<EOF>>" patterns
424: can only occur at the beginning of a pattern, and, as well as with '/' and '$',
425: cannot be grouped inside parentheses. The following are all illegal:
426: .Pp
427: .Ds C
428: foo/bar$
429: foo(bar$)
430: foo^bar
431: <sc1>foo<sc2>bar
432: .De
433: .Sh SUMMARY OF SPECIAL ACTIONS
434: In addition to arbitrary C code, the following can appear in actions:
435: .Tw Fl
436: .Tp Ic ECHO
437: Copies
438: .Va yytext
439: to the scanner's output.
440: .Tp Ic BEGIN
441: Followed by the name of a start condition places the scanner in the
442: corresponding start condition.
443: .Tp Ic REJECT
444: Directs the scanner to proceed on to the "second best" rule which matched the
445: input (or a prefix of the input).
446: .Va yytext
447: and
448: .Va yyleng
449: are set up appropriately. Note that
450: .Ic REJECT
451: is a particularly expensive feature in terms scanner performance;
452: if it is used in
453: .Em any
454: of the scanner's actions it will slow down
455: .Em all
456: of the scanner's matching. Furthermore,
457: .Ic REJECT
458: cannot be used with the
459: .Fl f
460: or
461: .Fl F
462: options.
463: .Pp
464: Note also that unlike the other special actions,
465: .Ic REJECT
466: is a
467: .Em branch ;
468: code immediately following it in the action will
469: .Em not
470: be executed.
471: .Tp Fn yymore
472: tells the scanner that the next time it matches a rule, the corresponding
473: token should be
474: .Em appended
475: onto the current value of
476: .Va yytext
477: rather than replacing it.
478: .Tp Fn yyless \&n
479: returns all but the first
480: .Ar n
481: characters of the current token back to the input stream, where they
482: will be rescanned when the scanner looks for the next match.
483: .Va yytext
484: and
485: .Va yyleng
486: are adjusted appropriately (e.g.,
487: .Va yyleng
488: will now be equal to
489: .Ar n ) .
490: .Tp Fn unput c
491: puts the character
492: .Ar c
493: back onto the input stream. It will be the next character scanned.
494: .Tp Fn input
495: reads the next character from the input stream (this routine is called
496: .Fn yyinput
497: if the scanner is compiled using
498: .Em C \&+\&+ ) .
499: .Tp Fn yyterminate
500: can be used in lieu of a return statement in an action. It terminates
501: the scanner and returns a 0 to the scanner's caller, indicating "all done".
502: .Pp
503: By default,
504: .Fn yyterminate
505: is also called when an end-of-file is encountered. It is a macro and
506: may be redefined.
507: .Tp Ic YY_NEW_FILE
508: is an action available only in <<EOF>> rules. It means "Okay, I've
509: set up a new input file, continue scanning".
510: .Tp Fn yy_create_buffer file size
511: takes a
512: .Ic FILE
513: pointer and an integer
514: .Ar size .
515: It returns a YY_BUFFER_STATE
516: handle to a new input buffer large enough to accomodate
517: .Ar size
518: characters and associated with the given file. When in doubt, use
519: .Ar YY_BUF_SIZE
520: for the size.
521: .Tp Fn yy_switch_to_buffer new_buffer
522: switches the scanner's processing to scan for tokens from
523: the given buffer, which must be a YY_BUFFER_STATE.
524: .Tp Fn yy_delete_buffer buffer
525: deletes the given buffer.
526: .Tp
527: .Sh \&VALUES\ AVAILABLE\ TO THE USER
528: .Tw Fl
529: .Tp Va \&char \&*yytext
530: holds the text of the current token. It may not be modified.
531: .Tp Va \&int yyleng
532: holds the length of the current token. It may not be modified.
533: .Tp Va FILE \&*yyin
534: is the file which by default
535: .Nm lex
536: reads from. It may be redefined but doing so only makes sense before
537: scanning begins. Changing it in the middle of scanning will have
538: unexpected results since
539: .Nm lex
540: buffers its input. Once scanning terminates because an end-of-file
541: has been seen,
542: .Fn void\ yyrestart FILE\ *new_file
543: may be called to point
544: .Va yyin
545: at the new input file.
546: .Tp Va FILE \&*yyout
547: is the file to which
548: .Ar ECHO
549: actions are done. It can be reassigned by the user.
550: .Tp Va YY_CURRENT_BUFFER
551: returns a
552: YY_BUFFER_STATE
553: handle to the current buffer.
554: .Tp
555: .Sh MACROS THE USER CAN REDEFINE
556: .Tw Fl
557: .Tp Va YY_DECL
558: controls how the scanning routine is declared.
559: By default, it is "int yylex()", or, if prototypes are being
560: used, "int yylex(void)". This definition may be changed by redefining
561: the "YY_DECL" macro. Note that
562: if you give arguments to the scanning routine using a
563: K&R-style/non-prototyped function declaration, you must terminate
564: the definition with a semi-colon (;).
565: .Tp Va YY_INPUT
566: The nature of how the scanner
567: gets its input can be controlled by redefining the
568: YY_INPUT
569: macro.
570: YY_INPUT's calling sequence is "YY_INPUT(buf,result,max_size)". Its
571: action is to place up to
572: .Ar max _size
573: characters in the character array
574: .Ar buf
575: and return in the integer variable
576: .Ar result
577: either the
578: number of characters read or the constant YY_NULL (0 on Unix systems)
579: to indicate EOF. The default YY_INPUT reads from the
580: global file-pointer "yyin".
581: A sample redefinition of YY_INPUT (in the definitions
582: section of the input file):
583: .Pp
584: .Ds I
585: %{
586: #undef YY_INPUT
587: #define YY_INPUT(buf,result,max_size) \\
588: result = ((buf[0] = getchar()) == EOF) ? YY_NULL : 1;
589: %}
590: .De
591: .Tp Va YY_INPUT
592: When the scanner receives an end-of-file indication from YY_INPUT,
593: it then checks the
594: .Fn yywrap
595: function. If
596: .Fn yywrap
597: returns false (zero), then it is assumed that the
598: function has gone ahead and set up
599: .Va yyin
600: to point to another input file, and scanning continues. If it returns
601: true (non-zero), then the scanner terminates, returning 0 to its
602: caller.
603: .Tp Va yywrap
604: The default
605: .Fn yywrap
606: always returns 1. Presently, to redefine it you must first
607: "#undef yywrap", as it is currently implemented as a macro. It is
608: likely that
609: .Fn yywrap
610: will soon be defined to be a function rather than a macro.
611: .Tp Va YY_USER_ACTION
612: can be redefined to provide an action
613: which is always executed prior to the matched rule's action.
614: .Tp Va YY_USER_INIT
615: The macro
616: .Va YY _USER_INIT
617: may be redefined to provide an action which is always executed before
618: the first scan.
619: .Tp Va YY_BREAK
620: In the generated scanner, the actions are all gathered in one large
621: switch statement and separated using
622: .Va YY _BREAK ,
623: which may be redefined. By default, it is simply a "break", to separate
624: each rule's action from the following rule's.
625: .Tp
626: .Sh FILES
627: .Dw lex.backtrack
628: .Di L
629: .Dp Pa lex.skel
630: skeleton scanner.
631: .Dp Pa lex.yy.c
632: generated scanner
633: (called
634: .Pa lexyy.c
635: on some systems).
636: .Dp Pa lex.backtrack
637: backtracking information for
638: .Fl b
639: .Dp Pa flag
640: (called
641: .Pa lex.bck
642: on some systems).
643: .Dp
644: .Sh SEE ALSO
645: .Xr lex 1 ,
646: .Xr yacc 1 ,
647: .Xr sed 1 ,
648: .Xr awk 1 .
649: .br
650: .Em lexdoc
651: .br
652: M.
653: E.
654: Lesk and E.
655: Schmidt,
656: .Em LEX \- Lexical Analyzer Generator
657: .Sh DIAGNOSTICS
658: .Tw Fl
659: .Tp Li reject_used_but_not_detected undefined
660: or
661: .Tp Li yymore_used_but_not_detected undefined
662: These errors can occur at compile time.
663: They indicate that the
664: scanner uses
665: .Ic REJECT
666: or
667: .Fn yymore
668: but that
669: .Nm lex
670: failed to notice the fact,
671: meaning that
672: .Nm lex
673: scanned the first two sections looking for occurrences of these actions
674: and failed to find any,
675: but somehow you snuck some in via a #include
676: file,
677: for example .
678: Make an explicit reference to the action in your
679: .Nm lex
680: input file.
681: Note that previously
682: .Nm lex
683: supported a
684: .Li %used/%unused
685: mechanism for dealing with this problem;
686: this feature is still supported
687: but now deprecated,
688: and will go away soon unless the author hears from
689: people who can argue compellingly that they need it.
690: .Tp Li lex scanner jammed
691: a scanner compiled with
692: .Fl s
693: has encountered an input string which wasn't matched by
694: any of its rules.
695: .Tp Li lex input buffer overflowed
696: a scanner rule matched a string long enough to overflow the
697: scanner's internal input buffer 16K bytes - controlled by
698: .Va YY_BUF_MAX
699: in
700: .Pa lex.skel .
701: .Tp Li scanner requires \&\-8 flag
702: Your scanner specification includes recognizing 8-bit characters and
703: you did not specify the -8 flag and your site has not installed lex
704: with -8 as the default .
705: .Tp Li too many \&%t classes!
706: You managed to put every single character into its own %t class.
707: .Nm Lex
708: requires that at least one of the classes share characters.
709: .Tp
710: .Sh HISTORY
711: A
712: .Nm lex
713: appeared in Version 6 AT&T Unix.
714: The version this man page describes is
715: derived from code contributed by Vern Paxson.
716: .Sh AUTHOR
717: Vern Paxson, with the help of many ideas and much inspiration from
718: Van Jacobson. Original version by Jef Poskanzer.
719: .Pp
720: See
721: .Em Lexdoc
722: for additional credits and the address to send comments to.
723: .Sh BUGS
724: .Pp
725: Some trailing context
726: patterns cannot be properly matched and generate
727: warning messages ("Dangerous trailing context"). These are
728: patterns where the ending of the
729: first part of the rule matches the beginning of the second
730: part, such as "zx*/xy*", where the 'x*' matches the 'x' at
731: the beginning of the trailing context. (Note that the POSIX draft
732: states that the text matched by such patterns is undefined.)
733: .Pp
734: For some trailing context rules, parts which are actually fixed-length are
735: not recognized as such, leading to the abovementioned performance loss.
736: In particular, parts using '\&|' or {n} (such as "foo{3}") are always
737: considered variable-length.
738: .Pp
739: Combining trailing context with the special '\&|' action can result in
740: .Em fixed
741: trailing context being turned into the more expensive
742: .Em variable
743: trailing context. This happens in the following example:
744: .Pp
745: .Ds C
746: %%
747: abc \&|
748: xyz/def
749: .De
750: .Pp
751: Use of
752: .Fn unput
753: invalidates yytext and yyleng.
754: .Pp
755: Use of
756: .Fn unput
757: to push back more text than was matched can
758: result in the pushed-back text matching a beginning-of-line ('^')
759: rule even though it didn't come at the beginning of the line
760: (though this is rare!).
761: .Pp
762: Pattern-matching of NUL's is substantially slower than matching other
763: characters.
764: .Pp
765: .Nm Lex
766: does not generate correct #line directives for code internal
767: to the scanner; thus, bugs in
768: .Pa lex.skel
769: yield bogus line numbers.
770: .Pp
771: Due to both buffering of input and read-ahead, you cannot intermix
772: calls to <stdio.h> routines, such as, for example,
773: .Fn getchar ,
774: with
775: .Nm lex
776: rules and expect it to work. Call
777: .Fn input
778: instead.
779: .Pp
780: The total table entries listed by the
781: .Fl v
782: flag excludes the number of table entries needed to determine
783: what rule has been matched. The number of entries is equal
784: to the number of DFA states if the scanner does not use
785: .Ic REJECT ,
786: and somewhat greater than the number of states if it does.
787: .Pp
788: .Ic REJECT
789: cannot be used with the
790: .Fl f
791: or
792: .Fl F
793: options.
794: .Pp
795: Some of the macros, such as
796: .Fn yywrap ,
797: may in the future become functions which live in the
798: .Fl lfl
799: library. This will doubtless break a lot of code, but may be
800: required for POSIX-compliance.
801: .Pp
802: The
803: .Nm lex
804: internal algorithms need documentation.
This archive runs on limited infrastructure. Preserving old code on modern bandwidth. Automated agents are requested to crawl responsibly.