Return to note CVS log

Up to [Research Unix] / researchv10no / cmd / sml / lib / lexgen

Annotation of researchv10no/cmd/sml/lib/lexgen/note, revision 1.1

1.1     ! root        1: From: sufrin%[email protected]
        !             2: Message-Id: <[email protected]>
        !             3: Date: Sat Dec 10 17:13:18 1988
        !             4: To: dbm <@RELAY.CS.NET:[email protected]>
        !             5: Subject: small suggestion for Andrew (re lexgen)
        !             6: Reply-To: Bernard Sufrin <sufrin%[email protected]>
        !             7: 
        !             8: 
        !             9: Dave: sorry to send stuff this way,
        !            10: but I can't find Andrew's email address. Bernard
        !            11: 
        !            12: Lexgen is very useful. A very simple impro-
        !            13: vement (which I've implemented here) is to
        !            14: give programmers the option of avoiding con-
        !            15: struction of a substring every time yytext
        !            16: is bound. Whilst "substring" is the simplest
        !            17: way to "internalise" a string, the program-
        !            18: mer may be able to do better by taking the
        !            19: "substring specification" (!yyb,i0,i-i0) and
        !            20: internalising it by some other means (which
        !            21: involves comparing the substring spec with
        !            22: already-internalised strings by any of the
        !            23: usual methods: I usually use open-hash +
        !            24: linear search).
        !            25: 
        !            26: 
        !            27: When employing this trick in a couple of
        !            28: compilers and our proof-support system I've
        !            29: found the consequent reduction of garbage
        !            30: collector cycles to be well worthwhile
        !            31: (instead of one substring construction per
        !            32: symbol of the text being analysed we get one
        !            33: substring constructed per DISTINCT symbol).
        !            34: 
        !            35: I've added an option in the lexgen defini-
        !            36: tions section: %nosubstring turns the binding
        !            37: of yytext from
        !            38: 
        !            39:        val yytext = substring(!yyb,i0,i-i0)
        !            40: into
        !            41:        val yytext = (!yyb,i0,i-i0:int)
        !            42:        
        !            43: 
        !            44: Here's the diff: 
        !            45: 
        !            46: 92c94
        !            47: <           COUNT | REJECT | FULLCHARSET | STRUCT
        !            48: ---
        !            49: >           COUNT | REJECT | FULLCHARSET | STRUCT | NOSUBSTRING
        !            50: 118c120
        !            51: <    val StrName = ref "Mlex"
        !            52: ---
        !            53: >    val StrName = ref "Mlex";
        !            54: 120,121c122
        !            55: <    val ResetFlags = fn () => (CountNewLines := false; HaveReject := false;
        !            56: <                              CharSetSize := 128; StrName := "Mlex")
        !            57: ---
        !            58: >    (* Can obliterate substring operator *)
        !            59: 122a124,129
        !            60: >    val SubStrName = ref "substring";
        !            61: > 
        !            62: >    val ResetFlags = fn () =>
        !            63: >    (CountNewLines := false; HaveReject := false;
        !            64: >     CharSetSize := 128; StrName := "Mlex"; SubStrName := "substring" )
        !            65: > 
        !            66: 325a333,334
        !            67: >                                  else if command = "nosubstring" then
        !            68: >                                         NOSUBSTRING
        !            69: 606a616,617
        !            70: >               | NOSUBSTRING =>
        !            71: >                          (SubStrName := ""; ParseDefs())
        !            72: 958c969
        !            73: <        sayln "\t\t\t(let val yytext = substring(!yyb,i0,i-i0)";
        !            74: ---
        !            75: >        say "\t\t\t(let val yytext = "; say(!SubStrName); sayln "(!yyb,i0,i-i0:int)";
        !            76: 
        !            77: 

This archive runs on limited infrastructure. Preserving old code on modern bandwidth. Automated agents are requested to crawl responsibly.