![[BACK]](/cvsweb/icons/back.gif){kind=icon}
Return to ch7.n CVS log ![[TXT]](/cvsweb/icons/text.gif){kind=icon}
![[DIR]](/cvsweb/icons/dir.gif){kind=icon}
Up to [CSRG BSD Unix] / 43BSDTahoe / ucb / lisp / doc|
Return to ch7.n CVS log | Up to [CSRG BSD Unix] / 43BSDTahoe / ucb / lisp / doc |
1.1 ! root 1: .\" Copyright (c) 1980 Regents of the University of California. ! 2: .\" All rights reserved. The Berkeley software License Agreement ! 3: .\" specifies the terms and conditions for redistribution. ! 4: .\" ! 5: .\" @(#)ch7.n 6.2 (Berkeley) 5/14/86 ! 6: .\" ! 7: ." $Header: ch7.n,v 1.3 83/07/01 11:22:58 layer Exp $ ! 8: .Lc The\ Lisp\ Reader 7 ! 9: .sh 2 Introduction \n(ch 1 ! 10: .pp ! 11: The ! 12: .i read ! 13: function is responsible for converting ! 14: a stream of ! 15: characters into a Lisp expression. ! 16: .i Read ! 17: is table driven and the table it uses is called a ! 18: .i readtable. ! 19: The ! 20: .i print ! 21: function does the ! 22: inverse of ! 23: .i read ; ! 24: it converts a Lisp expression into a stream of ! 25: characters. ! 26: Typically the conversion is done in such ! 27: a way that if that stream of characters were read by ! 28: .i read , ! 29: the ! 30: result would be an expression equal to the one ! 31: .i print ! 32: was given. ! 33: .i Print ! 34: must also refer to the readtable in order to determine ! 35: how to format its output. ! 36: The ! 37: .i explode ! 38: function, which returns a list of characters rather than ! 39: printing them, must also refer to the readtable. ! 40: .pp ! 41: A readtable is created ! 42: with the ! 43: .i makereadtable ! 44: function, modified with the ! 45: .i setsyntax ! 46: function and interrogated with the ! 47: .i getsyntax ! 48: function. ! 49: The structure of a readtable is hidden from the user - a ! 50: readtable should ! 51: only be manipulated with the three functions mentioned above. ! 52: .pp ! 53: There is one distinguished readtable called the ! 54: .i current ! 55: .i readtable ! 56: whose value determines what ! 57: .i read , ! 58: .i print ! 59: and ! 60: .i explode ! 61: do. ! 62: The current readtable is the value of the symbol ! 63: .i readtable . ! 64: Thus it is possible to rapidly change ! 65: the current syntax by lambda binding ! 66: a different readtable to the symbol ! 67: .i readtable. ! 68: When the binding is undone, the syntax reverts to its old form. ! 69: .sh +0 Syntax\ Classes ! 70: .pp ! 71: The readtable describes how each of the 128 ascii characters should ! 72: be treated by the reader and printer. ! 73: Each character belongs to a ! 74: .i syntax ! 75: .i class ! 76: which has three properties: ! 77: .ip character\ class\ - ! 78: Tells what the reader should do when it sees this character. ! 79: There are a large number of character classes. ! 80: They are described below. ! 81: .ip separator\ - ! 82: Most types of tokens the reader constructs are one character ! 83: long. ! 84: Four token types have an arbitrary length: number (1234), ! 85: symbol print name (franz), ! 86: escaped symbol print name (|franz|), and string ("franz"). ! 87: The reader can easily determine when it has ! 88: come to the ! 89: end of one of the last two types: it just looks for the ! 90: matching delimiter (| or "). ! 91: When the reader is reading a number or symbol print name, it ! 92: stops reading when it comes to a character with the ! 93: .i separator ! 94: property. ! 95: The separator character is pushed back into the input stream and will ! 96: be the first character read when the reader is called again. ! 97: .ip escape\ - ! 98: Tells the printer when to put escapes in front of, or around, a symbol ! 99: whose print name contains this character. ! 100: There are three possibilities: always escape a symbol with this character ! 101: in it, only escape a symbol if this is the only character in the symbol, ! 102: and only escape a symbol if this is the first character in the symbol. ! 103: [note: The printer will always escape a symbol which, if printed out, would ! 104: look like a valid number.] ! 105: .pp ! 106: When the Lisp system is built, Lisp code is added to a C-coded kernel ! 107: and the result becomes the standard lisp system. ! 108: The readtable present in the C-coded kernel, called the ! 109: .i raw ! 110: .i readtable , ! 111: contains the bare necessities for reading in Lisp code. ! 112: During the ! 113: construction of the complete Lisp system, ! 114: a copy is made of the raw readtable and ! 115: then the copy is modified by adding macro characters. ! 116: The result is what is called the ! 117: .i standard ! 118: .i readtable . ! 119: When a new readtable is created with ! 120: .i makereadtable, ! 121: a copy is made of either the ! 122: raw readtable ! 123: or the current readtable (which is likely to be the standard readtable). ! 124: .sh +0 Reader\ Operations ! 125: .pp ! 126: The reader has a very simple algorithm. ! 127: It is either ! 128: .i scanning ! 129: for a token, ! 130: .i collecting ! 131: a token, ! 132: or ! 133: .i processing ! 134: a token. ! 135: Scanning involves reading characters and throwing ! 136: away those which don't start tokens (such as blanks and tabs). ! 137: Collecting means gathering the characters which make up a ! 138: token into a buffer. ! 139: Processing may involve creating symbols, strings, lists, ! 140: fixnums, bignums or flonums or calling a user written function called ! 141: a character macro. ! 142: .pp ! 143: The components of the syntax class determine when the reader ! 144: switches between the scanning, collecting and processing states. ! 145: The reader will continue scanning as long as the character class ! 146: of the characters it reads is ! 147: .i cseparator. ! 148: When it reads a character whose character class is not ! 149: .i cseparator ! 150: it stores that character in its buffer and begins the collecting phase. ! 151: .pp ! 152: If the character class of that first character is ! 153: .i ccharacter , ! 154: .i cnumber , ! 155: .i cperiod , ! 156: or ! 157: .i csign . ! 158: then it will continue collecting until it runs into a character whose ! 159: syntax class has the ! 160: .i separator ! 161: property. ! 162: (That last character will be pushed back into the input buffer and will ! 163: be the first character read next time.) ! 164: Now the reader goes into the processing phase, checking to see if the ! 165: token it read is a number or symbol. ! 166: It is important to note that after ! 167: the first character is collected the component of the syntax class which ! 168: tells the reader to stop ! 169: collecting is the ! 170: .i separator ! 171: property, not the character class. ! 172: .pp ! 173: If the character class of the character which stopped the scanning is not ! 174: .i ccharacter , ! 175: .i cnumber , ! 176: .i cperiod , ! 177: or ! 178: .i csign . ! 179: then the reader processes that character immediately. ! 180: The character classes ! 181: .i csingle-macro , ! 182: .i csingle-splicing-macro , ! 183: and ! 184: .i csingle-infix-macro ! 185: will act like ! 186: .i ccharacter ! 187: if the following token is not a ! 188: .i separator. ! 189: The processing which is done for a given character class ! 190: is described in detail in the next section. ! 191: .sh +0 Character\ Classes ! 192: .de Cc ! 193: .sp 2v ! 194: .tl '\fI\\$1\fP''raw readtable:\\$2' ! 195: .tl '''standard readtable:\\$3' ! 196: .. ! 197: .pc ! 198: .Cc ccharacter A-Z\ a-z\ ^H\ !#$%&*,/:;<=>?@^_`{}~ A-Z\ a-z\ ^H\ !$%&*/:;<=>?@^_{}~ ! 199: .pc % ! 200: A normal character. ! 201: .Cc cnumber 0-9 0-9 ! 202: This type is a digit. ! 203: The syntax for an integer (fixnum or bignum) is a string of ! 204: .i cnumber ! 205: characters optionally followed by a ! 206: .i cperiod. ! 207: If the digits are not followed by a ! 208: .i cperiod , ! 209: then they are interpreted in base ! 210: .i ibase ! 211: which must be eight or ten. ! 212: The syntax for a floating point number is ! 213: either zero or more ! 214: .i cnumber 's ! 215: followed by a ! 216: .i cperiod ! 217: and then followed by one or more ! 218: .i cnumber 's. ! 219: A floating point number ! 220: may also be an integer or floating point number followed ! 221: by 'e' or 'd', an optional '+' or '\-' ! 222: and then zero or more ! 223: .i cnumber 's. ! 224: .Cc csign +\- +\- ! 225: A leading sign for a number. ! 226: No other characters should be given this class. ! 227: .Cc cleft-paren ( ( ! 228: A left parenthesis. ! 229: Tells the reader to begin forming a list. ! 230: .Cc cright-paren ) ) ! 231: A right parenthesis. ! 232: Tells the reader that it has reached the end of a list. ! 233: .Cc cleft-bracket [ [ ! 234: A left bracket. ! 235: Tells the reader that it should begin forming a list. ! 236: See the description of ! 237: .i cright-bracket ! 238: for the difference between cleft-bracket and cleft-paren. ! 239: .Cc cright-bracket ] ] ! 240: A right bracket. ! 241: A ! 242: .i cright-bracket ! 243: finishes the formation of the current ! 244: list and all enclosing lists until it finds one which ! 245: begins with a ! 246: .i cleft-bracket ! 247: or until it reaches the ! 248: top level list. ! 249: .Cc cperiod . . ! 250: The period is used to separate element of a cons cell ! 251: [e.g. (a\ .\ (b\ .\ nil)) is the same as (a\ b)]. ! 252: .i cperiod ! 253: is also used in numbers as described above. ! 254: .Cc cseparator ^I-^M\ esc\ space ^I-^M\ esc\ space ! 255: Separates tokens. When the reader is scanning, these character ! 256: are passed over. ! 257: Note: there is a difference between the ! 258: .i cseparator ! 259: character class and the ! 260: .i separator ! 261: property of a syntax class. ! 262: .Cc csingle-quote \\' \\' ! 263: This causes ! 264: .i read ! 265: to be called recursively and the list ! 266: (quote <value read>) to be returned. ! 267: .Cc csymbol-delimiter | | ! 268: This causes the reader to begin collecting characters and to stop only ! 269: when another identical ! 270: .i csymbol-delimiter ! 271: is seen. ! 272: The only way to escape a ! 273: .i csymbol-delimiter ! 274: within a symbol name is with a ! 275: .i cescape ! 276: character. ! 277: The collected characters are converted into a string which becomes ! 278: the print name of a symbol. ! 279: If a symbol with an identical print name already exists, then the ! 280: allocation is not done, rather the existing symbol is used. ! 281: .Cc cescape \e \e ! 282: This causes the next character to read in to be treated as a ! 283: .b vcharacter . ! 284: A character whose syntax class is ! 285: .b vcharacter ! 286: has a character class ! 287: .i ccharacter ! 288: and does not have ! 289: the ! 290: .i separator ! 291: property so it will not separate symbols. ! 292: .Cc cstring-delimiter """" """" ! 293: This is the same as ! 294: .i csymbol-delimiter ! 295: except the result is returned as a string instead of a symbol. ! 296: .Cc csingle-character-symbol none none ! 297: This returns a symbol whose print name is the the single character ! 298: which has been collected. ! 299: .Cc cmacro none `, ! 300: The reader calls the macro function associated with this character and ! 301: the current readtable, passing it no arguments. ! 302: The result of the macro is added to the structure the reader is building, ! 303: just as if that form were directly read by the reader. ! 304: More details on macros are provided below. ! 305: .Cc csplicing-macro none #; ! 306: A ! 307: .i csplicing-macro ! 308: differs from a ! 309: .i cmacro ! 310: in the way the result is incorporated in the structure the reader is ! 311: building. ! 312: A ! 313: .i csplicing-macro ! 314: must return a list of forms (possibly empty). ! 315: The reader acts as ! 316: if it read each element of ! 317: the list itself without ! 318: the surrounding parenthesis. ! 319: .Cc csingle-macro none none ! 320: This causes to reader to check the next character. ! 321: If it is a ! 322: .i cseparator ! 323: then this acts like a ! 324: .i cmacro. ! 325: Otherwise, it acts like a ! 326: .i ccharacter. ! 327: .Cc csingle-splicing-macro none none ! 328: This is triggered like a ! 329: .i csingle-macro ! 330: however the result is spliced in like a ! 331: .i csplicing-macro. ! 332: .Cc cinfix-macro none none ! 333: This is differs from a ! 334: .i cmacro ! 335: in that the macro function is passed a form representing what the reader ! 336: has read so far. ! 337: The result of the macro replaces what the reader had read so far. ! 338: .Cc csingle-infix-macro none none ! 339: This differs from the ! 340: .i cinfix-macro ! 341: in that the macro will only be triggered if the character following the ! 342: .i csingle-infix-macro ! 343: character is a ! 344: .i cseparator . ! 345: .Cc cillegal ^@-^G^N-^Z^\e-^_rubout ^@-^G^N-^Z^\e-^_rubout ! 346: The characters cause the reader to signal an error if read. ! 347: .sh +0 Syntax\ Classes ! 348: .pp ! 349: The readtable maps each character into a syntax class. ! 350: The syntax class contains three pieces of information: ! 351: the character class, whether this is a separator, and the escape ! 352: properties. ! 353: The first two properties are used by the reader, the last by ! 354: the printer (and ! 355: .i explode ). ! 356: The initial lisp system has the following syntax classes defined. ! 357: The user may add syntax classes with ! 358: .i add-syntax-class . ! 359: For each syntax class, we list the properties of the class and ! 360: which characters have this syntax class by default. ! 361: More information about each syntax class can be found under the ! 362: description of the syntax class's character class. ! 363: .de Sy ! 364: .sp 1v ! 365: .(b ! 366: .tl '\fB\\$1\fP''raw readtable:\\$2' ! 367: .tl '\fI\\$4\fP''standard readtable:\\$3' ! 368: .tl '\fI\\$5\fP''' ! 369: .if \n(.$>5 .tl '\fI\\$6\fP''' ! 370: .)b ! 371: .. ! 372: .pc ! 373: .Sy vcharacter A-Z\ a-z\ ^H\ !#$%&*,/:;<=>?@^_`{}~ A-Z\ a-z\ ^H\ !$%&*/:;<=>?@^_{}~ ccharacter ! 374: .pc % ! 375: .Sy vnumber 0-9 0-9 cnumber ! 376: .Sy vsign +- +- csign ! 377: .Sy vleft-paren ( ( cleft-paren escape-always separator ! 378: .Sy vright-paren ) ) cright-paren escape-always separator ! 379: .Sy vleft-bracket [ [ cleft-bracket escape-always separator ! 380: .Sy vright-bracket ] ] cright-bracket escape-always separator ! 381: .Sy vperiod . . cperiod escape-when-unique ! 382: .Sy vseparator ^I-^M\ esc\ space ^I-^M\ esc\ space cseparator escape-always separator ! 383: .Sy vsingle-quote \\' \\' csingle-quote escape-always separator ! 384: .Sy vsymbol-delimiter | | csingle-delimiter escape-always ! 385: .Sy vescape \e \e cescape escape-always ! 386: .Sy vstring-delimiter """" """" cstring-delimiter escape-always ! 387: .Sy vsingle-character-symbol none none csingle-character-symbol separator ! 388: .Sy vmacro none `, cmacro escape-always separator ! 389: .Sy vsplicing-macro none #; csplicing-macro escape-always separator ! 390: .Sy vsingle-macro none none csingle-macro escape-when-unique ! 391: .Sy vsingle-splicing-macro none none csingle-splicing-macro escape-when-unique ! 392: .Sy vinfix-macro none none cinfix-macro escape-always separator ! 393: .Sy vsingle-infix-macro none none csingle-infix-macro escape-when-unique ! 394: .Sy villegal ^@-^G^N-^Z^\e-^_rubout ^@-^G^N-^Z^\e-^_rubout cillegal escape-always separator ! 395: .sh +0 Character\ Macros ! 396: .pp ! 397: Character macros are ! 398: user written functions which are executed during the reading process. ! 399: The value returned by a character macro may or may not be used by ! 400: the reader, depending on the type of macro and the value returned. ! 401: Character macros are always attached to a single character with ! 402: the ! 403: .i setsyntax ! 404: function. ! 405: .sh +1 Types ! 406: There are three types of character macros: normal, splicing and infix. ! 407: These types differ in the arguments they are given or in what is done ! 408: with the result they return. ! 409: .sh +1 Normal ! 410: .pp ! 411: A normal macro ! 412: is passed no arguments. ! 413: The value returned by a normal macro is simply used by ! 414: the reader as if it had read the value itself. ! 415: Here is an example of a macro which returns the abbreviation ! 416: for a given state. ! 417: .Eb ! 418: \->\fI(de\kAfun stateabbrev nil ! 419: \h'|\nAu'(cdr (assq (read) '((california . ca) (pennsylvania . pa)))))\fP ! 420: stateabbrev ! 421: \-> \fI(setsyntax '\e! 'vmacro 'stateabbrev)\fP ! 422: t ! 423: \-> \fI'( ! california ! wyoming ! pennsylvania)\fP ! 424: (ca nil pa) ! 425: .Ee ! 426: Notice what happened to ! 427: \fI ! wyoming\fP. ! 428: Since it wasn't in the table, the associated function ! 429: returned nil. ! 430: The creator of the macro may have wanted to leave the ! 431: list alone, in such a case, but couldn't with this ! 432: type of reader macro. ! 433: The splicing macro, described next, allows a character macro function ! 434: to return a value that is ignored. ! 435: .sh +0 Splicing ! 436: .pp ! 437: The value returned from a splicing macro must be a list or nil. ! 438: If the value is nil, then the value is ignored, otherwise the reader ! 439: acts as if it read each object in the list. ! 440: Usually the list only contains one element. ! 441: If the reader is reading at the top level (i.e. not collecting elements ! 442: of list), ! 443: then it is illegal for a splicing macro to return more then one ! 444: element in the list. ! 445: The major advantage of a splicing macro over a normal macro is the ! 446: ability of the splicing macro to return nothing. ! 447: The comment character (usually ;) is a splicing macro bound to a ! 448: function which reads to the end of the line and always returns nil. ! 449: Here is the previous example written as a splicing macro ! 450: .Eb ! 451: \-> \fI(de\kAfun stateabbrev nil ! 452: \h'|\nAu'(\kC(lam\kBbda (value) ! 453: \h'|\nBu'(cond \kA(value (list value)) ! 454: \h'|\nAu'(t nil))) ! 455: \h'|\nCu'(cdr (assq (read) '((california . ca) (pennsylvania . pa))))))\fP ! 456: \-> \fI(setsyntax '! 'vsplicing-macro 'stateabbrev)\fP ! 457: \-> \fI'(!pennsylvania ! foo !california)\fP ! 458: (pa ca) ! 459: \-> \fI'!foo !bar !pennsylvania\fP ! 460: pa ! 461: \-> ! 462: .Ee ! 463: .sh +0 Infix ! 464: .pp ! 465: Infix macros are passed a ! 466: .i conc ! 467: structure representing what has been read so far. ! 468: Briefly, a ! 469: tconc ! 470: structure is a single list cell whose car points to ! 471: a list and whose cdr points to the last list cell in that list. ! 472: The interpretation by the reader of the value ! 473: returned by an infix macro depends on ! 474: whether the macro is called while the reader is constructing a ! 475: list or whether it is called at the top level of the reader. ! 476: If the macro is called while a list is ! 477: being constructed, then the value returned should be a tconc ! 478: structure. ! 479: The car of that structure replaces the list of elements that the ! 480: reader has been collecting. ! 481: If the macro is called at top level, then it will be passed the ! 482: value nil, and the value it returns should either be nil ! 483: or a tconc structure. ! 484: If the macro returns nil, then the value is ignored and the reader ! 485: continues to read. ! 486: If the macro returns a tconc structure of one element (i.e. whose car ! 487: is a list of one element), then that single element is returned ! 488: as the value of ! 489: .i read. ! 490: If the macro returns a tconc structure of more than one element, ! 491: then that list of elements is returned as the value of read. ! 492: .Eb ! 493: \-> \fI(de\kAfun plusop (x) ! 494: \h'|\nAu'(cond \kB((null x) (tconc nil '\e+)) ! 495: \h'|\nBu'(t (lconc nil (list 'plus (caar x) (read))))))\fP ! 496: ! 497: plusop ! 498: \-> \fI(setsyntax '\e+ 'vinfix-macro 'plusop)\fP ! 499: t ! 500: \-> \fI'(a + b)\fP ! 501: (plus a b) ! 502: \-> \fI'+\fP ! 503: |+| ! 504: \-> ! 505: .Ee ! 506: .sh -1 Invocations ! 507: .pp ! 508: There are three different circumstances in which you would like ! 509: a macro function to be triggered. ! 510: .ip \fIAlways\ -\fP ! 511: Whenever the macro character is seen, the macro should be invoked. ! 512: This is accomplished by using the character classes ! 513: .i cmacro , ! 514: .i csplicing-macro , ! 515: or ! 516: .i cinfix-macro , ! 517: and by using the ! 518: .i separator ! 519: property. ! 520: The syntax classes ! 521: .b vmacro , ! 522: .b vsplicing-macro , ! 523: and ! 524: .b vsingle-macro ! 525: are defined this way. ! 526: .ip \fIWhen\ first\ -\fP ! 527: The macro should only be triggered when the macro character is the first ! 528: character found after the scanning process. ! 529: A syntax class for a ! 530: .i when ! 531: .i first ! 532: macro would ! 533: be defined ! 534: using ! 535: .i cmacro , ! 536: .i csplicing-macro , ! 537: or ! 538: .i cinfix-macro ! 539: and not including the ! 540: .i separator ! 541: property. ! 542: .ip \fIWhen\ unique\ -\fP ! 543: The macro should only be triggered when the macro character is the only ! 544: character collected in the token collection ! 545: phase of the reader, ! 546: i.e the macro character is preceeded by zero or more ! 547: .i cseparator s ! 548: and followed by a ! 549: .i separator. ! 550: A syntax class for a ! 551: .i when ! 552: .i unique ! 553: macro would ! 554: be defined using ! 555: .i csingle-macro , ! 556: .i csingle-splicing-macro , ! 557: or ! 558: .i csingle-infix-macro ! 559: and not including the ! 560: .i separator ! 561: property. ! 562: The syntax classes so defined are ! 563: .b vsingle-macro , ! 564: .b vsingle-splicing-macro , ! 565: and ! 566: .b vsingle-infix-macro . ! 567: .sh -1 Functions ! 568: .Lf setsyntax 's_symbol\ 's_synclass\ ['ls_func] ! 569: .Wh ! 570: ls_func is the name of a function or a lambda body. ! 571: .Re ! 572: t ! 573: .Se ! 574: S_symbol should be a symbol whose print name is only one character. ! 575: The syntax class for ! 576: that character is ! 577: set to s_synclass in the current readtable. ! 578: If s_synclass is a class that requires a character macro, then ! 579: ls_func must be supplied. ! 580: .No ! 581: The symbolic syntax codes are new to Opus 38. ! 582: For compatibility, s_synclass can be one of the fixnum syntax codes ! 583: which appeared in older versions of the ! 584: .Fr ! 585: Manual. ! 586: This compatibility is only temporary: existing code which uses the ! 587: fixnum syntax codes should be converted. ! 588: .Lf getsyntax 's_symbol ! 589: .Re ! 590: the syntax class of the first character ! 591: of s_symbol's print name. ! 592: s_symbol's print name must be exactly one character long. ! 593: .No ! 594: This function is new to Opus 38. ! 595: It supercedes \fI(status\ syntax)\fP which no longer exists. ! 596: .Lf add-syntax-class 's_synclass\ 'l_properties ! 597: .Re ! 598: s_synclass ! 599: .Se ! 600: Defines the syntax class s_synclass to have properties l_properties. ! 601: The list l_properties should contain a character classes mentioned ! 602: above. ! 603: l_properties may contain one of the escape properties: ! 604: .i escape-always , ! 605: .i escape-when-unique , ! 606: or ! 607: .i escape-when-first . ! 608: l_properties may contain the ! 609: .i separator ! 610: property. ! 611: After a syntax class has been defined with ! 612: .i add-syntax-class , ! 613: the ! 614: .i setsyntax ! 615: function can be used to give characters that syntax class. ! 616: .Eb ! 617: ; Define a non-separating macro character. ! 618: ; This type of macro character is used in UCI-Lisp, and ! 619: ; it corresponds to a FIRST MACRO in Interlisp ! 620: \-> \fI(add-syntax-class 'vuci-macro '(cmacro escape-when-first))\fP ! 621: vuci-macro ! 622: \-> ! 623: .Ee
This archive runs on limited infrastructure. Preserving old code on modern bandwidth. Automated agents are requested to crawl responsibly.