Return to READ_ME CVS log

Up to [MW Coherent from dump] / coherent / d / bin / spell

Annotation of coherent/d/bin/spell/READ_ME, revision 1.1.1.1

1.1       root        1:        This directory contains the sources for the spell and typo commands.
                      2: This file is intended to aid the dictionary maintainer in tuning it
                      3: for his local site.
                      4: 
                      5: The Dictionary
                      6: 
                      7:        The dictionaries for both typo and spell are stored in the directory
                      8: `/usr/dict'.  We will not discuss the typo dictionary here as it is not
                      9: intended to be extended, rather being provided as a screen of the most common
                     10: words.  Remaining words are ordered statistically, rather than by dictionary
                     11: lookup.  Conversely, spell uses an exact membership tester rather than one of
                     12: the several approximate membership tester techniques found in the literature.
                     13: The resultant small speed penalty is more than offset by the ability to have an
                     14: almost perfect lookup record with essentially no misses.  The only misses
                     15: will be such misspelling as the wrong homonym (`their' for `there') or
                     16: the misuse of a proper name (e.g. `manuel' for `manual').
                     17: 
                     18:        The dictionary is maintained as a set of word lists.  As there
                     19: are really two dictionaries--a British one and an American one--a set
                     20: of common words (`common') is maintained as well as files with each of
                     21: American (`american') and British (`british') spellings.  More will be
                     22: said later about what words should go into each file.  In addition, there
                     23: is a file reserved for the local dictionary maintainer, called `local',
                     24: which allows him to add words without changing the distributed lists.
                     25: These files can be merged (`sort -mu common british local' or
                     26: `sort -mu common american local') to produce the two master lists.
                     27: These lists are never normally stored in this form, rather the compressed
                     28: lists are stored.  This compression reduces the amount of I/O that must
                     29: occur for the spell command to read in the dictionary.  These compressed
                     30: files are named `clista' and `clistb', for American and British, respectively.
                     31: The shell file `build' reconstructs the compressed lists from these components
                     32: by using the above `sort' commands and the internal comman
                     33: `/usr/src/cmd/spell/spellin' which takes a list of words on its
                     34: standard input and produces a compressed list on its standard output.
                     35: 
                     36:        In these words lists, a trailing asterisk (`#') on a word implies that
                     37: there is an optional `s'.  This may signify a plural or a different verb form.
                     38: This shorthand is included so that such plurals can be cheaply maintained
                     39: without the sacrifice in accuracy that would occur with arbitrary suffix
                     40: and prefix stripping code.
                     41: 
                     42: British Versus American
                     43: 
                     44:        Since there is some dissension in British spelling regarding words
                     45: that end in `ize' or `ise', the `ize' spelling is included in the common
                     46: file and a corresponding entry with `ise' is put into the British file
                     47: (e.g. optimize vs. optimise).  Such words as `honor' vs. `honour' are
                     48: put into the American and British files, respectively.  However,
                     49: in the US there are words which are spelled two ways (e.g. `glamour' vs.
                     50: `glamor') so that the former is put into the common file and the
                     51: latter is put into the American file.  There are no hard and fast rules
                     52: in any case, and thus there are bound to be disagreements no matter what
                     53: is done here.
                     54: 
                     55: Updating Word Lists
                     56: 
                     57:        Spell maintains a history file, `spellhist', which contains all
                     58: misspelled words that are found.  This will contain a large number of
                     59: words that should not go into the dictionary (e.g. opcodes, variable names,
                     60: very obscure technical terms, etc.) as well as words that should be added.
                     61: This list should be screened periodically and truncated.
                     62: 
                     63:        Some of the files, especially `common', are too large to edit with
                     64: `ed' and thus the stream editor (`sed') can be used to manipulate it.
                     65: Also, if it is desired to change `common' rather than `local', new words
                     66: can be put in by creating a new words file, say `new', and merging it
                     67: with sort into a new `common' file (e.g. `sort -mu -o common common new').
                     68: 
                     69:        As well, it is always a good idea to check when adding new words
                     70: that the singular is not already there to save adding a plural.
                     71: For example, if the word `macro' was already in the dictionary and
                     72: `macros' was to be added, one could issue the following `sed' command
                     73: to change `macro' to `macro#' (singular or plural):
                     74:        sed 's/^macro$/macro#/' common >new; mv new common
                     75: 
                     76:        One last word of caution is in order.  Word lists that are updated
                     77: should be scrupulously checked for errors with a dictionary.  Nothing is
                     78: worse than a spelling dictionary with misspellings in it.  This can happen
                     79: quite easily if a common misspelling is believed from the history file.

This archive runs on limited infrastructure. Preserving old code on modern bandwidth. Automated agents are requested to crawl responsibly.