Return to popbb.txt CVS log

Up to [CSRG BSD Unix] / 43BSD / contrib / mh / support / pop

Annotation of 43BSD/contrib/mh/support/pop/popbb.txt, revision 1.1

1.1     ! root        1: Request For Comments:  draft
        !             2: 
        !             3: 
        !             4: 
        !             5: 
        !             6: 
        !             7: 
        !             8: 
        !             9: 
        !            10: 
        !            11: 
        !            12: 
        !            13: 
        !            14:                      Post Office Protocol (revised)
        !            15:                        Extended Service Offerings
        !            16: 
        !            17: 
        !            18:                         Wed Dec 18 23:17:52 1985
        !            19: 
        !            20: 
        !            21:                             Marshall T. Rose
        !            22: 
        !            23:                 Northrop Research and Technology Center
        !            24:                            One Research Park
        !            25:                    Palos Verdes Peninsula, CA  90274
        !            26: 
        !            27:                            MRose%NRTC@USC-ECL
        !            28: 
        !            29: 
        !            30: 
        !            31: 
        !            32:     This memo suggests a simple method for workstations to dynamically
        !            33:     access mail from a discussion group server, as an extension to an
        !            34:     earlier memo which dealt with dynamically accessing mail from a
        !            35:     mailbox server using the Post Office Protocol (POP).  This RFC
        !            36:     specifies a proposed protocol for the ARPA Internet community, and
        !            37:     requests discussion and suggestions for improvements.  All of the
        !            38:     extensions described in this memo to the POP are OPTIONAL.  
        !            39: 
        !            40: Request For Comments:  draft                                    M. Rose
        !            41: POP (revised) Extended Service Offerings                           NRTC
        !            42: 
        !            43: 
        !            44: 
        !            45:                       Introduction and Motivation
        !            46: 
        !            47:     It is assumed that the reader is familiar with the previous memo
        !            48:     that discusses the Post Office Protocol (POP) [MRose84].  This memo
        !            49:     describes extensions to the POP which enhance the service it offers
        !            50:     to clients.  This additional service permits a client host to access
        !            51:     discussion group mail, which is often kept in a separate spool area,
        !            52:     using the general POP facilities.  
        !            53: 
        !            54:     The next section describes the evolution of discussion groups and
        !            55:     the technologies currently used to implement them.  To summarize:
        !            56: 
        !            57:        o An exploder is used to map from a single address to
        !            58:          a list of addresses which subscribe to the list, and redirects
        !            59:          any subsequent error reports associated with the delivery of
        !            60:          each message.  This has two primary advantages:
        !            61:                - Subscribers need know only a single address
        !            62:                - Responsible parties get the error reports and not
        !            63:                  the subscribers
        !            64: 
        !            65:        o Typically, each subscription address is not a person's private
        !            66:          maildrop, but a system-wide maildrop, which can be accessed
        !            67:          by more than one user.  This has several advantages:
        !            68:                - Only a single copy of each message need traverse the
        !            69:                  net for a given site (which may contain several local
        !            70:                  hosts).  This conserves bandwidth and cycles.
        !            71:                - Only a single copy of each message need reside on each
        !            72:                  subscribing host.  This conserves disk space.
        !            73:                - The private maildrop for each user is not cluttered
        !            74:                  with discussion group mail.
        !            75: 
        !            76:     Despite this optimization of resources, further economy can be
        !            77:     achieved at sites with more than one host.  Typically, sites with
        !            78:     more than one host either:
        !            79: 
        !            80:         1.  Replicate discussion group mail on each host.  This
        !            81:         results in literally gigabytes of disk space committed to
        !            82:         unnecessarily store redundant information.
        !            83: 
        !            84:         2.  Keep discussion group mail on one host and give all users a
        !            85:         login on that host (in addition to any other logins they may
        !            86:         have).  This is usually a gross inconvenience for users who
        !            87:         work on other hosts, or a burden to users who are forced to
        !            88:         work on that host.  
        !            89: 
        !            90:     As discussed in [MRose84], the problem of giving workstations
        !            91:     dynamic access to mail from a mailbox server has been explored in
        !            92:     great detail (originally there was [RFC918], this prompted the
        !            93:     author to write [MRose84], independently of this [RFC918] was
        !            94:     upgraded to [RFC937]).  A natural solution to the problem outlined
        !            95:     above is to keep discussion group mail on a mailbox server at each
        !            96:     site and permit different hosts at that site to employ the POP to
        !            97:     access discussion group mail.  If implemented properly, this
        !            98:     avoids the problems of both strategies outlined above.
        !            99: 
        !           100:        ASIDE:     It might be noted that a good distributed filesystem
        !           101:                   could also solve this problem.  Sadly, "good"
        !           102:                   distributed filesystems, which do not suffer
        !           103:                   unacceptable response time for interactive use, are
        !           104:                   few and far between these days!  
        !           105: 
        !           106:     Given this motivation, now let's consider discussion groups, both in
        !           107:     general and from the point of view of a user agent.  Following this,
        !           108:     extensions to the POP defined in [MRose84] are presented.  Finally,
        !           109:     some additional policy details are discussed along with some initial
        !           110:     experiences.
        !           111: 
        !           112:                       What's in a Discussion Group
        !           113: 
        !           114:     Since mailers and user agents first crawled out of the primordial
        !           115:     ARPAnet, the value of discussion groups have been appreciated,
        !           116:     (though their implementation has not always been well-understood).
        !           117: 
        !           118:     Described simply, an discussion group is composed of a number of
        !           119:     subscribers with a common interest.  These subscribers post mail to
        !           120:     a single address, known as a distribution address.  From this
        !           121:     distribution address, a copy of the message is sent to each
        !           122:     subscriber.  Each group has a moderator, which is the person that
        !           123:     administrates the group.  The moderator can usually be reached at a
        !           124:     special address, known as a request address.  Usually, the
        !           125:     responsibilities of the moderator are quite simple, since the mail
        !           126:     system handles the distribution to subscribers automatically.  In
        !           127:     some cases, the interest group, instead of being distributed
        !           128:     directly to its subscribers, is put into a digest format by the
        !           129:     moderator and then sent to the subscribers.  Although this requires
        !           130:     more work on the part of the moderator, such groups tend to be
        !           131:     better organized.  
        !           132: 
        !           133:     Unfortunately, there are a few problems with the scheme outlined
        !           134:     above.  First, if two users on the same host subscribe to the same
        !           135:     interest group, two copies of the message get delivered.  This is
        !           136:     wasteful of both processor and disk resources.  
        !           137: 
        !           138:     Second, some of these groups carry a lot of traffic.  Although
        !           139:     subscription to an group does indicate interest on the part of a
        !           140:     subscriber, it is usually not interesting to get 50 messages or so
        !           141:     delivered to the user's private maildrop each day, interspersed with
        !           142:     personal mail, that is likely to be of a much more important and
        !           143:     timely nature.  
        !           144: 
        !           145:     Third, if a subscriber on the distribution list for a group becomes
        !           146:     "bad" somehow, the originator of the message and not the moderator
        !           147:     of the group is notified.  It is not uncommon for a large list to
        !           148:     have 10 or so bogus addresses present.  This results in the
        !           149:     originator being flooded with "error messages" from mailers across
        !           150:     the ARPA Internet stating that a given address on the list was bad.
        !           151:     Needless to say, the originator usually could not care less if the
        !           152:     bogus addresses got a copy of the message or not.  The originator is
        !           153:     merely interested in posting a message to the group at large.
        !           154:     Furthermore, the moderator of the group does care if there are bogus
        !           155:     addresses on the list, but ironically does not receive notification.
        !           156: 
        !           157:     There are various approaches which can be used to solve some or all
        !           158:     of these problems.  Usually these involve placing an exploder agent
        !           159:     at the distribution source of the discussion group, which expands
        !           160:     the name of the group into the list of subscription addresses for
        !           161:     the group.  In the process, the exploder will also change the
        !           162:     address that receives error notifications to be the request address
        !           163:     or other responsible party.  
        !           164: 
        !           165:     A complementary approach, used in order to cut down on resource
        !           166:     utilization of all kinds, replaces all the subscribers at a single
        !           167:     host (or group of hosts under a single administration) with a single
        !           168:     address at that host.  This address maps to a file on the host,
        !           169:     usually in a spool area, which all users can access.  (Advanced
        !           170:     implementations can also implement private discussion groups this
        !           171:     way, in which a single copy of each message is kept, but is
        !           172:     accessible to only a select number of users on the host.)  
        !           173: 
        !           174:     The two approaches can be combined to avoid all of the problems
        !           175:     described above.
        !           176: 
        !           177:     Finally, a third approach can be taken, which can be used to aid
        !           178:     user agents processing mail for the discussion group:  In order to
        !           179:     speed querying of the maildrop which contains the local host's copy
        !           180:     of the discussion group, two other items are usually associated with
        !           181:     the discussion group, on a local basis.  These are the maxima and
        !           182:     the last-date.  Each time a message is received for the group on the
        !           183:     local host, the maxima is increased by at least one.  Furthermore,
        !           184:     when a new maxima is generated, the current date is determined.
        !           185:     This is called the last date.  As the message is entered into the
        !           186:     local maildrop, it is given the current maxima and last-date.  This
        !           187:     permits the user agent to quickly determine if new messages are
        !           188:     present in the maildrop.
        !           189: 
        !           190:        NOTE:      The maxima may be characterized as a monotonically
        !           191:                   increasing quanity.  Although sucessive values of the
        !           192:                   maxima need not be consecutive, any maxima assigned
        !           193:                   is always greater than any previously assigned value.
        !           194: 
        !           195:                           Definition of Terms
        !           196: 
        !           197:     To formalize these notions somewhat, consider the following 7
        !           198:     parameters which describe a given discussion group from the
        !           199:     perspective of the user agent (the syntax given is from [RFC822]):
        !           200: 
        !           201:        NAME            Meaning: the name of the discussion group
        !           202:                        Syntax:  TOKEN (ALPHA *[ ALPHA / DIGIT / "-" ])
        !           203:                                 (case-insensitive recognition)
        !           204:                        Example: unix-wizards
        !           205: 
        !           206:        ALIASES         Meaning: alternates names for the group, which
        !           207:                                 are locally meaningful; these are
        !           208:                                 typically used to shorten user typein
        !           209:                        Syntax:  TOKEN (case-insensitive recognition)
        !           210:                        Example: uwiz
        !           211: 
        !           212:        ADDRESS         Meaning: the primary source of the group
        !           213:                        Syntax:  822 address
        !           214:                        Example: Unix-Wizards@BRL
        !           215: 
        !           216:        REQUEST         Meaning: the primary moderator of the group
        !           217:                        Syntax:  822 address
        !           218:                        Example: Unix-Wizards-Request@BRL
        !           219: 
        !           220:        FLAGS           Meaning: locally meaningful flags associated
        !           221:                                 with the discussion group; this memo
        !           222:                                 leaves interpretation of this parameter
        !           223:                                 to each POP implementation
        !           224:                        Syntax:  octal number
        !           225:                        Example: 01
        !           226: 
        !           227:        MAXIMA          Meaning: the magic cookie associated with the
        !           228:                                 last message locally received for the
        !           229:                                 group; it is the property of the magic
        !           230:                                 cookie that it's value NEVER decreases,
        !           231:                                 and increases by at least one each time
        !           232:                                 a message is locally received
        !           233:                        Syntax:  decimal number
        !           234:                        Example: 1004
        !           235: 
        !           236:        LASTDATE        Meaning: the date that the last message was
        !           237:                                 locally received
        !           238:                        Syntax:  822 date
        !           239:                        Example: Thu, 19 Dec 85 10:26:48 -0800
        !           240: 
        !           241:     Note that the last two values are locally determined for the
        !           242:     maildrop associated with the discussion group and with each message
        !           243:     in that maildrop.  Note however that the last message in the
        !           244:     maildrop have a different MAXIMA and LASTDATE than the discussion
        !           245:     group.  This often occurs when the maildrop has been archived.
        !           246: 
        !           247:     Finally, some local systems provide mechanisms for automatically
        !           248:     archiving discussion group mail.  In some cases, a two-level archive
        !           249:     scheme is used:  current mail is kept in the standard maildrop,
        !           250:     recent mail is kept in an archive maildrop, and older mail is kept
        !           251:     off-line.  With this scheme, in addition to having a "standard"
        !           252:     maildrop for each discussion group, an "archive" maildrop may also
        !           253:     be available.  This permits a user agent to examine the most recent
        !           254:     archive using the same mechanisms as those used on the current mail.
        !           255: 
        !           256:                             The XTND Command
        !           257: 
        !           258:     The following commands are valid only in the TRANSACTION state of
        !           259:     the POP.  This implies that the POP server has already opened the
        !           260:     user's maildrop (which may be empty).  This maildrop is called the
        !           261:     "default maildrop".  The phrase "closes the current maildrop" has
        !           262:     two meanings, depending on whether the current maildrop is the
        !           263:     default maildrop or is a maildrop associated with a discussion
        !           264:     group. 
        !           265: 
        !           266:     In the former context, when the current maildrop is closed any
        !           267:     messages marked as deleted are removed from the maildrop currently
        !           268:     in use.  The exclusive-access lock on the maildrop is then released
        !           269:     along with any implementation-specific resources (e.g.,
        !           270:     file-descriptors).  
        !           271: 
        !           272:     In the latter context, a maildrop associated with a discussion group
        !           273:     is considered to be read-only to the POP client.  In this case, the
        !           274:     phrase "closes the current maildrop" merely means that any
        !           275:     implementation-specific resources are released.  (Hence, the POP
        !           276:     command DELE is a no-op.)
        !           277: 
        !           278:     All the new facilities are introduced via a single POP command,
        !           279:     XTND.  All positive reponses to the XTND command are multi-line.
        !           280: 
        !           281:     The most common multi-line response to the commands contains a
        !           282:     "discussion group listing" which presents the name of the discussion
        !           283:     group along with it's maxima.  In order to simplify parsing all POP
        !           284:     servers are required to use a certain format for discussion group
        !           285:     listings:  
        !           286: 
        !           287:                              NAME SP MAXIMA 
        !           288: 
        !           289:     This memo makes no requirement on what follows the maxima in the
        !           290:     listing.  Minimal implementations should just end that line of the
        !           291:     response with a CRLF pair.  More advanced implementations may
        !           292:     include other information, as parsed from the message.  
        !           293: 
        !           294:        NOTE:      This memo STRONGLY discourages implementations from
        !           295:                   supplying additional information in the listing.  
        !           296: 
        !           297: 
        !           298:     XTND BBOARDS [name]
        !           299:        Arguments: the name of a discussion group (optionally)
        !           300:        Restrictions: may only be given in the TRANSACTION state.
        !           301:        Discussion:
        !           302: 
        !           303:         If an argument was given, the POP server closes the current
        !           304:         maildrop.  The POP server then validates the argument as the
        !           305:         name of a discussion group.  If this is successful, it opens
        !           306:         the maildrop associated with the group, and returns a
        !           307:         multi-line response containing the discussion group listing.
        !           308:         If the discussion group named is not valid, or the associated
        !           309:         archive maildrop is not readable by the user, then an error
        !           310:         response is returned.  
        !           311: 
        !           312:         If no argument was given, the POP server issues a multi-line
        !           313:         response.  After the initial +OK, for each discussion group
        !           314:         known, the POP server responds with a line containing the
        !           315:         listing for that discussion group.  Note that only
        !           316:         world-readable discussion groups are included in the multi-line
        !           317:         response.  
        !           318: 
        !           319:         In order to aid user agents, this memo requires an extension to
        !           320:         the scan listing when an "XTND BBOARDS" command has been given.
        !           321:         Normally, a scan listing, as generated by the LIST, takes the
        !           322:         form:
        !           323: 
        !           324:                MSGNO SIZE
        !           325: 
        !           326:         where MSGNO is the number of the message being listed and SIZE
        !           327:         is the size of the message in octets.  When reading a maildrop
        !           328:         accessed via "XTND BBOARDS", the scan listing takes the form 
        !           329: 
        !           330:                MSGNO SIZE MAXIMA
        !           331: 
        !           332:         where MAXIMA is the maxima that was assigned to the message
        !           333:         when it was placed in the BBoard.
        !           334: 
        !           335:        Possible Responses:
        !           336:            +OK XTND
        !           337:            -ERR no such bboard
        !           338:        Examples:
        !           339:            C:    XTND BBOARDS
        !           340:            S:    +OK XTND
        !           341:            S:    system 10
        !           342:            S:    mh-users 100
        !           343:            S:    .
        !           344:            C:    XTND BBOARDS system
        !           345:            S:    + OK XTND
        !           346:            S:    system 10
        !           347:            S:    .
        !           348: 
        !           349:     XTND ARCHIVE name
        !           350:        Arguments: the name of a discussion group (required)
        !           351:        Restrictions: may only be given in the TRANSACTION state.
        !           352:        Discussion:
        !           353: 
        !           354:         The POP server closes the current maildrop.  The POP server
        !           355:         then validates the argument as the name of a discussion group.
        !           356:         If this is successful, it opens the archive maildrop associated
        !           357:         with the group, and returns a multi-line response containing
        !           358:         the discussion group listing.  If the discussion group named is
        !           359:         not valid, or the associated archive maildrop is not readable
        !           360:         by the user, then an error response is returned.  
        !           361: 
        !           362:         In addition, the scan listing generated by the LIST command is
        !           363:         augmented (as described above).
        !           364: 
        !           365:        Possible Responses:
        !           366:            +OK XTND
        !           367:            -ERR no such bboard
        !           368:        Examples:
        !           369:            C:    XTND ARCHIVE system
        !           370:            S:    + OK XTND
        !           371:            S:    system 3
        !           372:            S:    .
        !           373: 
        !           374:     XTND X-BBOARDS name
        !           375:        Arguments: the name of a discussion group (required)
        !           376:        Restrictions: may only be given in the TRANSACTION state.
        !           377:        Discussion:
        !           378: 
        !           379:         The POP server validates the argument as the name of a
        !           380:         discussion group.  If this is unsuccessful, then an error
        !           381:         response is returned.  Otherwise a multi-line response is
        !           382:         returned.  The first 14 lines of this response (after the
        !           383:         initial +OK) are defined in this memo.  Minimal implementations
        !           384:         need not include other information (and may omit certain
        !           385:         information, outputing a bare CRLF pair).  More advanced
        !           386:         implementations may include other information.  
        !           387: 
        !           388:                Line    Information (refer to "Definition of Terms")
        !           389:                ----    -----------
        !           390:                  1     NAME
        !           391:                  2     ALIASES, separated by SP
        !           392:                  3     system-specific: maildrop
        !           393:                  4     system-specific: archive maildrop
        !           394:                  5     system-specific: information
        !           395:                  6     system-specific: maildrop map
        !           396:                  7     system-specific: encrypted password
        !           397:                  8     system-specific: local leaders, separated by SP
        !           398:                  9     ADDRESS
        !           399:                 10     REQUEST
        !           400:                 11     system-specific: incoming feed
        !           401:                 12     system-specific: outgoing feeds
        !           402:                 13     FLAGS SP MAXIMA
        !           403:                 14     LASTDATE
        !           404: 
        !           405:         Most of this information is entirely too specific to the UCI
        !           406:         Version of the Rand MH Message Handling System[MRose85].
        !           407:         Nevertheless, lines 1, 2, 9, 10, 13, and 14 are of general
        !           408:         interest, regardless of the implementation.  
        !           409: 
        !           410:        Possible Responses:
        !           411:            +OK XTND
        !           412:            -ERR no such bboard
        !           413:        Examples:
        !           414:            C:    XTND X-BBOARDS system
        !           415:            S:    + OK XTND
        !           416:            S:    system
        !           417:            S:    local general
        !           418:            S:    /usr/bboards/system.mbox
        !           419:            S:    /usr/bboards/archive/system.mbox
        !           420:            S:    /usr/bboards/.system.cnt
        !           421:            S:    /usr/bboards/.system.map
        !           422:            S:    *
        !           423:            S:    mother
        !           424:            S:    system@nrtc
        !           425:            S:    system-request@nrtc
        !           426:            S:    
        !           427:            S:    dist-system@nrtc-gremlin
        !           428:            S:    01 10
        !           429:            S:    Thu, 19 Dec 85 00:08:49 -0800
        !           430:            S:    .
        !           431: 
        !           432:                               Policy Notes
        !           433: 
        !           434:     Depending on the particular entity administrating the POP service
        !           435:     host, two additional policies might be implemented:
        !           436: 
        !           437:     1.  Private Discussion Groups 
        !           438: 
        !           439:     In the general case, discussion groups are world-readable, any user,
        !           440:     once logged in (via a terminal, terminal server, or POP, etc.), is
        !           441:     able to read the maildrop for each discussion group known to the POP
        !           442:     service host.  Nevertheless, it is desirable, usually for privacy
        !           443:     reasons, to implement private discussion groups as well.  
        !           444: 
        !           445:     Support of this is consistent with the extensions outlined in this
        !           446:     memo.  Once the AUTHORIZATION state has successfully concluded, the
        !           447:     POP server grants the user access to exactly those discussion groups
        !           448:     the POP service host permits the authenticated user to access.  As a
        !           449:     "security" feature, discussion groups associated with unreadable
        !           450:     maildrops should not be listed in a positive response to the XTND
        !           451:     BBOARDS command.
        !           452: 
        !           453:     2.  Anonymous POP Users 
        !           454: 
        !           455:     In order to minimize the authentication problem, a policy
        !           456:     permitting "anonymous" access to the world-readable maildrops for
        !           457:     discussion groups on the POP server may be implemented.
        !           458: 
        !           459:     Support of this is consistent with the extensions outlined in this
        !           460:     memo.  The POP server can be modified to accept a USER command for a
        !           461:     well-known pseudonym (i.e., "anonymous") which is valid with any
        !           462:     PASS command.  As a "security" feature, it is advisable to limit
        !           463:     this kind of access to only hosts at the local site, or to hosts
        !           464:     named in an access list.
        !           465: 
        !           466:                       Experiences and Conclusions
        !           467: 
        !           468:     All of the facilities described in this memo and in [MRose84] have
        !           469:     been implemented in MH #6.1.  Initial experiences have been, on the
        !           470:     whole, very positive. 
        !           471: 
        !           472:     After the first implementation, some performance tuning was
        !           473:     required.  This consisted primarily of caching the datastructures
        !           474:     which describe discussion groups in the POP server.  A second
        !           475:     optimization pertained to the client:  the program most commonly
        !           476:     used to read BBoards in MH was modified to retrieve messages only
        !           477:     when needed.  Two schemes are used:
        !           478: 
        !           479:        o If only the headers (and the first few lines of the body) of
        !           480:         the message are required, e.g., for a scan listing, then only
        !           481:         these are retrieved.  The resulting output is then cached, on
        !           482:         a per-message basis.
        !           483: 
        !           484:        o If the entire message is required, then it is retrieved intact,
        !           485:         and cached locally.  
        !           486: 
        !           487:     With these optimizations, response time is quite adequate when the
        !           488:     POP server and client are connected via a high-speed local area
        !           489:     network.  In fact, the author uses this mechanism to access certain
        !           490:     private discussion groups over the ARPAnet.  In this case, response
        !           491:     is still good.  When a 9.6Kbps modem is inserted in the path,
        !           492:     response went from good to almost tolerable (fortunately the author
        !           493:     only reads a few discussion groups in this fashion).  
        !           494: 
        !           495:     To conclude:  the POP is a good thing, not only for personal mail
        !           496:     but for discussion group mail as well.
        !           497: 
        !           498:                                References
        !           499: 
        !           500:     [MRose84]   M.T. Rose.
        !           501:                "Post Office Protocol (revised)", University of Delaware.
        !           502:                (October, 1984)
        !           503: 
        !           504:     [MRose85]  M.T. Rose, J.L. Romine.
        !           505:                "The Rand MH Message Handling System: User's Manual",
        !           506:                University of California, Irvine.  (November, 1985)
        !           507: 
        !           508:     [RFC822]   D.H. Crocker.
        !           509:                "Standard for the Format of ARPA Internet Text
        !           510:                Messages", University of Delaware.  (August, 1982)
        !           511: 
        !           512:     [RFC918]    J.K. Reynolds.
        !           513:                "Post Office Protocol", USC/Information Sciences Institute.
        !           514:                (October, 1984)
        !           515: 
        !           516:     [RFC937]    M. Butler, J.B. Postel, et. al.
        !           517:                "Post Office Protocol - Version 2", USC/Information Sciences
        !           518:                Institute.
        !           519:                (February, 1985)

This archive runs on limited infrastructure. Preserving old code on modern bandwidth. Automated agents are requested to crawl responsibly.