Annotation of 43BSDReno/contrib/isode-beta/doc/quipu/distribution.tex, revision 1.1.1.1

1.1       root        1: 
                      2: \chapter {Distributed Operation}
                      3: \label {dist-op}
                      4: 
                      5: \section {Overview}
                      6: 
                      7: 
                      8: Distributed Operation is a major aspect of the QUIPU
                      9: Directory Service
                     10: Sadly, the OSI Directory specifications in this area are, in the author's
                     11: opinion, rather unsatisfactory.   
                     12: Therefore, the QUIPU distributed operations are described in a slightly
                     13: different manner.  The concept of ``Naming Context'' is not used, and the
                     14: significance of ``Knowledge'' is de-emphasised.  The external view of this
                     15: functionality is fully in line with the standard.   
                     16: 
                     17: Some of the concepts 
                     18: defined in this chapter  do {\em not} correspond to the
                     19: ISO/CCITT terms, and so new terminology is introduced. 
                     20: Standard terminology is used in the standard way.
                     21: 
                     22: 
                     23: \section {DSA/DUA Interaction Model}
                     24: 
                     25: There are some interesting choices to be made between DSA Referral and
                     26: Chaining.  A DUA will start by contacting a local DSA, specifying that
                     27: chaining is preferred (i.e., DSA referrals should not be passed back to the
                     28: DUA).  After that, the first DSA will proceed by use of DSA Referral, except
                     29: for operations where this is not possible in the QUIPU framework (some cases
                     30: of search).  The advantages of this approach are:
                     31: 
                     32: \begin {itemize}
                     33: \item
                     34: The DUA code can be kept to a minimum, as there is no need to handle
                     35: referrals.
                     36: This does mean that the DUA must always interact with the Directory
                     37: Service through a DSA which supports chaining (which might exclude
                     38: some implementations).
                     39: \item
                     40: Always going thorough a local DSA allows a ``per system'' cache to
                     41: be maintained in a coherent manner.
                     42: \item
                     43: The overhead of maintaining chained connections is not passed
                     44: on too far.
                     45: \end {itemize}
                     46: 
                     47: Note that whilst the DUA procedural does not handle referrals transparently,
                     48: they are defined in the service interface, so that an application can
                     49: choose to handle the referrals directly if it wishes to do so.
                     50: 
                     51: \section {Model of Data Distribution}
                     52: \label {model-dist}
                     53: 
                     54: This is a critical section of the design.   It is essential to understand it
                     55: before studying distributed operation.
                     56: 
                     57: \subsection {Entry Data Blocks}
                     58: 
                     59: For the root and every non-leaf vertex, there will be an
                     60: {\em Entry Data Block} (EDB)
                     61: which contains
                     62: {\em all}
                     63: information pertaining
                     64: to the next level down in the DIT.
                     65: Figure~\ref{edbf} gives and ASN.1 description of
                     66: the Entry Data Block format.
                     67: 
                     68: \tagrind [htbp] {edb}{Entry Data Block Format}{edbf}
                     69: 
                     70: 
                     71: It should be noted and remembered that 
                     72: the Entry Data Block associated with an entry and described as ``the Entry
                     73: Data Block of the entry'' contains the entries children (i.e., their
                     74: attributes and RDNs)
                     75: and not the attributes of the entry itself.
                     76: This is {\em not} necessarily intuitive.
                     77: 
                     78: 
                     79: 
                     80: \subsection {Masters and Slaves}
                     81: 
                     82: Every Entry Data Block has {\em Master}  and {\em Slave} copies.
                     83: There will typically be only one master (although there may be
                     84: multiple master copies, where data is maintained ``out of band'').
                     85: Slave copies are automatically replicated from a Master EDB.
                     86: This may be direct or indirect.  The full propagation path must be acyclic
                     87: (loop free).
                     88: 
                     89: A DSA has either all or none of an Entry Data Block 
                     90: as a Master or Slave
                     91: (viz: Entry Data Blocks are atomic).
                     92: Any other DIT information it contains is treated as cached
                     93: data.
                     94: A DSA does not need to have any Master or Slave data.
                     95: 
                     96: 
                     97: DSAs are named, and represented in the DIT.
                     98: One of the reasons for this, is to enable use of the
                     99: Directory to identify the OSI location of DSAs.
                    100: This OSI location can then be adjusted transparent to the
                    101: logical mapping of the DIT onto DSAs.
                    102: This can be seen as treating a DSA in the same manner as any other
                    103: Application Entity.
                    104: This simplifies the implementation, as there does not need  to be
                    105: specific storage of additional configuration information (knowledge).
                    106: 
                    107: An important piece of information stored in the entry of each DSA is the
                    108: list of EDBs and how they are replicated.  This information is the basis for
                    109: automatic replication.   
                    110: 
                    111: \subsection {QUIPU Subordinate References}
                    112: 
                    113: An entry has associated with it, attributes which indicate  the
                    114: DSAs which have Master or Slave Entry Data Blocks for the entry
                    115: in question.
                    116: These pointers are known as {\em Quipu Subordinate References} (QSR).
                    117: For every QSR, the DSA must have a Master or Slave copy of the EDB, as
                    118: implied  by the QSR.
                    119: The converse it not true:  DSAs may have copies of EDBs without there being
                    120: QSRs.
                    121: The  DSAs with QSRs have the information {\em and} are
                    122: prepared to answer public queries about the Entry Data Block in
                    123: question.
                    124: DSAa with EDBs (typically slave copies) and no QSRs usually have copies for
                    125: performance or robustness reasons.  
                    126: 
                    127: Quipu Subordinate References are similar to the standardised Non-Specific
                    128: Subordinate References (NSSR).  There are the following differences:
                    129: 
                    130: \begin {itemize}
                    131: \item QSRs admit to replication, and therefore there are Master QSRs and
                    132: Slave QSRs.
                    133: 
                    134: \item A QSR always points to all of the relevant information, whereas an NSSR
                    135: may only point to a part of it and must be used in ``and'' conjunction with
                    136: other NSSRs.
                    137: \end {itemize}
                    138: 
                    139: 
                    140: \subsection {Access to the root EDB}
                    141: 
                    142: There is no requirement for a given DSA to hold a copy of the
                    143: root Entry Data Block.
                    144: However, to be able to systematically process all queries, there
                    145: must be direct or indirect access to the root Entry Data Block.
                    146: Therefore, every DSA which does not have a copy of the root
                    147: Entry Data Block must know the name and address of one or more
                    148: DSA which either has a copy of the root Entry Data Block, or is
                    149: closer (in terms of these references) to a DSA which has a
                    150: copy.  This approach is similar to, but not the same as, use of superior
                    151: references defined in the standard.
                    152: 
                    153: \section {Standard Knowledge References}
                    154: 
                    155: 
                    156: In addition, to the QUIPU specific QSRs, an entry might also contain
                    157: standard Knowledge References, as defined in the OSI Directory.  This is
                    158: used to point to data not contained in a QUIPU DSA.  There are three types
                    159: of reference defined in the standard:
                    160: 
                    161: \begin {description}
                    162: \item[Subordinate Reference]  Pointer to an Entry
                    163: 
                    164: \item[Cross Reference] From the QUIPU standpoint, the same as a subordinate
                    165: reference
                    166: 
                    167: \item[Non Specific Subordinate Reference] Pointer to multiple subordinates
                    168: which must be queried for the next level down.  QSRs are similar to this
                    169: (see previous section).
                    170: 
                    171: \end {description}
                    172: 
                    173: In the first two cases, the entry in the Entry Data
                    174: Block is considered to be ``Knowledge only'' (although other entry
                    175: information may be cached). 
                    176: In the third case the entry will also have full information on itself.  
                    177: If any of these are present, there will be no QUIPU master or slave pointers.
                    178: These three types of pointer are mutually exclusive\footnote{Thought(SEK) ---
                    179: does the standard let you have a subordinate reference plus a cached NSSR?}.
                    180: 
                    181: These attributes are not supported in QUIPU 5.0.
                    182: 
                    183: \section {Navigation}
                    184: 
                    185: Given this data model, a straightforward navigation algorithm
                    186: can now be specified.
                    187: The requirement is to locate the entry associated with a
                    188: specified Distinguished Name.
                    189: When the entry is arrived at, the operations will behave
                    190: as proscribed.
                    191: 
                    192: The basis of the navigation strategy is that the first DSA (i.e., the one
                    193: accessed by the DAP) does all of the hard work.  Other DSAs, accessed by DSA
                    194: Referral, either answer the question or return an error.  This is important,
                    195: as it is the basic strategy by which the system ensures completion of
                    196: queries.
                    197: 
                    198: First consider the behaviour of a DSA accessed by the Directory
                    199: System Protocol (DSP):
                    200: 
                    201: \begin {enumerate}
                    202: \item 
                    203: Look up the Distinguished Name.
                    204: \item 
                    205: If the Distinguished Name is found, go to step 6.
                    206: \item 
                    207: If there is a local copy of
                    208: the Entry Data Block of the parent,
                    209: return a nameError.
                    210: The ``matched'' parameter should be set to the Distinguished Name
                    211: of the Entry Data Block (i.e., the level above the offered name).
                    212: This is an authoritative NO.
                    213: \item 
                    214: Strip the lowest component off the Distinguished Name, and go
                    215: to step 1.
                    216: If there are no more components, go to step 5.
                    217: This process checks for authoritative NO.
                    218: \item 
                    219: At this point, the name has not been found, and no relevant
                    220: Entry Data Block has been found.
                    221: This implies that the DSA does not hold the root Entry Data
                    222: Block.
                    223: Therefore the DSA should return a DSA Referral.
                    224: The DSA Referral should be the list of DSAs (names and
                    225: addresses) which are known
                    226: to be closer to the root.
                    227: \item 
                    228: We now have an entry which matches some or all of the original
                    229: Distinguished Name.
                    230: Consider this entry.
                    231: \item 
                    232: If the entry contains an Alias attribute, dereference, and goto step 1.
                    233: Note that if a referral is returned, that the appropriate parameters should
                    234: be set to indicate all dereferences.
                    235: \item 
                    236: If the entry is the one specified in the query, return the
                    237: answer to the query, or the appropriate (authoritative) error.
                    238: \item 
                    239: If the entry is of object class ``QuipuNonLeafObject'', return a Referral.
                    240: This is simply a redirect to a DSA which can take the query at least one
                    241: step further.  The names for the DSA Referral should be taken from the
                    242: master and slave Quipu Subordinate References.  Where the calling DSA is a
                    243: non-QUIPU DSA, the Presentation address of the Master DSA must be looked up,
                    244: and only this one returned.
                    245: \item
                    246: If the entry contains a reference, the appropriate referral should be
                    247: returned.
                    248: \item 
                    249: The query refers to an entry below the bottom of the DIT.
                    250: An authoritative nameError can be returned.
                    251: \end {enumerate}
                    252: 
                    253: Now consider the slightly more complex case of the initial
                    254: DSA (doing the DSA Referral).
                    255: Steps 1-4 are followed as above
                    256: as above, to determine authoritative NO.
                    257: 
                    258: \begin {enumerate}
                    259: \setcounter{enumi}{4}
                    260: \item
                    261: At this point, the name has not been found, and no relevant
                    262: Entry Data Block has been found.
                    263: This implies that the DSA does not hold the root Entry Data
                    264: Block.
                    265: The list of DSAs which are known
                    266: to be closer to the root, are the starting point for the
                    267: iterative query.
                    268: Go to step \ref{step-iter}.
                    269: \item 
                    270: We now have an entry which matches some or all of the original
                    271: Distinguished Name.
                    272: Consider this entry.
                    273: \item 
                    274: If the entry contains an Alias attribute, apply the relevant
                    275: dereference to the original query, and go back to the start.
                    276: \item 
                    277: If the entry is the one specified in the query, return the
                    278: answer to the query, or the appropriate (authoritative) error.
                    279: \item 
                    280: If the entry is of object class ``QuipuNonLeafObject'', 
                    281: this gives a list of QSRs to start the iterative query.
                    282: Go to step \ref{step-iter}.
                    283: \item
                    284: If the entry contains a standard knowledge reference, then
                    285: go to step \ref{step-iter}.  Note that for non-specific subordinate
                    286: references, {\em all} of the references must
                    287: be followed before giving up.
                    288: \item 
                    289: The query refers to an entry below the bottom of the DIT.
                    290: An authoritative nameError can be returned,
                    291: \item 
                    292: \label{step-iter}
                    293: Select one of the DSAs from the referral list.
                    294: The order to try the DSAs is arbitrary.
                    295: However, it is attempted to  select ones with the
                    296: topologically closest name first (e.g., a UK DSA will prefer to
                    297: query another UK DSA before asking a French one).
                    298: Try DSAs in turn until one gives an answer or you get bored.
                    299: Consider the answer.
                    300: Authoritative answers (yes or no) should be passed back to the
                    301: DUA.
                    302: If a Referral is received, recurse to step, watching
                    303: carefully for loops.
                    304: \end {enumerate}
                    305: 
                    306: It can be seen that this navigation is relying on data being
                    307: distributed correctly, and DSAs other than the one doing the
                    308: work behaving in a correct manner.
                    309: Information provided in the referral should be used to ensure that the
                    310: iteration is progressing, and thus detect livelock situations.
                    311: 
                    312: 
                    313: 
                    314: \section {List}
                    315: 
                    316: The Entry Data Block concept allows the list operation to fall out in a
                    317: straightforward manner.
                    318: Navigation to the Entry Data Block belonging to the name provided, will
                    319: give access to the full result for the list operation.
                    320: 
                    321: Note that where cross/subordinate references are involved, it will be
                    322: assumed that these are not alias entries (reasonable in practice).
                    323: This will allow list to be performed in a single DSA in all cases.
                    324: 
                    325: \section {Search}
                    326: 
                    327: This section describes how the OSI Directory search is supported.  This is
                    328: one of the hardest parts of the implementation, and care must be taken.
                    329: Note that the DAP argument in DSP is always that provided by the
                    330: DUA\footnote {Whilst this is in principle one of the key aspects of the way
                    331: the DSP works, the recommendations for distributed operations violate this
                    332: principle when dealing with aliases during search.}.  This means that the
                    333: work done by a given DSA must be in relation to the target object.  With
                    334: other operations, this is (fairly) straightforward, as the target object
                    335: always references the base object of the operation.  For searching, care
                    336: must be taken to correctly verify whether the base object has been reached.
                    337: This is done by use of the operation progress information, setting the name
                    338: resolution phase to completed.
                    339: 
                    340: The search operation functions by searching the part of the tree implied
                    341: by the ``subset'' specification.
                    342: Rather than returning all of the information, the queried DSA will apply
                    343: the associated filter to the entries in question, and return the
                    344: filtered result, along with appropriate continuation pointers.
                    345: This should minimise network traffic.
                    346: 
                    347: 
                    348: The case of ``subset = baseObject'' is handled by navigating to the Entry
                    349: Data Block of the object's parent, and applying the given filter.
                    350: If the entry is a cross reference or subordinate reference, the reference
                    351: should be followed (using the same query).
                    352: 
                    353: The case of ``subset = oneLevel'' is handled by navigating to the object's
                    354: own Entry Data Block.
                    355: There the filter is applied to each of its children.
                    356: If any of the children are alias entries, the alias should be
                    357: de-referenced, and a baseObject search applied to the new entry.
                    358: For each child which is a cross references or subordinate references, 
                    359: the references should be followed, setting the target object to be the child.
                    360: 
                    361: The case of ``subset = wholeSubtree'' is handled by navigating to the
                    362: object's own Entry Data Block.
                    363: There, the filter is applied to the object and to each of its
                    364: children\footnote{QUIPU 5.0 gets this wrong, and does not apply the filter
                    365: to the base object.}.
                    366: If any of the children are alias entries, the alias should be
                    367: de-referenced, and a wholeSubtree search applied to the new entry.
                    368: For each child which has QUIPU children (determined
                    369: by the prescence of a masterDSA attribute), the search should be applied to
                    370: the master or one of the slave DSAs, with target object set to the child.
                    371: For each child which is a cross reference or subordinate reference, 
                    372: the references should be followed, setting the target object to the child.
                    373: For each child which has non-specific subordinate references, the search
                    374: should be applied to {\em all} of the referenced DSAs, with the target
                    375: object set to the child.
                    376: 
                    377: There are three procedures for operating:
                    378: 
                    379: \begin {enumerate}
                    380: \item Everything handled by the first DSA, with other DSAs returning a
                    381: mixture of results and partial outcome qualifiers.  This is not currently
                    382: supported due to some awkward implications of holding state, but will be the
                    383: default in QUIPU 6.0.
                    384: 
                    385: \item Proceed by referral until a DSA is reached which has a copy of the
                    386: base object.  Then this DSA proceeds by referral, and returns the full
                    387: result to the first DSA.  This is how QUIPU 5.0 works.  It has the advantage
                    388: of often accumulating search results in a local environment, and so will be
                    389: selectable in QUIPU 6.0 (possibly in a complex manner).
                    390: 
                    391: \item Proceed by referral until a DSA is reached which has a copy of the
                    392: base object.  Then proceed entirely by chaining.  This is not done.
                    393: 
                    394: \end {enumerate}
                    395: 
                    396: 
                    397: There is potential for looping in this procedure.
                    398: This will be detected and broken by noting loops in the DSA trace
                    399: information.
                    400: This takes account of the fact that some distribution will allow
                    401: for a query to re-enter the same DSA a number of times.
                    402: 
                    403: 
                    404: The Search and list operations make use of the ``partial results''
                    405: functionality to return information if a time or size limit is reached.
                    406: Thus, setting a low size limit will allow a user to easily examine 
                    407: sample information (either by list or search).
                    408: 
                    409: \section {Unavailable DSAs}
                    410: 
                    411: In the case where a DSA is unavailable, and chaining is preferred, a reference
                    412: will be returned by a QUIPU DSA.  
                    413: A QUIPU DUA, which knows it is talking to a QUIPU DSA
                    414: can rely on this behaviour, and simply use the referral as a diagnostic to
                    415: the user.  It is hoped that the next version of the standard will add an
                    416: obvious extra parameter.
                    417: 
                    418: 
                    419: 
                    420: \section {Presentation Addresses}
                    421: 
                    422: In terms of the directory, presentation addresses are handled according to
                    423: the standard.  Presentation addresses are stored in text form according to 
                    424: \cite {PSAP.String}.   The network encoding follows \cite {NSAP.Approach}.
                    425: 
                    426: \section {Operating When DSAs are not fully interconnected}
                    427: \label {tcp-only}
                    428: 
                    429: Whilst global interconnection of all application entities is an OSI ideal,
                    430: it will not be achievable in the short or medium term.  Application relaying
                    431: by DSAs will be needed to achieve full directory connectivity.  
                    432: 
                    433: 
                    434: In general, it is not desirable for DSAs to proceed by chaining - it wastes
                    435: unnecessary application level resources.  Later, there may be policy reasons
                    436: to prefer chaining, but these are ignored for now.  The internal structure
                    437: of network addresses allows a DSA to determine if two DSAs can communicate
                    438: directly.  For QUIPU 5.0, referrals will be used if the calling and
                    439: referenced DSA can communicate directly and chaining otherwise.  For QUIPU
                    440: 6.0, authorisation will be required for chaining to take place.
                    441: 
                    442: 
                    443: \section {The External view of QUIPU}
                    444: 
                    445: To a non-QUIPU system, QUIPU will appear to work exactly according to the
                    446: standard.  This is not quite true for QUIPU 5.0, but is sufficiently close to
                    447: allow a high level of interworking with a non-QUIPU system.  This is an
                    448: optimistic statement, not written in the light of experience with
                    449: interoperability testing.
                    450: 
                    451: When a QUIPU DSA interacts with another DSA, it will look up its object
                    452: classes (and probably other information).  This will allow it to determine if
                    453: the other DSA is a QUIPU DSA.  When interacting with another QUIPU DSA, the
                    454: following deviations from the standard are possible.  These are primarily
                    455: concerned with the introduction of replication:
                    456: 
                    457: \begin {itemize}
                    458: \item Presentation Address might be omitted from Access Point (always
                    459: present in QUIPU 5.0, for consideration in QUIPU 6.0).
                    460: \item Cross References and Subordinate References have multiple values
                    461: (although QUIPU will probably never send these to itself).
                    462: \item Multiple values of Non Specific Subordinate Reference are assumed to
                    463: have OR conjunction (i.e., they are really QSRs).
                    464: \item Use of QUIPU Access control as describe in Section~\ref{dsp-acl}
                    465: \end {itemize}
                    466: 
                    467: When a QUIPU DSA returns references which are derived from reference
                    468: attributes, it will return them as specified.  If it returns information
                    469: derived from QUIPU internal pointers, it will return a non-specific
                    470: subordinate reference.  If the DSA being communicated with is not a QUIPU
                    471: DSA, it will return only a reference to the master DSA (as replication is
                    472: not admitted within the protocol).
                    473: 
                    474: \section {Use of ACLs in DSP}
                    475: \label{dsp-acl}
                    476: 
                    477: Within the DSP, between a pair of QUIPU DSAs, the ACL
                    478: attribute becomes special.  It is used as a roadmap of the entry for
                    479: a DSA which is caching information.  For this reason, the ACL is
                    480: always asked for in read operations invoked by DSP --- irrespective
                    481: of whether the corresponding DAP operation asked for it.  The ACL
                    482: will always be returned to the DSA, even if the DUA in question does
                    483: not have rights to it.  This will allow the DSA to perform caching
                    484: ``correctly''.  When an ACL is passed in the DSP, it will be encoded
                    485: so that ALL of the attributes of the entry are explicitly referred
                    486: to.  Thus, the caching DSA will be able to determine whether or not
                    487: it has all the attributes of a given entry.  This should be a useful
                    488: performance gain.
                    489: 
                    490: This will be added in QUIPU 6.0.
                    491: 
                    492: \section {Access Control and Authentication}
                    493: 
                    494: All operations must take account of access control prior to being performed.
                    495: Authentication must be done at BIND time, as this is the only point at which
                    496: the DSA can return an error to the DUA which indicates authentication
                    497: failure.
                    498: 
                    499: If there is public right on the information, then the effort of doing the
                    500: authentication was wasted.
                    501: Therefore, if a user is accessing information {\em known} to be publicly
                    502: readable, it will be optimal {\em not} to supply credentials.
                    503: 
                    504: For policy reasons, QUIPU 6.0 may make specification of DUA and the
                    505: associated simple authentication mandatory.  
                    506: 
                    507: \section {Cached Data}
                    508: \label {cache-all}
                    509: 
                    510: Cached data is not mentioned in the basic algorithm.
                    511: However, the algorithm can utilise cached
                    512: data in some circumstances.
                    513: This is because of the manner in which identification of copy data has been
                    514: introduced in the final stage of the OSI Directory specification.
                    515: Cached data may be used whenever this is not prohibited by the service
                    516: controls.
                    517: The standard does not clearly define what ``copy'' data is.  In general,
                    518: QUIPU treats master and slave data as authoritative.
                    519: Both slave and cached data are returned to the user as
                    520: ``copy'' data.  For this reason, the distinction between slave and copy data
                    521: can only be internal to QUIPU.
                    522: 
                    523: There is no time to live or age information in the OSI Directory Protocols.
                    524: Care must be taken when caching, that spurious information is not passed
                    525: around indefinitely between DSAs
                    526: 
                    527: When QUIPU holds cached data, it will note:
                    528: 
                    529: \begin {itemize}
                    530: \item How long it has had it
                    531: \item If it came from a master, slave, or cache.  
                    532: \item If it is complete (e.g., are all values of an attribute present)
                    533: \end {itemize}
                    534: 
                    535: This section is open ended.
                    536: The exact approaches to caching, and determining suitable timeout values
                    537: will be the subject of experiment.
                    538: 
                    539: QUIPU 5.0 supports the caching techniques described in section
                    540: \ref{cache-all}, except:
                    541: 
                    542: \begin {itemize}
                    543: \item Holding age information
                    544: \item Storage on disk
                    545: \item Timeouts 
                    546: \item Negative caching
                    547: \end {itemize}
                    548: 
                    549: 
                    550: \subsection {Caching Configuration Data}
                    551: 
                    552: 
                    553: When the DSA is using cached data (e.g., the Presentation Address of another
                    554: DSA), it should be aware of this fact, and should update the information
                    555: (from non-cache information) if it fails to establish an OSI connection,
                    556: or some inconsistency appears to occur in the data.  If there is some doubt
                    557: (e.g., if the error might be due to the DSA being unavailable), the age of
                    558: the cache should be take into account when determining whether or not to do
                    559: a refresh.
                    560: 
                    561: The important thing about managing cached data is to handle timeouts
                    562: sensibly.
                    563: Data cached from a cache may have an indeterminate age.
                    564: It is important that this data is given a relatively short timeout, to
                    565: prevent it being circulated indefinitely amongst a set of DSAs.
                    566: However, {\em if} the data can be verified by usage (e.g., correctly 
                    567: connecting to a DSA verifies a presentation address), it should then be
                    568: treated as if cached from a master/slave.  
                    569: Non-verified data should be treated in the same manner as user data, which
                    570: is described in the next section.
                    571: Data cached from master/slave information should be given a longer timeout.
                    572: Data is discarded, rather than refreshed automatically.  A timeout of some
                    573: number of days seems appropriate for most data.   
                    574: 
                    575: A special case is made for comparing passwords, because it is a user
                    576: expectation to use the new password immediately.  For this reason, if a
                    577: password compare fails, a check should be made against the master copy.
                    578: Note that old passwords will sometimes be valid for a
                    579: short while beyond their change.  
                    580: 
                    581: \subsection {Caching User Data}
                    582: 
                    583: User data is cached in a manner similar to configuration data.  Data cached
                    584: from a cache will only be held for a short period of time (perhaps a few
                    585: minutes).  This time should be short enough to ensure that spurious cached
                    586: information will die out, but long enough to give the necessary performance
                    587: improvements for repeated queries.   
                    588: 
                    589: Other data may be cached for longer.  If ever a query is made asking for
                    590: authoritative data, any cached values should be flushed immediately ---
                    591: irrespective of whether the query succeeds.  A query for authoritative data
                    592: is a strong hint about the inaccuracy of cached data, or that a recent
                    593: change might have occurred.
                    594: 
                    595: \subsection {Negative Data}
                    596: 
                    597: Usage of directories has shown that incorrect queries are often repeated,
                    598: particularly when the user is a mail system.  For this reason, negative
                    599: answers will be cached for a short period of time (QUIPU 6.0).
                    600: 
                    601: \subsection {Writing Caches to disk}
                    602: \label {disk-cache}
                    603: 
                    604: This is beyond the scope of QUIPU 5.0, but essential for QUIPU 6.0 for
                    605: improved performance. 
                    606: 
                    607: The astute will notice that without any cached information, DSA startup for
                    608: DSAs which do not have copies of a few relevant EDBs, will be an expensive
                    609: operation.  
                    610: Data cached from a cache will never be saved on disk.  
                    611: 
                    612: Disk caching is important to ensure that this overhead is only be incurred
                    613: in practice, on the occasion of the initial startup.  This means that QUIPU
                    614: 5.0 DSAs (without disk caching) should always be given copies of at least
                    615: their parent EDB.
                    616: 
                    617: \section {Configuration and Slave Update}
                    618: 
                    619: A given DSA will have copies of zero or more Entry Data Blocks.
                    620: A DSA may either be a master for a given Entry Data Block, or a
                    621: slave.
                    622: If there are multiple master copies, it is assumed that these
                    623: are kept coherent by some out of band mechanism.
                    624: For example, one of them is the ``real'' master, and the others
                    625: are updated by file transfer when modifications occur.
                    626: This discussion will proceed for the single master case.
                    627: 
                    628: There are three distinct types of DSA:
                    629: 
                    630: \begin {enumerate}
                    631: \item 
                    632: A DSA with a master copy of the root Entry Data Block.
                    633: \item 
                    634: A DSA with a slave copy of the root Entry Data Block.
                    635: \item 
                    636: A DSA with no copy of the root Entry Data Block.
                    637: \end {enumerate}
                    638: 
                    639: As noted in Section~\ref{model-dist}, DSAs of type 2 and 3 will have pointer(s) to
                    640: a DSA which is ``closer'' to the master copy of the root Entry
                    641: Data Block.
                    642: Specifying this hierarchical distribution, as opposed to requiring
                    643: direct access to the master (as in earlier versions of the OSI Directory)
                    644: means that there can be many copies of information which needs to be highly
                    645: replicated, without excessive
                    646: redundant copying across the Wide Area Network.   
                    647: This will be particularly important for the root Entry Data Block.
                    648: 
                    649: DSAs of type 2 will only need the pointer information for initial
                    650: startup or recovery after catastrophic corruption.
                    651: When the slave copy of the root Entry Data Block has been
                    652: obtained for the first time, this will supercede the pointers.
                    653: DSAs of type 3 will usually use cached information in preference to
                    654: these pointers, and will only need the pointers if cached information is
                    655: (or appears to be) invalid.
                    656: 
                    657: The only information which a DSA has to obtain locally at initial boot time,
                    658: other than the DSA pointers, is its own name.  All other information may be
                    659: obtained from the Directory.
                    660: Beyond this, the Directory Service manages its own
                    661: configuration.
                    662: There is little point in having a Directory Service providing
                    663: general high speed access to global information, and then
                    664: requiring an additional system (knowledge) to deal with its own
                    665: configuration.
                    666: 
                    667: Associated with the DSA's entry in the DIT is a specification of
                    668: the entries for which the DSA is a master, and for which it is
                    669: a slave.
                    670: A DSA will be able to derive the location of an Entry Data
                    671: Block for which it is master from this information.
                    672: Thus at initial boot, a DSA will utilise its initial DSA pointers
                    673: to read its own entry.
                    674: The location of master Entry Data Blocks will be derivable from
                    675: their name, and so their existence can then be verified by the
                    676: DSA in question.
                    677: A DSA which is a master for the root Entry Data Block will have
                    678: no pointers.
                    679: However, it can go straight to the master root Entry Data
                    680: Block, read the information about itself, and proceed as for
                    681: other DSAs.
                    682: 
                    683: 
                    684: It is believed that for early pilots, a high level of copying configuration
                    685: data will be desirable to achieve robustness.  The root and national EDBs
                    686: will be very highly replicated, even though QUIPU can operate with a rather
                    687: low level of replication.
                    688: 
                    689: \section {DSA Naming}
                    690: \label {dsa-naming}
                    691: 
                    692: \subsection {Choice of Names to prevent loops}
                    693: 
                    694: Care must be taken to prevent the situation where the location
                    695: of a DSA is only known through itself (and other more complex
                    696: variants).
                    697: A simple rule for naming DSAs will ensure that this cannot
                    698: happen.
                    699: The master DSA for a given entry (i.e., the DSA controlling the Entry Data Block of
                    700: containing 
                    701: the entry's children) should have its
                    702: name in the Entry Data Block of the entry's parent or at a
                    703: level higher in the DIT.
                    704: For example, the master DSA of 
                    705: \begin{quote}\small\begin{verbatim}
                    706: Country=UK, Org=University College London, OU=Computer Science
                    707: \end{verbatim}\end{quote}
                    708: which contains information on entries below Computer Science, may be labelled
                    709: \begin{quote}\small\begin{verbatim}
                    710: Country=UK, Org=University College London, DSA=Three Toed Sloth
                    711: \end{verbatim}\end{quote}
                    712: or
                    713: \begin{quote}\small\begin{verbatim}
                    714: Country=France, DSA=Capybara
                    715: \end{verbatim}\end{quote}
                    716: It may not be labelled
                    717: \begin{quote}\small\begin{verbatim}
                    718: Country=UK, Org=University College London, OU=Computer Science, 
                    719:             DSA=Alpaca
                    720: \end{verbatim}\end{quote}
                    721: or
                    722: \begin{quote}\small\begin{verbatim}
                    723: Country=France, Org=Inria, DSA=Llama
                    724: \end{verbatim}\end{quote}
                    725: 
                    726: 
                    727: A little more flexibility could be allowed;
                    728: However, this rule is simple, it prevents deadlock, and allows
                    729: for reasonable labelling practices.
                    730: The restriction may be relaxed somewhat, when the concept of Directory
                    731: Management Domains is introduced more formally.
                    732: 
                    733: \section {Local DSA Information}
                    734: 
                    735: 
                    736: The basic entry for the DSA contains information which must be widely known.
                    737: Essentially this is the OSI location of the DSA.  There is other information
                    738: which it is useful to store in the Directory, but which is primarily needed
                    739: by the DSA itself.  Because of the naming rules in the previous section, the
                    740: DSA's entry will usually be mastered in a different DSA.  Therefore, we have
                    741: a second entry for each DSA, which is the child of the prime DSA Entry.
                    742: This entry will always have common name ``Info'', and be mastered in the DSA
                    743: itself.  There will usually not be slave copies.  The DSA Address is also
                    744: stored in this entry.  This duplication (and therefore extra management) is
                    745: seen as worthwhile, as it allows a DSA to start without access to any other
                    746: DSA.  This information may be absent (usually only when a DSA is being
                    747: created).
                    748: 
                    749: Initially, the following information will be stored in this entry:
                    750: 
                    751: \begin {itemize}
                    752: \item The EDB management information
                    753: \item DSA Control (see Section~\ref{dsa-control})
                    754: \item Software Version --- this attribute is automatically maintained
                    755: by the DSA.
                    756: \end {itemize}
                    757: 
                    758: For QUIPU 5.0, these entries are supported, but are in the main DSA entry.
                    759: 
                    760: The following items will be added for QUIPU 6.0, to allow for more
                    761: sophisticated remote
                    762: management:
                    763: 
                    764: \begin {itemize}
                    765: \item Authorisation policy
                    766: \item Tailoring
                    767: \item The schema files (OID, Attribute, Object Class)
                    768: \end {itemize}
                    769: 
                    770: Replication will also allow common management of multiple DSAs (e.g., to share
                    771: a common schema).
                    772: 
                    773: 
                    774: \section {DSA Naming Architecture }
                    775: 
                    776: The following Naming Architecture components are needed in order to support
                    777: the QUIPU Mechanism for distributed operations.
                    778: These are defined here, as a convenient location.
                    779: Numbers are assigned in the QUIPU Naming Architecture contained in Volume~5
                    780: of the User's
                    781: Manual \cite{QUIPU.Manual}.
                    782: 
                    783: \tagrindfile {na}{Naming Architecture}{na}
                    784: 

unix.superglobalmegacorp.com

This archive runs on limited infrastructure. Preserving old code on modern bandwidth. Automated agents are requested to crawl responsibly.