Return to management.tex CVS log

Up to [CSRG BSD Unix] / 43BSDReno / contrib / isode-beta / doc / quipu

Annotation of 43BSDReno/contrib/isode-beta/doc/quipu/management.tex, revision 1.1

1.1     ! root        1: 
        !             2: \chapter {Management}
        !             3: \section {Introduction}
        !             4: 
        !             5: Perhaps the most significant change between QUIPU 5.0 and QUIPU 6.0 will be
        !             6: the addition of the management facilities described in this chapter.  Whilst
        !             7: most of the things specified here can be achieved in QUIPU 5.0, much needs to
        !             8: be done by direct editing of the EDB files.  It is expected to be able to
        !             9: move to a very high level of remote management in the next version.  Except
        !            10: where stated, this chapter applies to QUIPU 6.0.
        !            11: 
        !            12: Most of the focus in this section is in management of the directory itself.
        !            13: However, there is also discussion of some issues relating to management of
        !            14: user data.
        !            15: 
        !            16: 
        !            17: \section {DUA Configuration}
        !            18: 
        !            19: \label {dua-mgt}
        !            20: 
        !            21: Most of the issues of DUA management are those of standard software
        !            22: installation, and beyond the scope of this document.  The use of a Directory
        !            23: for software management might be an interesting research issue.   
        !            24: We assume conventional installation practices.  
        !            25: 
        !            26: In this section, we consider a DUA to be a system accessing the directory
        !            27: using the DAP, which may or may not have a name registered in the directory.
        !            28: The only information which a DUA needs is a list of DSAs to contact, in
        !            29: order to ``get into the system''.  This can be done by use of a system wide
        !            30: file, which may be overridden by the DUA.  This file must be updated
        !            31: manually at present.
        !            32: 
        !            33: A problem with this is that DSAs move, and there is no way to keep the DUA
        !            34: configuration up to date.  It is assumed that each host supporting DUAs has
        !            35: the concept of ``site'', which is a name (e.g.,
        !            36: ``@Country=GB@Organisation=University College London''), and that each host
        !            37: has a local name (e.g., ``pyr1.cs.ucl.ac.uk'').  This is mapped onto an
        !            38: entry in the directory with an attribute in the entry, giving a list of appropriate DSAs (typically 2-3, to
        !            39: give redundancy).  A process is provided which runs on the machine and
        !            40: updates the default address list (file) at intervals.  By automatically
        !            41: installing defaults which point to ``well known'' DSAs, the initial manual
        !            42: step can be avoided.  This leads to the following DUA installation procedure:
        !            43: 
        !            44: \begin {enumerate}
        !            45: \item Bootstrap  by use of default ``well known'' DSAs
        !            46: \item Define Default Site 
        !            47: \item Register Site and Machines in Directory, including list of default
        !            48: DSAs.   This may well require installation of local DSAs as a pre-requisite.
        !            49: \item Run DUA default update program at intervals
        !            50: \end {enumerate}
        !            51: 
        !            52: 
        !            53: DUAs overriding the host defaults should take analogous precautions to
        !            54: ensure information is up to date.  Explicit storing of presentation
        !            55: addresses, other than in a cache, is deprecated.
        !            56: 
        !            57: \section {DSA Management}
        !            58: 
        !            59: The basic level of management is of a single DSA.  This section discussed
        !            60: procedures for dealing with a single DSA.  Installation of a single DSA
        !            61: affects multiple DSAs, and is discussed in Section~\ref{add-dsa}.
        !            62: 
        !            63: \subsection {DSA Static Configuration}
        !            64: 
        !            65: There are tools to view the the configuration of a single DSA.  These are
        !            66: provided by DISH scripts (QUIPU 6.0).
        !            67: Dish is described in the QUIPU Manual \cite{QUIPU.Manual}.
        !            68: First there is an overall summary, which includes.
        !            69: 
        !            70: \begin {itemize}
        !            71: \item Software version
        !            72: \item List of EDBs held 
        !            73: \item Address
        !            74: \end {itemize}
        !            75: 
        !            76: Then for each EDB, the following information may be obtained:
        !            77: 
        !            78: \begin {itemize}
        !            79: \item Version number
        !            80: \item Master / Slave / Cache
        !            81: \item Number of Entries
        !            82: \item Volume of Data
        !            83: \item All RDNs 
        !            84: \item All data
        !            85: \end {itemize}
        !            86: 
        !            87: There will be a tool to list the tree (name only) for all the information
        !            88: held in a DSA.   
        !            89: 
        !            90: \subsection {DSA Dynamic Configuration}
        !            91: 
        !            92: There is a need to determine the operational state of a given DSA.  This
        !            93: will be done either by use of a QUIPU specific operation or by reading
        !            94: pseudo-attributes.  It will include information on:
        !            95: 
        !            96: \begin {itemize}
        !            97: \item  Listens available
        !            98: \item  Calls open
        !            99: \item  Queries in progress
        !           100: \end {itemize}
        !           101: 
        !           102: To be defined.  
        !           103: 
        !           104: \subsection {DSA Control}
        !           105: \label {dsa-control}
        !           106: 
        !           107: There is a need to control the operation of a DSA.  Rather than invent
        !           108: protocol, which would restrict operations to QUIPU DUAs, we utilise a special
        !           109: attribute stored in the DSAs own EDB (``DSAControl'').  This (virtual) attribute is only
        !           110: writable by the DSA Manager, and is not readable.   Values written into it
        !           111: control operation of the DSA.  These are described fully in the QUIPU Manual.
        !           112: The operations include:
        !           113: 
        !           114: \begin {itemize}
        !           115: \item Lock the DSA (to prevent updates).
        !           116: \item Alter the logging level (globally, or on a the current association).
        !           117: This is primarily for debugging.
        !           118: \item Reread EDBs from disk.
        !           119: \item Refresh: reread DSA parameters, and adapt to any changes.  This operation is
        !           120: used when changing the Directory Configuration.
        !           121: \item Stop DSA (cause process to exit)
        !           122: \end {itemize}
        !           123: 
        !           124: 
        !           125: QUIPU 6.0 will add the functionality to control specific EDBs stored in a
        !           126: DSA.
        !           127: 
        !           128: 
        !           129: \subsection {Consistency Checks}
        !           130: 
        !           131: The following consistency checks should be carried out by a DSA.
        !           132: Some are applied at DSA startup.  Others are costly, and so should be
        !           133: applied at intervals using special tools.
        !           134: The following checks should be made:
        !           135: 
        !           136: \begin {enumerate}
        !           137: \item Spurious (Unix) directories in the database
        !           138: \item Extra attribute files
        !           139: 
        !           140: \item For all master non-leaf entries:
        !           141: \begin {itemize}
        !           142: \item
        !           143: Check that the master and slave DSAs
        !           144: referenced (EDB Info) have copies of the implied EDB.
        !           145: \item Check that referenced DSAs are accessible (directly or indirectly).
        !           146: This might be expensive, and so not be undertaken too often.
        !           147: \end {itemize}
        !           148: 
        !           149: 
        !           150: \item For the DSAs own EDB Info (master and slave), as determined from the
        !           151: DSAs own entry:
        !           152: \begin {itemize}
        !           153: \item
        !           154: check that the EDB files
        !           155: are present (should happen at boot time)
        !           156: \item Check that from and to DSAs and correctly listed (they exist, have the
        !           157: implied EDB, and the distance is acceptable)
        !           158: \end {itemize}
        !           159: 
        !           160: Note that the introduction of replication by subtree may well have an effect
        !           161: on this.
        !           162: 
        !           163: \item Check that all file attributes referenced in each EDB  are present
        !           164: 
        !           165: 
        !           166: \end {enumerate}
        !           167: 
        !           168: \section {Configuration Inspection}
        !           169: 
        !           170: It will be important to have tools to enable viewing of the overall
        !           171: Directory Configuration.  These will be provided by the use of DISH scripts.
        !           172: The basic function should operate at an EDB level, and for an EDB should be
        !           173: able to provide:
        !           174: 
        !           175: \begin {itemize}
        !           176: \item Master DSA
        !           177: \item Registered Slave DSAs
        !           178: \item Unregistered Slave DSAs
        !           179: \item Propagation tree 
        !           180: \item References to non-QUIPU DSAs
        !           181: \item Spot shadow information 
        !           182: \item List of child EDBs (a basis for recursive operation)
        !           183: \end {itemize}
        !           184: 
        !           185: This is non-trivial.
        !           186: 
        !           187: 
        !           188: \section {Configuration Changes}
        !           189: 
        !           190: \subsection {General Approach}
        !           191: 
        !           192: This section describes how the directory is manipulated to give a coherent
        !           193: configuration.   There are some general principles:
        !           194: 
        !           195: \begin {itemize}
        !           196: \item  As much as possible is done by use of standard directory operations.
        !           197: \item These operations are for DSA managers, and not naiive users.  Access
        !           198: Control needs to be set up appropriately.
        !           199: \end {itemize}
        !           200: 
        !           201: Many of these operations will be provided by use of DISH scripts, giving
        !           202: manual directions where external action must be taken (e.g., loading
        !           203: software).   
        !           204: 
        !           205: Operations will often affect multiple DSAs.  The tools will be designed so
        !           206: that if later operations fail, earlier ones will be unrolled.  There is
        !           207: clearly some potential for getting the overall system into an incoherent
        !           208: state.  At this point the tool will complain loudly, and expect intelligent
        !           209: intervention.   
        !           210: 
        !           211: \subsection {Add DSA}
        !           212: \label {add-dsa}
        !           213: 
        !           214: Adding a new DSA proceeds in the following manner:
        !           215: 
        !           216: \begin {enumerate}
        !           217: \item Install software at OSI location.  No tailoring should be needed,
        !           218: other than that in the software configuration.
        !           219: Installation will include:
        !           220: \begin {itemize}
        !           221: \item All of the relevant DSA binaries 
        !           222: \item A default DSA tailor file, which will include pointers to ``well
        !           223: known'' DSAs.
        !           224: \item A base directory for the database, owned by an appropriate uid.
        !           225: \end {itemize}
        !           226: 
        !           227: \item Add DSA entry to the global directory (using a DUA pointed at an
        !           228: existing DSA).
        !           229: \label{global-step}
        !           230: 
        !           231: \item Boot new DSA.
        !           232: \item Add local DSA information (over DAP).
        !           233: \item Refresh DSA (see Section~\ref{dsa-control}).
        !           234: \end {enumerate}
        !           235: 
        !           236: For testing purposes, step \ref{global-step} may be replaced by adding an
        !           237: EDB which gives the DSA a dummy name stored locally.  This will enable a DSA
        !           238: to be tested in isolation from the rest of the directory.
        !           239: 
        !           240: \subsection {Remove DSA}
        !           241: 
        !           242: \begin {enumerate}
        !           243: \item Needs name of DSA
        !           244: \item Ensure no EDBs held by DSA
        !           245: \item Turn off DSA
        !           246: \item Remove DSA entry
        !           247: \item Remove software
        !           248: \end {enumerate}
        !           249: 
        !           250: 
        !           251: \subsection {Modify DSA info}
        !           252: 
        !           253: Most DSA information can be changed as an aspect of local configuration.
        !           254: The name of a DSA cannot be changed externally.  It is necessary to remove
        !           255: one DSA and then recreate a new one (or vice versa).  In practice, the tools
        !           256: for DSA creation and EDB manipulation will make this straightforward.  
        !           257: 
        !           258: Moving the OSI location of a DSA is straightforward, as the naming rules will
        !           259: ensure that this propagates without deadlock.  DUAs will adapt to the change
        !           260: by use of backup DSAs (which must not be moved at the same time) and the
        !           261: techniques described in Section~\ref{dua-mgt}.
        !           262: This will cause some level of disruption, and so OSI location changes should be
        !           263: avoided as far as possible.
        !           264: 
        !           265: Note that the presentation address of a DSA is stored in two entries, and
        !           266: that both must be modified.
        !           267: 
        !           268: \subsection {Add new EDB}
        !           269: 
        !           270: AddEntry will only work if the appropriate EDB is in place (although subtree
        !           271: propagation might enable a sensible defaulting mechanism to be achieved).   
        !           272: AddEntry should not create EDBs, as this affects multiple DSAs, and a failure
        !           273: could leave the directory in an inconsistent state.  This also gives
        !           274: management control over the addition of new layers into the DIT.
        !           275: The following steps are followed by the EDB creation tool.
        !           276: 
        !           277: \begin {enumerate}
        !           278: \item Need to specify Name of EDB and Master DSA
        !           279: \item Check the EDB of parent exists
        !           280: \item Add EDB info to DSA entry of the master DSA
        !           281: \item Add master pointer in parent Entry
        !           282: \item Refresh master DSA, which will cause EDB file to be created
        !           283: \end {enumerate}
        !           284: 
        !           285: 
        !           286: \subsection {Add EDB copy}
        !           287: 
        !           288: Usually, an EDB will be given a number of slave copies.
        !           289: 
        !           290: \begin {enumerate}
        !           291: \item Needs name of EDB, slave DSA, update mechanism, and whether EDB will be visible
        !           292: \item Add EDB info to slave DSA
        !           293: \item Modify EDB Info of DSA sending update (often, but not necessarily the
        !           294: master)
        !           295: \item Refresh slave DSA and DSA sending update
        !           296: \item Optionally add slave pointer to entry.  Every EDB will be given a
        !           297: small number of ``visible'' copies.  EDBs high in the tree will often have
        !           298: many private copies.   
        !           299: 
        !           300: \end {enumerate}
        !           301: 
        !           302: \subsection {Remove EDB copy}
        !           303: 
        !           304: An EDB copy is removed as follows:
        !           305: 
        !           306: \begin {enumerate}
        !           307: \item Needs name of EDB and DSA name
        !           308: \item Ensure no copies being sent from this DSA
        !           309: \item Modify EDB Info of sending DSA
        !           310: \item Modify pointer in Entry (if needed)
        !           311: \item Remove EDB Info from DSA
        !           312: \item Refresh DSA
        !           313: 
        !           314: \end {enumerate}
        !           315: 
        !           316: \subsection {Remove EDB}
        !           317: 
        !           318: After removing all copies, a master EDB is removed as follows:
        !           319: 
        !           320: \begin {enumerate}
        !           321: \item Ensure no copies being sent from the DSA
        !           322: \item Ensure that EDB has no remaining entries (might optionally
        !           323: remove all such entries)
        !           324: \item Remove pointer in entry
        !           325: \item Remove EDB from DSA
        !           326: \item Get DSA to refresh (or wait for reboot)
        !           327: \end {enumerate}
        !           328: 
        !           329: \subsection {Change EDB Master}
        !           330: 
        !           331: It is sometimes necessary to change a master or EDB propagation path, and
        !           332: their should be tools to facilitate this.
        !           333: 
        !           334: The propagation path can be viewed as a tree.  The basic tool acts like the
        !           335: unix mv command.  It works as follows:
        !           336: 
        !           337: \begin {enumerate}
        !           338: \item Needs the EDB/DSA pair being moved, and name of new feed DSA
        !           339: \item If incremental update is used, remove item from old feed DSA
        !           340: \item Optionally remove DSA from getEDB list of old feed DSA
        !           341: \item Change source in DSA
        !           342: \item If incremental update is used, add item to new feed DSA
        !           343: \item Optionally add DSA to getEDB list of new feed DSA
        !           344: \item Optionally modify the associated entry list of slaves
        !           345: \item Extra twiddles if moving a parent to a child
        !           346: \end {enumerate}
        !           347: 
        !           348: Changing the master is a special case of this.  In addition:
        !           349: \begin {itemize}
        !           350: \item The EDB entry must be updated to reflect change of master
        !           351: \item The new master must not
        !           352: \end {itemize}
        !           353: 
        !           354: 
        !           355: \subsection {References}
        !           356: 
        !           357: Facilities will be provided to establish, update, and remove references to
        !           358: non-QUIPU DSAs.
        !           359: 
        !           360: \subsection {Spot Shadows}
        !           361: 
        !           362: Facilities will be provided to establish, update, and remove spot shadows.
        !           363: 
        !           364: \section {Bootstrap}
        !           365: 
        !           366: The only thing {\em not} automated is installation of the software, and
        !           367: setup of tsapd info (for non-QUIPU OSI software).  The former might be
        !           368: tractable by use of QUIPU, but is not considered here.  
        !           369: 
        !           370: For the latter, tsapd might contact the Directory dynamically (probably at
        !           371: start time) or a special tool might down load the isoservices file from the
        !           372: directory.   It is almost certainly worth doing one of these.
        !           373: 
        !           374: \section {Authorisation}
        !           375: 
        !           376: QUIPU 6.0 will tackle authorisation of access.  The following will be
        !           377: considered:
        !           378: 
        !           379: \begin {enumerate}
        !           380: \item Who can connect
        !           381: \begin {itemize}
        !           382: \item DSA and DUA rights
        !           383: \item Specification by entry/subtree/filter
        !           384: \end {itemize}
        !           385: 
        !           386: \item Which networks the authorised entity is allowed access to
        !           387: 
        !           388: \item Level of access (authentication policy variations)
        !           389: 
        !           390: \end {enumerate}
        !           391: 
        !           392: 
        !           393: \section {User Data Management}
        !           394: 
        !           395: QUIPU 6.0 will provide additional facilities for checking data stored in the
        !           396: directory.
        !           397: 
        !           398: \subsection {Aliases}
        !           399: 
        !           400: A special interface is provided for managing aliases (built on DISH).
        !           401: This has two functions, achieved as described.
        !           402: 
        !           403: \begin {itemize}
        !           404: \item 
        !           405: 
        !           406: Add alias:
        !           407: \begin {enumerate}
        !           408: \item Check if object pointed to exists and is not an alias
        !           409: \item Add the alias
        !           410: \end {enumerate}
        !           411: 
        !           412: Delete alias:
        !           413: \begin {enumerate}
        !           414: \item Simply remove the entry.
        !           415: \end {enumerate}
        !           416: 
        !           417: \end {itemize}
        !           418: 
        !           419: 
        !           420: 
        !           421: \subsection {Externally mastered data}
        !           422: 
        !           423: It is important to recognise that in most early uses of the the directory,
        !           424: much data will be downloaded from external sources of data.  This needs to
        !           425: deal with:
        !           426: 
        !           427: \begin {itemize}
        !           428: \item There will be multiple external sources, and user modification of data
        !           429: in the {\em same} entry
        !           430: \item Different sources will have different names for the same object
        !           431: \item There must be a mechanism to delete old entries.  This needs to cope
        !           432: with multiple sources.
        !           433: \end {itemize}
        !           434: 
        !           435: The approach taken here, is intended to overcome some of the deficiencies
        !           436: found with earlier bulk loading techniques.
        !           437: 
        !           438: 
        !           439: A basic rule is: Each source of data views a single object to have one
        !           440: unique name;  This unique name is stored as a value of an attribute in an
        !           441: entry in the context of a Distinguished Name (e.g., ``Steve Kille'' in the
        !           442: context of ``@Country=GB@Organisation=University College London='').
        !           443: 
        !           444: This rule will facilitate unique merging from multiple sources.  Tools (DISH
        !           445: scripts) will be provided to facilitate this merging.  It is assumed that
        !           446: such scripts will be adapted to local data.  
        !           447: 
        !           448: The following two tools are expected:
        !           449: 
        !           450: \begin {enumerate}
        !           451: \item Given a list of values relative to a DN:
        !           452: \begin {itemize}
        !           453: \item If there is is an exact match to a single entry, do nothing.
        !           454: \item If there is a unique approximate match, add in the value given as an
        !           455: alternate value (typically of the common name).  Log this match (for 
        !           456: possible human verification).
        !           457: \item If there is no approximate match, create an entry.  Use the value as
        !           458: the distinguished name.  Other parameters should be a set of supplied
        !           459: defaults.  Optionally allocate a randomly generated password.
        !           460: \item If there are multiple approximate matches, or an exact match to
        !           461: multiple entries, break for human intervention
        !           462: to disambiguate.
        !           463: \item  Optionally, delete any children of the DN not identified.
        !           464: \end {itemize}
        !           465: This tool gives a general facility for namespace management.
        !           466: 
        !           467: \item Given an RDN {\em or} unique value relative to a DN, merge in
        !           468: specified attributes.  This can be used to merge in partial data from many
        !           469: sources, having used the previous tool to control the namespace.
        !           470: 
        !           471: \end {enumerate}
        !           472: 
        !           473: 
        !           474: 
        !           475: 
        !           476: \subsection {Consistency Checking}
        !           477: 
        !           478: Tools to check the data stored will include:
        !           479: 
        !           480: \begin {itemize}
        !           481: \item Presence of attributes
        !           482: \item Syntax checking
        !           483: \item Old entries
        !           484: \item Misc (e.g., searching for swear words)
        !           485: \end {itemize}
        !           486: 
        !           487: 
        !           488: \subsection {Changing Names and Arboriculture}
        !           489: 
        !           490: It is useful to be able to change non-leaf names, and to move around
        !           491: portions of the DIT.  This is rather easy to do locally, because of the
        !           492: nature of the QUIPU EDB format.  However, much care must be taken to
        !           493: correctly preserve external references.  Often, there will be the need to
        !           494: use aliases as a part of the transition.
        !           495: 
        !           496: Moving is not supported directly by the directory.  This needs to be done by 
        !           497: creation and deletion, with appropriate additional EDB management (pointers).
        !           498: Tools should be provided to:
        !           499: 
        !           500: \begin {itemize}
        !           501: \item Delete EDBs (recursively)
        !           502: \item Copy EDBs
        !           503: \item Move EDBs (combining the previous two)
        !           504: \end {itemize}
        !           505: 
        !           506: These sound easy, but much care must be taken with the propagation aspects.
        !           507: 

This archive runs on limited infrastructure. Preserving old code on modern bandwidth. Automated agents are requested to crawl responsibly.