![[BACK]](/cvsweb/icons/back.gif){kind=icon}
Return to udb.t CVS log ![[TXT]](/cvsweb/icons/text.gif){kind=icon}
![[DIR]](/cvsweb/icons/dir.gif){kind=icon}
Up to [Research Unix] / researchv8dc / cmd / adb|
Return to udb.t CVS log | Up to [Research Unix] / researchv8dc / cmd / adb |
1.1 ! root 1: .po .5i ! 2: .bd S B 3 ! 3: .de UX ! 4: .ie \\n(GA>0 \\$2UNIX\\$1 ! 5: .el \{\ ! 6: .if n \\$2UNIX\\$1* ! 7: .if t \\$2UNIX\\$1\\f1\(dg\\fP ! 8: .FS ! 9: .if n *UNIX ! 10: .if t \(dgUNIX ! 11: .ie \\$3=1 is a Footnote of Bell Laboratories. ! 12: .el is a Trademark of Bell Laboratories. ! 13: .FE ! 14: .nr GA 1\} ! 15: .. ! 16: .TL ! 17: Using ADB to Debug the ! 18: .UX ! 19: Kernel ! 20: .AU ! 21: Samuel J. Leffler ! 22: .AU ! 23: William N. Joy ! 24: .AI ! 25: Computer Systems Research Group ! 26: Department of Electrical Engineering and Computer Science ! 27: University of California, Berkeley ! 28: Berkeley, California 94720 ! 29: (415) 642-7780 ! 30: .de IR ! 31: \fI\\$1\fP\\$2 ! 32: .. ! 33: .de DT ! 34: .TA 8 16 24 32 40 48 56 64 72 80 ! 35: .. ! 36: .AB ! 37: .PP ! 38: .FS ! 39: *DEC and VAX are trademarks of ! 40: Digital Equipment Corporation. ! 41: .FE ! 42: This document describes the use of extensions made ! 43: to the 4.1bsd release of the VAX* ! 44: .UX ! 45: debugger adb for the purpose of debugging the ! 46: .UX ! 47: kernel. ! 48: It discusses the changes made to allow ! 49: standard adb commands to ! 50: function properly with the kernel and ! 51: introduces the basics necessary for users ! 52: to write adb command scripts which ! 53: may be used to augment the standard adb ! 54: command set. The examination techniques described here ! 55: may be applied to running systems, as well as ! 56: the post-mortem dumps automatically created ! 57: by the ! 58: .IR savecore (8) ! 59: program after a system crash. ! 60: The reader is expected to have at least a ! 61: passing familiarity with the debugger command ! 62: language. ! 63: .AE ! 64: .ds LH "Using ADB on the UNIX Kernel ! 65: .ds RH Introduction ! 66: .ds CF \*(DY ! 67: .LP ! 68: .nr H1 1 ! 69: .ds CH " ! 70: .bp ! 71: .nr % 1 ! 72: .ds CH "\(hy \\n(PN \(hy ! 73: .LG ! 74: .B ! 75: .ce ! 76: 1. INTRODUCTION ! 77: .sp 2 ! 78: .R ! 79: .NL ! 80: .PP ! 81: Modifications have been made to the ! 82: standard VAX ! 83: .UX ! 84: debugger adb to simplify ! 85: examination of post-mortem ! 86: dumps automatically generated following a system crash. ! 87: These changes may also be used when examining ! 88: .UX ! 89: in its normal operation. ! 90: This document serves as ! 91: an introduction to the \fBuse\fP of ! 92: these facilities, and ! 93: should not be construed as a description ! 94: of \fIhow to debug the kernel\fP. ! 95: .NH 2 ! 96: Invocation ! 97: .PP ! 98: When examining the ! 99: .UX ! 100: kernel a new option, \fB\-k\fP, should be used, e.g. ! 101: .sp 1 ! 102: .ti +5 ! 103: adb \-k /vmunix /dev/mem ! 104: .sp 1 ! 105: This flag causes adb to partially simulate ! 106: the VAX virtual memory hardware when ! 107: accessing the \fIcore\fP file. ! 108: In addition the internal state maintained ! 109: by the debugger is initialized from ! 110: data structures maintained by the ! 111: .UX ! 112: kernel explicitly for debugging\(dd. ! 113: A post-mortem dump may be examined in a similar ! 114: fashion, ! 115: .sp 1 ! 116: .ti +5 ! 117: adb \-k vmunix.? vmcore.? ! 118: .sp 1 ! 119: where the appropriate version of the saved operating system ! 120: image and core dump are supplied in place of ``?''. ! 121: .FS ! 122: \(dd If the \-k flag is not used when invoking ! 123: adb the user must explicitly calculate virtual ! 124: addresses. With the \-k option adb interprets ! 125: page tables to automatically ! 126: perform virtual to physical address translation. ! 127: .FE ! 128: .NH 2 ! 129: Establishing Context ! 130: .PP ! 131: During initialization adb attempts to establish the ! 132: context of the ``currently active process'' by examining ! 133: the value of the kernel variable \fImasterpaddr\fP. ! 134: This variable contains the virtual address of the ! 135: process context block of the last process which ! 136: was set executing by the \fISwtch\fP routine. ! 137: \fImasterpaddr\fP normally provides sufficient information ! 138: to locate the current stack frame (via the stack ! 139: pointers found in the context block). ! 140: By locating the VAX process context block for the ! 141: process, adb may then perform virtual to ! 142: physical address translation using that process's ! 143: in-core page tables. ! 144: .PP ! 145: When examining post-mortem dumps establishing the ! 146: context of the ``currently active process'' is nontrivial. ! 147: This is due to the different ways in which the ! 148: VAX may save state after a nonrecoverable error. ! 149: Crashes may or may not be ``clean'' (i.e. ! 150: the top of the interrupt stack contains the process's ! 151: kernel mode stack pointer and program counter); ! 152: an ``unclean'' crash will occur, for instance, ! 153: if the interrupt stack overflows. ! 154: Thus, one must manually try one of two possible techniques ! 155: to get a stack trace from a post-mortem dump. If the ! 156: crash was clean the current stack frame is present at the ! 157: ``top'' of the interrupt stack, \fIintstack\-4\fP, ! 158: and the command ! 159: .sp 1 ! 160: .ti +5 ! 161: *(\fBintstack\fP\-4)$c ! 162: .sp 1 ! 163: will generate a stack trace all the way from the kernel ! 164: to the top of the user process's stack (e.g. to the ! 165: \fImain\fP routine in the user process which was running). ! 166: Otherwise, one must scan through the interrupt stack ! 167: looking for the stack frame. This is usually indicated ! 168: by a zero longword entry (the argument count), ! 169: .sp 1 ! 170: .in +5 ! 171: .nf ! 172: \fBintstack\fP/\fBX\fP ! 173: .fi ! 174: .in -5 ! 175: .sp 1 ! 176: Once the stack frame has been located, the command ! 177: .sp 1 ! 178: .ti +5 ! 179: \fB*.$c\fP ! 180: .sp 1 ! 181: will generate a stack trace. ! 182: An alternate method may be used when a trace of a particular ! 183: process is required, see section 2.3. ! 184: .ds RH "Command Scripts ! 185: .LP ! 186: .nr H1 2 ! 187: .bp ! 188: .LG ! 189: .B ! 190: .ce ! 191: 2. ADB COMMAND SCRIPTS ! 192: .sp 2 ! 193: .R ! 194: .nr H2 0 ! 195: .NL ! 196: .NH 2 ! 197: Extending the Formatting Facilities ! 198: .PP ! 199: Once the process context has been established, the ! 200: complete adb command set is available for interpreting ! 201: data structures. In addition, a number of adb scripts have ! 202: been created to simplify the structured printing of commonly ! 203: referenced kernel data structures. The scripts normally ! 204: reside in ! 205: the directory \fI/usr/lib/adb\fP, and are invoked ! 206: with the ``$<'' operator. ! 207: (A later table lists the ``standard'' scripts.) ! 208: .PP ! 209: As an example, consider the following listing which ! 210: contains a dump of a faulty process's state ! 211: (our typing is shown emboldened). ! 212: .sp 1 ! 213: .nf ! 214: .DT ! 215: % \fBadb \-k vmunix.17 vmcore.17\fP ! 216: sbr 8001d064 slr d9c ! 217: p0br 800efa00 p0lr 34 p1br 7f8efe00 p1lr 1ffff2 ! 218: \fB*(intstack\-4)$c\fP ! 219: _boot() from 80004025 ! 220: _boot(0,4) from 80004025 ! 221: _panic(80021185) from 800057e2 ! 222: _soreceive(8017478c,0) from 80007c90 ! 223: _read() from 800098d7 ! 224: _syscall() from 8000b6e2 ! 225: _Xsyscall(3,7fffe834,258) from 80000f64 ! 226: ?() from c1c ! 227: ?() from 26a ! 228: ?(0,7fffef18,7fffef1c) from 1d3 ! 229: ?() from 2f ! 230: \fB800021185/s\fP ! 231: _icpreg+99: receive ! 232: \fBu$<u\fP ! 233: _u: ! 234: _u: ksp usp ! 235: 7fffff9c 7fffe59c ! 236: r0 r1 r2 r3 ! 237: 155c00 800237d4 80041800 3 ! 238: r4 r5 r6 r7 ! 239: 0 0 11090 80041800 ! 240: r8 r9 r10 r11 ! 241: 80021244 c 7fffe5b4 80000000 ! 242: ap fp pc psl ! 243: 7fffffe8 7fffffa4 8000b784 d80004 ! 244: p0br p0lr p1br p1lr ! 245: 800efa00 4000034 7f8efe00 1ffff2 ! 246: szpt cmap2 sswap ! 247: 2 94000307 0 ! 248: sigc1 sigc2 sigc3 ! 249: 1af03fb fa007f02 40cbc6c ! 250: _u+78: arg0 arg1 arg2 ! 251: 3 7fffe834 258 ! 252: _u+8c: segflg error uid gid ruid rgid procp ! 253: 0 0 4 a 4 a 80041800 ! 254: ! 255: _u+d4: uap rv1 rv2 ubase ! 256: 7ffff078 0 1 7fffe834 ! 257: count off cdir rdir ! 258: 258 150 8003cf00 0 ! 259: _u+f4: pathname ! 260: .netrc ! 261: dirp dino entry pdir ! 262: 3 1395 .netrc0 ! 263: 7ffff11c: ofiles ! 264: 80040818 80040818 80040818 800406b0 ! 265: 800406d4 800406ec 0 0 ! 266: 0 0 0 0 ! 267: 0 0 0 0 ! 268: 0 0 0 0 ! 269: ! 270: ofileflg ! 271: 0 0 0 0 0 0 0 0 ! 272: 0 0 0 0 0 0 0 0 ! 273: 0 0 0 0 ! 274: 7ffff180: sigs ! 275: 0 360c 1 360c ! 276: 0 0 0 aae ! 277: 0 0 0 0 ! 278: 0 0 0 0 ! 279: 0 0 0 0 ! 280: 1 0 0 0 ! 281: 0 0 0 0 ! 282: 0 0 0 0 ! 283: ! 284: code ar0 prbase prsize ! 285: 0 80000000 0 0 ! 286: ! 287: .ne 2 ! 288: 7ffff248: proff prscal eosys sep ttyp ! 289: 0 0 0 0 800288b4 ! 290: ! 291: 7ffff258: ttymin ttymaj ! 292: 0 0 ! 293: 7ffff25e: xmag xtsiz xdsiz xbsiz ! 294: 3c000000 10000000 108c0000 a680000 ! 295: ! 296: xssiz entloc relflg ! 297: 0 0 6c720000 ! 298: 7ffff27e: directory ! 299: ogin ! 300: start acflg fpflg cmsk tsiz dsiz ! 301: 11688 0 12 0 160000 60000 ! 302: ! 303: 7ffff2a2: ssiz ! 304: 80000 ! 305: \fB80041800$<proc\fP ! 306: 80041800: link rlink addr ! 307: 800237d4 0 800efde0 ! 308: 8004180c: upri pri cpu stat time nice slp cursig ! 309: 073 073 045 03 023 024 0 0 ! 310: 80041814: sig siga0 siga1 flag ! 311: 0 80002 45 8001 ! 312: 80041824: uid pgrp pid ppid poip szpt tsize ! 313: 4 bb bc bb 0 2 1e ! 314: 80041834: dsize ssize rssize maxrss ! 315: 16 6 14 3fffff ! 316: 80041844: swrss swaddr wchan textp ! 317: 0 0 0 80044ee0 ! 318: 80041854: clktim p0br xlink ticks ! 319: 0 800efa00 80041720 22 ! 320: 80041864: %cpu ndx idhash pptr ! 321: +5.1369253545999527e\-02 1c 8 80041720 ! 322: \fB80044ee0$<text\fP ! 323: 80044ee0: daddr ! 324: 7e2 0 0 0 ! 325: 0 0 0 0 ! 326: 0 0 0 0 ! 327: ! 328: ptdaddr size caddr iptr ! 329: 352 1e 80041800 8003cfa0 ! 330: ! 331: rssize swrss count ccount flag slptim poip ! 332: 1a 0 02 02 042 0 0 ! 333: .sp 1 ! 334: .fi ! 335: .PP ! 336: The cause of the crash was a ``panic'' ! 337: (see the stack trace) due to the 0 ! 338: argument passed the \fIsoreceive\fP routine. The majority ! 339: of the dump was done to illustrate the use of two command ! 340: scripts used to format kernel data structures. The ``u'' ! 341: script, invoked by the command ``u$<u'', is a lengthy series ! 342: of commands which pretty-prints the user vector. Likewise, ! 343: ``proc'' and ``text'' are scripts used to format the obvious ! 344: data structures. Let's quickly examine the ``text'' script (the ! 345: script has been broken into a number of lines for convenience ! 346: here; in actuality it is a single line of text). ! 347: .sp 1 ! 348: .nf ! 349: \&./"daddr"n12Xn\e ! 350: "ptdaddr"16t"size"16t"caddr"16t"iptr"n4Xn\e ! 351: "rssize"8t"swrss"8t"count"8t"ccount"8t"flag"8t"slptim"8t"poip"n2x4bx++n ! 352: .sp 1 ! 353: .fi ! 354: The first line produces the list of disk block addresses associated ! 355: with a swapped out text segment. The ``n'' format forces a new-line ! 356: character, with 12 hexadecimal integers printed immediately after. ! 357: Likewise, the remaining two lines of the command format the remainder ! 358: of the text structure. The expression ``16t'' causes adb to tab ! 359: to the next column which is a multiple of 16. ! 360: The last two plus operators are present ! 361: to round ``.'' to the end of the text structure. This allows the ! 362: user to reinvoke the format on consecutive text structures without ! 363: having to be concerned about proper alignment of ``.''. ! 364: .PP ! 365: The majority of the scripts provided are of this nature. ! 366: When possible, the formatting scripts print a data structure ! 367: with a single format to allow subsequent reuse when interrogating ! 368: arrays of structures. That is, the previous script could have ! 369: been written ! 370: .sp 1 ! 371: .nf ! 372: \&./"daddr"n12Xn ! 373: +/"ptdaddr"16t"size"16t"caddr"16t"iptr"n4Xn ! 374: +/"rssize"8t"swrss"8t"count"8t"ccount"8t"flag"8t"slptim"8t"poip"n2x4bx++n ! 375: .sp 1 ! 376: .fi ! 377: but then reuse of the format would have invoked only the last ! 378: line of the format. ! 379: .NH 2 ! 380: Traversing Data Structures ! 381: .PP ! 382: The adb command language can be used to traverse complex data ! 383: structures. One such data structure, a linked list, occurs ! 384: quite often in the kernel. By using adb variables and the ! 385: normal expression operators it is a simple matter to construct ! 386: a script which chains down the list printing each element ! 387: along the way. ! 388: .PP ! 389: For instance, the queue of processes awaiting timer events, ! 390: the callout queue, is printed with the following two scripts: ! 391: .sp 1 ! 392: .nf ! 393: .ne 4 ! 394: \fBcallout\fP: ! 395: .in +5 ! 396: .sp 1 ! 397: calltodo/"time"16t"arg"16t"func"12+ ! 398: *+$<callout.next ! 399: .sp 1 ! 400: .ne 6 ! 401: .ti -5 ! 402: \fBcallout.next\fP: ! 403: .sp 1 ! 404: \&./Dpp ! 405: *+>l ! 406: ,#<l$< ! 407: <l$<callout.next ! 408: .sp 1 ! 409: .in -5 ! 410: .fi ! 411: .PP ! 412: The first line of the script \fBcallout\fP starts the traversal ! 413: at the global symbol ! 414: \fIcalltodo\fP and prints a set of headings. ! 415: It then skips the empty portion of the structure used ! 416: as the head of the queue. ! 417: The second line then invokes the script \fBcallout.next\fP ! 418: moving ``.'' to ! 419: the top of the queue (``*+'' performs the indirection ! 420: through the link entry of the structure at the head of the queue). ! 421: .PP ! 422: \fBcallout.next\fP prints values for each column, then performs ! 423: a conditional test on the link to the next entry. This test ! 424: is performed as follows, ! 425: .IP "*+>l" 9 ! 426: Place the value of the ``link'' in the adb variable ``<l''. ! 427: .IP ",#<l$<" ! 428: If the value stored in ``<l'' is non-zero, then the current ! 429: input stream (i.e. the script \fBcallout.next\fP) is terminated. ! 430: Otherwise, the expression ``#<l'' will be zero, and the ``$<'' ! 431: will be ignored. That is, the combination of the logical negation ! 432: operator ``#'', adb variable ``<l'', and ``$<'' operator ! 433: creates a statement of the form, ! 434: .sp 1 ! 435: .ce ! 436: if (!link) exit; ! 437: .sp 1 ! 438: The remaining line of \fBcallout.next\fP simply reapplies the ! 439: script on the next element in the linked list. ! 440: .LP ! 441: A sample \fIcallout\fP dump is shown below. ! 442: .nf ! 443: .sp 1 ! 444: .ne 14 ! 445: % \fBadb \-k /vmunix /dev/mem\fP ! 446: sbr 8001f864 slr d9c ! 447: p0br 800efa00 p0lr 8e p1br 7f8efe00 p1lr 1ffff2 ! 448: \fB$<callout\fP ! 449: _calltodo: ! 450: _calltodo: time arg func ! 451: 8004ecfc: 26 0 _dzscan ! 452: 8004ed0c: 8 0 _upwatch ! 453: 8004ed1c: 0 0 _ip_timeo ! 454: 8004ed5c: 0 0 _tcp_timeo ! 455: 8004ed6c: 0 0 _rkwatch ! 456: 8004ecfc: 52 0 _dzscan ! 457: 8004ed2c: 68 _Syssize+70 _tmtimer ! 458: 8004ed3c: 2920 0 _memenable ! 459: .fi ! 460: .sp 1 ! 461: .NH 2 ! 462: Supplying Parameters ! 463: .PP ! 464: If one is clever, a command script may use the address ! 465: and count portions of an adb command as parameters. An example of ! 466: this is the \fBsetproc\fP script used to switch to the ! 467: context of a process with a known process-id; ! 468: .sp 1 ! 469: .ti +5 ! 470: \fB99$<setproc\fP ! 471: .sp 1 ! 472: The body of \fBsetproc\fP is ! 473: .sp 1 ! 474: .in +5 ! 475: .nf ! 476: \&.>4 ! 477: *nproc>l ! 478: *proc>f ! 479: $<setproc.nxt ! 480: .in -5 ! 481: .sp 1 ! 482: .fi ! 483: while \fBsetproc.nxt\fP is ! 484: .sp 1 ! 485: .nf ! 486: .in +5 ! 487: (*(<f+28))&0xffff="pid "X ! 488: ,#((*(<f+28)&0xffff)-<4)$<setproc.done ! 489: <l-1>l ! 490: <f+70>f ! 491: ,#<l$< ! 492: $<setproc.nxt ! 493: .in -5 ! 494: .sp 1 ! 495: .fi ! 496: The process-id, supplied as the parameter, is stored in the ! 497: variable ``<4'', the number of processes is placed in ``<l'', ! 498: and the base of the array of process structures in ``<f''. ! 499: \fBsetproc.nxt\fP then performs a linear search through the ! 500: array until it matches the process-id requested, or until ! 501: it runs out of process structures to check. The script ! 502: \fBsetproc.done\fP simply establishes the context of the ! 503: process, then exits. ! 504: .NH 2 ! 505: Standard Scripts ! 506: .PP ! 507: The following table summarizes the command scripts currently ! 508: available in the directory \fI/usr/lib/adb\fP. ! 509: .TS ! 510: center, box; ! 511: c s s ! 512: l | l | l ! 513: lb | l | l. ! 514: Standard Command Scripts ! 515: _ ! 516: Name Use Description ! 517: _ ! 518: buf \fIaddress\fP$<\fBbuf\fP format block I/O buffer ! 519: callout $<\fBcallout\fP print timer queue ! 520: clist \fIaddress\fP$<\fBclist\fP format character I/O linked list ! 521: dino \fIaddress\fP$<\fBdino\fP format directory inode ! 522: dir \fIaddress\fP$<\fBdir\fP format directory entry ! 523: file \fIaddress\fP$<\fBfile\fP format open file structure ! 524: filsys \fIaddress\fP$<\fBfilsys\fP format in-core super block structure ! 525: findproc \fIpid\fP$<\fBfindproc\fP find process by process id ! 526: ifnet \fIaddress\fP$<\fBifnet\fP format network interface structure ! 527: ifuba \fIaddress\fP$<\fBifuba\fP format UNIBUS resource structure ! 528: inode \fIaddress\fP$<\fBinode\fP format in-core inode structure ! 529: inpcb \fIaddress\fP$<\fBinpcb\fP format internet protocol control block ! 530: mact \fIaddress\fP$<\fBmact\fP show ``active'' list of mbuf's ! 531: mbstat $<\fBmbstat\fP show mbuf statistics ! 532: mbuf \fIaddress\fP$<\fBmbuf\fP show ``next'' list of mbuf's ! 533: mount \fIaddress\fP$<\fBmount\fP format mount structure ! 534: pcb \fIaddress\fP$<\fBpcb\fP format process context block ! 535: proc \fIaddress\fP$<\fBproc\fP format process table entry ! 536: setproc \fIpid\fP$<\fBsetproc\fP switch process context to \fIpid\fP ! 537: socket \fIaddress\fP$<\fBsocket\fP format socket structure ! 538: tcpcb \fIaddress\fP$<\fBtcpcb\fP format TCP control block ! 539: text \fIaddress\fP$<\fBtext\fP format text structure ! 540: traceall $<\fBtraceall\fP show stack trace for all processes ! 541: tty \fIaddress\fP$<\fBtty\fP format tty structure ! 542: u \fIaddress\fP$<\fBu\fP format user vector, including pcb ! 543: vtimes \fIaddress\fP$<\fBvtimes\fP format process times ! 544: .TE ! 545: .ds RH "Summary ! 546: .LP ! 547: .nr H1 2 ! 548: .bp ! 549: .LG ! 550: .B ! 551: .ce ! 552: 3. SUMMARY ! 553: .sp 2 ! 554: .R ! 555: .nr H2 0 ! 556: .NL ! 557: .PP ! 558: The extensions made to adb provide basic support for ! 559: debugging the ! 560: .UX ! 561: kernel by eliminating the need for a user to carry ! 562: out virtual to physical address translation. A collection ! 563: of scripts have been written to nicely format the major ! 564: kernel data structures and aid in switching between ! 565: process contexts. This has been carried out with ! 566: only minimal changes to the debugger. ! 567: .PP ! 568: More work is needed to provide enough information ! 569: for the debugger to automatically establish context ! 570: after a system crash. The system currently does not ! 571: always save enough state to allow the debugger to reliably ! 572: locate the stack frame just prior to an exception. ! 573: .PP ! 574: More work is also required on the user interface ! 575: to adb. It appears the inscrutable adb command language ! 576: has limited widespread use of much of the power of ! 577: adb. One possibility is to provide a more comprehensible ! 578: ``adb frontend'', just as \fIbc\fP(1) is used to ! 579: frontend \fIdc\fP(1). ! 580: .PP ! 581: Finally, adb could be significantly improved if it ! 582: were knowledgeable about a programs data structures. ! 583: This would eliminate the use of numeric offsets into ! 584: C structures.
This archive runs on limited infrastructure. Preserving old code on modern bandwidth. Automated agents are requested to crawl responsibly.