[ Next Article | Previous Article | Book Contents | Library Home | Legal | Search ]
Commands Reference, Volume 1

crash Command

Purpose

Displays system images for examining a dump.

Syntax

crash [-a ] [ -i IncludeFile ] [ SystemImageFile KernelFile ] ]

Description

The crash command is an interactive utility for examining an operating system image, or the running kernel. The crash command facility interprets and formats control structures in the system and certain miscellaneous functions for examining a dump.

The SystemImageFile parameter specifies the file that contains the system image. The default for the SystemImageFile parameter is the /dev/mem file. The KernelFile parameter contains the kernel symbol definitions. The default for the KernelFile parameter is the /unix file.

You can run the crash command with no arguments to examine an active system. If you specify a system image file, the crash command assumes it is a system dump file and sets the default thread to the thread running at the time of the crash.

Notes:
  1. When using the crash command a kernel file must be available.
  2. Stack tracing of the current process on a running system does not work.
  3. When crash is run on a running system, you will see a message about the possibility that crash may cause a system crash and/or data corruption, and there will be error log entries noting each time crash is started and stopped. These are done because of the risk posed by certain crash subcommands. Those subcommands are shown here with the messages, labeled Attention. If you limit yourself to subcommands without these messages, you can avoid problems.
  4. If the dump file has been compressed, the crash command will be unable to read it. If you are attempting to run crash directly against a dump device that contains a compressed dump, you will have to use the savecore command to copy the dump to a file first. After running the savecore command, uncompress the file and run crash against that file. If you have a compressed regular file that contains your dump, it must be uncompressed with the uncompress command before crash can read the dump. See the sysdumpdev command for more information on compressed dumps. This section applies to only AIX Version 4.3.2 and later versions.

The crash command recognizes the following aliases in subcommand format specifications:

Format Aliases Format Aliases
byte b instruction I
character char, c longdec ld, D
decimal dec, e longoct lo, O
directory direct, dir, d octal oct, o
hexadecimal hexadec, hex, h, x write w
i-node ino, i

Note:The instruction format disassembles instructions at a specified address. The crash command attempts to disassemble instructions to a PowerPC or POWER architecture instruction, depending upon the machine architecture on which the system dump was taken. If the instruction is not in the appropriate architecture, the crash command attempts to disassemble it to an instruction from the other architecture. This behavior can be modified with the set idarch subcommand. For more information, see "Interpreting an Assembler Listing" in AIX Assembler Language Reference.

Symbols displayed by crash

Several crash subcommands (such as trace, ds, ....) display symbol names and offsets corresponding to numeric addresses. All symbols from KernelFile display with the symbol name, a + sign, and the offset, for example:

.thread_terminate+3e0
vmmerrlog + 0x00000018

Text symbols from kernel extensions are shown in the form .[KernelExtensionName:FunctionName]+Offset, but only if the traceback tables for the kernel extension can be found in SystemImageFile. Offset is the hexadecimal offset (in bytes) from the beginning of the function. In the following example, the address being shown is 0x44c bytes past the beginning of the svc_run routine in the nfs_ext kernel extension.

.[nfs.ext:svc_run]+44c

Data symbols from kernel extensions cannot be displayed by crash, so the addresses of data items within kernel extensions display in the form .[KernelExtensionName]+Offset, where Offset is the offset of the address from the beginning of the kernel extension. This is also used for text symbols when traceback tables are not available in SystemImageFile. In the following example, the address being shown is 0x4c bytes past the beginning of the nfs.ext kernel extension:

.[nfs.ext] + 0x0000004c

Addresses in crash

Many of the commands in crash take addresses or kernel symbol names as parameters. Addresses are always specified in hexadecimal, and can usually be specified in one of the following forms:

addr An 8 digit hexadecimal number, taken to be as an effective address within the context of the current process and thread, or (in some cases) the context of the thread specified on a previous cm command. addr can be prefixed with the characters 0x.
segid:offset segid is the segment ID for a virtual memory segment. The maximum size is 6 hex digits. offset is the offset (in bytes) from the beginning of that segment. The maximum size is 7 hex digits.
r:realaddr r is the literal character "r". realaddr is a real memory address. This form can only be used when running crash against a system dump, and it only will display dump data areas that were dumped by real address instead of virtual address. readaddr can be up to 12 hexadecimal digits.
To enhance readability, you may include underscores ("_") anywhere within these values.

Examples:

18340050
2314:55300
r:14_3370_0560 (same as r:1433700560)

Command-line Editing

The crash command provides command line editing features similar to those provided by the Korn shell. vi mode provides vi-like editing features, while emacs mode gives you controls similar to emacs. You can turn these features on by using the crash subcommand set edit. So, to turn on vi-style command-line editing, you would type the subcommand set edit vi.

Output Redirection

The crash command provides a subset of Korn shell input/output redirection. Specifically, the following operators are provided:

| (pipe symbol)

Pipes all output of the command before the symbol to the input of the command after the symbol. Both standard output and error output are affected, which is different than standard shell behavior.

> filename

Writes the output of the command before the > to filename. Both standard and error output are written to the file.

>> filename

Adds the output of the command before the >> to the end of filename. Both standard and error output are written to the file.

Subcommands

The crash command recognizes several subcommands. The crash command presents a > (greater-than sign) prompt when it is ready to interpret subcommands entered from the workstation. The general subcommand format for the crash command is:

Subcommand [Flags] [StructuresToBeDisplayed]

When allowed, the Flag parameters modify the format of the data displayed. If you do not specify which structure elements you want to examine, all valid entries are displayed. In general, those subcommands that perform I/O operations with addresses assume hexadecimal notation.

Since the crash command only deals with kernel threads, the word thread when used alone will be used to mean kernel thread in the crash documentation that follows. The default thread for several subcommands is the current thread (the thread currently running). On a multiprocessor system, you can use the cpu subcommand to change the current processor: the default thread becomes the running thread on the selected processor.

The parameters ProcessTableEntry and ThreadTableEntry are used in many subcommands to indicate a process or thread respectively. These parameters are simply numbers for table entry indexes which can be displayed using the proc and thread subcommands.

Most of the subcommands recognized by the crash command have aliases (abbreviated forms that give the same result). The crash command recognizes the following subcommands:

alter [[-c] [-s] [-w] [-l] [L | ll] addr val
  Recognized by the poke subcommand alias. Put the value val at given address addr.

Attention:You should use this command with extreme caution. Errors in the use of this command may cause system failures.

-c
Indicates that val is a char.(1 byte) This is the default.
-s
Indicates that val is a short. (2 bytes)
-w
Indicates that val is a word. (4 bytes)
-l
Indicates that val is a long. (4 bytes)
-L | ll
Indicates that val is a long long. (8 bytes)
alter [[-c] [-s] [-w] [-l]] [L | ll] addr origval=val
  Recognized by the poke subcommand alias. Put value at given address only if origval is already stored at that address.

Attention:You should use this command with extreme caution. Errors in the use of this command may cause system failures.

Note: The flags indicate the val is a char, short, word, long, or L | ll for storing 64-bit information.

alter [[-c] [-s] [-w] [-l]] [L | ll] addr
  Recognized by the poke subcommand alias. Start altering memory at the given address, interactively,

Attention:You should use this command with extreme caution. Errors in the use of this command may cause system failures.

Note: The flags indicate the val is a char, short, word, long, or L | ll for storing 64-bit information.

buf [BufferHeaderNumber] . . .
  Recognized by the bufhdr and hdr subcommand aliases. Displays the system buffer headers. The buff.# file is created where the # variable file extension is the BufferHeaderNumber. Binary data is written to the buff.# file.
buffer [Format] [BufferHeaderNumber] . . .
  Recognized by the b subcommand alias. Displays the data in a system buffer according to the Format parameter. When specifying a buffer header number, the buffer associated with that buffer header is displayed. If you do not provide a Format parameter, the previous Format is used. Valid parameters are the decimal, octal, hex, character, byte, directory, i-node, and write formats. The write format creates a file in the current directory containing the buffer data.
calc expr Recognized by the @ subcommand alias. Expression can be any expression supported by the bc command, although mixed case is supported. Values are in hex.
callout Recognized by the c, call, calls, time, timeout,tout, and trb subcommand aliases. Displays all entries on the active trblist (rough equivalent of the cutoutable).
cm [ldron | ldroff] [vmmon | vmmoff] [ThreadTableEntry SegmentNumber]
  Changes the segment map of the crash command internal pointers for any process thread segment not paged out if you specify the process-thread-slot number and segment number. This allows the od subcommand to display data relative to the beginning of the segment desired for threads other than the current thread. Specification of vmmon or vmmoff allows selection of whether effective addresses in the range 0x70000000 through 0xafffffff are to be interpreted by the od subcommand as kernel or VMM data references. Similarly, selection of ldron or ldroff allows selection of whether effective addresses in segment 11 (0xbxxxxxxx) and segment 13 (0xdxxxxxxx) are to be interpreted by the od subcommand as references to loader data. When entering the cm subcommand without any parameters, it resets the map of internal pointers.
conv num Recognized by the % subcommand alias. If num is hex, convert to decimal, otherwise convert to hex. Bases are guessed by leading 0x for hex or base# for bases other than decimal.
conv [-] [bdxo] num Recognized by the % subcommand alias. Convert num to specified base binary, decimal, hex, or octal. Bases are guessed by leading 0x for hex or base# for bases other than decimal.
conv [-] [bdxo] [bdxo] num
  Recognized by the % subcommand alias. Convert num from base specified by the first flag to the base specified by the second flag, binary, decimal, hex, or octal.
cpu [ProcessorNumber]
  If no argument is given, the cpu subcommand displays the number of the currently selected processor. Initially, the selected processor is the processor that caused the system crash (when running crash against a dump), or processor 0 (when running against a running system). If the ProcessorNumber argument is given, the cpu subcommand selects the specified processor as the current processor. By extension, this selects the current thread (the running thread on the selected processor). Processor numbering starts from zero.
dblock Address Recognized by the dblk subcommand alias. The dblock subcommand displays the allocated streams data block headers. The Address parameter is required. If the Address is not supplied, this subcommand prints an error message stating that the address is required. See /usr/include/sys/stream.h header file for the datab structure definition. The freep and db_size descriptions are not included in /usr/include/sys/stream.h. These structure members and their descriptions are:
freep
Address of the free pointer.
db_size
Size of the data block.

There is no checking performed on the address passed in as the required parameter. The dblock subcommand will accept any address. It is up to the user to be sure that a valid address is specified.

To determine a valid address, you will need to run the mblock subcommand. From the output of the mblock subcommand, select a non-zero data block address from under the column heading, DATABLOCK.

decode instr Decode the given instruction word. instr is specified as a hex value.
devsw Show device switch table.
devsw major Show the device switch structure for device with the given major number.
dlock[ThreadIdentifier | -p [ProcessorNumber] ]
  Displays deadlock analysis information about all types of locks (simple, complex, and lockl). The dlock subcommand searches for deadlocks from a given start point. If ThreadIdentifier is given, the corresponding thread is the start point. If -p is given without a ProcessorNumber, the start point is the running thread on the current processor. If -p ProcessorNumber is given, the running kernel thread on the specified processor is the start point. If no arguments are given, dlock searches for deadlocks among all threads on all processors.

The first output line gives information about the starting thread, including the lock which is blocking the kernel thread, and a stack trace showing the function calls which led to the blocking lock request. Each subsequent line shows the lock held by the blocked thread from the previous line, and identifies the thread or interrupt handler which is blocked by those locks. If the information required for a full analysis is not available (paged out), an abbreviated display is shown; in this case, examine the stack trace to locate the locking operations which are causing the deadlock. The display stops when a lock is encountered for a second time, or no blocking lock is found for the current thread.

dmodsw Displays the streams drivers switch table. The information printed is contained in an internal structure. The members of this internal structure and their descriptions are:
address
Address of dmodsw.
d_next
Pointer to the next driver in the list.
d_prev
Pointer to the previous driver in the list.
d_name
Name of the driver.
d_flags
Flags specified at configuration time.
d_sqh
Pointer to synch queue for driver-level synchronization.
d_str
Pointer to streamtab associated with the driver.
d_sq_level
Synchronization level specified at configuration time.
d_refcnt
Number of open or pushed count.
d_major
Major number of a driver.

The flags structure member, if set, is based on one of the following values:

#define Value Description
F_MODSW_OLD_OPEN 0x1 Supports old-style (V.3)open/close parameters
F_MODSW_QSAFETY 0x2 Module requires safe timeout/bufcall callbacks
F_MODSW_MPSAFE 0x4 Non MP Safe drivers need funneling

The synchronization level codes are described in the /usr/include/sys/strconf.h header file.

ds [DataAddress] . . . Finds the data symbols closest to the given addresses.
du [ThreadTableEntry] Displays a combined hex and ASCII dump of the specified thread's uthread structure and of the user structure of the process which owns the thread. If the data is not available (paged out), a message is displayed. The default is the current thread.
dump Displays the name of each component for which there is data present. After you select a component name from the displayed list, the crash program loads and runs the associated formatting routine contained in the/usr/lib/ras/dmprtns directory. If there is more than one data area for the selected component, the formatting routine displays a list of the data areas and allows you to select one. The crash command then displays the selected data area. You may enter the quit subcommand to return to the previously displayed list and make another selection or enter the quit subcommand a second time to leave the dump subcommand loop.
errpt [ count] Displays messages in the error log. Count is the number of messages to print that have already been read by the errdemon process. (The default is 3 messages.) errpt always prints all messages that have not yet been read by the errdemon process.
file [FileTableEntry] . . . Recognized by the files and f subcommand aliases. Displays the file table. Unless specific file entries are requested, only those with a nonzero reference count are displayed.
find [-u] [-s] [-p slot] [-c context] [-a alignment] pattern
  Recognized by the x subcommand alias. Search user-space for a given pattern. The default is to search the GPR save areas in the mstsave areas which are both on the Current Save Area Chain (CSA) and in each uthread area for every thread.
-u
Search all process private segments, (Stack, Uarea,...)
-s
Search all process private segment from the current stack pointer.
-c context
Number of bytes of context to print on a match.
-a alignment
Byte alignment for pattern. The default is 4.
-p slot
Search only specified process. The default is to search all processes.

Attention: Using this command on a running system may cause system crashes.

Rules for pattern

pattern is a search pattern of any arbitrary length that contains either a hexadecimal number or a string. To specify a hexadecimal pattern, just type the hex digits. 'Don't care' digits can be represented with the character x. To specify a string pattern, enclose the pattern in double quotes. 'Don't care' characters can be represented with the sequence \x

Examples:

> find -k 02x4
00110a28: 02140008                             |....|
00110af0: 02640004                             |.d..|
00110c80: 02e40004                             |....|
003f0ed8: 02242ff8                             |.$/.|
...

> find -k "b\xt"
00012534: 6269745f                             |bit_|
00012618: 6269745f                             |bit_|
0001264c: 6269745f                             |bit_|
00021cb0: 62797465                             |byte|
00021d60: 62797465                             |byte|
...

> find -k "i_ena" 0 250000     
001ceaa8: 695f656e 61626c65                    |i_enable|
find -k [-c context] [-a alignment] pattern [start[end]]
  Recognized by the x subcommand alias. Search the kernel segments. The default range is the whole of each kernel segment.
-c context Number of bytes of context to print on a match.
-a alignment Byte alignment for pattern. The default is 4.
find -b branch_addr [start_addr[end_addr]]
  Recognized by the x subcommand alias. Search for a branch to the given address. The default range is the whole of each kernel segment.
find -m [-a addr] [-t type] [-c] [-i] [start[end]]
  Recognized by the x subcommand alias. Search the things that look like mbufs. The default search range is the network memory heap.
-a Search for mbufs that point to this cluster address.
-t type Only search for this type of mbuf.
-c Only search for clusters.
-i Ignore length sanity checks.
find -v [-f] wordval [start[end]]
  Recognized by the x subcommand alias. Search for the first word not matching the given value. The default is to search the kernel segments.
-f Force scan to continue when a region not in the dump is scanned.
find -U seg_id Recognized by the x subcommand alias. Search for processes whose segment registers contain the given segment ID.
fmodsw Displays the streams modules switch table. The information printed is contained in an internal structure. The members of this internal structure and their definitions are:
address Address of fmodsw.
d_next Pointer to the next module in the list.
d_prev Pointer to the previous module in the list.
d_name Name of the module.
d_flags Flags specified at configuration time.
d_sqh Pointer to synch queue for module-level synchronization.
d_str Pointer to streamtab associated with the module.
d_sq_level Synchronization level specified at configuration time.
d_refcnt Number of open or pushed count.
d_major -1.

The flags structure member, if set, is based on one of the following values:

#define Value Description
F_MODSW_OLD_OPEN 0x1 Supports old-style (V.3)open/close parameters
F_MODSW_QSAFETY 0x2 Module requires safe timeout/bufcall callbacks
F_MODSW_MPSAFE 0x4 Non MP Safe drivers need funneling

The synchronization level codes are described in the /usr/include/sys/strconf.h header file.

fs [ThreadTableEntry]
  Traces a kernel stack for the thread specified by the thread slot number. Displays the called subroutines with a hex dump of the stack frame for the subroutine that contains the parameters passed to the subroutine. The default thread is the thread currently running. This subcommand will not work on the current thread or a running system because it uses stack tracing; however, it does work on a dump image.
help Recognized by the ? subcommand alias. Print a list of a commands with short descriptions.
help cmd... Recognized by the ? subcommand alias. Print a long description of the specified command.
help /regex... Recognized by the ? subcommand alias. Print those long descriptions that match the specified extended, case-insensitive regular expression.
hide symbol... Hide the specified symbol from the crash commands that convert addresses to symbols and offsets. The main reason for this ability is to hide symbols that may show up in the middle of a function. This occurs in assembly routines. See the unhide subcommand.
hide Show all hidden symbols. See the unhide subcommand.
id[*...] addr [units]
id[*...] symname [units] Instruction Decode. Equivalent to od with the instruction format. The units are measured in number of instructions.
inode [-] [ <MAJ> <MIN> <INUMB> ] . . . 
  Recognized by the ino and i subcommand aliases. Displays the i-node table and the i-node data block addresses. A specific i-node can be displayed by specifying the major and minor device number of the device where the i-node resides and the i-node number. The i-node will only be displayed if it is currently on the system hash list.
kfp[FramePointer] . . .  Recognized by the fp and rl subcommand aliases. If the kfp subcommand is entered without parameters, it displays to the screen the last kernel frame pointer address that was set using kfp. If a frame pointer address is provided, then it sets the kernel frame pointer to the new address. This subcommand is used conjunction with the -r flag on the trace subcommand.
knlist[Symbol] . . . Displays the addresses of all the symbol names given. If the symbol is not found, a no-match message is displayed to the screen. This subcommand reads the kernel export list for either the running system or a dump image.
le [-l32 | -l64 | -p proc_slot | -a] [[ address | name]...]
  Displays load list entries. The default is to display load list entries starting at the kernel load anchor. If an address is specified, without the -a flag, only load list entries which include the address within the text or data area display. If a name is specified all load list entries which have a name that includes the input string display. If an attempt is made to display a paged-out loader entry the subcommand displays an error message. The following flags control the load list entry chain that is searched/displayed:
-l32
Uses the 32-bit shared library load list anchor.
-l64
Uses the 64-bit shared library load list anchor.
-p proc_slot
Uses the load list anchor contained in the indicated processes user area.
-a
Displays a single load list entry at a specified address (an address must be specified with this flag)
link addr num [next_offset [end_val]
  Recognized by the ll subcommand alias. Follows linked list starting at addr. Print num words for each entry. next_offset is the offset in words of the next pointer, the default is 0. end_val is the value of the next pointer that terminates the list, the default is 0.
linkblk Recognized by the lblk subcommand alias. The linkblk subcommand displays the streams linkblk table. See /usr/include/sys/stream.h header file for the linkblk structure definition. If there are no linkblk structures found on the system, the linkblk subcommand prints a message stating that no structures are found.
lock Recognized by the locks subcommand alias. Print status on global kernel locks as well as threads waiting on events and locks.
lock [-clsq] addr|symbol...
  Recognized by the locks subcommand alias. Print lock at addr, or address specified by symbol. The default format is that of a Simple_lock.
-c Print as a Complex_lock
-l Print as a lock_t
-s Print as a Simple_lock. This is the default.
-q Suppress instrumentation data.

Status Bits:

I INTERLOCK
W WAITING
WW WANT_WRITE (Complex_lock)
RD READ_MODE(Complex_lock)
L LOCKBIT (Simple_lock)
S INSTR_ON (Simple_lock)
mblock Address Recognized by the mblk subcommand alias. The mblock subcommand displays the allocated streams message block headers. The Address parameter is required. If Address is not supplied, the subcommand prints an error message stating that the address is required. See /usr/include/sys/stream.h header file for the msgb structure definitions.

There is very little checking performed on the address passed in as the required parameter. The mblock subcommand accepts any address that falls on a 128-byte boundary. It is up to the user to be sure that a valid address is specified.

To determine a valid address, run the queue subcommand. From the output of the queue subcommand, you will need to select a non-zero address in the head of the message queue, under the column heading HEAD, for either a read queue or a write queue.

mbuf Address  Displays system mbuf structures at the specified address.
mst [-f] [Address] . . .   Displays the mstsave portion of the uthread structure at the addresses specified (see the uthread.h and mstsave.h header files in /usr/include/sys). If you do not specify an address, it displays all of the mstsave entries on the CSA chain except the first. If you specify the -f flag the first mstsave area on the CSA chain displays.
ndb Displays network kernel data structures either for a running system or a system dump. The ndb subcommand, short for network debugger, supports the following options:
? Provides first-level help information.
help Provides additional help information.
tcb [Addr] Shows TCBs. The default is HEAD TCB.
udb [Addr] Shows UDBs. The default is HEAD UDB.
socketsAddr Shows sockets at the given address.
mbuf [Addr] Shows the mbuf at the specified address.
ifnet [Addr] Shows the ifnet structures at the specified address.
quit Stops the running option.
xit Exits the ndb submenu.
netm Displays the most recent net_malloc_police record.

Note: Requires that the Memory Overlay Detection System (MODS) be enabled or that the network option net_malloc_police be turned on. For additional information on the net_malloc_police option, see the no command.

netm -a Displays all records, starting with the most recent.

Note: Requires that the MODS be enabled or that the network option net_malloc_police be turned on. For additional information on the net_malloc_police option, see the no command.

netm addr Displays records whose address or caller fields match the given address.

Note: Requires that the MODS be enabled or that the network option net_malloc_police be turned on. For additional information on the net_malloc_police option, see the no command.

netstat Equivalent to the command line version of the netstat command.
nm[Symbol] . . . Displays symbol value and type as found in KernelFile.
od [ldr:][vmm:] [* . . .] [SymbolName | Address] [Count] [Format]
  The od subcommand dumps the number of data values specified by Count starting at SymbolName or Address according to Format. Possible formats are octal, longoct, decimal, longdec, character, hex, instruction, and byte. The default is hex.
Note: If you use the Format, you must also use Count. If the SymbolName or Address> is proceeded by an asterisk, then the symbol or address is dereferenced before displaying the data. Additionally, the strings ldr: and vmm: may be used to indicate that addresses are to be considered loader or VMM addresses, just as if the cm ldron and/or cm vmmon subcommand had been issued.
ppd [ProcessorNumber *]
  Displays per-processor data area (PPDA) structures for the specified processor. If no processor is specified, the current processor selected by the cpu subcommand is used. If the asterisk argument is given, the PPDA of every enabled processor is displayed.
prall Equivalent to crash -a from the command line.
print [Type] Address Recognized by the pr, str, or struct subcommand aliases. Does dbx-style printing of structures. The -i option must be given on the command line to use this feature. Type is the name of the structure to be displayed.
print -d type Recognized by the pr, str, or struct subcommand aliases. Sets the default type for subsequent print commands to type.
print -l offset|name [-e end_val] [type] address
  Recognized by the pr, str, or struct subcommand aliases. Displays a linked list, starting at address using offset or structure member name as the location of the next pointer. Stop when next value of equals end_val. The default end_val is 0.
proc [-] [-r] [ProcessTableEntry] . . . 
  Recognized by the ps and p subcommand aliases. Displays the process table, including the thread count (the number of threads in the process) and state of each process. (See the /usr/include/sys/proc.h file for this structure definition.) The -r flag displays only runnable processes. The - (minus) flag displays a longer listing of the process table.
qrun Displays the list of scheduled streams queues. If there are no queues found for scheduling, the qrun subcommand prints a message stating that there are no queues scheduled for service.
queue [Address] Recognized by the que subcommand alias. The queue subcommand displays the STREAMS queue. If the optional parameter, Address, is not supplied, crash will display information for all write queues available. Refer to the /usr/include/sys/stream.h header file for the queue structure definition.

If you wish to see the information stored for a read queue, issue the queue subcommand with the read queue address specified as the Address parameter.

When you issue the queue subcommand with the Address parameter, the column headings do not distinguish between the read queue and the write queue. One queue address will be displayed under the column heading, QUEUE and the other queue in the pair will be displayed under the column heading, OTHERQ. The write queue will have a numerically higher address than the read queue.

quit Recognized by the q subcommand alias. Exits from the crash command.
search[-sn] name Search the symbols table for name.
-s Prints symbols matching name in the nm format. Also prints the symbol table entry for the last symbol found.
-n Prevents the search from examining kernel extensions..
search[-n] addr Search for the symbol with the largest value less than or equal to addr.
-n Prevents the search from examining kernel extensions..
segst64 [-p pslot | -t tslot] [-l limit [-s segflag[:value][, segflag[:value]]...] [-n [start_esid [end_esid]]
Recognized by the adspace, as, and sr subcommand aliases. Displays segstate information for a 64-bit process. The segstate for the current process displays unless the -p or -t flags are specified. All of the segstate entries display unless limited by the -l flag or the starting esid, start_esid and possible ending esid, end_esid. Specifying the -s flag limits the display to only those segstate entries matching the given segflags, matching pattern types, as well as their corresponding values. The -l flag limits the display to a maximum number of entries. The -n flag also prints the segnodes for the displayed data. Segnode entries are not included in the count when limiting the data with -l.
-p pslot Specifies the process slot number.
-t tslot Specifies the thread slot number.
-s segflag:[value] Limites the display to the segstate entries matching that segflag and value.
-l limit Specifies the number of entries to print.
-n Prints the uadnodes for the displayed data.
select Recognized by the sel subcommand alias. Displays all select control blocks.
select p proc_slot Recognized by the sel subcommand alias. Displays select control blocks for process in specified slot.

Note: The p flag is not prefixed with a -.

select dev_id unique_id
  Recognized by the sel subcommand alias. Displays select control blocks matching the specified device and unique IDs
set Display crash variables and values.
set allhex [no] Causes crash to use only hex values for both input and output as opposed to a mixture of hex and decimal. Specify no to turn this option off.
set edit [emacs|gmacs|none|vi]
  Sets command line editing mode.
set fpregs [yes|no|auto]
  Specify whether or not floating-point registers should be displayed. If auto is used, the fpeu variable in the mstsave area determines when to display the registers.
set idarch [ppc|pwr|auto]
  Set instruction decode architecture. auto detects the architecture from the system.
set logfile [filename]
  Set logfile to given name, or turn off logging if no name is given.
set loglevel [0|1|2] Set logging granularity to:
0
coarse-only commands will be logged.
1
medium-commands and output to terminal will be logged.
2
fine-commands and all outputs will be logged, including redirected.
set prtype [type] Set the default print type. This is equivalent to print -d type.
set quiet [no] Suppress error messages concerning missing or swapped out threads and processes. Specify no to turn off this option.
socket[-] . . .  Recognized by the sock subcommand alias. Displays the system socket structures. If the - (minus sign) flag is used, the socket buffers will also be displayed.
sr64 [-p pslot | -t tslot] [-l limit [-n [start_esid [end_esid]]
Recognized by the segst and seg subcommand aliases. Displays the effective segment IDs (esid) and their corresponding segvals for a 64-bit process. If you do not specify the -p or -t flags, sr64 uses the current process. Otherwise, it uses pslot as the process slot number for the desired process, or tslot as the thread slot number of a thread contained within the desired process. It lists all entries in the adspace unless a starting esid, start_esid and possible ending esid, end_esid is given. Also, it stops listing if the number of entries specified by the -l flag have printed. Since adspace_t holds 16 entries, each line consists of an esid, its corresponding value, and the 3 subsequent values following it in the adspace_t. The -n flag also prints the uadnodes for the displayed data. uadnode entries are not included in the count when limiting the data with -l.
-p pslot Specifies the process slot number.
-t tslot Specifies the thread slot number.
-l limit Specifies the number of entries to print.
-n Prints the uadnodes for the displayed data.
stack[ThreadTableEntry ] . . .
  Recognized by the stk, s, kernel, and k subcommand aliases. Displays a dump of the kernel stack of the thread identified by ThreadTableEntry. If you do not specify an entry, information about the last running kernel thread is displayed. You cannot trace the stack of the current kernel thread on a running system.
stat Displays statistics found in the dump. These statistics include the panic message (if a panic occurred), time of crash, system name, and whether the MODS (here called xmalloc debug) is enabled.
status [ProcessorNumber] Displays a description of the kernel thread scheduled on the designated processor. If no processor is specified, the status subcommand displays information for all processors. The information displayed includes the processor number, thread identifier, thread table slot, process identifier, process table slot, and process name.
stream Displays the stream head table. The information printed is contained in an internal structure. The members of this internal structure are:
address Address of stream head
wq Address of streams write queue
dev Associated device number of the stream
read_error Read error on the stream
write_error Write error on the stream
flags Stream head flag values
push_cnt Number of modules pushed on the stream
wroff Write offset to prepend M_DATA
ioc_id ID of outstanding M_IOCTL request
pollq List of active polls
sigsq List of active M_SETSIGs

The flags structure member, if set, is based on combinations of the following values:

#define Value Description
F_STH_READ_ERROR 0x0001 M_ERROR with read error received, fail all read calls
F_STH_WRITE_ERROR 0x0002 M_ERROR with write error received, fail all writes
F_STH_HANGUP 0x0004 M_HANGUP received, no more data
F_STH_NDELON 0x0008 do TTY semantics for ONDELAY handling
F_STH_ISATTY 0x0010 this stream acts as a terminal
#define Value Description
F_STH_MREADON 0x0020 generate M_READ messages
F_STH_TOSTOP 0x0040 disallow background writes (for job control)
F_STH_PIPE 0x0080 stream is one end of a pipe or fifo
F_STH_WPIPE 0x0100 stream is the "write" side of a pipe
F_STH_FIFO 0x0200 stream is a fifo
F_STH_LINKED 0x040 stream has one or more lower streams linked
F_STH_CTTY 0x0800 stream controlling tty
F_STH_CLOSED 0x4000 stream has been closed, and should be freed
F_STH_CLOSING 0x8000 actively on the way down
symptom[-e] Displays the symptom string for a dump. It is not valid on a running system. The optional -e option will create an error log entry containing the symptom string, and is normally only used by the system and not entered manually. The symptom string can be used to identify duplicate problems.
tcb [ThreadTableEntry] . . . Displays the mstsave portion of the user structures of the named threads (see the user.h and mstsave.h header files). If you do not specify an entry, information about the last running thread displays. This subcommand replaces the pcb subcommand.
thread [-] [-r] [-p ProcessTableEntry | -a Address | ThreadTableEntry]
  Recognized by the th subcommand alias. Displays the contents of the thread table. The - (minus) flag displays a longer listing of the thread table. The -r flag displays only runnable threads. The -p flag displays only those threads which belong to the process identified by ProcessTableEntry. The -a flag displays the thread structure at Address. If ThreadTableEntry is given, only the corresponding thread is displayed.
trace [-r | -m [-f]] [-k | -s] [-r] [ThreadTableEntry ] . . .  
  Recognized by the t subcommand alias. Displays a kernel stack trace of the thread identified by ThreadTableEntry. If you do not specify a thread table entry, information about the current thread is displayed. When using the -k flag, the stack frame addresses indicate the stack frame containing the link register value pointing to the function that is displayed. The -m flag causes trace to display the traceback associated with each mstsave area on the Current Save Area Chain (CSA), except the first. To see a traceback from the first mstsave area, specify the -f flag. When either the -m or -k flags are used, trace may also show the LR (link register) and top stack frame pointer. These are not part of the stack trace and are therefore marked with an asterisk. The -r flag will cause trace to use the kernel frame pointer set up by the kfp subcommand as its starting address instead of the frame pointer found in the SystemImageFile. The trace subcommand will stop and an error will be reported if an invalid frame pointer is encountered. The -s flag displays saved register information for each stack frame. With no flags, trace prints address information.
ts[TextAddress] . . .  Finds the text symbols closest to the given addresses.
tty[ d ] [ l] [ e ] [ Name | Major [ Minor ] ]
  Displays the tty structures. If no parameters are specified, a short list of all open terminals is displayed. Selected terminals can be displayed by specifying the terminal name, such as tty1, or a major number with optional minor number. The flags modify the displayed information: the d flag displays driver information; the l flag displays line discipline information; and the e flag displays information for every module or driver present in the stream for the selected lines.
unhide symbol... Unhide the specified symbol. See the hide subcommand.
unhide Unhide all hidden symbols. See the hide subcommand.
user [-s] [ThreadTableEntry ] . . . 
  Recognized by the uarea, u_area, and u subcommand aliases. Displays the uthread structure and the associated user structure of the thread identified by ThreadTableEntry. (See the/usr/include/sys/user.h file for the user structure definition.) If you do not specify the entry, the information about the last running thread displays. The -s flag limits the output to segment register information.
var Recognized by the tunables, tunable, tune, and v subcommand aliases. Displays the tunable system parameters.
vfs [-] [VfsSlotNumber] Recognized by the mount, mnt, and m subcommand aliases. The vfs uses the specified VfsSlotNumber to display an entry in the vfs table. Use the - flag to display the v-nodes associated with the vfs. The default displays the entire vfs table. See the sys/vfs.h header file for structure definitions.

See examples 8 and 9 for samples of the vfs subcommand output.

vnode [VNodeAddress] Displays data at the specified v-node address as a v-node. The address must be specified in hexadecimal notation. The default is to display all v-node structures. See the sys/vfs.h header file for structure definitions.

See example 10 for a sample of the vnode subcommand output.

which symbol | addr Recognized by the wf subcommand alias. Displays the name of the kernel source file containing symbol or addr.
xmalloc Recognized by the xm and malloc subcommand aliases. Prints information concerning the allocation and usage of kernel memory (the pinned_heap and the kernel_heap).
xmalloc [addr] Recognized by the xm and malloc subcommand aliases. Prints xmalloc information about addr. If addr is not specified crash attempts to find the addresses involved in the system crash caused by the MODS.
xmalloc -s [addr] Recognized by the xm and malloc subcommand aliases. Prints debug xmalloc allocation records associated with addr.

Note: The -s flag requires that the memory overlay detection system (MODS) has been turned on.

xmalloc -h [addr] Recognized by the xm and malloc subcommand aliases. Prints MODS xmalloc free list records associated with addr.

Note: The -h flag requires that the memory overlay detection system (MODS) has been turned on.

xmalloc [-l] -f Recognized by the xm and malloc subcommand aliases. Prints allocation records on free list from earliest-freed to latest-freed. The -l flag prints a long listing.

Note: The -f flag requires that the memory overlay detection system (MODS) has been turned on.

xmalloc [-l] -a Recognized by the xm and malloc subcommand aliases. Prints the allocation record table. The -l flag prints a long listing.

Note: The -a flag requires that the memory overlay detection system (MODS) has been turned on.

xmalloc [-l] -p pageno Recognized by the xm and malloc subcommand aliases. Prints page descriptor information for page pageno. The -l flag prints additional information.
xmalloc -d [addr] Recognized by the xm and malloc subcommand aliases. Prints debug xmalloc allocation record hash chain associated with the record hash value for addr.
xmalloc -v Recognized by the xm and malloc subcommand aliases. Verify allocation trailers of allocated records, and free fill patterns of freed records.

Note: The -v flag requires that the memory overlay detection system (MODS) has been turned on.

! Runs shell commands.
? Displays summary of crash commands.

Flags

-a Prints an assortment of data structures without user interaction to standard output.
-i Reads this include file and to allow the structures declared in this file to be used with the print subcommand. You can supply several options on the command line with this flag.

Examples

  1. To invoke the crash command with the default system image and kernel image files, enter:
    crash

    The crash command returns a > prompt and waits for you to enter a subcommand.

  2. To invoke the crash command with the system image file and the UNIX kernel file, enter:
    crash sysimage /unix

    The crash program returns a > prompt and waits for you to enter a subcommand.

    The following examples show examples of the usage of crash subcommands:

  3. To write buffer 0 to the buff.0 file in binary format, enter:
    buffer write 0
    The crash command returns a > prompt and waits for you to enter a subcommand.
  4. To set the segment map to thread slot number 1 and segment 2, then display ten words starting from address 2ff3b400, using segment 2 of thread slot number 1, enter:
    >cm 1 2 
    t1,2>od 2ff3b400 10 
  5. To display the user structure from the unix kernel sys/user.h, enter:
    crash -i sys/user.h <print user *u 
  6. To display deadlocks anywhere on the system, enter:
    >dlock
  7. To display a deadlock involving a given thread 00d3f, enter:
    >dlock 00d3f
    The output is similar to:
    
    Deadlock from tid 00d3f.  This tid waits for the first line lock, 
    owned by Owner-Id that waits for the next line lock, and so on...  
         LOCK NAME    |   ADDRESS  |  OWNER-ID | WAITING FUNCTION 
               lockC1 | 0x001f79e0 |  Tid 113d | .lock_write_ppc 
                       called from :  .times + 0000020c 
    Dump data incomplete.Only 0 bytes found out of 4.  
                       called from :  .file + 0000000b 
                lockC2 | 0x001f79e8 | Tid d3f | .lock_write_ppc 
                         called from :  .times + 000001c8 
    Dump data incomplete.Only 0 bytes found out of 4.  
                       called from :  .file + 0000000b
  8. To display the third entry in the vfs table, enter:
    > vfs 3
    The output is similar to:
    
    VFS ADDRESS TYPE OBJECT STUB NUM FLAGS PATHS 
      3 1a62494  jfs 1a6d47c 1a6d650 5 D /dev/hd1 mounted over /u 
            flags:  C=disconnected D=device I=remote P=removable 
                    R=readonly     S=shutdown U=unmounted Y=dummy 
  9. To display the v-node table data associated with the third entry in the vfs table, enter:
    > vfs - 3
    The output is similar to:
    
    VFS ADDRESS TYPE  OBJECT    STUB NUM FLAGS PATHS 
      3 1a62494  jfs 1a6d47c 1a6d650   5 D   /dev/hd1 mounted over /u 
    ADDRESS VFS MVFS VNTYPE FSTYPE COUNT ISLOT    INODE FLAGS 
    1a6e0ac   3    -   vreg    jfs     1    -   18f82c0 
    1a6e218   3    -   vreg    jfs     1    -   18f8770 
    1a6e24c   3    -   vreg    jfs     1    -   18f8590 
    1a6e17c   3    -   vdir    jfs     3    -   18f7f00 
    1a6dea4   3    -   vreg    jfs     2    -   18f65b0 
    1a6dfa8   3    -   vdir    jfs     5    -   18f6100
    1a6d47c   3    -   vdir    jfs     1    -   18ea580 vfs_root 
  10. To display the v-node data for the address 1a6e078, enter
    
    > vnode 1a6e078 
    ADDRESS VFS MVFS VNTYPE FSTYPE COUNT ISLOT    DATAPTR FLAGS  
    1a6e078   0    -   vreg    jfs     4     -    18f6790 
     Total VNODES printed 1 

Files

/usr/sbin/crash Contains the crash command.
/dev/mem Default system image file
/unix Default kernel file
/usr/include/sys/*.h Header files for table and structure information.

Related Information

The ksh command, mount command, pstat command, ps command, savecore command, stty command, and sysdumpdev command.

Memory Overlay Detection System (MODS) in the AIX Version 4.3 Kernel Extensions and Device Support Programming Concepts.


[ Next Article | Previous Article | Book Contents | Library Home | Legal | Search ]