Clear tmp & cache in weblogic
ps -ef |grep test-memorypool | awk '{print $2}' | xargs kill -9;find . -name "tmp*" -exec rm -rf {} \;;find . -name "cache*" -exec {} \;
Your Linux or macOS computer is using virtual memory. Discover how it’s affecting your system’s use of physical memory, CPU, and hard disk resources.
What Is Virtual Memory?
Your computer is fitted with a finite amount of physical memory called random access memory (RAM). This RAM needs to be managed by the kernel and shared between the operating system and whatever applications happen to be running. If these combined demands are asking for more memory than is physically installed in your computer, what can the kernel do?
Linux and Unix-like operating systems such as macOS can use space on your hard disk to help them manage memory demands. A reserved area of hard drive space called “swap space” can be used as though it were an extension of RAM. This is virtual memory.
The Linux kernel can write the contents of a block of memory into swap space, and free up that region of RAM for use by another process. The swapped out—also called “paged” out—memory can be retrieved from the swap space and restored to RAM when it is required.
Of course, the speed of access for paged out memory is slower than that of memory held in RAM. And that’s not the only trade-off. Whilst virtual memory does provide a way for Linux to manage its memory demands, using virtual memory places increased burdens elsewhere on the computer.
Your hard drive must perform more reads and writes. The kernel—and hence, the CPU—must do more work as it swaps memory out, swaps memory in, and keeps all the plates spinning to satisfy the memory needs of the different processes.
Linux provides a way for you to monitor all of this activity in the shape of the
vmstat command, which reports on virtual memory statistics.The vmstat Command
If you type
vmstat as a command with no parameters, it will show you a set of values. These values are the averages for each of the statistics since your computer was last rebooted. These figures are not a snapshot of the values “right now.”vmstat
A short table of values is displayed.
There are columns headed Procs, Memory, Swap, IO, System, and CPU. The final column (most right-hand column) contains the data relating to the CPU.
Here is a list of the data items in each column.
Proc
- r: The number of runnable processes. These are processes that have been launched and are either running or are waiting for their next time-sliced burst of CPU cycles.
- b: The number of processes in uninterruptible sleep. The process isn’t sleeping, it is performing a blocking system call, and it cannot be interrupted until it has completed its current action. Typically the process is a device driver waiting for some resource to come free. Any queued interrupts for that process are handled when the process resumes its usual activity.
Memory
- swpd: the amount of virtual memory used. In other words, how much memory has been swapped out.,
- free: the amount of idle (currently unused) memory.
- buff: the amount of memory used as buffers.
- cache: the amount of memory used as cache.
Swap
- si: Amount of virtual memory swapped in from swap space.
- so: Amount of virtual memory swapped out to swap space.
IO
- bi: Blocks received from a block device. The number of data blocks used to swap virtual memory back into RAM.
- bo: Blocks sent to a block device. The number of data blocks used to swap virtual memory out of RAM and into swap space.
System
- in: The number of interrupts per second, including the clock.
- cs: The number of context switches per second. A context switch is when the kernel swaps from system mode processing into user mode processing.
CPU
These values are all percentages of the total CPU time.
- us: Time spent running non-kernel code. That is, how much time is spent in user time processing and in nice time processing.
- sy: Time spent running kernel code.
- id: Time spent idle.
- wa: Time spent waiting for input or output.
- st: Time stolen from a virtual machine. This is the time a virtual machine has to wait for the hypervisor to finish servicing other virtual machines before it can come back and attend to this virtual machine.
Using a Time Interval
We can have
vmstat provide regular updates to these figures by using a delay value. The delay value is provided in seconds. To have the statistics updated every five seconds, we’d use the following command:vmstat 5
Every five seconds
vmstat will add another line of data to the table. You’ll need to hit Ctrl+C to stop this.
Using a Count Value
Using too low a
delay value will put additional strain on your system. If you need to have rapid updates to try to diagnose a problem, it is recommended that you use a count value as well as a delay value.
The
count value tells vmstat how many updates to perform before it exits and returns you to the command prompt. If you do not provide a count value, vmstat will run until it is stopped by Ctrl+C.
To have
vmstat provide an update every five seconds—but only for four updates—use the following command:vmstat 5 4
After four updates
vmstat stops of its own accord.
Changing the Units
You can choose to have the memory and swap statistics displayed in kilobytes or megabytes using the
-S (unit-character) option. This must be followed by one of k , K , m, or M. These represent:- k:1000 bytes
- K: 1024 bytes
- m: 1000000 bytes
- M: 1048576 bytes
To have the statistics updated every 10 seconds with the memory and swap statistics displayed in megabytes, use the following command:
vmstat 10 -S M
The memory and swap statistics are now shown in megabytes. Note that the
-S option does not affect the IO block statistics. These are always displayed in blocks.
Active and Inactive Memory
If you use the
-a (active) option the buff and cache memory columns are replaced by the “inact” and “active” columns. As they would suggest, these show the amount of inactive and active memory.
To see these two columns instead of the buff and cache columns, include the
-a option, as shown:vmstat 5 -a -S M
The inact and active columns are affected by the -S (unit-character) option.
Forks
The
-f switch displays the number of forks that have happened since the computer was booted up.
In other words, this shows the number of tasks that have been launched (and, for the bulk of them, closed again) since the system was booted. Every process launched from the command line would increase this figure. Each time a task or process spawns or clones a new task, this figure will increase.
vmstat -f
The forks display does not update.
Displaying Slabinfo
The kernel has its own memory management to worry about as well as the memory management for the operating system and all of the applications.
As you might imagine the kernel is allocating and deallocating memory over and over for the many different types of data object that it must handle. To make this as efficient as possible, it uses a system called slabs. This is a form of caching.
Memory allocated, used, and no longer required for a specific type of kernel data object can be re-used for another data object of the same type without the memory being deallocated and reallocated. Think of slabs as pre-allocated, made to measure, segments of RAM for the kernel’s own needs.
To see the statistics for the slabs, use the
-m (slabs) option. You will need to use sudo, and you will be prompted for your password. As the output can be quite lengthy, we are piping it through less.sudo vmstat -m | less
The output has five columns. These are:
- Cache: Name of the cache.
- num: The number of currently active objects in this cache.
- total: The total number of available objects in this cache.
- size: The size of each object in the cache.
- pages: The total number of memory pages that have (at least) one object currently associated with this cache.
Press
q to leave less.Displaying Event Counters and Memory Statistics
To display a page of event counters and memory statistics, use the
-s (stats) option. Note that’s a lowercase “s.”vmstat -s
Although the statistics that are reported are largely the same as the information that makes up the default
vmstat output, some of them are split out in more detail.
For example, the default output combines both the nice and the non-nice user CPU time into the “us” column. The -s (stats) display lists these statistics separately.
Displaying Disk Statistics
You can obtain a similar listing of disk statistics using the
-d (disk) option.vmstat -d | less
For each disk, three columns are displayed, these are Reads, Writes, and IO.
IO is the rightmost column. Note that the sec column in IO is measured in seconds but the time-based statistics in the read and write columns are measured in milliseconds.
This is what the columns mean:
Reads
- total: The total count of disk reads.
- merged: The total count of grouped reads.
- sectors: The total count of sectors that have been read in.
- ms: Total count of time in milliseconds that were used reading data from the disk.
writes
- total: The total count of disk writes.
- merged: The total count of grouped writes.
- sectors: The total count of sectors written to.
- ms = Total count of time in milliseconds that were used writing data to the disk.
IO
- cur: Number of current disk reads or writes.
- sec: Time spent in seconds for any in-progress reads or writes.
Displaying Summary Disk Statistics
To see a quick display of summary statistics for your disk activity, use the
-D (disk-sum) option. Note the uppercase “D.”vmstat -D
The number of disks might look abnormally high. The computer used to research this article is running Ubuntu. With Ubuntu, each time you install an application from a Snap, a
squashfs pseudo-filesystem is created which is attached to a /dev/loop device.
Annoyingly these device entries are counted as hard drive devices by many of the Linux commands and utilities.
Displaying Partition Statistics
To see statistics related to a specific partition, use the
-p (partition) option and provide the partition identifier as a command line parameter.
Here we are going to look at the partition
sda1. The digit one indicates this is the first partition on device sda, which is the main hard drive for this computer.vmstat -p sda1
The information returned shows the total count of disk reads and disk writes to and from that partition, and the number of sectors included in disk read and disk write actions.
A Peek Under The Hood
It’s always good to know how to lift the hood and see what’s going on underneath. Sometimes you’ll be trying to problem solve, sometimes it’ll be out of interest because you want to know how your computer ticks.
vmstat can provide you with a ton of useful information. Now you know how to access it and what it means. And forewarned is forearmed—when you do need to roll your sleeves up and do some diagnostics, you’ll know you’ve got vmstat on your side.10+ “vmstat” Command Usage Examples in Linux
This tutorial explains Linux “vmstat” command, options and its usage with examples.
vmstat – Virtual Memory Statistics
Description:
Description:
vmstat command is used to report virtual memory statistics. vmstat reports information about processes, memory, paging, block IO, traps, and cpu activity.
Synopsis:
vmstat [-a] [-n] [-t] [-S unit] [delay [ count]] vmstat [-s] [-n] [-S unit] vmstat [-m] [-n] [delay [ count]] vmstat [-d] [-n] [delay [ count]] vmstat [-p disk partition] [-n] [delay [ count]] vmstat [-f] vmstat [-V]
vmstat [-a] [-n] [-t] [-S unit] [delay [ count]] vmstat [-s] [-n] [-S unit] vmstat [-m] [-n] [delay [ count]] vmstat [-d] [-n] [delay [ count]] vmstat [-p disk partition] [-n] [delay [ count]] vmstat [-f] vmstat [-V]
OPTIONS:
-a
switch displays active/inactive memory, given a 2.5.41 kernel or better.
-f
switch displays the number of forks since boot. This includes the fork, vfork, and clone system calls, and is equivalent to the total number of tasks created. Each process is represented by one or more tasks, depending on thread usage. This display does not repeat.
-t
switch adds timestamp to the output.
-m
switch displays slabinfo.
-n
switch causes the header to be displayed only once rather than periodically.
-s
switch displays a table of various event counters and memory statistics. This display does not repeat.
delay
delay between updates in seconds. If no delay is specified, only one report is printed with the average values since boot.
count
number of updates. If no count is specified and delay is defined, count defaults to infinity.
-d
reports disk statistics (2.5.70 or above required)
-w
enlarges field width for big memory sizes
-p
followed by some partition name for detailed statistics (2.5.70 or above required)
-S
followed by k or K or m or M switches outputs between 1000, 1024, 1000000, or 1048576 bytes
-V
switch results in displaying version information.
switch displays active/inactive memory, given a 2.5.41 kernel or better.
-f
switch displays the number of forks since boot. This includes the fork, vfork, and clone system calls, and is equivalent to the total number of tasks created. Each process is represented by one or more tasks, depending on thread usage. This display does not repeat.
-t
switch adds timestamp to the output.
-m
switch displays slabinfo.
-n
switch causes the header to be displayed only once rather than periodically.
-s
switch displays a table of various event counters and memory statistics. This display does not repeat.
delay
delay between updates in seconds. If no delay is specified, only one report is printed with the average values since boot.
count
number of updates. If no count is specified and delay is defined, count defaults to infinity.
-d
reports disk statistics (2.5.70 or above required)
-w
enlarges field width for big memory sizes
-p
followed by some partition name for detailed statistics (2.5.70 or above required)
-S
followed by k or K or m or M switches outputs between 1000, 1024, 1000000, or 1048576 bytes
-V
switch results in displaying version information.
Examples:
1. Simple example along-with field descriptions
$ vmstat procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------ r b swpd free buff cache si so bi bo in cs us sy id wa st 0 0 305416 260688 29160 2356920 2 2 4 1 0 0 6 1 92 2 0
Field Description For Vm Mode
Procs
r: The number of processes waiting for run time.
b: The number of processes in uninterruptible sleep.
b: The number of processes in uninterruptible sleep.
Memory
swpd: the amount of virtual memory used.
free: the amount of idle memory.
buff: the amount of memory used as buffers.
cache: the amount of memory used as cache.
inact: the amount of inactive memory. (-a option)
active: the amount of active memory. (-a option)
free: the amount of idle memory.
buff: the amount of memory used as buffers.
cache: the amount of memory used as cache.
inact: the amount of inactive memory. (-a option)
active: the amount of active memory. (-a option)
Swap
si: Amount of memory swapped in from disk (/s).
so: Amount of memory swapped to disk (/s).
so: Amount of memory swapped to disk (/s).
IO
bi: Blocks received from a block device (blocks/s).
bo: Blocks sent to a block device (blocks/s).
bo: Blocks sent to a block device (blocks/s).
System
in: The number of interrupts per second, including the clock.
cs: The number of context switches per second.
cs: The number of context switches per second.
CPU
These are percentages of total CPU time.
us: Time spent running non-kernel code. (user time, including nice time)
sy: Time spent running kernel code. (system time)
id: Time spent idle. Prior to Linux 2.5.41, this includes IO-wait time.
wa: Time spent waiting for IO. Prior to Linux 2.5.41, included in idle.
st: Time stolen from a virtual machine. Prior to Linux 2.6.11, unknown.
us: Time spent running non-kernel code. (user time, including nice time)
sy: Time spent running kernel code. (system time)
id: Time spent idle. Prior to Linux 2.5.41, this includes IO-wait time.
wa: Time spent waiting for IO. Prior to Linux 2.5.41, included in idle.
st: Time stolen from a virtual machine. Prior to Linux 2.6.11, unknown.
2. To execute every 2 seconds for 10 times, do the following. You don’t need to press Ctrl-C in this case. After executing 10 times, it will stop automatically.
$ vmstat 2 10 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 1 0 0 537144 182736 6789320 0 0 0 0 1 1 0 0 100 0 0 0 0 0 537004 182736 6789320 0 0 0 0 50 32 0 0 100 0 0 ..
3. Using vmstat, this next command will grep the memory and give a good overview of the memory used in the computer.
$ vmstat -s -S M | grep mem 5965 M total memory 5136 M used memory 3580 M active memory 1176 M inactive memory 829 M free memory 474 M buffer memory
4. Display active and inactive memory
By default vmstat doesn’t display this information. Use option -a, to display active and inactive memory information as shown below.
$ vmstat -a procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------ r b swpd free inact active si so bi bo in cs us sy id wa st 0 0 305416 253820 1052680 2688928 2 2 4 1 0 0 6 1 92 2 0
4. Display number of forks since last boot
This displays all the fork system calls made by the system since the last boot. This displays all fork, vfork, and clone system call counts.
$ vmstat -f 81651975 forks
5. Display timestamp
When you use vmstat to monitor the memory usage repeately, it would be nice to see the timestap along with every line item. Use option -t to display the time stamp as shown below.
$ vmstat -t 1 100 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------ ---timestamp--- r b swpd free buff cache si so bi bo in cs us sy id wa st 0 0 0 3608728 148368 3898200 0 0 0 0 1 1 0 0 100 0 0 2011-07-09 21:16:28 PDT 0 0 0 3608728 148368 3898200 0 0 0 0 60 15 0 0 100 0 0 2011-07-09 21:16:29 PDT 0 0 0 3608712 148368 3898200 0 0 0 0 32 28 0 0 100 0 0 2011-07-09 21:16:30 PDT
6. Display slab info
Use option -m, to display the slab info as shown below.
$ vmstat -m Cache Num Total Size Pages fib6_nodes 5 113 32 113 ip6_dst_cache 4 15 256 15 ndisc_cache 1 15 256 15 RAWv6 7 10 768 5 UDPv6 0 0 640 6 tw_sock_TCPv6 0 0 128 30 ...
7. Display statistics in a table format
Instead of displays the values in the record format, you can display the output of vmstat in table format using option -s as shown below.
$ vmstat -s 4149928 total memory 3864824 used memory 2606664 active memory 1098180 inactive memory 285104 free memory 19264 buffer memory 2326692 swap cache 4192956 total swap 274872 used swap 3918084 free swap 1032454000 non-nice user cpu ticks 14568 nice user cpu ticks 89482270 system cpu ticks 16674327143 idle cpu ticks 368965706 IO-wait cpu ticks 1180468 IRQ cpu ticks ..
8. Display disk statistics
Use option -d to display the disk statistics as shown below. This displays the reads, writes, and I/O statistics of the disk.
$ vmstat -d disk- ------------reads------------ ------------writes----------- -----IO------ total merged sectors ms total merged sectors ms cur sec sda 153189971 69093708 2719150864 737822879 329617713 157559204 3965687592 4068577985 0 1102243 sdb 501426305 97099356 2345472425 731613156 419220973 533565961 2661869460 1825174087 0 1510434 sdc 884213459 22078974 513390701 452540172 127474901 8993357 2411187300 2133226954 0 1569758
9. Changing width of output
The default output without increasing the width is shown below.
The default output without increasing the width is shown below.
$ vmstat 1 3 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 0 0 0 3608688 148368 3898204 0 0 0 0 1 1 0 0 100 0 0 0 0 0 3608804 148368 3898204 0 0 0 0 72 30 0 0 100 0 0 0 0 0 3608804 148368 3898204 0 0 0 0 60 27 0 0 100 0 0
Use option -w to increase the width of the output columns as shown below. This give better readability.
$ vmstat -w 1 3 procs -------------------memory------------------ ---swap-- -----io---- --system-- -----cpu------- r b swpd free buff cache si so bi bo in cs us sy id wa st 0 0 0 3608712 148368 3898204 0 0 0 0 1 1 0 0 100 0 0 0 0 0 3608712 148368 3898204 0 0 0 0 93 23 0 0 100 0 0 0 0 0 3608696 148368 3898204 0 0 0 0 35 34 0 0 100 0 0
10. Display statistics for a partition
To display the disk I/O statistics of a specific disk partition use option -p as shown below.
$ vmstat -p sdb1 sdb1 reads read sectors writes requested writes 501423248 2345417917 419221612 2661885948
11. Display in MB
By default vmstat displays the memory information in kb. To disply in MB, use the option “-S m” as shown below.
$ vmstat -S m procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------ r b swpd free buff cache si so bi bo in cs us sy id wa st 0 0 281 288 19 2386 0 0 4 1 0 0 6 1 92 2 0
How to read Vmstat output
Environment
- Red Hat Enterprise Linux
Issue
- How i can read vmstat output
Resolution
vmstat (virtual memory statistics) is a valuable monitoring utility, which also provides information about block IO and CPU activity in addition to memory.
vmstat Basics
vmstat provides a number of values and will typically be called using two numerical parameters.
Example: vmstat 1 5
1 -> the values will be re-measured and reported every second
5 -> the values will be reported five times and then the program will stop
The first line of the report will contain the average values since the last time the computer was rebooted. All other lines in the report will represent their respective current values. Vmstat does not need any special user rights. It can run as a normal user.
[user@fedora9 ~]$ vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
3 0 0 44712 110052 623096 0 0 30 28 217 888 13 3 83 1 0
0 0 0 44408 110052 623096 0 0 0 0 88 1446 31 4 65 0 0
0 0 0 44524 110052 623096 0 0 0 0 84 872 11 2 87 0 0
0 0 0 44516 110052 623096 0 0 0 0 149 1429 18 5 77 0 0
0 0 0 44524 110052 623096 0 0 0 0 60 431 14 1 85 0 0
[user@fedora9 ~]$
Meaning of the individual Values
(Source man vmstat):
Procs
r: The number of processes waiting for run time.
b: The number of processes in uninterruptible sleep.
Memory
swpd: the amount of virtual memory used.
free: the amount of idle memory.
buff: the amount of memory used as buffers.
cache: the amount of memory used as cache.
inact: the amount of inactive memory. (-a option)
active: the amount of active memory. (-a option)
Swap
si: Amount of memory swapped in from disk (/s).
so: Amount of memory swapped to disk (/s).
IO
bi: Blocks received from a block device (blocks/s).
bo: Blocks sent to a block device (blocks/s).
System
in: The number of interrupts per second, including the clock.
cs: The number of context switches per second.
CPU
These are percentages of total CPU time.
us: Time spent running non-kernel code. (user time, including nice time)
sy: Time spent running kernel code. (system time)
id: Time spent idle. Prior to Linux 2.5.41, this includes IO-wait time.
wa: Time spent waiting for IO. Prior to Linux 2.5.41, included in idle.
st: Time stolen from a virtual machine. Prior to Linux 2.6.11, unknown.
Examples
CPU User Load Example
A standard audio file will be encode as an MP3 file by means of the lame encoder[1] in this example. This process is quite CPU intensive and also demonstrates the execution of vmstat in parallel with a user CPU time of 97%:
[user@RHEL ~]$ vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
6 1 0 302380 16356 271852 0 0 561 568 80 590 43 7 43 7 0
1 0 0 300892 16364 273256 0 0 0 52 79 274 97 3 0 0 0
2 0 0 299544 16364 274532 0 0 0 0 78 372 97 3 0 0 0
1 0 0 298292 16368 275780 0 0 0 0 53 255 97 3 0 0 0
1 0 0 296820 16368 277192 0 0 0 0 77 377 97 3 0 0 0
[user@RHEL ~]$
CPU System Load Example
In this example, a file will be filled with random content using dd.
[user@fedora9 ~]$ dd if=/dev/urandom of=500MBfile bs=1M count=500
For this, /dev/urandom[2] will supply random numbers, which will be generated by the kernel. This will lead to an increased load on the CPU (sy – system time). At the same time, the vmstat executing in parallel will indicate that between 93% and 97% of the CPU time is being used for the execution of kernel code (for the generation of random numbers, in this case).
[user@RHEL ~]$ vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
2 0 402944 54000 161912 745324 5 14 54 59 221 867 13 3 82 2 0
1 0 402944 53232 161916 748396 0 0 0 0 30 213 3 97 0 0 0
1 0 402944 49752 161920 751452 0 0 0 0 28 290 4 96 0 0 0
1 0 402944 45804 161924 755564 0 0 0 0 29 188 2 98 0 0 0
1 0 402944 42568 161936 758608 0 0 0 17456 272 509 7 93 0 0 0
[user@RHEL ~]$
The time for executing system calls[3][4][5] will be counted as system time (sy).
RAM Bottleneck (swapping) Example
In this example, many applications will be opened (including VirtualBox with a Windows guest system, among others). Almost all of the working memory will be used. Then, one more application (OpenOffice) will be started. The Linux kernel will then swap out several memory pages to the swap file on the hard disk, in order to get more RAM for OpenOffice. Swapping the memory pages to the swap file will be seen in the so (swap out - memory swapped to disk) column as vmstat executes in parallel.
[user@RHEL ~]$ vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
3 1 244208 10312 1552 62636 4 23 98 249 44 304 28 3 68 1 0
0 2 244920 6852 1844 67284 0 544 5248 544 236 1655 4 6 0 90 0
1 2 256556 7468 1892 69356 0 3404 6048 3448 290 2604 5 12 0 83 0
0 2 263832 8416 1952 71028 0 3788 2792 3788 140 2926 12 14 0 74 0
0 3 274492 7704 1964 73064 0 4444 2812 5840 295 4201 8 22 0 69 0
[user@RHEL ~]$
High IO Read Load Example
A large file (such as an ISO file) will be read and written to /dev/null using dd.
[user@RHEL ~]$ dd if=bigfile.iso of=/dev/null bs=1M
Executed in parallel, vmstat will show the increased IO read load (the bi value).
[user@RHEL ~]$ vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
3 1 465872 36132 82588 1018364 7 17 70 127 214 838 12 3 82 3 0
0 1 465872 33796 82620 1021820 0 0 34592 0 357 781 6 10 0 84 0
0 1 465872 36100 82656 1019660 0 0 34340 0 358 723 5 9 0 86 0
0 1 465872 35744 82688 1020416 0 0 33312 0 345 892 8 11 0 81 0
0 1 465872 35716 82572 1020948 0 0 34592 0 358 738 7 8 0 85 0
[user@RHEL ~]$
High IO Write Load Example
In contrast with the previous example, dd will read from /dev/zero and write a file. The oflag=dsync will cause the data to be written immediately to the disk (and not merely stored in the page cache).
[user@RHEL ~]$ dd if=/dev/zero of=500MBfile bs=1M count=500 oflag=dsync
Executed in parallel, vmstat will show the increased IO write load (the bo value).
[user@RHEL ~]$ vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
1 1 0 35628 14700 1239164 0 0 1740 652 117 601 11 4 66 20 0
0 1 0 34852 14896 1239788 0 0 0 23096 300 573 3 16 0 81 0
0 1 0 32780 15080 1241304 0 0 4 21000 344 526 1 13 0 86 0
0 1 0 36512 15244 1237256 0 0 0 19952 276 394 1 12 0 87 0
0 1 0 35688 15412 1237180 0 0 0 18904 285 465 1 13 0 86 0
[user@RHEL ~]$
CPU Waiting for IO Example
In the following example, an updatedb process is already running. The updatedb utility is part of mlocate. It examines the entire file system and accordingly creates the database for the locate command (by means of which file searches can be performed very quickly). Because updatedb reads all of the file names from the entire file system, the CPU must wait to get data from the IO system (the hard disk). For that reason, vmstat running in parallel will display large values for wa (waiting for IO):
[user@RHEL ~]$ vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
2 1 403256 602848 17836 400356 5 15 50 50 207 861 13 3 83 1 0
1 0 403256 601568 18892 400496 0 0 1048 364 337 1903 5 7 0 88 0
0 1 403256 600816 19640 400568 0 0 748 0 259 1142 6 4 0 90 0
0 1 403256 600300 20116 400800 0 0 476 0 196 630 8 5 0 87 0
0 1 403256 599328 20792 400792 0 0 676 0 278 1401 7 5 0 88 0
[user@RHEL ~]$
Additional vmstat Options
vmstat --help
[user@RHEL ~]$ vmstat --help
usage: vmstat [-V] [-n] [delay [count]]
-V prints version.
-n causes the headers not to be reprinted regularly.
-a print inactive/active page stats.
-d prints disk statistics
-D prints disk table
-p prints disk partition statistics
-s prints vm table
-m prints slabinfo
-S unit size
delay is the delay between updates in seconds.
unit size k:1000 K:1024 m:1000000 M:1048576 (default is K)
count is the number of updates.
vmstat
[user@fedora9 ~]$ vmstat
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
2 0 14960 38016 6584 1069284 0 1 506 81 727 1373 12 4 81 3 0
[user@fedora9 ~]$
vmstat -V
[user@fedora9 ~]$ vmstat -V
procps version 3.2.7
[user@fedora9 ~]$
vmstat -a
[user@fedora9 ~]$ vmstat -a
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free inact active si so bi bo in cs us sy id wa st
3 0 14960 38024 988284 461704 0 1 506 81 726 1372 12 4 81 3 0
[user@fedora9 ~]$
vmstat -d
[user@fedora9 ~]$ vmstat -d
disk- ------------reads------------ ------------writes----------- -----IO------
total merged sectors ms total merged sectors ms cur sec
ram0 0 0 0 0 0 0 0 0 0 0
ram1 0 0 0 0 0 0 0 0 0 0
ram2 0 0 0 0 0 0 0 0 0 0
ram3 0 0 0 0 0 0 0 0 0 0
ram4 0 0 0 0 0 0 0 0 0 0
ram5 0 0 0 0 0 0 0 0 0 0
ram6 0 0 0 0 0 0 0 0 0 0
ram7 0 0 0 0 0 0 0 0 0 0
ram8 0 0 0 0 0 0 0 0 0 0
ram9 0 0 0 0 0 0 0 0 0 0
ram10 0 0 0 0 0 0 0 0 0 0
ram11 0 0 0 0 0 0 0 0 0 0
ram12 0 0 0 0 0 0 0 0 0 0
ram13 0 0 0 0 0 0 0 0 0 0
ram14 0 0 0 0 0 0 0 0 0 0
ram15 0 0 0 0 0 0 0 0 0 0
sda 136909 31536 13893867 1197609 58190 219323 2233264 7688807 0 677
sda1 35703 6048 1326394 511477 6728 16136 182984 419232 0 222
sda2 85 1489 2935 653 141 3603 29952 5254 0 1
sda3 101111 23961 12564154 685330 51321 199584 2020328 7264321 0 512
sr0 0 0 0 0 0 0 0 0 0 0
fd0 0 0 0 0 0 0 0 0 0 0
[user@fedora9 ~]
vmstat -D
[user@fedora9 ~]$ vmstat -D
22 disks
0 partitions
273820 total reads
63034 merged reads
27787446 read sectors
2395193 milli reading
116450 writes
438666 merged writes
4467248 written sectors
15377932 milli writing
0 inprogress IO
1412 milli spent IO
vmstat -p
vmstat -p will not work under Fedora: https://bugzilla.redhat.com/show_bug.cgi?id=485246. The following report comes from an Ubuntu 9.10 system.
user@RHEL:~$ vmstat -p /dev/sda9
sda9 reads read sectors writes requested writes
23420 411365 24464 530801
vmstat -s
[user@fedora9 ~]$ vmstat -s
1553972 total memory
1516180 used memory
461892 active memory
988304 inactive memory
37792 free memory
6644 buffer memory
1069388 swap cache
1052248 total swap
14960 used swap
1037288 free swap
161467 non-nice user cpu ticks
7586 nice user cpu ticks
46310 system cpu ticks
1108919 idle cpu ticks
46832 IO-wait cpu ticks
2694 IRQ cpu ticks
2452 softirq cpu ticks
0 stolen cpu ticks
6947021 pages paged in
1116896 pages paged out
183 pages swapped in
3744 pages swapped out
9985406 interrupts
18852586 CPU context switches
1239004323 boot time
15072 forks
[user@fedora9 ~]$
vmstat -m
[user@fedora9 ~]$ vmstat -m
Cache Num Total Size Pages
fuse_request 11 11 368 11
fuse_inode 9 9 448 9
rpc_inode_cache 8 8 512 8
nf_conntrack_expect 0 0 168 24
nf_conntrack 26 80 248 16
dm_uevent 0 0 2464 3
UDPv6 22 22 704 11
TCPv6 6 6 1344 6
kmalloc_dma-512 8 8 512 8
sgpool-128 12 12 2048 4
scsi_io_context 0 0 104 39
ext3_inode_cache 6822 8360 496 8
ext3_xattr 85 85 48 85
journal_handle 170 170 24 170
journal_head 76 219 56 73
revoke_record 256 256 16 256
flow_cache 0 0 80 51
bsg_cmd 0 0 288 14
mqueue_inode_cache 7 7 576 7
isofs_inode_cache 0 0 376 10
hugetlbfs_inode_cache 11 11 344 11
dquot 0 0 128 32
shmem_inode_cache 1058 1071 448 9
xfrm_dst_cache 0 0 320 12
UDP 19 21 576 7
TCP 17 24 1216 6
blkdev_queue 21 21 1080 7
biovec-256 2 2 3072 2
biovec-128 5 5 1536 5
biovec-64 7 10 768 5
sock_inode_cache 619 650 384 10
file_lock_cache 39 39 104 39
Acpi-Operand 2935 2958 40 102
Acpi-Namespace 1700 1700 24 170
Cache Num Total Size Pages
taskstats 25 26 312 13
proc_inode_cache 233 242 360 11
sigqueue 28 28 144 28
radix_tree_node 7888 8606 296 13
bdev_cache 24 24 512 8
inode_cache 370 462 344 11
dentry 6592 15390 136 30
names_cache 2 2 4096 2
avc_node 73 73 56 73
selinux_inode_security 9888 10030 48 85
idr_layer_cache 627 644 144 28
buffer_head 2308 2688 64 64
mm_struct 659 693 448 9
vm_area_struct 11110 11592 88 46
files_cache 115 130 384 10
sighand_cache 141 150 1344 6
task_struct 246 248 3696 2
anon_vma 4778 5120 16 256
kmalloc-4096 95 112 4096 8
kmalloc-2048 272 304 2048 16
kmalloc-1024 518 524 1024 4
kmalloc-512 764 888 512 8
kmalloc-256 198 208 256 16
kmalloc-128 629 832 128 32
kmalloc-64 4322 5568 64 64
kmalloc-32 1554 1664 32 128
kmalloc-16 2644 3584 16 256
kmalloc-8 3561 3584 8 512
kmalloc-192 6349 6930 192 21
kmalloc-96 885 1176 96 42
[user@fedora9 ~]$