Back to the main page

SAR (system activity reporter) on Solaris

As someone who's responsible for running and performance of servers, you'll probably one day find yourself in situation where you need to explain that your server is okay (if you are lucky), but performance problem is most likely within network or users desktop. The tool that can help you is SAR (system activity reporter).

It originates from Sun and basically gathers system activity and creates periodic reports. Run it from cronjob. On Solaris, there is predefined cronjob file /var/spool/cron/crontabs/sys, so you see that SAR is run through cronjob by user sys.

Solaris packages are:
# pkginfo | grep SUNWacc
system      SUNWaccr                         System Accounting, (Root)
system      SUNWaccu                         System Accounting, (Usr)

The SAR works in two steps.
The cronjob for sys user (file /var/spool/cron/crontabs/sys ) can for example look like:

# collects 1 sample every 120 sec (2 min) and write in file /var/adm/sa/sa`/usr/bin/date +%d`
* * * * * /usr/lib/sa/sa1 120 1
# creates report with interval (-i) of 900 sec (15 min) using all data (-A)
45 23 * * * /usr/lib/sa/sa2 -i 900 -A

Now when you have report you want to view if using some kind of graphs.
I've tried the kSar (version 5.0.6) so for example, a PDF report can be created.

Here are notes (from kSar docs) to better understand Solaris graphs:

CPU (sar -u)

The CPU spends time doing one of 4 things: So if CPU idle graph is at 0 for a long period of time, this probably means that your host needs to have more CPU power, but also look at the time spent in system mode.
If you are running NFS server and your system time is very high, maybe you could have a look at your NFS parameter and try to tune some parameters to get things running better.
If you got many waiting I/O you should have a look at the disks performance to see if your box is really wasting time on disk utilization. It was said that waiting I/O has been disabled on Solaris 10.

Disk (Disk Transfer and Disk Wait) (sar -d)

Disk has these two pages.

Disk Transfer has three graphs about data transferred from/to the disk. Disk Wait has statistics about queue and waiting I/O statistics.

Run Queue (sar -q)


The Swapping panel report information about lightweight process (LWP).
A LWP run in user space on top of kernel thread and shares its address space with other LWPs.
The graph report the number of 512 bytes pages swapped out to disk or swapped in to memory and the number of swap request in and out. There is also a graph reporting the number of LWP switch done by the CPU (review together with the 'Run Queue Size' graph).

Swap Queue (sar -w)

The Swap Queue is pretty always at zero, if you got some value, then you got a problem or you have had a problem.
In typical situation the swap queue is only used when there is big memory exhaustion.
The swpq-sz (swap queue size) is the number of process the system had to swap to the disk for freeing some memory. To find out which process has been swapped to the disk, you can search for process where rss (resident set size) is 0 either with "prstat" or "ps -efly".

Buffers (sar -b)

The Buffers has 4 graphs.

Syscalls (sar -c)

The Syscalls has 4 graphs.

File Access activity (sar -a)

The File panel report information about file-system utilization

TTY (Terminal) activity (sar -y)

This reports TTY activity per second.

Messages & Sempaphores

It shows shared memory activity. Note that Oracle is using semaphore a lot.

Paging (first page, page-OUT activity) (sar -g)

This Paging panel has four graphs.

Paging (second page, page-IN activity) (sar -p)

This panel has four graphs.

Memory Usage (sar -r)

The Memory usage panel has two graphs.

Kernel Memory Allocation (sar -k)

The KMA allows a kernel subsystem to allocate and free memory as needed.
Rather than statically allocating the maximum amount of memory it is expected to require under peak load, the KMA divides requests for memory into three categories: small (less than 256 bytes), large (512 to 4 Kbytes), and oversized (greater than 4 Kbytes).
It keeps two pools of memory to satisfy small and large requests. The oversized requests are satisfied by allocating memory from the system page allocator.

Back to the main page