If a system's time is adjusted backwards, then elapsed
time could appear as negative. This yielded a negative
%CPU value. Alternately if zeros were suppressed ('0')
the result was a blank %CPU column. In both cases that
distortion would last for one display cycle or until a
user forced a display refresh via some keyboard input.
The original recommendation was trading gettimeofday()
for clock_gettime() using CLOCK_MONOTONIC. But on some
systems that might not be possible, forcing the use of
CLOCK_REALTIME instead. Not only would that complicate
the build system, but it may leave us with minus %CPU.
Another approach was to ensure that elapsed time could
never be negative. Of course, this produced distortion
of %CPU values but it would be proportionally correct.
This wasn't dissimilar to a distortion already present
should the time be adjusted forward or backward within
any 'remaining' top delay intervals. These aberrations
would be avoided with clock_gettime & CLOCK_MONOTONIC,
but that is a less than ideal solution as noted above.
This final solution, which originated down under, will
simply rely on the /proc/uptime seconds, which will be
immune to *any* tampering with the system clock. Thus,
we now have a fix for the distortion we didn't know we
suffered plus a negative %CPU that began this odyssey.
Thanks to:
sk.alvin.x@gmail.com, for the original effort
jcapik@redhat.com, for a heads up on CLOCK_MONOTONIC
csmall-procps@enc.com.au, for the best suggestion of all
Reference(s):
. original post/patch
http://www.freelists.org/post/procps/PATCH-top-use-clock-gettime-instead-of-gettimeofday
. heads up on CLOCK_MONOTONIC
http://www.freelists.org/post/procps/PATCH-top-use-clock-gettime-instead-of-gettimeofday,2
. the final solution
http://www.freelists.org/post/procps/PATCH-top-use-clock-gettime-instead-of-gettimeofday,11
Signed-off-by: Jim Warner <james.warner@comcast.net>
I ran into this limit with net.ipv4.ip_local_reserved_ports ,
sysctl complained about the line after the long line, further
slowing down my error hunting.
Due to fgets usage, increase buffer size to 4096 chars with
minimum amount of code changes.
Signed-off-by: Sami Farin <hvtaifwkbgefbaei@gmail.com>
Minor fixes that the translator (Yuri) has found in some of the
strings. You only know how many typos and thinkos you have when
someone is trying to translate it.
Signed-off-by: Craig Small <csmall@enc.com.au>
tmpfs has become much more widely used since distributions use it for
/tmp (Fedora 18+). In /proc/meminfo, memory used by tmpfs is accounted
into "Cached" (aka "NR_FILE_PAGES",
http://lxr.free-electrons.com/source/mm/shmem.c#L301 ).
The tools just pass it on, so what top, free and vmstat report as
"cached" is the sum of page cache and tmpfs.
free has the extremely useful "-/+ buffers/cache" output. However, now
that tmpfs is accounted into "cached", those numbers are way off once
you have big files in /tmp.
Fortunately, kernel 2.6.32 introduces "Shmem", which makes tmpfs memory
usage accessible from userspace (
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=4b02108ac1b3354a22b0d83c684797692efdc395 ).
This patch substracts Shmem from Cached to get the actual page cache
memory. This makes both issues mentioned above disappear. For older
kernels, Shmem is not available (hence zero) and this patch is no-op.
Additionally:
* Update the man pages of free and vmstat to explain what is happening
* Finally drop "MemShared" from the /proc/meminfo parser, it has been
dead for 10+ years and is only causing confusion ( removed in kernel
2.5.54, see
https://git.kernel.org/cgit/linux/kernel/git/tglx/history.git/commit/?id=fe04e9451e5a159247cf9f03c615a4273ac0c571 )
Since its inception top has always used enumerators to
identify displayable fields. They've taken the form of
P_PID, etc. As it turns out, something has changed for
libc6-dev versions beyond 2.17-93 wherein 'P_PID' will
now be exposed via stdlib.h. I have not pinpointed the
exact cause but it may depend on header include order.
This patch just trades top's long standing 'P_' prefix
convention for that of 'EU_' (short for enumerator). I
cannot find *any* header under /usr/include/ currently
utilizing this particular three character combination.
And as a further safeguard top will henceforth include
'system' specific headers after the standard includes.
Reference(s):
http://www.freelists.org/post/procps/top-wont-compile-anymore
Signed-off-by: Jim Warner <james.warner@comcast.net>
The get-trans target rsyncs the latest copies of translation files
from the translation project. I put this target in because I always
forget the exact syntax of the command.
Signed-off-by: Craig Small <csmall@enc.com.au>
When top originally responded to the potential libnuma
stderr write, the library was consistently called with
each refresh cycle. That, in turn, guaranteed that any
warning message would be seen at program end by virtue
of: 1) having been issued before the 2nd refresh cycle
and; 2) benefiting from inherited /dev/null buffering.
A later efficiency refactor meant the numa library may
not always be called with every refresh cycle. Rather,
it was only called if top was in one of two numa views
(the '2' or '3' toggles). That, in turn, resulted in a
loss of any warning message at program end unless numa
mode had been preserved in the rcfile. In other words,
if top was started normally then a single cycle stderr
redirect would have long passed by the time the '2' or
'3' toggle was activated. The warning message actually
was spewed but quickly lost to the full screen refresh
which follows all keyboard interactions with the user.
This commit simply moves the restoration of our stderr
redirect to program end (instead of that first display
refresh). Now, any libnuma stderr warning message will
appear as the concluding output line upon quitting top
without regard to when any numa mode view was invoked.
And since this technique might be useful in some other
context (as an example of how to 'buffer' stderr) it's
been generalized with its own #define. But to maximize
its usefulness, the original redirect should be issued
much earlier in pgm startup than top has chosen to do.
Reference(s):
. original libnuma stderr response (msg seen)
commit 35dc6dcc49
. numa refractoring for efficiency (msg lost)
commit f12c0d5c6e
Signed-off-by: Jim Warner <james.warner@comcast.net>
Previously the headers were printed directly without
the format specifier. That way is considered insecure
and leads to build errors with -Werror=format-security
flag set.
Once the write side of the pipe has been duped to stdout for the child,
the original pipefd is no longer needed, so it can be closed to avoid
leaking to the child.
The leak can easily be seen with "watch ls -l /proc/self/fd", but I
found this due to "watch lvs" diagnosing itself:
File descriptor 4 (pipe:[3163616]) leaked on lvs invocation.
Signed-off-by: Josh Stone <jistone@redhat.com>
From now the vmstat can append a timestamp to each line in the
VMSTAT and DISKSTAT mode. You can achieve that with the '-t'
switch.
The '-w' switch now works in the DISKSTAT mode too.
Benno Schulenberg suggested some changes to the help messages
to provide some consistency and clarity for both the users and
translators of procps.
The test needed to be updated as the pmap output changed too.
Signed-off-by: Craig Small <csmall@enc.com.au>
pgrep does not show the full command line when the -a and -u flags are
combined. The -a flag is ignored when the -u flag is used as well.
In addition, the supplied patch by Elliot did not fix the problem
when invert flag ( -v ) was used; a very small tweak to the patch
fixed this problem as well. This problem existed before.
Signed-off-by: Craig Small <csmall@enc.com.au>
Commit 4359cf0698 restored kill's ability
to kill PID -1. This however left PIDs -2 to -9 (or rather process
groups 2 to 9) still having this problem. The check is now generically
looking for a digit and parses it correctly.
vmstat -d or vmstat -p would crash mysteriously under different
circumstances. The problem was eventually tracked down to /sys not
being mounted which meant is_disk() always returned false.
The partition would then be attempted to be linked to a non-existent
disk causing a segfault.
vmstat will now not link to a disk if none exists.
The change in testing will skip those tests when /sys/block doesn't
exist.
Many thanks to Daniel Schepler for his analysis and suggestions.
Bug-Debian: http://bugs.debian.org/736628
Both the man page and the shell builtin kill mention you can
use PID -1, which means nuke everything you can get at.
Alas this "fun" option was missing and the only way to get
around it was with "kill -HUP -- -1".
This small change means kill -HUP -1 is back for all those
destructive types. The error was introduced when the argument
parser was fixed for other problems.
Thanks to Mike for pointing this out.
Signed-off-by: Craig Small <csmall@enc.com.au>
Under some circumstances the ksh shell doesn't fork new processes
when executing scripts and the script is interpreted by the
parent process. That makes the execution faster, but it means
ksh needs to reuse the /proc/PID/cmdline for the new script name
and arguments while the file length needs to stay untouched.
The fork is skipped only when the new cmdline is shorter than
the parent's cmdline and the rest of the file is filled
with '\0'. This is perfectly ok until we try to read the cmdline
of such process. As the read_unvectored() function replaces
all zeros with chosen separator, these trailing zeros are replaced
with spaces in case of the ps tool. Consequently it appends
multiple spaces at the end of the arguments string even when these
zeros do not represent any separators and therefore shouldn't
be replaced.
With this commit the read_unvectored() function skips the
replacement of trailing zeros and separates valid content only.
Reference: https://bugzilla.redhat.com/show_bug.cgi?id=1057600
8 digits per memory column is apparently still too low
for systems with 128TB of RAM. Anyway, setting the limit
to 999TB (12 digits) must be sufficient for now as it
produces wide gaps on "regular" computers.
This commit also increases the number of digits per cpu
columns from 2 to 3 as some of them can hit the 100%
maximum in corner cases.
Apparently there are occasions when NUMA nodes may not
always be contiguous. Under such conditions nodes that
were not used would still occupy precious Summary Area
space showing 100% idle, under the '2' command toggle.
With this commit top will no longer display numa nodes
that have no associated cpu when the '2' toggle is on.
But just in case we wish to return to former behavior,
a new #define called OFF_NUMASKIP has been introduced.
And as an aside, a recent refactor mentioned below set
the stage for this patch to be 'self-tuning'. In other
words, if an inactive/non-displayed node should become
active (if even possible), then top will begin showing
such a node automatically with the next screen update.
Unfortunately, all inactive nodes now 'suppressed' are
still accessible via the '3' command. Those nodes will
just be displayed as empty (no associated cpus shown).
This is not really a top problem but more of a libnuma
and/or user deficiency. The library lacks the means to
validate a node id and the user then input a node that
was not even shown under a '2' toggle Summary display.
( too bad libnuma does not offer an 'is_node_active' )
( type function so top could warn a user when such a )
( discontinuous node was requested using his '3' cmd )
( sure, top could achieve this objective himself but )
( that would require making yet another array global )
( which i'm just not in the mood to do - besides, we )
( have already made enough concessions to libnuma.so )
Lastly, an existing #define (PRETEND_NUMA) was changed
to 'disable' node #1 so as to simulate a discontinuous
node. This allows testing of the '2' and '3' commands.
Reference(s):
http://www.spinics.net/lists/util-linux-ng/msg08671.html
. set stage for self tuning
commit f12c0d5c6e
Signed-off-by: Jim Warner <james.warner@comcast.net>
SIGCONT is a continue signal. It seems that some zsh setups can send
this signal, causing ps to abort. This is not what "continue" means.
This change just uses the default handler which will continue a stopped
process.
References:
http://bugs.debian.org/732410http://www.zsh.org/cgi-bin/mla/redirect?WORKERNUMBER=32251
Signed-off-by: Craig Small <csmall@enc.com.au>
To assist translators, the help lines are split so that each translation
chunk has one option. This gives bonus of if we add or change an option,
only that option remains untranslated rather than the entire help block.
Reference:
http://www.freelists.org/post/procps/procpsng-for-Translation-Project,1
Signed-off-by: Craig Small <csmall@enc.com.au>
A recent change involving a one cycle stderr redirect,
to handle a libnuma potential transgression, failed to
follow normal global variable naming conventions. This
patch will capitalize the 1st letter of 'Stderr_save'.
Reference(s):
commit 35dc6dcc49
Signed-off-by: Jim Warner <james.warner@comcast.net>
Excluding those special X_XON/X_XOF enums, which might
not even be present, restore strict collating order of
all the case labels in the task_show switch statement.
Also, adjust a few sort callbacks for the same reason.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Surprisingly the code for the 'st' column was added in the past,
but wasn't enabled. The vmstat manual already contains the 'st'
column and this commit finally enables the feature.
This is a rework of the merge request #5 that unconditionally
forced the output to cross the 80 chars border.
With this commit users can switch to the wide output mode
with the -w option.
The recent addition of namespaces, combined with those
potential suse out-of-memory fields, means that we are
close to the maximum number of fields poor ol' top can
display. Imagine, the really old top was limited to 26
fields (28 with the suse hack) and this top had neared
the version 'g' rcfile limits which were a healthy 55.
This patch adds another 15 fields to the maximum while
making it even easier to increase in the future. Also,
top still silently accommodates older config files all
the way back to the original pre-ng version top-3.2.8!
Signed-off-by: Jim Warner <james.warner@comcast.net>
While 'invisible' thread subdirectories are accessible
under /proc/ with stat/opendir calls, they have always
been treated as non-existent, as is true with readdir.
This patch trades the /proc/#/ns access convention for
the more proper /proc/#/task/#/ns approach when thread
access is desired. In addition some namespace code has
been simplified and made slightly more efficient given
the calloc nature of proc_t acquisition and its reuse.
Reference(s):
commit a01ee3c0b3
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch is inspired by the 'minimize numa overhead'
patch. It trades the use of subscripts for pointers to
avoid gcc repeated subscript offset calculation bloat.
Now, throughout the cpus_refresh function, a subscript
will be resolved just once & this will (dramatically?)
reduce the path-length taken for each and every frame!
For example, a non-optimized compilation could produce
400+ fewer machine instructions through pointer usage.
[ ok, optimized compiles only save 18+ instructions! ]
Lastly, any residual 'symmetry only' crap is now gone!
Signed-off-by: Jim Warner <james.warner@comcast.net>
A recent libnuma potential corruption problem solution
has caused me to reevaluate some associated numa logic
for efficiency. Here is a summary of the problems that
exist with current libnuma/user possible interactions:
. Whenever the numa library was present extra overhead
would always be incurred in maintaining the node stats
even when the '2' or '3' commands were not being used.
. As part of such overhead a separate loop was used to
reinitialize each cpu/node structure with each display
cycle so that prior accumulated totals were preserved.
Again, it didn't matter if numa data was really shown.
This commit attempts to refocus on the 'critical path'
costs in a running top by optimizing for the occasions
when numa node data is not being displayed. Under such
conditions, no extra overhead will be incurred whether
or not a distribution has the libnuma library present.
To achieve this goal, some additional overhead will be
incurred, but only when actually displaying numa data.
And all such new costs have been minimized in spite of
the gcc inclination to duplicate subscript resolution.
Reference(s):
commit 24bd950cb2e1722d459461f0f9c0c30a4b9ffdaa
Signed-off-by: Jim Warner <james.warner@comcast.net>
A recent libnuma potential corruption problem solution
suggests that libnuma could change in the future so as
to not spew to stderr. This then raises a question how
top could exploit any such library change since we are
currently locked into version #1 of the library by way
of our dlopen("libnuma.so.1", RTLD_LAZY) runtime call.
While not an ultimate solution, this commit will first
try for the most recent version of that library during
top's startup before trying the original libnuma.so.1.
We do this via the unqualified library soname symlink.
For this new dlopen() call to succeed, technically the
numa 'devel' package would usually have been required,
but that's not always true with every distro. And when
the libnuma.so symlink isn't present, it can always be
manually added should a newer & better behaved library
arrive & users tire of the stderr warning at top exit.
Reference(s):
commit 24bd950cb2e1722d459461f0f9c0c30a4b9ffdaa
Signed-off-by: Jim Warner <james.warner@comcast.net>
Confession is supposed to be good for the sole, right?
After a senior moment regarding the 'x' toggle quirks,
and thinking top had somehow regressed, I concluded an
additional explanatory note might well be appropriate.
Those quirks were already documented under the 5d & 5e
topics. But there was no such caution documented under
the 'x' command explanation itself, found in topic 4c.
Signed-off-by: Jim Warner <james.warner@comcast.net>
There is a chance that the libnuma library may corrupt
top's display with some stderr warning messages in the
event something under /sys/devices/system/node/ cannot
be accessed. And, while 2 overridable 'weak' functions
are provided to alter such behavior, we can't use them
since top dynamically links to the library via dlopen.
This commit will redirect stderr to '/dev/null' during
just the first screen display cycle. Thus we can avoid
the corruption which would have remained visible until
the underlining screen row's data had finally changed.
Lastly, this patch should allow such a library warning
to actually appear when one finally exits our program.
[ i think the libnuma folks should consider changing ]
[ the error/warning interfaces to accommodate dlopen ]
[ rather than forcing something like the ugly kludge ]
[ we have employed or libnuma dependency on everyone ]
Reference(s):
https://bugzilla.redhat.com/show_bug.cgi?id=998678
Signed-off-by: Jim Warner <james.warner@comcast.net>
