This is a modification of MR !122 by @renit1609 to fit the new
library.
Problem statement:
The procps library has no PROC_FILLIO flag to
fetch the proc field "/proc/[pid]/io" data
process-wise.
IO Accounting is not included as part of procps.
Requirement:
We have a requirement to fetch process wise
IO utilization which can be used for monitoring.
When looking through the procps library, I see
that IO Accounting (/proc/[pid]/io) is not being
included as part of procps. There is no such
flag like PROC_FILLIO being included in readproc.h .
Solution:
While looking at the implementation done for
other proc fields, I used the spare bits in app code.
I renamed PROC_SPARE_1 as PROC_FILLIO, the spare bit from
PROC_SPARE_* and used it for fetching /proc/[pid]/io
data as part of the procps library similar to other
fields. I moved the PROC_SPARE_* bits each by 1 bit
to retain the spare bits. Meanwhile added the IO fields
in proc_t structure.
References:
procps-ng/procps!122procps-ng/procps#184
nl_langinfo and CODESET are undefined in a musl system. Instead of
uncondionally including langinfo.h, this change includes include/nls.h
which has the tests and work-arounds for systems that don't have these
features. This is similar to how other programs within procps include
langinfo.h via nls.h
References:
procps-ng/procps!130
Signed-off-by: Craig Small <csmall@dropbear.xyz>
This simple two line code change fixes an intermittent
bug whereby %CPU for parent(s) with collapsed children
could be vastly understated from those displayed under
the current 3.3.17 publicly available top & libprocps.
If one started several top instances in the background
using very a small delay interval (zero?), then if the
shell under which they were running was collapsed, you
would see similar %CPU results for both the libraries.
However, when running a demanding 'make' like a kernel
compile (especially if backed by fast processors and a
SSD), then newlib would generally show only 1/3 to 1/2
of the collapsed %CPU values that appeared for 3.3.17.
Of course, now that the bug has been swatted with this
commit the disparities between those results is easily
explained. Since newly created tasks never contributed
tics during the interval where they were created, only
with many short lived tasks would differences surface.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Rather than run risks of more expensive and repetitive
address resolution, we will establish this local index
for a one time cost and avoid any potential gcc bloat.
[ this commit was made in pursuit of a bug involving ]
[ the distortion of elapsed task tics. but, it turns ]
[ out these changes had nothing to do with that bug. ]
[ however, the patch is being retained as desirable. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
That old library defined this field as 'unsigned int'.
However, here it was known as a 'signed int'. Thus for
consistency we'll now also treat it as 'unsigned int'.
[ this commit was made in pursuit of a bug involving ]
[ the distortion of elapsed task tics. but, it turns ]
[ out these changes had nothing to do with that bug. ]
[ however, the patch is being retained as desirable. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
After removing brackets from those 'derived' notations
I was surprised to discover that several origin/source
comments were wrong. So this patch fixes those errors.
[ along the way a couple enumerators were renamed to ]
[ better (i hope) reflect what they're representing. ]
[ that, in turn, also required a little rearranging. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
None of the other four new api headers use brackets on
derived items. With this patch we normalize the fifth.
[ it makes for a cleaner, less confusing, appearance ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
This small change was a result of some experimentation
trading our current 'hsearch' hash scheme for 'gperf'.
I discovered that when the ':' character was a part of
each 'gperf' key, that generated search logic was more
complicated and thus slower. But without a ':', it was
a little cleaner/leaner and therefore slightly faster.
Assuming that the same trailing ':' *might* affect the
current 'hsearch' logic, to be safe we will remove it.
[ while the 'gperf' version will slightly outperform ]
[ an 'hsearch', too many ugly implementation details ]
[ were exposed which complicates future maintenance. ]
[ thus, we'll retain our current 'hsearch' approach. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch will condense some logic in those functions
associated with the file input operations. The changes
will not, for the most part, alter any generated code.
More significantly (though not very) was the change to
two 'strtoul' calls. Since the returned 'endptr' value
isn't exploited, when that parm is set to NULL, we can
save one instruction on each side of such calls (wow).
Signed-off-by: Jim Warner <james.warner@comcast.net>
When #define QUICK_THREADS was introduced, for copying
some task data for a child thread, one proc_t pad byte
was used to mark, then later identify, those children.
Later the QUICK_THREADS was recycled as FALSE_THREADS,
and used for a different purpose, but a conditional in
the header file erroneously remained. Now, it is gone!
Reference(s):
. Jul, 2016 - QUICK_THREADS become FALSE_THREADS
commit c546d9dd44
. Aug, 2011 - QUICK_THREADS intruduced
commit bb4f08ba29
Signed-off-by: Jim Warner <james.warner@comcast.net>
What to call the new library?
Keep using libprocps wouldn't do, its a very different library from
the programs' point of view. It would also mean we could have some
clashes around the packages (two package names, same library name).
The ancient procps used libproc or libproc-a.b.c where a.b.c was the
package version. Kept the revision numbers down (it was always 0.0.0)
but the name of the library changed.
So if we use libproc-2 is there a clash with an ancient procps?
procps v 2.0.0 was around in 1999 so it was 22 years ago, also the
name of the library would have been libproc-2.0.0.so not libproc-2.so
so we're fine with that.
libproc-2 seems to fit, our second major re-work of the procps
library.
Signed-off-by: Craig Small <csmall@dropbear.xyz>
Since 'procps_uptime' will access the /proc filesystem
the <pids> 'new' guy should should protect against the
possibility /proc isn't mounted when 'boot_seconds' is
established. A zero is better than the negative value.
[ the only distortion would be to PIDS_TIME_ELAPSED. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
With the way those 'extents_free_all' guys were coded,
there's no real need to check for a NULL this->extents
before calling 'em. That's how <stat> already does it.
Signed-off-by: Jim Warner <james.warner@comcast.net>
In addition to copyright date, the initial descriptive
line was changed from a generic statement to one which
reflects the specific portion of the proc/ filesystem.
[ such descriptions alternate between 'declarations' ]
[ (h files) & corresponding 'definitions' (c files). ]
Also, a few missing copyright attributions were added.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This commit just ensures that the relatively expensive
ID to name conversions aren't performed unless they're
explicitly requested. It also internalizes those flags
that required the PROC_FILLSTATUS flag to also be set.
[ requiring a caller, in our case pids.c, to provide ]
[ two flags when a single field was the objective is ]
[ wrong & represents a future potential toe-stubber. ]
[ moreover, what's worse is that those two flags are ]
[ seemingly unrelated. but, without both, a SEGV can ]
[ can be expected when a result.str pointer is NULL. ]
[ by contrast, in the master branch those fields are ]
[ arrays which, when set to zeroes, produce an empty ]
[ string. So, there is no abend (but no name either) ]
[ when one of those two required flags were omitted. ]
[ and worth noting, in that branch it's not just one ]
[ caller required to observe a two flag requirement. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
With the 4 header files removed in the previous patch,
this commit just changes all those obsolete references
to that new consolidated 'misc.h' header file instead.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Prior to this patch, we had four separate header files
dealing with miscellaneous functions. Those four files
were documented in the single man page: procps_misc.3.
Now, we will have just a single header file documented
in a single man page (that is what's called symmetry).
[ and while we're at it, we will shorten that overly ]
[ long struct 'procps_namespaces' name to agree with ]
[ the function naming conventions, e.g. 'procps_ns'. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
The preceding commit made that 'esc_all' function more
efficient by using direct pointer manipulation instead
of an indexed string reference approach within a loop.
This commit applies the same approach to the 'esc_ctl'
function. Now we'll save 12 more iterated instructions
while decreasing the function's code size by 43 bytes.
Signed-off-by: Jim Warner <james.warner@comcast.net>
While this patch has some cosmetic whitespace changes,
more importantly it makes that 'esc_all' function more
efficient. By abandoning the indexed loop approach for
a direct pointer manipulation, we will save 9 iterated
machine instructions, for a total of 33 bytes of code.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Much of what was represented in the commit message for
the reference shown below was revisited in this patch.
It also means that the assertion in the last paragraph
of that message will only now be true with LANG unset.
[ and forget all the bullshit about not altering any ]
[ kernel supplied data. sometimes we must to avoid a ]
[ corrupt display due to a string we can not decode. ]
And while this commit still avoids the overhead of the
'mbrtowc', 'wcwidth' 'isprint, & 'iswprint' functions,
we achieve all the benefits with simple table lookups.
Plus such benefits are extended to additional strings.
For example, both PIDS_EXE and PIDS_CMD fields are now
also subject to being 'escaped'. If a program name did
contain multibyte characters, potential truncation may
corrupt it when it's squeezed into a 15/63 byte array.
Now, all future users of this new library only need to
deal with the disparities between string and printable
lengths. Such strings themselves are always printable.
[ the ps program now contains some unnecessary costs ]
[ with the duplicated former 'escape' functions. But ]
[ we retain that copied escape.c code for posterity. ]
[ besides, in a one-shot guy it's of little concern. ]
Note: Proper display of some multibyte strings was not
possible at the linux console. It would seem a concept
of zero length chars (like a 'combining acute accent')
is not recognized. Thus the display becomes corrupted.
But if utf8 decoding is disabled (via LANG=), then all
callers will now see '?', restoring correct alignment.
Reference(s):
. Dec 2020, newlib 'escape' logic refactored
commit a221b9084a
Signed-off-by: Jim Warner <james.warner@comcast.net>
This new library provides callers with pure strings or
string vectors. It is up to those callers to deal with
potential utf8 multibyte characters and any difference
between strlen and the corresponding printable widths.
So, it makes no sense for the library to go to all the
trouble of invoking those rather expensive 'mbrtowc' &
'wcwidth' functions to ultimately yield total 'cells'.
Thus, this patch will eliminate all the code and parms
that are involved with such possible multibyte issues.
[ Along the way we'll lose the ability to substitute ]
[ '?' for an invalid/unprintable multibyte sequence. ]
[ We will, however, replace ctrl chars with the '?'. ]
[ This presents no problem for that ps program since ]
[ it now duplicates all of the original escape code. ]
[ And, we'll no longer be executing that code twice! ]
[ As for the top program, it takes the position that ]
[ it is wrong to alter kernel supplied data. So with ]
[ potential invalid/unprintable stuff, he'll rely on ]
[ terminal emulators to properly handle such issues! ]
[ Besides, even using a proper multibyte string, not ]
[ all terminals generate the proper printable width. ]
[ This is especially true when it comes to an emoji. ]
[ And should callers chose not to be portable to all ]
[ locales by calling setlocale(LC_ALL, ""), they can ]
[ expect to see lots of "?", regardless of what this ]
[ library fixes in a faulty multibyte string anyway. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
After 'errno' management was standardized, a couple of
fields were added to the <pids> api. Only 1 (PIDS_EXE)
involved dynamic memory and, unfortunately, it did not
conform to that expected normalized ENOMEM convention.
Reference(s):
. Jun 2018, added 'exe' to library
commit ad4269f118
. Nov 2017, standardized 'errno' management
commit 06be33b43e
Signed-off-by: Jim Warner <james.warner@comcast.net>
Under the above #define this commit now also addresses
2 additional possible toe stubbers involving 'select'.
If some readproc.h constants were uncoupled from their
pids.h enumerators a 'make check-lib' will now detect.
Signed-off-by: Jim Warner <james.warner@comcast.net>
If a hash results report is output (via UNREF_RPTHASH)
a portion is devoted to occupied table entries ordered
by depth. There is a possibility that some depths will
not be found among existing occupied table entries and
to avoid any confusion probably should not be printed.
[ to illustrate the potential for confusion prior to ]
[ this patch, force a very small table size (like 8) ]
[ & then trigger the procps_pids_unref() eoj report. ]
So this patch ensures only 'in use' entries are shown.
[ admittedly, all of the remaining logic in the loop ]
[ could/should be subordinate to this new 'if' test, ]
[ but we will keep the change to a minimum. besides, ]
[ there's no harm subtracting/adding a zero numdepth ]
[ especially since the chance of a zero is very low. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch just raises the size of the hash table used
to calculate elapsed task stuff. The net result should
be less need for 'chaining' under pid hash collisions.
[ the hash scheme is intentionally kept as primitive ]
[ and, therefore, as fast as possible. it employs an ]
[ 'and' approach versus a 'mod' operation since both ]
[ yield similar distribution but the former approach ]
[ was 4 fewer cpu instructions in terms of overhead. ]
[ additionally, for hash collisions, 'chaining' uses ]
[ an array index rather than the usual pointer since ]
[ the HST_t guys may move when they are reallocated. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch separates the memory allocations into those
used initially from those used in later reallocations.
Thus, we can reduce that iterative realloc() overhead.
Additionally, we'll correct a long standing oops where
multiple history_info structures were created at 'new'
time when only one should have been allocated (jeeze).
[ originally the allocation was strangely based upon ]
[ number of 'items' (???) & later a #define constant ]
Reference(s):
. May, 2016 - subsequent bad history_info logic
commit 9ebadc1438
. Aug, 2015 - original faulty history_info code
commit 7e6a371d8a
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch may be a little misleading in terms of size
since most of the changes just reorder a little logic.
The most significant changes involve two GUEST values.
My original implementation excluded such tics from the
TOTAL calculation and, therefore, the BUSY figure too.
That decision was erroneously based on some code found
in ./kernel/sched/cputime.c which in hindsight applies
only to processes, not those system level cpu figures.
[ another likely oops classified STOLEN tics as IDLE ]
So, this patch attempts to bring those SUM values into
better agreement with the calculations performed for a
root cgroup (see ./kernel/cgroup/rstat.c source file).
[ we differ from those above in that we also include ]
[ the IDLE plus IOWAIT tics in our TOTAL calculation ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
There's a huge toe-stubber awaiting future maintenance
as reflected in that commit below which deals with the
addition of new enumerators to the Item_table. Namely,
whenever the table is grown, one must remember to also
change that existing 'logical_end' enumerator's value.
Well, not anymore! Since that MAXTABLE macro was added
to the procps-private.h header we can now also exploit
it so a 'logical_end' automatically tracks table size.
This change also renders some code associated with the
ITEMTABLE_DEBUG #define unnecessary. So it's gone too.
Reference(s):
. 08/2016, add new enumerators
commit 09e1886c9e
. 08/2020, added MAXTABLE macro
commit c865b06c30
. 08/2020, introduced ITEMTABLE_DEBUG
commit 92d0297e1e
Signed-off-by: Jim Warner <james.warner@comcast.net>
At 'new' time, the major API modules each ensure their
'info' parameter isn't NULL but what it pointed to was
except this single straggler, for some unknown reason.
So, this patch brings him into line with those others.
[ And, without going into the ugly details, this was ]
[ the reason I never experienced an abend originally ]
[ but Craig did. And, though related to stacks mgmt, ]
[ zero initialization was not a factor. Anyway, with ]
[ this patch, everybody would have experienced abend ]
[ under the original (faulty) test_Itemtables logic! ]
Reference(s):
https://www.freelists.org/post/procps/keep-on-patchin,13
Signed-off-by: Jim Warner <james.warner@comcast.net>
Me thinks Craig had the right idea but perhaps not the
most correct solution. As currently structured, all of
the tests now go way too far by checking every 'unref'
and 'new' call when what we're trying for is survival.
In the final analysis, it doesn't matter who issues an
EXIT_FAILURE - that run_tests guy or an early Exit out
of a procps_new() function. They both will produce the
same end result of the desired "FAIL" test diagnostic.
[ and this patch once again allows the slabinfo test ]
Reference(s):
commit 4eeed6dcff
Signed-off-by: Jim Warner <james.warner@comcast.net>
The referenced commit introduced a test program for the API
but it would often fail due to:
The given pointer for _new() not being NULL
The return value for _new not checked, so the subsequent _unref()
would free() random memory
slabinfo checks failing due to permission denied errors.
The first two are fixed, as well as returning a fail to the test
if they don't return correctly, with slabinfo waiting to see if there
is a way of initialising the structure without reading the slabinfo.
References:
commit e616409aa4
Signed-off-by: Craig Small <csmall@dropbear.xyz>
This change sets the stage for exploiting the recently
added ITEMTABLE_DEBUG #define. All tests are performed
in a single module (after trying 6 separate programs).
The chances of each test detecting errors is extremely
remote (at least while I'm maintaining these modules).
However, this single program approach has one flaw and
it relates to the response whenever an error is found.
Each of those six new API modules calls Exit() if they
detect an error. Otherwise, incorrect results would be
produced at the least or an abend encountered at most.
This means that multiple 'make check' invocations will
be needed if more than 1 module actually was in error.
All in all, it is a small price for a large assurance.
Reference(s):
https://www.freelists.org/post/procps/keep-on-patchin,7
Signed-off-by: Jim Warner <james.warner@comcast.net>
The recent work on updating the <meminfo> and <vmstat>
modules with some newly added linux fields reminded me
(again) of a need for some mechanism guaranteeing that
a header file agrees with the source file assumptions.
Sadly, in the past, if a table entry was omitted or if
the table and header are ordered differently, then the
library would silently return the wrong results values
or even potentially experience a SIGSEGV abnormal end.
This patch offers a much needed development assist for
ensuring that Item_table entries are synchronized with
header file enumerators in terms of number plus order.
It's intended solely for our use as libprocps evolves.
Now, by activating ITEMTABLE_DEBUG, either directly or
via ./configure CFLAGS='-DITEMTABLE_DEBUG', the number
and order will be verified. It is envisioned that this
feature will be used at least once prior to a release.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This macro should help the following patch be a little
less prolix. Besides, this private header could/should
do just a little more to help with our newlib efforts.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This cosmetic change just prepares for the later patch
introducing validation of Item_tables and enumerators.
[ and, we'll now have better 'set' function names of ]
[ 'set_diskstats_ENUM' instead of the more redundant ]
[ current 'set_diskstats_DISKSTATS_ENUM' convention. ]
[ now our only exception is the <slabinfo> api where ]
[ a full enumerator identifier with 'SLAB' & 'SLABS' ]
[ prefixes are used, and 'SLABINFO_noop/extra' guys. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Since last visited, there have been several new fields
added and one field deleted in the /proc/meminfo file.
[ references shown below represent linux git commits ]
Reference(s):
. 4/2020, added 'ShadowCallStack'
commit 628d06a48f57c36abdc2a024930212e654a501b7
. 9/2019, added 'FileHugePages' & 'FilePmdMapped'
commit 60fbf0ab5da1c360e02b7f7d882bf1c0d8f7e32a
. 9/2019, removed 'Quicklists'
commit 13224794cb0832caa403ad583d8605202cabc6bc
Signed-off-by: Jim Warner <james.warner@comcast.net>
This commit attempts to smooth some wrinkles impacting
any future libprocps user exploitation. The 2 problems
relate exclusively to our XTRA_PROCPS_DEBUG provision.
1. The 'xtra-procps-debug.h' header had an include for
'procps-private.h', which was not an installed header.
So the STRINGIFY macros will now be embedded directly.
2. Each of the new api headers referenced '<proc/...>'
rather than '<procps/...>' for the debugging #include.
So, we must drop that prefix in favor of a quoted file
name so that debugging builds work regardless of where
that 'xtra-procps-debug.h' header happens to be found.
Reference(s):
. Jun, 2020 - changed target install directory
commit d64755ea5f
. Sep, 2018 - #include migrated from procps.h
commit fab37662ef
Signed-off-by: Jim Warner <james.warner@comcast.net>
I almost feel like I should apologize for this kind of
patch ( *almost* ). But, since this comment was unique
to the <pids> api and since it was especially designed
to align with the following comment and since the next
comment already carried the alignment emphasis, I will
refrain from issuing any apology and submit it anyway.
[ plus to prove that i am not totally anal-retentive ]
[ and can exercise some restraint there are two more ]
[ places where i COULD have added alignment emphasis ]
[ see lines 804-806 and lines 1360-1361 for my proof ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch just trades the 'typestr' for the much more
meaningful 'valid range' when a relative enum has been
deemed invalid under that XTRA_PROCPS_DEBUG provision.
[ and we'll also make one comment a bit more generic ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
In that commit referenced below, a promise was made to
revisit an 'escape_str' function in efforts to make it
private to the library. The problem was it's needed by
both ps plus the library which is why it was exported.
So, in an effort to remove it from libprocps.sym, this
patch duplicates all the required code in ps/output.c.
Now, each version can be made private to their caller.
[ along the way we'll use this opportunity to remove ]
[ the 'restrict' qualifiers from function parameters ]
[ while swatting a compiler warning referenced below ]
Reference(s):
. April 2016, most escape functions made private
commit d916d5db86
proc/escape.c: In function `escape_command':
proc/escape.c:182:23: warning: initialization of `const char **' from incompatible pointer type `char **' [-Wincompatible-pointer-types]
182 | const char **lc = (char**)pp->cmdline;
| ^
Signed-off-by: Jim Warner <james.warner@comcast.net>
There was a time when that procps.h file served a more
traditional role. Prior to the commit referenced below
it held just macros plus manifest constants. But, with
that change, such items were replaced with a series of
includes embracing all the library exported functions.
That approach was known to disguise errors which would
have otherwise yielded a compiler warning. And without
such a warning, there was no way to address the error.
So this patch will trade the all inclusive header file
approach for individual includes only where necessary.
Reference(s):
. April 2016, procps.h header file revamped
commit ccb6ae8de1
. Sept 2018, top abandoned use of procps.h
commit a6dfc2382e
Signed-off-by: Jim Warner <james.warner@comcast.net>
The meminfo & vmstat file read buffers are now defined
through symbolic constants next to the symbolic names.
[ recent attention to the /proc/stat buffer size led ]
[ to this (hopefully) better approach to definitions ]
Reference(s):
commit 81f4a6acdf
Signed-off-by: Jim Warner <james.warner@comcast.net>
We won't go as far as the merge request shown below in
increasing buffer size, but we will reduce by half the
total iterations while growing it to achieve one read.
[ and this seems in line with what was recently done ]
[ to help top under a massively parallel environment ]
[ thru 2 abreast ('4') and combined cpus ('!') modes ]
Reference(s:
https://gitlab.com/procps-ng/procps/-/merge_requests/105
Signed-off-by: Jim Warner <james.warner@comcast.net>
Now that we may be getting serious with documentation,
that stat module was revisited with an eye toward user
friendliness. Heck, even this author puzzled over some
of the existing notes and naming conventions employed.
So, this patch will adjust some identifiers and expand
the notes to (hopefully) better serve potential users.
The most significant change was making the STAT_TIC_ID
always valid for numa nodes, even if any are inactive.
Thus the -22222 special STAT_NODE_INVALID constant now
is applied only to STAT_TIC_NUMA_NODE. It will be used
on the cpu summary and reaps with STAT_REAP_CPUS_ONLY.
And it will also mark any numa node that was inactive.
Signed-off-by: Jim Warner <james.warner@comcast.net>
In preparation for a documentation attempt, this patch
renames 'struct reap' to 'struct reaped' in two of our
interfaces. That will bring all the non-pid interfaces
into agreement since the <stat> api already used that.
Signed-off-by: Jim Warner <james.warner@comcast.net>
