This commit implements multiple performance improvements
to the transaction code:
- Don't process xbps_pkg_name() N times each time we access
its package dictionary (via pkgdb or rpool), just do it once
at xbps_pkgdb_init() time. At pkgdb init time, it just creates
a property in pkgdb, "pkgname". At rpool time, each time a
package is accessed, the "pkgname" string property is added.
- The package transaction dictionary contains the "transaction"
object to know what's the pkg type. This has been changed to an
uint8, this simplifies the logic and it's faster than checking
a string object. See xbps_trans_type_t and xbps_transaction_pkg_type().
- Fixed the issue that was marked with XXX in transaction shlibs
checking code. This has been fixed and improved and resources are
now just freed as expected.
- Simplified random code all over the place, avoiding unnecessary
allocations or operations.
- Rename some transaction files to have a better description.
This is my first rototill to the code in 2020.
The funcs xbps_pkg_name() and xbps_pkgpattern_name() were
using malloc(3) to return the result, until now.
They now have been changed to not allocate the result
via malloc, the caller is responsible to provide a buffer
at least of XBPS_NAME_SIZE (64).
If for whatever reason the pkgname can't be guessed,
returns false. This should avoid lots of small allocs
around libxbps.
New functions have the following prototype:
bool xbps_pkg_name(char *dst, size_t len, const char *pkg)
bool xbps_pkgpattern_name(char *dst, size_t len, const char *pkg)
as suggested by @duncaen.
As shown in https://github.com/void-linux/xbps/issues/110
/var might be rw, while / might be ro.
Check this via xbps_pkgdb_lock() for both things.
Close#110
Signed-off-by: Juan RP <xtraeme@gmail.com>
xbps_find_virtualpkg_in_conf() needs to look at the vpkgs set up in
configuration files, not from those set by pkgdb.
As a result of this the two test cases that were failing yesterday are now
fixed.
Those are a wrapper around xbps_{array,dictionary}_internalize_from_zfile()
that prints a debugging msg when the plist file cannot be internalized.
Update xbps to use these wrappers.
- Simplify xbps_repo_open::repo_get_dict().
- Use xbps_end() in the utils where necessary.
- Make xbps_end() call xbps_pkgdb_unlock() if necessary.
- Make xbps_end() release rpool resources.
- Make xbps_end() release resources from xbps_handle.
- Fixed 90% of reported leaks (still reachable at exit) from valgrind.
That was to silence valgrind's memcheck with --leak-check=full.
These routines return a xbps_array_t with a full sorted dependency graph
for the target pkg, by querying pkgdb or rpool.
Update xbps-query(8) to use the new libxbps API.
- Rather than using a POSIX named semaphore use a POSIX lock (lockf(3))
for pkgdb for writers. Writers that cannot acquire the pkgdb lock will
get EAGAIN rather then being blocked.
- Due to using a file lock we cannot write the pkgdb every time a package
is being unpacked, configured or removed. Instead pkgdb is only written
at the end of a specific point in the transaction (unpack, configure, remove)
or via xbps_pkgdb_unlock().
In some tasks the single threaded implementation outperms the multithreaded
one. Use it where it really makes a difference. The _multi() routines do not
spawn any thread if _SC_NPROCESSORS_ONLN == 1.
Bump XBPS_API_VERSION.
This routine will spawn a thread per core to process N items stored
in the specified array, the last thread gets the remainder of items left.
Results have shown that xbps benefits if there is a considerable amount
of items and number of threads being spawned.
Use it in xbps_pkgdb_foreach_cb(), xbps-pkgdb(8), xbps-query(8)
and xbps-rindex(8).
On UP systems there's no overhead because pthread(3) is not used at all.
WIP! investigate if it can be used in libxbps (xbps_rpool_foreach()),
and finish conversion of xbps-rindex(8) -c.
The list of required external deps is now confuse, libarchive and openssl.
libxbps now includes a wrapper for proplib prefixed with xbps_ rather than prop_.
These are the core interfaces in the new API:
rpool - Interface to interact with the repository pool.
rindex - Interface to interact with repository indexes.
pkgdb - Interface to interact with local packages.
transaction - Interface to interact with a transaction.
This also brings new repository index format, making the index file
per architecture and being incompatible with previous versions.
The transaction frequency flush option has been removed, and due to
the nature of package states it was causing more harm than good.
More changes coming soon, but the API shall remain stable from now on.
