This commit implements multiple performance improvements
to the transaction code:
- Don't process xbps_pkg_name() N times each time we access
its package dictionary (via pkgdb or rpool), just do it once
at xbps_pkgdb_init() time. At pkgdb init time, it just creates
a property in pkgdb, "pkgname". At rpool time, each time a
package is accessed, the "pkgname" string property is added.
- The package transaction dictionary contains the "transaction"
object to know what's the pkg type. This has been changed to an
uint8, this simplifies the logic and it's faster than checking
a string object. See xbps_trans_type_t and xbps_transaction_pkg_type().
- Fixed the issue that was marked with XXX in transaction shlibs
checking code. This has been fixed and improved and resources are
now just freed as expected.
- Simplified random code all over the place, avoiding unnecessary
allocations or operations.
- Rename some transaction files to have a better description.
This is my first rototill to the code in 2020.
The funcs xbps_pkg_name() and xbps_pkgpattern_name() were
using malloc(3) to return the result, until now.
They now have been changed to not allocate the result
via malloc, the caller is responsible to provide a buffer
at least of XBPS_NAME_SIZE (64).
If for whatever reason the pkgname can't be guessed,
returns false. This should avoid lots of small allocs
around libxbps.
New functions have the following prototype:
bool xbps_pkg_name(char *dst, size_t len, const char *pkg)
bool xbps_pkgpattern_name(char *dst, size_t len, const char *pkg)
as suggested by @duncaen.
We do not want to continue processing more threads
if pthread_create(3) fails, rather return an error.
This is for #182 but not yet fixed, there might be a
memleak somewhere.
This reverts commit 08aa44602f.
The element still has to be processed and ignored by one of the
threads, it is not the last entry this will make the actually last item
be ignored.
the first thread to finish will start again from 0 (`reserved`) until
0+slicecount, next thread will then start from the end of the previous
thread which is already wrong, resulting in processing the first
slicecount*maxthreads entries twice.
The first slicecount*maxthreads entries are accounted by the thread
creation loop and `reserved` has to start at the first unaccounted index.
Before this patch xbps had a constant slice count for every thread.
This resulted in unbalanced thread usage if the workload was
homogenous over the slices.
This patch replaces the naive slice count approach by spinlock based
reservation.
In some tasks the single threaded implementation outperms the multithreaded
one. Use it where it really makes a difference. The _multi() routines do not
spawn any thread if _SC_NPROCESSORS_ONLN == 1.
Bump XBPS_API_VERSION.
This routine will spawn a thread per core to process N items stored
in the specified array, the last thread gets the remainder of items left.
Results have shown that xbps benefits if there is a considerable amount
of items and number of threads being spawned.
Use it in xbps_pkgdb_foreach_cb(), xbps-pkgdb(8), xbps-query(8)
and xbps-rindex(8).
On UP systems there's no overhead because pthread(3) is not used at all.
WIP! investigate if it can be used in libxbps (xbps_rpool_foreach()),
and finish conversion of xbps-rindex(8) -c.
The list of required external deps is now confuse, libarchive and openssl.
libxbps now includes a wrapper for proplib prefixed with xbps_ rather than prop_.
These are the core interfaces in the new API:
rpool - Interface to interact with the repository pool.
rindex - Interface to interact with repository indexes.
pkgdb - Interface to interact with local packages.
transaction - Interface to interact with a transaction.
This also brings new repository index format, making the index file
per architecture and being incompatible with previous versions.
The transaction frequency flush option has been removed, and due to
the nature of package states it was causing more harm than good.
More changes coming soon, but the API shall remain stable from now on.
There's no need to prop_dictionary_copy the returned dictionary to
later have to free it again, just return directly the dictionary and
avoid the free(3)s.
- Configuration file 'xbps-conf.plist' has been splitted off into
two files: conf.plist and repositories.plist. By default they
are stored in etc/xbps.
- Changed some members in xbps_handle struct, mostly to make it easy
to change its value in {cache,root}dir and conffile.
- Made xbps_init() release proplib objects as soon as we don't need
them, that way it uses 35% less of memory or in some cases even more.
There will be another commit that will implement to read new virtualpkg
settings by the user, as specified in:
http://code.google.com/p/xbps/issues/detail?id=12
- A configuration file "xbps-conf.plist" replaces the (un)register target
in xbps-repo(8) and (un)set-prop in xbps-bin(8). For now, you can set
the repositories and prefered virtual packages.
- New package pattern matching code from NetBSD. Supports more ways of
matching patterns in packages.
- Multiple bugs fixed in virtual packages related matching code.
--HG--
rename : LICENSE => COPYING
That means that the following functions were removed:
- xbps_set_{cachedir,flags,rootdir}.
- xbps_get_{cachedir,flags,rootdir}.
With this change fixed an obvious typo that made -c argument to not work,
and now the cache directory is an absolute path not relative to rootdir.
