Commit Graph

3511 Commits

Author SHA1 Message Date
Juan RP
06c9891ae3 xbps_transaction_*: multiple performance improvements (v2).
This commit implements multiple performance improvements
to the transaction code:

- Don't process xbps_pkg_name() N times each time we access
  its package dictionary (via pkgdb or rpool), just do it once
  at xbps_pkgdb_init() time. At pkgdb init time, it just creates
  a property in pkgdb, "pkgname". At rpool time, each time a
  package is accessed, the "pkgname" string property is added.

- The package transaction dictionary contains the "transaction"
  object to know what's the pkg type. This has been changed to an
  uint8, this simplifies the logic and it's faster than checking
  a string object. See xbps_trans_type_t and xbps_transaction_pkg_type().

- Fixed the issue that was marked with XXX in transaction shlibs
  checking code. This has been fixed and improved and resources are
  now just freed as expected.

- Simplified random code all over the place, avoiding unnecessary
  allocations or operations.

- Rename some transaction files to have a better description.

This is my first rototill to the code in 2020.
2020-02-21 09:37:32 +01:00
sineemore
701132071d xbps-install: show overall download progress 2020-02-19 00:28:40 +01:00
Juan RP
4d6d328c62
xbps_transaction_prepare: fix for XBPS_FLAG_DOWNLOAD_ONLY.
We don't really care if all pkgs are on hold, so modify
logic to avoid this part and exit early.
2020-02-18 09:07:03 +01:00
Juan RP
1b9dc3cc4d
xbps_transaction_prepare: set "total-download-size" correctly.
- It was set only with XBPS_FLAG_DOWNLOAD_ONLY.
- Simplify the logic.

In response to #187 and #230
2020-02-18 08:34:41 +01:00
Duncan Overbruck
3ad575178e lib/initend.c: use new xbps_path_* functions 2020-02-14 19:20:54 +01:00
Duncan Overbruck
0f61a1a5a2 lib/util_path.c: add some util functions to work with paths 2020-02-14 19:20:54 +01:00
Juan RP
fba65ad9da xbps_transaction_store: ensure no multiple versions.
This change ensures that no multiple versions of the same pkg
are added to the transaction; if a new version of the same
package is being added as a dependency, compare stored
and current and use the greatest one.

This fixes the recent issue seen in the aarch64 builders, where
two versions of the same package were added to the transaction.

Added a new test case.
2020-02-14 08:41:35 +01:00
Juan RP
28154f488c xbps-uchroot: fix indentation 2020-02-13 09:06:29 +01:00
Juan RP
6ccee8fd5e xbps-uchroot: overlayfs fixes v2.
- Do not use nftw(3), this won't clean up the tree properly.
  Rather use scandir(3) with alphasort, like xbps-create(1).

- Don't drop privs in the parent to make sure that the
  temp overlayfs tree can be removed properly.

This fixes the issues while building go modules and others
with xbps-src.
2020-02-13 09:06:29 +01:00
Duncan Overbruck
fbba38b006 lib/util_hash.c: add conditions to avoid misuse without asserts 2020-02-10 15:03:54 +01:00
Duncan Overbruck
d99aae586a lib/download.c: fix digestlen assert and add if condition to avoid misuse 2020-02-10 15:03:54 +01:00
Duncan Overbruck
2ef79bcc09
lib/util_hash.c: change asserts to allow at least X buffer length 2020-02-10 13:32:17 +01:00
Duncan Overbruck
dd4eabe253
include/xbps.h.in: update descriptions for *_sha256* functions and defines 2020-02-10 13:32:17 +01:00
Duncan Overbruck
9efa72f5c8
lib/package_config_files.c: use _get_cstring_nocopy for sha256_orig 2020-02-10 13:32:17 +01:00
Duncan Overbruck
0d90534236
libxbps: ABI/API break due to hash function changes 2020-02-10 13:32:17 +01:00
Duncan Overbruck
aa4d726dca
lib/package_register.c: make sure to error out if copy fails 2020-02-10 13:32:17 +01:00
Duncan Overbruck
dca2223cb7
lib/transaction_store.c: store a copy in transaction packages array
This fixes multiple use after frees, found with AddressSanitizers.

* xbps_package_register: the pkgdb dictionary is replaced with
  the new one.
* xbps_pkgdb_update: here the pkgdb is externalized, freed and
  internalized.
2020-02-10 13:32:14 +01:00
Juan RP
3eed9bca59
xbps_array_foreach_cb_multi: use PTHREAD_MUTEX_INITIALIZER.
We use defaults, no need to use pthread_mutex_init().
2020-02-10 09:14:50 +01:00
Juan RP
c460df6011
xbps_array_foreach_cb_multi: switch to a mutex.
There's no reason to use a spin lock.
2020-02-10 09:08:33 +01:00
Juan RP
8c930a47dd
travis: disable tcc from build matrix.
tcc seems to cause more problems than benefits, so
get rid of it.
2020-02-10 09:07:33 +01:00
Juan RP
835834ff73
xbps_repo_open: fix memleak in error paths. 2020-02-10 09:07:00 +01:00
Andreas Kempe
02c9cb11c4 configuration: add keepconf option
Add configuration option keepconf that stops xbps from overwriting
unchanged configuration files. If keepconf=true, xbps will store the new
configuration as <name>.new-<version> instead of overwriting unchanged
configuration files.
2020-02-08 21:06:04 +01:00
Juan RP
71a594f681
Revert "xbps-uchroot: make sure to cleanup tempdir with overlayfs."
This reverts commit f6a6385b42.

Does not work as expected, and does not fix the real issue
which is still not understood.
2020-02-08 20:43:23 +01:00
Juan RP
c4019aa923
xbps_transaction_files: fixed another memleak.
Thanks valgrind
2020-02-08 20:22:42 +01:00
Juan RP
2d7e3d7b1a
xbps_transaction_remove_pkg: fix a memleak.
Thanks valgrind
2020-02-08 20:12:11 +01:00
Juan RP
58217d412d
xbps_transaction_files: fix a memleak.
Thanks valgrind!
2020-02-08 20:08:03 +01:00
Juan RP
460515bf41
lib/util.c: add assertions everywhere. 2020-02-08 20:00:56 +01:00
Juan RP
6010a24de6 libxbps: ABI/API break due to xbps_pkg{,pattern}_name changes.
The funcs xbps_pkg_name() and xbps_pkgpattern_name() were
using malloc(3) to return the result, until now.

They now have been changed to not allocate the result
via malloc, the caller is responsible to provide a buffer
at least of XBPS_NAME_SIZE (64).

If for whatever reason the pkgname can't be guessed,
returns false. This should avoid lots of small allocs
around libxbps.

New functions have the following prototype:

bool xbps_pkg_name(char *dst, size_t len, const char *pkg)
bool xbps_pkgpattern_name(char *dst, size_t len, const char *pkg)

as suggested by @duncaen.
2020-02-08 19:49:57 +01:00
Juan RP
1cda3017c3
README.md: sort badges 2020-02-08 13:43:47 +01:00
Juan RP
b23bff86d8
README.md: add repology xbps versions. 2020-02-08 13:42:39 +01:00
Juan RP
bbf113c492
README.md: add LGTM project badges 2020-02-08 13:39:43 +01:00
Juan RP
66bea965dc
xbps_register_pkg: switch to localtime_r(3).
lgtm suggested that localtime() is not reentrant,
so that we now use localtime_r() instead.

lgtm reports 0 alerts after this change, yay.
2020-02-08 13:34:57 +01:00
Juan RP
7d934c9290
xbps-create: fix two issues detected by lgtm 2020-02-07 17:16:47 +01:00
Juan RP
6ff9422134
configure: bump to 0.59 2020-02-07 10:11:11 +01:00
Juan RP
397f2b86e1
tests: new alternatives test case by @CameronNemo.
The test case has been provided by @CameronNemo via #185.
This is expected to fail currently.
2020-02-07 09:55:36 +01:00
Juan RP
732a0e6bdf
alternatives: fixed replace_alternative_with_symlink test case.
Only 1 expected failure now!
2020-02-07 09:43:47 +01:00
Juan RP
94b56d7539
alternatives: do not remove symlinks if provider != target pkg.
This fixes the keep_provider_on_update test case, submitted
by @st3r4g via #219.

While checking for obsoletes make sure current provider
also matches the current target pkg.

Fixes #219
2020-02-07 09:16:57 +01:00
Juan RP
86ce5a2084
tests: new test case for alternatives by @st3r4g.
This currently reproduces the tar/bsdtar alternatives
issue that appeared recently in void.

This is expected to fail currently.

From #219
2020-02-07 07:51:01 +01:00
Juan RP
8d2aaff3e1
mk/prog.mk: fix warning with clang; use PROG_CFLAGS. 2020-02-06 09:48:43 +01:00
Juan RP
4c22511d17
configure: simplify 2020-02-06 09:48:30 +01:00
Juan RP
3a0ca77264
xbps-uchroot: stop remounting / unnecessarily.
The remount is done to use nosuid, but we can get
rid of the remount just because of this specific
option that does not have too much value for our
general usage (xbps-src).
2020-02-06 08:50:12 +01:00
Juan RP
f6a6385b42
xbps-uchroot: make sure to cleanup tempdir with overlayfs.
Go modules seem to have too restrictive permissions on
its builddir, making xbps-uchroot(1) unable to remove
its directory tree due to insufficient permissions.

Run nftw() twice, the first run to set chmod 755, and
second run to remove the file.

Fixes `xbps-src -t` with any pkg that uses `build_style=go`.

This should avoid lots of temp dirs in the buildbot
builders while building go modules and others.
2020-02-04 13:17:26 +01:00
Duncan Overbruck
adebbf71e2
tests: add 3 new test cases for obsolete files with alternatives 2020-02-03 15:33:55 +01:00
Juan RP
749bdabc83
tests/alternatives: set atf_expect_fail for unfixed issue. 2020-02-03 09:58:57 +01:00
Johannes Brechtmann
354d45fca1 add test which replaces an alternative group with a symlink 2020-02-03 09:57:22 +01:00
Juan RP
5b43614e80
libxbps: fixed regression introduced in 0.58.
While looking for dependencies, we need to check
if xbps_rpool_get_pkg() returned a suitable match;
and then validate its result.

This fixes the update_and_install test case that
was reverted via #218.
2020-02-03 09:19:54 +01:00
Juan RP
2612cb9bbe
Makefile: add target deps 2020-02-03 08:14:10 +01:00
Duncan Overbruck
8637269b38
lib/transaction_revdeps.c: fix provides/replaces #218
This reverts a change that solved another issue and marks the test case
for the other issue as expected failure.

The other issue is not as important as this, as it blocks updating a lot
of systems.
2020-01-31 16:32:44 +01:00
Duncan Overbruck
8698735939
tests: add testcase for libglnvd replaces/provides from issue #218 2020-01-31 16:28:41 +01:00
Juan RP
d86e8dde47
NEWS: correct date for 0.58. 2020-01-30 17:51:57 +01:00