Before this patch xbps had a constant slice count for every thread.
This resulted in unbalanced thread usage if the workload was
homogenous over the slices.
This patch replaces the naive slice count approach by spinlock based
reservation.
This is a trickier situation.
The original message:
```
initend.c:423:10: error: ‘%s’ directive output may be truncated writing 15 bytes into a region of size between 1 and 512 [-Werror=format-truncation=]
"%s%s", strcmp(xhp->rootdir, "/") ? xhp->rootdir : "",
^~
initend.c:422:3: note: ‘snprintf’ output between 16 and 527 bytes into a destination of size 512
snprintf(xhp->confdir, sizeof(xhp->confdir),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
"%s%s", strcmp(xhp->rootdir, "/") ? xhp->rootdir : "",
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
XBPS_SYSCONF_PATH);
~~~~~~~~~~~~~~~~~~
initend.c:429:7: error: ‘snprintf’ output may be truncated before the last format character [-Werror=format-truncation ]
"%s/%s", strcmp(xhp->rootdir, "/") ? xhp->rootdir : "", buf);
^~~~~~~
initend.c:428:3: note: ‘snprintf’ output 2 or more bytes (assuming 513) into a destination of size 512
snprintf(xhp->confdir, sizeof(xhp->confdir),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
"%s/%s", strcmp(xhp->rootdir, "/") ? xhp->rootdir : "", buf);
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
initend.c:434:9: error: ‘%s’ directive output may be truncated writing 17 bytes into a region of size between 1 and 512 [-Werror=format-truncation=]
"%s%s", strcmp(xhp->rootdir, "/") ? xhp->rootdir : "",
^~
initend.c:433:2: note: ‘snprintf’ output between 18 and 529 bytes into a destination of size 512
snprintf(sysconfdir, sizeof(sysconfdir),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
"%s%s", strcmp(xhp->rootdir, "/") ? xhp->rootdir : "",
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
XBPS_SYSDEFCONF_PATH);
~~~~~~~~~~~~~~~~~~~~~
initend.c:455:11: error: ‘%s’ directive output may be truncated writing 14 bytes into a region of size between 0 and 511 [-Werror=format-truncation=]
"%s/%s", strcmp(xhp->rootdir, "/") ? xhp->rootdir : "",
^~
initend.c:454:3: note: ‘snprintf’ output between 16 and 527 bytes into a destination of size 512
snprintf(xhp->cachedir, sizeof(xhp->cachedir),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
"%s/%s", strcmp(xhp->rootdir, "/") ? xhp->rootdir : "",
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
XBPS_CACHE_PATH);
~~~~~~~~~~~~~~~~
initend.c:461:7: error: ‘snprintf’ output may be truncated before the last format character [-Werror=format-truncation ]
"%s/%s", strcmp(xhp->rootdir, "/") ? xhp->rootdir : "", buf);
^~~~~~~
initend.c:460:3: note: ‘snprintf’ output 2 or more bytes (assuming 513) into a destination of size 512
snprintf(xhp->cachedir, sizeof(xhp->cachedir),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
"%s/%s", strcmp(xhp->rootdir, "/") ? xhp->rootdir : "", buf);
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
initend.c:467:11: error: ‘%s’ directive output may be truncated writing 12 bytes into a region of size between 0 and 511 [-Werror=format-truncation=]
"%s/%s", strcmp(xhp->rootdir, "/") ? xhp->rootdir : "",
^~
initend.c:466:3: note: ‘snprintf’ output between 14 and 525 bytes into a destination of size 512
snprintf(xhp->metadir, sizeof(xhp->metadir),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
"%s/%s", strcmp(xhp->rootdir, "/") ? xhp->rootdir : "",
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
XBPS_META_PATH);
~~~~~~~~~~~~~~~
initend.c:473:7: error: ‘snprintf’ output may be truncated before the last format character [-Werror=format-truncation ]
"%s/%s", strcmp(xhp->rootdir, "/") ? xhp->rootdir : "", buf);
^~~~~~~
initend.c:472:3: note: ‘snprintf’ output 2 or more bytes (assuming 513) into a destination of size 512
snprintf(xhp->metadir, sizeof(xhp->metadir),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
"%s/%s", strcmp(xhp->rootdir, "/") ? xhp->rootdir : "", buf);
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
```
It's basically warning about dangerous operations on strings. And as
far as I could tell, is a valid warning and not a false alarm!
This fix makes the concept of `XBPS_MAXPATH` lose a little bit of sense
as now it doesn't necessarily represent the max size of the paths used
by xbps, but instead the max allowed size of the path configured.
I think this change is ok, but I wasn't able to find any reference to
why it was chosen to be 512. POSIX mandates at least 256, so I'm not
breaking anything that wasn't broken already, and Linux seems to have
a maximum size of 4096, which is pretty safe.
Therefore, this changes should be harmless. I think.
When trying to compile the file ftp.c, I get errors related with
warnings that were marked to be reported as error.
This was the original message:
```
fetch/ftp.c:444:8: error: this statement may fall through [-Werror=implicit-fallthrough=]
type = 'D';
~~~~~^~~~~
fetch/ftp.c:445:2: note: here
case 'D':
^~~~
fetch/ftp.c: In function ‘ftp_request’:
fetch/ftp.c:342:3: error: missed loop optimization, the loop counter may overflow [-Werror=unsafe-loop-optimizations]
for (i = 0; i <= len && i <= end - dst; ++i)
^~~
fetch/ftp.c:342:24: error: missed loop optimization, the loop counter may overflow [-Werror=unsafe-loop-optimizations]
for (i = 0; i <= len && i <= end - dst; ++i)
~~~~~~~~~^~~~~~~~~~~~~~~~~
```
$ARCH-repodata is currently not protected by a signature. While most of
the package metadata is also embedded into the .xbps files, which are
protected by a signature, xbps-install ignores it
(1670ff000d/lib/package_unpack.c (L123))
and relies entirely on $ARCH-repodata.
This enables anyone who is able to modify the $ARCH-repodata to
substitute packages. This patch adds a check that verifies the signed
pkgver matches the one in the repodata, so at least downgrades posing as
updates are detected.
This is an incomplete fix as the whole transaction is still set up with
the unsigned repodata and other issues surely exist. The real fix is
signing $ARCH-repodata.
Calling EVP_cleanup here undoes initialization already done in
lib/fetch/common.c ssl_init() causing subsequent attempts to call
SSL_CTX_new() to fail. Which ultimately is the root cause for #170.
Fixes#170
xbps_find_virtualpkg_in_conf() needs to look at the vpkgs set up in
configuration files, not from those set by pkgdb.
As a result of this the two test cases that were failing yesterday are now
fixed.
When adding packages to the index, xbps-rindex will check if the
consistency of shlibs is broken by a package. If so, rindex will create
a stage file and commit the packages there. Once the consistency is restored,
rindex -a will commit the stage area back to the public repo and delete the
stage file.
When adding packages to the index, xbps-rindex will check if the
consistency of shlibs is broken by a package. If so, rindex will create
a stage repository and commit the packages there. Once the consistency
is restored, rindex -a will commit the stage area back to the public
repo and delete the stage file.
This commit adds relative link support to lib/package_alternatives.c.
Instead of creating absolute links, xbps will create relative links
to the target. This allows to follow links on systems even if the
aren't mounted on /.
The default CA file set by FreeBSD is only available when using the
LibreSSL provided CA file, and we've decided to use the CA path
by default.
Discussed with @dominikh.
While xbps_fetch_file() creates the .part file and for whatever reason
it did not finish properly to rename the file, it could request the server
to restart the download with offset set to file size, resulting in HTTP 416
return code.
Handle this case by checking if the server returns 416 and then checking
if the returned file size matches the requested offset and just rename
the file.
Thanks to @beefcurtains for the test case.
Those are a wrapper around xbps_{array,dictionary}_internalize_from_zfile()
that prints a debugging msg when the plist file cannot be internalized.
Update xbps to use these wrappers.
We use a simple file lock that is created with O_CREAT|O_EXCL.
This should fix the concurrency issues with multiple processes
running xbps-rindex -a/-c on the same repository/arch combo.
At the time we've been searching for the pkg in a repo, no repos
were registered resulting in EINVAL. While here, if there's no
declared repos return ENOENT instead.
There's no reason to make them absolute, simply store in the metadata
the target file as is. This vastly simplifies the code and makes all
test pass correctly.
If xbps-create(8) did not guess the target file of relative symlinks for
some reason, just compare the current symlink and what's stored as is,
without converting it to absolute.
This might happen with dangling relative symlinks or existing binary
packages that were not created with a newer xbps-create(8).
This fixes the following case:
- A-1.0_1 is installed and owns bin/foo.
- A-1.1_1 is in the transaction and depends on B-1.0_1 that owns bin/foo.
- B-1.0_1 is in the transaction and replaces A<1.1_1.
- A-1.1_1 in the transaction now has "skip-obsoletes".
- A is updated to 1.1_1, its metadata file (<pkgname>-files.plist) wasn't removed.
Added two new test cases to verify moving files from/to a dependency.
If xbps_repo_open() was called with the lock arg set, xbps_repo_close()
will now unlock the repo file lock, without the need to set it.
This avoids the need to always unlock the file lock even if it wasn't
locked previously. This also introduceds an ABI/API break, but this
way it's cleaner.
- Simplify xbps_repo_open::repo_get_dict().
- Use xbps_end() in the utils where necessary.
- Make xbps_end() call xbps_pkgdb_unlock() if necessary.
- Make xbps_end() release rpool resources.
- Make xbps_end() release resources from xbps_handle.
- Fixed 90% of reported leaks (still reachable at exit) from valgrind.
That was to silence valgrind's memcheck with --leak-check=full.
Added a new test case to verify its correctness. Somehow virtual packages
declared via xbps.d(5) were not working correctly for dependencies.
This now works as expected with vpkgs declared with and without a version
component.
-i, --ignore can be specified multiple times and can be used to
ignore configuration of those packages while configuration of all
packages is being performed.
Close#67
This explicitly enables the in memory fetch/store of remote repository
data archives mode, ignoring existing on-disk repodata archives.
This changes the previous behaviour of falling back to this mode if no
on-disk repodata archives were found.
Thanks to @Gottox and @dominikh for comments.
These routines return a xbps_array_t with a full sorted dependency graph
for the target pkg, by querying pkgdb or rpool.
Update xbps-query(8) to use the new libxbps API.
The "architecture" configuration keyword is now available to override
the native machine architecture returned by (uname(2)).
The XBPS_ARCH environment variable still has preference.
This allows you to print to stdout any file stored in a binary package,
locally or remotely!
$ xbps-query -R --cat=/usr/bin/ls coreutils > ls
$ file ls
ls: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, BuildID[sha1]=7a195fc46d1d5cdca32bfccd3b30f81784e342ed, stripped
$
- This mode prints to stdout the matching FILE stored in a binary package.
- ABI break: renamed xbps_get_pkg_plist_from_binpkg() xbps_binpkg_get_plist().
- Added xbps_binpkg_get_file() as a generic way to get pkg file contents.
- Removed useless comments from xbps_api_impl.h.
The behaviour of this routine mimics the existing xbps_array_add() with
the difference that stored objects are moved to the right to insert
our object as the first element on the array.
Use this to add replaced packages in the transaction array at the head
rather than at the end, to preserve the proper sorting order.
Unpacking those symlinks is so cheap that there's no point in wasting
resources just to check if those have been modified, just unpack them
unconditionally.
There's no need to internalize props.plist because there are no changes
compared to the pkg dictionary stored in the repo index, simply ignore it.
The code only cares for the files.plist to check for obsolete files.
All package files even not owned by root should be removed, so make sure
that those checks return success while being root.
This fixes package removal for polkit, that had /etc/polkit/rules.d
files owned by the polkitd user.
