Refresh the CONTRIBUTING document with a more modern style and use it as
the single source of truth for how to contribute to the project.
Remove the AUTHORS file, as that file isn't contemporarily used and
its information is redundant.
Signed-off-by: Alex Stewart <alex.stewart@ni.com>
The opkg mailing list has moved to opkg@lists.yoctoproject.org. The old
Google Code/Group is being deprecated. Update the contributing document
to reflect this change.
Signed-off-by: Alex Stewart <alex.stewart@ni.com>
Update the CONTRIBUTING document markdown styling to something more
modern and fixup a spelling mistake in the text.
No substantial content changes in this commit.
Signed-off-by: Alex Stewart <alex.stewart@ni.com>
Cleanup the inconsistent use of tabs and spaces in the opkg-build script
for cleanliness, and homogenize the style of case and if-then
statements.
No functional changes intended in this commit.
Signed-off-by: Alex Stewart <alex.stewart@ni.com>
Add support for tar archives created with --acls and/or --xattrs options,
PAX header format.
GNU tar and libarchive already supports ACLs and extended attributes.
We can now add this support as well to opkg-build script in order to use
fsetattr or setcap inside do_install command and end up with a file in
an image with the relevant ACLs and xattrs.
Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
Signed-off-by: Alex Stewart <alex.stewart@ni.com>
Add support for tar archives created with --acls and/or --xattrs options,
PAX header format.
GNU tar and libarchive already supports ACLs and extended attributes.
We can now add this support as well to opkg-build script in order to use
fsetattr or setcap inside do_install command and end up with a file in
an image with the relevant ACLS and xattrs.
Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
Signed-off-by: Alex Stewart <alex.stewart@ni.com>
Attempting to add feeds with some special names that match
architecture names was returning an error:
"5 The named feed already exists"
Make sourceTypePattern in createFeedLineRegex only match
feed configuration lines src or dist
Signed-off-by: Petar Koynov <petar.koynov@ni.com>
Signed-off-by: Alex Stewart <alex.stewart@ni.com>
Add ztsd compressor option to support using zstandard for packages.
Zstandard compression offers a significant speed increase at
decompression time over the current default (xz). This is amplified on
less capable hardware as used in embedded devices. Further more, it
offers a slightly faster compression speed at the highest compression
levels with a slightly lower compression ratio and the ability to
choose your own compromise between fast compression speed with less
compression and slower compression speed with more compression.
Zstandard would therefore be a good solution for package compression when:
* install time matters, especially if you are installing on less capable
hardware
* disk space is not a concern: lower compression can speed up developer
builds, especially incremental builds
* disk space is a concern: release builds can make use of the highest
compression setting to reduce package sizes
Signed-off-by: Henry Kleynhans <hkleynhans@fb.com>
Signed-off-by: Alex Stewart <alex.stewart@ni.com>
The installation of opkg-build man page introduces a host dependency
on perl for the pod2man package to generate the man page.
To allow the opkg-utils scripts to be installed separately from the
manpages, break apart the install step into two install steps:
install-utils and install-docs
CC: Christian Hermann <mail@hermannch.dev>
Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com>
Signed-off-by: Alex Stewart <alex.stewart@ni.com>
Add support for the special tilde character. The current opkg version
allows specifying a version appended with a tilde e.g.
1.2.2~releasecandidate which is lower in version than 1.2.2 itself. With
this commit, this is now also supported by the opkg.py script.
Signed-off-by: Maarten <maarten.vanmegen@nedap.com>
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
* Add the opkg-feed script to UTILS so that it is installed with a `make
install`.
* Clean up the UTILS variable declaration to be a little more diffable.
Signed-off-by: Alex Stewart <alex.stewart@ni.com>
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
There is a sorting problem with opkg-build where the ipk generated is depending
upon the order of files on disk. The reason is the --sort option to tar only
influences the orders of files tar reads, not those passed by the -T option.
Add in a sort call to resolve this issue. To ensure consistent sorting we
also need to force to a specific locale (C) else the results are still not
deterministic.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
The previous implementation relied on python, it shelled out anyway
(using bashisms) to call ar/tar and couldn't handle paths != cwd.
This rewrite
* can handle packages elsewhere than cwd
* prints a summary like opkg-build does ("Unbuilt <IPK> into <DEST>")
* is written in valid POSIX shell syntax
* is compatible with opkg-build's non-esoteric options
* is linted with shellcheck.
Tested with
* different tar
* gnu tar 1.32
* bsdtar 3.4.1
* busybox-tar 1.31.1
* toybox-tar 0.8.2 (but prints a warning: tar: './' not under '/tmp/dest/foo')
* some opkg-build arguments, as of `opkg-build -o root -g root $args`, with
* args=
* args="-Z xz"
* args="-c"
* args="-c -Z xz"
Signed-off-by: Christian Hermann <mail@hermannch.dev>
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Email protection program incorrectly mangled the URLs used to test
opkg-feeds. Correct the URLs to make the test pass.
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Switch all py script shebangs to `#!/usr/bin/env python3`.
Testing: Successfully ran the following on an IPK feed in $PWD:
* opkg-make-index --checksum md5 --checksum sha256 -p Packages -m -l Packages.filelist .
* opkg-graph-deps ./Packages
* opkg-compare-indexes ./ ./
* opkg-list-fields font-alias_1.0.3-r2.3.204_all.ipk
* opkg-show-deps -p Packages font-alias
I'm not sure how to correctly run opkg-update-index and opkg-unbuild.
I attempted to run both with `-h`, no parse errors.
My motivation for this change is to add xz support to opkg filelists,
which requires python3's version of tarfile. See previous commits in
this series for details. However, this is probably a good change in
general since python2 support is starting to dwindle.
Signed-off-by: Haris Okanovic <haris.okanovic@ni.com>
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Open data.tar.xz if gz payload is not present. This change allows xz
payloads to be archived in filelist.
However, only python3's tarfile has xz support. This therefore only
works if opkg-make-index's shebang is overridden to run in python3.
Signed-off-by: Haris Okanovic <haris.okanovic@ni.com>
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
String split fails on newline (\n) chars in python3 unless subprocess
output string is decoded as text. This results in fn containing a bad
filename with trailing newline (\n) char, which subsequently breaks
open() in get_file_list().
Tested `opkg-make-index -l Packages.filelist ...` in python 2 and 3.
Signed-off-by: Haris Okanovic <haris.okanovic@ni.com>
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
We need opkg-build to fail if for example the tar command is passed invalid
options. Without this, we see silently created empty packaged where data.tar
is zero bytes in size. This creates hard to debug problems.
An example is when reproducible builds are enabled and run on old hosts like
centos7 which has tar < 1.28:
Subprocess output:tar: unrecognized option '--clamp-mtime'
Try `tar --help' or `tar --usage' for more information.
This change does make the script bash specific but that shouldn't an issue
for systems capable of building ipks.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
* printing opkg.Package directly doesn't return anything useful now
<opkg.Package object at 0x7fd49e9f2780>
* we need to call Package.print() function and specify which checksums
to print, we can include both md5 and sha256 for opkg-list-fields
* it was changed in this commit:
commit 601d691dd8
Author: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Date: Wed Dec 19 11:40:15 2018 -0600
opkg-make-index: add sha256sum support
which replaced the modified __str__ function with print(self, checksum)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
The install receipe would have copied all the manpages to each of the
section folders. This change will do what I believe the original author
had intended.
Signed-off-by: Joshua DeWeese <jdeweese@hennypenny.com>
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Opkg should make sure the binaries it creates are created in a
repoducibe way, but it shouldn't modify the mtime of the binaries it
consumes. Doing so has side effects, for exmaple, Python pyc files are
invalidated.
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
OE parallelism means that the number of hard links may increase or
decrease while a tar file is being created, which makes tar fail with
error 1. To avoid this problem, use tar -T option, which uses filelists.
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Argsparse is a newer library, it makes the code cleaner and easier to
expand. Only enabled short options, to leave the command line as it was
with getopt.
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Avoid an integer to string conversion (and back) by directly supplying
the Unix time as an integer to -mtime.
Fixes bugzilla# 12580
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Commit fcb2633900 which added Debian-style
support for link relocation tries to relocate symbolic link on host OS,
resulting in following-like error when two alternative packages have
symbolic link source located in different directories (/bin/rev ->
/bin/busybox.nosuid and /usr/bin/rev -> /usr/bin/rev.util-linux):
update-alternatives: renaming rev link from /bin/rev to /usr/bin/rev
mv: cannot stat '/bin/rev': No such file or directory
Fix the issue by prefixing 'olink' and 'link' variable references with
path to targeted root filesystem's root directory.
Signed-off-by: Niko Mauno <niko.mauno@iki.fi>
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
When using sstate, two parallel builds can produce two packages
with the same mtime but different checksums. When later one of
those two builds fetches the others ipk, the package index does
not get udpated properly (since mtime matches). This ends up with
messages such as:
Downloading file:/../tmp/work/../image/...ipk.
Removing corrupt package file /../sysroot/../var/cache/opkg/volatile/...ipk
However, in that case, ctime is different. Use ctime instead of
mtime to prevent failures like this.
Suggested-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
Acked-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Acked-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
