The '$' characters inside awk commands need to be $$ when appearing
inside a Makefile. Without this, we generate empty src and src-sys
packages.
Reported by: Stefan Grundmann
Fixes: a8be899289 ("build: Support got")
MFC after: 1 day
We bootstrap libz unconditionally on macOS and Linux, and conditionally
on FreeBSD if pre-1600029 or DISK_IMAGE_TOOLS_BOOTSTRAP is enabled, but
if building on new enough -CURRENT without DISK_IMAGE_TOOLS_BOOTSTRAP
then libz is not bootstrapped and so this libz dependency does not
exist. Add the dependency whenever we are bootstrapping libz but not
otherwise, and do so without duplicating the condition.
Fixes: 1dbcc7ee92 ("Makefile.inc1: Make sure mandoc bootstrap tool depends on libz")
MFC after: 1 week
It links against both libopenbsd and libz, so reflect this in the
bootstrap tool's dependencies in case of races from highly-parallel
builds.
Reported by: jkh
MFC after: 1 week
Add code for using got(1), matching how git(1) is currently used:
1. Determining the git branch,
2. Determining the checked-out hash,
3. Determining the commit count,
4. Determining the timestamp of the most recent commit,
5. Determining if the tree is "dirty",
6. Generating lists of files for the src and src-sys packages, and
7. Excluding .got directories from src.txz and ports.txz.
The release engineer hopes to use this functionality in release builds,
since got's minimalism makes it a far more manageable build tool than
git.
Reviewed by: naddy
MFC after: 1 week
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D60606
For rm -f it's not an error if the files do not exist. If there's any
other error it's presumably a real issue, so avoid ignoring all errors.
Reviewed by: brooks, adrian
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D56714
Build targets should generally be run as an unprivileged user. Even
though building as root is discouraged many users do so, so just start
with a warning.
Reviewed by: brooks
Differential Revision: https://reviews.freebsd.org/D50352
The utility targets described by this comment are in fact related to
building (buildworld), not installing (which is included in
`make world`).
Reviewed by: brooks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D56715
Also, adjust bootstrap tools code in Makefile.inc1 to reflect this; in
addition to rebuilding mkimg if it is out of date, we now need to build
libz, which was not previously needed by mkimg.
Reviewed by: kevans
MFC after: 2 weeks
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D60153
This is the last remaining piece of GPL software in the base system.
The installer transitioned to bsddialog four years ago, and the last
remaining dialog consumer, dpv, was turned off more than two years ago.
Retire dpv, libdpv, libfigpar (used only by dpv), and dialog itself.
Reviewed by: dteske
Differential Revision: https://reviews.freebsd.org/D55424
This is the last remaining piece of GPL software in the base system.
The installer transitioned to bsddialog four years ago, and the last
remaining dialog consumer, dpv, was turned off more than two years ago.
Retire dpv, libdpv, libfigpar (used only by dpv), and dialog itself.
(cherry picked from commit 73bb4a92985d747462d866248c1e7623f7ff1a7f)
If pkg is installed, for each old library found to still be present,
check if any installed packages either provide or require the library,
and inform the user.
MFC after: 1 week
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D53977
If the user runs "make update-packages" without bumping BRANCH, then it
isn't possible to copy packages from the old location to the new one
(because the two locations are the same). So just skip that step.
Sponsored by: ConnectWise
PR: 295085
MFC after: 1 week
Reviewed by: ivy, emaste
Differential Revision: https://reviews.freebsd.org/D56872
Some of the LLVM binary utilities were included in the Clang package
(because they did not set an explicit PACKAGE).
Add a new Makefile under clang/toolchain to create the symlinks and man
links for ar, c++filt, nm, and so on (without the llvm-* prefix) when
LLVM_BINUTILS is enabled (as it is by default).
PR: 293610
Reviewed by: bapt, ivy, brooks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D55692
Makefile.inc1 copies locales to ${INSTALLTMP} to avoid issues when
running make installworld on a live system. However, this can break
on non-FreeBSD systems, e.g. on openSUSE where /usr/share/locales
has mode 0555, which means after we copy it, we can't delete it,
so the build fails.
Since this functionality is only useful when installing over a
live system, disable it when the build host is not FreeBSD.
MFC after: 2 weeks
Reviewed by: kevans, emaste
Sponsored by: https://www.patreon.com/bsdivy
Differential Revision: https://reviews.freebsd.org/D56677
Previously we had a mix of ${PKG_CMD} and bare 'pkg', which is
wrong, and breaks the build when 'pkg' isn't in the tools path,
e.g. when cross-building.
MFC after: 2 weeks
Reviewed by: wosch, emaste
Sponsored by: https://www.patreon.com/bsdivy
Differential Revision: https://reviews.freebsd.org/D56655
We support both -h and -n, but GNU coreutils only supports -n,
so use that instead. This fixes the package build on Linux.
MFC after: 2 weeks
Reviewed by: (wosch, imp) (previous version), emaste
Better fix than the original patch suggested by: jrtc27
Sponsored by: https://www.patreon.com/bsdivy
Differential Revision: https://reviews.freebsd.org/D56656
Instead of driving the world package build from Makefile.inc1,
use a subdir build where each package has a subdirectory under
packages/ using the new <bsd.pkg.mk>.
Convert some metadata that was previously in the UCL files (e.g.
sets and dependencies) to Makefile variables.
Build the packages under objdir (not repodir), and use the new
stagepackages target to copy them to repodir when creating the
repository.
Determine an explicit list of packages to build in packages/Makefile
based on enabled src.conf options, and add logic to abort the build
if we attempt to build an empty package. This inverts the previous
logic in Makefile.inc1 which would simply skip empty packages.
There are a few advantages to doing it this way:
* The package build works more like the rest of the build system,
so it's more accessible to developers.
* We can customise the packages we build based on src.conf options,
e.g. skipping a package entirely, or adjusting its dependencies
based on what it actually requires.
* We have a specific list of packages that we want to build, and an
unexpectedly missing package results in a build error, instead of
silently producing a broken repository.
* It's possible to build (and in the future, install) an individual
package without having to rebuild the entire repository.
This doesn't apply to the dtb, kernel-* or src-* packages; those
have their own build systems in Makefile.inc1 and will be converted
later.
MFC after: 4 weeks (stable/15 only)
Reviewed by: jlduran, sjg, brooks
Sponsored by: https://www.patreon.com/bsdivy
Differential Revision: https://reviews.freebsd.org/D56087
Nobody will test it as long as it's not tied into the top-level target,
and we don't really have a good reason not to cut over to it since it
does produce identical results. Switch over but do not remove the old
sh implementation yet, just in case.
Note that this replacement is driven by a significant speedup from how
the new version is designed: tha lua version actually tries to
parallelize its data collection efforts to speed up generation.
Reviewed by: imp (previous version), emaste
Differential Revision: https://reviews.freebsd.org/D56558
To avoid breaking the build due to incompatible changes in flua or lua
libraries on the host, use the bootstrap flua for the package build.
MFC after: 2 weeks
Reviewed by: kevans, emaste
Sponsored by: https://www.patreon.com/bsdivy
Differential Revision: https://reviews.freebsd.org/D56271
We want to use flua from the source tree (not the host) during package
build, firstly to protect us from breaking changes in lua or libucl,
and secondly to allow (in future) cross-building of packages from
Linux or macOS.
Since we don't know if the user will be building packages during the
bootstrap phase, and because flua is fairly small and generally useful,
build it as a bootstrap tool unconditionally.
MFC after: 2 weeks
Reviewed by: kevans, emaste
Sponsored by: https://www.patreon.com/bsdivy
Differential Revision: https://reviews.freebsd.org/D56270
These are not used in our world and kernel build targets. We use the
compiler driver for assembly, and ar adds the archive index (symbol
table) automatically.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D55964
Because of this setting we were still using ELF Tool Chain tools for
buildworld. The sets of binary utilities are largely equivalent and
this went unnoticed after commit 1cae7121c6 ("Enable LLVM_BINUTILS
by default").
This was discovered recently because ELF Tool Chain objcopy produces
standalone debug files without phdrs and this caused an issue with a
3rd party ELF parser [1]. Remove the forced setting so that we use
LLVM's binutils to build the system.
[1] https://sourceware.org/bugzilla/show_bug.cgi?id=33876
Re-commit after fixing a bootstrapping issue with LLVM binutils (in
17494c6e6b "build: Boostrap LLVM_BINUTILS for cross-tools").
Reviewed by: imp, jhb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D55650
This reverts commit 858f53dd43.
It is not clear to me why building from Linux or MacOS fails to build
the toolchain, so reintroduce the long-standing slightly-broken
toolchain until that can be determined.
Reported by: vexeduxr, jrtc27
Because of this setting we were still using ELF Tool Chain tools for
buildworld. The sets of binary utilities are largely equivalent and
this went unnoticed after commit 1cae7121c6 ("Enable LLVM_BINUTILS
by default").
This was discovered recently because ELF Tool Chain objcopy produces
standalone debug files without phdrs and this caused an issue with a
3rd party ELF parser [1]. Remove the forced setting so that we use
LLVM's binutils to build the system.
[1] https://sourceware.org/bugzilla/show_bug.cgi?id=33876
Reviewed by: imp, jhb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D55650
Before this change, `make test-includes` (run as part of buildworld)
would place test files in the current directory, which would clutter up
git clones. Run `make obj` beforehand to ensure that the files are put
in `${.OBJDIR}` instead of `${.CURDIR}`. This helps cut down on the
noise significantly when running commands like `git status`.
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D55499
If a package plist only contains directories, but no files, do not
create the package.
This fixes an issue where setting "package=foo" in mtree causes the
"foo" package to always be created, even if nothing else installs in
that package, because the mtree entry is always added to the plist.
This most often happens:
* With architecture-specific directories, because mtree can't install
a directory conditionally based on architecture, and
* With packages that are completely empty when a particular src.conf
knob is disabled, because mtree will still create the directories.
Although it's theoretically possible that we might want to create a
package that only contains directories, there are no such packages
today.
MFC after: 2 weeks (stable/15 only)
Reviewed by: manu, des
Differential Revision: https://reviews.freebsd.org/D55412
Sponsored by: https://www.patreon.com/bsdivy
Set the MK_LLVM_LINK_STATIC_LIBRARIES knob to "yes" when building the
native-xtools target. This reverts to the behaviour prior to
2e47f35be5.
This avoids a build failure that occurs otherwise, where compilation
fails looking for a libllvmprivate.so that was not built.
It is unclear if this addresses the issue in all instances---some
replies in the PRs indicate otherwise. Still, some report success, and
in my own testing this fixed creation of a cross-compiled poudriere
jail. Commit this while we continue to investigate...
PR: 286710, 291409
Tested by: marck, [email protected]
Reviewed by: emaste
MFC after: 3 days
Fixes: 2e47f35be5 ("Convert libllvm, libclang and liblldb into private shared libraries").
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D54815
If BUILDKERNELS contains a single kernel, and NO_INSTALLEXTRAKERNELS=no,
then ${BUILDKERNELS:[2..-1]} returns the first kernel in the list and
that kernel ends up in both INSTALLKERNEL and INSTALLEXTRAKERNELS.
This breaks the package build because it means we generate that package
twice, once with an incorrect install path of /boot/kernel.NAME instead
of /boot/kernel.
Only set INSTALLEXTRAKERNELS if we're building at least two kernels.
Fixes: 2c0e9a76c1 ("Makefile.inc1: Make NO_INSTALLKERNEL less of a hack")
Reviewed by: jrtc27
Sponsored by: https://www.patreon.com/bsdivy
Differential Revision: https://reviews.freebsd.org/D54539
Commit 74a6bb524e added a check to install{world,kernel} to avoid
people accidentally running this on pkgbase systems and leaving their
system broken. This had two issues:
(1) The warning was not sufficiently scary, leading people to think
this was safe to do as long as they set DESTDIR=/.
(2) The installkernel check was too strict, and prevented installing
kernels that don't conflict with packaged kernels.
Fix (1) by rewording the warning to be scarier, and while here, add
two new variables (ALLOW_PKGBASE_INSTALL{WORLD,KERNEL}) which could
be set in /etc/make.conf for people who want to break their systems
by default.
Fix (2) by improving the installkernel check to see if the kernel(s)
we're actually installing are packaged. This means a new kernel can
be installed to /boot/kernel as long as there's no packaged kernel
installed there. This check understands INSTKERNNAME, so if there
is a packaged kernel in /boot/kernel, the new kernel can still be
installed using INSTKERNNAME=testkernel (or whatever).
MFC after: 2 weeks
Reported by: christos, jhb, others
Reviewed by: emaste, christos (previous version)
Sponsored by: https://www.patreon.com/bsdivy
Differential Revision: https://reviews.freebsd.org/D54346
Similar to the build epoch check, cache a list of source options in the
object tree, and force a clean build if the cached list does not match
the current list, after filtering out options which are known not to
affect the build (e.g. CLEAN, TESTS, WARNS).
This also adds a DEPEND_CLEANUP option (which defaults to yes unless
the CLEAN option is set) which can be used to skip depend-cleanup for
faster incremental builds.
MFC after: 1 week
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D52011
Rather than adding a dummy entry which requires everyone to know to skip
it, introduce a new INSTALLEXTRAKERNELS which contains the set of
kernels to install as kernel.KERNCONF.
Reviewed by: ivy
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D54291
Commit 35dd53a9e1 ("librdmacm/libibverbs: Statically bound libbnxtre.so.1 to rping")
causes a build failure, since src.libnames.mk declares a depencency on
libbnxtre.so, but the libpcap Makefile and prebuild lib dependencies
were not updated accordingly. Fix the declarations.
Fixes: 35dd53a9e1 ("librdmacm/libibverbs: Statically bound libbnxtre.so.1 to rping")
Reviewed by: ssaxena
Differential Revision: https://reviews.freebsd.org/D54048
We need to preserve modification times on bootstrap tools, but `cp -p`
also tries to preserve flags, which fails if OBJROOT is on NFS. A -N
option was added to cp for this purpose, but trying to use that would
break cross-building on hosts that don't have that option. The best
remaining option is `install -p`, which we already assume is present.
PR: 275030
Reviewed by: imp, emaste
Differential Revision: https://reviews.freebsd.org/D53751
In order to make sure that man pages are all installed before we run
makewhatis to generate mandoc.db files, we have long placed etc at the
end of the list of subdirectories being recursed into by the build.
In order to support installworld -jN, a .WAIT was more recently added
here.
With the recent adoption by the release engineering team of parallel
*release* builds (aka 'make release -jN') it is now also necessary to
add the same .WAIT before recursing for the 'distribute' target, as we
otherwise end up with distribution sets containing incomplete mandoc.db
files.
Reviewed by: bdrewery
PR: 289683
MFC after: 3 days
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D53533
Colin Percival