pkgbase splits programs and their tests into separate packages (e.g.
unifdef/yacc/indent/file2c -> toolchain, jail -> jail, etc), while their
tests always land in the generic tests package. Installing just the
tests package therefore leaves these suites unable to find the binary
they exercise, and they fail confusingly instead of skipping.
Rather than force all packages to be installed for the tests package,
make the tests cope with the missing packages. Add require.progs (or,
for TAP/PLAIN-style suites with no atf_set hook: TEST_METADATA
required_programs) for the binary under test in: bectl, certctl,
ctfconvert, dhclient, file2c, indent, ipfw, jail, lastcomm, mixer,
newsyslog, nvmecontrol, pfctl, praudit, sa, syslogd, tunefs, unifdef,
yacc, ztest.
I skipped adding data for all the binaries in the base runtime
package for simplicity.
Assisted-by: Claude Code (Sonnet 5)
Sponsored by: Netflix
Reviewed by: ngie
Differential Revision: https://reviews.freebsd.org/D58262
While here, use caph_rights_limit(), as syslogd already uses
caph_enter().
PR: 298104
Reported by: mi
Fixes: 24816abb87 ("syslogd: Limit rights on procdescs")
MFC after: 3 days
On SIGHUP reload, closelogfiles() frees each F_PIPE filed even when its
pipe process is still running. close_filed() sets f_type to F_UNUSED
before the check, so the condition f_type != F_PIPE is always true and
the filed is freed while its process descriptor is still on the dead
queue and registered in the kqueue. When the child later exits, the
NOTE_EXIT handler dereferences the freed filed (use-after-free) and
never closes the process descriptor, leaving the pipe child as a
persistent zombie.
Capture whether the filed is a pipe with an active process descriptor
before calling close_filed(), and defer the free in that case so the
NOTE_EXIT handler can reap the child and free the filed.
Reviewed by: markj
Fixes: 95381c0139 (syslogd: Use process descriptors)
Differential Revision: https://reviews.freebsd.org/D59319
Since syslogd was converted to run in a Capsicum sandbox, it needs to
explicitly connect() its forwarding sockets rather than using sendmsg().
At the time syslogd starts during boot, some of its forwarding
destinations may not be routable, in which case connect() fails.
Fix this by making connect() failures non-fatal, and use cap_net to
lazily connect sockets once something actually tries logging to the
destination.
Add a regression test.
Reported by: ae
Reviewed by: ae
Discussed with: jfree
Fixes: 4ecbee2760 ("syslogd: Open forwarding socket descriptors")
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D57394
* Instead of an error string, return the usual 0 or -1 and let the caller
figure out what, if anything, to tell the user.
* Avoid string manipulations by opening /dev first and using openat()
with O_RESOLVE_BENEATH.
* Add a boolean argument which, if false, causes ttymsg() to return
without sending the message if the tty's group-writable bit is not
set. This saves programs that respect this setting (like syslogd(8))
from having to check before calling ttymsg().
* Update all callers.
The observable effect of this change is minimal except for slightly
different error messages when ttymsg() fails. However, syslogd(8) will
no longer print spurious error messages on the console after trying and
failing to write a log message to an X11 session.
PR: 295171
MFC after: 1 week
Reviewed by: jfree, markj
Differential Revision: https://reviews.freebsd.org/D57018
nvlist_take_string_array(9) takes ownership of the array and its
strings. casper_ttymsg() freed neither, leaking memory on every
F_CONSOLE and F_TTY message. On long-running systems with high
error-rate syslog traffic routed to /dev/console, syslogd.casper grew
to hundreds of MB.
Use nvlist_get_string_array(9) to borrow the array instead. Update
casper_wallmsg() similarly.
Approved by: src (des)
Closes: https://github.com/freebsd/freebsd-src/pull/2222
Fixes: 61a29eca55 ("syslogd: Log messages using libcasper")
MFC after: 3 days
MFC to: stable/15
PR: 295488
Reported by: Pat Maddox <[email protected]>
Reviewed by: markj
Tested by: dch
This change fixes Bug 293879, where ereregex filters in syslogd
did not handle alternation correctly.
The issue appears to come from mixing up two different kinds of
flags: syslogd's internal serialized filter flags in
usr.sbin/syslogd/syslogd.h, and the regex compilation flags from
include/regex.h. ereregex was storing REG_EXTENDED in cmp_flags,
even though cmp_flags is meant to carry syslogd's own FILT_FLAG_*
values for configuration serialization and reconstruction.
REG_EXTENDED has the same bit value as FILT_FLAG_EXCLUDE, so the
filter could be reconstructed with the wrong semantics.
The fix stores FILT_FLAG_EXTENDED instead, allowing syslogd to
correctly REG_EXTENDED when compiling the regex.
A test was also added for both ereregex and !ereregex filters.
PR: 293879
Signed-off-by: tzyt <[email protected]>
Fixes: 2567168dc4
MFC after: 1 week
Reviewed-by: markj, ngie
Pull-Request: https://github.com/freebsd/freebsd-src/pull/2088
Wordsmith mentions throughout the manual of syslog and dmesg, to clearly
differentiate them in an externally consisteny way, increasing operator
onboarding speed and elegance.
The daemon that handles general system messages, syslog, describes them
as "system messages", and "messages" is the standard filename. Rewrite
syslog related manual titles to align search results with this, and hier
entries to align the index. Use care to maintain keywords and not add
extra lines. Newsyslog trades "maintain" with "rotate" for visibility.
MFC after: 3 days
Reviewed by: markj
Closes: https://github.com/freebsd/freebsd-src/pull/2067
Normally, syslogd reacts only to SIGTERM, and ignores SIGINT and SIGQUIT
unless in debug mode. Extend that to also apply when running in the
foreground. Take this opportunity to comment the event loop.
MFC after: 1 week
Reviewed by: jfree
Differential Revision: https://reviews.freebsd.org/D55886
Make parse_selector() print a warning to stderr and continue parsing the
config if it encounters an invalid facility or priority. Note that
because the parsing is done from a casper service, there isn't a good
mechanism to log errors; the warnings are visible only when syslogd is
started in debug mode.
Reported by: Doug Hardie <[email protected]>
MFC after: 1 week
Fixes: f4b4a10abb ("syslogd: Move selector parsing into its own function")
Reviewed by: jfree, jlduran, eugen, delphij
Differential Revision: https://reviews.freebsd.org/D55033
syslogd_start() waits for the local log socket to appear before
returning, to ensure that the daemon is ready to handle log messages.
Some tests start two daemons, so by default the socket already exists
when the second daemon is started, so syslogd_start() returns early.
The test subsequently sends a message to this second daemon, which
sometimes isn't ready.
Define a separate log socket for the second daemon. Add a check to
syslogd_start() to help catch this type of bug.
Reviewed by: jlduran
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D54800
- In syslogd_start(), assign the lo0 address in the specified jail.
- Use the correct netmask.
Reviewed by: jlduran
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D54799
I occasionally see failures in the syslogd test suite. The problem is
that the tests are racy: they send a message using logger(1), then
immediately check whether the message was logged to a log file. If the
syslogd instance under test doesn't get a chance to run before the
second step, the test fails.
This change reworks things to avoid the race while minimizing the amount
of time sleeping.
1) Each test uses a single logfile, so have them use a new common
variable, SYSLOGD_LOGFILE, instead of something test-specific.
2) In syslogd_start(), if the configuration references SYSLOGD_LOGFILE,
wait for it to be created by syslogd before returning.
3) Add a helper syslogd_check_log(), to check for a given log entry in
the last line of SYSLOGD_LOGFILE, instead of using atf_check
directly.
4) In syslogd_check_log(), poll the logfile until the desired log entry
appears, or the test times out.
With this change, I was able to run the tests 1000 times in a loop with
4-way parallelism without seeing any test failures. Without the change
I usually get a failure within 10 loops.
Reviewed by: jlduran
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D54779
This is just for consistency with all other logger(1) invocations, which
happen from the syslogd_log() function.
No functional change intended.
Reviewed by: jlduran
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D54778
Pipe actions spawn a process based on the command provided in the
syslogd configuration file. When a HUP signal is received, enter
the process into the deadq instead of immediately killing it.
This matches the behavior of syslogd prior to it being Capsicumized.
Fixes: d2d180fb77
This was previously deprecated and is slated for removal in 15.0.
Users who still need ftpd(8) can install the ftp/freebsd-ftpd port.
Retain the ftp(d) PAM services since other FTP daemons use them.
Update /etc/inetd.conf to point to /usr/local.
Add ftpd to ObsoleteFiles, but do not list configuration files since
users may want to preserve these to use with the freebsd-ftpd port.
There is still some language in the manual referring to ftpd(8)
which is relevant to the port, which has been retained but updated
to reference the port.
MFC after: 3 days
Relnotes: yes
Reviewed by: cperciva
Differential Revision: https://reviews.freebsd.org/D52739
This moves some math to where it logically makes more sense for skipping
over the ": " separator in the message formatted "app[PID]: MSG".
No functional change.
Fixes: 18bcf5a0 ("Restore local kernel "prog" filtering")
Sponsored by: Dell Inc.
When we're forwarding syslog messages to a remote host we can get temporary
failures such as EAGAIN or ECONNREFUSED from the sendmsg() call. These are not
fatal. Future attempts to send might succeed. Don't mark the remote target as
F_UNUSED so we'll try to log again on future messages.
While here remove the commented out ENOBUFS case, because we actually handle
that (by ignoring it, so we retry with future logs).
Reviewed by: markj, jfree
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D51995
Using .Aq is wrong here, because the protocol says the prefix must use
ASCII <>.
MFC after: 3 days
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D51880
Looks like during the capsicum setup the return value for cap_syslogd
is checked twice rather than checking the return value of cap_net
after initializing the casper service.
Signed-off-by: Nathan Huff <[email protected]>
Fixes: 897240c7d6 ("syslogd: Enter capability mode")
Pull-request: https://github.com/freebsd/freebsd-src/pull/1796
Take a cue from vnet.subr and abstract module loading, jail creation and
cleanup.
Note that the way jails are created differs slightly from vnet.subr, to
maintain the current style of the tests.
Reported by: Jenkins
Reviewed by: markj
Approved by: emaste (mentor)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D48949
Prior to commit 4ecbee2760, syslogd used its listening socket(s) to
forward messages to remote hosts, when so configured. As a consequence,
they are sent from the address+port to which those sockets are bound,
typically 0.0.0.0:514.
When in capability mode, sendto() is not permitted, so we instead
pre-create sockets and connect them to the forwarding addresses, letting
the kernel pick an ephemeral source port. However, this doesn't match
syslogd's previous behaviour, breaking some setups.
So, restore the old behaviour by binding forwarding sockets to the
addresses on which syslogd is listening. Since we cannot use the same
sockets for receiving messages and also for forwarding them, use
SO_REUSEPORT to enable duplicate bindings to port 514, relying on the
existing behaviour that the first socket bound to that port is the one
that actually receives messages.
Add some regression tests to cover this and related functionality of
syslogd's -a option.
Reviewed by: jfree
Reported by: Michael Butler <[email protected]>
Fixes: 4ecbee2760 ("syslogd: Open forwarding socket descriptors")
Differential Revision: https://reviews.freebsd.org/D48222
- nvlist_append_nvlist_array() makes a copy of the input nvlist, so the
pattern of nvlist_append_nvlist_array(... cfline(...)) would leak
memory and descriptors. Pass the entire config nvlist to cfline()
instead since this is needed for a future change.
- In parse_action(), free the linked list returned by getaddrinfo().
- Remove some checks at the beginning of close_filed(). For some log
types we'll always have f->f_file == -1, in which case we wouldn't
release other resources, such as forwarding sockets.
- After converting a filed to an nvlist, free the filed resources.
Fixes: 2567168dc4 ("syslogd: Refresh configuration using libcasper")
Reviewed by: jfree
Differential Revision: https://reviews.freebsd.org/D48250
The syslogd bsd/rfc3164 message format does not strictly conform to the
format recommended in RFC 3164 when syslog messages are forwarded from
remote hosts.
The new rfc3164-strict format generates messages that conform with the
RFC 3164 recommended format. The existing format is maintained for
backwards compatibility.
PR: 220246
Reviewed by: markj
MFC after: 3 weeks
Pull Request: https://github.com/freebsd/freebsd-src/pull/1536
RFC 3164 and RFC 5424 syslog message format tests and minor refactor of
existing tests to reuse components in new tests.
Reviewed by: markj
MFC after: 3 weeks
Pull Request: https://github.com/freebsd/freebsd-src/pull/1536
- Tests have been updated to use "jail" execution environment,
is_exclusive="true" is no longer required.
- tests/Makefile update to style.Makefile(5).
Reviewed by: markj
MFC after: 3 weeks
Pull Request: https://github.com/freebsd/freebsd-src/pull/1536
- When forwaring a message, don't assume that f->f_num_addr_fds > 0.
- Avoid calling free() on a stack-allocated object.
Fixes: 4ecbee2760 ("syslogd: Open forwarding socket descriptors")
Fixes: f4b4a10abb ("syslogd: Move selector parsing into its own function")
Reviewed by: jfree
Differential Revision: https://reviews.freebsd.org/D47844
The basic case previously tested if messages are correctly logged over
UNIX, INET, and INET6 transport. This single case can be split up into
three separate cases to decrease code complexity and offer more granular
results.
Both INET and INET6 cases will be skipped if the kernel does not
support the corresponding transport.
Reviewed by: zlei, markj
Differential Revision: https://reviews.freebsd.org/D47650
Test communication between hosts by setting up two syslogd instances,
each binded to their own addresses, and sending a message from one to
the other. The tests passes if the message was delivered and logged
successfully.
This required some tweaking to the syslogd start and stop routines so
they could handle launching multiple syslogd instances with different
runtime files.
Run the tests in jails with an isolated VNET so they don't fail due to
an address bind collision in the host's VNET.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D47649
Previously, when forwarding a message to a remote address, the target's
addrinfo was saved at config-parse-time. When message-deliver-time came,
the message's addrinfo was passed into sendmsg(2) and delivered by the
first available inet socket.
Passing an addrinfo into sendmsg(2) is prohibited in Capsicum capability
mode, so sockets are now opened and connected to their remote peers at
config-parse-time when executing outside of the capability sandbox.
These connected socket descriptors are saved and passed into sendmsg(2),
allowing forwarding to be performed inside of the capability sandbox.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D47104
For each new pipe process, add its process descriptor into the kqueue
with the EVFILT_PROCDESC filter and NOTE_EXIT event. When the pipe
process exits, the main kqueue loop will catch this, logging exit errors
and cleaning up the pipe process' filed node.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D41477
Create syslogd_cap_enter() function, which puts syslogd in capability
mode and opens necessary libcasper channels for resource acquisition.
The `syslogd.casper` and `system.net` services are used.
Substitute capability-mode-unfriendly functions with their libcasper
`cap_` equivalents.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D41467
Do not update local timezone data upon reload when syslogd is compiled
with libcasper support. Timezone files are pre-fetched using the
caph_cache_tzdata() function before entering capability mode. Attempts
to re-fetch these files will result in a capability violation.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D41466
Some logging operations require access to external resources to
complete. Logging to F_WALL requires on-demand access to the user
accounting database. Logging to F_CONSOLE requires access to the
console. Logging to F_PIPE prompts execution of a command outside
of capability mode.
These operations cannot be performed in capability mode, so the
"p_open", "ttymsg", and "wallmsg" commands may be sent to libcasper to
circumvent these limitations.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D41465
When a SIGHUP signal is sent to syslogd, the configuration is reparsed,
leading to new resource acquisition.
If syslogd is running in capability mode and a SIGHUP is received, new
resources cannot be acquired. To mitigate this issue, libcasper is used
to parse the configuration.
The libcasper process runs outside of capability mode and is capable of
parsing syslogd's configuration and obtaining new resources. These
resources are then sent to the syslogd process via nvlist.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D41464
Do not use macros to access unioned struct members. Use an anonymous
union and access union member names directly.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D41703
Once in capability mode, resource acquisition is not possible. To
mitigate this, introduce a libcasper service that is responsible for
fetching and returning new, requested resources to syslogd.
Some resources must be packed into an nvlist to be properly transferred
between syslogd and the libcasper process. The filed_to_nvlist() and
nvlist_to_filed() functions are included to ease the packing process for
filed structures.
Two additional syslogd.h and syslogd_cap.h header files are included
with shared declarations.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D41463
When bind() is called, the process umask is applied, so one has to
either clear the umask before binding or call chmod() to add permissions
after the fact. Do the former here to ensure that the socket always has
the correct mode.
Reported by: Lexi Winter <[email protected]>
Fixes: 2b8c3a05e0 ("syslogd: Set unix socket modes atomically")
This can arise if the jail doesn't have networking configured, and if
-ss is specified, syslogd won't listen on port 514 anyway.
Add a regression test case for this as well.
PR: 238006
MFC after: 1 month
We've ifdef'd out the copyright strings for some time now. Go ahead and
remove the ifdefs. Plus whatever other detritis was left over from other
recent removals. These copyright strings are present in the comments and
are largely from CSRG's attempt at adding their copyright to every
binary file (which modern interpretations of the license doesn't
require).
Sponsored by: Netflix
Warner Losh