The ThreadPool destructor joins all threads. Therefore, this
destructor should be called before the destructors of all static
variables that are used in those threads.
This was available on Serenity by sending SIGINFO to RequestServer. Add
an IPC so we may see this info while running the browser.
(cherry picked from commit e75ba44f377b9ba15186a0af89a6c52777038527)
Previously we made one thread pool per ipc client, which is excessive
and can eat up all the available fds very fast.
This is a manual port of commit 18499c4eac301db7e8915284f33766ca96cdeef2
from Ladybird.
Ultimately it's RequestServer who knows which kind of encodings it can
handle and decompress, so let's have it set the Accept-Encoding.
(cherry picked from commit 2c918b540db2efcef4751084592ae1cf3be0999f)
This allows RS to start connections in parallel without actively waiting
for any possible handshakes.
Doing so gives us a nearly-3x speedup on the average connection latency.
No longer just for response headers! The same type is obviously useful
and ergonomic when making requests as well.
(cherry picked from commit 260c5c50ad19f19d0d4c30984e512f56c055ecff)
Updated various SerenityOS components to make it build.
Instead of using a HashMap<ByteString, ByteString, CaseInsensitive...>
everywhere, we now encapsulate this in a class.
Even better, the new class also allows keeping track of multiple headers
with the same name! This will make it possible for HTTP responses to
actually retain all their headers on the perilous journey from
RequestServer to LibWeb.
(cherry picked from commit e636851481eabdf00953573a5eb459ee52feeacc)
Updated various SerenityOS components to make it build.
Fetch: Make sure we iterate over HeaderMap's headers()
This fixes a build failure when built with CMake option
'-DENABLE_ALL_THE_DEBUG_MACROS=ON'.
(cherry picked from commit c51d01bea712d75f9b2cd700be942935044e49b4)
A connection's socket is allowed to be null if it's being recreated
after a failed connect(), this was handled correctly in
request_did_finish but not in recreate_socket_if_needed; this commit
fixes this oversight.
Most of IPC::Connection is thread-safe, but the responsiveness timer is
very much not so, this commit makes sure only one thread can send stuff
through IPC to avoid having threads racing in IPC::Connection.
This is simply less work than making sure everything IPC::Connection
uses (now and later) is thread-safe.
Previously RS handled all the requests in an event loop, leading to
issues with connections being started in the middle of other connections
being started (and potentially blowing up the stack), ultimately causing
requests to be delayed because of other requests.
This commit reworks the way we handle these (specifically starting
connections) by first serialising the requests, and then performing them
in multiple threads concurrently; which yields a significant loading
performance and reliability increase.
Add factory functions to distinguish between when the owner of the File
wants to transfer ownership to the new IPC object (adopt) or to send a
copy of the same fd to the IPC peer (clone).
This behavior is more intuitive than the previous behavior. Previously,
an IPC::File would default to a shallow clone of the file descriptor,
only *actually* calling dup(2) for the fd when encoding or it into an
IPC MessageBuffer. Now the dup(2) for the fd is explicit in the clone_fd
factory function.
Part of this issue was fixed in 89877b3f40
but that only addressed the first layer of deferred_invoke, ignoring the
second one (which would cause a race if a request was sent to a host
immediately following a timeout event from the same host).
Fixes#23840.
This URL library ends up being a relatively fundamental base library of
the system, as LibCore depends on LibURL.
This change has two main benefits:
* Moving AK back more towards being an agnostic library that can
be used between the kernel and userspace. URL has never really fit
that description - and is not used in the kernel.
* URL _should_ depend on LibUnicode, as it needs punnycode support.
However, it's not really possible to do this inside of AK as it can't
depend on any external library. This change brings us a little closer
to being able to do that, but unfortunately we aren't there quite
yet, as the code generators depend on LibCore.
This causes a behavior change in which the read FD is now non-blocking.
This is intentional, as this change avoids a deadlock between RS and
WebContent, where WC could block while reading from the request FD,
while RS is blocked sending a message to WC.
The underlying issue here isn't quite understood yet, but for some
reason, when we defer this connection, we ultimately end up blocking
indefinitely on macOS when a subsequent StartRequest message tries to
send its request FD over IPC. We should continue investigating that
issue, but for now, this lets us use RequestServer more reliably on
macOS.
It's no change in application behavior to have these objects owned by
the function-scope static map in Protocol.cpp, while allowing us to
remove some ugly FIXMEs from time immemorial.
This makes it so the clients don't have to wait for RS to become
responsive, potentially allowing them to do other things while RS
handles the connections.
Fixes#23306.
This really only implements a heuristic, assuming that HTTP/1.0 servers
cannot handle having multiple active connections; this assumption has
lots of false positives, but ultimately HTTP/1.0 is an out-of-date HTTP
version and people using it should just switch to a newer standard
anyway.
Specifically, python's "SimpleHTTPRequestHandler" utilises a
single-threaded HTTP/1.0 server, which means no keepalive and more
importantly, hangs and races with more than a single connection present.
This commit makes it so we serialise all requests to servers that are
known to serve only a single request per connection (aka HTTP/1.0 with
our setup, as we unconditionally request keepalive)
Fixes#22582
Previously, the job and the (cache of them) would lead to a UAF, as
after `.start()` was called on the job it'd be immediately destroyed.
Example of previous bug:
```
// Note due to the cache &jobA == &jobB
auto& jobA = Job::ensure("https://r.bing.com/");
auto& jobB = Job::ensure("https://r.bing.com/");
// Previously, the first .start() free'd the job
jobA.start();
// So the second .start() was a UAF
jobB.start();
```
This commit un-deprecates DeprecatedString, and repurposes it as a byte
string.
As the null state has already been removed, there are no other
particularly hairy blockers in repurposing this type as a byte string
(what it _really_ is).
This commit is auto-generated:
$ xs=$(ack -l \bDeprecatedString\b\|deprecated_string AK Userland \
Meta Ports Ladybird Tests Kernel)
$ perl -pie 's/\bDeprecatedString\b/ByteString/g;
s/deprecated_string/byte_string/g' $xs
$ clang-format --style=file -i \
$(git diff --name-only | grep \.cpp\|\.h)
$ gn format $(git ls-files '*.gn' '*.gni')
Prior to this commit, it was possible to get a socket stuck in a state
with enqueued entries, waiting for a nonexistent request to finish.
This state could be entered by issuing a request immediately after the
completion of another one, and before deferred_invoke execution in the
event loop.
This commit closes this hole by making sure the socket is never in a
state where it can queue requests without an active job.
Since 2023-09-08, Clang trunk has had a bug which causes a segfault when
evaluating certain `requires` expressions inside templated lambdas.
There isn't an imminent fix on the horizon, so let's work around the
issue by specifying the type of the offending lambda arguments
explicitly.
See https://github.com/llvm/llvm-project/issues/67260
In order to follow spec text to achieve this, we need to change the
underlying representation of a host in AK::URL to deserialized format.
Before this, we were parsing the host and then immediately serializing
it again.
Making that change resulted in a whole bunch of fallout.
After this change, callers can access the serialized data through
this concept-host-serializer. The functional end result of this
change is that IPv6 hosts are now correctly serialized to be
surrounded with '[' and ']'.
This commit changes the variables used to represent the size and
progress of downloads from u32 to u64. This allows `pro` and
`Browser` to report the total size and progress of a download
correctly for downloads larger than 4GiB.
Starting a gemini request creates a pipe for the output stream for the
response, and a Core::Stream::File object is created from that pipe.
Previously, the length of the response was computed by calling
output_stream.size() which used lseek on the file descriptor. Doing that
returned an error from lseek. Computing the value by counting the
received bytes (via Gemini::Job::response_length) avoids that crash.
We have a new, improved string type coming up in AK (OOM aware, no null
state), and while it's going to use UTF-8, the name UTF8String is a
mouthful - so let's free up the String name by renaming the existing
class.
Making the old one have an annoying name will hopefully also help with
quick adoption :^)
Sönke Holz