[System]: Allow reading fully buffered `HttpWebResponse` body after close.
If we have already read the entire response body, then we should allow the
user to read it even after the request has been aborted.
* `WebConnectionStream.Read ()`: Move `Operation.ThrowIfClosedOrDisposed ()`
call down after argument checks and call a new protected abstract method
called `TryReadFromBufferedContent ()` prior to it.
* `WebResponseStream.TryReadFromBufferedContent ()`: Implement it here.
If the entire response body has been buffered, then `bufferedEntireContent`
is true and `innerStream is BufferedReadStream` and we can call the new
`BufferedReadStream.TryReadFromBuffer ()`.
* `WebResponseStream.ReadAllAsync ()`: Set `bufferedEntireContent = true`
after reading the entire response body.
* `BufferedReadStream.TryReadFromBuffer ()`: New internal method.
Fixes#10645.
* [System]: The big WebConnectionStream cleanup.
`WebResponseStream` has been broken down into smaller pieces and now
fully supports GZip / Deflate decompression.
The entire `contentLength`, `readBuffer` and `totalRead` logic has been
replaced by a nested layer of `WebReadStream` classes.
* `WebReadStream` is a new abstract base class.
* `BufferedReadStream` handles the situation where we have some buffered
data and an (optional) inner stream to read from.
* `ContentDecodeStream` takes care of GZip / Deflate decompression.
* `FixedSizeReadStream` handles the `contentLength` logic - ie. making sure
we are only reading up to `contentLength` bytes from the stream.
* The old `MonoChunkStream` has been renamed into `MonoChunkParser` (as it was
really just a parser and not a stream) and a new `MonoChunkStream` has been
added that's based on `WebReadStream`.
* Remove HttpWebRequestTest.TestLargeDataReading().
This test was using Reflection to poke around implementation internals.
* [csproj] Update project files
* Cosmetic cleanups.
* [System]: Enable some System.Net.Requests tests from CoreFX.
* tests.make: add `$(XTEST_HARNESS_PATH)` to `TEST_MONO_PATH`,
so we can find xunit.assert.
* corefx/SR.cs: add `System.Net.Requests` resources from corefx.
* HttpWebRequest: See below for detailed summary.
* Match parameter names of overridden base class members.
* Fixed order of some argument / state checks to match the
tested behavior.
* Most async methods now throw exceptions instead of returning
faulted tasks when called with incorrect arguments.
* HttpWebResponse.ContentType: use `string.empty` as default.
* WebConnectionStream.FlushAsync: override to return a finished task.
* HttpWebRequestTest.AllowReadStreamBuffering: removed this test.
* test-helpers/RemoteExecutorConsoleApp: silently ignore compiler-
generated helper methods.
Enabled tests from CoreFX:
--------------------------
Added the following tests from external/corefx/src/System.Net.Requests/tests:
* AuthorizationTest, FileWebRequestTest, GlobalProxySelectionTest,
HttpRequestCachePolicyTest, HttpWebRequestTest, HttpWebResponseHeaderTest,
HttpWebResponseTest, LoggingTest, RequestStreamTest, WebRequestTest
Not working yet:
* AuthenticationManagerTest, FtpWebRequestTest
We are currently using
../../../external/corefx/src/System.Net.Requests/tests/*.cs:AuthenticationManagerTest.cs,FtpWebRequestTest.cs
Summary of HttpWebRequest changes:
----------------------------------
* Host - we are now using the implementation from corefx.
* DefaultMaxResponseHeadersLength (unused) - fix default value.
* AllowReadStreamBuffering (unused) - allow this property to
be read/written.
* ContinueTimeout - likewise.
* DefaultCachePolicy (unused) - allow this property to be
read/written and set default value.
* DefaultMaximumErrorResponseLength (unused) - likewise.
* Connection - use implementation from corefx.
* Method - use implementation from corefx, which has additional
error checks.
* MaximumResponseHeadersLength - use implementation from corefx,
adding error checks.
* ReadWriteTimeout - likewise.
* ProtocolVersion - include parameter name in ArgumentException.
* TransferEncoding - use implementation from corefx.
* BeginGetRequestStream / GetRequestStreamAsync / GetRequestStream:
throw exceptions when called with incorrect arguments rather than
returning faulted tasks.
* Disable the corefx tests for the moment.
* Bump API snapshot submodule
* Update tests.make
* [WebConnection] Make ReadDone and InitRead instance methods to avoid passing cnc around
* [WebConnection] Inline only call to InitConnection
* [WebConnection] Fix race condition between Close and BeginWrite
Make sure we always complete the async operations on error and
also send callbacks outside the lock.
(cherry picked from commit 9027f0ad28bf733ebb8ac9964c78c22853ca93c7)
When HttpWebRequest is used to download infinite (ContentLength not set) response from web server, we can see that after reading about 2 GB (Int32.MaxValue) of data, totalRead variable can be equal to contentLength (which is int32.MaxValue). Then WebConnectionStream thinks that all data is received (see soft condition totalRead >= contentLength), but this is not correct.
This change is released under the MIT license.
When the stream returned by WebClient.OpenWrite() / OpenWriteAsync() /
OpenWriteTaskAsync(), we need to call GetResponse() and read the full
response.
Unfortunately, there is no async API available for this, so this API should
not be used in async applications.
* Merged WebConnection.EndWrite() and EndWrite2().
* WebConnectionStream.BeginWrite(): Use the same async-write
callback for both chunked and non-chunked; remove the single-use
small WriteRequestAsync() and inline it here to make the code
more readable.
* WebConnectionStream.WriteRequestAsyncCB(): Now called for all
async writes, call WebConnection.EndWrite() here.
The code is now more readable and easier to understand. This is
the first part of a series of cleanups, which should eventually
lead towards full async and 100-continue.
Revert "Revert "Merge pull request #829 from symform/httpwebreq-async-ssl""
This reverts commit 56d9a057eb, which was a
revert of 35899c07ff, thus putting 35889 back.
* ServicePoint now uses an idle timer to close idle connections.
* WebConnectionGroup has been mostly rewritten to use a linked
list of open connections and their idle times.
This fixes both the problem that we were leaking sockets and
that we were not reusing existing connections.
* ServicePoint now correctly keeps track of the number of open
connections.
* ServicePointManager now uses the correct default idle time
of 100 seconds (previously 15 minutes).
* WebConnection uses the new WebConnectionGroup's busy logic
to correctly flag connections as busy / reusable.
More details:
* HttpWebRequest now writes the HTTP request headers asynchronously. This will avoid blocking a thread pool thread until the headers are written.
* SslClientStream was doing the TLS negotiations in a blocking manner. Converted the negotiation method to an async state machine where all operations are done asynchronously.
* The test case identified in #15451 now completes quickly with less than 15 threads created, vs. 100s of threads used before this fix.
Future work if needed:
* I did not convert SslServerStream but the same pattern could be used here.
* SendAlert is still blocking. I thought it was not worth converting it to non-blocking.
Modulo the GUID in the assembly and the GUID embedded in some
field initializers that use the assembly GUID for their name, the
assemblies are idential before and after the patch.
When we get an error from the server before writing an entire
unbuffered response, the server might close the connection leading to
write errors. Ignore those and return the actual HTTP status code from
the original server error.
* HttpWebRequest.cs:
* WebConnectionStream.cs: if there's an error writing the headers from
a buffered request, propagate the error instead of hiding it and
causing a different one later (ie, ObjectDisposedException instead of
the original WebException). This happened when posting without
Content-Length set to an HTTPS url without validating the server
certificate.
svn path=/trunk/mcs/; revision=139885
* WebConnectionStream.cs: when sending a 0-length POST, ignore further
writes instead of throwing.
Fixes bug #524049.
svn path=/trunk/mcs/; revision=138447
* WebClient.cs: set the content length when uploading values. This
allows reusing the connection.
* WebConnectionStream.cs: SetHeaders() takes ownership of the buffer
and does not need to copy it any more. WriteRequest() wasn't always
setting 'requestWritten' to true.
* HttpWebRequest.cs: change in SetHeaders call.
svn path=/trunk/mcs/; revision=137497
Modified CanRead and CanWrite to return false when instance is disposed.
* HttpWebRequestTest.cs: Enabled tests, and removed test that is now
covered by two separate CanTimeout tests for request and response
stream.
* HttpWebResponseStreamTest.cs: Enabled tests.
svn path=/trunk/mcs/; revision=137450
* WebConnection.cs: provide better error message when there's an error
writing.
* WebConnectionStream.cs: don't nullify the buffer until
CheckFinalStatus() decides that there are no more authentication
retries or redirects.
svn path=/trunk/mcs/; revision=137332
* HttpWebRequest.cs: avoid duplicating Accept-Encoding if GetHeaders()
is called twice.
* WebConnectionStream.cs: don't send a 0 content length for non-write
requests.
svn path=/trunk/mcs/; revision=137233