121 Commits
Author SHA1 Message Date
Martin Baulig cf609799f0 [System]: Allow reading fully buffered HttpWebResponse body… (#15220)
[System]: Allow reading fully buffered `HttpWebResponse` body after close.

If we have already read the entire response body, then we should allow the
user to read it even after the request has been aborted.

* `WebConnectionStream.Read ()`: Move `Operation.ThrowIfClosedOrDisposed ()`
   call down after argument checks and call a new protected abstract method
   called `TryReadFromBufferedContent ()` prior to it.

* `WebResponseStream.TryReadFromBufferedContent ()`: Implement it here.
  If the entire response body has been buffered, then `bufferedEntireContent`
  is true and `innerStream is BufferedReadStream` and we can call the new
  `BufferedReadStream.TryReadFromBuffer ()`.

* `WebResponseStream.ReadAllAsync ()`: Set `bufferedEntireContent = true`
   after reading the entire response body.

* `BufferedReadStream.TryReadFromBuffer ()`: New internal method.

Fixes #10645.
2019-07-09 19:26:42 +02:00
Martin Baulig d125afca84 [System]: The big WebConnectionStream cleanup. (#8569)
* [System]: The big WebConnectionStream cleanup.

`WebResponseStream` has been broken down into smaller pieces and now
fully supports GZip / Deflate decompression.

The entire `contentLength`, `readBuffer` and `totalRead` logic has been
replaced by a nested layer of `WebReadStream` classes.

* `WebReadStream` is a new abstract base class.

* `BufferedReadStream` handles the situation where we have some buffered
  data and an (optional) inner stream to read from.

* `ContentDecodeStream` takes care of GZip / Deflate decompression.

* `FixedSizeReadStream` handles the `contentLength` logic - ie. making sure
  we are only reading up to `contentLength` bytes from the stream.

* The old `MonoChunkStream` has been renamed into `MonoChunkParser` (as it was
  really just a parser and not a stream) and a new `MonoChunkStream` has been
  added that's based on `WebReadStream`.

* Remove HttpWebRequestTest.TestLargeDataReading().

This test was using Reflection to poke around implementation internals.

* [csproj] Update project files

* Cosmetic cleanups.
2018-05-10 20:17:21 -04:00
Martin Baulig 9648645fb3 [System]: Enable some System.Net.Requests tests from CoreFX. (#7080)
* [System]: Enable some System.Net.Requests tests from CoreFX.

* tests.make: add `$(XTEST_HARNESS_PATH)` to `TEST_MONO_PATH`,
  so we can find xunit.assert.

* corefx/SR.cs: add `System.Net.Requests` resources from corefx.

* HttpWebRequest: See below for detailed summary.

* Match parameter names of overridden base class members.

* Fixed order of some argument / state checks to match the
  tested behavior.

* Most async methods now throw exceptions instead of returning
  faulted tasks when called with incorrect arguments.

* HttpWebResponse.ContentType: use `string.empty` as default.

* WebConnectionStream.FlushAsync: override to return a finished task.

* HttpWebRequestTest.AllowReadStreamBuffering: removed this test.

* test-helpers/RemoteExecutorConsoleApp: silently ignore compiler-
  generated helper methods.

Enabled tests from CoreFX:
--------------------------

Added the following tests from external/corefx/src/System.Net.Requests/tests:

* AuthorizationTest, FileWebRequestTest, GlobalProxySelectionTest,
  HttpRequestCachePolicyTest, HttpWebRequestTest, HttpWebResponseHeaderTest,
  HttpWebResponseTest, LoggingTest, RequestStreamTest, WebRequestTest

Not working yet:

* AuthenticationManagerTest, FtpWebRequestTest

We are currently using
../../../external/corefx/src/System.Net.Requests/tests/*.cs:AuthenticationManagerTest.cs,FtpWebRequestTest.cs

Summary of HttpWebRequest changes:
----------------------------------

* Host - we are now using the implementation from corefx.

* DefaultMaxResponseHeadersLength (unused) - fix default value.

* AllowReadStreamBuffering (unused) - allow this property to
  be read/written.

* ContinueTimeout - likewise.

* DefaultCachePolicy (unused) - allow this property to be
  read/written and set default value.

* DefaultMaximumErrorResponseLength (unused) - likewise.

* Connection - use implementation from corefx.

* Method - use implementation from corefx, which has additional
  error checks.

* MaximumResponseHeadersLength - use implementation from corefx,
  adding error checks.

* ReadWriteTimeout - likewise.

* ProtocolVersion - include parameter name in ArgumentException.

* TransferEncoding - use implementation from corefx.

* BeginGetRequestStream / GetRequestStreamAsync / GetRequestStream:
  throw exceptions when called with incorrect arguments rather than
  returning faulted tasks.

* Disable the corefx tests for the moment.

* Bump API snapshot submodule

* Update tests.make
2018-02-28 00:32:06 +01:00
Martin Baulig cd09ddbc55 New PR for the new web stack (#6125) 2017-12-14 10:06:47 +01:00
Ludovic Henry 52b00f2922 [WebConnection] Fix race condition between Close and BeginWrite (#4693)
* [WebConnection] Make ReadDone and InitRead instance methods to avoid passing cnc around

* [WebConnection] Inline only call to InitConnection

* [WebConnection] Fix race condition between Close and BeginWrite
2017-04-14 14:51:42 -04:00
Marek Safar 400b5eb8b4 [System] Change WebRequest::BeginGetRequestStream driven IAsyncResult::CompletedSynchronously to match .net. Fixes #32374 2016-08-17 13:47:47 +02:00
Marek Safar cc3f4c6037 [System] Replace few System.Net types with referencesource 2016-05-13 18:09:51 +02:00
Martin Baulig 9af4b08290 Revert "[System]: WebConnection: improve chunked reads and async callbacks."
This reverts commit 9027f0ad28bf733ebb8ac9964c78c22853ca93c7.
2015-10-14 19:26:06 -04:00
Martin Baulig 77e1889d20 [System]: WebConnection: improve chunked reads and async callbacks.
Make sure we always complete the async operations on error and
also send callbacks outside the lock.

(cherry picked from commit 9027f0ad28bf733ebb8ac9964c78c22853ca93c7)
2015-10-14 15:18:27 -04:00
BogdanovKirill 1abf5baf16 [WebConnectionStream] Fix connection loss for >2GB reads
When HttpWebRequest is used to download infinite (ContentLength not set) response from web server, we can see that after reading about 2 GB (Int32.MaxValue) of data, totalRead variable can be equal to contentLength (which is int32.MaxValue). Then WebConnectionStream thinks that all data is received (see soft condition totalRead >= contentLength), but this is not correct.

This change is released under the MIT license.
2015-08-26 11:09:15 +05:00
Martin Baulig dd6f03681e [System]: Fix WebConnectionStream.SetHeadersAsync() logic for #31830.
This fixes commits 8ccfe0f8ff and
c109f9ca03.

Added new test:
https://github.com/xamarin/web-tests/commit/d60bc3f637ca2bf4aa1379b839d878e224d344c8

(cherry picked from commit c133880bc4)
2015-07-30 15:41:10 +02:00
Martin Baulig bf1e02d4d4 Revert "[System] Don't assume WebConnectionStream of unknown http methods require write stream. Fixes #31830"
This reverts commit 8ccfe0f8ff.

This commit breaks the Content-Type logic for existing HTTP verbs such as for instance
using DELETE with an empty body [1].  Added two new test cases [2] and [3] to the stable
branch of the web-tests (https://github.com/xamarin/web-tests/tree/stable) for the particular
scenario from bug #31830 as well as the default HttpClient.SendAsync() without a Content [4].

The rules are as follows:

1.) HttpWebRequest adds the Content-Length header if and only if you're calling
    GetRequestStream(); a length of 0 will be used if you don't actually write
    any content.

    This is true for POST, PUT, DELETE and "obscute" verbs.

2.) HttpClient.SendAsync() always adds a Content-Length header.

Tests:

[1] https://github.com/xamarin/web-tests/blob/31b8fed0ec1283b8c0693bb14d1ac266792c1020/Xamarin.WebTests/Xamarin.WebTests.Tests/TestPost.cs#L84

[2] https://github.com/xamarin/web-tests/blob/31b8fed0ec1283b8c0693bb14d1ac266792c1020/Xamarin.WebTests/Xamarin.WebTests.Tests/TestPost.cs#L296

[3] https://github.com/xamarin/web-tests/blob/31b8fed0ec1283b8c0693bb14d1ac266792c1020/Xamarin.WebTests/Xamarin.WebTests.Tests/TestHttpClient.cs#L134

[4] https://github.com/xamarin/web-tests/blob/31b8fed0ec1283b8c0693bb14d1ac266792c1020/Xamarin.WebTests/Xamarin.WebTests.Tests/TestHttpClient.cs#L127

(cherry picked from commit 3f1ae72f1b385c3c5e4b5ac2b93d13418c020f02)
2015-07-28 21:14:03 +02:00
Marek Safar 7faba3ca52 [System] Don't assume WebConnectionStream of unknown http methods require write stream. Fixes #31830 2015-07-13 14:11:25 +02:00
Martin Baulig 20d845ca07 Merge pull request #1156 from felfert/master
Fixed Bug #21135 - Deadlock in WebConnectionStream
2015-01-22 17:21:24 +01:00
Martin Baulig 7d8798c15d [System]: WebClient.OpenWrite() must get the response on close; fixes #10163.
When the stream returned by WebClient.OpenWrite() / OpenWriteAsync() /
OpenWriteTaskAsync(), we need to call GetResponse() and read the full
response.

Unfortunately, there is no async API available for this, so this API should
not be used in async applications.
2014-08-25 18:15:49 +02:00
Fritz Elfert ce9db9db3f Fixed Bug #21135 - Deadlock in WebConnectionStream 2014-07-11 19:20:11 +02:00
Marek Safar d1d81c9cb5 [system] Fix warnings 2014-05-27 17:04:24 +02:00
Martin Baulig e49d56e8d1 [Http]: Cleanup chained async operations; bug #19161. 2014-05-21 02:26:48 +02:00
Miguel de Icaza 705da187cf [System] Fix build warnings 2014-05-04 17:15:18 -04:00
Martin Baulig 461855eddd [Http]: Clear the 'SendChunked' flag when redirecting. 2014-04-08 08:24:21 +02:00
Martin Baulig faaa8a7c94 Async-ify the remaining WebConnectionStream pieces. 2014-02-25 18:57:30 +01:00
Martin Baulig da6c88b49b [System] WebConnection and WebConnectionStream code cleanups.
* Merged WebConnection.EndWrite() and EndWrite2().

* WebConnectionStream.BeginWrite(): Use the same async-write
  callback for both chunked and non-chunked; remove the single-use
  small WriteRequestAsync() and inline it here to make the code
  more readable.

* WebConnectionStream.WriteRequestAsyncCB(): Now called for all
  async writes, call WebConnection.EndWrite() here.

The code is now more readable and easier to understand.  This is
the first part of a series of cleanups, which should eventually
lead towards full async and 100-continue.
2014-02-19 11:08:18 +01:00
Martin Baulig 5e03c107bc Put back and fix the SSL async patch.
Revert "Revert "Merge pull request #829 from symform/httpwebreq-async-ssl""

This reverts commit 56d9a057eb, which was a
revert of 35899c07ff, thus putting 35889 back.
2014-02-13 18:48:16 +01:00
Martin Baulig 4b5f5f96c8 [System]: Reset NTLM auth state when redirecting; distinguish host and proxy. 2014-02-13 15:48:44 +01:00
Martin Baulig c6a70a9586 Rewrite and fix the web stack's connection reuse and idle logic.
* ServicePoint now uses an idle timer to close idle connections.

* WebConnectionGroup has been mostly rewritten to use a linked
  list of open connections and their idle times.

  This fixes both the problem that we were leaking sockets and
  that we were not reusing existing connections.

* ServicePoint now correctly keeps track of the number of open
  connections.

* ServicePointManager now uses the correct default idle time
  of 100 seconds (previously 15 minutes).

* WebConnection uses the new WebConnectionGroup's busy logic
  to correctly flag connections as busy / reusable.
2014-02-13 14:17:14 +01:00
Martin Baulig 861d495951 Fix WebConnectionStream.CheckComplete() logic for chunked encoding. 2014-02-13 13:22:19 +01:00
Marek Safar 56d9a057eb Revert "Merge pull request #829 from symform/httpwebreq-async-ssl"
This reverts commit 35899c07ff, reversing
changes made to 80166fd02f.
2014-02-10 18:48:03 +01:00
Bassam Tabbara 90f9595967 Convert blocking operations in HttpWebRequest and SslClientStream to non-blocking operations. This fixes #15451.
More details:
  * HttpWebRequest now writes the HTTP request headers asynchronously. This will avoid blocking a thread pool thread until the headers are written.
  * SslClientStream was doing the TLS negotiations in a blocking manner. Converted the negotiation method to an async state machine where all operations are done asynchronously.
  * The test case identified in #15451 now completes quickly with less than 15 threads created, vs. 100s of threads used before this fix.

Future work if needed:
  * I did not convert SslServerStream but the same pattern could be used here.
  * SendAlert is still blocking. I thought it was not worth converting it to non-blocking.
2013-12-06 09:32:22 -08:00
Tammo 'kb' Hinrichs 170fc3aaf7 Reverted whitespace changes 2013-06-12 19:52:14 +02:00
Tammo 'kb' Hinrichs 907b13aa04 Fix race condition in WebConnectionStream
See https://bugzilla.xamarin.com/show_bug.cgi?id=1553
2013-05-08 16:34:17 +02:00
Miguel de Icaza aa4af9833a Add some defensive exceptions to WebConnectionStream to try to track down #6329 2012-11-07 13:22:18 -05:00
Miguel de Icaza a6b3391227 Add some defensive exceptions to WebConnectionStream to try to track down #6329 2012-11-07 13:22:12 -05:00
Martin Baulig f7db37a5ed Fix #5655: Always send HTTP headers for WebDAV methods. 2012-09-05 22:20:00 +02:00
Miguel de Icaza f48ceb9860 [System] Removal of the NET_2_0 in the source code
Modulo the GUID in the assembly and the GUID embedded in some
field initializers that use the assembly GUID for their name, the
assemblies are idential before and after the patch.
2012-06-02 13:16:40 -04:00
Gonzalo Paniagua Javier 4cb9ef3186 DELETE can have a request stream
Fixes bug #3276.
2012-02-27 10:53:41 -05:00
Gonzalo Paniagua Javier 4b72b76703 Return the correct length for input streams 2011-10-17 00:45:17 -04:00
Gonzalo Paniagua Javier 8e67b8c2fc Don't send the buffered request until GetResponse is called
Allows changing some header values after writing the request until the
	call to (Begin)GetResponse()
2011-10-16 01:57:41 -04:00
Gonzalo Paniagua Javier b1248b22e4 [HttpWebRequest] Can't ignore erros here 2011-02-10 17:25:15 -05:00
Gonzalo Paniagua Javier b92b165ddb [HttpWebRequest] Error handling for POST/PUT
When we get an error from the server before writing an entire
	unbuffered response, the server might close the connection leading to
	write errors. Ignore those and return the actual HTTP status code from
	the original server error.
2011-02-10 17:25:15 -05:00
Gonzalo Paniagua Javier 604d5faa0b More StringComparison.Ordinal
Use StringComparison.Ordinal when calling IndexOf to find the index of
	an ASCII string.
2010-10-31 17:23:18 -04:00
Gonzalo Paniagua Javier b24939c61c Create only 1 callback instance for read/write
Read/Write were creating one callback instance per call. Now we keep
	the same delegate around for all the calls.
2010-10-02 01:41:40 -04:00
Gonzalo Paniagua Javier 6a4600c8ec Adapt to changes in exceptions policy for TP.
These exception were being thrown, sometimes, in a threadpool thread and then
ignored.
2010-07-30 01:54:00 -04:00
Gonzalo Paniagua Javier eafc6c8b08 2009-10-17 Gonzalo Paniagua Javier <[email protected]>
* WebConnectionStream.cs: add 3 missing DoCallback() calls in
	EndRead/EndWrite.


svn path=/trunk/mcs/; revision=144315
2009-10-18 02:09:20 +00:00
Gonzalo Paniagua Javier ae20ad47f0 2009-08-13 Gonzalo Paniagua Javier <[email protected]>
* HttpWebRequest.cs:
	* WebConnectionStream.cs: if there's an error writing the headers from
	a buffered request, propagate the error instead of hiding it and
	causing a different one later (ie, ObjectDisposedException instead of
	the original WebException). This happened when posting without
	Content-Length set to an HTTPS url without validating the server
	certificate.


svn path=/trunk/mcs/; revision=139885
2009-08-13 22:12:18 +00:00
Gonzalo Paniagua Javier 27e633cf32 2009-07-22 Gonzalo Paniagua Javier <[email protected]>
* WebConnectionStream.cs: when sending a 0-length POST, ignore further
	writes instead of throwing.
	Fixes bug #524049.


svn path=/trunk/mcs/; revision=138447
2009-07-22 19:58:51 +00:00
Gonzalo Paniagua Javier 547e70a03b 2009-07-07 Gonzalo Paniagua Javier <[email protected]>
* WebClient.cs: set the content length when uploading values. This
	allows reusing the connection.
	* WebConnectionStream.cs: SetHeaders() takes ownership of the buffer
	and does not need to copy it any more. WriteRequest() wasn't always
	setting 'requestWritten' to true.
	* HttpWebRequest.cs: change in SetHeaders call.


svn path=/trunk/mcs/; revision=137497
2009-07-07 14:57:51 +00:00
Gert Driesen 8d403f0381 * WebConnectionStream.cs: Move and improve argument checks to match MS.
Modified CanRead and CanWrite to return false when instance is disposed.
* HttpWebRequestTest.cs: Enabled tests, and removed test that is now
covered by two separate CanTimeout tests for request and response
stream.
* HttpWebResponseStreamTest.cs: Enabled tests.

svn path=/trunk/mcs/; revision=137450
2009-07-06 19:58:00 +00:00
Gonzalo Paniagua Javier 781e4faf6c 2009-07-02 Gonzalo Paniagua Javier <[email protected]>
* WebConnection.cs: provide better error message when there's an error
	writing.
	* WebConnectionStream.cs: don't nullify the buffer until
	CheckFinalStatus() decides that there are no more authentication
	retries or redirects.


svn path=/trunk/mcs/; revision=137332
2009-07-02 23:03:11 +00:00
Gonzalo Paniagua Javier 77e7d3d183 fix
svn path=/trunk/mcs/; revision=137239
2009-07-02 07:34:09 +00:00
Gonzalo Paniagua Javier 7ff2c1978d 2009-07-02 Gonzalo Paniagua Javier <[email protected]>
* HttpWebRequest.cs: avoid duplicating Accept-Encoding if GetHeaders()
	is called twice.
	* WebConnectionStream.cs: don't send a 0 content length for non-write
	requests.


svn path=/trunk/mcs/; revision=137233
2009-07-02 07:33:12 +00:00