16 Commits
Author SHA1 Message Date
Martin Baulig cf609799f0 [System]: Allow reading fully buffered HttpWebResponse body… (#15220)
[System]: Allow reading fully buffered `HttpWebResponse` body after close.

If we have already read the entire response body, then we should allow the
user to read it even after the request has been aborted.

* `WebConnectionStream.Read ()`: Move `Operation.ThrowIfClosedOrDisposed ()`
   call down after argument checks and call a new protected abstract method
   called `TryReadFromBufferedContent ()` prior to it.

* `WebResponseStream.TryReadFromBufferedContent ()`: Implement it here.
  If the entire response body has been buffered, then `bufferedEntireContent`
  is true and `innerStream is BufferedReadStream` and we can call the new
  `BufferedReadStream.TryReadFromBuffer ()`.

* `WebResponseStream.ReadAllAsync ()`: Set `bufferedEntireContent = true`
   after reading the entire response body.

* `BufferedReadStream.TryReadFromBuffer ()`: New internal method.

Fixes #10645.
2019-07-09 19:26:42 +02:00
Martin Baulig 13540da80b [System]: Correctly dispose inner stream in WebResponseStream. #11928. (#12459) 2019-01-21 23:24:57 +01:00
Martin Baulig 6e75b0c5aa [System]: Fix WebResponseStream.ReadAllAsyncInner() return value.
We need to call `ToArray()` instead of `GetBuffer()` to truncate the buffer
to the amount of bytes that we actually read.
2018-09-13 13:14:46 +02:00
Martin Baulig 0a8b1b2666 [System]: HttpWebRequest now throws WebExceptionStatus.RequestCanceled on abort. (#9212)
* [System]: `HttpWebRequest` now throws `WebExceptionStatus.RequestCanceled` on abort.  #9031.

Cleanup the internal `HttpWebRequest.RunWithTimeout()` function and use
`GetWebException()` everywhere to throw `WebException` with
`WebExceptionStatus.RequestCanceled` on abort.

New tests for this will be added to the web-tests by extending the existing
`Xamarin.WebTests.HttpInstrumentationTests.AbortDuringHandshake` to check the
synchronous and begin/end async APIs as well, using both `GET` and `POST`.

See https://github.com/xamarin/web-tests/blob/master/Xamarin.WebTests.Tests/Xamarin.WebTests.HttpInstrumentationTests/AbortDuringHandshake.cs.

Fixes #9031.

* Fix exception checks, setting `TransferEncoding` without `SendChunked` is supposed to throw `InvalidOperationException`.
2018-06-25 23:16:32 +02:00
Martin Baulig 2dc7e89078 [System]: Fix bug in WebResponseStream.GetResponse() that will make some people happy. (#9228)
Test: https://github.com/xamarin/web-tests/commit/0777c6bbe3adc2b678ee286cd8e06067f29e548f

You need to construct the response like in the test.  Total size of the response must be somewhere between 4082 and 4092 bytes and you must send this as a blob - that is, using a single write operation.
2018-06-22 18:09:23 +02:00
Martin Baulig d125afca84 [System]: The big WebConnectionStream cleanup. (#8569)
* [System]: The big WebConnectionStream cleanup.

`WebResponseStream` has been broken down into smaller pieces and now
fully supports GZip / Deflate decompression.

The entire `contentLength`, `readBuffer` and `totalRead` logic has been
replaced by a nested layer of `WebReadStream` classes.

* `WebReadStream` is a new abstract base class.

* `BufferedReadStream` handles the situation where we have some buffered
  data and an (optional) inner stream to read from.

* `ContentDecodeStream` takes care of GZip / Deflate decompression.

* `FixedSizeReadStream` handles the `contentLength` logic - ie. making sure
  we are only reading up to `contentLength` bytes from the stream.

* The old `MonoChunkStream` has been renamed into `MonoChunkParser` (as it was
  really just a parser and not a stream) and a new `MonoChunkStream` has been
  added that's based on `WebReadStream`.

* Remove HttpWebRequestTest.TestLargeDataReading().

This test was using Reflection to poke around implementation internals.

* [csproj] Update project files

* Cosmetic cleanups.
2018-05-10 20:17:21 -04:00
Martin Baulig 762af4e5b0 [System]: Fix memory leak in the web stack related to Task.Delay (partial fix of #7356).
Using `Task.Delay (timeout)` has the potential of possibly allocating and holding large
chunks of memory throughout the timeout period.  While this is not strictly speaking a
memory leak because those objects will eventually be disposed and garbage collected, it
can cause memory usage of apps to grow quite significantly in the short run.

The default timeout values are as follows:

* `HttpWebRequest.ReadWriteTimeout`: 5 minutes.

* `HttpWebRequest.Timeout`: 100 seconds.

* `ServicePointManager.MaxServicePointIdleTime`: 100 seconds.

And these values can be increased by the user.  This means that we're potentially holding
hundreds, maybe even thousands of timeout objects in memory for 5+ minutes.

This has now been fixed by using the `Task.Delay(int,CancellationToken)` constructor.

Note that disposing a `CancellationTokenSource` will not cancel it, so the following code
will still leak:

	using (var cts = new CancellationTokenSource ()) {
		Task.Delay (TimeSpan.FromDays (3), cts.Token);
		// Work
	}

To actually make the `Task.Delay` instance go away when we're done with it, we need to
use this pattern:

	var cts = new CancellationTokenSource ();
	try {
		Task.Delay (TimeSpan.FromDays (3), cts.Token);
		// Work
	} finally {
		cts.Cancel ();
		cts.Dispose ();
	}

We also cannot use `CancellationToken.CancelAfter()` for timeouts because unfortunately
some of the underlying framework code that we're using does not honor cancellation
requests - like for instance `NetworkStream`.

This is a partial fix for #7356.  While this commit significantly improves the problem,
there may be other unrelated issues left that needs to be investigated.
2018-04-06 20:46:24 +02:00
Martin Baulig e480620040 [System]: Fix a race condition that was exposed by #7358 (see #7668).
The latest `WebCompletionSource<T>` changes (PR #7358) exposed and
intensified the side-effects of a race condition between `WebOperation`
and `ServicePointScheduler`.

* WebCompletionSource<T> now contains new `CurrentResult` and `CurrentStatus`
  properties, which allow completion status to be checked in a non-blocking way.

* WebOperation.CompleteResponseRead() has been renamed into Finish() and
  contains the continuation that was previouly known as FinishReading().
  Calling this new Finish() method will now complete the `finishedTask`
  completion, which has a new accessor called `Finished`.

* The ServicePointScheduler's main-loop is still woken up by completion of
  `WebOperation.Finished.Task`, but we use the new `CurrentResult` property
  to check state in a non-blocking way.  This ensures the the clean-up and
  bookkeeping work that's necessary after `WebCompletion.Finish()` always
  happens prior to starting new connections.
2018-03-20 16:55:25 +01:00
Martin Baulig 4b88e2a0e2 [System]: Introduce WebCompletionSource<T> and use it in WebOperation. (#7358) 2018-03-15 21:50:32 +01:00
Martin Baulig 8facc4afb2 [System]: Use new WebCompletionSource instead of TaskCompletionSource. (#7293)
This new helper class uses `TaskCompletionSource<>` internally, but instead
of using TrySetException(), it sets a result containing an `ExceptionDispatchInfo`.

The problem with using TrySetException() is that it wraps the exception
object in an `AggregateException`, that would be unobserved if the caller
throws the original exception.
2018-02-28 09:37:24 +01:00
Martin Baulig 9648645fb3 [System]: Enable some System.Net.Requests tests from CoreFX. (#7080)
* [System]: Enable some System.Net.Requests tests from CoreFX.

* tests.make: add `$(XTEST_HARNESS_PATH)` to `TEST_MONO_PATH`,
  so we can find xunit.assert.

* corefx/SR.cs: add `System.Net.Requests` resources from corefx.

* HttpWebRequest: See below for detailed summary.

* Match parameter names of overridden base class members.

* Fixed order of some argument / state checks to match the
  tested behavior.

* Most async methods now throw exceptions instead of returning
  faulted tasks when called with incorrect arguments.

* HttpWebResponse.ContentType: use `string.empty` as default.

* WebConnectionStream.FlushAsync: override to return a finished task.

* HttpWebRequestTest.AllowReadStreamBuffering: removed this test.

* test-helpers/RemoteExecutorConsoleApp: silently ignore compiler-
  generated helper methods.

Enabled tests from CoreFX:
--------------------------

Added the following tests from external/corefx/src/System.Net.Requests/tests:

* AuthorizationTest, FileWebRequestTest, GlobalProxySelectionTest,
  HttpRequestCachePolicyTest, HttpWebRequestTest, HttpWebResponseHeaderTest,
  HttpWebResponseTest, LoggingTest, RequestStreamTest, WebRequestTest

Not working yet:

* AuthenticationManagerTest, FtpWebRequestTest

We are currently using
../../../external/corefx/src/System.Net.Requests/tests/*.cs:AuthenticationManagerTest.cs,FtpWebRequestTest.cs

Summary of HttpWebRequest changes:
----------------------------------

* Host - we are now using the implementation from corefx.

* DefaultMaxResponseHeadersLength (unused) - fix default value.

* AllowReadStreamBuffering (unused) - allow this property to
  be read/written.

* ContinueTimeout - likewise.

* DefaultCachePolicy (unused) - allow this property to be
  read/written and set default value.

* DefaultMaximumErrorResponseLength (unused) - likewise.

* Connection - use implementation from corefx.

* Method - use implementation from corefx, which has additional
  error checks.

* MaximumResponseHeadersLength - use implementation from corefx,
  adding error checks.

* ReadWriteTimeout - likewise.

* ProtocolVersion - include parameter name in ArgumentException.

* TransferEncoding - use implementation from corefx.

* BeginGetRequestStream / GetRequestStreamAsync / GetRequestStream:
  throw exceptions when called with incorrect arguments rather than
  returning faulted tasks.

* Disable the corefx tests for the moment.

* Bump API snapshot submodule

* Update tests.make
2018-02-28 00:32:06 +01:00
monojenkins dd5b5ca52e Set NetworkStream.ReadTimeout in WebConnection.InitConnection(). (#6904)
Setting ReadTimeout in WebResponseStream.ctor throws System.Net.Sockets.SocketException : Invalid arguments on iOS.

Set it in WebConnection.InitConnection() immediately after establishing a connection, before we read/write anything.
2018-02-09 21:21:52 +01:00
Martin Baulig 8bfcb27147 [System]: Revert my latest WebResponseStream changes.
This reverts PR #6662 (commit 18a1a25) and #6771 (commit da03780).

These two changes introduced a regression:
https://github.com/mono/mono/issues/6788
2018-02-08 12:52:36 -05:00
Martin Baulig da03780370 [System]: WebResponseStream now correctly reads the chunk trailer when using gzip/deflate. (#6771)
* [System]: WebResponseStream now correctly reads the chunk trailer when using gzip/deflate.

This should actually fix bug #59779.

* Add missing ConfigureAwait(false)
2018-02-05 14:45:56 -05:00
Martin Baulig 18a1a25345 [System]: Refactor and cleanup WebResponseStream (#6662)
We are now using nested streams to take care of buffering, chunked encoding and automatic decompression.

GZip / Deflate decompression have been moved inside `WebResponseStream`, the `HttpWebResponse` now always returns a `WebResponseStream` (and not a `GZipStream`) because we have to do extra book-keeping after reading the entire content.
2018-02-01 23:39:46 +01:00
Martin Baulig cd09ddbc55 New PR for the new web stack (#6125) 2017-12-14 10:06:47 +01:00